It’s Crystal Clear: New Methods Emerge to Predict the Distribution Pattern of Prime Numbers

Despite the unsettling effects of figuring out fractions, performing long division and taking the square roots of numbers have had for an eternity upon many students in the middle grades, some of these people continue on to adulthood with at least an appreciation of what mathematics can do in the real world. While they might still break into a slight sweat if challenged to quickly calculate the equivalent of 3/8, they still realize the importance of doing so and, moreover, applying that value to solve a problem.

So too, just as math teachers everywhere exhort their students to “put on their thinking caps”, sometimes a math story appears in the news that takes a bit more concentration to fully comprehend, but nonetheless really does have a certain technological cool and practicality to it. What is equally intriguing is when such a new development has the potential to eventually impact other areas of innovation that appear at first to be disparate or even unrecognizable. On its face, scientific advance X could not possibly be related to mathematical outcome Y until, by virtue of some very unconventional thinkers in another field, the real possibility emerges of a workable application of X to achieve Y.

Let’s take our virtual calculators out of their pocket protectors and have a look at such a recent advancement that is not only useful as party fun for math geeks. Rather, it may have meaningful significant in encryption science and, in turn, online security, e-commerce and data privacy. This achievement was reported in a fascinating article entitled Researchers Discover a Pattern to the Seemingly Random Distribution of Prime Numbers, by Liv Boeree, posted on Motherboard.com on September 14, 2018.

I will summarize and annotate this, and then pose several of my own equation-free questions.

Prime Time

Image from Pixabay.com

First, the basics: Prime numbers (“primes”) are whole numbers that are only divisible by 1 and themselves. They start out small as 2, 3 and 5 and range upwards towards infinity.¹ As these primes are plotted out along on a graph they appear to be increasingly random with no discernible or predictable pattern.

Nonetheless, one of the greatest unsolved math problems is called the Reimann Hypothesis which, among its other brain-bending complexities, posits that there may well be a pattern to the distribution of primes but it has not yet been derived.² Discovering such a pattern would be a monumental accomplishment with major significance in mathematics, physics and modern cryptography, the latter of which is based upon large prime numbers. (More about this below).

Recently, three researchers at Princeton University have made such a discovery about an atomic pattern in a physical material comparable to the distribution of primes. They have found similarities involving primes and “certain naturally occurring crystalline materials”.³ Their recent scientific paper detailing this work is entitled Uncovering Multiscale Order in the Prime Numbers Via Scattering, by Salvatore Torquato, Ge Zhang and Matthew de Courcy-Ireland, was published in the Journal of Statistical Mechanics: Theory and Experiment, on September 5, 2018.  

The unpredictability of finding new primes is not always necessarily a detriment. For example, modern cryptography methods such as the RSA encryption algorithm depends upon this factor when it comes to very large primes. This relies upon the principle that it is simple enough to take two large prime numbers and multiply them but intensely difficult to reverse this in an effort to determine exactly which two primes were used.

[While this post was being drafted, an article was posted on BusinessInsider.com on September 25, 2018 entitled An Eminent Mathematician Claims to Have Solved One of Math’s Greatest Mysteries — and It’s One of 6 Problems With a $1 Million Prize, by Andy Kiersz, reported that Sir Michael Atiyah has achieved a solution to the Reimann Hypothesis. However, this remains to be vetted by other mathematicians in this field. This problem is one of six remaining great unsolved math problems, termed the “Millennial Problems”, for which the Clay Mathematics Institute has offered a $1 million prize for the solution to each.4 This article also contains concise descriptions of the other five problems.]

Fine Crystal Settings

Fractal Rhombic Ring, Image by fdcomite

In a process known as X-ray diffraction, chemists and physicists study the atomic structure of a material by exposing it to x-rays and observing how the beams “scatter off the atoms within it”. Different materials will produce a variety of such patterns and indicate “how symmetrically their atoms are arranged”. In the case of a crystal, whose atomic structure is more firm than other materials such as liquids, the x-ray’s pattern of diffraction is “more orderly”.

In 2017, the lead author of the paper, Professor Salvatore Torquato, wondered whether primes could be “modeled as atom-like particles” and whether they would also form a pattern. Along with his co-authors, together they “computationally represented the primes as a one-dimensional string of atoms” and then “scattered light off them”.

They found that this created a “quasicrystal-like inference pattern” that was also a previously unseen form of fractal pattern termed “hyperuniformity“. It is exhibited by only a several “materials and systems in nature”. Included among them are prime numbers. This finding might turn out to be useful in studying such non-repeating patterns in a new field of research called “aperiodic order“.

Professor Torquato said in an article in Quanta Magazine entitled A Chemist Shines Light on a Surprising Prime Number Pattern, by Natalie Wolchover, dated May 14, 2018, that there is a resulting implication that primes “are a completely new category of structures” when viewing them as a form of physical system.

Much of the interest surrounding the new paper is its “unique intersection between the physical and more abstract mathematical realms”. As well, it contains a new algorithm that permits the prediction of primes “with high accuracy”. In time this may prove to be another advance in decisively solving the mysteries of the primes.

Image from Pixabay.com (2)

My Questions

  • If Professor Torquato’s and his co-authors’ paper and algorithm prove to be genuinely able to predict the patterns of the appearance of primes, does this actually strengthen and/or weaken the foundation of RSA-based encryption?
  • Moreover, if Sir Atiyah’s has, in fact, solved the Reimann Hypothesis, what are the potential positive and negative effects upon the whole field of cryptography? Are there any additional impacts on other fields of science, math, physics and technology?
  • If and when practical quantum computing becomes a reality and results in the capability to much more rapidly factor primes used in encryption, how will the work of Professor Torquato and Sir Atiyah be affected?
  • So, how much is 3/8 anyway?

 


1.  Currently, the largest prime number ever discovered was identified in 2017 and has 23,239,425 digits. That’s a lot.

2.  For an outstanding history of the pursuit of prime numbers and the mathematical quest to discover a pattern in their distribution, I very high recommend reading The Music of the Primes: Searching to Solve the Greatest Mystery in Mathematics, by Marcus du Sautoy,  Harper Perennial; Reprint edition (August 14, 2012). This is a very accessible and literate book that presents a variety of engaging stories and deep insights into what might otherwise have otherwise appeared to have been a rather dry subject.

3.  A more technical report on this story was posted on Princeton’s website entitled Surprising Hidden Order Unites Prime Numbers and Crystal-like Materials, by Kevin McElwee, on September 5, 2018.

4.  I suggest adding a seventh intractable problem to this list that will likely never be solved: Finding a parking spot in my neighborhood.

Concrete Data Sets: New Online Map of Building Construction Metrics Across New York

Image from Pixabay.com

There is an age-old expression among New Yorkers that their city will really be a great place one day if someone ever finishes building it. I have heard this many times during my life as a native and lifelong resident of this remarkable place.

Public and private construction goes on each day on a vast scale throughout the five boroughs of NYC. Over the past several decades under successive political administrations, many areas have been re-zoned to facilitate and accelerate this never-ending buildup and built-out. This relentless activity produces many economic benefits for the municipal economy. However, it also results in other detrimental effects including housing prices and rents that continue to soar upward, disruptive levels of noise and waste materials affecting people living nearby, increased stresses upon local infrastructure, and just as regrettably, steady erosion of the unique characters and spirits of many neighborhoods.¹

In a significant technological achievement intended to focus and consolidate the massive quantities of location, scope and cost data about the  plethora of structures sprouting up everywhere, on August 22, 2018 the New York City Buildings Department launched an interactive NYC Active Major Construction Map (“The Map”). Full coverage of its inauguration was provided in a very informative article in The New York Times entitled A Real-Time Map Tracks the Building Frenzy That’s Transforming New York, by Corey Kilgannon, on August 22, 2018. (Here, too, is the Building Department’s press release.) I highly recommend both a click-through and full read of it and further online exploration of The Map itself.

I will also summarize and annotate this report, and then pose some of my own code compliant questions.

Home on the [Data] Range

Construction on Lexington Avenue, Image by Jeffrey Zeldman

As the ubiquitous pounding of steel and pouring of concrete proceeds unabated, there is truly little or no getting around it. The Map is one component of a $60 million digital initiative established in 2015 which is intended to produce an “impressive level of detail” on much of this cityscape altering activity.

The recent inception of The Map provides everyone in the metro area an online platform to track some of the key details of the largest of these projects plotted across a series of key metrics.  An accompanying grid of tables below it lists and ordinates the largest projects based upon these dimensions.

The Map’s user interface presents this “overview of the frenzy of construction” dispersed across the city’s communities using the following configurations:

  • Each project’s location represented by a blue dot that can be clicked to reveal the property’s contractor, history and any violations.
  • Cumulative real-time totals of square footage under construction, permits and dwelling units involved. This data can be further filtered by borough.
  • Scrollable and clickable Top 10 lists by project square footage, size, cost and dwelling units

As well, it provides residents a virtual means to identify who is making all of that real-world blaring construction noise in their neighborhood.²

If I Had a Hammer

Executives, organizations and community advocates representing a diversity of interests have expressed their initial support for The Map.

Second Avenue Subway Update, Image by MTA (2)

The NYC Building Commissioner, Rick D. Chandler, believes this new resource is a means to provide transparency to his department’s tremendous quantity of construction data. Prior to the rollout of The Map, accessing and processing this information required much greater technical and professional skills. Furthermore, the data will be put to use to “improve and streamline the department’s operations”.

According to Andrew Berman, the Executive Director of the non-profit advocacy group Greenwich Village Society for Historic Preservation, he finds The Map to be both useful and “long overdue”. It is providing his group with a more convenient means to discover additional information about the proliferation of project sites in the Village. He also noted that under the previously existing municipal databases, this data was far more challenging to extract. Nonetheless, the new map remains insufficient for him and “other measures were needed” for the city government to increase oversight and enforcement of construction regulations concerning safety and the types of projects are permitted on specific properties.

Local real estate industry trade groups such as the Real Estate Board of New York, are also sanguine about this form of digital innovation, particularly for it accessibility. The group’s current president, John H. Banks, finds that it is “more responsive to the needs of the private sector”, raises transparency and the public’s “awareness of economic activity, jobs and tax revenues” flowing from the city’s construction projects.

Plans are in place to expand The Map based upon user feedback. As well, it will receive daily updates thus providing “a real-time advantage over analyst and industry reports”.

Image from Pixabay.com

My Questions

  • Does a roadmap currently exist for the projected development path of The Map’s content and functionality? If so, how can all interested parties provide ongoing commentary and support for it?
  • Are there other NYC databases and data sources that could possibly be integrated into the map? For example, tax, environmental and regulatory information might be helpful.
  • Can other cities benefit from the design and functionality of The Map to create or upgrade their own versions of similar website initiatives?
  • What new entrepreneurial, academic and governmental opportunities might now present themselves because of The Map?
  • How might artificial intelligence and/or machine learning capabilities be, well, mapped into The Map’s functionalities? Are there any plans to add chatbot scripting capabilities to The Map?

 


Two related Subway Fold posts covering other aspects of construction include:


1.  For a deeply insightful analysis and passionate critique of the pervasive and permanent changes to many of New York’s neighborhoods due to a confluence of political, economic and social forces and interests, I highly recommend reading Vanishing New York: How a Great City Lost Its Soul, by Jeremiah Moss, (Dey Street Books, 2017). While I did not agree with some aspects of his book, the author has expertly captured and scrutinized how, where and why this great city has been changed forever in many ways. (See also the author’s blog Jeremiah’s Vanishing New York for his continuing commentary and perspectives.)

2.  Once I lived in a building that had been mercifully quiet for a long time until the adjacent building was purchased, gutted and totally renovated. For four months during this process, the daily noise level by comparison made a typical AC/DC concert sound like pin drop.