Hey, What’s the Big Idea: Modeling the the Networks of Emerging Innovations

“Connex Labyrinth”, Image by fdecomite.

When you want to invite someone new into your LinkedIn network, the social business platform provides users with a simple formatted email making it easier to do this. It reads in part “I’d like to add you to my network”. Recently, a research team published the results of their work proving that a similar network effect 1 occurs not only among people, but also in a comparable manner among new ideas. As a result of such inventive schmoozing, ideas plus other new ideas can now be seen as yielding all kinds of unexpected links to innovation 2.

This “mathematical model for the emergence of innovations” mapping out all these processes was published by the School of Mathematical Sciences at Queen Mary University of London, as reported in a fascinating article about an exciting advance on Phys.Org in a story entitled Mathematicians Develop Model for How New Ideas Emerge, posted January 24, 2018. (No writer is credited.)  This article is a summary of the full paper by the research team responsible entitled Network Dynamics of Innovation Processes that appeared in the Physical Review Letters (subscription required), published on January 26, 2018.

I highly recommend a click-through and full read of the Phys.org article, including its two accompanying visualizations. I will recap and annotate some of the key points in this piece and then pose some of own questions about how and why ideas may either swipe left or swipe right when trying to meet up with each other.

New Modeling Combining Two Established Approaches

The research team was led by Professor Vito Latora, who was also the lead author of the paper. They found that by “studying creative processes and understanding how innovations arise” and, furthermore, how [mathematical] “novelties” can lead to additional discoveries, the results of these interactions can lead to “effective interventions” that could support the “success and sustainable growth” in our society. Similar patterns have been seen in scientific and artistic fields 3.

This was accomplished by first transposing the theory of the ‘adjacent possible’ from its original field of biological systems into, second, the “language of complex networks“. The former describes the “set of all novel opportunities” that arise whenever a new discovery appears. The latter has become a reliable means to study actual systems in the real world by examining the key relationships “between the components” and, in turn, modeling the “hidden structure behind many complex social phenomena”.

In effect, network modeling has been applied here to construct the “underlying space of relations among concepts”. Indeed, some very cool and productive connections are occurring within such idea-fueled networks.

Professor Latora further believes that understanding the key elements of a successful idea are critical to later making decisions, forming strategies and supporting successes.  He thinks that such results can be part of “sustainable growth in our society”.

Potential Benefits for Multiple Fields

Another new methodology was derived during the course of his team’s research: The concept of “reinforced walks” (as a subset of “random walks“, a form of mathematical object 4), was used as the basis to model the interaction among concepts and ideas. During all of this activity “innovation corresponds to the first visit of a site on the network”, as well as every time such a “walker”, as termed in the article, transits from one concept to another concept. The more this type of path is traveled, the more it becomes reinforced and thus productive. This network dynamic, named by the team as the “edge-reinforced random walk“, is clearly diagrammed and further described in the Phys.Org article’s first graphic.

In an actual case study applying this methodology, the research team built a database of 20 years’ worth of publications from a diversity of fields including “astronomy, ecology, economics and mathematics”. This was done to examine the emergence of new concepts. Their results of applying their edge-reinforced random walk model to this compilation was that they succeeded in reproducing evidence of the growth of knowledge in contemporary science.

Professor Latora and his team are currently working to extend their model by studying network spaces where several of these “walkers” are operating simultaneously 5.

My Questions

  • Can edge-reinforced random walks be adapted and applied to additional non-scientific and non-artistic domains such as history, politics and culture? That is, can it yield meaningful results and insights in sectors of society that are less data-derived?
  • How is this methodology distinguishable from various branches of artificial intelligence where vast stores of data are used to “train” the capabilities of these systems?
  • Can edge-reinforced random walks also be deployed as a form of predictive device? For example, what if a great deal more data from a larger diversity of fields was similarly compiled and tested, would this provide a partial preview into the future in science, technology and biology?
  • Taking this a step further, could edge-reinforced random walks be enhanced to include the capacity to predict or at least sense the possibilities and/or probabilities of entirely unpredictable major events such as market crashes? 6

 


For a comparative perspective, albeit a dated one published 18 years ago, on how human social systems and their interactions will still be needed in the all-encompassing “Information Age”, I suggest a book that is still considered a significant achievement for its time entitled The Social Life of Information, by John Seely Brown and David Duguid (Harvard Business Review, 2000).


1.  Network effects among people, populations and technologies have also been explored in these 10 Subway Fold posts.

2.  See also this April 28, 2016 Subway Fold post entitled Book Review of “Inventology: How We Dream Up Things That Change the World”

3.  Another example is the study of laws and legal precedents  in this manner as described in this May 15, 2015 Subway Fold post entitled Recent Visualization Projects Involving US Law and The Supreme Court.

4.  This concept as applied in securities trading is the subject of a classic text on this subject entitled A Random Walk Down Wall Street, by Burton Malkiel (W. W. Norton & Company, Eleventh Edition, 2016)

5.  In an entirely different context, fans of The Walking Dead will also appreciate how large groups of walkers manifest their own distinctly emergent behaviors.

6.  The leading text on this subject is The Black Swan: Second Edition: The Impact of the Highly Improbable, by Nassim Nicholas Taleb (Random House, 2010).

Watson, is That You? Yes, and I’ve Just Demo-ed My Analytics Skills at IBM’s New York Office

IMAG0082

My photo of the entrance to IBM’s office at 590 Madison Avenue in New York, taken on July 29, 2015.

I don’t know if my heart can take this much excitement. Yesterday morning, on July 29, 2015, I attended a very compelling presentation and demo of IBM’s Watson technology. (This AI-driven platform has been previously covered in these five Subway Fold posts.) Just the night before, I saw I saw a demo of some ultra-cool new augmented reality systems.

These experiences combined to make me think of the evocative line from Supernaut by Black Sabbath with Ozzie belting out “I’ve seen the future and I’ve left it behind”. (Incidentally, this prehistoric metal classic also has, IMHO, one of the most infectious guitar riffs with near warp speed shredding ever recorded.)

Yesterday’s demo of Watson Analytics, one key component among several on the platform, was held at IBM’s office in the heart of midtown Manhattan at 590 Madison Avenue and 57th Street. The company very graciously put this on for free. All three IBM employees who spoke were outstanding in their mastery of the technology, enthusiasm for its capabilities, and informative Q&A interactions with the audience. Massive kudos to everyone involved at the company in making this happen. Thanks, too, for all of attendees who asked such excellent questions.

Here is my summary of the event:

Part 1: What is Watson Analytics?

The first two speakers began with a fundamental truth about all organizations today: They have significant quantities of data that are driving all operations. However, a bottleneck often occurs when business users understand this but do not have the technical skills to fully leverage it while, correspondingly, IT workers do not always understand the business context of the data. As a result, business users have avenues they can explore but not the best or most timely means to do so.

This is where Watson can be introduced because it can make these business users self-sufficient with an accessible, extensible and easier to use analytics platform. It is, as one the speakers said “self-service analytics in the cloud”. Thus, Watson’s constituents can be seen as follows:

  • “What” is how to discover and define business problems.
  • “Why” is to understand the existence and nature of these problems.
  • “How” is to share this process in order to affect change.

However, Watson is specifically not intended to be a replacement for IT in any way.

Also, one of Watson’s key capabilities is enabling users to pursue their questions by using a natural language dialog. This involves querying Watson with questions posed in ordinary spoken terms.

Part 2: A Real World Demo Using Airline Customer Data

Taken directly from the world of commerce, the IBM speakers presented a demo of Watson Analytics’ capabilities by using a hypothetical situation in the airline industry. This involved a business analyst in the marketing department for an airline who was given a compilation of market data prepared by a third-party vendor. The business analyst was then assigned by his manager with researching and planning how to reduce customer churn.

Next, by enlisting Watson Analytics for this project, the two central issues became how the data could be:

  • Better understand, leveraged and applied to increase customers’ positive opinions while simultaneously decreasing the defections to the airline’s competitors.
  • Comprehensively modeled in order to understand the elements of the customer base’s satisfaction, or lack thereof, with the airline’s services.

The speakers then put Watson Analytics through its paces up on large screens for the audience to observe and ask questions. The goal of this was to demonstrate how the business analyst could query Watson Analytics and, in turn, the system would provide alternative paths to explore the data in search of viable solutions.

Included among the variables that were dexterously tested and spun into enlightening interactive visualizations were:

  • Satisfaction levels by other peer airlines and the hypothetical Watson customer airline
  • Why customers are, and are not, satisfied with their travel experience
  • Airline “status” segments such as “platinum” level flyers who pay a premium for additional select services
  • Types of travel including for business and vacation
  • Other customer demographic points

This results of this exercise as they appeared onscreen showed how Watson could, with its unique architecture and tool set:

  • Generate “guided suggestions” using natural language dialogs
  • Identify and test all manner of connections among the population of data
  • Use predictive analytics to make business forecasts¹
  • Calculate a “data quality score” to assess the quality of the data upon which business decisions are based
  • Map out a wide variety of data dashboards and reports to view and continually test the data in an effort to “tell a story”
  • Integrate an extensible set of analytical and graphics tools to sift through large data sets from relevant Twitter streams²

Part 3: The Development Roadmap

The third and final IBM speaker outlined the following paths for Watson Analytics that are currently in beta stage development:

  • User engagement developers are working on an updated visual engine, increased connectivity and capabilities for mobile devices, and social media commentary.
  • Collaboration developers are working on accommodating work groups and administrators, and dashboards that can be filtered and distributed.
  • Data connector developers are working on new data linkages, improving the quality and shape of connections, and increasing the degrees of confidence in predictions. For example, a connection to weather data is underway that would be very helpful to the airline (among other industries), in the above hypothetical.
  • New analytics developers are working on new functionality for business forecasting, time series analyses, optimization, and social media analytics.

Everyone in the audience, judging by the numerous informal conversations that quickly formed in the follow-up networking session, left with much to consider about the potential applications of this technology.


1.  Please see these six Subway Fold posts covering predictive analytics in other markets.

2.  Please see these ten Subway Fold posts for a variety of other applications of Twitter analytics.