The Mediachain Project: Developing a Global Creative Rights Database Using Blockchain Technology

Image from Pixabay

Image from Pixabay

When people are dating it is often said that they are looking for “Mr. Right” or “Ms. Right”. That is, finding someone who is just the right romantic match for them.

In the case of today’s rapid development, experimentation and implementation of blockchain technology, if a startup’s new technology takes hold, it might soon find a highly productive (but maybe not so romantic) match in finding Mr. or Ms. [literal] Right by deploying the blockchain as a form of global registry of creative works ownership.

These 5 Subway Fold posts have followed just a few of the voluminous developments in bitcoin and blockchain technologies. Among them, the August 21, 2015 post entitled Two Startups’ Note-Worthy Efforts to Adapt Blockchain Technology for the Music Industry has drawn the most number of clicks. A new report on on February 23, 2016 entitled Mediachain is Using Blockchain to Create a Global Rights Database by Pete Rizzo provides a most interesting and worthwhile follow on related to this topic. I recommend reading it in its entirety. I will summarize and annotate it to provide some additional context, and then pose several of my own questions.

Producing a New Protocol for Ownership, Protection and Monetization

Applications of blockchain technology for the potential management of economic and distribution benefits of “creative professions”, including writers, musicians and others, that have been significantly affected by prolific online file copying still remains relatively unexplored. As a result, they do not yet have the means to “prove and protect ownership” of their work. Moreover, they do have an adequate system to monetize their digital works. But the blockchain, by virtue of its structural and operational nature, can supply these creators with “provenance, identity and micropayments“. (See also the October 27, 2015 Subway Fold post entitled Summary of the Bitcoin Seminar Held at Kaye Scholer in New York on October 15, 2015 for some background on these three elements.)

Now on to the efforts of a startup called Mine ( @mine_labs ), co-founded by Jesse Walden and Denis Nazarov¹. They are preparing to launch a new metadata protocol called Mediachain that enables creators working in digital media to write data describing their work along with a timestamp directly onto the blockchain. (Yet another opportunity to go out on a sort of, well, date.)  This system is based upon the InterPlanetary File System (IPFS). Mine believes that IPSF is a “more readable format” than others presently available.

Walden thinks that Mediachain’s “decentralized nature”, rather than a more centralized model, is critical to its objectives. Previously, a very “high-profile” somewhat similar initiative to establish a similarly global “database of musical rights and works” called the Global Repertoire Database (GDR) had failed.

(Mine maintains this page of a dozen recent posts on about their technology that provides some interesting perspectives and details about the Mediachain project.)

Mediachain’s Objectives

Walden and Nazarov have tried to innovate by means of changing how media businesses interact with the Internet, as opposed to trying to get them to work within its established standards. Thus, the Mediachain project has emerged with its focal point being the inclusion of descriptive data and attribution for image files by combining blockchain technology and machine learning². As well, it can accommodate reverse queries to identify the creators of images.

Nazarov views Mediachain “as a global rights database for images”. When used in conjunction with, among others, Instagram, he and Walden foresee a time when users of this technology can retrieve “historic information” about a file. By doing so, they intend to assist in “preserving identity”, given the present challenges of enforcing creator rights and “monetizing content”. In the future, they hope that Mediachain inspires the development of new platforms for music and movies that would permit ready access to “identifying information for creative works”. According to Walden, their objective is to “unbundle identity and distribution” and provide the means to build new and more modern platforms to distribute creative works.

Potential Applications for Public Institutions

Mine’s co-founders believe that there is further meaningful potential for Mediachain to be used by public organizations who provide “open data sets for images used in galleries, libraries and archives”. For example:

  • The Metropolitan Museum of Art (“The Met” as it is referred to on their website and by all of my fellow New York City residents), has a mandate to license the metadata about the contents of their collections. The museum might have a “metadata platform” of its own to host many such projects.
  • The New York Public Library has used their own historical images, that are available to the public to, among other things, create maps.³ Nazarov and Walden believe they could “bootstrap the effort” by promoting Mediachain’s expanded apps in “consumer-facing projects”.

Maintaining the Platform Security, Integrity and Extensibility

Prior to Mediachain’s pending launch, Walden and Nazarov are highly interested in protecting the platform’s legitimate users from “bad actors” who might wrongfully claim ownership of others’ rightfully owned works. As a result, to ensure the “trust of its users”, their strategy is to engage public institutions as a model upon which to base this. Specifically, Mine’s developers are adding key functionality to Mediachain that enables the annotation of images.

The new platform will also include a “reputation system” so that subsequent users will start to “trust the information on its platform”. In effect, their methodology empowers users “to vouch for a metadata’s correctness”. The co-founders also believe that the “Mediachain community” will increase or decrease trust in the long-term depending on how it operates as an “open access resource”. Nazarov pointed to the success of Wikipedia to characterize this.

Following the launch of Mediachain, the startup’s team believes this technology could be integrated into other existing social media sites such as the blogging platform Tumblr. Here they think it would enable users to search images including those that may have been subsequently altered for various purposes. As a result, Tumblr would then be able to improve its monetization efforts through the application of better web usage analytics.

The same level of potential, by virtue of using Mediachain, may likewise be found waiting on still other established social media platforms. Nazarov and Walden mentioned seeing Apple and Facebook as prospects for exploration. Nazarov said that, for instance, could set its own terms for its usage and consumption on Facebook Instant Articles (a platform used by publishers to distribute their multimedia content on FB). Thereafter, Mediachain could possibly facilitate the emergence of entirely new innovative media services.

Nazarov and Walden temper their optimism because the underlying IPFS basis is so new and acceptance and adoption of it may take time. As well, they anticipate “subsequent issues” concerning the platform’s durability and the creation of “standards for metadata”. Overall though, they remain sanguine about Mediachain’s prospects and are presently seeking developers to embrace these challenges.

My Questions

  • How would new platforms and apps using Mediachain and IPSF be affected by the copyright and patent laws and procedures of the US and other nations?
  • How would applications built upon Mediachain affect or integrate with digital creative works distributed by means of a Creative Commons license?
  • What new entrepreneurial opportunities for startup services might arise if this technology eventually gains web-wide adoption and trust among creative communities?  For example, would lawyers and accountants, among many others, with clients in the arts need to develop and offer new forms of guidance and services to navigate a Mediachain-enabled marketplace?
  • How and by whom should standards for using Mediachain and other potential development path splits (also known as “forks“), be established and managed with a high level of transparency for all interested parties?
  • Does analogizing what Bitcoin is to the blockchain also hold equally true for what Mediachain is to the blockchain, or should alternative analogies and perspectives be developed to assist in the explanation, acceptance and usage of this new platform?

June 1, 2016 Update:  For an informative new report on Mediachain’s activities since this post was uploaded in March, I recommend clicking through and reading Mediachain Enivisions a Blockchain-based Tool for Identifying Artists’ Work Across the Internet, by Jonathan Shieber, posted today on

1.   This link from Mine’s website is to an article entitled Introducing Mediachain by Denis Nazarov, originally published on on January 2, 2016. He mentions in his text an earlier startup called Diaspora that ultimately failed in its attempt at creating something akin to the Mediachain project. This December 4, 2014 Subway Fold post entitled Book Review of “More Awesome Than Money” concerned a book that expertly explored the fascinating and ultimately tragic inside story of Diaspora.

2.   Many of the more than two dozen Subway Fold posts in the category of Smart Systems cover some of the recent news, trends and applications in machine learning.

3.  For details, see the January 5, 2016 posting on the NY Public Library’s website entitled Free for All: NYPL Enhances Public Domain Collections for Sharing and Reuse, by Shana Kimball and Steven A. Schwarzman.

Visionary Developments: Bionic Eyes and Mechanized Rides Derived from Dragonflies

"Transparency and Colors", Image by coniferconifer

“Transparency and Colors”, Image by coniferconifer

All manner of software and hardware development projects strive to diligently take out every single bug that can be identified¹. However, a team of researchers who is currently working on a fascinating and potentially valuable project is doing everything possible to, at least figuratively, leave their bugs in.

This involves a team of Australian researchers who are working on modeling the vision of dragonflies. If they are successful, there could be some very helpful implications for applying their work to the advancement of bionic eyes and driverless cars.

When the design and operation of biological systems in nature are adapted to improve man-made technologies as they are being here, such developments are often referred to as being biomimetic².

The very interesting story of this, well, visionary work was reported in an article in the October 6, 2015 edition of The Wall Street Journal entitled Scientists Tap Dragonfly Vision to Build a Better Bionic Eye by Rachel Pannett. I will summarize and annotate it, and pose some bug-free questions of my own. Let’s have a look and see what all of this organic and electronic buzz is really about.

Bionic Eyes

A research team from the University of Adelaide has recently developed this system modeled upon a dragonfly’s vision. It is built upon a foundation that also uses artificial intelligence (AI)³. Their findings appeared in an article entitled Properties of Neuronal Facilitation that Improve Target Tracking in Natural Pursuit Simulations that was published in the June 6, 2015 edition of The Royal Society Interface (access credentials required). The authors include Zahra M. Bagheri, Steven D. Wiederman, Benjamin S. Cazzolato, Steven Grainger, and David C. O’Carroll. The funding grant for their project was provided by the Australian Research Council.

While the vision of dragonflies “cannot distinguish details and shapes of objects” as well as humans, it does possess a “wide field of vision and ability to detect fast movements”. Thus, they can readily track of targets even within an insect swarm.

The researchers, including Dr. Steven Wiederman, the leader of the University of Adelaide team, believe their work could be helpful to the development work on bionic eyes. These devices consist of an  artificial implant placed in a person’s retina that, in turn, is connected to a video camera. What a visually impaired person “sees” while wearing this system is converted into electrical signals that are communicated to the brain. By adding the software model of the dragonfly’s 360-degree field of vision, this will add the capability for the people using it to more readily detect, among other things, “when someone unexpectedly veers into their path”.

Another member of the research team and one of the co-authors of their research paper, a Ph.D. candidate named Zahra Bageri, said that dragonflies are able to fly so quickly and be so accurate “despite their visual acuity and a tiny brain around the size of a grain of rice”4 In other areas of advanced robotics development, this type of “sight and dexterity” needed to avoid humans and objects has proven quite challenging to express in computer code.

One commercial company working on bionic eye systems is Second Sight Medical Products Inc., located in California. They have received US regulatory approval to sell their retinal prosthesis.

Driverless Cars

In the next stage of their work, the research team is currently studying “the motion-detecting neurons in insect optic lobes”, in an effort to build a system that can predict and react to moving objects. They believe this might one day be integrated into driverless cars in order to avoid pedestrians and other cars5. Dr. Wiederman foresees the possible commercialization of their work within the next five to ten years.

However, obstacles remain in getting this to market. Any integration into a test robot would require a “processor big enough to simulate a biological brain”. The research team believes that is can be scaled down since the “insect-based algorithms are much more efficient”.

Ms. Bagheri noted that “detecting and tracking small objects against complex backgrounds” is quite a technical challenge. She gave as an example of this a  baseball outfielder who has only seconds to spot, track and predict where a ball hit will fall in the field in the midst of a colorful stadium and enthusiastic fans6.

My Questions

  • As suggested in the article, might this vision model be applicable in sports to enhancing live broadcasts of games, helping teams review their game day videos afterwards by improving their overall play, and assisting individual players to analyze how they react during key plays?
  • Is the vision model applicable in other potential safety systems for mass transportation such as planes, trains, boats and bicycles?
  • Could this vision model be added to enhance the accuracy, resolution and interactivity of virtual reality and augmented reality systems? (These 11 Subway Fold posts appearing in the category of Virtual and Augmented Reality cover a range of interesting developments in this field.)


1.  See this Wikipedia page for a summary of the extraordinary career Admiral Grace Hopper. Among her many technological accomplishments, she was a pioneer in developing modern computer programming. She was also the originator of the term computer “bug”.

2For an earlier example of this, see the August 18, 2014 Subway Fold post entitled IBM’s New TrueNorth Chip Mimics Brain Functions.

3The Subway Fold category of Smart Systems contains 10 posts on AI.

4Speaking of rice-sized technology, see also the April 14, 2015 Subway Fold post entitled Smart Dust: Specialized Computers Fabricated to Be Smaller Than a Single Grain of Rice.

5While the University of Adelaide research team is not working with Google, nonetheless the company has been a leader in the development of autonomous cars with their Google’s Self-Driving Car Project.

6New York’s beloved @Mets might also prove to be worthwhile subjects to model because of their stellar play in the 2015 playoffs. Let’s vanquish those dastardly LA Dodgers on Thursday night. GO METS!

NASA is Providing Support for Musical and Humanitarian Projects

"NASA - Endeavor 2", Image by NASA

“NASA – Endeavor 2”, Image by NASA

In two recent news stories, NASA has generated a world of good will and positive publicity about itself and its space exploration program. It would be an understatement to say their results have been both well-grounded and out of this world.

First, NASA astronaut Chris Hadfield created a vast following for himself online when he uploaded a video onto YouTube of him singing David Bowie’s classic Space Oddity while on a mission on the International Space Station (ISS).¹ As reported on the October 7, 2015 CBS Evening News broadcast, Hadfield will be releasing an album of 12 songs he wrote and performed in space, today on October 9. 2015. He also previously wrote a best-selling book entitled An Astronaut’s Guide to Life on Earth: What Going to Space Taught Me About Ingenuity, Determination, and Being Prepared for Anything (Little, Brown and Company, 2013). I highly recommend checking out his video, book and Twitter account @Cmdr_Hadfield.

What a remarkably accomplished career in addition to his becoming an unofficial good will ambassador for NASA.

The second story, further enhancing the agency’s reputation, concerns a very positive program affecting many lives that was reported in a most interesting article on on September 28, 2015 entitled How NASA Data Can Save Lives From Space by Issie Lapowsky. I will summarize and annotate it, and then pose some my own terrestrial questions.

Agencies’ Partnership

According to a NASA administrator Charles Bolden, astronauts frequently look down at the Earth from space and realize that borders across the world are subjectively imposed by warfare or wealth. These dividing lines between nations seem to become less meaningful to them while they are in flight. Instead, the astronauts tend to look at the Earth and have a greater awareness everyone’s responsibilities to each other. Moreover, they wonder what they can possibly do when they return to make some sort of meaningful difference on the ground.

Bolden recently shared this experience with an audience at the United States Agency for International Development (USAID) in Washington, DC, to explain the reasoning behind a decade-long partnership between NASA and USAID. (This latter is the US government agency responsible for the administration of US foreign aid.) At first, this would seem to be an unlikely joint operation between two government agencies that do not seem to have that much in common.

In fact, this combination provides “a unique perspective on the grave need that exists in so many places around the world”, and a special case where one agency sees it from space and the other one sees it on the ground.

They are joined together into a partnership known as SERVIR where NASA supplies “imagery, data, and analysis” to assist developing nations.  They help these countries with forecasting and dealing “with natural disasters and the effects of climate change”.

Partnership’s Results

Among others, SERVIR’s tools have produced the following representative results:

  • Predicting floods in Bangladesh that gives citizens a total of eight days notice in order to make preparations that will save lives. This reduced the number to 17 during the last year’s monsoon season whereas previously it had been in the thousands.
  • Predicting forest fires in the Himalayas.
  • For central America, NASA created  a map of ocean chlorophyll concentration that assisted public officials in identifying and improving shellfish testing in order to deal with “micro-algae outbreaks” responsible for causing significant health issues.

SERVIR currently operates in 30 countries. As a part of their network, there are regional hubs working with “local partners to implement the tools”. Last week it opened such a hub in Asia’s Mekong region. Both NASA and USAID are hopeful that the number of such hubs will continue to grow.

Google is also assisting with “life saving information from satellite imagery”. They are doing this by applying artificial intelligence (AI)² capabilities to Google Earth. This project is still in its preliminary stages.

My Questions

  • Should SERVIR reach out to the space agencies and humanitarian organizations of other countries to explore similar types of humanitarian joint ventures?
  • Do the space agencies of other countries have similar partnerships with their own aid agencies?
  • Would SERVIR benefit from partnerships with other US government agencies? Similarly, would it benefit from partnering with other humanitarian non-governmental organizations (NGO)?
  • Would SERVIR be the correct organization to provide assistance in global environmental issues? Take for example the report on the October 8, 2015 CBS Evening News network broadcast of the story about the bleaching of coral reefs around the world.


1.  While Hatfield’s cover and Bowie’s original version of Space Oddity are most often associated in pop culture with space exploration, I would like to suggest another song that also captures this spirit and then truly electrifies it: Space Truckin’ by Deep Purple. This appeared on their Machine Head album which will be remembered for all eternity because it included the iconic Smoke on the Water. Nonetheless, Space Truckin‘ is, in my humble opinion, a far more propulsive tune than Space Oddity. Its infectious opening riff will instantly grab your attention while the rest of the song races away like a Saturn Rocket reaching for escape velocity. Furthermore, the musicianship on this recording is extraordinary. Pay close attention to Richie Blackmore’s scorching lead guitar and Ian Paice’s thundering drums. Come on, let’s go space truckin’!

2. These eight Subway Fold posts cover AI from a number of different perspectives involving a series of different applications and markets.

New Startup’s Legal Research App is Driven by Watson’s AI Technology

"Supreme Court, 60 Centre Street, Lower Manhattan", Image by Jeffrey Zeldman

[New York] “Supreme Court, 60 Centre Street, Lower Manhattan”, Image by Jeffrey Zeldman

May 9, 2016: An update on this post appears below.

Casey Stengel had a very long, productive and colorful career in professional baseball as a player for five teams and later as a manager for four teams. He was also consistently quotable (although not to the extraordinary extent of his Yankee teammate Yogi Berra). Among the many things Casey said was his frequent use of the imperative “You could look it up”¹.

Transposing this gem of wisdom from baseball to law practice², looking something up has recently taken on an entirely new meaning. According to a fascinating article posted on on August 8, 2015 entitled Your Lawyer May Soon Ask for This AI-Powered App for Legal Help by Davey Alba, a startup called ROSS Intelligence has created a unique new system for legal research. I will summarize, annotate and pose a few questions of my own.

One of the founders of ROSS, Jimoh Ovbiagele (@findingjimoh), was influenced by his childhood and adolescent experiences to pursue studying either law or computer science. He chose the latter and eventually ended up working on an artificial intelligence (AI) project at the University of Toronto. It occurred to him then that machine learning (a branch of AI), would be a helpful means to assist lawyers with their daily research requirements.

Mr. Ovbiagele joined with a group of co-founders from diverse fields including “law to computers to neuroscience” in order to launch ROSS Intelligence. The legal research app they have created is built upon the AI capabilities of IBM’s Watson as well as voice recognition. Since June, it has been tested in “small-scale pilot programs inside law firms”.

AI, machine learning, and IBM’s Watson technology have been variously taken up in these nine Subway Fold posts. Among them, the September 1, 2014 post entitled Possible Futures for Artificial Intelligence in Law Practice covered the possible legal applications of IBM’s Watson (prior to the advent of ROSS), and the technology of a startup called Viv Labs.

Essentially, the new ROSS app enables users to ask legal research questions in natural language. (See also the July 31, 2015 Subway Fold post entitled Watson, is That You? Yes, and I’ve Just Demo-ed My Analytics Skills at IBM’s New York Office.) Similar in operation to Apple’s Siri, when a question is verbally posed to ROSS, it searches through its data base of legal documents to provide an answer along with the source documents used to derive it. The reply is also assessed and assigned a “confidence rating”. The app further prompts the user to evaluate the response’s accuracy with an onscreen “thumbs up” or “thumbs down”. The latter will prompt ROSS to produce another result.

Andrew Arruda (@AndrewArruda), another co-founder of ROSS, described the development process as beginning with a “blank slate” version of Watson into which they uploaded “thousands of pages of legal documents”, and trained their system to make use of Watson’s “question-and-answer APIs³. Next, they added machine learning capabilities they called “LegalRank” (a reference to Google’s PageRank algorithm), which, among others things, designates preferential results depending upon the supporting documents’ numbers of citations and the deciding courts’ jurisdiction.

ROSS is currently concentrating on bankruptcy and insolvency issues. Mr. Ovbiagele and Mr. Arruda are sanguine about the possibilities of adding other practice areas to its capabilities. Furthermore, they believe that this would meaningfully reduce the $9.6 billion annually spent on legal research, some of which is presently being outsourced to other countries.

In another recent and unprecedented development, the global law firm Dentons has formed its own incubator for legal technology startups called NextLaw Labs. According to this August 7, 2015 news release on Denton’s website, the first company they have signed up for their portfolio is ROSS Intelligence.

Although it might be too early to exclaim “You could look it up” at this point, my own questions are as follows:

  • What pricing model(s) will ROSS use to determine the cost structure of their service?
  • Will ROSS consider making its app available to public interest attorneys and public defenders who might otherwise not have the resources to pay for access fees?
  • Will ROSS consider making their service available to the local, state and federal courts?
  • Should ROSS make their service available to law schools or might this somehow impair their traditional teaching of the fundamentals of legal research?
  • Will ROSS consider making their service available to non-lawyers in order to assist them in represent themselves on a pro se basis?
  • In addition to ROSS, what other entrepreneurial opportunities exist for other legal startups to deploy Watson technology?

Finally, for an excellent roundup of five recent articles and blog posts about the prospects of Watson for law practice, I highly recommend a click-through to read Five Solid Links to Get Smart on What Watson Means for Legal, by Frank Strong, posted on The Business of Law Blog on August 11, 2015.

May 9, 2016 Update:  The global law firm of Baker & Hostetler, headquartered in Cleveland, Ohio, has become the first US AmLaw 100 firm to announce that it has licensed the ROSS Intelligence’s AI product for its bankruptcy practice. The full details on this were covered in an article posted on May 6, 2016 entitled AI Pioneer ROSS Intelligence Lands Its First Big Law Clients by Susan Beck, on

Some follow up questions:

  • Will other large law firms, as well as medium and smaller firms, and in-house corporate departments soon be following this lead?
  • Will they instead wait and see whether this produces tangible results for attorneys and their clients?
  • If so, what would these results look like in terms of the quality of legal services rendered, legal business development, client satisfaction, and/or the incentives for other legal startups to move into the legal AI space?

1.  This was also the title of one of his many biographies,  written by Maury Allen, published Times Books in 1979.

2.  For the best of both worlds, see the legendary law review article entitled The Common Law Origins of the Infield Fly Rule, by William S. Stevens, 123 U. Penn. L. Rev. 1474 (1975).

3For more details about APIs see the July 2, 2015 Subway Fold post entitled The Need for Specialized Application Programming Interfaces for Human Genomics R&D Initiatives

Watson, is That You? Yes, and I’ve Just Demo-ed My Analytics Skills at IBM’s New York Office


My photo of the entrance to IBM’s office at 590 Madison Avenue in New York, taken on July 29, 2015.

I don’t know if my heart can take this much excitement. Yesterday morning, on July 29, 2015, I attended a very compelling presentation and demo of IBM’s Watson technology. (This AI-driven platform has been previously covered in these five Subway Fold posts.) Just the night before, I saw I saw a demo of some ultra-cool new augmented reality systems.

These experiences combined to make me think of the evocative line from Supernaut by Black Sabbath with Ozzie belting out “I’ve seen the future and I’ve left it behind”. (Incidentally, this prehistoric metal classic also has, IMHO, one of the most infectious guitar riffs with near warp speed shredding ever recorded.)

Yesterday’s demo of Watson Analytics, one key component among several on the platform, was held at IBM’s office in the heart of midtown Manhattan at 590 Madison Avenue and 57th Street. The company very graciously put this on for free. All three IBM employees who spoke were outstanding in their mastery of the technology, enthusiasm for its capabilities, and informative Q&A interactions with the audience. Massive kudos to everyone involved at the company in making this happen. Thanks, too, for all of attendees who asked such excellent questions.

Here is my summary of the event:

Part 1: What is Watson Analytics?

The first two speakers began with a fundamental truth about all organizations today: They have significant quantities of data that are driving all operations. However, a bottleneck often occurs when business users understand this but do not have the technical skills to fully leverage it while, correspondingly, IT workers do not always understand the business context of the data. As a result, business users have avenues they can explore but not the best or most timely means to do so.

This is where Watson can be introduced because it can make these business users self-sufficient with an accessible, extensible and easier to use analytics platform. It is, as one the speakers said “self-service analytics in the cloud”. Thus, Watson’s constituents can be seen as follows:

  • “What” is how to discover and define business problems.
  • “Why” is to understand the existence and nature of these problems.
  • “How” is to share this process in order to affect change.

However, Watson is specifically not intended to be a replacement for IT in any way.

Also, one of Watson’s key capabilities is enabling users to pursue their questions by using a natural language dialog. This involves querying Watson with questions posed in ordinary spoken terms.

Part 2: A Real World Demo Using Airline Customer Data

Taken directly from the world of commerce, the IBM speakers presented a demo of Watson Analytics’ capabilities by using a hypothetical situation in the airline industry. This involved a business analyst in the marketing department for an airline who was given a compilation of market data prepared by a third-party vendor. The business analyst was then assigned by his manager with researching and planning how to reduce customer churn.

Next, by enlisting Watson Analytics for this project, the two central issues became how the data could be:

  • Better understand, leveraged and applied to increase customers’ positive opinions while simultaneously decreasing the defections to the airline’s competitors.
  • Comprehensively modeled in order to understand the elements of the customer base’s satisfaction, or lack thereof, with the airline’s services.

The speakers then put Watson Analytics through its paces up on large screens for the audience to observe and ask questions. The goal of this was to demonstrate how the business analyst could query Watson Analytics and, in turn, the system would provide alternative paths to explore the data in search of viable solutions.

Included among the variables that were dexterously tested and spun into enlightening interactive visualizations were:

  • Satisfaction levels by other peer airlines and the hypothetical Watson customer airline
  • Why customers are, and are not, satisfied with their travel experience
  • Airline “status” segments such as “platinum” level flyers who pay a premium for additional select services
  • Types of travel including for business and vacation
  • Other customer demographic points

This results of this exercise as they appeared onscreen showed how Watson could, with its unique architecture and tool set:

  • Generate “guided suggestions” using natural language dialogs
  • Identify and test all manner of connections among the population of data
  • Use predictive analytics to make business forecasts¹
  • Calculate a “data quality score” to assess the quality of the data upon which business decisions are based
  • Map out a wide variety of data dashboards and reports to view and continually test the data in an effort to “tell a story”
  • Integrate an extensible set of analytical and graphics tools to sift through large data sets from relevant Twitter streams²

Part 3: The Development Roadmap

The third and final IBM speaker outlined the following paths for Watson Analytics that are currently in beta stage development:

  • User engagement developers are working on an updated visual engine, increased connectivity and capabilities for mobile devices, and social media commentary.
  • Collaboration developers are working on accommodating work groups and administrators, and dashboards that can be filtered and distributed.
  • Data connector developers are working on new data linkages, improving the quality and shape of connections, and increasing the degrees of confidence in predictions. For example, a connection to weather data is underway that would be very helpful to the airline (among other industries), in the above hypothetical.
  • New analytics developers are working on new functionality for business forecasting, time series analyses, optimization, and social media analytics.

Everyone in the audience, judging by the numerous informal conversations that quickly formed in the follow-up networking session, left with much to consider about the potential applications of this technology.

1.  Please see these six Subway Fold posts covering predictive analytics in other markets.

2.  Please see these ten Subway Fold posts for a variety of other applications of Twitter analytics.


IBM’s Watson is Now Data Mining TED Talks to Extract New Forms of Knowledge

"sydneytocairns_385", Image by Daniel Dimarco

“sydneytocairns_385”, Image by Daniel Dimarco

Who really benefited from the California Gold Rush of 1849? Was it the miners, only some of whom were successfully, or the merchants who sold them their equipment? Historians have differed as to the relative degree, but they largely believe it was the merchants.

Today, it seems we have somewhat of a modern analog to this in our very digital world: The gold rush of 2015 is populated by data miners and IBM is providing them with access to its innovative Watson technology in order for these contemporary prospectors to discover new forms of knowledge.

So then, what happens when Watson is deployed to sift through the thousands of incredibly original and inspiring videos of online TED Talks? Can the results be such that TED can really talk and, when processed by Watson, yield genuine knowledge with meaning and context?

Last week, the extraordinary results of this were on display at the four-day World of Watson exposition here in New York. A fascinating report on it entitled How IBM Watson Can Mine Knowledge from TED Talks by Jeffrey Coveyduc, Director, IBM Watson, and Emily McManus, Editor, was posted on the TED Blog on May 5, 2015. This was the same day that the newfangled Watson + TED system was introduced at the event. The story also includes a captivating video of a prior 2014 TED Talk by Dario Gil of IBM entitled Cognitive Systems and the Future of Expertise that came to play a critical role in launching this undertaking.

Let’s have a look and see what we can learn from the initial results. I will sum up and annotate this report, and then ask a few additional questions.

One of the key objectives of this new system is to enable users to query it in natural language. An example given in the article is “Will new innovations give me a longer life?”. Thus, users can ask questions about ideas expressed among the full database of TED talks and, for the results, view video excerpts where such ideas have been explored. Watson’s results are further accompanied by a “timeline” of related concepts contained in a particular video clip permitting users to “tunnel sideways” if they wish and explore other topics that are “contextually related”.

The rest of the article is a dialog between the project’s leaders Jeffrey Coveyduc from IBM and editor Emily McManus that took place at Watson World.  They discussed how this new idea was transformed into a “prototype” of a fresh new means to extract “insights” from within “unstructured video”.

Ms. McManus began by recounting how she had attended Mr. Dario’s TED Talk about cognitive computing. Her admiration of his presentation led her to wonder whether Watson could be applied to TED Talks’ full content whereby users would be able to pose their own questions to it in natural language. She asked Mr. Dario if this might be possible.

Mr. Coveyduc said that Mr. Dario then approached him to discuss the proposed project. They agreed that it was not just the content per se, but rather, that TED’s mission of spreading ideas was so compelling. Because one of Watson’s key objectives is to “extract knowledge” that’s meaningful to the user, it thus appeared to be “a great match”.

Ms. McManus mentioned that TED Talks maintains an application programming interface (API) to assist developers in accessing their nearly 2,000 videos and transcripts. She agreed to provide access to TED’s voluminous content to IBM. The company assembled its multidisciplinary project team in about eight weeks.

They began with no preconceptions as to where their efforts would lead. Mr. Coveyduc said they “needed the freedom to be creative”. They drew from a wide range of Watson’s existing technical services. In early iterations of their work they found that “ideas began to group themselves”. In turn, this led them to “new insights” within TED’s vast content base.

Ms. McManus recently received a call from Mr. Dario asking her to stop by his office in New York. He demo-ed the new system which had completely indexed the TED content. Moreover, he showed how it could display, according to her “a universe of concepts extracted” from the content’s core. Next, using the all important natural language capabilities to pose questions, they demonstrated how the results in the form of numerous short clips which, taken altogether, were compiling “a nuanced and complex answer to a big question”, as she described it.

Mr. Coveyduc believes this new system simplifies how users can inspect and inquire about “diverse expertise and viewpoints” expressed in video. He cited other potential areas of exploration such as broadcast journalism and online courses (also known as MOOCs*). Furthermore, the larger concept underlying this project is that Watson can distill the major “ideas and concepts” of each TED Talk and thus give users the knowledge they are seeking.

Going beyond Watson + TED’s accomplishments, he believes that video search remains quite challenging but this project demonstrates it can indeed be done. As a result, he thinks that mining such deep and wide knowledge within massive video libraries may turn into “a shared source of creativity and innovation”.

My questions are as follows:

  • What if Watson was similarly applied to the vast troves of video classes used by professionals to maintain their ongoing license certifications in, among others, law, medicine and accounting? Would new forms of potentially applicable and actionable knowledge emerge that would benefit these professionals as well as the consumers of their services? Rather than restricting Watson to processing the video classes of each profession separately, what might be the results of instead processing them together in various combinations and permutations?
  • What if Watson was configured to process the video repositories of today’s popular MOOC providers  such as Coursera or edX? The same as well for universities around the world who are putting their classes online. Their missions are more or less the same in enabling remote learning across the web in a multitude of subjects. The results could possibly hold new revelations about subjects that no one can presently discern.

Two other recent Subway Fold posts that can provide additional information, resources and questions that I suggest checking out include Artificial Intelligence Apps for Business are Approaching a Tipping Point posted on March 31, 2015, and Three New Perspectives on Whether Artificial Intelligence Threatens or Benefits the World posted on December 27, 2014.

*  See the September 18, 2014 Subway Fold post entitled A Real Class Act: Massive Open Online Courses (MOOCs) are Changing the Learning Process for the full details and some supporting links.

Three New Perspectives on Whether Artificial Intelligence Threatens or Benefits the World

"Gritty Refraction", Image by Mark Oakley

“Gritty Refraction”, Image by Mark Oakley

As the velocity of the rate of change in today’s technology steadily continues to increase, one of the contributing factors behind this acceleration the rise of artificial intelligence (AI). “Smart” attributes and functionalities are being baked into a multitude of systems that are affecting our lives in many visible and, at other times, transparent ways. Just to name one well-known example of an AI-enabled app is Siri, the voice recognition system in the iPhone. Two recent Subway Fold posts have also examined AI’s applications in law (1) and music (2).

However, notwithstanding all of the technological, social and commercial benefits produced by AI, a widespread reluctance, if not fear, of its capabilities to produce negative effects still persists. Will the future produce consequences resembling those in the Terminator or Matrix movie franchises, the “singularity” predicted by Ray Kurzweil where machine intelligence will eventually surpass human intelligence, or perhaps other more benign and productive outcomes?

During the past two weeks, three articles have appeared where their authors have expressed more upbeat outlooks about AI’s potential. They believe that smarter systems are not going to become the world’s new overlords (3) and, moreover, there is a long way to go before computers will ever achieve human-level intelligence or even consciousness. I highly recommend reading them all in their entirety for their rich content, insights and engaging prose.

I will sum up, annotate and comment upon some of the key points in these pieces, which have quite a bit in common in their optimism, analyses and forecasts.

First is a reassuring column by Dr. Gary Marcus, a university professor and corporate CEO, entitled Artificial Intelligence Isn’t a Threat—Yet, that appeared in the December 12, 2014 edition of The Wall Street Journal. While acknowledging the advances in machine intelligence, he still believes that computers today are nowhere near “anything that looks remotely like human intelligence”. However,  computers do not necessarily need to be “superintelligent” to do significant harm such as wild swings in the equities markets resulting from programming errors.(4)

He is not calling for an end to further research and development in AI. Rather, he urges proceeding with caution with safeguards carefully in place focusing upon on the apps access to other networked systems, in areas such as, but not limited to, medicine and autos. Still, the design, implementation and regulation of such “oversight” has yet to be worked out.

Dr. Marcus believes that we might now be overly concerned about any real threats from AI while still acknowledging potential threats from it. He poses questions about levels of transparency and technologies that assess whether AI programs are functioning as intended. Essentially, a form of “infrastructure” should be  in place to evaluate and “control the results” if needed.

Second, is an article enumerating five key reasons why the AI apocalypse is not nearly at hand right now. It is aptly entitled Will Artificial Intelligence Destroy Humanity? Here are Reasons Not to Worry, by Timothy B. Lee, which was posted on on December 19, 2014. The writer asserts that the fears and dangers of AI are far overstated based on his research and interviews with some AI experts. To sum up these factors:

  • Actual “intelligence” is dependent on real world experience such that massive computing power alone will not produce comparable capabilities in machines. The example cited here is studying a foreign language well enough to pass as a native speaker. This involves both book learning and actually speaking with locals in order to include social elements and slang. A computer does not and never will have these experiences nor can they simulate them.
  • Computers, by their very nature, must reply on humans for maintenance, materials, repairs and ultimately, replacement. The current state of robotics development is unable to handle these responsibilities. Quite simply, machines need us and will continue to do so for a long time.
  • Creating a computerized equivalent of a real human’s brain is very tough and remains beyond the reach of today’s circuitry and programming.  Living neurons are indeed quite different in their behaviors and responses than digital devices.  The author cites the modeling of weather simulations as one where progress has been relatively small despite the huge increases in available processing capacity. Moreover, simulating brain activity in the an effort to generate a form of intelligence is relatively far more difficult than modeling weather systems.(5)
  • Relationships, more than intelligence, are needed to acquire power in the real world. Looking at the achievements of recent US presidents, the author states that they gained their achievements by virtue of their networks, personalities and skills at offering rewards and penalties. Thus, machines assist in attaining great technological breakthroughs, but only governments and companies can assemble to capital and resources to implement great projects. Taking this logic further, machines could never take over the world because they utterly lack the capability to work with the large numbers of people needed to even attempt this. (Take that, SkyNet.)
  • Intelligence will become less valuable as its supply increases according to the laws of supply and demand. As the pricing of computing continues to fall, their technological capabilities continues to rise. As the author interprets these market forces, the availability of “super-intelligent computers” will become commoditized and, in turn, produce even more intelligent machines where pricing is competitive. (6)

The third article presents a likewise sanguine view on the future of AI entitled Apocalypse or Golden Age: What Machine Intelligence Will Do to Us, by Patrick Ehlen, was posted on on December 23, 2014. He drew his research from a range of leaders, projects and studies to arrive at similar conclusions that the end of the world as we know it is not at hand because of AI. This piece overlaps with the others on a number of key points. It provides the following additional information and ideas:

  • Well regarded university researchers and tech giants such as Google are pursuing extensive and costly AI research and development programs in conjunction with their ongoing work into such areas as robotics, machine learning, and modeling simple connectomes (see fn.5 below).
  • Unintended bad consequence of well-intentioned research are almost always inevitable. Nonetheless, experts believe that the rate of advancement in this field will continue to accelerate and may well have significant impacts upon the world during the next 20 years.
  • On August 6, 2014, the Pew Internet Research Project published a comprehensive report that was directly on point entitled AI, Robotics, and the Future of Jobs by Aaron Smith and Janna Anderson. This was compiled based on surveys of nearly 1,900 AI experts. To greatly oversimplify the results, while there was largely a consensus view on the progress in this field and ever-increasing integration of AI into numerous areas, there was also a significant split of opinion as the economic,  employment and educational effects of AI in conjunction with robotics. (I highly recommend taking some time to read through this very enlightening report because of its wealth of insights and diversity of perspectives.)
  • Today we are experiencing a “perfect storm” where AI’s progress is further being propelled by the forces of computing power and big data. As a result, we can expect “to create new services that will redefine our expectations”. (7)
  • Certain sectors of our economy will realize greater benefits from the surge in AI than others.(8) This, too, will be likely to cause displacements and realignments in employment in these areas.
  • Changes to relevant social and public policies will be needed in order to successfully adapt to AI-driven effects upon the economy. (This is similar to Dr. Marcus’s views, above, that news forms of safeguards and infrastructure will become necessary.)

I believe that authors Marcus, Lee and Ehlen have all made persuasive cases that AI will continue to produce remarkable new goods, services and markets without any world threatening consequences. Yet they all alert their readers about the unintended and unforeseeable economic and social impacts that likely await us further down the road. My own follow up questions are as follows:

  • Who should take the lead in coordinating the monitoring of these pending changes? Whom should they report to and what, if any, regulatory powers should they have?
  • Will any resulting positive or negative changes attributable to AI be global if and when they manifest themselves, or will they be unevenly distributed in among only certain nations, cities, marketplaces, populations and so on?
  • Is a “negative” impact of AI only in the eye of the beholder? That is, what metrics and analytics exist or need to be developed in order to assess the magnitudes of plus or minus effects? Could such standards be truly objective in their determinations?
  • Assuming that AI development and investment continues to race ahead, will this lead to a possible market/investment bubble or, alternatively, some form of AI Industrial Complex?
  • So, is everyone looking forward to the July 2015 release of Terminator Genisys?


1.  See Possible Futures for Artificial Intelligence in Law Practice posted on September 1, 2014.

2.  See Spotify Enhances Playlist Recommendations Processing with “Deep Learning” Technology posted  on August 14, 2014.

3.  The origin of the popular “I, for one, welcome our new robot overlords” meme originated in the Season 5 episode 15 of The Simpsons entitled Deep Space Homer. (My favorite scene in this ep is where the – – D’oh! – – potato chips are floating all around the spaceship.)

4.   In Flash Boys (W.W. Norton & Company, 2014), renowned author Michael Lewis did an excellent job of reporting on high-speed trading and the ongoing efforts  to reform it. Included is coverage of the “flash crash” in 2010 when errant program trading caused a temporary steep decline in the stock market.

 5For an absolutely fascinating deep and wide analysis of current and future projects to map out all of the billions of connections among the neurons in the human brain, I suggest reading Connectome: How the Brain’s Wiring Makes Us Who We Are (Houghton Mifflin, 2012), by Sebastian Seung.  See also a most interesting column about the work of Dr. Seung and others by James Gorman in the November 10, 2014 edition of The New York Times entitled Learning How Little We Know About the Brain. (For the sake of all humanity, let’s hope these scientists don’t decide to use Homer J. Simpson, at fn.3 above, as a test subject for their work.)

6.  This factor is also closely related to the effects of Moore’s Law which states that the number of transistors that can be packed onto a chip doubles almost doubles in two years (later revised to 18 months). This was originally conceived by Gordon E. Moore, a legendary computer scientists and one of the founders of Intel. This principal has held up for nearly fifty years since it was first published.

7.  This technological convergence is fully and enthusiastically  explored in an excellent article by Kevin Kelly entitled The Three Breakthroughs That Have Finally Unleashed AI on the World in the November 2014 issue of WIRED.

8This seems like a perfect opportunity to invoke the often quoted maxim by master sci-fi and speculative fiction author William Gibson that “The future is already here – it’s just not very evenly distributed.”