Along with the exponential growth of big data in terms of its quantity, myriad of collection points, nearly limitless storage capabilities, and complex analytics, investors are keenly interested in discovering unique advantages from this phenomenon to be applied in the securities markets.* While financial institutions of all types have used sophisticated metrics and predictions to gain tactical advantages in their trading operations for many decades, burgeoning big data methodologies have recently created new opportunities for entrepreneurs to provide the financial services industry with ever more original and arcane forms of predictive analytics.
Investors now have data services available to them offering insights never previously feasible or even imaginable. Yesterday’s (November 21, 2014) edition of The Wall Street Journal carried a fascinating report highlighting three of these operations entitled Startups Tip Investors to Hidden Data Pearls by Bradley Hope. (A subscription to the WSJ Online is required for full access to this report on WSJ.com, but this piece was available here in slightly different version on CBS’s Marketwatch.com.) This additional extract page from the article is also available online and contains explanatory graphics of their formats and analyses.
How are these new data points being mined, examined and spun into forecasts? To briefly sum up the work of these startups covered in this article:
- Orbital Insight analyzes satellite photos of building sites in 30 cities in China, cornfields, and parking lots in order to assess how their capacities might influence the markets in various ways. They are seeking to intuit “early indicators” of trends and influences. Their clients include hedge funds.
- Dataminr sifts through a half a billion daily tweets in order to spot potential market moving trends ahead of the news services.** The link above to the graphics from the WSJ article contains a very effective infographic on this process.*** The company’s proprietary systems categorize and analyze all tweets in real time, discerns potentially useful patterns, and then distributes the results to their clients.
- Premise Inc. uses a global system, now in 18 countries, that provides cell phone credits as payments to individuals who monitor the prices of various goods. From this input, the company tracks early inflation rates and other economic data. They believe that their data can differ from official government sources.
I recommend reading this story in full for all of its compelling details.
My follow up questions include:
- Who watches these watchmen? Will market forces determine which of them are producing valid and actionable collection and analytics or should they somehow be subject to regulatory oversight?
- Because these data types and analytics are so new, how are these companies and others like them addressing the distinctions between correlation and causation in their reports to their clients? Would it be beneficial for them to form a trade association to address this and other issues that might arise in the future for this nascent industry?
- Are there entrepreneurial opportunities here for another type of new startups to vet the practices and products of such companies? That is, analysts who produce no new data types themselves, but rather, apply existing and, perhaps develop new, analytical tools for such assessments?
- What other fields, markets and professions might benefit from this trend to discover and assess new data types in addition to finance?
* Please see this April 9, 2014 Subway Fold post entitled Roundup of Some Recent Books on Big Data, Analytics and Intelligent Systems.
** Please see this July 31, 2014 Subway Fold post entitled New Analytical Twitter Traffic Report on US TV Shows During the 2013 – 2014 Season.
*** Please see this January 30, 2015 Subway Fold post entitled Timely Resources for Studying and Producing Infographics.