I Can See for Miles: Using Augmented Reality to Analyze Business Data Sets

matrix-1013612__340, Image from Pixabay

While one of The Who’s first hit singles, I Can See for Miles, was most certainly not about data visualization, it still might – – on a bit of a stretch – – find a fitting a new context in describing one of the latest dazzling new technologies in the opening stanza’s declaration “there’s magic in my eye”.  In determining Who’s who and what’s what about all this, let’s have a look at report on a new tool enabling data scientists to indeed “see for miles and miles” in an exciting new manner.

This innovative approach was recently the subject of a fascinating article by an augmented reality (AR) designer named Benjamin Resnick about his team’s work at IBM on a project called Immersive Insights, entitled Visualizing High Dimensional Data In Augmented Reality, posted on July 3, 2017 on Medium.com. (Also embedded is a very cool video of a demo of this system.) They are applying AR’s rapidly advancing technology1 to display, interpret and leverage insights gained from business data. I highly recommend reading this in its entirety. I will summarize and annotate it here and then pose a few real-world questions of my own.

Immersive Insights into Where the Data-Points Point

As Resnick foresees such a system in several years, a user will start his or her workday by donning their AR glasses and viewing a “sea of gently glowing, colored orbs”, each of which visually displays their business’s big data sets2. The user will be able to “reach out select that data” which, in turn, will generate additional details on a nearby monitor. Thus, the user can efficiently track their data in an “aesthetically pleasing” and practical display.

The project team’s key objective is to provide a means to visualize and sum up the key “relationships in the data”. In the short-term, the team is aiming Immersive Insights towards data scientists who are facile coders, enabling them to visualize, using AR’s capabilities upon time series, geographical and networked data. For their long-term goals, they are planning to expand the range of Immersive Insight’s applicability to the work of business analysts.

For example, Instacart, a same-day food delivery service, maintains an open source data set on food purchases (accessible here). Every consumer represents a data-point wherein they can be expressed as a “list of purchased products” from among 50,000 possible items.

How can this sizable pool of data be better understood and the deeper relationships within it be extracted and understood? Traditionally, data scientists create a “matrix of 2D scatter plots” in their efforts to intuit connections in the information’s attributes. However, for those sets with many attributes, this methodology does not scale well.

Consequently, Resnick’s team has been using their own new approach to:

  • Lower complex data to just three dimensions in order to sum up key relationships
  • Visualize the data by applying their Immersive Insights application, and
  • Iteratively label and color-code the data” in conjunction with an “evolving understanding” of its inner workings

Their results have enable them to “validate hypotheses more quickly” and establish a sense about the relationships within the data sets. As well, their system was built to permit users to employ a number of versatile data analysis programming languages.

The types of data sets being used here are likewise deployed in training machine learning systems3. As a result, the potential exists for these three technologies to become complementary and mutually supportive in identifying and understanding relationships within the data as well as deriving any “black box predictive models”.

Analyzing the Instacart Data Set: Food for Thought

Passing over the more technical details provided on the creation of team’s demo in the video (linked above), and next turning to the results of the visualizations, their findings included:

  • A great deal of the variance in Instacart’s customers’ “purchasing patterns” was between those who bought “premium items” and those who chose less expensive “versions of similar items”. In turn, this difference has “meaningful implications” in the company’s “marketing, promotion and recommendation strategies”.
  • Among all food categories, produce was clearly the leader. Nearly all customers buy it.
  • When the users were categorized by the “most common department” they patronized, they were “not linearly separable”. This is, in terms of purchasing patterns, this “categorization” missed most of the variance in the system’s three main components (described above).

Resnick concludes that the three cornerstone technologies of Immersive Insights – – big data, augmented reality and machine learning – – are individually and in complementary combinations “disruptive” and, as such, will affect the “future of business and society”.


  • Can this system be used on a real-time basis? Can it be configured to handle changing data sets in volatile business markets where there are significant changes within short time periods that may affect time-sensitive decisions?
  • Would web metrics be a worthwhile application, perhaps as an add-on module to a service such as Google Analytics?
  • Is Immersive Insights limited only to business data or can it be adapted to less commercial or non-profit ventures to gain insights into processes that might affect high-level decision-making?
  • Is this system extensible enough so that it will likely end up finding unintended and productive uses that its designers and engineers never could have anticipated? For example, might it be helpful to juries in cases involving technically or financially complex matters such as intellectual property or antitrust?


1.  See the Subway Fold category Virtual and Augmented Reality for other posts on emerging AR and VR applications.

2.  See the Subway Fold category of Big Data and Analytics for other posts covering a range of applications in this field.

3.  See the Subway Fold category of Smart Systems for other posts on developments in artificial intelligence, machine learning and expert systems.

4.  For a highly informative and insightful examination of this phenomenon where data scientists on occasion are not exactly sure about how AI and machine learning systems produce their results, I suggest a click-through and reading of The Dark Secret at the Heart of AI,  by Will Knight, which was published in the May/June 2017 issue of MIT Technology Review.

Timely Resources for Studying and Producing Infographics

Image by Nicho Design

Image by Nicho Design

[This post was originally uploaded on October 21, 2014. It has been updated below with new information on January 30, 2015.]

Infographics seem to be appearing in a steadily increasing frequency in many online and print publications. Collectively they are an expressive informational phenomenon where art and data science intersect to produce often strikingly original and informative results. In two previous Subway Fold posts concerning new visual perspectives and covering user data about LinkedIn, I highlighted two examples that struck me as being particularly effective in transforming complex data sets into clear and convincing visual displays.

Recently, I have come across the following resources about inforgraphics I believe are worth exploring:

  • A new book entitled Infographics Designers’ Sketchbooks by authors Steven Heller and Rick Landers is being published today, October 14, 2014, by Princeton Architectural Press. An advanced review, including quotes from the authors, was posted on October 7, 2014 entitled A Behind-the-Scenes Look at How Infographics Are Made on Wired.com by Liz Stinson. To quickly recap this article, the book compiles a multitude of resources, sketches, how-to’s, best practices guidelines, and insights from more than 200 designers of infographics. Based upon the writer’s description, there is much value and motivation to be had within these pages to learn and put to good use the aesthetic and explanatory powers of infographics.
  • DailyInfographic.com provides thousands of exceptional examples of infographics, true to its name updated daily, that are valuable for both the information they present and, moreover, the inspiration they provide to consider trying to design and prepare your own for your online and print efforts. This page on Wikipedia provides an excellent exploration of the evolution and effectiveness of infographics.
  • Edward Tufte is considered to be one of the foremost experts in the visual presentation of data and information and I highly recommend checking out his link rich biography and bibliography page on Wikipedia and more of his work and other offerings on his own site edwardtufte.com.
  • October 15, 2014 UPDATE:  Yesterday, soon after I added this post, I read about the publication of another compilation of the year’s best in this field in US entitled The Best American Infographics 2014 by Gareth Cook (Houghton Mifflin Harcourt). This appeared in an article about the publication of this new book on Scientific American.com in a post there entitled SA Recognized for Great Infographics  by Jen Christiansen. This collection includes two outstanding infographics that have recently appeared in  Scientific American about the locations of wild bees and the increasing levels of caffeine in various drinks, both of which are reproduced on this page. (One location where I would not like to, well, bee, is where these two topics intersect to produce over-caffeinated wild bees. Run!)

Please post any comments here to share examples of infographics that have impressed you or impacted your understanding of particular concepts and information.

January 30, 2015 Update:

Consisely Getting to the heart of succeeding with this web-ubiquitous form of visual display of information is a very practical new column by Sarah Quinn entitled What Makes a Great Infographic? , posted on January 28, 2015, on SocialMediaToday.com.  I highly recommend clicking through and reading it in full for all of its valuable details. I believe it is a timely addition to anyone’s infographic toolkit.

I will briefly sum up, annotate and add some comments to Ms. Quinn’s five elements to get an infographic to potential greatness. (The anagram I have come up to help commit these points to memory by using their first letters is: Try make your effort a GooD ACT):

1.  A Targeted Audience:  Research your audience well so that your infographic becomes a must share for them. As a part of this, focus upon what problem they may have that you can solve for them and use the infographic to provide solutions to it. Further, establish a persona define the ideal audience you intend to reach and then address them. (Personas are often the cornerstones of marketing and content strategy campaigns.)

2.  A Compelling Theme: Your infographic depicts “your story’ and must strongly relate with your brand’s identity.  The representative sample used in this article is entitled “Food Safety at the Grill” which does an effective job of guiding and educating the reader while simultaneously representing the infographic author’s brand.

3.  Actionable DataThis should be thoroughly researched and the numbers threaded throughout the graphical display. In effect, the data should support the solution and/or brand you are presenting.

4.  Awesome GraphicsQuite simply, it must be aesthetically pleasing while presenting the message. Indeed, the graphics’ quality will form an effective narrative. If you are outsourcing this, Ms. Quinn provides seven helpful guidelines to help instruct the graphics contractor.

5.  Powerful Copy:  This is just as important as the display and should include “powerful headlines” is presenting your message. As with the targeted audience in 1. above, so to should the text be compelling enough so that readers will be motivated to share the infographic with others.