Prints Charming: A New App Combines Music With 3D Printing

"Totem", Image by Brooke Novak

“Totem”, Image by Brooke Novak

What does a song actually look like in 3D? Everyone knows that music has always been evocative of all kinds of people, memories, emotions and sensations. In a Subway Fold post back on November 30, 2014, we first looked at Music Visualizations and Visualizations About Music. But can a representation of a tune now be taken further and transformed into a tangible object?

Yes, and it looks pretty darn cool. A fascinating article was posted on Wired.com on July 15, 2015, entitled What Songs Look Like as 3-D Printed Sculptures by Liz Stinson, about a new Kickstarter campaign to raise funding for the NYC startup called Reify working on this. I will sum up, annotate and try to sculpt a few questions of my own.

Reify’s technology uses sound waves in conjunction with 3D printing¹ to shape a physical “totem” or object of it. (The Wired article and the Reify website contain pictures of samples.) Then an augmented reality² app in a mobile device will provide an on-screen visual experience accompanying the song when the camera is pointed towards it. This page on their website contains a video of a demo of their system.

The firm is led by Allison Wood and Kei Gowda. Ms. Wood founded it in order to study “digital synesthesia”. (Synthesia is a rare condition where people can use multiple senses in unusual combinations to, for example, “hear” colors, and was previously covered in the Subway Fold post about music visualization linked to above.) She began to explore how to “translate music’s ephemeral nature” into a genuine object and came up with the concept of using a totem.

Designing each totem is an individualized process. It starts with analyzing a song’s “structure, rhythm, amplitude, and more” by playing it through the Echo Nest API.³ In turn, the results generated correspond to measurements including “height, weight and mass”. The tempo and genre of a song also have a direct influence on the shaping of the totem. As well, the musical artists themselves have significant input into the final form.

The mobile app comes into play when it is used to “read” the totem and interpret its form “like a stylus on a record player or a laser on a CD”. The result is, while the music playing, the augmented reality component of the app captures and then generates an animated visualization incorporating the totem on-screen.  The process is vividly shown in the demo video linked above.

Reify’s work can also be likened to a form of information design in the form of data visualization4. According to Ms. Wood, the process involves “translating data from one form into another”.

My questions are as follows:

  • Is Reify working with, or considering working with, Microsoft on its pending HoloLens augmented reality system and/or companies such as Oculus, Samsung and Google on their virtual reality platforms as covered in the posts linked to in Footnote 2 below?
  • How might Reify’s system be integrated into the marketing strategies of musicians? For example, perhaps printing up a number of totems for a band and then distributing them at concerts.
  • Would long-established musicians and performers possibly use Reify to create totems of some their classics? For instance, what might a totem and augmented reality visualization for Springsteen’s anthem, Born to Run, look like?

1.  See these two Subway Fold posts mentioning 3D printing.

2.  See these eight Subway Fold posts covering some of the latest developments in virtual and augmented reality.

3API’s in a medical and scientific context were covered in a July 2, 2015 Subway Fold Post entitled The Need for Specialized Application Programming Interfaces for Human Genomics R&D Initiatives.

4.  This topic is covered extensively in dozens of Subway Fold posts in the Big Data and Analytics and Visualization categories.

Advertisers Looking for New Opportunities in Virtual and Augmented Spaces

"P1030522.JPG", Image by Xebede

“P1030522.JPG”, Image by Xebede

Are virtual reality (VR) and augmented reality (AR) technologies about to start putting up “Place Your Ad Here” signs in their spaces?

Today’s advertising firms and their clients are constantly searching for new venues and the latest technologies with which to compete in evermore specialized global marketplaces. With so many current and emerging alternatives, investing their resources to reach their optimal audiences and targeted demographics requires highly nimble planning and anticipating risks. Effective strategies for both of these factors were recently explored in-depth in the March 22, 2015 Subway Fold post entitled What’s Succeeding Now in Multi-Level Digital Strategies for Companies.

Just such a new venue to add to the media buying mix might soon become virtual worlds. With VR is the early stages of going more mainstream in the news media (see the May 5, 2015 Subway Fold post entitled The New York Times Introduces Virtual Reality Tech into Their Reporting Operations), and even more so in film (see the March 26, 2015 Subway Fold post entitled Virtual Reality Movies Wow Audiences at 2015’s Sundance and SXSW Festivals), it seems inevitable that VR might turn out to be the next frontier for advertising.

This new marketplace will also include augmented reality, involving the projection of virtual/holographic images within a field of view of the real world. Microsoft recently introduced a very sleek-looking headset for this called the HoloLens, which will be part of their release of Windows 10 expected sometime later this year.

A fascinating report on three new startups in this nascent field entitled Augmented Advertising, by Rachel Metz, appeared in the May/June 2015 edition of MIT Technology Review. (Online, the same article is entitled “Virtual Reality Advertisements Get in Your Face”.) I will sum up, annotate and pose a few additional questions to this. As well, I highly recommend clicking through on the links below to these new companies to fully explore all of the resources on their truly innovative and imaginative sites.

As the VR and AR headsets are set to enter the consumer marketplace later in 2015, manufactured by companies including Oculus, Sony, Microsoft (see the above links), Magic Leap and Samsung, consumers will soon be above to experience video games and movie formatted for these new platforms.

The first company in the article working in this space is called Mediaspike. They develop apps and tools for mobile VR. The demo that the writer Metz viewed with a VR headset placed that her in a blimp flying over a city containing billboards for an amusement ride based on the successful movie franchise that began with Despicable Me. The company is developing product placement implementations within these environments using billboards, videos and other methods.

One of the billboards was showing a trailer for the next movie in this series called Minions. While Metz became a bit queasy during this experience (a still common concern for VR users), she nonetheless found it “a heck of a lot more interesting” than the current types of ads seen on websites and mobile.

The second new firm is called Airvertise. They are developing “virtual 3-D models that are integrated with real world locations”. It uses geographic data to create constructs where, as a virtual visitor, you can readily walk around in them. Their first platform will be smartphone apps followed by augmented reality viewers. At the SXSW Festival last March (please see the link again in the third paragraph above to the post about VR at SXSW), the company demo-ed an iPad app that, using the tablet’s motion sensors, produced and displayed a virtual drone “hovering above the air about 20 feet away” with a banner attached to it. As the user/viewer walks closer to it the relative size and spatial orientation of it correspondingly increases.

The third startup is called Blippar. Their AR-based app permits commercial content to be viewed on smartphones. Examples include seeing football players when the phone is held up to a can of Pepsi, and shades of nail polish from the cosmetics company Maybelline. The company is currently strategizing about how to create ads in this manner that will appropriately consumers engage consumers but not put them off in any way.

My questions are as follows:

  • Will VR and AR advertising agencies and sponsors open up this field to user-generated ads and commercial content which has already been successful in a number of ad campaigns for food and cars? Perhaps by open-sourcing their development platforms, crowdsourcing the ads, and providing assistance with such efforts this new advertising space can gain some additional attention and traction.
  • What is exactly about VR and AR experience that will provide the most leverage to advertising agencies and their clients? Is it only limited to the novelty of it – – that might well wear off after a while – – or is there something unique about these technologies that will inform and entertain consumers about goods and services in ways neither previously conceived of nor achieved? Is a critical must-have app or viral ad campaign going to be needed for this to reach a tipping point?
  • Might countering technologies also appear to block VR and AR advertising? For example, Ad Block Plus is a very popular browser add-on that enables users to filter out today’s banner ads and pop-ups online. How might advertisers positively reaction to such avoidance?
  • Just as the leading social media services such as, among others, Facebook (which now owns Oculus), Twitter and Intagram, where advertisers have major presences, do VR and AR similarly lend themselves to being populated by advertisers on such a web-wide scale?