Virtual Reality Universe-ity: The Immersive “Augmentarium” Lab at the U. of Maryland

"A Touch of Science", Image by Mars P.

“A Touch of Science”, Image by Mars P.

Got to classes. Sit through a series of 50 minute lectures. Drink coffee. Pay attention and take notes. Drink more coffee. Go to the library to study, do research and complete assignments. Rinse and repeat for the rest of the semester. Then take your final exams and hope that you passed everything. More or less, things have traditionally been this way in college since Hector was a pup.

Might students instead be interested in participating at the new and experimental learning laboratory called the Augmentarium at the University of Maryland where immersing themselves in their studies takes on an entirely new meaning? This is a place where virtual reality (VR)  is being tested and integrated into the learning process. (There 14 Subway Fold posts cover a range of VR and augmented reality [AR] developments and applications.)

Where do I sign up for this?¹

The story was covered in a fascinating report that appeared on December 8, 2015 on the website of the Chronicle of Higher Education entitled Virtual-Reality Lab Explores New Kinds of Immersive Learning, by Ellen Wexler. I highly recommend reading this in its entirety as well as clicking on the Augmentarium link to learn about some these remarkable projects. I also suggest checking out the hashtag #Augmentarium on Twitter the very latest news and developments. I will summarize and annotate this story, and pose some of my own questions right after I take off my own imaginary VR headset.

Developing VR Apps in the Augmentarium

In 2014, Brendan Iribe, the co-founder of the VR headset company Oculus², as well as a University of Maryland alumni, donated $31 million to the University for its development of VR technology³. During the same year, with addition funding obtained from the National Science Foundation, the Augmentarium was built. Currently, researchers at the facility are working on applications of VR to “health care, public safety, and education”.

Professor Ramani Duraiswami, a PhD and co-founder of a startup called VisiSonics (developers of 3D audio and VR gaming systems), is involved with the Augmentarium. His work is in the area of audio, which he believes has a great effect upon how people perceive the world around them. He further thinks that an audio or video lecture presented via distance learning can be greatly enhanced by using VR to, in his words make “the experience feel more immersive”. He feels this would make you feel as though you are in the very presence of the instructor4.

During a recent showcase there, Professor Duraiswami demo-ed 3D sound5 and a short VR science fiction production called Fixing Incus. (This link is meant to be played on a smartphone that is then embedded within a VR viewer/headset.) This implementation showed the audience what it was like to be immersed into a virtual environment where, when they moved their heads and line of sight, what they were viewing corresponding and seamlessly changed.

Enhancing Virtual Immersions for Medicine and Education

Amitabh Varshney, the Director of the University’s Institute for Advanced Computer Studies, is now researching “how the brain processes information in immersive environments” and how is differs from how this is done on a computer screen.6 He believes that VR applications in the classroom will enable students to immerse themselves in their subjects, such as being able to “walk through buildings they design” and “explore” them beyond “just the equations” involved in creating these structures.

At the lab’s recent showcase, he provided the visitors with (non-VR) 3D glasses and presented “an immersive video of a surgical procedure”. He drew the audience’s attention to the doctors at the operating table who were “crowing around” it. He believes that the use of 3D headsets would provide medical students a better means to “move around” and get an improved sense of what this experience is actually like in the operating room. (The September 22, 2015 Subway Fold post entitled VR in the OR: New Virtual Reality System for Planning, Practicing and Assisting in Surgery is also on point and provides extended coverage on this topic.)

While today’s early iterations of VR headsets (either available now or early in 2016), are “cumbersome”, researchers hope that they will evolve (in a manner similar to mobile phones which, in turn and as mentioned above, are presently a key element in VR viewers), and be applied in “hospitals, grocery stores and classrooms”.  Director Varshney can see them possibly developing along an even faster timeline.

My Questions

  • Is the establishment and operation of the Augmentarium a model that other universities should consider as a means to train students in this field, attract donations, and incubate potential VR and AR startups?
  • What entrepreneurial opportunities might exist for consulting, engineering and tech firms to set up comparable development labs at other schools and in private industry?
  • What other types of academic courses would benefit from VR and AR support? Could students now use these technologies to create or support their academic projects? What sort of grading standards might be applied to them?
  • Do the rapidly expanding markets for VR and AR require that some group in academia and/or the government establish technical and perhaps even ethical standards for such labs and their projects?
  • How are relevant potential intellectual property and technology transfer issues going to be negotiated, arbitrated and litigated if needed?

 


1.  Btw, has anyone ever figured out how the very elusive and mysterious “To Be Announced (TBA)”, the professor who appears in nearly all course catalogs, ends up teaching so many subjects at so many schools at the same time? He or she must have an incredibly busy schedule.

2.  These nine Subway Fold posts cover, among other VR and AR related stories, the technology of Oculus.

3.  This donation was reported in an article on September 11, 2014 in The Washington Post in an article entitled Brendan Iribe, Co-founder of Oculus VR, Makes Record $31 Million Donation to U-Md by Nick Anderson.

4.  See also the February 18, 2015 Subway Fold post entitled A Real Class Act: Massive Open Online Courses (MOOCs) are Changing the Learning Process.

5.  See also Designing Sound for Virtual Reality by Todd Baker posted on Medium.com on December 21, 2015, for a thorough overview of this aspect of VR, and the August 5, 2015 Subway Fold post entitled  Latest Census on Virtual Senses: A Seminar on Augmented Reality in New York covering, among other AR technologies, the development work and 3D sound wireless headphones of Hooke Audio.

6.  On a somewhat related topic, see the December 18, 2015 Subway Fold post entitled Mind Over Subject Matter: Researchers Develop A Better Understanding of How Human Brains Manage So Much Information.

Movie Review of “The Human Face of Big Data”

"Blue and Pink Fractal", Image by dev Moore

“Blue and Pink Fractal”, Image by dev Moore

What does big data look like, anyway?

To try to find out, I was very fortunate to have obtained a pass to see a screening of a most enlightening new documentary called The Human Face of Big Data. The event was held on October 20, 2015 at Civic Hall in the Flatiron District in New York.

The film’s executive producer, Rick Smolan, (@ricksmolan), first made some brief introductory remarks about his professional work and the film we were about to see. Among his many accomplishments as a photographer and writer, he was the originator and driving force behind the A Day in the Life series of books where teams of photographers were dispatched to take pictures of different countries for each volume in such places as, among others, the United States, Japan and Spain.

He also added a whole new meaning to a having a hand in casting in his field by explaining to the audience that he had recently fallen from a try on his son’s scooter and hence his right hand was in a cast.

As the lights were dimmed and the film began, someone sitting right in front of me did something that was also, quite literally, enlightening but clearly in the wrong place and at the wrong time by opening up a laptop with a large and very bright screen. This was very distracting so I quickly switched seats. In retrospect, doing so also had the unintentional effect of providing me with a metaphor for the film: From my new perspective in the auditorium, I was seeing a movie that was likewise providing me with a whole new perspective on this important subject.

This film proceeded to provide an engrossing and informative examination of what exactly is “big data”, how it is gathered and analyzed, and its relative virtues and drawbacks.¹ It accomplished all of this by addressing these angles with segments of detailed expositions intercut with interviews of leading experts. In his comments afterwards, Mr. Smolan described big data as becoming a form of “nervous system” currently threading out across our entire planet.

Other documentarians could learn much from his team’s efforts as they smartly surveyed the Big Dataverse while economically compressing their production into a very compact and efficient package. Rather than a paint by, well, numbers production with overly long technical excursions, they deftly brought their subject to life with some excellent composition and editing of a wealth of multimedia content.

All of the film’s topics and transitions between them were appreciable evenhanded. Some segments specifically delved into how big data systems vacuum up this quantum of information and how it positively and negatively affects consumers and other demographic populations. Other passages raised troubling concerns about the loss of personal privacy in recent revelations concerning the electronic operations conducted by the government and the private sector.

I found the most compelling part of the film to be an interview with Dr. Eric Topol, (@EricTopol), a leading proponent of digital medicine, using smart phones as a medical information platform, and empowering patients to take control of their own medical data.² He spoke about the significance of the massive quantities and online availability of medical data and what this transformation  mean to everyone. His optimism and insights about big data having a genuine impact upon the quality of life for people across the globe was representative of this movie’s measured balance between optimism and caution.

This movie’s overall impression analogously reminded me of the promotional sponges that my local grocery used to hand out.  When you returned home and later added a few drops of water to these very small, flat and dried out novelties, they quickly and voluminously expanded. So too, here in just a 52-minute film, Mr. Smolan and his team have assembled a far-reaching and compelling view of the rapidly expanding parsecs of big data. All the audience needed to access, comprehend and soak up all of this rich subject matter was an open mind to new ideas.

Mr. Smolan returned to the stage after the movie ended to graciously and enthusiastically answer questions from the audience. It was clear from the comments and questions that nearly everyone there, whether they were familiar or unfamiliar with big data, had greatly enjoyed this cinematic tour of this subject and its implications. The audience’s well-informed inquiries concerned the following topics:

  • the ethics and security of big data collection
  • the degrees to which science fiction is now become science fact
  • the emergence and implications of virtual reality and augment reality with respect to entertainment and the role of big data in these productions³
  • the effects and influences of big data in medicine, law and other professions
  • the applications of big data towards extending human lifespans

Mr. Smolan also mentioned that his film will be shown on PBS in 2016. When it becomes scheduled, I very highly recommend setting some time aside to view it in its entirety.

Big data’s many conduits, trends, policies and impacts relentlessly continue to extend their global grasp. The Human Face of Big Data delivers a fully realized and expertly produced means for comprehending and evaluating this crucial and unavoidable phenomenon. This documentary is a lot to absorb yet an apt (and indeed fully app-ed), place to start.

 


One of the premiere online resources for anything and everything about movies is IMDB.com. It has just reached its 25th anniversary which was celebrated in a post in VentureBeat.com on October 30, 2015, entitled 25 Years of IMDb, the World’s Biggest Online Movie Database by Paul Sawers.


1These 44 Subway Fold Posts covered many of the latest developments in different fields, marketplaces and professions in the category of Big Data and Analytics.

2.  See also this March 3, 2015 Subway Fold post reviewing Dr. Topol’s latest book, entitled Book Review of “The Patient Will See You Now”.

3These 11 Subway Fold Posts cover many of the latest developments in the arts, sciences, and media industries in the category of Virtual and Augmented Reality. For two of the latest examples, see an article from the October 20, 2015 edition of The New York Times entitled The Times Partners With Google on Virtual Reality Project by Ravi Somaiya, and an article on Fortune.com on September 27, 2015 entitled Oculus Teams Up with 20th Century Fox to Bring Virtual Reality to Movies by Michael Addady. (I’m just speculating here, but perhaps The Human Face of Big Data would be well-suited for VR formatting and audience immersion.)

Prints Charming: A New App Combines Music With 3D Printing

"Totem", Image by Brooke Novak

“Totem”, Image by Brooke Novak

What does a song actually look like in 3D? Everyone knows that music has always been evocative of all kinds of people, memories, emotions and sensations. In a Subway Fold post back on November 30, 2014, we first looked at Music Visualizations and Visualizations About Music. But can a representation of a tune now be taken further and transformed into a tangible object?

Yes, and it looks pretty darn cool. A fascinating article was posted on Wired.com on July 15, 2015, entitled What Songs Look Like as 3-D Printed Sculptures by Liz Stinson, about a new Kickstarter campaign to raise funding for the NYC startup called Reify working on this. I will sum up, annotate and try to sculpt a few questions of my own.

Reify’s technology uses sound waves in conjunction with 3D printing¹ to shape a physical “totem” or object of it. (The Wired article and the Reify website contain pictures of samples.) Then an augmented reality² app in a mobile device will provide an on-screen visual experience accompanying the song when the camera is pointed towards it. This page on their website contains a video of a demo of their system.

The firm is led by Allison Wood and Kei Gowda. Ms. Wood founded it in order to study “digital synesthesia”. (Synthesia is a rare condition where people can use multiple senses in unusual combinations to, for example, “hear” colors, and was previously covered in the Subway Fold post about music visualization linked to above.) She began to explore how to “translate music’s ephemeral nature” into a genuine object and came up with the concept of using a totem.

Designing each totem is an individualized process. It starts with analyzing a song’s “structure, rhythm, amplitude, and more” by playing it through the Echo Nest API.³ In turn, the results generated correspond to measurements including “height, weight and mass”. The tempo and genre of a song also have a direct influence on the shaping of the totem. As well, the musical artists themselves have significant input into the final form.

The mobile app comes into play when it is used to “read” the totem and interpret its form “like a stylus on a record player or a laser on a CD”. The result is, while the music playing, the augmented reality component of the app captures and then generates an animated visualization incorporating the totem on-screen.  The process is vividly shown in the demo video linked above.

Reify’s work can also be likened to a form of information design in the form of data visualization4. According to Ms. Wood, the process involves “translating data from one form into another”.

My questions are as follows:

  • Is Reify working with, or considering working with, Microsoft on its pending HoloLens augmented reality system and/or companies such as Oculus, Samsung and Google on their virtual reality platforms as covered in the posts linked to in Footnote 2 below?
  • How might Reify’s system be integrated into the marketing strategies of musicians? For example, perhaps printing up a number of totems for a band and then distributing them at concerts.
  • Would long-established musicians and performers possibly use Reify to create totems of some their classics? For instance, what might a totem and augmented reality visualization for Springsteen’s anthem, Born to Run, look like?

1.  See these two Subway Fold posts mentioning 3D printing.

2.  See these eight Subway Fold posts covering some of the latest developments in virtual and augmented reality.

3API’s in a medical and scientific context were covered in a July 2, 2015 Subway Fold Post entitled The Need for Specialized Application Programming Interfaces for Human Genomics R&D Initiatives.

4.  This topic is covered extensively in dozens of Subway Fold posts in the Big Data and Analytics and Visualization categories.

Advertisers Looking for New Opportunities in Virtual and Augmented Spaces

"P1030522.JPG", Image by Xebede

“P1030522.JPG”, Image by Xebede

Are virtual reality (VR) and augmented reality (AR) technologies about to start putting up “Place Your Ad Here” signs in their spaces?

Today’s advertising firms and their clients are constantly searching for new venues and the latest technologies with which to compete in evermore specialized global marketplaces. With so many current and emerging alternatives, investing their resources to reach their optimal audiences and targeted demographics requires highly nimble planning and anticipating risks. Effective strategies for both of these factors were recently explored in-depth in the March 22, 2015 Subway Fold post entitled What’s Succeeding Now in Multi-Level Digital Strategies for Companies.

Just such a new venue to add to the media buying mix might soon become virtual worlds. With VR is the early stages of going more mainstream in the news media (see the May 5, 2015 Subway Fold post entitled The New York Times Introduces Virtual Reality Tech into Their Reporting Operations), and even more so in film (see the March 26, 2015 Subway Fold post entitled Virtual Reality Movies Wow Audiences at 2015’s Sundance and SXSW Festivals), it seems inevitable that VR might turn out to be the next frontier for advertising.

This new marketplace will also include augmented reality, involving the projection of virtual/holographic images within a field of view of the real world. Microsoft recently introduced a very sleek-looking headset for this called the HoloLens, which will be part of their release of Windows 10 expected sometime later this year.

A fascinating report on three new startups in this nascent field entitled Augmented Advertising, by Rachel Metz, appeared in the May/June 2015 edition of MIT Technology Review. (Online, the same article is entitled “Virtual Reality Advertisements Get in Your Face”.) I will sum up, annotate and pose a few additional questions to this. As well, I highly recommend clicking through on the links below to these new companies to fully explore all of the resources on their truly innovative and imaginative sites.

As the VR and AR headsets are set to enter the consumer marketplace later in 2015, manufactured by companies including Oculus, Sony, Microsoft (see the above links), Magic Leap and Samsung, consumers will soon be above to experience video games and movie formatted for these new platforms.

The first company in the article working in this space is called Mediaspike. They develop apps and tools for mobile VR. The demo that the writer Metz viewed with a VR headset placed that her in a blimp flying over a city containing billboards for an amusement ride based on the successful movie franchise that began with Despicable Me. The company is developing product placement implementations within these environments using billboards, videos and other methods.

One of the billboards was showing a trailer for the next movie in this series called Minions. While Metz became a bit queasy during this experience (a still common concern for VR users), she nonetheless found it “a heck of a lot more interesting” than the current types of ads seen on websites and mobile.

The second new firm is called Airvertise. They are developing “virtual 3-D models that are integrated with real world locations”. It uses geographic data to create constructs where, as a virtual visitor, you can readily walk around in them. Their first platform will be smartphone apps followed by augmented reality viewers. At the SXSW Festival last March (please see the link again in the third paragraph above to the post about VR at SXSW), the company demo-ed an iPad app that, using the tablet’s motion sensors, produced and displayed a virtual drone “hovering above the air about 20 feet away” with a banner attached to it. As the user/viewer walks closer to it the relative size and spatial orientation of it correspondingly increases.

The third startup is called Blippar. Their AR-based app permits commercial content to be viewed on smartphones. Examples include seeing football players when the phone is held up to a can of Pepsi, and shades of nail polish from the cosmetics company Maybelline. The company is currently strategizing about how to create ads in this manner that will appropriately consumers engage consumers but not put them off in any way.

My questions are as follows:

  • Will VR and AR advertising agencies and sponsors open up this field to user-generated ads and commercial content which has already been successful in a number of ad campaigns for food and cars? Perhaps by open-sourcing their development platforms, crowdsourcing the ads, and providing assistance with such efforts this new advertising space can gain some additional attention and traction.
  • What is exactly about VR and AR experience that will provide the most leverage to advertising agencies and their clients? Is it only limited to the novelty of it – – that might well wear off after a while – – or is there something unique about these technologies that will inform and entertain consumers about goods and services in ways neither previously conceived of nor achieved? Is a critical must-have app or viral ad campaign going to be needed for this to reach a tipping point?
  • Might countering technologies also appear to block VR and AR advertising? For example, Ad Block Plus is a very popular browser add-on that enables users to filter out today’s banner ads and pop-ups online. How might advertisers positively reaction to such avoidance?
  • Just as the leading social media services such as, among others, Facebook (which now owns Oculus), Twitter and Intagram, where advertisers have major presences, do VR and AR similarly lend themselves to being populated by advertisers on such a web-wide scale?

The New York Times Introduces Virtual Reality Tech into Their Reporting Operations

"Mobile World Congress 2015", Image by Jobopa

“Mobile World Congress 2015”, Image by Jobopa

As incredibly vast as New York City is, it has always been a great place to walk around. Its multitude of wonderfully diverse neighborhoods, streets, buildings, parks, shops and endless array of other sites can always be more fully appreciated going on foot here and there in – – as we NYC natives like call it – – “The City”.

The April 26, 2015 edition of The New York Times Magazine was devoted to this tradition. The lead off piece by Steve Duenes was entitled How to Walk in New York.  This was followed by several other pieces and then reports on 15 walks around specific neighborhoods. (Clicking on the Magazine’s link above and then scrolling down to the second and third pages will produce links to nearly all of these articles.) I was thrilled by reading this because I am such an avid walker myself.

The very next day, on May 27, 2015, Wired.com carried a fascinating story about how one of the issues’ accompanying and rather astonishing supporting graphics was actually done in a report by Angela Watercutter entitled How the NY Times is Sparking the VR Journalism Revolution.  But even that’s not the half of it – – the NYTimes has made available for downloading a full virtual reality file of the full construction and deconstruction of the graphic. The Wired.com post contains the link as well as a truly mind-boggling high-speed YouTube video of the graphic’s rapid appearance and disappearance and a screen capture from the VR file itself. (Is “screen capture” really accurate to describe it or is something more like “VR  frame”?)  This could take news reporting into an entirely new dimension where viewers literally go inside of a story.

I will sum up, annotate and pose a few questions about this story. (For another other enthusiastic Subway Fold post about VR, last updated on March 26, 2015, please see Virtual Reality Movies Wow Audiences at 2015’s Sundance and SXSW Festivals.)

This all began on April 11, 2015 when a French artist named JR pieced together and then removed in less than 24 hours, a 150-foot photograph right across the street from the landmark Flatiron Building. This New York Times commissioned image was of “a 20-year-old Azerbaijani immigrant named Elmar Aliyev”. It was used on the cover of this special NYTimes Magazine edition. Upon its completion JR then photographed from a helicopter hovering above. (See the March 19, 2015 Subway Fold post entitled  Spectacular Views of New York, San Francisco and Las Vegas at Night from 7,500 Feet Up for another innovative project inject involving highly advanced photography of New York also taken from a helicopter.)

The NYTimes deployed VR technology from a company called VRSE.tools to transform this whole artistic experience into a fully immersive presentation entitled Walking New York. The paper introduced this new creation at a news conference on April 27th. To summarize the NYTimes Magazine’s editor-in-chief, Jake Silverstein, this project was chosen for a VR implementation because it would so dramatically enhance a viewer’s experience of it. Otherwise, pedestrians walking over the image across the sidewalk would not nearly get the full effect of it.

Viewing Walking New York in full VR mode will require an app from VRSE’s site (linked above), and a VR viewer such as, among others, Google Cardboard.

The boost to VR as an emerging medium by the NYTimes‘ engagement on this project is quite significant. Moreover, this demonstrates how it can now be implemented in journalism. Mr. Silverman, to paraphrase his points of view,  believes this demonstrates how it can be used to literally and virtually bring someone into a story. Furthermore, by doing so, the effect upon the VR viewer is likely to be an increased amount of empathy for certain individuals and circumstances who are the subjects of these more immersive reports.

There will more than likely be a long way to go before “VR filming rigs” can be sent out by news organizations to cover stories as they occur. The hardware is just now that widespread or mainstream yet. As well, the number of people who are trained and know how to use this equipment is still quite small and, even for those who do, preparing such a virtual presentation lags behind today’s pace of news reporting.

Another journalist venturing into VR work is Newsweek reporter Nonny de la Pena’s reconstruction of the shooting in the Trayvon Martin case. (See ‘Godmother of VR’ Sees Journalism as the Future of Virtual Reality by Edward Helmore, posted on The Guardian’s website on March 11, 2015, for in-depth coverage of her innovative efforts.)

Let’s assume that out on the not too distant horizon that VR journalism gains acceptance, its mobility and ease-of-use increases, and the rosters of VR-trained reporters and producers increases so that this field undergoes some genuine economies of scale. Then, as with many other life cycles of emergent technologies, the applications in this nascent field would only become limited by the imaginations by its professionals and their audiences. My questions are as follows:

  • What if the leading social media platforms such as Twitter, Facebook (which already purchased Oculus, the maker of VR headsets for $2B last year),  LinkedIn, Instagram (VR Instgramming, anyone?), and others integrate VR into their capabilities? For example, Twitter has recently added a live video feature called Periscope that its users have quickly and widely embraced. In fact, it is already being used for live news reporting as users turn their phones towards live events as they happen. Would they just as likely equally swarm to VR?
  • What if new startup social media platforms launch that are purely focused on experiencing news, commentary, and discussion in VR?
  • Will previously unanticipated ethical standards be needed and likewise dilemmas result as journalists move up the experience curve with VR?
  • How would the data and analytics firms that parse and interpret social media looking for news trends add VR newsfeeds into their operations and results? (See the Subway Fold posts on January 21, 2015 entitled The Transformation of News Distribution by Social Media Platforms in 2015 and on December 2, 2014 entitled Startup is Visualizing and Interpreting Massive Quantities of Daily Online News Content.)
  • Can and should VR be applied to breaking news, documentaries and news shows such as 60 Minutes? What could be the potential risks in doing so?
  • Can drone technology and VR news gathering be blended into a hybrid flying VR capture platform?

I am also looking forward to seeing what other applications, adaptations and markets for VR journalism will emerge that no one can possibly anticipate at this point.

Virtual Reality Movies Wow Audiences at 2015’s Sundance and SXSW Festivals

Image by mconnors

Image by mconnors

[This post was originally uploaded on December 12, 2014. It has been updated below with new information on December 19, 2014,  January 13, 2015 and March 27, 2015.]

December 12, 2014 Post:

At the 2015 Sundance Film Festival to be held in Park City Utah from January 22, 2015 through February 1, 2015, part of this major annual film event is a program called New Frontier. This year it will be presenting 13 virtual reality (VR) films and “experiences”. Advanced coverage of this event was reported in an article on Wired.com on December 4, 2014 entitled VR Films Are Going to Be All Over Sundance in 2015 by Angela Watercutter. After reading this exciting preview I wanted to immediately pack a bag and start walking there.

To sum up, annotate and comment upon some of the key points in this story, the platforms being used for these presentations will mostly be the Oculus, while Google Cardboard and Samsung’s Gear VR will also deployed. While the Oculus Rift headset has not yet released to the consumer public, developers currently do have had access to it. As a result, they were able to create and format these soon-to-be-premiered experimental works. This year’s offerings are a much deeper and wider lineup than the much more limited sampling of Ocolus-based experiments presented during the 2012 Sundance Festival.

(In a recent Subway Fold post on November 26, 2014 entitled Robots and Diamonds and Drones, Aha! Innovations on the Horizon for 2015, one of the startups briefly mentioned is called Jaunt which is described in the blog post as “… developing an entirely new platform and 360 degree camera to create fully immersive virtual reality movies to be viewed using the versatile new Oculus Rift headset.”)

Attendees at some other recent industry events have responded very favorably to Oculus demonstrations. They included a HBO’s presentation of a Game of Thrones experience at this year’s South by Southwest festival, a Jaeger-piloting simulation ¹ at the 2014 Comic-Con in San Diego , and at the 2014 Electronic Entertainment Expo (E3).

To read what some of the creators involved in Sundance’s VR movies have to say about their creations and some brief descriptions and 2-D graphics of this immersive fare, I very highly recommend clicking through and reading this report in its entirety. They include, among others, news and documentaries, bird flights, travel landscapes, rampaging Kaiju, and several social situations.

I wanna go!

My follow-up questions include:

  • Because VR movie production is entirely digital, can this experience be securely distributed online to other film festival and film schools to share with and, moreover, inspire new VR cinematic works by writers, directors, producers and actors?
  • Can the Hyve-3D virtual development platform covered in this August 28, 2014 Subway Fold Post entitled Hyve-3D: A New 3D Immersive and Collaborative Design System, be adapted and formatted for the cinema so that audiences can be fully immersed in virtual firms without the need for a VR headset?
  • If entertainment companies, movie producers, investors and other supporters line up behind the development and release of VR movies, will this be seen by the public as being more like 3-D movies where the novelty has quickly worn off ², or more like a fundamental shift in movie production, presentation and marketing? What if, using the Oculous Rift, users could experience movie trailers, if the entire film at any location? Would this be a market that might draw the attention of Netflix, Hulu, Amazon, Google and other online content distributors and producers?

____________________________
1.  In another Jaeger and Kaiju-related update, there is indeed good news as reported on June 27, 2014 on the HuffingtonPost.com by Jessica Goodman in a story entitled ‘Pacific Rim 2’ Confirmed For 2017 Release Date.

2.  See 2014 Box Office Will Be Hurt By Diminishing Popularity Of 3D Movies: Analyst by David Lieberman, posted on Deadline.com on February 3, 2014. For other new theater experience innovations, see also To Lure Young, Movie Theaters Shake, Smell and Spritz by Brooks Barnes in the November 29, 2014 edition  of The New York Times.

____________________________

December 19, 2014 Update:

The current release of the movie adaptation of the novel Wild by Cheryl Strayed (Knopf, 2011), has been further formatted into 3-minute supplemental virtual reality movie as reported in the December 15, 2014 edition of The New York Times by Michael Cieply in an article entitled Virtual Reality ‘Wild’ Trek. This short film is also scheduled to be presented at the 2015 Sundance festival. Using Oculus and Samsung VR technology, this is an immersive meeting with the lead character, played by actress Reese Witherspoon, while she is hiking in the wilderness. She is quoted as being very pleased with the final results of this VR production.

January 13, 2015 Update:

While VR’s greatest core ability is in placing viewers within a totally immersive digital  environments, this also presents a challenge in keeping them fully focused upon the main narrative.That is, something happening off to the left or right may draw their attention away and thus detract from the experience.

A startup called Visionary VR has developed a system to reconcile this challenge. It enables creators of VR entertainment to concentrate the viewer’s attention upon the action occurring in the stories and games. This was reported in a most interesting article posted on Recode.com on January 5, 2015 entitled In Virtual Reality Movies, You Are the Camera. That Can Be a Problem, but Here’s One Solution, by Eric Johnson. I believe this will keep your attention as a reader, even in the three dimensions in the real world, and recommend clicking through for all of the details. As well, there is a rather spectacular video presented by the founders of the company on the capabilities of their system.

To recap the key points, Visionary VR creates an invisible boundary around the main narrative that alerts the viewer that they are looking away into other “zones” within the environment. When this occurs, the narrative is suspended but viewers can venture into these interactive peripheral areas and further explore elements of the story. Just as easily, they can return their gaze back to the story which will then re-engage and move forward. Visionary VR has created platform and toolkit for VR authors and storytellers to generate and edit their work while within a virtual environment itself. When viewing the accompanying video, the interface reminded me of something out of Minority Report.

(Btw, it has just been announced that this movie is going to be turned into a TV pilot for Fox according to a story posted on Deadline.com entitled ‘Minority Report’ Gets Fox Pilot Order, by Nellie Andreeva on January 9, 2015. This post also contains a photo from the movie showing this then fictional and now real interface. How cool would it be to see this new pilot in full VR?!)

March 27, 2015 Update:

VR movie technology continues to gather momentum and accolades at 2015’s artistic festivals. Its latest display was held at last week’s (March 13 through 17, 2015) South By Southwest Festival (SXSW). The page for the VR panel and speakers is linked here. Coverage of the event was posted in a very informative and enthusiastic article on VentureBeat.com entitled The Future of Interactive Cinematic VR is Coming, and Fast by Daniel Terdiman, on March 18, 2015.

Those in attendance were truly wowed by what they saw, and, moreover, the potential of fully immersive experiences and storytelling. Please click-through to this story for the full details. I will briefly sum up some of the main points.

The article mostly highlights and highly praises the demo by Jaunt, a startup emerging as one of the innovators in VR movies, mentioned in the initial December 12, 2014 post above. Other VR companies also presented their demos at SXSW.

The Jaunt demo consisted of Paul McCartney playing Live and Let Die in concert. Here’s the link to Jaunt’s Content page containing the stream for this and eight other VR movies (including the Kaiju Fury! film also mentioned in the December 12th post above). In order to immerse yourself in ay of these you will need either an Oculus Rift headset or a Google Cardboard device.

VR movie technology is indeed presenting filmmakers with “opportunities that have not been possible before”. This is likewise so for a range of content creators including, among others sure to come, musicians, athletes, interviewers and documentary makers.

Another panelist, Jason Rubin, the head of worldwide studios for Oculus, spoke about the level of progress being made to make these narrative experiences more genuinely interactive with viewers. He believes this will lead to entirely new forms of cinematic experiences.

Arthur van Hoff, Jaunt’s founder and CTO, stated the possibility of VR films where users can follow one particular actor’s perspective and story within the production. (Visionary VR’s technology, described in the January 13, 2015 Update above, might also be helpful in this regard.)

While new “companies, technologies and investors” in this nascent field are expected, Jaunt believes its current two-year lead will give its technology and productions an advantage.

Updates on Recent Posts Re: Music’s Big Data, Deep Learning, VR Movies, Regular Movies’ Effects on Our Brains, Storytelling and, of Course, Zombies

This week has seen the publication of an exciting series of news stories and commentaries that provide a very timely opportunity to update six recent Subway Fold posts. The common thread running through the original posts and these new pieces is the highly inventive mixing, mutating and monetizing of pop culture and science. Please put on your virtual 3-D glasses let’s see what’s out there.

The December 10, 2014 Subway Fold post entitled Is Big Data Calling and Calculating the Tune in Today’s Global Music Market? explored the apps, companies and trends that have become the key drivers in the current global music business. Adding to the big data strategies and implementations for three more major music companies and their rosters of artists was a very informative report in the December 15, 2014 edition of The Wall Street Journal by Hannah Karp entitled Music Business Plays to Big Data’s Beat. (A subscription for the full text required a subscription to WSJonline.com, but the story also appeared in full on Nasdaq.com clickable here.) As described in detail in this report, Universal Music, Warner Music, and Sony Music have all created sophisticated systems to parse numerous data sources and apply customized analytics for planning and executing marketing campaigns.

Next for an alternative and somewhat retro approach, a veteran music retailer named Sal Nunziato wrote a piece on the Op Ed page of The New York Times on the very same day entitled Elegy for the ‘Suits’. He blamed the Internet more than the music labels for the current state of music where “anyone with a computer, a kazoo and an untuned guitar” can release their music  online regardless of its quality. Thus, the ‘suits’ he nostalgically misses were the music company execs who exerted  more controlled upon the quantity and quality of music available to the public.

Likewise covering the tuning up of another major force in today’s online music streaming industry was an August 14, 2014 Subway Fold post entitled Spotify Enhances Playlist Recommendations Processing with “Deep Learning” Technology. This summarized a report about how deep learning technology was being successfully applied to improve the accuracy and responsiveness of Spotify’s recommendation engine. Presenting an even stronger case that you-ain’t-seen-nothing-yet in this field was an engaging analysis of some still largely unseen developments in deep learning posted on December 15, 2014, on Gigaom.com entitled What We Read About Deep Learning is Just the Tip of the Iceberg by Derrick Harris. These include experimental systems being tested by the likes of Google, Facebook and Microsoft. As well, there were a series of intriguing presentations and demos at the recent Neural Information Processing Systems conference held in Montreal. As detailed here with a wealth of supporting links, many of these advanced systems and methods are expected to gain more press and publicity in 2015.

Returning to the here and now at end of 2014, the current release of the movie adaptation of the novel Wild by Cheryl Strayed (Knopf, 2011), has been further formatted into 3-minute supplemental virtual reality movie as reported in the December 15, 2014 edition of The New York Times by Michael Cieply in an article entitled Virtual Reality ‘Wild’ Trek. This fits right in with the developments covered in the December 10, 2014 Subway Fold post entitled A Full Slate of Virtual Reality Movies and Experiences Scheduled at the 2015 Sundance Film Festival as this short film is also scheduled to be presented at the 2015 Sundance festival. Using Oculus and Samsung VR technology, this is an immersive meeting with the lead character, played by actress Reese Witherspoon, while she is hiking in the wilderness. She is quoted as being very pleased with the final results of this VR production.

The next set of analyses and enhancements to our cinematic experience, continuing right along with the September 3, 2014 Subway Fold post entitled Applying MRI Technology to Determine the Effects of Movies and Music on Our Brains, concerns a newly published book that explains the science of how movies affect our brains entitled Flicker: Your Brain on Movies (Oxford University Press, 2014), by Dr. Jeffrey Zacks. The author was interviewed during a fascinating segment of the December 18, 2014 broadcast of The Brian Lehrer Show on WYNC radio. Among other things, he spoke about why audiences cry during movies (even when the films are not very good), sometimes root for the villain, and move to duck out of the way when an object on the screen seems to be coming right at them such as the giant bolder rolling after Indiana Jones at the start of Raiders of the Lost Ark. Much of this is intentionally done by the filmmakers to manipulate audiences into heightened emotional responses to key events as they unfold on the big screen.

Of course, all movie making involves the art and science of storytelling skills as discussed in the November 4, 2014 Subway Fold post entitled Say, Did You Hear the Story About the Science and Benefits of Being an Effective Storyteller?. In a very practical and insightful article in the December 12, 2014 edition of The New York Times by Alina Tugend entitled Storytelling Your Way to a Better Job or a Stronger Start-Up there are some helpful applications for today’s marketplace. As concisely stated in this piece “You need to have a good story.” It describes in detail how there are now consultants, charging meaningful fees, with new approaches and techniques who assist people in improving their skills in order to become more persuasive storytellers. Among others interviewed for this story was Dr. Paul J. Zak, who wrote the recent article on The Harvard Business Review Blog which was the basis for the November 4th Subway Fold post. It concludes with five helpful pointers to spin a compelling yarn for your listeners.

Finally, the best story told on TV during the 2014 season was – – in a fictional world where brains take on an entirely different significance – –  The Walking Dead on AMC in terms of the extraordinary number of tweets about ongoing adventures Sheriff Rick and the Grimes Gang. This was covered on Nielsen.com on December 15, 2014 in a post entitled Tops of 2014: Social TV.  TWD averaged twice as many tweets as its next competitor in the ongoing series category. This follows up directly with the July 31, 2014 Subway Fold post entitled New Analytical Twitter Traffic Report on US TV Shows During the 2013 – 2014 Season.  As I read scores of TWD tweets on the mid-season finale myself, everyone will miss you, Beth.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As a major fan of TWD, I would like to take the opportunity add my own brief review about the tragic events in Episode 5.8:

I think that in the end, Beth was a form of avatar for the entire show. She traveled many miles from lying on her bed in Season 2 completely unable to function and progressing to Season 5 as a realist concerning herself and the group’s survival. Rather than resigning herself to be held a captive ward in the hospital, she was determined to escape no matter what and was so proud of helping Jonah to escape.

She awakened and arose to be a survivor and a committed member of the Grimes Gang, just as everyone else has done during the past five years. That is, Beth’s journey reflects the entire group’s journey. She, and the Grimes Gang, up to this point have survived all of the threats they faced and endured all of the horrors they have seen. They will all survive but this death with have more serious repercussions than perhaps any other death up until this point. Maggie, Daryl, Rick, Carol and Carl, the core of the GG, will not soon recover from this.

What I still do not understand is why, given that she was finally free in the hospital’s hallway, did she jeopardize her life by going after the lead officer with a scissors. It seemed to be somewhat at odds with Beth’s character as someone who had survived until now on her own determination and close bond with the group. She had nothing to gain by such a reckless act in the middle of a very volatile situation. Was it a sacrifice to save Jonah? Did she realize that the cop was holding a gun at that point? Was she just overtaken by the motivation that desperate times sometimes call for desperate measures?

Consider, too, that she was Herschel’s daughter and her character reflected what she had learned from him: 1. Both learned to see things differently and adapted when the circumstances changed. 2. Both faced sacrifices and danger with great dignity. (Recall Herschel’s acknowledging grin towards Rick right before the Governor murdered the elder of the survivors, and then Beth’s defiant grin when she saw that Jonah had escaped.) 3. Both were resilient insofar as Herschel adapting to the loss of his leg and Beth recovering from her father’s murder. 4. Both sought to comfort others as Herschel stayed with the flu patients and Beth finally drew Daryl out about his terrible family life. Recall also, the three very effective times during her history on the show when Beth’s singing gave great comfort to the others. Indeed, she was a saintly figure but as this story arc wore on, her demise seemed to be foretold.

TWD remains, for me, an absolutely brilliant show in terms of its characters, narrative and presentation.