Virtual Reality Universe-ity: The Immersive “Augmentarium” Lab at the U. of Maryland

"A Touch of Science", Image by Mars P.

“A Touch of Science”, Image by Mars P.

Got to classes. Sit through a series of 50 minute lectures. Drink coffee. Pay attention and take notes. Drink more coffee. Go to the library to study, do research and complete assignments. Rinse and repeat for the rest of the semester. Then take your final exams and hope that you passed everything. More or less, things have traditionally been this way in college since Hector was a pup.

Might students instead be interested in participating at the new and experimental learning laboratory called the Augmentarium at the University of Maryland where immersing themselves in their studies takes on an entirely new meaning? This is a place where virtual reality (VR)  is being tested and integrated into the learning process. (There 14 Subway Fold posts cover a range of VR and augmented reality [AR] developments and applications.)

Where do I sign up for this?¹

The story was covered in a fascinating report that appeared on December 8, 2015 on the website of the Chronicle of Higher Education entitled Virtual-Reality Lab Explores New Kinds of Immersive Learning, by Ellen Wexler. I highly recommend reading this in its entirety as well as clicking on the Augmentarium link to learn about some these remarkable projects. I also suggest checking out the hashtag #Augmentarium on Twitter the very latest news and developments. I will summarize and annotate this story, and pose some of my own questions right after I take off my own imaginary VR headset.

Developing VR Apps in the Augmentarium

In 2014, Brendan Iribe, the co-founder of the VR headset company Oculus², as well as a University of Maryland alumni, donated $31 million to the University for its development of VR technology³. During the same year, with addition funding obtained from the National Science Foundation, the Augmentarium was built. Currently, researchers at the facility are working on applications of VR to “health care, public safety, and education”.

Professor Ramani Duraiswami, a PhD and co-founder of a startup called VisiSonics (developers of 3D audio and VR gaming systems), is involved with the Augmentarium. His work is in the area of audio, which he believes has a great effect upon how people perceive the world around them. He further thinks that an audio or video lecture presented via distance learning can be greatly enhanced by using VR to, in his words make “the experience feel more immersive”. He feels this would make you feel as though you are in the very presence of the instructor4.

During a recent showcase there, Professor Duraiswami demo-ed 3D sound5 and a short VR science fiction production called Fixing Incus. (This link is meant to be played on a smartphone that is then embedded within a VR viewer/headset.) This implementation showed the audience what it was like to be immersed into a virtual environment where, when they moved their heads and line of sight, what they were viewing corresponding and seamlessly changed.

Enhancing Virtual Immersions for Medicine and Education

Amitabh Varshney, the Director of the University’s Institute for Advanced Computer Studies, is now researching “how the brain processes information in immersive environments” and how is differs from how this is done on a computer screen.6 He believes that VR applications in the classroom will enable students to immerse themselves in their subjects, such as being able to “walk through buildings they design” and “explore” them beyond “just the equations” involved in creating these structures.

At the lab’s recent showcase, he provided the visitors with (non-VR) 3D glasses and presented “an immersive video of a surgical procedure”. He drew the audience’s attention to the doctors at the operating table who were “crowing around” it. He believes that the use of 3D headsets would provide medical students a better means to “move around” and get an improved sense of what this experience is actually like in the operating room. (The September 22, 2015 Subway Fold post entitled VR in the OR: New Virtual Reality System for Planning, Practicing and Assisting in Surgery is also on point and provides extended coverage on this topic.)

While today’s early iterations of VR headsets (either available now or early in 2016), are “cumbersome”, researchers hope that they will evolve (in a manner similar to mobile phones which, in turn and as mentioned above, are presently a key element in VR viewers), and be applied in “hospitals, grocery stores and classrooms”.  Director Varshney can see them possibly developing along an even faster timeline.

My Questions

  • Is the establishment and operation of the Augmentarium a model that other universities should consider as a means to train students in this field, attract donations, and incubate potential VR and AR startups?
  • What entrepreneurial opportunities might exist for consulting, engineering and tech firms to set up comparable development labs at other schools and in private industry?
  • What other types of academic courses would benefit from VR and AR support? Could students now use these technologies to create or support their academic projects? What sort of grading standards might be applied to them?
  • Do the rapidly expanding markets for VR and AR require that some group in academia and/or the government establish technical and perhaps even ethical standards for such labs and their projects?
  • How are relevant potential intellectual property and technology transfer issues going to be negotiated, arbitrated and litigated if needed?

 


1.  Btw, has anyone ever figured out how the very elusive and mysterious “To Be Announced (TBA)”, the professor who appears in nearly all course catalogs, ends up teaching so many subjects at so many schools at the same time? He or she must have an incredibly busy schedule.

2.  These nine Subway Fold posts cover, among other VR and AR related stories, the technology of Oculus.

3.  This donation was reported in an article on September 11, 2014 in The Washington Post in an article entitled Brendan Iribe, Co-founder of Oculus VR, Makes Record $31 Million Donation to U-Md by Nick Anderson.

4.  See also the February 18, 2015 Subway Fold post entitled A Real Class Act: Massive Open Online Courses (MOOCs) are Changing the Learning Process.

5.  See also Designing Sound for Virtual Reality by Todd Baker posted on Medium.com on December 21, 2015, for a thorough overview of this aspect of VR, and the August 5, 2015 Subway Fold post entitled  Latest Census on Virtual Senses: A Seminar on Augmented Reality in New York covering, among other AR technologies, the development work and 3D sound wireless headphones of Hooke Audio.

6.  On a somewhat related topic, see the December 18, 2015 Subway Fold post entitled Mind Over Subject Matter: Researchers Develop A Better Understanding of How Human Brains Manage So Much Information.

Mind Over Subject Matter: Researchers Develop A Better Understanding of How Human Brains Manage So Much Information

"Synapse", Image by Allan Ajifo

“Synapse”, Image by Allan Ajifo

There is an old joke that goes something like this: What do you get for the man who has everything and then where would he put it all?¹ This often comes to mind whenever I have experienced the sensation of information overload caused by too much content presented from too many sources. Especially since the advent of the Web, almost everyone I know has also experienced the same overwhelming experience whenever the amount of information they are inundated with everyday seems increasingly difficult to parse, comprehend and retain.

The multitudes of screens, platforms, websites, newsfeeds, social media posts, emails, tweets, blogs, Post-Its, newsletters, videos, print publications of all types, just to name a few, are relentlessly updated and uploaded globally and 24/7. Nonetheless, for each of us on an individualized basis, a good deal of the substance conveyed by this quantum of bits and ocean of ink somehow still manages to stick somewhere in our brains.

So, how does the human brain accomplish this?

Less Than 1% of the Data

A recent advancement covered in a fascinating report on Phys.org on December 15, 2015 entitled Researchers Demonstrate How the Brain Can Handle So Much Data, by Tara La Bouff describes the latest research into how this happens. I will summarize and annotate this, and pose a few organic material-based questions of my own.

To begin, people learn to identify objects and variations of them rather quickly. For example, a letter of the alphabet, no matter the font or an individual regardless of their clothing and grooming, are always recognizable. We can also identify objects even if the view of them is quite limited. This neurological processing proceeds reliably and accurately moment-by-moment during our lives.

A recent discover by a team of researchers at Georgia Institute of Technology (Georgia Tech)² found that we can make such visual categorizations with less than 1% of the original data. Furthermore, they created and validated an algorithm “to explain human learning”. Their results can also be applied to “machine learning³, data analysis and computer vision4. The team’s full findings were published in the September 28, 2015 issue of Neural Computation in an article entitled Visual Categorization with Random Projection by Rosa I. Arriaga, David Rutter, Maya Cakmak and Santosh S. Vempala. (Dr. Cakmak is from the University of Washington, while the other three are from Georgia Tech.)

Dr. Vempala believes that the reason why humans can quickly make sense of the very complex and robust world is because, as he observes “It’s a computational problem”. His colleagues and team members examined “human performance in ‘random projection tests'”. These measure the degree to which we learn to identify an object. In their work, they showed their test subjects “original, abstract images” and then asked them if they could identify them once again although using a much smaller segment of the image. This led to one of their two principal discoveries that the test subjects required only 0.15% of the data to repeat their identifications.

Algorithmic Agility

In the next phase of their work, the researchers prepared and applied an algorithm to enable computers (running a simple neural network, software capable of imitating very basic human learning characteristics), to undertake the same tasks. These digital counterparts “performed as well as humans”. In turn, the results of this research provided new insight into human learning.

The team’s objective was to devise a “mathematical definition” of typical and non-typical inputs. Next, they wanted to “predict which data” would be the most challenging for the test subjects and computers to learn. As it turned out, they each performed with nearly equal results. Moreover, these results proved that “data will be the hardest to learn over time” can be predicted.

In testing their theory, the team prepared 3 different groups of abstract images of merely 150 pixels each. (See the Phys.org link above containing these images.) Next, they drew up “small sketches” of them. The full image was shown to the test subjects for 10 seconds. Next they were shown 16 of the random sketches. Dr. Vempala of the team was “surprised by how close the performance was” of the humans and the neural network.

While the researchers cannot yet say with certainty that “random projection”, such as was demonstrated in their work, happens within our brains, the results lend support that it might be a “plausible explanation” for this phenomenon.

My Questions

  • Might this research have any implications and/or applications in virtual reality and augment reality systems that rely on both human vision and processing large quantities of data to generate their virtual imagery? (These 13 Subway Fold posts cover a wide range of trends and applications in VR and AR.)
  • Might this research also have any implications and/or applications in medical imaging and interpretation since this science also relies on visual recognition and continual learning?
  • What other markets, professions, universities and consultancies might be able to turn these findings into new entrepreneurial and scientific opportunities?

 


1.  I was unable to definitively source this online but I recall that I may have heard it from the comedian Steven Wright. Please let me know if you are aware of its origin. 

2.  For the work of Georgia’s Tech’s startup incubator see the Subway Fold post entitled Flashpoint Presents Its “Demo Day” in New York on April 16, 2015.

3.   These six Subway Fold posts cover a range of trends and developments in machine learning.

4.   Computer vision was recently taken up in an October 14, 2015 Subway Fold post entitled Visionary Developments: Bionic Eyes and Mechanized Rides Derived from Dragonflies.

Summary of the Media and Tech Preview 2016 Discussion Panel Held at Frankfurt Kurnit in NYC on December 2, 2015

"dtv svttest", Image by Karl Baron

“dtv svttest”, Image by Karl Baron

GPS everywhere notwithstanding, there are still maps on the walls in most buildings that have a red circle somewhere on them accompanied by the words “You are here”. This is to reassure and reorient visitors by giving them some navigational bearings. Thus you can locate where you are at the moment and then find your way forward.

I had the pleasure of attending an expert panel discussion last week, all of whose participants did an outstanding job of analogously mapping where the media and technology are at the end of 2015 and where their trends are heading going into the New Year. It was entitled Digital Breakfast: Media and Tech Preview 2016, was held at the law firm of Frankfurt Kurnit Klein & Selz in midtown Manhattan. It was organized and presented by Gotham Media, a New York based firm engaged in “Digital Strategy, Marketing and Events” as per their website.

This hour and a half presentation was a top-flight and highly enlightening event from start to finish. My gratitude and admiration for everyone involved in making this happen. Bravo! to all of you.

The panelists’ enthusiasm and perspectives fully engaged and transported the entire audience. I believe that everyone there appreciated and learned much from all of them. The participants included:

The following is a summary based on my notes.

Part 1:  Assessments of Key Media Trends and Events in 2015

The event began on an unintentionally entertaining note when one of the speakers, Jesse Redniss, accidentally slipped out his chair. Someone in the audience called out “Do you need a lawyer?”, and considering the location of the conference, the room erupted into laughter.¹

Once the ensuing hilarity subsided, Mr. Goldblatt began by asking the panel for their media highlights for 2015.

  • Ms. Bond said it was the rise of streaming TV, citing Netflix and Amazon, among other industry leaders. For her, this is a time of interesting competition as consumers have increasing control over what they view. She also believes that this is a “fascinating time” for projects and investments in this market sector. Nonetheless, she does not think that cable will disappear.
  • Mr. Kurnit said that Verizon’s purchase of AOL was one of the critical events of 2015, as Verizon “wants to be 360” and this type of move might portend the future of TV. The second key development was the emergence of self-driving cars, which he expects to see implemented within the next 5 to 15 years.
  • Mr. Redniss concurred on Verizon’s acquisition of AOL. He sees other activity such as the combination of Comcast and Universal as indicative of an ongoing “massive media play” versus Google and Facebook. He also mentioned the significance of Nielsen’s Total Audience Measure service.²
  • Mr. Sreenivasan stated that social media is challenging, as indicated by the recent appearance of “Facebook fatigue” affecting its massive user base. Nonetheless, he said “the empire strikes back” as evidenced in their strong financial performance and the recent launch of Chan Zuckerberg LLC to eventually distribute the couple’s $45B fortune to charity. He also sees that current market looking “like 2006 again” insofar as podcasts, email and blogs making it easy to create and distribute content.

Part 2: Today’s Golden Age of TV

Mr. Goldblatt asked the panel for their POVs on what he termed the current “Golden Age of TV” because of the increasing diversity of new platforms, expanding number of content providers and the abundance of original programming. He started off by asking them for their market assessments.

  • Ms. Bond said that the definition of “television” is now “any video content on any screen”. As a ubiquitous example she cited content on mobile platforms. She also noted proliferation of payment methods as driving this market.
  • Mr. Kurnit said that the industry would remain a bit of a “mess” for the next three or four years because of the tremendous volume of original programming, businesses that operate as content aggregators, and pricing differentials. Sometime thereafter, these markets will “rationalize”. Nonetheless, the quality of today’s content is “terrific”, pointing to examples by such media companies as the programs on AMC and HBO‘s Game of Thrones. He also said that an “unbundled model” of content offerings would enable consumers to watch anywhere.
  • Mr. Redniss believes that “mobile transforms TV” insofar as smartphones have become the “new remote control” providing both access to content and “disoverability” of new offerings. He predicted that content would become “monetized across all screens”.
  • Mr. Sreenivasan mentioned the growing popularity of binge-watching as being an important phenomenon. He believes that the “zeitgeist changes daily” and that other changes are being “led by the audience”.

The panel moved to group discussion mode concerning:

  • Consumer Content Options: Ms. Bond asked how will the audience pay for either bundled or unbundled programming options. She believes that having this choice will provide consumers with “more control and options”. Mr. Redniss then asked how many apps or services will consumers be willing to pay for? He predicted that “everyone will have their own channel”. Mr. Kurnit added that he thought there are currently too many options and that “skinny bundles” of programming will be aggregated. Mr. Sreenivasan pointed towards the “Amazon model” where much content is now available but it is also available elsewhere and then Netflix’s offering of 30 original shows. He also wanted to know “Who will watch all of this good TV?”
  • New Content Creation and Aggregation: Mr. Goldblatt asked the panelists whether a media company can be both a content aggregator and a content creator. Mr. Kurnit said yes and Mr. Redniss immediately followed by citing the long-tail effect (statistical distributions in business analytics where there are higher numbers of data points away from the initial top or central parts of the distribution)³. Therefore, online content providers were not bound by the same rules as the TV networks. Still, he could foresee some of Amazon’s and Netflix’s original content ending up being broadcast on them. He also gave the example of Amazon’s House of Cards original programming as being indicative of the “changing market for more specific audiences”. Ultimately, he believes that meeting such audiences’ needs was part of “playing the long game” in this marketplace. 
  • Binge-Watching: Mr. Kurnit followed up by predicting that binge-watching and the “binge-watching bucket” will go away. Mr. Redniss agreed with him and, moreover, talked about the “need for human interaction” to build up audiences. This now takes the form of “superfans” discussing each episode in online venues. For example, he pointed to the current massive marketing campaign build upon finding out the fate of Jon Snow on Games of Thrones.
  • Cord-Cutting: Mr. Sreenivasan believes that we will still have cable in the future. Ms. Bond said that service offerings like Apple TV will become more prevalent. Mr. Kunit said he currently has 21 cable boxes. Mr. Redniss identified himself as more of a cord-shaver who, through the addition of Netflix and Hulu, has reduced his monthly cable bill.

Part 3: Virtual Reality (VR) and Augmented Reality (AR)

Moving on to two of the hottest media topics of the day, virtual reality and augmented reality, the panelist gave their views.

  • Mr. Sreenivasan expressed his optimism about the prospects of VR and AR, citing the pending market launches of the Oculus Rift headset and Facebook 360 immersive videos. The emergence of these technologies is creating a “new set of contexts”. He also spoke proudly of the Metropolitan Museum Media Lab using Oculus for an implementation called Diving Into Pollack (see the 10th project down on this page), that enables users to “walk into a Jackson Pollack painting”.
  • Mr. Kurnit raised the possibility of using Oculus to view Jurassic Park. In terms of movie production and immersion, he said “This changes everything”.
  • Mr. Redniss said that professional sports were a whole new growth area for VR and AR, where you will need “goggles, not a screen”. Mr. Kurnit followed up mentioning a startup that is placing 33 cameras at Major League Baseball stadiums in order to provide 360 degree video coverage of games. (Although he did not mention the company by name, my own Googling indicates that he was probably referring to the “FreeD” system developed by Replay Technologies.)
  • Ms. Bond posed the question “What does this do for storytelling?”4

(See also these 12 Subway Fold posts) for extensive coverage of VR and AR technologies and applications.)

Part 4: Ad-Blocking Software

Mr. Goldblatt next asked the panels for their thoughts about the impacts and economics of ad-blocking software.

  • Mr. Redniss said that ad-blocking apps will affect how advertisers get their online audience’s attention. He thinks a workable alternative is to use technology to “stitch their ads into content” more effectively.
  • Mr. Sreenivasan believes that “ads must get better” in order to engage their audience rather than have viewers looking for means to avoid them. He noted another alternative used on the show Fargo where network programming does not permit them to use fast-forward to avoid ads.
  • Mr. Kurnit expects that ads will be blocked based on the popularity and extensibility of ad-blocking apps. Thus, he also believes that ads need to improve but he is not confident of the ad industry’s ability to do so. Furthermore, when advertisers are more highly motivated because of cost and audience size, they produce far more creative work for events like the NFL Super Bowl.

Someone from the audience asked the panel how ads will become integrated into VR and AR environments. Mr. Redniss said this will happen in cases where this technology can reproduce “real world experiences” for consumers. An example of this is the Cruise Ship Virtual Tours available on Carnival Cruise’s website.

(See also this August 13, 2015 Subway Fold post entitled New Report Finds Ad Blockers are Quickly Spreading and Costing $Billions in Lost Revenue.)

Part 5: Expectations for Media and Technology in 2016

  • Mr. Sreenivasan thinks that geolocation technology will continue to find new applications in “real-life experiences”. He gave as an example the use of web beacons by the Metropolitan Museum.
  • Ms. Bond foresees more “one-to-one” and “one-to-few” messaging capabilities, branded emjois, and a further examination of the “role of the marketer” in today’s media.
  • Mr. Kurnit believes that drones will continue their momentum into the mainstream. He sees the sky filling up with them as they are “productive tools” for a variety of commercial applications.
  • Mr. Redniss expressed another long-term prospect of “advertisers picking up broadband costs for consumers”. This might take the form of ads being streamed to smart phones during NFL games. In the shorter term, he can foresee Facebook becoming a significant simulcaster of professional sporting events.

 


1.  This immediately reminded of a similar incident years ago when I was attending a presentation at the local bar association on the topic of litigating cases involving brain injuries. The first speaker was a neurologist who opened by telling the audience all about his brand new laptop and how it was the latest state-of-the-art-model. Unfortunately, he could not get it to boot up no matter what he tried. Someone from the back of audience then yelled out “Hey doc, it’s not brain surgery”. The place went into an uproar.

2.  See also these other four Subway Fold posts mentioning other services by Nielsen.

3.  For a fascinating and highly original book on this phenomenon, I very highly recommend reading
The Long Tail: Why the Future of Business Is Selling Less of More (Hyperion, 2005), by Chris Anderson. It was also mentioned in the December 10, 2014 Subway Fold post entitled Is Big Data Calling and Calculating the Tune in Today’s Global Music Market?.

4.  See also the November 4, 2014 Subway Fold post entitled Say, Did You Hear the Story About the Science and Benefits of Being an Effective Storyteller?

New Strata-Gem Produces “Q-Carbon” – A Substance Stronger and Brighter than Diamonds

"Diamond", Image by Hisham Alqawsi

“Diamond”, Image by Hisham Alqawsi

There are eight Bruce Springsteen songs specifically containing the word “diamond” in their lyrics.¹ My favorite among them is from his song Better Days containing the line “Your heart like a diamond shone”. It is a highly evocative image from a deeply powerful song that has a special meaning of hope to many of his fans.

This line quickly sprang to mind when I read a fascinating new article posted on Smithsonian.com on December 2, 2015 entitled Weird New Type of Carbon Is Harder (and Brighter) Than Diamond by Maya Wei-Haas. This is a story about scientists at North Carolina State University² who have just announced the fabrication of Q-Carbon, a substance that is harder and brighter than diamonds as well as having magnetic and glowing properties. I will summarize and annotate it, and pose a few carbon-based questions of my own.

While “Q-carbon” surely sounds far less romantic and is not likely to appear in Bruce’s lyrics anytime soon  – – “Your heart like Q-carbon shone”? – – nah, I just don’t think so – – these four adjectives still seem fitting for both Bruce’s rock and this new material’s roll. For now, let’s focus on the latter in its new diamond setting.

It has taken the North Carolina State University team decades to develop and fabricate Q-carbon. Their invention can make diamonds quickly and at room temperature without using traditional methods of creating industrial diamonds by apply high pressure and high heat to carbon. Moreover, their work also led them to develop this additional “new phase of carbon” called Q-carbon. Their findings were published online on November 30, 2015 in the Journal of Applied Physics in a paper entitled Novel Phase of Carbon, Ferromagnetism, and Conversion into Diamond by Jagdish Narayan and Anagh Bhaumik.

According to Mr. Narayan, the key to this new method’s success is in its speed. At normal room temperature, the team applied “extremely short laser pulses” to amorphous carbon (which has no crystal structure). This heated the material to 6,740°F. When this “puddle” was very rapidly cooled, the Q-carbon then formed.

The scientists found it was harder than regular diamonds, exhibited ferroelectric (magnetic) properties, and was able to give off “small amounts of light”.  Among the anticipated applications for Q-carbon are in developing electronic displays, electronic components, and assisting in “understanding the magnetism of other planets”.

Its most current (no pun intended) use is in improving diamond fabrication. By varying the rates at which the carbon “puddles” are cooled, diamonds can be formed into specialized structure including “nanoneedles, microneedles³, nanodots and films”. Other fields such as medicine and abrasives4 may also benefit from this development.

This new methodology is also relatively inexpensive since its uses an existing  laser system otherwise used in eye surgery. It is likewise very fast as it can produce “a carat in about 15 minutes” according to Mr. Narayan. He is also optimistic that the diamonds can be scaled up from their current size of 70 microns by widening the side of the laser’s beam. (The Smithsonian.com article linked to above contains a photo of these new diamonds.)

The team is now working on further understanding and testing the properties of Q-carbon.

My questions are as follows:

  • What other industries and marketplaces might benefit from Q-carbon and/or the innovative methods used to produce it?
  • Are there other new molecular forms of carbon still to be found in addition to Q-carbon given that the last 30 years have brought us the discoveries and development of graphene. nanotubes and buckminsterfullerines?
  • Should Q-carbon be evaluated as a possible metamaterial because it appears to have some of the physical qualities of these substances as described in the April 10, 2015 Subway Fold post entitled The Next Wave in High Tech Materials Science ?

 


A sampling of additional press coverage on this story from the past week includes:


1.  See 10 Bruce Springsteen Songs That Reference Diamonds and Jewelry by Daniel Ford, posted on JCKonline.com on September 11, 2013.

2.  North Carolina State University scientists were also involved in the June 27, 2015 Subway Fold post entitled Medical Researchers are Developing a “Smart Insulin Patch”.

3.  “Microneedles” were also first mentioned in the same post as Footnote 2 above.

4Does this include car alarms that only seem to go off at 4:00 am in the morning in my neighborhood?