Virtual, Augmented, and Hybrid
REALITY
Cathleen Galas
June 2014
CDG Labs/SAP
Virtual Reality
• Virtual reality is all about the creation of a virtual
world that users can interact with. This virtual
world should be designed in such a way that
users would find it difficult to tell the difference
from what is real and what is not. Furthermore,
VR is usually achieved by the wearing of a VR
helmet or goggles similar to the Oculus Rift.
• http://www.techtimes.com/articles/5078/20140406/augmented-reality-vs-virtual-reality-
what-are-the-differences-and-similarities.htm
The Virtual World Framework:
A Foundation for the
Augmented Conversation
• The Virtual World Framework (VWF) is an HTML-5
based virtual worlds environment and virtual worlds
construction environment. The VWF kernel allows
seamless real-time collaboration and communication
between any users that have a modern web browser
on virtually any device.
• developed by David Smith et. al. Lockheed Martin and Department
of Defense
• David A. Smith (computer scientist) (Lockheed Martin Chief Innovation Officer – project oversight, design, architecture), David Easter (LM Engineer -
Development/Architecture), Richard Boyd (Lockheed Martin Virtual world Labs Chief Architect - governance and ecosystem design), Allison Corey (LM Engineer –
development), Scott R. Haynes (LM Engineer - Development), Michael J. Vacirca (LM Engineer - Development), Steven D. Carr (LM Engineer - Development), Cynthia
L. Moore (contractor, OSD(P&R) TRS – design, future specs, governance), Carlton Rosengrant OSD(P&R) TRS – overall Program Manager, Frank C. DiGiovanni
(Director, OSD(P&R)TRS),[9] Coles/RDECOM (Edge interfacing), Robert Chadwick (ADL – code efficiency and applications), and emerging partners as new functions and
libraries are proposed.
VWF: Virtual World Framework
• Collaborative
• Immersive
• Integrated
Demo: Sandbox to build your own environment
• https://virtual.wf/demos.html
• https://virtual.wf/
Immersion
Headset wearer is immersed
in scenarios in 3D virtual
worlds that feel lifelike
VWF Command Center App
• A virtual representation of a joint op center, a
collaborative training classroom.
• https://virtual.wf/demos.html
• GitHub@virtual-world-framework
Virtual Reality
An attendee tries out the Virtuix Oculus Rift setup this month at the Electronic Entertainment
Expo in Los Angeles. CNN June 17, 2014
http://www.cnn.com/2014/06/16/tech/gaming-gadgets/virtual-reality-e3/
Sony’s
“Project Morpheus”
Virtual Reality System
Announced for the PlayStation 4 in March 2014
Oculus Rift
first version 2012
http://en.wikipedia.org/wiki/Oculus_Rift
“The more that virtual world
wraps around you, the more
important it will be for it to move
immediately and exactly to
reflect the motion of your head.
If it doesn’t it leads to something
called “simulator sickless”. And
the culprit is something called
“latency”.”
• Peter Rubin, Senior Editor,
Wired
Oculus Rift 2014
The Inside Story of Oculus Rift and
how Virtual Reality became Reality
May 20, 2014
http://www.wired.com/2014/05/oculus-rift-4/
“It’s only
worthwhile if
you have a
wide view.
Otherwise, it’s
just a
television on
your head.”
--Palmer
Luckey
Oculus has found a way to combine stereoscopic
3-D, 360-degree visuals, and a wide field of
view—along with a supersize dose of
engineering and software magic—it hacks your
visual cortex.
Your brain cannot perceive a difference
between experiencing something on the Rift
and experiencing it in the real world.
“This is the first time that we’ve
succeeded in stimulating parts of the
human visual system directly, I don’t
get vertigo when I watch a video of the
Grand Canyon on TV, but I do
when I stand on a ledge in VR.”
Abrash, the Valve engineer
• “Imagine enjoying a courtside seat at a game,
studying in a classroom of students and
teachers all over the world, or
• consulting with a doctor face-to-face—just by
putting on goggles in your home.” That’s the
true promise of VR: going beyond the idea of
immersion and achieving true presence—the
feeling of actually existing in a virtual space.
• Mark Zuckerberg
Facebook buys Oculus VR for
$2 billion
"This is really a new communication platform,"
Zuckerberg wrote. "By feeling truly present, you
can share unbounded spaces and experiences
with the people in your life. Imagine sharing not
just moments with your friends online, but
entire experiences and adventures.”
• http://techradar.com/news/portable-devices/other-devices/facebook-buys-
oculus-vr-for-2-billion-1237102
• June 24, 2014: released a first of its kind: a 3D
fully spherical virtual reality video player for
your browser.
• Vi Hart, Emily Eifler, Andrea Hawksley, CDG Labs/SAP
Augmented Reality
• blending of virtual reality and real life, as
developers can create images within
applications that blend in with contents in the
real world.
• users are able to interact with virtual contents
in the real world, and are able to distinguish
between the two.
• http://www.techtimes.com/articles/5078/20140406/augmented-reality-vs-virtual-
reality-what-are-the-differences-and-similarities.htm
Difference and similarities
• Both VR and AR immerse the user
• With AR, users continue to be in touch with
the real world while interacting with virtual
objects around them.
• With VR, the user is isolated from the real
world while immersed in a world that is
completely fabricated.
• http://www.techtimes.com/articles/5078/20140406/augmented-
reality-vs-virtual-reality-what-are-the-differences-and-similarities.htm
Simple Augmented Reality
Gesture recognition
Enables humans to communicate with the machine
http://www.engadget.com/gallery/microsoft-surface-surface-and-gesture-based-computing-lands/
Reality-Virtuality Continuum
• Reality-Virtuality Continuum CC BY 3.0
• By Paul Milgram and Fumio Kishino in 1994
Marker Based
AR
QR codes, 2D tags,
Popcode,
Google goggles
Geotagging and Geolocation
• Geotags are GPS
coordinates that
associates media
content to a
location.
• AR applications draw
on tags created by
companies and by
everyday users.
Microsoft’s 2D geotag
MARKERLESS AUGMENTED REALITY PLATFORM
DEVELOPED AT CAMBRIDGE UNIVERSITY
POPCODE
www.popcode.info
AR Furniture Instructions
Shows the user steps to assemble furniture and allows
Inspecting a virtual representation at each stage.
Can be used for navigation and fun!
ARIS
(Augmented Reality and Interactive Storytelling)
Open source authoring platform that enables
users to create place-based or narrative gaming
activities designed for teaching and learning.
ARIS Education Examples
• Middle school students walk the University of Wisconsin-Madison
campus using mobile phones to view footage of Vietnam war
protests that occurred in the same locations.
• Albuquerque students are using the ARIS engine to practice Spanish
language skills by talking with real people and virtual characters
while visiting a local neighborhood.
• The Smithsonian has expressed interest in building an interactive
narrative to help kids relate to artifacts in the collection.
• DOW DAY
•arisgames.org, www.newlearninginstitute.org/digital-media-
programs/community-based-programs/aris-platform-debuts-placework-studio-
program-rolls-out
DOW DAY VIRTUALLY PLACES YOU IN 1967, ON THE CAMPUS OF THE
UNIVERSITY OF WISCONSIN – MADISON, WHEN THE STUDENTS BEGAN A
PROTEST AGAINST THE DOW CHEMICAL CORPORATION FOR MAKING NAPALM
FOR THE WAR. YOU TAKE THE ROLE OF A NEWS REPORTER AND INVESTIGATE
THE DIFFERENT INTERESTS AND PERSPECTIVES OF STUDENTS, POLICE AND
DOW EMPLOYEES.
Image Recognition System
Aurasma AR
Aurasma uses advanced image and pattern
recognition to blend the real-world and GPS
with rich interactive content such as videos and
animations called “Auras”.
Layar AR
A mobile browser that allows a digitally
enhanced view of the real world and
interactive print.
LAYAR
Uses a combination of the mobile
phone’s camera, compass and GPS
data to identify the user’s location and
field of view, retrieve data based on
those geographical coordinates, and
overlay that data over the
camera view.
Layar
Experience history, art, how things looked
15 years ago
Google co-founder Sergey Brin models the
Google Glass project
during Google I/O in June 2012.
Virgin Atlantic will permanently implement
Google Glass at London Heathrow airport
following a successful six-week trial.
Drchrono AR
• a complete healthcare EHR and EMR platform,
with medical billing, and a patient portal.
• Includes drawing tools, referrals, labs, alerts,
photos, and videos.
Drchrono
enables doctors to record and save surgeries
and consultations as videos
Start-up Drchrono has launched its “wearable health record” Google Glass app
Innovega:
Augmented-reality contacts
Contact lenses interact with full HD glasses.
AR Glasses Future?
If all goes as planned, people may
eventually be able to watch HD movies or
become fully immersed in a video game
entirely through a pair of lightweight
eyeglasses that make it seem like you're
watching a 240-inch TV.
http://www.cnet.com/news/future-vision-wearable-tech-that-requires-fda-approval/
Hybrid Reality
• Also called Mixed reality
• Encompasses both augmented reality and
augmented virtuality
• Merges real and virtual world to produce new
environments and visualizations where
physical and digital objects co-exist and
interact in real time.
• http://en.wikipedia.org/wiki/Mixed_reality
Reality-Virtuality Continuum
• Reality-Virtuality Continuum CC BY 3.0
• By Paul Milgram and Fumio Kishino in 1994
Reality-virtuality continuum
The area between the two extremes, where
both the real and the virtual are mixed, is the
so-called mixed reality. This consists of both
augmented reality, where the virtual augments
the real, and augmented virtuality, where the
real augments the virtual.
• http://en.wikipedia.org/wiki/Virtuality_Continuum
The Hybrid Age
•a new sociotechnical era
unfolding as technologies merge
and humans merge with
technology.
•technology is ubiquitious (trillions
of sensors coating the
environment)
•technology is intelligent(devices
communicating with each other
and us)
•technology is social (encouraging
us to have relationships with it)
Hybrid Reality: Thriving in the Emerging
Human-Technology Civilization
Technology no longer just processes our
instruction; it has its own agency, and we
respond to it as much as it responds to us. What
this means for societies and individuals, as well
as communities and nations, is truly
world changing.
How will we respond and adapt?
http://www.amazon.com/dp/B0085BLPW8
Augmented & Hybrid Reality
Affordances for learning?
• Immersive
• Gives students an up close look at objects
• Creative platforms
• Makes learning visible
• Increases engagement, motivation
Hybrid Reality
Learning Affordances BEYOND AR
• Makes the physical location more relevant to the task
• Bridging of physical and digital spaces
• Fosters collaboration by making them CRUCIAL to the
tasks at hand
• Forces players to look at familiar spaces from an
unfamiliar perspective
• Looks at classroom content from a different viewpoint
using social, experiential and situated learning
• De Souza e Silva, A., Delacruz, G.C., “Hybrid Reality Games Reframed”, Games and Culture, Volume
1, Number 3, 2006. http://gac.sagepub.com/content/1/3/231.abstract

Virtual, Augmented, and Hybrid Reality

  • 1.
    Virtual, Augmented, andHybrid REALITY Cathleen Galas June 2014 CDG Labs/SAP
  • 2.
    Virtual Reality • Virtualreality is all about the creation of a virtual world that users can interact with. This virtual world should be designed in such a way that users would find it difficult to tell the difference from what is real and what is not. Furthermore, VR is usually achieved by the wearing of a VR helmet or goggles similar to the Oculus Rift. • http://www.techtimes.com/articles/5078/20140406/augmented-reality-vs-virtual-reality- what-are-the-differences-and-similarities.htm
  • 3.
    The Virtual WorldFramework: A Foundation for the Augmented Conversation • The Virtual World Framework (VWF) is an HTML-5 based virtual worlds environment and virtual worlds construction environment. The VWF kernel allows seamless real-time collaboration and communication between any users that have a modern web browser on virtually any device. • developed by David Smith et. al. Lockheed Martin and Department of Defense • David A. Smith (computer scientist) (Lockheed Martin Chief Innovation Officer – project oversight, design, architecture), David Easter (LM Engineer - Development/Architecture), Richard Boyd (Lockheed Martin Virtual world Labs Chief Architect - governance and ecosystem design), Allison Corey (LM Engineer – development), Scott R. Haynes (LM Engineer - Development), Michael J. Vacirca (LM Engineer - Development), Steven D. Carr (LM Engineer - Development), Cynthia L. Moore (contractor, OSD(P&R) TRS – design, future specs, governance), Carlton Rosengrant OSD(P&R) TRS – overall Program Manager, Frank C. DiGiovanni (Director, OSD(P&R)TRS),[9] Coles/RDECOM (Edge interfacing), Robert Chadwick (ADL – code efficiency and applications), and emerging partners as new functions and libraries are proposed.
  • 4.
    VWF: Virtual WorldFramework • Collaborative • Immersive • Integrated Demo: Sandbox to build your own environment • https://virtual.wf/demos.html • https://virtual.wf/
  • 5.
    Immersion Headset wearer isimmersed in scenarios in 3D virtual worlds that feel lifelike
  • 6.
    VWF Command CenterApp • A virtual representation of a joint op center, a collaborative training classroom. • https://virtual.wf/demos.html • GitHub@virtual-world-framework
  • 7.
    Virtual Reality An attendeetries out the Virtuix Oculus Rift setup this month at the Electronic Entertainment Expo in Los Angeles. CNN June 17, 2014 http://www.cnn.com/2014/06/16/tech/gaming-gadgets/virtual-reality-e3/
  • 8.
    Sony’s “Project Morpheus” Virtual RealitySystem Announced for the PlayStation 4 in March 2014
  • 9.
    Oculus Rift first version2012 http://en.wikipedia.org/wiki/Oculus_Rift
  • 10.
    “The more thatvirtual world wraps around you, the more important it will be for it to move immediately and exactly to reflect the motion of your head. If it doesn’t it leads to something called “simulator sickless”. And the culprit is something called “latency”.” • Peter Rubin, Senior Editor, Wired
  • 11.
  • 12.
    The Inside Storyof Oculus Rift and how Virtual Reality became Reality May 20, 2014 http://www.wired.com/2014/05/oculus-rift-4/
  • 13.
    “It’s only worthwhile if youhave a wide view. Otherwise, it’s just a television on your head.” --Palmer Luckey
  • 14.
    Oculus has founda way to combine stereoscopic 3-D, 360-degree visuals, and a wide field of view—along with a supersize dose of engineering and software magic—it hacks your visual cortex. Your brain cannot perceive a difference between experiencing something on the Rift and experiencing it in the real world.
  • 15.
    “This is thefirst time that we’ve succeeded in stimulating parts of the human visual system directly, I don’t get vertigo when I watch a video of the Grand Canyon on TV, but I do when I stand on a ledge in VR.” Abrash, the Valve engineer
  • 16.
    • “Imagine enjoyinga courtside seat at a game, studying in a classroom of students and teachers all over the world, or • consulting with a doctor face-to-face—just by putting on goggles in your home.” That’s the true promise of VR: going beyond the idea of immersion and achieving true presence—the feeling of actually existing in a virtual space. • Mark Zuckerberg
  • 17.
    Facebook buys OculusVR for $2 billion "This is really a new communication platform," Zuckerberg wrote. "By feeling truly present, you can share unbounded spaces and experiences with the people in your life. Imagine sharing not just moments with your friends online, but entire experiences and adventures.” • http://techradar.com/news/portable-devices/other-devices/facebook-buys- oculus-vr-for-2-billion-1237102
  • 18.
    • June 24,2014: released a first of its kind: a 3D fully spherical virtual reality video player for your browser. • Vi Hart, Emily Eifler, Andrea Hawksley, CDG Labs/SAP
  • 21.
    Augmented Reality • blendingof virtual reality and real life, as developers can create images within applications that blend in with contents in the real world. • users are able to interact with virtual contents in the real world, and are able to distinguish between the two. • http://www.techtimes.com/articles/5078/20140406/augmented-reality-vs-virtual- reality-what-are-the-differences-and-similarities.htm
  • 22.
    Difference and similarities •Both VR and AR immerse the user • With AR, users continue to be in touch with the real world while interacting with virtual objects around them. • With VR, the user is isolated from the real world while immersed in a world that is completely fabricated. • http://www.techtimes.com/articles/5078/20140406/augmented- reality-vs-virtual-reality-what-are-the-differences-and-similarities.htm
  • 23.
  • 24.
    Gesture recognition Enables humansto communicate with the machine http://www.engadget.com/gallery/microsoft-surface-surface-and-gesture-based-computing-lands/
  • 26.
    Reality-Virtuality Continuum • Reality-VirtualityContinuum CC BY 3.0 • By Paul Milgram and Fumio Kishino in 1994
  • 27.
    Marker Based AR QR codes,2D tags, Popcode, Google goggles
  • 28.
    Geotagging and Geolocation •Geotags are GPS coordinates that associates media content to a location. • AR applications draw on tags created by companies and by everyday users.
  • 29.
  • 30.
    MARKERLESS AUGMENTED REALITYPLATFORM DEVELOPED AT CAMBRIDGE UNIVERSITY POPCODE www.popcode.info
  • 31.
    AR Furniture Instructions Showsthe user steps to assemble furniture and allows Inspecting a virtual representation at each stage.
  • 32.
    Can be usedfor navigation and fun!
  • 33.
    ARIS (Augmented Reality andInteractive Storytelling) Open source authoring platform that enables users to create place-based or narrative gaming activities designed for teaching and learning.
  • 34.
    ARIS Education Examples •Middle school students walk the University of Wisconsin-Madison campus using mobile phones to view footage of Vietnam war protests that occurred in the same locations. • Albuquerque students are using the ARIS engine to practice Spanish language skills by talking with real people and virtual characters while visiting a local neighborhood. • The Smithsonian has expressed interest in building an interactive narrative to help kids relate to artifacts in the collection. • DOW DAY •arisgames.org, www.newlearninginstitute.org/digital-media- programs/community-based-programs/aris-platform-debuts-placework-studio- program-rolls-out
  • 35.
    DOW DAY VIRTUALLYPLACES YOU IN 1967, ON THE CAMPUS OF THE UNIVERSITY OF WISCONSIN – MADISON, WHEN THE STUDENTS BEGAN A PROTEST AGAINST THE DOW CHEMICAL CORPORATION FOR MAKING NAPALM FOR THE WAR. YOU TAKE THE ROLE OF A NEWS REPORTER AND INVESTIGATE THE DIFFERENT INTERESTS AND PERSPECTIVES OF STUDENTS, POLICE AND DOW EMPLOYEES.
  • 36.
  • 37.
    Aurasma AR Aurasma usesadvanced image and pattern recognition to blend the real-world and GPS with rich interactive content such as videos and animations called “Auras”.
  • 39.
    Layar AR A mobilebrowser that allows a digitally enhanced view of the real world and interactive print.
  • 40.
    LAYAR Uses a combinationof the mobile phone’s camera, compass and GPS data to identify the user’s location and field of view, retrieve data based on those geographical coordinates, and overlay that data over the camera view.
  • 41.
    Layar Experience history, art,how things looked 15 years ago
  • 42.
    Google co-founder SergeyBrin models the Google Glass project during Google I/O in June 2012.
  • 43.
    Virgin Atlantic willpermanently implement Google Glass at London Heathrow airport following a successful six-week trial.
  • 44.
    Drchrono AR • acomplete healthcare EHR and EMR platform, with medical billing, and a patient portal. • Includes drawing tools, referrals, labs, alerts, photos, and videos.
  • 45.
    Drchrono enables doctors torecord and save surgeries and consultations as videos Start-up Drchrono has launched its “wearable health record” Google Glass app
  • 46.
  • 47.
    AR Glasses Future? Ifall goes as planned, people may eventually be able to watch HD movies or become fully immersed in a video game entirely through a pair of lightweight eyeglasses that make it seem like you're watching a 240-inch TV. http://www.cnet.com/news/future-vision-wearable-tech-that-requires-fda-approval/
  • 48.
    Hybrid Reality • Alsocalled Mixed reality • Encompasses both augmented reality and augmented virtuality • Merges real and virtual world to produce new environments and visualizations where physical and digital objects co-exist and interact in real time. • http://en.wikipedia.org/wiki/Mixed_reality
  • 49.
    Reality-Virtuality Continuum • Reality-VirtualityContinuum CC BY 3.0 • By Paul Milgram and Fumio Kishino in 1994
  • 50.
    Reality-virtuality continuum The areabetween the two extremes, where both the real and the virtual are mixed, is the so-called mixed reality. This consists of both augmented reality, where the virtual augments the real, and augmented virtuality, where the real augments the virtual. • http://en.wikipedia.org/wiki/Virtuality_Continuum
  • 51.
    The Hybrid Age •anew sociotechnical era unfolding as technologies merge and humans merge with technology. •technology is ubiquitious (trillions of sensors coating the environment) •technology is intelligent(devices communicating with each other and us) •technology is social (encouraging us to have relationships with it)
  • 52.
    Hybrid Reality: Thrivingin the Emerging Human-Technology Civilization Technology no longer just processes our instruction; it has its own agency, and we respond to it as much as it responds to us. What this means for societies and individuals, as well as communities and nations, is truly world changing. How will we respond and adapt? http://www.amazon.com/dp/B0085BLPW8
  • 53.
    Augmented & HybridReality Affordances for learning? • Immersive • Gives students an up close look at objects • Creative platforms • Makes learning visible • Increases engagement, motivation
  • 54.
    Hybrid Reality Learning AffordancesBEYOND AR • Makes the physical location more relevant to the task • Bridging of physical and digital spaces • Fosters collaboration by making them CRUCIAL to the tasks at hand • Forces players to look at familiar spaces from an unfamiliar perspective • Looks at classroom content from a different viewpoint using social, experiential and situated learning • De Souza e Silva, A., Delacruz, G.C., “Hybrid Reality Games Reframed”, Games and Culture, Volume 1, Number 3, 2006. http://gac.sagepub.com/content/1/3/231.abstract

Editor's Notes

  • #26 Many homes have gesture recognition devices such as WII or Kinect.
  • #28 QR=Quick Response, 2D tags is Microsofts own tag:More advanced marker based AR involves image recognition with or without a custom marker.The following applications are examples of more advanced marker based AR.
  • #30 Microsofts 2D code system