Augmented Human 2018

Augmented Human:
Augmented Reality & Beyond
Woontack Woo (禹雲澤), Ph.D.
KI-ITC ARRC & GSCT UVR Lab
KAIST
ACMMM_AltMM @ KAIST, Oct. 21. 2018
Augmented Humans As A New Medium

Review VR/AR in 2017-2018
VR/AR as a New Medium
2012-2017 © Woo, KAIST GSCT UVR Lab., Daejeon 34141, Korea

Augmented City

Human Augmentation

Multi-Media Alternate Realities allow Us
To access other worlds, to live other's stories, to
communicate with or experience alternate realities
To enter into different spaces, times or situations
To experience immersion, e.g., VR/AR
 VR vs. AR
(out there) VR puts people in a
computer-generated world
•To immerse yourself in a computer-generated
virtual space and interact with it in 5+ senses
(here & now) AR puts virtual content
in a physical world
•To interact with VR, feeling reality in real life.

Multi-Media Alternate Realities!
VR, AR, Ubiquitous VR, and Augmented Human
Augmented
Reality
• WW Board
• Metaverse
• UVR
Augmented
Human
• Body+
• Brain+
• Social+
Ubiquitous
Computing
• IoT, IoE
• Context-
aware UI
Wearable
Robotics
• Cyborg
• Bionics,
• Implantable,
Virtual Augmentation
Physical Augmentation
HumanEnvironment

Where are We Now? [ISMAR2018]
(H. Fuchs) VR/AR Technologies?
•MM, CV, CG, HCI, AI, Authoring are almost ready?
•Killer Apps and platforms(HW, social) are in early stage!
(H. Benko) Compelling VR/AR Experiences?
•Adaptive, believable, computational, personalized UIs
(Woo) In addition to core-Tech & smart UI
•Context-aware information augmentation, instead of immersive CG!
What else?
MM alternate realities!
MM alternate humans?

VR/AR : What Should We Do?
Changing thoughts on VR/AR!
•Core Technologies are ready: CV, CG, HCI, IoT/BD, AI-UI, etc.
• Content authoring with Vuforia, ARCore, ARKit, Sumerian, etc.
•Then, move from ‘WoW Content’ to ‘Social UX Platform’
• Social UX (awa information/knowledge) capturing/sharing on 3D Maps
Remaining Challenges: Platform and eco-system
•To build Eco-System to create/experience VR/AR and share UX
AH, a new viewpoint on VR/AR!
•From ‘AR’ to ‘Augmented Human’
• To enhance human’s quality of life
by augmenting Body/Brain, Mind,
Social Relationship, etc.)
• To symbiose with personal companion
utilizing Holistic QS and e-Personality
Take-home Message

Content
VR/AR comes Back!
Ubiquitous VR as AR2.0: Beyond VR/AR
UVR: Main Technical Challenges
Augmented Human: What about Human?
What Should We Do?

Media are …
Human’s Sensory Extension [Marshall McLuhan ’64]
Personal Media Trends
Smaller, cheaper, faster, smarter, more intimate
•Desktop, Laptop, Mobile and many problems…
What’s Next? Wearable? Then wearable AR?
http://goo.gl/K6xAIA
Personal New Media

New Trend: Personal Computing
Wearables: From ‘HC Interface’ to ‘HC Integration’
R-V Convergence: From IoT to ‘Internet of Everything’
Smart Companion: From AI to ‘Assistive Intelligence’
How do Humans evolve with Personal Media?
Personal New Media and Human
Inevitable
C
R
C
R
C
R
C C C
C

Where are New Media going?
Milgram's Reality-Virtuality Continuum [94]
Azuma's Definition on AR [97]
combines real and virtual
is interactive in real time
is registered in real 3D world
R. Azuma, A Survey of Augmented Reality, Presence, Vol.6, No.4, Aug. 1997, pp. 355-385.
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays, IEICE Trans. on I&S, E77-D(12), pp. 1321-1329, 1994.

VR/AR comes Back!
VR in 1960s, 1980s, 1990s, and 2010s

VR was Hot in 1990s!
VR/AR is Hot Again! Why?
Where are New Media going?
AR1.0 (Desktop, 1990s-)
Augment Real
Objects with Virtual
Content
AR2.0 (Mobile, 2000s-)
Augment Real Space
with Content
AR3.0 (Smart phone, 2010s-)
Augment Real Place
with Dynamic
(Social) Content
AR4.0 (Glass, 2020s-)
Augment Human
with Context-aware
(Social) Content

Google Tango (Lenovo Phab 2 Pro)
AR as a DigiLog Medium
Visual Positioning
Service
A-commerce by GAP

Apple ARKit
Google ARCore

Multi-platform AR
Social AR with Apple ARKit2.0
Sharing with Google ARCore1.5
•ARCore Cloud Anchor Test
(iOS + Android)

Amazon Sumerian

Aero Project by Adobe

HoloLens: Augmented Robot

HoloLens: Augmented Learning

Magic Leap One

DAQRI

Meta 2 Collaboration

Facebook F8, 2017

Facebook F8, 2018
Social VR, bringing VR mainstream
•Oculus Go Available Now
•New Apps for Shared VR Experiences
•More Immersive Ways to Share
 More AR
•to Instagram, Messenger, Adds Tools

Beyond AR/VR
World Board, Metaverse, 2nd Life, etc.
Ubiquitous Virtual Reality
•VR in ubiComp Environment
•Context-aware MAR
UVR as a DigiLog Medium
UVR
Measured
Space
Physical
Space
Virtual
Space
SensingModeling
Augmentation
& Interaction
World Board [’99]:
Information in places
by J. C. Spohrer (IBM)

Mirror World Concept by Leap Motion
AVR continuum

How to Achieve UVR in DigiLog Era [Woo01]
3D Link with IoT/IoE between twin (real & virtual) worlds
CoI (Context-of-Interest) Augmentation with Data, not just
sight (sound, haptics, smell, taste, etc.) but information
Bidirectional Interaction/Collaboration with Intelligence for
O2O/H2H Communications
SPACES (3rd skin)
Intimate/Private
Personal
Social
General/public
Real
Space Social Networks
Seamless
Augmentation btw
dual spaces
How
to
Link
Seaml
essly?
CoI
LIN
K
U-
Content
Virtual space

UVR, Beyond VR/AR
What I have Worked… since 2001

Holistic Layers of Physical Spaces/Places
.
.
.
.
.
.
Real/Physical
Virtual/Cyber
Information/
Knowledge
SPACE
TIME
Layers Social networks
(wisdom)

What Should We Do?
How does UVR, Context-Aware MAR, work?
Get video
from camera
Recognize
Object of
Interest
Estimate position
and orientation of
the camera
Render the
augmented
scene
Process the
interaction
Update the
status
Heterogeneous AR DB

Level of Augmented Reality
Level 0
No
augmentatio
n
Level 1
Pre-defined
single content
augmentation
Object-
specific
Predefined
rendering
Predefined
UI/interaction
Level 2
Multi-modal
& combined
function
augmentation
Space-specific
SLAM
Limited light
source-aware
rendering
Partially
context-aware
Level 3
Limited
organic
augmentation
Town-level
Dynamic
SLAM
Limited
environment-
aware
rendering
Limited
context-aware
Level 4
Full organic
augmentation
World-level
Organic
SLAM
Full
environment-
aware
rendering
Full context-
aware
Sensing,
Modeling
Tracking,
Recognizing,
Rendering, Interaction Update Sensing

CAMAR: Context-aware MAR
How does Context-aware AR work?
Get video
from camera
Recognize
Object of
Interest
Estimate position
and orientation of
the camera
Render the
augmented
scene
Process the
interaction
Update the
status
•Context-aware & Visualization
•Quantified Self for AH Platform
•Social Agent for Quality of Social
Life

UVR Core Technologies
Summary : Remaining Key Challenges
Wearable platforms: Smart watch, Glasses, etc.
D-SALM: Simultaneous Object-aware Tracking and Mapping
Smart UI: Context-aware Multimodal Augmentation & infoViz
Natural Multimodal Interaction and Collaboration
UX Authoring, Capturing and Augmented Human
Social UX: Co-presence and Tele-collaboration w/ Organic & Emphatic UI
3D map-based in-situ Authoring, Storytelling & Standard for eco-system

If Technologies are Ready, Then?
Will AR/VR improve the quality of our lives?
How can we use UVR(or CAMAR) to improve QoL?
How to make a good balance?
What’s Next?
Human Technology

From Augmented ‘Reality’ to ‘Human’
VR/AR and UVR: extended space-time
Augmented Human in UVR Era?
•Slave of technology? Isolation?
•Upgrading humans into GODS?
•Or have a right to be a bit lazy and to enjoy social life?
Augmented Human
http://goo.gl/yJPDtC

Homo Sapiens
Human: social animal with body/brain and mind
Physical vs. Virtual Augmentation
Body+
•To act, respond, interact,
move spatio-temporally
Brain+
•To perceive, recognize, decide,
memorize/retrieve
Mind/Social+
•To communicate, collaborate
with empathy
Augmented Human
Act
ST Move
Body
Activity
Brain
Activity
Perceive/Cognize Decide
Collaborate
Communicate
Empathic
Social
Activity
Interact

Then, How to achieve AH?
Body+ Brain+ and Social+
First of all,
Augmented Human in UVR Era
There are 3 Things extremely hard:
i)Steel,
ii)A Diamond, and
iii)To know one's self.
- Benjamin Franklin (1750)
“Knowing yourself is the beginning
of all wisdom.”
- Aristotle
“Know thyself.”
- Socrates

HQS monitors/tracks the User!
Holistic Quantified Self (HQS) [Woo 2016]
•Small (Personal) Data but Big ‘Challenge & Opportunity’
Physical, Emotional, Social, and Digital behavior state
E-personality based on HQS
Personal agent based on E-personality
•Agent that monitors, negotiates, recommends
Integratio
n
User profile,
Sensors &
Context
Holistic
Quantified Self
Analysis
E-Personality
Bank
Personalized
Agent
E-Personality

Applications of E-Personality with HQS
Personal AR Guardian
(healthcare, life coach/guard, etc.)
Sustainable self-management
with game-like competition
Personal black-box for
protecting self from criminal

Social+ with e-Personality
From Quantified Self to Quality of Social Life!
Recommendation for deliberative decision
Experience Sharing: Empathy
Communication, Negotiation and Collaboration
Family
Friends
Groups
Communities
Companies
Countries
…
Trusted
Organization
Personalized
Augmentation
UX
Sharing
Sensing and
Monitoring
Context-
aware
Negotiation
Quantified Self

What Should We Do?
Deploy or Die! Obsolete or Augment?
"Do you want to sell sugared water
for the rest of your life?
Or do you want to come with me and
change the world?“
- Steve Jobs

Summary
VR/AR comes Back!
Ubiquitous VR as AR2.0: Beyond VR/AR
UVR: Main Technical Challenges
Augmented Human: What about Human?
What Should We Do?

The future is already here. It is just not
uniformly distributed” by William Gibson (SF writer)
More Information
Woontack Woo, Ph.D.
wwoo@kaist.ac.kr
http://uvr.kaist.ac.kr
http://arrc.kaist.ac.kr
Related Events
10th Augmented Human 2019, France, Mar. 11-12, 2019
14th ISUVR2019, Korea, Jun. 2019
Welcome to AH in UVR Era!

Augmented Human 2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Augmented Human 2018

Similar to Augmented Human 2018 (20)

More from Woontack Woo

More from Woontack Woo (20)

Recently uploaded

Recently uploaded (20)

Augmented Human 2018