The Virtual Reality Medium
and Creations
Jeff Shaw
VR Developer - VRX
Virtual Reality is
a New Artistic
Medium with
Examples
Introduction
 Jeff Shaw – VR Developer
 VRX – Virtual Reality eXperiences (VR Studio)
 30+ years experience in VR
 Singer Link Flight Simulators
 Intel VR Development
 VR HMD and other Patents
 Oculus Kickstarter
 VRX started, full time VR
 The Virtual Reality Cafe
 Computer Graphics and 360 Video creations
A Brief History of VR
 1792 – First Panoramic Painting- Edinboro
 1838 – Stereoscopic Photos and Viewers
 1929 – Flight Simulator (Edward Link – Singer Link)
 1950’s - Sensorama
 1965 – Ultimate Display (Ivan Sullivan)
 1987 – “Virtual Reality” term Coined (Jarad Lanier VPL)
 1991 – Virtual Group Arcade Machines
 !992 – The Lawnmower Man Movie
 1995 – Nintendo Virtual Boy
 2012 – Oculus Rift Kickstarter
 2015+ - HTC Vive, Playstation, +++
 2020 – Alyx – Half-Life VR
Different Types of VR
 Photographic 360 video – Photographic detail and
realism, no interactivity or freedom of movement, quicker
and easier to generate.
 Computer Graphics – interaction and freedom of
movement, CG graphics apparent not real. Interactive
controllers.
 *Lines blurring - Fusion of CG and Photographic – Green screen,
Photogrammetry, LIDAR
Ways to See VR Today
 Rotation - 3 degrees of freedom (DoF) means you can view any
direction in an experience by rotating and turning your head. But you cannot
move around in the experience by moving around in real life.
 Rotation and Movement - 6 degrees of freedom (DoF) means
you can view any direction AND you can move around in virtual reality space by
moving around in real life.
Platform Examples
The Degrees of Freedom (DoF) of an object represent its
ability to move around in a space. In total, 3D space has
six degrees of freedom. Three of them represent rotational
movement (yaw, pitch and roll) and the other three represent
translational movement (elevate, straff, and surge).
Capabilities and Limitations
 Capabilities - Audio Visual - Combines elements of
audio and visual artistic expression like a fusion of
painting, sculpting, photography, film making, music
composition, Story Telling, +
 Limitations – Other senses – Touch, Smell, Taste,
Orientation/Velocity (the gap is slowly closing).
 Small Audience/Market - A very small market right
now, difficult to make money selling apps to end users
and having wide visibility of work.
Challenges
 Elements of composition (Visual
direction, balance – many principles are
based on a frame)
 Telling a story in VR
 Immature technology – Expensive,
Bulky, Limitations (Sensory, Cost, Bulk,
Audience Size)
 Simulator sickness
Complete EnvironmentUsers View
A documentary style VR experience about the Dutch
painter Hieronymus Bosch
Examples:
The Eye of the Owl
A mix of photographic and CG elements in 3D VR
harmony. Combined fantastic fantasy locations,
creatures, and choreographed dance and music.
Fantasy Dance & Fashion
Playing with ultra-realistic psychedelic liquids in zero gravity
on the International Space Station
Liquid Space
The Eye of the Owl
A documentary
style VR
experience about
the Dutch painter
Hieronymus
Bosch
Examples:
Bosch VR Project Goals
 To make a documentary style VR experience to accompany the Bosch
documentary film.
 To present the actual Bosch masterpiece The Garden of Earthly Delights in a
realistic way without rendering or animating the painting content.
 Integrate a soundscape into the painting
 Provide expert narration
 Provide a guided tour and free exploration functionality
 Provide multiple languages
 Distribute on multiple VR platforms
Making a VR Application - Development
Tools & Methods
 Bosch VR Tools/Methods
 Agile development – frequent iterations
 Game Engine - Unity 3D
 Programming - C#, JavaScript
 Audio Recording – Blue Yeti + Audacity
 Graphics Rendering – 3D Max, Blender, Sketch up
 3D Assets – Turbosquid, Unity Asset Store
 3D/Binaural Audio Plugin – Two Big Ears, Unity 3D
The Garden of Earthly Delights by Hieronymus Bosch
Exterior – Shutters Closed
Bosch VR Scenes
 Owl Flight
 Bosch’s Studio
 St Jan’s Cathedral in Den Bosch
 Art Historian Laboratory
 Prado Museum
 Interior of Tree Man
Owl flight
The owl flight
over the market
square into
Bosch’s studio
Research – Market Square Layout
Satellite Image of Den Bosch VR Market Square Building Layout
Studio Interior
The Garden of Earthly
Delights has just been
finished inside Bosch’s
studio
Available on Steam: https://store.steampowered.com/app/420020/Eye_of_the_Owl__Bosch_VR/
Studio Interior
The Garden of Earthly
Delights has just been
finished inside Bosch’s
studio (Gear VR Version)
https://www.youtube.com/watch?v=lbHs3ghTLMs360 Video Prototype Example:
Research - Studio Setting
Creating the painters
workshop from the
period
VR Magnifying Glass
Use the magnifying glass
to see minute details and
brush strokes
Select areas for magic
carpet miniaturization
exploration
Magic Carpet Shrinking Observer
Shrink to a fraction of your
normal size and perceive
levels of detail like never
before. See into the
cracks in the painting and
brush strokes.
Stand with the characters
in the individual scenes
Bosch Painting Soundscape
Sounds are placed
in 3D position on
the painting.
Research – Bosch as the owl
The many faces of the owl
Research - Butt Song
A musical score is
printed on a
characters rear end
and played in period
instruments in the
VR soundscape.
Research - Binaural Sound
A binaural “microphone”
exhibiting head blocking.
Controls – Navigating in the VR World
The standard Samsung
controller provides an
automatic setup and usable
interface to map your
controls
The Vive Controllers for
interaction. Movement is
natural with the Vive.
Video Clips of Demos
https://www.youtube.com/watch?v=lbHs3ghTLMs
Gear VR Prototype
360 Video from Vive Prototype
https://vimeo.com/130895622
Using 2D Sprites and
simplified animation.
Flat and no character
animation rigging.
Bosch 500 - H Bosch VR App
Similar Applications
Similar Applications
Castle VRuin
Photogrammetric
reconstructions with a
RC helicopter
Walk around a
ancient castle
reconstructed in VR
https://www.youtube.com/watch?v=Ccx1RBv7UFA
VR application renders Van Gogh’s
Painting to a 3D world we can explore
(http://www.dailymail.co.uk)
https://www.youtube.com/watch?v=BMzh-s8qeSo
Video Demo – The Night Cafe
Similar Applications
Van Gogh VR Application – The Night Cafe
Van Gogh VR Application – Reviews
Disappointing negative
criticism for this
wonderful creation.
Everyone has different
views.
Listen to those who
don’t share your
views.
A new musical
instrument which
creates
synesthetic
visuals as you
create music
The Sonotrope
Examples:
https://www.youtube.com/watch?time_continue=49&v=NjfxwIURnkM&feature=emb_logo
An online
platform where
two participants
use avatars of
animal deities.
An art
installation by
Artist Faysal
Mroueh
exhibited in
Amsterdam
Set Animal
Examples:
https://www.amsterdamart.com/events/792/opening-set-animal-by-faysal-mroueh
Green Screen in 3D VR
Mixing 3D 360 video with Computer
Graphics for Victoria Vitalie Fashion SF
https://www.youtube.com/watch?v=H_ytExf5wP4
A mix of
photographic
and CG
elements in 3D
VR harmony.
Combined
fantastic fantasy
locations,
creatures, and
choreographed
dance and
music.
Examples:
Fantasy Dance & Fashion
Interact with
ultra-realistic
liquids in zero
gravity on the
international
space station.
Liquid Space
Examples:
https://store.steampowered.com/app/1183750/Liquid_Space/
Apollo 17 - Moonbuggy
Examples:
https://store.steampowered.com/app/1060460/Apollo_17__Moonbuggy_VR/
Other VR Projects this Year (Steam)
Pumpkin Smasher
VR Garage
https://store.steampowered.com/app/1179400/Basic_Car_Repair_Garage_VR/
https://store.steampowered.com/app/1157650/Pumpkin_Smasher_VR/
Do your thing!
 Follow your passion!

The VR Medium

  • 1.
    The Virtual RealityMedium and Creations Jeff Shaw VR Developer - VRX Virtual Reality is a New Artistic Medium with Examples
  • 2.
    Introduction  Jeff Shaw– VR Developer  VRX – Virtual Reality eXperiences (VR Studio)  30+ years experience in VR  Singer Link Flight Simulators  Intel VR Development  VR HMD and other Patents  Oculus Kickstarter  VRX started, full time VR  The Virtual Reality Cafe  Computer Graphics and 360 Video creations
  • 3.
    A Brief Historyof VR  1792 – First Panoramic Painting- Edinboro  1838 – Stereoscopic Photos and Viewers  1929 – Flight Simulator (Edward Link – Singer Link)  1950’s - Sensorama  1965 – Ultimate Display (Ivan Sullivan)  1987 – “Virtual Reality” term Coined (Jarad Lanier VPL)  1991 – Virtual Group Arcade Machines  !992 – The Lawnmower Man Movie  1995 – Nintendo Virtual Boy  2012 – Oculus Rift Kickstarter  2015+ - HTC Vive, Playstation, +++  2020 – Alyx – Half-Life VR
  • 4.
    Different Types ofVR  Photographic 360 video – Photographic detail and realism, no interactivity or freedom of movement, quicker and easier to generate.  Computer Graphics – interaction and freedom of movement, CG graphics apparent not real. Interactive controllers.  *Lines blurring - Fusion of CG and Photographic – Green screen, Photogrammetry, LIDAR
  • 5.
    Ways to SeeVR Today  Rotation - 3 degrees of freedom (DoF) means you can view any direction in an experience by rotating and turning your head. But you cannot move around in the experience by moving around in real life.  Rotation and Movement - 6 degrees of freedom (DoF) means you can view any direction AND you can move around in virtual reality space by moving around in real life. Platform Examples The Degrees of Freedom (DoF) of an object represent its ability to move around in a space. In total, 3D space has six degrees of freedom. Three of them represent rotational movement (yaw, pitch and roll) and the other three represent translational movement (elevate, straff, and surge).
  • 6.
    Capabilities and Limitations Capabilities - Audio Visual - Combines elements of audio and visual artistic expression like a fusion of painting, sculpting, photography, film making, music composition, Story Telling, +  Limitations – Other senses – Touch, Smell, Taste, Orientation/Velocity (the gap is slowly closing).  Small Audience/Market - A very small market right now, difficult to make money selling apps to end users and having wide visibility of work.
  • 7.
    Challenges  Elements ofcomposition (Visual direction, balance – many principles are based on a frame)  Telling a story in VR  Immature technology – Expensive, Bulky, Limitations (Sensory, Cost, Bulk, Audience Size)  Simulator sickness Complete EnvironmentUsers View
  • 8.
    A documentary styleVR experience about the Dutch painter Hieronymus Bosch Examples: The Eye of the Owl A mix of photographic and CG elements in 3D VR harmony. Combined fantastic fantasy locations, creatures, and choreographed dance and music. Fantasy Dance & Fashion Playing with ultra-realistic psychedelic liquids in zero gravity on the International Space Station Liquid Space
  • 9.
    The Eye ofthe Owl A documentary style VR experience about the Dutch painter Hieronymus Bosch Examples:
  • 10.
    Bosch VR ProjectGoals  To make a documentary style VR experience to accompany the Bosch documentary film.  To present the actual Bosch masterpiece The Garden of Earthly Delights in a realistic way without rendering or animating the painting content.  Integrate a soundscape into the painting  Provide expert narration  Provide a guided tour and free exploration functionality  Provide multiple languages  Distribute on multiple VR platforms
  • 11.
    Making a VRApplication - Development Tools & Methods  Bosch VR Tools/Methods  Agile development – frequent iterations  Game Engine - Unity 3D  Programming - C#, JavaScript  Audio Recording – Blue Yeti + Audacity  Graphics Rendering – 3D Max, Blender, Sketch up  3D Assets – Turbosquid, Unity Asset Store  3D/Binaural Audio Plugin – Two Big Ears, Unity 3D
  • 12.
    The Garden ofEarthly Delights by Hieronymus Bosch Exterior – Shutters Closed
  • 13.
    Bosch VR Scenes Owl Flight  Bosch’s Studio  St Jan’s Cathedral in Den Bosch  Art Historian Laboratory  Prado Museum  Interior of Tree Man
  • 14.
    Owl flight The owlflight over the market square into Bosch’s studio
  • 15.
    Research – MarketSquare Layout Satellite Image of Den Bosch VR Market Square Building Layout
  • 16.
    Studio Interior The Gardenof Earthly Delights has just been finished inside Bosch’s studio Available on Steam: https://store.steampowered.com/app/420020/Eye_of_the_Owl__Bosch_VR/
  • 17.
    Studio Interior The Gardenof Earthly Delights has just been finished inside Bosch’s studio (Gear VR Version) https://www.youtube.com/watch?v=lbHs3ghTLMs360 Video Prototype Example:
  • 18.
    Research - StudioSetting Creating the painters workshop from the period
  • 19.
    VR Magnifying Glass Usethe magnifying glass to see minute details and brush strokes Select areas for magic carpet miniaturization exploration
  • 20.
    Magic Carpet ShrinkingObserver Shrink to a fraction of your normal size and perceive levels of detail like never before. See into the cracks in the painting and brush strokes. Stand with the characters in the individual scenes
  • 21.
    Bosch Painting Soundscape Soundsare placed in 3D position on the painting.
  • 22.
    Research – Boschas the owl The many faces of the owl
  • 23.
    Research - ButtSong A musical score is printed on a characters rear end and played in period instruments in the VR soundscape.
  • 24.
    Research - BinauralSound A binaural “microphone” exhibiting head blocking.
  • 25.
    Controls – Navigatingin the VR World The standard Samsung controller provides an automatic setup and usable interface to map your controls The Vive Controllers for interaction. Movement is natural with the Vive.
  • 26.
    Video Clips ofDemos https://www.youtube.com/watch?v=lbHs3ghTLMs Gear VR Prototype 360 Video from Vive Prototype
  • 27.
    https://vimeo.com/130895622 Using 2D Spritesand simplified animation. Flat and no character animation rigging. Bosch 500 - H Bosch VR App Similar Applications
  • 28.
    Similar Applications Castle VRuin Photogrammetric reconstructionswith a RC helicopter Walk around a ancient castle reconstructed in VR https://www.youtube.com/watch?v=Ccx1RBv7UFA
  • 29.
    VR application rendersVan Gogh’s Painting to a 3D world we can explore (http://www.dailymail.co.uk) https://www.youtube.com/watch?v=BMzh-s8qeSo Video Demo – The Night Cafe Similar Applications Van Gogh VR Application – The Night Cafe
  • 30.
    Van Gogh VRApplication – Reviews Disappointing negative criticism for this wonderful creation. Everyone has different views. Listen to those who don’t share your views.
  • 31.
    A new musical instrumentwhich creates synesthetic visuals as you create music The Sonotrope Examples: https://www.youtube.com/watch?time_continue=49&v=NjfxwIURnkM&feature=emb_logo
  • 32.
    An online platform where twoparticipants use avatars of animal deities. An art installation by Artist Faysal Mroueh exhibited in Amsterdam Set Animal Examples: https://www.amsterdamart.com/events/792/opening-set-animal-by-faysal-mroueh
  • 33.
    Green Screen in3D VR Mixing 3D 360 video with Computer Graphics for Victoria Vitalie Fashion SF https://www.youtube.com/watch?v=H_ytExf5wP4
  • 34.
    A mix of photographic andCG elements in 3D VR harmony. Combined fantastic fantasy locations, creatures, and choreographed dance and music. Examples: Fantasy Dance & Fashion
  • 35.
    Interact with ultra-realistic liquids inzero gravity on the international space station. Liquid Space Examples: https://store.steampowered.com/app/1183750/Liquid_Space/
  • 36.
    Apollo 17 -Moonbuggy Examples: https://store.steampowered.com/app/1060460/Apollo_17__Moonbuggy_VR/ Other VR Projects this Year (Steam) Pumpkin Smasher VR Garage https://store.steampowered.com/app/1179400/Basic_Car_Repair_Garage_VR/ https://store.steampowered.com/app/1157650/Pumpkin_Smasher_VR/
  • 37.
    Do your thing! Follow your passion!