A comprehensive approach to immersive engagement - An overview of the Immersive industry including Cinematic VR, Virtual Production and the Metaverse.

1. What’s up with the Immersive Industry?!
A comprehensive approach to immersive engagement
Chaitanya Chinchlikar
chaitanya.c@whistlingwoods.net
Vice President & Business Head, CTO & Head of Emerging Media

2. Let’s get to the point…
Or the sphere… (since it’s all going to be about immersive from here on) 😁
What is the immersive industry?

3. No rocket science to this…
It is the 4th content consumption (experience) platform
Cinema TV Digital/OTT Immersive

4. How did we get here?
What started first?:
- The journey towards OTT/Streaming platforms
OR
- The journey towards the Metaverse?
Common thought would say – OTT / streaming. Right?
After all, the digital content industry started waaaay back when the internet was invented… Right?
Right?

5. How did we get here?
Well, NO!
The immersive industry in-fact started much earlier…
While the metaverse may seem like something that has appeared recently, but it hasn’t.
Over the past 50+ years almost ALL the technological advances in content consumption & creation have
actually lead us to this point. Over the past decades our goal has been to draw the viewer closer & closer
INSIDE the content and make the content experience more & more immersive.
Eg: Screen size, Digital-isation, Projection quality, Surround sound, Imax, Stereoscopy, Atmos, Seats, etc.

6. Content ‘Experience’
• The consumpIon experience is
becoming as important is the
content.
• And as expected, the market has
responded to give the consumer
what they’re seeking.
The most potent tool for delivering experiences is IMMERSION

7. What is all this talk about immersion? 🤔

8. SENSES
How many senses do humans have?

9. SENSES
How many senses do humans have?
The obvious answer is five: SIGHT, HEARING, TOUCH, SMELL & TASTE.
But in reality, there are 27.
In addition to these 5 organ-based senses, there are:
¤ 4 Perception-based senses
¤ 18 other Organ based senses

10. Senses
◻ Perception based senses
¤ Time
¤ Agency (control)
¤ Familiarity
¤ Sensing gravity
Organ Based Senses
¤ Balance
¤ Temperature
¤ Proprioception (Kinesthetic
sense)
¤ Pain
¤ Sexual stimulation
¤ Hunger
¤ Thirst
¤ Breathing
¤ Suffocation
¤ Intoxication
¤ Vasodilation (Blushing)
¤ Digestion
¤ Swallowing / Vomiting
¤ Gagging
¤ Urination
¤ Vasodilation caused Pain
¤ Cardioception
¤ UV Radiation Detection

11. Senses
Perception based senses
¤ Time
¤ Agency (control)
¤ Familiarity
¤ Sensing gravity
¤ IMMERSION
Organ Based Senses
¤ Balance
¤ Temperature
¤ Proprioception (Kinesthetic
sense)
¤ Pain
¤ Sexual stimulation
¤ Hunger
¤ Thirst
¤ Breathing
¤ Suffocation
¤ Intoxication
¤ Vasodilation (Blushing)
¤ Digestion
¤ Swallowing / Vomiting
¤ Gagging
¤ Urination
¤ Vasodilation caused Pain
¤ Cardioception
¤ UV Radiation Detection

12. IMMERSION
IMMERSION
The impact of immersion on other senses (the ability of activating the sense gating
mechanism of the brain) has caused science to treat immersion as a sense.
Two types of Immersion:
¤ Physical / Technical Immersion
¤ Cognitive Immersion:
• Spatial Immersion
• Temporal Immersion
• Spacio-Temporal Immersion
• Emotional Immersion

13. Journey of the immersive ecosystem
And where are we now?

14. Gartner Hype
Cycle for
Emerging
Technologies
2015

15. Gartner Hype
Cycle for
Emerging
Technologies
2016

16. Gartner Hype
Cycle for
Emerging
Technologies
2017

17. Gartner Hype
Cycle for
Emerging
Technologies
2018

18. Gartner Hype
Cycle for
Emerging
Technologies
2019

19. Gartner Hype
Cycle for
Emerging
Technologies
2020

20. Gartner Hype
Cycle for
Emerging
Technologies
2021

21. In Summary
Year when these technologies weren’t hype anymore:
¤ Virtual Reality – 2017
¤ Blockchain – 2018
¤ Decentralised Web – 2019

22. The Dunning-Kruger Curve
Cinematic VR
Functional
Augmented Reality
Production

23. So, what do we do in/with this immersive world? 🤔

24. So, what do we do in/with this immersive world? 🤔
◻ We ENTERTAIN ourselves
◻ We LIVE our lives
◻ We use its TOOLS to create content

25. Immersive Industry: Experiential – Functional – Production
Functional
¤ Going to ‘work’ /
running a business
¤ Shopping
¤ Education
¤ Social experiences
¤ and more…
Experiences
¤ Cinematic VR
¤ Live VR (Concerts,
Sports, etc)
¤ Gaming
¤ Tourism?
¤ and more…
Production
¤ Photogrammetry
¤ Volumetric Capture
¤ Virtual Production
¤ and more…

26. That’s a lot!

27. Is the immersive platform unique?
YES!

28. Content & Platforms
WHAT
NEXT?
Cinema TV Digital/OTT
Birth Late 1800s 1930s Late 1990s
Narrative
Influence
High Medium Very High
Visual Influence Very High Medium Medium – to – High
Audio Influence Very High Low Low – to – Medium
Immersion
Impact
Mid – to – High Low Very Low

29. Content & Platforms
Cinema TV Digital/OTT Immersive
Birth Late 1800s 1930s Late 1990s ?
Narrative
Influence
High Medium Very High Medium
Visual Influence Very High Medium Medium – to – High Very High
Audio Influence Very High Low Low – to – Medium Very High
Immersion
Impact
Mid – to – High Low Very Low Very High

30. Content & Platforms
• We need to understand that there are unique production / narrative / experiential needs of
each experience
• The idea-to-consumption pipelines for these experiences
Also, we should know these 4 RULES:
• Don’t bore people – experience without engagement is pointless
• Don’t confuse people – immersive content is not a treasure hunt
• Don’t give people whiplash – contextually 360deg but content-wise between 160-210deg
• Don’t give people a headache / nausea or make them vomit

31. Understand the ‘experience levers’
AGENCY /
CONTROL
LIVE

32. So, what have we learnt?
About Cinematic VR, Live Sports Production for VR, Virtual Production

33. Cinematic VR
• A VR 'film' is an 'experience' of which the Story, Visuals, Sound, Immersion & Interactivity are
all parts. Work on all of them during the ‘writing’.
• Cinematic grammar needs to be re-written for frameless / immersive content.
– Develop a spatial visual view – a 360 mindset
– A deeper study of Immersion as a sense is needed by all filmmakers.
– Directing Attention is the new craft to be learnt by the filmmaker.
– The viewers eyes are your viewfinder & your frame
– Solving people’s FOMO is the big challenge.
• Intuitive Filmmaking is needed - Think of why & how you would look at something.
• Every VR shot is a VFX shot.

34. Cinematic VR
Film / Theatrical TV Digital/OTT Cinematic VR
Nature of
Viewing
Captive Community viewing
Non-captive Family
viewing
Non-captive Individual
viewing
Captive Individual viewing
Platform
Viewing Details
Fixed frame, large screen,
2D & 3D
Fixed frame, small screen,
2D
Fixed frame, mini screen,
2D
FRAMELESS, 2D & 3D,
Duration &
Structure
110-180 mins
60-100 scenes
1-8 shots / scene
22–44 mins
8-10 scenes
1-20 shots / scene
3–60 mins
2-20 scenes
1-4 shots / scene
60secs - 60mins
Fiction Content
Structure
Primarily stand-alone,
marginally serial
Primarily serial, marginally
stand-alone
Equal amount of stand-
alone & serial content
Equal amount of stand-
alone & serial content
USP
Larger Than Life,
Audio-visual narrative
spectacle
Appointment Viewing,
Story & Character
development
Individuality,
High concept, less
formulaic, writing-focused,
pace is critical
Immersive & interactive

35. Live Sports Production
• Don’t conflate production for broadcast with production for immersive
• Go back to first principles of:
– Information
– Experience
And produce for them, given the pros / cons of the immersive production pipeline
• Don’t shun the existing content feeds. Build on them.
• Use the frameless medium appropriately to do more than is possible on broadcast

36. What is Virtual Production?
Virtual Production is a filmmaking pipeline for creation of blended (live action + computer generated)
moving pictures. It is where the physical and digital worlds interact in real-time as they are composed &
captured on ‘set’, by combining traditional production tools of physical sets & props, actors, cameras & lights,
with virtual and augmented reality, game-engines, virtual sets, photogrammetry, LED walls, volumetrically
captured or animated characters, virtual cameras, virtual lights and camera, object & motion tracking.
VP broadens BOTH Imagination & Efficiency
FOR
Live Produced content AND Filmed Entertainment.

37. What is Virtual Production?
There’s 3 kinds/types/levels of Virtual Production, based on the combination of inputs, whether they
are real-time or not and whether the final output emerges from a computer, or from the camera:
- All-CG Virtual Production
- On-Set Virtual Production
- In-Camera VFX

38. How does one approach Virtual Production?
- Understanding the Virtual Production workflows for all 3 kinds of VP – Full-CG Virtual
Production / On Set Virtual Production / In-camera VFX
- Disaggregation of the workflow into its elements
- Script Breakdown
- VAD Assets including workflows of Photogrammetry, Volumetric Capture, Motion Capture
- Camera, Object, Motion Tracking
- ‘On-Set’ shot-taking
- Finishing
- Skillsets needed for the above

39. The
Wall
Chart

40. Virtual Production Pipeline reimagined
Gaming Engine
(Virtual ProducJon
enabled)
Action Assets
- Live Action actors
- Physical Props
- Volumetric Captured assets
- CG created Animated Models
- CG created MoCap assets
- CG created photogrammetry
assets
The World
- Real Set
- Blended Set – Real + Virtual
- Virtual Set (Green Screen/LED)
- Photogrammetry Set
- 360 photograph
- Pre-shot video / photo
- Game-engine created world
360 VR video (3DoF)
Camera, Motion &
Object tracking
Flatscreen video 2D / 3D 6DoF volumetric experience
Output Format
+

41. Application of Virtual Production?
• The biggest mistake that the industry makes is to assume that ALL content should be produced
using this pipeline.
• Virtual Production is not meant only for BIG productions like Mandalorian or Blade Runner
2049, etc. Virtual Production has a specific function, a specific use and that is what it should be
used for, and for that ONLY. Right use of it even for just one or 2 shots can add huge value.
• It definitely has a place in the film production ecosystem and producers / directors / technicians
need to make specific decisions whether to deploy VP for certain shots.

42. Are there any problems in the Immersive Industry?

43. Problems… a plenty…
• Biological safety about the wearable headset
• Privacy – biohacking – iris tracking / pupil dilation tracking
• Business Model is linked to cryptos / tokens
• DAOs (Decentralised Autonomous Organisations) vs Investors
• The Virtual Production pipeline is still too scattered with very little standardisation
• Judicious use of Virtual Production

44. Is there money at the end of these troubles?

45. VR – Industry / Revenue / Units
The number of VR devices shipped was
13.48 million units in 2020, expected to
reach 112.62 mn units by 2026, with a
CAGR of 33.85% between 2021-26.
The global VR content market is
expected to reach US$ 84.09bn by 2028,
exhibiBng a CAGR of 44.8% between
2021-2028.
As of January 2020
As of January 2022

46. The Virtual Production Market
220bn$ - global content
spend in 2021

47. Some more good news…

48. There is consumer interest in immersive experiences

49. Headsets are becoming sleeker

50. We have decent competition in VR
Headset image
not found 🤷

51. In Summary…
The journey must be made…
Fail repeatedly, fail fast, until you stop failing…
We need to learn from our mistakes and move on…

52. The Dunning-Kruger Curve
Cinematic VR
Metaverse
Augmented Reality
Virtual Produc8on

53. THANK YOU
Chaitanya Chinchlikar
Vice President & Business Head
Chief Technology Officer & Head of Emerging Media
chaitanya.c@whistlingwoods.net