Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The rise of VR & AR era. Why this time is different?

12,506 views

Published on

When Facebook bought virtual reality company Oculus in early 2014, virtual reality blew up. While game and movie studios began reimagining the future, others looked back at the "old days" of VR — a loosely remembered period in the 1990s when gloves and goggles were super cool and everyone was going to get high on 3D graphics.

Computer enthusiasts and science fiction writers have dreamed about VR for decades. But earlier attempts to develop it, especially in the 1990s, were disappointing. It turns out the technology wasn’t ready yet. What’s happening now  — because of Moore’s Law, and also the rapid improvement of processors, screens, and accelerometers, driven by the smartphone boom  —  is that VR is finally ready to go mainstream. Now, we’re in the midst of a virtual reality revolution.

So the question I want to pose here – is this time different?
Are we going to this future or it’s only media hype?

Published in: Technology
  • Thanks for sharing! Can you share me the latest progress on VR/AV with hon.jamin@gmail.com ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I’m so excited! Thanks to Dr.Unity for bringing back my Ex-husband and brought great joy to me today! Unityspelltemple@gmail.com is certainly the best spell caster online,if you need your Ex lover back fast!and his result is 100% guarantee.. After 12years of marriage, me and my husband has been into one quarrel or the other until he finally left me and moved to California to be with another woman. I felt my life was over and my kids thought they would never see their father again. i tried to be strong just for the kids but i could not control the pains that torments my heart, my heart was filled with sorrows and pains because i was really in love with my husband. Every day and night i think of him and always wish he would come back to me, I was really upset and i needed help, so i searched for help online and I came across a website that suggested that Dr Unity can help get ex back fast. So, I felt I should give him a try. I contacted him and he told me what to do and i did it then he did a (Love spell) for me. 28 hours later, my husband really called me and told me that he miss me and the kids so much, So Amazing!! So that was how he came back that same day,with lots of love and joy,and he apologized for his mistake,and for the pain he caused me and the kids. Then from that day,our Marriage was now stronger than how it were before, All thanks to Dr Unity. he is so powerful and i decided to share my story on the internet that Dr.Unity real and powerful spell caster who i will always pray to live long to help his children in the time of trouble, if you are here and you need your Ex back or your husband moved to another woman, do not cry anymore, contact this powerful spell caster now. Here’s his contact: Email him at: Unityspelltemple@gmail.com , you can also call him or add him on Whats-app: +2348071622464 , you can also visit his website: https://unityspelltempe.blogspot.com .
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Thanks for sharing!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • In page 11, DHL: Using Augmented and Virtual Reality, DHL can optimise parcel and load handling, customer delivery efficiency and provide detailed tracking. What's the detailed scenario? How DHL can optimise parcel and load handling?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Good summary.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

The rise of VR & AR era. Why this time is different?

  1. 1. The rise of AR & VR: why this time is different? Vasily Ryzhonkov, Director of Mobile Tech Center, Skolkovo Foundation vryzhonkov@sk.ru http://mobile.sk.ru
  2. 2. The Metaverse is coming… Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Realtime networked 3D gaming was a big step 19 years ago… Source: Neal Stephenson, Snow Crash, 1992 Source: Quake game by John Carmack et al., 1996 …and now we got this. Metaverse (coined in 1994 by Stephenson) is a collective virtual shared space, created by the convergence of virtually enhanced physical reality and physically persistent virtual space, including the sum of all virtual worlds, augmented reality, and the internet. (Wikipedia)
  3. 3. Is this our future? Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Fantasts have been dreaming about immersive computing technologies for a long time… Source: Minority Report, 2002 Source: Microsoft, 2015 …and in 2015 Microsoft presented it’s vision of the future computing platform (along with other concepts from Facebook, Google, Sony, HTC, Valve …)
  4. 4. VR/AR are the Next Big Thing The Next Mega Tech Theme is Virtual Reality (VR). • We believe virtual reality and augmented reality are the next mega tech themes through 2030. • We liken the state of virtual and augmented reality today as similar to the state of mobile phones 15 years ago. • It likely will take a decade before mainstream adoption as necessary improvements in displays and applications as well as lower pricing are needed to drive demand. • The first phase will be virtual reality, which we believe will start over the next several years, and is the primary focus of this report; the second phase will be augmented reality. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  5. 5. Technology Definitions Augmented Reality Augmented reality, or AR, is when real reality, or RR, is enhanced with virtual images that appear to coexist with the real world. Virtual Reality Virtual reality, or VR, is when you’re immersed in a purely virtual world 5 Mixed Reality (MR) or Diminished Reality, refers to a system that combines real and virtual objects and information using augmented virtuality, which is combines some reality with virtual world. Mixed Reality Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  6. 6. Technology Definitions What is Total Immersion? According to Wikipedia, Immersion into virtual reality is a perception of being physically present in a non- physical world. This combines the sensory perceptions of the human mind to give a surreal impression of living in another world, constructed of Virtual or Augmented reality. Additionally, Virtual reality glasses can produce a visceral feeling of being in a simulated world, a form of spatial immersion called Presence. According to Oculus VR, the technology requirements to achieve this visceral reaction are low-latency and precise tracking of movements 6Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  7. 7. Ontology of AR / VR Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Source: “Mediated Reality with implementations for everyday life”, Steve Mann, 2002 Source: "Augmented Reality: A class of displays on the reality-virtuality continuum“, Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino, 1994 Father of Augmented Reality, Steve Mann
  8. 8. 0 WAVE: Infancy Mirror stereoscope: Sir Charles Wheatstone 1840 Stereoscope: An optical illusion instrument, giving depth of field through two pictures of the same object Sir David Brewster created a new stereoscope by utilizing prismatic lenses 1841 - 1860 Viewing 3D movies through stereoscope at home 1922 First 3D film produced in Hollywood: The Power of Love 1923 - Present Mainstream 3D Cinema begins 8 Stereoscope: Sir David Brewster Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  9. 9. 1st WAVE: Inventors & Dreamers 1957 Sensorama: First VR machine developed by Morton Heilig, gives visual and sensory perceptions of virtual reality, including vibrations, smell and touch. 1930 1965 Link trainer: Invented by Edwin Link for training military pilots, using simulation “Ultimate Display”, a VR helmet with two CRT screens, was created by Ivan Sutherland. The two CRT screens enabled the viewer to experience the 3D images 9Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  10. 10. 2nd WAVE: Industrial & Military Applications 1966-68 Thomas A. Furness III developed the first VR head mounted display cockpit for the US Air Force, with Bob Sproull. 1980- Present NASA: Virtual Visual Environment Display, head mounted unit for VR graphics and simulation using LCD display and wide angle lens. 10Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru General Electric Corporation built one of the first computerized flight simulators. It provided 180-degree FOV (Field-of-View) by using three screens surrounding the training cockpit. 1972
  11. 11. 3rd WAVE: Industrial Applications & Gaming DHL: Using Augmented and Virtual Reality, DHL can optimise parcel and load handling, customer delivery efficiency and provide detailed tracking ESI: Improved data handling, warehouse management and capacity forecasting using AR and VR technologies Using an iPad, with an application specially developed by Alstom Transport, all maintenance tasks are described with precision superimposing in real time a 3D virtual model on the operator’s perception of reality 11Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Nintendo Virtual Boy : 1994, price - $180. Sega VR: 1994, prototype
  12. 12. 4rd WAVE: Next computing platform The 4th wave of virtual and augmented reality began, when Facebook bought, the VR pioneering company Oculus for $ 2 Billion US Dollars. The acquisition sparked a market speculation and generated a renewed interest in VR and AR technology. The Google Trend graph depicted in adjacent, shows the interest in the technology over time since 2014. Google Glass, launched in 2013 is another example built on (optical head-mounted display ) the next generation computing platform using Ubiquitous computing for displaying real time AR information through lens. 12 Google Trends: Virtual Reality Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  13. 13. Predictions and hype present and in the past Source: Gartner (2008) Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Source: Gartner (2015) The only outcome I can make is …predictions are always wrong…but they can indicate media hype and technological advances
  14. 14. Predictions and hype present and in the past Source: Frost & Sullivan, 4th Wave in:Bauer (1996), OVUM in Leston et al. (1996), Machover (1996) “Are these expectations realistic or do they result from the initial euphoria of an innovative technology?”, - Economist, 1996 Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  15. 15. 7 Failed Virtual Reality Technologies The Sensorama : Morton Heilig, inv - 1957, patent - 1962 The Sword of Damocles : Ivan Sutherland , 1968 Nintendo Virtual Boy : 1994, price - $180. Power Glove : Nintendo, 1989, price - $75 The Virtusphere : Ray and Nurulla Latypov, 1995, price - $100K Omnidirectional Treadmills VRML, 1995 VRML was billed as a 3- D alternative to HTML. The idea was that users could interact freely with 3-D worlds on the internet, described by text and interpreted by modeling software. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  16. 16. Why AR & VR didn't get broad adoption? • Technical imperfection • Cost • Bulky & Bad design • Failed to deliver value to the end user Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  17. 17. What's now and why 2014th is the cornerstone? Facebook’s Oculus Rift and Sony Playstation.VR is only beginning… Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  18. 18. What's now and why 2014th is the cornerstone? Critical mass of elements has come together Organizational and economic: • critical mass of companies is working together • price and affordability • ergonomics and design (not bulky) Technical: • flat-panel displays • cell phone panels (good megapixels) • optics (wide fov) • mobile CPUs & GPUs • batteries and power management • wireless • gyroscopes and accelerometers • cameras • projectors Technical: • computer vision • tracking hardware • content 3d games • latency*** Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  19. 19. AR & VR Ecosystem 19Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  20. 20. AR & VR Ecosystem : Cont. 20Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  21. 21. AR & VR Ecosystem : Cont. DISPLAY HARDWARE: [HMD]: AlterGaze, AntVR, FOVE, AVEGANT, HTC, Oculus, Sony, Panasonic, VRVANA, Razer, SENSICS, Immersion VRELLA, Light & Shadows & Mindmaze. [Wireless/ Mobile VR]: Moar, LG, ZEISS, Wearality, USENS, Homido, Samsung, GearVR & VR One. [GAMING]: Activision, Blizzard, CCP, Epic Games, nDreams, Otherworld, Reload, Owlchemy Labs, Vertigo Games & VR Bits. [CAVE]: Visbox, Worldviz, Christie and mechdyne. 21Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  22. 22. VR Ecosystem 22Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Source: Greenlight VR, UploadVR Number of VR companies Investments in VR Top 10 companies in VR based on Total Funding Raised
  23. 23. VR Ecosystem 23Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Source: Greenlight VR, UploadVR Total funding raised by all VR companies VR Investment landscape
  24. 24. Russian Landscape There are more than 60 companies in Russian AR/VR market. Mostly these companies are digital studios. However, there are high-tech companies with proprietary technology developed in-house. 24Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  25. 25. Russian Landscape: Cont. Augmented reality companies: 25 За 2014 год произошел резкий всплеск образования компаний, занимающихся интерактивными технологиями (40% компаний). Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  26. 26. Russian Landscape: Cont. Virtual reality companies: 26Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  27. 27. Russian Landscape: Cont. AR / VR continuum: controllers / 3D visualization & 3D content creation / haptic & sensory feedback / images & video search and analysis / gesture recognition, computer vision etc. 27Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  28. 28. Current Market Analytics :- Revenue Users and Investments Given the scale of customer adoption, it has given rise to steady revenue over time, totalling over 62 $ Million US dollars by 2017 with AR revenue share consisting of 600 $ Million US Dollars by 2017. The agencies suggest that a total of revenue of 160 $ Million US Dollars by 2018, with head mounted display sales leading to 117 4 Million US Dollars. 28Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  29. 29. Current Market Analytics :- Market Share Different analytical companies show different estimates. Piper Jaffray suggests that VR market can grow up to $7 bln by 2025; some agencies assess market as $150 bln by 2020 consisting of AR and VR market share. Upon further analysis, we see that market will consist of VR Hardware, VR games, VR Film & Concerts, VR amusement parks, VR Entertainment, VR speciality products (such as adult content), VR industrial applications. 29Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  30. 30. Market Analytics : Revenue generation and adoption Piper Jaffray report: “We believe technology advances over the next 3-5 years will continue to drive down the cost of these technologies and enable significant improvement in tracking and rendering. We expect CPUs and GPUs of computers to continue along Moore’s Law which should enable cheaper computers to eventually run wired headsets headsets. We expect VR will gradually drop from an all in entry cost of $1000+ (VR headset $400 + high powered computer $1,000) to a more reasonable $500+ (VR headset $250 + computer $300) for VR headsets that offer high levels of immersion” Conclusion: HMD price will be key driver of the mass adoption. 30Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  31. 31. Market Analytics : Revenue generation from HMD Based on the information above, we can conclude that VR HMD will drive the industry forward. However, content will be king. That’s why a lot of VR companies invest heavily in VR studios, create SDK for developers. And recently Oculus Rift announced that hundreds of games are in the process of the development and thousands of developers are involved in this process. 31Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  32. 32. Gaming market dissection During the third quarter of 2014 , KZero, a market analytics firm conducted a market research within the gaming section of VR sectors. It was found that RPG style games were most popular amongst gamers who are older than 30 years old, with simulation and exploration type games being popular between 20-30 year old gamer Based on the red colour depiction within Simulation, it can be inferred that majority of the games are in late developmental or in released stage, whilst RPG (Role Playing Games) are in early development or alpha stages. 32Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  33. 33. Analysis Conclusion 1. Strong correlation can be seen with other market technologies like gaming, 3D, motion tracking, which are interconnected. 2. It has been estimated that, there will be 170 million active users in 2018, accounting to the total gaming population of 5.2 billion at large. 3. Gaming industry is dominated by software sector and is believed to remain that way 4. Most opportune and profitable moment in cycle is between 2017 and 2018, for technology will be developed and available to bigger mass, leading to a predominant early majority generating the highest fixed return per user. Highest return on adoption is probably somewhere near the start of next year, 5. To conclude, Augmented Reality is expected to rise faster than VR, due less complexity, increased adaptability, more interoperability, less motion sickness and simulation sickness 33Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  34. 34. VR/ AR Market drivers and projections 34 1. An estimated 3000+ million mobile subscribers are expected in 2020, having a penetration rate of 51% at a global average. 2. This provides Augmented Reality, a development scope in games, enterprise application and location based services. 3. It has been estimated that USA, Japan and SE Asia will be the most profitable markets for AR. 4. The key market drivers for AR are, mobile devices, app stores and gaming apps. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  35. 35. Challenges and barriers of AR & VR Economic and organizational risks of Virtual reality: • Physical Risk. Any discrepancy between what the user sees and feels could lead to motion sickness. • Security Risks. For example, if users are able to modify the appearance of their avatars, they could potentially impersonate other users. • Privacy Risks. A virtual environment can dramatically expand the scope of potential privacy violations because every single behavior in a virtual environment can be tracked, and every element of a virtual environment can be manipulated. • Price. Many experts think that the limiting factor for virtual reality is going to be the price. The reality is a bit more complicated. • Value for customers. Consumer appetite for VR is hard to predict and need to be stimulated. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  36. 36. Challenges and barriers of VR There are four really hard problems that if solved would make VR far more convincing: • Tracking • Latency • Stimulating the human perceptual system to produce results indistinguishable from the real world • Virtual reality sickness Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Barriers from technological perspective: • GPU and CPU horsepower • Input • Tracking • Computer vision • Display • Audio • Haptics • Sense of empathy • Etc…
  37. 37. Use case: GPU horsepower needed for mobile VR Goal is to calculate GPU horsepower needed for low-end but realistic mobile VR immersive HMD Information given: 1. Display Resolution: 1080p 2. Frames per second: 60-90 3. Depth of resolution: 24 bit 4. Texture information: 12 million per scene 5. Latency: <20 ms 6. Pixel Density: 300 dpi Calculation: 1. The GPU needs 50 bytes per polygon, which then is multiplied for 12 million polygons, would be 600 Megabytes per second, with 60 frames per second. 2. In order to calculate the Bandwidth, it will be : 600 megabytes x 60 frames/sec = 36 GB/sec [Pass 1]. For 90 frames/sec it would be 54 GB/sec 3. According to OpenGL and DirectX best practices, a minimum of 3 passes on each polygon must be made. This is done in order to draw the polygon, emulate its animation properties and to display its texture 37Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Tesla GPU NVIDIA Pascal GPU
  38. 38. Use case: GPU horsepower needed - Cont. Information given: 1. Display Resolution: 1080p 2. Frames per second: 60-90 3. Depth of resolution: 24 bit 4. Texture information: 12 million per scene 5. Latency: <20 ms 6. Pixel Density: 300 dpi Calculation: 4. Therefore, the bandwidth required by the GPU would be 36 GB/sec x 3 = 108 GB/sec. For 90 frames it would be 54 GB/sec x 3 = 162 GB/sec 5. Additional considerations: Texture characteristics and attributes, such as shining, shading, colour and material need to be analysed and incorporated within these parameters 6. In order to rasterize, 1 polygon, it would need around 100 floating- point operations. Therefore, for 12 million polygons, it would be: 12,000,000 x 100 flops x 60 frames/sec x 3 raster passes, it would equate to around: 216 GFLOP/ sec For 90 fps – 324 GFLOP/s Based on the information above, NVIDIA have suggested GTX 970 to be the capable GPU according to the requirements. GTX 970 has 196+28 GB/sec – 3494 GFLOP/s – 145 Wt - - $329 – June, 2014 Top is GTX Titan X – 336 GB/sec – 6144 GFLOP/s – 250 Wt - $999 – 1kg – Mar, 2015 38Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru CPU vs GPU Performance
  39. 39. Use case: Power consideration. Information given: 1. Power Required: 145 Watts per Hour 2. Duration: 1, 2, or 8 hours per cycle 3. Battery Type: Lithium Ion 39Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Therefore, energy stored in the battery for scenario 1 would be: 145 Wt*h Scenario 2: 290 Wt*h Scenario 3: 1160 Wt*h For commercial implementation, Lithium Ion gives best performance and is supported by wide industry standards. The typical energy density stored in a mobile phone battery is between 0,9 – 2,3 MJ/L. This equates to 740 Wt*h/L.
  40. 40. Use case: Battery required to power GPU for realistic scene 40Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Energy density of Lithium Ion batteries over the years, Battery University, 2014 Scenario 2: Volume = 0,000391 m3 Equal dimensions = 0,073m Scenario 3: Volume = 0,001567 m3 Equal dimensions = 0,12 m Outcome: 1. For mobile VR system with realistic graphics working for 1 hour we will need battery with 6 cm x 6 cm x 6 cm dimensions. 2. For mobile VR HMD working at least 8 hours we will need a battery with 12 cm x 12 cm x 12 cm.
  41. 41. Complexities within VR & AR IMAGE DISPLAY IMAGE RENDERING DRIVER PROCESSING on CPU SPATIAL AUDIO PIXEL PERSISTENCE HEAD TRACKING HAPTICS INPUT BODY TRACKING Graphics Sub- system VR Headset Other Peripherals Based on the data obtained from AMD, with Oculus being the benchmark standard, it is evident from the information table above, that a ordinary gaming PC would be unable to suffice the demanding computations for this technology. Additionally, in order to improve the visual experience the whole sub- system, starting from input control to image display needs to be optimised for minimal latency and maximum throughput. This includes, rendering in 2D, 3D and immersing displays with texture based rendering. 41Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  42. 42. CPU and GPU Performance over time The adjacent CPU/GPU graph suggests, using Moore’s Law, coupled with Nanotechnology together with advancements in parallel computation has boosted the overall performance of GPU in relation to graphic intensive computation. Based on both graphs, it can be inferred that price of computing will do down significantly, whilst the performance gains are expected to be exponential over time. According to the computation by AI Impact, it suggests that the price of a gigaFLOPS falls by an order of magnitude roughly every four years (14 orders of magnitude in 54 years is 3.9 years per order of magnitude). Since 1997, each order of magnitude only took three years (5.7 orders of magnitude in 18 years). 42Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  43. 43. Key outcome of the GPU and Battery case GPU horsepower case 1. There are cards that already can satisfy minimum requirements for realistic immersive VR systems. But they will work only with high-end laptops and PC for the next 5 years. 2. According to current trends the GPU horsepower will be increasing, but the power and weight will also be increasing – 250 Wt - $1000 price and 1 kg weight are limiting factors for mobile VR. 3. This leads us to the total cost of VR system for the end user (gaming market). It will consist of laptop + high-end GPU + HMD + content = $2000 + $1000 + $ 1500 + $1000 = $5500 per year per 10 pieces of content. A bit expensive, isn’t it? 4. Mobile VR will be developing with low-end graphics and will not give opportunity to be totally immersed. 5. Mixed systems will be developing where computational power will be in the backpack of the gamer Battery case 1. Batteries will still be the problematic part of the equation. The energy density/weight and energy density/volume ratios are progressing not so fast as energy density/cost ratio. It’s very critical issue for mobile VR systems and will not be solved for the next 5-10 years according to current trends in technology. 43Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru The only way I see is rethinking of computing architectures. What AMD, Nvidia, and Intel already are doing….
  44. 44. Progress within VR & AR “LiquidVR is a silky smooth player” – The Guardian “GameWorks VR: Smarter graphics for virtual reality” - CNET Recently, AMD has launched Project: LiquidVR® dedicated to VR and AR technology. According to AMD, GPU software and hardware sub-systems to tackle the common issues and pitfalls of achieving presence, such as reducing motion-to-photon latency to less than 10 milliseconds. In other developing events, Intel has partnered with Oculus Rift to form VR alliance in association with Altspace VR to establish a common platform for better user experience. Additionally, 6th generation Intel processors have incorporated Intel RealSense® Technology which is a gesture based Human- Computer Interaction model to process and compute AR/VR information. Microsoft has also made extensive efforts to incorporate VR and AR techno in the latest release of Windows 10, and are working on releasing the HoloLens (VR HMD) device logy in 2015. 44Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  45. 45. Challenges and barriers of AR These are AR technological barriers: • Wide-field-of-view see-through display • Input • Mobility • GPU and CPU horsepower • Battery • Room-scale tracking • Walk-around tracking Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru The ODG Smart Glasses Source: ODG Group
  46. 46. Critical Success Factors : Vision 2020 Cheaper cost of devices Wider availability of content Elimination of motion sickness Portability Increased battery life Affordable Manufacturing Affordable Price Mobility Vision Immersion Usability Flexibility Wearability “AR needs to become more mobile than mobile to compete,“ – Binocular IO “While internal market competition is inevitable, the fight that really matters is between AR/VR’s installed base and the huge incumbent smartphone and tablet market” – DIGI Capital 46Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  47. 47. What are key areas for innovation in AR & VR? VR will kick off graphics into hyperdrive : • 90 hz stereo, Can't miss frames • 110 fov • Resolution of 10k x 10k • High rendering quality • New rendering techniques • GPU performance 2 to 3 orders of magnitude • Transmission links • Display ( 100 megapixels and more, thousands of pixels per inch, HDR) • Rethinking art and animation technology • Optics • Rendering engines and content creation platforms Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  48. 48. What are key areas for innovation in AR & VR? • Wide field of view (immersion, peripheral cues, 100-110 degrees) • Adequate resolution (problematic for VR due to wide FOV, one-seventh the pixel density per degree of a wide-screen TV - 26p, 1080p works, more is better) • Low pixel persistence (necessary to avoid blurring with eye motion, no more than 3 ms) • A high enough refresh rate (to avoid flicker with low persistance, 95 Hz seems to be sufficient) • Global display (illuminates all pixels simultaneously, avoid motion-induced compression, stretching, and skewing) • Optics (only one or at most two lenses per eye, not enough to allow simultaneous control of focal length, viewing distance, size, distortion, various forms of aberration) • Mobility Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  49. 49. What are key areas for innovation in AR & VR? • Optical calibration ( the human visual system is amazingly sensitive to deviations, many problems are impossible to identify until calibration is done ) • Rock-solid tracking (must support translation (position in x, y, z) as well as orientation, position with at least milimeter accuracy, orientation with at least quarter-degree accuracy, volume at least 1,5 meters on a side) • Low latency (20 ms motion-to-last-photon works, 25 ms may be good enough) • Resolution (The math for resolution together with wide fields of view is brutal; divide 1K by 1K resolution – about the best an affordable head mounted display is likely to be able to do in the next year – into a 100 - degree FOV and you get a display with less than 1/50th the pixel density of a phone at normal viewing distance) • Input Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  50. 50. Implications and how it will change any aspect of our life VR will influence all aspects that it touches: • Presentation • Art and creation methods • Ergonomics • Productivity (the way we work) • Locomotion • Social interaction • Total immersion and mixed reality Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  51. 51. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru What is total immersion? Mixed reality as a new media…
  52. 52. Total Immersion Required Characteristics Unique characteristics of immersive virtual reality: • Head-referenced viewing provides a natural interface for the navigation in three-dimensional space and allows for look-around, walk-around, and fly-through capabilities in virtual environments. • Stereoscopic viewing enhances the perception of depth and the sense of space. • The virtual world is presented in full scale and relates properly to the human size. • Realistic interactions with virtual objects via data glove and similar devices allow for manipulation, operation, and control of virtual worlds. • The convincing illusion of being fully immersed in an artificial world can be enhanced by auditory, haptic, and other nonvisual technologies. • Networked applications allow for shared virtual environments. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  53. 53. What is total immersion? Mixed reality as a new media… Technical requirements: • Face tracking • Eye tracking • Head tracking • Hands tracking • Body tracking • Position tracking Apply that to virtual avatar: • Input • Audio • Interaction • Haptics • Perceptual research : psychology and other human factors Multimodal interactions are key to VR: • Feedback loop (see the object, touch it, sense it) • Correspondence between insights Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  54. 54. AR & VR as Next Computing Platform • Drives the perceptual system as its built to be driven • Different and more powerful mode of interaction • Proper superset of all existing means of communication • In the limit, its as big as reality ... or bigger Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  55. 55. What the future of VR & AR will look like? ● Virtual reality will be most popular in the gaming market, because of the high level of immersion possible in VR, as well as the high demand for interactive experiences. ● Augmented reality will be most successful in the enterprise market, for applications in logistics, engineering, and automotive. ● Applications such as education, travel, and design are served well by both AR and VR, and success depends more on specific needs than general application. ● New content mediums, such as interactive VR video, are becoming more popular and catching the attention of content creators as a potentially viable medium. Existing software solutions, such as CAD software, can be outfitted to support AR and VR devices, and open a new approach to using the software. Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  56. 56. What's next? Possible roadmaps Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru Scenario 1. • VR will be the first in mass adoption. • The roadmap for VR is likely to be pretty straightforward: gaming market, entertainment market. Scenario 2. • AR will be second and after 2020. • Industrial applications of AR find it’s slow way to the market overcoming technical & organizational barriers. • Information glasses succeed. Scenario 3. • Console AR revolutionizes living-room gaming. • Next step – whole new computing platform. Laptops and PCs are replaced by HMD.
  57. 57. The future of VR & AR “The return of virtual reality: 'this is as big an opportunity as the internet” – The Guardian “The Future Of Virtual Reality Might Be Present” – The Forbes “Another use case for VR photography could be news journalism” – ABC News “Virtual Reality is inevitable” - Palmer Luckey “Ten years ago, this capability was purely in the realm of futurists and seemed like a dream. Now, it feels like a very real possibility that will enable us to experience nature, architecture, history and culture in new and meaningful ways, and possibly lead to a new renaissance of art and discovery”. – The Washington Post “The Future Is Right In Front Of Our Eyes” - TechCrunch “A natural next step for VR is full-immersion home entertainment” - TechCrunch As we progress into the 21st Century. Augmented and Virtual Reality will play an integral part of our lives, helping and assisting us in every way imaginable. Given the significant advancements within the same, this technology is set to become multi- faceted stream of communication sectors, from news media to transport and logistics to name a few. 57Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
  58. 58. Our Team 58 Vasily Ryzhonkov, Director of Mobile Tech Center, Skolkovo Foundation vryzhonkov@sk.ru http://mobile.sk.ru Joshua Levy Intern at Mobile Tech Center, Skolkovo Foundation 58Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

×