Virtual Reality & Sim Racing in Assetto Corsa - Romagnoli

1,777 views

Published on

Slides from Giovanni Romagnoli talk @ codemotion roma 2014

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,777
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Virtual Reality & Sim Racing in Assetto Corsa - Romagnoli

  1. 1. ROME 11-12 april 2014ROME 11-12 april 2014 Virtual Reality & Sim Racing in Assetto Corsa g.romagnoli@kunos-simulazioni.com Giovanni Romagnoli Kunos Simulazioni
  2. 2. VR & SimRacing in Assetto CorsaROME 11-12 april 2014
  3. 3. VR & SimRacing in Assetto CorsaROME 11-12 april 2014
  4. 4. 1. Assetto Corsa VR & SimRacing in Assetto Corsa I’ll be talking about : 1) Assetto Corsa 2) VR & SimRacing 3) Stereo Theory 4) Implementation
  5. 5. 1. Assetto Corsa VR & SimRacing in Assetto Corsa 1. Assetto Corsa
  6. 6. 1. Assetto Corsa VR & SimRacing in Assetto Corsa Simulated racing is a type of software that attempts to accurately simulate auto racing. Kunos Simulazioni 13 people working 3 Games published
  7. 7. 1. Assetto Corsa VR & SimRacing in Assetto Corsa Latest product : Assetto Corsa proprietary technology Graphic, Physics, Sound engine, …
  8. 8. 1. Assetto Corsa VR & SimRacing in Assetto Corsa 1) Consumer SimRacers & Casual 2 different market categories
  9. 9. 1. Assetto Corsa VR & SimRacing in Assetto Corsa 2) Professional Companies interested in vehicle simulation 2 different market categories
  10. 10. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 2. SimRacing & VR
  11. 11. 2. SimRacing & VR VR & SimRacing in Assetto Corsa On the professional VR isn’t the “new thing” VR is a computer-created environment that can simulate the physical presence of the user in another world Presence VS Immersion VR = Physical + Display
  12. 12. 2. SimRacing & VR VR & SimRacing in Assetto Corsa
  13. 13. 2. SimRacing & VR VR & SimRacing in Assetto Corsa
  14. 14. 2. SimRacing & VR VR & SimRacing in Assetto Corsa
  15. 15. 2. SimRacing & VR VR & SimRacing in Assetto Corsa State of the art : multiple screen rendering Let’s focus on how we display information Physical feedback is a solved problem
  16. 16. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Single screen rendering
  17. 17. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Triple screen rendering
  18. 18. 2. SimRacing & VR VR & SimRacing in Assetto Corsa 1) Reduce the problem to 2D 2) Make math work: For each Screen : Angle = asin ( dir ); VM = rotate( VM, Angle); Screen = rotate(Screen, -Angle); Proj(Screen, NearPlane); 3) Create the Off-Center Proj-Matrix It’s easy to do : dir
  19. 19. 2. SimRacing & VR VR & SimRacing in Assetto Corsa Triple screen drawbacks : 1) Expensive rendering 2) Expensive setup 3) Merge & Blending 4) No presence 5) (brute force)
  20. 20. 2. SimRacing & VR VR & SimRacing in Assetto Corsa What’s the alternative to multiple rendering? Let’s start to talk about optics…
  21. 21. 3. Stereo Theory VR & SimRacing in Assetto Corsa 3. Stereo Theory
  22. 22. 3. Stereo Theory VR & SimRacing in Assetto Corsa 3.1 Optics How we perceive DEPTH 2 signals Focal Plane + 2D info+
  23. 23. 3. Stereo Theory VR & SimRacing in Assetto Corsa Convergence : The meeting of lines of sight through the eyes at a common point 2 signals :
  24. 24. 3. Stereo Theory VR & SimRacing in Assetto Corsa No FusionFusionFusion
  25. 25. 3. Stereo Theory VR & SimRacing in Assetto Corsa Stereoscopy Stereoscopy is a technique for creating the illusion of depth in an image by means of binocular vision. 2 signals Focal Plane + 2D info+
  26. 26. 3. Stereo Theory VR & SimRacing in Assetto Corsa
  27. 27. 3. Stereo Theory VR & SimRacing in Assetto Corsa Safe distance : Closest point > Stereo base * focal length. Other consequences of fixed focal plane :
  28. 28. 3. Stereo Theory VR & SimRacing in Assetto Corsa There are several ways to achieve Stereoscopy
  29. 29. 3. Stereo Theory VR & SimRacing in Assetto Corsa The device : Oculus Rift
  30. 30. 3. Stereo Theory VR & SimRacing in Assetto Corsa It’s simple : • OLED Display 75 Hz, low persistence • Lenses to adjust the image to what the viewer expects • 60 Hz Positional tracking • 1000 Hz update rate for orientation
  31. 31. 3. Stereo Theory VR & SimRacing in Assetto Corsa Why not Before? • Fast switching displays • Computer Vision algorithms for positional tracking • Computing power wasn’t there Latency Price
  32. 32. 3. Stereo Theory VR & SimRacing in Assetto Corsa DK2 and SDK 2 way more powerful respect to DK1! Higher level of abstraction, tons of utilites e.g.: set low persistence mode sensorDevice->SetDisplayReport(…); e.g.: get user info pHMD->GetProfile() How do we develop on it?
  33. 33. 3. Stereo Theory VR & SimRacing in Assetto Corsa …and there’s also a c interface for the library
  34. 34. 4. Implementation VR & SimRacing in Assetto Corsa 4. Implementation
  35. 35. 4. Implementation VR & SimRacing in Assetto Corsa Problematics to deal with Oculus Rift 1) Rendering in stereo 2) Resolution 3) Latency 4) GUI 5) Motion Sickness
  36. 36. 4. Implementation VR & SimRacing in Assetto Corsa 4.1) Rendering Stereo
  37. 37. 3. Stereo Theory VR & SimRacing in Assetto Corsa 3D Device User
  38. 38. 3. Stereo Theory VR & SimRacing in Assetto Corsa View Matrix Projection Matrix Distortion
  39. 39. 4. Implementation VR & SimRacing in Assetto Corsa Rendering in stereo : view matrix We need to render twice the scene with 2 different view matrix with no convergence Just shift the different view matrices by the interpupillary distance V’ = T V
  40. 40. 4. Implementation VR & SimRacing in Assetto Corsa Rendering in stereo : proj matrix a = aspect ratio FOV :
  41. 41. 4. Implementation VR & SimRacing in Assetto Corsa The problem is that the FOV in humans is not symmetric! We need to reason in terms of half fov (s) and set our proj matrix accordingly.
  42. 42. 4. Implementation VR & SimRacing in Assetto Corsa
  43. 43. 4. Implementation VR & SimRacing in Assetto Corsa Rendering in Stereo : distortion New SDK function to do this right (lens-independent) Pincushion Barrel Physical Software
  44. 44. 4. Implementation VR & SimRacing in Assetto Corsa The new SDK makes our life easier : It gives you a 3d mesh where to map your render target. (+ corrects for chromatic Ab.) It’s easier to use & improve, for example adding a fade effect on the border
  45. 45. 4. Implementation VR & SimRacing in Assetto Corsa Trivial shader VS_OUTPUT ksVS(VS_INPUT input) { VS_OUTPUT Output; Output.TexCoord0 =float2(RED.X,RED.Y) ; […] Output.Color = float4(0,0,0,alpha); return Output; } float4 ksPS(VS_OUTPUT In) : SV_TARGET { float ResultR = Texture.Sample(Linear, In.TexCoord0).r; float ResultG = Texture.Sample(Linear, In.TexCoord1).g; float ResultB = Texture.Sample(Linear, In.TexCoord2).b; return float4(ResultR * In.Color.r, ResultG * In.Color.g, ResultB * In.Color.b, 1.0); }
  46. 46. 4. Implementation VR & SimRacing in Assetto Corsa
  47. 47. 4. Implementation VR & SimRacing in Assetto Corsa 4.2) Resolution
  48. 48. 4. Implementation VR & SimRacing in Assetto Corsa Resolution We used : Downsampling MSAA 2X FXAA 2X EYE : 1 pixel per 5/1000 degrees. OCULUS : 1 pixel per 1/6 degree Screen structure : Pixel Mask Grid vs Brick
  49. 49. 4. Implementation VR & SimRacing in Assetto Corsa 4.3) Latency
  50. 50. 4. Implementation VR & SimRacing in Assetto Corsa Latency & performance motion-to-photon time= “time between movement of the user’s head and the updated image being displayed on the screen” <= 20ms. Eye : Fast, really fast (900 degrees/s) Many talks on the subject
  51. 51. 4. Implementation VR & SimRacing in Assetto Corsa How AC engine works: Shadow Map Pass Cube Map Pass Main render Skybox Opaque Transparent Camera Matrix Post Process Pass GUI Render Time
  52. 52. 4. Implementation VR & SimRacing in Assetto Corsa Shadow Map Pass Cube Map Pass Main render Pass Distortion & Post Process Pass Skybox Opaque Transparent GUI For each eye get Eye Camera Matrix For Each eye : Render Time
  53. 53. 4. Implementation VR & SimRacing in Assetto Corsa Shadow Map Pass Cube Map Pass Opaque Transparent GUI Fetch data Middle Eye Main render Pass Fetch data : for each eye (alternate) Render Time Skybox Distortion & Post Process Pass RT2
  54. 54. 4. Implementation VR & SimRacing in Assetto Corsa 4.4) GUI
  55. 55. 4. Implementation VR & SimRacing in Assetto Corsa
  56. 56. 4. Implementation VR & SimRacing in Assetto Corsa GUI : This problem is quite difficult to deal in AC Remember the stereo rule : closest point = stereo base * focal length About 50 cm with standard oculus settings
  57. 57. 4. Implementation VR & SimRacing in Assetto Corsa It could be very clean…
  58. 58. 4. Implementation VR & SimRacing in Assetto Corsa … or very messy.
  59. 59. 4. Implementation VR & SimRacing in Assetto Corsa HUD : This is a problem that is quite difficult to deal with : Direct Approach : FAILED Just the minimum info : FAILED Remember, the objective is PRESENCE
  60. 60. 4. Implementation VR & SimRacing in Assetto Corsa
  61. 61. 4. Implementation VR & SimRacing in Assetto Corsa
  62. 62. 4. Implementation VR & SimRacing in Assetto Corsa 4.5) Simulation Sickness
  63. 63. 4. Implementation VR & SimRacing in Assetto Corsa Simulation Sickness : It happends when a disagreement exists between visually perceived movement and the vestibular system's sense of movement Follow the Oculus Best Practices!
  64. 64. 4. Implementation VR & SimRacing in Assetto Corsa The problem is that games like AC breaks most of the Oculus Best Practices! The experience turns out to be “not that bad”at first Acceleration, movement of the camera, …
  65. 65. 4. Implementation VR & SimRacing in Assetto Corsa Head Model
  66. 66. 4. Implementation VR & SimRacing in Assetto Corsa Calibrate the camera for each car / 3D driver
  67. 67. 4. Implementation VR & SimRacing in Assetto Corsa
  68. 68. 4. Implementation VR & SimRacing in Assetto Corsa
  69. 69. 4. Implementation VR & SimRacing in Assetto Corsa
  70. 70. 4. Implementation VR & SimRacing in Assetto Corsa The resetting dilemma : It took a long time to figure out a very simple thing : the user should only be able to reset yaw. We want a 1:1 mapping between head movement & camera movement
  71. 71. 4. Implementation VR & SimRacing in Assetto Corsa Attenuancy of car pitch movement for small HF values Up vector Up vector No problems for yaw, roll : the user expects them while in a car Spline normal
  72. 72. 4. Implementation VR & SimRacing in Assetto Corsa Faded Render Borders Removed any Bloom effects Removed Motion Blur
  73. 73. 4. Implementation VR & SimRacing in Assetto Corsa
  74. 74. 4. Implementation VR & SimRacing in Assetto Corsa
  75. 75. 4. Implementation VR & SimRacing in Assetto Corsa Use as much help from Oculus SDK as possible Prediction : fixed to 40 ms (iRacing 20ms) User profile : take all the data from there Drift Correction : just on DK1
  76. 76. 4. Implementation VR & SimRacing in Assetto Corsa VR Legs The programmer gets easily used to non-correct stereo rendering
  77. 77. 4. Implementation VR & SimRacing in Assetto Corsa
  78. 78. 4. Implementation VR & SimRacing in Assetto Corsa
  79. 79. 4. Implementation VR & SimRacing in Assetto Corsa More on motion sickness : Use testers! Force your users to play in a defined setting Haptic feedback is a blessing
  80. 80. 4. Implementation VR & SimRacing in Assetto Corsa There’s nothing you can do to combat motion on this: 1) Low resolution -> high distances mess 2) Blur when you turn your head quickly or your eyes moves quickly on the screen (display latency)
  81. 81. 4. Implementation VR & SimRacing in Assetto Corsa Open points - Motion sickness ? - Professional use ?
  82. 82. 4. Implementation VR & SimRacing in Assetto Corsa Thanks ! I’ll be in the exhibition area with the device. Do you have any questions?

×