Virtual Reality & Sim Racing in Assetto Corsa - Romagnoli
Upcoming SlideShare
Loading in...5
×
 

Virtual Reality & Sim Racing in Assetto Corsa - Romagnoli

on

  • 408 views

Slides from Giovanni Romagnoli talk @ codemotion roma 2014

Slides from Giovanni Romagnoli talk @ codemotion roma 2014

Statistics

Views

Total Views
408
Views on SlideShare
408
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Virtual Reality & Sim Racing in Assetto Corsa - Romagnoli Virtual Reality & Sim Racing in Assetto Corsa - Romagnoli Presentation Transcript

    • ROME 11-12 april 2014ROME 11-12 april 2014 Virtual Reality & Sim Racing in Assetto Corsa g.romagnoli@kunos-simulazioni.com Giovanni Romagnoli Kunos Simulazioni
    • VR & SimRacing in Assetto CorsaROME 11-12 april 2014
    • VR & SimRacing in Assetto CorsaROME 11-12 april 2014
    • 1. Assetto Corsa VR & SimRacing in Assetto Corsa I’ll be talking about : 1) Assetto Corsa 2) VR & SimRacing 3) Stereo Theory 4) Implementation
    • 1. Assetto Corsa VR & SimRacing in Assetto Corsa 1. Assetto Corsa
    • 1. Assetto Corsa VR & SimRacing in Assetto Corsa Simulated racing is a type of software that attempts to accurately simulate auto racing. Kunos Simulazioni 13 people working 3 Games published
    • 1. Assetto Corsa VR & SimRacing in Assetto Corsa Latest product : Assetto Corsa proprietary technology Graphic, Physics, Sound engine, …
    • 1. Assetto Corsa VR & SimRacing in Assetto Corsa 1) Consumer SimRacers & Casual 2 different market categories
    • 1. Assetto Corsa VR & SimRacing in Assetto Corsa 2) Professional Companies interested in vehicle simulation 2 different market categories
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa 2. SimRacing & VR
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa On the professional VR isn’t the “new thing” VR is a computer-created environment that can simulate the physical presence of the user in another world Presence VS Immersion VR = Physical + Display
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa State of the art : multiple screen rendering Let’s focus on how we display information Physical feedback is a solved problem
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa Single screen rendering
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa Triple screen rendering
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa 1) Reduce the problem to 2D 2) Make math work: For each Screen : Angle = asin ( dir ); VM = rotate( VM, Angle); Screen = rotate(Screen, -Angle); Proj(Screen, NearPlane); 3) Create the Off-Center Proj-Matrix It’s easy to do : dir
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa Triple screen drawbacks : 1) Expensive rendering 2) Expensive setup 3) Merge & Blending 4) No presence 5) (brute force)
    • 2. SimRacing & VR VR & SimRacing in Assetto Corsa What’s the alternative to multiple rendering? Let’s start to talk about optics…
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa 3. Stereo Theory
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa 3.1 Optics How we perceive DEPTH 2 signals Focal Plane + 2D info+
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa Convergence : The meeting of lines of sight through the eyes at a common point 2 signals :
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa No FusionFusionFusion
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa Stereoscopy Stereoscopy is a technique for creating the illusion of depth in an image by means of binocular vision. 2 signals Focal Plane + 2D info+
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa Safe distance : Closest point > Stereo base * focal length. Other consequences of fixed focal plane :
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa There are several ways to achieve Stereoscopy
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa The device : Oculus Rift
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa It’s simple : • OLED Display 75 Hz, low persistence • Lenses to adjust the image to what the viewer expects • 60 Hz Positional tracking • 1000 Hz update rate for orientation
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa Why not Before? • Fast switching displays • Computer Vision algorithms for positional tracking • Computing power wasn’t there Latency Price
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa DK2 and SDK 2 way more powerful respect to DK1! Higher level of abstraction, tons of utilites e.g.: set low persistence mode sensorDevice->SetDisplayReport(…); e.g.: get user info pHMD->GetProfile() How do we develop on it?
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa …and there’s also a c interface for the library
    • 4. Implementation VR & SimRacing in Assetto Corsa 4. Implementation
    • 4. Implementation VR & SimRacing in Assetto Corsa Problematics to deal with Oculus Rift 1) Rendering in stereo 2) Resolution 3) Latency 4) GUI 5) Motion Sickness
    • 4. Implementation VR & SimRacing in Assetto Corsa 4.1) Rendering Stereo
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa 3D Device User
    • 3. Stereo Theory VR & SimRacing in Assetto Corsa View Matrix Projection Matrix Distortion
    • 4. Implementation VR & SimRacing in Assetto Corsa Rendering in stereo : view matrix We need to render twice the scene with 2 different view matrix with no convergence Just shift the different view matrices by the interpupillary distance V’ = T V
    • 4. Implementation VR & SimRacing in Assetto Corsa Rendering in stereo : proj matrix a = aspect ratio FOV :
    • 4. Implementation VR & SimRacing in Assetto Corsa The problem is that the FOV in humans is not symmetric! We need to reason in terms of half fov (s) and set our proj matrix accordingly.
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa Rendering in Stereo : distortion New SDK function to do this right (lens-independent) Pincushion Barrel Physical Software
    • 4. Implementation VR & SimRacing in Assetto Corsa The new SDK makes our life easier : It gives you a 3d mesh where to map your render target. (+ corrects for chromatic Ab.) It’s easier to use & improve, for example adding a fade effect on the border
    • 4. Implementation VR & SimRacing in Assetto Corsa Trivial shader VS_OUTPUT ksVS(VS_INPUT input) { VS_OUTPUT Output; Output.TexCoord0 =float2(RED.X,RED.Y) ; […] Output.Color = float4(0,0,0,alpha); return Output; } float4 ksPS(VS_OUTPUT In) : SV_TARGET { float ResultR = Texture.Sample(Linear, In.TexCoord0).r; float ResultG = Texture.Sample(Linear, In.TexCoord1).g; float ResultB = Texture.Sample(Linear, In.TexCoord2).b; return float4(ResultR * In.Color.r, ResultG * In.Color.g, ResultB * In.Color.b, 1.0); }
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa 4.2) Resolution
    • 4. Implementation VR & SimRacing in Assetto Corsa Resolution We used : Downsampling MSAA 2X FXAA 2X EYE : 1 pixel per 5/1000 degrees. OCULUS : 1 pixel per 1/6 degree Screen structure : Pixel Mask Grid vs Brick
    • 4. Implementation VR & SimRacing in Assetto Corsa 4.3) Latency
    • 4. Implementation VR & SimRacing in Assetto Corsa Latency & performance motion-to-photon time= “time between movement of the user’s head and the updated image being displayed on the screen” <= 20ms. Eye : Fast, really fast (900 degrees/s) Many talks on the subject
    • 4. Implementation VR & SimRacing in Assetto Corsa How AC engine works: Shadow Map Pass Cube Map Pass Main render Skybox Opaque Transparent Camera Matrix Post Process Pass GUI Render Time
    • 4. Implementation VR & SimRacing in Assetto Corsa Shadow Map Pass Cube Map Pass Main render Pass Distortion & Post Process Pass Skybox Opaque Transparent GUI For each eye get Eye Camera Matrix For Each eye : Render Time
    • 4. Implementation VR & SimRacing in Assetto Corsa Shadow Map Pass Cube Map Pass Opaque Transparent GUI Fetch data Middle Eye Main render Pass Fetch data : for each eye (alternate) Render Time Skybox Distortion & Post Process Pass RT2
    • 4. Implementation VR & SimRacing in Assetto Corsa 4.4) GUI
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa GUI : This problem is quite difficult to deal in AC Remember the stereo rule : closest point = stereo base * focal length About 50 cm with standard oculus settings
    • 4. Implementation VR & SimRacing in Assetto Corsa It could be very clean…
    • 4. Implementation VR & SimRacing in Assetto Corsa … or very messy.
    • 4. Implementation VR & SimRacing in Assetto Corsa HUD : This is a problem that is quite difficult to deal with : Direct Approach : FAILED Just the minimum info : FAILED Remember, the objective is PRESENCE
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa 4.5) Simulation Sickness
    • 4. Implementation VR & SimRacing in Assetto Corsa Simulation Sickness : It happends when a disagreement exists between visually perceived movement and the vestibular system's sense of movement Follow the Oculus Best Practices!
    • 4. Implementation VR & SimRacing in Assetto Corsa The problem is that games like AC breaks most of the Oculus Best Practices! The experience turns out to be “not that bad”at first Acceleration, movement of the camera, …
    • 4. Implementation VR & SimRacing in Assetto Corsa Head Model
    • 4. Implementation VR & SimRacing in Assetto Corsa Calibrate the camera for each car / 3D driver
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa The resetting dilemma : It took a long time to figure out a very simple thing : the user should only be able to reset yaw. We want a 1:1 mapping between head movement & camera movement
    • 4. Implementation VR & SimRacing in Assetto Corsa Attenuancy of car pitch movement for small HF values Up vector Up vector No problems for yaw, roll : the user expects them while in a car Spline normal
    • 4. Implementation VR & SimRacing in Assetto Corsa Faded Render Borders Removed any Bloom effects Removed Motion Blur
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa Use as much help from Oculus SDK as possible Prediction : fixed to 40 ms (iRacing 20ms) User profile : take all the data from there Drift Correction : just on DK1
    • 4. Implementation VR & SimRacing in Assetto Corsa VR Legs The programmer gets easily used to non-correct stereo rendering
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa
    • 4. Implementation VR & SimRacing in Assetto Corsa More on motion sickness : Use testers! Force your users to play in a defined setting Haptic feedback is a blessing
    • 4. Implementation VR & SimRacing in Assetto Corsa There’s nothing you can do to combat motion on this: 1) Low resolution -> high distances mess 2) Blur when you turn your head quickly or your eyes moves quickly on the screen (display latency)
    • 4. Implementation VR & SimRacing in Assetto Corsa Open points - Motion sickness ? - Professional use ?
    • 4. Implementation VR & SimRacing in Assetto Corsa Thanks ! I’ll be in the exhibition area with the device. Do you have any questions?