Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

14,263 views

Published on

Talk by Sebastien Hillaire at SIGGRAPH 2016 in the Physically Based Shading in Theory and Practice course.

  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

  1. 1. empowers game creators to shape the future of gaming
  2. 2. Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite SEBASTIEN HILLAIRE - ELECTRONIC ARTS / FROSTBITE
  3. 3. 3SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice An EA wide collaboration within Frostbite  Sky and atmosphere systems  Master’s thesis: Gustav Bodare & Edvard Sandberg (Ghost)  Bioware (Mass Effect Andromeda), DICE (Mirror’s Edge Catalyst), Ghost (Need for Speed)  Volumetric cloud systems  Master’s thesis: Rurik Högfeldt  Bioware: Marc-Andre Loyer (Programmer), Don Arceta (Lead Environment Art) and Soren Hesse (Tech Environment Art)  The rendering research community
  4. 4. 4SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Context  Physically based rendering in Frostbite  Transition done in 2014 [Lagarde&deRousiers14]  Huge increase in visual quality
  5. 5. 5SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Context  Previous techniques in Frostbite  Sky  HDRI captures, color gradients  Atmosphere  Depth, height fog, screen space light shafts  Clouds  Panoramic texture + Visual flow [Guerrette14]  Translated planar cloud layers  Cons  Lots of manual work to get components to match  Not dynamic, difficult to achieve time of day
  6. 6. 6SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Real-life sky, atmosphere and clouds With dynamic time of day and evolving weather Photo by DAVID ILIFF. License: CC-BY-SA 3.0
  7. 7. 7SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Scope and motivation  Dynamic time of day and evolving weather at 60 FPS  Physically based rendering  Meaningful material parameters  Decouple material from lighting  Unified interactions  Sky, atmosphere and cloud interact together  Interaction with opaque and transparent objects, global illumination, etc.
  8. 8. 8SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Mirror’s Edge Catalyst - DICE [Christin16]
  9. 9. 9SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice
  10. 10. 10SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline  Sky and atmosphere rendering  Sun rendering  Volumetric cloud rendering Photo by DAVID ILIFF. License: CC-BY-SA 3.0
  11. 11. 11SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline  Sky and atmosphere rendering   Photo by DAVID ILIFF. License: CC-BY-SA 3.0
  12. 12. 12SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky and atmosphere Atmosphere Rayleigh scattering Mie scattering Components
  13. 13. 13SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Previous work  Analytical models [CIESky][Hosek12][Preetham99]  Simpler but limited to ground view  Spectral rendering [Elek09][Preetham99]  Sky atmosphere simulation [Bruneton08]  Pre integrated multi-scattering/absorption in 4D LUTs  Scattering occlusion from terrain  Ignore earth shadow  3D LUTs [Elek09]  Improve LUT parameterization [Yusov13] [Bruneton08] [Hosek12]
  14. 14. 14SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky – algorithm overview Transmittance LUT (h, θv) Single scattering LUT (h, θv, θl) L θl V θv H h Final multi ScatteringLUT (Rayleigh, Mie) Accumulate scattering order Scattered light integration N-order scattering [Bruneton08] [Bruneton08] [Elek09] [Yusov13]
  15. 15. 15SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Atmosphere simulation coefficients [Nishita93][Riley04][Bruneton08][Adams74][Kutz12] Type Scattering (𝒎−𝟏 ) Extinction (𝒎−𝟏 ) distribution Rayleigh (standard air) σ 𝒔 𝒓𝒂𝒚 = 𝟓. 𝟖𝑒−6, 𝟏. 𝟑𝟓𝑒−5, 𝟑. 𝟑𝟏𝑒−5 σ 𝒔 𝒓𝒂𝒚 𝒆 −𝒉 𝟖.𝟎 𝒌𝒎 Mie (pollution, water, dust, etc.) σ 𝒔 𝒎𝒊𝒆 >= 3𝑒−6 a × σ 𝒔 𝒎𝒊𝒆 𝒆 −𝒉 𝟏.𝟐 𝒌𝒎 Ozone --- σ 𝒂 𝒐𝒛𝒐 𝒆 −𝒉 𝟖.𝟎 𝒌𝒎 a = 1.11 σ 𝒂 𝒐𝒛𝒐 = 𝟑. 𝟒𝟐𝟔, 𝟖. 𝟐𝟗𝟖, 𝟎. 𝟑𝟓𝟔 𝒎−𝟏 × 𝟔𝒆−𝟓% [Kutz12] [Bruneton08]
  16. 16. Rayleigh scattering examples
  17. 17. Mie scattering example
  18. 18. 18SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Ozone absorption  “Essential […] to reproduce the blue of the zenith sky” [Adams74][Kutz12] Ozone absorption: “Essential […] to reproduce the blue of the zenith sky”[Adams74]
  19. 19. 19SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky and aerial perspective rendering  Render the sky using ScatteringLUTs(h,θv,θl)  Camera aerial perspective volumes  32x32 x 16 depth slices  Integrate scattering/transmittance for each slices  Easy to apply on transparents (per vertex evaluation) Scattered luminance Transmittance
  20. 20. 20SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Time of day - Moving sun LUTs + moving sun automatically updates:  Sky  Aerial perspective Triggers updates of  Sky environment map  Local reflection volumes  Global illumination  Light probes Sky environment lighting debug
  21. 21. 21SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky rendering performance  LUT updates  Needed when changing atmospheric properties  Distribute evaluation over several frames  Each scattering order, sub table parts  While updating, interpolate last results Complete update frame count Mean frame GPU cost 1 3.50 ms 19 0.22 ms Component GPU cost AP camera volume 32x32x16 0.05 ms Sky/AP rendering 0.42 ms LUT updates with 4th order scattering measured on XB1 Sky/Atmosphere rendering cost on XB1 720p
  22. 22. 22SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline   Sun rendering  Photo by DAVID ILIFF. License: CC-BY-SA 3.0
  23. 23. 23SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sun disk luminance and ground illuminance In Frostbite, artists specify illuminance E for sun at zenith for sun solid angle 𝜴 [Lagarde&deRousiers14] E = 100000 Lux (default sun) L outer space luminance L outer space luminance E current sun position
  24. 24. 24SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Atmosphere transmittance on sun  luminance = apply transmittance LUT per pixel to outer space luminance
  25. 25. 25SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Outline    Volumetric cloud rendering Photo by DAVID ILIFF. License: CC-BY-SA 3.0
  26. 26. 26SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Previous work  Mesh + HyperTexture [Bouthors08]  High quality but overall expensive  Splatting based rendering [Yusov14]  Particle based, depth aware blending  Specific look did not match our target  Cloud layer [Schneider15]  Completely dynamic solution  Lighting, shadowing, weather [Yusov14] [Bouthors08] [Schneider15]
  27. 27. 27SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Volumetric cloud rendering  Similar to [Schneider15]  Cloud layer with weather animation  Dynamic lighting and shadowing  Perlin-Worley noise for realistic cloud details  Beer-Powder  Improvements  Scattering energy conservation  2-lobe phase functions  Time of day and weather interaction with sky and aerial perspective [Schneider15]
  28. 28. 28SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Volumetric cloud lighting components Transmittance Ambient Sun scattering Cloud self shadow Phase function Reality
  29. 29. 29SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Scattering integration improvement  Ray marching: improved integration from [Hillaire15] float3 scattering = float3(0.0, 0.0, 0.0); float transmittance = 1.0; for (s= 0; s< StepCount; ++s) { 𝑆 = sample scattered luminance sampleTransmittance = 𝑒−𝝈𝒕 𝐷 = sample transmittance scattering += 𝑆 * transmittance transmittance *= sampleTransmittance } Wrong?  Analytically integrate lighting w.r.t. transmittance over depth D (ShaderToy demo) 0 𝐷 𝑒−𝝈𝒕 𝑥 × 𝑆 𝑑𝑥 = 𝑆−𝑆×𝑒−𝝈𝒕 𝐷 𝝈𝒕 𝝈𝒕 : extinction over range 𝑆 : scattered luminance 𝐷 : integration distance
  30. 30. 30SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice scattering += 𝑆 * transmittance transmittance *= sampleTransmittance Non energy conservative
  31. 31. 31SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice transmittance *= sampleTransmittance scattering += 𝑆 * transmittance Over attenuation dark media
  32. 32. 32SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Analytic integration 𝑆−𝑆×𝑒−𝝈𝒕 𝐷 𝝈𝒕 Energy conservative
  33. 33. 33SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Default scattering integration, 21 samples Default scattering integration, 512 samples Improved scattering integration, 21 samples Improved scattering integration, 512 samples
  34. 34. 34SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function Two-lobes HG phase function  𝒑𝒉𝒂𝒔𝒆 𝑔0, 𝑔1, α, 𝜇 = 𝑙𝑒𝑟𝑝(𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔0, 𝜇 , 𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔1, 𝜇 , α)  Tried best fit  Back-scattering still not very visible   Would require multi-scattering…  Two-lobes HG phase + artist control  More freedom to control cloud style  Default: 𝑔0 = 0.8, 𝑔1 = −0.5, α = 0.5 [Bouthors08] Cloud Henyey/Greenstein Rayleigh 0.0 0.5 1.0 1.5 2.0 2.5 3.0 10 4 0.01 1 100 Two-lobe HG fitted to cloud phase Angle Phase
  35. 35. 35SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.3 0.2 0.1 0.1 0.2 0.3 HG phase with g=0.8
  36. 36. 36SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 0.5 1.0 1.5 2.0 0.3 0.2 0.1 0.1 0.2 0.3 2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5
  37. 37. 37SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.3 0.2 0.1 0.1 0.2 0.3 HG phase with g=0.8
  38. 38. 38SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Phase function0.5 0.5 1.0 1.5 2.0 0.3 0.2 0.1 0.1 0.2 0.3 2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5
  39. 39. 39SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Aerial perspective and clouds  Must look consistent under all time of day and weather conditions Visual mismatch (missing fog on clouds) Clouds without aerial perspective
  40. 40. 40SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Aerial perspective and clouds  Per sample: expensive to evaluate  Instead sampled once on cloud front interface  Compute mean depth on cloud interface weighted by transmittance
  41. 41. 41SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Aerial perspective on cloud Clouds with aerial perspective
  42. 42. 42SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Cloud coverage and scattering influence Cloud hemisphere sampling around camera • Integrate cloud luminance • Mean cloud transmittance New AP scattered luminance = oldAP x transmittance + luminance
  43. 43. 43SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Cloud coverage and scattering influence RealityClouds with aerial perspective interactions
  44. 44. 44SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Volumetric clouds performance 720p 900p Hemisphere sampling: scattering & coverage 0.090 ms 0.090 ms Main view (with temporal re-projection) 0.720 ms 1.090 ms Planar reflection (optional) 0.035 ms 0.048 ms Total 0.835 ms 1.228 ms  Measured on XB1  Cloud main view resolution = 640x360 (720p/2)  Planar reflection resolution scale = 128x72 (720p/10)  Default quality seen in this presentation  Looking toward horizon
  45. 45. 45SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Final result Sun rise with animated clouds
  46. 46. 46SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Final result Mars blue sunset (eye-balled atmosphere) [NASA]
  47. 47. 47SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Conclusion  Physically based sky, atmosphere and cloud rendering solution  Participating media material definition  Physically based and energy conservative scattering formulation  Supports dynamic time of day and evolving weather  Simulation of interactions between sky, sun, aerial perspective and clouds  Default: earth/sun/moon physical values  Simulation parameters can all be changed to simulate extra-terrestrial worlds  Practical use in Mirror’s Edge Catalyst: [Christin16]
  48. 48. 48SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Used widely in production by EA Games Need for Speed, Ghost Mass Effect Andromeda, BiowareFIFA 2017
  49. 49. 49SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Future work  Aerial perspective scattering shadowing by opaque and translucent clouds  Can’t use epipolar sampling min/max optimisation due to translucent clouds  Directional sky ambient by using cloud local gradient as main direction?  Convolution of sky envmap with phase function for cloud ambient lighting  Dynamic weather texture for cloud animation  GI and reflection volumes update  Still a problem when it comes to delay and performance  A complex problem with different priorities for each games
  50. 50. 50SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice References [Adams74] The influence of Ozone and Aerosol on the Brightness and Color of the twilight Sky, JAS 1974. [Bouthors08] Interactive Multiple Anisotropic Scattering in Clouds, I3D08. [Bruneton08] Precomputed Atmospheric scattering, EGSR 2008. [Christin16] Lighting the City of Glass, GDC 2016. [CIESky] Spatial distribution of Daylight [Elek09] Rendering Parametrizable Planetary Atmospheres with Multiple Scattering in Real-time, CESCG 2009. [Guerrette14] Moving the heavens, GDC 2014. [Harris02] Real-Time Cloud Rendering for Games, GDC 2002. [Hestroffer98] Wavelength dependency of the Solar limb darkening, Astronomy and Physics 1998. [Hillaire15] Physically Based and Unifier Volumetric Rendering in Frostbite, SIGGRAPH 2015. [Hosek12] An Analytic Model for Full Spectral Sky-Dome Radiance, SIGGRAPH 2012 [Kutz12] Blog post, 2012. [Lagarde&deRousiers14] Moving Frostbite to PBR, SIGGRAPH 2014. [NASA] http://www.nasa.gov/multimedia/imagegallery/image_feature_347.html [Nishita93] Display of the Earth Taking into Account Atmospheric Scattering, SIGGRAPH 1993. [PBR] Physically Based Rendering book, www.pbrt.org. [Preetham99] A Practical Analytic Model for Daylight, SIGGRAPH 99. [Riley04] Efficient Rendering of Atmospheric Phenomena, EGSR 2004. [Schneider15] The Real-time Volumetric Cloudscapes of Horizon: Zero Dawn, SIGGRAPH 2015. [Yusov13] Outdoor Light Scattering, GDC 2013. [Yusov14] High-Performance Rendering of realistic Cumulus Clouds using Pre-Computed Lighting, HPG14. [Wenzel07] Real time atmospheric effects in game revisited, GDC 2007. [Wrenninge10] Oz: The Great and Volumetric, SIGGRAPH 2010.
  51. 51. 51SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Battlefield, Mass Effect, Need for Speed, FIFA, Plant vs Zombies, and many more. We are hiring!
  52. 52. 52SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Questions?  Thanks!  The Frostbite rendering Team and game teams  Stephen Hill and Stephen McAuley for the course organisation  All the reviewers  Eric Bruneton and Fabrice Neyret  For further discussions, get in touch   sebastien.hillaire@frostbite.com  https://twitter.com/SebHillaire  Additional notes and Mathematica notebook to follow soon…
  53. 53. 53SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Bonus slides
  54. 54. 54SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sky – how many scattering orders? Single Scattering 2nd scattering order 4th scattering order
  55. 55. 55SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Scattering texture packing  ScatteringLUTs(h, θv, θl) is not storing (RayScat 𝒓𝒈𝒃, MieScat 𝒓𝒈𝒃)  But (RayScat 𝒓𝒈𝒃, MieScat) MieScat 𝒓𝒈𝒃 = RayScat 𝒓𝒈𝒃 × MieScat RayScat 𝒓 × σ 𝒔 𝒓𝒂𝒚 𝒓 σ 𝒔 𝒓𝒂𝒚 (RayScat 𝒓𝒈𝒃, MieScat 𝒓𝒈𝒃) (RayScat 𝒓𝒈𝒃, MieScat)
  56. 56. 56SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Height fog  Uses scattered luminance at horizon  Seamless sky/fog transition  Per pixel coverage  Per pixel phase function  Approximations  Assumes low altitude fog  Ignores opaque shadows and self shadowing Horizon P ScatteredLightP ≈ ScatteredLightHorizon
  57. 57. 57SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sun disk limb darkening  Attenuate luminance near edge of sun disk (Default = sun model [Hestroffer98]) (Very low exposure) Nolimbdarkening Sunmodel Stronger
  58. 58. 58SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Night time  Moon and stars rendered as sprite using celestial system  Moon  Angular radius ≈ 0,24 to 0,28 degree  Luminance ≈ 2500 cd/m2  Secondary sky/atmosphere scattering source  Stars  Luminance? Angular radius?
  59. 59. 59SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Night time Sun set
  60. 60. 60SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Ambient contribution  Ambient component  Capture the sky environment without sun disk  Integrate luminance to scatter  Approximations  Phase function as uniform  Single environment color ignoring cloud self occlusion
  61. 61. 61SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Multi-scattering approximation  Use [Wrenninge10]: multi-octaves single scattering / extinction / phase Without multi-scattering
  62. 62. 62SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Multi-scattering approximation  Use [Wrenninge10]: multi-octaves single scattering / extinction / phase With multi-scattering
  63. 63. 63SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Cloud coverage and scattering influence Thick cloud + aerial perspective + hemisphere coverage/scattering Thick cloud only Clouds do not fit Thick cloud + aerial perspective Aerial perspective too bright  
  64. 64. 64SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice Sunny 16 validation  Sun illuminance at ground level: 100000  Camera: ISO 100, aperture f/16 and shutter speed 1/125

×