5 Major Challenges in Real-time Rendering (2012)

339,973 views

Published on

Talk by Johan Andersson (DICE/EA) in the Beyond Programmable Shading Course at SIGGRAPH 2012.

The other talks in the course can be found here: http://bps12.idav.ucdavis.edu/

2 Comments
35 Likes
Statistics
Notes
  • ThX
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • my roomate's aunt makes $83/hr on the laptop. She has been without work for 8 months but last month her pay was $8682 just working on the laptop for a few hours. Read more on this site...NuttÿRîçhDÖtc om
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
339,973
On SlideShare
0
From Embeds
0
Number of Embeds
321,851
Actions
Shares
0
Downloads
373
Comments
2
Likes
35
Embeds 0
No embeds

No notes for slide
  • Shader should be able to decide what to output per sample, while executing per pixelCan we keep deferred MSAA in large local store tiles?
  • Texture space shading?
  • Particles would be the last to be included in an OIT solution
  • Prohibitive to include all transparent surfaces50x+ particle overdraw commonCombine with half-resHybrid OIT: pre-sorted particles blend on closest OIT sample?
  • For performance, would use pragmatic max clamps
  • GPU Compute performance portability Challenge between to run same code efficiently on different architectures and platformsMajor changes can sometimes be requiredNot unexpected, but can be a challenge
  • 5 Major Challenges in Real-time Rendering (2012)

    1. 1. Beyond Programmable Shading Course ACM SIGGRAPH 2012 5 MAJOR CHALLENGES IN REAL-TIME RENDERING Johan Andersson, DICEBeyond Programmable Shading, SIGGRAPH 2012 1
    2. 2. Overview• What are the major challenges for us in the next 5-10 years? – Real-time rendering for games as well as other areas• Which problems do we want to solve?• What do we want to achieve & focus on?• Based on own thoughts & feedback from people in the industry Beyond Programmable Shading, SIGGRAPH 2012 2
    3. 3. Beyond Programmable Shading Course ACM SIGGRAPH 201225 5 MAJOR CHALLENGES IN REAL-TIME RENDERING Johan Andersson, DICE Beyond Programmable Shading, SIGGRAPH 2012 3
    4. 4. Challenges 2012Same as 2010!1. Cinematic Image Quality2. Illumination3. Programmability4. Production costs5. Scaling Beyond Programmable Shading, SIGGRAPH 2012 4
    5. 5. Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #1CINEMATIC IMAGE QUALITY
    6. 6. Cinematic Image Quality• Goal is to achieve Cinematic Image Quality – Same smooth & rich pictures that CG movies have• Need significant improvements to GPU primary visibility – Antialiasing – Transparency – Defocus blur – Motion blur• A future solution needs to include all together Beyond Programmable Shading, SIGGRAPH 2012 6
    7. 7. Antialiasing• Single most visible issue to improve on – Aliasing breaks the illusion – Less aliasing = more pleasing & easier to see visuals• Sources of aliasing: – Geometric aliasing – Proxy geometry aliasing (alpha test) – Shader aliasing – Mixed resolution rendering Beyond Programmable Shading, SIGGRAPH 2012 7
    8. 8. Post-process antialiasing• Lots of developments in post-AA techniques – MLAA, FXAA, SMAA and more – Good quality / performance ratio – See SIGGRAPH’11 course: “Filtering Approaches for Real-Time Anti-Aliasing” http://iryoku.com/aacourse/• But need solutions for the full problem Beyond Programmable Shading, SIGGRAPH 2012 8
    9. 9. Geometric aliasing• Current solutions: MSAA, SSAA, temporal – Fixed quality techniques, not adaptive – Problematic to scale up to very high quality• 16x MSAA is really good quality but expensive – Need high rate if using coverage masks• MSAA + deferred – Massive memory usage & bandwidth – Want access to MSAA compression surface to avoid computing ourselves – See Andrew’s talk in the course: “Intersecting Lights with Pixels” Beyond Programmable Shading, SIGGRAPH 2012 9
    10. 10. Geometric aliasing• Other alternatives? – Analytical antialiasing • Would be interesting to see more research – Pre-filtered Sparse Voxel Octrees • Requires very high resolution / large storage • See Cyril’s talk in the course: “Dynamic Sparse Voxel Octrees for Next-Gen Real-time Rendering” Beyond Programmable Shading, SIGGRAPH 2012 10
    11. 11. Shader aliasing• Shader aliasing becoming more of a problem – High-frequency specular highlights – High-frequency shadows – Amplified by HDR Bloom & Bokeh• What is needed to make sure shaders do not output aliased values? – Careful handling of derivatives when texture mapping – LEAN mapping, EVSM shadows – Wednesday: “Rock-Solid Shading: Image Stability without Sacrificing Detail” Beyond Programmable Shading, SIGGRAPH 2012 11
    12. 12. Proxy geometry aliasing• Alpha-testing for proxy geometry results in major aliasing – MSAA per-sample evaluation is costly & requires many samples• Real transparency can directly solve the aliasing – But we need to sort, need order-independent transparency Assets from Valve Beyond Programmable Shading, SIGGRAPH 2012 12
    13. 13. Transparency • Order-dependent transparency has always been a big limitation for content creators & developers – Restrictive art pipeline: • No glass houses • Even windows on cars & buildings can be painful – Restrictive interaction between objects & effects • Meshes vs particles vs volumetrics • Lack of sorting prevents usage of other transparent techniques • Order-independent transparency is must going forward – With good performance & determinism Beyond Programmable Shading, SIGGRAPH 2012 13
    14. 14. Order-independent Transparency• Adaptive Transparency [Salvi11] is promising – Currently requires multi-pass & unbounded memory – Need render target read/modify/write to get single- pass & bounded memory – Composite as you go (forward lighting) Beyond Programmable Shading, SIGGRAPH 2012 14
    15. 15. Motion blur• Important for sense of speed & direction• Velocity vectors + post-process holds up quite well 15
    16. 16. Defocus blur• Key visual cue to perceive depth & focus – Guide & emotional storytelling tool• Sprite splatting [Igarashi08] is popular – Works great for out of focus background – Very sensitive to aliasing – Sharp edges on strong foreground blur Beyond Programmable Shading, SIGGRAPH 2012 16
    17. 17. Defocus blur – visibility issueIncorrect visibility = hard edge Correct visibility Beyond Programmable Shading, SIGGRAPH 2012 17
    18. 18. Defocus & motion blur – beyond post• Raytrace geometry• Stochastic rasterization – Lots of samples required for foreground, too little = noisy • Critical to have fast & temporally stable image reconstruction – Is defocus or motion blur the most important? • Leaning toward defocus due to the visibility issue • Though MB should not be applied after DOF, need a solution that handles both properly• Pre-filtered Sparse Voxel Octrees Beyond Programmable Shading, SIGGRAPH 2012 18
    19. 19. Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #2ILLUMINATION
    20. 20. Illumination challenges Dynamic Global Shadows Reflections Illumination Beyond Programmable Shading, SIGGRAPH 2012 20
    21. 21. Dynamic Global Illumination• Key visual component• Multiple dynamic alternatives now: – Light Propagation Volume – Voxel cone tracing – Reflective Shadow Maps + VPLs – Geometry pre-compute based: Enlighten• Major trade-offs depending on perf/memory/quality Beyond Programmable Shading, SIGGRAPH 2012 21
    22. 22. Dynamic GI wanted characteristics• Static & dynamic geometry• No pre-computation required – Major tradeoff with performance• Handle large scales: indoor to large outdoor – With minimal light leakage• Multiple indirect bounces – Single is not enough for in-door• Indirect specular reflections Beyond Programmable Shading, SIGGRAPH 2012 22
    23. 23. Shadows• General shadows continue to be a major challenge – “Efficient Real-time Shadows” course• As a game developer, I’m tired of shadows  – Not what the graphics pipeline is built for – Time consuming tweaking, optimizations, compromises – Current techniques don’t scale up – Are there Graphics/Compute extensions that could help? Beyond Programmable Shading, SIGGRAPH 2012 23
    24. 24. Shadows - wanted characteristicsRobust: General: – Stable under object, light & – Works with all light types camera motion – Supports dynamic geometry – No light leakage – Supports alpha-test – No flickering – Supports transparent receivers – No magic constants & casters – Scalable from small to large light sourcesHigh-quality: – Variable penumbra – No aliasing Fast! – Motion blurred – Sparse sampling – Good culling Beyond Programmable Shading, SIGGRAPH 2012 24
    25. 25. The Many Shadow problem• Want shadows on all lights – Easier to author • No light leaking through walls – Doesn’t limit content creators – Higher quality & more interactive• Current issues – Amount of geometry – Culling – Draw calls Super simple scene, 10 spotlights – Non-sparse rendering Beyond Programmable Shading, SIGGRAPH 2012 25
    26. 26. Many Shadow – potential solutions• Efficient rasterization – Smart logarithmic triangle culling with spatial data structure – Lazy on-demand scene culling, geometry culling & graphics dispatch • with CPU or GPU Compute• Raytrace geometry – Render gbuffer – For each pixel affected by a light, cast ray to light – Evaluate & composite directly in pixel shader, or output to light visibility masks• Cone trace into SVO – Once SVO data structure is built, easy to cone trace to query light visibility – Soft shadows = fast!  Beyond Programmable Shading, SIGGRAPH 2012 26
    27. 27. Reflections – categoriesGlossy reflections on arbitrary surfaces Perfect reflections on mostly-planar surfaces Beyond Programmable Shading, SIGGRAPH 2012 27
    28. 28. Reflections – use cases• Glossy reflections – Most surfaces, rough metal – Screen-space reflection • Fully dynamic • Simple & cheap • Visibility problems – Voxel Cone Tracing • Can be a good fit • Expensive to build SVO of scene Beyond Programmable Shading, SIGGRAPH 2012 28
    29. 29. Reflections – use cases• Perfect reflections – Mostly planar surfaces: windows, water – Render reflected view(s) • Prohibitive to render scenes upfront with multiple planes • Want more sparse rendering – Raytracing is elegant, but impractical • Performance • Have to switch entire game graphics pipeline – Voxel Cone Tracing • Requires massive resolution • Not practical until we can use it for primary visibility Beyond Programmable Shading, SIGGRAPH 2012 29
    30. 30. Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #3PROGRAMMABILITY
    31. 31. Programmability• Need major innovations to solve the other challenges Beyond Programmable Shading, SIGGRAPH 2012 31
    32. 32. Programmability - Pipelines• Graphics pipeline is fast but fixed – No conservative rasterization – No programmable blending – No flexible texture filtering (min/max/derivative)• GPU Compute can’t efficiently simulate a full graphics pipeline – Use the graphics pipeline when possible – Need to enable building your own efficient GPU Compute pipelines • When going beyond graphics pipeline capabilities • Esp. important with long HW & OS lead times Beyond Programmable Shading, SIGGRAPH 2012 32
    33. 33. Programmability - Areas• Need a virtual data-parallel ISA – Separate front-end from back-end • Can use same front-end (language) on all platforms (back-ends) • Run on all platforms and all (modern) architectures – Both CPU and GPU! • HSA with HSAIL is one promising solution – Most developers are multi-platform • Won’t write serious code in platform- or vendor-specific languages Beyond Programmable Shading, SIGGRAPH 2012 33
    34. 34. Programmability - Areas• Strip down the GPU SW stack & hardware abstractions – Only bindless access to resources • Need standard non-opaque texture layouts – Virtual memory – Both CPU or GPU Compute can generate GPU work – Layer existing fat APIs on top of it (DirectX & OpenGL) Beyond Programmable Shading, SIGGRAPH 2012 34
    35. 35. Programmability - Areas• GPU Compute spawning fine-grained tasks for itself – Build your own pipelines, independently of CPU – Kepler GK110 ‘Dynamic Parallelism’ on all chips, platforms and compute languages• Need mechanisms to build SIMD coherency – Not ideal to write out giant sample lists to memory and sort – Queues as a language & HW abstraction primitive Beyond Programmable Shading, SIGGRAPH 2012 35
    36. 36. Programmability - Areas• Low-latency CPU/GPU collaboration – Balance work, run where most efficient • Frostbite uses it on consoles [Coffin11] [Brisebois11] – GPU spawning work for CPU – CPU inserting more work for GPU within the frame – Simple use case for Sample Distribution Shadow Maps: 1. Render z+gbuffer 2. Analyze zbuffer to determine ideal shadowmap distribution 3. Kick of CPU to cull & create shadowmap graphics display list 4. GPU renders something else while waiting for CPU Beyond Programmable Shading, SIGGRAPH 2012 36
    37. 37. Programmability - Areas• Render target read/modify/write – A must have base operation – OIT & programmable blending Beyond Programmable Shading, SIGGRAPH 2012 37
    38. 38. Beyond Programmable Shading Course ACM SIGGRAPH 2012Challenge #4PRODUCTION COSTS Challenge #4 PRODUCTION COSTS
    39. 39. Production costs• Games are getting bigger & more complex – More content – More variation – Higher quality/detail – More complex content production process• What are the next big step forward for content production? – Quick iteration times = quality Beyond Programmable Shading, SIGGRAPH 2012 39
    40. 40. Production costs• If we had the ultimate real-time renderer that solves primary visibility and illumination, how much artist time would we save? – Probably not that much overall unfortunately (but increase quality) • Having shadows everywhere and dynamic GI saves some artist time • Will save engineering time & support – Content creation is the biggest time sink• What can save significant amount of time? – Scalable geometry representation – Procedural texturing – Procedural geometry – Content acquisition Beyond Programmable Shading, SIGGRAPH 2012 40
    41. 41. Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #5 SCALING
    42. 42. Scaling• Games & rendering use cases are needing more and more scaling. Both up and down! – Detail: mm to km – Resolution: 320x480 (IPhone3) to 5760x1200 (Eyefinity). 45x – Power: 1W to 300W. 300x• Requires significant scaling in performance• Which techniques, algorithms & pipelines are scalable? Beyond Programmable Shading, SIGGRAPH 2012 42
    43. 43. Scaling: Detail• How can we increase detail while building even larger interactive worlds? – Scalable geometry is difficult, discrete LODs suck • Want an inherently scalable & filterable primary data representation. Dynamic SVOs? – Can’t author everything • Use procedural detail up close Beyond Programmable Shading, SIGGRAPH 2012 43
    44. 44. Scaling: Resolution• Some of the lowest powered devices have the highest resolution screens – Consumers  – Developers • Graphics pipeline need a more flexible decoupling of shading rate vs visibility rate! – MSAA and fixed upsampling is not enough Beyond Programmable Shading, SIGGRAPH 2012 44
    45. 45. Scaling: Power• Marketplace is shifting from 100+ W to 1-45 W – Phones (1 W), Tablets (3 W), Ultrabooks (17 W), Laptops (45 W)• Developers typically don’t care about power usage – But hardware/device manufacturers and consumers do – With cloud rendering, power is a direct cost for developer• Need power efficient algorithms, techniques & pipelines – Grand challenge for the next 10 years: photo-realistic rendering at 1W Beyond Programmable Shading, SIGGRAPH 2012 45
    46. 46. Questions? email: johan.andersson@dice.se blog: http://repi.se twitter: @repi course page: http://bps12.idav.ucdavis.edu Beyond Programmable Shading, SIGGRAPH 2012 46
    47. 47. Thanks for the feedback!Christina Coffin Sebastien Lagarde Pål-Kristian EngstadCyril Crassin Sébastien Hillaire Jonathan Ragan-KellyAaron LefohnMarco Salvi Jim Vaughn Matthijs De SmedtAndrew Lauritzen Steven Tovey Doug BinksColin Barré-Brisebois Aras Pranckevičius Morgan McGuireIvan van Assen Julien Guertault Andrew RichardsMatt Collins Sam Martin Daniel WexlerCharles de RousiersStephen Hill Niklas Lundberg Eric SmolikowskiNathan Reed Michael Nicolella David MöllerstedtTimothy Lottes Aaron Miller ’ Beyond Programmable Shading, SIGGRAPH 2012 47
    48. 48. References• [Igarashi08]. Real-Time Depth-of-Field Rendering Using Point Splatting on Per-Pixel Layers. http://dxp.korea.ac.kr/homewiki/images/f/f5/PG-PointDof- submit.pdf• [Coffin11]. SPU Based Deferred Shading in Battlefield 3 for Playstation 3. Christina Coffin. http://publications.dice.se/attachments/Christina_Coffin_Programming_SPU _Based_Deferred.pptx• [Brisebois11] More Performance Five Rendering Ideas from Battlefield 3 and Need For Speed The Run. Barre-Brisebois et al. http://publications.dice.se/attachments/BF3_NFS_WhiteBarreBrisebois_Sig graph2011.pptx• [Salvi11] Adaptive Transparency. Salvi et al. http://software.intel.com/en- us/articles/adaptive-transparency-hpg-2011/ Beyond Programmable Shading, SIGGRAPH 2012 48

    ×