• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
5 Major Challenges in Real-time Rendering (2012)
 

5 Major Challenges in Real-time Rendering (2012)

on

  • 269,709 views

Talk by Johan Andersson (DICE/EA) in the Beyond Programmable Shading Course at SIGGRAPH 2012.

Talk by Johan Andersson (DICE/EA) in the Beyond Programmable Shading Course at SIGGRAPH 2012.

The other talks in the course can be found here: http://bps12.idav.ucdavis.edu/

Statistics

Views

Total Views
269,709
Views on SlideShare
13,554
Embed Views
256,155

Actions

Likes
23
Downloads
268
Comments
2

75 Embeds 256,155

http://dice.se 118566
http://www.frostbite.com 115370
http://publications.dice.se 11766
http://3dnchu.jimdo.com 5322
http://masafumi.cocolog-nifty.com 1399
http://www.frostbite.com.cluster.triggerfish.se 1061
http://www.shader.jp 703
http://translate.googleusercontent.com 442
http://leegoonz.blogspot.kr 328
http://new-devweb.eng.capcom.co.jp 323
http://kindtis.tistory.com 246
http://192.168.25.100 101
https://twitter.com 62
http://sin-logs.blogspot.jp 54
https://si0.twimg.com 34
http://leegoonz.blogspot.com 33
http://www.hanrss.com 33
http://192.168.25.101 29
http://www.dice.se 27
http://webcache.googleusercontent.com 27
https://twimg0-a.akamaihd.net 23
http://sin-logs.blogspot.com 20
http://www.twylah.com 20
http://131.253.14.98 18
http://157.109.200.99 13
http://app.m-cocolog.jp 12
http://www.dice.se.cluster.triggerfish.se 12
http://www19.jimdo.com 10
http://leegoonz.blogspot.co.uk 9
http://131.253.14.66 7
http://t.co 5
http://us-w1.rockmelt.com 5
http://www.lab.nbgi.jp 4
https://www.google.de 4
http://leegoonz.blogspot.ru 4
http://intra.te.capcom.co.jp 3
http://leegoonz.blogspot.jp 3
https://www.google.com 3
http://www.newsblur.com 3
http://cloud.feedly.com 3
http://newsblur.com 3
http://leegoonz.blogspot.co.at 3
https://translate.googleusercontent.com 2
https://www.google.co.uk 2
http://battlelog.battlefield.com 2
http://sin-logs.blogspot.co.uk 2
http://frostbiteengine.com 2
http://www.google.se 2
http://127.0.0.1 2
http://leegoonz.blogspot.com.br 2
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • ThX
    Are you sure you want to
    Your message goes here
    Processing…
  • my roomate's aunt makes $83/hr on the laptop. She has been without work for 8 months but last month her pay was $8682 just working on the laptop for a few hours. Read more on this site...NuttÿRîçhDÖtc om
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Shader should be able to decide what to output per sample, while executing per pixelCan we keep deferred MSAA in large local store tiles?
  • Texture space shading?
  • Particles would be the last to be included in an OIT solution
  • Prohibitive to include all transparent surfaces50x+ particle overdraw commonCombine with half-resHybrid OIT: pre-sorted particles blend on closest OIT sample?
  • For performance, would use pragmatic max clamps
  • GPU Compute performance portability Challenge between to run same code efficiently on different architectures and platformsMajor changes can sometimes be requiredNot unexpected, but can be a challenge

5 Major Challenges in Real-time Rendering (2012) 5 Major Challenges in Real-time Rendering (2012) Presentation Transcript

  • Beyond Programmable Shading Course ACM SIGGRAPH 2012 5 MAJOR CHALLENGES IN REAL-TIME RENDERING Johan Andersson, DICEBeyond Programmable Shading, SIGGRAPH 2012 1
  • Overview• What are the major challenges for us in the next 5-10 years? – Real-time rendering for games as well as other areas• Which problems do we want to solve?• What do we want to achieve & focus on?• Based on own thoughts & feedback from people in the industry Beyond Programmable Shading, SIGGRAPH 2012 2
  • Beyond Programmable Shading Course ACM SIGGRAPH 201225 5 MAJOR CHALLENGES IN REAL-TIME RENDERING Johan Andersson, DICE Beyond Programmable Shading, SIGGRAPH 2012 3
  • Challenges 2012Same as 2010!1. Cinematic Image Quality2. Illumination3. Programmability4. Production costs5. Scaling Beyond Programmable Shading, SIGGRAPH 2012 4
  • Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #1CINEMATIC IMAGE QUALITY
  • Cinematic Image Quality• Goal is to achieve Cinematic Image Quality – Same smooth & rich pictures that CG movies have• Need significant improvements to GPU primary visibility – Antialiasing – Transparency – Defocus blur – Motion blur• A future solution needs to include all together Beyond Programmable Shading, SIGGRAPH 2012 6
  • Antialiasing• Single most visible issue to improve on – Aliasing breaks the illusion – Less aliasing = more pleasing & easier to see visuals• Sources of aliasing: – Geometric aliasing – Proxy geometry aliasing (alpha test) – Shader aliasing – Mixed resolution rendering Beyond Programmable Shading, SIGGRAPH 2012 7
  • Post-process antialiasing• Lots of developments in post-AA techniques – MLAA, FXAA, SMAA and more – Good quality / performance ratio – See SIGGRAPH’11 course: “Filtering Approaches for Real-Time Anti-Aliasing” http://iryoku.com/aacourse/• But need solutions for the full problem Beyond Programmable Shading, SIGGRAPH 2012 8
  • Geometric aliasing• Current solutions: MSAA, SSAA, temporal – Fixed quality techniques, not adaptive – Problematic to scale up to very high quality• 16x MSAA is really good quality but expensive – Need high rate if using coverage masks• MSAA + deferred – Massive memory usage & bandwidth – Want access to MSAA compression surface to avoid computing ourselves – See Andrew’s talk in the course: “Intersecting Lights with Pixels” Beyond Programmable Shading, SIGGRAPH 2012 9
  • Geometric aliasing• Other alternatives? – Analytical antialiasing • Would be interesting to see more research – Pre-filtered Sparse Voxel Octrees • Requires very high resolution / large storage • See Cyril’s talk in the course: “Dynamic Sparse Voxel Octrees for Next-Gen Real-time Rendering” Beyond Programmable Shading, SIGGRAPH 2012 10
  • Shader aliasing• Shader aliasing becoming more of a problem – High-frequency specular highlights – High-frequency shadows – Amplified by HDR Bloom & Bokeh• What is needed to make sure shaders do not output aliased values? – Careful handling of derivatives when texture mapping – LEAN mapping, EVSM shadows – Wednesday: “Rock-Solid Shading: Image Stability without Sacrificing Detail” Beyond Programmable Shading, SIGGRAPH 2012 11
  • Proxy geometry aliasing• Alpha-testing for proxy geometry results in major aliasing – MSAA per-sample evaluation is costly & requires many samples• Real transparency can directly solve the aliasing – But we need to sort, need order-independent transparency Assets from Valve Beyond Programmable Shading, SIGGRAPH 2012 12
  • Transparency • Order-dependent transparency has always been a big limitation for content creators & developers – Restrictive art pipeline: • No glass houses • Even windows on cars & buildings can be painful – Restrictive interaction between objects & effects • Meshes vs particles vs volumetrics • Lack of sorting prevents usage of other transparent techniques • Order-independent transparency is must going forward – With good performance & determinism Beyond Programmable Shading, SIGGRAPH 2012 13
  • Order-independent Transparency• Adaptive Transparency [Salvi11] is promising – Currently requires multi-pass & unbounded memory – Need render target read/modify/write to get single- pass & bounded memory – Composite as you go (forward lighting) Beyond Programmable Shading, SIGGRAPH 2012 14
  • Motion blur• Important for sense of speed & direction• Velocity vectors + post-process holds up quite well 15
  • Defocus blur• Key visual cue to perceive depth & focus – Guide & emotional storytelling tool• Sprite splatting [Igarashi08] is popular – Works great for out of focus background – Very sensitive to aliasing – Sharp edges on strong foreground blur Beyond Programmable Shading, SIGGRAPH 2012 16
  • Defocus blur – visibility issueIncorrect visibility = hard edge Correct visibility Beyond Programmable Shading, SIGGRAPH 2012 17
  • Defocus & motion blur – beyond post• Raytrace geometry• Stochastic rasterization – Lots of samples required for foreground, too little = noisy • Critical to have fast & temporally stable image reconstruction – Is defocus or motion blur the most important? • Leaning toward defocus due to the visibility issue • Though MB should not be applied after DOF, need a solution that handles both properly• Pre-filtered Sparse Voxel Octrees Beyond Programmable Shading, SIGGRAPH 2012 18
  • Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #2ILLUMINATION
  • Illumination challenges Dynamic Global Shadows Reflections Illumination Beyond Programmable Shading, SIGGRAPH 2012 20
  • Dynamic Global Illumination• Key visual component• Multiple dynamic alternatives now: – Light Propagation Volume – Voxel cone tracing – Reflective Shadow Maps + VPLs – Geometry pre-compute based: Enlighten• Major trade-offs depending on perf/memory/quality Beyond Programmable Shading, SIGGRAPH 2012 21
  • Dynamic GI wanted characteristics• Static & dynamic geometry• No pre-computation required – Major tradeoff with performance• Handle large scales: indoor to large outdoor – With minimal light leakage• Multiple indirect bounces – Single is not enough for in-door• Indirect specular reflections Beyond Programmable Shading, SIGGRAPH 2012 22
  • Shadows• General shadows continue to be a major challenge – “Efficient Real-time Shadows” course• As a game developer, I’m tired of shadows  – Not what the graphics pipeline is built for – Time consuming tweaking, optimizations, compromises – Current techniques don’t scale up – Are there Graphics/Compute extensions that could help? Beyond Programmable Shading, SIGGRAPH 2012 23
  • Shadows - wanted characteristicsRobust: General: – Stable under object, light & – Works with all light types camera motion – Supports dynamic geometry – No light leakage – Supports alpha-test – No flickering – Supports transparent receivers – No magic constants & casters – Scalable from small to large light sourcesHigh-quality: – Variable penumbra – No aliasing Fast! – Motion blurred – Sparse sampling – Good culling Beyond Programmable Shading, SIGGRAPH 2012 24
  • The Many Shadow problem• Want shadows on all lights – Easier to author • No light leaking through walls – Doesn’t limit content creators – Higher quality & more interactive• Current issues – Amount of geometry – Culling – Draw calls Super simple scene, 10 spotlights – Non-sparse rendering Beyond Programmable Shading, SIGGRAPH 2012 25
  • Many Shadow – potential solutions• Efficient rasterization – Smart logarithmic triangle culling with spatial data structure – Lazy on-demand scene culling, geometry culling & graphics dispatch • with CPU or GPU Compute• Raytrace geometry – Render gbuffer – For each pixel affected by a light, cast ray to light – Evaluate & composite directly in pixel shader, or output to light visibility masks• Cone trace into SVO – Once SVO data structure is built, easy to cone trace to query light visibility – Soft shadows = fast!  Beyond Programmable Shading, SIGGRAPH 2012 26
  • Reflections – categoriesGlossy reflections on arbitrary surfaces Perfect reflections on mostly-planar surfaces Beyond Programmable Shading, SIGGRAPH 2012 27
  • Reflections – use cases• Glossy reflections – Most surfaces, rough metal – Screen-space reflection • Fully dynamic • Simple & cheap • Visibility problems – Voxel Cone Tracing • Can be a good fit • Expensive to build SVO of scene Beyond Programmable Shading, SIGGRAPH 2012 28
  • Reflections – use cases• Perfect reflections – Mostly planar surfaces: windows, water – Render reflected view(s) • Prohibitive to render scenes upfront with multiple planes • Want more sparse rendering – Raytracing is elegant, but impractical • Performance • Have to switch entire game graphics pipeline – Voxel Cone Tracing • Requires massive resolution • Not practical until we can use it for primary visibility Beyond Programmable Shading, SIGGRAPH 2012 29
  • Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #3PROGRAMMABILITY
  • Programmability• Need major innovations to solve the other challenges Beyond Programmable Shading, SIGGRAPH 2012 31
  • Programmability - Pipelines• Graphics pipeline is fast but fixed – No conservative rasterization – No programmable blending – No flexible texture filtering (min/max/derivative)• GPU Compute can’t efficiently simulate a full graphics pipeline – Use the graphics pipeline when possible – Need to enable building your own efficient GPU Compute pipelines • When going beyond graphics pipeline capabilities • Esp. important with long HW & OS lead times Beyond Programmable Shading, SIGGRAPH 2012 32
  • Programmability - Areas• Need a virtual data-parallel ISA – Separate front-end from back-end • Can use same front-end (language) on all platforms (back-ends) • Run on all platforms and all (modern) architectures – Both CPU and GPU! • HSA with HSAIL is one promising solution – Most developers are multi-platform • Won’t write serious code in platform- or vendor-specific languages Beyond Programmable Shading, SIGGRAPH 2012 33
  • Programmability - Areas• Strip down the GPU SW stack & hardware abstractions – Only bindless access to resources • Need standard non-opaque texture layouts – Virtual memory – Both CPU or GPU Compute can generate GPU work – Layer existing fat APIs on top of it (DirectX & OpenGL) Beyond Programmable Shading, SIGGRAPH 2012 34
  • Programmability - Areas• GPU Compute spawning fine-grained tasks for itself – Build your own pipelines, independently of CPU – Kepler GK110 ‘Dynamic Parallelism’ on all chips, platforms and compute languages• Need mechanisms to build SIMD coherency – Not ideal to write out giant sample lists to memory and sort – Queues as a language & HW abstraction primitive Beyond Programmable Shading, SIGGRAPH 2012 35
  • Programmability - Areas• Low-latency CPU/GPU collaboration – Balance work, run where most efficient • Frostbite uses it on consoles [Coffin11] [Brisebois11] – GPU spawning work for CPU – CPU inserting more work for GPU within the frame – Simple use case for Sample Distribution Shadow Maps: 1. Render z+gbuffer 2. Analyze zbuffer to determine ideal shadowmap distribution 3. Kick of CPU to cull & create shadowmap graphics display list 4. GPU renders something else while waiting for CPU Beyond Programmable Shading, SIGGRAPH 2012 36
  • Programmability - Areas• Render target read/modify/write – A must have base operation – OIT & programmable blending Beyond Programmable Shading, SIGGRAPH 2012 37
  • Beyond Programmable Shading Course ACM SIGGRAPH 2012Challenge #4PRODUCTION COSTS Challenge #4 PRODUCTION COSTS
  • Production costs• Games are getting bigger & more complex – More content – More variation – Higher quality/detail – More complex content production process• What are the next big step forward for content production? – Quick iteration times = quality Beyond Programmable Shading, SIGGRAPH 2012 39
  • Production costs• If we had the ultimate real-time renderer that solves primary visibility and illumination, how much artist time would we save? – Probably not that much overall unfortunately (but increase quality) • Having shadows everywhere and dynamic GI saves some artist time • Will save engineering time & support – Content creation is the biggest time sink• What can save significant amount of time? – Scalable geometry representation – Procedural texturing – Procedural geometry – Content acquisition Beyond Programmable Shading, SIGGRAPH 2012 40
  • Beyond Programmable Shading Course ACM SIGGRAPH 2012 Challenge #5 SCALING
  • Scaling• Games & rendering use cases are needing more and more scaling. Both up and down! – Detail: mm to km – Resolution: 320x480 (IPhone3) to 5760x1200 (Eyefinity). 45x – Power: 1W to 300W. 300x• Requires significant scaling in performance• Which techniques, algorithms & pipelines are scalable? Beyond Programmable Shading, SIGGRAPH 2012 42
  • Scaling: Detail• How can we increase detail while building even larger interactive worlds? – Scalable geometry is difficult, discrete LODs suck • Want an inherently scalable & filterable primary data representation. Dynamic SVOs? – Can’t author everything • Use procedural detail up close Beyond Programmable Shading, SIGGRAPH 2012 43
  • Scaling: Resolution• Some of the lowest powered devices have the highest resolution screens – Consumers  – Developers • Graphics pipeline need a more flexible decoupling of shading rate vs visibility rate! – MSAA and fixed upsampling is not enough Beyond Programmable Shading, SIGGRAPH 2012 44
  • Scaling: Power• Marketplace is shifting from 100+ W to 1-45 W – Phones (1 W), Tablets (3 W), Ultrabooks (17 W), Laptops (45 W)• Developers typically don’t care about power usage – But hardware/device manufacturers and consumers do – With cloud rendering, power is a direct cost for developer• Need power efficient algorithms, techniques & pipelines – Grand challenge for the next 10 years: photo-realistic rendering at 1W Beyond Programmable Shading, SIGGRAPH 2012 45
  • Questions? email: johan.andersson@dice.se blog: http://repi.se twitter: @repi course page: http://bps12.idav.ucdavis.edu Beyond Programmable Shading, SIGGRAPH 2012 46
  • Thanks for the feedback!Christina Coffin Sebastien Lagarde Pål-Kristian EngstadCyril Crassin Sébastien Hillaire Jonathan Ragan-KellyAaron LefohnMarco Salvi Jim Vaughn Matthijs De SmedtAndrew Lauritzen Steven Tovey Doug BinksColin Barré-Brisebois Aras Pranckevičius Morgan McGuireIvan van Assen Julien Guertault Andrew RichardsMatt Collins Sam Martin Daniel WexlerCharles de RousiersStephen Hill Niklas Lundberg Eric SmolikowskiNathan Reed Michael Nicolella David MöllerstedtTimothy Lottes Aaron Miller ’ Beyond Programmable Shading, SIGGRAPH 2012 47
  • References• [Igarashi08]. Real-Time Depth-of-Field Rendering Using Point Splatting on Per-Pixel Layers. http://dxp.korea.ac.kr/homewiki/images/f/f5/PG-PointDof- submit.pdf• [Coffin11]. SPU Based Deferred Shading in Battlefield 3 for Playstation 3. Christina Coffin. http://publications.dice.se/attachments/Christina_Coffin_Programming_SPU _Based_Deferred.pptx• [Brisebois11] More Performance Five Rendering Ideas from Battlefield 3 and Need For Speed The Run. Barre-Brisebois et al. http://publications.dice.se/attachments/BF3_NFS_WhiteBarreBrisebois_Sig graph2011.pptx• [Salvi11] Adaptive Transparency. Salvi et al. http://software.intel.com/en- us/articles/adaptive-transparency-hpg-2011/ Beyond Programmable Shading, SIGGRAPH 2012 48