Your SlideShare is downloading. ×
0
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
5 Major Challenges in Interactive Rendering
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

5 Major Challenges in Interactive Rendering

56,704

Published on

Talk from SIGGRAPH 2010 and the Beyond Programmable Shading course. …

Talk from SIGGRAPH 2010 and the Beyond Programmable Shading course.

Also see http://publications.dice.se for more material and other DICE talks.

Published in: Entertainment & Humor
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
56,704
On Slideshare
0
From Embeds
0
Number of Embeds
58
Actions
Shares
0
Downloads
119
Comments
0
Likes
10
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Hi, I’m going to talk today a bit about what type of major challenges we have ahead for us in the next 5-10 years, what type of problems we want to solve and things to focus on. Sort of why we want to go Beyond Programable Shading? 
  • 5 orders of a magnitude difference
  • MLAA
  • MLAA
  • Particles would be the last to be included in an OIT solution
  • Show zbrush character Kayvon’s talk on Quad-fragment merging
  • Explicit braided parallelism GRAMPS-style queuing
  • Not the most sexy of topics, but the single most important one for game development Show undergrowth?
  • For us, that is a lot of time and a lot of money. But the single most important improvement in this new version of the engine. Editor framework, intutitive & fast workflows, features to simplify development
  • Examples: Automatic LOD solutions or directly rendering giant meshes without LODs Physically based rendering
  • Not the most sexy of topics, but the single most important one for game development Show undergrowth?
  • Memory usage is major cliff, render to virtual memory/cache/edram?
  • PantaRay - Fast Ray Traced Occlusion for Massive Scenes
  • Transcript

    • 1. 5 Major Challenges in Interactive Rendering Johan Andersson DICE
    • 2. Overview <ul><li>What are the major challenges for us in the next 5-10 years ? </li></ul><ul><ul><li>Interactive rendering for games as well as other interactive areas </li></ul></ul><ul><li>Which problems do we want to solve ? </li></ul><ul><li>What do we want to achieve &amp; focus on ? </li></ul><ul><li>Based on own thoughts &amp; feedback from people in the industry </li></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 3. THE 5 CHALLENGES <ul><li>(in no particular order) </li></ul>
    • 4. CINEMATIC IMAGE QUALITY <ul><li>Challenge #1 </li></ul>
    • 5. Challenge #1 - Cinematic image quality <ul><li>Want to get to that smooth visual feel CG movies have </li></ul><ul><ul><li>Consumers are viewing them on same device as they play games on </li></ul></ul><ul><li>Visual realism of real-time is still far from offline CG </li></ul><ul><ul><li>33 ms vs ~60 minutes per frame </li></ul></ul><ul><li>Areas with big quality difference : </li></ul><ul><ul><li>Aliasing </li></ul></ul><ul><ul><li>Motion blur &amp; depth of field </li></ul></ul><ul><ul><li>Transparency </li></ul></ul><ul><ul><li>Geometry </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 6. Aliasing – we have it! 08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 7. Aliasing <ul><li>Aliasing is one of the biggest visual artifacts </li></ul><ul><ul><li>Most games have lots of it  </li></ul></ul><ul><ul><li>Eyes distracted by flickering </li></ul></ul><ul><ul><li>Aliasing within a frame is typically very variable </li></ul></ul><ul><li>Multiple current antialiasing techniques, but no complete solution yet </li></ul><ul><ul><li>MSAA does not scale well (storage, bandwidth) </li></ul></ul><ul><ul><li>Post-effect based techniques only solve part of the aliasing problem </li></ul></ul><ul><li>How can we get to a pipeline that scales up to much higher quality AA ? </li></ul><ul><ul><li>Without breaking performance, memory storage or bandwidth </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 8. Everybody loves bokeh Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 9. Motion blur and Depth of Field <ul><li>Important visual cues to percieve depth, focus &amp; motion! </li></ul><ul><ul><li>The movie people know this </li></ul></ul><ul><li>Games only have post-process based versions </li></ul><ul><ul><li>Lots of artifacts </li></ul></ul><ul><ul><li>Not possible to implement complete effect as post-process </li></ul></ul><ul><li>Ideal to have a rendering pipeline that can naturally support motion blur and depth of field </li></ul><ul><ul><li>Stochastic rasterization? </li></ul></ul><ul><ul><li>Raytracing? </li></ul></ul><ul><ul><li>REYES? </li></ul></ul><ul><ul><li>Other? </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 10. Transparency <ul><li>Order- dependent transparency has always been a big limitation for content creators &amp; developers </li></ul><ul><ul><li>Restrictive art pipeline: no glass houses </li></ul></ul><ul><ul><li>Even windows on cars &amp; buildings can be painful </li></ul></ul><ul><ul><li>Restrictive interaction between objects &amp; effects </li></ul></ul><ul><ul><ul><li>Meshes vs particles vs volumetrics </li></ul></ul></ul><ul><li>Order- independent transparency is must going forward </li></ul><ul><ul><li>Big challenge! Gradual process </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 11. Micropolygon rendering <ul><li>Massively detailed geometry </li></ul><ul><ul><li>Render directly instead of simplify to normalmaps </li></ul></ul><ul><ul><li>True silhouettes &amp; no faceted edges </li></ul></ul><ul><li>DX11 tessellation is a good step forward </li></ul><ul><ul><li>But still quite complex full pipeline in practice </li></ul></ul><ul><ul><li>Shading quad efficiency issues with small triangles </li></ul></ul><ul><li>We’re getting closer! </li></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 12. ILLUMINATION <ul><li>Challenge #2 </li></ul>
    • 13. Challenge #2 - Illumination 08/01/10 Beyond Programmable Shading, SIGGRAPH 2010 Global Illumination Shadows Reflections
    • 14. Global Illumination 08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 15. Global Illumination <ul><li>Key visual component </li></ul><ul><ul><li>Build mood, ground environment </li></ul></ul><ul><ul><li>Current generation almost always static (or non-existing) </li></ul></ul><ul><li>Need dynamic GI solution(s) </li></ul><ul><ul><li>Dynamic environments &amp; for quick iteration times </li></ul></ul><ul><ul><li>Starting to see real-time dynamic solutions! </li></ul></ul><ul><ul><li>Multiple types of algorithms &amp; levels of pre-computation </li></ul></ul><ul><li>Interesting &amp; difficult example use cases: </li></ul><ul><ul><li>Large-scale destructible environment </li></ul></ul><ul><ul><li>Single frame instant muzzle flash </li></ul></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 16. Beyond Programmable Shading, SIGGRAPH 2010 08/01/10 Shadows - ouch
    • 17. Shadows <ul><li>Shadowmaps are still not a completely solved problem </li></ul><ul><ul><li>Non-trivial implementations: </li></ul></ul><ul><ul><li>Aliasing, resolution-matching, filtering, management, culling </li></ul></ul><ul><ul><li>Translucent shadows </li></ul></ul><ul><li>Oh and where are the penumbras ? </li></ul><ul><ul><li>Area light source shadows are more pleasing </li></ul></ul><ul><ul><li>Variable penumbra &amp; overlapping casters </li></ul></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 18. Massive local shadowing <ul><li>We can light with 1000 light sources </li></ul><ul><ul><li>Without shadows!  </li></ul></ul><ul><ul><li>Or only a few with shadows </li></ul></ul><ul><ul><li>Handled separately </li></ul></ul><ul><li>Next step: have 100s of lights with shadows </li></ul><ul><ul><li>Requires rethinking and much improved culling &amp; dispatch efficiency </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 19. Reflections Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 20. Reflections <ul><li>Currently only have 2 methods: </li></ul><ul><ul><li>Envmaps </li></ul></ul><ul><ul><ul><li>Scale up to 100s of dynamic envmaps? Similar to point light shadow problem </li></ul></ul></ul><ul><ul><ul><li>Doesn’t solve concave or large flat surfaces </li></ul></ul></ul><ul><ul><li>Planar reflections </li></ul></ul><ul><ul><ul><li>Good for big flat single/few surfaces </li></ul></ul></ul><ul><ul><ul><li>But restricted to that as well, how to handle slopes &amp; multiple walls/windows? </li></ul></ul></ul><ul><li>Need solutions for local reflections on arbitrary surfaces </li></ul><ul><ul><li>Both glossy &amp; perfect reflections </li></ul></ul><ul><ul><li>Not that much research in this area except with raytracing? </li></ul></ul><ul><li>Hybrid rasterization/raytracing pipeline? </li></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 21. PROGRAMMABILITY <ul><li>Challenge #3 </li></ul>
    • 22. Programmability <ul><li>Graphics pipeline is fast but fixed </li></ul><ul><ul><li>No conservative rasterization </li></ul></ul><ul><ul><li>No programmable blending </li></ul></ul><ul><ul><li>No flexible texture filtering (min/max/derivative) </li></ul></ul><ul><li>Compute pipeline can’t simulate full graphics pipeline efficiently today </li></ul><ul><li>How do we get to a hybrid/reconfigurable pipeline? </li></ul><ul><ul><li>What are the actual use cases &amp; requirements ? </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 23. Examples of what we like to solve <ul><li>Irregular workloads / user-mode scheduling </li></ul><ul><ul><li>Key building block for many advanced techniques </li></ul></ul><ul><li>GPU-based scene culling &amp; rendering </li></ul><ul><ul><li>GPU feeding itself (on a high-level) </li></ul></ul><ul><ul><li>For performance and flexibility. </li></ul></ul><ul><li>Half-res rendering without depth artifacts </li></ul><ul><ul><li>Depth test per sample, shade per quad, upsample to pixel </li></ul></ul><ul><li>For more use cases: </li></ul><ul><ul><li>Bending the Graphics Pipeline at 11:45 am </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 24. PRODUCTION COSTS <ul><li>Challenge #4 </li></ul>
    • 25. Production costs <ul><li>We are increasing quality &amp; richness in all areas, but can’t continue to increase costs at same rate </li></ul><ul><ul><li>Turn the trend of more &amp; more expensive content creation ! </li></ul></ul><ul><ul><li>Not as sexy, but single most important challenge in practice for many (game) developers </li></ul></ul><ul><li>Linked with iteration times which is critical for both quality, quantity &amp; low costs </li></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 26. Importance of this challenge <ul><li>An example: </li></ul><ul><ul><li>We’ve spent 20 man years on improving workflows, iteration times &amp; reducing production costs for our next game engine - Frostbite 2 </li></ul></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 27. Production cost reductions <ul><li>Improvements: </li></ul><ul><ul><li>Faster workflows &amp; tools </li></ul></ul><ul><ul><li>Procedural content amplification / generation </li></ul></ul><ul><ul><li>Fewer custom pipelines, techniques &amp; solutions </li></ul></ul><ul><ul><li>More sharing &amp; reuse of content </li></ul></ul><ul><li>Giant important topic but only something we cover indirectly in the course </li></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 28. Procedural foliage distribution 08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 29. SCALING UP <ul><li>Challenge #5 </li></ul>
    • 30. Scaling up <ul><li>GPU model has become quite flexible </li></ul><ul><ul><li>Main problem is often not capabilities but performance/bw </li></ul></ul><ul><li>Want more of everything: </li></ul><ul><ul><li>Performance, bandwidth &amp; memory </li></ul></ul><ul><ul><li>Content, detail &amp; quality </li></ul></ul><ul><li>Scalability without performance cliffs </li></ul><ul><ul><li>Assumed reasonableness in fixed pipelines </li></ul></ul><ul><ul><li>Graceful performance degradation strongly preferred </li></ul></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 31. Scaling up - Techniques <ul><li>Some techniques break down when scaling up, for example: </li></ul><ul><ul><li>Deferred shading with +4x MSAA </li></ul></ul><ul><ul><li>Quad-based forward shading with ~1 pixel triangles </li></ul></ul><ul><ul><li>GPU dispatch APIs when doing 100s of small scene renders for shadows &amp; reflections (batch counts) </li></ul></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 32. Scaling up - Worlds <ul><li>Move from static worlds to immersive interactive worlds </li></ul><ul><ul><li>Unique </li></ul></ul><ul><ul><li>Detailed </li></ul></ul><ul><ul><li>Changeable / Destructible </li></ul></ul><ul><ul><li>(Procedural) </li></ul></ul><ul><li>Massive scenes are a challenge for many industries </li></ul><ul><ul><li>Movies, Scientific, Games </li></ul></ul><ul><ul><li>Esp. interaction with memory &amp; performance cliffs </li></ul></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 33. The Challenges <ul><li>Illumination </li></ul><ul><li>Cinematic Image Quality </li></ul><ul><li>Programmability </li></ul><ul><li>Production costs </li></ul><ul><li>Scaling up </li></ul>08/01/10 Beyond Programmable Shading Course, ACM SIGGRAPH 2010
    • 34. Conclusions <ul><li>Real-time rendering is far from a solved problem </li></ul><ul><li>We need major improvements to the real-time rendering pipeline(s) and programming model </li></ul>08/01/10 Beyond Programmable Shading, SIGGRAPH 2010
    • 35. Thanks for all the feedback! <ul><li>Christina Coffin (@christinacoffin) </li></ul><ul><li>Colin Barré-Brisebois (@ZigguratVertigo) </li></ul><ul><li>Daniel Collin (@daniel_collin) </li></ul><ul><li>Flavius Alecu (@flawe) </li></ul><ul><li>Sander van Rossen ( @logicalerror) </li></ul><ul><li>Rob Jones (@bobvodka) </li></ul><ul><li>Stephen Tovey (@nonchaotic) </li></ul><ul><li>Colin Riley (@domipheus) </li></ul><ul><li>Joe Tidmarsh (@mrjovis) </li></ul><ul><li>Stephen Hill (@self_shadow) </li></ul><ul><li>Federico B P ( @nocturndragon) </li></ul><ul><li>Stefan Boberg (@bionicbeagle) </li></ul><ul><li>Noel Llopis (@snappytouch) </li></ul><ul><li>Björn Knafla (@bjoernknafla) </li></ul><ul><li>Andrew Richards (@codeandrew) </li></ul><ul><li>Juan Manuel Alvarez (@the_naicigam) </li></ul><ul><li>David Luebke (@davedotluebke) </li></ul><ul><li>Jonathan Ragan-Kelley (@jrk) </li></ul><ul><li>Pat Wilson (@pat_wilson) </li></ul><ul><li>Rachel Blum (@groby) </li></ul><ul><li>Brian Karis (@briankaris) </li></ul><ul><li>Matt Collins (@matt_c_) </li></ul><ul><li>Sam Martin (@palgorithm) </li></ul><ul><li>Aaron Lefohn </li></ul><ul><li>Andrew Lauritzen </li></ul><ul><li>Luca Fascione </li></ul><ul><li>Steve Anichini </li></ul><ul><li>Simon Taylor </li></ul><ul><li>Matt Swoboda </li></ul><ul><li>Cody Ritchie </li></ul><ul><li>Mattias Kylen </li></ul><ul><li>Oscar Carlén </li></ul>Beyond Programmable Shading Course, ACM SIGGRAPH 2010 08/01/10
    • 36. Contact details <ul><li>email: [email_address] </li></ul><ul><li>blog: http://repi.se </li></ul><ul><li>twitter: @repi </li></ul><ul><li>For more DICE talks: </li></ul><ul><li>http://publications.dice.se </li></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10
    • 37. BONUS
    • 38. Surface lighting <ul><li>More advanced surface shading &amp; lighting </li></ul><ul><ul><li>Sub-surface scattering </li></ul></ul><ul><ul><li>Hair </li></ul></ul><ul><ul><li>Foliage </li></ul></ul><ul><ul><li>Arbitrary / generalized BRDFs? </li></ul></ul><ul><li>How can they work with deferred rendering ? </li></ul>Beyond Programmable Shading, SIGGRAPH 2010 08/01/10

    ×