The document discusses various anti-aliasing techniques for real-time graphics, focusing on approaches suitable for CryENGINE 3. It describes the requirements like being orthogonal, having low memory overhead, and working with deferred rendering. Temporal anti-aliasing and distributed supersampling are covered, including techniques like reprojection and velocity weighting to minimize artifacts at low frame rates. Challenges with multi-sampling anti-aliasing on consoles are also discussed. Future work could combine supersampling with post-process anti-aliasing to handle disoccluded regions.
Graphics Gems from CryENGINE 3 (Siggraph 2013)Tiago Sousa
This lecture covers rendering topics related to Crytek’s latest engine iteration, the technology which powers titles such as Ryse, Warface, and Crysis 3. Among covered topics, Sousa presented SMAA 1TX: an update featuring a robust and simple temporal antialising component; performant and physically-plausible camera related post-processing techniques such as motion blur and depth of field were also covered.
Secrets of CryENGINE 3 Graphics TechnologyTiago Sousa
In this talk, the authors will describe an overview of a different method for deferred lighting approach used in CryENGINE 3, along with an in-depth description of the many techniques used. Original file and videos at http://crytek.com/cryengine/presentations
Game engines have long been in the forefront of taking advantage of the ever increasing parallel compute power of both CPUs and GPUs. This talk is about how the parallel compute is utilized in practice on multiple platforms today in the Frostbite game engine and how we think the parallel programming models, hardware and software in the industry should look like in the next 5 years to help us make the best games possible
Past, Present and Future Challenges of Global Illumination in GamesColin Barré-Brisebois
Global illumination (GI) has been an ongoing quest in games. The perpetual tug-of-war between visual quality and performance often forces developers to take the latest and greatest from academia and tailor it to push the boundaries of what has been realized in a game product. Many elements need to align for success, including image quality, performance, scalability, interactivity, ease of use, as well as game-specific and production challenges.
First we will paint a picture of the current state of global illumination in games, addressing how the state of the union compares to the latest and greatest research. We will then explore various GI challenges that game teams face from the art, engineering, pipelines and production perspective. The games industry lacks an ideal solution, so the goal here is to raise awareness by being transparent about the real problems in the field. Finally, we will talk about the future. This will be a call to arms, with the objective of uniting game developers and researchers on the same quest to evolve global illumination in games from being mostly static, or sometimes perceptually real-time, to fully real-time.
Graphics Gems from CryENGINE 3 (Siggraph 2013)Tiago Sousa
This lecture covers rendering topics related to Crytek’s latest engine iteration, the technology which powers titles such as Ryse, Warface, and Crysis 3. Among covered topics, Sousa presented SMAA 1TX: an update featuring a robust and simple temporal antialising component; performant and physically-plausible camera related post-processing techniques such as motion blur and depth of field were also covered.
Secrets of CryENGINE 3 Graphics TechnologyTiago Sousa
In this talk, the authors will describe an overview of a different method for deferred lighting approach used in CryENGINE 3, along with an in-depth description of the many techniques used. Original file and videos at http://crytek.com/cryengine/presentations
Game engines have long been in the forefront of taking advantage of the ever increasing parallel compute power of both CPUs and GPUs. This talk is about how the parallel compute is utilized in practice on multiple platforms today in the Frostbite game engine and how we think the parallel programming models, hardware and software in the industry should look like in the next 5 years to help us make the best games possible
Past, Present and Future Challenges of Global Illumination in GamesColin Barré-Brisebois
Global illumination (GI) has been an ongoing quest in games. The perpetual tug-of-war between visual quality and performance often forces developers to take the latest and greatest from academia and tailor it to push the boundaries of what has been realized in a game product. Many elements need to align for success, including image quality, performance, scalability, interactivity, ease of use, as well as game-specific and production challenges.
First we will paint a picture of the current state of global illumination in games, addressing how the state of the union compares to the latest and greatest research. We will then explore various GI challenges that game teams face from the art, engineering, pipelines and production perspective. The games industry lacks an ideal solution, so the goal here is to raise awareness by being transparent about the real problems in the field. Finally, we will talk about the future. This will be a call to arms, with the objective of uniting game developers and researchers on the same quest to evolve global illumination in games from being mostly static, or sometimes perceptually real-time, to fully real-time.
Talk by Graham Wihlidal (Frostbite Labs) at GDC 2017.
Checkerboard rendering is a relatively new technique, popularized recently by the introduction of the PlayStation 4 Pro. Many modern game engines are adding support for it right now, and in this talk, Graham will present an in-depth look at the new implementation in Frostbite, which is used in shipping titles like 'Battlefield 1' and 'Mass Effect Andromeda'. Despite being conceptually simple, checkerboard rendering requires a deep integration into the post-processing chain, in particular temporal anti-aliasing, dynamic resolution scaling, and poses various challenges to existing effects. This presentation will cover the basics of checkerboard rendering, explain the impact on a game engine that powers a wide range of titles, and provide a detailed look at how the current implementation in Frostbite works, including topics like object id, alpha unrolling, gradient adjust, and a highly efficient depth resolve.
Course presentation at SIGGRAPH 2014 by Charles de Rousiers and Sébastian Lagarde at Electronic Arts about transitioning the Frostbite game engine to physically-based rendering.
Make sure to check out the 118 page course notes on: http://www.frostbite.com/2014/11/moving-frostbite-to-pbr/
During the last few months, we have revisited the concept of image quality in Frostbite. The core of our approach was to be as close as possible to a cinematic look. We used the concept of reference to evaluate the accuracy of produced images. Physically based rendering (PBR) was the natural way to achieve this. This talk covers all the different steps needed to switch a production engine to PBR, including the small details often bypass in the literature.
The state of the art of real-time PBR techniques allowed us to achieve good overall results but not without production issues. We present some techniques for improving convolution time for image based reflection, proper ambient occlusion handling, and coherent lighting units which are mandatory for level editing.
Moreover, we have managed to reduce the quality gap, highlighted by our systematic reference comparison, in particular related to rough material handling, glossy screen space reflection, and area lighting.
The technical part of PBR is crucial for achieving good results, but represents only the top of the iceberg. Frostbite has become the de facto high-end game engine within Electronic Arts and is now used by a large amount of game teams. Moving all these game teams from “old fashion” lighting to PBR has required a lot of education, which have been done in parallel of the technical development. We have provided editing and validation tools to help the transition of art production. In addition, we have built a flexible material parametrisation framework to adapt to the various authoring tools and game teams’ requirements.
A technical deep dive into the DX11 rendering in Battlefield 3, the first title to use the new Frostbite 2 Engine. Topics covered include DX11 optimization techniques, efficient deferred shading, high-quality rendering and resource streaming for creating large and highly-detailed dynamic environments on modern PCs.
With the highest-quality video options, Battlefield 3 renders its Screen-Space Ambient Occlusion (SSAO) using the Horizon-Based Ambient Occlusion (HBAO) algorithm. For performance reasons, the HBAO is rendered in half resolution using half-resolution input depths. The HBAO is then blurred in full resolution using a depth-aware blur. The main issue with such low-resolution SSAO rendering is that it produces objectionable flickering for thin objects (such as alpha-tested foliage) when the camera and/or the geometry are moving. After a brief recap of the original HBAO pipeline, this talk describes a novel temporal filtering algorithm that fixed the HBAO flickering problem in Battlefield 3 with a 1-2% performance hit in 1920x1200 on PC (DX10 or DX11). The talk includes algorithm and implementation details on the temporal filtering part, as well as generic optimizations for SSAO blur pixel shaders. This is a joint work between Louis Bavoil (NVIDIA) and Johan Andersson (DICE).
Progressive Lightmapper: An Introduction to Lightmapping in UnityUnity Technologies
In 2018.1 we removed the preview label from the Progressive Lightmapper – we’ve made memory improvements, optimizations, and have had customers battle test it. We are now also working on a GPU accelerated version of the lightmapper. In this session, Tobias and Kuba will provide an intro to the basics of lightmapping and address of the most common issues that users struggle with and how to solve them. They will also provide an update on the future roadmap for lightmapping in Unity.
Tobias Alexander Franke & Kuba Cupisz (Unity Technologies)
Talk from SIGGRAPH 2010 and the <a />Beyond Programmable Shading course</a>
Also see <a />publications.dice.se</a> for more material and other DICE talks.
We present the technology and ideas behind the unique lighting in MIRRORS EDGE from DICE. Covering how DICE adopted Global illumination into their lighting process and Illuminate Labs current toolbox of state of the art lighting technology.
Talk by Fabien Christin from DICE at GDC 2016.
Designing a big city that players can explore by day and by night while improving on the unique visual from the first Mirror's Edge game isn't an easy task.
In this talk, the tools and technology used to render Mirror's Edge: Catalyst will be discussed. From the physical sky to the reflection tech, the speakers will show how they tamed the new Frostbite 3 PBR engine to deliver realistic images with stylized visuals.
They will talk about the artistic and technical challenges they faced and how they tried to overcome them, from the simple light settings and Enlighten workflow to character shading and color grading.
Takeaway
Attendees will get an insight of technical and artistic techniques used to create a dynamic time of day system with updating radiosity and reflections.
Intended Audience
This session is targeted to game artists, technical artists and graphics programmers who want to know more about Mirror's Edge: Catalyst rendering technology, lighting tools and shading tricks.
This presentation gives an overview of the rendering techniques used in KILLZONE 2. We put the main focus on the lighting and shadowing techniques of our deferred shading engine and how we made them play nicely with anti-aliasing.
Talk by Graham Wihlidal (Frostbite Labs) at GDC 2017.
Checkerboard rendering is a relatively new technique, popularized recently by the introduction of the PlayStation 4 Pro. Many modern game engines are adding support for it right now, and in this talk, Graham will present an in-depth look at the new implementation in Frostbite, which is used in shipping titles like 'Battlefield 1' and 'Mass Effect Andromeda'. Despite being conceptually simple, checkerboard rendering requires a deep integration into the post-processing chain, in particular temporal anti-aliasing, dynamic resolution scaling, and poses various challenges to existing effects. This presentation will cover the basics of checkerboard rendering, explain the impact on a game engine that powers a wide range of titles, and provide a detailed look at how the current implementation in Frostbite works, including topics like object id, alpha unrolling, gradient adjust, and a highly efficient depth resolve.
Course presentation at SIGGRAPH 2014 by Charles de Rousiers and Sébastian Lagarde at Electronic Arts about transitioning the Frostbite game engine to physically-based rendering.
Make sure to check out the 118 page course notes on: http://www.frostbite.com/2014/11/moving-frostbite-to-pbr/
During the last few months, we have revisited the concept of image quality in Frostbite. The core of our approach was to be as close as possible to a cinematic look. We used the concept of reference to evaluate the accuracy of produced images. Physically based rendering (PBR) was the natural way to achieve this. This talk covers all the different steps needed to switch a production engine to PBR, including the small details often bypass in the literature.
The state of the art of real-time PBR techniques allowed us to achieve good overall results but not without production issues. We present some techniques for improving convolution time for image based reflection, proper ambient occlusion handling, and coherent lighting units which are mandatory for level editing.
Moreover, we have managed to reduce the quality gap, highlighted by our systematic reference comparison, in particular related to rough material handling, glossy screen space reflection, and area lighting.
The technical part of PBR is crucial for achieving good results, but represents only the top of the iceberg. Frostbite has become the de facto high-end game engine within Electronic Arts and is now used by a large amount of game teams. Moving all these game teams from “old fashion” lighting to PBR has required a lot of education, which have been done in parallel of the technical development. We have provided editing and validation tools to help the transition of art production. In addition, we have built a flexible material parametrisation framework to adapt to the various authoring tools and game teams’ requirements.
A technical deep dive into the DX11 rendering in Battlefield 3, the first title to use the new Frostbite 2 Engine. Topics covered include DX11 optimization techniques, efficient deferred shading, high-quality rendering and resource streaming for creating large and highly-detailed dynamic environments on modern PCs.
With the highest-quality video options, Battlefield 3 renders its Screen-Space Ambient Occlusion (SSAO) using the Horizon-Based Ambient Occlusion (HBAO) algorithm. For performance reasons, the HBAO is rendered in half resolution using half-resolution input depths. The HBAO is then blurred in full resolution using a depth-aware blur. The main issue with such low-resolution SSAO rendering is that it produces objectionable flickering for thin objects (such as alpha-tested foliage) when the camera and/or the geometry are moving. After a brief recap of the original HBAO pipeline, this talk describes a novel temporal filtering algorithm that fixed the HBAO flickering problem in Battlefield 3 with a 1-2% performance hit in 1920x1200 on PC (DX10 or DX11). The talk includes algorithm and implementation details on the temporal filtering part, as well as generic optimizations for SSAO blur pixel shaders. This is a joint work between Louis Bavoil (NVIDIA) and Johan Andersson (DICE).
Progressive Lightmapper: An Introduction to Lightmapping in UnityUnity Technologies
In 2018.1 we removed the preview label from the Progressive Lightmapper – we’ve made memory improvements, optimizations, and have had customers battle test it. We are now also working on a GPU accelerated version of the lightmapper. In this session, Tobias and Kuba will provide an intro to the basics of lightmapping and address of the most common issues that users struggle with and how to solve them. They will also provide an update on the future roadmap for lightmapping in Unity.
Tobias Alexander Franke & Kuba Cupisz (Unity Technologies)
Talk from SIGGRAPH 2010 and the <a />Beyond Programmable Shading course</a>
Also see <a />publications.dice.se</a> for more material and other DICE talks.
We present the technology and ideas behind the unique lighting in MIRRORS EDGE from DICE. Covering how DICE adopted Global illumination into their lighting process and Illuminate Labs current toolbox of state of the art lighting technology.
Talk by Fabien Christin from DICE at GDC 2016.
Designing a big city that players can explore by day and by night while improving on the unique visual from the first Mirror's Edge game isn't an easy task.
In this talk, the tools and technology used to render Mirror's Edge: Catalyst will be discussed. From the physical sky to the reflection tech, the speakers will show how they tamed the new Frostbite 3 PBR engine to deliver realistic images with stylized visuals.
They will talk about the artistic and technical challenges they faced and how they tried to overcome them, from the simple light settings and Enlighten workflow to character shading and color grading.
Takeaway
Attendees will get an insight of technical and artistic techniques used to create a dynamic time of day system with updating radiosity and reflections.
Intended Audience
This session is targeted to game artists, technical artists and graphics programmers who want to know more about Mirror's Edge: Catalyst rendering technology, lighting tools and shading tricks.
This presentation gives an overview of the rendering techniques used in KILLZONE 2. We put the main focus on the lighting and shadowing techniques of our deferred shading engine and how we made them play nicely with anti-aliasing.
Siggraph2016 - The Devil is in the Details: idTech 666Tiago Sousa
A behind-the-scenes look into the latest renderer technology powering the critically acclaimed DOOM. The lecture will cover how technology was designed for balancing a good visual quality and performance ratio. Numerous topics will be covered, among them details about the lighting solution, techniques for decoupling costs frequency and GCN specific approaches.
Original slides at: http://crytek.com/cryengine/presentations
For Crysis 2, the R&D team at Crytek created the third iteration of CryENGINE. This lecture covers various topics related to Crytek’s latest engine iteration. Tiago Sousa provides an overview of the rendering pipeline, and the successful transition to a multiplatform friendly deferred lighting approach; how gamma-correct HDR rendering and its multiplatform details have been handled, with a focus on performance and quality. Other topics include deferred lighting techniques such as efficiently handling skin-rendering, and overcoming alpha-blending problems for hair/fur rendering for current-generation hardware; water-rendering and dynamic interaction; batched HDR post-processing; and how AA was handled. The lecture also includes multiplatform comparisons on final image quality, optimization strategies, and performance analysis insights. It also unveils the DX11 implementation of certain features.
Computer Graphics and its applications, Elements of a Graphics, Graphics Systems: Video Display Devices, Raster Scan Systems, Random Scan Systems, Input devices.
: Introduction of Rendering, Raytracing, Antialiasing, Fractals
Rendering Technologies from Crysis 3 (GDC 2013)Tiago Sousa
This talk covers changes in CryENGINE 3 technology during 2012, with DX11 related topics such as moving to deferred rendering while maintaining backward compatibility on a multiplatform engine, massive vegetation rendering, MSAA support and how to deal with its common visual artifacts, among other topics.
Look Ma, No Jutter! Optimizing Performance Across Oculus MobileUnity Technologies
The introduction of mobile AR means the arrival of more accessible devices, and for developers, a broader range of consumers to target. The good news is that you're "already ready." A stable of universal techniques and best practices can help reduce draw calls and maximize performance without sacrificing fidelity across Gear VR, Oculus Go, and Project Santa Cruz. We'll start with an overview of the devices and basic considerations, and go step by step through the process of reviewing and optimizing textures, scene geometry, and lighting. Attendees interested in Project Santa Cruz will also benefit from an introduction to Unity's profiling workflow.
Gabor Szauer - Oculus
More Performance! Five Rendering Ideas From Battlefield 3 and Need For Speed:...Colin Barré-Brisebois
This talk covers techniques from Battlefield 3 and Need for Speed: The Run. Includes chroma sub-sampling for faster full-screen effects, a novel DirectX 9+ scatter-gather approach to bokeh rendering, HiZ reverse-reload for faster shadow, improved temporally-stable dynamic ambient occlusion, and tile-based deferred shading on Xbox 360.
2. Filtering Approaches for Real-Time Anti-Aliasing Anti-Aliasing Methods in CryENGINE 3 Tiago SousaR&D Principal Graphics Engineer Crytek tiago@crytek.com
3. CryENGINE 3 AA Requirements Orthogonal and general solutions No per-platform AA solution Play nice with HDR/Deferred techniques Sub-pixel accuracy is important for us Schimering was the biggest offender on Crysis 1 and 2 levels Crysis had imensively aliased assets: alpha tested/tiny sub-pixel details HDR makes it even worse, big range of lighting contrast/color variation Low memory footprint Cost less than 2 ms on low end GPUs Every ms counts for consoles
4. MSAA Troubles for this HW Generation Memory requirements 2x, 4x, etc Multiplatform + Non conventional rendering [Sousa 2011] 0 support on PS3 for FP16 (for alpha blending passes) 10 MB EDRAM on x360 + Tilling + Resolves cost overhead Alpha testing AA, requires ATOC Tone mapping should be performed per sub-sample Else noticeable wrong results on high contrast regions Too expensive for older platforms
6. Temporal Anti-Aliasing (aka Motion Blur) Directional blur along screen space velocity vector [Green 2003] Delta from prev/cur screen space position, per-pixel or per vertex Image space motion blur Main benefict: Less noticeable aliasing during movement Pt Pt-1
8. A-Buffer SSAA [Haeberli90] Add sub-pixel jitter to camera frustum Brute force: Render scene multiple times N sub samples N scene renders Robust and best quality Also more uses besides SSAA (TSSAA/DOF/Soft-Shadows) Base concept used for our techniques Problem: Cannot afford render scene multiple times (yet) Great for reference/marketing quality shots though
11. Distribute A-Buffer SSAA Overframes Running at 60 fps ? Add sub-pixel jitter to camera frustum every frame Store previous/current frame and linear blend them Light-speed 2x SSAA: ~0.5 ms on current consoles 2 frames 2x SSAA, 4 frames 4x SSAA, etc But... not many reach 60 fps on consoles Lower fps results in extremelly noticeable image ghosting
13. Minimizing Artifacts Improving blending: Reprojection Velocity vector fetches from previous frame sub-sample target Exactly same as in TAA (but single tap) Deformable geometry slightly more expensive to handle Output pixel velocity into a render target Could not affort for vegetation Problem: Disocluded regions ghosting
15. Minimizing Artifacts (2) Disable blending if ||V|| > 0? Very rare the case when player not moving And we still want AA during camera movement Weighting using color/edge tagging ? Sub-pixel/hi frequency detail results in noticeable schimering Reprojection range clamping Pixel weight proportional to reprojection limit Eg: fBlendW = saturate( 1 - (fVLen / fVMaxLen) ) Coarse Depth stored in sub-sample buffer alpha channel Mask out if fVLen > fMaxVThreshold and fCurrD > fPrevD
22. Distributed A-Buffer SSAA: Caveats Not temporally stable No AA on disocluded regions Input signal changes (color/lighting), no robust solution yet Alpha blending problematic Withouth OIT, only possible to handle correctly for first hit Additional overhead Multi-GPU Additional frame latency to address For Crysis 2, we switched to Nvidia’s FXAA when in MGPU Schimering was again, biggest complain from MGPU users
23. Future Work SSAA combo with post processed AA Maybe similarly to DLAA: horizontal/vertical edges, blend taps This means at least 4 additional taps AA on disocluded regions
34. Distributed A-Buffer SSAA: Current Results Far from perfect, but: Orthogonal Sub-pixel accuracy Shader anti-aliasing bonus 2x Quincunx SSAA: 1 ms for consoles 0.2 ms at 1080p on pc’s 2x SSAA + edge AA: 1.7 ms 4x SSAA + edge AA: 2.2 ms 3 MB additional memory footprint
35. Acknowledgements Nick Kasyan,Nicolas Schulz, Vaclav Kyba, Michael Kopietz, Carsten Wenzel, Vladimir Kajalin, Andrey Konich, Ivo Zoltan Frey Jorge Jimenez, Diego Guitierrez, Naty Hoffman And to the entire Crytek team
36. Further Readings Haeberli, P, Akeley, K “The Accumulation Buffer: Hardware Support for High-Quality Rendering”, 1990 Siggraph’96 Course , Blythe, D et al “Programming with OpenGL: Advanced Rendering”, 1996 Green, S “Stupid OpenGL Shader Tricks”, 2003 Sousa, T. “Crysis Next Gen Effects”, 2008 Swoboda, M “Deferred Rendering in FrameRanger”, 2009 Yang, G et al “Amortized Super Sampling”, 2010 Binks, D. “Dynamic Resolution Rendering”, 2011 Sousa, T., Kasyan, N. and Schulz, N. “Secrets of the CryENGINE 3 Technology”, 2011
39. Bonus: Marketing Screenshots Always some trickery On CryENGINE 2 rendered multiple tiles at big resolutions and downsampled to get SSAA On CryENGINE 3 distributed SSAA with many samples Random sub-pixel jitter Almost perfect SSAA All Crysis 2 marketing shots used this variation
Editor's Notes
Multitude of workarounds and added complexityEach platform with its own solution, workarounds and optimizationsPerformance costFor current Gen, situation doens’t look promising...Minimal G-BufferRGBA8 World Space BF Normals + GlossinessReadback Z24S8 Depth + Stencil for tagging interior areasDeferred lighting buffersXbox 360: 2x RGB10F resolved to RGB10A2 for performance/memoryPS3: 2x RGBA8 encoded in RGBKPC: 2x FP16Scene bufferX360: RGB10F resolved to FP16 for performancePS3: RGBA8 encoded in RGBK for opaque, FP16 for transparentsPC: FP16X360: 28 MB; PS3: 31.5MB; PC: 49 MB at 720p, 110 MB at 1080p and so on.
Increasingly popularity since Crysis 1 and KZ2Aproximated as directional blur along screen space velocity vectorDelta from prev/cur screen space position, per-pixel or per vertex Added bonus: Great cinematic lookUser for all platforms
Alternatively, the usual approach of rendering higher resolution and downscale
Used in few older 60 fps games (DMC4?) , but who was first?And even at 60 fps, still noticeable ghosting
Strafing is the worst case, we’ll use if for images
Disable accumulation if ||V|| > 0?Very rare the case when player not movingAnd we still want AA during camera movementWeighting using color ?Sub-pixel/hi frequency detail results in noticeable schimeringReprojection range clamping ? (used for Crysis 2)Pixel weight proportional to reprojection limitEg: fBlendW = saturate( 1 - (fVLen / fVMaxLen) )Coarse Depth stored in sub-sample buffer alpha channelMask out if fVLen > fMaxVThreshold and fCurrD > fPrevD8 bit insufficient to cover large range, limited to nearby