Syysgraph 2018 - Modern Graphics Abstractions & Real-Time Ray Tracing

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Modern Graphics Abstractions
& Real-Time Ray Tracing
Syysgraph 2018
Graham Wihlidal, Colin Barré-Brisebois
SEED – Electron...

Modern Graphics Abstractions

Halcyon Rendering
▪ “PICA PICA” and Halcyon built from scratch
▪ Implemented lots of bespoke technology
▪ Only modern abst...

Halcyon Rendering
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Handles
Render Commands
Render Ba...

Render Backend

Render Backend
▪ Live-reloadable DLLs
▪ Enumerates adapters and capabilities
▪ Swap chain support
▪ Extensions (i.e. ray t...

Render Backend
▪ Provides debugging and profiling
▪ RenderDoc integration, validation layers, …
▪ Create and destroy rende...

Render Backend
S E E D // Halcyon + Vulkan
▪ Direct3D 12
▪ Vulkan 1.1
▪ Metal 2
▪ Proxy
▪ Mock

Render Backend
S E E D // Halcyon + Vulkan
▪ Direct3D 12
▪ Shader Model 6.X
▪ DirectX Ray Tracing
▪ Bindless Resources
▪ E...

Render Backend
S E E D // Halcyon + Vulkan
▪ Vulkan 1.1
▪ Sub-groups
▪ Descriptor indexing
▪ External memory
▪ Multi-draw ...

Render Backend
S E E D // Halcyon + Vulkan
▪ Metal 2
▪ Early development
▪ Primarily desktop
▪ Argument buffers
▪ Machine ...

Render Backend
S E E D // Halcyon + Vulkan
▪ Proxy
▪ Discussed later in the presentation

Render Backend
S E E D // Halcyon + Vulkan
▪ Mock
▪ Performs resource tracking and validation
▪ Command stream is parsed a...

Render Device

Render Device
S E E D // Halcyon + Vulkan
▪ Abstraction of a logical GPU adapter
▪ e.g. VkDevice, ID3D12Device, …
▪ Provid...

Render Device
S E E D // Halcyon + Vulkan
▪ Ownership of GPU resources
▪ Create & Destroy
▪ Lifetime tracking of resources...

Render Handles

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Handles
▪ Resources associated by handle
▪ Lightwei...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Handles
ID3D12Resource ID3D12Resource
DX12: Adapter...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Handles
▪ Can query if a device has a handle loaded...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Handles
▪ Shared resources are supported
▪ Primary ...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Render Handles
▪ Can also mix and match backends in the sa...

Render Commands

Render Commands
▪ Draw
▪ DrawIndirect
▪ Dispatch
▪ DispatchIndirect
▪ UpdateBuffer
▪ UpdateTexture
▪ CopyBuffer
▪ CopyText...

Render Commands
▪ Queue type specified
▪ Spec validation
▪ Allowed to run?
▪ e.g. draws on compute
▪ Automatic scheduling
...

Render Commands
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing

Render Command List
▪ Encodes high level commands
▪ Tracks queue types encountered
▪ Queue mask indicating scheduling rule...

Render Compilation
▪ Render command lists are “compiled”
▪ Translation to low level API
▪ Can compile once, submit multipl...

Render Graph

Render Graph
▪ Inspired by FrameGraph [O’Donnell 2017]
▪ Automatically handle transient resources
▪ Import explicitly mana...

Render Graph
▪ Basic memory management
▪ Not targeting current consoles
▪ Fine grained memory reuse sub-optimal with curre...

Render Graph
▪ Frame Graph → Render Graph: No concept of a “frame”
▪ Fully automatic transitions and split barriers
▪ Sing...

Render Graph
▪ Composition of multiple graphs at varying frequencies
▪ Same GPU: async compute
▪ mGPU: graphs per GPU
▪ Ou...

$Render Graph ▪ Composition of multiple graphs at varying frequencies ▪ e.g. translucency, refraction, global illumination ...$

Render Graph
▪ Two phases
▪ Graph construction
▪ Specify inputs and outputs
▪ Serial operation (by design)
▪ Graph evaluat...

 Construction phase
 Evaluation phase

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Explicit heterogeneous mGPU
▪ Parallel fork...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Workloads are divided into partitions
▪ Bas...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Run ray generation on primary GPU
▪ Copy re...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Only width is divided
▪ Simplifies textures...

▪ Lots of fun coordinate snapping bugs
▪ i.e. 3 GPUs partitioned to 0.33333…

▪ Lots of fun coordinate snapping bugs
▪ 16 GPUs! (because, why not?)

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ RenderGraphSchedule
▪ NoDevices → Pass is d...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ RenderTransferPartition
▪ PartitionAll → Se...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ PartitionAll → PartitionAll
▪ Copies full r...

Devices this pass will run on
Schedule transfers in or out
Scaling work dimensions for each GPU

Some bugs were obvious

Some bugs were obvious

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Some bugs were subtle
▪ Weird cell shading?...

Render Graph
Some of our render graph passes:
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Bloom
▪ Bo...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Implicit data flow via explicit scopes
▪ “L...

Render Graph
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ Lookup by type
▪ scope.get<T>() -> &T
▪ Par...

Render Graph DSL
▪ Experimental
▪ Macro Magic
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing

Virtual Multi-GPU

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
▪ Most developers have single GPU
▪ Unco...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
▪ Build device indirection table
▪ Virtu...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
▪ Create multiple instances of a device
...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
▪ Increases overall wall time (don’t use...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
▪ PICA PICA developers all had 1 GPU
▪ L...

S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
Virtual Multi-GPU
▪ Develop and debug multi-GPU with only ...

Render Proxy

▪ Remote render backend
▪ Any API / Any OS
Render Proxy
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing

▪ Render API calls are routed remotely
▪ Uses gRPC (high performance RPC framework)
▪ Use an API on an incompatible OS
▪ e...

▪ Scale large workloads with a GPU cluster
▪ Some API as render graph mGPU
▪ Only rendering is routed, scene state is loca...

Render device → gRPC

Protobuf 3
Schema

▪ The possibilities are endless!
Render Proxy
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing

Machine Learning

Machine Learning
▪ Deep reinforcement learning
▪ Rendering 36 semantic views
▪ Training with TensorFlow
▪ On-premise GPU c...

Machine Learning
▪ Adding inferencing with DirectML
▪ Hardware accelerated inferencing operators
▪ Resource management
▪ S...

Machine Learning
▪ Treat trained ML models like any other 3D asset
▪ Render Graph abstractions
▪ Reference the same render...

Machine Learning
▪ Provide operators as render commands
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ ...

Asset Pipelines

Asset Pipelines
▪ Geometry
▪ Animations
▪ Shaders
▪ Sounds
▪ Music
▪ Textures
▪ Scenes
▪ etc.
S E E D // Modern Graphics A...

Asset Pipelines
▪ Everything is content addressable
▪ Hash of data is the identity
▪ Sha256
▪ Merkle trees!
▪ Dependency e...

https://en.wikipedia.org/wiki/Merkle_tree

Asset Pipelines
▪ Containerized, running on Kubernetes
▪ Google Cloud Platform
▪ On-Premise Cluster
▪ AMD 1950X TR
▪ NV Ti...

Asset Pipelines
▪ Analytics with Prometheus and Grafana
▪ Publish custom metrics
▪ Scraped into rich UI
▪ Collecting data ...

Shaders

Shaders
▪ Exclusively HLSL
▪ Shader Model 6.X
▪ Majority are compute shaders
▪ Performance is critical
▪ Group shared memo...

Shaders
▪ No reflection
▪ Avoid costly lookups
▪ Only explicit bindings
▪ … except for validation
▪ Extensive use of HLSL ...

Shaders
S E E D // Halcyon + Vulkan
SPIR-VDXIL
Vulkan 1.1Direct3D 12
ISPCMSL
SPIRV-CROSS
AVX2, …Metal 2
DXC
HLSL

Real-Time Ray Tracing

Hybrid Rendering
Direct Shadows
(ray trace or raster)
Direct Lighting
(compute)
Reflections
(ray trace or compute)
G-Buffe...

No Ray Tracing

Path Tracing (1 spp)

Path Tracing Accumulation (15s)

Hybrid Rendering

No Ray Tracing

Hybrid Rendering

• Launch rays from G-Buffer
• Trace at half resolution
• Supports arbitrary normal & roughness
• Extensive spacial & tempo...

Raw Reflections (1/4 rpp)

After Reconstruction & Filtering

▪ Launch ray towards light
▪ Handled by Miss Shader (DXR)
▪ Soft shadows?
▪ Random direction from cone [PBRT]
▪ SVGF-style...

Hard Raytraced Shadows Soft Raytraced Shadows (Unfiltered) Soft Raytraced Shadows (Filtered)
S E E D // Modern Graphics Ab...

Transparent
Shadows
▪ Hard to get right with raster
▪ [McGuire17]
▪ Trace towards light
▪ Like opaque shadow tracing
▪ Acc...

Transparent Shadows
▪ Keep tracing until
▪ Hit opaque surface or all light is absorbed or miss
S E E D // Modern Graphics ...

Transparent Shadows (hard)

Transparent Shadows (soft)

ො𝑛
𝑝
𝐴 𝑝 =
1
𝜋
න
Ω
𝑉𝑝, ෝ𝑤(ො𝑛 ⋅ ෝ𝑤)𝑑𝜔

Screen-Space AO

Raytraced AO

Transparency & Translucency
▪ Raytracing enables accurate light scattering
▪ Transparency
▪ Order-independent (OIT)
▪ Mult...

$Transparency Works for clear and rough glass ▪ Clear ▪ No filtering required ▪ Rough ▪ Microfacet Models for Refraction th...$

Translucency
▪ For every valid position & normal
▪ Flip normal and push (ray) inside
▪ Launch rays in uniform sphere dist....

GI
▪ Can’t solve GI every frame
▪ Our GI budget: 250k rays / frame
▪ World space surfels
▪ Easy to accumulate
▪ Distribute...

Round 1: Victory!
▪ Initial DXR-enabled games to have a set of ray tracing features
▪ Reflections
▪ Shadows
▪ AO
▪ …
▪ Dev...

Round 2:
Open Problems

Transparency
▪ Transparency is far from being solved!
▪ Glass
▪ Clear/rough + filtered + shadowed
▪ Particles & Volumetric...

Partial Coverage
▪ Foliage
▪ Can still do alpha test (i.e.: any-hit)
▪ Animated becomes a real problem
▪ Defocus effects s...

Specialized Denoising
▪ PICA PICA: specialized denoising & reconstruction for
view-dependent, light-dependent or any other...

Geometry++
▪ Ray-Triangle intersection is the fast path
▪ Well-known problem [Möller 1997] [Baldwin 2016]
▪ Intersection o...

Taking Advantage of DXR
▪ DirectX offers easy interoperability between raster, compute and raytracing
▪ Raytracing, raster...

Performance (1/)
▪ Best tracing performance comes from one massive static BVH?
▪ Viable for a big static scene, but most g...

Performance (2/)
▪ 2+ levels hierarchies?
▪ + : Allow to split worlds into update frequencies, and avoid updates if nothin...

Performance (3/)
▪ Example: 1K materials, fire 100k secondary rays
▪ (Potentially) need to run 1k shaders each on a batch ...

Future Ray Tracing Research
▪ RTRT literature has focused on accelerating “correct” raytracing
▪ Lots new challenges with ...

Global Illumination
▪ Open problem even in offline rendering
▪ Variance still too high
▪ Can reduce frequency
▪ Prefilteri...

Summary
▪ Modern graphics abstractions are important for efficiency without verbosity
▪ Real-time ray tracing significantl...

References
S E E D // Modern Graphics Abstractions & Real-Time Ray Tracing
▪ [Andersson 2018] Johan Andersson, Colin Barré...

Thanks
▪ Matthäus Chajdas
▪ Rys Sommefeldt
▪ Timothy Lottes
▪ Tobias Hector
▪ Neil Henning
▪ John Kessenich
▪ Hai Nguyen
▪...

Thanks
▪ SEED
▪ Johan Andersson
▪ Jasper Bekkers
▪ Joakim Bergdahl
▪ Ken Brown
▪ Dean Calver
▪ Dirk de la Hunt
▪ Jenna Fri...

S E E D / / S E A R C H F O R E X T R A O R D I N A R Y E X P E R I E N C E S D I V I S I O N
S T O C K H O L M – L O S A ...

Questions?
Graham Wihlidal
graham@ea.com
@gwihlidal
Colin Barré-Brisebois
cbbrisebois@ea.com
@ZigguratVertigo