「原神」におけるコンソールプラットフォーム開発

Zhenzhong Yi
Studio Technical Director, miHoYo
Head of Genshin Impact Console Development
From Mobile to Console: Genshin
Impact’s rendering technology on
Console

Genshin Impact
• Open-world action RPG
• Multiplatform support, including
PS4, PC, and mobile versions
• Long game life-cycle with
numerous updates over time
• Features an “anime” rendering
style

About Myself
• Zhenzhong Yi
• Over 10 years of console game development experience. Worked for companies both in China
and the US before joining miHoYo, including：
• Microsoft Xbox - Seattle, WA
• Avalanche Studios - New York City, NY
• Zindagi Games - Los Angeles, CA
• Ubisoft Shanghai
• Returned to China and joined miHoYo in early 2019 to build the console development team

Presentation Outline
• Console rendering pipeline overview
• Analysis of key technical points
• Summary and conclusion

• Unity is an extremely flexible engine
• Unity’s concise coding style allowed us to more conveniently customize
the development of Genshin Impact's rendering pipeline
• Unity China’s technical support also provided great assistance and
cooperation, so we truly appreciate their help

• More than half of our efforts were focused on making full use of the
console’s hardware architecture in development and optimization
• We accumulated tons of experience through this process and had several
technical breakthroughs. However…
• According to Sony NDA restrictions:
• We cannot share hardware-related content
• We cannot share information concerning low-level optimization
• We will not be discussing the CPU, I/O, or other such modules today

• Genshin Impact’s engine has two different customized rendering pipelines
• PC and console ➔ “console rendering pipeline”
• Android and iOS ➔ “mobile rendering pipeline”
• The overall tone is PBR based stylized rendering
• The game’s versions are under simultaneous development, with mobile serving as the
primary development platform, and console development following close behind
• The selection of technologies must consider resource production and possible
runtime costs of all platforms
Console Rendering Pipeline Overview

• Based on PBR, which keeps the light and shadow effects
of the entire environment uniform
• Not completely energy conservative
• The lighting models of different materials are altered based on art
requirements
• The game’s light and shadow effects are calculated in
real-time
• Time of day
• Dynamic weather

• High-Resolution output
• PS4 Pro – native 4k resolution
• Standard PS4 – 1440P rendering resolution, 1080P output resolution
• A clearer image is more conducive to our game’s art style
• Extensive use of compute shaders
• Over half of the features in the console pipeline were implemented via
compute shaders

• Lighting and materials appear very different from more realistic-looking
games, since the art team had very special requirements for them
• Dirty, noisy, dark
• Fresh, bright, clean, anime-like
• We kept these 2 groups of words in mind at all times

• A deeply customized Unity engine for mobile
• We couldn’t just rely on Unity's own PS4 platform implementation
• Nearly non-existent development resources
• Initially, the console team consisted of one member: me!
• Whole studio severely lacked console development experience
• Many other things to handle: Sony Accounts, PSN Store, PS4 TRC, etc.
• A tight development deadline
• Starting from zero with a timeline of approximately a year and a half
What We Started With

Principles We Followed
• When transforming the rendering pipeline for console, we adhered to the following ideas:
• Avoid excessive features
• Cool-looking technologies do not necessarily match the style of our game
• Choose practical and mature technologies whenever possible
• No time for trial and error
• It’s best to have interaction between multiple technologies
• Systematic transformation allows the resulting picture to appear more unified
• The benefits of improved picture are exponential: 1 + 1 > 2

• Selected a few technical points from scene lighting and
shadow effects
• Shared ideas on how to upgrade Genshin Impact’s
visual quality
• Focused on methodology

Shadows
• To match the art style, shadows needed to provide enough
details up close while still covering a large enough area

• Cascaded shadow map + Poisson noise soft shadows
• Did not use the usual 4 cascades, instead we used up to 8 cascades
• More cascades bring better shadow effects, but also bring more
performance overhead
• More draw calls resulting in more CPU overhead
• More cascades resulting in more GPU overhead
Shadows

• CPU optimization
• Used shadow caching to reduce draw call count
• First 4 cascades are updated with every frame
• Last 4 cascades are updated via interleaving
• Total 5 cascades updated each frame
• All cascades are updated at least once every 8 frames
Shadows

Shadows
• GPU optimization
• We used a screen space shadow map
• Each pixel will do 11 samples based on Poisson disc to generate soft shadows,
and to eliminate banding, sampling patterns are rotated
• The cost of the entire pass is about 2 to 2.6ms
• Do we really need to do such intensive calculations for every single pixel?

Shadows
• GPU optimizations
• Soft shadows are only useful at edges of shadows
• A full-screen mask map is generated to mark the shadow, non-shadow, and
penumbra areas
• Soft shadows are only calculated for pixels marked in penumbra areas
• 2 – 2.6ms ➔ 1.3 – 1.7ms

⚫ Low resolution calculation
⚫ ¼ x ¼ resolution
⚫ 16 pixels correspond to a mask value
⚫ Calculate each pixel to determine if it’s in shadow or not,
merge the results to get the mask pixels
⚫ Accurate, but slow, and must be done 16 times
⚫ Optimization: Select a small number of sample
points to calculate, get approximate results
⚫ A few samples cannot perfectly represent the 4x4 tile
⚫ Blur the mask map to enlarge the penumbra area
Mask Generation

• Characters and scenery that are already in shadow appear to be floating
• Using ambient occlusion (AO), characters and scenery in the shadows will cast
faint soft shadows around them
• The game uses 3 different kinds of AO technology
• HBAO provides more details in small areas
• AO Volume provides a wide range of AO for static objects
• Capsule AO provides a wide range of AO for characters
Multiple AO Technologies

• AO Volume generates a larger range of AO than HBAO
• For example, a tabletop can cast large AO on the floor
• HBAO cannot provide such effects
• Occlusion information for each object is generated offline in
object local space
• AO value is calculated during runtime via this local space
occlusion information
AO Volume

• AO Volume solves the large range AO problem for static objects
• But the game’s characters have skeletal animations
• The occlusion information cannot be just generated offline
• We used capsule AO technology for characters
• Skeletal animations are used to update capsules
• The occlusion is directional
Capsule AO

• ½ x ½ resolution AO render target, bilateral upsampled to the full res. AO texture
• Bilateral filtered Gaussian blur is applied to eliminate noise. Remember, no noise!
• Points for further optimization:
• The bilateral filter has lots of repeated calculations
• 2 pass blur + 1 pass upsample = multiple AO reads and writes
• Solution：
• Complete the compute shader in a single pass
Applying AO

• Implemented clustered deferred lighting, supports 1024 lights in view
• Screen is divided into 64 x 64 pixel tiles, with each tile sliced into 16 clusters in depth
direction
Local Lighting

• Nearly 100 lights in view can cast real-time shadow
• We could support more, but these are enough
• Dynamically adjust shadow map resolution based on distance and priorities
• Baked static shadow texture + dynamic object shadows
• With lots of local light, the large amount of baked shadow textures
puts a heavy load on both the game’s capacity and I/O
Local Light Shadows

• Static scene shadow texture is baked offline and then compressed
• Use compute shader to decompress at runtime, the decompression speed is
very fast
• On a base PS4, about 0.05ms to decompress a 1k x 1k shadow texture
Local Light Shadows

Local Light Shadow Texture Compression
• 2 x 2 block encode into 32 bits used for every 4 depth values
• Plane equation mode or packed floating-point mode
• 64 bits with optional high-precision compression
• Quadtree is used to merge encoded data, further increasing compression rate
• 16 x 16 blocks form a single tile, each tile has 1 quadtree
• Reference: [Bo Li, 2019]

• Compression Rate: In default precision mode, compression rate of a typical
indoor scene is about 20:1 – 30:1, high precision mode is about 40 – 70% of that
• Compression of shadow texture is essential, the capacity can be reduced by an
order of magnitude

Default precision
compression

High-Precision
compression

• The size of a 2k x 2k texture is reduced from 8MB to 274.4KB
• The default precision compression rate reaches up to 29.85:1
• The difference between the high-precision compressed texture and the
uncompressed texture is indistinguishable to the naked eye
• The resulting texture size is 583.5KB with an approximate compression ratio
of 14:1

Volumetric Fog
• Volumetric fog can be illuminated by a light source

Volumetric Fog
• If the local light has a projection texture, volumetric fog will produce the
corresponding effect

• Physically-based light scattering
• The volumetric fog can be controlled with different parameters in different areas
• Volumetric fog is illuminated by the light source
• Temporal filter is used for multi-frame blending which results in a smoother and more
stable fog image
• The GPU cost is less than 1ms
Volumetric Fog

Volumetric Fog
• Camera view based
• The view frustum is divided into voxels and aligned with the clusters of clustered
deferred lighting
• The fog parameters are saved in texture and loaded into the world via streaming
• Injected into voxel while calculating
• Take local lights into account
• Ray marching to get volumetric information

• Occluded directional light produces the god ray effect
God Rays

God Rays
• A separate pass to generate god rays
• ½ x ½ resolution
• Generated via ray marching, sampling up to 5 shadow cascades
• Provides adjustable parameters for the art team to add god rays on top of
volumetric fog

God Rays
• Volumetric fog can also generate god rays
• But the resulting effect in-game was not satisfactory to the art team
• The voxels’ resolution wasn’t enough
• The intensity of god rays relies on the density of the
volumetric fog, but dense fog on screen looks dirty
• Separately generated god rays have a higher resolution, sharper
edges, and more room for adjustment by the art team

God Rays
God Ray From
Volumetric Fog

God Rays
God Ray From
GodRay Pass

• Used reflection probes to provide reflection information for the scene
• Time of day + dynamic weather means we cannot use baked cubemap
• Offline bake scene data into a mini G-Buffer
• Runtime generates cubemaps using real-time lighting condition
• The art team can place multiple reflection probes wherever necessary
Reflection Probes

• Reflection probes are updated at runtime
• The process consists of three steps in total: relight, convolve, then compress
• Compute shader is used to simultaneously process all 6 faces of a cubemap
• Calculations are done in multiple frames with one probe being processed at a
time, looping continuously
Reflection Probes

Reflection Probes
+
Relight
=
• Relight: Mini G-Buffer is lit based on the current lighting condition

Reflection Probes
• Convolve: Generate the cubemap's mipmap chain, convolution applied on each mip
level
• Compress: Performs a BC6H compression on cubemap, 4x4 block ➔ 128 bits

• After relighting, the reflection probes contain the current lighting condition,
from which we can extract ambient information
• This is then saved as 3-band SH
• After the reflection probes are updated, the corresponding ambient probes
are then automatically updated
• Implemented using compute shader
Ambient Probes

• Relight does not consider shadows for the sake of performance and size of data
• Both reflection probes and ambient probes will leak light
• Ground located within a shadow will appear too bright
• Shadows are baked offline and saved as shadow SH
• In the same way, we save the local light's lighting information into a local light SH
• Finally, during the relight step we add the shadow SH and local light SH
Improvements on Image-Based Lighting

Shadow SH OFF

Shadow SH ON

Shadow SH OFF
Local Light SH OFF

Shadow SH OFF
LocalLight SH OFF
Local Light SH ON

• Reflection probes are divided into indoor and outdoor types in order to handle
different indoor and outdoor lighting conditions
• Our art team uses the interior mesh to mark which pixels are affected by an
indoor lighting environment
• The ambient probes will accordingly generate different ambient lighting for
indoor and outdoor environments
Interior Marks

Screen Space Reflection (SSR)
• Different technique than that used for water surface reflections
• On a PS4 Pro, the GPU overhead is around 1.5ms
• A temporal filter is used to increase stability
• Hi-Z buffer is used for acceleration and allow rays trace up to the full screen
• During the reflection, the color buffer of the previous frame is sampled

• As was visible in the previous images, even without SSR, the game will still
have the reflection information provided by reflection probes
• Deferred reflection pass simultaneously performs the reflection and ambient
calculations
• The AO value is considered when reflection information is added to the
lighting calculation, which then effectively reduces issues of leaking light
Runtime Reflection System

• HDR: High Dynamic Range Display
• SMPTE ST 2084 Transfer Function
• Supports a max brightness of 10,000 nits
• WCG: Wide Color Gamut
• The Rec. 2020 color space covers 75.8% of the
CIE 1931 color space, compared to Rec. 709,
which is commonly used by HDTV and only
covers 35.9%
• (Image source: Wikipedia)
HDR Display

HDR Display
• Here’s a breakdown of Genshin Impact’s console rendering pipeline for SDR
and HDR modes:

• Starting from 1.2, Genshin Impact will support HDR10 on PS4
• No SDR tone mapping, color grading or gamma correction
• WCG color grading LUT is made in software like DaVinci
• WCG color LUT pass takes less than 0.05ms
• Including white balance, WCG color grading and color expansion
• ACES pipeline (RRT + ODT) was not used as it does not fit our game’s style
• By blending HDR output with tone mapping results, the scene within the SDR
brightness range looks closer to the SDR version
HDR Display

• The issue of consistency within the SDR brightness range
• SDR: EOTF_BT1886 (OETF_sRGB (color)) != color
• HDR: Inverse_PQ(PQ(color)) = color
• An OOTF curve is added at the end of the HDR pipeline to simulate
the error caused by EOTF_BT1886 (OETF_sRGB (color)) conversion
HDR Display

• Many art resources rely on hue shift generated by the tone mapping curve
• There is no tone map in a hue-preserving HDR pipeline
Addressing the Issue of Hue Shift
╳
SDR (Hue Shift)
=
HDR (Hue Preserving)

• Blackbody radiation solution
• Temperature is used to calculate color
• Requires the art team to modify their assets
• Genshin Impact uses a method of simulating hue shift in the shader
• Does not require any modification of assets
• Final color grading now comes with a tone mapping curve, so there’s no need
for mixing tone mapping as we mentioned before
• Calculations are combined in the LUT
Addressing the Issue of Hue Shift

• Console rendering pipeline overview
• Analysis of key technical points
• Summary and conclusion
Presentation Outline

• Global players' acceptance of Genshin Impact on the PlayStation 4 has far
exceeded our expectations
• However, there are still many areas that require further optimization, and
we will continue to make more improvements
• Better performance
• Faster loading times
• More stability
• More graphics features that suit the game’s style

• Genshin Impact is only the beginning and is our first venture in console
development
• Time is limited, resources are limited
• The team requires talent of all sorts
• We’ve gained experience in combining realistic rendering technology with
anime rendering style
• There is still much we’d like to do, especially with the next generation of
consoles arriving now
• We look forward to more opportunities to exchange development
experiences

• We will continue to develop future products
• The game programming team is continually recruiting
new members
• Especially hiring console development-related
positions
• Join our team of tech otakus saving the world!
We Are Hiring!

With Special Thanks To:
• The entire Genshin Impact engine team
• Every member who contributed to Genshin
Impact’s console development ♥
• Special member of the console team, Lulu🐱
• A very special thanks to Wenli Chen, Terry Liu,
and all the global tech support specialists at
Sony for all the support they have given us

References：
• Josiah Manson, 2016,“Fast Filtering of Reflection Probes”
• Li Bo, SIGGRAPH 2019, “A Scalable Real-Time Many-Shadowed-Light Rendering System”
• Michal Iwanicki, SIGGRAPH 2013, “Lighting Technology of The Last Of Us”
• Paul Malin, 2018, “HDR Display in Call of Duty®”
• Yasin Uludag, 2014, <GPU Pro 5>,“Hi-Z Screen-Space Cone-Traced Reflections”
• Nathan Reed, GDC 2012, “Ambient Occlusion Fields and Decals in Infamous 2”
• Fabian Bauer, SIGGRAPH 2019, “Creating the Atmospheric World of Red Dead Redemption 2: A
Complete and Integrated Solution”

「原神」におけるコンソールプラットフォーム開発

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 「原神」におけるコンソールプラットフォーム開発

Similar to 「原神」におけるコンソールプラットフォーム開発 (20)

More from Unity Technologies Japan K.K.

More from Unity Technologies Japan K.K. (20)

Recently uploaded

Recently uploaded (20)

「原神」におけるコンソールプラットフォーム開発