SlideShare a Scribd company logo
1 of 71
Killzone Shadow Fall:
Threading the Entity
Update on PS4
Jorrit Rouwé
Lead Game Tech, Guerrilla Games
Introduction
• Killzone Shadow Fall is a First Person Shooter
• PlayStation 4 launch title
• In SP up to 60 characters @ 30 FPS
• In MP up to 24 players @ 60 FPS
• Gameplay logic has lots of
• Branches
• Virtual functions
• Cache misses
• Not suitable for PS3 SPU’s but PS4 has 6 x86 cores
What do we cover?
• What is an Entity?
• What we did on PS3
• Multi threading on PS4
• Balancing Entities across frames
• Performance issues found
• Performance results achieved
• Debug tools
What is an Entity?
What is an Entity?
• Base class for most game objects
• E.g. Player, Enemy, Weapon, Door
• Not used for static world
• Has Components
• E.g. Model, Mover, Destructibility
• Update at a fixed frequency
• 15, 30, 60 Hz
• Almost everything at 15 Hz
• Player updated at higher frequency to avoid lag
What is a Representation?
• Entities and Components have Representation
• Controls rendering, audio and VFX
• State is interpolated in frames where entity not updated
• Cheaper to interpolate than to update
• Introduces latency
• Always blend towards last update
Movie
Multi Threading Approach
PS3: 1 Entity = 1 Fiber
• Most time spent on PPU
• No clear concurrency model
• Read partial updated state
• Entities deadlock waiting for each other
yield fiber resume fiber
Time
PPU
SPU 1
SPU 2
Entity 1 Entity 2 Entity 3 Entity 1
Animation
Ray Cast
Entity 2
PS4: 1 Entity = 1 Job
• No fibers
• Entity updates as a whole
• How to solve race conditions?
Time
CPU 1
CPU 2
CPU 3
Entity 1
Entity 2
Entity 3
Entity 1Animation
Ray Cast Entity 2

job
• Make update order explicit:
Strong Dependencies
Missile
MissileMissile LauncherMissile Launcher Soldier
Time
Soldier
strong
dependency
• No (indirect) dependency = no access
• Works two ways: Weapon can access Soldier too
• Create dependency has 1 frame latency
• Global systems need locks
Non-dependent Entities can Execute Concurrently
Soldier1
Weapon1
Time
Soldier2
Weapon2
CPU 2
CPU 1 Soldier1 Weapon1
Soldier2 Weapon2
• A few entities cause huge bottleneck
What about this?
Time
Soldier1
Pistol1
Soldier2
Pistol2
Bullet System
Soldier1 Pistol1 Soldier2 Pistol2 Bullet System
Non-exclusive Dependencies
• Access to ‘Bullet System’ must be lock protected
Soldier1 Soldier2
Bullet System
non-exclusive
Time
CPU 2
CPU 1 Soldier1 Pistol1
Soldier2 Pistol2
Bullet System
Barrier
Pistol1 Pistol2
Weak Dependencies
• 2 tanks fire at each other:
• Update order reversed when circular dependency occurs
• Not used very often (< 10 per frame)
Time
Missile2Missile1
Tank1 Tank2 Missile2Missile1
Tank2 Tank1 Missile1Missile2
or
weak
strong
Tank1 Tank2
Non-updating Entities
• Entity can skip updates (LOD)
• Entity can update in other frame
• Do normal scheduling!
Entity1
Entity3
Entity1
Entity2 Entity3
Time
not updating
Summarizing Dependencies
Strong
Exclusive
Weak
Exclusive
Strong
Non-excl.
Weak
Non-excl.
Symbol
Two way access ✓ ✓ ✓ ✓
Order guaranteed ✓ ✓
Allow concurrency + + ++ ++
Require lock ✓ ✓
Referencing Entities
• Dev build: CheckedPtr<Entity>
• Acts as normal pointer
• Check dependency on dereference
• Retail build: Entity *
• No overhead!
• Doesn’t catch everything
• Can use pointers to members
• Bugs were easy to find
Working with Entities without dependency
• ThreadSafeEntityInterface
• Mostly read only
• Often used state (name, type, position, …)
• Mutex per Entity
• Send message (expensive)
• Processed single threaded when no dependency
• Schedule single threaded callback (expensive)
• Everything can be accessed
Scheduling Algorithm
Scheduling Algorithm
• Entities with exclusive dependencies merged to 1 job
• Dependencies determine sorting
• Non-exclusive dependencies become job dependencies
• Expensive jobs kicked first!
Missile
Missile Launcher
Tank
Bullet System
Pistol
Soldier
Missile Tank
Bullet System
PistolSoldier
Job1
Job2
job dependency
not updating
weak
non-exclusive
strong
Scheduling Algorithm – Edge Case
Entity1 Entity3
Entity4Entity2
non-exclusive
exclusive job dependency
Entity1 Entity2
Entity3 Entity4
Job1
Job2
Scheduling Algorithm – Edge Case
• Non cyclic dependency becomes cyclic job dependency
• Job1 and Job2 need to be merged
Entity1 Entity3
Entity4Entity2
non-exclusive
exclusive
Entity1 Entity2Entity3 Entity4
Job1
Balancing Entities Across Frames
Balancing Entities Across Frames
• Prevent all 15 Hz entities from updating in same frame
• Entity can move to other frame
• Smaller delta time for 1 update
• Keep parent-child linked entities together
• Weapon of soldier
• Soldier on mounted gun
• Locked close combat
Balancing Entities – In Action
Time in even frame
(sum across cores,
running average)
Balancing Entities – In Action
Time in odd frame
(should be equal)
Balancing Entities – In Action
Civilian @ 15Hz
update even frame
Balancing Entities – In Action
Civilian @ 15Hz
update odd frame
Balancing Entities – In Action
Flash on switch odd / even
Movie
Movie
Performance Issues
Performance Issues
• Memory allocation mutex
• Eliminated many dynamic allocations
• Use stack allocator
• Locking physics world
• R/W mutex for main simulation world
• Second ‘bullet collision’ broadphase + lock
• Large groups of dependent entities
• Player update very expensive
Cut Scene - Problem
• Cut scene entity requires dependencies
• 10+ characters in cut scene creates huge job!
Civilian 1
Cut Scene
Civilian 2 Player Camera
Cut Scene - Solution
• Create sub cut scenes for non-interacting entities
• Master cut scene determines time and flow
• Scan 1 frame ahead in timeline to create dependency
Civilian 1
Cut Scenenon-exclusive
Sub Cut Scene 1 Sub Cut Scene 2
Civilian 2
Sub Cut Scene 3
Player Camera
Using an Object
• Dependencies on useable objects not possible (too many)
• Get list of usable objects
• Global system protected by lock
• ‘Use’ icon appears on screen
• Player selects
• Create dependency
• Start ‘use’ animation
• Start interaction 1 frame later (dependency valid)
• Hides 1 frame delay!
Grenade
• Explosion damages many entities
• Creating dependencies not option (too many)
• ThreadSafeEntityInterface not an option
• Need knowledge of parts
• Run line of sight checks inside update
• Uses scheduled callback to apply damage
Performance Results
5000 Crates
Performance Results - Synthetic
100 Soldiers 500 Flags
Level
Counts Dependencies Max
Entities
in Job
SpeedupNumber
Entities
Updating
Entities
Number
Humans
Strong
Excl
Strong
Non-Excl
5000 Crates (20 µs each) 5019 5008 1 12 4 13 2.8X
100 Soldiers (700 µs each) 326 214 105 212 204 19 4.2X
500 Flags (160 µs each) 519 508 1 12 4 13 5.2X
6 cores!
Performance Results - Levels
Level
Counts Dependencies Max
Entities
in Job
SpeedupNumber
Entities
Updating
Entities
Number
Humans
Strong
Excl
Strong
Non-Excl
The Helghast (You Owe Me) 1141 206 32 71 23 20 4.1X
The Patriot (On Vectan Soil) 435 257 44 199 107 15 4.3X
The Remains (12p Botzone) 450 128 14 97 44 18 3.7X
The Helghast The RemainsThe Patriot
Game Frame - The Patriot
Game Frame - Global
Time
18 ms
Game Frame - Global
CPU 1 (main thread)
Game Frame - Global
CPU 2
Game Frame - Global
CPU 3
Game Frame - Global
AI
Physics
Entity Update VFX & Draw
Barrier
Game Frame – Entity Update
Fix Weak Dependency Cycles
Game Frame – Entity Update
Prepare Jobs
Game Frame – Entity Update
Link and Order Jobs
Game Frame – Entity Update
Execute Jobs
Game Frame – Entity Update
Single Threaded Callbacks
Game Frame – Entity Update
Player Job
Game Frame – Entity Update
Player Entity
Game Frame – Entity Update
Animation
Game Frame – Entity Update
Capsule Collision
Game Frame – Entity Update
Player Representation
Game Frame – Entity Update
OWL (Helper Robot)
Game Frame – Entity Update
Inventory
Game Frame – Entity Update
AI Soldier
Game Frame – Entity Update
Cloth Simulation
Game Frame – Entity Update
Cut Scene
Game Frame – Entity Update
Sub Cut Scenes
Game Frame – Entity Update
Cheap Entities (Destructibles)
Debug Tools
Debug Tools
• Dependencies get complex, we use yEd to visualize!
Debug Tools
Player
Debug Tools
Bullet System
Debug ToolsCut scenes
Conclusions
• Easy to implement in existing engine
• Game programmers can program as if single threaded
• Very few multithreading issues
Questions?
jorrit@guerrilla-games.com

More Related Content

What's hot

Rendering AAA-Quality Characters of Project A1
Rendering AAA-Quality Characters of Project A1Rendering AAA-Quality Characters of Project A1
Rendering AAA-Quality Characters of Project A1Ki Hyunwoo
 
Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Physically Based Sky, Atmosphere and Cloud Rendering in FrostbitePhysically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Physically Based Sky, Atmosphere and Cloud Rendering in FrostbiteElectronic Arts / DICE
 
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...Johan Andersson
 
A Bit More Deferred Cry Engine3
A Bit More Deferred   Cry Engine3A Bit More Deferred   Cry Engine3
A Bit More Deferred Cry Engine3guest11b095
 
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)Johan Andersson
 
Secrets of CryENGINE 3 Graphics Technology
Secrets of CryENGINE 3 Graphics TechnologySecrets of CryENGINE 3 Graphics Technology
Secrets of CryENGINE 3 Graphics TechnologyTiago Sousa
 
FrameGraph: Extensible Rendering Architecture in Frostbite
FrameGraph: Extensible Rendering Architecture in FrostbiteFrameGraph: Extensible Rendering Architecture in Frostbite
FrameGraph: Extensible Rendering Architecture in FrostbiteElectronic Arts / DICE
 
Rendering Tech of Space Marine
Rendering Tech of Space MarineRendering Tech of Space Marine
Rendering Tech of Space MarinePope Kim
 
Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019
Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019 Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019
Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019 Unity Technologies
 
BitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven rendererBitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven renderertobias_persson
 
State-Based Scripting in Uncharted 2: Among Thieves
State-Based Scripting in Uncharted 2: Among ThievesState-Based Scripting in Uncharted 2: Among Thieves
State-Based Scripting in Uncharted 2: Among ThievesNaughty Dog
 
Triangle Visibility buffer
Triangle Visibility bufferTriangle Visibility buffer
Triangle Visibility bufferWolfgang Engel
 
Lighting Shading by John Hable
Lighting Shading by John HableLighting Shading by John Hable
Lighting Shading by John HableNaughty Dog
 
How we optimized our Game - Jake & Tess' Finding Monsters Adventure
How we optimized our Game - Jake & Tess' Finding Monsters AdventureHow we optimized our Game - Jake & Tess' Finding Monsters Adventure
How we optimized our Game - Jake & Tess' Finding Monsters AdventureFelipe Lira
 
Physically Based and Unified Volumetric Rendering in Frostbite
Physically Based and Unified Volumetric Rendering in FrostbitePhysically Based and Unified Volumetric Rendering in Frostbite
Physically Based and Unified Volumetric Rendering in FrostbiteElectronic Arts / DICE
 
Multiprocessor Game Loops: Lessons from Uncharted 2: Among Thieves
Multiprocessor Game Loops: Lessons from Uncharted 2: Among ThievesMultiprocessor Game Loops: Lessons from Uncharted 2: Among Thieves
Multiprocessor Game Loops: Lessons from Uncharted 2: Among ThievesNaughty Dog
 

What's hot (20)

Rendering AAA-Quality Characters of Project A1
Rendering AAA-Quality Characters of Project A1Rendering AAA-Quality Characters of Project A1
Rendering AAA-Quality Characters of Project A1
 
Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Physically Based Sky, Atmosphere and Cloud Rendering in FrostbitePhysically Based Sky, Atmosphere and Cloud Rendering in Frostbite
Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite
 
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
 
A Bit More Deferred Cry Engine3
A Bit More Deferred   Cry Engine3A Bit More Deferred   Cry Engine3
A Bit More Deferred Cry Engine3
 
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
Parallel Graphics in Frostbite - Current & Future (Siggraph 2009)
 
DirectX 11 Rendering in Battlefield 3
DirectX 11 Rendering in Battlefield 3DirectX 11 Rendering in Battlefield 3
DirectX 11 Rendering in Battlefield 3
 
Secrets of CryENGINE 3 Graphics Technology
Secrets of CryENGINE 3 Graphics TechnologySecrets of CryENGINE 3 Graphics Technology
Secrets of CryENGINE 3 Graphics Technology
 
FrameGraph: Extensible Rendering Architecture in Frostbite
FrameGraph: Extensible Rendering Architecture in FrostbiteFrameGraph: Extensible Rendering Architecture in Frostbite
FrameGraph: Extensible Rendering Architecture in Frostbite
 
Rendering Tech of Space Marine
Rendering Tech of Space MarineRendering Tech of Space Marine
Rendering Tech of Space Marine
 
Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019
Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019 Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019
Intrinsics: Low-level engine development with Burst - Unite Copenhagen 2019
 
BitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven rendererBitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven renderer
 
Shiny PC Graphics in Battlefield 3
Shiny PC Graphics in Battlefield 3Shiny PC Graphics in Battlefield 3
Shiny PC Graphics in Battlefield 3
 
A Real-time Radiosity Architecture
A Real-time Radiosity ArchitectureA Real-time Radiosity Architecture
A Real-time Radiosity Architecture
 
State-Based Scripting in Uncharted 2: Among Thieves
State-Based Scripting in Uncharted 2: Among ThievesState-Based Scripting in Uncharted 2: Among Thieves
State-Based Scripting in Uncharted 2: Among Thieves
 
Triangle Visibility buffer
Triangle Visibility bufferTriangle Visibility buffer
Triangle Visibility buffer
 
Lighting Shading by John Hable
Lighting Shading by John HableLighting Shading by John Hable
Lighting Shading by John Hable
 
Hair in Tomb Raider
Hair in Tomb RaiderHair in Tomb Raider
Hair in Tomb Raider
 
How we optimized our Game - Jake & Tess' Finding Monsters Adventure
How we optimized our Game - Jake & Tess' Finding Monsters AdventureHow we optimized our Game - Jake & Tess' Finding Monsters Adventure
How we optimized our Game - Jake & Tess' Finding Monsters Adventure
 
Physically Based and Unified Volumetric Rendering in Frostbite
Physically Based and Unified Volumetric Rendering in FrostbitePhysically Based and Unified Volumetric Rendering in Frostbite
Physically Based and Unified Volumetric Rendering in Frostbite
 
Multiprocessor Game Loops: Lessons from Uncharted 2: Among Thieves
Multiprocessor Game Loops: Lessons from Uncharted 2: Among ThievesMultiprocessor Game Loops: Lessons from Uncharted 2: Among Thieves
Multiprocessor Game Loops: Lessons from Uncharted 2: Among Thieves
 

Similar to Killzone Shadow Fall: Threading the Entity Update on PS4

4Developers 2015: Gamedev-grade debugging - Leszek Godlewski
4Developers 2015: Gamedev-grade debugging - Leszek Godlewski4Developers 2015: Gamedev-grade debugging - Leszek Godlewski
4Developers 2015: Gamedev-grade debugging - Leszek GodlewskiPROIDEA
 
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...Terrance Cohen
 
Gdc gameplay replication in acu with videos
Gdc   gameplay replication in acu with videosGdc   gameplay replication in acu with videos
Gdc gameplay replication in acu with videosCharles Lefebvre
 
Maximize Your Production Effort (English)
Maximize Your Production Effort (English)Maximize Your Production Effort (English)
Maximize Your Production Effort (English)slantsixgames
 
Supersize Your Production Pipe
Supersize Your Production PipeSupersize Your Production Pipe
Supersize Your Production Pipeslantsixgames
 
Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022Stefano Stabellini
 
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay
GDC 2010 - A Dynamic Component Architecture for High Performance GameplayGDC 2010 - A Dynamic Component Architecture for High Performance Gameplay
GDC 2010 - A Dynamic Component Architecture for High Performance GameplayTerrance Cohen
 
Create a Scalable and Destructible World in HITMAN 2*
Create a Scalable and Destructible World in HITMAN 2*Create a Scalable and Destructible World in HITMAN 2*
Create a Scalable and Destructible World in HITMAN 2*Intel® Software
 
Sephy engine development document
Sephy engine development documentSephy engine development document
Sephy engine development documentJaejun Kim
 
Unity - Internals: memory and performance
Unity - Internals: memory and performanceUnity - Internals: memory and performance
Unity - Internals: memory and performanceCodemotion
 
Supersize your production pipe enjmin 2013 v1.1 hd
Supersize your production pipe    enjmin 2013 v1.1 hdSupersize your production pipe    enjmin 2013 v1.1 hd
Supersize your production pipe enjmin 2013 v1.1 hdslantsixgames
 
Advances in Game AI
Advances in Game AIAdvances in Game AI
Advances in Game AILuke Dicken
 
Making a game with Molehill: Zombie Tycoon
Making a game with Molehill: Zombie TycoonMaking a game with Molehill: Zombie Tycoon
Making a game with Molehill: Zombie TycoonJean-Philippe Doiron
 
How shit works: the CPU
How shit works: the CPUHow shit works: the CPU
How shit works: the CPUTomer Gabel
 
Lagom - Mircoservices "Just Right"
Lagom - Mircoservices "Just Right"Lagom - Mircoservices "Just Right"
Lagom - Mircoservices "Just Right"Markus Jura
 
Developing Next-Generation Games with Stage3D (Molehill)
Developing Next-Generation Games with Stage3D (Molehill) Developing Next-Generation Games with Stage3D (Molehill)
Developing Next-Generation Games with Stage3D (Molehill) Jean-Philippe Doiron
 
SPU Physics
SPU PhysicsSPU Physics
SPU PhysicsSlide_N
 
2017.09.07 Orleans - PWL Seattle
2017.09.07 Orleans - PWL Seattle2017.09.07 Orleans - PWL Seattle
2017.09.07 Orleans - PWL SeattleVictor Hurdugaci
 

Similar to Killzone Shadow Fall: Threading the Entity Update on PS4 (20)

4Developers 2015: Gamedev-grade debugging - Leszek Godlewski
4Developers 2015: Gamedev-grade debugging - Leszek Godlewski4Developers 2015: Gamedev-grade debugging - Leszek Godlewski
4Developers 2015: Gamedev-grade debugging - Leszek Godlewski
 
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay - M...
 
InfectNet Technical
InfectNet TechnicalInfectNet Technical
InfectNet Technical
 
Gdc gameplay replication in acu with videos
Gdc   gameplay replication in acu with videosGdc   gameplay replication in acu with videos
Gdc gameplay replication in acu with videos
 
Maximize Your Production Effort (English)
Maximize Your Production Effort (English)Maximize Your Production Effort (English)
Maximize Your Production Effort (English)
 
Supersize Your Production Pipe
Supersize Your Production PipeSupersize Your Production Pipe
Supersize Your Production Pipe
 
Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022
 
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay
GDC 2010 - A Dynamic Component Architecture for High Performance GameplayGDC 2010 - A Dynamic Component Architecture for High Performance Gameplay
GDC 2010 - A Dynamic Component Architecture for High Performance Gameplay
 
Create a Scalable and Destructible World in HITMAN 2*
Create a Scalable and Destructible World in HITMAN 2*Create a Scalable and Destructible World in HITMAN 2*
Create a Scalable and Destructible World in HITMAN 2*
 
Sephy engine development document
Sephy engine development documentSephy engine development document
Sephy engine development document
 
Unity - Internals: memory and performance
Unity - Internals: memory and performanceUnity - Internals: memory and performance
Unity - Internals: memory and performance
 
Supersize your production pipe enjmin 2013 v1.1 hd
Supersize your production pipe    enjmin 2013 v1.1 hdSupersize your production pipe    enjmin 2013 v1.1 hd
Supersize your production pipe enjmin 2013 v1.1 hd
 
Advances in Game AI
Advances in Game AIAdvances in Game AI
Advances in Game AI
 
Making a game with Molehill: Zombie Tycoon
Making a game with Molehill: Zombie TycoonMaking a game with Molehill: Zombie Tycoon
Making a game with Molehill: Zombie Tycoon
 
How shit works: the CPU
How shit works: the CPUHow shit works: the CPU
How shit works: the CPU
 
Lagom - Mircoservices "Just Right"
Lagom - Mircoservices "Just Right"Lagom - Mircoservices "Just Right"
Lagom - Mircoservices "Just Right"
 
Gamedev-grade debugging
Gamedev-grade debuggingGamedev-grade debugging
Gamedev-grade debugging
 
Developing Next-Generation Games with Stage3D (Molehill)
Developing Next-Generation Games with Stage3D (Molehill) Developing Next-Generation Games with Stage3D (Molehill)
Developing Next-Generation Games with Stage3D (Molehill)
 
SPU Physics
SPU PhysicsSPU Physics
SPU Physics
 
2017.09.07 Orleans - PWL Seattle
2017.09.07 Orleans - PWL Seattle2017.09.07 Orleans - PWL Seattle
2017.09.07 Orleans - PWL Seattle
 

Recently uploaded

Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...IJECEIAES
 
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Lovely Professional University
 
Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...IJECEIAES
 
Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1T.D. Shashikala
 
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdflitvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdfAlexander Litvinenko
 
Augmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxAugmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxMustafa Ahmed
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)NareenAsad
 
analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxKarpagam Institute of Teechnology
 
handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailingAshishSingh1301
 
Introduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoIntroduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoAbhimanyu Sangale
 
Interfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfInterfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfragupathi90
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfJNTUA
 
Lab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docxLab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docxRashidFaridChishti
 
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesLinux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesRashidFaridChishti
 
Geometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdfGeometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdfJNTUA
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AISheetal Jain
 
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...josephjonse
 
Worksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxWorksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxMustafa Ahmed
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2T.D. Shashikala
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...archanaece3
 

Recently uploaded (20)

Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...Fuzzy logic method-based stress detector with blood pressure and body tempera...
Fuzzy logic method-based stress detector with blood pressure and body tempera...
 
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
 
Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...Developing a smart system for infant incubators using the internet of things ...
Developing a smart system for infant incubators using the internet of things ...
 
Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1
 
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdflitvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
 
Augmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxAugmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptx
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)
 
analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptx
 
handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailing
 
Introduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of ArduinoIntroduction to Arduino Programming: Features of Arduino
Introduction to Arduino Programming: Features of Arduino
 
Interfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfInterfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdf
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
 
Lab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docxLab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docx
 
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesLinux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
 
Geometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdfGeometric constructions Engineering Drawing.pdf
Geometric constructions Engineering Drawing.pdf
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AI
 
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
 
Worksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxWorksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptx
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...
 

Killzone Shadow Fall: Threading the Entity Update on PS4

  • 1. Killzone Shadow Fall: Threading the Entity Update on PS4 Jorrit Rouwé Lead Game Tech, Guerrilla Games
  • 2. Introduction • Killzone Shadow Fall is a First Person Shooter • PlayStation 4 launch title • In SP up to 60 characters @ 30 FPS • In MP up to 24 players @ 60 FPS • Gameplay logic has lots of • Branches • Virtual functions • Cache misses • Not suitable for PS3 SPU’s but PS4 has 6 x86 cores
  • 3. What do we cover? • What is an Entity? • What we did on PS3 • Multi threading on PS4 • Balancing Entities across frames • Performance issues found • Performance results achieved • Debug tools
  • 4. What is an Entity?
  • 5. What is an Entity? • Base class for most game objects • E.g. Player, Enemy, Weapon, Door • Not used for static world • Has Components • E.g. Model, Mover, Destructibility • Update at a fixed frequency • 15, 30, 60 Hz • Almost everything at 15 Hz • Player updated at higher frequency to avoid lag
  • 6. What is a Representation? • Entities and Components have Representation • Controls rendering, audio and VFX • State is interpolated in frames where entity not updated • Cheaper to interpolate than to update • Introduces latency • Always blend towards last update
  • 9. PS3: 1 Entity = 1 Fiber • Most time spent on PPU • No clear concurrency model • Read partial updated state • Entities deadlock waiting for each other yield fiber resume fiber Time PPU SPU 1 SPU 2 Entity 1 Entity 2 Entity 3 Entity 1 Animation Ray Cast Entity 2
  • 10. PS4: 1 Entity = 1 Job • No fibers • Entity updates as a whole • How to solve race conditions? Time CPU 1 CPU 2 CPU 3 Entity 1 Entity 2 Entity 3 Entity 1Animation Ray Cast Entity 2  job
  • 11. • Make update order explicit: Strong Dependencies Missile MissileMissile LauncherMissile Launcher Soldier Time Soldier strong dependency
  • 12. • No (indirect) dependency = no access • Works two ways: Weapon can access Soldier too • Create dependency has 1 frame latency • Global systems need locks Non-dependent Entities can Execute Concurrently Soldier1 Weapon1 Time Soldier2 Weapon2 CPU 2 CPU 1 Soldier1 Weapon1 Soldier2 Weapon2
  • 13. • A few entities cause huge bottleneck What about this? Time Soldier1 Pistol1 Soldier2 Pistol2 Bullet System Soldier1 Pistol1 Soldier2 Pistol2 Bullet System
  • 14. Non-exclusive Dependencies • Access to ‘Bullet System’ must be lock protected Soldier1 Soldier2 Bullet System non-exclusive Time CPU 2 CPU 1 Soldier1 Pistol1 Soldier2 Pistol2 Bullet System Barrier Pistol1 Pistol2
  • 15. Weak Dependencies • 2 tanks fire at each other: • Update order reversed when circular dependency occurs • Not used very often (< 10 per frame) Time Missile2Missile1 Tank1 Tank2 Missile2Missile1 Tank2 Tank1 Missile1Missile2 or weak strong Tank1 Tank2
  • 16. Non-updating Entities • Entity can skip updates (LOD) • Entity can update in other frame • Do normal scheduling! Entity1 Entity3 Entity1 Entity2 Entity3 Time not updating
  • 17. Summarizing Dependencies Strong Exclusive Weak Exclusive Strong Non-excl. Weak Non-excl. Symbol Two way access ✓ ✓ ✓ ✓ Order guaranteed ✓ ✓ Allow concurrency + + ++ ++ Require lock ✓ ✓
  • 18. Referencing Entities • Dev build: CheckedPtr<Entity> • Acts as normal pointer • Check dependency on dereference • Retail build: Entity * • No overhead! • Doesn’t catch everything • Can use pointers to members • Bugs were easy to find
  • 19. Working with Entities without dependency • ThreadSafeEntityInterface • Mostly read only • Often used state (name, type, position, …) • Mutex per Entity • Send message (expensive) • Processed single threaded when no dependency • Schedule single threaded callback (expensive) • Everything can be accessed
  • 21. Scheduling Algorithm • Entities with exclusive dependencies merged to 1 job • Dependencies determine sorting • Non-exclusive dependencies become job dependencies • Expensive jobs kicked first! Missile Missile Launcher Tank Bullet System Pistol Soldier Missile Tank Bullet System PistolSoldier Job1 Job2 job dependency not updating weak non-exclusive strong
  • 22. Scheduling Algorithm – Edge Case Entity1 Entity3 Entity4Entity2 non-exclusive exclusive job dependency Entity1 Entity2 Entity3 Entity4 Job1 Job2
  • 23. Scheduling Algorithm – Edge Case • Non cyclic dependency becomes cyclic job dependency • Job1 and Job2 need to be merged Entity1 Entity3 Entity4Entity2 non-exclusive exclusive Entity1 Entity2Entity3 Entity4 Job1
  • 25. Balancing Entities Across Frames • Prevent all 15 Hz entities from updating in same frame • Entity can move to other frame • Smaller delta time for 1 update • Keep parent-child linked entities together • Weapon of soldier • Soldier on mounted gun • Locked close combat
  • 26. Balancing Entities – In Action Time in even frame (sum across cores, running average)
  • 27. Balancing Entities – In Action Time in odd frame (should be equal)
  • 28. Balancing Entities – In Action Civilian @ 15Hz update even frame
  • 29. Balancing Entities – In Action Civilian @ 15Hz update odd frame
  • 30. Balancing Entities – In Action Flash on switch odd / even
  • 31. Movie
  • 32. Movie
  • 34. Performance Issues • Memory allocation mutex • Eliminated many dynamic allocations • Use stack allocator • Locking physics world • R/W mutex for main simulation world • Second ‘bullet collision’ broadphase + lock • Large groups of dependent entities • Player update very expensive
  • 35. Cut Scene - Problem • Cut scene entity requires dependencies • 10+ characters in cut scene creates huge job! Civilian 1 Cut Scene Civilian 2 Player Camera
  • 36. Cut Scene - Solution • Create sub cut scenes for non-interacting entities • Master cut scene determines time and flow • Scan 1 frame ahead in timeline to create dependency Civilian 1 Cut Scenenon-exclusive Sub Cut Scene 1 Sub Cut Scene 2 Civilian 2 Sub Cut Scene 3 Player Camera
  • 37. Using an Object • Dependencies on useable objects not possible (too many) • Get list of usable objects • Global system protected by lock • ‘Use’ icon appears on screen • Player selects • Create dependency • Start ‘use’ animation • Start interaction 1 frame later (dependency valid) • Hides 1 frame delay!
  • 38. Grenade • Explosion damages many entities • Creating dependencies not option (too many) • ThreadSafeEntityInterface not an option • Need knowledge of parts • Run line of sight checks inside update • Uses scheduled callback to apply damage
  • 40. 5000 Crates Performance Results - Synthetic 100 Soldiers 500 Flags Level Counts Dependencies Max Entities in Job SpeedupNumber Entities Updating Entities Number Humans Strong Excl Strong Non-Excl 5000 Crates (20 µs each) 5019 5008 1 12 4 13 2.8X 100 Soldiers (700 µs each) 326 214 105 212 204 19 4.2X 500 Flags (160 µs each) 519 508 1 12 4 13 5.2X 6 cores!
  • 41. Performance Results - Levels Level Counts Dependencies Max Entities in Job SpeedupNumber Entities Updating Entities Number Humans Strong Excl Strong Non-Excl The Helghast (You Owe Me) 1141 206 32 71 23 20 4.1X The Patriot (On Vectan Soil) 435 257 44 199 107 15 4.3X The Remains (12p Botzone) 450 128 14 97 44 18 3.7X The Helghast The RemainsThe Patriot
  • 42. Game Frame - The Patriot
  • 43. Game Frame - Global Time 18 ms
  • 44. Game Frame - Global CPU 1 (main thread)
  • 45. Game Frame - Global CPU 2
  • 46. Game Frame - Global CPU 3
  • 47. Game Frame - Global AI Physics Entity Update VFX & Draw Barrier
  • 48. Game Frame – Entity Update Fix Weak Dependency Cycles
  • 49. Game Frame – Entity Update Prepare Jobs
  • 50. Game Frame – Entity Update Link and Order Jobs
  • 51. Game Frame – Entity Update Execute Jobs
  • 52. Game Frame – Entity Update Single Threaded Callbacks
  • 53. Game Frame – Entity Update Player Job
  • 54. Game Frame – Entity Update Player Entity
  • 55. Game Frame – Entity Update Animation
  • 56. Game Frame – Entity Update Capsule Collision
  • 57. Game Frame – Entity Update Player Representation
  • 58. Game Frame – Entity Update OWL (Helper Robot)
  • 59. Game Frame – Entity Update Inventory
  • 60. Game Frame – Entity Update AI Soldier
  • 61. Game Frame – Entity Update Cloth Simulation
  • 62. Game Frame – Entity Update Cut Scene
  • 63. Game Frame – Entity Update Sub Cut Scenes
  • 64. Game Frame – Entity Update Cheap Entities (Destructibles)
  • 66. Debug Tools • Dependencies get complex, we use yEd to visualize!
  • 70. Conclusions • Easy to implement in existing engine • Game programmers can program as if single threaded • Very few multithreading issues