The document discusses the integration of Perforce into a AAA game engine, focusing on its C++ API to manage file systems and support game production workflows. It highlights the challenges faced with previous systems and outlines the goals of performance, stability, scalability, and user experience in the new engine. The benefits include a streamlined user interface and integrated operations, while costs involve high maintenance and complex testing.

1. 1
Perforce integration in a AAA game engine
The Strength of the C++ API
Marc Desfossés
Lead Tools Programmer
Jean-Sébastien Pelletier
Technical Lead
Perforce integration in a AAA game engine
Ubisoft Entertainment

2. 2
Introduction
The organized chaos of the game industry

3. 3
History – Prince of Persia
• How it worked on Prince of Persia (2003)
• In-house source control
• What we loved
• Bigfile: File system stored in a single file
• Integrated in editor
• What did not work
• Data corruption
• Not scalable : No submit queue

4. 4
History – Starting from scratch
Foundations of a new engine (2004)
• Full Perforce Integration
• Keep Bigfile’s concepts
• AAA game title
• Big teams
• Lots of assets

5. 5
Game Production Workflow
Creating a AAA game

6. 6
Engine overview
Guildlib
(Data
Exchange)
Scimitar
(Engine)
Anvil
(Editor)
3DStudio
Max
MoAon
Builder
P4
Photoshop
Bigfile

7. 7
Objectives
• Performance
• Stability
• Get rid of data corruption
• Scalability
• User Experience

8. 8
Objectives - Scalability
• Over 2 millions of files on AC3
• ~ 500 users submitting code or data
• 150 Gigs of data
• Multi-Studio support (5 studios)

9. 9
Objectives - Scalability

11. 11
Objectives – User experience
• Simple User Interface
• Exposing only the basics features
• Add, Edit, Delete
• Limited branching and resolve for power users

12. 12
User Experience - Custom dialogs
Anvil
Editor

13. 13
User Experience - Custom dialogs
Visual
Diff
Submit
Assistant

14. 14
Architecture
Leveraging the C++ API

15. 15
Architecture – The Bigfile
• File system embedded in ONE file similar to a zip file.
• Database driven
• Transactions
• It contains relationships between all game objects
• Source control metadata for each file stored in FAT
• Revision number
• File status

16. 16
Architecture – Clientview Sync Process
- Mirror bigfile and its associated perforce client
- Run commands such as fstat, flush, edit, revert, delete,
add.
- Once this is completed we can run any Perforce
command.

17. 17
Architecture – Splitting game object files
Name
=
texture.psd
Object
Type
=
Texture
Folder
=
foo
Meta
Data
Data
/foo/texture.psd
BIGFILE
Perforce

18. 18
Architecture – Splitting game object files
• Avoid branching when moving or renaming
• Move or Rename becomes edit of the meda-data file
• Save disk space on server
• No data file submit for rename or move
• Allow custom file statuses
• Only the meta-data file is synced

19. 19
Architecture – File Mapping
//ac3/main/d/3/00/01/300012ce1.d
//ac3/main/a/3/00/01/300012ce1.a
/foo/texture.psd
File
texture.psd
with
ID
0x300012ce1
Perforce
Depot
BIGFILE

20. 20
• FileSys class
• Used to redirect I/O
• Lets us improve performance
• Callback call for each file when running
commands
Architecture – The C++ API

21. 21
• P4 sync //ac3/main/…
• Callbacks in FileSys class for each file synced
• Open()
• Stat() – Return false (no temp file)
• Write() – Append buffer in memory
• Close() – Commit buffer and update metadata in
bigfile
Architecture – Sync Example

22. 22
Architecture – Sync Performance

23. 23
Architecture – Custom File Statuses
• Ghosted files

24. 24
Architecture – Custom File Statuses
• Hijacked files
• Clobbered files that can be reverted
• Skipped during sync
• Contained in a special changelist

25. 25
Demo

26. 26
Benefits vs Costs
Is it really worth it ?

27. 27
Benefits
• Integrated User Experience
• Embedded Perforce operations
• Submit Assistant dialog
• 5 minutes setup
• Custom Statuses
• Ghosted files
• Hijacked files
• Performance

28. 28
Costs
• High maintenance
• Always behind Perforce’s new features
• Shelve and Streams for instance
• Testing is complex
• Upgrades need to be fully tested

29. 29
Final words

30. 30
Final words
• The Perforce C++ API is powerful and lets you
manage your own File System.
• Perforce can support the load required by the
gaming industry.
• A tight integration allowed us to simplify our
users workflows.

31. 31
Questions ?