Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Photogrammetry and Star Wars Battlefront


Published on

Presentation by Andrew Hamilton and Ken Brown from DICE at GDC 2016.

Photogrammetry has started to gain steam within the Games Industry in recent years. At DICE, this technique was first used on Battlefield and they fully embraced the technology and workflow for Star Wars: Battlefront. This talk will cover their research and development, planning and production, techniques, key takeaways and plans for the future. The speakers will cover photogrammetry as a technology, but more than that, show that it's not a magic bullet but instead a tool like any other that can be used to help achieve your artistic vision and craft.

Come and learn how (and why) photogrammetry was used to create the world of Star Wars. This talk will cover Battlefront's use of of the technology from pre-production to launch as well as some of their philosophies around photogrammetry as a tool. Many visuals will be included!

Intended Audience
A content creator friendly talk intended for pretty much any developer, especially those involved in 3D content creation. It is not a technical talk focused on the code or engineering of photogrammetry. The speakers will quickly cover all basics, so absolutely no prerequisite knowledge required.

Published in: Entertainment & Humor

Photogrammetry and Star Wars Battlefront

  1. 1. Photogrammetry and Star Wars Battlefront Andrew Hamilton Lead Environment Artist Kenneth Brown Technical Art Director EA DICE
  2. 2. Buckle Up. Photogrammetry
  3. 3. Using photos for measurements and, in our case, modelling/texturing. Photogrammetry
  4. 4. Photogrammetry
  5. 5. Photogrammetry
  6. 6. Photogrammetry
  7. 7. Photogrammetry
  8. 8. Photogrammetry
  9. 9. The End.
  10. 10. A Retrospective • The beginning: research and development • Scanning plans and goals • Capturing the planets • Mentality and workflows • Game assets in a “photogrammetry world” • Terrain • Level Construction Kits • Key learnings and takeaways
  11. 11. Big Changes • New console generation • New franchise • Very high expectations to deliver after BF4 • A clean slate; we want to do it right! • PBR on the way but not ready (in the beginning)
  12. 12. • Star Wars! • Learning from the past • Photogrammetry pushing the boundaries • A really tight schedule • If we’re saying 60, we need to be 60 • Keeping it together Challenges
  13. 13. Dev Structure • 3 month development cycles • Content development until the end • Need for 2 week buffer discovered • Approval lock • Stabilization
  14. 14. The beginning
  15. 15. Into Star Wars: Battlefront
  16. 16. Scan everything!..? • Bridge the gap between scanned characters and their surroundings. • Make the best looking Star Wars game ever! 60fps  • We have so little time…
  17. 17. From here...
  18. 18. here!
  19. 19. • Capture the complete asset libraries of all planets. • Focus and capture only what is essential, not everything we see. • Capture meaningful variation and features. • Visit the real film locations for true inspiration and understanding. Location capture goals
  20. 20. • Simple kit (24mm lens, Canon 6D, tripod, color chart, etc) • Ideal weather conditions • On-location breakdown • Thinking in terms of usable pieces • Aware of Frostbite strengths/weaknesses • Focus on quality over quantity of captures Capturing on location
  21. 21. A. Dirt ground B. Leaves and twigs C. Clover ground D. Clover leaves E. Fern large F. Fern medium G. Fern small H. Log large I. Log medium J. Log small K. Tree large L. Tree medium M. Tree small N. Tree stump O. Vegetation AC D B E F G H I J K L M N O
  22. 22. • Overlap!! • Covering all angles to reduce cleanup. • Avoid cast shadow - ambient light is easily removed later in the process. • Most reliable results occurred when full asset was visible in most photos. • Avoid objects with too intricate details. • Be sure that all capture methods are consistent and clean. • Prepare yourself and equipment for various conditions – weather can make the process sloppy and destroy consistent results. Best practices
  23. 23. Capturing the planets
  24. 24. Successful location scanning • A small and manageable group with light equipment is ideal. • Structured and focused on location. • Absolutely key to connect and understand how to recreate real locations.
  25. 25. Managing the data • Over 100,000 photos to manually sort • Over 2000 assets to manually review • Over 14 hours of video to categorize • Less than ideal storage solution • A lot of room for improvement here!
  26. 26. Workflow
  27. 27. Workflow Now we have the data, so let’s go.
  28. 28. High Poly High Poly Re-Topo Re-Topo UV Layout UV Layout Texture Texture Setup Setup 0 2 4 6 8 10 12 14 16 BF4 SW:BF Days Per Asset High Poly Re-Topo UV Layout Texture Setup How long does an asset take? Is Photogrammetry a savings? (Straw Poll)
  29. 29. How long does an asset take? With Automation High Poly High Poly Re-Topo Re-Topo UV Layout UV Layout Texture Setup 0 2 4 6 8 10 12 14 16 BF4 SW:BF Days Per Asset High Poly Re-Topo UV Layout Texture Setup
  30. 30. Photogrammetry workflow • Remove vignette, keep lens distortion. • Batch calibration to color chart. • This entire step should be automated! PS Camera RAW
  31. 31. Photogrammetry workflow • Most assets need to be masked – very time consuming if not automatic. • Baking at High gave most useable and manageable results for Objects. • Baking at Ultra gave best results for Terrain – files are huge though. Agisoft PhotoScan
  32. 32. Photogrammetry workflow • Mosaic rather than Average blend gives much sharper results. • Shredded UVs – take to Zbrush and reproject, or bake high res. Agisoft PhotoScan
  33. 33. Photogrammetry workflow • Some assets completely automated in MeshLab. • Some needed fine tuning in Topogun – ok if very limited asset library. Topogun / MeshLab
  34. 34. Photogrammetry workflow • Detail textures applied later means UVs demand very little stretching. • UV direction important for consistent detail texture alignment, sometimes may need a second UV just for detail tiling. Maya / UVLayout
  35. 35. Photogrammetry workflow • Color, Tangent Normals, Object Normals, Heightmap, AO, Cavity. XNormal
  36. 36. Photogrammetry workflow • Object Space Normals used as a mask to remove ambient lighting. • AO or cavity map for fine tune lighting removal. XNormal
  37. 37. • 16 bit depth is your friend • Remove as much as possible from camera raw • Once removed, move on to ambient • Object space normal maps ftw Photogrammetry workflow Light Removal
  38. 38. Light Removal Object Normal • In this example, shadows are not so distinct • Generate an object normal map • This gives distinct colors for the different object normals
  39. 39. Light Removal Object Normal • Using Black & White mask in Photoshop you can pull out different “directions”
  40. 40. Workflow Now we have the base “source” data…
  41. 41. Texel Density • Graphical representation of a pixel • An initial pitfall for us • The following screenshots are 1080p • Illustrates texture pixel (texel) to screen pixel ratio
  42. 42. ~10 texture pixels per screen pixel (excluding mips)
  43. 43. For Comparison: Two 4k Textures. 55mb
  44. 44. For Comparison: Two 1k Textures. 3.7mb
  45. 45. Texel Density • You’d be surprised what you can get away with • Detail maps are our friends • Large textures have a very big streaming overhead • We have an 800mb texture pool on Gen4 • One 8k texture is almost 90mb Takeaways
  46. 46. Texture Arrays • The photoscanned asset as a base is generally quite low res. • All environmental assets use a consistent preset shader. • Planet-specific Texture Array instance shaders add the detail on top. Object Preset Shader
  47. 47. Texture Arrays • This texture array instance shader is set up for normal, reflectance, and color. Texture Array Instance Shader
  48. 48. Texture Arrays • Brightness is painted in 5 even steps (as there are 5 detail textures). • The order of these layers must correspond to the order in the Array. Tree Normals (Texture Array mask in Alpha) Brightness % 0 25 50 75 100 Detail Leaves Bark Burnt Bark Rock Moss
  49. 49. Texture Arrays • Detail textures are fading away towards the end of LOD0 with ‘LodFade’. • LOD1 has a new simplified shader without detail textures. Texture Array Instance Shader
  50. 50. Texture Arrays • Like most texture assets in Battlefront, we heavily pack into channels. • The detail texture uses RG for normals, B for Reflectance, A for Color. Detail Texture within the Texture Array
  51. 51. Texture Arrays • Unique PhotoScan base is surface blurred, while keeping important details. • Drastically reduces texture memory.
  52. 52. Texture Arrays • Texture Array tiling detail on top fill in the details without overpowering the base. • Drastically increases texture resolution.
  53. 53. How do we ensure all assets fit together consistently? “Old” Asset Creation QUALITY TECHEFFORT Photogrammetry TECH+TECHNIQUES QUALITY EFFORT “New” Asset Creation QUALITY TECHEFFORT NEEDED EFFORTTECH+TECHNIQUES Photoscan vs. Everything Else
  54. 54. • Helps us bridge that gap • Lessons from Criterion, Guerilla Games, Dontnod etc • Trying to get everyone to switch together can backfire. • Some people will adapt instantly. Let them lead! • For PBR, there isn’t really a middle road. PBR Quick Thoughts
  55. 55. PBR • Assets generally fit well but still needed to be balanced against the scene. • Finding and trusting true PBR values at times was a challenge • Constant reinforcement helps prevent inconsistencies
  56. 56. BaseColor
  57. 57. Normals
  58. 58. Reflectance
  59. 59. Smoothness
  60. 60. SubSurfaceTranslucency
  61. 61. Shadow
  62. 62. Ambient Occlusion
  63. 63. Light + Indirect
  64. 64. Vegetation • Reaching 60fps – lets make a dense forest! • Integrating hand-built vegetation into a PhotoScanned world was tough to reach comparable quality! • Difficult to get good results from scanned vegetation – complex and noisy, resulting in horrible scans. • Based on photos of vegetation from location captured on blue board.
  65. 65. Vegetation • All vegetation share the same textures • There are textures created separately for close up, mid, and far distances. • There is a lot of texture packing within vegetation assets to reduce complexity of shaders. • Normals created by broadly modelling around leaves and branches Maya.
  66. 66. Vegetation • Unique PhotoScanned base blends to tiling texture that extends upwards. • Tree base is generally evenly tessellated for best displacement results. Tiling trunk texture Unique base texture
  67. 67. Terrain/asset blending • A lot of work put in to reduce the visible disconnect between assets and terrain. • Most assets have terrain radiosity projected on to them. • Texture Arrays world projected and details match. • Terrain color and normals applied to most assets.
  68. 68. Terrain/asset blending • A lot of work put in to reduce the visible disconnect between assets and terrain. • Most assets have terrain radiosity projected on to them. • Texture Arrays world projected and details match. • Terrain color and normals applied to most assets.
  69. 69. Mesh Displacement • Great quality boost for close up detail. • Object displacement stopped at ~7 meters. • An ongoing balance between quality and performance.
  70. 70. Mesh Displacement OFF
  71. 71. Mesh Displacement ON
  72. 72. • Ability to define displacement shape (via mask) • One of our “Must Have” features • Critical for ATAT footprints • Terrain Decal Component • Combined with detail displacement shader • (More on this a little later) Tessellation Heightfield Displacement
  73. 73. • Quality vs. Speed ratio is extremely high • Feel free to get creative Tessellation and Photogrammetry Heightfield Displacement
  74. 74. Speaking of terrain. Terrain
  75. 75. Terrain Final big chunk of our “photogrammetry” puzzle.
  76. 76. Terrain • 8x8km with a 1x1km playable space. • Huge background terrain meshes. • Terrain textures captured from real world. • Layers limited to control draw calls.
  77. 77. Moving (mostly) away from WorldMachine..
  78. 78. real-world topography data.
  79. 79. Processing the data
  80. 80. Extending the horizon 100x80km
  81. 81. Texture Capture
  82. 82. Best practices - Terrain • We aimed for 3x3 meter patches • 5x5 photo grid ensures enough overlap • Capture only 1 defined element at a time • Low ISO and high F-Stop for sharp image • Avoid noisy surfaces such as grass
  83. 83. Terrain Tessellation • Great quality boost for close up detail on terrain • Major upgrade for us. Can never go back! • Terrain displacement stopped at ~15 meters.
  84. 84. Terrain Tessellation OFF
  85. 85. Terrain Tessellation ON
  86. 86. Level Construction Kits • Content and documentation packages. • Provides fast level iteration. • Shifts focus to building levels rather than content during Production. • Allows us to more quickly understand performance and keep track of it.
  87. 87. Level Construction Kits • 8x8km highly detailed terrain. • Ready to paint terrain materials. • Easily accessible limited library of content. • Lighting presets based on the films.
  88. 88. HothEndor Tatooine Sullust
  89. 89. Endor
  90. 90. Endor
  91. 91. Hoth
  92. 92. Hoth
  93. 93. Tatooine
  94. 94. Tatooine
  95. 95. Sullust
  96. 96. Sullust
  97. 97. Key learnings and takeaways • Solid processes and workflows in place before Production was a big win. • Well planned and structured Photogrammetry trips results in clean and manageable assets. • Photogrammetry is not a silver bullet – it was only one part of achieving the final results in game. • It was crucial to have artists own and be responsible for their locations/planets - Consistency is key when working with scanned assets. • Working towards PBR was an ongoing process to success. • Much of our creative process was incredibly repetitive, much of which can and should be automated. • There was way too much manual labor, with levels being built by brute force without the help of workflow-improving tools.
  98. 98. Special Thanks John Troive - Pontus Ryman - Petter Sköld - Tim McLeod - Simon Barsky - Darko Pracic - Daniel Cambrand - Oscar Carlen - Madina Chionidi - Björn Arvidsson - Robert Kihl - … and of course, the entire Battlefront team at DICE! Senior Environment Artist Senior Environment Artist Senior Environment Artist Environment Artist Environment Artist Environment Artist Environment Artist Senior Lighting Artist Character Artist Character Artist Rendering Engineer
  99. 99. Thanks! Andrew Hamilton – Lead Environment Artist Kenneth Brown – Technical Art Director