Successfully reported this slideshow.

Photogrammetry and Star Wars Battlefront

108

Share

1 of 164
1 of 164

Photogrammetry and Star Wars Battlefront

108

Share

Download to read offline

Description

Presentation by Andrew Hamilton and Ken Brown from DICE at GDC 2016.

Photogrammetry has started to gain steam within the Games Industry in recent years. At DICE, this technique was first used on Battlefield and they fully embraced the technology and workflow for Star Wars: Battlefront. This talk will cover their research and development, planning and production, techniques, key takeaways and plans for the future. The speakers will cover photogrammetry as a technology, but more than that, show that it's not a magic bullet but instead a tool like any other that can be used to help achieve your artistic vision and craft.

Takeaway
Come and learn how (and why) photogrammetry was used to create the world of Star Wars. This talk will cover Battlefront's use of of the technology from pre-production to launch as well as some of their philosophies around photogrammetry as a tool. Many visuals will be included!

Intended Audience
A content creator friendly talk intended for pretty much any developer, especially those involved in 3D content creation. It is not a technical talk focused on the code or engineering of photogrammetry. The speakers will quickly cover all basics, so absolutely no prerequisite knowledge required.

Transcript

  1. 1. Photogrammetry and Star Wars Battlefront Andrew Hamilton Lead Environment Artist Kenneth Brown Technical Art Director EA DICE
  2. 2. Buckle Up. Photogrammetry
  3. 3. Using photos for measurements and, in our case, modelling/texturing. Photogrammetry
  4. 4. Photogrammetry
  5. 5. Photogrammetry
  6. 6. Photogrammetry
  7. 7. Photogrammetry
  8. 8. Photogrammetry
  9. 9. The End.
  10. 10. A Retrospective • The beginning: research and development • Scanning plans and goals • Capturing the planets • Mentality and workflows • Game assets in a “photogrammetry world” • Terrain • Level Construction Kits • Key learnings and takeaways
  11. 11. Big Changes • New console generation • New franchise • Very high expectations to deliver after BF4 • A clean slate; we want to do it right! • PBR on the way but not ready (in the beginning)
  12. 12. • Star Wars! • Learning from the past • Photogrammetry pushing the boundaries • A really tight schedule • If we’re saying 60, we need to be 60 • Keeping it together Challenges
  13. 13. Dev Structure • 3 month development cycles • Content development until the end • Need for 2 week buffer discovered • Approval lock • Stabilization
  14. 14. The beginning
  15. 15. Into Star Wars: Battlefront
  16. 16. Scan everything!..? • Bridge the gap between scanned characters and their surroundings. • Make the best looking Star Wars game ever! ..at 60fps  • We have so little time…
  17. 17. From here...
  18. 18. ..to here!
  19. 19. • Capture the complete asset libraries of all planets. • Focus and capture only what is essential, not everything we see. • Capture meaningful variation and features. • Visit the real film locations for true inspiration and understanding. Location capture goals
  20. 20. • Simple kit (24mm lens, Canon 6D, tripod, color chart, etc) • Ideal weather conditions • On-location breakdown • Thinking in terms of usable pieces • Aware of Frostbite strengths/weaknesses • Focus on quality over quantity of captures Capturing on location
  21. 21. A. Dirt ground B. Leaves and twigs C. Clover ground D. Clover leaves E. Fern large F. Fern medium G. Fern small H. Log large I. Log medium J. Log small K. Tree large L. Tree medium M. Tree small N. Tree stump O. Vegetation AC D B E F G H I J K L M N O
  22. 22. • Overlap!! • Covering all angles to reduce cleanup. • Avoid cast shadow - ambient light is easily removed later in the process. • Most reliable results occurred when full asset was visible in most photos. • Avoid objects with too intricate details. • Be sure that all capture methods are consistent and clean. • Prepare yourself and equipment for various conditions – weather can make the process sloppy and destroy consistent results. Best practices
  23. 23. Capturing the planets
  24. 24. Successful location scanning • A small and manageable group with light equipment is ideal. • Structured and focused on location. • Absolutely key to connect and understand how to recreate real locations.
  25. 25. Managing the data • Over 100,000 photos to manually sort • Over 2000 assets to manually review • Over 14 hours of video to categorize • Less than ideal storage solution • A lot of room for improvement here!
  26. 26. Workflow
  27. 27. Workflow Now we have the data, so let’s go.
  28. 28. High Poly High Poly Re-Topo Re-Topo UV Layout UV Layout Texture Texture Setup Setup 0 2 4 6 8 10 12 14 16 BF4 SW:BF Days Per Asset High Poly Re-Topo UV Layout Texture Setup How long does an asset take? Is Photogrammetry a savings? (Straw Poll)
  29. 29. How long does an asset take? With Automation High Poly High Poly Re-Topo Re-Topo UV Layout UV Layout Texture Setup 0 2 4 6 8 10 12 14 16 BF4 SW:BF Days Per Asset High Poly Re-Topo UV Layout Texture Setup
  30. 30. Photogrammetry workflow • Remove vignette, keep lens distortion. • Batch calibration to color chart. • This entire step should be automated! PS Camera RAW
  31. 31. Photogrammetry workflow • Most assets need to be masked – very time consuming if not automatic. • Baking at High gave most useable and manageable results for Objects. • Baking at Ultra gave best results for Terrain – files are huge though. Agisoft PhotoScan
  32. 32. Photogrammetry workflow • Mosaic rather than Average blend gives much sharper results. • Shredded UVs – take to Zbrush and reproject, or bake high res. Agisoft PhotoScan
  33. 33. Photogrammetry workflow • Some assets completely automated in MeshLab. • Some needed fine tuning in Topogun – ok if very limited asset library. Topogun / MeshLab
  34. 34. Photogrammetry workflow • Detail textures applied later means UVs demand very little stretching. • UV direction important for consistent detail texture alignment, sometimes may need a second UV just for detail tiling. Maya / UVLayout
  35. 35. Photogrammetry workflow • Color, Tangent Normals, Object Normals, Heightmap, AO, Cavity. XNormal
  36. 36. Photogrammetry workflow • Object Space Normals used as a mask to remove ambient lighting. • AO or cavity map for fine tune lighting removal. XNormal
  37. 37. • 16 bit depth is your friend • Remove as much as possible from camera raw • Once removed, move on to ambient • Object space normal maps ftw Photogrammetry workflow Light Removal
  38. 38. Light Removal Object Normal • In this example, shadows are not so distinct • Generate an object normal map • This gives distinct colors for the different object normals
  39. 39. Light Removal Object Normal • Using Black & White mask in Photoshop you can pull out different “directions”
  40. 40. Workflow Now we have the base “source” data…
  41. 41. Texel Density • Graphical representation of a pixel • An initial pitfall for us • The following screenshots are 1080p • Illustrates texture pixel (texel) to screen pixel ratio
  42. 42. ~10 texture pixels per screen pixel (excluding mips)
  43. 43. For Comparison: Two 4k Textures. 55mb
  44. 44. For Comparison: Two 1k Textures. 3.7mb
  45. 45. Texel Density • You’d be surprised what you can get away with • Detail maps are our friends • Large textures have a very big streaming overhead • We have an 800mb texture pool on Gen4 • One 8k texture is almost 90mb Takeaways
  46. 46. Texture Arrays • The photoscanned asset as a base is generally quite low res. • All environmental assets use a consistent preset shader. • Planet-specific Texture Array instance shaders add the detail on top. Object Preset Shader
  47. 47. Texture Arrays • This texture array instance shader is set up for normal, reflectance, and color. Texture Array Instance Shader
  48. 48. Texture Arrays • Brightness is painted in 5 even steps (as there are 5 detail textures). • The order of these layers must correspond to the order in the Array. Tree Normals (Texture Array mask in Alpha) Brightness % 0 25 50 75 100 Detail Leaves Bark Burnt Bark Rock Moss
  49. 49. Texture Arrays • Detail textures are fading away towards the end of LOD0 with ‘LodFade’. • LOD1 has a new simplified shader without detail textures. Texture Array Instance Shader
  50. 50. Texture Arrays • Like most texture assets in Battlefront, we heavily pack into channels. • The detail texture uses RG for normals, B for Reflectance, A for Color. Detail Texture within the Texture Array
  51. 51. Texture Arrays • Unique PhotoScan base is surface blurred, while keeping important details. • Drastically reduces texture memory.
  52. 52. Texture Arrays • Texture Array tiling detail on top fill in the details without overpowering the base. • Drastically increases texture resolution.
  53. 53. How do we ensure all assets fit together consistently? “Old” Asset Creation QUALITY TECHEFFORT Photogrammetry TECH+TECHNIQUES QUALITY EFFORT “New” Asset Creation QUALITY TECHEFFORT NEEDED EFFORTTECH+TECHNIQUES Photoscan vs. Everything Else
  54. 54. • Helps us bridge that gap • Lessons from Criterion, Guerilla Games, Dontnod etc • Trying to get everyone to switch together can backfire. • Some people will adapt instantly. Let them lead! • For PBR, there isn’t really a middle road. PBR Quick Thoughts
  55. 55. PBR • Assets generally fit well but still needed to be balanced against the scene. • Finding and trusting true PBR values at times was a challenge • Constant reinforcement helps prevent inconsistencies
  56. 56. BaseColor
  57. 57. Normals
  58. 58. Reflectance
  59. 59. Smoothness
  60. 60. SubSurfaceTranslucency
  61. 61. Shadow
  62. 62. Ambient Occlusion
  63. 63. Light + Indirect
  64. 64. Vegetation • Reaching 60fps – lets make a dense forest! • Integrating hand-built vegetation into a PhotoScanned world was tough to reach comparable quality! • Difficult to get good results from scanned vegetation – complex and noisy, resulting in horrible scans. • Based on photos of vegetation from location captured on blue board.
  65. 65. Vegetation • All vegetation share the same textures • There are textures created separately for close up, mid, and far distances. • There is a lot of texture packing within vegetation assets to reduce complexity of shaders. • Normals created by broadly modelling around leaves and branches Maya.
  66. 66. Vegetation • Unique PhotoScanned base blends to tiling texture that extends upwards. • Tree base is generally evenly tessellated for best displacement results. Tiling trunk texture Unique base texture
  67. 67. Terrain/asset blending • A lot of work put in to reduce the visible disconnect between assets and terrain. • Most assets have terrain radiosity projected on to them. • Texture Arrays world projected and details match. • Terrain color and normals applied to most assets.
  68. 68. Terrain/asset blending • A lot of work put in to reduce the visible disconnect between assets and terrain. • Most assets have terrain radiosity projected on to them. • Texture Arrays world projected and details match. • Terrain color and normals applied to most assets.
  69. 69. Mesh Displacement • Great quality boost for close up detail. • Object displacement stopped at ~7 meters. • An ongoing balance between quality and performance.
  70. 70. Mesh Displacement OFF
  71. 71. Mesh Displacement ON
  72. 72. • Ability to define displacement shape (via mask) • One of our “Must Have” features • Critical for ATAT footprints • Terrain Decal Component • Combined with detail displacement shader • (More on this a little later) Tessellation Heightfield Displacement
  73. 73. • Quality vs. Speed ratio is extremely high • Feel free to get creative Tessellation and Photogrammetry Heightfield Displacement
  74. 74. Speaking of terrain. Terrain
  75. 75. Terrain Final big chunk of our “photogrammetry” puzzle.
  76. 76. Terrain • 8x8km with a 1x1km playable space. • Huge background terrain meshes. • Terrain textures captured from real world. • Layers limited to control draw calls.
  77. 77. Moving (mostly) away from WorldMachine..
  78. 78. ..to real-world topography data.
  79. 79. Processing the data
  80. 80. Extending the horizon 100x80km
  81. 81. Texture Capture
  82. 82. Best practices - Terrain • We aimed for 3x3 meter patches • 5x5 photo grid ensures enough overlap • Capture only 1 defined element at a time • Low ISO and high F-Stop for sharp image • Avoid noisy surfaces such as grass
  83. 83. Terrain Tessellation • Great quality boost for close up detail on terrain • Major upgrade for us. Can never go back! • Terrain displacement stopped at ~15 meters.
  84. 84. Terrain Tessellation OFF
  85. 85. Terrain Tessellation ON
  86. 86. Level Construction Kits • Content and documentation packages. • Provides fast level iteration. • Shifts focus to building levels rather than content during Production. • Allows us to more quickly understand performance and keep track of it.
  87. 87. Level Construction Kits • 8x8km highly detailed terrain. • Ready to paint terrain materials. • Easily accessible limited library of content. • Lighting presets based on the films.
  88. 88. HothEndor Tatooine Sullust
  89. 89. Endor
  90. 90. Endor
  91. 91. Hoth
  92. 92. Hoth
  93. 93. Tatooine
  94. 94. Tatooine
  95. 95. Sullust
  96. 96. Sullust
  97. 97. Key learnings and takeaways • Solid processes and workflows in place before Production was a big win. • Well planned and structured Photogrammetry trips results in clean and manageable assets. • Photogrammetry is not a silver bullet – it was only one part of achieving the final results in game. • It was crucial to have artists own and be responsible for their locations/planets - Consistency is key when working with scanned assets. • Working towards PBR was an ongoing process to success. • Much of our creative process was incredibly repetitive, much of which can and should be automated. • There was way too much manual labor, with levels being built by brute force without the help of workflow-improving tools.
  98. 98. Special Thanks John Troive - Pontus Ryman - Petter Sköld - Tim McLeod - Simon Barsky - Darko Pracic - Daniel Cambrand - Oscar Carlen - Madina Chionidi - Björn Arvidsson - Robert Kihl - … and of course, the entire Battlefront team at DICE! Senior Environment Artist Senior Environment Artist Senior Environment Artist Environment Artist Environment Artist Environment Artist Environment Artist Senior Lighting Artist Character Artist Character Artist Rendering Engineer
  99. 99. Thanks! Andrew Hamilton – Lead Environment Artist Andrew.Hamilton@dice.se Kenneth Brown – Technical Art Director Kenneth.Brown@dice.se

Editor's Notes

  • Photogrammetry -  the science of making measurements from photographs, especially for recovering the exact positions of surface points. Moreover, it may be used to recover the motion pathways of designated reference points located on any moving object, on its components and in the immediately adjacent environment. Photogrammetry may employ high-speed imaging and remote sensing in order to detect, measure and record complex 2-D and 3-D motion fields (see also sonar, radar, lidar, etc.). Photogrammetry feeds the measurements from remote sensing and the results of imagery analysis into computational models in an attempt to successively estimate, with increasing accuracy, the actual, 3-D relative motions within the researched field. (source: Wikipedia)
  • Photogrammetry -  the science of making measurements from photographs, especially for recovering the exact positions of surface points. Moreover, it may be used to recover the motion pathways of designated reference points located on any moving object, on its components and in the immediately adjacent environment. Photogrammetry may employ high-speed imaging and remote sensing in order to detect, measure and record complex 2-D and 3-D motion fields (see also sonar, radar, lidar, etc.). Photogrammetry feeds the measurements from remote sensing and the results of imagery analysis into computational models in an attempt to successively estimate, with increasing accuracy, the actual, 3-D relative motions within the researched field. (source: Wikipedia)
  • A detailed breakdown of our step-by-step workflow can be found here.
  • Align photos.
  • Generate point cloud.
  • Generate highpoly mesh.
  • Generate/project textures.
  • Opportunity to change the way we think and create content
    Available manpower (artists transitioning from BF4)



  • Photogrammetry can make great high res, largely organic assets
    We still need man (alien?) made asset
    We need to significantly increase our efficiency
    Let’s look at were we are and re-evaluate everything we know. Nothing is off limits.
    Today we’ll discuss some of the software we settled on
  • The beginnings of photogrammetry with Battlefield 4
  • An example of the range of captured assets from the Lucas archives.
  • An example of the range of captured assets from the Lucas archives.
  • Supporting scanned assets with other tools such as Marvellous Designer
  • Hard-Surface materials created with traditional methods
  • An example of the level of detail we’re able to get with combining photoscan with new (to us) software.
  • An example of the level of detail we’re able to get with combining photoscan with new (to us) software.
  • An example of the ingame assets using the combination of techniques.
  • A screenshot from Battlefield: Armored Kill that was considered ‘realistic’ during its time…
  • …to a screenshot from Star Wars: Battlefront, using photogrammetry on a very limited number of selective content to build up a realistic environment
  • A slideshow of locations we have visited on scanning trips, for a quick glimpse into the range and variety of settings at our disposal. Iceland
  • Iceland
  • Death Valley, USA
  • Utah, USA
  • Death Valley, USA
  • National Redwood Forests , USA
  • National Redwood Forests , USA
  • Finse, Norway
  • Finse, Norway
  • Iceland
  • Iceland
  • Iceland
  • A small selection from our large library of scanned materials that would be used for Sullust terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Tatooine terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Endor terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Hoth terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Hoth terrain surfaces
  • Vegetation and ground scattering assets captured on a blue board for easy background extraction
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • Opportunity to change the way we think and create content
    Available manpower (artists transitioning from BF4)



  • Opportunity to change the way we think and create content
    Available manpower (artists transitioning from BF4)



  • Data taken from a cross sampling of content creator disciplines. Straw poll – not necessarily accurate 1:1, but representative of reality.
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • Shadows
     
    Need to mask out shadows
    Super important to have 16 bit depth
    Before that, remove as much as possible from the camera raw (shadow slider photoshop camera raw plugin)
    Create mask from what’s dark.
    Flatten luminance in the colors first
    Shadows tend to be colder, so you can use color to grab them.
     
    Once you’ve gotten the shadows removed, you can treat it like something that’s been captured overcast
     
    Get rid of ambient light
    Two ways
    Use the normal map (baked out in object space) to get the angle.
    Use the occlusion, baked from the geometry.
    Also need to use the mask to remove the color (bounced light)
    Overcast is best
  • Let’s look at other aspects of the asset creation
  • Texel refers to the graphical representation of a pixel on a 2d object (eg. a texture). Basically, when you look at a pixel of a texture rendered on a 3d model, you are looking at a“texel”.
  • In this shot ~25x’s more pixels available than the screen can render.
  • Mips always try to keep a 1:1 ratio. So detail will get mipped away anyways!
  • Not only is this extremely close for 1p, it’s an even more extreme distance for 3p. At this resolution, there is certainly an absolute difference in quality, but a trivial relative difference in quality.
  • To give you a sense of the memory impact (and subsequent streaming impact) that this has.
  • The quality of these assets are great. This just reinforces what we’ve been saying – focus on the craftsmanship. More resolution isn’t what’s going to make these assets look better – the overall quality of the asset is what’s going to sell it.
  • In this example, the multiply node is set to a factor of 4, give a range from 0-4 (or 5 distinct integers). Color information (only greyscale though) is stored in the alpha.
  • - Anything we can do to increase efficiency
  • PBR is a talk in and of itself, but there a few things worth mentioning.
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Battlefield titles extensively used WorldMachine for the terrain, but we needed to support our up-close Photoscanned assets with realistic landscape in the distance
  • We instead complemented Photoscanned assets with real-world landscape data
  • The square in the middle represents the 8x8km Frostbite terrain, where mesh extends the backdrop a further 50km in all directions
  • Example of a scanned surface
  • Example of a scanned surface
  • Example of a scanned surface
  • A Level Construction Kit was created for each planet or biome
  • LCK video
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • Description

    Presentation by Andrew Hamilton and Ken Brown from DICE at GDC 2016.

    Photogrammetry has started to gain steam within the Games Industry in recent years. At DICE, this technique was first used on Battlefield and they fully embraced the technology and workflow for Star Wars: Battlefront. This talk will cover their research and development, planning and production, techniques, key takeaways and plans for the future. The speakers will cover photogrammetry as a technology, but more than that, show that it's not a magic bullet but instead a tool like any other that can be used to help achieve your artistic vision and craft.

    Takeaway
    Come and learn how (and why) photogrammetry was used to create the world of Star Wars. This talk will cover Battlefront's use of of the technology from pre-production to launch as well as some of their philosophies around photogrammetry as a tool. Many visuals will be included!

    Intended Audience
    A content creator friendly talk intended for pretty much any developer, especially those involved in 3D content creation. It is not a technical talk focused on the code or engineering of photogrammetry. The speakers will quickly cover all basics, so absolutely no prerequisite knowledge required.

    Transcript

    1. 1. Photogrammetry and Star Wars Battlefront Andrew Hamilton Lead Environment Artist Kenneth Brown Technical Art Director EA DICE
    2. 2. Buckle Up. Photogrammetry
    3. 3. Using photos for measurements and, in our case, modelling/texturing. Photogrammetry
    4. 4. Photogrammetry
    5. 5. Photogrammetry
    6. 6. Photogrammetry
    7. 7. Photogrammetry
    8. 8. Photogrammetry
    9. 9. The End.
    10. 10. A Retrospective • The beginning: research and development • Scanning plans and goals • Capturing the planets • Mentality and workflows • Game assets in a “photogrammetry world” • Terrain • Level Construction Kits • Key learnings and takeaways
    11. 11. Big Changes • New console generation • New franchise • Very high expectations to deliver after BF4 • A clean slate; we want to do it right! • PBR on the way but not ready (in the beginning)
    12. 12. • Star Wars! • Learning from the past • Photogrammetry pushing the boundaries • A really tight schedule • If we’re saying 60, we need to be 60 • Keeping it together Challenges
    13. 13. Dev Structure • 3 month development cycles • Content development until the end • Need for 2 week buffer discovered • Approval lock • Stabilization
    14. 14. The beginning
    15. 15. Into Star Wars: Battlefront
    16. 16. Scan everything!..? • Bridge the gap between scanned characters and their surroundings. • Make the best looking Star Wars game ever! ..at 60fps  • We have so little time…
    17. 17. From here...
    18. 18. ..to here!
    19. 19. • Capture the complete asset libraries of all planets. • Focus and capture only what is essential, not everything we see. • Capture meaningful variation and features. • Visit the real film locations for true inspiration and understanding. Location capture goals
    20. 20. • Simple kit (24mm lens, Canon 6D, tripod, color chart, etc) • Ideal weather conditions • On-location breakdown • Thinking in terms of usable pieces • Aware of Frostbite strengths/weaknesses • Focus on quality over quantity of captures Capturing on location
    21. 21. A. Dirt ground B. Leaves and twigs C. Clover ground D. Clover leaves E. Fern large F. Fern medium G. Fern small H. Log large I. Log medium J. Log small K. Tree large L. Tree medium M. Tree small N. Tree stump O. Vegetation AC D B E F G H I J K L M N O
    22. 22. • Overlap!! • Covering all angles to reduce cleanup. • Avoid cast shadow - ambient light is easily removed later in the process. • Most reliable results occurred when full asset was visible in most photos. • Avoid objects with too intricate details. • Be sure that all capture methods are consistent and clean. • Prepare yourself and equipment for various conditions – weather can make the process sloppy and destroy consistent results. Best practices
    23. 23. Capturing the planets
    24. 24. Successful location scanning • A small and manageable group with light equipment is ideal. • Structured and focused on location. • Absolutely key to connect and understand how to recreate real locations.
    25. 25. Managing the data • Over 100,000 photos to manually sort • Over 2000 assets to manually review • Over 14 hours of video to categorize • Less than ideal storage solution • A lot of room for improvement here!
    26. 26. Workflow
    27. 27. Workflow Now we have the data, so let’s go.
    28. 28. High Poly High Poly Re-Topo Re-Topo UV Layout UV Layout Texture Texture Setup Setup 0 2 4 6 8 10 12 14 16 BF4 SW:BF Days Per Asset High Poly Re-Topo UV Layout Texture Setup How long does an asset take? Is Photogrammetry a savings? (Straw Poll)
    29. 29. How long does an asset take? With Automation High Poly High Poly Re-Topo Re-Topo UV Layout UV Layout Texture Setup 0 2 4 6 8 10 12 14 16 BF4 SW:BF Days Per Asset High Poly Re-Topo UV Layout Texture Setup
    30. 30. Photogrammetry workflow • Remove vignette, keep lens distortion. • Batch calibration to color chart. • This entire step should be automated! PS Camera RAW
    31. 31. Photogrammetry workflow • Most assets need to be masked – very time consuming if not automatic. • Baking at High gave most useable and manageable results for Objects. • Baking at Ultra gave best results for Terrain – files are huge though. Agisoft PhotoScan
    32. 32. Photogrammetry workflow • Mosaic rather than Average blend gives much sharper results. • Shredded UVs – take to Zbrush and reproject, or bake high res. Agisoft PhotoScan
    33. 33. Photogrammetry workflow • Some assets completely automated in MeshLab. • Some needed fine tuning in Topogun – ok if very limited asset library. Topogun / MeshLab
    34. 34. Photogrammetry workflow • Detail textures applied later means UVs demand very little stretching. • UV direction important for consistent detail texture alignment, sometimes may need a second UV just for detail tiling. Maya / UVLayout
    35. 35. Photogrammetry workflow • Color, Tangent Normals, Object Normals, Heightmap, AO, Cavity. XNormal
    36. 36. Photogrammetry workflow • Object Space Normals used as a mask to remove ambient lighting. • AO or cavity map for fine tune lighting removal. XNormal
    37. 37. • 16 bit depth is your friend • Remove as much as possible from camera raw • Once removed, move on to ambient • Object space normal maps ftw Photogrammetry workflow Light Removal
    38. 38. Light Removal Object Normal • In this example, shadows are not so distinct • Generate an object normal map • This gives distinct colors for the different object normals
    39. 39. Light Removal Object Normal • Using Black & White mask in Photoshop you can pull out different “directions”
    40. 40. Workflow Now we have the base “source” data…
    41. 41. Texel Density • Graphical representation of a pixel • An initial pitfall for us • The following screenshots are 1080p • Illustrates texture pixel (texel) to screen pixel ratio
    42. 42. ~10 texture pixels per screen pixel (excluding mips)
    43. 43. For Comparison: Two 4k Textures. 55mb
    44. 44. For Comparison: Two 1k Textures. 3.7mb
    45. 45. Texel Density • You’d be surprised what you can get away with • Detail maps are our friends • Large textures have a very big streaming overhead • We have an 800mb texture pool on Gen4 • One 8k texture is almost 90mb Takeaways
    46. 46. Texture Arrays • The photoscanned asset as a base is generally quite low res. • All environmental assets use a consistent preset shader. • Planet-specific Texture Array instance shaders add the detail on top. Object Preset Shader
    47. 47. Texture Arrays • This texture array instance shader is set up for normal, reflectance, and color. Texture Array Instance Shader
    48. 48. Texture Arrays • Brightness is painted in 5 even steps (as there are 5 detail textures). • The order of these layers must correspond to the order in the Array. Tree Normals (Texture Array mask in Alpha) Brightness % 0 25 50 75 100 Detail Leaves Bark Burnt Bark Rock Moss
    49. 49. Texture Arrays • Detail textures are fading away towards the end of LOD0 with ‘LodFade’. • LOD1 has a new simplified shader without detail textures. Texture Array Instance Shader
    50. 50. Texture Arrays • Like most texture assets in Battlefront, we heavily pack into channels. • The detail texture uses RG for normals, B for Reflectance, A for Color. Detail Texture within the Texture Array
    51. 51. Texture Arrays • Unique PhotoScan base is surface blurred, while keeping important details. • Drastically reduces texture memory.
    52. 52. Texture Arrays • Texture Array tiling detail on top fill in the details without overpowering the base. • Drastically increases texture resolution.
    53. 53. How do we ensure all assets fit together consistently? “Old” Asset Creation QUALITY TECHEFFORT Photogrammetry TECH+TECHNIQUES QUALITY EFFORT “New” Asset Creation QUALITY TECHEFFORT NEEDED EFFORTTECH+TECHNIQUES Photoscan vs. Everything Else
    54. 54. • Helps us bridge that gap • Lessons from Criterion, Guerilla Games, Dontnod etc • Trying to get everyone to switch together can backfire. • Some people will adapt instantly. Let them lead! • For PBR, there isn’t really a middle road. PBR Quick Thoughts
    55. 55. PBR • Assets generally fit well but still needed to be balanced against the scene. • Finding and trusting true PBR values at times was a challenge • Constant reinforcement helps prevent inconsistencies
    56. 56. BaseColor
    57. 57. Normals
    58. 58. Reflectance
    59. 59. Smoothness
    60. 60. SubSurfaceTranslucency
    61. 61. Shadow
    62. 62. Ambient Occlusion
    63. 63. Light + Indirect
    64. 64. Vegetation • Reaching 60fps – lets make a dense forest! • Integrating hand-built vegetation into a PhotoScanned world was tough to reach comparable quality! • Difficult to get good results from scanned vegetation – complex and noisy, resulting in horrible scans. • Based on photos of vegetation from location captured on blue board.
    65. 65. Vegetation • All vegetation share the same textures • There are textures created separately for close up, mid, and far distances. • There is a lot of texture packing within vegetation assets to reduce complexity of shaders. • Normals created by broadly modelling around leaves and branches Maya.
    66. 66. Vegetation • Unique PhotoScanned base blends to tiling texture that extends upwards. • Tree base is generally evenly tessellated for best displacement results. Tiling trunk texture Unique base texture
    67. 67. Terrain/asset blending • A lot of work put in to reduce the visible disconnect between assets and terrain. • Most assets have terrain radiosity projected on to them. • Texture Arrays world projected and details match. • Terrain color and normals applied to most assets.
    68. 68. Terrain/asset blending • A lot of work put in to reduce the visible disconnect between assets and terrain. • Most assets have terrain radiosity projected on to them. • Texture Arrays world projected and details match. • Terrain color and normals applied to most assets.
    69. 69. Mesh Displacement • Great quality boost for close up detail. • Object displacement stopped at ~7 meters. • An ongoing balance between quality and performance.
    70. 70. Mesh Displacement OFF
    71. 71. Mesh Displacement ON
    72. 72. • Ability to define displacement shape (via mask) • One of our “Must Have” features • Critical for ATAT footprints • Terrain Decal Component • Combined with detail displacement shader • (More on this a little later) Tessellation Heightfield Displacement
    73. 73. • Quality vs. Speed ratio is extremely high • Feel free to get creative Tessellation and Photogrammetry Heightfield Displacement
    74. 74. Speaking of terrain. Terrain
    75. 75. Terrain Final big chunk of our “photogrammetry” puzzle.
    76. 76. Terrain • 8x8km with a 1x1km playable space. • Huge background terrain meshes. • Terrain textures captured from real world. • Layers limited to control draw calls.
    77. 77. Moving (mostly) away from WorldMachine..
    78. 78. ..to real-world topography data.
    79. 79. Processing the data
    80. 80. Extending the horizon 100x80km
    81. 81. Texture Capture
    82. 82. Best practices - Terrain • We aimed for 3x3 meter patches • 5x5 photo grid ensures enough overlap • Capture only 1 defined element at a time • Low ISO and high F-Stop for sharp image • Avoid noisy surfaces such as grass
    83. 83. Terrain Tessellation • Great quality boost for close up detail on terrain • Major upgrade for us. Can never go back! • Terrain displacement stopped at ~15 meters.
    84. 84. Terrain Tessellation OFF
    85. 85. Terrain Tessellation ON
    86. 86. Level Construction Kits • Content and documentation packages. • Provides fast level iteration. • Shifts focus to building levels rather than content during Production. • Allows us to more quickly understand performance and keep track of it.
    87. 87. Level Construction Kits • 8x8km highly detailed terrain. • Ready to paint terrain materials. • Easily accessible limited library of content. • Lighting presets based on the films.
    88. 88. HothEndor Tatooine Sullust
    89. 89. Endor
    90. 90. Endor
    91. 91. Hoth
    92. 92. Hoth
    93. 93. Tatooine
    94. 94. Tatooine
    95. 95. Sullust
    96. 96. Sullust
    97. 97. Key learnings and takeaways • Solid processes and workflows in place before Production was a big win. • Well planned and structured Photogrammetry trips results in clean and manageable assets. • Photogrammetry is not a silver bullet – it was only one part of achieving the final results in game. • It was crucial to have artists own and be responsible for their locations/planets - Consistency is key when working with scanned assets. • Working towards PBR was an ongoing process to success. • Much of our creative process was incredibly repetitive, much of which can and should be automated. • There was way too much manual labor, with levels being built by brute force without the help of workflow-improving tools.
    98. 98. Special Thanks John Troive - Pontus Ryman - Petter Sköld - Tim McLeod - Simon Barsky - Darko Pracic - Daniel Cambrand - Oscar Carlen - Madina Chionidi - Björn Arvidsson - Robert Kihl - … and of course, the entire Battlefront team at DICE! Senior Environment Artist Senior Environment Artist Senior Environment Artist Environment Artist Environment Artist Environment Artist Environment Artist Senior Lighting Artist Character Artist Character Artist Rendering Engineer
    99. 99. Thanks! Andrew Hamilton – Lead Environment Artist Andrew.Hamilton@dice.se Kenneth Brown – Technical Art Director Kenneth.Brown@dice.se

    Editor's Notes

  • Photogrammetry -  the science of making measurements from photographs, especially for recovering the exact positions of surface points. Moreover, it may be used to recover the motion pathways of designated reference points located on any moving object, on its components and in the immediately adjacent environment. Photogrammetry may employ high-speed imaging and remote sensing in order to detect, measure and record complex 2-D and 3-D motion fields (see also sonar, radar, lidar, etc.). Photogrammetry feeds the measurements from remote sensing and the results of imagery analysis into computational models in an attempt to successively estimate, with increasing accuracy, the actual, 3-D relative motions within the researched field. (source: Wikipedia)
  • Photogrammetry -  the science of making measurements from photographs, especially for recovering the exact positions of surface points. Moreover, it may be used to recover the motion pathways of designated reference points located on any moving object, on its components and in the immediately adjacent environment. Photogrammetry may employ high-speed imaging and remote sensing in order to detect, measure and record complex 2-D and 3-D motion fields (see also sonar, radar, lidar, etc.). Photogrammetry feeds the measurements from remote sensing and the results of imagery analysis into computational models in an attempt to successively estimate, with increasing accuracy, the actual, 3-D relative motions within the researched field. (source: Wikipedia)
  • A detailed breakdown of our step-by-step workflow can be found here.
  • Align photos.
  • Generate point cloud.
  • Generate highpoly mesh.
  • Generate/project textures.
  • Opportunity to change the way we think and create content
    Available manpower (artists transitioning from BF4)



  • Photogrammetry can make great high res, largely organic assets
    We still need man (alien?) made asset
    We need to significantly increase our efficiency
    Let’s look at were we are and re-evaluate everything we know. Nothing is off limits.
    Today we’ll discuss some of the software we settled on
  • The beginnings of photogrammetry with Battlefield 4
  • An example of the range of captured assets from the Lucas archives.
  • An example of the range of captured assets from the Lucas archives.
  • Supporting scanned assets with other tools such as Marvellous Designer
  • Hard-Surface materials created with traditional methods
  • An example of the level of detail we’re able to get with combining photoscan with new (to us) software.
  • An example of the level of detail we’re able to get with combining photoscan with new (to us) software.
  • An example of the ingame assets using the combination of techniques.
  • A screenshot from Battlefield: Armored Kill that was considered ‘realistic’ during its time…
  • …to a screenshot from Star Wars: Battlefront, using photogrammetry on a very limited number of selective content to build up a realistic environment
  • A slideshow of locations we have visited on scanning trips, for a quick glimpse into the range and variety of settings at our disposal. Iceland
  • Iceland
  • Death Valley, USA
  • Utah, USA
  • Death Valley, USA
  • National Redwood Forests , USA
  • National Redwood Forests , USA
  • Finse, Norway
  • Finse, Norway
  • Iceland
  • Iceland
  • Iceland
  • A small selection from our large library of scanned materials that would be used for Sullust terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Tatooine terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Endor terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Hoth terrain surfaces
  • A small selection from our large library of scanned materials that would be used for Hoth terrain surfaces
  • Vegetation and ground scattering assets captured on a blue board for easy background extraction
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • An example of a scanned asset, showing how intentionally generic and tilable they are.
  • Opportunity to change the way we think and create content
    Available manpower (artists transitioning from BF4)



  • Opportunity to change the way we think and create content
    Available manpower (artists transitioning from BF4)



  • Data taken from a cross sampling of content creator disciplines. Straw poll – not necessarily accurate 1:1, but representative of reality.
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • A basic step by step guide through the Photogrammetry workflow
  • Shadows
     
    Need to mask out shadows
    Super important to have 16 bit depth
    Before that, remove as much as possible from the camera raw (shadow slider photoshop camera raw plugin)
    Create mask from what’s dark.
    Flatten luminance in the colors first
    Shadows tend to be colder, so you can use color to grab them.
     
    Once you’ve gotten the shadows removed, you can treat it like something that’s been captured overcast
     
    Get rid of ambient light
    Two ways
    Use the normal map (baked out in object space) to get the angle.
    Use the occlusion, baked from the geometry.
    Also need to use the mask to remove the color (bounced light)
    Overcast is best
  • Let’s look at other aspects of the asset creation
  • Texel refers to the graphical representation of a pixel on a 2d object (eg. a texture). Basically, when you look at a pixel of a texture rendered on a 3d model, you are looking at a“texel”.
  • In this shot ~25x’s more pixels available than the screen can render.
  • Mips always try to keep a 1:1 ratio. So detail will get mipped away anyways!
  • Not only is this extremely close for 1p, it’s an even more extreme distance for 3p. At this resolution, there is certainly an absolute difference in quality, but a trivial relative difference in quality.
  • To give you a sense of the memory impact (and subsequent streaming impact) that this has.
  • The quality of these assets are great. This just reinforces what we’ve been saying – focus on the craftsmanship. More resolution isn’t what’s going to make these assets look better – the overall quality of the asset is what’s going to sell it.
  • In this example, the multiply node is set to a factor of 4, give a range from 0-4 (or 5 distinct integers). Color information (only greyscale though) is stored in the alpha.
  • - Anything we can do to increase efficiency
  • PBR is a talk in and of itself, but there a few things worth mentioning.
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Render passes that compose the final image
  • Battlefield titles extensively used WorldMachine for the terrain, but we needed to support our up-close Photoscanned assets with realistic landscape in the distance
  • We instead complemented Photoscanned assets with real-world landscape data
  • The square in the middle represents the 8x8km Frostbite terrain, where mesh extends the backdrop a further 50km in all directions
  • Example of a scanned surface
  • Example of a scanned surface
  • Example of a scanned surface
  • A Level Construction Kit was created for each planet or biome
  • LCK video
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • This scene appears to be comprised of many elements with much variation…
  • …but it is only comprised of a very limited amount of generic assets that work well to populate a large area without appearing repetitive.
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • A slideshow showing the variety of locations in Star Wars: Battlefront, created with a limited set of assets using the Level Construction Kits
  • More Related Content

    More from Electronic Arts / DICE

    ×