Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Star Ocean 4 - Flexible Shader Managment and Post-processing


Published on

Published in: Technology, Art & Photos
  • I'd advise you to use this service: ⇒ ⇐ The price of your order will depend on the deadline and type of paper (e.g. bachelor, undergraduate etc). The more time you have before the deadline - the less price of the order you will have. Thus, this service offers high-quality essays at the optimal price.
    Are you sure you want to  Yes  No
    Your message goes here
  • Yes you are right. There are many research paper writing services available now. But almost services are fake and illegal. Only a genuine service will treat their customer with quality research papers. ⇒ ⇐
    Are you sure you want to  Yes  No
    Your message goes here
  • If you have any problems with writing, feel free to ask our writers for help! ⇒ ⇐ is ready to help with any kind of academic writing!
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ♥♥♥ ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❶❶❶ ❶❶❶
    Are you sure you want to  Yes  No
    Your message goes here

Star Ocean 4 - Flexible Shader Managment and Post-processing

  1. 2. STAR OCEAN 4 : Flexible Shader Management and Post-processing Yoshiharu Gotanda Research and Development Dept., tri-Ace Inc.
  2. 3. What is STAR OCEAN 4? <ul><li>Star Ocean: The Last Hope </li></ul><ul><ul><li>The latest installment of an RPG series from Square-Enix </li></ul></ul><ul><ul><li>Developed by tri-Ace Inc. </li></ul></ul><ul><ul><li>On Xbox 360 </li></ul></ul><ul><ul><li>Released in February 2009 </li></ul></ul><ul><ul><ul><li>For North America and Japan </li></ul></ul></ul><ul><ul><ul><li>Coming soon for Europe </li></ul></ul></ul><ul><ul><li>Developed on an in-house game engine </li></ul></ul><ul><ul><ul><li>Also used for Infinite Undiscovery (Xbox 360) and Valkyrie Profile : Covenant of the Plume (NDS) </li></ul></ul></ul>
  3. 4. History of STAR OCEAN <ul><li>Drives the studio’s development environment </li></ul><ul><ul><li>Each game developed for each platform </li></ul></ul><ul><ul><li>Changed development environments every time </li></ul></ul>
  4. 5. STAR OCEAN <ul><li>Released in 1996 </li></ul><ul><ul><li>On SNES (only for Japan) </li></ul></ul><ul><ul><li>No clear game engine </li></ul></ul><ul><ul><ul><li>All game code written in assembly </li></ul></ul></ul><ul><ul><ul><li>Used in-house script language for events </li></ul></ul></ul><ul><ul><ul><li>Used in-house art tools </li></ul></ul></ul>
  5. 6. STAR OCEAN : The Second Story <ul><li>Released in 1998-99 </li></ul><ul><ul><li>On PlayStation </li></ul></ul><ul><ul><li>Used TPFC (1 st gen.) </li></ul></ul><ul><ul><ul><li>Basic I/O (virtualization) and memory management, threading, math functions. </li></ul></ul></ul><ul><ul><ul><li>Written in C++ </li></ul></ul></ul><ul><ul><ul><li>Also used for Valkyrie Profile </li></ul></ul></ul><ul><ul><ul><li>Used in-house script language for events </li></ul></ul></ul><ul><ul><li>“ STAR OCEAN : Blue Sphere” released for Game Boy in 2001(only for Japan) </li></ul></ul>
  6. 7. STAR OCEAN: Till the End of Time <ul><li>Released in 2003-04 </li></ul><ul><ul><li>On PlayStation 2 </li></ul></ul><ul><ul><li>Used TPFC2 and tpgcLib (2 nd gen.) </li></ul></ul><ul><ul><ul><li>TPFC for PS2 and Graphics Engine </li></ul></ul></ul><ul><ul><ul><ul><li>Supported 3dsmax </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Custom shader engine </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Flexible Particle System </li></ul></ul></ul></ul><ul><ul><ul><ul><li>HDR Rendering and Spherical Harmonics Lighting </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Animation driven physics </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Most cloth and accessories simulated in real-time, even within cutscenes </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Also used for Radiata Stories and Valkyrie Profile : Silmeria </li></ul></ul></ul></ul><ul><ul><ul><li>Used in-house interactive cutscene tool </li></ul></ul></ul>
  7. 8. STAR OCEAN: The Last Hope <ul><li>Released in 2009 </li></ul><ul><ul><li>On Xbox 360 </li></ul></ul><ul><ul><li>Used the 3 rd generation engine </li></ul></ul><ul><ul><ul><li>Also used for Infinite Undiscovery (Xbox 360) and Valkyrie Profile : Covenant of the Plume (NDS) </li></ul></ul></ul><ul><ul><li>Used in-house interactive cutscene tool </li></ul></ul><ul><ul><ul><li>99.5% of cutscenes are rendered in real-time </li></ul></ul></ul><ul><ul><ul><ul><li>Some scenes used FMV by Square-Enix </li></ul></ul></ul></ul>
  8. 9. 3 rd Generation Engine <ul><li>Aska </li></ul><ul><ul><li>Designed based upon the 2 nd gen engine </li></ul></ul><ul><ul><li>Integrated Game Development Environment </li></ul></ul><ul><ul><li>Supports Maya </li></ul></ul><ul><ul><li>Fully multi-threaded engine </li></ul></ul><ul><ul><li>Flexible shader system </li></ul></ul><ul><ul><li>Camera simulation </li></ul></ul><ul><ul><li>Flexible Particle System </li></ul></ul><ul><ul><li>HDR Rendering and flexible light/shadow system </li></ul></ul><ul><ul><li>Animation driven physics </li></ul></ul><ul><ul><li>Multi-platform </li></ul></ul><ul><ul><ul><li>PC </li></ul></ul></ul><ul><ul><ul><li>Xbox 360 </li></ul></ul></ul><ul><ul><ul><li>Nintendo DS </li></ul></ul></ul><ul><ul><ul><li>Other platforms…  </li></ul></ul></ul>
  9. 10. Video (Dev env.)
  10. 11. Flexible Shader Management <ul><li>Fully flexible shader management </li></ul><ul><ul><li>Artists can create materials in Maya </li></ul></ul><ul><ul><ul><li>Using Hypershade interface </li></ul></ul></ul><ul><ul><ul><li>Shader binary is automatically generated according to artists setting </li></ul></ul></ul>
  11. 12. Design Policy <ul><li>Artists create shaders in Maya </li></ul><ul><ul><li>They can create shaders without a programmer </li></ul></ul><ul><ul><ul><li>Not limited by programmers </li></ul></ul></ul><ul><ul><ul><li>Can try new ideas immediately </li></ul></ul></ul><ul><ul><li>Need to train artists </li></ul></ul><ul><ul><ul><li>How to set parameters and construct shaders </li></ul></ul></ul><ul><ul><ul><li>Knowledge of physics (a little bit) </li></ul></ul></ul><ul><ul><ul><li>Templates </li></ul></ul></ul>
  12. 13. Pros and cons <ul><li>Shader generated at run-time (during development) </li></ul><ul><ul><li>Pros </li></ul></ul><ul><ul><ul><li>Don’t have to include shader binary in resource files </li></ul></ul></ul><ul><ul><ul><li>Artists can freely create shaders </li></ul></ul></ul><ul><ul><ul><li>Easily support the variation of shaders at run-time </li></ul></ul></ul><ul><ul><ul><li>Easy to manage a shader binary file </li></ul></ul></ul><ul><ul><li>Cons </li></ul></ul><ul><ul><ul><li>Explosion of the number of shader variations </li></ul></ul></ul><ul><ul><ul><ul><li>Large shader binary </li></ul></ul></ul></ul><ul><ul><ul><li>Must create possible shader variations </li></ul></ul></ul><ul><ul><ul><ul><li>Must play through all possible content in the game. </li></ul></ul></ul></ul>
  13. 14. Subdivided shaders <ul><li>Implement small shader nodes with specific functionality </li></ul><ul><ul><li>Correspond to Shader Node in Maya </li></ul></ul><ul><ul><li>Artists can connect each input and output freely </li></ul></ul><ul><ul><ul><li>UV, Color, Normal, Alpha… </li></ul></ul></ul>
  14. 15. Shader samples - Lighting <ul><li>Shade surfaces with the result from lighting </li></ul><ul><ul><li>Phong </li></ul></ul><ul><ul><li>Anisotropic Phong </li></ul></ul><ul><ul><li>Blinn-Phong </li></ul></ul><ul><ul><li>Normalized Blinn </li></ul></ul><ul><ul><li>Ashikhmin </li></ul></ul><ul><ul><li>Kajiya-Kay </li></ul></ul><ul><ul><li>Marschner </li></ul></ul><ul><ul><ul><li>Albedo Map </li></ul></ul></ul><ul><ul><ul><li>Specular Map </li></ul></ul></ul><ul><ul><ul><li>Gloss(shininess) Map </li></ul></ul></ul><ul><ul><ul><li>Fresnel Map </li></ul></ul></ul><ul><ul><ul><li>Offset Map </li></ul></ul></ul><ul><ul><ul><li>Translucency </li></ul></ul></ul><ul><ul><ul><li>Ambient Occlusion </li></ul></ul></ul>
  15. 16. Shader samples – Normal, UV <ul><li>Normal generation </li></ul><ul><ul><li>Normal Mapping </li></ul></ul><ul><ul><li>Parallax Mapping </li></ul></ul><ul><ul><li>Parallax Occlusion Mapping </li></ul></ul><ul><li>UV generation </li></ul><ul><ul><li>Reflection </li></ul></ul><ul><ul><ul><li>Sphere </li></ul></ul></ul><ul><ul><ul><li>Dual Paraboloid </li></ul></ul></ul><ul><ul><ul><li>Cube </li></ul></ul></ul><ul><ul><li>UV offset (refraction) </li></ul></ul><ul><ul><li>UV animation </li></ul></ul><ul><li>Blur effects </li></ul><ul><ul><li>Rectangle Distortion </li></ul></ul><ul><ul><li>Spherical Distortion </li></ul></ul><ul><ul><li>Radial </li></ul></ul><ul><ul><li>Others… </li></ul></ul>
  16. 17. Shader samples – Shadow, projection <ul><li>Shadow </li></ul><ul><ul><li>Multiple Uniform </li></ul></ul><ul><ul><li>Multiple LiSPSM </li></ul></ul><ul><ul><li>Cascade Shadow </li></ul></ul><ul><ul><ul><li>PCF </li></ul></ul></ul><ul><ul><li>Projection Shadow </li></ul></ul><ul><ul><li>Omni directional Cube Shadow </li></ul></ul><ul><ul><li>Static Vertex Shadow </li></ul></ul><ul><ul><li>Static Shadow </li></ul></ul><ul><li>Projector </li></ul><ul><ul><li>Color </li></ul></ul><ul><ul><li>Cube </li></ul></ul><ul><ul><li>Normal map </li></ul></ul>
  17. 18. Shader samples – Computation <ul><li>Arbitrary Blending </li></ul><ul><ul><li>Shader node that simply blends computed color, normal and so on. </li></ul></ul><ul><ul><li>Additive, subtractive, alpha blending, Photoshop-like blending and so on. </li></ul></ul><ul><ul><li>Normal blending </li></ul></ul><ul><ul><li>Absorption </li></ul></ul><ul><li>Swizzle </li></ul><ul><li>Depth processing </li></ul><ul><ul><li>Detail Mapping </li></ul></ul><ul><ul><ul><li>Blend normal maps based on the depth </li></ul></ul></ul><ul><ul><ul><ul><li>Can blend any kind of textures </li></ul></ul></ul></ul><ul><ul><li>Shader LOD based on the depth </li></ul></ul><ul><li>Branch </li></ul><ul><ul><li>Conditional shader branching </li></ul></ul><ul><ul><ul><li>Can be used for optimization (depend on hardware) </li></ul></ul></ul>
  18. 19. Shader samples – Others <ul><li>Alpha generation </li></ul><ul><ul><li>Fresnel </li></ul></ul><ul><ul><li>Soft Polygon </li></ul></ul><ul><li>Texture sampling </li></ul><ul><ul><li>Shader node fetching a texture </li></ul></ul><ul><ul><ul><li>Standard texture </li></ul></ul></ul><ul><ul><ul><li>Multipass rendered texture </li></ul></ul></ul><ul><ul><ul><li>Global texture </li></ul></ul></ul><ul><ul><li>Compressed HDR texture </li></ul></ul><ul><li>Vertex Shaders </li></ul><ul><ul><li>Skinning </li></ul></ul><ul><ul><li>Volume Rendering </li></ul></ul><ul><ul><li>Vertex Color </li></ul></ul><ul><ul><li>Vertex blending </li></ul></ul><ul><li>Particle rendering shader </li></ul><ul><ul><li>Many kinds of particle renderers </li></ul></ul><ul><li>And so on… </li></ul>
  19. 20. Shader samples – Post-processing effects <ul><li>Tone mapping </li></ul><ul><ul><li>Standard tone mapping </li></ul></ul><ul><ul><li>Film Simulation </li></ul></ul><ul><ul><ul><li>Reproduce specifications of films or C-MOS sensors </li></ul></ul></ul><ul><ul><ul><li>Reproduce film grain or digital noise </li></ul></ul></ul><ul><ul><li>Dithering </li></ul></ul><ul><li>Lens simulation </li></ul><ul><ul><li>Focus Blur (DOF) </li></ul></ul><ul><ul><li>Glare </li></ul></ul><ul><ul><li>Physically-based Lens Structure </li></ul></ul><ul><ul><li>Motion blur </li></ul></ul><ul><ul><ul><li>Camera </li></ul></ul></ul><ul><ul><ul><li>Object </li></ul></ul></ul><ul><li>Color filters </li></ul><ul><ul><li>Contrast </li></ul></ul><ul><ul><li>Brightness </li></ul></ul><ul><ul><li>Monotone </li></ul></ul><ul><ul><li>Tone Curve </li></ul></ul><ul><ul><li>Color Temperature </li></ul></ul><ul><li>Other effects </li></ul><ul><ul><li>Outdoor Light Scattering </li></ul></ul><ul><ul><li>Light Shaft simulation </li></ul></ul><ul><ul><li>Screen-space Ambient Occlusion </li></ul></ul>
  20. 21. Other features <ul><li>Shader flexibility </li></ul><ul><ul><li>Arbitrary number of… </li></ul></ul><ul><ul><li>Or any combination of… </li></ul></ul><ul><ul><ul><li>Per-pixel lights and light type </li></ul></ul></ul><ul><ul><ul><li>Per-vertex lights </li></ul></ul></ul><ul><ul><ul><li>Shadows and shadow algorithms </li></ul></ul></ul><ul><ul><ul><li>Projectors </li></ul></ul></ul>
  21. 22. Shader node example – simple Specular Map Texture Normal Map Texture Albedo Texture Ashikhmin BRDF Shader
  22. 23. Shader node example – artistic Color Texture for albedo Tangent space normal map texture Specular color texture Phong BRDF Shader Fresnel Shader Output Diffuse Color Gray scaled color output from Fresnel Shader
  23. 24. Shader node example - background Parallax Occlusion Map Shader Output Normal Dual paraboloid environment map texture Color map for albedo with modified UV Blinn-Phong BRDF Shader
  24. 25. Shader node example - effect Animated UV Offset Blending 2 textures UVs are shifted based off normal map
  25. 26. Shader node example – detail map LOD factor computation node Base Normal Map Texture Normal Map blended with three other textures using LOD factor Albedo Texture
  26. 27. AHSL <ul><li>In-house shader language based on HLSL and Cg </li></ul><ul><ul><li>Each shader node has its own AHSL source file </li></ul></ul><ul><ul><ul><li>Some shader nodes correspond to multiple source files </li></ul></ul></ul><ul><ul><li>AHSL Shader Manager class generates shader binaries by linking all code specified in Maya materials at run-time </li></ul></ul>
  27. 28. Shader Immediate Constants <ul><li>Change some constant values to immediate values </li></ul><ul><ul><li>For optimization </li></ul></ul><ul><ul><li>Use immediate values for generation of source code instead of using constant registers for specific constant vectors </li></ul></ul><ul><ul><ul><li>(0,0,0,0) </li></ul></ul></ul><ul><ul><ul><li>(0,0,0,1) </li></ul></ul></ul><ul><ul><ul><li>(1,1,1,0) </li></ul></ul></ul><ul><ul><ul><li>(1,1,1,1) </li></ul></ul></ul>float4 eConstCol0; float4 LoadConst0() { return eConstCol0;} float4 LoadConst0() { return float4 (0,0,0,0)}
  28. 29. Shader Cache <ul><li>Store complied shaders in the shader cache file on devkit </li></ul><ul><ul><li>Cache components </li></ul></ul><ul><ul><ul><li>ShaderKey </li></ul></ul></ul><ul><ul><ul><ul><li>Specifies a unique ID (includes shader structure) </li></ul></ul></ul></ul><ul><ul><ul><li>Constant table </li></ul></ul></ul><ul><ul><ul><ul><li>Information of constants </li></ul></ul></ul></ul><ul><ul><ul><li>Shader binary </li></ul></ul></ul><ul><ul><li>Assumes that this file contains all possible shader combinations which are used in a game </li></ul></ul>
  29. 30. Shader profile data <ul><li>Supports development </li></ul><ul><ul><li>Stored in different file from the cache file </li></ul></ul><ul><ul><li>Store information corresponding to each ShaderKey </li></ul></ul><ul><ul><ul><li>Total time used, Time last used </li></ul></ul></ul><ul><ul><ul><li># of queries, Time to generate </li></ul></ul></ul><ul><ul><ul><li>Name of build devkit </li></ul></ul></ul><ul><ul><ul><li>Region information </li></ul></ul></ul><ul><ul><ul><li># of merges </li></ul></ul></ul><ul><ul><ul><ul><li>And so on… </li></ul></ul></ul></ul>
  30. 31. Previous problems… <ul><li>Development period </li></ul><ul><ul><li>Star Ocean: TLH was developed almost simultaneously with Infinite Undiscovery </li></ul></ul><ul><ul><ul><li>Infinite Undiscovery finished slightly earlier </li></ul></ul></ul><ul><ul><li>Most problems related to shader management observed in Infinite Undiscovery </li></ul></ul><ul><ul><ul><li>Same problems occurred in Star Ocean: TLH </li></ul></ul></ul><ul><ul><ul><li>Most problems were solved in Infinite Undiscovery </li></ul></ul></ul><ul><ul><ul><ul><li>Some information came from Infinite Undiscovery </li></ul></ul></ul></ul>
  31. 32. Problems <ul><li>This implementation requires shader generation </li></ul><ul><ul><li>Essentially compile shaders at run-time </li></ul></ul><ul><ul><ul><li>Generated shaders stored in the cache file </li></ul></ul></ul><ul><ul><ul><li>Rebuild in the case of a version update </li></ul></ul></ul><ul><ul><ul><ul><li>Or remove the cache file </li></ul></ul></ul></ul><ul><ul><li>A foreseen problem with cache file generation and management when beginning engine design </li></ul></ul><ul><ul><ul><li>Becomes tangible as approaching to end of project (Infinite Undiscovery) </li></ul></ul></ul>
  32. 33. The size of cache file <ul><li>Don’t compile shaders in the shipped build </li></ul><ul><ul><li>Fundamentally the shader cache is created during QA </li></ul></ul><ul><ul><ul><li>Shaders CAN be compiled in the shipped build </li></ul></ul></ul><ul><ul><ul><ul><li>Didn’t use, looks ugly </li></ul></ul></ul></ul><ul><ul><li>The size of the cache file increased as approached end of project </li></ul></ul><ul><ul><ul><li>Estimated 10Mbytes at first </li></ul></ul></ul><ul><ul><ul><ul><li>Actually, seemed to exceed estimate </li></ul></ul></ul></ul>
  33. 34. Solutions <ul><li>To solve the size problem </li></ul><ul><ul><li>Decompress each shader binary at run-time </li></ul></ul><ul><ul><li>Separated shader cache to L1 and L2 </li></ul></ul><ul><ul><li>Supported multiple shader cache files </li></ul></ul><ul><ul><li>Created a tool to manage shader files in Windows </li></ul></ul><ul><ul><li>Implemented performance vs. size control parameters </li></ul></ul>
  34. 35. But… <ul><li>Increased the size </li></ul><ul><ul><li>Despite a lot of effort </li></ul></ul><ul><ul><ul><li>The size exceeded 50Mbytes! (Infinite Undiscovery) </li></ul></ul></ul><ul><ul><ul><li>More than 30,000 cache combinations </li></ul></ul></ul><ul><ul><ul><li>Separated the cache files but they still exceeded acceptable file size </li></ul></ul></ul><ul><ul><li>Started analyzing the details of the shader cache files </li></ul></ul>
  35. 36. The problem of Shader Adaptors <ul><li>Shader Adaptors dominated most of the shader combinations </li></ul><ul><ul><li>What is a Shader Adaptor? </li></ul></ul><ul><ul><ul><li>Shaders added at run-time </li></ul></ul></ul><ul><ul><ul><ul><li>Shadows, projectors and so on… </li></ul></ul></ul></ul><ul><ul><ul><ul><li>These kinds of shaders occupied 80% </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Especially shadows </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>There was a shader supporting 5 shadows for an object </li></ul></ul></ul></ul></ul>
  36. 37. Example All Adaptors are shadow maps in this shader Many shaders using 5 Shader Adaptors
  37. 38. Limitation <ul><li>Supported a limitation to the number of Shader Adaptors (shadows) </li></ul><ul><ul><li>To limit the number during generation or in the tool </li></ul></ul><ul><ul><ul><li>The size was dramatically decreased </li></ul></ul></ul><ul><ul><ul><ul><li>Caused appearance problems </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Manually adjusted </li></ul></ul></ul></ul></ul>
  38. 39. Non-generated shaders <ul><li>Implemented functionality for shaders not generated </li></ul><ul><ul><li>In case of Shader Adaptors </li></ul></ul><ul><ul><ul><li>Used base shaders </li></ul></ul></ul><ul><ul><ul><li>Better than disappearing </li></ul></ul></ul>
  39. 40. Cache file creation <ul><li>The cache file was created by the QA team </li></ul><ul><ul><li>Created the cache file when the specifications and resources relating to the shader were fixed </li></ul></ul><ul><ul><ul><li>Using debug functionality </li></ul></ul></ul><ul><ul><ul><li>Play through the game </li></ul></ul></ul><ul><ul><ul><li>Merge files created by multiple testers </li></ul></ul></ul><ul><ul><li>This process was much tougher than expected </li></ul></ul><ul><ul><ul><li>Several weeks with tens of testers </li></ul></ul></ul><ul><ul><ul><li>Done again and again because it was an unfamiliar system </li></ul></ul></ul>
  40. 41. Statistics <ul><li>Infinite Undiscovery separated the file into 4 files. </li></ul><ul><ul><li>Common Region contains shaders shared for both areas </li></ul></ul>11120 14.8M Area2 6804 9.0M Area1 9158 9.8M Common 26 18k Boot # of shaders Size Name of Region
  41. 42. Statistics <ul><li>Star Ocean: TLH separated the file into 4 files </li></ul><ul><ul><li>Star Ocean: TLH only has Boot Region as a common shader binary </li></ul></ul><ul><ul><ul><li>Disc1, 2 and 3 shader files include common shaders </li></ul></ul></ul>15810 19.8M Disc3 20417 25.4M Disc2 18882 24.1M Disc1 356 293k Boot # of shaders Size Name of region
  42. 43. Summary of size problem <ul><li>The engine was developing simultaneously with game development for earlier projects </li></ul><ul><ul><li>Variations in Shader Adaptors </li></ul></ul><ul><ul><ul><li>Many shadow algorithms implemented </li></ul></ul></ul><ul><ul><li>Variations in Shader Immediate Constants </li></ul></ul><ul><ul><ul><li>Found a solution </li></ul></ul></ul><ul><ul><ul><ul><li>Didn’t use at this time </li></ul></ul></ul></ul><ul><ul><li>Increased the number of shaders as new shaders were implemented </li></ul></ul><ul><ul><ul><li>Final shaders differed dramatically despite similar appearance </li></ul></ul></ul><ul><ul><ul><li>Artists repeated a trial and error process because it was their first experience </li></ul></ul></ul>
  43. 44. Disadvantages <ul><li>The cost of shader cache creation </li></ul><ul><ul><li>Limitation of automatic creation </li></ul></ul><ul><ul><li>The problem of file size </li></ul></ul><ul><ul><ul><li>Cache file size too big </li></ul></ul></ul><ul><ul><ul><ul><li>Probably no problem for next-gen? </li></ul></ul></ul></ul><ul><ul><ul><li>Consequently, should be conscious of this in the case of creating resources </li></ul></ul></ul><ul><ul><ul><li>Try to cut down the shader count </li></ul></ul></ul><ul><li>Difficulty of creating shaders </li></ul><ul><ul><li>Artists must know the mechanism of shaders </li></ul></ul>
  44. 45. Advantages <ul><li>High flexibility </li></ul><ul><ul><li>Artists can create various shaders without programmers </li></ul></ul><ul><ul><ul><li>Can create an unreasonable combination of shaders </li></ul></ul></ul><ul><ul><li>Performance optimization </li></ul></ul><ul><ul><ul><li>Shader Immediate Constants… </li></ul></ul></ul><ul><ul><li>Optimal shader code can be automatically generated using a shader complier </li></ul></ul>
  45. 46. More problems <ul><li>Solution for shader creation </li></ul><ul><ul><li>More automation </li></ul></ul><ul><ul><ul><li>Create estimated shader combinations off line? </li></ul></ul></ul><ul><ul><ul><li>Create shaders using scene information or database? </li></ul></ul></ul><ul><ul><li>Optimize shader generation </li></ul></ul><ul><ul><ul><li>Rebuild </li></ul></ul></ul><ul><ul><ul><li>Distributed shader generation </li></ul></ul></ul><ul><ul><li>Reduce combinations </li></ul></ul><ul><ul><ul><li>Run-time shader profiler </li></ul></ul></ul><ul><ul><li>GPU support? </li></ul></ul><ul><ul><ul><li>DX11? </li></ul></ul></ul>
  46. 47. This system is… <ul><li>Also used for </li></ul><ul><ul><li>Particle rendering </li></ul></ul><ul><ul><ul><li>Many variations of particles </li></ul></ul></ul><ul><ul><li>Post-processing shader </li></ul></ul><ul><ul><ul><li>Flexible post-processing </li></ul></ul></ul>
  47. 48. Flexible post-processing <ul><li>Our engine tries to implement </li></ul><ul><ul><li>Camera system simulation </li></ul></ul><ul><ul><ul><li>Physically plausible </li></ul></ul></ul><ul><ul><ul><li>Visually plausible </li></ul></ul></ul><ul><ul><li>In order to achieve photo-realistic camera simulation, HDR rendering is very important </li></ul></ul><ul><ul><ul><li>Camera simulation is an efficient approach to utilize a HDR rendering result </li></ul></ul></ul><ul><ul><li>Post-processing is a realistic solution to simulate phenomena occurring in a camera </li></ul></ul><ul><ul><ul><li>Ray-tracing is an ideal solution </li></ul></ul></ul>
  48. 49. Photo-realistic Camera System <ul><ul><li>Our engine tries to implement a physically-based camera system </li></ul></ul>200mm F2.8
  49. 50. Why physics? <ul><li>Unnatural effects from the camera system </li></ul><ul><ul><li>An audience feels incongruity from situations which are impossible to shoot using a real camera </li></ul></ul><ul><ul><ul><li>Grow up watching a lot of visuals through a real camera </li></ul></ul></ul><ul><ul><ul><ul><li>TV programs </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Films </li></ul></ul></ul></ul><ul><ul><ul><li>Very shallow depth of field with a wide photo lens or a far focal distance? </li></ul></ul></ul><ul><ul><ul><li>Large F value when shooting a dark scene </li></ul></ul></ul><ul><ul><ul><ul><li>Long time exposure? Motion Blur? </li></ul></ul></ul></ul><ul><ul><ul><li>No noise or grain within dark scenes </li></ul></ul></ul><ul><ul><ul><ul><li>Need motion blur or hand vibration when carrying a camera by hand? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Be conscious of shooting situations </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Time value can’t exceed the frame rate in the case of movies </li></ul></ul></ul></ul><ul><ul><ul><li>No diffraction in bright pixels </li></ul></ul></ul><ul><ul><ul><li>And so on… </li></ul></ul></ul><ul><ul><li>Physically plausible effects make visuals persuasive and give presence </li></ul></ul><ul><ul><ul><li>Has been used in animation for a long time </li></ul></ul></ul>Effect from diffraction Less noise but long motion blur Noisy but short motion blur
  50. 51. Before physics <ul><li>Rendering must be processed in linear color space </li></ul><ul><ul><li>Difficult to process properly without linear color space </li></ul></ul><ul><ul><ul><li>Shading (BRDF) </li></ul></ul></ul><ul><ul><ul><li>Post-processing </li></ul></ul></ul><ul><ul><ul><ul><li>The law of energy conservation </li></ul></ul></ul></ul><ul><ul><ul><li>Proper texture filtering </li></ul></ul></ul><ul><ul><ul><li>Be aware of the color space </li></ul></ul></ul><ul><ul><ul><li>sRGB space of textures should be converted to linear </li></ul></ul></ul><ul><ul><ul><li>Ideally spectrum based rendering? </li></ul></ul></ul><ul><ul><li>Finally apply gamma correction </li></ul></ul><ul><ul><ul><li>Appropriate gamma correction for the output device </li></ul></ul></ul>
  51. 52. Bokeh Effect (DOF)
  52. 53. High quality bokeh <ul><li>Characteristics of our bokeh </li></ul><ul><ul><li>Can adjust actual camera parameters </li></ul></ul><ul><ul><ul><li>F-stop, Focal length, real lens, ISO sensitivity, etc… </li></ul></ul></ul><ul><ul><li>Quality settings hardly affect look of bokeh </li></ul></ul><ul><ul><ul><li>Reduced buffer size, tap count, etc… </li></ul></ul></ul><ul><ul><li>High quality, fine blur </li></ul></ul><ul><ul><ul><li>Try to focus on only one eye in close-up shots </li></ul></ul></ul><ul><ul><ul><ul><li>Very shallow depth of field </li></ul></ul></ul></ul><ul><ul><li>Easy shader management with AHSL system </li></ul></ul>
  53. 54. Bokeh – Low profile 1/16 & 1/64 Reduced buffer 33 tap GPU 1.3[ms]
  54. 55. Bokeh – Standard profile 1/4 & 1/16 Reduced buffer 21 tap GPU 3.1[ms]
  55. 56. Bokeh – Reference profile Use full size buffer 227 tap GPU 246.6[ms]
  56. 57. Lens Equation <ul><li>The size of bokeh can be computed with the Lens Equation </li></ul>x: diameter of blur (CoC) o: object distance p: focal distance f: focal length F: F-stop
  57. 58. Blur kernel <ul><li>Blurred light is distributed equally </li></ul><ul><ul><li>Without considering aberrations </li></ul></ul><ul><ul><li>Blur kernel should be flat </li></ul></ul><ul><ul><ul><li>Not Gaussian </li></ul></ul></ul>In focus (focus on the plane) Out of focus (Further distances focus off the plane) Image plane
  58. 59. Gaussian blur <ul><li>Bokeh with Gaussian blur is often seen </li></ul><ul><ul><li>Not visually pleasing </li></ul></ul><ul><ul><li>Looks softer than reality </li></ul></ul><ul><ul><ul><li>No sharp edge on blur (especially in bright places) </li></ul></ul></ul>Gauss Flat
  59. 60. Blend with blurred image <ul><li>Bokeh by simply blending with a blurred image </li></ul><ul><ul><li>Also not visually pleasing </li></ul></ul><ul><ul><li>Looks like the soft focus effect </li></ul></ul>
  60. 61. Blend vs. Per-pixel blur Blend Per-pixel (21taps)
  61. 62. Bokeh diagram Output (Gathered during Tonemapping Process) Blend Mipmapped color buffers Reduced color buffer Fixed tap count blur on Large Reduced Buffer High tap count blur on Small Reduced Buffer Create masks for all the bokeh shaders from depth buffer
  62. 63. Bokeh diagram (detail) Output (Gathered during Tonemapping Process) Fixed tap count blur and blend with SRB on large reduced buffer High tap count blur on Small Reduced Buffer (SRB) Create mipmap src. (put mask to A) Generate A lot of Masks (A,R,G,B)x2 from Depth Dilate Front Mask Create Half Reduced Buffer
  63. 64. Mipmapped source textures <ul><li>Create mipmapped textures of the back buffer </li></ul><ul><ul><li>Trade off between the tap count and sharpness using tri-linear filtering </li></ul></ul>Different Source Tex LodBias, same tap count More correct blur radius Smoother but less correct
  64. 65. Bleeding artifacts <ul><li>A phenomenon where foreground pixels bleed into the background </li></ul>Fixed
  65. 66. Solution for bleeding artifact <ul><li>Create blurred image from the source image and the mask image </li></ul><ul><ul><li>Bleeding occurs on bright boundaries because of bilinear filtering </li></ul></ul><ul><li>Create blurred image with the source image pre-multiplied by the mask image </li></ul><ul><ul><li>Expensive, need to process the foreground blur and background blur separately </li></ul></ul>Using Source Image & Mask Using pre-multiplied Source Image Bleeding Remains
  66. 67. Creating foreground mask <ul><li>Expanding mask texture with 10 taps for foreground bokeh </li></ul><ul><li>fetch 10 surrounding points for dilate </li></ul><ul><li>9 tap Gaussian blur (4 fetches) </li></ul>1 2
  67. 68. Bokeh AHSL shader variation <ul><li>Max 3*2 3 = 24 variations </li></ul><ul><ul><li>Region (Front, Back, Both) </li></ul></ul><ul><ul><li>Tap count (odd, even) </li></ul></ul><ul><ul><li>Mask type </li></ul></ul><ul><ul><li>Eclipse </li></ul></ul>
  68. 69. Aperture shape <ul><li>F-Stop describes how wide the aperture is </li></ul><ul><li>The shape of aperture is… </li></ul><ul><ul><li>More circular when opened </li></ul></ul><ul><ul><li>Tends towards a polygonal shape as it is closed </li></ul></ul>F5.6 F8.0 F13 Fully opened
  69. 70. Variable bokeh shape <ul><li>Bokeh shape changes with F-stop </li></ul>100mm 1/30 F12.5 (our simulation) 100mm 1/250 F4.3 (our simulation) 28-200mm F3.5-5.6 (6 blades) (91taps)
  70. 71. Comparison with Real Photo Our simulation Real photo F2.8 F3.2 F3.5 F4 F5.6 A light source of 5mm in diameter at a distance of 3m from the camera. The focus distance is 1.5m, with 70-200mm F2.8 lens at 200mm
  71. 72. Comparison with Real Photo Our simulation Real photo F8 F11 F16 F22 F32
  72. 73. One more thing <ul><li>During the experiment, the difference in size between F2.8 and F3.5 in the real photo is smaller than the theoretical calculation </li></ul><ul><ul><li>Actually, bokeh at F2.8 is optically vignetted </li></ul></ul>F2.8 (without vignetting simulation) F2.8 F3.5 Real Photo F2.8 & F3.5
  73. 74. Optical Vignetting <ul><li>The phenomenon where light entering near the edge of the lens at an obtuse angle can’t reach the film because it is obstructed by the lens barrel </li></ul><ul><ul><li>The actual physical phenomenon is very complicated </li></ul></ul><ul><ul><li>Our implementation for lens structures is approximated rather than using real physical properties </li></ul></ul>Blue line indicates that ray is obstructed Film
  74. 75. Optical Vignetting <ul><li>Recognized by… </li></ul><ul><ul><li>The corners of the picture are darkened </li></ul></ul><ul><ul><li>“ Cat eye effect” (eclipsed bokeh) </li></ul></ul>24mm F2.8 24mm F5.6
  75. 76. Implementation <ul><li>Compute the attenuation curve and eclipsing ratio using a virtual lens database </li></ul>Our simulation Falloff illuminance from Cosine Fourth Law and optical vignetting
  76. 77. Struct LensParameter struct LensParameter { string szName[64]; u8 nAppertureAngleNumber; f32 fDesignedFilmSize; f32 fMinFStop; f32 fMaxFStop; f32 fFStopZoom; f32 fMinFocusDepth; f32 fMinProjectionDistance; f32 fMaxProjectionDistance; // ----------------- Vignetting Distance // || --------------- <-----------> // || ---------------- | // || || | || | // Entrance || Open Ap ||Vignetting|| | // Size || Size ||Size || | // || || | || | // || ---------------- | // || --------------- <-Frange Back -> // ----------------- ^Iris // <- Iris Distance -> // <------------ Entrance Distance ------------> f32 fEntranceDistance; f32 fEntranceSize; f32 fApertureDistance; f32 fOpenApertureSize;
  77. 78. View angle change <ul><li>The focal distance changes with the focal length in a real camera </li></ul><ul><ul><li>Means that changing the focal distance affects the view angle </li></ul></ul>Focus on this glass Easily recognize view angle change This lens can focus between 1.5m – 10m – Inf. Even with a 50cm focal distance change as shown in these pictures, recognizable view change appears. This change can be easily calculated with the lens equation.
  78. 79. Comparison of view angle When focusing on a closer point, focal length gets longer
  79. 80. Comparison of view angle
  80. 81. Theory <ul><li>If you want to change the focal distance (a) and don’t want to change the view angle (m), change the distance to the image plane (b) </li></ul><ul><ul><li>However the distance to the image plane can’t be moved because it’s based on camera design </li></ul></ul><ul><ul><ul><li>For some lenses, distance to the image plane can be “virtually” changed </li></ul></ul></ul><ul><ul><li>Therefore, the view angle changes when changing the focal distance </li></ul></ul>a b f Lens Formula a: distance to object b: distance to image plane f: focal length magnification (FOV)
  81. 82. Motion Blur 35mm 1/60sec F14
  82. 83. What is motion blur? <ul><li>Film is exposed to light while the shutter is open </li></ul><ul><ul><li>As the camera or subjects move during exposure, blur appears </li></ul></ul><ul><li>Dt ≠ Time value </li></ul><ul><li>Implementation is easy and produces high quality visuals </li></ul>
  83. 84. Camera motion blur <ul><li>Create a velocity map from the depth buffer using the different matrices of the camera across frames </li></ul>
  84. 85. Object Motion Blur <ul><li>Create a velocity map of the object in another pass </li></ul><ul><ul><li>Smear the velocity map based on each velocity vector </li></ul></ul>(1/64 reduced buffer)
  85. 86. Without motion blur
  86. 87. With motion blur
  87. 88. Tonemapping & filters
  88. 89. Glare filter <ul><li>Multiple Gaussian Filter </li></ul><ul><ul><li>Not physically-based </li></ul></ul><ul><ul><li>“ Log base” emphasis instead of “Threshold base” </li></ul></ul><ul><ul><li>“ Energy conservation” is implemented (optional) </li></ul></ul>+ =
  89. 90. Tonemapping curve <ul><li>Instead of using a specific algorithm, use specifications from films or C-MOS sensors to create the curve table </li></ul><ul><ul><li>The curve is compressed based on the log basis </li></ul></ul><ul><ul><ul><li>float u = saturate(log2(vInCol.r+1)/2.32); vOutCol.r = tex1D(s, u).r; </li></ul></ul></ul>
  90. 91. Print <ul><li>Negative films are developed in negative colors </li></ul><ul><ul><li>Negative film requires printing for correct colors </li></ul></ul><ul><ul><ul><li>Orange removal is also required </li></ul></ul></ul><ul><ul><li>Film duplication process was implemented </li></ul></ul>
  91. 92. Film profile comparison Original (used in the game) Company K, Reversal Company F, Reversal
  92. 93. Film profile comparison Original (used in the game) Company K, Reversal Company K, Negative
  93. 94. Film profile comparison Company F, Negative Company C, Digital
  94. 95. Film profile comparison Original (used in the game) Company K, Reversal No film profile (Reinhard)
  95. 96. Film profile comparison (all) Original K, Rev. K, Neg. F, Rev. F, Neg. C, Digital Reinhard
  96. 97. Film grain <ul><li>Developed a tool to create film grain based on a simulation </li></ul><ul><ul><li>“ Multiply ” blending </li></ul></ul>ISO 50 ISO 400
  97. 98. C-MOS noise <ul><li>“ Additive ” blending </li></ul><ul><li>Bayer pattern × White noise </li></ul><ul><ul><li>Considering color interpolation </li></ul></ul>
  98. 99. Results of grain Digital noise Film grain Enlarged and brightened in Photoshop (Digital camera profile)
  99. 100. Other simulations <ul><li>Auto Focus / Auto Exposure </li></ul><ul><ul><li>AF and AE are also implemented based on a real camera </li></ul></ul><ul><ul><ul><li>Professionally, AF and AE are essentially not used </li></ul></ul></ul><ul><ul><ul><li>Only used for special cases </li></ul></ul></ul><ul><ul><ul><ul><li>AE was used for in-game exposure control </li></ul></ul></ul></ul>
  100. 101. Custom Filters <ul><li>Combination of matrix and texture fetch </li></ul><ul><ul><li>Easy to edit in Maya </li></ul></ul><ul><ul><ul><li>Brightness & Contrast </li></ul></ul></ul><ul><ul><ul><li>Tone curve </li></ul></ul></ul><ul><ul><ul><li>Color temperature </li></ul></ul></ul><ul><ul><ul><li>Gamma </li></ul></ul></ul><ul><ul><ul><li>Chroma correction </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul>
  101. 102. Dither <ul><li>Keep color information during dither process when processing gamma correction </li></ul><ul><ul><li>HDR should have more than 8 bits of information </li></ul></ul><ul><ul><li>Fp32 in the shader </li></ul></ul><ul><ul><li>Disadvantage of noisy image caused by dithering </li></ul></ul><ul><ul><li>Probably make visuals worse with dithering and the display’s visual processing </li></ul></ul><ul><ul><li>Can use information of more than 8 bits with DeepColor </li></ul></ul><ul><ul><ul><li>In case of using analog interfaces, depends on A/D converter in the output device </li></ul></ul></ul>Emphasized dithering
  102. 103. Tonemapping flow Gather Buffers (DOF, Glare) Pre matrix transform (Spectral film characteristic) Added Noise (Film grain, C-MOS noise ) Tex fetched transform (Film curve, Tone curve, gamma …) Matrix transform (chroma correction, …) Dithering Output to LDR buffer Input HDR buffers
  103. 104. Post-process AHSL shader variations <ul><li>Max 2 10 = 1024 variations </li></ul><ul><ul><li>Focus blur enabled (+ vignetting) </li></ul></ul><ul><ul><li>Textured tonemap </li></ul></ul><ul><ul><li>Pre matrix transform </li></ul></ul><ul><ul><li>Matrix transform </li></ul></ul><ul><ul><li>Film grain (or CCD noise) </li></ul></ul><ul><ul><li>Dithering </li></ul></ul><ul><ul><li>Depth buffer (input, output) </li></ul></ul>
  104. 105. Results without post-process with post-process (high quality) with post-process (original) 85mm F6.3
  105. 106. Results without post-process with post-process (high quality) with post-process (original) 50mm F2.8
  106. 107. Results without post-process with post-process (high quality) with post-process (original) 24-70mm F12 58mm
  107. 108. Problem <ul><li>Artists couldn’t handle many parameters </li></ul><ul><ul><li>The parameters are based on real camera parameters </li></ul></ul><ul><ul><ul><li>F-stop </li></ul></ul></ul><ul><ul><ul><li>Time Value </li></ul></ul></ul><ul><ul><ul><li>Focal distance </li></ul></ul></ul><ul><ul><ul><li>Lens choice </li></ul></ul></ul><ul><ul><ul><li>Film or sensor choice </li></ul></ul></ul><ul><ul><ul><li>Sensitivity </li></ul></ul></ul><ul><ul><ul><li>Etc… </li></ul></ul></ul><ul><ul><li>However, in order to control quality, a lot of other (non-physical) parameters are implemented </li></ul></ul><ul><ul><ul><li>Templates were implemented </li></ul></ul></ul>
  108. 109. Finding a solution <ul><li>In-house seminars were held </li></ul><ul><ul><li>Although it was difficult to cover all parameters </li></ul></ul><ul><ul><li>However, for Star Ocean: TLH </li></ul></ul><ul><ul><ul><li>Some simulations weren’t used </li></ul></ul></ul><ul><ul><ul><li>Some quality parameters weren’t set efficiently </li></ul></ul></ul><ul><ul><ul><ul><li>The quality got worse, less than needed in some cases </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Lower quality than prepared templates for the performance </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Although, no gain performance-wise </li></ul></ul></ul></ul></ul>Unnatural bokeh caused by too few taps
  109. 110. Summary <ul><li>Camera parameters affect each other </li></ul><ul><ul><li>Large F-stop number </li></ul></ul><ul><ul><ul><li>Long time value or high sensitivity </li></ul></ul></ul><ul><ul><ul><ul><li>Long motion blur or coarser grain </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Small bokeh (Focus blur) </li></ul></ul></ul></ul><ul><ul><li>Small F-stop number </li></ul></ul><ul><ul><ul><li>Short time value or low sensitivity </li></ul></ul></ul><ul><ul><ul><ul><li>Short motion blur or finer grain </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Large bokeh </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Optical Vignetting </li></ul></ul></ul></ul><ul><ul><li>Changing focal distance </li></ul></ul><ul><ul><ul><li>Affects the angle of view and the size of bokeh at the same time </li></ul></ul></ul><ul><ul><li>Various optical phenomena occur due to limitations in lens structure and camera </li></ul></ul>
  110. 111. More simulations <ul><li>More physically based simulations </li></ul><ul><ul><li>Diffraction simulation </li></ul></ul><ul><ul><ul><li>Rather than simple glare and stars </li></ul></ul></ul><ul><ul><li>Aberration simulation </li></ul></ul><ul><ul><li>Ghost simulation </li></ul></ul><ul><ul><ul><li>Rather than image based one </li></ul></ul></ul><ul><ul><li>Scatter-based implementation </li></ul></ul>
  111. 112. Acknowledgements <ul><li>Copyrights </li></ul><ul><ul><li>© 1996 tri-Ace Ltd./MEIMU/SQUARE ENIX CO., LTD. All Rights Reserved. </li></ul></ul><ul><ul><li>© 1998 tri-Ace Inc./LINKS/Minato Koio/SQUARE ENIX CO., LTD. All Rights Reserved. </li></ul></ul><ul><ul><li>© 2001 tri-Ace Inc./Mayumi Azuma/SQUARE ENIX CO., LTD. All Rights Reserved. </li></ul></ul><ul><ul><li>© 2003 tri-Ace Inc./SQUARE ENIX CO., LTD. All Rights Reserved. </li></ul></ul><ul><ul><li>© 2004 tri-Ace Inc./SQUARE ENIX CO., LTD. All Rights Reserved. </li></ul></ul><ul><ul><li>© 2009 SQUARE ENIX CO., LTD. All Rights Reserved. Developed by tri-Ace Inc. </li></ul></ul>
  112. 113. Acknowledgements <ul><li>Assistance for slides </li></ul><ul><ul><li>Tatsuya Shoji, R&D </li></ul></ul><ul><ul><li>Daisuke Sugiura, R&D </li></ul></ul><ul><ul><li>Paul Caristino, R&D </li></ul></ul><ul><ul><li>Chuk Tang, R&D </li></ul></ul><ul><ul><li>Masaki Kawase, Silicon Studio Corporation </li></ul></ul><ul><ul><li>Many screenshots and video from Star Ocean: TLH team </li></ul></ul>
  113. 114. Questions? <ul><li>These slides can be found on </li></ul><ul><ul><li> </li></ul></ul><ul><ul><ul><li>More details about flexible shader management and camera simulation are described in the other slides </li></ul></ul></ul><ul><ul><ul><ul><li>Unfortunately, written in Japanese  </li></ul></ul></ul></ul><ul><ul><li>Questions are welcome </li></ul></ul><ul><ul><ul><li>[email_address] </li></ul></ul></ul>