Mobile optimization techniques
•Download as PPTX, PDF•
2 likes•1,181 views
Mobile optimization techniques presented by Nordeus at Unite 2015 conference.
Recommended
Recommended
More Related Content
Viewers also liked
Similar to Mobile optimization techniques
Similar to Mobile optimization techniques (20)
Recently uploaded
Recently uploaded (20)
Mobile optimization techniques
- 1. MOBILE OPTIMISATION TECHNIQUES TOMISLAV RAKIĆ SRĐA ŠTETIĆ-KOZIĆ Unite Europe 2015
- 2. INTRODUCTION ● Nordeus ● Belgrade based ● Developer of Top Eleven MOBILE OPTIMIZATION TECHNIQUES
- 3. ● Scene loading time ● Draw calls ● Batching ● Instancing MOBILE OPTIMIZATION TECHNIQUES OVERVIEW
- 4. ● Our flagship product ● 110 million registered users ● Global ● 5 years old MOBILE OPTIMIZATION TECHNIQUES
- 5. MOBILE OPTIMIZATION TECHNIQUES ● One codebase ● GUI heavy game ● Fragmented mobile market ● Custom GUI framework TIME TO UPDATE: T11 2015
- 9. MOBILE OPTIMIZATION TECHNIQUES ● Unity Profiler ● System.Stopwatch ● norde.us/unite2015 PROFILING
- 10. MOBILE OPTIMIZATION TECHNIQUES ● Discoveries ○ private vs public fields ○ null fields ○ Hierarchy overhead ○ Fixed costs ● All fixed in Unity 5! DESERIALIZATION
- 11. MOBILE OPTIMIZATION TECHNIQUES ● Solutions ○ Custom binary format ○ Code generation ● Almost there... DESERIALIZATION
- 12. MOBILE OPTIMIZATION TECHNIQUES ● Well known problem ● Instantiation is slow ● Use GameObject pools ● norde.us/unite2015 INSTANTIATION
- 15. Transform setup order matters TRANSFORM HIERARCHY MOBILE OPTIMIZATION TECHNIQUES 10000 msBOTTOM - UP TOP - DOWN 200 ms t [ms]
- 20. MOBILE OPTIMIZATION TECHNIQUES ● You can have pointers in C# ● Create smcs.rsp ○ -unsafe ● Write unsafe code ● norde.us/unite2015 ARRAY RESIZING
- 25. MOBILE OPTIMIZATION TECHNIQUES ● Expensive on mobile devices ● The future ○ Metal ○ OpenGL AEP ○ Vulkan ● The present DRAW CALLS
- 26. MOBILE OPTIMIZATION TECHNIQUES ● Huge scene ● Lots of vertices ● Fast moving camera ● Lots of objects DRAW CALLS
- 27. ● Atlasing ● NVIDIA - Improve Batching Using Texture Atlases STATIC BATCHING MOBILE OPTIMIZATION TECHNIQUES
- 28. MOBILE OPTIMIZATION TECHNIQUES ● Tiling ○ Vertex approach STATIC BATCHING
- 30. TILING ● Tiling ○ Vertex approach ○ Fragment approach MOBILE OPTIMIZATION TECHNIQUES
- 32. MOBILE OPTIMIZATION TECHNIQUES ● Script - Batch meshes ● ??? ● Shader - Read tiling data TILING
- 33. MOBILE OPTIMIZATION TECHNIQUES ● Color ● Normals ● UV0..3 ● Tangents VERTEX ATTRIBUTES
- 34. MOBILE OPTIMIZATION TECHNIQUES ● Script - Batch meshes ● Script - Bake tiling data to tangents ● Shader - Read tiling data TILING
- 40. MOBILE OPTIMIZATION TECHNIQUES ● Mipmaps problem ● Obvious solution - ddx, ddy ● GL_EXT_shader_texture_lod ○ Supported on iOS ○ “Supported” on Mali TILING
- 41. MOBILE OPTIMIZATION TECHNIQUES Try #1 - Expand texture to the edge of atlas TILING
- 42. Try #2 - Mirror texture reads TILING MOBILE OPTIMIZATION TECHNIQUES
- 43. Try #3 - Sort textures by color TILING MOBILE OPTIMIZATION TECHNIQUES
- 44. MOBILE OPTIMIZATION TECHNIQUES ● Environments look nicer ● Less draw calls than before ● Added complexity in fragment shader TILING
- 45. MOBILE OPTIMIZATION TECHNIQUES ● Scenario - Traffic MOVING OBJECTS
- 47. MOBILE OPTIMIZATION TECHNIQUES ● Static batching? ● Unity’s dynamic batching? ● OpenGL ES 3.0 - Instancing ● OpenGL ES 2.0? MOVING OBJECTS
- 48. MOBILE OPTIMIZATION TECHNIQUES ● OpenGL ES 3.0 ○ Instance ID ○ Draw Elements Instanced ● Instance ID - tangents ● Draw elements Instanced - custom mesh ● Shader for each N INSTANCING
- 50. MOBILE OPTIMIZATION TECHNIQUES ● Vertex shader complexity ● Mesh memory ● norde.us/unite2015 INSTANCING
- 52. BILLBOARDS ● Pivot ○ Colors? ○ Normals! ○ #pragma glsl_no_auto_normalization ● Offset ○ Vertex position ● Transform matrix in shader ○ Rotate vertices MOBILE OPTIMIZATION TECHNIQUES
- 53. MOBILE OPTIMIZATION TECHNIQUES ● Tradeoff ○ Vertex shader complexity ○ Doesn’t work with transparency ● norde.us/unite2015 BILLBOARDS
- 54. MOBILE OPTIMIZATION TECHNIQUES ● Measure ● Improve ● No silver bullet CONCLUSION
- 55. MOBILE OPTIMIZATION TECHNIQUES ● tomislavr@nordeus.eu ● srdjas@nordeus.eu ● norde.us/unite2015 Q&A
Editor's Notes
- We are Tomislav, and Srdja, developers from Nordeus. Company based in Belgrade, with offices around the world in: London, Dublin, Skopje and San Francisco. Nordeus is the developer of Top Eleven, most popular MMO football management game.
- Today we share our experiences on common poblems War story from the trenches of Top Eleven product development, where we had problems with scene loading times aong other things Also had problems with draw calls. Who doesn’t have a problem with draw calls on mobile. Then we will share some ideas on how you can improve static batching with tiling. And we will present a technique we call poor man's instancing.
- In our case we have Top Eleven. It’s a game about football (or soccer, for guys from the other continent), you become a manager and you battle others if you like football you will probably recognize the face from the picture, it’s Jose Mourinho. it’s a cross platform game. As football (or soccer) is played in *every* country, so is our game. We even had logins from Antarctica. What does that mean. . . . Having success in emerging markets and being 5 years old led to the fact that 25% of users have old phones, whose performance is just terrible. Which mean we have to be careful when it comes to performance Organic growth and availability are big drivers for our product It’s already been around for 5 years. 3 years ago switch to unity.
- At one point we decided to update the app. To improve gameplay, and to improve visualls drastically. Biiiig update. Besides improving player experience one of the production goals was to unify the codebase, and to have one team instead of ios, android and flash teams, all implementing the same features. But Top Eleven is a GUI heavy game, and especially from our own android teams experience we know how mobile market is fragmented. There are lots of different resolutions, different DPI, different screens out there. This was before 4.6 so write our own GUI framework to minimize the UI production time and to improve quality for TE and future games.
- We chose Unity because it would definitely help us unify the codebase, but most importantly because it is flexible enough. Like John said in the keynote unity is a flexible tool We actually started transitioning to Unity 3 years ago with basics, we grew, today full unity. Last year we had implemented our UI framwork. philosophically it is similar to Unity’s new UI system, but coming from Flash programming background we expanded it with some concepts like Groups, and Layouts from flash. Here you can see it in action, these are screenshots from two different screens with different DPIs but the UI is automagically layouted.
- We were prototyping gameplay from our most complex scene, the Squad scene. You have many players on a squad. List many rows, many columns, each column has many elements, and all of that has to be aligned and layouted and there are multiple screens in this scene. I’m trying to justify that there are as much as 4k game objects on this scene, maybe pictures don’t do the job. We coulnd go below 4k, and 15k monob. But we had a problem.
- This is an early alpha build. Managers looking at this were asking should we kill this project. We had no idea why this was happening. We suspected id had something to do with 4k gos.
- Unity’s profiler first tool you should use. Works in dev mode on devices. Sometimes you need to get precise measurements during runtime on devices, most rudimentary way would be to use System.stopwatch to measure milisecs. We needed precise measurements, because were making really small steps, 10ms at a time. We had days where same code would give different results, and it was not a problem in code, we were just measuring wrong things. Share, we would love to. This is a tech talk but we wont show much code on slides, we will share examples and tools, because we love this community.
- But you don’t have to worry about these things Unity 5 fixed all of these problems. Maybe useful if you use old... Even though it was better, it wasn’t in a blink of an eye. We needed the best thing possible for those lowes end phones.
- What we did to mitigate these problems is write GUI metadata to a special binary format. Pipeline: Design -> build -> generate code -> embed code to all monobehaviours -> write our supre tight scene format -> when run, read scene. And then you have to instantiate all those objects
- Instantiation of new objects during runtime is slow in general Instantiation is slow because you are allocating on heap. Then when you delete some objects, garbage collector might trigger at one point, freezing the game. Pool is a well know creational pattern. Rather than allocating and destroying them on demand, you take an object from the pool, activate it. When you no longer need it you deactivate it and return it to the pool. I am sure that there are different game object pool implementations, but on this link you can find a very neat generic pool and game object pool implementation. Top Eleven uses a pool of 5k game objects, because we know that’s our maximum. So we have read the scene, instantiated all objects, now we need to recreate the transform hierarchy of objects to reconstruct the scene.
- If you are (re-)constructing hierarchy from code and you are using a simple recursive algorithm, you will be doin’ it bottoms up. 3 to 2. subtree gets attached to 1.
- Or you can do it town down. Where you would first attach 2 to 1, 3 to 2, and so on.
- Worst case scenario 1k game objects. And it isn’t linear. This was unexpected. Iako oba resenja deluju naizgled dobro. Anegdota. I don’t know what to make of this, or why this happens, but I avoid reparenting transforms bottom's up from now on.
- Now we have recontructed scene. We have read alll layouting metadata, code is running. To show anything on screen, we use layouting metadata, we create UI meshes on the fly. If something is moving we recreate meshes every frame.
- Mesh.vertices requires array type. You need to pass exact number of vertices you want to render. If you create a new array every frame, you get ton’s of garbage.
- This is how arrays are laid out in the memory. In the header we have the length of an array.
- If we only had pointers in C#, we would be able to rewrite that size=4 using pointer arithmetics, and thus be able to resize arrays free of cost.
- Share VaryingList, our implementation of a list that uses this concept, is able to be resized free of cost, and can behave like an array, and it’s very useful for procedural generation Super useful for procedurally generated meshes.
- Just to refresh you memory we started with this.
- We released top eleven 2015 unity update in January this year, and we have been featured several times on the stores so far.
- So far we have talked about top eleven as an UI heavy game, but it features a fully 3D ground. It has had similar requirements as our GUI, and that is to be supported on lowest end of devices, to be available to our huge audience. Developing in 3D brought a different set of challenges compared to the 2D GUI development which also need to be solved. Let’s welcome srdja who will share his experience on 3d graphics optimizations.
- For every object the GPU has to draw, you have to call certain API functions to set up everything. Those API functions very often have to, in turn, call the graphics driver. The functions themselves have CPU overhead and some drivers add even more overhead on top of that. Since mobile phone CPUs are not that fast, this causes the CPU to become a bottleneck from simply drawing too many objects on the screen. Just to illustrate how bad the problem is, going over 150 draw calls with no game logic running can put you below 60FPS on Adreno devices, which are terrible at handling draw calls. And 150 draw calls is not even that many if you’re trying to make, let’s say an open world game. Lower level APIs that have considerably less overhead are currently being developed, such as Metal for iOS, OpenGL AEP for Android and Vulkan. That’s what the future holds, but we’re currently developing in the present and only Metal is somewhat enough widespread to be considered. Old devices that don’t support these technologies will take a while to die off, so we have to solve the problems with draw calls by ourselves right now.
- The scenario we have in our new game is Huge, open scene that was around 5x5 km in size Lots of vertices Fast moving camera Lots of objects All of these factors combine to produce a lot of draw calls. I’m mentioning fast moving camera because while not a problem by itself, Unity 4 static batching can reduce performance when your camera moves very fast. Luckily that’s fixed by Unity 5, but if you’re still using 4, you should be aware of this.
- One of the techniques used to reduce the number of draw calls is static batching. Static batching is used to combine static objects that share the same material into one big object and thus make it one draw call. Since one prerequisite is for the material to be shared between objects, you want for as many objects as possible to use the same material. This is where another technique comes in, called Atlasing. Atlasing is the process of putting a bunch of textures into one texture called the atlas. As you can see on the picture, this is a texture atlas made up out of smaller textures. This allows you to create a single material with the atlas as its texture, which in turn allows you to batch more objects together, reducing the number of draw calls even further. There’s a great whitepaper by Nvidia that you can check out if you want to know more.
- In our game, we have a lot of urban environments and to make them look good you have to use texture tiling extensively. Tiling is a technique where you repeat a texture several times side by side. This is done because a lot of objects cover wide areas and creating a texture for the entire surface would take up a lot of memory. With tiling, you can create a smaller texture and repeat it on the entire surface. A good example of a material that tiles very well is asphalt. Tiling does not work in combination with atlasing and static batching by default and if you need it, you’ll have to implement it yourself. This and the problems with the performance due to the camera were our primary motivations for doing a custom static batching technique. There are two possible approaches when doing tiling with atlases. The first one is by adding additional vertices to the models.
- A single quad would be broken up into, in the case in the picture, 3 separate quads, and each quad’s U coordinate would go from 0 to 1. Effectively, this object is now tiled 3 times. The problem with this approach is that it usually has to be done manually by artists themselves, it can add a lot of vertices and it’s really hard to do for irregular shapes. Due to these downsides, you don’t want to use this as a general purpose solution.
- You can also calculate the UV coordinates in the fragment shader and this is how you would usually do tiling without atlases.
- As you can see on the picture, we have one big quad, but the U coordinate is calculated by the fragment shader using the parameters set in the material. These parameters might be familiar to you, they are called scale and offset in the Unity editor and they are specific to a material and that’s the main problem. Since we’re dealing with atlases, all the objects share one material but each texture in the atlas needs separate parameters which means that we can’t use them.
- The first step in the process would be to batch the meshes together. The last step is for the shader to read the tiling data and draw the mesh. And now we have to figure out how to pass the data from our script to the shader.
- Vertex attributes to the rescue! They are per vertex so they solve the problem material parameters had. What you should know about vertex attributes is that you can encode any type of that into any of them. Vertex colors are usually used to make the scene look more diverse. Normals are used for lighting Unity 4 had only 2 UV channels and they are usually used for texture mapping and lightmapping. With Unity 5 you have 2 additional UV channels available so you can use those. We had some precision issues on iOS devices when storing large numbers in UV channels, so be mindful of that. And lastly, tangents! In our case we didn’t use tangents at all so they were a natural place to put our tiling data.
- And now we have everything setup and everything should work properly, right?
- Not quite, since our textures could be looked at from both very far and very close, we needed mipmaps for the atlas and that’s where a problem appeared. Picture on the left is how our shader looks like when mipmaps for the atlas are disabled and on the right is a picture with them enabled. Notice the faint horizontal lines that appeared.
- Horizontal line on the picture on the right. This happens because at the exact point where we get to the end of the texture and need to wrap back to the beginning, a lower mip level is used.
- Not quite, since our textures could be looked at from both very far and very close, we needed mipmaps for the atlas and that’s where a problem appeared. Picture on the left is how our shader looks like when mipmaps for the atlas are disabled and on the right is a picture with them enabled. Notice the faint horizontal lines that appeared.
- To better illustrate, look at this diagram. This is a texture in UV space, and we need to wrap back to 0.1 after we get to 0.4 to tile it. Green line is the coordinate of the previously read pixel and the red line is the coordinate of the currently read pixel. Notice how their difference is only 0.01. This difference is important because that is how the GPU decides what mip level to use. This difference indicates to the GPU how far away the texture is. Lower numbers means that the texture is close and thus higher mip levels will be used.. Let’s take a look at our problem case now.
- The first pixel we read is at 0.4. The second is at 0.1. Their difference is now 0.3 which is much larger than in the previous case. Because of this, the GPU will choose a much lower mip level, because it believes that the texture is very far away from the observer.
- Luckily, this is a solved problem. Calculating the actual distance is relatively easy and there’s a way to force the GPU to use them. But this is where I have some good news and some bad news for you. The good news is that this functionality is supported in an OpenGL extension. The bad news is that i t’s an extension so manufacturers don’t have to support it, and even if they do, they might not do it correctly. From what we’ve seen, it’s supported on all modern iOS devices and it’s “supported” on some Mali devices. I used air quotes here because on a Galaxy S3, we had strange graphical corruption if we tried using it even though it is supported. This is not good, we need a way to solve this problem on all devices. You could try a couple of different things.
- In the red square on the picture is the texture we want tile. It’s been copied multiple times, all the way to the right, to the very edge of the atlas. This way, when we get to the end and need to repeat a tile, the GPU will do the wrapping automatically, and it does not suffer from the same problem that we had. Unfortunately, this only allows for horizontal wrapping. If we wanted vertical wrapping as well, it would have to be expanded vertically. This wastes a ton of space on the textures, and in turn memory, so this approach won’t work.
- What if instead of jumping from 1 straight to 0, we do it gradually, over the course of one tile? This way, every second tile would be mirrored, like in the picture. This completely solved the issue with lines, but created a bigger problem. Human eye is extremely good at noticing symmetry, and most textures looked completely uncanny when tiled like this.
- What if, instead of eliminating the problem, we masked it? Look inside the white square for example. Notice how all the textures in there are blue-ish. If we sort them by color, when they get minimized for lower mip levels, they will remain approximately the same color which makes the lines almost or completely invisible in most cases. On the few objects where this didn’t work, we simply didn’t use batching at all. This added a negligible amount of draw calls, so it was worth it.
- This is a lot of effort to go through, so let’s see what the end benefits are. There are less draw calls than before, which makes it easier to hit performance targets or even add more objects to the scene to make it look nicer. Since you can use tiling on everything without fear of additional draw calls, your objects look nicer as well. This also solved the performance issue with the fast moving camera in Unity 4. One trade off, that you have to make here is the added complexity in the fragment shader. From our testing, you lose about 5% performance if you’re GPU bound. Now that we have solved the problem with static objects, what about the objects that are moving?
- This picture is a pretty good representation of the problem we had. A bunch of cars, each with 4 wheels and the wheels needed to move and rotate so it doesn’t look uncanny. Let’s take a look at another example as well.
- Look at this monstrosity. We usually target around 150 draw calls at most so our games would work on lower end devices and this thing would take over 10% of the budget.
- How to solve this problem? Static batching can’t be used since objects need to move each frame. Unity’s dynamic batching has very low limits about the number of vertices that the objects that will be batched can have, due to performance consideration. Also, you can’t actually control what will be batched and what won’t, so the use case is very limited and it won’t work here. Luckily, there’s a technique that can help us here and it’s called Instancing. Instancing allows you to render multiple copies of the same mesh with different parameters, such as position and rotation in a single draw call. It is usually used for trees, grass or any repeatable object that does not appear repetitive when rendered like this. This would be perfect for our use case, and it’s available in OpenGL ES 3.0. But what about OpenGL ES 2.0? According to the most recent data, ⅔ of Android devices on the market are OpenGL ES 2.0 only. We don’t want to exclude ⅔ of the market, unless absolutely necessary, and I’m guessing neither do you. So we started thinking, can we somehow replicate this technique in OpenGL ES 2.0?
- Instancing is implemented using two important parts 1. InstanceID variable available in the vertex shader that indicates what instance the current vertex belongs to. 2. Draw Elements Instanced which accepts the mesh, number of copies and transformation matrices for each copy and draws them in a single draw call. We need to replicate both of those. Similarly to static batching we can bake instance id in the vertices themselves by putting it into tangents. DrawElementsInstanced is a little bit harder. You can do it by creating a custom mesh at runtime which contains N copies of the original mesh where N is the number of objects that should be drawn. Other than those two you need shader support as well, so you have to write a shader that accepts transformation matrices and that draws the custom mesh.
- So, the steps of drawing 4 wheels of a car in a single draw call are: Create a custom mesh that contains 4 copies of the wheel Bake IDs into tangents in the new mesh These two steps are only done once. The following two are done each frame: Pass the transformation matrices for each individual wheel into the shader. Draw the custom mesh using that shader. Sum the bounding box of all objects, so Unity knows whether it should draw them or not. The shader will according to the ID stored in tangents choose the correct transformation matrix. You can use dummy game objects and get the transformation matrices from their transforms.
- Vertex shader is a bit more complex due to the big switch. Mesh is larger due to the N * original meshes
- Explain what billboards are, what they are used for (mostly foliage), similar problem to static batching. Static batching can’t be used due to the required rotation to the camera. If we rotate each object individually, we’ll end up with a lot of draw calls. Extensions of the previous method (instancing). Static batching doesn’t work – would batch in world positions (messes up normals and tangents)
- Transform matrix is constructed in the vertex shader. Pivot stored in normal, vertex position is offset from pivot. Normals – pivot Vertex position – offsets from pivot Transform matrix – constructed in shader because objects are batched in a single mesh
- Works with opaque and cutout objects. Transparency doesn’t work due to sorting. Pribilizavamo se kraju, pozvao bih Tomislava da sumira
- We told you a story on how we almost never released TE. But we saved it, because we got good at measuring. Srdja told you how you can improve some features No silver bullet.