Presentation given to CocoaHeads Berlin May 11 2023

1. Georgi Nikolov 11.05.2023
Drawing with Metal
Insights after years of web development with WebGL

2. • From Sofia, Bulgaria
• Living in Berlin for about 5 years now
• Working as a freelance developer
• Background is traditional web development
• True passion is computer graphics, animation and experimental UIs

4. What is Metal?
• Low level access to the GPU on Apple Platforms
• Graphics rendering
• Parallel computing
• Underlying framework for all of Apple graphics and game APIs
• SpriteKit
• CoreGraphics
• CoreAnimation
• More and more games for Apple platforms support it

5. What is Metal?
• Part of the modern pipeline based rendering APIs (in fact the first one)
• Metal
• Vulkan
• DirectX 12
• WebGPU
• As low level as you can get on Apple hardware, extremely efficient
• Still relatively easy to pick up and be productive with
• Easily integrable into SpriteKit, SceneKit, UIKit, SwiftUI

6. What is Metal?
• Easily integrable with Model I/O
• Importing and exporting 3D assets
• Loading model animations, textures, etc
• Has C++ interface metal-cpp
• Different Apple GPU families have different capabilities
• Need to check at runtime if a given shiny feature is present
• If not, build a workaround for it

7. Anatomy of a Metal app
1. Create a CAMetalLayer to render its pixel contents programmatically via Metal
2. Get a reference to a MTLDevice that is a direct link to the device GPU
3. Create a vertex buffer holding the data you want to render
struct Vertex {
float2 position;
float3 colorRGB;
};
let vertices: [Vertex] = [
Vertex(position: float2(-0.5, 0.5), colorRGB: float3(1, 0, 0)),
Vertex(position: float2(0.5, 0.5), colorRGB: float3(0, 1, 0)),
Vertex(position: float2(0, -0.5), colorRGB: float3(0, 0, 1)),
]

8. Anatomy of a Metal app
4. Write your shaders to be ran on the GPU
1. Vertex shader to process each vertex in parallel
2. Fragment shader to process each pixel in parallel
3. Written as small programs in Metal Shading Language
1. Based on C++14
5. Compile your shaders and rendering settings into MTLRenderPipelineState
6. Set your viewport size
7. Set your newly created MTLRenderPipelineState as active

9. Anatomy of a Metal app
8. Send argument data to your vertex shader on the GPU
9. Submit a drawing command to the GPU
1. Primitive mode - point, lines, triangles
2. How many vertices to render

10. Differences with WebGL
1. Significantly faster
1. Made for Apple hardware
1. Offline resource creation and compilation
2. System memory shared between CPU and GPU
3. Faster resource loading
2. Lower level
3. Compute / tessellation shaders, ray tracing, MetalFX upscaling
4. Incredibly better tooling and debugging experience in Xcode

11. Similarities with WebGL
1. The maths
2. GPUs excel at 3 things, regardless of API used:
1. Floating point math
2. Primitive rasterisation
3. Textures blending
3. Apple devices use Metal behind the scenes to render WebGL

13. App architecture
1. Very thin SwiftUI layer
1. Compromised mostly of text and buttons
2. Metal view
1. Drawing the demo graphics and running the animations
2. Capturing user input such as taps and dragging
These two layers stay synced using published properties via Combine
ObservableObject which acts as aa central dispatcher

14. App architecture
class Options: ObservableObject {
@Published var activeProjectName: String = WelcomeScreen.SCREEN_NAME
}

15. Point light shadows demo
Cube shadows with depth cubemaps in a single draw via Metal layer selection
https://learnopengl.com/

16. Point light shadows demo
Different spheres, same underlying shaders: Metal function constants
constant bool is_sphere_back_side [[function_constant(0)]];
constant bool is_shaded_and_shadowed [[function_constant(1)]];
constant bool is_cut_off_alpha [[function_constant(2)]];
fragment float4 fragment_main() {
// ...
if (is_cut_off_alpha) {
float a = computeOpacity(in.uv);
if (a < 0.5) {
discard_fragment();
}
}
}

17. Xcode tooling & debugging
1. Incredibly nice Metal Debugger
1. Geometry viewer
2. Shader debugger
3. Render frame capture
4. GPU timeline
2. Texture creation
1. Different compression formats just work
2. Offline mips generation

18. Swift vs JS
1. More expressive syntax with more constructs
2. Statically typed
3. Proper OOP as opposed to prototypical inheritance
4. Protocols and extensions

19. References and readings

20. georgi-nikolov.com