Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
GPU PROGRAMMING WITH
GPUIMAGE AND METAL
JANIE CLAYTON-HASZ
What is a GPU?
A Graphics Processing Unit (GPU) is a small super
computer that does one thing really really well. That
one...
What is Parallel Computing
The default processes in a project is serialized
computing. One instruction is processed at a t...
But I Have Concurrency!
Concurrency is about dealing with a lot of things
at once.

Parallelism is about doing lots of thi...
Shader Basics
Shaders are the programs that determine what
gets displayed on your screen.

Shaders determine shapes, color...
GRAPHICS ON IOS DEVICES
There are many levels of abstraction
for graphics on iOS.
!
Some frameworks are more
abstracted than others.
UIKit
Sprite Kit
Core Animation/Graphics
OpenGL ES/GLKit
A BRIEF HISTORY OF TIME,
UH, OPENGL…
OpenGL Origins
First released in 1992

Was an attempt to formalize a 3D graphic
specification across platforms
Problems with OpenGL
Was created back when GPUs were not very
powerful and existed on external graphics cards
that could b...
Creation of OpenGL ES
ES: Embedded Systems

Wanted to strip out all of the dead code from
OpenGL

Was specifically tailored...
We don’t need a dozen turtles that all do the same thing
OpenGL ES Specifics
Streamlined version of OpenGL

Everything you can do in OpenGL ES can directly
be ported to OpenGL

Bas...
CPU VS GPU PROGRAMMING
CPU Expensive Tasks
Sending hardware commands to the GPU
(Changing State Vectors)

Confirming that API usage is valid

Comp...
What the Heck is “State”??
Try to envision a client-server process. Instead of
your program sending an instruction over th...
What the Heck is “State”??
For example, in most Twitter client applications the
client batches 20 or more Tweets in one ca...
Fixed Function Pipeline
Present in OpenGL ES 1.1

Shaders were hard-coded into OpenGL

Easier to use, but were very limited
Programmable Pipeline
Introduced in OpenGL ES 2.0

Shaders are now the responsibility of the
programmer

Harder to do, but...
GLSL SHADER BUILDING
GLSL
OpenGL Shading Language (GLSL)

Introduced in OpenGL 2.0 in 2004

C-like language for building shaders, which are
sma...
Two Flavors of GLSL
Shaders
Vertex

Fragment
Vertex Shaders
Vertex Shaders
The Vertex Shader would record the vertices of the
star (which would be broken down into a series of
triang...
Fragment Shaders
Fragment Shaders
Fragment Shaders determine what pixels receive
which color.

If you look carefully at the star, there are...
Inputs and Outputs
Uniforms

Attributes

Varyings
Uniforms
Values that don’t change during rendering

Read-Only

Available in both Vertex and Fragment Shaders
Attributes
Vertex Shader only

Input values that change with every vertex, like
their position, color, and texture coordin...
Varyings
Used to pass data between the Vertex and the
Fragment Shaders

Read-only in the Fragment Shader

Read-Write in th...
GPU IMAGE
Creating GPUImage
GPUImage dates back to iOS 5.

Unlike Core Image (at the time), GPUImage utilized
shaders more efficiently...
Why is GPUImage so
Efficient?
OpenGL ES tasks must be performed on one
thread

Many people utilize locks to manage the thre...
Demo
METAL: THE NEW KID IN
TOWN
What does Metal Promise?
Deep hardware integration between Apple chips
and Apple frameworks

General Purpose GPU programmi...
What Specifically are the
CPU Expensive Tasks?
Compiling Shaders

Validating State

Start Work on the GPU
Life Before Metal
All three of these expensive tasks were done on
each and every single draw call.

All of these tasks don...
Life After Metal
Compiling Shaders: Now done when the
applications builds

Validating State: Now done when the content
loa...
Why is This Important?
Before Metal, you would have to balance CPU
time with GPU time. Tasks were so expensive that
the GP...
Where Does Metal Help
You?
Metal helps you when you have a lot of objects
that need to work independently of one another.
...
ZEN GARDEN DEMO
EPIC GAMES
Secret Sauce
Okay, so one of the big, obscure black boxes in
this scheme is the promise of deep software/
hardware integra...
IS THERE ANY POINT IN LEARNING
OPENGL ES ANYMORE?
Yes, absolutely. Metal’s API is very similar to OpenGL ES.
!
It will take a while for everyone to transition over
to devic...
Well crap.
Think Different.
The End
Gpu Programming With GPUImage and Metal
Gpu Programming With GPUImage and Metal
Gpu Programming With GPUImage and Metal
Upcoming SlideShare
Loading in …5
×

Gpu Programming With GPUImage and Metal

4,391 views

Published on

These are the slides from the talk I did on GPU Programming for CocoaConf Columbus 2014

Published in: Technology

Gpu Programming With GPUImage and Metal

  1. 1. GPU PROGRAMMING WITH GPUIMAGE AND METAL JANIE CLAYTON-HASZ
  2. 2. What is a GPU? A Graphics Processing Unit (GPU) is a small super computer that does one thing really really well. That one thing is processing floating point math in parallel. There are several applications for being able to do really fast floating point math: Graphics processing, bioinformatics, molecular dynamics, etc… Most people are going to primarily focus on graphics processing, as we will today
  3. 3. What is Parallel Computing The default processes in a project is serialized computing. One instruction is processed at a time and then the CPU moves on to the next one. Parallel computing is the process of allowing multiple instructions to be carried out at once. Can be done on different threads, cores, and even at the bit level.
  4. 4. But I Have Concurrency! Concurrency is about dealing with a lot of things at once. Parallelism is about doing lots of things at once.
  5. 5. Shader Basics Shaders are the programs that determine what gets displayed on your screen. Shaders determine shapes, colors, textures, lighting… Everything you see and use comes down to shaders.
  6. 6. GRAPHICS ON IOS DEVICES
  7. 7. There are many levels of abstraction for graphics on iOS. ! Some frameworks are more abstracted than others.
  8. 8. UIKit Sprite Kit Core Animation/Graphics OpenGL ES/GLKit
  9. 9. A BRIEF HISTORY OF TIME, UH, OPENGL…
  10. 10. OpenGL Origins First released in 1992 Was an attempt to formalize a 3D graphic specification across platforms
  11. 11. Problems with OpenGL Was created back when GPUs were not very powerful and existed on external graphics cards that could be swapped out The computer system architecture was vastly different when OpenGL was created. Things that were not very efficient then, like the GPU, are vastly more efficient now. Nothing is ever deprecated (Don’t ask Java programmers what that means, they don’t know)
  12. 12. Creation of OpenGL ES ES: Embedded Systems Wanted to strip out all of the dead code from OpenGL Was specifically tailored to work on less powerful devices like mobile phones
  13. 13. We don’t need a dozen turtles that all do the same thing
  14. 14. OpenGL ES Specifics Streamlined version of OpenGL Everything you can do in OpenGL ES can directly be ported to OpenGL Basically an optimized version of OpenGL
  15. 15. CPU VS GPU PROGRAMMING
  16. 16. CPU Expensive Tasks Sending hardware commands to the GPU (Changing State Vectors) Confirming that API usage is valid Compiling the shaders Interaction between the state and the shaders
  17. 17. What the Heck is “State”?? Try to envision a client-server process. Instead of your program sending an instruction over the network to a server and getting data back, you are sending instructions from your CPU to your GPU to be executed. Since you are sending instructions away from your client to be done elsewhere, you want to minimize this as much as possible.
  18. 18. What the Heck is “State”?? For example, in most Twitter client applications the client batches 20 or more Tweets in one call. This allows the application to feed tweets to the user without them having to wait for the network to deliver each and every tweet individually.
  19. 19. Fixed Function Pipeline Present in OpenGL ES 1.1 Shaders were hard-coded into OpenGL Easier to use, but were very limited
  20. 20. Programmable Pipeline Introduced in OpenGL ES 2.0 Shaders are now the responsibility of the programmer Harder to do, but provides far more flexibility and options for effects
  21. 21. GLSL SHADER BUILDING
  22. 22. GLSL OpenGL Shading Language (GLSL) Introduced in OpenGL 2.0 in 2004 C-like language for building shaders, which are small, efficient programs to run on the GPU Includes some specific data types and methods for processing geometry and graphics math that are not included in C
  23. 23. Two Flavors of GLSL Shaders Vertex Fragment
  24. 24. Vertex Shaders
  25. 25. Vertex Shaders The Vertex Shader would record the vertices of the star (which would be broken down into a series of triangles) The Vertex Shader would also specify that the area between the vertices is yellow. If there was a texture instead of a color, the shader would keep track of the texture coordinates.
  26. 26. Fragment Shaders
  27. 27. Fragment Shaders Fragment Shaders determine what pixels receive which color. If you look carefully at the star, there are areas outside the star that are yellow and areas inside that are white. If there is a gradient, the Fragment Shader will calculate what specific color each individual pixel will be.
  28. 28. Inputs and Outputs Uniforms Attributes Varyings
  29. 29. Uniforms Values that don’t change during rendering Read-Only Available in both Vertex and Fragment Shaders
  30. 30. Attributes Vertex Shader only Input values that change with every vertex, like their position, color, and texture coordinates
  31. 31. Varyings Used to pass data between the Vertex and the Fragment Shaders Read-only in the Fragment Shader Read-Write in the Vertex Shader Varyings are the variables that determine the pixel color for the Fragment Shader
  32. 32. GPU IMAGE
  33. 33. Creating GPUImage GPUImage dates back to iOS 5. Unlike Core Image (at the time), GPUImage utilized shaders more efficiently to make image processing faster. Core Image has been improved over the years and they are now comparable.
  34. 34. Why is GPUImage so Efficient? OpenGL ES tasks must be performed on one thread Many people utilize locks to manage the thread or, God forbid, only use the main thread. <shudder> NSLock is expensive to the CPU GPUImage utilizes a serial dispatch queue through GCD to manage anything that touches the GPU to keep everything happy and thread safe.
  35. 35. Demo
  36. 36. METAL: THE NEW KID IN TOWN
  37. 37. What does Metal Promise? Deep hardware integration between Apple chips and Apple frameworks General Purpose GPU programming (GPGPU) Precompiled Shaders up to 10 times more draw calls per frame Being able to perform draw calls on multiple threads
  38. 38. What Specifically are the CPU Expensive Tasks? Compiling Shaders Validating State Start Work on the GPU
  39. 39. Life Before Metal All three of these expensive tasks were done on each and every single draw call. All of these tasks don’t have to be done thousands of times a frame. Many can be done once, as long as the program knows that it does not have to continually check them.
  40. 40. Life After Metal Compiling Shaders: Now done when the applications builds Validating State: Now done when the content loads Start Work on the GPU: Still happens on each draw call. We can’t win them all…
  41. 41. Why is This Important? Before Metal, you would have to balance CPU time with GPU time. Tasks were so expensive that the GPU would usually not be used to capacity. Now that the CPU tasks are less expensive, you can take that time to generate more AI and do more programming logic.
  42. 42. Where Does Metal Help You? Metal helps you when you have a lot of objects that need to work independently of one another. Certain tasks, like image processing, do not involve a lot of objects, so you aren’t going to gain much with Metal.
  43. 43. ZEN GARDEN DEMO EPIC GAMES
  44. 44. Secret Sauce Okay, so one of the big, obscure black boxes in this scheme is the promise of deep software/ hardware integration. One thing I have not had the chance to study in depth is kernel programming and chip architecture. Knowing the idiosyncrasies of the chips and only having to support one type allows for more targeted processes.
  45. 45. IS THERE ANY POINT IN LEARNING OPENGL ES ANYMORE?
  46. 46. Yes, absolutely. Metal’s API is very similar to OpenGL ES. ! It will take a while for everyone to transition over to devices with A7 chips. ! Apple will continue to support its developers who work with OpenGL ES, especially since the Mac uses OpenGL and won’t be able to use Metal (yet). ! It isn’t like they would just throw out an older technology that works perfectly well and replace it with something that barely works, right??
  47. 47. Well crap.
  48. 48. Think Different.
  49. 49. The End

×