try! Swift Tokyo 2018 での発表資料です。Metalの使い方の話「ではなく」、Metalを通じて**普段意識する機会の少ないGPUレイヤに目を向けてみる**という内容となっておりますので、Metal使わないし興味ないという方々もぜひ。 英語版は https://www.slideshare.net/t26v0748/uiimageview-vs-metal-89418399/ にあります。 補足記事を書きました： http://shu223.hatenablog.com/entry/2018/03/05/124639 【概要】 MetalはGPUへのアクセスを提供するAPIで、OpenGLより10倍速いという謳い文句で登場しました。本セッションではMetalの基礎を解説しつつ、iOSにおけるグラフィックス描画性能をUIImageViewと比較してみます。 MetalのAPIを直接利用する機会がなくても、Metalはあなたのアプリの裏で暗躍しています。身近なクラスとの比較を通じて、普段我々が意識することのないGPUのレイヤで何が起きているのか、目を向けてみるきっかけになればと思います。

1. try! Swift Tokyo 2018
Shuichi Tsutsumi
@shu223
UIImageView vs Metal
[ ]

2. Shuichi Tsutsumi @shu223
• iOS Developer
- @Fyusion Inc.
- @Freelance

3. • Metal
• GPU

4. Agenda
• UIImageView vs Metal
→ GPU
1. UIKit is optimized well with GPU.
2. Consider also the GPU, when measuring the performance.
3. Pay attention to the processing flow between CPU and GPU.
4. Be careful where the resource is.

5. imageView.image = image

8.
1
1 60

9. CPU GPU
CPU
•
•
GPU
•
• CPU

10. • CPU
- 100%
• GPU GPU

11.
1 60
• CPU
•
OK
GPU
1

12. What’s ?

13. GPU
GPU
Your app
???

14. OpenGL

15. OpenGL
• API
•
• Apple

16. Metal
• Apple
• Apple
• OpenGL 10

17. Sounds great!

18. Metal

19. imageView.image = image

21. Metal

22. func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else {return}
guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()}
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()}
let w = min(texture.width, drawable.texture.width)
let h = min(texture.height, drawable.texture.height)
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSizeMake(w, h, texture.depth),
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
blitEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}
private let device = MTLCreateSystemDefaultDevice()!
private func setup() {
commandQueue = device.makeCommandQueue()
let textureLoader = MTKTextureLoader(device: device)
texture = try! textureLoader.newTexture(
name: "highsierra",
scaleFactor: view.contentScaleFactor,
bundle: nil)
mtkView.device = device
mtkView.delegate = self
mtkView.colorPixelFormat = texture.pixelFormat
}

24. private let device = MTLCreateSystemDefaultDevice()!
private func setup() {
commandQueue = device.makeCommandQueue()
let textureLoader = MTKTextureLoader(device: device)
texture = try! textureLoader.newTexture(
name: "highsierra",
scaleFactor: view.contentScaleFactor,
bundle: nil)
mtkView.device = device
mtkView.delegate = self
mtkView.colorPixelFormat = texture.pixelFormat
}
func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else {return}
guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalE
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fa
let w = min(texture.width, drawable.texture.width)
let h = min(texture.height, drawable.texture.height)
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSizeMake(w, h, texture.depth),
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
blitEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}
imageView.image = image

25. private let device = MTLCreateSystemDefaultDevice()!
private func setup() {
commandQueue = device.makeCommandQueue()
let textureLoader = MTKTextureLoader(device: device)
texture = try! textureLoader.newTexture(
name: "highsierra",
scaleFactor: view.contentScaleFactor,
bundle: nil)
mtkView.device = device
mtkView.delegate = self
mtkView.colorPixelFormat = texture.pixelFormat
}
func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else {return}
guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalE
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fa
let w = min(texture.width, drawable.texture.width)
let h = min(texture.height, drawable.texture.height)
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSizeMake(w, h, texture.depth),
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
blitEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}
imageView.image = image
💡

26. My Idea:
Metal
✓ Easy to use as UIImageView
✓ Metal Accelerated
“MetalImageView”
metalImageView.texture = texture

27. Powered by

28. UIImageView vs Metal

29. •
- 5120 x 3200 (elcapitan.jpg)
- 1245 x 1245 (sierra.png)

30. Measuring Code
let time1 = CACurrentMediaTime()
if isMetal {
let metalCell = cell as! MetalTableViewCell
metalCell.metalImageView.textureName = name
} else {
let uikitCell = cell as! TableViewCell
uikitCell.uiImageView.image = UIImage(named: name)
}
let time2 = CACurrentMediaTime()
print("time:(time2-time1)")
Time
Interval
Render with UIImageView
Render with Metal

31. Results
• Metal is 10x - 20x faster!
Time to render an image
UIImageView 0.4 - 0.6 msec
Metal 0.02 - 0.05 msec
iPhone 6s

32. • Metal
UIImageView Metal

33. Measuring Code
let time1 = CACurrentMediaTime()
if isMetal {
let metalCell = cell as! MetalTableViewCell
metalCell.metalImageView.textureName = name
} else {
let uikitCell = cell as! TableViewCell
uikitCell.uiImageView.image = UIImage(named: name)
}
let time2 = CACurrentMediaTime()
print("time:(time2-time1)")

34. CPU GPU

35. CPU GPU
CPU/GPU GPU
GPU

36. let time1 = CACurrentMediaTime()
if isMetal {
let metalCell = cell as! MetalTableViewCell
metalCell.metalImageView.textureName = name
} else {
let uikitCell = cell as! TableViewCell
uikitCell.uiImageView.image = UIImage(named: name)
}
let time2 = CACurrentMediaTime()
print("time:(time2-time1)")

37. CPU GPU
CPU/GPU GPU
GPU

38. • GPU
func draw(in view: MTKView) {
// Prepare the command buffer
...
// Push the command buffer
commandBuffer.commit()
// Wait
commandBuffer.waitUntilCompleted()
// Measure
let endTime = CACurrentMediaTime()
print(“Time: (endTime - startTime)”)
}
GPU
GPU

39. Results
• Metal !?
- 30 fps
→
• UIImageView
Time to render an image
UIImageView 0.4 - 0.6 msec
Metal 40 - 200 msec

40. UIImageView

41. ※WWDC17 Platforms State of the Union
UIKit Metal

42. • UIKit
•

43. Point 1:
UIKit is optimized well
with GPU

44. Point 2:
Consider also the GPU,
when measuring the performance

45. MetalImageView Metal

46. Profile using Instruments
Metal System Trace

48. On CPU
On GPU
Create command buffers etc. on CPU
Submit command buffers etc. on CPU
Process shaders on GPU

49. On CPU
On GPU

50. Problem 1

51. Resize
(MPSImageLanczosScale)
Render
(MTLBlitCommandEncoder)
Unexpected interval
Measuring Time

52. • MPSImageLanczosScale
• setNeedsDisplay()
• MTKViewDelegate draw(in:)
• draw(in:)
Problem

53. GPU CPU
On CPU
On GPU

55. Metal 4:
GPU

56. Resize
Render
2. CPU creates GPU commands
as a command buffer
4. GPU processes
the commands
3. Push it to
GPU

57. GPU
•
• GPU

58. 2. CPU creates GPU commands
as a command buffer
4. GPU processes
the commands
3. Push it to
GPU
Resize
Render

59. Resize Render

60. Unexpected interval
Combine
Resize Render Resize+Render

61. Point 3:
Pay attention to the processing flow
between CPU and GPU

62. Problem 2

64. CPU/GPU

65. • 20 - 500 msec
• →
let startTime = CACurrentMediaTime()
textureLoader.newTexture(name: name, scaleFactor: scaleFactor, bundle: nil) { (texture,
error) in
let endTime = CACurrentMediaTime()
print("Time to load (name): (endTime - startTime)")
// Do something
...
}

66. • UIImage(named:)
•
Metal/GPU

67. Metal/GPU :
private var cachedTextures: [String: MTLTexture] = [:]OK
private var cachedImages: [String: UIImage] = [:]NG

68. After adopting Cache

69. Point 4:
Be careful where the resource is.

72. • Metal
• GPU

73. • UIImageView vs Metal
→ GPU
1. UIKit is optimized well with GPU.
2. Consider also the GPU, when measuring the performance.
3. Pay attention to the processing flow between CPU and GPU.
4. Be careful where the resource is.

74. Thank you!
https://github.com/shu223