iOSDC 2017での登壇資料です。 [概要] WWDC17でもMetalのセッション会場は閑古鳥が鳴いてました。Metalって3Dゲームとかカメラアプリとかのトリッキーなエフェクト用でしょ、うちには関係ないなぁ・・・と思われている方も多いかもしれません。しかし、UIKitの下回りもMetalですし、色んな所で実はMetalは暗躍しています。そんなMetalにもっと多くの開発者に興味を持ってもらうため、非ゲーム・非カメラアプリ開発者にも関係しそうな切り口からMetalやGPUについて紹介してみたいと思います。

1. iOSDC 2017
Shuichi Tsutsumi
@shu223

3. WWDC 2014

7. Swift Metal !

8. WWDC2014 Metal

9. WWDC2014 Swift

10. !
!

11. Metal

12. "
"

13. Metal

14. ※

16. ✋

18. Metal
GPU
• Core ML
• Vision
• Core Image
• SceneKit
• SpriteKit
• MapKit

19. Metal
GPU
• Core ML
• Vision
• Core Image
• SceneKit
• SpriteKit
• MapKit
• Core Animation
• UIKit

20. 143,000
Metal API

21. 143,000
17,000,000
Metal API
Metal API

22. Metal

24. ✓Metal
✓Metal
✓Metal

25. imageView.image = image

33. CPU GPU
CPU
•
•
GPU
•
• CPU

34. • CPU
- 100%
• GPU GPU
-
-
-
-

35.
1 60
• CPU
•
OK
1

36.
1 60
• CPU
•
OK
GPU
1

37. : GPU GPU

38. GPU

40. GPU
GPU
Your app
???

41. OpenGL

42. OpenGL
• API
•
• Apple

43. Metal
• Apple
• Apple
• OpenGL 10
• OpenGL Metal

45. imageView.image = image

47. Metal API

48. func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else {return}
guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()}
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()}
let targetW = min(texture.width, drawable.texture.width)
let targetH = min(texture.height, drawable.texture.height)
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSizeMake(targetW, targetH, texture.depth),
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
blitEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}
private let device = MTLCreateSystemDefaultDevice()!
private func setup() {
commandQueue = device.makeCommandQueue()
let textureLoader = MTKTextureLoader(device: device)
texture = try! textureLoader.newTexture(name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil)
mtkView.device = device
mtkView.delegate = self
mtkView.colorPixelFormat = texture.pixelFormat
}

49. • MTLDevice
• MTLCommandQueue
• MTLCommandBuffer
• MTLCommandEncoder
• MTLTexture
• MTKView
• MTKTextureLoader
• etc…

51. $

58. MTLCommandBuffer
MTLTexture
MTLDevice

59. MTLCommandBuffer
MTLTexture
MTLDevice
MTLCommandEncoder

60. MTLCommandBuffer
MTLTexture
MTLDevice
MTLCommandQueue
MTLCommandEncoder

61. MTLCommandBuffer
MTLTexture
MTLDevice
MTLCommandQueue
MTKTextureLoader
MTLCommandEncoder

63. MTLDevice
let device = MTLCreateSystemDefaultDevice()!
※

64. MTLTexture, MTKTextureLoader
//
loader = MTKTextureLoader(device: device)
//
texture = try! loader.newTexture(
name: “hoge",
scaleFactor: view.contentScaleFactor,
bundle: nil)
※

65. MTLCommandQueue, MTLCommandBuffer
//
let commandBuffer = commandQueue.makeCommandBuffer()!
// …
//
commandBuffer.commit()
※
//
commandQueue = device.makeCommandQueue()

66. MTLCommandEncoder
•
MTLRenderCommandEncoder
MTLComputeCommandEncoder
MTLBlitCommandEncode

67. MTLCommandEncoder (MTLBlitCommandEncoder)
let blitEncoder = commandBuffer.makeBlitCommandEncoder()!
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: srcOrigin,
sourceSize: srcSize,
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin:dstOrigin)
blitEncoder.endEncoding()
※

68. MTLCommandEncoder (MTLBlitCommandEncoder)
let blitEncoder = commandBuffer.makeBlitCommandEncoder()!
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: srcOrigin,
sourceSize: srcSize,
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin:dstOrigin)
blitEncoder.endEncoding()
※

69. //
let commandBuffer = commandQueue.makeCommandBuffer()!
//
let blitEncoder = commandBuffer.makeBlitCommandEncoder()!
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: srcOrigin,
sourceSize: srcSize,
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin:dstOrigin)
//
blitEncoder.endEncoding()
//
commandBuffer.commit()
※

70. private let device = MTLCreateSystemDefaultDevice()!
private func setup() {
commandQueue = device.makeCommandQueue()
let textureLoader = MTKTextureLoader(device: device)
texture = try! textureLoader.newTexture(name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil)
mtkView.device = device
mtkView.delegate = self
mtkView.colorPixelFormat = texture.pixelFormat
}
func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else {return}
guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()}
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()}
let targetW = min(texture.width, drawable.texture.width)
let targetH = min(texture.height, drawable.texture.height)
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSizeMake(targetW, targetH, texture.depth),
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
blitEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}

71. private let device = MTLCreateSystemDefaultDevice()!
private func setup() {
commandQueue = device.makeCommandQueue()
let textureLoader = MTKTextureLoader(device: device)
texture = try! textureLoader.newTexture(name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil)
mtkView.device = device
mtkView.delegate = self
mtkView.colorPixelFormat = texture.pixelFormat
}
→
→ GPU
→ GPU
→
func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else {return}
guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()}
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()}
let targetW = min(texture.width, drawable.texture.width)
let targetH = min(texture.height, drawable.texture.height)
blitEncoder.copy(from: texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0),
sourceSize: MTLSizeMake(targetW, targetH, texture.depth),
to: drawable.texture,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
blitEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
}

72. %

73. Powered by

74. GPU
GPU
Your app
???

75. Metal
• ARKit
• MPS CNN / RNN
• Core Image

76. GPU Metal

77. • Metal - UIKit
• GPU GPU
• Metal - Metal / OpenGL
• Metal -

78. https://github.com/shu223