Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UIImageView vs Metal [日本語版] #tryswiftconf

3,888 views

Published on

try! Swift Tokyo 2018 での発表資料です。Metalの使い方の話「ではなく」、Metalを通じて**普段意識する機会の少ないGPUレイヤに目を向けてみる**という内容となっておりますので、Metal使わないし興味ないという方々もぜひ。

英語版は https://www.slideshare.net/t26v0748/uiimageview-vs-metal-89418399/ にあります。

補足記事を書きました: http://shu223.hatenablog.com/entry/2018/03/05/124639

【概要】

MetalはGPUへのアクセスを提供するAPIで、OpenGLより10倍速いという謳い文句で登場しました。本セッションではMetalの基礎を解説しつつ、iOSにおけるグラフィックス描画性能をUIImageViewと比較してみます。

MetalのAPIを直接利用する機会がなくても、Metalはあなたのアプリの裏で暗躍しています。身近なクラスとの比較を通じて、普段我々が意識することのないGPUのレイヤで何が起きているのか、目を向けてみるきっかけになればと思います。

Published in: Technology
  • Be the first to comment

UIImageView vs Metal [日本語版] #tryswiftconf

  1. 1. try! Swift Tokyo 2018 Shuichi Tsutsumi @shu223 UIImageView vs Metal [ ]
  2. 2. Shuichi Tsutsumi @shu223 • iOS Developer - @Fyusion Inc. - @Freelance
  3. 3. • Metal • GPU
  4. 4. Agenda • UIImageView vs Metal → GPU 1. UIKit is optimized well with GPU. 2. Consider also the GPU, when measuring the performance. 3. Pay attention to the processing flow between CPU and GPU. 4. Be careful where the resource is.
  5. 5. imageView.image = image
  6. 6. 
 1 1 60 

  7. 7. CPU GPU CPU • • GPU • • CPU
  8. 8. • CPU - 100% • GPU GPU
  9. 9. 
 1 60 • CPU • OK GPU 1
  10. 10. What’s ?
  11. 11. GPU GPU Your app ???
  12. 12. OpenGL
  13. 13. OpenGL • API • • Apple
  14. 14. Metal • Apple • Apple • OpenGL 10
  15. 15. Sounds great!
  16. 16. Metal
  17. 17. imageView.image = image
  18. 18. Metal
  19. 19. func draw(in view: MTKView) { guard let drawable = view.currentDrawable else {return} guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()} guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()} let w = min(texture.width, drawable.texture.width) let h = min(texture.height, drawable.texture.height) blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSizeMake(w, h, texture.depth), to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0)) blitEncoder.endEncoding() commandBuffer.present(drawable) commandBuffer.commit() commandBuffer.waitUntilCompleted() } private let device = MTLCreateSystemDefaultDevice()! 
 private func setup() { commandQueue = device.makeCommandQueue() let textureLoader = MTKTextureLoader(device: device) texture = try! textureLoader.newTexture( name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil) mtkView.device = device mtkView.delegate = self mtkView.colorPixelFormat = texture.pixelFormat }
  20. 20. private let device = MTLCreateSystemDefaultDevice()! 
 private func setup() { commandQueue = device.makeCommandQueue() let textureLoader = MTKTextureLoader(device: device) texture = try! textureLoader.newTexture( name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil) mtkView.device = device mtkView.delegate = self mtkView.colorPixelFormat = texture.pixelFormat } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else {return} guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalE guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fa let w = min(texture.width, drawable.texture.width) let h = min(texture.height, drawable.texture.height) blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSizeMake(w, h, texture.depth), to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0)) blitEncoder.endEncoding() commandBuffer.present(drawable) commandBuffer.commit() commandBuffer.waitUntilCompleted() } imageView.image = image
  21. 21. private let device = MTLCreateSystemDefaultDevice()! 
 private func setup() { commandQueue = device.makeCommandQueue() let textureLoader = MTKTextureLoader(device: device) texture = try! textureLoader.newTexture( name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil) mtkView.device = device mtkView.delegate = self mtkView.colorPixelFormat = texture.pixelFormat } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else {return} guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalE guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fa let w = min(texture.width, drawable.texture.width) let h = min(texture.height, drawable.texture.height) blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSizeMake(w, h, texture.depth), to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0)) blitEncoder.endEncoding() commandBuffer.present(drawable) commandBuffer.commit() commandBuffer.waitUntilCompleted() } imageView.image = image 💡
  22. 22. My Idea: Metal ✓ Easy to use as UIImageView ✓ Metal Accelerated “MetalImageView” metalImageView.texture = texture
  23. 23. Powered by
  24. 24. UIImageView vs Metal
  25. 25. • - 5120 x 3200 (elcapitan.jpg) - 1245 x 1245 (sierra.png)
  26. 26. Measuring Code let time1 = CACurrentMediaTime() if isMetal { let metalCell = cell as! MetalTableViewCell metalCell.metalImageView.textureName = name } else { let uikitCell = cell as! TableViewCell uikitCell.uiImageView.image = UIImage(named: name) } let time2 = CACurrentMediaTime() print("time:(time2-time1)") Time Interval Render with UIImageView Render with Metal
  27. 27. Results • Metal is 10x - 20x faster! Time to render an image UIImageView 0.4 - 0.6 msec Metal 0.02 - 0.05 msec iPhone 6s
  28. 28. • Metal UIImageView Metal
  29. 29. Measuring Code let time1 = CACurrentMediaTime() if isMetal { let metalCell = cell as! MetalTableViewCell metalCell.metalImageView.textureName = name } else { let uikitCell = cell as! TableViewCell uikitCell.uiImageView.image = UIImage(named: name) } let time2 = CACurrentMediaTime() print("time:(time2-time1)")
  30. 30. CPU GPU
  31. 31. CPU GPU 
 CPU/GPU GPU GPU
  32. 32. let time1 = CACurrentMediaTime() if isMetal { let metalCell = cell as! MetalTableViewCell metalCell.metalImageView.textureName = name } else { let uikitCell = cell as! TableViewCell uikitCell.uiImageView.image = UIImage(named: name) } let time2 = CACurrentMediaTime() print("time:(time2-time1)")
  33. 33. CPU GPU 
 CPU/GPU GPU GPU
  34. 34. • GPU func draw(in view: MTKView) { // Prepare the command buffer ... // Push the command buffer commandBuffer.commit() // Wait commandBuffer.waitUntilCompleted() // Measure let endTime = CACurrentMediaTime() print(“Time: (endTime - startTime)”) } GPU GPU
  35. 35. Results • Metal !? - 30 fps → • UIImageView Time to render an image UIImageView 0.4 - 0.6 msec Metal 40 - 200 msec
  36. 36. UIImageView
  37. 37. ※WWDC17 Platforms State of the Union UIKit Metal
  38. 38. • UIKit •
  39. 39. Point 1: UIKit is optimized well with GPU
  40. 40. Point 2: Consider also the GPU, 
 when measuring the performance
  41. 41. MetalImageView Metal
  42. 42. Profile using Instruments Metal System Trace
  43. 43. On CPU On GPU Create command buffers etc. on CPU Submit command buffers etc. on CPU Process shaders on GPU
  44. 44. On CPU On GPU
  45. 45. Problem 1
  46. 46. Resize (MPSImageLanczosScale) Render (MTLBlitCommandEncoder) Unexpected interval Measuring Time
  47. 47. • MPSImageLanczosScale • setNeedsDisplay() • MTKViewDelegate draw(in:) • draw(in:) Problem
  48. 48. GPU CPU On CPU On GPU
  49. 49. Metal 4: GPU
  50. 50. Resize Render 2. CPU creates GPU commands 
 as a command buffer 4. GPU processes the commands 3. Push it to GPU
  51. 51. GPU • • GPU
  52. 52. 2. CPU creates GPU commands 
 as a command buffer 4. GPU processes the commands 3. Push it to GPU Resize Render
  53. 53. Resize Render
  54. 54. Unexpected interval Combine Resize Render Resize+Render
  55. 55. Point 3: Pay attention to the processing flow between CPU and GPU
  56. 56. Problem 2
  57. 57. CPU/GPU
  58. 58. • 20 - 500 msec • → let startTime = CACurrentMediaTime() textureLoader.newTexture(name: name, scaleFactor: scaleFactor, bundle: nil) { (texture, error) in let endTime = CACurrentMediaTime() print("Time to load (name): (endTime - startTime)") // Do something ... }
  59. 59. • UIImage(named:) • Metal/GPU
  60. 60. Metal/GPU : private var cachedTextures: [String: MTLTexture] = [:]OK private var cachedImages: [String: UIImage] = [:]NG
  61. 61. After adopting Cache
  62. 62. Point 4: Be careful where the resource is.
  63. 63. • Metal • GPU
  64. 64. • UIImageView vs Metal → GPU 1. UIKit is optimized well with GPU. 2. Consider also the GPU, when measuring the performance. 3. Pay attention to the processing flow between CPU and GPU. 4. Be careful where the resource is.
  65. 65. Thank you! https://github.com/shu223

×