Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mopcon2017 - AppDevKit x CameraKit

229 views

Published on

This presentation that we're going to talk about how to use CameraKit to implement high performance real-time camera app. We also will cover the tips that you should know when you implement customized camera feature in your app.

Published in: Mobile
  • Be the first to comment

  • Be the first to like this

Mopcon2017 - AppDevKit x CameraKit

  1. 1. 1 AppDevKit x CameraKit Anistar Sung - Yahoo Senior Engineer Manager 7 Making Camera Amazing
  2. 2. Self Introduction 2 Working for Yahoo Top 10 Camera apps - The New York Times AppDevKit iOS open source project iOS app development experience Making Camera Amazing
  3. 3. What the CameraKit can do? High performance real-time computing with camera CPU & GPU rendering solutions Demos with sample apps 3 Agenda Making Camera Amazing
  4. 4. 4Making Camera Amazing UIImagePickerController
  5. 5. 5Making Camera Amazing
  6. 6. 6Making Camera Amazing
  7. 7. Customization Functional limitation Memory usage 7 Issues of Using UIImagePickController API Making Camera Amazing
  8. 8. 8Making Camera Amazing AVFoundation
  9. 9. What the CameraKit can do? 9Making Camera Amazing
  10. 10. 10 R-Finder 2013
  11. 11. 11Making Camera Amazing SNAP & BUY 2014
  12. 12. 12 CameraKit
  13. 13. 13 AppDevKit x CameraKit Making Camera Amazing UI Image Camera Animation List Common
  14. 14. 14Making Camera Amazing ADKCameraPosition ADKCameraFlashMode ADKCameraTorchMode ADKCameraMirrorMode ADKCameraFocusMode ADKCameraExposureMode ADKCameraWhiteBlanceMode ADKCameraErrorCodealignDeviceOrientation trackLiveVideoData cameraQuality captureVideoPreviewLayer recording exposureBias shutterSpeed ISO zoomFactor lensPosition whiteBalanceTemperature whiteBalanceTint lowLightBoost stabilization cameraPermission microphonePermission frontCameraAvailable rearCameraAvailable optimizeForHighestFrameRate flashAvailable torchAvailable focusAtPoint exposureAtPoint startCamera stopCamera captureImage startCaptureVideo stopCaptureVideo ADKCamera
  15. 15. 15 Initializing a camera Making Camera Amazing camera = [[ADKCamera alloc] initCameraWithDelegate:self quality:AVCaptureSessionPresetLow position:ADKCameraPositionRear]; camera.alignDeviceOrientation = YES;
  16. 16. 16Making Camera Amazing
  17. 17. 17 Camera Viewfinder Preview Making Camera Amazing // Adding preview layer on the screen. [view.layer addSublayer:camera.captureVideoPreviewLayer] // Starting to capture images. [camera startCamera];
  18. 18. 18 Taking photos / videos Making Camera Amazing // Capturing a high quality photo. [camera captureImage:^(UIImage *image, NSDictionary *exifDic, NSError *error) { // Using the image to do something… }
  19. 19. 19 Demo Making Camera Amazing Powering up a simple camera app by CameraKit Catherine Shu, Lead Engineer
  20. 20. High performance real-time computing with camera 20Making Camera Amazing
  21. 21. 21Making Camera Amazing
  22. 22. 22Making Camera Amazing
  23. 23. 23 Goal Making Camera Amazing 1. Making an awesome effect to make girls happy 2. Could be previewed effects immediately 3. Supporting selfies mode
  24. 24. 24 Demo Making Camera Amazing Using CameraKit to create a Selfies CAM Catherine Shu, Lead Engineer
  25. 25. 25Making Camera Amazing camera = [[ADKCamera alloc] initCameraWithDelegate:self quality:AVCaptureSessionPresetHigh position:ADKCameraPositionFront]; camera.trackLiveVideoData = YES; camera.liveVideoDataDlegate = self; Getting Real-time Images
  26. 26. 26Making Camera Amazing
  27. 27. 27 Doing Face Detection Making Camera Amazing - (void)ADKCamera:(ADKCamera *)camera didUpdateSampleBuffer: (CMSampleBufferRef)sampleBuffer { // Converting sample buffer to core image. CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); CIImage *sourceCoreImage = [CIImage imageWithCVPixelBuffer: (CVPixelBufferRef)imageBuffer options:nil];
  28. 28. 28 CoreImage
  29. 29. 29 Doing Face Detection Making Camera Amazing faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:opts]; // Fetching features NSArray *features = [faceDetector featuresInImage:sourceCoreImage];
  30. 30. 30Making Camera Amazing
  31. 31. 31 Using CoreImage Making Camera Amazing CIFilter *filter = [CIFilter filterWithName:@“CIBumpDistortion"]; [filter setValue:filteredCoreImage forKey:kCIInputImageKey]; [filter setValue:[CIVector vectorWithCGPoint:leftEyePoint] forKey:kCIInputCenterKey]; [filter setValue:@(eyeRadius) forKey:kCIInputRadiusKey]; [filter setValue:@(0.55) forKey:kCIInputScaleKey]; filteredCoreImage = filter.outputImage;
  32. 32. 32Making Camera Amazing
  33. 33. 33 Add Instant Effect Making Camera Amazing CIFilter *filter = [CIFilter filterWithName:@"CIPhotoEffectInstant"]; [filter setValue:filteredCoreImage forKey:kCIInputImageKey]; filteredCoreImage = filter.outputImage;
  34. 34. 34Making Camera Amazing
  35. 35. 35 Display Images Making Camera Amazing dispatch_async(dispatch_get_main_queue(), ^{ // Actually, this is a bad idea! UIImage *image = [UIImage imageWithCIImage:filteredCoreImage]; imageView.image = image; });
  36. 36. CPU & GPU rendering solutions 36Making Camera Amazing
  37. 37. 37 Poor Performance Flow Making Camera Amazing CVPixelBuffer CIImage CVPixelBuffer CIContext EAGLContext UIImage UIImageView
  38. 38. 38 Requirements in Real-time Process Making Camera Amazing 60 fps Frame 1 Frame 2 Frame 3 Frame 60Frame 4 16.6 ms 16.6 ms 16.6 ms 16.6 ms 16.6 ms
  39. 39. 39Making Camera Amazing
  40. 40. 40Making Camera Amazing
  41. 41. 41 Better Performance Flow Making Camera Amazing CVPixelBuffer CIImage CVPixelBuffer CIContext EAGLContext GLKView UIImage UIImageView
  42. 42. 42 ADKOpenGLImageView Making Camera Amazing CIImage *image UIColor *backgroundColor ADKOpenGLImageViewContentMode *contentMode
  43. 43. 43Making Camera Amazing ADKOpenGLImageViewContentModeScaleToFill ADKOpenGLImageViewContentModeScaleAspectFit ADKOpenGLImageViewContentModeScaleAspectFill ADKOpenGLImageViewContentModeCenter ADKOpenGLImageViewContentModeTop ADKOpenGLImageViewContentModeBottom ADKOpenGLImageViewContentModeLeft ADKOpenGLImageViewContentModeRight ADKOpenGLImageViewContentModeTopLeft ADKOpenGLImageViewContentModeTopRight ADKOpenGLImageViewContentModeBottomLeft ADKOpenGLImageViewContentModeBottomRight
  44. 44. 44 Better Performance Flow Making Camera Amazing CVPixelBuffer CIImage CVPixelBuffer CIContext EAGLContext ADKOpenGLImageView UIImage UIImageView
  45. 45. 45 Using ADKOpenGlImageView Making Camera Amazing // Set up config and assign a image for displaying. glImageView.contentMode = ADKOpenGLImageViewContentModeScaleAspectFill; glImageView.backgroundColor = [UIColor black]; glImageView.image = finalCoreImage;
  46. 46. 46 Perfomance comparison Making Camera Amazing 300% + * It only for testing the performance of image assignment and rendering
  47. 47. ADKCamera makes things simple Using GPU to process image will be better (EX: CoreImage…) ADKOpenGLImageView provides an OpenGL solution 47 Summary Making Camera Amazing
  48. 48. 48 Q + A Making Camera Amazing

×