Building a Native Camera Access Library - Part IV - Transcript.pdf

Building Native Camera Access - Part IV
In this part we continue the iOS Port work starting with CameraKitView

#import <Foundation/Foundation.h>
#import <AVFoundation/AVFoundation.h>
@interface CameraKitView : UIView {
AVCaptureVideoPreviewLayer *innerLayer;
}
-(void)layoutSubviews;
-(void)setLayer:(AVCaptureVideoPreviewLayer*)l;
@end
CameraKitView.h
Before I go into the 4 new update methods I'd like to detour through the CameraKitView class first.

This is pretty standard. We just derive from UIView which is the standard component type for iOS UI's

}
@end
CameraKitView.h
This is the layer we set, keeping it as a variable is convenient

}
@end
CameraKitView.h
This is a method we're overriding from UIView it's invoked when the UIView is arranged by iOS which also has something similar to a layout manager

}
@end
CameraKitView.h
This is the setLayer method we discussed before

#import "CameraKitView.h"
@implementation CameraKitView
-(void)setLayer:(AVCaptureVideoPreviewLayer*)l {
innerLayer = l;
}
-(void)layoutSubviews {
innerLayer.frame = self.bounds;
}
@end
CameraKitView.m
The implementation of the class is even simpler. We store the layer into the member field in set layer.

#import "CameraKitView.h"
@implementation CameraKitView
-(void)setLayer:(AVCaptureVideoPreviewLayer*)l {
innerLayer = l;
}
-(void)layoutSubviews {
innerLayer.frame = self.bounds;
}
@end
CameraKitView.m
When laying out the view I update the fame based on the bounds. Both represent the physical location of the view on the screen. 
Codename One positions the UIView automatically but the CALayer within is positioned by this class. So when Codename One places the UIView based on the layout
manager, the bounds of the UIView are copied into the layer so it shows in the same place

-(void)updateDirection {
[previewLayer removeFromSuperlayer];
[container setLayer:nil];
[captureSession stopRunning];
[previewLayer release];
[captureSession release];
[self lazyInitPostAuthorization];
}
com_codename1_camerakit_impl_CameraNativeAccessImpl.m
Now that this is out of the way lets go into the update methods. Before I go into the ones we already saw there is a hidden one which is invoked when a user invokes
setFacing to change the direction. It's not in the code from before as the functionality was embedded into that code.

When we change the camera direction we need to pick a new device and eﬀectively start over again which is what lazyInitPostAuthorization does. Here we discard the
preview layer & stop the capture session

}
Objective-C uses reference counting, we need to release objects we allocated. This is handled automatically by ARC normally but ARC collides with the GC

}
Because this is essentially recreating the UI we go through the second part of initialization over again

Reference Counting & ARC
© Codename One 2017 all rights reserved
✦Objective-C Doesn’t have garbage collection
✦They use reference counting
✦ARC (Automatic Reference Counter) Hides manual
reference counting
✓ Reference counters are problematic with cyclic references
✓ Reference counting provides more deterministic behavior
✓ Garbage collectors are faster but sometimes stall
✦We tried a hybrid approach
Objective-C and Swift don't have a garbage collector like Java does.

Instead they use reference counting which means every object has a number representing the areas of the code that need it. When I don't want an object deleted I invoke
[myObject retain]; and it's saved when I don't care about it anymore I do [myObject release];. A retain operation increments a number and a release call decrements it.
When the number reaches 0 the object is released. There is more to it but that's the basic GIST of it.

A few years back Apple introduced a compiler enhancement that automatically injects retain/release calls into the code so developers don't "see" this and it feels more
like working with a GC. This is called ARC which standard for "Automatic Reference Counting". Unfortunately we can't get ARC to play nicely with our Garbage Collector.

I won't go to deep into the subject of GC vs. Reference Counting as it's a problematic subject but here's the GIST of it:

Reference counting can fail with cyclic references (Object-A needs Object-B and visa versa). GC's are immune to such cases

Reference counting provides more deterministic behavior. That means it will always perform at exactly the same speed so we can rely on its behavior

Garbage collectors are faster but sometimes stall. For UI’s this can sometimes be a problem as a GC will behave one way in one execution and diﬀerently in another
execution.

We looked at using a hybrid reference counter/GC solution when developing our VM and eventually scrapped that as there were no benefits in that approach. Both
approaches are workable and you need to be ready to debug their pitfalls

-(void)updateVideoQuality {
if(device == nil) {
return;
}
[device lockForConfiguration:nil];
switch(videoQuality) {
case VIDEO_QUALITY_480P:
captureSession.sessionPreset = AVCaptureSessionPreset640x480;
break;
break;
case VIDEO_QUALITY_LOWEST:
case VIDEO_QUALITY_QVGA:
captureSession.sessionPreset = AVCaptureSessionPresetLow;
break;
break;
case VIDEO_QUALITY_HIGHEST:
break;
}
[device unlockForConfiguration];
}
Moving on lets look at the updateVideoQuality method. While it's a bit bigger the core concepts are relatively simple.

If this is invoked before start() it's totally fine, this method will be invoked again when start() is invoked.

if(device == nil) {
return;
}
break;
break;
break;
break;
break;
}
}
When we manipulate some configurations we need to acquire a device lock to prevent concurrent modifications

if(device == nil) {
return;
}
break;
break;
break;
break;
break;
}
}
The rest is a standard switch case to map the standard constants to iOS constants.

-(void)updateFlash {
if(device == nil) {
return;
}
switch(flash) {
case FLASH_ON:
if([device isFlashModeSupported:AVCaptureFlashModeOn]) {
[device setFlashMode:AVCaptureFlashModeOn];
}
break;
case FLASH_OFF:
if([device isFlashModeSupported:AVCaptureFlashModeOff]) {
[device setFlashMode:AVCaptureFlashModeOff];
}
break;
case FLASH_AUTO:
if([device isFlashModeSupported:AVCaptureFlashModeAuto]) {
[device setFlashMode:AVCaptureFlashModeAuto];
}
break;
}
}
This was pretty simple. Next on the line is updateFlash which is also as simple. I can go over the method but there is really nothing here that we didn't discuss in the
previous method. We return for null device, we lock for configuration & we convert the constant type.

-(void)updateFocus {
if(device == nil) {
return;
}
switch(focus) {
case FOCUS_OFF:
[device setFocusMode:AVCaptureFocusModeLocked];
break;
case FOCUS_CONTINUOUS:
[device setFocusMode:AVCaptureFocusModeContinuousAutoFocus];
break;
case FOCUS_TAP_WITH_MARKER:
case FOCUS_TAP:
UITapGestureRecognizer *tapGR = [[UITapGestureRecognizer alloc]
initWithTarget:self
action:@selector(tapToFocus:)];
[tapGR setNumberOfTapsRequired:1];
[tapGR setNumberOfTouchesRequired:1];
[container addGestureRecognizer:tapGR];
break;
}
}
Surprisingly updateFocus does have something new to say despite being pretty identical to the first two. I'll skip the identical part and discuss the final section.

There is no builtin focus on tap in iOS so we need to use some code to do this

-(void)updateFocus {
if(device == nil) {
return;
}
switch(focus) {
case FOCUS_OFF:
[device setFocusMode:AVCaptureFocusModeLocked];
break;
case FOCUS_CONTINUOUS:
break;
case FOCUS_TAP_WITH_MARKER:
case FOCUS_TAP:
UITapGestureRecognizer *tapGR = [[UITapGestureRecognizer alloc]
initWithTarget:self
action:@selector(tapToFocus:)];
[tapGR setNumberOfTapsRequired:1];
[tapGR setNumberOfTouchesRequired:1];
[container addGestureRecognizer:tapGR];
break;
}
}
This invoked the tapToFocus method on self (which is this object in Objective-C) when a user taps container

// converted from this answer: https://stackoverflow.com/a/17025083/756809
-(void)tapToFocus:(UITapGestureRecognizer *)singleTap {
CGPoint touchPoint = [singleTap locationInView:container];
CGPoint convertedPoint = [previewLayer
captureDevicePointOfInterestForPoint:touchPoint];
if([device isFocusPointOfInterestSupported] &&
[device isFocusModeSupported:AVCaptureFocusModeAutoFocus]){
NSError *error = nil;
[device lockForConfiguration:&error];
if(!error){
[device setFocusPointOfInterest:convertedPoint];
[device setFocusMode:AVCaptureFocusModeAutoFocus];
}
}
}
So lets look at the tapToFocus method. Notice that this logic isn't something I came up with. I took it from a stackoverflow answer which is very convenient for
implementing these sort of things.

A tap returns a point on the screen which we can convert to a point relative to the coordinate space of the previewLayer

// converted from this answer: https://stackoverflow.com/a/17025083/756809
-(void)tapToFocus:(UITapGestureRecognizer *)singleTap {
CGPoint touchPoint = [singleTap locationInView:container];
CGPoint convertedPoint = [previewLayer
captureDevicePointOfInterestForPoint:touchPoint];
if([device isFocusPointOfInterestSupported] &&
[device isFocusModeSupported:AVCaptureFocusModeAutoFocus]){
NSError *error = nil;
[device lockForConfiguration:&error];
if(!error){
[device setFocusPointOfInterest:convertedPoint];
[device setFocusMode:AVCaptureFocusModeAutoFocus];
}
}
}
If we can declare a focus point of interest we can just set the focus to the right point. It's not necessarily trivial but mostly boilerplate

-(void)updateZoom {
if(device == nil) {
return;
}
device.videoZoomFactor = MAX(1.0,
MIN(zoom, device.activeFormat.videoMaxZoomFactor));
}
This brings us to the last and simplest of these methods. This is mostly a rehash of the other methods so I won't discuss it. Notice it guards against zooming too much or
too little.

And that's it, with these changes camera will basically work! We just need to fill in a few more "details" which are mostly boilerplate.

Building a Native Camera Access Library - Part IV - Transcript.pdf

Recommended

Recommended

More Related Content

Similar to Building a Native Camera Access Library - Part IV - Transcript.pdf

Similar to Building a Native Camera Access Library - Part IV - Transcript.pdf (20)

More from ShaiAlmog1

More from ShaiAlmog1 (20)

Recently uploaded

Recently uploaded (20)

Building a Native Camera Access Library - Part IV - Transcript.pdf