Camera API Proposal May 2013


Published on

The Khronos Camera working group is creating an open, royalty-free standard for advanced, low-level control of mobile and embedded cameras and sensors.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Camera API Proposal May 2013

  1. 1. © Copyright Khronos Group 2013 - Page 1Camera Control APIProvisional Working GroupMay 2013
  2. 2. © Copyright Khronos Group 2013 - Page 2Advanced Camera Control Use Cases• High-dynamic range (HDR) and computational flash photography- High-speed burst with individual frame control over exposure and flash• Rolling shutter elimination- High-precision intra-frame synchronization between camera and motion sensor• HDR Panorama, photo-spheres- Continuous frame capture with constant exposure and white balance• Subject isolation and depth detection• High-speed burst with individual frame control over focus• Time-of-flight or structured light depth camera processing- Aligned stacking of data from multiple sensors• Augmented Reality- 60Hz, low-latency capture with motion sensor synchronization- Multiple Region of Interest (ROI) capture- Multiple sensors for scene scaling- Detailed feedback on camera operation per frame
  3. 3. © Copyright Khronos Group 2013 - Page 3Camera Control API Complements AccelerationPre-ISPProcessing(Bayer Space)ISPPost-ISPProcessing(YUV Space)ApplicationNeed Camera API to feedprocessing pipeline foradvanced use casesImage/VisionAccelerationAPIs?SensorLensFlash
  4. 4. © Copyright Khronos Group 2013 - Page 4Precursor APIs for Camera Control Initiative• FCAM – Open source project- Capture of stream of camera images with precision control- A pipeline that converts requests into image stream- All parameters packed into the requests - no global state- Programmer has full control over sensor settings for each frame in stream- Control over focus and flash- No hidden daemon running- Control ISP- Can access supplementalstatistics from ISP if available• Android New Camera HAL (2013)- Uses some of these concepts
  5. 5. © Copyright Khronos Group 2013 - Page 5Potential Camera Control API Functionality• Burst Sensor Control- Exposure, time, gain, CFA pattern …• Burst Lens control- Target focus distance, aperture, focal length, position state …• Burst Flash Control- Brightness, duration, burst, activity state …• ISP Control- Demosaic-ing quality, denoising quality, 3A, CCM, Gamma, color space …• Output(s) Control- Resolution, ROI extraction, quality, format (including Bayer, YUV) …• Frame timestamp- To be used to synch with motion sensors• Parameter access- Standard-defined parameters AND Vendor-specific extensions registry• Feedback Statistics- Histogram, sharpness map …• Multi-sensor control- Synchronization, master sensor controlling other sensors, image stacking• Metadata- Per frame: Focal length (fx, fy), principal point (cx, cy), skew (s), image resolution (h, w), exposure- Per device: Cameras and sensors physical layout, calibration and lens distortion
  6. 6. © Copyright Khronos Group 2013 - Page 6Camera Control API UsageBurst control ofsensor, flash, lenswith precisiontimestamps on framesApplicationStatisticsFeedbackOutputScalersand FormattersDownstreamProcessingISP ImageProcessingMEMSSensorsPrecision timestampson positional sensorsamplesSensorLensFlashEGLStreamand EGL accessto sampletimestampsCamera Control API