SlideShare a Scribd company logo
Camera 3
The New Camera Subsystem in Android KitKat

Balwinder Kaur,

Senior Member, Technical Staff, Aptina Imaging

AnDevCon, Burlingame, CA
11.15.13
Slide Deck v 1.2 last updated 12.27.13

© 2013 Aptina Imaging Corporation. All rights reserved. Products are warranted only to meet Aptina’s production data sheet specifications. Information, products, and/or
specifications are subject to change without notice. All information is provided on an “AS IS” basis without warranties of any kind. Dates are estimates only. Drawings not to
scale. Aptina and the Aptina logo are trademarks of Aptina Imaging Corporation. All other trademarks are the property of their respective owners.

1

|

© 2013 Aptina Imaging Corporation
Agenda
I.

Camera 3
‣ Introduction
‣ History

‣ Computational Photography Use Cases

II.

The APIs
‣ {@hide} – Sneak Peek into coming Android Pro Camera APIs

III. The Platform
‣ Architecture
‣ 3A

IV. Emerging Trends

V.

Demo

VI. Q&A
2

|

© 2013 Aptina Imaging Corporation
Camera 3 – The New Camera
Framework

3

|

© 2013 Aptina Imaging Corporation
Limitations of Camera 1
• 3 Operating Modes
‣ Preview
‣ Capture
‣ Video

• Burst Mode Photography – hard to
implement
‣ Zero Shutter Lag
‣ Multi-Shot HDR
‣ Panoramic Stitch

• Metadata: Practically, only face
detection data could be returned
to the application
4

|

© 2013 Aptina Imaging Corporation
Camera 3
• Android 4.3 (Jelly Bean MR2) and Android 4.4 (KitKat)
have shipped with a new Camera Hardware Interface
‣ Android 4.3 => Camera 3 Hardware Interface
‣ Android 4.4 => Camera 3.1 Hardware Interface
What
happened
to
Camera2
?

5

|

© 2013 Aptina Imaging Corporation
Overview of Camera 3
•

Camera Framework till Android 4.0 was modeled after a point & shoot camera.

•

Expanded Camera Subsystem is modeled after a Professional Camera
‣

•

6

But it is more than that

It is the basis of a potential paradigm shift in photography

|

© 2013 Aptina Imaging Corporation
A Bit of History
FCAM

F2 Frankencamera
Project

Lead - Stanford Professor Marc Levoy
Main architects - Eino-Ville (Eddy) Talvala and Andrew Adams
Eddy Talvala is the Tech Lead for Camera Framework on the
Android Team
Industry Partner - Nokia

Computational Photography
‣ Out of scope for this talk
‣ Good starting point - A white paper on Computational Photography Platforms
by Marc Levoy
‣ Use and relevance of software increases in photography
‣ Processing happens in near real time instead of post-processing
‣ Open platforms facilitate innovation
7

|

© 2013 Aptina Imaging Corporation
Computational Photography
Definition
• Computational photography refers to computational image
capture, processing, and manipulation techniques that
enhance or extend the capabilities of digital photography.
• Good examples of image capture and manipulation include
panoramas, and high-dynamic-range imaging which is the use
of differently exposed pictures of the same scene to extend
dynamic range beyond even that of analog film-based media.
Source: Wikipedia

8

|

© 2013 Aptina Imaging Corporation
Multi Shot HDR
• HDR = High Dynamic Range
• Multiple shots at different exposures and then blended together

9

|

© 2013 Aptina Imaging Corporation
Panoramic Stitching

Source: commons.wikimedia.org

10

|

© 2013 Aptina Imaging Corporation
Multi-Flash Photography

Source: Stanford Graphics Research
11

|

© 2013 Aptina Imaging Corporation
Flash No-Flash Photography

Source: Microsoft Research

12

|

© 2013 Aptina Imaging Corporation
Flash No-Flash Photography

Source: Microsoft Research
13

|

© 2013 Aptina Imaging Corporation
Focus Bracketing

Focus Stacking for macro photography

Source: Wikipedia
14

|

© 2013 Aptina Imaging Corporation
View Finder Editing
Today’s smartphones have features like

•

‣

Computational Photography can take it to the next level

•

‣

•

15

Touch to Expose & Touch to Focus

Edit the frame in the view finder by selecting areas to sharpen, blur, brighten
and maintain these during capture mode

Source: SIGGRAPH 2012

|

© 2013 Aptina Imaging Corporation
Open Platform Requirements
for Computation Photography
• Fine grained control of the Sensor, Flash, Lens and Image Signal
Processing pipeline
• Ability to program
‣ exposure time, auto white balance, focus, frame-rate etc.
‣ metering algorithm, autofocus algorithm etc.

• Meta data
‣ Every frame is returned with the settings that it was taken with (verses
settings take effect sometime in future)

• Platform should have enough compute and memory power.
‣ With multi core devices and GPU capability increasing, Android becomes a
very attractive platform as an open computational photography platform.

16

|

© 2013 Aptina Imaging Corporation
Other use cases
• Using Imagers for Information
‣ Gesture Recognition
‣ Face Recognition
‣ Augmented Reality

• Requirements differ for this class of applications
‣ High and steady Frame Rate under all light conditions is
Important.
‣ Of less importance
•
•

17

|

High Resolution
Image Quality & Color Accuracy

© 2013 Aptina Imaging Corporation
The APIs

18

|

© 2013 Aptina Imaging Corporation
Camera API
Camera APIs
Package - android.hardware.camera
Android Version

Additions to
android.hardware.camera

Kit Kat (4.4)

API level 19

None

Jelly Bean MR2 (4.3)

API level 18

None

Jelly Bean MR1 (4.2)

API level 17

Minor (HDR, shutterSound)

Jelly Bean (4.1)

API level 16

Minor (AutoFocus Move
Callback)

Ice-cream Sandwich

API level 15

Minor (Video Stabilization)

Ice-cream Sandwich
19

API Level

API level 14

Major (Face Detection)

|

© 2013 Aptina Imaging Corporation
A Sneak Peek of the New Camera APIs
• Kitkat shipped with a set of Experimental Java camera
APIs with the {@hide} tag.
‣ https://android.googlesource.com/platform/frameworks/base
/+/kitkat-release/core/java/android/hardware/camera2/

• Package name is android.hardware.camera2
‣ But in other parts of AOSP many references to ProCamera
(Professional Camera) are found.

20

|

© 2013 Aptina Imaging Corporation
Pro Camera APIs ?
• They are experimental and not part of the SDK, but
available in the platform.

• Likely to change before released in the SDK.
• Camera 3 Hardware Interface is fairly stable now.
• It’s a good time to start thinking about new Camera
applications

21

|

© 2013 Aptina Imaging Corporation
android.hardware.camera2
Interface
Camera Device
Exception
CameraAccessException

22

|

© 2013 Aptina Imaging Corporation

Classes
CameraCharacteristics
CameraDevice.CaptureListener
CameraDevice.StateListener
CameraManager
CameraManager.AvailabilityListener
CameraMetadata
CameraMetadata.Key<t>
CaptureFailure
CaptureRequest
CaptureRequest.Builder
CaptureResult
Face
Rational
Size
Camera Manager
Interface for iterating, listing and accessing Camera Devices
CameraManager manager = (CameraManager)
getSystemService(Context.CAMERA_SERVICE);
//Iterating through Camera Devices
manager.getCameraIdList();
// Query a device for Capabilities
manager.getCameraCharacteristics(cameraId);
// Open a Camera
manager.openCamera ( cameraId,...);
// add an Availability Listener as Cameras become available or unavailable
manager.addAvailabilityListener(CameraManager.AvailabilityListener);

23

|

© 2013 Aptina Imaging Corporation
Camera Device
•

Device is modeled as a pipeline
‣

Input : CaptureRequest for a single Frame

‣

Output: CaptureResult Metadata packet + output image buffers for the request.
These buffers can go to one or multiple output targets
•

SurfaceView

•

SurfaceTexture

•

MediaCodec

•

MediaRecorder

•

Renderscript Allocation et al

•

ImageReader (Use this for Still Images)

‣

‣
24

Requests are processed in order, multiple requests are in flight to maintain full
frame rate ( generally means 30 frames per second )

Android 4.4 added the flush() command to drop all inflight requests.
|

© 2013 Aptina Imaging Corporation
Capture Request
// Step 1 : Configure Output Surfaces
camera.configureOutputs (List<Surfaces> outputs)
// Set the size and format.
// Step 2: Build a CaptureRequest
// Use default Templates – Manual, Preview, Still_Capture,
// Record, Video_Snapshot, Zero_Shutter_Lag
CaptureRequest.Builder bldr =
camera.createCaptureRequest(CameraDevice.TEMPLATE_VIDEO_SNAPSHOT);
//no chaining of calls available
bldr.addTarget();
bldr.setTag(); // to identify the result when it is returned
bldr.set (Metadata.Key …);
CaptureRequest req = bldr.build();

25

|

© 2013 Aptina Imaging Corporation
Capture Modes
//Step 3: Submit a request
//Pick 1 of the 4
// capture, captureBurst, setRepeatingRequest, setRepeatingBurst

camera.capture(CaptureRequest, CameraDevice.CaptureListener) //one time capture
camera.captureBurst(…) // Bracketing Modes. Guaranteed that no other request will be
interspersed in this queue
camera.setRepeatingRequest (…) // Preview. Lower priority. Cancelled with stopRepeating() or
flush() call. videoRates – needs clarification

camera.setRepeatingBurst (…) // same as setRepeatingBurst but at full frame rates

26

|

© 2013 Aptina Imaging Corporation
CaptureListener
//Step 4: Process the progress of the capture
//4 callback methods available
onCaptureStarted(CameraDevice, CaptureRequest, long timestamp);
onCaptureCompleted(CameraDevice, CaptureRequest, CaptureResult);
//or
onCaptureFailed(CameraDevice, CaptureRequest, CaptureFailure);
// once all the capture requests are processed
onCaptureSequenceCompleted(CameraDevice, int seqid, int framenumber)

27

|

© 2013 Aptina Imaging Corporation
Capture Result
//Step 5: Investigate the CaptureResult or CaptureFailure Object Received
// The results of a single image capture from the
//image sensor. Contains the final configuration for
//the capture hardware (sensor, lens, flash), the
//processing pipeline, the control algorithms,
//and the output buffers.
get(CameraMetadata.Key<T> key) ;
getRequest() ;

28

|

© 2013 Aptina Imaging Corporation
ImageReader
•

Introduced in KitKat API Level 19

•

Android.media.ImageReader

•

Allows direct application access to image data rendered into a Surface

•

maxImages determines the maximum number of Image objects that can be be
acquired from the ImageReader simultaneously. Use cautiously !

•

ImageFormat is defined by ImageFormat or PixelFormat

•

ImageReader.OnImageAvailableListener provides callback for when the next
Image is available

•

Use acquireLatestImage, rather than acquireNextImage. It removes old images

ImageReader imgreader = ImageReader.newInstance(width, height, format,
maxImages);
imgreader.getSurface() ; // Pass this into the CameraRequest Class

29

|

© 2013 Aptina Imaging Corporation
Camera Metadata
• Base class for Camera Controls and Information
• Metadata is of 2 types
‣ Static and Dynamic

• Sensor Control
• Flash Control
• Lens Control
• Pipeline Control
• 3 Subclasses
‣ CameraCharacteristics

‣ CaptureRequest
‣ CaptureResult

30

|

© 2013 Aptina Imaging Corporation
Camera Metadata – Static
• Characteristics for an Individual Camera Device
• Static – does not change for a device, so no need to
open a Camera to get this information.
‣ Can be queried from Camera Manager or Camera Device

• Keys are of the format - “Available” or “Info”
• Some interesting Metadata Keys
‣ INFO_SUPPORTED_HARDWARE_LEVEL : Limited or Full
‣ 3A : CONTROL_AE_*, CONTROL_AWB_*, CONTROL_AF_*
‣ LENS_INFO_*, FLASH_INFO_*, SENSOR_INFO_*

31

|

© 2013 Aptina Imaging Corporation
Camera Metadata - Dynamic
• Dynamic Metadata is “per frame”
• Set in CaptureRequest via the Builder to specify
settings while capturing a frame.
• Returned as part of the CaptureResult
• One has to be a “pro” with Image Sensors and Image
Sensing Pipelines to use this.
• Very long list of options available

32

|

© 2013 Aptina Imaging Corporation
How do we use these APIs?
•

To query whether the APIs are available or not on a platform, use
reflection.

•

{@hide} only means not available in SDK.

•

Steps to setup development environment
‣

Connect KitKat Device or start KitKat AVD
adb pull /system/framework/core.jar .
adb pull /system/framework/framework.jar .

‣

Convert from Dex to Jar using tool such as dex2jar
dex2jar core.jar
dex2jar framework.jar

‣

In Eclipse, Project → Properties → Java Build Path → Libraries → Add External
JARs... → core-dex2jar.jar & framework-dex2jar.jar
Source : stackoverflow

•
33

APIs don’t work on AVD because the HAL is a Version 1 Implementation
|

© 2013 Aptina Imaging Corporation
• https://github.com/lbk003/Cam3HiddenAPIs

34

|

© 2013 Aptina Imaging Corporation
The Platform

35

|

© 2013 Aptina Imaging Corporation
AOSP Architecture
Application
Framework/SDK

Java Classes Exposed to the
application for interacting with
the hardware

frameworks/base/core/java/and
roid/hardware

JNI

Glue Code between Java Classes
and Native Classes

frameworks/base/core/jni/andro
id_hardware_Camera.cpp

Native
Framework

Native counterpart of the Java
Camera Classes. Manages all
Binder Interactions

frameworks/av/camera/Camera.
cpp

IPC Binder

3 Binder Interfaces
ICameraService and Icamera
from Application to Framework.
ICameraClient for callbacks into
the Application

frameworks/av/camera

Camera Service

The Camera Service that
manages permissions and
lifecycle of the Camera Devices

frameworks/av/services/camera
/libcameraservice/CameraServic
e.cpp

HAL Interface

Hardware Interface. Multiple
Versions

platform/hardware/libhardware/
include

HAL
Implementation

Hardware specific
implementations. Depends on
host processor, ISP and Sensor

platform/hardware/<vendor>/<pl
atform>

Kernel drivers

http://source.android.com/devices/camera/camera.html
36

|

© 2013 Aptina Imaging Corporation
Camera Version Saga
Camera
HAL
Version

Description

1.0

Android 4.0 (Icecream Sandwich)

Converted from C++
CameraHardwareInterface abstraction
layer to camera.h

2.0

Android 4.2 (Jelly
Bean MR1)

Initial Release of expanded capability HAL. camera2.h
Allowed ZSL in the service layer.

3.0

Android 4.3 (Jelly
Bean MR2)

First revision of expanded-capability
HAL camera3.h. Operational mode
same as Camera 2, but ABI different,
and optimized to support multiple
host architectures

3.1
37

Android Version

Android 4.4 (Kit
Kat)

Minor Revision. Added flush() call to
drop all inflight requests immediately

|

© 2013 Aptina Imaging Corporation
Camera Version Saga (contd)
•

platform/hardware/libhardware/include/camera_common.h

•

Camera HAL version = CAMERA_DEVICE_API_VERSION

•

CAMERA_MODULE_API_VERSION is different
CAMERA_MODULE_API_
VERSION

CAMERA_DEVICE_A
PI_VERSION

Android Version

(Camera HAL
Version)

1.0

1.0

Android 4.0 (Ice-cream Sandwich)

2.0

2.0

Android 4.2 (Jelly Bean MR1)

2.1

3.0

Android 4.3 (Jelly Bean MR2)

2.2

3.1

Android 4.4 (Kit Kat)

• CAMERA_DEVICE_API_VERSION 2.x is outdated.

38

|

© 2013 Aptina Imaging Corporation
Camera 3 HAL
• Supports 2 modes – Limited and Full
• Limited just means Camera version 1

• Camera HAL version 3 and 1 can co-exist. For example,
a front facing camera may be a version 1 device,
whereas the rear facing one could be a version 3
device.

39

|

© 2013 Aptina Imaging Corporation
Camera 1 HAL

http://source.android.com/devices/camera/camera.html
40

|

© 2013 Aptina Imaging Corporation
Camera 3 Core Operation Model

http://source.android.com/devices/camera/camera.html
41

|

© 2013 Aptina Imaging Corporation
Camera 3 HAL Model

http://source.android.com/devices/camera/camera.html
42

|

© 2013 Aptina Imaging Corporation
Camera Pipeline

http://source.android.com/devices/camera/camera.html
43

|

© 2013 Aptina Imaging Corporation
3A
•

3A = Auto Focus, Auto Exposure, Auto White Balance

•

Along with demosaicing and denoising, forms a crucial part of the
image pipeline.

•

The 3A algorithms are typically the “secret sauce” of the ISP

•

The State Machine for each of the 3A is defined by the HAL.

•

When a device is opened the 3A is INACTIVE

•

3A algorithms at the top level are controlled by
ANDROID_CONTROL_MODE key
1.

ANDROID_CONTROL_MODE_OFF

2.

ANDROID_CONTROL_MODE_AUTO

3.

ANDROID_CONTROL_USE_SCENE_MODE
•

44

|

SCENE_MODE_FACE_PRIORITY -the AE/AWB/AFMODE controls work as in
ANDROID_CONTROL_MODE_AUTO, but the 3A routines must bias toward metering and focusing
on any detected faces in the scene.

© 2013 Aptina Imaging Corporation
Open Source Implementation of HAL3
Nexus 5
• https://android.googlesource.com/platform/hardware/
qcom/camera/+/kitkat-release/QCamera2/HAL3/
• Released for the Qualcomm MSM8974 Snapdragon 800
based device

45

|

© 2013 Aptina Imaging Corporation
Implementing Camera3 HALs
• Overall with Camera3 specification, the Camera HAL
implementations have become simpler, since most of the
functionality like stream management, etc. are defined and have
moved into the Service Layer
• Most Camera HAL implementations have had required considerable
amount of cleanup and refactoring.
• Most of the work is getting the Hardware vendors (AP, ISPs) ready
to support these features
• Devices with Camera 3 implementations - Nexus 7 (4.3) and Nexus
5
• Major progress but there are areas of investigations - like
processing of non-Bayer sensors, multi-stream, multi – array
sensors etc.

46

|

© 2013 Aptina Imaging Corporation
A Peek into the Future

47

|

© 2013 Aptina Imaging Corporation
Emerging Trends
• Photography is going to become a lot about the software as
processing from offline computers to smart devices
• Imagers for information
‣ Computer Vision, Gesture Recognition, Augmented Reality

• Non-Bayer Sensors
‣ Clarity+

• High Dynamic Range
‣ HDR in Preview, Video

• Multi Sensor Imagers – Array imagers, Stereo

48

|

© 2013 Aptina Imaging Corporation
DEMO
49

|

© 2013 Aptina Imaging Corporation
Q&A
50

|

© 2013 Aptina Imaging Corporation
References
•

Computational Photography
‣
‣

http://graphics.stanford.edu/projects/lightfield/

‣

http://fcam.garage.maemo.org/

‣

http://graphics.stanford.edu/courses/cs178/applets/

‣

https://graphics.stanford.edu/papers/fcam/fcam-sigtalk.pdf

‣

•

http://graphics.stanford.edu/projects/camera-2.0/

http://graphics.stanford.edu/papers/camera20/levoy-platforms-cga10.pdf

Android
‣
‣

http://source.android.com/devices/camera/camera.html

‣

•

http://source.android.com
http://android.googlesource.com

Clarity+
‣

51

http://www.aptina.com/Aptina_ClarityPlus_WhitePaper.pdf

|

© 2013 Aptina Imaging Corporation
About
Aptina is a global provider of CMOS imaging solutions that enable
Imaging Everywhere™. Using performance enhancing technologies like
Aptina A-Pix™, Aptina A-PixHS™ , DR-Pix™ and Clarity+™ technology,
Aptina has created a market-leading portfolio of image sensor
products found in leading consumer electronics like smart phones,
tablets, laptops, digital and video cameras, as well as applications in
surveillance, automotive, medical, video conferencing, and gaming.
Clarity+ is Aptina’s breakthrough clear pixel technology that
provides 2x improvement in image quality under low light conditions.

Contact Information
Balwinder Kaur bkaur@aptina.com
52

|

© 2013 Aptina Imaging Corporation
Hidden Camera 3 APIs in Android 4.4 (KitKat)

More Related Content

What's hot

Android Boot Time Optimization
Android Boot Time OptimizationAndroid Boot Time Optimization
Android Boot Time Optimization
Kan-Ru Chen
 
Embedded Android : System Development - Part IV
Embedded Android : System Development - Part IVEmbedded Android : System Development - Part IV
Embedded Android : System Development - Part IV
Emertxe Information Technologies Pvt Ltd
 
Init of Android
Init of AndroidInit of Android
Init of Android
Tetsuyuki Kobayashi
 
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
Nanik Tolaram
 
Using the Android Native Development Kit (NDK)
Using the Android Native Development Kit (NDK)Using the Android Native Development Kit (NDK)
Using the Android Native Development Kit (NDK)
Xavier Hallade
 
ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?
Tetsuyuki Kobayashi
 
From Android NDK To AOSP
From Android NDK To AOSPFrom Android NDK To AOSP
From Android NDK To AOSP
Min-Yih Hsu
 
Android graphic system (SurfaceFlinger) : Design Pattern's perspective
Android graphic system (SurfaceFlinger) : Design Pattern's perspectiveAndroid graphic system (SurfaceFlinger) : Design Pattern's perspective
Android graphic system (SurfaceFlinger) : Design Pattern's perspective
Bin Chen
 
Embedded Android : System Development - Part III (Audio / Video HAL)
Embedded Android : System Development - Part III (Audio / Video HAL)Embedded Android : System Development - Part III (Audio / Video HAL)
Embedded Android : System Development - Part III (Audio / Video HAL)
Emertxe Information Technologies Pvt Ltd
 
Android 10
Android 10Android 10
Low Level View of Android System Architecture
Low Level View of Android System ArchitectureLow Level View of Android System Architecture
Low Level View of Android System Architecture
National Cheng Kung University
 
Overview of Android binder IPC implementation
Overview of Android binder IPC implementationOverview of Android binder IPC implementation
Overview of Android binder IPC implementation
Chethan Pchethan
 
Android Internals
Android InternalsAndroid Internals
Android Internals
Opersys inc.
 
Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?
Opersys inc.
 
Android Binder IPC for Linux
Android Binder IPC for LinuxAndroid Binder IPC for Linux
Android Binder IPC for Linux
Yu-Hsin Hung
 
Embedded Graphics Drivers in Mesa (ELCE 2019)
Embedded Graphics Drivers in Mesa (ELCE 2019)Embedded Graphics Drivers in Mesa (ELCE 2019)
Embedded Graphics Drivers in Mesa (ELCE 2019)
Igalia
 
Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...
Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...
Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...
Linaro
 
Android presentation
Android presentationAndroid presentation
Android presentation
Siva Ramakrishna kv
 
Binder: Android IPC
Binder: Android IPCBinder: Android IPC
Binder: Android IPC
Shaul Rosenzwieg
 

What's hot (20)

Android Boot Time Optimization
Android Boot Time OptimizationAndroid Boot Time Optimization
Android Boot Time Optimization
 
Embedded Android : System Development - Part IV
Embedded Android : System Development - Part IVEmbedded Android : System Development - Part IV
Embedded Android : System Development - Part IV
 
Init of Android
Init of AndroidInit of Android
Init of Android
 
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
 
Using the Android Native Development Kit (NDK)
Using the Android Native Development Kit (NDK)Using the Android Native Development Kit (NDK)
Using the Android Native Development Kit (NDK)
 
ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?ADB(Android Debug Bridge): How it works?
ADB(Android Debug Bridge): How it works?
 
Android binder-ipc
Android binder-ipcAndroid binder-ipc
Android binder-ipc
 
From Android NDK To AOSP
From Android NDK To AOSPFrom Android NDK To AOSP
From Android NDK To AOSP
 
Android graphic system (SurfaceFlinger) : Design Pattern's perspective
Android graphic system (SurfaceFlinger) : Design Pattern's perspectiveAndroid graphic system (SurfaceFlinger) : Design Pattern's perspective
Android graphic system (SurfaceFlinger) : Design Pattern's perspective
 
Embedded Android : System Development - Part III (Audio / Video HAL)
Embedded Android : System Development - Part III (Audio / Video HAL)Embedded Android : System Development - Part III (Audio / Video HAL)
Embedded Android : System Development - Part III (Audio / Video HAL)
 
Android 10
Android 10Android 10
Android 10
 
Low Level View of Android System Architecture
Low Level View of Android System ArchitectureLow Level View of Android System Architecture
Low Level View of Android System Architecture
 
Overview of Android binder IPC implementation
Overview of Android binder IPC implementationOverview of Android binder IPC implementation
Overview of Android binder IPC implementation
 
Android Internals
Android InternalsAndroid Internals
Android Internals
 
Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?
 
Android Binder IPC for Linux
Android Binder IPC for LinuxAndroid Binder IPC for Linux
Android Binder IPC for Linux
 
Embedded Graphics Drivers in Mesa (ELCE 2019)
Embedded Graphics Drivers in Mesa (ELCE 2019)Embedded Graphics Drivers in Mesa (ELCE 2019)
Embedded Graphics Drivers in Mesa (ELCE 2019)
 
Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...
Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...
Secure Boot on ARM systems – Building a complete Chain of Trust upon existing...
 
Android presentation
Android presentationAndroid presentation
Android presentation
 
Binder: Android IPC
Binder: Android IPCBinder: Android IPC
Binder: Android IPC
 

Viewers also liked

How to create a camera2
How to create a camera2How to create a camera2
How to create a camera2
Booch Lin
 
Android Camera Architecture
Android Camera ArchitectureAndroid Camera Architecture
Android Camera Architecture
Picker Weng
 
Camera2 introdction
Camera2 introdctionCamera2 introdction
Camera2 introdction
Booch Lin
 
Camera2 API, SHIM, and HAL 3.2 in Android 5.1
Camera2 API, SHIM, and HAL 3.2 in Android 5.1Camera2 API, SHIM, and HAL 3.2 in Android 5.1
Camera2 API, SHIM, and HAL 3.2 in Android 5.1
Cheng Hsien Chen
 
あ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていた
あ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていたあ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていた
あ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていた
Yamashita Takeshi
 
Introduction of Android Camera1
Introduction of Android Camera1Introduction of Android Camera1
Introduction of Android Camera1
Booch Lin
 
Camera API 3.0を触ってみた
Camera API 3.0を触ってみたCamera API 3.0を触ってみた
Camera API 3.0を触ってみた
Masaki Otsuki
 
IoT in the Enterprise
IoT in the EnterpriseIoT in the Enterprise
IoT in the Enterprise
Balwinder Kaur
 
Going Beyond the Device Heart Beat
Going Beyond the Device Heart BeatGoing Beyond the Device Heart Beat
Going Beyond the Device Heart Beat
Balwinder Kaur
 
Camera2
Camera2Camera2
カメラアプリ実装のコツ
カメラアプリ実装のコツカメラアプリ実装のコツ
カメラアプリ実装のコツ
tac0x2a
 
Kitkat
KitkatKitkat
Kitkat
Risha Bagchi
 
GreenDao Introduction
GreenDao IntroductionGreenDao Introduction
GreenDao Introduction
Booch Lin
 
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernelKernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
Anne Nicolas
 
Embedded Linux Multimedia
Embedded Linux MultimediaEmbedded Linux Multimedia
Embedded Linux Multimedia
Caglar Dursun
 
Kernel Recipes 2014 - Testing Video4Linux Applications and Drivers
Kernel Recipes 2014 - Testing Video4Linux Applications and DriversKernel Recipes 2014 - Testing Video4Linux Applications and Drivers
Kernel Recipes 2014 - Testing Video4Linux Applications and Drivers
Anne Nicolas
 
Multimedia con GNU/Linux
Multimedia con GNU/LinuxMultimedia con GNU/Linux
Multimedia con GNU/Linux
Stefano Sabatini
 
Android internals 03 - Build system, emulator (rev_1.1)
Android internals 03 - Build system, emulator (rev_1.1)Android internals 03 - Build system, emulator (rev_1.1)
Android internals 03 - Build system, emulator (rev_1.1)
Egor Elizarov
 
안드로이드 와 디바이스 드라이버 적용 기법
안드로이드 와 디바이스 드라이버 적용 기법안드로이드 와 디바이스 드라이버 적용 기법
안드로이드 와 디바이스 드라이버 적용 기법
chon2010
 
Understanding the Android System Server
Understanding the Android System ServerUnderstanding the Android System Server
Understanding the Android System Server
Opersys inc.
 

Viewers also liked (20)

How to create a camera2
How to create a camera2How to create a camera2
How to create a camera2
 
Android Camera Architecture
Android Camera ArchitectureAndroid Camera Architecture
Android Camera Architecture
 
Camera2 introdction
Camera2 introdctionCamera2 introdction
Camera2 introdction
 
Camera2 API, SHIM, and HAL 3.2 in Android 5.1
Camera2 API, SHIM, and HAL 3.2 in Android 5.1Camera2 API, SHIM, and HAL 3.2 in Android 5.1
Camera2 API, SHIM, and HAL 3.2 in Android 5.1
 
あ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていた
あ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていたあ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていた
あ・・・ありのまま今起こったことを話すぜ!おれはTextureViewでプロジェクトを開始していたと思ったらいつのまにかSurfaceViewに戻っていた
 
Introduction of Android Camera1
Introduction of Android Camera1Introduction of Android Camera1
Introduction of Android Camera1
 
Camera API 3.0を触ってみた
Camera API 3.0を触ってみたCamera API 3.0を触ってみた
Camera API 3.0を触ってみた
 
IoT in the Enterprise
IoT in the EnterpriseIoT in the Enterprise
IoT in the Enterprise
 
Going Beyond the Device Heart Beat
Going Beyond the Device Heart BeatGoing Beyond the Device Heart Beat
Going Beyond the Device Heart Beat
 
Camera2
Camera2Camera2
Camera2
 
カメラアプリ実装のコツ
カメラアプリ実装のコツカメラアプリ実装のコツ
カメラアプリ実装のコツ
 
Kitkat
KitkatKitkat
Kitkat
 
GreenDao Introduction
GreenDao IntroductionGreenDao Introduction
GreenDao Introduction
 
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernelKernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
 
Embedded Linux Multimedia
Embedded Linux MultimediaEmbedded Linux Multimedia
Embedded Linux Multimedia
 
Kernel Recipes 2014 - Testing Video4Linux Applications and Drivers
Kernel Recipes 2014 - Testing Video4Linux Applications and DriversKernel Recipes 2014 - Testing Video4Linux Applications and Drivers
Kernel Recipes 2014 - Testing Video4Linux Applications and Drivers
 
Multimedia con GNU/Linux
Multimedia con GNU/LinuxMultimedia con GNU/Linux
Multimedia con GNU/Linux
 
Android internals 03 - Build system, emulator (rev_1.1)
Android internals 03 - Build system, emulator (rev_1.1)Android internals 03 - Build system, emulator (rev_1.1)
Android internals 03 - Build system, emulator (rev_1.1)
 
안드로이드 와 디바이스 드라이버 적용 기법
안드로이드 와 디바이스 드라이버 적용 기법안드로이드 와 디바이스 드라이버 적용 기법
안드로이드 와 디바이스 드라이버 적용 기법
 
Understanding the Android System Server
Understanding the Android System ServerUnderstanding the Android System Server
Understanding the Android System Server
 

Similar to Hidden Camera 3 APIs in Android 4.4 (KitKat)

COSCUP 2017 FACE OFF
COSCUP 2017 FACE OFFCOSCUP 2017 FACE OFF
COSCUP 2017 FACE OFF
PRADA Hsiung
 
Implementing Computational Camera
Implementing Computational CameraImplementing Computational Camera
Implementing Computational Camera
Qualcomm Developer Network
 
UplinQ - implementing computational camera
UplinQ - implementing computational cameraUplinQ - implementing computational camera
UplinQ - implementing computational camera
Satya Harish
 
"Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App...
"Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App..."Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App...
"Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App...
Edge AI and Vision Alliance
 
xmobots at the Scade User Group Conference 2013
xmobots at the Scade User Group Conference 2013xmobots at the Scade User Group Conference 2013
xmobots at the Scade User Group Conference 2013
Ansys
 
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a...
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a..."Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a...
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a...
Edge AI and Vision Alliance
 
IBM Mobile Quality Assurance Open Beta Study Group Session 3 (Instrumenting...
IBM Mobile Quality Assurance   Open Beta Study Group Session 3 (Instrumenting...IBM Mobile Quality Assurance   Open Beta Study Group Session 3 (Instrumenting...
IBM Mobile Quality Assurance Open Beta Study Group Session 3 (Instrumenting...
Roger Snook
 
Avigilon acc6.10.0.24-release-notes
Avigilon acc6.10.0.24-release-notesAvigilon acc6.10.0.24-release-notes
Avigilon acc6.10.0.24-release-notes
Danilo Moreira
 
Video Surveillance Report
Video Surveillance ReportVideo Surveillance Report
Video Surveillance Report
Mihika Shah
 
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
Edge AI and Vision Alliance
 
Computer Vision Technology and Expertise
Computer Vision Technology and ExpertiseComputer Vision Technology and Expertise
Computer Vision Technology and Expertise
Rhonda Software
 
IRJET- Optimization of Surveillance Camera for Low Cost Storage Device
IRJET- Optimization of Surveillance Camera for Low Cost Storage DeviceIRJET- Optimization of Surveillance Camera for Low Cost Storage Device
IRJET- Optimization of Surveillance Camera for Low Cost Storage Device
IRJET Journal
 
Securescape Introduction
Securescape IntroductionSecurescape Introduction
Securescape Introduction
Daden Limited
 
ARM: Enhancing your Unity mobile VR experience
ARM: Enhancing your Unity mobile VR experienceARM: Enhancing your Unity mobile VR experience
ARM: Enhancing your Unity mobile VR experience
DevGAMM Conference
 
Machine Vision Cameras
Machine Vision CamerasMachine Vision Cameras
Machine Vision Cameras
📘 Parth Desai
 
"Addressing Corner Cases in Embedded Computer Vision Applications," a Present...
"Addressing Corner Cases in Embedded Computer Vision Applications," a Present..."Addressing Corner Cases in Embedded Computer Vision Applications," a Present...
"Addressing Corner Cases in Embedded Computer Vision Applications," a Present...
Edge AI and Vision Alliance
 
Mopcon2017 - AppDevKit x CameraKit
Mopcon2017 - AppDevKit x CameraKitMopcon2017 - AppDevKit x CameraKit
Mopcon2017 - AppDevKit x CameraKit
anistar sung
 
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
IRJET Journal
 
Video Stabilization using Python and open CV
Video Stabilization using Python and open CVVideo Stabilization using Python and open CV
Video Stabilization using Python and open CV
IRJET Journal
 
Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools
Matteo Valoriani
 

Similar to Hidden Camera 3 APIs in Android 4.4 (KitKat) (20)

COSCUP 2017 FACE OFF
COSCUP 2017 FACE OFFCOSCUP 2017 FACE OFF
COSCUP 2017 FACE OFF
 
Implementing Computational Camera
Implementing Computational CameraImplementing Computational Camera
Implementing Computational Camera
 
UplinQ - implementing computational camera
UplinQ - implementing computational cameraUplinQ - implementing computational camera
UplinQ - implementing computational camera
 
"Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App...
"Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App..."Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App...
"Snapdragon Hybrid Computer Vision/Deep Learning Architecture for Imaging App...
 
xmobots at the Scade User Group Conference 2013
xmobots at the Scade User Group Conference 2013xmobots at the Scade User Group Conference 2013
xmobots at the Scade User Group Conference 2013
 
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a...
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a..."Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a...
"Accessing Advanced Image Processing Feature Sets with Alvium Cameras Using a...
 
IBM Mobile Quality Assurance Open Beta Study Group Session 3 (Instrumenting...
IBM Mobile Quality Assurance   Open Beta Study Group Session 3 (Instrumenting...IBM Mobile Quality Assurance   Open Beta Study Group Session 3 (Instrumenting...
IBM Mobile Quality Assurance Open Beta Study Group Session 3 (Instrumenting...
 
Avigilon acc6.10.0.24-release-notes
Avigilon acc6.10.0.24-release-notesAvigilon acc6.10.0.24-release-notes
Avigilon acc6.10.0.24-release-notes
 
Video Surveillance Report
Video Surveillance ReportVideo Surveillance Report
Video Surveillance Report
 
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
 
Computer Vision Technology and Expertise
Computer Vision Technology and ExpertiseComputer Vision Technology and Expertise
Computer Vision Technology and Expertise
 
IRJET- Optimization of Surveillance Camera for Low Cost Storage Device
IRJET- Optimization of Surveillance Camera for Low Cost Storage DeviceIRJET- Optimization of Surveillance Camera for Low Cost Storage Device
IRJET- Optimization of Surveillance Camera for Low Cost Storage Device
 
Securescape Introduction
Securescape IntroductionSecurescape Introduction
Securescape Introduction
 
ARM: Enhancing your Unity mobile VR experience
ARM: Enhancing your Unity mobile VR experienceARM: Enhancing your Unity mobile VR experience
ARM: Enhancing your Unity mobile VR experience
 
Machine Vision Cameras
Machine Vision CamerasMachine Vision Cameras
Machine Vision Cameras
 
"Addressing Corner Cases in Embedded Computer Vision Applications," a Present...
"Addressing Corner Cases in Embedded Computer Vision Applications," a Present..."Addressing Corner Cases in Embedded Computer Vision Applications," a Present...
"Addressing Corner Cases in Embedded Computer Vision Applications," a Present...
 
Mopcon2017 - AppDevKit x CameraKit
Mopcon2017 - AppDevKit x CameraKitMopcon2017 - AppDevKit x CameraKit
Mopcon2017 - AppDevKit x CameraKit
 
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
 
Video Stabilization using Python and open CV
Video Stabilization using Python and open CVVideo Stabilization using Python and open CV
Video Stabilization using Python and open CV
 
Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools
 

Recently uploaded

CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
Claudio Di Ciccio
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 

Recently uploaded (20)

CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 

Hidden Camera 3 APIs in Android 4.4 (KitKat)

  • 1. Camera 3 The New Camera Subsystem in Android KitKat Balwinder Kaur, Senior Member, Technical Staff, Aptina Imaging AnDevCon, Burlingame, CA 11.15.13 Slide Deck v 1.2 last updated 12.27.13 © 2013 Aptina Imaging Corporation. All rights reserved. Products are warranted only to meet Aptina’s production data sheet specifications. Information, products, and/or specifications are subject to change without notice. All information is provided on an “AS IS” basis without warranties of any kind. Dates are estimates only. Drawings not to scale. Aptina and the Aptina logo are trademarks of Aptina Imaging Corporation. All other trademarks are the property of their respective owners. 1 | © 2013 Aptina Imaging Corporation
  • 2. Agenda I. Camera 3 ‣ Introduction ‣ History ‣ Computational Photography Use Cases II. The APIs ‣ {@hide} – Sneak Peek into coming Android Pro Camera APIs III. The Platform ‣ Architecture ‣ 3A IV. Emerging Trends V. Demo VI. Q&A 2 | © 2013 Aptina Imaging Corporation
  • 3. Camera 3 – The New Camera Framework 3 | © 2013 Aptina Imaging Corporation
  • 4. Limitations of Camera 1 • 3 Operating Modes ‣ Preview ‣ Capture ‣ Video • Burst Mode Photography – hard to implement ‣ Zero Shutter Lag ‣ Multi-Shot HDR ‣ Panoramic Stitch • Metadata: Practically, only face detection data could be returned to the application 4 | © 2013 Aptina Imaging Corporation
  • 5. Camera 3 • Android 4.3 (Jelly Bean MR2) and Android 4.4 (KitKat) have shipped with a new Camera Hardware Interface ‣ Android 4.3 => Camera 3 Hardware Interface ‣ Android 4.4 => Camera 3.1 Hardware Interface What happened to Camera2 ? 5 | © 2013 Aptina Imaging Corporation
  • 6. Overview of Camera 3 • Camera Framework till Android 4.0 was modeled after a point & shoot camera. • Expanded Camera Subsystem is modeled after a Professional Camera ‣ • 6 But it is more than that It is the basis of a potential paradigm shift in photography | © 2013 Aptina Imaging Corporation
  • 7. A Bit of History FCAM F2 Frankencamera Project Lead - Stanford Professor Marc Levoy Main architects - Eino-Ville (Eddy) Talvala and Andrew Adams Eddy Talvala is the Tech Lead for Camera Framework on the Android Team Industry Partner - Nokia Computational Photography ‣ Out of scope for this talk ‣ Good starting point - A white paper on Computational Photography Platforms by Marc Levoy ‣ Use and relevance of software increases in photography ‣ Processing happens in near real time instead of post-processing ‣ Open platforms facilitate innovation 7 | © 2013 Aptina Imaging Corporation
  • 8. Computational Photography Definition • Computational photography refers to computational image capture, processing, and manipulation techniques that enhance or extend the capabilities of digital photography. • Good examples of image capture and manipulation include panoramas, and high-dynamic-range imaging which is the use of differently exposed pictures of the same scene to extend dynamic range beyond even that of analog film-based media. Source: Wikipedia 8 | © 2013 Aptina Imaging Corporation
  • 9. Multi Shot HDR • HDR = High Dynamic Range • Multiple shots at different exposures and then blended together 9 | © 2013 Aptina Imaging Corporation
  • 11. Multi-Flash Photography Source: Stanford Graphics Research 11 | © 2013 Aptina Imaging Corporation
  • 12. Flash No-Flash Photography Source: Microsoft Research 12 | © 2013 Aptina Imaging Corporation
  • 13. Flash No-Flash Photography Source: Microsoft Research 13 | © 2013 Aptina Imaging Corporation
  • 14. Focus Bracketing Focus Stacking for macro photography Source: Wikipedia 14 | © 2013 Aptina Imaging Corporation
  • 15. View Finder Editing Today’s smartphones have features like • ‣ Computational Photography can take it to the next level • ‣ • 15 Touch to Expose & Touch to Focus Edit the frame in the view finder by selecting areas to sharpen, blur, brighten and maintain these during capture mode Source: SIGGRAPH 2012 | © 2013 Aptina Imaging Corporation
  • 16. Open Platform Requirements for Computation Photography • Fine grained control of the Sensor, Flash, Lens and Image Signal Processing pipeline • Ability to program ‣ exposure time, auto white balance, focus, frame-rate etc. ‣ metering algorithm, autofocus algorithm etc. • Meta data ‣ Every frame is returned with the settings that it was taken with (verses settings take effect sometime in future) • Platform should have enough compute and memory power. ‣ With multi core devices and GPU capability increasing, Android becomes a very attractive platform as an open computational photography platform. 16 | © 2013 Aptina Imaging Corporation
  • 17. Other use cases • Using Imagers for Information ‣ Gesture Recognition ‣ Face Recognition ‣ Augmented Reality • Requirements differ for this class of applications ‣ High and steady Frame Rate under all light conditions is Important. ‣ Of less importance • • 17 | High Resolution Image Quality & Color Accuracy © 2013 Aptina Imaging Corporation
  • 18. The APIs 18 | © 2013 Aptina Imaging Corporation
  • 19. Camera API Camera APIs Package - android.hardware.camera Android Version Additions to android.hardware.camera Kit Kat (4.4) API level 19 None Jelly Bean MR2 (4.3) API level 18 None Jelly Bean MR1 (4.2) API level 17 Minor (HDR, shutterSound) Jelly Bean (4.1) API level 16 Minor (AutoFocus Move Callback) Ice-cream Sandwich API level 15 Minor (Video Stabilization) Ice-cream Sandwich 19 API Level API level 14 Major (Face Detection) | © 2013 Aptina Imaging Corporation
  • 20. A Sneak Peek of the New Camera APIs • Kitkat shipped with a set of Experimental Java camera APIs with the {@hide} tag. ‣ https://android.googlesource.com/platform/frameworks/base /+/kitkat-release/core/java/android/hardware/camera2/ • Package name is android.hardware.camera2 ‣ But in other parts of AOSP many references to ProCamera (Professional Camera) are found. 20 | © 2013 Aptina Imaging Corporation
  • 21. Pro Camera APIs ? • They are experimental and not part of the SDK, but available in the platform. • Likely to change before released in the SDK. • Camera 3 Hardware Interface is fairly stable now. • It’s a good time to start thinking about new Camera applications 21 | © 2013 Aptina Imaging Corporation
  • 22. android.hardware.camera2 Interface Camera Device Exception CameraAccessException 22 | © 2013 Aptina Imaging Corporation Classes CameraCharacteristics CameraDevice.CaptureListener CameraDevice.StateListener CameraManager CameraManager.AvailabilityListener CameraMetadata CameraMetadata.Key<t> CaptureFailure CaptureRequest CaptureRequest.Builder CaptureResult Face Rational Size
  • 23. Camera Manager Interface for iterating, listing and accessing Camera Devices CameraManager manager = (CameraManager) getSystemService(Context.CAMERA_SERVICE); //Iterating through Camera Devices manager.getCameraIdList(); // Query a device for Capabilities manager.getCameraCharacteristics(cameraId); // Open a Camera manager.openCamera ( cameraId,...); // add an Availability Listener as Cameras become available or unavailable manager.addAvailabilityListener(CameraManager.AvailabilityListener); 23 | © 2013 Aptina Imaging Corporation
  • 24. Camera Device • Device is modeled as a pipeline ‣ Input : CaptureRequest for a single Frame ‣ Output: CaptureResult Metadata packet + output image buffers for the request. These buffers can go to one or multiple output targets • SurfaceView • SurfaceTexture • MediaCodec • MediaRecorder • Renderscript Allocation et al • ImageReader (Use this for Still Images) ‣ ‣ 24 Requests are processed in order, multiple requests are in flight to maintain full frame rate ( generally means 30 frames per second ) Android 4.4 added the flush() command to drop all inflight requests. | © 2013 Aptina Imaging Corporation
  • 25. Capture Request // Step 1 : Configure Output Surfaces camera.configureOutputs (List<Surfaces> outputs) // Set the size and format. // Step 2: Build a CaptureRequest // Use default Templates – Manual, Preview, Still_Capture, // Record, Video_Snapshot, Zero_Shutter_Lag CaptureRequest.Builder bldr = camera.createCaptureRequest(CameraDevice.TEMPLATE_VIDEO_SNAPSHOT); //no chaining of calls available bldr.addTarget(); bldr.setTag(); // to identify the result when it is returned bldr.set (Metadata.Key …); CaptureRequest req = bldr.build(); 25 | © 2013 Aptina Imaging Corporation
  • 26. Capture Modes //Step 3: Submit a request //Pick 1 of the 4 // capture, captureBurst, setRepeatingRequest, setRepeatingBurst camera.capture(CaptureRequest, CameraDevice.CaptureListener) //one time capture camera.captureBurst(…) // Bracketing Modes. Guaranteed that no other request will be interspersed in this queue camera.setRepeatingRequest (…) // Preview. Lower priority. Cancelled with stopRepeating() or flush() call. videoRates – needs clarification camera.setRepeatingBurst (…) // same as setRepeatingBurst but at full frame rates 26 | © 2013 Aptina Imaging Corporation
  • 27. CaptureListener //Step 4: Process the progress of the capture //4 callback methods available onCaptureStarted(CameraDevice, CaptureRequest, long timestamp); onCaptureCompleted(CameraDevice, CaptureRequest, CaptureResult); //or onCaptureFailed(CameraDevice, CaptureRequest, CaptureFailure); // once all the capture requests are processed onCaptureSequenceCompleted(CameraDevice, int seqid, int framenumber) 27 | © 2013 Aptina Imaging Corporation
  • 28. Capture Result //Step 5: Investigate the CaptureResult or CaptureFailure Object Received // The results of a single image capture from the //image sensor. Contains the final configuration for //the capture hardware (sensor, lens, flash), the //processing pipeline, the control algorithms, //and the output buffers. get(CameraMetadata.Key<T> key) ; getRequest() ; 28 | © 2013 Aptina Imaging Corporation
  • 29. ImageReader • Introduced in KitKat API Level 19 • Android.media.ImageReader • Allows direct application access to image data rendered into a Surface • maxImages determines the maximum number of Image objects that can be be acquired from the ImageReader simultaneously. Use cautiously ! • ImageFormat is defined by ImageFormat or PixelFormat • ImageReader.OnImageAvailableListener provides callback for when the next Image is available • Use acquireLatestImage, rather than acquireNextImage. It removes old images ImageReader imgreader = ImageReader.newInstance(width, height, format, maxImages); imgreader.getSurface() ; // Pass this into the CameraRequest Class 29 | © 2013 Aptina Imaging Corporation
  • 30. Camera Metadata • Base class for Camera Controls and Information • Metadata is of 2 types ‣ Static and Dynamic • Sensor Control • Flash Control • Lens Control • Pipeline Control • 3 Subclasses ‣ CameraCharacteristics ‣ CaptureRequest ‣ CaptureResult 30 | © 2013 Aptina Imaging Corporation
  • 31. Camera Metadata – Static • Characteristics for an Individual Camera Device • Static – does not change for a device, so no need to open a Camera to get this information. ‣ Can be queried from Camera Manager or Camera Device • Keys are of the format - “Available” or “Info” • Some interesting Metadata Keys ‣ INFO_SUPPORTED_HARDWARE_LEVEL : Limited or Full ‣ 3A : CONTROL_AE_*, CONTROL_AWB_*, CONTROL_AF_* ‣ LENS_INFO_*, FLASH_INFO_*, SENSOR_INFO_* 31 | © 2013 Aptina Imaging Corporation
  • 32. Camera Metadata - Dynamic • Dynamic Metadata is “per frame” • Set in CaptureRequest via the Builder to specify settings while capturing a frame. • Returned as part of the CaptureResult • One has to be a “pro” with Image Sensors and Image Sensing Pipelines to use this. • Very long list of options available 32 | © 2013 Aptina Imaging Corporation
  • 33. How do we use these APIs? • To query whether the APIs are available or not on a platform, use reflection. • {@hide} only means not available in SDK. • Steps to setup development environment ‣ Connect KitKat Device or start KitKat AVD adb pull /system/framework/core.jar . adb pull /system/framework/framework.jar . ‣ Convert from Dex to Jar using tool such as dex2jar dex2jar core.jar dex2jar framework.jar ‣ In Eclipse, Project → Properties → Java Build Path → Libraries → Add External JARs... → core-dex2jar.jar & framework-dex2jar.jar Source : stackoverflow • 33 APIs don’t work on AVD because the HAL is a Version 1 Implementation | © 2013 Aptina Imaging Corporation
  • 35. The Platform 35 | © 2013 Aptina Imaging Corporation
  • 36. AOSP Architecture Application Framework/SDK Java Classes Exposed to the application for interacting with the hardware frameworks/base/core/java/and roid/hardware JNI Glue Code between Java Classes and Native Classes frameworks/base/core/jni/andro id_hardware_Camera.cpp Native Framework Native counterpart of the Java Camera Classes. Manages all Binder Interactions frameworks/av/camera/Camera. cpp IPC Binder 3 Binder Interfaces ICameraService and Icamera from Application to Framework. ICameraClient for callbacks into the Application frameworks/av/camera Camera Service The Camera Service that manages permissions and lifecycle of the Camera Devices frameworks/av/services/camera /libcameraservice/CameraServic e.cpp HAL Interface Hardware Interface. Multiple Versions platform/hardware/libhardware/ include HAL Implementation Hardware specific implementations. Depends on host processor, ISP and Sensor platform/hardware/<vendor>/<pl atform> Kernel drivers http://source.android.com/devices/camera/camera.html 36 | © 2013 Aptina Imaging Corporation
  • 37. Camera Version Saga Camera HAL Version Description 1.0 Android 4.0 (Icecream Sandwich) Converted from C++ CameraHardwareInterface abstraction layer to camera.h 2.0 Android 4.2 (Jelly Bean MR1) Initial Release of expanded capability HAL. camera2.h Allowed ZSL in the service layer. 3.0 Android 4.3 (Jelly Bean MR2) First revision of expanded-capability HAL camera3.h. Operational mode same as Camera 2, but ABI different, and optimized to support multiple host architectures 3.1 37 Android Version Android 4.4 (Kit Kat) Minor Revision. Added flush() call to drop all inflight requests immediately | © 2013 Aptina Imaging Corporation
  • 38. Camera Version Saga (contd) • platform/hardware/libhardware/include/camera_common.h • Camera HAL version = CAMERA_DEVICE_API_VERSION • CAMERA_MODULE_API_VERSION is different CAMERA_MODULE_API_ VERSION CAMERA_DEVICE_A PI_VERSION Android Version (Camera HAL Version) 1.0 1.0 Android 4.0 (Ice-cream Sandwich) 2.0 2.0 Android 4.2 (Jelly Bean MR1) 2.1 3.0 Android 4.3 (Jelly Bean MR2) 2.2 3.1 Android 4.4 (Kit Kat) • CAMERA_DEVICE_API_VERSION 2.x is outdated. 38 | © 2013 Aptina Imaging Corporation
  • 39. Camera 3 HAL • Supports 2 modes – Limited and Full • Limited just means Camera version 1 • Camera HAL version 3 and 1 can co-exist. For example, a front facing camera may be a version 1 device, whereas the rear facing one could be a version 3 device. 39 | © 2013 Aptina Imaging Corporation
  • 41. Camera 3 Core Operation Model http://source.android.com/devices/camera/camera.html 41 | © 2013 Aptina Imaging Corporation
  • 42. Camera 3 HAL Model http://source.android.com/devices/camera/camera.html 42 | © 2013 Aptina Imaging Corporation
  • 44. 3A • 3A = Auto Focus, Auto Exposure, Auto White Balance • Along with demosaicing and denoising, forms a crucial part of the image pipeline. • The 3A algorithms are typically the “secret sauce” of the ISP • The State Machine for each of the 3A is defined by the HAL. • When a device is opened the 3A is INACTIVE • 3A algorithms at the top level are controlled by ANDROID_CONTROL_MODE key 1. ANDROID_CONTROL_MODE_OFF 2. ANDROID_CONTROL_MODE_AUTO 3. ANDROID_CONTROL_USE_SCENE_MODE • 44 | SCENE_MODE_FACE_PRIORITY -the AE/AWB/AFMODE controls work as in ANDROID_CONTROL_MODE_AUTO, but the 3A routines must bias toward metering and focusing on any detected faces in the scene. © 2013 Aptina Imaging Corporation
  • 45. Open Source Implementation of HAL3 Nexus 5 • https://android.googlesource.com/platform/hardware/ qcom/camera/+/kitkat-release/QCamera2/HAL3/ • Released for the Qualcomm MSM8974 Snapdragon 800 based device 45 | © 2013 Aptina Imaging Corporation
  • 46. Implementing Camera3 HALs • Overall with Camera3 specification, the Camera HAL implementations have become simpler, since most of the functionality like stream management, etc. are defined and have moved into the Service Layer • Most Camera HAL implementations have had required considerable amount of cleanup and refactoring. • Most of the work is getting the Hardware vendors (AP, ISPs) ready to support these features • Devices with Camera 3 implementations - Nexus 7 (4.3) and Nexus 5 • Major progress but there are areas of investigations - like processing of non-Bayer sensors, multi-stream, multi – array sensors etc. 46 | © 2013 Aptina Imaging Corporation
  • 47. A Peek into the Future 47 | © 2013 Aptina Imaging Corporation
  • 48. Emerging Trends • Photography is going to become a lot about the software as processing from offline computers to smart devices • Imagers for information ‣ Computer Vision, Gesture Recognition, Augmented Reality • Non-Bayer Sensors ‣ Clarity+ • High Dynamic Range ‣ HDR in Preview, Video • Multi Sensor Imagers – Array imagers, Stereo 48 | © 2013 Aptina Imaging Corporation
  • 49. DEMO 49 | © 2013 Aptina Imaging Corporation
  • 50. Q&A 50 | © 2013 Aptina Imaging Corporation
  • 52. About Aptina is a global provider of CMOS imaging solutions that enable Imaging Everywhere™. Using performance enhancing technologies like Aptina A-Pix™, Aptina A-PixHS™ , DR-Pix™ and Clarity+™ technology, Aptina has created a market-leading portfolio of image sensor products found in leading consumer electronics like smart phones, tablets, laptops, digital and video cameras, as well as applications in surveillance, automotive, medical, video conferencing, and gaming. Clarity+ is Aptina’s breakthrough clear pixel technology that provides 2x improvement in image quality under low light conditions. Contact Information Balwinder Kaur bkaur@aptina.com 52 | © 2013 Aptina Imaging Corporation