Describe the relationship of Camera2 API, SHIM, and HAL 3.2 in Android 5.1, also provides some changes in framework to get better result.

1. OUTLINE
Android camera API history
What’s new of Camera API 2
HAL 3.2
Titan Camera API 2 situation
Solutions & Hacks
Conclusions

2. API Lv 21
Camera API 1
Camera API 1(camera)
Camera API 2(camera2)
API Lv ##
Camera API 2
HISTORY

3. ICS
HAL 1.0
JB 4.2 JB 4.3
HAL 1.0 - 3.1
HAL 3.2
HAL 1.0
HAL 2.0
HAL 3.0
KK
HAL 1.0
HAL 2.0
HAL 3.0
HAL 3.1
L
HISTORY

4. WHAT’S NEW OF CAMERA API 2?

5. camera & camera2 are both available to App Developer
BUT - only HAL 3.2 enables all features/properties of camera2
Shim layer
being used to as the Facade layer between new camera2 API
and older HAL 1.0-3.1 by using camera API
WHAT’S NEW OF CAMERA API 2?

6. WHAT’S NEW OF CAMERA API 2?

7. WHAT’S NEW OF CAMERA API 2?
much more metadata, and per-frame metadata.
more direct controls on sensor and Post Processing
Synchronisation of setting and captures
Full rate, full-res captures in various formats
Support raw image capturing and saving
Camera API 1 is necessary for legacy applications

8. WHAT’S NEW OF CAMERA API 2?

9. WHAT’S NEW OF CAMERA API 2?

10. HAL 3.2
support vendor tags by camera module HAL
support partial metadata result
enables all features/properties of camera2
Not support stream configurations for still captures
Framework assumes high-res JPEG stream can be configured at all
times without showing down preview framerate.

11. HAL 3.2
HAL 3.2
FULL mode
LIMITED mode
HAL 1.0 - 3.1
LEGACY Mode

12. HAL 3.2

13. HAL 3.2

14. HAL 3.2

15. Step 1
Step 2Step 3
TITAN CAMERA API 2 SITUATION

16. Step 1
Step 2Step 3
TITAN CAMERA API 2 SITUATION

17. TITAN CAMERA API 2 SITUATION
CaptureRequest
frame meta
CaptureRequest
frame meta
camera2 HAL 3
CaptureRequest
frame meta
CaptureRequest
frame meta
CaptureRequest
frame meta
CaptureRequest
frame meta

18. TITAN CAMERA API 2 SITUATION
CaptureRequest
frame meta
CaptureRequest
frame meta
camera2 HAL 1
CaptureRequest
frame meta
CaptureRequest
frame meta
SHIM
CaptureRequest
frame meta
CaptureRequest
frame meta

19. TITAN CAMERA API 2 SITUATION
Arcsoft involved to use camera2 only for its camera app.
QCT never supportYUV sensor with HAL 3, and Titan’s camera
solution was forced to use HAL 1.
Vertu decide to use HAL 1 + shim + camera 2 in Apr 2015
Vertu would like to have a Legacy+ device.

20. TITAN CAMERA API 2 SITUATION
Arcsoft involved to use camera2 only for its camera app.
QCT never supportYUV sensor with HAL 3, and Titan’s camera
solution was forced to use HAL 1.
Vertu decide to use HAL 1 + shim + camera 2 in Apr 2015
Vertu would like to have aVertu specific Legacy+ mode for Titan

21. TITAN CAMERA API 2 SITUATION
LEGACY+ mode
slowmo/HFR support
Cont. shot/ZSL/BurstCap support (abandoned byVertu at wk 41)
PICAUTO support
AS HDR support
YUV420 to NV12/21, notYV12 format

22. TITAN CAMERA API 2 SITUATION
mediaserver
android.hardware.Camera
android.hardware.ca
mera2.*
android.hardware.ca
mera2.legacy.*
CameraService(Native)
App
Camera2 Legacy Mode
SHIM Layer

23. TITAN CAMERA API 2 SITUATION
CameraDeviceUserShimCameraManager
LegacyCameraDevice
RequestThread
Manager
GLThread
Manager
Capture
Collector

24. TITAN CAMERA API 2 SITUATION
RequestThread
Manager
GLThread
Manager
LegacyMetadata
Mapper
SurfaceTexture
Render
LegacyRequest
Mapper
LegacyResult
Mapper
LegacyXXX
Mapper
LegacyCameraDevice

25. SOLUTIONS & HACKS
LegacyMetadataMapper
be used for create camera static characteristic from HAL 1
Legacy+ mode
convert HFR support from HAL 1 to HIGH_SPEED_VIDEO
property and export it to camera characteristic
export AS HDR, PICAUTO to camera characteristic
export scene mode FOOD/BACKLIT to camera characteristic

26. SOLUTIONS & HACKS
LegacyRequestMapper
be used for mapping camera2 request to parameters for HAL 1
Legacy+ mode
adjust fps and HFR mode from HIGH_SPEED_VIDEO setting
set AS HDR and PICAUTO flag
wrapper and convertVERTU specific scene mode to pass CTS
Cont. shot/ZSL/Burst shot request process

27. SOLUTIONS & HACKS
LegacyResultMapper
be used for mapping camera2 request to parameters for HAL 1
Legacy+ mode
based on HIGH_SPEED_VIDEO setting, convert correct camera2 result
back.
based on AS HDR and PICAUTO flag, convert correct camera2 result
back.
wrapper and convertVERTU specific scene mode to pass CTS

28. SOLUTIONS & HACKS
More and more?
camera2 settings are stored in native space and extensions are
supported by vendor tags. (not available for Legacy mode)
We must revise the accessor to native space for extra supported
features with Java space support
camera native framework need to be extended (LegacyXXXMapper)
HAL 1 must be greatly modified for supports of all extra features.

29. SOLUTIONS & HACKS
More and more?
Performance for shim is really not good
parts of shim are not completed, cause most of all stability
problem
not only HAL 1, but underly QCOM camera system also need
to be customised repetitively

30. CONCLUSIONS
Shim is just a start, what is the next?
Performance tuning, shim is really not good in performance
Complete the shim, parts of shim are not done by Google, which cause
most of all stability problem
Go to lower layer. Not only HAL 1, but underly QCOM camera system
also need to be customised repetitively
PICAUTO Integration
AS HDR Integration