Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Image Fusion - How to Best Utilize Dual Cameras

17,256 views

Published on

The future of mobile dual cameras is brighter than ever. Adoption rates surpassed analysts' predictions, and all major OEMs plan to launch or have already launched handsets with zoom dual cameras. Now the race begins for higher zoom factors, better optical stabilization, lower Z-heights and better low-light performance – anything that improves the overall camera and handset user experience. In this paper, we discussed some ways to increase the zoom factor and reduce the Z-height while maintaining superior image quality.
The road to improved camera hardware does not stop here. From reducing the overall footprint of the camera module, finding new ways to optically stabilize the cameras, giving the camera autonomous sensing capabilities and other innovative hardware improvements, Corephotonics will continue to provide state-of-the-art, never-before-seen camera module designs to help drive mobile imaging innovation forward.

Published in: Technology
  • Be the first to comment

Image Fusion - How to Best Utilize Dual Cameras

  1. 1. Image Fusion How to Best Utilize Dual Cameras Corephotonics Proprietary Information
  2. 2. • UpdatedAdoptionRates: 30%in2018and50%in2019 • AllTopOEMsflagshipscontain DualCameras • NewlyreleasediPhone8&iPhoneX containDualCameras  DigiTimesResearch(July2017) Source:CLSA,April2016 1584 1641 1516 1463 0 77 318 488 0% 4% 17% 25% 0% 5% 10% 15% 20% 25% 30% 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2015 2016CL 2017CL 2018CL <Dual Cam Market Growth, CLSA, April , 2016 (Mil units)> Smartphones without dual cam Smartphones with dual cam Dual cam adoption (%) Dual Camera Adoption  More Then Just a Trend Dual-Camera Smartphone Market Growth 2Corephotonics Proprietary Information
  3. 3. Image Fusion - Overview 3 • Whatisit? o ImageFusionistheprocessof combiningtwoormoreinput imagesintoasingleimage • Whydoweneedit? o Themainreasonforcombining theimagesistogetamore informativeoutputimage Corephotonics Proprietary Information
  4. 4. Image Fusion – Example 1 4 • Mono+ColorDualCamera o Themonochromaticsensorcaptures 2.5Xmorelight ReachesbetterResolution&SNR o Combinedoutputimagehasbetter Resolution&SNR,especiallyin lowlightconditions Corephotonics Proprietary Information
  5. 5. Image Fusion – Example 2 5 • Wide+TeleDualCamera o SimulatesacontinuousOpticalZoom effect,asfoundinDSLRs o Combinedoutputimagewillimprove theResolution&SNRfromnoZoomup tothepointtheTelephotoCamera’s Field-of-Viewisthedominantone Corephotonics Proprietary Information
  6. 6. Corephotonics Confidential | 6 Full FOV  Corephotonics Dual Camera Module Corephotonics Proprietary Information
  7. 7. Corephotonics Confidential | 7 2X Digital Zoom Corephotonics Proprietary Information
  8. 8. Corephotonics Confidential | 8 2X Fusion Corephotonics Proprietary Information
  9. 9. Image Fusion – Algorithm Flow 9 • GenericschemeforFusion(Othermethodsexist) • SomedifferencesbetweenWide + Tele & Color + Mono Corephotonics Proprietary Information
  10. 10. Rectification 10 • Rectificationcorrectsfordistortion,scaleandshiftthatmaybeintroducedby theOpticsandAFMechanism • Thealgorithmusespre-computedrectificationdatathatisstoredinthe sensors'OTPMemory (FactoryCalibration) • Output:Afterthisstep,correspondingpoints inthetwoimageslieonEPI-Polarlines Corephotonics Proprietary Information
  11. 11. Global Registration 11 • The algorithmcalculates and compensates forglobal differencesbetweenthe 2 cameras which could be attributedto dynamic changes between images and specific properties of eachcamera Corephotonics Proprietary Information
  12. 12. Local Registration & Parallax Correction 12 • Fine Registration in which the parallax (i.e.shiftinXandYdimensions,whichisbasedonobjectdistance) is determined for each Pixel in the input images Corephotonics Proprietary Information
  13. 13. Decision & Fusion 13 • Decision: Handlesocclusions,correctsforRegistrationerrorsandhasan adaptivealgorithmthateliminatesartifactsfromthefinalimage • Fusion: BlockthatimplementstheactualdataFusioninawaythatthe2 originalimagesareseamlesslyblendedtogether X X Corephotonics Proprietary Information
  14. 14. Image Fusion Challenges: Occlusions 14 • Howtobesthandleareaseach cameraseesdifferently? 1. Inordertofusetwoimagesthe algorithmneedstousematchingpixels betweentheleftandrightimages 2. Incaseanareaisoccludedfromone cameraamatchingpointcannotbe detectedforthatarea 3. Handledbythedecisionblock Corephotonics Proprietary Information
  15. 15. Image Fusion Challenges: Lens Imperfections 15 • ThedifferencesinDepthofField areasthatcanbein-Focusinone cameraandout-of-Focusinthe 2nd camera • Registrationbecomesmuch trickierbecauseofthedifficultyto findfeaturepointsandmatching Pixels • TheDecisionBlockneedstoselect thecamerathatisin-Focus Corephotonics Proprietary Information
  16. 16. 16 Wide Fused X1.5 Image Fusion Challenges: Lens Imperfections Corephotonics Proprietary Information
  17. 17. Image Fusion Challenges: Overlap Transition 17 ThedifferenceintheField-of-ViewandtheResolutioncreates aninevitableinclusionofoneimageinsidetheother FusionwillnotcovertheentireField-of-Viewandthe algorithmwillhavetoadjusttheFusion“Strength”graduallyto allowanaturaluserexperienceinthetransitionareabetween theimages. Corephotonics Proprietary Information
  18. 18. Resolution & SNR Benefits Corephotonics Proprietary Information
  19. 19. Apple iPhone 7+ Fusion FusionZoom Factor 1.8 with 570 Lux • Tele & Wide images taken on iPhone • Images processed by Corephotonics Image Fusion (Right) compared to iPhone Fusion (Left) Corephotonics Proprietary Information
  20. 20. FusionHuawei P10 Fusion Corephotonics Proprietary Information
  21. 21. Mature & Artifact Free User Experience Corephotonics Proprietary Information
  22. 22. Show “thin lines” artifacts Corephotonics Confidential |22 iPhone7+ creates Fusion Artifacts in such complex depth scenes Apple iPhone 7+ Fusion FusionZoom Factor 1.8 Corephotonics Proprietary Information
  23. 23. What’s Next? 23 • OEMs push for higher Zoomfactor Increasedneed for Image Fusion • Verticalmarketapplications(AR / VR / MR) o LowLightEnhancement o Real-TimeFusion • Largebaseline to accommodatedepth apps Corephotonics Proprietary Information
  24. 24. Reference 24 To learnmore about Image Fusion CorephotonicsImage Fusion WhitePaper Corephotonics Proprietary Information
  25. 25. Thank You! 25 RoyFridman | DirectorofProductMarketing | rfridman@corephotonics.com Corephotonics Proprietary Information

×