Ei2004 presentation


Published on

Electronic Imaging International SPIE Conference, San Jose, 2004 on my doctoral work achievements.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Efficient data reduction=transferring just important (central) image information. Enhance Image Information Transfer=creating enough room for transferring important image information (details)
  • Viewpoint: Sampling architecture is not necessarily the same as the display architecture. Raster scanning was originally suggested for CRT displays but was adopted since 60’s for MOS imagers. Biological sampling architectures are very promising to implement efficient smart sampling architectures. Human fovea is a clear example.
  • 1- Integration time is the time elapsed between successive ring reset and ring signal readout. 2- Ring integration time in the bouncing scanning scheme is deduced including the bouncing at the inner and outer rings. 3- Due to the different of cardinals of the inner and outer rings, bouncing scanning at the inner ring will result lower integration time for outward scanning than bouncing at the outer ring for the inward scanning.
  • Ei2004 presentation

    1. 1. Foveated Architectures for CMOS Image Sensors Fayçal Saffih1, Richard Hornsey21 Integrated Camera Group, Electrical & Computer EngineeringDepartment, University of Waterloo, ON, Canada2 Centre for Vision Research, Department of Computer Science,York University, Toronto, Ontario, Canada Electronic Imaging 2004, San Jose
    2. 2. OutlineIntroduction & MotivationMotivation ImplementationPyramidal CMOS Image SensorApplicationsUniversal Multiresolution CMOS Image SensorApplicationsFinal Conclusion. Electronic Imaging 2004, San Jose
    3. 3. Introduction & MotivationCMOS image sensor development levels: Device level: Optical charge devices such as photodiodes.  Goal: Dynamic range enhancement, Fill factor increase…etc. Circuit Level: Amplifiers, buffers, ADC’s…etc.  Goal: Efficient charge transfer, high speed acquisition, noise filtering…etc System Level: Signal processing blocks such as motion detection, image processing blocks, …etc.  Goal: Image information extraction (motion, segmentation…etc), performance enhancement (data reduction, multiresolution, foveated vision…etc).Motivation Imitation of biological visual systems such as Human Visual System.  Efficient Data Reduction.  Efficient Image Information Transfer.  Minimization of Power Consumption.  Emphasize on the importance of architecture design in CMOS Imagers.Philosophy: Image acquisition is a Vision needs vision Electronic Imaging 2004, San Jose
    4. 4. Motivation ImplementationSystem level implementation The pixel structure is the 3T Active Pixel Sensor (APS). The acquisition system is a non-classical (non- orthogonal) architecture.  Solution: Pyramidal CMOS image sensorDevice level implementation The pixel structure is a non-classical (3T active pixel sensor). The acquisition system is a classical orthogonal architecture.  Solution: Universal Multiresolution CMOS Image Sensor Electronic Imaging 2004, San Jose
    5. 5. System Level approach PyramidalCMOS Image Sensor Electronic Imaging 2004, San Jose
    6. 6. Classical Acquisition SystemThe orthogonal acquisitionarchitecture along with rasterscanning suffers from thefollowing issues: 1D sampling architecture. Unique integration time Different sampling speed between horizontal and vertical axis of the image. This leads to anisotropic distribution of motion blur, being higher in the vertical axis than in the horizontal. Electronic Imaging 2004, San Jose
    7. 7. Human Fovea ArchitectureDynamic Range is higher in cones than Circular symmetry of the fovea photocellsrods. (rods and cones) has lead to symmetrical yetHVS system DR (~106) is much larger co-centric image sampling.than the photoreceptor DR (~102). Electronic Imaging 2004, San Jose
    8. 8. Pyramidal AcquisitionArchitecture Electronic Imaging 2004, San Jose
    9. 9. Pyramidal Acquisition ArchitectureFloorplan of the imager iscomposed of squarepixels (16µmx16µm)orthogonally compacted.Reset and select signalsare shared among eachringThe output buses arediagonals to dump the ringoutput into 8 CDS block atthe base of the pyramid Electronic Imaging 2004, San Jose
    10. 10. Bouncing Scanning Scheme Estimated Integration time of a ring r for inward Scanning:  r +1 Trin = 2  ∑ iT s + ( R − r )Tspl  + rT s i = R   i → i −1    Estimated Integration time of a ring r for outward Scanning:  r −1 Trout = 2  ∑ iTs + ( r − 1)Tspl  + rTs  i =1   i → i +1    Electronic Imaging 2004, San Jose
    11. 11. Ring Integration TimeEstimated of all rings’ integration time for inward (ISc) and Rings’ RMS of inward (ISc) and bounced scanningbounced scanning (BSc) at 40Fps. (BSc) at 8Fps under light intensity of 43.33uW (current testing) Electronic Imaging 2004, San Jose
    12. 12. Dynamic Range Fovea Each ring in the pyramidal imager The resulting dynamic range will have two different integration enhancement is made with two time. Fusing their two output will acquired scenes. Thus, this type enhance the ring’s dynamic range of dynamic range enhancement is by:20log(Tint1/Tint2), where Tint1 ≥Tint2 called, intrascene dynamic range. Electronic Imaging 2004, San Jose
    13. 13. Experimental ResultsInward and Bounced Images fromthe Pyramid Imager at 14lux@29Fps Fused Image of the inward Pyramidal CMOS Image Sensor and bounced images Layout Electronic Imaging 2004, San Jose
    14. 14. ApplicationsApplications that need optimal data transfer such as: Video-Phone Internet camerasApplications that need foveated vision: IndustrialInspection Surveillance Cameras Low Vision Enhancement Consumers Cameras Electronic Imaging 2004, San Jose
    15. 15. DisadvantagesComplex data structure for image re-constructionMismatch between sampling capacitors between thedifferent pyramid clusters may create some undesiredartifacts.The central diagonal in the pyramid is sampled bycapacitors with twice the capacity of their neighbor pixels. Although the above are real problems in our CMOS imager, they are not limitations as they are all solvable problems. Electronic Imaging 2004, San Jose
    16. 16. ConclusionNew architecture (hardware) and scanning scheme(software) for CMOS imagers have been suggested.The benefits of the new approach in image sampling aredue to: The 2D nature of the sampling rings Centricity of the sampling units has lead to foveated dynamic range enhancement topology.The difference in sampling architectures in CMOSimagers greatly impacts their performances. Electronic Imaging 2004, San Jose
    17. 17. Device Level approach Universal MultiresolutionCMOS Image Sensor Electronic Imaging 2004, San Jose
    18. 18. Multiresolution CMOS Image SensorMotivation Only small parts of image are regions of interest. Regions of low interest need to be sub-sampled or averaged. With increasing image resolution the ultimate limitation is image transfer especially for video broadcasting. Minimizing of power consumption requires a minimization of power consumption of the imager OR minimization of “data of interest” to be readout Programmability and expandability are the most important feature of such an architecture in order for it to span the largest field of applications: Universality Electronic Imaging 2004, San Jose
    19. 19. The Kernelling Just two clock cycles are enough for the kernel averaging: Fast Kernelling Electronic Imaging 2004, San Jose
    20. 20. Building Block: The Pixel Electronic Imaging 2004, San Jose
    21. 21. Multiresolution Decoder for Row reset and select CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block The Architecture CDS Block CDS Block Multiresolution Decoder for CDS Block CDS Block Decoder for Column Select of CDS Block CDS Block Correlated Double Sampling (CDS) CDS Block CDS Block Row-Average Support & Column-Average CDS Block Multiresolution Decoder for Row-Average and SamplingElectronic Imaging 2004, San Jose Image Output
    22. 22. Application: ProgrammabilityFundamental Foveated Multiresolution Random Foveated Multiresolution Multi-Foveated Image Sampling Electronic Imaging 2004, San Jose
    23. 23. Application: ProgrammabilityFundamental Foveated Multiresolution Fundamental Foveated Multiresolution With horizontally-rectangular kernels With vertically-rectangular kernels Effect of kernel topology on spatial filtering Electronic Imaging 2004, San Jose
    24. 24. ApplicationsMultiple Object TrackingIndustrial inspectionImage processing prototypingRemote image acquisition requiring minimalimage transfer bandwidth. Electronic Imaging 2004, San Jose
    25. 25. Final ConclusionTwo different approaches for implementingfoveated imaging have been suggested.The two imagers were fabricated in dual voltage1P6M standard 0.18µm CMOS technology. Thetwo imagers under test.Parallelism, programmability and expandabilitywere the keys behind the proposed CMOSimage sensors’ architectures Electronic Imaging 2004, San Jose
    26. 26. AcknowledgementThe authors are grateful to:  Canadian Microelectronics Corporation.  NSERC Canada.  Betacom Inc. Thank you ! Electronic Imaging 2004, San Jose