CMOS IMAGE SENSORSFrom the Biological Eye to Smart Machine Vision         .Fayçal Saffih, Ph.D. , Visiting scholarCommunic...
Outline   CMOS Imaging Technology   New Advancements in Smart Cameras   Applications in the Medical Domain   Contribut...
CMOS Imaging Technology             InStat/MDR   Two main electronic imaging technologies: Complementary Metal-    Oxide-...
Why CMOS imaging is important ?   Architectures flexibility allows a lot of flexibility of image acquisition in    CMOS i...
Research Contribution and      Future Work   Foveated Sampling Architectures      for CMOS Image Sensors Ph. D. Thesis ava...
Philosophical Statements   Image Scanning Statement       Image Scanning should be more adapted to image sampling/acquis...
Why Foveated CMOS Imagers?   Foveated image sampling architectures are needed due to:      1. Limited imaging system reso...
Why Foveated CMOS Imagers?Ramon Cajal identified the retina basic anatomicalstructure. Shown here is his sketch of the int...
Our Research Contribution Outline   Standard CMOS Image Sensor   System Level Approach (Time Domain Fovea)      Pyramid...
The Classical Architecture    & Raster Scanning                              Standard CMOS Image Sensor
Active Pixel Sensor   Standard CMOS Image Sensor
The Seminar Outline   Standard CMOS Image Sensor   System Level Approach (Time Domain Fovea)      Pyramidal CMOS Image ...
System Level Approach   Time Domain Approach:       Because Dynamic Range enhancement can be        realised by multi-ex...
Pyramidal CMOS Image Sensor    Architecture …Classical CMOS Imager Architecture   Pyramidal CMOS Imager Architecture     ...
Pyramidal CMOS Image Sensor                    Physical Design …Physical Design
Pyramidal CMOS Image Sensor                              Scanning …Rolling and Bouncing Scan
Foveated Dynamic Range Enhancement                    Mathematical Foundations: Prediction…Definitions:Integration time o...
Foveated Dynamic Range Enhancement Mathematical Foundations: Prediction…                                                 ...
Foveated Dynamic Range Enhancement Mathematical Foundations: Prediction…                            max (Tin( r , R, Ts ...
Foveated Dynamic Range Enhancement Mathematical Foundations: Controllability…                              Pinned Foveate...
Foveated Dynamic Range Enhancement             Mathematical Foundations: Advantage …Dynamic Range is usuallyassociated to...
Foveated Dynamic Range Enhancement Experimental Setup…                        CMOS Pyramidal Imager under test inside bla...
Foveated Dynamic Range Enhancement              Experimental Results…The model extraction will provide us:1.   Lmin: Mini...
Foveated Dynamic Range Enhancement               Experimental Results…Vcds in RMS voltage the whole pyramidal imager for ...
Foveated Dynamic Range Enhancement        Experimental Results…Measured and modeled values for Vcds   Correlation of the ...
Foveated Dynamic Range Enhancement                       Experimental Results…Definitions:Most researchers in CMOS imager...
Pyramidal CMOS Image Sensor                Experimental Results: Ring Sampling & Blur SymmetrizationMotion Blur effect on...
Pyramidal CMOS Image Sensor            Experimental Results: Ring Sampling & Blur SymmetrizationThe PixeLINK CMOS imager ...
Foveated Dynamic Range Enhancement          Experimental Results…(                 +                         )x½=        ...
Pyramidal CMOS Image Sensor               High-Speed Imaging …               FRPyr        R + 1.5                        ...
Low Pyramidal Imager FPN Perception by        Human Visual System Oblique effect found in the contrast sensitivity of the ...
Low Pyramidal Imager FPN Perception by                                         Human Visual SystemClassical CMOS Imager F...
Low Pyramidal Imager FPN Perception by                                         Human Visual SystemPyramidal CMOS Imager F...
HVS Spatial Filter Application on  Pyramidal FPN images: Model                Human Visual System                        ...
HVS Spatial Filter Application on Pyramidal FPN images: Results               Human Visual System                        ...
HVS Spatial Filter Application on Pyramidal FPN images: Results               Human Visual System                        ...
HVS Spatial Filter Application on Pyramidal FPN images: Results               Human Visual System                        ...
HVS Spatial Filter Application on Pyramidal FPN images: Results               Human Visual System                        ...
Low Pyramidal Imager FPN Perception by                                                                         Human Visua...
The Seminar Outline   Standard CMOS Image Sensor   System Level Approach (Time Domain)      Pyramidal CMOS Image Sensor...
Pixel Level Approach   Spatial Domain Approach:        Only regions of interest are sampled at the highest possible reso...
Multiresolution CMOS Image Sensor   Image Sampled at highest resolution in all pixels   Only Regions-Of-Interest (ROI) a...
The Seminar Outline   Standard CMOS Image Sensor   System Level Approach (Time Domain)      Pyramidal CMOS Image Sensor...
Conclusion and Future WorkConclusion:The architecture flexibility of CMOS imaging technology was:       Key factor:      ...
Thank you
Upcoming SlideShare
Loading in …5
×

From Biological to Smart CMOS Imaging: Architectural approach

954 views
838 views

Published on

From Biological to Smart CMOS Imaging: Architectural approach

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
954
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Prepare from [4] some extra info
  • 1. Shown is only the discharge site of the generated e-h pairs at the depletion region 2. The e-h pairs can be generated at: depletion region, peripheral regions, the substrate 3. The e-h pairs are drifted towards the discharging site by the electrical field or by diffusion
  • Although integration time could be controlled at the pixel level, we choose the system level (control) to minimize the drawbacks of pixel integration such as: 1. low fill factor, 2. extra noise sources, 3-decreased resolution
  • Why we suggested this pyramidal based on :1-our 1 st philosophical statement : Image info first 2-circular symmetry of biological vision systems (output diagonal busses are consequence of ring sampling) 3- the low dimension (1D) of the raster scan
  • This is the first prototype of the pyramidal CMOS imager  not optimized in: a-area, b-power consumption. Layout symmetry was a very interesting/useful factor. Pixel size = 16umx16um=256um 2 Fill Factor =78% Chip size =4mmx4mm
  • Raster scan (also called rolling scan) is also possible in pyramidal imager Shown is : the readout path between the pyramidal and the classical CMOS imager. Bouncing scan is possible in classical imager but with being its purpose (the circular symmetry and foveated DR enhancement)
  • The bouncing edges are the inner ring and the outer ring. Integration time of a pixel (or of RST-SEL shared sampling entities)= time between consecutive RST & SEL signals.
  • Photo-Signal= Integration time x Light intensity x Sensitivity Linearity: As long as the photo-response is linear (no saturation) the fused image is: 1- also linear 2- independent on ring order r (in contrast to inward and outward scan)
  • The minimum of the FDR enhancement IS where Tin=Tout. This point is called the fovea border FOVborder fast reaching its limit at α = 1 and above Fovea Border :1- Is situated at (R/sqrt(2)) or (71%)R, 2-can be programmed to pin at the outer/peripheral ring for a perfect foveated DR enhancement shape, 3- will be used later on for determining the system electrical DR and the FDR enhancement.
  • Controllability concerns the profile of the FDR enhancement by pinning its border to the outer ring and other alternatives such as reverse-FDR
  • DR measures an interval with a ratio approach that can also be described by a ratio of maximal binary number to the smallest (of course 1). Dual sampling technique has inspired this FDR approach however the whole imager will benefit from the extra binary bits (DR enhancement) which also cost the resultant image to be in a large size. Hence, due to philosophy statement 2: only region of interest is given the appropriate attention and resources (memory).
  • The real time acquisition of the system was realized by setting Labview process to real time within windows2000 environment. Tool used is called TaskInfo.
  • This is very simplistic model but serving the targeted objectives.
  • This is for rolling scan (Non-bouncing scan). Tin=124.8 ms, 62.4ms, 49.92ms, 24.96ms, 12.48ms, 6.24ms, 2.496ms, and 1.248ms for 10KHz, 20KHz, 25KHz, 50KHz, 100KHz, 200KHz, 500KHz and 1MHz respectively or 8Fps, 16Fps, 20Fps, 40.1Fps, 80.1Fps. 160.Fps, 400.6Fps, 801.3Fps (NB-FRAMErate = SPLfreq/1248;BS-FRAMErate = SPLfreq / 2496) Model speaks through the correlation values: 1- When noise discarded the model failed miserably.
  • CalculDR : is the system DR which is based on reachable Lmax and detectable Lmin (shared for all Tint –see slide 21-22) OptDRMx_MnDet : 1- to show the linearity of the system Max Detectable Light Vs Tint 2-That this approach is Dynamic range enhancement towards brighter light intensities. Theory_DRenh: The formula used to expect the dynamic range enhancement (from dual sampling DR enhancement)
  • Raster scanning is just an extreme case of the Rolling shutter.
  • Tint is the same for both, therefore its should be faire parameter: Shows the impact of the sampling architecture Imagine 2 pointers reading the 2 images simultaneously. Think of it as film image integration  The reason of the blur is therefore a result of the architecture sampling.
  • Use PowerPoint to show DR by changing the contrast or brightness The first set acquired at: 270 lux, 8 fps (10KHz) The second set sampled at: 402 lux, 400 fps (1MHz)
  • Because R 2 is the dominant player even without assuming Tspl=3Ts the results is likely not affected.
  • Because R 2 is the dominant player even without assuming Tspl=3Ts the results is likely not affected.
  • The present analysis shows that pyramidal CMOS image sensor is always faster than classical CMOS imager of equivalent size and scanning timing parameters. However, this high speed feature is more prominent in the case of acquisition segmentation into parallel readout channels, as originally designed, than in the case of serial readout. To conclude this section, its is worth mentioning that parallel segment readout scheme of the pyramidal imager is the readout of choice not only because of the fast frame rates its achieved, furthermore, because it is the most natural scheme to this architecture due to the fact that every segment has its own independent sample and hold banks of capacitors.
  • From top-right picture: Oblique bandwidth is about ½ of the cardinal bandwidth. It shows the Coincidence or Adaptation of the HVS spatial resolution power bandwidth to that its surrounding nature.
  • Fixed pattern noise (FPN) is the spatial noise distribution with no illumination of the image sensor array that is explicitly time independent, and hence “fixed” Causes: 1- photo-signal generation mismatch (variation of the photo-sensing areas -mask & process variations-, dark current: impurities, thermal generation, lattice defects) 2- photo-signal transportation mismatch (S&H mismatch, column amplifiers mismatch) This correlated double sampling suppresses reset noise, 1/ f noise, and Fixed Pattern Noise (FPN) due to threshold voltage variations.
  • Filter D and E for all 8 direction to mimic the circular shape of the filter and ADD F (half the bandwidth) to the cardinal axes to implement the oblique effect Filters D, E and F were chosen because: there expected to introduce the most significant response based on their Peak Frequency.
  • Using empirical data and well accepted spatial filter suggested by prof. Hugh at York University a modeled spatial filter was built. The oblique effect is introduced based on the experimental facts shown in slide 32 (bandwidth fact)
  • The Fourier spectrum modules were shown with modules sliced between 0 3(or4) to see Matlab was used. Inverse FFT show no discernable differences between B and C  used their difference.
  • Color-map used full range of the difference : information extraction.
  • Observe ellipses
  • FPN increases with decreasing SNR (increasing FR = decreasing integration time)
  • Information content in the spatial frequencies can be preserved if kernel averaging is done adequately. Could save large amounts of image data that is not needed and allocate those resources to high spatial frequency content areas.
  • From Biological to Smart CMOS Imaging: Architectural approach

    1. 1. CMOS IMAGE SENSORSFrom the Biological Eye to Smart Machine Vision .Fayçal Saffih, Ph.D. , Visiting scholarCommunication Research Laboratory (CRL)McMaster University
    2. 2. Outline CMOS Imaging Technology New Advancements in Smart Cameras Applications in the Medical Domain Contributions and Future Research
    3. 3. CMOS Imaging Technology InStat/MDR Two main electronic imaging technologies: Complementary Metal- Oxide-Semiconductor (CMOS) and Charge-Coupled-Devices (CCD) Total unit shipments of CMOS image sensors surpassed CCDs for the first time in 2005. (In-STAT/MDR) CMOS image sensor shipments will grow at roughly seven times the rate of CCDs through 2008 (In-Stat/MDR)
    4. 4. Why CMOS imaging is important ? Architectures flexibility allows a lot of flexibility of image acquisition in CMOS imagers compared to their CCD counter parts. CMOS Imagers are well known for their low power (20-50 mW) compared to CCD’s (2-5W) for the same pixel throughput. This is why CMOS imagers are dominating portable devices market and wireless medical imaging . Integration of acquisition and data processing is only possible in CMOS imaging technology. Compatibility with main stream CMOS technology. As such, every CMOS foundry has CMOS image sensor division.
    5. 5. Research Contribution and Future Work Foveated Sampling Architectures for CMOS Image Sensors Ph. D. Thesis available at: http://www.cs.yorku.ca/~visor/pdf/FaycalPhDThesis.pdf
    6. 6. Philosophical Statements Image Scanning Statement  Image Scanning should be more adapted to image sampling/acquisition rather than image display compatibility.  Image Information Acquisition is the priority of the imager. Image Sampling Statement  As image resolutions get higher and with it the amount of transmitted image data for display or processing, new architectures are needed for down-scaling the sampling resolution for regions of reduced interest. Innovative architectures are also needed to exploit the Human Visual System for transmitting only the most important regions in the acquired image.  Only Regions of Interest are sampled at highest imaging requirement such as dynamic range, spatial resolution ... etc.
    7. 7. Why Foveated CMOS Imagers? Foveated image sampling architectures are needed due to: 1. Limited imaging system resources (memory, processing power & time) 2. Not all image areas are of the same degree of importance (or of interest) 3. Only area of interest are sampled at the highest resolutions, highest dynamic range…etc. 4. Biologically inspired “Nature does nothing uselessly.” uselessly –Aristotle, Politics “The diversity of the phenomena of nature is so great, and the treasures hidden in the heavens so rich, precisely in order that the human mind shall never be lacking in fresh nourishment” nourishment –Kepler, from Carl Sagan’s book “Cosmos”
    8. 8. Why Foveated CMOS Imagers?Ramon Cajal identified the retina basic anatomicalstructure. Shown here is his sketch of the interconnectivityconfigurations of photocells rodes (f) and cones (e) [4]
    9. 9. Our Research Contribution Outline Standard CMOS Image Sensor System Level Approach (Time Domain Fovea)  Pyramidal CMOS Image Sensor  Architecture  Physical Design  Scanning  Foveated Dynamic Range Enhancement  High-Speed Imaging of Pyramidal CMOS Imager  Low Pyramidal Imager FPN Perception by HVS Pixel Level Approach (Spatial Domain Fovea)  Multiresolution CMOS Image Sensor  Architecture  Multiresolution Active Pixel Sensor  Results Conclusion and Future Work
    10. 10. The Classical Architecture & Raster Scanning Standard CMOS Image Sensor
    11. 11. Active Pixel Sensor Standard CMOS Image Sensor
    12. 12. The Seminar Outline Standard CMOS Image Sensor System Level Approach (Time Domain Fovea)  Pyramidal CMOS Image Sensor  Architecture  Physical Design  Scanning  Foveated Dynamic Range Enhancement  High-Speed Imaging of Pyramidal CMOS Imager  Low Pyramidal Imager FPN Perception by HVS Pixel Level Approach (Spatial Domain Fovea)  Multiresolution CMOS Image Sensor  Architecture  Multiresolution Active Pixel Sensor  Results Conclusion and Future Work
    13. 13. System Level Approach Time Domain Approach:  Because Dynamic Range enhancement can be realised by multi-exposure concept, controlling the topology of imager’s integration time profile may lead to a non-uniform and interesting Dynamic Range enhancement Design Approach:  Keep the standard APS structure while changing the Architectural Sampling structure Pyramidal CMOS Image Sensor
    14. 14. Pyramidal CMOS Image Sensor  Architecture …Classical CMOS Imager Architecture Pyramidal CMOS Imager Architecture 1D-Row Sampling 2D-Ring Sampling Vertical Busses Diagonal Busses
    15. 15. Pyramidal CMOS Image Sensor  Physical Design …Physical Design
    16. 16. Pyramidal CMOS Image Sensor  Scanning …Rolling and Bouncing Scan
    17. 17. Foveated Dynamic Range Enhancement  Mathematical Foundations: Prediction…Definitions:Integration time of a pixel (or of RST-SEL shared sampling entities): Timebetween consecutive RST & SELsignals.Ts: Pixel scanning time (0.1µs ~ 1µs)Tspl: Row sampling time rowsampling including Vs sampling, reset,Vr sampling (1µs ~ 10µs)r: ring orderR: Total number of ringsTin(r): Inward scan integration time.Tout(r): Outward scan integrationtime.  r +1   r −1  Tin( r , R, Ts , Tspl ) = 2  ∑ iTs + ( R − r )Tspl  + rTs , Tout ( r , Ts , Tspl ) = 2 ∑ s  iT + ( r − 1)T  + rT i =1  i = R  spl s i→i −1    i →i +1    …and after simplification we get: ( Tin ( r , R, Ts , Tspl ) = −Ts r 2 − 2Tspl r + R 2Ts + RTs + 2 RTspl ) Tout ( r , Ts , Tspl ) = Ts r 2 + 2Tspl r − 2Tspl
    18. 18. Foveated Dynamic Range Enhancement Mathematical Foundations: Prediction… ( Tin ( r , R, Ts , Tspl ) = −Ts r 2 − 2Tspl r + R 2Ts + RTs + 2 RTspl ) Tout ( r , Ts , Tspl ) = Ts r 2 + 2Tspl r − 2Tspl ToT ( R, Ts , Tspl ) = Tin( r , R, Ts , Tspl ) + Tout ( r , Ts , Tspl ) = R 2 Ts + R (Ts + 2Tspl ) − 2 Tspl
    19. 19. Foveated Dynamic Range Enhancement Mathematical Foundations: Prediction…  max (Tin( r , R, Ts , Tspl ), Tout ( r , Ts , Tspl ) )  DRenh (r ) = 20 log    min (Tin ( r , R, Ts , Tspl ), Tout ( r , Ts , Tspl ) )    1 1 2  FOVborder (α , R ) =  1 + α R ( R + 1) + α ( R + 1) − 1 , where α = Ts T spl α 2 
    20. 20. Foveated Dynamic Range Enhancement Mathematical Foundations: Controllability… Pinned Foveated Dynamic Range Enhancement
    21. 21. Foveated Dynamic Range Enhancement  Mathematical Foundations: Advantage …Dynamic Range is usuallyassociated to the number of“meaningful” bits of thedigital output of an imager.  DRdB  DRbits = log 2 10 20      3D view of the pinned FDR enhancement expressed in binary bits.
    22. 22. Foveated Dynamic Range Enhancement Experimental Setup… CMOS Pyramidal Imager under test inside black box
    23. 23. Foveated Dynamic Range Enhancement  Experimental Results…The model extraction will provide us:1. Lmin: Minimum detectable light intensity2. Lmax: Maximum saturation light intensity3. Ss: Sensitivity of the imager The Objective of this methodology is the extraction of: 1. The System electrical dynamic range 2. The Optical dynamic range enhancement 3. Verification of the mathematical predictions of dynamic range enhancement
    24. 24. Foveated Dynamic Range Enhancement  Experimental Results…Vcds in RMS voltage the whole pyramidal imager for 8 Slopes of linear regions of photon transfer curveintegration times
    25. 25. Foveated Dynamic Range Enhancement  Experimental Results…Measured and modeled values for Vcds Correlation of the model to the measured values
    26. 26. Foveated Dynamic Range Enhancement  Experimental Results…Definitions:Most researchers in CMOS imagersagree that Lmin is constant withintegration time.Lmin ≈ 5LuxLmax is calculated based on the model.It is at which light intensity the pixelwill reach saturation System and Enhancement Dynamic Ranges Extraction of the Pyramidal CMOS imager at 1MHz (800 Fps)CalculDR = 20 Log ( Lmax Lmin )  max( MaxDetLI inward , MaxDetLI outward ) OptDRMx _ MnDet (r ) = 20 log    min ( MaxDetLI inward , MaxDetLI outward )   max (Tin( r , R, Ts , Tspl ), Tout ( r , Ts , Tspl ) ) Theory_DRenh = 20 Log    min (Tin ( r , R, T , T ), Tout ( r , T , T ) )   s spl s spl 
    27. 27. Pyramidal CMOS Image Sensor  Experimental Results: Ring Sampling & Blur SymmetrizationMotion Blur effect on rolling-shutter classical CMOS imagers Global shutter versus rolling shutter and motion blur Rolling shutter is used to separate integration time from frame time (speed) To cancel motion blur, Global shutter is used but mainly for still imaging applications To have a motion-blur free video (continuous) images, global shutter is not adequate due to : Higher transmission rate requirements to transfer high frame rate images. Higher power consumption Excessive heat (increasing dark current). The above factors are more pronounced with increasing CMOS imagers resolutions.
    28. 28. Pyramidal CMOS Image Sensor  Experimental Results: Ring Sampling & Blur SymmetrizationThe PixeLINK CMOS imager window (784x784) exposure (or integration) time = 43.2msThe integration time the 64x64 pyramidal CMOS imager =43.24ms
    29. 29. Foveated Dynamic Range Enhancement  Experimental Results…( + )x½= , Inward scan Outward scan Fused Image Rolling scan Demonstration of Foveated Dynamic Range Enhancement at the Foveal Rings( + )x½= , Inward scan Outward scan Fused Image Rolling scan Demonstration of Foveated Dynamic Range Enhancement at the Peripheral Rings
    30. 30. Pyramidal CMOS Image Sensor  High-Speed Imaging … FRPyr R + 1.5 Inner ring =8 R →∞ → 8   FRClass R+7 Outer ring R=1024 The Pyramidal CMOS imager is designed primarily for parallel ring sampling with 8 output channels  This property of multi-channel readout is inherent to the pyramidal architecture in to contrast to multi-channel technique included in high speed classical CMOS imagers. In serial readout of sampled rings:  Pyramidal imager is faster at its fovea (central rings) than a classical CMOS imager of similar size of the fovea.  Pyramidal imager is very close in image sampling speed with the classical imager at higher imager resolutions.
    31. 31. Low Pyramidal Imager FPN Perception by Human Visual System Oblique effect found in the contrast sensitivity of the HVS at relatively high frequencies [94] Average spatial power spectrum distribution of about 500 natural scenes [100]
    32. 32. Low Pyramidal Imager FPN Perception by Human Visual SystemClassical CMOS Imager FPN 2D-Fourier Topology Spectrum
    33. 33. Low Pyramidal Imager FPN Perception by Human Visual SystemPyramidal CMOS Imager FPN Topology
    34. 34. HVS Spatial Filter Application on Pyramidal FPN images: Model Human Visual System Low Pyramidal Imager FPN Perception by
    35. 35. HVS Spatial Filter Application on Pyramidal FPN images: Results Human Visual System Low Pyramidal Imager FPN Perception by
    36. 36. HVS Spatial Filter Application on Pyramidal FPN images: Results Human Visual System Low Pyramidal Imager FPN Perception by
    37. 37. HVS Spatial Filter Application on Pyramidal FPN images: Results Human Visual System Low Pyramidal Imager FPN Perception by
    38. 38. HVS Spatial Filter Application on Pyramidal FPN images: Results Human Visual System Low Pyramidal Imager FPN Perception by
    39. 39. Low Pyramidal Imager FPN Perception by Human Visual SystemHVS Spatial Filter Application on Pyramidal FPN images: Conclusion  FPN is known to increase with increasing sampling frequency (or with decreasing integration time). This is why more oblique stripes are more visible with increasing sampling frequency.  The positive value of the oblique FPN stripes infers the presence of the FPN noise stripes in the tilted HVS filter and its absence from the normal HVS filter. Therefore, the pyramidal imager FPN noise is less perceived by normal human observer in contrast to the normal CMOS imager counter part.  The importance of sampling architectures is proven again not only to provide better imaging performances, but can make some inherent limitation of CMOS image sensors, such as higher FPN compared to CCD’s, less noticeable.
    40. 40. The Seminar Outline Standard CMOS Image Sensor System Level Approach (Time Domain)  Pyramidal CMOS Image Sensor  Architecture  Physical Design  Scanning  Foveated Dynamic Range Enhancement  High-Speed Imaging of Pyramidal CMOS Imager  Low Pyramidal Imager FPN Perception by HVS Pixel Level Approach (Spatial Domain)  Multiresolution CMOS Image Sensor  Architecture  Multiresolution Active Pixel Sensor  Results Conclusion and Future Work
    41. 41. Pixel Level Approach Spatial Domain Approach:  Only regions of interest are sampled at the highest possible resolution. Other regions of less (or no) interest are sampled at lower resolutions.  human Eye has variable interconnectivity configurations between the photo-cells and the ganglion cells (processing neurons):  one-to-many configuration for cones (highest resolution needed for reading)  Many-to-one configuration for rods (low resolution used for low light vision) Design Approach:  Keep the CMOS Image Sensor architecture while changing the APS Sampling structure Multiresolution CMOS Image Sensor
    42. 42. Multiresolution CMOS Image Sensor Image Sampled at highest resolution in all pixels Only Regions-Of-Interest (ROI) are kept at their highest resolution (no-charge distribution) Regions-Of-Less-Interest (ROLI) are selectively down-resolved by a charge-sharing mechanism to get a single value representing their average. Previous implementations were chip and column level approaches using the same concept. The suggested approach is pixel-based to ensure the expandability of the architecture to any imager size without increasing complexity.  Further details about the mechanism implementation, layout design as well as simulated Multiresolution foveated images are discussed in the thesis.
    43. 43. The Seminar Outline Standard CMOS Image Sensor System Level Approach (Time Domain)  Pyramidal CMOS Image Sensor  Architecture  Physical Design  Scanning  Foveated Dynamic Range Enhancement  High-Speed Imaging of Pyramidal CMOS Imager  Low Pyramidal Imager FPN Perception by HVS Pixel Level Approach (Spatial Domain)  Multiresolution CMOS Image Sensor  Architecture  Multiresolution Active Pixel Sensor  Results Conclusion and Future Work
    44. 44. Conclusion and Future WorkConclusion:The architecture flexibility of CMOS imaging technology was: Key factor:  In enhancing the optical dynamic range following a foveated profile mimicking human eye (biological) vision. Key factor:  In minimizing the perception of the CMOS imager FPN noise by human observer benefiting from HVS spatial discrimination limitation (Oblique Effect) Key factor:  In developing similar multiresolution mechanisms as those found at retina of the human eye which lead to an expandable multiresolution CMOS imager.Future Work Biological vision systems can give a great help in synthesising novel architectures exhibiting more adaptability, to light intensity for higher ranges and spatial resolving power. The Pyramidal CMOS imager and the multiresolution CMOS imager are first steps towards this vision.
    45. 45. Thank you

    ×