Camera , Visual , Imaging Technology : A Walk-through Presentation Transcript
Camera / Visual/ Imaging Technology:
A Walk-through ...
- Human Visual System
- Camera Technology and Features
- Future of Camera system and Technology
- Multimedia System Software Engineer ; with specialisation and passion for Camera/Imaging! :)
• Primarily for humans eye (visible spectrum)
• Machines (visible + invisible spectrum)
• Human Visual System. (HVS model)
• Analog and Digital (conversion & storage)
Artifacts / issues / adjustments with Digital capture.
• Comparison of Human Eye. (Photography need for Humans)
• Not for humans eye.
Basic Items in digital image capture. (just capture aspect – Part-I)
• Camera front end:
• Image sensors :
» CMOS/CCD (2D conventional): dynamic range , format, types, etc.
» 3D sensors
» D/A artifacts introduced.
» Resolution: benefit and disadvantage.
» Need for lens.
» Artifacts introduced.
• Specification of the captured image:
» Exposure, Focus, White balance (colorness aspect).
• Image pipe-line : raw to yuv or jpeg
• Typical digital imaging pipeline. (interface, algorithms)
» Raw, cfa, lens,
Camera : Intelligent/ advanced processing aspect: (Part-II)
Fundamental Intelligence: MUST HAVE
• Intelligent 3A : camera HW is not human eye.
Advanced imaging processing: Computer Vision
• Note on Computer vision – for human and for machine.
• Video/ image stabilization
• Reg-eye reduction, Effects ,
• Panorama/ 360view stiching.
• High Dynamic Range Imaging/ Automatic local Brightness, contrast control.
• Multi focus capture. (Pelica/ )
• 2D to 3D conversion.
• Multi-View capture. (3D)
• Face/ eye/ smile detection.
• Object /shape/ scene detection and recognition.
• Scene and object comparison.
• face recognition.
• Gesture recognition.
• Machine learning getting to machine/computer vision.
Computer Vision, OpenCV and the future of Camera Technology.
Rods and cones
• 120 million receptors in each eye.
– Cones– red, green, blue cones. Colour/Day vision.
– Rods - low light - night vision.
- Peak at 498 nm.
- Red or "L" cones peak at 564 nm.
- Green or "M" cones peak at 533 nm.
- Blue or "S" cones peak at 437 nm.
Hue, Saturation and Brightness
• The curved surfaces of the eye focus the image
onto the back surface of the eye rest is up to the
brain to make sense of the information received.
Sensitivity and Dynamic Range:
Variable range for different scenes.
Brain helps in creating final impression.
Much larger than digital camera.
Resolution details & color : the human eye
Capable of resolving up to 53Mpix; But human eye scan of a scene is not one shot.
It will be keep on scanning at different regions. And brain forms the image of total picture.
Camera pipeline : sensor module : Bayer filter
Issues and Need for improvement:
• Image Noise (photon, thermal, electrical, silicon defect)
• Image Distortion (Lens property)
• Image sharpness (focus aspect)
• Image brightness/ lightness (exposure aspect)
• Image colour mismatch (white balance and color correction aspect)
Bayer to RGB : CFA interpolation
More sensitive to Green and that dominates the content details.
Luma and chroma
Luma component is more important and most sensitive
Chroma is not that important as Luma:
Thus, YUV444 can give the same information as YUV422 and
Image Noise – from Sensor
- Amplifier Noise.
- Salt and pepper Noise – ADC , pixel silicon defect.
- Short noise - quantum fluctuations.
- Quantization Noise.
Effect of sensor size: and manufacturing : cheaper, costlier, pixel size , pixel to pixel
How much light able to collect – FSI, BSI sensors.
NOISE Filter of different capability would be needed to remove these.
Exposure/ Focus / White balance
• Camera needs to adjust the parameters to simulate human eye/brain.
• Exposure control goes to sensor: after evaluation is made by software.
• Exposure time/ shutter speed.
• Analog gain / ISO speed.
• Aperture size
• (mobile phone cameras doesn’t have variable aperture)
Focus control goes to Lens: after evaluation is made by software.
• Lens position is adjusted to achieve best focus.
- Different lighting conditions.
Image artifact – from CCD Sensor
Image artifact – from CMOS Sensor rolling shutter
Spatial image aliasing/moire noise
Lens Sharpness: finally its lens – multi element lens
Quantization effect. (quality factor)
Video Compression also has similar artifacts.
JPEG compression artifacts
High Dynamic Range Imaging: (HDR)
Next Level Advanced Enhancements/ Algorithms
• Video correction is easy.
• Image correction is complex.
• Morpho Movie Solid Demo:
Video/ Image Stabilization / anti-shake
Using Intelligent algorithms to “detect” “analyzing” and “recognizing” the image frame
It is a subjective classification with accuracy information.
Accuracy can be improved by making the machine/computer to learn and see multiple
scenarios of the same case.
This is machine Learning.
What was there in PC and desktop implementation and was with researchers are coming
on to hand-held devices.
Machine Vision / Computer Vision
Intelligent processing & understanding captured image.
Robotic vision and 3D camera/ advanced vision:
3D camera – 2 camera based and 1 camera based.
Depth sensing camera.
123d catch – 2D to 3D scan: https://
Augmented reality based application growth in Handheld devices.
AR: (Qualcomm SDK apps) : https://www.youtube.com/watch?v=_