The human eye is quite similar to a photographic camera. The cornea and the eye lens are the optical elements responsible for forming an image in the back of the eye. The iris is like the diaphragm of the camera, where the opening (or the aperture) controls the amount of light entering the eye. The retina, located at the back of the eye, is like the film, detecting the photons that entered the eye and then turning them into electrical impulses that exits out to the brain through the optic nerve. Now let us look at each part of the eye in detail.
So far in the course we have been analyzing various imaging systems in a system chain analogy where the imaging chain consisted of different steps in the whole system. Human visual system can also be considered as an imaging chain, where there are optical elements for image formation, anatomy and physiology responsible for exposure control, detectors responsible for capturing photons and turning them into electrical impulses, and processing. This section will cover the first three boxes responsible for image formation, exposure control, and detection. The later chapters will cover the processing and perception that the brain is responsible for.
The image is formed at the back of the eye using the cornea and the eye lens. The image formed is upside down and real. As we will see, the cornea is responsible for the most of the refraction of the light, while the eye lens is the fine tune used to focus between far and close objects.
Camera , Visual , Imaging Technology : A Walk-through
Camera / Visual/ Imaging Technology:
A Walk-through ...
- Human Visual System
- Camera Technology and Features
- Future of Camera system and Technology
- Multimedia System Software Engineer ; with specialisation and passion for Camera/Imaging! :)
• Primarily for humans eye (visible spectrum)
• Machines (visible + invisible spectrum)
• Human Visual System. (HVS model)
• Analog and Digital (conversion & storage)
Artifacts / issues / adjustments with Digital capture.
• Comparison of Human Eye. (Photography need for Humans)
• Not for humans eye.
Basic Items in digital image capture. (just capture aspect – Part-I)
• Camera front end:
• Image sensors :
» CMOS/CCD (2D conventional): dynamic range , format, types, etc.
» 3D sensors
» D/A artifacts introduced.
» Resolution: benefit and disadvantage.
» Need for lens.
» Artifacts introduced.
• Specification of the captured image:
» Exposure, Focus, White balance (colorness aspect).
• Image pipe-line : raw to yuv or jpeg
• Typical digital imaging pipeline. (interface, algorithms)
» Raw, cfa, lens,
Camera : Intelligent/ advanced processing aspect: (Part-II)
Fundamental Intelligence: MUST HAVE
• Intelligent 3A : camera HW is not human eye.
Advanced imaging processing: Computer Vision
• Note on Computer vision – for human and for machine.
• Video/ image stabilization
• Reg-eye reduction, Effects ,
• Panorama/ 360view stiching.
• High Dynamic Range Imaging/ Automatic local Brightness, contrast control.
• Multi focus capture. (Pelica/ )
• 2D to 3D conversion.
• Multi-View capture. (3D)
• Face/ eye/ smile detection.
• Object /shape/ scene detection and recognition.
• Scene and object comparison.
• face recognition.
• Gesture recognition.
• Machine learning getting to machine/computer vision.
Computer Vision, OpenCV and the future of Camera Technology.
Rods and cones
• 120 million receptors in each eye.
– Cones– red, green, blue cones. Colour/Day vision.
– Rods - low light - night vision.
- Peak at 498 nm.
- Red or "L" cones peak at 564 nm.
- Green or "M" cones peak at 533 nm.
- Blue or "S" cones peak at 437 nm.
Sensitivity and Dynamic Range:
Variable range for different scenes.
Brain helps in creating final impression.
Much larger than digital camera.
Resolution details & color : the human eye
Capable of resolving up to 53Mpix; But human eye scan of a scene is not one shot.
It will be keep on scanning at different regions. And brain forms the image of total picture.
Camera pipeline : sensor module : Bayer filter
Issues and Need for improvement:
• Image Noise (photon, thermal, electrical, silicon defect)
• Image Distortion (Lens property)
• Image sharpness (focus aspect)
• Image brightness/ lightness (exposure aspect)
• Image colour mismatch (white balance and color correction aspect)
Bayer to RGB : CFA interpolation
More sensitive to Green and that dominates the content details.
Luma and chroma
Luma component is more important and most sensitive
Chroma is not that important as Luma:
Thus, YUV444 can give the same information as YUV422 and
Image Noise – from Sensor
- Amplifier Noise.
- Salt and pepper Noise – ADC , pixel silicon defect.
- Short noise - quantum fluctuations.
- Quantization Noise.
Effect of sensor size: and manufacturing : cheaper, costlier, pixel size , pixel to pixel
How much light able to collect – FSI, BSI sensors.
NOISE Filter of different capability would be needed to remove these.
Exposure/ Focus / White balance
• Camera needs to adjust the parameters to simulate human eye/brain.
• Exposure control goes to sensor: after evaluation is made by software.
• Exposure time/ shutter speed.
• Analog gain / ISO speed.
• Aperture size
• (mobile phone cameras doesn’t have variable aperture)
Focus control goes to Lens: after evaluation is made by software.
• Lens position is adjusted to achieve best focus.
- Different lighting conditions.
Lens Sharpness: finally its lens – multi element lens
Quantization effect. (quality factor)
Video Compression also has similar artifacts.
JPEG compression artifacts
High Dynamic Range Imaging: (HDR)
Next Level Advanced Enhancements/ Algorithms
• Video correction is easy.
• Image correction is complex.
• Morpho Movie Solid Demo:
Video/ Image Stabilization / anti-shake
Using Intelligent algorithms to “detect” “analyzing” and “recognizing” the image frame
It is a subjective classification with accuracy information.
Accuracy can be improved by making the machine/computer to learn and see multiple
scenarios of the same case.
This is machine Learning.
What was there in PC and desktop implementation and was with researchers are coming
on to hand-held devices.
Machine Vision / Computer Vision
Intelligent processing & understanding captured image.
Robotic vision and 3D camera/ advanced vision:
3D camera – 2 camera based and 1 camera based.
Depth sensing camera.
123d catch – 2D to 3D scan: https://
Augmented reality based application growth in Handheld devices.
AR: (Qualcomm SDK apps) : https://www.youtube.com/watch?v=_
Aperture 1/∞ DOF (out of order in this slide:) )