This document summarizes key concepts related to television imaging and the human visual system. It discusses how television aims to accurately present distant scenes in terms of geometry, brightness, contrast and color. It also explains fundamentals of human vision that television design is based on. Key aspects covered include the electromagnetic spectrum, color temperature, the definition of white, saturation, contrast, scanning and synchronization, color displays, and common video codecs.

1. IMAGES
Television- "seeing from afar" hopes to
present an image of a distant scene as
accurately as possible in terms of:
-- Geometry
-- Brightness and contrast
-- Colour conditions
 It is necessary to understand the fundamentals of
human vision from which the design criteria for
television originate

2. The Electromagnetic Spectrum
Based on http://www.colourware.co.uk/cpfaq/q1-1.htm
Light is the visible part of the electromagnetic
spectrum

3. Colour Temperature
Objects generate a spectral power distribution
(SPD) according to how hot they are.
This is black body radiation, light sources can be matched against
these colours, eg:
2800 degrees K- Tungsten filament light bulb
6500 degrees K - Standard Daylight
This can be done even if the light source is not a black body, e.g. a
fluorescent tube, LED.
Colour temperature is used to calibrate the white balance for
lighting and exposure control

4. The definition of white
One of the interesting properties of the
human eye is its ability to see white.
White is scientifically defined as the
equality of all wavelengths.

5.  In reality, and as far as the human observer is
concerned, several colours are acceptable as
white.
 It is for this reason that the lighting for
television must be strictly controlled in terms
of the spectrum.

6. Standardisation is to D6500
Red, at 650 nm
Green, at 530 nm
Blue, at 460 nm

7.  D6500 is the white usually used in colour TV and
corresponds to a mixture of direct sunlight and diffuse
"skylight".
 It can be generated using pixels on a colour TV
screen.
 In a TV studio, tungsten lighting is almost always
used, which is deficient in the short wavelengths.
 The colour of the received picture uses electronic
processing which corrects for “white balance”

8. The Human Eye
http://webvision.med.utah.edu/anatomy.html

9. Visual Perception
daylight - cones (colour - sensitive down to 1 Lux)
3 types - each has different response curve
responsible for normal colour vision
6-7 million in each eye
night - rods (monochrome only - 10,000 times
more sensitive)
sensitive to incidental light and NOT colour
responsible for brightness and contrast information
100-120 million in each eye

10. Visual Sensitivity
 Spectral sensitivity functions derive from photoreceptor
sensitivities.
Peak sensitivities
R 580nm
G 550nm
B 440nm
Overall 555nm
Sensitivity of an individual From Wald G: Science 145:1007-1016, 1964,
URL: http://www.unmc.edu/Physiology/Mann/mann7.html

11. Visual Perception
day & night
vision
non-
uniform
response

12. Colour mixing
It has been known for some time that
the subjective effect of a particular
colour can be simulated by light of other
wavelengths.
As an example, a red light and a green
one, in varying proportions, can
produce a wide range from red through
to orange, yellow, and finally green.

13. http://www.colour.org.uk/additive.jpg

14. Subtractive colour mixture:
overlapping cyan, magenta and yellow filters

17. Saturation
 Another aspect is the definition of the depth of a
colour. This is known as the saturation.
 The hue of a colour is the apparent colour of the
dominant wavelength, eg blue, red, yellow, etc.
 Saturation can describe a whole range of colours,
from white, through pastel shades, right through to a
full deepness. A pale pink is a desaturated red.
 Desaturated colours contain a proportion of white, as
well as the predominant colour.

18. Saturation =
Maximum value - minimum value
---------------------------------------- X 100%
Maximum value
eg if Red = 0.5, Green = 0.5, Blue = 0.1
Saturation = 0.5 (Red or green) - 0.1 x 100% = 80%
0.5
 The colour would be described as a 50% amplitude,
80% saturated, yellow, (Red = Green), ie the hue is
"yellow".

20. Light and Colour
Luminous Intensity (Candela)
one Candela (cd) = one candle power (c.p.)
Luminous Flux (φ ) (Lumens)
one Lumen (lm) is the quantity of luminous
flux which falls upon a surface

21. Light and Colour
Illuminance (E)
measure of the concentration of luminous flux
falling on a surface
Illuminance expressed as Lux
one Lux = one lumen per square metre (lm/m2
)

22. Brightness
Another parameter is the brightness of
the perceived colour. If light of a certain
wavelength is added to light of the
same wavelength, then the total
brightness is perceived as the sum of
the two (Grassman's law).
There is incoherent addition of the
energies in the light.

23. Contrast
Contrast is a phenomenon, which
relates to, or is a comparison of, the
difference in the colour and brightness
of the object and other objects within
the same field of view.
The acceptability of a given contrast
depends on light level
Expressed as a ratio, eg 100:1.

24. Television
TV cameras are used to convert light from
a scene to an electrical signal.
TV displays are reciprocal to the TV
camera at the other end.

26. TV Image Scanning
The image has to be scanned and
reproduced at a rate without flicker and
appear as continuous motion.
The persistence of vision is such that a
minimum rate for the reproduction of
images is about 12 images/s.
At this rate movement appears jerky and
so a higher rate is used.

27. Basic Television System
Raster Scan system - Cathode Ray Tube
• electron beam deflected horizontally and
vertically to trace the image of a picture
onto a phosphor screen
• intensity of the beam modulated in
synchronism with the scan

28. Basic Television System
Broadcast television standards take into
account historical developments and the
need to produce material from films.
UK PAL (phase alternation line)
625 Lines, field scan rate 50Hz
2 Fields - 0dd and Even
Other standards NTSC, SECAM

31. Line scanning
 When line scanning is used the picture is
scanned from top to bottom sequentially using
625 lines.
 With interlaced scanning the image is renewed
twice in the time taken to complete the scan.
Two fields are scanned for each frame.
 Thus for a field rate of 50Hz the frame rate is
25Hz. This frame rate is compatible with a cine-
film frame rate of 24 frames/s.

32. Aspect ratio
 The aspect ratio of the image is the ratio of the
horizontal dimension to the vertical dimension.
An aspect ratio of 4 x 3 is used for standard
broadcast television (widescreen uses 16 x 9).
 If there are 625 lines vertically then for a similar
resolution horizontally the number of resolvable
points or picture elements along each line
should be 625 x 4/3 or 833. With equal
resolution in each direction the total number of
picture elements or pixels in the image is
520,833.

33. The colour display
In a colour display, there is a need to
display 3 coloured images simultaneously.
 The principle of operation is that a red,
green, and blue image will additively
together fool the eye into believing a wide
range of colours is actually present.

35. Light emitting dots
To do this, light emitting dots are used over
the whole screen.
The light emitting zones are in groups of 3.
One dot emits red light, another green, and
the third blue.
Each phosphor in a CRT is individually
activated by its own unique electron beam.
LCD, LED, and plasma work to same
principle

37. Synchronisation
 When a TV camera has formed a complete
image, it has to send this in a way that any
receiver can re-assemble it correctly, i.e. the
right lines at the correct vertical position.
 The receiver display scanning must be
synchronised with that of the transmitted
waveform. Synchronising pulses are
introduced into each line and these are used
to identify the start of each field and frame.

38. The sync pulse
 At the start of each line there is a
synchronising pulse, followed by the line
signal.
 The entire line and sync pulse comprises a
signal which is analysed at a standard 1 Volt
peak-to-peak amplitude.
 The image signal lies between 0.3 and 1.0
Volt and the line sync signal lies between 0
and 0.3 v. The 0.3 V level is the black level
and 1.0 V is the peak white level.

39. 625/50 signal, horizontal timing
|<------->|<----------------->|
horizontal active line
blanking 52us
12us
* ---
*** * |
* * -- video signal |
** * |
* 4.7us * * * 0.7 volts
* sync * * |
* pulse * * |
*** | *** *** *** ---
| * | * | * * |
| * | * | * * 0.3 volts
| ******* | ******* ---
| |
1.5us 5.8us
front back
porch porch

40. 625/50 signal, horizontal timing
http://graffiti.virgin.net/ljmayes.mal/var/tvsync.htm

41. Basic Television System
Video signal processing
RF (radio type - e.g. TV aerial)
Composite
YC
YUV (component)
RGB

42. Basic Television System
RF= video and audio together (on 2 slightly
separated frequencies)
Composite = video information combined
ie.
luminance (Y) + chrominance (C)

43. S-Video
• YC = luminance (Y) and chrominance (C)
processed separately
• S-Video (also known as Y/C) is a higher quality
signal than composite video, but a lower quality
than component video.
•
This mid-level format divides the signal into two
channels - luminance and chrominance.

44. Component video
YUV (component) = chrominance in 2 parts
(colour difference signals)
luminance (Y)
chrominance = (U) (V)
Why U & V, rather than RGB?
RGB requires 3 signals and greater
bandwidth

45. Component video
How the Colour Difference Signals are calculated
U = B-Y (i.e. blue colour minus luminance value)
V =R-Y (i.e. red colour minus luminance value)
Green is calculated electronically Y-B-R = G
Component digital video signals are sometimes referred to
as 4:2:2, meaning that in an 8-bit environment, 4 bits are
given to the Y component while 2 bits each are
dedicated to the B & R components.
The luminance or Y channel carries most of the image
detail and is, therefore, assigned more bits.

46. Codec
 Codec is an abbreviation / acronym of
"coder/decoder" or "compressor/decompressor",
which describes a device or program algorithm
that compresses data when you are recording and
producing a movie, and then decompresses the
data when the movie is being viewed.
 The codec you use when recording and producing
your movie will affect both the quality and the size
of the movie when it is completed and viewed.

47.  Over the years many different codecs have been
developed. Each codec has its strengths and
weaknesses.
 Video codecs can be divided into two categories:
lossless and lossy.
 Lossless codecs maintain perfect image quality
when the video is compressed, while lossy
codecs sacrifice image quality for compression.
 A still image JPG file is an example of lossy
compression. The image file size is much smaller,
but the image quality is degraded. Microsoft
Video 1 is a lossy codec.

48. Using a lossy video codec, each time you
reproduce the video the quality will
degrade.
This is the same as making a copy of
videotape and then copying from that copy.
The video quality of the second generation
copy is severely degraded, and subsequent
copies degrade further.

49. Identifying the Codecs on your System
In Windows XP:
Select Windows Start > Control Panel > Sounds and
Audio Device Properties > Video Codecs or Audio
Codecs.
The Codec a video file is compressed with:
Windows Explorer, right-click on the video file name,
select Properties > Summary tab > Advanced. The
codec is listed in the Video Compression entry.

50. Some examples of codecs include the
following:
AVS
Theora
Tarkin
H.261
H.263
H.263v2
H.264
MPEG-1 Video
MPEG-2 Video
MPEG-4 Video
Sorenson codec
Cinepak
Indeo 3/4/5
MJPEG

51. The codec is not to be confused with the
file format used to store the a/v information
encoded by the codec.
File formats like ".ogg", ".mpg", ".avi",
".mov", etc. are used to store information
encoded by a codec.

52. video file format
A video file format is a standard of
encoding video, audio and some auxiliary
information into a file.
In contrast to audio and image formats,
most video file formats allow a lot of
codecs, both audio and video, to be used.
Auxiliary information often includes data
required for correct synchronization
between audio and video subtitles

53. Popular video file formats and codecs
AVI (.avi)
 XviD
 DivX
 FFMPEG
 Indeo
 Cinepak
 Windows Media
MPEG-1 and
MPEG-2,
MP4 (MPEG4)
(.mpg)
QuickTime (.mov)
 Sorenson
 Cinepak
RealMedia (.rm)
 RealVideo

54. AVI - Audio Video Interleave, (defined by Microsoft)
 AVI files consist of still images called frames. Frames are
combined sequentially in one file so when it is played
using Media Player for example, each consecutive image
is played in the same way that a video player displays a
video.
 AVI files usually contain many thousands of frames, and
this produces very large file sizes. The video data can be
compressed using various compression algorithms called
codecs.
 Uncompressed, AVI produces enormous files, so to keep
sizes more reasonable, the codec should compress the
data stored in the AVI file.

55. Audio/Video Interleave File
 The AVI file is the most widely used audio/video format on
Windows platforms. However it is not compressed with one
specific codec, rather it is a file that can be compressed (or
completely uncompressed) with any one of numerous codecs
(examples: DivX, MPEG-4v2, Indeo 3.2, I263, Cinepak etc.).
 To play an AVI file in Windows you must have the correct codec
installed. There are many Windows applications available that
can play AVI files with the most widely used being Microsoft
Windows Media Player
 If you do not have the codec needed to play the file, Windows
Media Player will not play it. However, it may be possible to
play the audio stream within the file.

56. MPEG
Acronym for Moving Picture Experts Group
A group charged with the development of video and
audio encoding standards.
Since its first meeting in 1988, MPEG has grown to
include members from various industries and
universities.
 http://www.mpeg.org/
 http://en.wikipedia.org/wiki/Moving_Picture_Experts_Group

57. Before MPEG, there was the looming threat
of world domination by proprietary
standards cloaked in mystery.
Lossy compression is an inexact science
which always requires visual tweaking and
implementation tradeoffs
You never know what's behind any such
scheme (a lot of marketing hype).

58. How MPEG works
 The MPEG codecs use lossy data compression
 The moving picture coding systems such as MPEG-1, MPEG-2,
and MPEG-4 add an extra step, where the picture content is
predicted from past reconstructed images before coding, and
only the differences from the reconstructed pictures, and any
extra information needed to perform the prediction, are coded.

59. MPEG video compression
 The basic idea behind MPEG video compression is to
remove spatial redundancy within a video frame and
temporal redundancy between video frames.
 The images in a video stream usually do not change
much within small time intervals.
 The idea of motion-compensation is to encode a
video frame based on other video frames temporally
close to it.

60.  Frames can be encoded in three types: intra-frames (I-frames),
forward predicted frames (P-frames), and bi-directional predicted
frames (B-frames).
 An I-frame is encoded as a single image, with no reference to any
past or future frames. The encoding scheme used is similar to JPEG
compression.
 A P-frame is encoded relative to the past reference frame. A
reference frame is a P- or I-frame. The past reference frame is the
closest preceding reference frame.
 A B-frame is encoded relative to the past reference frame, the future
reference frame, or both frames. The future reference frame is the
closest following reference frame (I or P). The encoding for B-frames
is similar to P-frames, except that motion vectors may refer to areas in
the future reference frames.

61. MPEG has standardized the following
compression formats:
 MPEG-1
 Includes the popular Layer 3 (MP3) audio compression format.
 MPEG-2
 Video and audio standards for broadcast-quality television. Used on most
DVD movies.
 MPEG-3
 Originally designed for HDTV, but abandoned in favor of MPEG-2.
 MPEG-4
 Expands MPEG-1 to support video/audio "objects", 3D content, low bitrate
encoding and support for Digital Rights Management.
 MPEG-7 A formal system for describing multimedia content.
 MPEG-21 Future standard as a Multimedia Framework.

62. MPEG-1 VIDEO
 MPEG-1 was optimized for CD-ROM or
applications at about 1.5 Mbit/sec, which was
strictly non-interlaced.
 MPEG-1 video is used by the Video CD format.

63. MPEG-2 - Video Standard
 MPEG-2 specifies encoding for high-quality digital video.
 MPEG-2 Video builds on the completed MPEG-1 Video
Standard and works with I, P and B fields.
 MPEG-2 supports both interlaced and progressive scan
video streams. In progressive scan streams, the basic unit
of encoding is a frame, while in interlaced streams, the
basic unit is a field.
 MPEG-2 is typically used to encode audio and video for
broadcast TV signals, including digital satellite and Cable.
 MPEG-2, with some modifications, is also the coding
format used by standard commercial DVD movies.

64. MPEG-1/2 AUDIO – aka MP3 audio
 To compress audio, MPEG removes the redundant parts
of the sound signal that we do not hear (lossy).
 To do this MPEG Audio uses psyco- acoustic principles.
 Unlike video there is no perceivable quality loss. MPEG
Layer 3 is an excellent audio codec. It produces small files
sizes and maintains good audio quality.
 Many listeners accept the MP3 bitrate of 128 kilobits per
second (kbit/s) as near CD quality at a compression ratio
of approximately 11:1.
 Different mp3 codecs produce different results at the
same bitstream rates. (Black art).
 http://mp3licensing.com/

65. MPEG-4
 MPEG-4 (1998) is the designation for a group of audio and
video coding standards agreed upon by MPEG
 MPEG-4 is primarily designed to handle low bit-rate content,
from 4800 bit/s to approximately 4 Mbit/s.
 The primary uses for the MPEG-4 standard are web (streaming
media) and CD distribution, conversational (videophone) uses,
and broadcast television.
 MPEG-4 absorbs many of the features of MPEG-1 and MPEG-
2, adding new features such as (extended) VRML support for
3D rendering, object-oriented composite files (including audio,
video and VRML objects), support for Digital Rights
Management and various types of interactivity.

66. MJPEG
 Motion JPEG (MJPEG) is a video codec where
each frame is separately compressed into a
JPEG image.
 The resulting stream quality is independent from
the motion in the image, different from MPEG
video, where quality often decreases when small
images move rapidly.
 Typical data rate (29 Mbit/s) quality is very high,
files are very large.

67. DV
 Codec for DV video, the encoding format used by most
digital camcorders, typically those that support the IEEE
1394 (a.k.a. FireWire or i.Link) interface.
 DV is a digital video format, supported by a consortium of
over 50 companies, launched in late 1995.
 The codec sits in the camcorder or VCR. With DV, capture
and compression happens in the recorder.
 FireWire™ is a serial data transfer protocol and
interconnection system, originally developed by Apple
Computer, used (amongst other things) to transmit DV.
 In 1995, FireWire was standardized by the Institute of
Electrical and Electronic Engineers as IEEE 1394-1995.
http://en.wikipedia.org/wiki/DV

68. DV editing
 DV editing uses the Firewire I/O present in many DV
camcorders and DVCRs, connected to Firewire I/O boards
 You edit and stay in DV end-to-end. The digital data is
copied to your computer via the Firewire. This isn't
"capture," it's data transmission.
 With DV, the video is already captured and compressed.
No generation loss.
 You need:
 a DV camcorder or DVCR equipped with Firewire (IEEE 1394).
 a Firewire interface board.
 A non-linear editing application such as Adobe Premiere.
 a suitable computer with lots of hard drive space, fast enough to cope with
a data stream of 3.7 Megabytes per second.

69. VCD SVCD X(S)
VCD
DivX DV DVD
Formal
standard?
Yes Yes No No Yes Yes
Resolution
PAL/NTSC
352x240
352x288
480x480
480x456
720x480
720x576
or lower
640x480
or lower
720x480
720x576
720x480
720x576
Video
compression
MPEG-1 MPEG-2 MPEG-1
or
MPEG-2
MPEG-4 DV MPEG-2
Audio
compression
MPEG-1 MPEG-1 MPEG-1 MP3
WMA
DV MPEG-2
AC3
MB/min 10 10-20 5-20 1-10 216 30-70
DVD Player
compatibility
Very good Good Good None None Excellent
How CPU
intensive
Low High High Very high High Very High
Quality Good Very good Very good Very good Excellent Excellent

70. Comparison of audio codecs
http://en.wikipedia.org/wiki/Comparison_of_audio_codecs

71. Comparison of container formats
http://en.wikipedia.org/wiki/Comparison_of_c
ontainer_formats

72. List of Open source codecs
http://en.wikipedia.org/wiki/Open_source_cod
ecs_and_containers

73. Comparison of video codecs
http://en.wikipedia.org/wiki/Comparison_of_vi
deo_codecs