Multimedia Components

EC2037 - MULTIMEDIA
COMPRESSION AND
COMMUNICATION
Dr.C.Arun
07/31/16 Dr.C.Arun, REC

UNIT I: MULTIMEDIA COMPONENTS
• Introduction - Multimedia Skills -
Multimedia Components and their
Characteristics - Text, Sound, Images,
Graphics, Animation, Video, Hardware.

Introduction
What is Multimedia?
1. Multimedia is the integration of multiple forms of
media. This includes text, graphics, audio, video, etc.
2. Multimedia is any combination of text, art, sound,
animation, and video delivered to us by computer or
other electronic or digitally manipulated means. It is
a richly presented sensation.
3. Multimedia is the seamless integration of text, sound,
images of all kinds and control software within a
single digital information environment.
• For example, a presentation

The basic five Elements of
Multimedia
1. Text
2. Images
3. Audio
4. Video
5. Animation

What are online, off-line and hybrid
multimedia products?
• An “offline” project is self-contained, does
not interact with anything outside its
immediate environment
• An “online” project needs to communicate
with distant resources over a network.
• Some projects have elements of both
online and offline projects and we will refer
these as “hybrid” projects.

• Interactive (Nonlinear) Multimedia
Multimedia is combination of digitally manipulated text, photographs,
graphic art, sound, animation, and video elements. When you allow
an end user (user) multimedia project to control what and when the
elements are delivered, it is called interactive or nonlinear
multimedia.
Non-interactive (Linear) Multimedia:
A multimedia project need not be interactive to be called Non-
interactive multimedia: users can sit back and watch it just as they
do a movie or the television. In such cases a project is linear, or
starting at the beginning and running through to the end.

07/31/16 Dr.C.Arun,
Multimedia- Applications
1. Multimedia in Business
• Items difficult to stock like glass utensils, industrial equipment
• Business applications for multimedia include presentations,
training, marketing, advertising, product demos, simulations,
databases, instant messaging, and networked communications.
• Medical doctors can practice surgery methods via simulation
prior to actual surgery. Mechanics learn to repair engines.
2. Multimedia in Education
• These include learning packages and simulation of lab
experiments.
• Different aspects of the course curriculum which cannot be
explained easily through simple text and images could be
presented through video clips, animation, 3D modelling, audio
annotation, etc.
• E-learning

3.Multimedia in Home Entertainment
• computer based games for kids, interactive encyclopaedias,
storytelling, cartoons, etc.
• Wii, X-box, or Sony PlayStation machine.
4.Multimedia in Public Places
• information is accessed through a touch screen and viewed on a
monitor. In hotels, train stations, shopping malls, museums,
libraries, and grocery stores, multimedia is already available at
stand-alone terminals
5.Virtual Reality
• At the convergence of technology and creative invention in
multimedia is virtual reality
• Take a step forward, and the view gets closer
6.Content Based Storage and Retrieval (CBSR) System
• efficient methods of searching non-textual media are being
developed
• matching of a fingerprint form police records to identify the criminal
• identifying person from photograph07/31/16 Dr.C.Arun, REC

In Medicine
Source:
Cardiac Imaging,
YALE centre for
advanced cardiac
imaging

In training

Public awareness
campaign
Source
Interactive Multimedia Project
Department of food science&
nutrition, Colorado State Univ

Multimedia Project Development
Stages:
four basic stages in a multimedia project
1.Planning and costing
2. Designing and producing
3.Testing
4.Delivering

The extended version
1. Choosing a topic:
2. Writing a story:
3. Writing a script
4. Preparing a story board
5. Preparing a flow line
6. Implementation:
7. Testing and feedback:
8. Final delivery:

Multimedia Skills
1. Executive Producer
2. Producer/Project Manager
3. Creative Director/Multimedia Designer
4. Art Director/Visual Designer
5. Artist
6. Interface Designer
7. Game Designer
8. Subject Matter Expert
9. Instructional Designer/Training Specialist
10. Scriptwriter
11. Animator (2-D/3-D)
12. Sound Producer
13. Music Composer
14. Video Producer
15. Multimedia Programmer
16. HTML Coder
17. Media Acquisition
18. Marketing Director

Roles of a Multimedia Project
Manager
(1) Planner – Devises a cost effective method for developing a project within the constraints of schedule
and budget.
(2) Team Builder – Assembles a team of developers and then motivates them to work together.
(3) Organizer-Structures the project by applying the best mix of talent to meet the demands of a schedule
as well as technical requirements.
(4) Negotiator-Balances the needs of the project, the customer, and the development team.
(5) Flexible and assertive coach: Knows how to get the best out of his or her team. This is accomplished
by taking control when necessary to get the job done.
(6) Work Flow Manager: Schedules activities and tasks in a logical sequence.
(7) Sales Person-Understands the needs of the customer and delivers the solution on time and within the
budget.
(8) Problem Solver-Identifies and rectifies technical and management difficulties.
(9) Committed to Quality-Ensures that multimedia products are error free.
(10) Goal Setter- Identifies specific tasks and sees to it that they are completed on time.
(11) Possesses a Positive Attitude- Believes a complex project, despite the things that can go wrong, will
get done, no matter what.
(12) Listener-hears out customers, team members, management, and everyone who has a say in the
project, and then makes a decision that gets the job done.
(13) Multi-tasker- Judges a number of things at once, including technical, management, schedule, and
budget issues. Possessing this skill is critical and may be the most important skill of all.

Digital Image Representation & Processing(1/2)
Lecture-2

To discuss…
 Image fundamentals
 Image Formation
 1-bit & 8 bit image
 Color image

Digital Image
• Image can be defined as a 2-D function f(x,y), where x and y are
spatial coordinates and the amplitude of f at any pair of
coordinates (x,y) is called the intensity/gray level of the image
at that point
– When the image is gray scale, intensity values represent the range of
shades from black to white.
– For a color image the intensity values are represented as a combination of
R, G, B
• Can be considered as comprising x x y number of elements
(picture elements, image elements, pels, pixels), each of which has a
location and value.

Image Formation
• Pixel values are proportional to the energy/ electromagnetic
waves radiated from the source
– It implies this value cannot be negative, ranges from 0 to +ve infinity
• Function f(x,y) characterized by components
– Illumination i(x,y), value ranging from 0 to infinity
– Reflectance r(x,y), value ranging from 0 to 1
– f(x,y)= i(x,y) x r(x,y)
• f(x,y) lies between Lmin to Lmax scaled to [0,L-1], where 0
representing black and L-1 representing white, the intermediate
values are the shades of gray from black to white

Image Formation [2]

Image Formation [3]

• 256 gray levels (8bits/pixel) 32 gray levels (5 bits/pixel) 16 gray levels (4 bits/pixel)
• 8 gray levels (3 bits/pixel) 4 gray levels (2 bits/pixel) 2 gray levels (1 bit/pixel)
Image Formation [4]

Image Formation[5]
• Output of image sensors are continuous voltage
waveform, digitization is necessary for further
processing
• Digitizing the coordinate positions are called sampling
• Digitizing the amplitude values are called quantization
– Number of gray levels will be in an integer power of 2
– L=2k
, [0…L-1]
– Number of bits needed to store an image b=M x N
• Image is k bit image if it has 2k
gray levels
– 8 bit image has 256 gray levels

1-bit image
• Simplest type of image
• Each pixel consist of only ON / OFF information
• Called 1-bit monochrome (since no color) image
• Suitable for simple graphics & text
– JBIG (Joint Bi-level Image experts Group ), A compression
standard for binary image

8-bit image
• Gray levels between 0 to 255
(black to white)
• Image resolution refers the number
of pixels in an image. The higher
the resolution, the more pixels in
the image. Higher resolution
allows for more detail and subtle
color transitions in an image
• Shown is 512x512 byte image

Color image
• 24- bit color image
– Each pixel is represented by 3 bytes, RGB
– Each R, G, B are in the range 0-255
– 256 x 256 x 256 possible colors
– If space is a concern, reasonably accurate color image can be
obtained by quantizing the color information
• 8- bit color image
– Carefully chosen 256 colors represent the image
– We get information can be received from the color histogram

Color image [2]
• For 640 x 480 image represented with
– 24 bits requires 921.6 kbytes
– 8 bit requires 300 kbytes
• The 8-bit color image stores only the index of the color,
the file header will contain the mapping information.
• The table where the color information for all the 256
indices is called color lookup table (LUT)

Color Lookup Table [2]
• Median-Cut Algorithm
1. Find the smallest box that contains all the colors in the image
2. Sort the enclosed colors along the longest dimension of the box
3. Split the box into two regions at the median of the sorted list
4. Repeat steps 2 & 3 until the original color space has divided in to 256
regions
5. For every box, the mean of R,G & B will represent the box
6. Based on the Euclidean distance between these representatives, assign
every pixel a representative value
The new image generated is called color quantized image

Digital Audio

Topics today…
 Digitization of sound
 PCM
 Lossless predictive coding

Sound
• Sound is a pressure wave, taking continuous values
• Increase / decrease in pressure can be
measured in amplitude, which can be digitized
• Measure the amplitude at equally spaced time
intervals (sampling) and represent it with one of
finite digital values (quantization)
• Sampling frequency refers the rate at which the
sampling is performed

Digitizing Sound
• Start with the following questions
– Sampling Rate/Frequency??
– Degree of quantization??
– Uniform quantization or Nonuniform Quantization??
– File Format???

Nyquist Theorem
• Named after Harry Nyquist, mathematician at
Bell Labs
• For lossless digitization, the sampling rate should
be at least twice the maximum frequency responses.
– Indeed many times more the better.
– The frequency equal to half the Nyquist rate is called
Nyquist Frequency.
• If signal is band limited (minimum frequency=f1,
minimum frequency=f2), then the required
sampling rate is at least 2(f2 - f1)

Signal-to-Noise Ratio (SNR)
• Random fluctuation leads to noise in analog
systems
• Ratio of the power of the correct signal to the power
of the noise is called the Signal-to-Noise Ratio
(SNR), measured in decibel (dB).
• SNR is the measure of quality of signal
noise
signal
noise
signal
V
V
V
V
SNR 102
2
10 log20log10 ==

Signal-to-Quantization-Noise Ratio (SQNR)
• Digital signals stores only quantized values
• Number of bits / sample dictates the precision
• Dividing voltage in a fixed range like 0 to 1 into
256 levels leads a round off error.
– The difference between the value of the analog signal, for a
particular sampling time, and the nearest quantization
interval value quantization error/noise
– Quality of quantization is characterized by SQNR

Signal-to-Quantization-Noise Ratio (SQNR)
[2]
• N bits/sample can represent the digital signal in the
range –2N-1
to 2N-1
-1 corresponding the the analog signal in
the range –Vmax to +Vmax.
• Each quantization level represent the voltage of 2Vmax
/ 2N
– Mapping Vsignal to 2N-1 and Vquan_noice to ½
– Since the quantization error is statistically independent &
uniformly distributed from 0 to ½ , SQNR=6.02N+1.76 (dB)
)(02.62log20
2
1
2
log20log20
1
10
_
10 dBNxNx
V
V
SQNR
N
noisequan
signal
====
−

Non Linear Quantization
• Better quantization with limited bits
– Use of weber’s Law
∆response α ∆stimulus/stimulus
– Can be written as dr = k(1/s)ds
– Integrating, r=k ln s + C
– Stated otherwise, r=k ln (s/s0
)
• s0
is the lowest stimulus that causes the response
• Non linear quantization takes advantage of this
logarithmic relation
• Find the stimulus for every unit response from
the stimulus-response plot

Non Linear Quantization [2]
• This leads more
concentrated
quantization near the low
end of stimulus & more
compressed
quantization towards the
higher end of the
stimulus

Audio Filtering [2]
• Filter the audio signal before sampling (A to D)
to remove unwanted frequencies
– For speech signal, retain 50Hz to 10Khz
– For Music signal, retain 20Hz to 20KHz
• Filter after D to A conversion
– Use low pass filter

Coding of Audio using PCM
• PCM – Pulse Code Modulation
• Produces sampled, quantized output of the
audio signal
• Standard telephony assumes the highest
frequency of speech to reproduce is 4kHz, the
sampling frequency is 8kHz. Now 8 bit sample
size leads the bit rate 64kbps
• Before the digitization process to begin, the
signal is filtered to have sounds only up to 4kHz

Coding of Audio using PCM [2]

Coding of Audio using PCM [3]

Introduction to MIDI (Musical
Instrument Digital Interface ) & Music
Synthesis

MIDI
• Efficient method for representing musical
performance information
• One minute music commonly requires 5
Mbytes, whereas MIDI for 1 min requires 1
Kbytes.
– MIDI does not contain sampled audio data,
instead the instructions (MIDI Messages) that the
synthesizer can use to generate sound
– Easy to edit the music, change the playback
speed and the pitch or key of the sounds
independently.

MIDI Basics
• Provide the means for conveying the musical
performances efficiently
• 10 bits transmitted per byte (including 1 start &
stop bits)
• MIDI interface on a MIDI device contains
connectors
– IN
– OUT
– THRU

MIDI Basics [2]
• MIDI data stream originates
from MIDI controller such as
MIDI Sequencer, Keyboard
• The recipient of the MIDI
stream is the MIDI Sound
generator or sound module
• 16 logical channels, identified
by including 4 bit channel
number with the MIDI
messages
MIDI System

MIDI Messages
• A message is composed of a
status byte, followed by one
or two bytes of data
• Classified as
– Channel messages
• Channel voice messages
• Channel Mode messages
– System Messages
• System Common messages
• System Real-time Messages
• System Exclusive Messages
MIDI messages tables
http://www.midi.org/about-midi/table1.shtml

MIDI Sequencers &
synthesizers
• Sequencers
– Helps sending MIDI messages to MIDI synthesizer
– Adds time-stamping to the MIDI messages
• Synthesizer
– Polyphonic
• Ability to play more than one note at a time
– Multitimbral Mode
• Capable of producing two or more different instrument sounds
simultaneously
• If a synthesizer can play five notes simultaneously, and it can
produce a piano sound and an acoustic bass sound at the same
time, then it is multitimbral.

Standard MIDI Files
• MIDI messages are stored in disks as MIDI
files
• MIDI file is the major source of music in
computer games, CD Entertainment titles
• The file format is SMF (Standard MIDI Format)
• SMF file stores standard MIDI messages
along with the appropriate timing information
• SMF files organizes data in chunks, preceded
by ID and size

Standard MIDI Files [2]
• A chunk is a group of related data items
• Each chunk begins with 4 characters ID, saying the type
of the chunk, next 4 bytes forms the size of the chunk. ID
& size forms the chunk header
• MIDI Chunks are of following types
– MThd
– MTrk
• What does the header following header mean?
– 4D 54 68 64 00 00 00 06
• More on MIDI file format
– http://jedi.ks.uiuc.edu/~johns/links/music/midifile.htm

Wavetable Synthesis
• Stores high quality sound samples digitally &
plays it on demand
• Table of sound waveforms which may be looked
up and utilized when needed, hence the name
Wavetable.
• Amount of memory required to store these
samples can be reduced by techniques
collectively called as Wavetable Synthesis
Techniques

To discuss…
 Types of video signals
 Analog Video
 Digital Video

Types of Video Signals
Video Signals can be classified as
1. Composite Video
2. S-Video
3. Component Video

Types - Composite Video
• Used in broadcast TV’s
• Compatible with B/W TV
• Chrominance (color)( I & Q or U
& V) & Luminance (intensity)
signals are mixed into a single
carrier wave, which can be
separated at the receiver end
• Mixing of signals leads
interference & create dot crawl
Male F-Connector, Connecting
co-axial cable with the device
Dot Crawl, due to interference
in composite video

Types - S-Video
• S stands (Super / Separated video)
• Uses 2 wires, one for luminance & the other for chrominance signals
– Less cross talk
• Humans are able to differentiate spatial resolution in gray-
scale images with a much higher acuity than for the color
part of color images.
• As a result, we can send less accurate color information than
must be sent for intensity information

Types - Component Video
• Each primary is sent as a
separate video signal.
– The primaries can either be RGB
or a luminance-chrominance
transformation of them (e.g.,
YIQ, YUV).
– Best color reproduction
– No crosstalk
– Requires more bandwidth and
good synchronization of the three
components

Analog Video
• Represented as a continuous (time varying) signal

Analog Video [2]
Interlaced Scan
With interlaced scan, the
odd and even lines are
displaced in time from
each other.

Analog Video [3]
From T1

NTSC (National Television System Committee)
• It uses the familiar 4:3 aspect ratio (i.e., the ratio of picture
width to its height)
• Uses 525 scan lines per frame at 30 frames per second (fps).
• NTSC follows the interlaced scanning system, and each frame
is divided into two fields, with 262.5 lines/field.
• Thus the horizontal sweep frequency is 525x 29.97 =15,734
lines/sec, so that each line is swept out in 63.6 µ sec
(1/15.734 x 103
sec )
• 63.6 µ sec = 10.9 µ sec for Horizontal retrace + 52.7 µ sec active
line signal
• For the active line signal during which image data is
displayed

NTSC (National Television System Committee) [2]
• 20 lines at the beginning of every
field is for Vertical retrace control
information leaving 485 lines per
frame
• 1/6 of the raster at the left side is
blanked for horizontal retrace and
sync. The non-blanking pixels are
called active pixels.
•Pixels often fall in-between the scan
lines. NTSC TV is only capable of
showing about 340 (visually distinct)
lines

• NTSC video is an analog signal with no fixed
horizontal resolution
• Pixel clock is used to divide each horizontal line of
video into samples. Different video formats provide
different numbers of samples per line
• Uses YIQ Color Model
• Quadrature Modulation is used to combine I & Q to
produce a single chroma signal

• Fsc is 3.58MHz
• Composite signal is formed by
• The available bandwidth is 6MHz, in which the
audio is signal centered at 5.75MHz and the
lower spectrum carries picture information

PAL (Phase Alternating Line)
• Widely used in Western Europe, China, India, and
many other parts of the world.
• Uses 625 scan lines per frame, at 25 frames/second, with
a 4:3 aspect ratio and interlaced fields
• Uses the YUV color model
• Uses an 8 MHz channel and allocates a bandwidth of
5.5 MHz to Y, and 1.8 MHz each to U and V.

Digital Video
• Advantages over analog:
– Direct random access --> good for nonlinear video editing
– No problem for repeated recording
– No need for blanking and sync pulse
• Almost all digital video uses component video

High Definition TV (HDTV)
• The main thrust of HDTV (High Definition TV) is not
to increase the definition in each unit area, but rather to
increase the visual field especially in its width.
– The first generation of HDTV was based on an analog
technology developed by Sony and NHK in Japan in the late
1970s.
– Uncompressed HDTV will demand more than 20 MHz
bandwidth, which will not fit in the current 6 MHz or 8 MHz
channels
– More than one channels even after compression.

High Definition TV (HDTV) [2]
From T1

High Definition TV (HDTV) [3]
• The salient difference between conventional TV and
HDTV:
– HDTV has a much wider aspect ratio of 16:9 instead of 4:3.
– HDTV moves toward progressive (non-interlaced) scan.
The rationale is that interlacing introduces serrated edges to
moving objects and flickers along horizontal edges.

• Thank u

Multimedia Components

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Multimedia Components

Similar to Multimedia Components (20)

More from ArunChokkalingam

More from ArunChokkalingam (16)

Recently uploaded

Recently uploaded (20)

Multimedia Components