Efficient Methods of Multimodal Biometric
Speech and Face Recognition
This presentation proposes the efficient methods in multimodal biometric i.e. fingerprint,
Multimodal system is developed through fusion of fingerprint, speech and face recognition.
The proposed system is designed for applications where the training database contains a face,
fingerprint images and voice data .
The multimodal biometric security system may be used in various application areas such as, for
authentication number of employees working in offices, in military applications and also in all
possible security applications.
I. Introduction of Biometrics
IV. Multimodal Biometric Systems
V. Biometric Techniques
VI. Modules in Multimodal Biometric Systems
VII. Levels of Fusion
The term biometric is usually associated with the use of unique physiological characteristics to
identify an individual. One of the applications which most people associate with biometrics is
It is an automated method of recognizing a person based on a physiological or behavioral
characteristic such as face ,fingerprints, hand geometry, handwriting, iris, voice etc.
As password or PIN can lost or forgotten, biometrics cannot be forgotten or lost and requires
physical presence of the person to be authenticated.
Thus personal authentication systems using biometrics are more reliable, convenient and
efficient than the traditional identification methods.
However, even the best biometric traits till date are facing numerous problems biometric
authentication systems generally suffer from enrolment problems due to non-universal
biometric traits, susceptibility to biometric spoofing or insufficient accuracy caused by noisy
data acquisition in certain environments.
Thus to overcome these problems ,multimodal biometric system can be used to reduce/remove
the limitations of unimodal systems.
As the level of security breaches and transaction fraud increases, the need for highly secure
identification and personal verification technologies is becoming apparent.
In recent years, biometrics authentication has seen considerable improvements in reliability
and accuracy, with some of the traits offering good performance.
The reason to combine different modalities is to improve recognition rate.
The aim of multimodal biometrics is to reduce one or more of the following:
FAR(False Acceptance Rate) : It is a measure of the percent of invalid inputs that are
FRR(False Reject Rate) : It is a measure of the percent of valid inputs that are incorrectly
CER(Crossover Error Rate) : The rate at which both the accept and reject errors are equal.
- a lower value of the CER is more accurate for Biometric System.
MULTIMODAL BIOMETRIC SYSTEMS
Multimodal biometric systems are those that utilize more than one physiological or behavioural
characteristic for enrolment , verification, or identification.
In applications such as border entry/exit, access control, civil identification, and network
security, multi-modal biometric systems are looked to as a means of reducing false non-match
and false match rates, providing a secondary means of enrolment , verification, and
A multi biometric system uses multiple sensors for data acquisition. This allows capturing
multiple samples of a single biometric trait (called multi-sample biometrics) and/or samples of
multiple biometric traits (called multi source or multimodal biometrics).
A unimodal biometric system consists of three major modules: sensor module, feature
extraction module and matching module. The performance of a biometric system is largely
affected by the reliability of the sensor used and the degrees of freedom offered by the features
extracted from the sensed signal.
Further, if the biometric trait being sensed or measured is noisy (a fingerprint with a scar or a
voice altered by a cold, for example), the resultant matching score computed by the matching
module may not be reliable. This problem can be solved by installing multiple sensors that
capture different biometric traits. Such systems, known as multimodal biometric systems , are
expected to be more reliable due to the presence of multiple pieces of evidence.
It is the oldest and most widely used method.
It needs a fingerprint reader.
Registered points are located and compared.
Optical sensors are used for scanning purpose.
It can be used for many applications like pc login security, voting system, attendance system
Uses the ridge endings and bifurcation's on a persons finger to plot points known as Minutiae
The number and locations of the minutiae vary from finger to finger in any particular person,
and from person to person for any particular finger
Finger Image + Minutiae Minutiae
FACE RECOGNITION TECHNOLOGY:
Face Recognition is a biometric technique for automatic identification or verification of a
person from a digital image.
These include the position/size/shape of the eyes, nose, cheekbones and jaw line.
SPEECH RECOGNITION TECHNOLOGY:
It is a biometric process of validating a user's claimed identity using characteristics
extracted from their voices.
It uses the pitch, pattern, tone, frequency, rhythm of speech for identification purposes.
During the enrollment phase, the spoken words are converted from analog to digital
format, and the distinctive vocal characteristics such as pitch, frequency, and tone, are
extracted, and a speaker model is established.
A template is then generated and stored for future comparisons.
MODULES IN MULTIMODEL BIOMETRIC SYSTEM
A common biometric system mainly involves the following major modules-
1. Sensor Module
At sensor module a suitable user interface incorporating the biometric sensor or scanner is needed
to measure or record the raw biometric data of the user. This raw biometric data is captured and
then it is transferred to the next module for feature extraction. The design of the sensor module
influences the various factors like cost and size.
2. Feature Extraction Module
At feature extraction module the quality of the acquired biometric data from the sensor is assessed
initially for further processing. Thus generating a synoptic but indicative digital representation of
the underlying traits or modalities. After extracting the features it is given as input to the matching
module for further comparison.
3. Matching Module
The extracted features when compared with the templates in the database generate a match score.
This match score may be controlled by the quality of the given biometric data. The matching
module also condensed a decision making module in which the generated match score is used to
validate the claimed identity.
4. Decision making module
Decision making module identifies whether the user is a genuine user or an impostor based on the
match scores. These are used to either validate the identity of a person or provides a ranking of the
enrolled identities for identifying an individual.
A simple block diagram for multi-modal biometric system is shown in Fig
MODE OF OPERATION
The two major mode of operation in multi-modal biometric systems are
In serial mode of operation, multiple sources of information is not acquired simultaneously,
that is the user goes through stage by stage authentication process.
Thus the recognition time is improved in serial mode as decision is made before getting all the
In case of parallel mode of operation, recognition is performed by acquiring multiple sources o
This will reduce the efficiency of the system and in turn cause inconvenience to the user.
Study reveals that combined use of both modes may result a system which provides high
efficiency and user convenience.
LEVELS OF FUSION
By employing the information available in any of the modules like sensor level, feature
extraction level, matching level and Decision making level, fusion can be developed in multi-
modal biometric system like sensor level fusion, feature level fusion, matching score level
fusion and decision level fusion.
The different biometric identifier used in the multimodal biometric system, their information
from the individual identifier is taken together and can be fused at different levels of fusion
such as fusion at sensor level , fusion at feature level, fusion at matching score level and the
fusion at decision level
The following figure shows the fusion at Sensor level which involves combining raw data from
various sensors and this fusion can be appropriate for multi-sample and multi-sensor systems
Fig. Sensor level
Feature level fusion shown in Fig refers to combining the different feature sets extracted from
multiple biometric modalities into a single feature vector.
Fig. Feature level fusion
Matching score level fusion shown in Fig refers to the combination of similarity scores
provided by a matching module for each input features and template biometric feature
vectors in the database.
Fig. Matching score level fusion
In decision level fusion as shown in Fig, the information integration occurs when each
biometric system makes an independent decision about the identity of the user or verifies the
Fig: Decision level fusion
A multimodal biometrics system helps us to reduce:
• False accept rate (FAR)
• False reject rate (FRR)
• Failure to enrol rate (FTE)
It also increases:
• Sensor cost
• Enrolment time
• Transit times
• Need for a prior knowledge/data
• System development and complexity
Though there are many multi-modal biometric systems in practice for authentication of a
person, selection of appropriate modal, choice of optimal fusion level and redundancy in the
extracted features are still some of the shortcomings faced in the design of multi-modal
biometric system that needs to be addressed.
The different approaches that are possible in multi-modal biometric systems, the suitable
fusion levels, and the integration strategies that can be chosen to consolidate information
The combination of more than one biometrics can apply to enhance the security.
Performance and the advanced security level made the multi-modal biometric systems
popular in these days and has great scope in future.
1. L. Hong, A. Jain & S. Kumar, Can multimode biometric Improve performance,
Proceedings of Auto ID 99, pp. 59-64, 1999.
2. A. Ross & A. K. Jain, Information Fusion in Biometrics, Pattern Recognition
Letters, 24 (13), pp. 2115-2125, 2003.
3. A.S. & A. A. Raza , Combined Classifier for Invariant Face Recognition,
Pattern Analysis and Applications, 3(4), pp. 289-302, 2000
4. [A. Ross, A. K. Jain & J.A. Riesman, Hybrid fingerprint matcher, Pattern
Recognition, 36, pp. 1661–1673, 2003
5. W. T. Tan & A. K. Jai , Combining Face and Iris Biometrics for Identity
Verification, Proceedings of Fourth International Conference on AVBPA, pp.