Noninvesive Lie Detection

S. Sumriddetchkajorn
TAD2: The First Truly Non-Intrusive Lie Detection
System Deployed in Real Crime Cases
Sarun Sumriddetchkajorn
Armote Somboonkaew
Photonics Technology Laboratory
Intelligent Devices and Systems Research Unit
National Electronics and Computer Technology Center (NECTEC)
National Science and Technology Development Agency (NSTDA)
112 Thailand Science Park, Phahonyothin Rd., Klong 1, Klong Luang
Pathumthani 12120, Thailand
E-mails: sarun.sumriddetchkajorn@nectec.or.th, sarunphotonics@gmail.com

Today Lie Detection System
(Polygraph)
Ref: J. Reicherter, Sci. Am., p. 104, December 1997.
A Century Old Approach

Today Lie Detection System
Analyzes Our Involuntary Reflexes
Invasive Approach
- From a human rights and liberty point of view, if the subject refuses to have these
sensors on his or her body, the polygraph is useless
Problems
- Signals from these sensors can easily be interfered by the movement of the fingertips and
the subject’s body
- This intrusive nature makes subject feel
discomfort during investigation
Affects the accuracy of the system
Measure Breath Rate
- Requires a Blood Pressure Cuff Determine Cardiovascular Activity
- Requires Tubes Strapped
Around Chest and Abdomen
- Requires Electrodes on the
Fingertips
Measure Perspiration

Better Solutions
?

Time, Sept. 2006

IEEE Spectrum, Aug. 2010

Our Motivations
Implement A Non-Invasive Lie
Detection System
via
Thermal Imaging Analysis

Previous Thermal Imaging Analysis for Lie Detection Application
Pavlidis’s Work: analyzes a blood flow (Vs) rate in the periorbital area
Problems
• If Ts ~ TB, an exaggerated response signal occurs, affecting the accuracy for
classifying signals into normal and abnormal levels
• An experimental paradigm may also not respond in the same way as that of
an individual being tested for in the real case
Hence, A More Simpler Approach is Needed
dt
dT
TTdt
dV s
sB
s
2
)(
1


dt
dT
TTdt
dV s
sB
s
2
)(
1

 Ts: Skin Temperature
TB: Human Core Temperature
Ref: I. Pavlidis, J. Levine, and P. Baukol, Proc. IEEE in Computer Vision Beyond the Visible Spectrum: Methods and
Applications, pp. 104-109, 2000.

Our Solution
Blood Flow Rate in the
Periorbital Area
Respiration Pattern
dt
dT
dt
dV ss

• Both blood flow rate and
respiration pattern are
simultaneously analyzed
• More than one parameters are
investigated similar to the current
standard polygraph test
Thus,
+
Around the
Nostril Area min,
max,
s
s
s
T
T
R 
Ref: S. Sumriddetchkajorn and A. Somboonkaew, Patent Pending, 0601002047, May 8, 2006
• It should be tested under the
real crime scenes
• True performance is obtained

- Image segmentations and neural
network
- Multiclass-multifeature fuzzy
connectedness
- Support vector machine
- Maximum likelihood estimation
through Bayesian’s approach
and particle filtering
Previous Techniques
One Important Issue: Face Detection & Tracking
• Good matching and tracking
results
• Limited speed at 3-5 fps thus
not suitable for real time face
detection
• High complicated
mathematical functions
- Normalized cross correlation
(NCC) used in our TAD2 1.0
- Haar-feature approach
(OpenCV)
• Need to learn some reference
objects
• Small change in shape,
orientation and color lead to
false tracking

No need in learning
and matching
with reference patterns
Image
thresholding
Blob
filtering
• Simplicity
• Fast tracking speed
• High tracking efficiency
Ref: S. Sumriddetchkajorn and K. Chaitavon, Infrared Physics and Technology, Vol. 52, pp. 119-123
(2009).
Our Face Detection & Tracking Approach

Temperature thresholding
Morphological processing
Hole filtering
Blob filtering
Coordinate & orientation
of the face
Thermal
image
Assign three
region- of-
interests (ROIs)
New positions
and orientations
of all three ROIs
New positions
and orientations
of all three ROIs

Tracking Improvement
With Previous Normalized Cross Correlation (NCC) used in V.1.0
With New Tracking Algorithm used in V.2.7
Missing Missing Missing
Ref: S. Sumriddetchkajorn and A. Somboonkaew, Proc. AsiaSense, 183-187 (2009.)

Performance Comparisons
(only during data collection)
Nose
Face
detection / tracking
methods
NCC
(V.1.0)
New Algorithm
(V.2.7)
False detection and tracking rate (%)
Left
periorbital
Right
periorbital
Nostril Overall
Tracking
time
(ms)
27.60 1.98 26.94 18.84 14.8
0.70 0.70 1.81 1.07 6.1
17.6 times
for false error rate
17.6 times
for false error rate
2.4 times
for speed
2.4 times
for speed
Much Better Improvement

Another Important Issue: Standard Tests
Embedded with Three Standard Tests
10 Questions: Qs 1, 2, 4, 7  Irrelevant Qs
Qs 6, 10  Control Qs
Qs 3, 5, 8, 9  Relevant Qs
• Modified General Question Test (MGQT): Test Time  5-6 minutes
10 Questions: Qs 1, 2  Irrelevant Qs
Qs 4, 6, 9  Control Qs
Qs 5, 7, 10  Relevant Qs
Qs 3, 8  Specific (Relevant) Issues
• Modified Zone Comparison Test (MZCT): Test Time  5-6 minutes
Multiple Questions: Qs 1, 2, 5, 8, 11, …  Irrelevant Qs
Qs 4, 7, 10, 13, …  Control Qs
Qs 3, 6, 9, 12, …  Relevant Qs
• Irrelevant & Relevant Test (I&R): Test Time depends on number of questions

Our Baseline Classification Criteria
An Average Response
Associated with Qs 1 & 2
Baseline Signal for
Relevant Q3
Response from Q4 Baseline Signal for
Qs 5 & 6
Response from Q7 Baseline Signal for Qs
8, 9 & 10
• MGQT
An Average Response
Associated with Qs 1 & 2
Baseline Signal for All
Remaining Qs
• MZCT
Because each issue begins with an irrelevant question and follows
by a relevant and a control questions.
Baseline Signal for Relevant &
Control Qs for That Issue
• I&R
Irrelevant Q for That Issue

Our Scoring Criteria
Comparison between Abnormal
Responses from the Relevant and
the Control Questions
Comparison between Abnormal
Responses from the Relevant and
the Control Questions
Scores: -1 Response from Relevant Q > Control Q
+1 Response from Relevant Q < Control Q
0 Response from Relevant Q = Control Q
• MGQT
• MZCT
Zone 1: contains Qs 4, 5, 6
Zone 2: contains Qs 6, 7, 8
Zone 3: contains Qs 9, 10
• I&R
A Total Score of > -2 Non-Deceptive Indication (NDI)
Otherwise Deceptive Indication (DI)

Our Thermal Image Analyzer for Deception Detection (TAD2)
Version 2.7Profile Section
MGQT, MZCT, I&R
FLIR A-40M:
FireWire Interface
320x240 Pixels
7.5-13 um Waveband
0.08 mK NETD
60 fps
Thermal Imaging Camera
With our face tracking
approach: 20 fps
Computer:
Intel Core2Duo T8300
2.4 GHz, 3MB RAM,
and a video processor
GeForce 9300M G

Question Section (MGQT)

Question Section (MZCT)

Question Section (I&R)

Test Section
Left Periorbital
Right Periorbital
Nostril
Envt.
Q Started Q Stop
Subject responses

Analysis Section (for Current Chart)

Analysis Section (uses 3 Charts)

Far-Infrared
Camera
Current Polygraph
System
Chair for
Subject
Far-Infrared
Camera
Chair for
Subject Processing
Unit
Chair for
Officer
Our Field Test Results
• Our Examination Scenario
A Tool to prove that you are
innocent NOT telling us a lie

Nostril Region
Periorbital Area
Nostril Region
Periorbital Area
Exhale
• Examples of Thermal Images for the Subject “A
Inhale
Periorbital Area: 15x15 Pixels
Nostril Area: 40x15 Pixels
Note: All 14 subjects (32 tests) signed the permission to perform the test.

0
0.1
0.2
0.3
0.4
0 1 2 3 4 5 6 7 8 9 10 11 12
Time (seconds)
RelativeBloodVelocity
• Example of the Relative Blood Velocity in the Periorbital Area
Linear Fit
Raw Data
Q A
Find Slope Ratio (Q/A)
Shows Abnormal Response
in this Question
Periorbital Areas
Questioning Section
Answering Section
Subject “A” in the Question 5 Section of MGQT

1.01
1.02
1.03
1.04
1.05
2 3 4 5 6 7 8 9 10 11 12
Ts,max/Ts,min
• Example of Our Measured Tmax/Tmin Around the Norstril Area
Subject “A” in the Question 5 Section
- Pattern Enhancement via Its Derivative
- Number of Peaks, Valleys
- Period of Peak, Valley
Subject’s Response
Time (seconds)
-2.5
-1.5
-0.5
0.5
1.5
2.5
2 3 4 5 6 7 8 9 10 11 12
Time (seconds)
d(Ts,max/Ts,min)
dt

0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
1 2 3 4 5 6 7 8 9 10
Question
|Slope|
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1 2 3 4 5 6 7 8 9 10
Question
|Slope|
Valley
AmplitudeBaseline
Valley
Period
Slope Value
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10
Question
|SlopeRatio|
Baseline
Slope Ratio
Subject “A” First Examination Chart of MGQT
• Examples of Our Calculated Slope Ratios and Slope Values
From Periorbital Areas: 0-1-1-1 = -3
Scores
From Nostril Area: +1+1+1+1= 4 (Amplitude)
-1-1+0+0 = -2 (Period)

For 7 Subjects under MGQT
• Only the Periorbital Signal: 42.9% Accuracy
• Only the Data from Nostril Area: 57.1% Accuracy
• Combination of Both Signals: 85.7% Accuracy
Periorbital
Nostril
Combined

Conclusion
• A Non-Invasive Lie Detection System is Developed (MGQT, MZCT, I&R) and
Ready to be Tested More in the Field
2D Data from Periorbital Areas
2D Data from Nostril Area
Relative Blood Flow Velocity
Respiration Pattern
Our Baseline Classification &
Scoring Criteria
Simple Tracking Algorithm
+
• Our Field Test Results from 14 Subjects with 32 Tests show:
A Successful Classification Rate of 84%
• Our Future Work:
- Increase the Number of Subjects and Number of Tests
- Develop a More Intelligent Baseline and Scoring Adjustment Method
- User Friendlier Interface

References
1. S. Sumriddetchkajorn and A. Somboonkaew, “A non invasive method for analyzing psychophysiological
data,” Thailand Patent Application, 0601002047, May 8, 2006.
2. S. Sumriddetchkajorn and A. Somboonkaew, “A non invasive deceptive detection system and methods for
assigning regions of interest and for distinguishing deceptive levels,” Thailand Patent Application,
0701000585, Feb. 9, 2007.
3. S. Sumriddetchkajorn, A. Somboonkaew, T. Sodsong, I. Promduang, and N. Sumriddetchkajorn, “A field test
study of our non-invasive thermal image analyzer for deceptive detection,” Proc. SPIE 6633, 66331F
(2007).
4. S. Sumriddetchkajorn and A. Somboonkaew, “A study of normalized cross correlation pattern matching
algorithm in thermal imagery,” Proc. ECTI 2, 1125-1127 (2007).
5. S. Sumriddetchkajorn and A. Somboonkaew, “A field test study of our non-invasive thermal image
analyzer for deceptive detection,” Proc. SPIE 6633, 66331F (2007).
6. S. Sumriddetchkajorn and A. Somboonkaew, “Face detection in thermal imagery using an Open Source
Computer Vision Library,” Proc. SPIE 7299, 729906 (2009).
7. S. Sumriddetchkajorn and A. Somboonkaew, “Tracking improvement in our far infrared-based non-invasive
lie sensing system,” Proc. AsiaSense, 183-187 (2009).
8. S. Sumriddetchkajorn and K. Chaitavon, “Field test studies of our infrared-based human temperature
screening system embedded with a parallel measurement approach,” Infra.Phys. and Technol., Vol. 52, pp.
119-123 (2009).

Noninvesive Lie Detection

More Related Content

Similar to Noninvesive Lie Detection

More from Sarun Sumriddetchkajorn

Recently uploaded

Noninvesive Lie Detection