Combining Optical Brain Imaging and Physiological Signals to Study Cognitive Function

Combining Optical Brain Imaging and
Physiological Signals to Study Cognitive Function
Experts discuss the fundamentals of fNIRS and
present new research capabilities enabled through
the integration of optical brain imaging technology
and physiological recording systems.

InsideScientific is an online educational environment
designed for life science researchers. Our goal is to aid in
the sharing and distribution of scientific information
regarding innovative technologies, protocols, research tools
and laboratory services.

Continuous Wave Functional Near Infrared (fNIR) Spectroscopy Imaging Systems
www.fnirdevices.com
3© fNIR Devices LLC. 2016
Optical Brain Imaging:
Physiological and Physical
Principles
Hasan Ayaz, PhD
Associate Research Professor, Drexel University

www.fnirdevices.com
Functional Near-Infrared Monitor [fNIR]
>> Cortical activation-related to hemodynamic changes in
natural environments and diverse field conditions
Brain Activity Monitor

www.fnirdevices.com
Physiological Principles of fNIR:
• Neurons consume energy (glucose) when activated
• Oxygen is required to metabolize the glucose
• As clusters of neurons are activated, there is an
increased need for oxygen in that area
• Oxygen is transported to neural tissue via oxy-
hemoglobin in the blood
Oxy-Hb and deoxy-Hb are correlates of brain activity
through oxygen consumption by neurons
• The oxygen exchange occurs in the capillary beds
• As oxy-hemoglobin gives up oxygen to the neural
tissue, it is transformed into deoxygenated
hemoglobin
David J. Heeger & David Ress. (2002) What does fMRI tell us about neuronal activity?
Nature Reviews Neuroscience 3, 142-151
Neural Activity & Hemodynamic Response

www.fnirdevices.com
Physical Principles of fNIR: Photon Migration in Tissue

www.fnirdevices.com
Physical Principles of fNIR: Photon Migration in Tissue
H Obrig , R Wenzel, M Kohl, S Horst, P Wobst, J Steinbrink, F Thomas, A Villringer. Near-infrared
spectroscopy: does it function in functional activation studies of the adult brain? International
Journal of Psychophysiology 35(2-3):125-142, 2000
Photons that enter the tissue
undergo two types of
interaction:
1. Scattering
(cell membranes)
2. Absorption
(Hb, HbO2, water)

www.fnirdevices.com
Modern Portable System (Desktop)
2010…
fNIR Instrumentation

www.fnirdevices.com
2010…

www.fnirdevices.com
2015…

www.fnirdevices.com
McKendrick, R., Ayaz, H., Olmstead, R., & Parasuraman, R.
(2014). Enhancing dual-task performance with verbal and
spatial working memory training: Continuous monitoring of
cerebral hemodynamics with NIRS. Neuroimage, 85, Part
3(0), 1014-1026.
Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A.,
McKendrick, R., & Parasuraman, R. (2013). Continuous
monitoring of brain dynamics with functional near infrared
spectroscopy as a tool for neuroergonomic research:
Empirical examples and a technological development.
Frontiers in Human Neuroscience, 7, 1-13.
Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B.,
& Onaral, B. (2012). Optical brain monitoring for operator
training and mental workload assessment. Neuroimage,
59(1), 36-47.
Select References
2015…

www.fnirdevices.com
• Cell-phone sized,
• Battery operated,
• Fast-setup/no gel
required
• Suitable for field
applications
Modern Wireless system (Pocket)

www.fnirdevices.com
• Cell-phone sized,
• Battery operated,
• Fast-setup/no gel
required
• Suitable for field
applications
Modern Wireless system (Pocket)
fNIR Instrumentation Select References
Mckendrick, R., Parasuraman, R., Murtza, R., Formwalt, A.,
Baccus, W., Paczynski, M., & Ayaz, H. (2016). Into The Wild:
Neuroergonomic Differentiation of Hand-Held and
Augmented Reality Wearable Displays During Outdoor
Navigation with Functional Near Infrared Spectroscopy.
Frontiers in Human Neuroscience, 10.
McKendrick, R., Parasuraman, R., & Ayaz, H. (2015).
Wearable functional Near Infrared Spectroscopy (fNIRS)
and transcranial Direct Current Stimulation (tDCS):
Expanding Vistas for Neurocognitive Augmentation.
Frontiers in Systems Neuroscience, 9(27).
Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A.,
McKendrick, R., & Parasuraman, R. (2013). Continuous
monitoring of brain dynamics with functional near infrared
spectroscopy as a tool for neuroergonomic research:
Empirical examples and a technological development.
Frontiers in Human Neuroscience, 7, 1-13.

www.fnirdevices.com
fNIR Signal Measurement
Absorption spectrum of main
chromophores in tissue:
• Low absorption between
700-900nm provides an
optical window to the tissue

www.fnirdevices.com
Modified Beer Lambert Law
(MBLL) allows calculation of
oxygenation changes from
light intensity measures.
Modified Beer Lambert Law
Can be solved for concentration changes for non-
singular F matrix

www.fnirdevices.com
MBLL is repeated
continuously to generate
time-series each for oxy-
Hb, deox-Hb, total-Hb,
and oxygenation.

www.fnirdevices.com
• Raw fNIR signals are light intensity values at
detectors
• The physiologically irrelevant data (such as
respiration and heart pulsation effects) and
equipment noise, etc. are first eliminated
from the raw fNIR measurements
• Saturation, motion artifact and other noise
needs to be eliminated (or excluded from
analysis
Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., & Onaral, B. (2011).
Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in
Spatial Navigation. J Vis Exp(56), e3443. doi: 10.3791/3443
fNIR Signal Processing

www.fnirdevices.com
Motion Artifact Detection
& Removal
• Low-pass / Band-pass Filters
• Wavelet Analysis
• Independent Component Analysis
(ICA)
• Principle Component Analysis (PCA)
• Coefficient of Variance related
(SMAR, etc.)
• Optimal Filtering
(Adaptive; Wiener; Kalman)

www.fnirdevices.com
& Removal
(ICA)
(SMAR, etc.)

www.fnirdevices.com
Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., &
Onaral, B. (2011). Using MazeSuite and Functional Near Infrared
Spectroscopy to Study Learning in Spatial Navigation. J Vis
Exp(56), e3443. doi: 10.3791/3443
Izzetoglu, M., Chitrapu, P., Bunce, S., & Onaral, B. (2010). Motion
artifact cancellation in NIR spectroscopy using discrete Kalman
filtering. Biomed Eng Online, 9(1), 16.
Ayaz, H., Izzetoglu, M., Shewokis, P. A., & Onaral, B. (2010).
Sliding-window Motion Artifact Rejection for Functional Near-
Infrared Spectroscopy. Conf Proc IEEE Eng Med Biol Soc, 6567-
6570.
Izzetoglu, M., Devaraj, A., Bunce, S., & Onaral, B. (2005). Motion
artifact cancellation in NIR spectroscopy using Wiener filtering.
IEEE Trans Biomed Eng, 52(5), 934-938.
& Removal
(ICA)
(SMAR, etc.)
Select References

www.fnirdevices.com
Raw light intensity HbO/HbR Changes
Filtered HbO/HbRTask/condition blocks (time synced markers)
Mean, Median, Stddev, Time-Peak Range, Min, Max
Preprocessing
(And. LP, detrending, etc.)
Beer–Lambert Law
Subject X, Ch1
0 100 200 300 400 500
-1.5
-1
-0.5
0
HbR/HbO Raw
HbR
HbO
0 100 200 300 400 500
-0.2
-0.1
0
0.1
0.2
HbR/HbO filtred (10Hz)
HbR
HbO
0 100 200 300 400 500
-0.2
-0.1
0
0.1
0.2
HbR/HbO filtred (1Hz)
HbR
HbO Feature Extraction
0 100 200 300 400 500
2
2.2
2.4
2.6
2.8
695nm
830nm
730nm
850nm
3000
2500
2000

www.fnirdevices.com
To learn more about fNIR Devices technology, visit www.fnirdevices.com

www.fnirdevices.com
Optical Brain Imaging:
Field Applications
Kurtulus Izzetoglu, PhD
Associate Research Professor, Drexel University

www.fnirdevices.com
Non-invasive ‘anesthesia care’
monitoring to detect depth of
anesthesia and over sedation
fNIR to Monitor Depth
of Anesthesia

www.fnirdevices.com
deoxy-Hb displayed very slow
rate of change in deep
anesthesia,
whereas this rate of change is
significantly increased when
the patient emerges to
wakefulness.
of Anesthesia

www.fnirdevices.com
deoxy-Hb Changes in Voxel 12
0.000
0.500
1.000
1.500
2.000
2.500
3.000
3.500
X-4min X-3min X-2min X-1min X Y-
4min
Y-
3min
Y-
2min
Y-
1min
Y
Time
microMolar
Wound Closure Eye Opening
………..
3.4% {
48.8% {
Deep Anesthesia Light Anesthesia
deoxy-Hb displayed very slow
rate of change in deep
anesthesia,
whereas this rate of change is
significantly increased when
the patient emerges to
wakefulness.
of Anesthesia
Izzetoglu K, et al (2011). The Evolution of Field Deployable fNIR Spectroscopy
From Bench to Clinical Settings. J of Innovative Optical Health Sciences

www.fnirdevices.com
fNIR for assessment of
cognitive impairment
and recovery following
traumatic brain injury
(TBI)
Healthy Subject TBI Patient
Neurorehabilitation
Merzagora A.C., Butti M. et al., Clinical
Neurophysiology , 2009

www.fnirdevices.com
Cz
-300 700
-20
20
Time (ms)
Amplitude(V)
Pz
Non-Target
Target
-300 700
-20
20
Time (ms)
Amplitude(V)
-0.015
0.01
12
Time (s)
oxygenation(M)
fNIR EEG
fNIR & EEG combined to
study and assess the
cognitive impairments
of the patients in their
everyday life activities.
Neurorehabilitation
Merzagora A.C., Butti M. et al., Clinical
Neurophysiology , 2009

www.fnirdevices.com
Holtzer, R, et al (2015) Online fronto-cortical control of simple and attention-demanding locomotion in humans. Neuroimage
Assessment of cortical control of
locomotion, using fNIR, was highly
reproducible.
Oxygenation levels were increased
and maintained in attention-
demanding locomotion.
Definitive role for the PFC in higher-
order control of locomotion.
fNIR to determine online cortical
control of locomotion in humans.
Cognitive Aging

www.fnirdevices.com
Air Traffic Controller
Cognitive Workload
Monitor
UAV Ground Operator
Training & Workload
Monitor
Pilot
Expertise Development
Monitor
fNIR can “bring more
of brain on task”*
…enable human in the
loop studies
*DARPA: Dylan Schmorrow, Ph.D. LCDR, MSC, US Navy,
Program Manager, ITO
Human
Performance
Assessment
Safe & Effective Piloting

www.fnirdevices.com
FAA Next Generation Air Transportation System
(NextGen) Study
ATC Part-Task
Communication type: Voice and Data
Workload manipulated by number of
aircrafts in the sector: n= 6,12,18
Ayaz, H., et al., Optical brain monitoring for operator training
and mental workload assessment , NeuroImage (2012),
Air Traffic Controller Workload

www.fnirdevices.com
FAA Next Generation Air Transportation System
(NextGen) Study
ATC Part-Task
Communication type: Voice and Data
Workload manipulated by number of
aircrafts in the sector: n= 6,12,18
Air Traffic Controller Workload

www.fnirdevices.com
Oden, K., et al (2015) Empirical Support for Brain-Based Assessment in Simulation-Based
Training
objective measure of expertise level
to validate adequate training
real-time monitor of mental
workload
Pilot Expertise-Workload
Assessment

www.fnirdevices.com
workload
Assessment

www.fnirdevices.com
• Amenable to integration with other sensors
– multi modality monitoring
• Shorter preparation time – easy to calibrate and baseline
• Less intrusive – critical in field studies
• Easy to interpret and relate to task-dependent analyses – direct measure
of the activity
• Lends itself to easy engineering/customization – important to support
‘translational’ research
fNIR For Natural Environments & Field Conditions

Demonstration of
fnirSoft Software
Click To
Access Video

How To Integrate
fNIR Data With Other
Physiological Signals
Frazer Findlay CEO, BIOPAC Systems, Inc.

BIOPAC—Inspiring people and enabling discovery about life
© BIOPAC Systems, Inc. 2016
www.biopac.com
40
Over 97% of top universities run BIOPAC Systems
THE WORLD DISCOVERS
WITH BIOPAC
Solutions for life science research and education

www.biopac.com
41
Combining fNIR with other Physiological Signals
BioNomadix
Wireless
Physiological
Data
Mobita System
32-Channel EEG /
Biopotentials
B-Alert System
9-Channels of EEG
and Cognitive
States Metrics
MP36R System
4-Universal
Channels
MP160 System
16-Channels of
Tethered or
Wireless Signals
To learn more about BIOPAC solutions for signal integration, visit www.biopac.com

www.biopac.com
42
Stimulus Presentation with
Triggers for Synchronization
SuperLab® - E-Prime®
Presentation®
Virtual Reality
BIOPAC VR Solutions
offer real-time feedback
loops and event marking

www.biopac.com
43
Stationary Eye Tracking
with Visual Stimulation
Unrestrained Mobile
Eye Tracking

www.biopac.com
44
Facial Expressions and Video Synchronization

www.biopac.com
45
fNIR Imager 1000
Continuous wave NIR
spectroscopy control box
and 16 channel sensor
STP100C
Trigger BNC;
fNIR trigger via
BNC
MP160 System
16-Channels
Wired or
Wireless Unit
Hardware Communication
Front
FrontBack

www.biopac.com
46
Front
FrontBack
Hardware Communication
MP160 receives
select trigger
information
from fNIR
control unit…
 START
 STOP
 BASELINE BEGIN
 BASELINE END
 RECORD

www.biopac.com
47
Hardware Communication with Visual Stimulus
Visual stimulus is
delivered to subject at
a stationary monitor
or by VR...
Front
FrontBack

www.biopac.com
48
Hardware Communication with Visual Stimulus
Visual stimulus is
delivered to subject at
a stationary monitor
or by VR...
Triggers sent
to the MP160
System and
fNIR computer

Demonstration of
AcqKnowledge Software
Click To
Access Video

www.biopac.com
50
• Combine fNIR with a variety physiological monitoring
systems for– ECG, EEG, EMG, EOG, EDA, Respiration, etc.
• Interface with stimulus presentation systems such as E-Prime,
SuperLab, Presentation and BIOPAC Virtual Reality systems
• Monitor subject eye position, gaze path and facial expressions
• Maintain synchronization across devices
Summary

Thank You!
Kurtulus Izzetoglu, PhD Frazer FindlayHasan Ayaz, PhD
If you have questions for the presenters
please contact them by email.
For additional information on the solutions
presented in this webinar please visit:
http://www.biopac.com
http://www.fnirdevices.com Hasan Ayaz, PhD ayaz@drexel.edu
Kurtulus Izzetoglu, PhD ki25@drexel.edu
Frazer Findlay info@biopac.comClick To View Full Recording

Combining Optical Brain Imaging and Physiological Signals to Study Cognitive Function

More Related Content

What's hot

Similar to Combining Optical Brain Imaging and Physiological Signals to Study Cognitive Function

More from InsideScientific

Recently uploaded

Combining Optical Brain Imaging and Physiological Signals to Study Cognitive Function