1. SBIR Phase I Final Progress Report
Grant Number: 1R43NS69310-1A1
Project Title: BOLD-Related EEG Signal Estimation Software
Grantee Organization: Source Signal Imaging, Inc.
Project Period: 3/15/2011 – 9/30/2012
PI: Mark E. Pflieger
Goal, requirements, and impact. The scientific-technical goal of this SBIR project is to develop a cohesive
set of new algorithms for analyzing simultaneous EEG-fMRI experimental data in order to detect and
estimate EEG space-time-frequency signals that are coupled specifically with BOLD signals of interest,
such as BOLD signals associated with a brain region of interest. Commercially, BOLD-Related EEG (BRE)
signal detection and estimation algorithms shall be provided on the EMSE
software platform to cognitive,
systems, and translational neuroscientists, who shall be supported in their research efforts by Source Signal
Imaging (SSI) application engineers (see Attachment [1]). The BRE algorithms shall respect general
principles that constrain biophysical models (such as hemodynamic response functions and neuroelectric
volume conductors) without, however, depending on specific modeling details. Rather, the computational
theory shall serve to promote discoveries of empirical relationships in EEG-fMRI experimental data. We
aspire to provide and support data-driven BRE algorithms that will help neuroscientists discover new
relationships between the vast realms of neuroelectric and hemodynamic phenomena—new relationships
that can promote a deeper understanding of neuroimaging of the adaptive and maladaptive human brain.
Assessment of progress and current status. We have made substantial progress with algorithm theory and
methods (two major revisions), software technologies for bridging EEG and fMRI preprocessing tools,
engagement with EEG-fMRI experimental data, and computational infrastructure. However, algorithmic
methods in the third generation (second major revision) are implemented partially, and analysis of the
experimental data is incomplete. Therefore, this project needs—and we have been giving it—more effort
beyond Phase I in order to demonstrate feasibility as required for Phase II. We are encouraged by
incremental progress with implementation and application of the current generation of algorithms.
Algorithm theory and methods: First generation. The Phase I application described our first generation of
algorithms for BRE (BOLD-related EEG) signal detection and estimation. Key features of our first approach
have been maintained throughout this project: (i) the researcher specifies a BOLD signal of interest (SOI) in
the hemodynamic domain of phenomena comprising frequencies ~0.01-0.10 Hz; (ii) specificity of the BOLD
SOI is determined by its contrast with BOLD signal not of interest (SnOI) in the hemodynamic domain; (iii)
the BOLD signal subspace potentially relevant to quasi-statically generated EEG signals is constrained by
gray matter voxels located principally in cerebral cortex; (iv) the EEG signal is analyzed spatially one
frequency at a time across a spectrum ranging ~0.10-100 Hz; (v) the applied math problem is to find a
temporal filter (hemodynamic response function = HRF) and a spatial filter (beamformer-like estimator) such
that a flexible nonlinear function of spatially filtered EEG power, convolved with the HRF, is maximally
coupled with SOI-specific BOLD signal; (vi) a nonparametric statistical test is needed to detect BRE signal
and to assess the strength of EEG-BOLD coupling; (vii) fast BRE signal in the neuroelectric time domain is
estimated by synthesizing multi-frequency spatial filters across the EEG spectrum; and (viii) a composite
HRF is estimated in the hemodynamic time domain by via its correspondence with fast BRE signal. We
learned that our first approach was limited significantly in practice due to two highly general yet “brute force”
nonlinear optimization methods: specification of the objective function as conditional mutual information,
and numerical optimization by conjugate gradient ascent method. The highly general character of these
nonlinear methods comes at a high computational cost; in particular, up-front over-reduction of the BOLD
and EEG signal spaces are required in practice. Consequently, too many space-time-frequency degrees of
freedom available in each domain (hemodynamic and neuroelectric) are lost before the analysis can
proceed. A second related issue is that—although we seek model-independent characterizations of
temporal and spatial filters in (v) above—our methods had to be initialized using a canonical hemodynamic
response model used in the fMRI field, and a standard volume conductor and source space model used in

2. the EEG field. However, a priori models may introduce biases that can hinder discoveries of empirical
neurovascular coupling relationships to be found in concurrent EEG-fMRI data. Finally, a third issue we
encountered through engagement with the data is that—in spite of the relative high spatial resolution of
fMRI—regional specificity of the ongoing BOLD signal can be difficult to achieve in practice due to high
inter-regional correlations with a region of interest (ROI). In fact, whole-brain BOLD signal correlations are
the well-known basis for studies of resting and task-related functional connectivity networks using fMRI.
However, this presents a problem with the core-shell approach to regional specificity proposed in our Phase
I application, because a “core” ROI may be more highly correlated with distant regions than with its
immediately surrounding “shell”; yet distant regions also may contribute to the generation of EEG signals in
synchrony with ROI generation.
Algorithm theory and methods: Second generation. The second generation of algorithms (our first major
revision) was based on the following insight: since BOLD SOI is coupled with some (generally nonlinear)
function of EEG power, and EEG power is a second order statistic, and second order statistics in EEG are
generally characterized by a matrix whose elements are sums of cross-products between pairs of EEG
channels, we can initiate the analysis process by maximizing BOLD SOI coupling for each EEG channel
pair separately. When the separately optimized couplings are assembled into a full inter-channel coupling
matrix, that matrix can be analyzed for its principal spatial components, which are closely related to the
desired EEG spatial filter. Thus, the second approach estimated the EEG spatial filter empirically from the
start, without model-based initialization or multivariate nonlinear gradient ascent. On the other hand, we still
needed a canonical HRF model for the coupling analysis; and we still conceived of the BOLD SOI in terms
of an associated ROI. This hybrid approach was an intermediate step that lead to our current third
generation of algorithms (second major revision).
Algorithm theory and methods: Third generation. Our current algorithms (second major revision) are
described in detail in Attachment [2], a manuscript in preparation entitled “Data-driven detection and
estimation of BOLD-related EEG signal: computational theory”. The new approach begins (1) by
characterizing the total BOLD signal space empirically, based on a temporal component analysis such as
ICA, as used typically in fMRI functional connectivity studies (see Attachment [3]). The components define
basis functions that span the total BOLD signal space, and the BOLD SOI specified by the researcher must
“stand out” against this total background. In particular, any ROI-based SOI must be specified as a linear
combination of components so that ROI correlations with distant regions are taken into account and
canceled. Thus, although voxel space is still important for SOI specification, the total BOLD signal space as
defined by components is primary. Next (2), a linear technique in the hemodynamic frequency domain is
used to maximize nonlinear dynamic specificity of the SOI in the total BOLD signal space. This technique
works by forming a matrix that characterizes interactions between low frequencies (~0.01-0.10 Hz) in the
BOLD signal space. Interactions between different frequencies occur only for nonlinear systems; however,
the matrix is analyzed using standard linear methods. Next (3), the insight described above for the second
generation approach is extended as follows. Second-order inter-channel statistics in the neuroelectric
frequency domain are estimated separately; however, a static nonlinear function (designated by φ) is
applied to the matrix as a whole. This is another way in which nonlinearities may be introduced while
preserving the computational efficiency and stability of linear methods. Next (4), we depart from the second
generation by directly optimizing the EEG-to-BOLD HRF in the hemodynamic frequency domain, without
using a canonical HRF model. Once again, linear methods are employed, this time due to the standard
assumption that the unknown HRF is convolved with some function of neural activity. This assumption is
consistent with our Phase I application, and so is adequate for feasibility assessment. However, from other
work, we also know how to extend linear methods to solve nonlinear Volterra models of neurovascular
coupling (reference [25] of Attachment [2]). Next (5), we employ phase-randomization in the hemodynamic
frequency domain to test nonparametrically for statistically significant neurovascular coupling of the entire
channel-by-channel HRF matrix obtained in the previous step. Importantly, this method is used to detect
BRE signal and—if so detected—the method determines the “beamspace dimension” of the BRE spatial
filter (whereas in our Phase I application, we were resigned to use only a beamspace dimension of 1). Next
(6), we synthesize a spatial filter across all (or a range of) neuroelectric frequencies in order to estimate fast
BRE in the continuous EEG data. This spatial filter may be used to derive virtual EEG channels of derived
data—specifically related to BOLD SOI—which in turn may be analyzed using standard methods of task-

3. related EEG analysis. Finally (7), we use the fast BRE signal to refocus the HRF analysis across a broader
band of EEG frequencies. The ensuing neuroelectric HRF (as opposed to a canonical HRF) may then be
incorporated into the design matrix of standard task-related general linear model analyses of the fMRI data.
In summary, our revised algorithms employ stable methods of linear matrix algebra to solve nonlinear
problems in the hemodynamic frequency domain, where second-order EEG cross-spectra have been
doubly transformed (first to the neuroelectric frequency domain, and then to the hemodynamic frequency
domain). The resulting algorithms preserve the objectives of our Phase I application while overcoming the
computational limitations of our original approach.
Software technologies for bridging EEG and fMRI preprocessing tools. In addition to fundamental revisions
of our computational approach, we have addressed the important practical matter of how to perform
integrated BRE signal detection and estimation using SSI’s Windows-based C++ development platform (the
technological basis for EMSE
, which contains a complete set of preprocessing tools for EEG) together with
commercial-friendly-licensed software for fMRI preprocessing. To this end, we have consulted with
Neuroimaging in Python (NiPy) developers giving particular attention to the Nipype pipelines and interfaces
(http://nipy.sourceforge.net/nipype/). The NiPy project contains native commercial-friendly Python code for
neuroimaging analysis as well as interfaces to other standard fMRI analysis packages, notably, FSL, AFNI,
and FreeSurfer. Of these, FreeSurfer has a commercial-friendly license; AFNI does not; and FSL has a
mixed license (one for academic use and another for commercial use; we have obtained permission from
Oxford University for our current use of FSL as described in Attachment [3]). Python code is portable
across computing platforms (Windows, Unix), and we have therefore implemented a Python interface to
SSI’s scripting engine technology. That is, native EMSE scripts may incorporate existing Python scripts, or
generate and execute Python scripts “on the fly”. Because most available fMRI software runs best on Unix,
we have been using VMware
(http://www.vmware.com) to run fMRI software on an Ubuntu virtual machine
(https://help.ubuntu.com/community/VMware) running on Windows concurrently with EMSE, using a shared
data folder to merge EEG and fMRI analysis results.
Engagement with EEG-fMRI experimental data. Using standard preprocessing methods, we have engaged
with two high-quality EEG-fMRI datasets provided by Prof. G. Ron Mangun, acquired at the UC Davis
Imaging Research Center. See reference [1] of Attachment [2] for a description of the experimental
paradigm. Eight further datasets are standing by, awaiting completion of our analyses. With respect to the
64-channel EEG data, MRI gradient artifacts and ballistocardiogram artifacts were corrected successfully
and consistently using two different commercial software implementations (Scan by Compumedics
Neuroscan’s Scan; and EMSE by SSI). Standard analyses of the data produced high-quality ERPs using
EMSE. With respect to the fMRI data, we engaged Colleen Chen, a graduate student of the Computational
Science Research Program at San Diego State University, who routinely performs fMRI analyses at the
nearby Brain Development Imaging Lab (http://www.sci.sdsu.edu/bdil/web/BDIL.html). Preprocessing steps
and intermediate results for the two UC Davis datasets are provided in Attachment [3]. In brief, we
analyzed these multi-run task-related datasets using standard methods using for functional connectivity
analysis—in particular, the MELODIC ICA method provided in the FSL software, which we are using for
data-driven estimation of the total BOLD signal space (together with exclusion of artifact components and
components related to subcortical networks outside of the EEG source space). Importantly, as shown, we
were able to identify standard functional connectivity networks across runs for each individual subject.
Computational infrastructure. Finally, a significant proportion of our efforts were devoted to designing and
implementing C++ classes for BRE integration EEG and fMRI. Except for fundamental data structures,
much of the design parallels the computational theory. Consequently, new C++ classes and modifications
to classes have been required for each generation of algorithms.
In conclusion, based on the current status, we feel optimistic that continuing efforts beyond Phase I will
enable us to prepare a competitive Phase II proposal.
Attachments
[1] EMSE
as a platform for EEG translational research: from theory to clinical practice.
[2] Data-driven detection and estimation of BOLD-related EEG signal: computational theory.
[3] fMRI preprocessing of UC Davis datasets.

4. SSI Vision Statement 130929
EMSE
as a Platform for EEG Translational Research:
From Theory to Clinical Practice
Translational Research is the engagement of basic research with clinical practice: in particular, the
engagement of cognitive, behavioral, and social electrophysiological research with best clinical practices
for preventing, assessing, diagnosing, and treating neurological, psychiatric, and neurodevelopmental
disorders.
(a) Clinical practice needs more effective combinations of diagnostic constructs and therapeutic
interventions which are based on substantiated theories of adaptive and maladaptive brain-behavioral
circuit dynamical mechanisms. These clinical needs motivate the design of EEG experiments that use
special behavioral paradigms (including baseline or resting conditions). The EEG experiments are
designed to test etiological hypotheses, to devise EEG-based diagnostic biomarkers, to assess the
effectiveness of pharmacologic interventions, or to develop therapeutic neurofeedback protocols.
(b) Series of well-designed experiments, incisive data analyses, and sound theoretical
interpretations in cognitive electrophysiology and related fields lead systematically to improved clinical
practices.

5. SSI Vision Statement 130929
Technical Support at Source Signal Imaging (SSI) is the engagement of SSI personnel with
researchers who use the EMSEsoftware platform to design, conduct, analyze, visualize, and interpret
behavioral electrophysiology experiments.
From its classical embodiment as a GUI-intensive suite of software tools for analysis, visualization,
and modeling of neurophysiological data and integration with MRI neuroimaging, EMSEhas been
evolving to serve as a complete software platform for EEG translational research, including: (i)
integration of experimental design with nonlinear system modeling (SystemID module); (ii) hardware-
neutral, real-time data acquisition integrated with closed-loop behavioral experimentation (rtEMSE );
(iii) fully automated high-throughput analysis (EM|SE|2
= EMSE Scripting Engine); and (iv) multimodal
functional integration via hybrid analysis pipelines leveraging other neuroimaging packages (EEG-fMRI).
(c) Researchers have specific use cases that need to be supported on the EMSEplatform. SSI
technical support personnel are available to help.
(d) The EMSEplatform enables various research scenarios, both classical and new.
Software Development leverages and organizes SSI software technologies (algorithm libraries and
programming frameworks) in order to create, maintain, and evolve the EMSEplatform.
(e) SSI software developers translate functional requirements of EMSEinto detailed designs that
call upon programming frameworks and algorithm libraries.
(f) SSI software developers implement the detailed designs and SSI quality engineers test against
the functional requirements.
Algorithms Research at SSI engages applied mathematics to formulate theories that solve advanced
computational problems of scientific interest. Solutions are implemented as algorithms within suitable
programming frameworks. Computational performance characteristics are studied and optimized.
(g) A scientific computational problem is formulated and solved in mathematical terms.
(h) A theoretical solution to the computational problem is embodied as a testable algorithm.
In conclusion: Sound mathematical theory is the foundation for computational algorithms which,
when provided as accessible content on the EMSEplatform, can facilitate basic research that leads
systematically to improved clinical practices.
Interaction between translational research labs and SSI technical support occasionally can lead to
formal research collaborations.
Interaction between SSI technical support and SSI software development can generate requests for
new features to be implemented and reports of bugs to be fixed.
Interaction between SSI software development and SSI algorithms research can give rise to a
creative and productive research & development cycle.
Workshops sponsored by SSI & partners are fruitful occasions that can engender collaborations and
generate feature requests.
Product planning is a formal lifecycle process at SSI that also engages marketing partners to
prioritize implementation of routine feature requests and commercialization of advanced R&D projects.

6. Abstract—Given a concurrent EEG-fMRI experiment with
multiple runs, we describe a data-driven computational
procedure for detecting neuroelectric-hemodynamic couplings
associated with specific EEG frequencies and fMRI brain
signals of interest (SOIs). For each SOI and frequency, a
hemodynamic response function (HRF) is estimated for every
pair of EEG channels with respect to a second-order quantity
derived from spectral cross-products. A resulting HRF matrix
is submitted to a nonparametric randomization procedure to
assess the statistical significance of its principal EEG spatial
components. SOI-specific BOLD-related EEG (sBRE) signal is
detected when one or more components exceed a statistical
threshold. From the above-threshold components, spatial filters
are constructed which may be used to estimate fast sBRE signal
from the EEG data over a range of frequencies. In addition, a
reduced SOI-specific HRF is obtained which may be used to
improve general linear modeling of the fMRI data. Thus, EEG-
related HRFs (temporal convolutions) and BOLD-related
neuroelectric source estimators (spatial matrices) are jointly
derived from the concurrent data without biophysical modeling.
I. INTRODUCTION
Computational integration of experimental data from
electrodynamic neurophysiology and hemodynamic
neuroimaging modalities is needed for noninvasive
estimation of neurovascular coupling dynamics [26], mutual
improvement of spatial-temporal resolution [30], and a
deeper theoretical understanding of each modality [29].
Joint EEG-fMRI has contributed to advances in basic
cognitive neuroscience [7], epilepsy {Lemieux}, and
schizophrenia [5].
Here we describe a new approach for computational
integration of simultaneously recorded EEG and fMRI.
II.COMPUTATIONAL THEORY
Motivation of the computational approach begins with
multi-run concurrent EEG-fMRI data. The researcher
specifies one or more BOLD signals of interest (SOIs) which
may include brain regions of interest or large-scale networks
of interest. Each BOLD SOI is enhanced in order to
maximize its specificity with respect to other BOLD signals
throughout the brain. During each fMRI TR interval (i.e.,
repetition time period), second-order neuroelectric quantities
are derived for all EEG channel pairs and for each
neuroelectric frequency. A static nonlinear function (such as
a logarithm) may be applied to the entire matrix of second-
*Research reported in this manuscript was supported by the National
Institute of Neurological Disorders and Stroke of the National Institutes of
Health under Award Number R43NS69310. The content is solely the
responsibility of the author(s) and does not necessarily represent the official
views of the National Institutes of Health.
M. E. Pflieger is with Source Signal Imaging Inc., La Mesa, CA 91942
USA (phone: 619-464-0003; e-mail: mep@sourcesignal.com).
order quantities. Each resulting time series is input to a
linear time-invariant (LTI) system which has BOLD SOI as
output. For each neuroelectric (high) frequency, the LTI
system is characterized via a channel-by-channel matrix of
hemodynamic response functions (HRFs) which are
estimated in the hemodynamic (low) frequency domain. A
nonparametric randomization test is used to identify how
many (if any) EEG spatial components are coupled with the
BOLD SOI, and the results are used to construct a space-
frequency filter bank for estimating SOI-specific BOLD-
related EEG (sBRE) signal and its hemodynamic response.
A. Given experimental data
We begin with data from a multi-run concurrent EEG-
fMRI experiment which may (but need not) involve a task to
be performed by the participant. We suppose that MRI
scanner and ballistocardiogram artifacts have been removed
from the EEG data, and that a suitable preprocessing
pipeline has been applied to the fMRI BOLD data.
Symbols related to the data are tabulated as follows:
TABLE 1. PARAMETER INDICES AND SIZES
Symbol Table for Discrete Parameters: Indices and Sizes
Parameter Description Index Size
Neuroelectric (fast) time (~ms) t

T

Hemodynamic (slow) time (~s ) t T
Neuroelectric (high) frequency (>~0.1Hz) f

F

Hemodynamic (low) frequency (<~0.1 Hz) f F
fMRI gray matter voxel v V
BOLD spatial basis function (component) b B
BOLD SOI subspace dimension s S
EEG subinterval of fMRI scan interval i I
EEG channel j J
Experimental run k K
BOLD data phase randomization r R
BOLD-related EEG beamspace dimension d D
TABLE 2. SYMBOLS FOR EXPERIMENTAL DATA
Symbol Table for Time and Frequency Domain Data
Description Symbol
EEG potential on run k for channel j at fast
time t

within subinterval i of scan interval t
[ ]( )kijX t t


Fourier transform of [ ]( )kijX t t

 at frequency f

[ ]( )kijX t f


BOLD signal on run k at time t for voxel v ,
component b , or SOI dimension s
( ), ( ), ( )kv kb ksY t Y t Y t  
Fourier transform of ( )kY t  at frequency f ( )kY f 
Note that time and frequency domain interpretations are resolved via the parameter []
and variable () names. Parameters and variables may be swapped as needed.
Data-Driven Detection and Estimation of
BOLD-Related EEG Signal: Computational Theory*
Mark E. Pflieger et al. (in preparation)

7. B. BOLD signal of interest (SOI)
BOLD signals measured at gray matter voxels may be
coupled with neuroelectric generators of scalp-recorded
EEG. However, due to both local and distant voxel-voxel
correlations of BOLD signals over the entire brain, the
effective dimension of the BOLD signal space B is much
smaller than the total number of gray matter voxels V . In
practice, the BOLD signal space dimension B may be
obtained by temporal component analysis such as temporal
principal component analysis (PCA) or independent
component analysis (ICA). Temporal component analysis
produces B components or basis functions that collectively
span the entire BOLD signal space. Each component b has
an associated time series ( )kbY t (for each experimental run
k ) and an associated spatial pattern ( )bP v distributed over
all gray matter voxels v V∈ .
The spatial pattern of a component may coincide with a
brain region. More generally, however, it corresponds to a
distributed network of brain regions which have correlated
(or anti-correlated) BOLD signals. If the BOLD signal of
interest (SOI) does not coincide with one particular
component, then the SOI’s desired spatial pattern ( )sP v
(e.g., a region of interest) may be reconstructed
approximately as a linear combination of component spatial
patterns with β coefficients obtained via linear least squares
1
( ) ( )
B
s sb b
b
P v P vβ
=
≈ ∑ (1)
so that the time series associated with the BOLD SOI is
1
( ) ( )
B
ks sb kb
b
Y t Y tβ
=
= ∑  . (2)
In sum, the BOLD SOI may correspond to a localized
brain region or a distributed brain network, as specified by
the researcher. As a special case, the researcher’s BOLD
SOI s may coincide with a single component b .
More generally, the BOLD SOI may be a projection from
the total BOLD signal space to a subspace comprising a
small number S of dimensions.
C. Maximally specific BOLD SOI
We aim to derive a bank of EEG spatial filters that
maximize neuroelectric-hemodynamic coupling with the
BOLD SOI specifically. Therefore, because EEG is
sensitive to whole-cortex activity, we must consider and
factor out BOLD signal not of interest (SnOI). This
subsection describes a hemodynamic frequency domain filter
that maximizes specificity of BOLD SOI versus SnOI.
Consider a thought experiment in which BOLD signal is
perfectly correlated in time across all voxels. Because no
temporal information is available for making discriminations,
it would be impossible to identify SOI-specific BOLD-
related EEG signal using such data. Although this case is
extreme, large-scale correlations are well known in fMRI
data and, in fact, are the basis for identifying resting state
network phenomena [3]. Hence, only the distinctive SOI
features that remain after removing correlations with SnOI
may be used to detect SOI-specific neuroelectric-
hemodynamic coupling. Our filter for extracting the
distinctive SOI features is designed as follows.
Let ,SOIkY be the F S× SOI matrix where each element
( )ksY f is the Fourier transform of (2), and form the F F× 
average SOI inter-frequency cross-spectral matrix
SOI ,SOI ,SOI
1
1 K
k k
kK
∗
=
≡ ∑G Y Y . Similarly, let kY be the F B×
total signal (SOI + SnOI) matrix for run k where each
element is ( )kbY f ; then form the average total signal inter-
frequency cross-spectral matrix
1
1 K
k k
kK
∗
=
≡ ∑G Y Y . Next,
perform complex singular value decomposition (SVD) [13]
( )0.5 0.5
SOI 1 1 1svd − − ∗
=G G G U W U (3)
of the SOI matrix normalized (i.e., sphered) with respect to
the total signal matrix via the inverse symmetric square root
0.5−
G . Examine the largest singular values from the
diagonal of 1W and retain S S′ ≤ of these. The Appendix
provides a statistical specificity test for setting a threshold on
the singular values. If 0S′ = , then the SOI is not
sufficiently distinctive to be discriminated from SnOI, and
thus there is no basis for further analysis with respect to the
SOI. Otherwise, form the matrix S′U comprising the first
S′ columns of 1U . The F F×
 
SOI projection matrix is
0.5
SOI S S
∗ −
′ ′≡P U U G . (4)
Finally, obtain the F S× SOI-specific BOLD signal matrix
in the hemodynamic frequency domain:
,SOI SOI ,SOIk k=Y P Y . (5)
The filtered hemodynamic time domain signal ( )ksY t
 is
obtained by inverse transforming the matrix elements
( )ksY f
 . However, we note that (16) below uses the
frequency domain representation.
In addition, we note that cross-frequency interactions
reflect nonlinear dynamics of the BOLD signal. Thus, this
section uses linear methods (SVD) in the frequency domain
to extract distinctive nonlinear features of BOLD SOI, i.e.,
SOI patterns of cross-frequency interactions that stand out
against average patterns found in the total BOLD signal.
D. Second-order EEG-derived quantities
A fundamental second-order quantity derived from the
EEG data is the inter-channel cross-spectrum for a scan
interval
1 2 1 2 1 2
1
1
[ ]( ) [ ]( ) [ ]( ) [ ]( )
I
kj j kj j kij kij
i
C f t C t f X t f X t f
I
∗
=
= ≡ ∑
   
    (6)
which is the subinterval-averaged Fourier transform of the
cross-correlation function of EEG channels 1j and 2j for
fMRI scan interval t of run k . For example, if the fMRI
scan interval (TR) is 2 s and the EEG subinterval is 250 ms,

8. then there are 15I = 50%-overlapping subintervals, and the
neuroelectric frequency bin width is 4 Hz (= 1/0.25s).
Let [ ]( )k f tC

 be the J J× matrix whose element at row
1j and column 2j is 1 2
[ ]( )kj jC f t

 , and estimate the
experimental average inter-channel cross-spectral matrix
1 1
1
[ ]( ) [ ]( ) [ ]
T K
k
t k
f f t f
TK
•
= =
= ≡∑∑C C C


  
 

. (7)
Then the spatially normalized (or sphered) inter-channel
cross-spectral matrix is
0.5 0.5
[ ]( ) [ ] [ ]( ) [ ]k kf t f f t f− −
≡C C C C
   
  (8)
where 0.5
[ ]f−
C

is the inverse symmetric square root of
[ ]fC

stabilized via complex SVD. The element at row 1j
and column 2j is denoted by 1 2
[ ]( )kj jC f t

 . Next perform
SVD on (8)
( ) 2 2 2svd [ ]( )k f t ∗
=C U W U

 (9)
where 2 21 22 23diag{ , , , }w w w=W  , a diagonal matrix of
rank-ordered non-negative real singular values. Indices k ,
f

, and t have been suppressed on the right-hand side.
To the extent that EEG spatial patterns and their
amplitudes on scan interval t of run k resemble the
experimental average patterns and amplitudes estimated by
(7), we expect typical singular values around 1.0 after spatial
normalization. A singular value greater than 1.0 indicates a
larger-than-average amplitude of the corresponding pattern;
whereas a fractional singular value less than 1.0 indicates a
smaller-than-average amplitude of the corresponding pattern.
Because both decreases and increases relative to the
experimental average may be relevant to BOLD-related
coupling, a log (e.g., base 2) transform (2.01.0, 1.00.0,
0.5-1.0) may be applied to attenuate average values while
accentuating both larger- and smaller-than-average values.
More generally, consider any invertible scalar function
: (0, ] [ , ]φ ∞ → −∞ ∞ such as the identity function
1
( ) ( )x x xφ φ−
= = (10)
or a power function
1 1/
( )
( )
x x
y y
β
β
φ
φ−
=
=
(11)
or a logarithmic function
1
( ) log( )
( ) exp( / )
x x
y y
φ β
φ β−
=
=
. (12)
Any such function may be applied to the whole matrix
( )
2 21 22 2
[ ]( ) [ ]( )
diag{ ( ), ( ), }
k kf t f t
w w
φ φ
φ φ ∗
≡
=
C C
U U
 
 

(13)
that is, by applying φ to each singular value of equation (9).
The resulting element at row 1j and column 2j is denoted
by 1 2
[ ]( )kj jC f tφ

 .
Empirically, our motivation is to discover forms of φ that
optimize coupling with BOLD signal.
E. HRF estimation in EEG sensor-sensor space
In the hemodynamic time domain, for each pair of EEG
sensors, a φ-transformed EEG inter-channel cross-spectral
time series is convolved as input with an unknown
hemodynamic response function (HRF) H in order to
approximate the SOI-specific BOLD time series as output:
1 2 1 2
( ) [ ]( ) [ ]( )ks sj j kj jY t H f C f t dτ φ τ τ≈ −∫
 

     . (14)
By the convolution theorem, this transforms to a linear
relationship in the hemodynamic frequency domain
1 2 1 2
( ) [ ]( ) [ ]( )ks sj j kj jY f H f f C f fφ≈
 

   (15)
so that a best-fitting HRF can be estimated across runs as
1 2
1 2
1 2
2
( ) [ ]( )
[ ]( )
[ ]( )
ks kj jk
sj j
kj jk
Y f C f f
H f f
C f f
φ
φ
≈
∑
∑

  


. (16)
Thus, the time domain HRF is 1 2
[ ]( )sj jH f t

 . Let 1 2
max
sj jt be the
time at which the HRF peaks in a specified time interval, and
define the J J× peak HRF matrix for SOI component s :1
1 2 1 2
max
[ ] [ ]( )s sj j sj jf H f t ≡  H
 
 . (17)
As a step toward identifying EEG spatial filters (i.e.,
channel vectors) which are most strongly coupled with
BOLD SOI component s , we perform complex singular
value decomposition [13] on [ ]s fH

:
( ) 3 3 3svd [ ]s f ∗
=H U W U

(18)
where 3 31 32 33diag{ , , , }w w w=W  are rank-ordered singular
values. Indices s and f

have been suppressed on the right-
hand side.
F. BOLD signal phase randomization procedure
Our next step is to assess the statistical significance of the
singular values 3W nonparametrically by generating an
empirical distribution of nonsignificant singular values
obtained from surrogate data sets constructed by destroying
the possibility of BOLD-EEG coupling. This is achieved by
phase-randomizing the BOLD data across runs and
hemodynamic frequencies as follows:
( ) ( ) exp{ }rks ks rkfY f Y f iθ=

 
  (19)
where r indexes one of R randomizations, and
0 2rkfθ π≤ <

is a random phase angle. Note that each
randomization remains fixed for all BOLD SOI components
and EEG frequencies.
For each randomization r , substitute ( )rksY f
 for ( )ksY f

in equation (16) and proceed to compute
3 31 32 33diag{ , , , }r r r rw w w=W  as in equation (18). By
1
Alternatively, the HRF matrix may be constructed (a) at a specified
hemodynamic latency; (b) by stacking matrices obtained in (a) for a range
of latencies; or (c) by stacking matrices across hemodynamic frequencies.

9. construction, these singular values are sampled from the null
distribution that obtains when BOLD-EEG coupling is zero.
G. Detection of SOI-specific BOLD-related EEG signal
To assess the statistical significance of coupling, let d be
an integer less than the number J of EEG channels, and let
( )dN w be the number of randomizations for which
3r dw w≥ , where 3r dw is the rank d singular value obtained
from one of the randomizations. In addition, let 3d dw w≡
be the corresponding the rank d singular value from the
original matrix 3W . Then the estimated probability that a
the rank d singular value in the null distribution exceeds or
equals dw is ( ) ( ( ) 1) / ( 1)d d dp w N w R= + + . Thus, we
detect SOI-specific BOLD-related EEG (BRE) signal when
( )dp w α< , where α is a probability threshold criterion
such as 0.05α = . We proceed sequentially
1, 2, ,d d d D= = = until the criterion fails to be satisfied
at 1d D= + . The beamspace dimension D is the largest
integer for which the criterion is satisfied consecutively.
H. Space-frequency filter bank for sBRE signal estimation
We proceed to construct a bank of EEG spatial filters—
one set of filters for each neuroelectric frequency f

—that
estimate SOI-specific BOLD-related EEG (sBRE) signal,
i.e., EEG signal optimally and significantly coupled with the
BOLD SOI. The channel-by-channel peak HRF matrix
constructed in (17) contains EEG spatial patterns that
optimize peak neuroelectric-hemodynamic coupling with the
BOLD SOI. The principal patterns are the columns of
matrix 3 [ ]s fU

of (18), and the phase randomization
procedure is applied to detect [ ] 0sD f ≥

EEG patterns that
are statistically coupled with BOLD SOI component s .
Thus, we construct the [ ]sD f J×

sBRE estimation matrix as
[ ] [ ]s Dsf f∗
≡E U
 
(20)
where [ ]Ds fU

comprises [ ]sD f

columns of 3 [ ]s fU

.
I. Estimation of fast sBRE signal from EEG data
The sBRE estimation matrix [ ]s fE

is a spatial filter that
may be applied in the neuroelectric frequency domain to
EEG subintervals ( , , )k t i of spatially normalized multi-
channel data 0.5
[ ] [ ]( )kif t f−
C X
 
 , where [ ]( )ki t fX

 is a
column vector having J elements [ ]( )kijX t f

 . The result
0.5
[ ]( ) [ ] [ ] [ ]( )ski s kit f f f t f−
=Z E C X
   
  (21)
estimates the sBRE signal (a [ ]sD f

-dimensional vector) in
the neuroelectric frequency domain at frequency f

on
experimental run k , scan interval t , and subinterval i .
It may appear that [ ]( )ski t fZ

 can be converted directly to
the neuroelectric time domain via the inverse Fourier
transform. However, direct conversion to the neuroelectric
time domain is not meaningful because the number of
beamspace dimensions [ ]sD f

and the EEG spatial patterns
associated with beamspace dimensions vary with f

.
One solution is to obtain a set of common spatial patterns
across frequencies by SVD of the stacked estimator matrix
0.5
4 4 4
0.5
[ 1] [ 1]
svd
[ ] [ ]
s
s
f f
f F f F
−
∗
−
 = =
 
= 
 
= =  
E C
U W V
E C
 

  
(22)
so that sBRE signal is estimated in the frequency domain as
[ ]( ) [ ]( )ski s kit f t f′ ′=Z E X
 
  (23)
where s
′E ( D J′× ) restricts the rows of 4
∗
V to D′ common
beamspace dimensions. Now the fast sBRE signal can be
obtained via the inverse Fourier transform as [ ]( )ski t t′Z

 .
We note that finding a frequency-independent sBRE
estimator matrix s
′E is consistent with the more restrictive
quasi-static approximation that is assumed almost always
when volume conductor models are constructed to solve
EEG and MEG forward and inverse problems.
In addition, because the Fourier transform is linear, we
can almost write a direct and much simplified time domain
estimator as
( ) ( )s st t′ ′=Z E X
 
(24)
where indices for the experimental run, scan interval, and
EEG subinterval are no longer needed: “almost” because s
′E
is in general a complex matrix whereas ( )tX

is recorded as a
real vector time series. Thus, (24) obtains with the further
assumption—also consistent with the quasi-static
approximation—that the imaginary part of s
′E is negligible.
J. Hemodynamic response function of the sBRE signal
Let [ ]( )skidZ t f

 be beamspace component d of the
estimated BRE signal vector [ ]( )ski t fZ

 of (21); then form
[ ] 2
1 1
1
[ ]( ) [ ]( )
sD fI
sk skid
i d
C f t Z t f
I
φ
= =
 
′ ≡   
 
∑ ∑

 
  (25)
which is function φ applied to the subinterval averaged sum
of squared magnitudes of the [ ]sD f

orthogonal beamspace
dimensions of the sBRE estimate. After Fourier
transformation, substitution of [ ]( )skC f f′

 for 1 2
[ ]( )kj jC f fφ


in (16) produces the desired hemodynamic response function
[ ]( )sH f f

 (time domain equivalent = [ ]( )sH f t

 ) as the
counterpart of 1 2
[ ]( )sj jH f f

 .
III. DISCUSSION
We have detailed a computational theory that aims to
bridge a considerable gap between EEG and fMRI BOLD
phenomena, which are measured on temporal frequency
scales separated by about three orders of magnitude: the
neuroelectric high frequency range (~0.1-100 Hz) and the

10. hemodynamic low frequency range (<~0.01-0.1 Hz). Our
approach employs the Fourier transform twice. The first
transform converts continuous EEG (~ms time scale) to
spectra in subintervals (~250 ms) which are averaged to
estimate inter-channel cross-spectra for each fMRI TR (~2
s). On this ~0.5 Hz scale, the second transform is used to
solve a convolution equation in order to estimate a
neuroelectric-hemodynamic response function for each pair
of EEG channels. The resulting channel-by-channel matrix
is associated with a particular EEG frequency and is
analyzed for EEG spatial patterns which are most strongly
coupled with BOLD signal of interest.
Thus, taking advantage of concurrent EEG-fMRI
measurements, we seek to identify space-frequency
properties of EEG generated by cortical neuroelectric
sources that are coupled specifically with BOLD
hemodynamic signals of interest. However, functional
connectivity analyses with fMRI reliably disclose
hemodynamic correlations between distant brain regions that
may, in turn, be coupled with concurrent generators of scalp
EEG. Consequently, it appears vital to take into account
BOLD signal correlations throughout the brain, and
especially correlations with the BOLD signal of interest
(SOI).
{Discussion to be continued…}
ACKNOWLEDGMENT
Research reported in this manuscript was supported by the
National Institute of Neurological Disorders and Stroke of
the National Institutes of Health under Award Number
R43NS69310. The content is solely the responsibility of the
author(s) and does not necessarily represent the official
views of the National Institutes of Health.
APPENDIX
We describe a statistical test of SOI specificity for
deciding how many singular values to retain from the rank-
ordered singular value matrix 1 11 12 13diag{ , , , }w w w=W  of
equation (3) in section II-C. Perform K random
permutations ( K is the number of summands in the
formation of SOIG and G ) over all B basis functions
(components) R times. Note that each permutation has no
effect on the total signal matrix G , because the total signal
uses all basis functions, and their order of indexing is
immaterial. However, the SOI matrices ,SOIrkY that compose
,SOIrG will be changed for each K -fold randomization r ,
and thus the rank-ordered singular value matrix
1 11 12 13diag{ , , , }r r r rw w w=W  will be altered.
Note that each randomization r constructs K different
signal subspaces, each of which has dimensional
characteristics similar to the SOI subspace. Hence, the
singular values obtained after each randomization are not
specific to one particular signal subspace. Consequently, the
randomization procedure destroys SOI subspace specificity,
and may be used to construct a distribution for the null
hypothesis that the observed singular values were obtained in
the absence of SOI subspace specificity.
For each integer s S′ ≤ , let ( )sN w′ be the number of
randomizations for which rsw w′ ≥ , where 1rs r sw w′ ′≡ is the
rank s′ singular value from the diagonal of 1rW for some
randomization r ; and let 1s sw w′ ′≡ be the corresponding
rank s′ singular value from the original matrix 1W . Then
the estimated probability that a nonsignificant rank s′
singular value (obtained from nonspecific signal subspaces)
exceeds or equals sw ′ is ( ) ( ( ) 1) / ( 1)s sp w N w R′ ′= + + .
Thus, an acceptance criterion for SOI specificity is
( )sp w α′ < , where α is a probability threshold (e.g.,
0.05α = ). We choose the largest s S′ ′= (number of
retained singular values) for which the criterion is satisfied
consecutively. If 0S′ = , then the test indicates that SOI
cannot be differentiated adequately from SnOI.
REFERENCES
[1] J. J. Bengson, G. R. Mangun, and A. Mazaheri, “The neural markers
of an imminent failure of response inhibition,” Neuroimage, vol. 59,
pp. 1534-1439, January 2012.
[2] D. A. Birdwell, L. Wu, T. Eichele, and V. D. Calhoun, “The
spatiospectral characterization of brain networks: fusing concurrent
EEG spectra and fMRI maps,” Neuroimage, e-pub December 2012.
[3] B. B. Biswal, M. Mennes, X. N. Zuo, S. Gohel S, C. Kelly, S. M.
Smith, et al., “Toward discovery science of human brain function,”
Proc Natl Acad Sci U S A, vol. 107, pp. 4734-4739, March 2010.
[4] J. Britz, D. Van De Ville, and C. M. Michel, “BOLD correlates of
EEG topography reveal rapid resting-state network dynamics,”
Neuroimage, vol. 52, pp. 1162-1170, October 2010.
[5] V. Calhoun, L. Wu, K. Kiehl, T. Eichele, and G. Pearlson, “Aberrant
processing of deviant stimuli in schizophrenia revealed by fusion of
fMRI and EEG data,” Acta Neuropsychiatr., vol. 22, pp. 127-138,
June 2010.
[6] N. M. Correa, T. Eichele, T. Adali, Y. O. Li, and V. D. Calhoun,
“Multi-set canonical correlation analysis for the fusion of concurrent
single trial ERP and functional MRI,” Neuroimage, vol. 50, pp. 1438-
1445, May 2010.
[7] A. M. Dale and E. Halgren, “Spatiotemporal mapping of brain
activity by integration of multiple imaging modalities,” Curr. Opin.
Neurobiol., vol. 11, pp. 202-208, April 2001.
[8] J. Daunizeau, C. Grova, G. Marrelec, J. Mattout, S. Jbadi, M.
Pélégrini-Issac, J. M. Lina, and H. Benali, “Symmetrical event-related
EEG/fMRI information fusion in a variational Bayesian framework,”
Neuroimage, vol. 36, pp. 69-87, May 2007.
[9] J. C. de Munck, S. I. Gonçalves, L. Huiboom, J. P. Kuijer, P. J.
Pouwels, R. M. Heethaar, and F. H. Lopes da Silva, “The
hemodynamic response of the alpha rhythm: an EEG/fMRI study,”
Neuroimage, vol. 35, pp. 1142-1151, April 2007.
[10] T. Eichele, V. D. Calhoun, and S. Debener, “Mining EEG-fMRI using
independent component analysis,” Int. J. Psychophysiol., vol. 73, pp.
53-61, July 2009.
[11] E. Glerean, J. Salmi, J. M. Lahnakoski, I. P. Jääskeläinen, and M.
Sams, “Functional magnetic resonance imaging phase
synchronization as a measure of dynamic functional connectivity,”
Brain Connectivity, vol. 2, pp. 91-101, June 2012.
[12] R. I. Goldman, C. Y. Wei, M. G. Philiastides, A. D. Gerson, D.
Friedman, T. R. Brown, and P. Sajda, “Single-trial discrimination for
integrating simultaneous EEG and fMRI: identifying cortical areas
contributing to trial-to-trial variability in the auditory oddball task,”
Neuroimage, vol. 47, pp. 136-147, August 2009.
[13] G. H. Golub and C. F. Van Loan, Matrix Computations, Third
Edition. Johns Hopkins University Press, 1996.

11. [14] H. Laufs, A. Kleinschmidt, A. Beyerle, E. Eger, A. Salek-Haddadi, C.
Preibisch, and K. Krakow, “EEG-correlated fMRI of human alpha
activity,” Neuroimage, vol. 19, pp. 1463-1476, August 2003.
[15] X. Lei, D. Ostwald, J. Hu, C. Qiu, C. Porcaro, A. P. Bagshaw, and D.
Yao, “Multimodal functional network connectivity: an EEG-fMRI
fusion in network space,” PLoS One, vol. 6, e24642, September 2011.
[16] X. Lei, P. A. Valdes-Sosa, and D. Yao, “EEG/fMRI fusion based on
independent component analysis: integration of data-driven and
model-driven methods,” J. Integr. Neurosci., vol. 11, pp. 313-337,
September 2012.
[17] X. Lei, C. Qiu, P. Xu, and D. Yao, “A parallel framework for
EEG/fMRI analysis: methodology and simulation,” Neuroimage, vol.
52, pp. 1123-1134, September 2010.
[18] X. Lei, P. Xu, C. Luo, J. Zhao, D. Zhou, and D. Yao, “fMRI
functional networks for EEG source imaging,” Human Brain
Mapping, vol. 32, pp. 1141-1160, July 2011.
[19] M. Lenz, M. Musso, Y. Linke, O. Tüscher, J. Timmer, C. Weiller, and
B. Schelter, “Joint EEG/fMRI state space model for the detection of
directed interactions in human brains—a simulation study,” Physiol.
Meas., vol. 32, pp. 1725-1736, November 2011.
[20] Y. Lu, A. P. Bagshaw, C. Grova, E. Kobayashi, F. Dubeau, and J.
Gotman, “Using voxel-specific hemodynamic response function in
EEG-fMRI data analysis,” Neuroimage, vol. 32, pp. 238-247, August
2006.
[21] D. Mantini, L. Marzetti, M. Corbetta, G. L. Romani, and C. Del
Gratta, “Multimodal integration of fMRI and EEG data for high
spatial and temporal resolution analysis of brain networks,” Brain
Topography, vol. 23, pp. 150-158, June 2010.
[22] E. Martinez-Montes, P. A. Valdés-Sosa, F. Miwakeichi, R. I.
Goldman, and M. S. Cohen, “Concurrent EEG/fMRI analysis by
multiway partial least squares,” Neuroimage, vol. 22, pp. 1023-1034,
July 2004.
[23] F. Musso, J. Brinkmeyer, A. Mobascher, T. Warbrick, and G.
Winterer, “Spontaneous brain activity and EEG microstates. A novel
EEG/fMRI analysis approach to explore resting-state networks,”
Neuroimage, vol. 52, pp. 1149-1161, October 2010.
[24] D. Ostwald, C. Porcaro, and A. P. Bagshaw, “Voxel-wise information
theoretic EEG-fMRI feature integration,” Neuroimage, vol. 55, pp.
1270-1286, April 2011.
[25] M. E. Pflieger, “Efficient identification of oblique Volterra systems,”
Int. J. Bioelectromagnetism, vol. 14, pp. 51-55, February 2012.
[26] M. E. Pflieger and R. L. Barbour, “Multimodal integration of fMRI,
EEG, and NIRS,” Biomedical Optics (BIOMED) 2012 paper:
BSu2A.1 (opticsinfobase.org), April 2012.
[27] P. Sajda, “Signal processing challenges for single-trial analysis of
simultaneous EEG/fMRI,” Conf. Proc. IEEE EMBS 2009, pp. 29-30.
[28] J. R. Sato, C. Rondinoni, M. Sturzbecher, D. B. Araujo, and E.
Amaro Jr., “From EEG to BOLD: brain mapping and estimating
transfer functions in simultaneous EEG-fMRI acquisitions,”
Neuroimage, vol. 50, pp. 1416-1426, May 2010.
[29] R. Scheeringa, P. Fries, K.-M. Petersson, R. Oostenveld, I. Grothe,
D.G. Norris, P. Hagoort, and M. C. M. Bastiaansen, “Neuronal
dynamics underlying high- and low-frequency EEG oscillations
contribute independently to the human BOLD signal,” Neuron, vol.
69, pp. 572-583, February 2011.
[30] H. Shibasaki, “Human brain mapping: hemodynamic response and
electrophysiology,” Clin. Neurophysiol., vol. 119, pp. 731-743, April
2008.
[31] M. Ullsperger and S. Debener, eds., Simultaneous EEG and fMRI:
Recording, Analysis, and Application. Oxford University Press,
2010.
[32] P. A. Valdes-Sosa, J. M. Sanchez-Bornot, R. C. Sotero, Y. Iturria-
Medina, Y. Aleman-Gomez, J. Bosch-Bayard, F. Carbonell, and T.
Ozaki, “Model driven EEG/fMRI fusion of brain oscillations,” in
Human Brain Mapping, vol. 30, pp. 2701-2721, September 2009.
[33] K. Whittingstall, A. Bartels, V. Singh, S. Kwon, and N. K. Logothetis,
“Integration of EEG source imaging and fMRI during continuous
viewing of natural movies,” Magn. Reson. Imaging, vol. 28, pp.
1135-1142, October 2010.
[34] L. Wu, T. Eichele, and V. Calhoun, “Parallel independent component
analysis using an optimized neurovascular coupling for concurrent
EEG-fMRI sources,” Conf. Proc. IEEE EMBS 2011, pp. 2542-2545.
[35] L. Yang, Z. Liu, and B. He, “EEG-fMRI reciprocal functional
neuroimaging,” Clin. Neurophysiol., vol. 121, pp. 1240-1250, August
2010.
[36] H. Yuan, V. Zotev, R. Phillips, W. C. Drevets, and J. Bodurka,
“Spatiotemporal dynamics of the brain at rest—exploring EEG
microstates as electrophysiological signatures of BOLD resting state
networks,” Neuroimage, vol. 60, pp. 2062-2072, May 2012.

12. 1
1
fMRI Preprocessing of UC Davis Datasets
Colleen Chen, graduate student, SDSU Computational Science Research Center;
prepared for Source Signal Imaging, Inc.
Introduction
The functional neuroimaging data was collected from two participants across
multiple scans of task stimulus. The goal is to identify functional connectivity
networks in individual participants by combining within-participant scans of task
data. The networks are parcellated into spatial maps for feature extraction. We
analyzed the task data by applying “resting state” methods whereby ignoring the
events. We then applied subject-level ICA by extracting components across runs.
Multi-sessions temporal concatenation (MSTC), a procedure based on independent
component analysis, extracts common spatial maps across different scans within the
same subject. We want to see if it is possible to identify canonical brain networks in
individual subjects who are engaged in task.
Methods
Data were processed using the Analysis of Functional NeuroImages software [11]
(afni.nimh.nih.gov) and FSL 5.0 [12] (www.fmrib.ox.ac.uk/fsl). Functional images
were slice-time corrected, motion corrected (3dvolreg) to align to the middle time
point, and aligned to the anatomical image using FLIRT [13][14] with six degrees of
freedom. FSL’s nonlinear registration tool (FNIRT) was then used to standardize
images to the MNI152 standard image (3mm isotropic) using sine interpolation, and
the outputs were blurred to a global full-width-at-half-maximum of 10mm. Given
recent concerns that traditional filtering approaches can cause rippling of motion
confounds to neighboring time points [15], we used a second-order band-pass
Butterworth filter [16][17] to isolate low-frequency BOLD fluctuations (.008 < f < .1
Hz) [18].
Regression of a total of 16 nuisance variables was performed to improve data
quality [17]. Nuisance regressors included six rigid-body motion parameters
derived from motion correction and the derivatives. White matter (WM) and
ventricular masks were created at the participant level using FSL’s FAST image
segmentation [19] and trimmed to avoid partial-volume effects. An average time
series was extracted from each mask and was removed using regression, along with
its corresponding derivative. All nuisance regressors were band-pass filtered using
the second-order Butterworth filter (.008 < f < .08 Hz) [16][17]. We did not regress
out the global signal.
Motion was quantified as the Euclidean distance between the six rigid-body
motion parameters for two consecutive time points. For any instance greater than

13. 2
2
1mm, considered excessive motion, the time point as well as the preceding and
following time points were censored, or “scrubbed” [16]. If two censored time
points occurred within ten time points of each other, all time points between them
were also censored. Both subjects had more than 90% of time points and retained
more than 150 total time points remaining after censoring. Runs were then
truncated at the point where 150 usable time points was reached. Motion over the
truncated run was summarized for each participant as the average Euclidean
distance moved between time points (including areas that were censored).
Analysis-ICA
Independent component analysis (ICA) is a data-driven approach to defining
functional networks that does not require a priori information. Spatial components
extracted using ICA have been shown to correspond to known functional networks
and to be consistent within and across individuals. Temporally concatenated ICA
will be run by concatenating the multiple scans of each subject and using FSL
MELODIC to extract a set of subject-level components. Dimensionality will be
automatically estimated using Laplace approximation to the Bayesian evidence.
Once data were preprocessed to remove noise artifacts from in-scanner head
motion and those of physiological origin, we applied ICA to the datasets from the
two participants across multiple scans. We concatenated the time courses across
scans for each participant and the normalized the data by subtracting the mean and
dividing by the standard deviation. This standardized data produces cleaner looking
components. We also ran ICA without this normalization, in order to compare
results.
Results:
Subject #1- normalized
Visual- medial
Frontoparietal- left lateralized

14. 3
3
Executive control- medial frontal
areas (anterior cingulate and
paracingulate)
Visual- occipital pole
Sensorimotor
Default Mode Network- medial
parietal (Precuneous and posterior
cinculate), bilateral inferior,
ventromedial frontal cortex

15. 4
4
Frontoparietal- right lateralized
Auditory- superior temporal gyrus
and posterior insular

16. 5
5
Executive control- medial frontal
areas

17. 6
6
Subject #2- normalized
Visual- medial
Frontoparietal- right lateralized
Default Mode Network
Visual- occipital pole

18. 7
7
Auditory
Executive Control

19. 8
8
Auditory

20. 9
9
Discussion
Using automatic dimensionality estimation, MELODIC detected a total of 35
independent components (ICs) for subject #1, and 23 ICs for subject #2. The
difference in the number of components detected reflects the variability across
subjects and that ICA is susceptible to this cross-subject variability. Further, ICA
may be useful in detecting noisy components due to head-motion, as seen in figures
below:
It may be useful to defect noise using this analysis and remove the noisy
components from further statistical analysis.
Overall, ICA seems a promising tool for parcellations of the brain into networks that
can later be used as feature components in combining datasets from different
modalities, such as EEG. With the addition of more participants and data, it would be
possible to conduct group ICA for extraction of most robust canonical networks
(refer to figure below from Smith et al.).

21. 10
10
The 10 maps correspond to interpretable functional categories and can be
considered the ‘‘major representative’’ functional networks as derived
independently from both activation meta-analysis and resting data (Smith et al.).
The maps’ functional networks will be listed:
1. Visual
2. Visual
3. Visual
4. Default Mode Network
5. Cerebellum
6. Sensorimotor
7. Auditory
8. Executive control
9. Frontoparietal
10. Frontoparietal

22. 11
11
References
[11] Cox, R. W. (1996). "AFNI: Software for analysis and visualization of
functional magnetic resonance neuroimages." Computers and Biomedical Research
29: 162-173.
[12] Smith, S. M., M. Jenkinson, et al. (2004). "Advances in functional and
structural MR image analysis and implementation as FSL." Neuroimage 23 Suppl 1:
S208-219.
[13] Jenkinson, M., P. Bannister, et al. (2002). "Improved optimization for the
robust and accurate linear registration and motion correction of brain images."
Neuroimage 17(2): 825-841.
[14] Jenkinson, M. and S. Smith (2001). "A global optimisation method for robust
affine registration of brain images." Med Image Anal 5(2): 143-156.
[15] Carp, J. (2011). "Optimizing the order of operations for movement scrubbing:
Comment on Power et al." Neuroimage.
[16] Power, J. D., K. A. Barnes, et al. (2012). "Steps toward optimizing motion
artifact removal in functional connectivity MRI; a reply to Carp." Neuroimage.
[17] Satterthwaite, T. D., M. A. Elliott, et al. (2013). "An improved framework for
confound regression and filtering for control of motion artifact in the preprocessing
of resting-state functional connectivity data." Neuroimage 64: 240-256.
[18] Cordes, D., V. M. Haughton, et al. (2001). "Frequencies contributing to
functional connectivity in the cerebral cortex in "resting-state" data." AJNR Am J
Neuroradiol 22(7): 1326-1333.
[19] Zhang, Y., M. Brady, et al. (2001). "Segmentation of brain MR images through
a hidden Markov random field model and the expectation-maximization algorithm."
IEEE Trans Med Imaging 20(1): 45-57.