This is slide set of my Octopus-ReEL (Realtime Encephalography Lab) presentation in GDG-Izmir event held on Nov 3rd 2018 at Ege University Computer Engineering Dept.

1. “Octopus-ReEL”:
Open-source (Re)al-time (E)ncephalography (L)ab
Barkın İlhan
Computational Cognitive Neuroscience Lab
Meram Medical Faculty, Department of Biophysics, Konya, Turkey

2. PhD Thesis in Biophysics (2005-2009)
”Finding out the cortical equivalent neuroelectrical «sources» of
«directional hearing» (Neuro-electrical Source Imaging, NESI)”:
• EEG, equivalent dipole(s), modeling of realistic head geometry and
conductivity, forward modeling of tissue conductivity, optimization (i.e.
inverse problem solutions), optimal # of sensors, etc.
”Holistic low-cost networked source localization system with Beowulf
parallel computer interface, and a framework for real-time stimulus
creation & electrode measurement with no head restraining” (2009)
• Truely open-source, GNU-style «free-software» (i.e. GPLv3)
• Proper OS, library, etc. choices w/o any constraint as possible; aiming
elegance and quality, not quantity (of community, developers, ...) i.e.
Debian/GNU Linux, NFS, NIS, BIND, C, RTAI, COMEDI, C++, Qt,
OpenGL, OpenCL, OpenMPI, PetSc, NFS, NIS, BIND, …
• Real-time (@ it’s hardest) for neuro-feedback, making possible even
closed-loops for low-level brain structures
• Network-based, distributed, scalable nature, high-throughput, high-yield,
high-availability, etc.
Short history of Octopus-ReEL

3. Head Conductance
Model
Electrode coords
& EEG/ERP data
+
+
Task-related “a-priori information”
(i.e. confining region to auditory cortices)
INVERSE PROBLEM
SOLUTION
INVERSE PROBLEM
“Equivalent Current Dipole(s): PSPs of many
synchronously firing neurons cause the scalp
potential field, which we measure as sEEG”
(DeMunck, et al., IEEE Trans Biomed Eng, 1988)
ECD
NEUROELECTRICAL FORWARD MODELING
What is Neuro-electrical Source Imaging?

4. Modeling of cranial tissue geometry
(i.e. boundaries of scalp, skull & brain)
MRI data
(dense, covering all of the head)
EEG/ERP data
Measurement of
electrode coordinates
Synthesis of event-related stimuli
(REALTIME)
Subjectinvolvement
Timeline
Observed scalp
potential-field Co-registration of
electrodes & model
FORWARD MODELING
Computation of scalp potential-field
associated with i.e. var. source(s)
INVERSE PROBLEM
Solution for source(s) associated with
the observed scalp potential-field
through optimization
Computation
HPC TOOLS/HARDWARE
(for memory storage and faster
linear system solutions)
1. HEAD MODELING
2. EEG/ERP
(scalp map)
3. COMPUTATION
(localization of source(s))
General Outline of Neuro-electrical Source Imaging (NESI)

5. 1. HEAD MODELING for NESI
STEP 1: K-MEANS ALGORITHM FOR TISSUE SEGMENTATION
V = xj - mi
2
xj ÎSi
å
i=1
k
å
• Initially define k=3 class centroids in gray scale space (Sc,Sk,Br)
REPEAT UNTIL {
• Assign each voxel to the nearest centroid,
• Compute new centroids after the changes,
} (centroids DO NOT change)
(+) Fast, Simple & Suitable to modeling for conductivity (since we know the # of clusters
(- ) Dependence upon initial conditions, not guaranteeing globally minimum variances
(1) Data  k Gaussian mixtures (2) Minimization of inter-cluster variance (3) Identical to PCA
Modality: T1, T2, PD – Slice Thickness: 1 (volumetrical) 3, 5, 7 or 9mm
Noise rate (rel. to brightest tissue): %0, 1, 3, 5, 7 or 9 -- Intensity non-uniformity ("RF"): %0, 20 or 40
Validation using
BrainWeb Application
http://www.bic.mni.mcgill.ca
McConnell Brain Imaging Centre
Montréal Neurological Institute,
McGill University

6. Tissue classification: K-Means Segmentation + 3D Flood-
filling
1. HEAD MODELING for NESI

7. 3D Parametric curve
Energy functional to be minimized
Elasticity/Rigidity
Potential Field (to lead model on boundary)
I(x,y,z): Volume consisting edges
Gs : 3D Gaussian function
Generation of BEM Model: Gradient Vector Flow (GVF) Deformable Models
Most naïve approach is to generate a Gaussian
field in proximity of boundary such as:
Deformable Model
Surface Tension
1. HEAD MODELING for NESI

8. 3D Parametric curve
Energy functional to be minimized
Elasticity/Rigidity Surface Tension
Potential Field (to lead model on boundary)
Generation of BEM Model: Gradient Vector Flow (GVF) Deformable Models
GVF performs better in
penetrating into cavities
Deformable Model
3.Gradient Vector Flow (GVF) (Xu and Prince, IEEE Trans Image Process, 1998)
1. HEAD MODELING for NESI

9. How to solve?
(Equilibrium of internal and external forces on model)
(Euler-Lagrange Equation)
Friction term/viscosity coeff.
Mass (=0)
Numerical Solution: Change into dynamic form [ R(s) R(s,t), t  ],
then solve by Finite Difference Method)
Generation of BEM Model: Gradient Vector Flow (GVF) Deformable Models
3D Parametric curve
Energy functional to be minimized
1. HEAD MODELING for NESI

10. Generation of BEM Model: Gradient Vector Flow (GVF) Deformable Models
(z-component of GVF vector field)(edges)
1. HEAD MODELING for NESI

11. Generation of BEM Model: Gradient Vector Flow (GVF) Deformable Models)
1. HEAD MODELING for NESI

12. Modeling of cranial tissue geometry
(i.e. boundaries of scalp, skull & brain)
MRI data
(dense, covering all of the head)
EEG/ERP data
Measurement of
electrode coordinates
Synthesis of event-related stimuli
(REALTIME)
Subjectinvolvement
Timeline
Observed scalp
potential-field Co-registration of
electrodes & model
INVERSE PROBLEM
Solution for source(s) associated with
the observed scalp potential-field
through optimization
Computation
HPC TOOLS/HARDWARE
(for memory storage and faster
linear system solutions)
1. HEAD MODELING
2. EEG/ERP
(scalp map)
3. COMPUTATION
(localization of source(s)) FORWARD MODEL
Computation of scalp potential-field
associated with i.e. var. source(s)
General Outline of Neuroelectrical Source Imaging (NESI)

13. OCTOPUS EEG Front-end - Continuous EEG w/ events on Monitor #1
2. EEG/ERP – Recording the scalp potential field

14. 2. EEG/ERP – Electrode locations
OCTOPUS EEG Front-end - ERPs/Elec-coords/Gizmos on 2nd Monitor

15. OCTOPUS EEG Front-end - Easy measurement of electrode positions over “Gizmo’s”
Polhemus 3D Digitizer
2. EEG/ERP – Electrode locations

16. OCTOPUS EEG Front-end - Closed-loop head stabilization during measurement
2. EEG/ERP – Electrode locations

17. OCTOPUS EEG Front-end - ERPs/Elec-coords/Gizmos on 2nd Monitor
2. EEG/ERP – Electrode locations

18. OCTOPUS EEG Front-end - Easy measurement of electrode positions over “Gizmo’s”
2. EEG/ERP – Electrode locations

19. What is realtime?
Interrupts
OS Kernel
User
process
User
process
User
process
Hardware Abstraction Layer
System Hardware
Real-time
task
Real-time
task
Real-time
task
User
process
APII/O
Real-time Scheduler
Direct access
to Hardware
Interrupts
OS Kernel
User
process
User
process
User
process
System Hardware
User
processAPII/O
Non-realtime-scheduler
Interrupts
Non-realtime
Realtime
(RTAI nano-kernel)
2. EEG/ERP – Real-time Stimulation

20. The «directional hearing» study:
“Are bilateral dipoles for the IID and ITD are different?”
IID: Interaural Intensity Diff,, ITD: Interaural Time Diff.
Creation of dichotic IID & ITD stimuli over a 44100/s real-time loop.
2. EEG/ERP – Real-time Stimulation

21. L/R **Isolated** Audio Amplifier design for the presentation of stimuli
2. EEG/ERP – Real-time Stimulation

22. ERPs for IID L/R & ITD L/R on OCTOPUS EEG Front-end
2. EEG/ERP – Directional Auditory ERPs

23. Monaural Binaural (dichotic)
2. EEG/ERP – Directional Auditory ERPs

24. Modeling of cranial tissue geometry
(i.e. boundaries of scalp, skull & brain)
MRI data
(dense, covering all of the head)
EEG/ERP data
Measurement of
electrode coordinates
Synthesis of event-related stimuli
(REALTIME)
Subjectinvolvement
Timeline
Observed scalp
electric-field Co-registration of
electrodes & model
FORWARD MODELING
Computation of scalp electric-field
associated with i.e. random source(s)
INVERSE PROBLEM
Solution for source(s) associated with
the observed scalp electric-field
through optimization
Computation
HPC TOOLS/HARDWARE
(for memory storage and faster
linear system solutions)
1. HEAD MODELING
2. EEG/ERP
(scalp map)
3. COMPUTATION
(localization of source(s))
General Outline of Neuroelectrical Source Imaging (NESI)

25. 2. EEG/ERP – Co-registration of electrode and model spaces
Projection of electrodes over scalp triangles

26. Modeling of cranial tissue geometry
(i.e. boundaries of scalp, skull & brain)
MRI data
(dense, covering all of the head)
EEG/ERP data
Measurement of
electrode coordinates
Synthesis of event-related stimuli
(REALTIME)
Subjectinvolvement
Timeline
Observed scalp
potential-field Co-registration of
electrodes & model
Computation
HPC TOOLS/HARDWARE
(for memory storage and faster
linear system solutions)
3. COMPUTATION
(localization of source(s)) FORWARD MODEL
Computation of scalp potential-field
associated with i.e. var. source(s)
INVERSE PROBLEM
Solution for source(s) associated with
the observed scalp potential-field
through optimization
General Outline of Neuroelectrical Source Imaging (NESI)

27. PARAMETRIC
3. COMPUTATION (localization of sources)
Parametric
Modeling for BEM
Geometric
Modeling for
BEM
(Legendre Polynomials)
Electric potential on
surface
Electric potential on
surface

28. Solving for Lead Matrix Inverse (A-1) on a Beowulf Cluster (MPI)
• Gauss-Seidel (row writes in each iteration - not parallelizable)
• Jacobi (bad convergence rates)
• Krylov Subspace Methods (KSMs) specifically Generalized Minimum Residuals
(GMRES) (PETSc)  (Perfectly parallelizable + fast convergence)
Triangulated
Head Model
Electrode
triangle offsets
after
coregistration
“octopus-fc”
Pre-computes IPA and
Accelerated BEM Matrices in
parallel
for faster processing of
Inverse Problem
(Ataseven, et al., 2008)
Accelerated BEM Matrix
(nelec X # of elements)
3. COMPUTATION (localization of sources)

29. Inverse Solutions (“octopus-fe”)
• Fast forward-computation engine calls from GNU/Octave, which transparently
distributes workload among Beowulf nodes.
• Down-hill Simplex (Nelder-Mead algorithm) was used for error function minimization
• RDM between measured and forward computed potentials is used as the error function.
Search dipoles bilaterally in
|x|=5cm, r=2cm,
with all 6-degrees of
freedom
3. COMPUTATION (localization of sources)

30. Octopus – Main Hardware Model for developers

31. Octopus System v0.9.5
STIM
ACQ
OP
Monitor #2
Monitor #1
Audio Amp
Buffered EEG Input
Beowulf cluster
Octopus – Main Hardware Model for developers

32. 128-ELECTRODE EEG AMPLIFIER (Designed & Developed at HU between 1991-1997)
• 128 channels, test input
• CMRR > 80dB
• HPF=0.3Hz, LPF=70Hz
• Either battery or isolated
AC powered operation
Octopus – Main Hardware Model for developers

33. ACQ & STIM Servers
• Debian GNU/Linux 5.0 (Lenny)
• RTAI (Real-time Application Interface)
patched over Linux kernel 2.6)
• COMEDI (Control & Measurement
Device Interface)
GUI Workstation
• Debian GNU/Linux 5.0 (Lenny)
• Linux Kernel 2.6
Octopus – Main Software Model for developers

34. PhD Thesis in Biophysics (2005-2009)
”Finding out the cortical equivalent neuroelectrical «sources» of
«directional hearing» ”:
• EEG, equivalent dipole(s), modeling of realistic head geometry and
conductivity, forward modeling of tissue conductivity, optimization (i.e.
inverse problem solutions), optimal # of sensors, etc.
”Holistic low-cost networked source localization system with Beowulf
parallel computer interface, and a framework for real-time stimulus
creation & electrode measurement with no head restraining” (2009)
• Truely open-source, GNU-style «free-software» (i.e. GPLv3)
• Proper OS, library, etc. choices w/o any constraint as possible; aiming
elegance and quality, not quantity (of community, developers, ...) i.e.
Debian/GNU Linux, NFS, NIS, BIND, C, RTAI, COMEDI, C++, Qt,
OpenGL, OpenCL, OpenMPI, PetSc, NFS, NIS, BIND, …
• Real-time (@ it’s hardest) for neuro-feedback, making possible even
closed-loops for low-level brain structures
• Network-based, distributed, scalable nature, high-throughput, high-yield,
high-availability, etc.
Short history of Octopus-ReEL

35. PhD Thesis in Biophysics (2005-2009)
”Finding out the cortical equivalent neuroelectrical «sources» of
«directional hearing» ”:
• EEG, equivalent dipole(s), modeling of realistic head geometry and
conductivity, forward modeling of tissue conductivity, optimization (i.e.
inverse problem solutions), optimal # of sensors, etc.
”To design the most generalized gold standard” framework ” (2018+):
• Interface with any complementary brain imaging modality, s.a. EEG-fMRI
• Brain mapping and deep neural-net weighing cortical connectivity
Possible use for BCI and neuro-feedback w/ possibility to shrink (i.e.
embedded, nano-scale, FPGA, etc.)
• Proper API creation
• ”precursor” design for HPC-ready architecture for machine learning
(leading to BCI)
• Scripting support for several task subsystems new lang for stim, batch
procesing via Python (e.g. Some «RSPL»: Realtime Stimulus
Presentation Language (or abstraction of psychtoolbox on Octave with
other modality extensions)
«Long» history of Octopus-ReEL

36. PhD Thesis in Biophysics (2005-2009)
”Finding out the cortical equivalent neuroelectrical «sources» of
«directional hearing» ”:
• EEG, equivalent dipole(s), modeling of realistic head geometry and
conductivity, forward modeling of tissue conductivity, optimization (i.e.
inverse problem solutions), optimal # of sensors, etc.
”To design the most generalized gold standard” framework ” (2018+):
• Subject-space normalization for precise anatomical interpretation of
sources (i.e. MNI warping).
• Incorporation of a photogrammetric methods; will decrease costs
eliminating the need for Polhemus FasTrak,
• Different deformable model algorithms and optimizations on the
parameters of 3D GVF, specifically for BEM/FEM model generation,
«Long» history of Octopus-ReEL

37. Assoc. Prof. Süha YağcıoğluProf. Pekcan Ungan
My PhD Supervisors
Thank you!
(ad honorum)