2. Repeated and unpredictable seizures
Varying intensities of seizures
Treatment Options
Medical treatment
Resection Surgery
Evaluation
Magnetic Resonance Imaging (MRI) or Computed
Tomography (CT)
Intracranial Monitoring using Electrocorticography
(ECoG) and Electrocoritcal Stimulation Mapping (ESM)
3. Electrocorticography (ECoG)
Used to monitor cerebral activity
Power vs. Time graph
Electrodes implanted subdurally
Electrode Grids and Size
Electrode Grid Implantation
4. Functions correspond to regions
Brain reorganization in epileptic patients
Brain Mapping – Relationship between function and
anatomy of brain
Electrocortical Stimulation Mapping (ESM)
Direct electric stimulation of the cortex
“Gold Standard”
Drawbacks
Invasive
Tedious and time consuming
Risk of unnatural seizures
5. Language Region
Broca’s Area - speaking
Wernicke’s Area – listening
Language Task
Listens to and repeats words
Identified language region in previous
investigations (Towle et al. 2008)
Used for comparison
Suitable format
6. Purpose
The purpose of this experiment is to find a less tedious
and more patient-friendly way to localize language region
Focusing question
"Can the Electrocorticography (ECoG) captured while the patient is
conversing with staff/visitors during the hospital stay be used to
localize language region?"
Hypothesis
ECoG during Natural Conversation will localize the
language region
Language task localized language region
7. Patient’s video recordings
Video Player
Patient’s ECoG data
PC with MS-Windows &
Ubuntu
Computer Applications
EEGvue
Neuroscan 4.3
Loc3D Jr.
EEG View
MS-Excel
Neuroscan
Loc3D Jr.
8. Independent Variable
Type of method used to localize the language region
Dependent Variable
Brain activation (µV)
Constants
Patient
Single hospital stay
Electrode grid
Frequency band range
Lobes observed
Comparison Group
Language task
9. Overview of localization of language region using Natural Conversation
Timestamps for
different states
from the videos
EEG
Vue
Neuroscan
4.3
1 Continuous
file for Listening
Neuroscan
ASCII Data fileLoc3D Jr
Transpose
using
MS-Excel
Neuroscan
ASCII Data file
CT Scan
parameters
INFO file
Electrode
Marker File
CSV File
EEG View
1 Continuous
file for Talking
1 Continuous
file for Resting
1 Continuous
file for Listening
Only
1 Continuous
file for Talking
Only
Electrode
Spatial location
CSV file
Specific
electrodes
AMP file
REG file
Brain model
VTK file
ECoG data
Epoch FileEpoch FileEpoch FileEpoch File
Average File
Epoch FileEpoch FileEpoch FileEpoch FileCContinuous File
Nicolet BMSI file
containing the EEG
segment
<patient>.<state>.<da
te>.<cd#>.<trial#>
Epoch FileEpoch FileEpoch FileEpoch File
Epoch File
CT Scans
10. Operational definitions for each state
Talking
Listening
Resting
ECoG segments from patient’s videos
Load and position data
Compare and contrast brain activation
11. CD # Date CD Start CD End Event Start Event End State Duration
29 7/27/2004 22:44:48 23:24:21 23:10:26 23:10:43 Listening to Conversation 0:00:17
29 7/27/2004 22:44:48 23:24:21 23:11:52 23:11:55 Listening to Conversation 0:00:03
29 7/27/2004 22:44:48 23:24:21 23:12:45 23:12:53 Talking 0:00:08
29 7/27/2004 22:44:48 23:24:21 23:12:58 23:13:10 Talking 0:00:12
28 7/27/2004 22:05:15 22:44:48 22:20:15 22:20:20 Talking 0:00:05
28 7/27/2004 22:05:15 22:44:48 22:22:21 22:22:41 Listening to Conversation 0:00:20
28 7/27/2004 22:05:15 22:44:48 22:24:44 22:25:16 Listening to Conversation 0:00:32
28 7/27/2004 22:05:15 22:44:48 22:25:18 22:25:25 Talking (w/ hand) 0:00:07
28 7/27/2004 22:05:15 22:44:48 22:25:28 22:25:33 Talking (w/ hand) 0:00:05
28 7/27/2004 22:05:15 22:44:48 22:26:49 22:26:51 Talking 0:00:02
27 7/27/2004 21:25:42 22:05:15 22:01:21 22:01:26 Resting (eyes closed) 0:00:05
27 7/27/2004 21:25:42 22:05:15 22:01:44 22:02:36 Resting (eyes closed) 0:00:52
27 7/27/2004 21:25:42 22:05:15 22:02:40 22:03:02 Resting (eyes closed) 0:00:22
27 7/27/2004 21:25:42 22:05:15 22:03:04 22:03:13 Talking 0:00:09
27 7/27/2004 21:25:42 22:05:15 22:03:38 22:03:53 Talking (w/ hand) 0:00:15
27 7/27/2004 21:25:42 22:05:15 22:04:03 22:04:08 Talking 0:00:05
27 7/27/2004 21:25:42 22:05:15 22:04:18 22:04:23 Talking 0:00:05
26 7/27/2004 20:46:09 21:25:42 20:56:16 20:58:27 Listening to Conversation 0:02:11
25 7/27/2004 20:06:36 20:46:09 20:14:06 20:14:22 Resting (eyes open) 0:00:16
25 7/27/2004 20:06:36 20:46:09 20:14:35 20:16:10 Resting (eyes open) 0:01:35
23 7/27/2004 18:47:30 19:27:03 18:51:25 18:54:45 Resting (eyes open) 0:03:20
23 7/27/2004 18:47:30 19:27:03 18:55:21 18:55:25 Listening to Conversation 0:00:04
23 7/27/2004 18:47:30 19:27:03 18:55:43 18:55:49 Listening to Conversation 0:00:06
21 7/27/2004 17:28:24 18:07:57 17:59:53 18:00:00 Listening to Conversation 0:00:07
21 7/27/2004 17:28:24 18:07:57 18:00:03 18:00:16 Listening to Conversation 0:00:13
20 7/27/2004 16:48:51 17:28:24 16:48:53 16:49:14 Resting (eyes open) 0:00:21
20 7/27/2004 16:48:51 17:28:24 16:49:52 16:50:14 Resting (eyes open) 0:00:22
20 7/27/2004 16:48:51 17:28:24 16:50:17 16:52:51 Resting (eyes open) 0:02:34
18 7/27/2004 15:29:45 16:09:18 15:30:30 15:30:32 Talking 0:00:02
18 7/27/2004 15:29:45 16:09:18 15:30:35 15:30:38 Talking 0:00:03
18 7/27/2004 15:29:45 16:09:18 15:31:05 15:31:20 Talking 0:00:15
18 7/27/2004 15:29:45 16:09:18 15:33:23 15:33:27 Listening to Conversation 0:00:04
18 7/27/2004 15:29:45 16:09:18 15:34:11 15:34:16 Listening to Conversation 0:00:05
17 7/27/2004 14:50:12 15:29:45 14:59:49 14:59:57 Listening to Conversation 0:00:08
17 7/27/2004 14:50:12 15:29:45 15:09:24 15:09:26 Listening to Conversation 0:00:02
17 7/27/2004 14:50:12 15:29:45 15:09:51 15:09:56 Listening to Conversation 0:00:05
16 7/27/2004 14:10:39 14:50:12 14:10:51 14:10:57 Listening to Conversation 0:00:06
16 7/27/2004 14:10:39 14:50:12 14:11:11 14:11:20 Talking 0:00:09
16 7/27/2004 14:10:39 14:50:12 14:11:35 14:11:38 Listening to Conversation 0:00:03
The highlighted times are the samples that were averaged and used for comparison. Some
identified samples were not used for multiple reasons. Either they were too short or the patient
was performing another action such as moving his or her hand while talking or listening.
Patient Activity Log
12. Example of Power Spectrum
(Frequency vs. Power)
Power Spectra of all electrodes in
talking after resting was removed
15. Electrode Activation (Natural Conversation vs. Language Task)
Grid Location Electrode #
Natural
Conversation
Language task
Frontal Grid
33 X -
40 - X
44 - X
45 - X
46 - X
53 X X
Parietal/Upper
Temporal Grid
78 X -
79 - X
80 X X
83 X X
84 X -
88 X -
91 X -
92 X -
95 X X
96 X X
Note: ‘X’ in the column indicates Electrode Activation. ‘–‘ indicates absence of
Electrode Activation. Highlighted in orange are the electrodes that were activated
in both Natural Conversation and Language task.
16. Hypothesis not supported
Natural conversation could not localize the same
language region that language task did
However, natural conversation activated more regions in
parietal grid
Not ready for clinical applications
Can be used alongside ESM to refine the
technique
Future work
Automated mapping (Ziegler et al. 2011)
Wireless data transmission
Software automation
17. Difficult to find test subjects
Lateralization of brain function
Right handed with left side grids/left handed with right
side grids
Lack of video-ECoG recordings
Debate over definition of language
Various levels (Poeppel et al. 2012)
18. I would like to thank…
Dr. Vernon Leo Towle, University of Chicago
Falcon Dai & Weili Zheng, University of Chicago
Dr. Judith Scheppler & SIR Department, Illinois
Mathematics and Science Academy
Parents
19. Bauer P., Vansteensel M., Bleichner M., Hermes D., Ferrier C., Aarnoutse E., & Ramsey N (2013). Mismatch between electrocortical
stimulation and Electrocorticography frequency mapping of language. Brain Stimulation. xxx:1-8.
Borchers S., Himmelbach M., Logothetis N., & Karnath H. (2012). Direct electrical stimulation of human cortex- the gold standard for
mapping brain functions? Nat. Rev. Neurosci. 13:63-70.
Carter R., Aldridge S., Page M., & Parker S. (2009). The Human Brain Book: An illustrated guide to its structure, functions, and
disorders. New York, NY: DK Publishing.
Crone N., Sinai A., Korzeniewska A. (2006). High-frequency gamma oscillations and human brain mapping with electrocorticography.
Prog. Brain Res. 159:275-95.
"Epilepsy". Fact Sheets. World Health Organization. October 2012.
<http://www.who.int/mediacentre/factsheets/fs999/en/index.html> Accessed 24 Jul. 2013.
Gaona C.M., Sharma M., Freudenburg Z.V., Breshears J.D., Bundy D.T., Roland J., Barbour D.L., Schalk G., & Leuthardt E.C. (2011).
Nonuniform high-gamma (60–500 Hz) power changes dissociate cognitive task and anatomy in human cortex. J. Neurosci. 31:2091–2100.
Geschwind N. (1970). The Organization of Language and the Brain. Science. 170:940-944.
Penfield W. (1958). Some mechanisms of consciousness discovered during electrical stimulation of the brain. Proceedings of the National
Academy of Sciences of the United States of America. 44(2):51-56
Poeppel D., Emmorey K., Hickok G., & Pylkkanen L. (2012). Towards a new neurobiology of language. J. Neurosci. 32(41):14125-14131.
Ruescher J., Iljina O., Altenmuller D., Aertsen A., Schulze-Bonhage A., & Ball T. (2013). Somatotopic mapping of natural upper- and
lower-extremity movements and speech production with high gamma Electrocorticography. NeruoImage. 81:164-177.
Sinai A., Bowers C.W., Crainiceanu C.M., Boatman D., Gordon B., Lesser R.P., Lenz F.A., & Crone N.E. (2005). Electrocorticographic high
gamma activity versus electrical cortical stimulation mapping of naming. Brain. 128:1556-1570.
Towle V., Yoon H., Castelle M., Edgar J.C., Biassou N.M., Frim D., Spire J., & Kohrman M. (2008). ECoG gamma activity during a
language task: differentiating expressive and receptive speech areas. Brain. 131:2013-27.
Watkins K. & Paus T. (2004). Modulation of Motor Excitability during Speech perception: The Role of Broca’s Area. Journal of Cognitive
Neuroscience. 16(6):978-987.
Wu M.,Wisneski K., Schalk G., Sharma M., Roland J., Breshears J., Gaona C., Leuthardt E.C. (2010). Electrocorticographic frequency
alteration mapping for extraoperative localization of speech cortex. Neurosurgery 66:407–409.
Ziegler J., Kretzschmar H., Stachniss C., Grisetti G., & Burgard W. (2011). Accurate human motion capture in large areas by combining
IMU- and laser-based people tracking. Intelligent Robots and Systems (IROS). 86–91.
Editor's Notes
Epilepsy, a brain disorder which causes repeated and unpredictable seizures, affects about 50 million people around the world. These seizures occur because of disturbances in brain activity. The degree of a seizure varies from patient to patient. Some patients have simple staring spells while others have violent shaking.
Certain types of epilepsy can be treated with medication, while others need surgery. During this process, the neurosurgeon resects the epileptic regions if the function of that region is not important to the patient’s life style. For example, the patient can choose to opt out of the surgery if the epileptic regions are in the area which affect comprehension or movement.
The electrodes which are placed on the cerebrum are laid out as a grid, as shown in the picture to the right. There are different size grids, 8 electrodes by 8 electrodes, 1 by 6, and 1 by 4. Grids are placed on the frontal lobe, the parietal lobe, the temporal lobe, and the occipital lobe. These lobes make up the cerebrum.
Each of these electrodes are connected by a wire to a data acquisition system which collected the brain waves from the patient. These waves are visually represented as Electrocorticography waves as shown to your left by the power spectrum.
Different regions of the brain correspond to various functions. Not all patient’s have the same regions corresponding to the same functions, although they are located in the relative area. Also, patients who have being having seizures for a long time have brains which have reorganized vital tasks. Because of this, each individual's brain must be mapped to find the exact location for each function.
In order to find out this exact location, Electrocortical Stimulation mapping is performed. This technique, discovered by Wilder Penfield, is currently the gold standard in neuroscience. Prior to the technique, the neurosurgeon surgically implants electrodes onto the cerebrum of the brain. Then during the procedure, the surgeon probes the area of the cerebrum underneath each electrode, as shown by the picture to your right to determine the function of that region. For example, when the region of the patient’s brain which controls arm movement is stimulated by the probe, then the patient’s arm will jerk straight up.
There are many drawbacks to this invasive procedure. During the procedure, the patient must be conscious with his or her scalp peeled back, cerebrum bare. Although the patient is under local anesthesia, there is still moderate pain such as severe migraines. Also, the procedure itself is frightening, as the patient watches his or her limbs jerk suddenly. The procedure is also tedious and time consuming. The neurosurgeon has to stimulate each and every one of the electrodes on the patients brain. Also, if the neurosurgeon stimulates an area causing a seizure, this data cannot be used, since it is unnatural. If this occurs, the entire procedure must be stopped, and continued the next day. Because of this, the procedure can take many days.
So what exactly is the language region? Well, our definition of language basically consists of speaking and listening. Speaking is controlled by Broca’s area, located in the frontal lobe and listening is controlled by Wernicke’s area located in the upper temporal and parietal lobes. By using Electrocortical stimulation mapping, neuroscientists have learned where perception, language, memory, emotion, and movement occur in the different regions of the brain and have found Broca’s and Wernicke’s areas in individual patients.
Another term that was mentioned in the focusing question was a language task. During a language task, the patient listens to a series of words and repeats those words. This specific language task was used in previous investigations to find the language region. I used the data from the language task rather than Electrocortical Stimulation Mapping as a reference because it was found that the two provided accurate results and language task data was in a more suitable format.
A downside to using language task to find the language region is its over-simplicity. During the language task, the patient simply repeats words. On the other hand, natural conversation is.. Natural, meaning that it is more similar to real life scenarios. When involved in conversation, there is a certain level of comprehension that takes place as well which is not apparent in a language task. An unnatural environment of a language task also influences the patient’s actions or thoughts as well.
So the purpose of this research is to find a less tedious and more patient-friendly method compared to Electrocortical stimulation mapping in order to find the functions of different regions. For this investigation, the region for talking and listening will be used to determine the entire language region. To see whether analyzing natural conversation can be used as an alternative, I observed how brain activity during natural conversation compares to that during a language task.
The applications that I used were EEGvue, Neuroscan 4.3, Loc3D Jr and EEG View, spelled differently.
So to begin, I first defined each state so that the segments were consistent. Using the video and audio recordings, I was able to determine when the patient was talking, listening, or resting. The resting state was used a baseline. This process was quite tedious, as I had to go through each of the videos which were recorded over a span of 24 hours each day for 2 weeks. Also, I had to ensure that while the patient was speaking, there was no other movement going on which is difficult to control, as it is natural to use hands while speaking. It was also difficult to determine when the patient was resting, or sleeping. By looking at the electrocorticography data, I was able to somewhat distinguish between the two. Sleep could not be used as the baseline because of the various stages of sleep.
After sifting through all the videos, each segment from the patient’s data was saved for each state. After manipulating the format of the files, the data was loaded and position onto a figure of the brain which allowed me to compare and contrast the activation using a visual representation as well.
From the videos and the operational definitions, the start and stop times for each of the states were obtained. Using these time stamps and EEG vue, the ECoG data was segmented for each of the states and saved as a Nicolet BMSI file. Each of these files contains waveforms for all the electrodes in the grid for a specific segment.
Then, the Nicolet BMSI file was transformed into an Epoch file by applying Fast Fourier Transform using Neuroscan 4.3. This changed the file from a time domain to a frequency domain. The Epoch file was then converted into an Average file to create and compare power spectra. Neuroscan 4.3 provides the capability to load multiple files and graph them onto one Power vs. Frequency graph, which was very helpful to compare power spectra of different states. This figure is shown here *click*. Here is the data from one of the specific electrodes. The high gamma range at the end was used.
So to begin, I first defined each state so that the segments were consistent. Using the video and audio recordings, I was able to determine when the patient was talking, listening, or resting. The resting state was used a baseline. This process was quite tedious, as I had to go through each of the videos which were recorded over a span of 24 hours each day for 2 weeks. Also, I had to ensure that while the patient was speaking, there was no other movement going on which is difficult to control, as it is natural to use hands while speaking. It was also difficult to determine when the patient was resting, or sleeping. By looking at the electrocorticography data, I was able to somewhat distinguish between the two. Sleep could not be used as the baseline because of the various stages of sleep.
After sifting through all the videos, each segment from the patient’s data was saved for each state. After manipulating the format of the files, the data was loaded and position onto a figure of the brain which allowed me to compare and contrast the activation using a visual representation as well.
So to begin, I first defined each state so that the segments were consistent. Using the video and audio recordings, I was able to determine when the patient was talking, listening, or resting. The resting state was used a baseline. This process was quite tedious, as I had to go through each of the videos which were recorded over a span of 24 hours each day for 2 weeks. Also, I had to ensure that while the patient was speaking, there was no other movement going on which is difficult to control, as it is natural to use hands while speaking. It was also difficult to determine when the patient was resting, or sleeping. By looking at the electrocorticography data, I was able to somewhat distinguish between the two. Sleep could not be used as the baseline because of the various stages of sleep.
After sifting through all the videos, each segment from the patient’s data was saved for each state. After manipulating the format of the files, the data was loaded and position onto a figure of the brain which allowed me to compare and contrast the activation using a visual representation as well.
On the frontal grid, it was found that during natural conversation, only electrodes 33 and 53 were activated whereas during a language task, more electrodes were activated.
On the parietal grid, more electrodes in addition to the language task electrodes were activated during natural conversation.
Here is a visual representation of this data. On the figure to your left, the brain activation during natural conversation is shown and on the figure to your right, the brain activation during the language task is shown for just the frontal grid where Broca’s area, the area for speech, is located. You can see that there is more activation in the frontal grid during language task when compared to natural conversation.
The figure to your left is the same figure as on the previous slide. This time, the activation during natural conversation is being compared with activation during a language task in the parietal and upper temporal lobes where Wernicke’s area, the area for listening, is located. As you can see, there is much more activation during natural conversation when compared to that of a language task in the parietal and upper temporal lobes.
On the frontal grid, it was found that during natural conversation, only electrodes 33 and 53 were activated whereas during a language task, more electrodes were activated.
On the parietal grid, more electrodes in addition to the language task electrodes were activated during natural conversation.
From these results, it was found that because natural conversation could not identify the entire language region that the language task did, this method cannot be used in place of Electrocortical stimulation mapping and therefore, not ready for clinical applications. Until further investigation is performed on more patients, analyzing natural conversation can be used alongside ESM to refine the technique of analyzing natural conversation.
One of the problems that I came across was that I did not have enough samples. Generally, only epileptic patients are willing to undergo Electrocortical Stimulation mapping because of its invasive nature. I don’t know about you, but I would not want my brain cut open. Specific to the patient whose data I analyzed, the cable connections of the electrodes were switched between the left and right side for the frontal and parietal grids in the beginning. This error was fixed half way through the patient’s stay at the hospital, so this limited the amount of data I could work with as well.
Also, the location of the grids on the cerebrum is important. The patient must be right handed with left side grids or left handed with right side grids because for a right handed person, their Broca’s and Wernicke’s area is on the left side of their brain, and for a left handed person, their broca’s and wernicke’s area is on the right side of their brain.
For some other patients, the video and sound recordings along with the ECoG data was missing, or non existent. These recordings are necessary to accurately identify each state using the proper definitions.
Currently, there is also a debate over what should and should not be considered language because of the various levels and aspects of language processing such as perception, production, and sign language.
In the future, the mapping of language region may be automated. For example, a microphone could be placed on the patient and whenever the patient speaks, the starting and ending times can be recorded automatically. This would facilitate collecting segments of each state efficiently and increase the number of trials.
Radio technology may also be used in the future, where the data from electrodes is sent using a wireless connection. This allows the patient to return home and live comfortably. Also, the electrodes can remain on the cerebrum for a longer period of time, because the main risk of the entire Electrocortical stimulation mapping is infection which is why the patient can only remain in the hospital for maximum 6 weeks.
The technique of analyzing data from the videos of the patient’s hospital stay can be used for states other than talking and listening. Other tasks such as eating are vital to the patient as well and the region which controls these functions are important to know.
I would like to thank my advisor, Dr. Towle, for guiding me with my research project and for providing me with access to the Neurophysiologic Functional Localization Laboratory at the University of Chicago.
I would also like to thank Falcon Dai and Weili Zheng for helping me in understand the various computer applications used.
I would like to thank Dr. Judith Scheppler and the Student Inquiry and Research department for their help and encouragement during my research.
Lastly, I want to thank my parents for supporting me all the way.
