BRAIN COMPUTER INTERFACEAbstract: Brain–computer interfaces (BCIs) enable users to control devices withelectroencephalographic (EEG) activity from the scalp or with single-neuron activity from withinthe brain. Both methods have disadvantages: EEG has limited resolution and requires extensivetraining, while single-neuron recording entails significant clinical risks and has limited stability.We demonstrate here for the first time that electrocorticographic (ECoG) activity recorded fromthe surface of the brain can enable users to control a one-dimensional computer cursor rapidlyand accurately. We first identified ECoG signals that were associated with different types ofmotor and speech imagery. Over brief training periods of 3–24 min, four patients then used thesesignals to master closed-loop control and to achieve success rates of 74–100% in a one-dimensional binary task. Our results suggest that an ECoG-based BCI could provide for peoplewith severe motor disabilities a non-muscular communication and control option that is morepowerful than EEG-based BCIs and is potentially more stable and less traumatic than BCIs thatuse electrodes penetrating the brain. In additional open-loop experiments, we found that ECoGsignals at frequencies up to 180 Hz encoded substantial information about the direction of two-dimensional joystick movements.
INTRODUCTION Man machine interface has been one of the growing fields of research anddevelopment in recent years. Most of the effort has been dedicated to the design of user-friendlyor ergonomic systems by means of innovative interfaces such as voice recognition, virtualreality. A direct brain-computer interface would add a new dimension to man-machineinteraction. A brain-computer interface, sometimes called a direct neural interface or a brainmachine interface, is a direct communication pathway between a human or animal brain(or braincell culture) and an external device. In one BCIs, computers either accept commands from thebrain or send signals to it but not both. Two way BCIs will allow brains and external devices toexchange information in both directions but have yet to be successfully implanted in animals orhumans. Brain-Computer interface is a staple of science fiction writing. In its earliestincarnations no mechanism was thought necessary, as the technology seemed so far fetched thatno explanation was likely. As more became known about the brain however, the possibility hasbecome more real and the science fiction more technically sophisticated. Recently, thecyberpunk movement has adopted the idea of jacking in, sliding biosoft chips into slotsimplanted in the skull(Gibson,W.1984).Although such biosofts are still science fiction, therehave been several recent steps toward interfacing the brain and computers. In this definition, theword brain means the brain or nervous system of an organic life form rather than the mind.Computer means any processing or computational device, from simple circuits to silicon chips(including hypothetical future technologies like quantum computing). Research on BCIs hasbeen going on for more than 30 years but from the mid 1990’s there has been dramatic increaseworking experimental implants. The common thread throughout the research is the remarkablecortical-plasticity of the brain, which often adapts to BCIs treating prostheses controlled byimplants and natural limbs. With recent advances in technology and knowledge, pioneeringresearches could now conceivably attempt to produce BCIs that augment human functions ratherthan simply restoring them, previously only the realm of science fiction. Fig. 1.1: Schematicdiagram of a BCI system
Brain Computer Interface What is a Brain Computer Interface? As mentioned in the preface a BCI represents a directinterface between the brain and a computer or any other system. BCI is a broad concept andcomprehends any communication between the brain and a machine in both directions: effectivelyopening a completely new communication channel without the use of any peripheral nervoussystem or muscles. In principle this communication is thought to be two way. But present day BCI is mainlyfocusing on communication from the brain to the computer. To communicate in the otherdirection, inputting information in to the brain, more thorough knowledge is required concerningthe functioning of the brain. Certain systems like implantable hearing-devices that convert sound
waves to electrical signal which in turn directly stimulate the hearing organ already exist today.These are the first steps. The brain on the other hand is on a whole other complexity levelcompared to the workings of the inner ear. From here on the focus is on communication directly from the brain to the computer.Most commonly the electrical activity (fields) generated by the neurons is measured, thismeasuring technique is known as EEG (Electroencephalography). An EEG-based BCI systemmeasures specific features of the EEG-activity and uses these as control signals. Over the past 15 years the field of BCI has seen a rapidly increasing development rateand obtained the interest of many research groups all over the world. Currently in BCI-researchthe main focus is on people with severe motor disabilities. This target group has little (other)means of communication and would be greatly assisted by a system that would allow control bymerely thinking. Basic BCI layout.The concept of thinking is perhaps too broad a concept and can actually better be replaced bygenerating brain patterns. The general picture of a BCI thus becomes that the subject is activelyinvolved with a task which can be measured and recognized by the BCI. This task consists of thefollowing: evoked attention, spontaneous mental performance or mental imagination. The BCIthen converts the ‘command’ into input control for a device (see figure 1.1). This is the basic idea. With the continuously increasing knowledge of the brain andadvances in BCI over time, perhaps BCI will be able to extract actual intentions and thoughts.This however does not appear to be on the cards for the very near future.
The nature of EEG The reasons for selecting EEG as a measurement method of brain activity are based on theease of appliance, portability, excellent time resolution and the financial picture. From all theoptions available, ranging from among others: MRI, PET, MEG to EEG, EEG is the cheapestvariant and requires neither professional training nor the personnel to apply it. It consists of a capof simple electrodes (10-20 system consisting of over 20 electrodes is used at the TUD, seefigure 1.2) covering the cortex of the brain on the scalp. One of the main advantages of EEG is that it gives an excellent temporal resolution(milliseconds range); any change in brain dynamics will be registered almost instantaneous. Onthe other hand the biggest disadvantage compared to other methods is the very poor spatialresolution (centimeter range), which makes it hard to locate the exact location of the activity. Aside from the fact that the skull causes spatial smearing of the signal, two third of anyactivity generated by the neurons is lost due to misalignment of the firing neurons and the factthat any activity can only be measured on the surface of the cortex, which leaves out the majorityof the neurons, since the voltages being measured are extremely low. The EEG and therefore the combined activity from the neurons are characterized by highvariability in the signal and large amounts of noise and artifacts. This does not only impose theneed for heavy data processing but also makes it more difficult to predict and model the signal.
Basic BCI elements BCI consists of several distinct elements (see figure 1.1). All these elements combinedmake up the BCI. Basically the system consists of two adaptive controllers; on the one handthere is the user that generates the commands via electrophysiological input to the BCI. And onthe other hand the BCI that recognizes the commands and expresses these as output devicecontrols. In this BCI overview each element will be explained to such a level that the reader knowswhat its functions are and why it is required. As well as views on different approaches toproblems that arises in designing and implementing a BCI.BCI can be decomposed into four basic elements: The input, measuring the activity from the human brain. The signal pre-processing, the first step of acquiring usable signals by amplification, applying filters and reducing noise and artifacts of the input signals. The translation algorithm, this step compromises feature extraction which extracts the most valuable signals from the processed input. And feature classification which tries to classify the features into usable output for the next step. The output, from the classification is used as a control signal for various applications.1.1. The input The goal is to acquire knowledge of the intentions of the user either consciously orunconsciously by means of measurement of brain activity. This goal can be achieved in variousways, but it all starts with the brain and thus with the most basic element of the brain: the neuron. 1.1.1. The neuron A neuron is a cell that uses biochemical reactions to receive process and transmitinformation. It consists of the cell body (Soma) in which the cell core (Nucleus) resides (seefigure 2.1). Each neuron has one axon; this is a long ‘cable’-like part of the neuron which is usedto reach other neurons. The soma of a neuron is branched out into dendrites to which axon-endsfrom other neurons connect.
The dendrites are not in actual physical contact with the axons of other neurons; a small cleftexists between them: the synaptic gap. This is the location where the impulse is transferred. Overview of the neuronWhen a neuron fires, it sends signals to all the neurons that are connected to its axon via thedendrites. The dendrites can be connected to thousands of axons; all incoming signals combinedare added through spatial and temporal summation. If the aggregate input reaches a certainthreshold, the neuron will fire and send a signal along its own axon. The strength of this outputsignal is always the same, no matter the magnitude of the input.This single signal of a neuron is very weak. The numerous neurons in the brain are constantlyactive. The generated activity can be measured. It appears to be impossible to measure theindividual activity of every neuron. Moreover it is questionable whether it would be a real gain,since neurons work in groups to achieve a certain goal. The activity from a group of neuronshowever can be measured. For the signals of neurons to be visible using EEG in particular, acouple of conditions need to be met, which are summarized schematically in figure 2.2: The electrical activity of the neuron must be perpendicular to the scalp in order for the EEG to fully pick up the field. A large number of neurons must fire parallel to each other. The neurons must fire in synchrony with the same polarity, in order not to cancel each other out.
Cross-cut of the head: only the green neuronal activity can be measured using EEG.Because of these requirements the larger part of the total neuronal activity remains invisible forEEG measurement. 1.1.2. The brainCombining about 100 billion neurons results in what is called the human brain.The brain consists of the following elementsThe brainstem is an important relay station. It controls the reflexes and automatic functions, likeheart rate and blood pressure and also sleep control. The Cerebellum integrates information about position and movement from the vestibular system to coordinate limb movement and maintaining equilibrium. Mid-brain: amongst others the Hypothalamus and pituitary gland control visceral functions, body temperature and behavioral functions like, the body’s appetite, sleep patterns, the sexual drive and response to anxiety, aggression and pleasure. The Cerebrum (or cerebral cortex) receives and integrates information from all of the sense organs and controls the motor functions. Furthermore it contains the higher cerebral functions like: language, cognitive functions and memories. Emotions are also processed in the cerebrum.
Brain overview, the Hypothalamus is localized in the center of the brainand not depicted here; image from “Heart and Stroke Foundation”.The cortex of the cerebrum is part of the brain which is of the most interest for BCI. It isresponsible for the higher order cognitive tasks and is near the surface of the scalp. In additionthat functionality in the brain appears to be highly local. The cerebrum is divided into two hemispheres, left and right. The left halve senses andcontrols the right half of the body and vice versa. Each hemisphere can be divided into fourlobes, the frontal, the parietal, the occipital and the temporal (see figure 2.3). The cortex can alsoby divide in certain areas each of which is specialized for a different function. Especially thesensorimotor cortex is important for BCI. Over this part the entire human body is represented.The size of area corresponds with the importance and complexity of movement of that particularbody Homunculus: body drawn over sensorimotor cortex
In the light of BCI it is important to know in advance in which area to search for activity bothspatially and in the frequency domain.Brain activity measurementTo measure activity in the brain, several different approaches can be applied. Because differentphenomena can be measured in different ways: ranging from direct measures like detecting theelectrical currents or magnetic fields to indirect measures like measuring metabolism or bloodflow. Overview of the spatial/time resolution of various measurement techniques. Here follows a list of the most commonly used methods (see figure 2.5 for an overview ofthe spatial and time resolution of the mentioned methods). EEG, Electroencephalography involves recording the (very weak) electrical field generated by action potentials of neurons in the brain using small metal electrodes. The advantages of EEG are the high temporal resolution and (the possibility of) non-invasive measurement. Low spatial resolution, caused by spatial smearing of the skull and high variability in the EEG signal are disadvantages. ECoG, Electrocorticography involves the electrophysiology of extra-cellular currents. Has both high temporal as good spatial resolution. It is a form of invasive EEG where electrodes are placed directly on the brain. This technique is invasive and therefore requires expensive surgery and comes with significant safety risks for the patient.
PET, Positron Emission Tomography indirectly measures metabolism on a cellular levelby tracking injected radioactive isotopes. It is based on the principle that in areas ofincreased activity the metabolism is on a higher level and more isotopes are supplied bythe blood flow. This knowledge can be used to determine which areas are generatingactivity. Good spatial resolution is an advantage of PET. The really bad temporalresolution (about 2 minutes) is a distinct disadvantage. This is due to the fact thatmetabolism is a relatively slow process. Moreover ionizing radiation makes this methodharmful for the human body and thus unusable for applications like BCI.MEG, magneto encephalography directly measures the cortical magnetic fields producedby electrical currents. This method is non-invasive and has good spatial and temporalresolution. However the equipment is extremely expensive and due to the very weakmagnetic fields requires a very impractical isolation/shielding room (cage of Faraday).The real-time properties for analysis are poor.fMRI, functional Magnetic Resonance Imaging provides information on brain metabolismusing BOLD (Blood Oxygen Level Dependent). fMRI uses strong fluctuating magneticfield to measure the whereabouts of dipoles . The advantages are good spatialresolution and the non-invasiveness. But the temporal resolution is poor (about onesecond) and this method requires very expensive equipment and procedures.NIR, Near-infrared light penetrates the human head to sufficient depths to allowfunctional mapping of the cerebral cortex. Changes in tissue oxygenation causemodulation of absorption and scattering of photons, which can be measured . NIRcan measure two responses. o The slow response corresponds with the attenuation changes due to cerebral heamodynamics. o The fast response could be used for BCI and is due to changes in scattering properties. This response has a good temporal resolution, but is not yet feasible.To date the only experimental setup that has been used uses the slow response, which hasa poor temporal resolution. Advantages of optical techniques: no interference, user iselectrically isolated, non-invasive, non-ionizing and no gel is required. Overall this lookslike a promising technique.
1.1.3. EEG 126.96.36.199. Selecting a measurement method – Why EEG?The best brain measurement method would have a high spatial and temporal resolution, be verycheap, portable and easy to apply non-invasively. This method does not (yet) exists. Of all methods listed in the previous section, EEG is by far the most commonly used inBCI. The prime reason for this is the excellent temporal resolution which is a necessity for real-time BCI. And although the spatial data resulting from EEG is often distorted and far fromperfect, EEG offers direct functional correlation of brain activity. Another major plus is the ease of applying this method. With a cap containing only a fewelectrodes measurements can start. For practical uses and applications it is small and relativelyportable, which improves prospects of future applications Aside from the ease of appliance, thisis also a relatively low-cost method, certainly compared to methods like PET, MEG or MRI,which require expensive equipment and skilled professionals to operate. Subject one with EEG cap. Although EEG is the most commonly used, this does not mean that others methods arenot feasible. With the continuous improvement of the techniques involved, they can become aviable option in the future; like for instance Near Infrared measurement.
EEG comes in two flavors; the most commonly used in BCI is the non-invasive variant. Theelectrode is placed on the scalp. The obvious advantage is that it can be safely applied to anyoneat any moment without a lot of preparation. The second variant is the invasive EEG. Instead of attaching the electrode on the skull, itis placed inside. The advantage of this variant is the higher spatial resolution obtained by it. Withnon-invasive EEG, the skull causes significant spatial smearing of the measured activity: leadingto more difficult localization of the original signal, which degrades the quality of the signal. Invasive electrodes offer the possibility to locate activity far more precise. The obviousdrawback is that surgery is required to implant the electrodes. This comes with safety risks andhigh costs compared to non-invasive EEG. For application there must be large gain from theincreased accuracy to validate invasive EEG on human subjects.Brain Gate Dummy unit illustrating the design of a Brain Gate interface Brain Gate is a brain implantsystem developed by the bio-tech company Cyberkinetics in 2003 in conjunction with theDepartment of Neuroscience at Brown University. The device was designed to help those whohave lost control of their limbs, or other bodily functions, such as patients with amyotrophiclateral sclerosis (ALS) or spinal cord injury. The computer chip, which is implanted into thebrain, monitors brain activity in the patient and converts the intention of the user into computercommands. Currently the chip uses 100 hair-thin electrodes that sense the electromagneticsignature of neurons firing in specific areas of the brain, for example, the area that controls armmovement. The activities are translated into electrically charged signals and are then sent anddecoded using a program, which can move either a robotic arm or a computer cursor. Accordingto the Cyberkinetics website, three patients have been implanted with the Brain Gate system.The company has confirmed that one patient (Matt Nagle) has a spinal cord injury, whilstanother has advanced ALS. Division of Computer Science and Engineering, SOE, CUSAT 19Brain Computer Interface In addition to real-time analysis of neuron patterns to relay movement,the Brain gate array is also capable of recording electrical data for later analysis. A potential useof this feature would be for a neurologist to study seizure patterns in a patient with epilepsy.Cyberkinetics has a vision, CEO Tim Surgenor explained to Gizmag, but it is not promising"miracle cures", or that quadriplegic people will be able to walk again - yet. Their primary goal is
to help restore many activities of daily living that are impossible for paralysed people and toprovide a platform for the development of a wide range of other assistive devices. "Todayquadriplegic people are satisfied if they get a rudimentary connection to the outside world. Whatwere trying to give them is a connection that is as good and fast as using their hands. Weregoing to teach them to think about moving the cursor using the part of the brain that usuallycontrols the arms to push keys and create, if you will, a mental device that can input informationinto a computer. That is the first application, a kind of prosthetic, if you will. Then it is possibleto use the computer to control a robot arm or their own arm, but that would be down the road."Existing technology stimulates muscle groups that can make an arm move. The problemSurgenor and his team faced were in creating an input or control signal. With the right controlsignal they found they could stimulate the right muscle groups to make arm movement. "Another application would be for somebody to handle a tricycle or exercise machineto help patients who have a lot of trouble with their skeletal muscles. But walking, I have to say,would be very complex. There are a lot of issues with balance and thats not going to be an easything to do, but it is a goal." Cyberkinetics hopes to refine the Brain Gate in the next two years todevelop a wireless device that is completely implantable and doesnt have a plug, Division ofComputer Science and Engineering, SOE, CUSAT 20Brain Computer Interface making it saferand less visible. And once the basics of brain mapping are worked out there is potential for awide variety of further applications, Surgenor explains. "If you could detect or predict the onsetof epilepsy that would be a huge therapeutic application for people who have seizures, whichleads to the idea of a pacemaker for the brain. So eventually people may have this technology intheir brains and if something starts to go wrong it will take a therapeutic action. That could beavailable by 2007 to 2008." Surgenor also sees a time not too far off where normal humans areinterfacing with Brain Gate technology to enhance their relationship with the digital world - iftheyre willing to be implanted. "If we can figure out how to make this device cheaper, theremight be applications for people to control machines, write software or perform intensiveactions. But thats a good distance away. Right now the only way to get that level of detail fromthese signals is to actually have surgery to place this on the surface of the brain. Its not possibleto do this with a non-invasive approach. For example, you can have an EEG and if youconcentrate really hard you can think about and move a cursor on a screen, but if someone makesa loud noise or you get interrupted, you lose that ability. What were trying to make here is a
direct connection. The [Brain Gate] is going to be right there and you wont have to think aboutit."DARPA The Brown University group was partially funded by the Defense Advanced Research ProjectsAgency (DARPA), the central research and development organization for the US Department ofDefense (DoD). DARPA has been interested in Brain-Machine- Interfaces (BMI) for a numberof years for military applications like wiring fighter pilots directly to their planes to allowautonomous flight from the safety of the ground. Future developments are also envisaged inwhich humans could download memory implants for skill enhancement, allowing actions to beperformed that have not been learned directly.BCI APPLICATIONS:After we go through the various techniques of BCI the first question that comes to our mind is,what does BCI do to us and what are its applications. So BCI in today’s time turns useful to us inmany ways. Whether it is in any medical field or in a field leading to enhancement of humanenvironment. Some of the BCI applications are discussed below. 1) Adaptive BCI for Augmented Cognition and Action 2) BCI offers paralyzed patients improved quality of life 3) The Mental Typewriter Corresponding real-time adaptive interfaces with sub-second latency are being designed toevaluate this concept of an adaptive brain-computer interface in three specific applications 1) Error and conflict perception 2) Working memory encoding. 3) Rapid visual recognition
Experimental Brain Computer Interface Software for the Modular EEG (The ABI software)Introduction: ABI is simple software for the Modular EEG that implements an experimental BrainComputer Interface (BCI). Nowadays, BCI research is an highly active field, but the existingtechnology is still immature for its use outside of a labs settings. The ABI software tries toprovide a simple tool for hobbyists to do experiments on its own with BCI technology.Work of the software: The ABI is a BCI based on trials. A trial is a time interval where the user generatesbrainwaves to perform an action. The BCI tries to process this signal and to associate it to agiven class. The association is done by feeding a neural net with the preprocessed EEG data. Theneural nets output is then further processed and this final output corresponds to the given class.The neural net should be trained in order to learn the association. Division of Computer Scienceand Engineering, SOE, CUSAT 30 Brain Computer Interface The classifiers idea is heavilybased on Christin Schäfers design (winner of the BCI Competition II, Motor Imaginery Trials).The ABI software allows you to • Do simple Biofeedback. You can display raw EEG channels,narrow band frequency amplitudes and classes. • Simulate trials. • Record trials for a number ofchoices of different classes. • Train the interface.The classification achieved by this software:The method has been previously applied to the data provided by the BCI Competition II data(dataset III, Graz University, Motor Imaginary) and compared against the results obtained by thecontributors. The method has outperformed the results achieved by them, obtaining a higherMutual Information (which was the criterion used in the competition) of 0.67 bits (the winner ofthe competition obtained (0.61 bits). Of course, it is very important that more people test thesoftware and report its results to improve the method. Statistical stability can only be guaranteedif more people try it out.
CONCLUSION Brain-Computer Interface (BCI) is a method of communication based on voluntary neuralactivity generated by the brain and independent of its normal output pathways of peripheralnerves and muscles. The neural activity used in BCI can be recorded using invasive orNoninvasive techniques. We can say as detection techniques and experimental designs improve,the BCI will improve as well and would provide wealth alternatives for individuals to Interactwith their environment.