Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

1

Share

Download to read offline

Brain Computer Interface & It's Applications | NeuroSky Minwave | Raspberry Pi

Download to read offline

Brain Controlled Game Simulator
✓ Brain Controlled Robot
✓ Brain Keyboard
✓ Brain Visualization using Open Vibe & Python

Github: https://github.com/vsltech/braingamesimulator

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Brain Computer Interface & It's Applications | NeuroSky Minwave | Raspberry Pi

  1. 1. A PROJECT REPORT ON “Brain Computer Interface & its Applications” Guide- Submitted by- Mrs. Preeti Gupta Vishal Aditya Assistant Professor (ASET) B.Tech (CSE, VIII Semester) AMITY SCHOOL OF ENGINEERING & TECHNOLOGY AMITY UNIVERSITY RAJASTHAN
  2. 2. ii ABSTRACT Brain Computer Interface is a leading technology in today’s world. In simple words, it creates an interface between the brain and the machine, allowing the machine to get input from human brain. This technology has been up in the airs since decades and a lot of work has already been done, but a lot is also remaining. The main use of this technology is in the field of medical science. Works are being done in the field of military and skill improvement domains as well. This technology is growing day by day and is proving to be one of the most important technologies in the current scenario. This project shows how Brain Computer Interfaces (BCI) works and how it can be used in certain applications. EEG (Electroencephalography) was used to read brain waves and then those waves were analyzed to fetch fruitful results which are used in certain applications. The applications which have been demonstrated in this project are Brain Controlled Game Simulator, Brain Controlled Keyboard (for handicapped).
  3. 3. iii D E C L A R A T I O N I hereby declare that the project entitled “Brain Computer Interface & its Applications” submitted for the partial fulfillment of B. Tech Degree is my original work and the project has not formed the basis for the award of any degree, associate ship, fellowship or any other similar titles. Signature of the Student: Vishal Aditya Programme: B. Tech (CSE, VIII Semester) Place: Jaipur Date:
  4. 4. iv ACKNOWLEDGEMENT It is indeed a great pleasure and matter of immense satisfaction for us to express our deep sense profound gratitude towards all the people who have helped, inspired us in our project work. First I would like to give my gratitude to Mrs. Preeti Gupta (Assistant Professor, ASET) for the effort taken by her right from the selection of the project to its completion. She spent her precious time whenever we were in need of guidance. Moreover I would like to thank Prof. (Dr.) D.D.Shukla (Director, ASET) and Prof. (Dr.) Tarun Kumar Sharma (Head Of Department, CSE/IT), who were always there whenever I needed any support and were a constant source of inspiration for accomplishment of this project. Vishal Aditya
  5. 5. v Table Of Contents Title page i Abstract ii Declaration iii Acknowledgment iv Table of Contents v List of Figures vii 1 Introduction 1 1.1 Introduction 2 1.2 Aims & Objective 2 1.3 Brain Computer Interface 3 1.3.1 Invasive BCI 3 1.3.2 Non Invasive BCI 4 1.4 Electroencephalography (EEG) 5 2 Reading Brain Waves 7 2.1 Types Of Waves 8 2.1.1 Gamma Waves (31-50 Hz) 8 2.1.2 Beta Waves (13-30 Hz) 8 2.1.3 Alpha Waves (8-12 Hz) 8 2.1.4 Theta Waves (4-7 Hz) 8 2.1.5 Delta Waves (1-3 Hz) 9 2.2 Comparison Between EEG Bands 9 2.3 BCI System 10 2.3.1 Signal Acquisition 11 2.3.2 Feature Extraction 11 2.3.3. Feature Translation 11
  6. 6. vi 2.3.4 Output Device 11 3 Requirement & Feasibility Analysis 12 3.1 Requirement Analysis 13 3.1.1 Hardware Requirement 13 3.1.2 Software Requirement 13 3.2 Feasibility Analysis 13 3.2.1 Technical Feasibility 13 3.2.2 Economic Feasibility 14 4 Research Methodology 15 4.1 Literature Review 16 4.2 Objective 18 4.3 Algorithm 18 4.3.1 Attention 18 4.3.2 Meditation 18 4.3.3 Blink Detection 19 4.3.4 Mental Effort 19 4.3.5 Familiarity 19 4.3.6 Appreciation 19 4.3.7 Emotional Spectrum 20 4.3.8 Cognitive Preparedness 20 4.3.9 Creativity 20 4.3.10 Alertness 20 5 Working & Applications 21 5.1 EEG & Mindwave 22 5.1.1 The Origin 22
  7. 7. vii 5.1.2 EEG Signals 24 5.1.3 Clinical EEG 25 5.1.4 Emotiv Epoch EEG 26 5.1.5 NeuroSky Mindwave 27 5.2 How Does The Technology Work ? 30 5.3 Applications Of Our Project 31 5.4 How to Setup/Configure 31 5.4.1 Mind Controlled Robot 32 5.4.1.1 How to Do ? 33 5.4.1.2 What is RaspberryPi ? 33 5.4.1.3 Circuit Connection 34 5.4.1.4 Motor Direction Logic 35 5.4.1.5 Software Setup 35 5.4.2 Brain Keyboard 37 5.4.2.1 How to Setup/Configure 38 5.4.3 Data Visualization 38 5.4.3.1 Python 38 5.4.3.2 Open Vibe 39 5.4.3.3 MATLAB 41 5.4.3.4 Visual Studio 43 6 Conclusion 45 6.1 Conclusion 46 6.1.1 Medical Uses of EEG 46 6.1.2 EEG Technology & Research 47 6.1.3 EEG-Enabled Prosthetics 47 6.1.4 EEG-Enabled Communication Devices 48
  8. 8. viii 6.1.5 EEG & Education 49 6.1.6 EEG Entertainment 49 7 Future Scope 50 7.1 BCI – Advantages & Disadvantages 51 7.2 EEG – Future Scope 51 7.3 BCI – Future Scope 51 References 53
  9. 9. vii List Of Figures Sr. no. Figure No. Figure Title Page No 1 Figure 1.2.1 Invasive BCI 4 2 Figure 1.2.2 Non Invasive BCI 5 3 Figure 2.2 EEG Bands of Frequencies 9 4 Figure 2.3 BCI System 10 6 Figure 5.1 Brain Game Simulator Setup 22 6 Figure 5.1.1.1 Human Brain Lobes 23 7 Figure 5.1.1.2 Different layers of the brain cortex 23 8 Figure 5.1.2 Example of EEG Bands 24 9 Figure 5.1.3 Common Areas of Bipolar EEG Sensory 26 10 Figure 5.1.4 Emotiv Epoch EEG NeuroHeadset 26 11 Figure 5.1.5 Mindwave EEG NeuroHeadset 27 12 Figure 5.4 Brain Controlled Car Racing Game 32 13 Figure 5.4.1 Mind Controlled Robot 32 14 Figure 5.4.1.1 Mind Controlled Robot – Hardware Setup 33 15 Figure 5.4.1.2 RaspberryPi 33 16 Figure 5.4.1.3 Circuit Connection 34 17 Figure 5.4.1.4 Booting Raspbian/Noobs 35 18 Figure 5.4.1.5 Installing XRDP 36 19 Figure 5.4.2 Brain Keyboard 37 20 Figure 5.4.3.1 Screenshot of Plot in Python 39 21 Figure 5.4.3.2 Acquisition Server & Clients in Open Vibe 40 22 Figure 5.4.3.2.1 Running & Setting-up Acquisition Server 40 23 Figure 5.4.3.2.2 Acquisition Client & Signal Display Connection 41 24 Figure 5.4.3.3 NeuroSky Mindwave and MATLAB 41 24 Figure 5.4.3.4 Raw Value using Visual Studio 44 25 Figure 6.1.1 EEG Testing 46 25 Figure 6.1.4 Stephen Hawking 48 26 Figure 7.1 Future Scope (Robotic Assistance) 52
  10. 10. viii 27 Figure 7.2 Future Scope (Google Glass & BCI) 53
  11. 11. ASET Brain Computer Interface & its Applications 1 Chapter 1 Introduction
  12. 12. ASET Brain Computer Interface & its Applications 2 1.1 INTRODUCTION The project titled ‘Brain Computer Interface & its Applications’ is processed and designed under multiple softwares, MATLAB and Visual Studio. Core programming languages used are Python and C#. The project aims to deal with creating an interface between human brain and a computer using Electroencephalography (EEG) technique, a technique to read brain waves using sensors that read delta, alpha, beta, gamma & theta waves from the brain, amplifies it and gives the output in form of numerical values to the machine. The users just need to think of movement in order to drive the system. Several brain scanning techniques like fMRI, PET, or EEG are capable to analyze what you are thinking, dreaming, or seeing. EEG device that is used to read the brain waves is NeuroSky Mindwave Headset (MW001). It comprises of a Wi-Fi dongle and two lobes, frontal lobe for forehead and another lobe that connects to left ear. In all, the device can detect and give three values which can be used to develop mind-controlled applications:  Raw Values  Attention Values  Meditation Values  Eye Blink Strength 1.2 AIMS AND OBJECTIVE The core objectives which have been designated as fundamental to the project are: • To acquire EEG signal from BCI technique using NeuroSky Mindwave Headset. Read the brain waves using the EEG device. • Filter those Raw Values and understand the basic functionalities of how those brain waves are giving the particular output as per the reaction of the brain. Using the raw values obtained gathered as a base, we then used it to fetch Attention, Meditation and Eye Blink Strengths values. . • Analyze the values and use them with certain applications to control them by mind, such as Game Control, Keyboard Control for the disabled and show the use of BCI in Military. After the Raw Values are fetched and analyzed, we now need to use them in our applications enabling mind controlled applications.
  13. 13. ASET Brain Computer Interface & its Applications 3 1.3 BRAIN COMPUTER INTERFACE A Brain Computer Interface (BCI) often called a Mind-Machine Interface (MMI) or sometimes called a Direct Neural Interface (DNI), Synthetic Telepathy Interface (STI) or a Brain Machine Interface (BMI) is a direct communication pathway between the brain and an external device. BCIs are often directed at assisting, augmenting or repairing human cognitive or sensory- motor functions. Humans’ brain is filled with neurons, individual nerve cells connected to one another by dendrites and axons. Every action like think, move, feel or remember something make neurons are at work. That work is carried out by small electric signals that zip from neuron to neuron as fast as 250 mph. The signals are generated based on the differences in electric potential carried by ions on the membrane of each neuron. The signals then can be detected, interpreted to what they mean and use them to direct a device of some purpose. Therefore, BCI is a system that provides direct interface between the human brain and the computer. In order to develop the BCI system, the feasible technique should be studied. BCI systems are broadly classified into invasive and non-invasive techniques. 1.3.1 INVASIVE BCI: Invasive BCI are Neuroprosthetics where electrode arrays heads are buried within the brain during neurosurgery and left there on a permanent basis. Invasive devices produce the highest quality signals of BCI device because they lie in the grey matter of brain. They have by far the best signal to noise ratio and accuracy of any BCI method. Unfortunately invasive BCI is costly and require complex surgery to implant. They are require a permanent hole in the skull, build-up prone to scar-tissue, causing the signal to become weaker or even non-existent, as the body reacts to a foreign object in the brain. Electrocorticography (ECoG) is one of the invasive BCI. It also known as partially invasive as the device is implanted inside the skull but rest outside the brain rather than within the grey matter. ECoG is a very promising intermediate BCI modality because it has higher spatial resolution, better signal-to-noise ratio, wider frequency range, and less training requirements than scalp-recorded EEG, and at the same time has lower technical difficulty, lower clinical risk, and probably superior long-term stability than intracortical single-neuron recording. This feature profile shows potential for real world application for people with motor disabilities. Unfortunately ECoG is also costly and required
  14. 14. ASET Brain Computer Interface & its Applications 4 dangerous nature of surgeries for such system. Figure 1.3.1 Invasive BCI 1.3.2 NON INVASIVE BCI: Non-invasive BCI is the most popular technique where the electrodes need to be placed outside of the skull or on the scalp. Non-invasive methods are limited in that they are often susceptible to noise, have worse signal resolution due to distance from the brain, and have difficulty recording the inner workings of the brain. However they have the advantages that can combat these difficulties by lower cost, greater portability and the fact that they do not require any special surgery. Most non-invasive BCI systems use electroencephalogram (EEG) signals. EEG is the first non-invasive neuron imaging technique discovered which is used for measuring the electrical activity of the brain. Besides electrical activity, neural activity also produces other types of signals such as magnetic and metabolic that could be used in a BCI. Magnetic fields can be recorded by using magnetoencephalography (MEG), while brain metabolic activity which is reflected in changes in blood flow can be observed by using positron emission tomography
  15. 15. ASET Brain Computer Interface & its Applications 5 (PET), functional magnetic resonance imaging (fMRI), and optical imaging [1]. Unfortunately, such alternative techniques require sophisticated devices that can be operated only in special facilities. Moreover, techniques for measuring blood flow have long latencies and thus are less appropriate for interaction. Figure 1.3.2 Non-Invasive BCI 1.4 ELECTROENCEPHALOGRAPHY (EEG): EEG is the first non-invasive neuron imaging technique discovered which is used for measuring the electrical activity of the brain. EEG signals are detected from the scalp and contain noise as a result of electrical interference and movement of electrodes. Applying a large number of EEG channels may include noisy and redundant signals that degrade the BCI performance and also involve a prolonged preparation time that directly impacts the convenience in the use of the BCI. Therefore, selecting the least number of channels that yield the best or required accuracy can balance both needs for performance and convenience. Due to its ease of use, cost and high temporal resolution this method is the most widely used one in BCIs nowadays. The advantages of using EEG technique are: a) Hardware costs are significantly lower than those of most other techniques.
  16. 16. ASET Brain Computer Interface & its Applications 6 b) EEG sensors can be used in more places than fMRI, SPECT, PET, MRS, or MEG, as these techniques require bulky and immobile equipment. For example, MEG requires equipment consisting of liquid helium-cooled detectors that can be used only in magnetically shielded rooms, altogether costing upwards of several million dollars and fMRI requires the use of a 1-ton magnet in, again, a shielded room. c) EEG has very high temporal resolution, on the order of milliseconds rather than seconds. EEG is commonly recorded at sampling rates between 250 and 2000 Hz in clinical and research settings, but modern EEG data collection systems are capable of recording at sampling rates above 20,000 Hz if desired. MEG and EROS are the only other non-invasive cognitive neuroscience techniques that acquire data at this level of temporal resolution. d) EEG is silent, which allows for better study of the responses to auditory stimuli. e) EEG does not aggravate claustrophobia, unlike fMRI, PET, MRS, SPECT, and sometimes MEG. f) EEG does not involve exposure to high-intensity (>1 Tesla) magnetic fields, as in some of the other techniques, especially MRI and MRS. These can cause a variety of undesirable issues with the data, and also prohibit use of these techniques with participants that have metal implants in their body, such as metal-containing pacemakers. g) Extremely non-invasive, unlike ECoG which actually requires electrodes to be placed on the surface of the brain. The characteristics of EEG that compare favourably with behavioural testing: a) EEG can detect covert processing (i.e., processing that does not require a response) b) EEG can be used in subjects who are incapable of making a motor response. c) EEG is a powerful tool for tracking brain changes during different phases of life. EEG sleep analysis can indicate significant aspects of the timing of brain development, including evaluating adolescent brain maturation. d) In EEG there is a better understanding of what signal is measured as compared to other research techniques, i.e. the BOLD response in MRI.
  17. 17. ASET Brain Computer Interface & its Applications 7 Chapter 2 Reading Brain Waves
  18. 18. ASET Brain Computer Interface & its Applications 8 2.1 TYPES OF WAVES: You have around 86 billion neurons in your brain which control your whole body and generate thoughts in your head. These neurons fire with certain frequencies. The EEG headset reads these frequencies by reading the electrical activity that the neurons produce. Scientists have agreed to split these frequencies in different frequency bands. All the frequency bands have been studied intensively and are associated with several states of mind [2]:  Delta waves (1-3 Hz): Deepest meditation and deep sleep  Theta waves (4-7 Hz): Normal sleeping and normal meditation  Alpha waves (8-12 Hz): Relaxation/reflection  Beta waves (13-30 Hz): Active thinking, focus, hi alert, anxiety  Gamma waves (31-50 Hz): Conscious perception 2.1.1 GAMMA WAVES (31-50 Hz):  Too much: Anxiety, high arousal, stress  Too little: ADHD, depression, learning disabilities  Optimal: Binding senses, cognition, information processing, learning, perception, REM sleep 2.1.2 BETA WAVES (13-30 Hz):  Too much: Adrenaline, anxiety, high arousal, inability to relax, stress  Too little: ADHD, daydreaming, depression, poor cognition  Optimal: Conscious focus, memory, problem solving 2.1.3 ALPHA WAVES (8-12 Hz):  Too much: Daydreaming, inability to focus, too relaxed  Too little: Anxiety, high stress, insomnia, OCD  Optimal: Relaxation 2.1.4 THETA WAVES (4-7 Hz):  Too much: ADHD, depression, hyperactivity, impulsivity, inattentiveness  Too little: Anxiety, poor emotional awareness, stress
  19. 19. ASET Brain Computer Interface & its Applications 9  Optimal: Creativity, emotional connection, intuition, relaxation 2.1.5 DELTA WAVES (1-3 Hz):  Too much: Brain injuries, learning problems, inability to think, severe ADHD  Too little: Inability to rejuvenate body, inability to revitalize the brain, poor sleep  Optimal: Immune system, natural healing, restorative / deep sleep 2.2 COMPARISON BETWEEN EEG BANDS: The following is a comparison between the different bands of frequencies of Alpha, Beta Gamma, Theta and Delta waves: Figure 2.2 EEG Bands of Frequencies
  20. 20. ASET Brain Computer Interface & its Applications 10 All the frequencies are further divided as follows:  Delta: 1-3Hz  Theta: 4-7Hz  Alpha1: 8-9Hz  Alpha2: 10-12Hz  Beta1: 13-17Hz  Beta2: 18-30Hz  Gamma1: 31-40Hz  Gamma2: 41-50Hz 2.3 BCI SYSTEM: A BCI system consists of three components: Signal or Data Acquisition, Signal Processing (Feature Extraction, Feature Translation), and Output Device. These components are controlled by a protocol which defines the timing for operation, signal processing details, nature of device commands and the performance. Figure 2.3 BCI System
  21. 21. ASET Brain Computer Interface & its Applications 11 2.3.1 SIGNAL ACQUISITION: Signal acquisition in a BCI helps in the measurement of brain signals using a sensor modality. The sensor is basically a device implanted in the brain usually multi-electrode arrays that records the signals directly related to the movement. The signals can be amplified to levels suitable for electronic processing. Also, they can be subjected to filtering to remove electrical noise or other undesirable signals. After amplification and filtering process, the signals can be digitized and transmitted to a computer. 2.3.3 FEATURE EXTRACTION: Feature extraction in Brain Computer Interface (BCI) is the process of analyzing the digital signals to distinguish signal characteristics and represent them in a compact form suitable for translation into output commands. These features been extracted should have good correlations with the users intent. 2.3.3 FEATURE TRANSLATION: Resulting signal features are passed to the feature translation algorithm, which converts the features into the commands for the output device (i.e., commands that accomplish the users need). 2.3.4 OUTPUT DEVICE: The commands from the feature translation algorithm operate the external device of the Brain Computer Interface (BCI), providing functions such as cursor control, letter selection, robotic arm operation etc. The device operation then provides feedback to the user finally, thus completing the closed loop of Brain Computer Interface (BCI).
  22. 22. ASET Brain Computer Interface & its Applications 12 Chapter 3 Requirement & Feasibility Analysis
  23. 23. ASET Brain Computer Interface & its Applications 13 3.1 REQUIREMENT ANALYSIS The requirement gathering process is intensified and focused specifically on input. To understand the nature of the project to be built, the researcher (“analyst”) must understand the information domain for the research, as well as required function, behaviour, performance and interface, Requirement for both the system and the research project are documented and reviewed with the expertise to bias and due to variance [3]. 3.1.1 Hardware Requirement  Processor: Pentium 4 1.6GHz  Memory: 1GB  Hard disk: 200MB  Bluetooth: Bundled USB dongle  Keyboard  Mouse 3.1.2 Software Requirement  Operating System: Windows XP, Vista, or later.  Documentation: Any Office Products  Softwares: MATLAB, Visual Studio  Languages: Python, C#. 3.2 FEASIBILITY ANALYSIS All projects are feasible given unlimited resources and infinite time. Unfortunately the development of a computer-based system is more likely to be plagued by scarcity of resources and difficult delivery dates. It is both necessary and prudent to evaluate the feasibility of a project at the earliest possible time. We concentrate our attention on two primary areas of interest. 3.2.1 Technical Feasibility A study of function, performance and constraints may affect the ability to achieve an acceptable system. This is the most difficult area to access at this stage of the system development process. The consideration that are normally attached with this are: 1. Development risk 2. Resource availability 3. Technology During technical analysis the analyst evaluates technical merits of the system concept, while
  24. 24. ASET Brain Computer Interface & its Applications 14 at the same time collecting information about performance, reliability, maintainability and productibility. 3.2.2 Economic Feasibility An evaluation of development cost weight against the ultimate income or benefit derived from the development system. Development of this Research project is economically feasible and is achievable within the cost limit.
  25. 25. ASET Brain Computer Interface & its Applications 15 Chapter 4 Research Methodology
  26. 26. ASET Brain Computer Interface & its Applications 16 4.1 LITERATURE REVIEW Research work on Brain Computer Interface (BCI) began early in the year 1970 itself. This research work was started at the University of California Los Angeles (UCLA) under the permission from the National Science Foundation after a contract from DARPA. Thanks to the development of brain imaging technologies like EEG, ECOG fMRI, MEG etc. This is because they were one of the major milestones which helped the research workers to continue their work on BCI. At first, BCI work was carried on many animals and today it is tested on human beings too. Today, one of the major goal of BCI research is to develop applications that helps disabled people (suffering from brainstem stroke, spinal cord injury, blindness etc.) to communicate and interact with people and external environments. Recently, brain-computer interfaces (BCIs) based on visual evoked potentials (VEPs) have been shown to achieve remarkable communication speeds. As they use electroencephalography (EEG) as non-invasive method for recording neural signals, the application of gel-based EEG is time-consuming and cumbersome. In order to achieve a more user-friendly system, this work explores the usability of dry EEG electrodes with a VEP-based BCI. While the results show a high variability between subjects, they also show that communication speeds of more than 100 bit/min are possible using dry EEG electrodes. To reduce performance variability and deal with the lower signal-to-noise ratio of the dry EEG electrodes, an averaging method and a dynamic stopping method were introduced to the BCI system. Those changes were shown to improve performance significantly, leading to an average classification accuracy of 76% with an average communication speed of 46 bit/min, which is equivalent to a writing speed of 8.8 error-free letters per minute. Although the BCI system works substantially better with gel-based EEG, dry EEG electrodes are more user-friendly and still allow high-speed BCI communication. A Brain-Computer Interface (BCI) is a device that allows to control a computer by brain activity only, without the need for muscle control. Its main purpose is to restore or improve communication for paralysed patients. While there are different kinds of BCI that either rely on motor imagery or the detection of event-related potentials like the P300, BCI systems based on visual evoked potentials (VEPs) are currently the fastest method to establish non-invasive BCI control. Using code-modulated visual evoked potentials (c-VEPs), it was shown that participants
  27. 27. ASET Brain Computer Interface & its Applications 17 could spell more than 20 error-free characters per minute (> 140 bit/min). When using steady state visual evoked potentials (SSVEPs), the communication speed can further be improved by using frequency and phase information of the SSVEP to reach communication speeds up to 60 characters per minute (> 300 bit/min). While these results show that VEP-based BCIs allow for high-speed communication in a lab environment, there are only few efforts to transfer those systems out of the lab and make them useable for a broader audience. One of those few approaches is the Tübingen c-VEP BCI, which was previously shown to reach spelling speeds of more than 20 error-free letters per minute and in another work was improved to control mouse and keyboard of the Windows®operating system, thereby enabling a user to completely control a computer and use arbitrary applications by means of his brain activity only. While this is an important step towards improved useability of a BCI system, the use of gel-based EEG devices still limits the applicability, as gel-based EEG is time- consuming to apply, cannot be applied by the user itself, and “few people are ready to wash their hair each time they wish to use Word or a LaTeX editor”. To solve this problem, dry EEG electrodes have been developed that are quick to setup and allow to measure EEG without the need for conductive gel. While there are a variety of different dry electrodes being developed, there are also commercially available dry EEG electrodes that deliver good results and have been shown to work for BCI applications using P300, SSVEP and motor imagery. Further, there are consumer-grade EEG headsets with dry electrodes or easy to apply wet electrodes that don’t need the application of conductive gel like the Emotiv EPOC. While such systems have also been used for BCI applications, the signal quality of such EEG hardware is not comparable to professional devices and their use is not recommended for serious applications. A good comparison of different dry EEG electrodes developed and tested can be found in Baek et al. who review and discuss several publications. They conclude that dry electrodes allow the acquisition of spontaneous EEG signals with lower signal quality than gel-based EEG electrodes and that movement artefacts are a larger problem in dry electrodes. Nevertheless, signal quality is good enough to record P300 and SSVEP response to use them for BCI applications. While dry electrodes haven’t been tested with a c-VEP BCI system so far, the aim of this study was to test if commercially available medical-grade dry EEG electrodes can be used with a high- speed c-VEP BCI and what algorithmic improvements of the system are necessary to deal with the increased signal-to-noise ratio of dry EEG electrodes [4.1].
  28. 28. ASET Brain Computer Interface & its Applications 18 4.2 OBJECTIVE The objective of this assignment is to extract and then analyze the Raw Values obtained by EEG. 4.3 ALGORITHM NeuroSky algorithms provide the foundation of a universe of applications that can be built to optimize brain health, education, alertness and overall function. General algorithm is as follows: 1. First, for the raw EEG signals, first you need to apply pre-processing tasks like feature extraction. Various methods are available for the same like VLC methods, PCA or wavelets etc 2. Then apply classification methods over the transformed data. 3. Generally if you apply classification algorithms directly on the raw data, then it does not yield good accuracy and good models. NeuroSky provides inbuilt algorithms that are used to record certain types of activities [4.2]. 4.3.1 Attention: The Attention Meter algorithm indicates the intensity of mental “focus” or “attention.” The value ranges from 0 to 100. The attention level increases when a user focuses on a single thought or an external object, and decreases when distracted. Users can observe their ability to concentrate using the algorithm. In educational settings, attention to lesson plans can be tracked to measure their effectiveness in engaging students. In gaming, attention has been used to create “push” control over virtual objects. 4.3.2 Meditation: The Meditation Meter algorithm indicates the level of mental “calmness” or “relaxation.” The value ranges from 0 to 100, and increases when users relax the mind and decreases when they are uneasy or stressed. The Meditation Meter quantifies the ability to find an inner state of mindfulness, and can thus help users learn how to self correct and find inner balance to overcome the stresses of everyday life. The algorithm is also used in a variety of game-design controls.
  29. 29. ASET Brain Computer Interface & its Applications 19 4.3.3 Blink Detection: The Blink Detection algorithm signals a user’s blinks. A higher number indicates a “stronger” blink, while a smaller number indicates a “lighter” or “weaker” blink. The frequency of blinking is often correlated with nervousness or fatigue. Eye blinks are akin to a standard on/off binary system and therefore are valuable for controls that require definitive responses. For instance, in communication applications, one blink means no, two mean yes — giving individuals with a special needs a simple way to communicate [5]. 4.3.4 Mental Effort: The Mental Effort algorithm measures the mental workload while performing a task. The harder a user’s brain works on a task, the higher the value. The algorithm works well with both physical (e.g., drawing) and mental (e.g., reciting) tasks, and can be used for continuous real-time tracking and between-trial comparisons to measure the effects of multi-tasking, workload variability, and more. The algorithm can be used to track the effects of diverse cognitive loads on the ability to learn and provide feedback for user self-improvement. 4.3.5 Familiarity: The Familiarity algorithm tracks learning processes to measure the relative level of understanding, learning, or comfort with a task. It is a measure of the subconscious learning of procedural (motor) and mental tasks. In some cases, it reflects how well a user is doing with the task. By observing trends, users can better understand and assess their learning process. It can be applied to tasks that are physical in nature (e.g., drawing) or mental (e.g., recitation), and enable data-tracking assessments to gauge learning status. 4.3.6 Appreciation: The Appreciation algorithm provides real-time measurement of the level of enjoyment or appreciation a subject feels towards an external audiovisual stimulus. The algorithm allows moment-by-moment detection of emotions. In marketing applications, it can be used to track and understand a users level of appreciation, providing insights and strategic direction for marketing efforts.
  30. 30. ASET Brain Computer Interface & its Applications 20 4.3.7 Emotional Spectrum: Emotional spectrum measures the intensity (low-high) and pleasantness (pleasant/unpleasant) of emotional activity. Pleasant/Unpleasant identifies whether a user is having a pleasant or unpleasant emotion. Pleasant emotions could be happy, serene, relaxed, etc. Unpleasant emotions could be anger, disgust, depressed, etc. Intensity measures how strong a user’s emotions are in real time. The emotion could be happy, angry, calm, or any other emotion. 4 eTensity levels Very Weak – Weak – Strong – Very Strong. 4.3.8 Cognitive Preparedness: Cognitive Preparedness measures the brain’s capacity for optimal cognitive performance on a relatively complex task or in other words, the brain’s capacity for higher level cognitive functions. 4.3.9 Creativity: Creativity measures the brainwave activities underlying creative cognition. The higher the Creativity value, the stronger the brainwave activities promoting innovative and creative thinking. 4.3.10 Alertness: Alertness measures user’s alertness or vigilance level at the moment. High alertness value indicates you are at a state of focus while low value represents a relaxing state of mind.
  31. 31. Brain Computer Interface & its Applications 21 Chapter 5 Working & Applications
  32. 32. Brain Computer Interface & its Applications 22 5.1 EEG – ELECTROENCEPHALOGRAPHY AND MINDWAVE Figure 5.1: Brain Game Simulator Setup Our brain has always been fascinating and a lot of researchers are still trying to explore what we can do if we can capture some meaningful data from brain. It's a simple approach used by NeuroSky to capture raw data (frequency) that can further be analyzed for various applications. The working principle of our project is based on earlier discussed eSense Algorithms for capturing Attention, Meditation & Eye Blink Strengths. 5.1.1 The Origins EEG or electroencephalography are since Hans Berger in 1929 exposed that the activity of the brain could be measured from electrodes situated in the human skull. With EEG we could measure in fact, the functional state of the brain and diagnose some future or actual problems. This is the most common way to measure injuries in the brain and functional brain disturbances, but the creation of the signal is not well understood. Different regions of the cortex have different cytoarchitectures and each region has its own morphological patterns, aspects of
  33. 33. Brain Computer Interface & its Applications 23 intrinsic organization of the cortex are general. Most of the cortical cells are arranged in the form of columns, in which the neurons are distributed with the main axes of the dendritic trees parallel to each other and perpendicular to the cortical surface. This radial orientation is an important condition for the appearance of powerful dipoles. Figures below lists the parts of human brain cortex and zones of interest.. These layers are places of specialized cell structures and within places of different functions and different behaviors in electrical response. An area of very high activity is, for example, layer IV, which neurons function to distribute information locally to neu- rons located in the more superficial (or deeper) layers. Neurons in the superficial layers receive information from other regions of the cortex. Neurons in layers II, III, V, and VI serve to output the information from the cortex to deeper structures of the brain [6]. Figure 5.1.1.1: Lobes of the human brain, external cerebrum, midbrain areas such as the dience- phalon and the hindbrain areas such as the cerebellum, medulla, etc. Figure 5.1.1.2: Different layers (columns) of the brain cortex.
  34. 34. Brain Computer Interface & its Applications 24 Pyramidal cells in layers III and V are mainly responsible for the generation of the EEG. 5.1.2 EEG Signals The EEG signal consists of spontaneous potential fluctuations that also appear without a sensory input. It seems to be a stochastic signal, but it is also composed of quasi- sinusoidal rhythms. The synchrony of cerebral rhythms may occur from pacemaker centers in deeper cortical layers like the thalamus or in sub cortical regions, acting through diffuse synaptic linkages, reverberatory circuits incorporating axonal pathways with extensive ramifications, or electrical coupling of neuronal elements. The range of amplitudes is normally from 10mV to 150mV, when recorded from electrodes attached to the scalp. The EEG signal consists of a clinical relevant frequency range of 0.5–50 Hz (10). The most common frequency bands of EEG are the most common way of analysis. This information can reveal physiological and statistical evidence but each band could vary on people and animals with is behaviors and metal sanity, age, etc. The most important patterns of human EEG are described below [7]. Figure 5.1.2: Example of EEG Bands
  35. 35. Brain Computer Interface & its Applications 25 The phenomena of alpha de-synchronization channel could be used to get the eyes closed/open detection. Delta Waves The appearance of delta waves are common in neonatal and infant EEGs and during in sleep stages in adult EEGs. If delta EEGs appears by itself in a adult it means cerebral injury Theta Waves In the beginning where part of the delta waves, but scientists discovered the importance activity of these waves. Its region of interaction is between thalamic region and play dominant part in childhood and infancy. The normal adult waking of theta waves are a few or small amount of these frequencies observed in drowsiness and sleep. Large amount of theta waves are associated between different amount of pathologies. Alpha Waves These are originated on the posterior half back of the head and are from occipital an parietal regions. These waves are observed during conditions of awakeness, physical relaxation and mental inactivity can be blocked by mental activity or an influx of light when eyes are opened. Beta Waves Are presented in a healthy adult and the area of formation are in the frontal and central region of the cortex. Typical voltage of beta waves are less than 30uV. Beta activity increase when the organism is added with barbiturates, some non barbiturates sedatives and minor tranquilizers. It also appears during mental activity and tension. 5.1.3 Clinical EEG The most common EEG uses up to 30 landmarks on the skull using bipolar derivation (two electrodes on the skull and the difference is the gradient of potential). Unipolar derivation is done with an electrode or group of electrodes with the active part (activity) and the inactive part (usually nose or ear). The advantage of unipolar derivation are that the amplitude of each deflection is proportional to the magnitude of the potential change that causes it and the
  36. 36. Brain Computer Interface & its Applications 26 demonstration of small time differences between the occurrence of a widespread discharge at several electrode. Figure 5.1.3: Common areas of bipolar EEG sensory. 5.1.4 Emotiv Epoch EEG Based on the latest developments in neuro-technology, Emotiv presents a revolutionary personal interface for human computer interaction. Emotiv EPOC is a high resolution, multi- channel, wireless NeuroHeadset. The EPOC uses a set of 14 sensors plus 2 references to tune into electric signals produced by the brain to detect the user’s thoughts, feelings and expressions in real time. The EPOC connects wirelessly to PCs running Windows or MAC OS X. Experience the fantasy of controlling and influencing the virtual environment with your mind. Access applications and play games developed specifically for the EPOC, or use the EmoKey to connect to current PC games and experience them in a completely new way [8]. If you or any of your 3rd party applications require access to raw EEG data, you will need to purchase the Emotiv EEG NeuroHeadset. Figure 5.1.4: Emotiv Epoch EEG NeuroHeadset
  37. 37. Brain Computer Interface & its Applications 27 5.1.5 NeuroSky Mindwave Measuring Electroencephalogram (EEG) activity has historically required complex, intimidating and immovable equipment costing thousands of dollars. NeuroSky is unlocking a new world of solutions for education and entertainment with our research-grade, mobile, embeddable EEG biosensor solutions. Precisely accurate, portable and noise-filtering, our EEG biosensors translate brain activity into action. Our EEG solution digitizes analog electrical brainwaves to power the user-interface of games, education and research applications. We amplify and process raw brain signals to deliver concise input to the device. Our brainwave algorithms, developed by NeuroSky researcher and our partner universities and research institutions are uncovering new ways to interact with our world. Figure 5.1.5: Mindwave EEG NeuroHeadset Both EEGs are good. NeuroSky Mindwave is cheap, easy to hack and useful for develop simple applications and filters for study brainwave signals. One thing to note is that NeuroSky EEG is only for develop games and must not be used to study the signals as a medical device, this is not the goal of this company. The cost of the NeuroSky Mindwave is about 100 USD. Emotiv EEG is good at a clinical level because gives you a lot of information of regions from F1 to F15, has an gyroscope to sense orientation of the head and a SDK for developer or researcher in Linux and Windows. The cost of the EEG rounds about 300 USD. But if you need to get raw
  38. 38. Brain Computer Interface & its Applications 28 data for better analysis you need to buy the complete package of EEG NeuroHeadset, and software, it’s around 750 USD. There are two frames that are outputted at variable rates by the device: AA 04 80 02 00 56 27 AA AA 04 80 02 00 53 2A AA AA 20 02 38 83 18 02 43 EA 00 03 90 00 00 89 00 00 47 00 00 1E 00 00 28 00 00 3B 00 00 27 04 00 05 00 E7 AA AA 04 80 02 00 53 2A AA AA 04 80 02 00 55 28 AA AA 04 80 02 00 54 29 AA AA 04 80 02 00 54 29 AA Purple frame is outputted every 512 Hz and is not exchangeable the frequency time of output. Green frame is outputted every 1 Hz and is not exchangeable the frequency time of output. The frames contain useful information about raw values and calculated values. The below table lists the different frames that are output of every frame. For the 512 Hz frame the information output is: byte: value // Explanation [ 0]: 0xAA // [SYNC] [ 1]: 0xAA // [SYNC] [ 2]: 0x04 // [PLENGTH] (payload length) of 8 bytes [ 3]: 0x80 // [RAW_WAVE_VALUE] 16-bit two's-compliment signed value (high-order byte followed by low-order byte) (-32768 to 32767) [ 4]: 0x02 // [VLENGHT] (payload variable length) of 'n' bytes [ 5]: 0x00 // [RAW_HIGH] high order byte of raw data two's compliment signed value [ 6]: 0x53 // [RAW_LOW] low order byte of raw data two's compliment signed value [ 7]: 0x2A // [CHKSUM] (1's comp inverse of 8-bit Payload sum)
  39. 39. Brain Computer Interface & its Applications 29 For the 1 Hz frame the information output is: byte: value // Explanation [ 0]: 0xAA // [SYNC] [ 1]: 0xAA // [SYNC] [ 2]: 0x20 // [PLENGTH] (payload length) of 32 bytes [ 3]: 0x02 // [POOR_SIGNAL_QUALITY] (0 to 255) [ 4]: 0x38 // 56 of 255 [ 5]: 0x83 // [ASIC_EEG_POWER] eight big-endian 3-byte unsigned integer values representing delta, theta, low-alpha, high-alpha, low-beta, high-beta, low-gamma, and mid- gamma EEG band power values [ 6]: 0x18 // upper byte of EEG_POWER_DELTA [ 7]: 0x02 // middle byte of EEG_POWER_DELTA [ 8]: 0x43 // lower byte of EEG_POWER_DELTA [ 9]: 0xEA // upper byte of EEG_POWER_THETA [10]: 0x00 // middle byte of EEG_POWER_THETA [11]: 0x03 // lower byte of EEG_POWER_THETA [12]: 0x90 // upper byte of EEG_POWER_LOW_ALPHA [13]: 0x00 // middle byte of EEG_POWER_LOW_ALPHA [14]: 0x00 // lower byte of EEG_POWER_LOW_ALPHA [15]: 0x89 // upper byte of EEG_POWER_HIGH_ALPHA [16]: 0x00 // middle byte of EEG_POWER_HIGH_ALPHA [17]: 0x00 // lower byte of EEG_POWER_HIGH_ALPHA
  40. 40. Brain Computer Interface & its Applications 30 [18]: 0x47 // upper byte of EEG_POWER_LOW_BETA [19]: 0x00 // middle byte of EEG_POWER_LOW_BETA [20]: 0x00 // lower byte of EEG_POWER_LOW_BETA [21]: 0x1E // upper byte of EEG_POWER_HIGH_BETA [22]: 0x00 // middle byte of EEG_POWER_HIGH_BETA [23]: 0x00 // lower byte of EEG_POWER_HIGH_BETA [24]: 0x28 // upper byte of EEG_POWER_LOW_GAMMA [25]: 0x00 // middle byte of EEG_POWER_LOW_GAMMA [26]: 0x3B // lower byte of EEG_POWER_LOW_GAMMA [27]: 0x00 // upper byte of EEG_POWER_MID_GAMMA [28]: 0x00 // middle byte of EEG_POWER_MID_GAMMA [29]: 0x27 // lower byte of EEG_POWER_MID_GAMMA [31]: 0x04 // [ATTENTION] eSense (0 to 100) [32]: 0x00 // Attention level [33]: 0x05 // [MEDITATION] eSense (0 to 100) [34]: 0x00 // Meditation level [35]: 0xE7 // [CHKSUM] (1's comp inverse of 8-bit Payload sum) 5.2 HOW DOES THE TECHNOLOGY WORK ? Brainwaves are tiny electrical impulses released when a neuron fires in the brain. NeuroSky’s brain-computer interface (BCI) technology works by monitoring these electrical impulses with a forehead sensor. The neural signals are input into our ThinkGear chip, and interpreted with our patented Attention and Meditation algorithms. The measured electrical signals and calculated interpretations are then output as digital messages to the computer, toy, or
  41. 41. Brain Computer Interface & its Applications 31 mobile device, allowing you to see your brainwaves on the screen, or use your brainwaves to affect the device’s behavior. 5.3 APPLICATIONS OF OUR PROJECT: Brain Game Simulator: We are trying to automate the windows games first racing games controls using attention & meditation level values that we have captured from the frontal lobe of brain using NeuroSky Mindwave Headset. # Python Direct X game controller server # DirectX works with Scancodes for Direct Input 5.4 HOW TO SETUP/CONFIGURE?  Install python2.7 64-bit/32-bit (recommended) or later.  Install dependency packages for python o PyWin32: To automate windows keys using sendkeys & directX programming o Mindwave: To interface with ThinkGear socket connection & capture data from NeuroSky Mindwave Headset.  Check your port number from Device Manager & Edit the line from “braingamesimulator.py” file: headset = mindwave.Headset('COM13', 'CC0E')  Run file “braingamesimulator.py”  Calibration for individual person can be done by changing threshold values of if-elif conditions.  Start any racing game preferred Asphalt8.
  42. 42. Brain Computer Interface & its Applications 32 Figure 5.4: A brain controlled car racing game 5.4.1 Mind Controlled Robot: We are trying to control a robot using BCI through the threshold values of attention & meditation. Further, the data is sent to Raspberry Pi micro-controller through socket connection over Wireless network & GPIO triggers are sent to drive motors of Robot. Figure 5.4.1: Mind Controlled Robot
  43. 43. Brain Computer Interface & its Applications 33 5.4.1.1 How to do? Hardware Setup: Figure 5.4.1.1: Mind Controlled Robot – Hardware Setup 5.4.1.2 What is RaspberryPi? Figure 5.4.1.2: Raspberry Pi
  44. 44. Brain Computer Interface & its Applications 34 The Raspberry Pi is a low cost, credit-card sized computer that plugs into a computer monitor or TV, and uses a standard keyboard and mouse. It is a capable little device that enables people of all ages to explore computing, and to learn how to program in languages like Scratch and Python. It’s capable of doing everything you’d expect a desktop computer to do, from browsing the internet and playing high-definition video, to making spreadsheets, word-processing, and playing games. Using Python for RaspberryPi. Python is a wonderful and powerful programming language that's easy to use (easy to read and write) and with Raspberry Pi lets to connect projects to the real world. Python syntax is very clean, with an emphasis on readability and uses standard English keywords. Start by opening IDLE from the desktop. The easiest introduction to Python is through IDLE, a Python development environment. 5.4.1.3 Circuit Connection with MOTOR Driver L298n Dual H-Bridge Figure 5.4.1.3: Circuit Connection
  45. 45. Brain Computer Interface & its Applications 35 5.4.1.4 Motor Direction Logic 5.4.1.5 Software Setup:  Boot Raspbian/Noobs in Raspberry Pi. Figure 5.4.1.4: Booting Raspbian/Noobs
  46. 46. Brain Computer Interface & its Applications 36  Install XRDP to enable remote desktop connection. Figure 5.4.1.5: Installing XRDP  Install Dependencies: Python2.7, Mindwave, GPIO, PyQt4.  Start Hotspot from laptop using Connectify or CMD.(for wireless communication)  Connect to Remote Desktop Connection with the IP scanned for wlan0 of RPi.  Run the program “robotcontrol.py” to start connecting to NeuroSky Headset & send signals to drive motors of Robot.  Calibration: “Try to focus on Robot & increase your attention levels for FORWARD control & distract yourself to increase your meditation level for REVERSE control.
  47. 47. Brain Computer Interface & its Applications 37 5.4.2 Brain Keyboard: This application is an attempt to develop a keyboard interface for physically handicapped: Lost his/her hand & they can type using virtual keyboard through eye blinks. Figure 5.4.2: Brain Keyboard People who have severe movement disorders like Amyotrophic Lateral Sclerosis (ALS) or Locked in Syndrome, is a condition where a patient is awake and aware of its surrounding but unable to communicate or perform any action due to paralysis of almost every voluntary muscles in the body (with the exception of eye movements and blinking). The people who are suffering from severe Cerebral Palsy disorder are not able to speak or not understandable enough so they could not communicate and interact with others. These people also have severe movement disorder.Imagine these patients having a fully functional brain trapped within a non-functioning body. The brain of the patient would be fully conscious and aware of its surroundings, it could think and process stimuli, but unable to translate thought into action. This is caused by a gradual degeneration of the nerve cells in the central nervous system that controls voluntary muscle movement. The application has 26 alphabetical keys with a space and a delete key. Here, a square selection box scans all the keys one by one in a sequence. There is a time delay of about 100ms to move the square selection box from one key to another key. The application takes the input when user blinks his
  48. 48. Brain Computer Interface & its Applications 38 eye. The eye blinks triggers the current selected key (as shown in the picture) and the letter gets print on the text box 5.4.2.1 How to setup/configure?  Install python2.7 64-bit/32-bit (recommended) or later.  Install dependency packages for python o PyWin32: To automate windows keys using sendkeys & directX programming o Mindwave: To interface with ThinkGear socket connection & capture data from NeuroSky Mindwave Headset. o Check your port number from Device Manager & Edit the line from brainkeyboard.py: headset = mindwave.Headset('COM13', 'CC0E')  Run & start training your mind & eye blink strength for better controls 5.4.3. Data Visualization: Capture & Plot RAW Data from NeuroSky Mindwave Headset Our brain has always been fascinating & lot of researchers are still trying to explore what we can do if we can capture some meaningful data from brain. It's a simple approach used by NeuroSky to capture raw data (frequency) that can further be analyzed for various applications. 5.4.3.1 Python: Dependencies  Python2.7 or later version  Packages: PyQtGraph, Matplotlib, PyQt4  Drivers: ThinkGear Connector
  49. 49. Brain Computer Interface & its Applications 39 GitHub: https://github.com/vsltech/neuroskymindwavecapture/ Figure 5.4.3.1: Screenshot of Plot in Python 5.4.3.2 OpenVibe: The purpose of OpenViBE is to get data from the acquisition device through the Acquisition Server and then send it to one or more clients. This client is usually, for now, the OpenViBE designer. The Acquisition Server and the clients (Designers) can be either on the same machine or different machines on the same network, or any combination of these. The following diagram explains the possibilities:
  50. 50. Brain Computer Interface & its Applications 40 Figure 5.4.3.2: Acquisition Server and Clients in OpenVibe Procedure Run and setup the Acquisition Server Figure 5.4.3.2.1: Running and setting up the acquisition server in OpenVibe Run the designer and create your first scenario Now search for the Signal Display box and drag it into the scenario. This box displays the signal it gets on the input. Now connect the two boxes. Click on the signal output (pink arrow) on the Acquisition Client box and drag a connector to the matrix input (green arrow) of the Signal Display box.
  51. 51. Brain Computer Interface & its Applications 41 Figure 5.4.3.2.2: Connecting acquisition client with Signal Display 5.4.3.3 MATLAB Figure 5.4.3.3: NeuroSky Mindwave and MATLAB MATLAB is a part of the code decode Attention value calculated by e-sense algorithm from ThinkGear packets  Matlab Code to Read Eyeblink using Mindwave: This part of the code decode eye blink value calculated by e-sense algorithm from ThinkGear packets.
  52. 52. Brain Computer Interface & its Applications 42  Matlab code to read Meditation using Mindwave: This part of the code decode meditation value calculated by e-sense algorithm from ThinkGear packets.  Mind-Attention Control LED using Arduino.  Blinking led using EEG.  Interfacing Mindwave with Arduino. MATLAB Code to read attention values [9]: Matlab code to read attention using Mindwave: TG_DATA_ATTENTION = 2; %Clear Screen clc; %Clear Variables clear all; %Close figures close all; %Preallocate buffer data_att = zeros(1,256); %Comport Selection portnum1 = 7; %COM Port # comPortName1 = sprintf('.COM%d', portnum1); % Baud rate for use with TG_Connect() and TG_SetBaudrate(). TG_BAUD_115200 = 115200; % Data format for use with TG_Connect() and TG_SetDataFormat(). TG_STREAM_PACKETS = 0; % Data type that can be requested from TG_GetValue(). TG_DATA_ATTENTION = 2; %load thinkgear dll loadlibrary('Thinkgear.dll'); %To display in Command Window fprintf('Thinkgear.dll loadedn'); %get dll version dllVersion = calllib('Thinkgear', 'TG_GetDriverVersion'); %To display in command window fprintf('ThinkGear DLL version: %dn', dllVersion ); % Get a connection ID handle to ThinkGear connectionId1 = calllib('Thinkgear', 'TG_GetNewConnectionId');
  53. 53. Brain Computer Interface & its Applications 43 if ( connectionId1 < 0 ) error( sprintf( 'ERROR: TG_GetNewConnectionId() returned %d.n', connectionId1 ) ); end; % Attempt to connect the connection ID handle to serial port "COM3" errCode = calllib('Thinkgear', 'TG_Connect', connectionId1,comPortName1,TG_BAUD_115200, TG_STREAM_PACKETS ); if ( errCode < 0 ) error( sprintf( 'ERROR: TG_Connect() returned %d.n', errCode ) ); end fprintf( 'Connected. Reading Packets...n' ); i=0; j=0; %To display in Command Window disp('Reading Brainwaves'); figure; while i < 20 if (calllib('Thinkgear','TG_ReadPackets',connectionId1,1) == 1) %if a packet was read... if (calllib('Thinkgear','TG_GetValueStatus',connectionId1,TG_DATA_ATTENTION ) ~= 0) j = j + 1; i = i + 1; %Read attention Valus from thinkgear packets data_att(j) = calllib('Thinkgear','TG_GetValue',connectionId1,TG_DATA_ATTENTION ); %To display in Command Window disp(data_att(j)); %Plot Graph plot(data_att); title('Attention'); %Delay to display graph pause(1); end end end %To display in Command Window disp('Loop Completed') %Release the comm port calllib('Thinkgear', 'TG_FreeConnection', connectionId1 ); 5.4.3.4 Visual Studio Similarly, there is the visual studio code as well, which is coded in C#. The same steps are followed as above but in C#:
  54. 54. Brain Computer Interface & its Applications 44 Figure 5.4.3.4: Getting Raw Values using C# in Visual Studio
  55. 55. ASET Brain Computer Interface & its Applications 45 Chapter 6 Conclusion
  56. 56. ASET Brain Computer Interface & its Applications 46 6.1 CONCLUSION: As we have seen the working of BCI using EEG, it can be used in several modern applications as follows: 6.1.1 Medical Uses of EEG: EEG testing is used to diagnose and evaluate various neurological conditions, such as epilepsy. The monitoring of a patient’s brain waves can give their doctor insight into any abnormalities in brain function. Other conditions which may be tested for using EEG include Alzheimer’s disease and narcolepsy. Figure 6.1.1 EEG Testing EEG is also used to monitor brain activity when a patient is in a coma. If the coma is natural and the patient is on complete life support, EEG can detect brain death. If a patient is in a medically induced coma, EEG is used to determine the appropriate amount of sedation needed to keep the patient in the coma. If extensive brain damage is suspected due to physical trauma, drug abuse, or any other factor, EEG testing can help determine the actual extent of the damage. EEG cannot measure the performance of the brain beyond its electrical activity, and cannot be used to determine intelligence or to diagnose most mental health conditions (with the exception of certain kinds of psychoses which affect electrical activity in the brain) [10].
  57. 57. ASET Brain Computer Interface & its Applications 47 Modern EEG tests, unlike the invasive procedures of the 18th and 19th centuries, are completely painless and safe. No electricity is actually administered to the patient—it is simply measured. 6.1.2 EEG Technology and Research: EEG technology enables research in a variety of fields, from medicine to marketing. Recent research into sports-related concussions has used EEG testing to determine the short- and long-term results of sustaining multiple concussions or other head traumas. EEG has also played an important role in recent sleep studies, helping researchers learn the true effects of various lengths and qualities of sleep on the human brain. EEG testing is also being used in advertising. Researchers who observe eye movement, facial expressions, and brain waves to determine the effectiveness of various pieces of media are pioneering a field called neuromarketing, which seeks to use biometrics to better the odds of advertising success in a world already oversaturated by media. These are just a few examples of new and growing fields of research enabled by EEG technology—the measuring of human engagement and brain activity has nearly universal applications. 6.1.3 EEG-Enabled Prosthetics: Controlling a prosthetic limb with your mind was the stuff of science fiction until 1999, when a collaboration between researchers at the MCP Hahnemann School of Medicine in Philadelphia and Duke University in Durham (North Carolina) reported that they were successful in getting rats to control a simple robot using the rats’ brain activity alone. The field of neuroprosthetics is young, but promising. 15 years later, a paraplegic wearing a robotic exoskeleton controlled by his own brain signals kicked off the 2014 World Cup in Brazil. In 2015, researchers from the University of Houston developed an algorithm which allowed an amputee to grasp objects using a robotic hand which he controlled with his own mind. Early research in the field of neuroprosthetics required more invasive methods of recording brain signals (much like the early work of Richard Caton and Hans Berger). However, the robotic hand
  58. 58. ASET Brain Computer Interface & its Applications 48 at the University of Houston was enabled by EEG alone, signalling a drastic advancement in neuroprosthetic technology and a new hope for people living with loss of limbs or mobility. 6.1.4 EEG-Enabled Communication Devices: Most people are familiar with the story of theoretical physicist Stephen Hawking, who lost the ability to speak after gradually becoming paralyzed due to ALS. His speech deteriorated rapidly in the late 70s, at which point he required a translator of sorts to communicate clearly with others. In the 80s, a serious bout of pneumonia necessitated a tracheotomy, which rendered him completely unable to speak. Figure 6.1.4 Stephen Hawking Hawking used a computer program to help him speak for several years, which was fairly high-tech for the late 1980s and early 90s. In 2005 he began using a program which allowed him to communicate via computer using only the movement of his cheek muscles, and in 2012 he began experimenting with brainwave-powered communication devices. A communication device powered by brain activity alone would restore speech to patients even with complete paralysis. The technology has yet to be perfected, but new research is promising, and may one day lead to perfect technology-enabled communication for patients with ALS and other degenerative illnesses [11].
  59. 59. ASET Brain Computer Interface & its Applications 49 6.1.5 EEG and Education: Learning about how someone’s brain works is an incredible educational tool. The education system suffers from a lack of resources and understanding, which hurts teachers’ ability to effectively instruct students who have learning disabilities or simply don’t learn well from conventional instruction methods. Some connections have been made between ADHD diagnoses and EEG readings showing dysregulated cortical electrical activity. While diagnostics are not the responsibility of a teacher, applications of EEG technology which monitor focus and concentration could lead to a greater understanding of which educational techniques work best for individual students. Additionally, educational EEG-enabled games requiring intense focus could help unfocused students train their brains to concentrate better [12]. 6.1.6 EEG Entertainment: What if you could play video games with your mind? It may not carry the scientific gravity of neuroprosthetics, but it sure is fun. Neurogaming (are you starting to get the hang of this naming convention?) is another new tech field—and definitely the most exciting growth area for the gaming industry—enabled by EEG. Game development company Crooked Tree Studios used NeuroSky’s EEG technology to build “Throw Trucks With Your Mind”, the objective of which is probably self-explanatory. Players focus their thoughts on a single object, and the NeuroSky MindWave Mobile Headset interprets the electrical activity of their brains to pick up and hurl objects at opponents. Lat Ware, the creator of “Throw Trucks With Your Mind”, explains that the inspiration for the game came from his own experience with experimental neurofeedback therapy to treat his Attention Deficit Disorder. Learning that his brain activity could be fed into and interpreted by a computer fascinated Ware, eventually leading him to pitch his idea to Johnny Liu, the Director of NeuroSky’s Developer Program. Ware then raised nearly $50,000 via Kickstarter to get his project off the ground. The technology to support more applications of neurogaming exists but, until recently, it was too expensive to appeal to the average consumer of video games. Now that EEG headsets can be purchased for less than the cost of the newest traditional gaming console, neurogaming is poised to be the next big thing in video game entertainment [13].
  60. 60. ASET Brain Computer Interface & its Applications 50 Chapter 7 Future Scope
  61. 61. ASET Brain Computer Interface & its Applications 51 7.1 BCI ADVANTAGES & DISADVANTAGES Some of the good advantages of BCI are direct communication between brain and devices, better living for paralyzed people etc. While the disadvantages of BCI are; ill effects in the brain due to viral attacks, requiring excessive training for proper usage, high cost, slow speed, lack of better sensor modality, invasive BCIs are risky since it requires neurosurgery etc. 7.2 EEG FUTURESCOPE: As more exciting developments are made with EEG applications and the cost of hardware decreases, the future of EEG for device manufacturers seems full of promise. Consumers increasingly turn to mobile apps and wearable devices (such as MyFitnessPal, Clue, MySugr, orthe LifeBeat™ fitness tracking wristband) to gain insight into their health and wellness metrics, and help them meet their personal health goals. As EEG data can be interpreted to measure biometrics such as focus, attention, mental fatigue, stress, and more, it presents a new opportunity to give consumers insight into their personal wellness metrics. As mentioned above, with personal EEG devices becoming affordable for the average consumer, the neurogaming applications are virtually endless. Augmented and virtual reality games which can be controlled with the mind may seem like science fiction, but are solidly within the reach of modern game developers. Combined with other biometrics such as heart rate, pupil dilation, and facial expression analysis, it seems anything is possible–if you can observe, measure, and interpret it, you can make a game of it. 7.3 BCI FUTURESCOPE: As the coming step of technological advancement, many research workers and scientists are trying to bring out wide variety of BCI applications useful for the society. Coming years, we can make BCI restore and augment human functions thereby improving quality of living. Some applications like flying an aeroplane just by thinking, a blind driving a vehicle etc. will be coming to a reality. In the medical sector, research workers are trying to bring out miniaturized equipments and introduction of wireless BCI. It’s said that in the coming future, we can replace the robotic
  62. 62. ASET Brain Computer Interface & its Applications 52 devices and directly bypass the signals to the nerves in the damaged part of the brain, thereby allowing the paralyzed patient to move their body. Last but not the least, development in BCI can bring out drastic and attractive changes to the society [14]. Future applications of BCI: • Bionic eye • Transfer the hearing impulse to brain. • Robotic assistance to old and disabled. • Real time gaming. • Use a device similar to Google Glass. Include a BCI device within it. • Use a BCI bar code reader in a product. • When BCI bar code reader is read by a Google glass device activate the BCI interface in the device and record and process the signal on a server level. Figure 7.1: BCI - Future Scope (Robotic assistance to old and disabled)
  63. 63. ASET Brain Computer Interface & its Applications 53 Figure 7.2: BCI - Future Scope (Google Glass and BCI)
  64. 64. Brain Computer Interface & its Applications 54 REFERENCES [1] A Brain-Computer Interface for Shared Vehicle Control on TORCS Car Racing Game, 2014 10th International Conference on Natural Computation [2] NeuroSky SDK for .NET: Development Guide and API Reference. [3] Flappy Brain - Cambridge Jam Raspberry Pi project. WEB REFERENCES [4.1] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0172400 [4.2] http://neurosky.com/biosensors/eeg-sensor/algorithms/ [5] http://blog.makezine.com/archive/2010/04/tan_le_demos_mind-controller_on_mar.html [6] https://www.cnet.com/uk/news/moving-objects-with-mattels-brainwave-reading- mindflex/ [7] http://www.knight-of-pi.org/raspberry-pi-mindcontrol-neurosky-mindwave-as-simple- eeg-interface/ [8] http://neurosky.com/biosensors/eeg-sensor/algorithms/ [9] https://github.com/BarkleyUS/mindwave-python [10] http://developer.neurosky.com/docs/doku.php?id=app_notes_and_tutorials [11] https://www.pantechsolutions.net/blog/list-of-source-codes-available-for-neurosky- mindwave-mobile/ [12] https://store.neurosky.com/products/pc-developer-tools [13] http://openvibe.inria.fr/ [14] https://cofindtira.jimdo.com/2017/05/03/drivers-in-labview/
  • ShashiKiran127

    Feb. 19, 2018

Brain Controlled Game Simulator ✓ Brain Controlled Robot ✓ Brain Keyboard ✓ Brain Visualization using Open Vibe & Python Github: https://github.com/vsltech/braingamesimulator

Views

Total views

3,069

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

195

Shares

0

Comments

0

Likes

1

×