Exploring the Brain Computer Interface
Upcoming SlideShare
Loading in...5
×
 

Exploring the Brain Computer Interface

on

  • 443 views

Computer-brain interface is a mainstay of science fiction, and devices are available today to use our brainwaves as a computer input. But is it practical? How far away is it? Will "Big Brother" read ...

Computer-brain interface is a mainstay of science fiction, and devices are available today to use our brainwaves as a computer input. But is it practical? How far away is it? Will "Big Brother" read our thoughts and hack our brains?

In this class, we will dive into the future of thought as input for wearable devices with real-world examples and code. Demonstrations will be shown using the Emotiv EPOC headset, a revolutionary high resolution, neuro-signal acquisition and processing wireless neuroheadset that uses a set of sensors to tune into electric signals produced by the brain to detect thoughts, feelings and expressions.

You will see the EEG neuroheadset and computer interface with examples of interfacing with desktop, mobile and wearable apps. We will dive into the roots of the technology, showing code and examples along with big pictures of the technology. You will walk away with an understanding of how this still evolving and largely unknown technology really works, how it can be used, as well as longer-term implications.

Statistics

Views

Total Views
443
Views on SlideShare
439
Embed Views
4

Actions

Likes
4
Downloads
20
Comments
0

2 Embeds 4

https://twitter.com 2
http://www.slideee.com 2

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Exploring the Brain Computer Interface Exploring the Brain Computer Interface Presentation Transcript

  • Exploring the Brain Computer Interface JIM MCKEETH EMBARCADERO TECHNOLOGIES JIM.MCKEETH@EMBARCADERO.COM @JIMMCKEETH
  • ‹#› Agenda • The Science of Reading the Brain • The Science of Direct Brain Input • Consumer Brain-Computer Interfaces • Demonstration - Volunteer flying the Quadricopter • Summary & MindTunes 2
  • ‹#› About Jim McKeeth • Lead World Wide Developer Evangelist • Host of Podcast at Delphi.org • Longtime developer ◦ Object Pascal, Java, C#, JavaScript, Objective-C, etc. • Invented and patented swipe to unlock in 2000 ◦ US Patent # 8352745 & 6766456, etc. • Improvisational performer • I am not a neuroscientist, but I find this all fascinating! 3
  • ‹#4› Embarcadero Technologies • Founded 1993 • Provides Architect, Database and Developer Tools • 3.2 Million Customers including 97% of Fortune 2000 • 35+ Product Awards – Consistent Innovation • 500+ Employees in 29 Countries •www.embarcadero.com
  • ‹#› About Appmethod • Integrated Development Environment • Native compiler for Win32, Win64, OS X, iOS & Android • Multi-platform visual designer • Framework for rapid prototyping and accelerated development with C++ or Object Pascal • Architecture allowing shared code & UI across platforms • Advanced language features like ARC, Generics, etc. 5 www.appmethod.com
  • ‹#6›
  • ‹#7› Gartner's 2014 Hype Cycle for Emerging Technologies August 11, 2014 http://www.gartner.com/newsroom/id/2819918 Hype Cycle
  • The Brain-Computer-Interface in Science Fiction
  • ‹#› Brain-Computer Interface ST:TNG Episode 93 (4x19): The Nth Degree (1991) 9
  • Science Fiction is Science Fact TODAY!
  • The Science of Reading the Brain
  • 12 ‹#› Understanding the Brain Delta 4 Hz Hemodynamics (Blood flow) Theta 4-­‐7 Hz Alpha 7-­‐14 Hz Mu 8-­‐13 Hz Beta 15-­‐30 Hz Gamma 30-­‐100 Hz Neural Oscillation
  • ‹#› Polygraph (aka Lie Detector) • Measures physical manifestations of mental state. • Blood pressure, pulse, respiration, skin conductivity, etc. • Looks for changes during questioning. • Requires specialized training by operator. 13
  • ‹#› Magnetic Resonance Imaging (MRI) • Uses oscillating magnetic fields at different resonant frequencies. • Functional MRI uses software to analyze images. • Shows activity (blood flow) in different parts of the brain. • Combined with brain mapping can reveal thought information. 14
  • ‹#› Electroencephalography (EEG) •Measures electrical activity along the scalp. • Detects voltage fluctuations resulting from ionic current flows within neurons of brain. •Millisecond-range resolution (faster than CT or MRI) • Typical clinical usage has 19+ input electrodes as well as ground and reference electrodes. 15
  • ‹#› Electroencephalography (continued) • Series of electrodes placed next to scalp. • Difference in voltage is measured between each electrode and the reference. 16
  • ‹#› Magnetoencephalography (MEG) 17 Uses Arrays of SQUIDs (Superconducting QUantum Interference Devices)
  • The Science of Direct Brain Input
  • ‹#› Change Perception of Reality The “headjack” from The Matrix movies -­‐ 1999 19
  • Transcranial Direct-Current Stimulation (tDCS) • Constant, low current delivered via electrodes. • Results: ◦ Increase cognitive performance ◦ Enhance language & math ability ◦ Improve attention span ◦ Aid problem solving, memory & coordination • Recent DIY “Brain Hacking” Craze ◦ http://www.reddit.com/r/tDCS 20 ‹#› tdcsplacements.com
  • ‹#› Transcranial Magnetic Stimulation (TMS) • Similar strength to MRI •Magnetic induction of electrical currents. • Used to treat neurological conditions. • Produces: ◦ Flashes of lights in vision ◦Muscular activity ◦ Slight cognitive degradation 21
  • Ultrasonic Transcranial Stimulation Device • 2000 patent from Sony (US6584357 B1) ◦ Finer targeting than TMS ◦ Theoretically invoking sensory experiences ◦ Continued research and patents (US7350522 B2) 22 ‹#› http://www.newscientist.com/article/mg18624944.600 Speculative work, “There experiments were not any done.”
  • Ultrasonic Neuromodulation for Spatial Resolution • Virginia Tech Carilion Research Institute study published in Jan 2014 Nature Neuroscience ◦ Used “targeted ultrasound” to boost “sensory perception” ◦ The ultrasound waveform used in the study “alters the balance of synaptic inhibition and excitation between neighboring neurons within the cerebral cortex.” ◦ Targeted an area of the brain as small as the size of an “M&M” ◦ “This finding represents a new way of non-invasively modulating human brain activity with a better spatial resolution than anything currently available.” ‹#› http://www.eurekalert.org/pub_releases/2014-­‐01/vt-­‐udt011014.php 23
  • 2‹#4› "In neuroscience, it's easy to disrupt things. We can distract you, make you feel numb, trick you with optical illusions. It's easy to make things worse, but it's hard to make them better. These findings make us believe we're on the right path." -- William "Jamie " Tyler, Assistant Professor Virginia Tech Carilion Research Institute
  • Science Fiction Becoming Reality Interesting Milestones
  • ‹#› Brain-to-Brain Input The “Hat” from Brainstorm -­‐ 1983 26
  • ‹#› Brain-to-Brain Input at University of Washington • Input from Computer • EEG from brain • Transmitted over campus internet • Replayed through TMS to another brain • Output to computer More information http://bit.ly/EEG2TMS 27
  • ‹#› Cybernetic Limbs Captain America: The Winter Soldier -­‐ 2014 28
  • ‹#› DARPA’s Brain Controlled Prosthetic Arm • Part of DARPA’s Innovation Pathway. • Uses microchip implanted in brain. • 27 different fine grained controls to arm, hand and fingers. • Expected on market by 2016 29 http://www.fastcompany.com/1725799/darpas-­‐mind-­‐controlled-­‐ prosthetic-­‐arm-­‐could-­‐be-­‐market-­‐four-­‐years
  • ‹#› See What You See - Hear What You Hear The “Monitor” from Ender’s Game – 2013 30
  • ‹#› See What You See • In 2010 another team lead by Shinji Nishimoto partially reconstructed YouTube videos based on brain recordings. • Overlaid similar videos to create composite video. • http://youtu.be/nsjDnYxJ0bo & http://youtu.be/EdGjB6XCXOA 31
  • ‹#› See What You See 32 • In 2008 IBM used a feedback loop based on the fusiform face area of the observer’s brain for facial recognition. • Patent # US20100049076 A1
  • ‹#› Hear What You Hear • 2012 - UC Berkely & Brian Pasley decoded internal neural processing of auditory information. • Reconstructed sound on computer from electrical signals in observer’s brains. • Resulting sound was recognizable when compared to original. 33
  • ‹#› Identifying Thoughts • 2008 & 2009 Using fMRI recognized thoughts within the human brain. • Discovered similar thoughts appear similar is different subject’s brains. • Was able to discern which of 10 objects subject was thinking about. • Also predicted subject intentions up to 10 seconds before subject expressed them. 34
  • Read-Only Consumer Brain-Computer Interfaces
  • ‹#› Emotiv EPOC • 16 “wet” electrodes ◦ 14 EEG – read brain waves + 2 reference electrodes • Two-axis gyroscope to read head movements • 4 mental states, 13 conscious thoughts & facial expressions – 4 Processing suites • Wireless (encrypted) connection to Windows, Linux or Mac • $500 Dev kit, or $299 headset alone (no raw EEG) • www.emotiv.com 36
  • NeuroSky’s MindWave (also in XWave headsets) 3‹#7› • EEG with 1 “dry” electrode • Measures “attention,” “meditation” and eye-blinks • Bluetooth communication • SDK for iOS, Android, PC & Mac • $129.99 to $79.95 retail neurosky.com • Hacking: ◦ http://makezine.com/2011/05/10/behind-the-brain-blinker/ ◦ http://hackaday.com/2012/12/20/modifying-an-eeg-headset-for-lucid- dreaming/
  • ‹#› InteraXon’s Muse • 7 dry EEG sensors via Bluetooth • Focuses on “brain fitness” • 4 hour battery • SDK for Java, Python, Objective-C, C, C++ on iOS, Android, OS X, Windows, Ubuntu • $299 available now • choosemuse.com 38
  • ‹#› OpenBCI • Open source hardware & software • Bluetooth, Arduino, EEG • 8 wet Electrodes per board (can daisy chain boards) • Full open EEG & BCI platform • $500 - $800 (preorder) • www.openbci.com 39
  • ‹#› Emotiv Insight • Expected 2015 • Starting at $299 • 5+2 channels dry sensors • Bluetooth 4.0 LE • Data rate: 128 samples per second per channel • Battery Life: 4+ hours • SDK: Android, iOS, Mac, Linux and Windows Platforms www.emotiv.com -­‐ Comparison http://bit.ly/1cfliTn 40
  • More Details on the Emotiv EPOC
  • ‹#› Expressiv Suite • Detect facial expressions ◦ Eyelid & eyebrow positions – Wink, blink, furrow ◦ Horizontal eye movement ◦ Smile, laugh, clenching & smirking • EEG sensors picking up signals to muscles (not brain waves) • Very fast (10ms) 42
  • ‹#› Affectiv Suite • Detect emotions / mental states ◦ Excitement ◦ Engagement / Boredom ◦ Meditation ◦ Frustration • Passive detection • Variable levels of each • Short & long term tracking 43
  • ‹#› Cognitiv Suite • Detect conscious thoughts ◦ Requires training • Detect 13 thought patterns ◦ 6 Movements: Left, right, up, down, forward, pull ◦ 6 Turns: CW, CCW, left, right, sway-back, sway-forward ◦ 1 Visualization: Disappear • Not as fast as Expresiv 44
  • Demonstration Emotiv EPOC SDK + Parrot AR.Drone
  • ‹#› MindTunes • Three Tetraplegics Musicians • Collaborated with DJ Fresh and Julien Castet (tech) • Created dance track • http://youtu.be/HyF4ZxGhPHw • Full documentary • http://youtu.be/PgfxKZiSCDQ 46
  • More Information • Email: jim.mckeeth@embarcadero.com • Emotiv EPOC: www.emotiv.com • Parrot AR.Drone: projects.ardrone.org • My blog: delphi.org • Slides: http://www.slideshare.net/jimmckeeth/jim-mc-keeth-wearable- ‹#› thought-input • Code: https://github.com/jimmckeeth/Delphi-Emotiv-EPOC 47