SlideShare a Scribd company logo
1 of 132
Download to read offline
BBB Workshop Overview
Audrey Girouard, Erin Treacy Solovey,
   Regan Mandryk, Desney Tan,
 Lennart Nacke, Robert J.K. Jacob
BBB Workshop: Bringing Together
BCI& Physiological Computing




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   3
Workshop Goals
1. Provide a platform for creating synergies
   between two related and emerging HCI
   disciplines (PC and BCI).

2. Kick-start BBB research in HCI by identifying
   key research questions and application areas

3. Enhancing HCI methodologies by adding these
   new methods and techniques to the toolbox of
   HCI researchers

        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   4
About Us




    Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   5
About you
• 25 Participants                                    •    X EEG papers
                                                     •    Y fNIRS papers
                                                     •    Z EMG papers
• 9 Brain papers
                                                     •    A GSR
• 19 Body papers
                                                     •    B Eye tracking
             Brain   Body


                            32%



    68%




          Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   6
Workshop Discussion Questions



                      <on your tables>




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   7
Workshop Discussion Questions


 What terms/acronyms should we use to
          describe our work?
    Psychophysiological computing seems a bit long…
                         BBB?
                          BBI?
                  Other suggestions?




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   8
Today’s Schedule
09:00-10:30      Introduction of organizers & participants, CHI madness style

10:30-11:00      Break

11:00-12:30      Group Discussion (assign groups + topic)

12:30-14:00      Lunch

14:00-15:30      Small group discussion (1h of discussion + 30 minutes of reporting)

15:30-16:00      Break

16:00-17:00      Report from small groups, recap, group discussion – barriers to
                 publication in HCI




              Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   9
Staying connected
Website     http://www.eecs.tufts.edu/~agirou01/workshop/

Twitter     http://twitter.com/bbbCHI2010

Twitter     #bbbCHI2010
Hashtag
Facebook    http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and-
            Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786

Flickr      http://www.flickr.com/groups/bbbchi/

Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological-
            user-interaction




              Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   10
And now… 2 Minute Introductions

• About your research:
  – Application or domain space? (e.g. video games)
  – Signal you are measuring? (e.g. fNIRS)
  – Device/hardware you are using? (e.g. ISS, Inc)


• What would you like to get out of workshop?
  – What problems are you facing?
  – What do you see in common with others here?
  – What device would you like to know more about?

        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   11
List of Presentations
1.    Adriane Randolph                        16.     Katarzyna Wac
2.    Krzysztof Z. Gajos                      17.     Feng Tian
3.    Saraswathi Bellur                       18.     Veronica Zammitto
4.    Danny Plass-Oude Bos                    19.     Johanna Octavia
5.    Kai Kuikkaniemi                         20.     Bob Wray
6.    Mitchel Benovoy                         21.     Marjolein van der Zwaag
7.    Yee Chieh Chew
8.    Martha Crosby
9.    Jan Kallenbach
10.   Eva Oliveira
11.   Lucia Filgueiras
12.   Marc Grootjen
13.   Melody Moore Jackson
14.   Johann Schrammel
15.   Daniel Sjölie


             Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   12
Brain-Computer Interfaces as Child’s
Play: Using Mental Training to Improve
        Motor Skills in Children


       Adriane B. Randolph, Ph.D.
            Ashley Ingraham
       Kennesaw State University
Outline
•   Research Position
     – Examine ties between sensorimotor area and physical ability to uncover a
       training tool to aid children’s motor skill development

•   Pilot Study
     –   3 able-bodied adults (2 Male, 1 Female)
     –   Acquire baseline Wii tennis score
     –   Conduct mental training with BCI2000 mu training protocol for 5 days
     –   Retest Wii tennis score for improvements      Participant  Session One                   Session Five

     –   Mixed results                                 AZ               98                            60
                                                                      CM                    70        64
                                                                      MW                    103       118
•   Future Work
     – Extend study with 20 able-bodied adults (currently in progress)
     – Analyze link between training and baseline ability
     – Test with elementary-aged children



               Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010              14
Screening for Mu
Ashley prepares participant to use mental imagery of hand and foot
movements to determine his best channels for use in mu BCI training before
he plays a match of Wii Tennis.


     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   15
Thank you


      Adriane B. Randolph, Ph.D.
       Kennesaw State University BrainLab

                   (770) 423-6083
              brainlab@kennesaw.edu
        http://coles.kennesaw.edu/brainlab/




    Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   16
Understanding How to Design Complex
    Brain-Controlled Applications

           Krzysztof Z. Gajos
How to Design Brain-Controlled
Applications?
• Basic operations




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   18
How to Design Brain-Controlled
Applications?




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   19
How to Design Brain-Controlled
Applications?




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   20
How to Design Brain-Controlled
Applications?
• Basic operations




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   21
How to Design Brain-Controlled
Applications?
• Interaction between primary task and BCI
  performance




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   22
How to Design Brain-Controlled
Applications?
• Error prevention, recovery, and actions
  with side effects




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   23
Thank you
Krzysztof Gajos
kgajos@eecs.harvard.edu
http://www.eecs.harvard.edu/~kgajos/




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   24
Psychophysiological Responses
     to Media Interfaces




       Saraswathi Bellur     S. Shyam Sundar


     Media Effects Research Lab
     http://www.psu.edu/dept/medialab
    Pennsylvania State University
Biopac MP 35 & MP 150
ACQknowledge software
                                                   Skin Conductance (EDA)
                                                   Heart Rate (ECG)
                                                   Facial EMG
                                                   Brain Wave (EEG)
                                                   Eye Movement (EOG)

                                                     Application Areas
                                                       Interface presentation
                                                       Information Processing
                                                       Usability Testing &
                                                       Evaluation
                                                        User Experience

Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   26
Physiological Measures: Affect, Arousal & Attention
                                                                                                      3. ECG: Heart
                                                                                                      Rate recording




1. EEG: Bipolar recordings      Occipital Alpha Blocking
                                                                                                       Orienting
                                                                                                       Response via
                                                                                                       BPM change




2. Skin Conductance (EDA)         Tonic arousal levels                                                  4. Facial EMG:
                                                                                                        Corrugator &
                                                                                                        Zygomatic
                                                                                                        recordings

                   Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010             27
Theoretical Frameworks
•   The Media Equation: Mediated Life = Real Life (Reeves & Nass,
    1996).
•   Structural aspects of media technologies: Automatic or
    Controlled processing? Voluntary or Involuntary responses?
•   Specific cues triggered via interactions with four types of
    media affordances, MAIN model (Sundar, 2008)

                                                                           Animation
    M: MODALITY                              Customization                                      Heuristics

      A: AGENCY
                                             Information                                        Perceptual
                                                 Scent                                          Bandwidth
    I: INTERACTIVITY

    N: NAVIGABILITY                                   Contingency                   UGC & Sourceness


             Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010                28
Thank you
Saras Bellur (saras@psu.edu)
S. Shyam Sundar (sss12@psu.edu)

              Media Effects Research Lab
               http://www.psu.edu/dept/medialab
            Pennsylvania State University




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   29
BCI for Games

   Danny Plass-Oude Bos
 Boris Reuderink, Bram van de Laar,
  Hayrettin Gürkök, Christian Mühl,
Mannes Poel, Anton Nijholt, Dirk Heylen
  HMI, University of Twente
Target Group: the general population




Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   31
BCI: issues




Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   32
BCI: new information, new features




Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   33
Acceptance: Usability & Realistic Settings




Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   34
Subjects  Users




Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   35
Thank you
d.plass@ewi.utwente.nl
http://hmi.ewi.utwente.nl/~oudebos




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   36
Designing Biofeedback for
Games and Playful Applications
              Kai Kuikkaniemi
     Helsinki Institute for Information
                Technology
Emoshooter




    Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   38
Emoshigeru - cinema games
                                                                        Real-time
                                                                        Public display
                                                                        Biosignal adaptation
                                                                        Massive co-located
                                                                        Shooter / Strategy
                                                                        Part of Shigeru-games
                     local
                     wlan



   Public display
 Connected to game
      server




         Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Presemo – presentation engine




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emolisten – biosignal sonification




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Research statement
• Social Games and Playful Applications,
  which Utilize Biosignals
• Grey Area between Adaptation and
  Biofeedback
  – Implicit and Explicit Biofeedback
• Biosignal Synchrony (Inter-Personal)



       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   42
Our team and projects
Professor Marko Turpeinen
Professor Timo Saari
Ilkka Kosunen, Toni Laitinen, Petri Lievonen, Pauli Ojala

Fun of Gaming (EU FP6) (2006-2009)
Enactive Social Media and Games (Aalto) (2009)
Emokeitai (Tekes – National Innovation Agency) (2009-2011)
Thanks:
CKIR / Niklas Ravaja


          Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   43
Psychophysiological signal
  analysis and classification
                 Mitchel Benovoy
Centre for Intelligent Machine, McGill University
Outline
• Signals

• Hardware

• Application domains




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   45
Signals
                                 Galvanic Skin Response                                                                            Phalange Temperature
               1.5
∆ Cond. (µS)




                    1

               0.5

                    0

               -0.5
                               0.5            1              1.5            2               2.5
                                                    Sample                                  4
                                                                                         x 10




                                                                                    Electromyogram




                   42
                        Blood Volume Pulse – HR & Amp.                                                                     Respiration – Rate & Amp.
                   40                                                                                         2.2
                                                                                                                2
                   38
      Refl.




                                                                                                  Exp.
                                                                                                              1.8
                   36                                                                                         1.6
                   34                                                                                         1.4
                   32                                                                                         1.2
                        1000   2000   3000   4000   5000     6000   7000   8000   9000    10000                     1000    2000   3000   4000   5000   6000   7000   8000   9000        10000
                   7
                                                                                                               1
                                                                                                              0.8
            Amp.




                                                                                                  Amp.
                   6                                                                                          0.6
                                                                                                              0.4
                   5                                                                                          0.2
                        1000   2000   3000   4000   5000     6000   7000   8000   9000    10000                     1000    2000   3000   4000   5000   6000   7000   8000   9000        10000
               105
                                                                                                              30
 HR (bpm)




                                                                                                   RR (bpm)




               100
                                                                                                              25
                   95                                                                                         20

                   90                                                                                         15
                        1000   2000   3000   4000   5000     6000   7000   8000   9000    10000                     1000    2000   3000   4000   5000   6000   7000   8000   9000        10000




                                             Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010                                                       46
Hardware
• Thought Technology ProCom Infinity




                                                                       Images via www.thoughttechnology.com
       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010            47
Application Domains
• Affective Computing




  – Interaction evaluation (UI, devices, latency)
  – Music & emotion research
  – Psychoneurology
        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   48
Thank you




         benovoym@cim.mcgill.ca




    Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   49
BCI and Creativity

  Yee Chieh (Denise) Chew
Georgia Institute of Technology
Outline
• fNIR and Creative Expression
• EEG and Music




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   51
Character Templates for fNIR
Created with a mouse, these are templates that subjects traced
using fNIR



Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   52
Thank you
ychew@gatech.edu




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   53
Physiological Measures Used for
Identification of Cognitive States and
     Continuous Authentication
   Curtis S. Ikehara & Martha E. Crosby
     University of Hawaii at Manoa
Outline
• Overview of our Research Using
  Physiological sensors
  – Assessing Cognitive States
  – Continuous Authentication
• Possible Applications




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   55
Multi-modal Physiological Sensors System
                                                                                      Eye Tracker
                                                                            (Gaze Position, Fixation Number, Fixation
                                                                           Duration, Repeat Fixations, Search Patterns,
         MTF                                                                  Pupil Size, Blink Rate, Blink Duration)
(Moving Target Fractions)                  HUMAN
                                                                              Pressure Mouse
                                                                          (Pressures on Mouse Button and Body)



           Laptop
(Data Collection, Analysis & Display)
                                                         EDA / TSA Finger Sensors
                                                           (Electrodermal and Temperature Sensor Activity)

                 Oximeter
              (Pulse, Blood Oxygen)

                                                            Custom Electronics
                                                                (Analog to Digital Converter)
                      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   56           56
A Composite of Over 20 Mouse Clicks of
       Three People (Click Signature)
US Patent # 7,245,218




                                                                                         Subject 4




                         Subject 5                                                       Subject 7
                                                                                                     57
      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Possible Applications
• The student’s confidence can be assessed by
  evaluating the distortions of the mouse click
  signature.
• High Value Vehicle Control
  – Pressure and other physiological sensors can be
    attached to a steering (e.g. aircraft yoke or
    steering wheel) .
  – Sensors can be used to indicate that:
     • The authorized pilot or driver is in control.
     • The cognitive state of the pilot is consistent with the
       vehicle’s activity.

        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   58
Thank you
   Curtis S. Ikehara                                  Martha E. Crosby
cikehara@hawaii.edu                                  crosby@hawaii.edu


       University of Hawaii at Manoa
     1680 East-West Road - POST 317
      Honolulu, Hawaii 96822 USA




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   59
Measuring Interaction Experiences:
     Integration of Multiple
  Psychophysiological Methods
          Jan Kallenbach
          Aalto University
          Helsinki, Finland
Outline
               User Experience
                       =
             Choice & Interaction
                       +
            Cost-Benefit Evaluations

   Measuring Interaction Experiences?
                   
        Software Framework
     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   61
Multimodal
Measurements




               Client Application for UX Research
               Sampling frequency: max 1000Hz
               Psychophysiological Measures: Gaze, Pupil Size, EDA, EMG, EEG, …
               Behavioral Measures: Keyboard, Mouse, Slider, …
               System Measures: Window Events


               Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   62
Thank you
Jan Kallenbach
Research Scholar


Aalto University
School of Science & Technology
Department of Media Technology
P.O. Box 15500
FIN - 00076 Aalto




          Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   63
Towards enhanced video access and
 recommendation through emotions

             Eva Oliveira
            Nuno Ribeiro
          Teresa Chambel
           IPCA | LASIGE
We Want
• To access to videos based on emotions
  – user actual emotional (psychophysiological) state
  – video emotional content
  – director’s perspective
• To recommend videos based on emotions
  – create a set of rules from every affective
    classification perspective (user and video, director)




          Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   65
Affective video recommendation
system requirements ()
• Every video must be classified affectively from the user
  and from the content perspective.
• Every user has an emotional profile constituted by all his
  implicit (physiological) classification of every classified
  movie.
• User physiological signals can change the way we watch
  the videos




           Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   66
User Classification Issues
• Automatic Classification
  – Users emotional state
     • emotional model
  – Classification techniques
     • Result output
     • Accuracy
     • Lab contexts




         Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   67
Current State
• Capturing users phychophysiological signals
  (BIOPAC System) while watching movies




                        This is my exciting picture


• Next step: several classification techniques to
  convert signal result into emotional categories
         Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   68
Thank you
• Eva Oliveira :


• eva.oliveira@gmail.com




        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   69
Your users can’t talk!
How can you tell they liked
     your system?
 Lucia Filgueiras and Claudia Tambascia
            Escola Politecnica
        Universidade de São Paulo
What we want to do
• Develop products for cerebral palsy users
• Perform participatory design
• Understand non-verbal communication




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   71
Direct measures and usability evaluation
HRV and workload measures                                                                                                                                                                                                               Eye tracking in iTV apps
in control room operability                                                                                                                                                                                                             evaluation
evaluation
                                                                                                                                                                                                                                          Comparison of eye tracker
                                               NASA TLX, SWAT, Heart Rate
                                                                                                                                                                                                                                          findings to conventional
                                               Variability                                                                                                                                                                                usability tests results
                                                                                                    Variabilidade da frequência cardíaca OP13

                                                                                                                       LF nu / HF nu                 LSC           Média

                                               18,0
                                                                                                                                Operador analisa sistema após vários
 Índice do Balanço Simpático? Parassimpático




                                                                                            Ação do operador para                alarmes: temperatura alta (10:29);                            Ação do operador para
                                               16,0
                                                                                                  correção                         nível alto (10:32) e segurança                                    correção
                                               14,0                                                                                    (10:32; 10:37; 10:43)                     14,1

                                               12,0                        Alarme de                              11,7
                                                                        temperarura alta:
                                               10,0                          10:15
                                                                                                                                9,1
                                                                8,3                                                                                                                                                        8,5
                                                8,0
                                                                                                                                                                                                             7,3
                                                                                                                                                      6,6
                                                6,0                                                                                                                                                                               5,9
                                                                        5,2                                                                                                             5,3 5,6
                                                                                                    4,6 5,0
                                                4,0                                          3,9                                       3,8                  4,0 4,4                                   4,1
                                                                                                                                              2,9                                                                   3,3
                                                          2,6                 2,6 2,6                                    2,3                                              2,4
                                                2,0

                                                0,0
                                                      5

                                                            0

                                                                    5

                                                                          0

                                                                                  5

                                                                                        0

                                                                                                5

                                                                                                      0

                                                                                                              5

                                                                                                                    0

                                                                                                                            5

                                                                                                                                   0

                                                                                                                                          5

                                                                                                                                                 0

                                                                                                                                                        5

                                                                                                                                                               0

                                                                                                                                                                      5

                                                                                                                                                                             0

                                                                                                                                                                                    5

                                                                                                                                                                                           0

                                                                                                                                                                                                  5

                                                                                                                                                                                                         0

                                                                                                                                                                                                                5

                                                                                                                                                                                                                       0

                                                                                                                                                                                                                              5
                                                  :3

                                                          :4

                                                                :4

                                                                        :5

                                                                              :5

                                                                                      :0

                                                                                            :0

                                                                                                    :1

                                                                                                          :1

                                                                                                                  :2

                                                                                                                         :2

                                                                                                                                :3

                                                                                                                                       :3

                                                                                                                                              :4

                                                                                                                                                     :4

                                                                                                                                                            :5

                                                                                                                                                                   :5

                                                                                                                                                                          :0

                                                                                                                                                                                 :0

                                                                                                                                                                                        :1

                                                                                                                                                                                               :1

                                                                                                                                                                                                      :2

                                                                                                                                                                                                             :2

                                                                                                                                                                                                                    :3

                                                                                                                                                                                                                           :3
                                                 09

                                                       09

                                                               09

                                                                     09

                                                                             09

                                                                                   10

                                                                                           10

                                                                                                 10

                                                                                                         10

                                                                                                               10

                                                                                                                       10

                                                                                                                              10

                                                                                                                                     10

                                                                                                                                            10

                                                                                                                                                   10

                                                                                                                                                          10

                                                                                                                                                                 10

                                                                                                                                                                        11

                                                                                                                                                                               11

                                                                                                                                                                                      11

                                                                                                                                                                                             11

                                                                                                                                                                                                    11

                                                                                                                                                                                                           11

                                                                                                                                                                                                                  11

                                                                                                                                                                                                                         11




                                                                                                                                       HORA


 Heart Rate Variability of an operator in a unstable unit situation



                                                                                                                    Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our questions:
•   How personalized are body expressions for cerebral palsy patients?

•   Are there general body expressions that can mean the same for everyone?

•   Is it possible to determine a generic way of measuring these expressions?

•   Is there any other liaison besides gestures and facial expressions , like an
    unconscious connection of limbic systems that can be responsible for the
    mutual sensitive communication?

•   Is there a reference body signal that can be used as a reliable parameter to
    evaluate body expressions?

•   How can we establish a correlation between body expressions and
    patient´s opinion of approval or disapproval of a design solution?




              Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   73
Thank you!
Contact us:

Lucia V. L. Filgueiras
Escola Politecnica, Universidade de São Paulo
lucia.filgueiras@poli.usp.br

Claudia A.Tambascia
Fundação CPqD
claudiat@cpqd.com.br


                                                                     http://lts-i.pcs.usp.br

         Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   74
Human Computer Symbiosis
 by Mutual Understanding
        Marc Grootjen
      EagleScience-TNO
Outline
• Symbiosis & mutual understanding
• Information models
• Interfaces




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   76
Maximum task performance



  Symbiotic entity
                                                                                      Disturbance
                               Mutual
        human               understanding               computer




                             Tasks (goals)

Human-computer symbiosis with mutual understanding
as key issue for maximum task performance

   Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010                 77
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   78
Domesticating Brain-
Computer Interfaces: Systems
      for the Home
        Melody Moore Jackson
                BrainLab
    School of Interactive Computing
             Georgia Tech
Extended Home-based BCI study

• fNIR-based BCI for communication
• 32 People with ALS (so far) over 1 year
• 78% (25) could operate the device on
  initial visit with accuracy > 70%
• Accuracy to 100%, average 76.5%
• Issues:
  – Motivation
  – Data Collection
  – Remote Support
       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   80
The Kokoro Gatari
Brain Computer Interface based on Functional Near Infrared
imaging



Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   81
Thank you
Melody Moore Jackson
melody@cc.gatech.edu
404-277-1324

Ian McClendon
imclend@cc.gatech.edu
404-641-7712

      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   82
Get up, move on! Using
 Electromyography to Explore
the Relationship of Experience
          and Motion
         Johann Schrammel
   Centre for Usability Research and
              Engineering
Outline
• Compare experiences and emotions in
  conditions with and without movements
  with Electromyography(EMG) and other
  methods




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   84
Outline
• Can EMG successfully be used to
  estimate a user‘s emotional experience in
  motion contexts (artefacts, feasibility,
  etc.)?




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   85
Outline
• Is a more embodied way of controlling
  interaction actually amplifying experience?




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   86
Results
• EMG is suitable for motion contexts.
• Both EMG and self-report data indicate
  that natural body movement intensifies
  emotions.
• It’s crucial that body movements are in
  accordance with activity.



       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   87
Thank you
schrammel@cure.at




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   88
Reality-Based Brain-
Computer Interaction
        Daniel Sjölie
Umeå University, Umeå, Sweden
Reality-Based Brain-Computer
Interaction
• Complex, reality-based, interaction with the
  ability to adapt to the user in real-time
  – Using brain measurements to evaluate
• Direct brain-computer interaction
  – The user can be unaware
• We’re using a combination of virtual
  reality (VR) and functional MRI (fMRI)


       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   90
HCI and cognition
• An improved understanding of the brain in this
  interactive and complex context is needed
• Build on theories of cognition as a foundation
  for the development of advanced HCI
  – As interaction moves towards being reality-based
    we move closer to “interacting with reality”, and
    thus to general cognition
• Cognitive neuroscience and computational
  models of the brain are important tools

        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   91
Working with fMRI and Virtual Reality
Working with fMRI requires specialist competence and forces us to
deal with powerful magnetic fields and restricted mobility, but the
ability to measure the whole brain is an invaluable research tool.


Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   92
Thank you
Daniel Sjölie, daniel@cs.umu.se




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   93
emoBAN:
    Improving Quality of Life
via Psychophysiological Mobile
          Computing
    Katarzyna Wac, Anind K. Dey
   Carnegie Mellon University, HCII
emoBAN idea
• “what always speaks silently is the body” (Brown, 1990)
   – measure QoL via psychophysiology




         Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   95
emoBAN: Usable Useful Functional
• Body Area Network
  – sensors
    • vital signs, e.g., ECG, temp, resp
    • context, e.g., location, time, activity
  – actuators
    • feedback via, e.g., AV, tactile, light




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   96
Thank you
Dr Katarzyna Wac
Carnegie Mellon University
Human-Computer Interaction Institute
katewac@cmu.cs.edu
www.cui.unige.ch/~wac




           Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   97
Leveraging Psychophysical Data in
     Monitoring and Analyzing the
      States of Badminton Players
Feng Tian, Wencan Luo, Jing Dai, Reza Naima, Yongmin Cheng, Hongan
                          Wang, Guozhong Dai

         Institute of Software, Chinese Academy of Sciences
         Physical Education Collage, Beijing Sport University
               Berkeley Institute of Design, UC. Berkeley
   China Badminton Team, General Administration of Sport of China
                                4/20/2010
Traces and physical
                                    actions
                                    - Camera, accelerometers




                                        Psychophysical data
                                        - Berkeley Tricorder
                                         ECG, EMG, Pulse Oximetry,
                                         BioImpedance Respiration

Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Exploring Quantitative Methods
  for Evaluating Sports Games
           Veronica Zammitto
         Simon Fraser University
            Magy Seif El-Nasr
         Simon Fraser University
              Paul Newton
        Electronic Arts Canada
Current Study: NBA Live 10
• Eye tracking
  – Eye Tech TM3
• Psycho-physiological data
  – ProComp5 Infiniti
  – EMG (corrugator supercilii and zygomatic major),
    EKG (heart rate), GSR and BVP
• Gameplay Telemetry Data
  – event-based RPC telemetry collection system
• Triangulation
         Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   102
What was the user
                                                                                                       the
Data Triangulation - NBA Live 10                                                           emotional response?
                                                                                           looking at?
Game user experience
                                                                                          Event, when
                                                                                           Was positive?
                                                                                           Where user’s
                                                                                           Was negative?
                                                                                           attention was?
                Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010            103
Data Triangulation - NBA Live 10
Game user experience




                Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   104
Thank you
Veronica Zammitto
vzammitt@sfu.ca

Magy Seif El-Nasr
magy@sfu.ca

Paul Newton
pnewton@ea.com

     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   105
Exploring Psycho-physiological Measures for
the Design and Behavior of Socially-Aware
     Avatars in Ubicomp Environments

            Johanna Renny Octavia
            Hasselt University, Belgium
Avatars in Ubicomp Environment
 User’s representation  Mediator between system and user (companion)
 To design a socially-aware avatar that adapts its behavior according to user’s emotion
 Any incorrect actions of the avatar can lead to loss of identification of the user



             Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   107
Our informal experiment
 ProComp Infiniti device, measured GSR, EMG and BVP
 Higher physiological values when playing against a friend
 How different social relationships between players influence those values?


      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   108
Thank you




Expertise Centre for Digital Media (EDM)
Web: www.edm.uhasselt.be
Email:     johanna.octavia@uhasselt.be



        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   109
Tailoring Practice for
Individualized Learning via
      Cognitive State
          Bob Wray
    Soar Technology, Inc.
Why I need Psychophysiological
Computing
• Practice environments are becoming
  common-place
• Adaptive tailoring of practice is important for
  individualized experience/learning
• Need more than traditional adaptations (e.g.,
  scaffolding)
  – Adapting to engage learner / maintain interest
  – Adaptation to reducing “gaming” in environment
  – Adaptation to elicit learner affect/arousal (“real”)

        Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   111
Research Testbed
Narrative, conversational, inter-cultural training application

Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   112
Thank you
Bob Wray
Soar Technology
wray@soartech.com, 919.967.5079

Brian Magerko
Georgia Institute of Technology
magerko@git.edu

       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   113
Guidelines to biosignal
     driven HCI
      Marjolein van der Zwaag

Philips Research Europe, University of
             Groningen
Guidelines to biosignal
     driven HCI
   Marjolein van der Zwaag
    Egon L. van den Broek
       Joris H. Janssen
Outline
• Signal(s) measured: psycho-physiological
• Application domain: Affective computing
• Device used: Nexus 10, mind media

• Guidelines for processing biosignals
  –   Triangulation
  –   Identification of users
  –   Biosignal characteristics
  –   Temporal construction

          Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   116
Unobtrusive affective computing
Music direct your mood concept




Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   117
Thank you




Marjolein.van.der.zwaag@philips.com

      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   118
Workshop Discussion Questions



 What is brain/body computing good for?

How can we think systematically about this
                space?



      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   120
Workshop Discussion Questions




What are the major themes of HCI that can
 be explored using brain/body interaction?




      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   121
Workshop Discussion Questions

What specific brain & body sensing modalities
 are interesting and how can we use them?

What are most innovative technologies in this
                  space?

How do these compare with existing practices
 (NASA-TLX, Likert-style questionnaires, etc)?

      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   122
Workshop Discussion Questions
How can BCI and PC be defined within an HCI
                 context?

    How is a HCI context different from
 neuropsychology, biomedicine or biomedical
               engineering?

        How does HCI contribute to
   psychophysiological computing (or more
     specifically neural engineering, bio-
              engineering, etc)?
      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   123
Workshop Discussion Questions
If we could measure anything on brain and
  body, what would we want to measure and
                   why?

 What cognitive states and emotions have
     been investigated with sensors?

  What methodologies are best for what
               signals?
      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   124
Workshop Discussion Questions



    How can accurate physiological,
   psychological and neurophysiological
      measurements improve HCI?




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   125
Workshop Discussion Questions

   With what software techniques and
 hardware technologies could these signals
         be optimally processed?

   How can software account for the
   combination of user actions taken to
  accomplish a task versus actions that
 would surprise the user, yet optimize their
               experience?
      Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   126
Workshop Discussion Questions



What are the issues that must be considered
    when using brain/body computing for
 direct control, passive sensing, adaptation,
                     etc?



       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   127
Groups
• How to go from raw data to something
  else
  – Emotions: TABLE 1
  – Data Fusion: TABLE 4
• How to use these things in interactive
  interfaces: TABLE 3



       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   128
BBB Workshop: Bringing Together
BCI& Physiological Computing




     Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   129
Staying connected
Website     http://www.eecs.tufts.edu/~agirou01/workshop/

Twitter     http://twitter.com/bbbCHI2010

Twitter     #bbbCHI2010
Hashtag
Facebook    http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and-
            Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786

Flickr      http://www.flickr.com/groups/bbbchi/

Grouplist   http://groups.google.com/group/brain-body-interfaces/

Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological-
            user-interaction


              Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   130
BBB at CHI2010
Brains and Brawn session: Tuesday 11:30-1pm

Making Muscle-Computer Interfaces More Practical
          T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, James A. Landay
          University of Washington, USA
          Microsoft Corporation, USA
A Novel Brain-Computer Interface Using a Multi-Touch Surface
          Beste F. Yuksel, Michael Donnerer, James Tompkin, Anthony Steed.
          University College London, UK
The Influence of Implicit and Explicit Biofeedback in First-Person Shooter Games
          Kai Kuikkaniemi, Toni Laitinen, Marko Turpeinen, Timo Saari, Ilkka Kosunen, Niklas Ravaja,
          Helsinki Institute for Information Technology, Finland
          Center for Knowledge and Innovation Research, Finland
Effects of Interactivity and 3D-motion on Mental Rotation Brain Activity in an Immersive Virtual
Environment
          Daniel Sjölie, Kenneth Bodin, Eva Elgh, Johan Eriksson, Lars-Erik Janlert, Lars Nyberg
          Umeå University, Sweden


                Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010       131
THANK YOU
• Keep in touch!




       Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010   132

More Related Content

Viewers also liked

Towards a Predictive Model for Mobile Internet Quality
Towards a Predictive Model for Mobile Internet QualityTowards a Predictive Model for Mobile Internet Quality
Towards a Predictive Model for Mobile Internet QualityKatarzyna Wac & The QoL Lab
 
mHealth Services Science - Science in the Service of Life Quality
mHealth Services Science - Science in the Service of Life QualitymHealth Services Science - Science in the Service of Life Quality
mHealth Services Science - Science in the Service of Life QualityKatarzyna Wac & The QoL Lab
 
iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
iSenseStress: Assessing Stress Through Human-Smartphone Interaction AnalysisiSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
iSenseStress: Assessing Stress Through Human-Smartphone Interaction AnalysisKatarzyna Wac & The QoL Lab
 
Factors Influencing Quality of Experience of Commonly-Used Mobile Applications
Factors Influencing Quality of Experience of Commonly-Used Mobile ApplicationsFactors Influencing Quality of Experience of Commonly-Used Mobile Applications
Factors Influencing Quality of Experience of Commonly-Used Mobile ApplicationsKatarzyna Wac & The QoL Lab
 
Star Trek’s Tricorder: Science Fiction or Future Science?
Star Trek’s Tricorder: Science Fiction or Future Science?Star Trek’s Tricorder: Science Fiction or Future Science?
Star Trek’s Tricorder: Science Fiction or Future Science?Katarzyna Wac & The QoL Lab
 
Innovations for Global Health Challenges: Private Sector and Research View
Innovations for Global Health Challenges: Private Sector and Research ViewInnovations for Global Health Challenges: Private Sector and Research View
Innovations for Global Health Challenges: Private Sector and Research ViewKatarzyna Wac & The QoL Lab
 
Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...
Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...
Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...Katarzyna Wac & The QoL Lab
 

Viewers also liked (8)

Towards a Predictive Model for Mobile Internet Quality
Towards a Predictive Model for Mobile Internet QualityTowards a Predictive Model for Mobile Internet Quality
Towards a Predictive Model for Mobile Internet Quality
 
mHealth Services Science - Science in the Service of Life Quality
mHealth Services Science - Science in the Service of Life QualitymHealth Services Science - Science in the Service of Life Quality
mHealth Services Science - Science in the Service of Life Quality
 
iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
iSenseStress: Assessing Stress Through Human-Smartphone Interaction AnalysisiSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
iSenseStress: Assessing Stress Through Human-Smartphone Interaction Analysis
 
Factors Influencing Quality of Experience of Commonly-Used Mobile Applications
Factors Influencing Quality of Experience of Commonly-Used Mobile ApplicationsFactors Influencing Quality of Experience of Commonly-Used Mobile Applications
Factors Influencing Quality of Experience of Commonly-Used Mobile Applications
 
MIQModel: Predictive Model for Mobile Internet
MIQModel: Predictive Model for Mobile InternetMIQModel: Predictive Model for Mobile Internet
MIQModel: Predictive Model for Mobile Internet
 
Star Trek’s Tricorder: Science Fiction or Future Science?
Star Trek’s Tricorder: Science Fiction or Future Science?Star Trek’s Tricorder: Science Fiction or Future Science?
Star Trek’s Tricorder: Science Fiction or Future Science?
 
Innovations for Global Health Challenges: Private Sector and Research View
Innovations for Global Health Challenges: Private Sector and Research ViewInnovations for Global Health Challenges: Private Sector and Research View
Innovations for Global Health Challenges: Private Sector and Research View
 
Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...
Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...
Quality of Life Technologies: From Fundamentals of Mobile Computing to Patter...
 

Similar to Brain, Body, and Bytes CHI 2010 Workshop Presentations

Brain Computer Interface
Brain Computer Interface Brain Computer Interface
Brain Computer Interface Deepti Singh
 
Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...
Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...
Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...Md Jobair Hossain Faruk
 
What are most promising lifestyle and tech options to harness lifelong neurop...
What are most promising lifestyle and tech options to harness lifelong neurop...What are most promising lifestyle and tech options to harness lifelong neurop...
What are most promising lifestyle and tech options to harness lifelong neurop...SharpBrains
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacingKirtan Shah
 
neuralinterfacing-130331081952-phpapp01.pdf
neuralinterfacing-130331081952-phpapp01.pdfneuralinterfacing-130331081952-phpapp01.pdf
neuralinterfacing-130331081952-phpapp01.pdfPrasannaKumarpanda2
 
Brain computer interfaces in medicine
Brain computer interfaces in medicineBrain computer interfaces in medicine
Brain computer interfaces in medicineKarlos Svoboda
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacingBrujath Bru
 
asjadpresentation
asjadpresentationasjadpresentation
asjadpresentationAsjad K.
 
SharpBrains @ New York Public Library
SharpBrains @ New York Public LibrarySharpBrains @ New York Public Library
SharpBrains @ New York Public LibrarySharpBrains
 
Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...
Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...
Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...HCI Lab
 
Brain controlled-interfaces
Brain controlled-interfaces Brain controlled-interfaces
Brain controlled-interfaces N0696142
 
BRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACEBRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACERushi Prajapati
 
Kaplan alexander brain computer interface
Kaplan alexander brain computer interfaceKaplan alexander brain computer interface
Kaplan alexander brain computer interfaceigorod
 
Individual functional atlasing of the human brain with multitask fMRI data: l...
Individual functional atlasing of the human brain with multitask fMRI data: l...Individual functional atlasing of the human brain with multitask fMRI data: l...
Individual functional atlasing of the human brain with multitask fMRI data: l...Ana Luísa Pinho
 
Growing evidence for separate neural mechanisms for attention and consciousne...
Growing evidence for separate neural mechanisms for attention and consciousne...Growing evidence for separate neural mechanisms for attention and consciousne...
Growing evidence for separate neural mechanisms for attention and consciousne...Nao (Naotsugu) Tsuchiya
 
BRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACEBRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACEnitish_kumar
 

Similar to Brain, Body, and Bytes CHI 2010 Workshop Presentations (20)

Brain Computer Interface
Brain Computer Interface Brain Computer Interface
Brain Computer Interface
 
Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...
Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...
Poster Presentation: An Investigation on Non-Invasive Brain-Computer Interfac...
 
What are most promising lifestyle and tech options to harness lifelong neurop...
What are most promising lifestyle and tech options to harness lifelong neurop...What are most promising lifestyle and tech options to harness lifelong neurop...
What are most promising lifestyle and tech options to harness lifelong neurop...
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacing
 
Neural Interfacing
Neural Interfacing Neural Interfacing
Neural Interfacing
 
neuralinterfacing-130331081952-phpapp01.pdf
neuralinterfacing-130331081952-phpapp01.pdfneuralinterfacing-130331081952-phpapp01.pdf
neuralinterfacing-130331081952-phpapp01.pdf
 
Brain computer Interface
Brain computer InterfaceBrain computer Interface
Brain computer Interface
 
Brain computer interfaces in medicine
Brain computer interfaces in medicineBrain computer interfaces in medicine
Brain computer interfaces in medicine
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacing
 
asjadpresentation
asjadpresentationasjadpresentation
asjadpresentation
 
SharpBrains @ New York Public Library
SharpBrains @ New York Public LibrarySharpBrains @ New York Public Library
SharpBrains @ New York Public Library
 
Brain computer interface
Brain computer interfaceBrain computer interface
Brain computer interface
 
Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...
Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...
Augmenting Speech-Language Rehabilitation with Brain Computer Interfaces: An ...
 
Brain controlled-interfaces
Brain controlled-interfaces Brain controlled-interfaces
Brain controlled-interfaces
 
BRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACEBRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACE
 
Kaplan alexander brain computer interface
Kaplan alexander brain computer interfaceKaplan alexander brain computer interface
Kaplan alexander brain computer interface
 
BRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACEBRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACE
 
Individual functional atlasing of the human brain with multitask fMRI data: l...
Individual functional atlasing of the human brain with multitask fMRI data: l...Individual functional atlasing of the human brain with multitask fMRI data: l...
Individual functional atlasing of the human brain with multitask fMRI data: l...
 
Growing evidence for separate neural mechanisms for attention and consciousne...
Growing evidence for separate neural mechanisms for attention and consciousne...Growing evidence for separate neural mechanisms for attention and consciousne...
Growing evidence for separate neural mechanisms for attention and consciousne...
 
BRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACEBRAIN COMPUTER INTERFACE
BRAIN COMPUTER INTERFACE
 

More from Lennart Nacke

How to Write a CHI Paper 2016 (CHI PLAY 2016)
How to Write a CHI Paper 2016 (CHI PLAY 2016)How to Write a CHI Paper 2016 (CHI PLAY 2016)
How to Write a CHI Paper 2016 (CHI PLAY 2016)Lennart Nacke
 
Games institute: University of California Visit: Game Thinking Overview
Games institute: University of California Visit: Game Thinking OverviewGames institute: University of California Visit: Game Thinking Overview
Games institute: University of California Visit: Game Thinking OverviewLennart Nacke
 
Graphics Interface 2019: Invited Speaker: Lennart Nacke - Game Thinking
Graphics Interface 2019: Invited Speaker: Lennart Nacke - Game ThinkingGraphics Interface 2019: Invited Speaker: Lennart Nacke - Game Thinking
Graphics Interface 2019: Invited Speaker: Lennart Nacke - Game ThinkingLennart Nacke
 
GAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification researchGAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification researchLennart Nacke
 
Introduction to Game Thinking (Fluxible 2018)
Introduction to Game Thinking (Fluxible 2018)Introduction to Game Thinking (Fluxible 2018)
Introduction to Game Thinking (Fluxible 2018)Lennart Nacke
 
GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...
GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...
GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...Lennart Nacke
 
Game UX Summit '17: Challenges of Evaluating Player Cognition & Emotion
Game UX Summit '17: Challenges of Evaluating Player Cognition & EmotionGame UX Summit '17: Challenges of Evaluating Player Cognition & Emotion
Game UX Summit '17: Challenges of Evaluating Player Cognition & EmotionLennart Nacke
 
DAC 300: Chance and Skill in Game Design
DAC 300: Chance and Skill in Game DesignDAC 300: Chance and Skill in Game Design
DAC 300: Chance and Skill in Game DesignLennart Nacke
 
DAC 305: Choice and Agency in Gamification
DAC 305: Choice and Agency in GamificationDAC 305: Choice and Agency in Gamification
DAC 305: Choice and Agency in GamificationLennart Nacke
 
Brief Introduction to Game Design
Brief Introduction to Game DesignBrief Introduction to Game Design
Brief Introduction to Game DesignLennart Nacke
 
The 5 Gamification Languages: The secret to gameful experiences that last (Ga...
The 5 Gamification Languages: The secret to gameful experiences that last (Ga...The 5 Gamification Languages: The secret to gameful experiences that last (Ga...
The 5 Gamification Languages: The secret to gameful experiences that last (Ga...Lennart Nacke
 
#GamesUR Conference: From Body Signals to Brainy Player Insights
#GamesUR Conference: From Body Signals to Brainy Player Insights#GamesUR Conference: From Body Signals to Brainy Player Insights
#GamesUR Conference: From Body Signals to Brainy Player InsightsLennart Nacke
 
Social Player Analytics in a Facebook Health Game (HCI Korea)
Social Player Analytics in a Facebook Health Game (HCI Korea)Social Player Analytics in a Facebook Health Game (HCI Korea)
Social Player Analytics in a Facebook Health Game (HCI Korea)Lennart Nacke
 
Biofeedback Gaming: The Future of Game Interaction?
Biofeedback Gaming: The Future of Game Interaction?Biofeedback Gaming: The Future of Game Interaction?
Biofeedback Gaming: The Future of Game Interaction?Lennart Nacke
 
On the Usability of Psychophysiological User Research for the Games Industry
On the Usability of Psychophysiological User Research for the Games IndustryOn the Usability of Psychophysiological User Research for the Games Industry
On the Usability of Psychophysiological User Research for the Games IndustryLennart Nacke
 
CHI 2011: Biofeedback Game Design
CHI 2011: Biofeedback Game DesignCHI 2011: Biofeedback Game Design
CHI 2011: Biofeedback Game DesignLennart Nacke
 
Affective Ludology: Affective Measurement of User Experience in Games
Affective Ludology: Affective Measurement of User Experience in GamesAffective Ludology: Affective Measurement of User Experience in Games
Affective Ludology: Affective Measurement of User Experience in GamesLennart Nacke
 
Playability & Player Experience Research
Playability & Player Experience ResearchPlayability & Player Experience Research
Playability & Player Experience ResearchLennart Nacke
 
Game Metrics and Biometrics: The Future of Player Experience Research
Game Metrics and Biometrics: The Future of Player Experience ResearchGame Metrics and Biometrics: The Future of Player Experience Research
Game Metrics and Biometrics: The Future of Player Experience ResearchLennart Nacke
 
From Playability to a Hierarchical Game Usability Model
From Playability to a Hierarchical Game Usability ModelFrom Playability to a Hierarchical Game Usability Model
From Playability to a Hierarchical Game Usability ModelLennart Nacke
 

More from Lennart Nacke (20)

How to Write a CHI Paper 2016 (CHI PLAY 2016)
How to Write a CHI Paper 2016 (CHI PLAY 2016)How to Write a CHI Paper 2016 (CHI PLAY 2016)
How to Write a CHI Paper 2016 (CHI PLAY 2016)
 
Games institute: University of California Visit: Game Thinking Overview
Games institute: University of California Visit: Game Thinking OverviewGames institute: University of California Visit: Game Thinking Overview
Games institute: University of California Visit: Game Thinking Overview
 
Graphics Interface 2019: Invited Speaker: Lennart Nacke - Game Thinking
Graphics Interface 2019: Invited Speaker: Lennart Nacke - Game ThinkingGraphics Interface 2019: Invited Speaker: Lennart Nacke - Game Thinking
Graphics Interface 2019: Invited Speaker: Lennart Nacke - Game Thinking
 
GAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification researchGAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification research
 
Introduction to Game Thinking (Fluxible 2018)
Introduction to Game Thinking (Fluxible 2018)Introduction to Game Thinking (Fluxible 2018)
Introduction to Game Thinking (Fluxible 2018)
 
GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...
GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...
GAMES USER RESEARCH: Guest Lecture in UX Design Class at Wilfried Laurier Uni...
 
Game UX Summit '17: Challenges of Evaluating Player Cognition & Emotion
Game UX Summit '17: Challenges of Evaluating Player Cognition & EmotionGame UX Summit '17: Challenges of Evaluating Player Cognition & Emotion
Game UX Summit '17: Challenges of Evaluating Player Cognition & Emotion
 
DAC 300: Chance and Skill in Game Design
DAC 300: Chance and Skill in Game DesignDAC 300: Chance and Skill in Game Design
DAC 300: Chance and Skill in Game Design
 
DAC 305: Choice and Agency in Gamification
DAC 305: Choice and Agency in GamificationDAC 305: Choice and Agency in Gamification
DAC 305: Choice and Agency in Gamification
 
Brief Introduction to Game Design
Brief Introduction to Game DesignBrief Introduction to Game Design
Brief Introduction to Game Design
 
The 5 Gamification Languages: The secret to gameful experiences that last (Ga...
The 5 Gamification Languages: The secret to gameful experiences that last (Ga...The 5 Gamification Languages: The secret to gameful experiences that last (Ga...
The 5 Gamification Languages: The secret to gameful experiences that last (Ga...
 
#GamesUR Conference: From Body Signals to Brainy Player Insights
#GamesUR Conference: From Body Signals to Brainy Player Insights#GamesUR Conference: From Body Signals to Brainy Player Insights
#GamesUR Conference: From Body Signals to Brainy Player Insights
 
Social Player Analytics in a Facebook Health Game (HCI Korea)
Social Player Analytics in a Facebook Health Game (HCI Korea)Social Player Analytics in a Facebook Health Game (HCI Korea)
Social Player Analytics in a Facebook Health Game (HCI Korea)
 
Biofeedback Gaming: The Future of Game Interaction?
Biofeedback Gaming: The Future of Game Interaction?Biofeedback Gaming: The Future of Game Interaction?
Biofeedback Gaming: The Future of Game Interaction?
 
On the Usability of Psychophysiological User Research for the Games Industry
On the Usability of Psychophysiological User Research for the Games IndustryOn the Usability of Psychophysiological User Research for the Games Industry
On the Usability of Psychophysiological User Research for the Games Industry
 
CHI 2011: Biofeedback Game Design
CHI 2011: Biofeedback Game DesignCHI 2011: Biofeedback Game Design
CHI 2011: Biofeedback Game Design
 
Affective Ludology: Affective Measurement of User Experience in Games
Affective Ludology: Affective Measurement of User Experience in GamesAffective Ludology: Affective Measurement of User Experience in Games
Affective Ludology: Affective Measurement of User Experience in Games
 
Playability & Player Experience Research
Playability & Player Experience ResearchPlayability & Player Experience Research
Playability & Player Experience Research
 
Game Metrics and Biometrics: The Future of Player Experience Research
Game Metrics and Biometrics: The Future of Player Experience ResearchGame Metrics and Biometrics: The Future of Player Experience Research
Game Metrics and Biometrics: The Future of Player Experience Research
 
From Playability to a Hierarchical Game Usability Model
From Playability to a Hierarchical Game Usability ModelFrom Playability to a Hierarchical Game Usability Model
From Playability to a Hierarchical Game Usability Model
 

Recently uploaded

ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvRicaMaeCastro1
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...Nguyen Thanh Tu Collection
 
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxDIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxMichelleTuguinay1
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxVanesaIglesias10
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...DhatriParmar
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management SystemChristalin Nelson
 
ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfVanessa Camilleri
 
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Association for Project Management
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmStan Meyer
 
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptx
Unraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptxUnraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptx
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptxDhatriParmar
 
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptxBIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptxSayali Powar
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptxmary850239
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptxmary850239
 

Recently uploaded (20)

ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
31 ĐỀ THI THỬ VÀO LỚP 10 - TIẾNG ANH - FORM MỚI 2025 - 40 CÂU HỎI - BÙI VĂN V...
 
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxDIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptx
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management System
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 
ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdf
 
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and Film
 
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptx
Unraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptxUnraveling Hypertext_ Analyzing  Postmodern Elements in  Literature.pptx
Unraveling Hypertext_ Analyzing Postmodern Elements in Literature.pptx
 
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptxBIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
BIOCHEMISTRY-CARBOHYDRATE METABOLISM CHAPTER 2.pptx
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
 
4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx
 

Brain, Body, and Bytes CHI 2010 Workshop Presentations

  • 1.
  • 2. BBB Workshop Overview Audrey Girouard, Erin Treacy Solovey, Regan Mandryk, Desney Tan, Lennart Nacke, Robert J.K. Jacob
  • 3. BBB Workshop: Bringing Together BCI& Physiological Computing Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 3
  • 4. Workshop Goals 1. Provide a platform for creating synergies between two related and emerging HCI disciplines (PC and BCI). 2. Kick-start BBB research in HCI by identifying key research questions and application areas 3. Enhancing HCI methodologies by adding these new methods and techniques to the toolbox of HCI researchers Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 4
  • 5. About Us Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 5
  • 6. About you • 25 Participants • X EEG papers • Y fNIRS papers • Z EMG papers • 9 Brain papers • A GSR • 19 Body papers • B Eye tracking Brain Body 32% 68% Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 6
  • 7. Workshop Discussion Questions <on your tables> Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 7
  • 8. Workshop Discussion Questions What terms/acronyms should we use to describe our work? Psychophysiological computing seems a bit long… BBB? BBI? Other suggestions? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 8
  • 9. Today’s Schedule 09:00-10:30 Introduction of organizers & participants, CHI madness style 10:30-11:00 Break 11:00-12:30 Group Discussion (assign groups + topic) 12:30-14:00 Lunch 14:00-15:30 Small group discussion (1h of discussion + 30 minutes of reporting) 15:30-16:00 Break 16:00-17:00 Report from small groups, recap, group discussion – barriers to publication in HCI Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 9
  • 10. Staying connected Website http://www.eecs.tufts.edu/~agirou01/workshop/ Twitter http://twitter.com/bbbCHI2010 Twitter #bbbCHI2010 Hashtag Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and- Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786 Flickr http://www.flickr.com/groups/bbbchi/ Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological- user-interaction Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 10
  • 11. And now… 2 Minute Introductions • About your research: – Application or domain space? (e.g. video games) – Signal you are measuring? (e.g. fNIRS) – Device/hardware you are using? (e.g. ISS, Inc) • What would you like to get out of workshop? – What problems are you facing? – What do you see in common with others here? – What device would you like to know more about? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 11
  • 12. List of Presentations 1. Adriane Randolph 16. Katarzyna Wac 2. Krzysztof Z. Gajos 17. Feng Tian 3. Saraswathi Bellur 18. Veronica Zammitto 4. Danny Plass-Oude Bos 19. Johanna Octavia 5. Kai Kuikkaniemi 20. Bob Wray 6. Mitchel Benovoy 21. Marjolein van der Zwaag 7. Yee Chieh Chew 8. Martha Crosby 9. Jan Kallenbach 10. Eva Oliveira 11. Lucia Filgueiras 12. Marc Grootjen 13. Melody Moore Jackson 14. Johann Schrammel 15. Daniel Sjölie Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 12
  • 13. Brain-Computer Interfaces as Child’s Play: Using Mental Training to Improve Motor Skills in Children Adriane B. Randolph, Ph.D. Ashley Ingraham Kennesaw State University
  • 14. Outline • Research Position – Examine ties between sensorimotor area and physical ability to uncover a training tool to aid children’s motor skill development • Pilot Study – 3 able-bodied adults (2 Male, 1 Female) – Acquire baseline Wii tennis score – Conduct mental training with BCI2000 mu training protocol for 5 days – Retest Wii tennis score for improvements Participant Session One Session Five – Mixed results AZ 98 60 CM 70 64 MW 103 118 • Future Work – Extend study with 20 able-bodied adults (currently in progress) – Analyze link between training and baseline ability – Test with elementary-aged children Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 14
  • 15. Screening for Mu Ashley prepares participant to use mental imagery of hand and foot movements to determine his best channels for use in mu BCI training before he plays a match of Wii Tennis. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 15
  • 16. Thank you Adriane B. Randolph, Ph.D. Kennesaw State University BrainLab (770) 423-6083 brainlab@kennesaw.edu http://coles.kennesaw.edu/brainlab/ Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 16
  • 17. Understanding How to Design Complex Brain-Controlled Applications Krzysztof Z. Gajos
  • 18. How to Design Brain-Controlled Applications? • Basic operations Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 18
  • 19. How to Design Brain-Controlled Applications? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 19
  • 20. How to Design Brain-Controlled Applications? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 20
  • 21. How to Design Brain-Controlled Applications? • Basic operations Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 21
  • 22. How to Design Brain-Controlled Applications? • Interaction between primary task and BCI performance Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 22
  • 23. How to Design Brain-Controlled Applications? • Error prevention, recovery, and actions with side effects Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 23
  • 24. Thank you Krzysztof Gajos kgajos@eecs.harvard.edu http://www.eecs.harvard.edu/~kgajos/ Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 24
  • 25. Psychophysiological Responses to Media Interfaces Saraswathi Bellur S. Shyam Sundar Media Effects Research Lab http://www.psu.edu/dept/medialab Pennsylvania State University
  • 26. Biopac MP 35 & MP 150 ACQknowledge software  Skin Conductance (EDA)  Heart Rate (ECG)  Facial EMG  Brain Wave (EEG)  Eye Movement (EOG)  Application Areas Interface presentation Information Processing Usability Testing & Evaluation  User Experience Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 26
  • 27. Physiological Measures: Affect, Arousal & Attention 3. ECG: Heart Rate recording 1. EEG: Bipolar recordings Occipital Alpha Blocking Orienting Response via BPM change 2. Skin Conductance (EDA) Tonic arousal levels 4. Facial EMG: Corrugator & Zygomatic recordings Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 27
  • 28. Theoretical Frameworks • The Media Equation: Mediated Life = Real Life (Reeves & Nass, 1996). • Structural aspects of media technologies: Automatic or Controlled processing? Voluntary or Involuntary responses? • Specific cues triggered via interactions with four types of media affordances, MAIN model (Sundar, 2008) Animation M: MODALITY Customization Heuristics A: AGENCY Information Perceptual Scent Bandwidth I: INTERACTIVITY N: NAVIGABILITY Contingency UGC & Sourceness Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 28
  • 29. Thank you Saras Bellur (saras@psu.edu) S. Shyam Sundar (sss12@psu.edu) Media Effects Research Lab http://www.psu.edu/dept/medialab Pennsylvania State University Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 29
  • 30. BCI for Games Danny Plass-Oude Bos Boris Reuderink, Bram van de Laar, Hayrettin Gürkök, Christian Mühl, Mannes Poel, Anton Nijholt, Dirk Heylen HMI, University of Twente
  • 31. Target Group: the general population Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 31
  • 32. BCI: issues Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 32
  • 33. BCI: new information, new features Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 33
  • 34. Acceptance: Usability & Realistic Settings Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 34
  • 35. Subjects  Users Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 35
  • 36. Thank you d.plass@ewi.utwente.nl http://hmi.ewi.utwente.nl/~oudebos Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 36
  • 37. Designing Biofeedback for Games and Playful Applications Kai Kuikkaniemi Helsinki Institute for Information Technology
  • 38. Emoshooter Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 38
  • 39. Emoshigeru - cinema games Real-time Public display Biosignal adaptation Massive co-located Shooter / Strategy Part of Shigeru-games local wlan Public display Connected to game server Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 40. Presemo – presentation engine Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 41. Emolisten – biosignal sonification Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 42. Research statement • Social Games and Playful Applications, which Utilize Biosignals • Grey Area between Adaptation and Biofeedback – Implicit and Explicit Biofeedback • Biosignal Synchrony (Inter-Personal) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 42
  • 43. Our team and projects Professor Marko Turpeinen Professor Timo Saari Ilkka Kosunen, Toni Laitinen, Petri Lievonen, Pauli Ojala Fun of Gaming (EU FP6) (2006-2009) Enactive Social Media and Games (Aalto) (2009) Emokeitai (Tekes – National Innovation Agency) (2009-2011) Thanks: CKIR / Niklas Ravaja Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 43
  • 44. Psychophysiological signal analysis and classification Mitchel Benovoy Centre for Intelligent Machine, McGill University
  • 45. Outline • Signals • Hardware • Application domains Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 45
  • 46. Signals Galvanic Skin Response Phalange Temperature 1.5 ∆ Cond. (µS) 1 0.5 0 -0.5 0.5 1 1.5 2 2.5 Sample 4 x 10 Electromyogram 42 Blood Volume Pulse – HR & Amp. Respiration – Rate & Amp. 40 2.2 2 38 Refl. Exp. 1.8 36 1.6 34 1.4 32 1.2 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 7 1 0.8 Amp. Amp. 6 0.6 0.4 5 0.2 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 105 30 HR (bpm) RR (bpm) 100 25 95 20 90 15 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 46
  • 47. Hardware • Thought Technology ProCom Infinity Images via www.thoughttechnology.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 47
  • 48. Application Domains • Affective Computing – Interaction evaluation (UI, devices, latency) – Music & emotion research – Psychoneurology Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 48
  • 49. Thank you benovoym@cim.mcgill.ca Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 49
  • 50. BCI and Creativity Yee Chieh (Denise) Chew Georgia Institute of Technology
  • 51. Outline • fNIR and Creative Expression • EEG and Music Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 51
  • 52. Character Templates for fNIR Created with a mouse, these are templates that subjects traced using fNIR Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 52
  • 53. Thank you ychew@gatech.edu Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 53
  • 54. Physiological Measures Used for Identification of Cognitive States and Continuous Authentication Curtis S. Ikehara & Martha E. Crosby University of Hawaii at Manoa
  • 55. Outline • Overview of our Research Using Physiological sensors – Assessing Cognitive States – Continuous Authentication • Possible Applications Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 55
  • 56. Multi-modal Physiological Sensors System Eye Tracker (Gaze Position, Fixation Number, Fixation Duration, Repeat Fixations, Search Patterns, MTF Pupil Size, Blink Rate, Blink Duration) (Moving Target Fractions) HUMAN Pressure Mouse (Pressures on Mouse Button and Body) Laptop (Data Collection, Analysis & Display) EDA / TSA Finger Sensors (Electrodermal and Temperature Sensor Activity) Oximeter (Pulse, Blood Oxygen) Custom Electronics (Analog to Digital Converter) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 56 56
  • 57. A Composite of Over 20 Mouse Clicks of Three People (Click Signature) US Patent # 7,245,218 Subject 4 Subject 5 Subject 7 57 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 58. Possible Applications • The student’s confidence can be assessed by evaluating the distortions of the mouse click signature. • High Value Vehicle Control – Pressure and other physiological sensors can be attached to a steering (e.g. aircraft yoke or steering wheel) . – Sensors can be used to indicate that: • The authorized pilot or driver is in control. • The cognitive state of the pilot is consistent with the vehicle’s activity. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 58
  • 59. Thank you Curtis S. Ikehara Martha E. Crosby cikehara@hawaii.edu crosby@hawaii.edu University of Hawaii at Manoa 1680 East-West Road - POST 317 Honolulu, Hawaii 96822 USA Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 59
  • 60. Measuring Interaction Experiences: Integration of Multiple Psychophysiological Methods Jan Kallenbach Aalto University Helsinki, Finland
  • 61. Outline User Experience = Choice & Interaction + Cost-Benefit Evaluations Measuring Interaction Experiences?  Software Framework Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 61
  • 62. Multimodal Measurements Client Application for UX Research Sampling frequency: max 1000Hz Psychophysiological Measures: Gaze, Pupil Size, EDA, EMG, EEG, … Behavioral Measures: Keyboard, Mouse, Slider, … System Measures: Window Events Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 62
  • 63. Thank you Jan Kallenbach Research Scholar Aalto University School of Science & Technology Department of Media Technology P.O. Box 15500 FIN - 00076 Aalto Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 63
  • 64. Towards enhanced video access and recommendation through emotions Eva Oliveira Nuno Ribeiro Teresa Chambel IPCA | LASIGE
  • 65. We Want • To access to videos based on emotions – user actual emotional (psychophysiological) state – video emotional content – director’s perspective • To recommend videos based on emotions – create a set of rules from every affective classification perspective (user and video, director) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 65
  • 66. Affective video recommendation system requirements () • Every video must be classified affectively from the user and from the content perspective. • Every user has an emotional profile constituted by all his implicit (physiological) classification of every classified movie. • User physiological signals can change the way we watch the videos Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 66
  • 67. User Classification Issues • Automatic Classification – Users emotional state • emotional model – Classification techniques • Result output • Accuracy • Lab contexts Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 67
  • 68. Current State • Capturing users phychophysiological signals (BIOPAC System) while watching movies This is my exciting picture • Next step: several classification techniques to convert signal result into emotional categories Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 68
  • 69. Thank you • Eva Oliveira : • eva.oliveira@gmail.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 69
  • 70. Your users can’t talk! How can you tell they liked your system? Lucia Filgueiras and Claudia Tambascia Escola Politecnica Universidade de São Paulo
  • 71. What we want to do • Develop products for cerebral palsy users • Perform participatory design • Understand non-verbal communication Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 71
  • 72. Direct measures and usability evaluation HRV and workload measures Eye tracking in iTV apps in control room operability evaluation evaluation Comparison of eye tracker NASA TLX, SWAT, Heart Rate findings to conventional Variability usability tests results Variabilidade da frequência cardíaca OP13 LF nu / HF nu LSC Média 18,0 Operador analisa sistema após vários Índice do Balanço Simpático? Parassimpático Ação do operador para alarmes: temperatura alta (10:29); Ação do operador para 16,0 correção nível alto (10:32) e segurança correção 14,0 (10:32; 10:37; 10:43) 14,1 12,0 Alarme de 11,7 temperarura alta: 10,0 10:15 9,1 8,3 8,5 8,0 7,3 6,6 6,0 5,9 5,2 5,3 5,6 4,6 5,0 4,0 3,9 3,8 4,0 4,4 4,1 2,9 3,3 2,6 2,6 2,6 2,3 2,4 2,0 0,0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 :3 :4 :4 :5 :5 :0 :0 :1 :1 :2 :2 :3 :3 :4 :4 :5 :5 :0 :0 :1 :1 :2 :2 :3 :3 09 09 09 09 09 10 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11 11 11 HORA Heart Rate Variability of an operator in a unstable unit situation Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 73. Our questions: • How personalized are body expressions for cerebral palsy patients? • Are there general body expressions that can mean the same for everyone? • Is it possible to determine a generic way of measuring these expressions? • Is there any other liaison besides gestures and facial expressions , like an unconscious connection of limbic systems that can be responsible for the mutual sensitive communication? • Is there a reference body signal that can be used as a reliable parameter to evaluate body expressions? • How can we establish a correlation between body expressions and patient´s opinion of approval or disapproval of a design solution? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 73
  • 74. Thank you! Contact us: Lucia V. L. Filgueiras Escola Politecnica, Universidade de São Paulo lucia.filgueiras@poli.usp.br Claudia A.Tambascia Fundação CPqD claudiat@cpqd.com.br http://lts-i.pcs.usp.br Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 74
  • 75. Human Computer Symbiosis by Mutual Understanding Marc Grootjen EagleScience-TNO
  • 76. Outline • Symbiosis & mutual understanding • Information models • Interfaces Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 76
  • 77. Maximum task performance Symbiotic entity Disturbance Mutual human understanding computer Tasks (goals) Human-computer symbiosis with mutual understanding as key issue for maximum task performance Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 77
  • 78. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 78
  • 79. Domesticating Brain- Computer Interfaces: Systems for the Home Melody Moore Jackson BrainLab School of Interactive Computing Georgia Tech
  • 80. Extended Home-based BCI study • fNIR-based BCI for communication • 32 People with ALS (so far) over 1 year • 78% (25) could operate the device on initial visit with accuracy > 70% • Accuracy to 100%, average 76.5% • Issues: – Motivation – Data Collection – Remote Support Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 80
  • 81. The Kokoro Gatari Brain Computer Interface based on Functional Near Infrared imaging Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 81
  • 82. Thank you Melody Moore Jackson melody@cc.gatech.edu 404-277-1324 Ian McClendon imclend@cc.gatech.edu 404-641-7712 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 82
  • 83. Get up, move on! Using Electromyography to Explore the Relationship of Experience and Motion Johann Schrammel Centre for Usability Research and Engineering
  • 84. Outline • Compare experiences and emotions in conditions with and without movements with Electromyography(EMG) and other methods Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 84
  • 85. Outline • Can EMG successfully be used to estimate a user‘s emotional experience in motion contexts (artefacts, feasibility, etc.)? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 85
  • 86. Outline • Is a more embodied way of controlling interaction actually amplifying experience? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 86
  • 87. Results • EMG is suitable for motion contexts. • Both EMG and self-report data indicate that natural body movement intensifies emotions. • It’s crucial that body movements are in accordance with activity. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 87
  • 88. Thank you schrammel@cure.at Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 88
  • 89. Reality-Based Brain- Computer Interaction Daniel Sjölie Umeå University, Umeå, Sweden
  • 90. Reality-Based Brain-Computer Interaction • Complex, reality-based, interaction with the ability to adapt to the user in real-time – Using brain measurements to evaluate • Direct brain-computer interaction – The user can be unaware • We’re using a combination of virtual reality (VR) and functional MRI (fMRI) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 90
  • 91. HCI and cognition • An improved understanding of the brain in this interactive and complex context is needed • Build on theories of cognition as a foundation for the development of advanced HCI – As interaction moves towards being reality-based we move closer to “interacting with reality”, and thus to general cognition • Cognitive neuroscience and computational models of the brain are important tools Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 91
  • 92. Working with fMRI and Virtual Reality Working with fMRI requires specialist competence and forces us to deal with powerful magnetic fields and restricted mobility, but the ability to measure the whole brain is an invaluable research tool. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 92
  • 93. Thank you Daniel Sjölie, daniel@cs.umu.se Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 93
  • 94. emoBAN: Improving Quality of Life via Psychophysiological Mobile Computing Katarzyna Wac, Anind K. Dey Carnegie Mellon University, HCII
  • 95. emoBAN idea • “what always speaks silently is the body” (Brown, 1990) – measure QoL via psychophysiology Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 95
  • 96. emoBAN: Usable Useful Functional • Body Area Network – sensors • vital signs, e.g., ECG, temp, resp • context, e.g., location, time, activity – actuators • feedback via, e.g., AV, tactile, light Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 96
  • 97. Thank you Dr Katarzyna Wac Carnegie Mellon University Human-Computer Interaction Institute katewac@cmu.cs.edu www.cui.unige.ch/~wac Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 97
  • 98. Leveraging Psychophysical Data in Monitoring and Analyzing the States of Badminton Players Feng Tian, Wencan Luo, Jing Dai, Reza Naima, Yongmin Cheng, Hongan Wang, Guozhong Dai Institute of Software, Chinese Academy of Sciences Physical Education Collage, Beijing Sport University Berkeley Institute of Design, UC. Berkeley China Badminton Team, General Administration of Sport of China 4/20/2010
  • 99. Traces and physical actions - Camera, accelerometers Psychophysical data - Berkeley Tricorder ECG, EMG, Pulse Oximetry, BioImpedance Respiration Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 100. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
  • 101. Exploring Quantitative Methods for Evaluating Sports Games Veronica Zammitto Simon Fraser University Magy Seif El-Nasr Simon Fraser University Paul Newton Electronic Arts Canada
  • 102. Current Study: NBA Live 10 • Eye tracking – Eye Tech TM3 • Psycho-physiological data – ProComp5 Infiniti – EMG (corrugator supercilii and zygomatic major), EKG (heart rate), GSR and BVP • Gameplay Telemetry Data – event-based RPC telemetry collection system • Triangulation Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 102
  • 103. What was the user the Data Triangulation - NBA Live 10 emotional response? looking at? Game user experience Event, when Was positive? Where user’s Was negative? attention was? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 103
  • 104. Data Triangulation - NBA Live 10 Game user experience Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 104
  • 105. Thank you Veronica Zammitto vzammitt@sfu.ca Magy Seif El-Nasr magy@sfu.ca Paul Newton pnewton@ea.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 105
  • 106. Exploring Psycho-physiological Measures for the Design and Behavior of Socially-Aware Avatars in Ubicomp Environments Johanna Renny Octavia Hasselt University, Belgium
  • 107. Avatars in Ubicomp Environment  User’s representation  Mediator between system and user (companion)  To design a socially-aware avatar that adapts its behavior according to user’s emotion  Any incorrect actions of the avatar can lead to loss of identification of the user Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 107
  • 108. Our informal experiment  ProComp Infiniti device, measured GSR, EMG and BVP  Higher physiological values when playing against a friend  How different social relationships between players influence those values? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 108
  • 109. Thank you Expertise Centre for Digital Media (EDM) Web: www.edm.uhasselt.be Email: johanna.octavia@uhasselt.be Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 109
  • 110. Tailoring Practice for Individualized Learning via Cognitive State Bob Wray Soar Technology, Inc.
  • 111. Why I need Psychophysiological Computing • Practice environments are becoming common-place • Adaptive tailoring of practice is important for individualized experience/learning • Need more than traditional adaptations (e.g., scaffolding) – Adapting to engage learner / maintain interest – Adaptation to reducing “gaming” in environment – Adaptation to elicit learner affect/arousal (“real”) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 111
  • 112. Research Testbed Narrative, conversational, inter-cultural training application Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 112
  • 113. Thank you Bob Wray Soar Technology wray@soartech.com, 919.967.5079 Brian Magerko Georgia Institute of Technology magerko@git.edu Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 113
  • 114. Guidelines to biosignal driven HCI Marjolein van der Zwaag Philips Research Europe, University of Groningen
  • 115. Guidelines to biosignal driven HCI Marjolein van der Zwaag Egon L. van den Broek Joris H. Janssen
  • 116. Outline • Signal(s) measured: psycho-physiological • Application domain: Affective computing • Device used: Nexus 10, mind media • Guidelines for processing biosignals – Triangulation – Identification of users – Biosignal characteristics – Temporal construction Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 116
  • 117. Unobtrusive affective computing Music direct your mood concept Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 117
  • 118. Thank you Marjolein.van.der.zwaag@philips.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 118
  • 119.
  • 120. Workshop Discussion Questions What is brain/body computing good for? How can we think systematically about this space? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 120
  • 121. Workshop Discussion Questions What are the major themes of HCI that can be explored using brain/body interaction? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 121
  • 122. Workshop Discussion Questions What specific brain & body sensing modalities are interesting and how can we use them? What are most innovative technologies in this space? How do these compare with existing practices (NASA-TLX, Likert-style questionnaires, etc)? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 122
  • 123. Workshop Discussion Questions How can BCI and PC be defined within an HCI context? How is a HCI context different from neuropsychology, biomedicine or biomedical engineering? How does HCI contribute to psychophysiological computing (or more specifically neural engineering, bio- engineering, etc)? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 123
  • 124. Workshop Discussion Questions If we could measure anything on brain and body, what would we want to measure and why? What cognitive states and emotions have been investigated with sensors? What methodologies are best for what signals? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 124
  • 125. Workshop Discussion Questions How can accurate physiological, psychological and neurophysiological measurements improve HCI? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 125
  • 126. Workshop Discussion Questions With what software techniques and hardware technologies could these signals be optimally processed? How can software account for the combination of user actions taken to accomplish a task versus actions that would surprise the user, yet optimize their experience? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 126
  • 127. Workshop Discussion Questions What are the issues that must be considered when using brain/body computing for direct control, passive sensing, adaptation, etc? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 127
  • 128. Groups • How to go from raw data to something else – Emotions: TABLE 1 – Data Fusion: TABLE 4 • How to use these things in interactive interfaces: TABLE 3 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 128
  • 129. BBB Workshop: Bringing Together BCI& Physiological Computing Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 129
  • 130. Staying connected Website http://www.eecs.tufts.edu/~agirou01/workshop/ Twitter http://twitter.com/bbbCHI2010 Twitter #bbbCHI2010 Hashtag Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and- Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786 Flickr http://www.flickr.com/groups/bbbchi/ Grouplist http://groups.google.com/group/brain-body-interfaces/ Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological- user-interaction Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 130
  • 131. BBB at CHI2010 Brains and Brawn session: Tuesday 11:30-1pm Making Muscle-Computer Interfaces More Practical T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, James A. Landay University of Washington, USA Microsoft Corporation, USA A Novel Brain-Computer Interface Using a Multi-Touch Surface Beste F. Yuksel, Michael Donnerer, James Tompkin, Anthony Steed. University College London, UK The Influence of Implicit and Explicit Biofeedback in First-Person Shooter Games Kai Kuikkaniemi, Toni Laitinen, Marko Turpeinen, Timo Saari, Ilkka Kosunen, Niklas Ravaja, Helsinki Institute for Information Technology, Finland Center for Knowledge and Innovation Research, Finland Effects of Interactivity and 3D-motion on Mental Rotation Brain Activity in an Immersive Virtual Environment Daniel Sjölie, Kenneth Bodin, Eva Elgh, Johan Eriksson, Lars-Erik Janlert, Lars Nyberg Umeå University, Sweden Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 131
  • 132. THANK YOU • Keep in touch! Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 132