0
Evaluating brain signal input
for Kinect-based games

Dr Brett Wilkinson
Presenting the work of:
Mr Peter Mitchell, Dr Bre...
Research Overview
• Used Natural User Interfaces (NUIs) through a
combination of the Microsoft Kinect and Emotiv
EPOC to p...
Presentation Overview
• Background
• Testing methodology
• The Games
– Tile Puzzle
– Street Puzzle
– River Puzzle

• Resul...
Background on Existing
Studies
• BrainBasher (van de Laar, 2009)
• Used actual and imagined movement to
have participants ...
Goals for Testing
• Does BCI input with Kinect-based
games modify the experience?
• What signals are most appropriate for
...
Testing Approach
• 15 participants from Flinders University
– Students and academics
– Primarily male

• Play three puzzle...
Application Overview: Tile Puzzle
Application Overview: Tile Puzzle
• Time-based task
– Freedom to explore the interaction techniques with
the Emotiv and Ki...
Original Mock-up Example
Image of initial state

Initial State

Image of moving
squares

Swapping Tiles

Image of Calm vie...
Finished Tile Puzzle (Video)
Street Puzzle Overview

Street Puzzle
Application Overview: Street Puzzle
• Motion – goal – control brain state
• Rail-based task
– Set, randomised path

• Conc...
River Puzzle Overview

River Puzzle
Application Overview: River Puzzle
• Time-based task
– Selection of appropriate game items within a set
time
– The more co...
Results

General Gameplay Responses
Results Continued

Ease of use with specific HCI components
Results Continued

Combination of Inputs and Marketability
Results Continued

Comparison of Percent Time Spent in EEG States
Results Continued
• Technical issues encountered with BCI
equipment:
– Cheap headset resulted in limited
performance
– Cal...
Future Work
• Look at the potential of other BCI devices
• Look at the potential of other platforms
• Extended evaluation ...
Conclusion
• Pilot study indicated that the technology
can work together
• Developed a functional test platform
• User eva...
Upcoming SlideShare
Loading in...5
×

Evaluation of a Natural User Interaction Gameplay System Using the Microsoft Kinect Augmented with Non-invasive Brain Computer Interfaces

306

Published on

Evaluation of a Natural User Interaction Gameplay System Using the Microsoft Kinect Augmented with Non-invasive Brain Computer Interfaces by Peter Mitchell, Dr. Brett Wilkinson, and Dr. Sean Fitzgibbon

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
306
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Evaluation of a Natural User Interaction Gameplay System Using the Microsoft Kinect Augmented with Non-invasive Brain Computer Interfaces"

  1. 1. Evaluating brain signal input for Kinect-based games Dr Brett Wilkinson Presenting the work of: Mr Peter Mitchell, Dr Brett Wilkinson, Dr Sean Fitzgibbon and Mr Lawrence Sambrooks
  2. 2. Research Overview • Used Natural User Interfaces (NUIs) through a combination of the Microsoft Kinect and Emotiv EPOC to provide a full body Human Computer Interaction experience (HCI). • Research completed as a pilot study to determine the usability of the combination of hardware to explore whether there is future potential for the combination. Emotiv 2010, Arizona State University, viewed 9 September 2013, <http://lsrl.lab.asu.edu/site/?p=848> Microsoft Kinect 2012, Microsoft, viewed 9 September 2013 , <http://www.microsoft.com/en-us/kinectforwindows/>
  3. 3. Presentation Overview • Background • Testing methodology • The Games – Tile Puzzle – Street Puzzle – River Puzzle • Results
  4. 4. Background on Existing Studies • BrainBasher (van de Laar, 2009) • Used actual and imagined movement to have participants attempt to match actions. (seen top right) • BacteriaHunt (Bos et al., 2010) • BCI interaction used to provide a speed modifier in combination with keyboard interaction. • AlphaWoW (Bos et al. 2010) • Used a variety of BCI methods for character interaction in the game World of Warcraft. Inner speech, association, and mental states. VAN DE LAAR, B. L. 2009. Actual and imagined movement in BCI gaming. BOS, D.-O., REUDERINK, B., VAN DE LAAR, B., GURKOK, H., MUHL, C., POEL, M., HEYLEN, D. & NIJHOLT, A. Human-computer interaction for BCI games: Usability and user experience. Cyberworlds (CW), 2010 International Conference on, 2010. IEEE, 277-281.
  5. 5. Goals for Testing • Does BCI input with Kinect-based games modify the experience? • What signals are most appropriate for gameplay? • Can individuals maintain control over their own brain waves?
  6. 6. Testing Approach • 15 participants from Flinders University – Students and academics – Primarily male • Play three puzzle games • Complete post experiment survey • Complete post experiment NASA TLX
  7. 7. Application Overview: Tile Puzzle
  8. 8. Application Overview: Tile Puzzle • Time-based task – Freedom to explore the interaction techniques with the Emotiv and Kinect • Concentration and Relaxation used as input – Relax: reveal hidden image – Concentrate: hide image • Kinect used to map movements and speech to interaction – Control cursor – Select, place, rotate tiles
  9. 9. Original Mock-up Example Image of initial state Initial State Image of moving squares Swapping Tiles Image of Calm view Relaxed State Image of complete Completed Puzzle Free Sandstone Image 2012, viewed 2/04/2012, http://www.hoskingindustries.com.au/blog/tag/grunge/page/2/ Free Sandstone Image 2012, viewed 2/04/2012, http://www.spiralgraphics.biz/packs/stone_muted/index.htm?36 Hieroglify font, http://www.fontspace.com/download/1123/e17737daec4347e0b3edd50cd5c47df6/barmee_hieroglify.zip
  10. 10. Finished Tile Puzzle (Video)
  11. 11. Street Puzzle Overview Street Puzzle
  12. 12. Application Overview: Street Puzzle • Motion – goal – control brain state • Rail-based task – Set, randomised path • Concentration and Relaxation used as input – Relax: slow down game time – Concentrate: speed up game time • Kinect used to map movements to interaction – Sideway step to jump rail – Both hands used to halt motion
  13. 13. River Puzzle Overview River Puzzle
  14. 14. Application Overview: River Puzzle • Time-based task – Selection of appropriate game items within a set time – The more collected the higher the score • Concentration and Relaxation used as input – Relax: slow down game time – Concentrate: speed up game time • Kinect used to map movements to interaction – Control cursor – Select treasure and place in inventory
  15. 15. Results General Gameplay Responses
  16. 16. Results Continued Ease of use with specific HCI components
  17. 17. Results Continued Combination of Inputs and Marketability
  18. 18. Results Continued Comparison of Percent Time Spent in EEG States
  19. 19. Results Continued • Technical issues encountered with BCI equipment: – Cheap headset resulted in limited performance – Calibration difficulties and inconsistencies – Delay between updates – Muscle movement heavily contaminated data.
  20. 20. Future Work • Look at the potential of other BCI devices • Look at the potential of other platforms • Extended evaluation to investigate if training of state can be achieved
  21. 21. Conclusion • Pilot study indicated that the technology can work together • Developed a functional test platform • User evaluation conducted to suggest the potential for training brain state
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×