• Save
Real-time Sensory Substitution to Enable Players who are Blind to Play Video games using Whole-Body Gestures
Upcoming SlideShare
Loading in...5
×
 

Real-time Sensory Substitution to Enable Players who are Blind to Play Video games using Whole-Body Gestures

on

  • 826 views

A technique that allows blind users to pla

A technique that allows blind users to pla

Statistics

Views

Total Views
826
Views on SlideShare
825
Embed Views
1

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Real-time Sensory Substitution to Enable Players who are Blind to Play Video games using Whole-Body Gestures Real-time Sensory Substitution to Enable Players who are Blind to Play Video games using Whole-Body Gestures Presentation Transcript

  • Real-time Sensory Substitution to Enable Players who are Blind to Play Video games using Whole-Body Gestures Tony Morelli, Eelke Folmer - FDG’11, Bordeaux Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Gesture Based Interaction Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Reliance on Visual Cues Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Reliance on Visual Cues dodge Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Reliance on Visual Cues dodge ♫ ♫ Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • visual impairment & obesity 60,000 Blind Children in US More obese Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Barriers to physical activity Rely on sighted guide Safety Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Exercise games ★ Moderate to Vigorous PA (MVPA) ★Independently ★60 min of MVPA daily (CDC) ★Safer Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Research Question Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Gestures w/o visual cues? Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Gestures w/o visual cues? ? Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Sensory Substitution Visual information Haptic Audio Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Sensory Substitution closed captioning Visual information Haptic Audio Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Sensory Substitution closed captioning Visual information Haptic Audio braille speech synthesis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • SS with Gestures? Wii Sports VI Tennis serve ♫ bounce ♫ Bu zz return Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • User studies vigorous moderate light VI Tennis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • User studies vigorous moderate light VI Tennis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • User studies vigorous moderate Upper Body light VI Tennis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • User studies vigorous moderate Upper Body light VI Tennis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • User studies vigorous moderate light VI Tennis Whole Body Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Whole Body Exergames Yield Vigorous PA [Biddis & Irwin 2010] Track Gestures using Computer Vision Problems with Sensory Substitution »Systems are Controller less »Access to Source code Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real Time Sensory Substitution Le*!! Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Video Analysis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Instrumentation Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis <SECTION> <XMIN>100</XMIN> <XMAX>300</XMAX> <YMIN>150</YMIN> <YMAX>300</YMAX> <MINR>230</MINR> <MAXR>999</MAXR> <MING>230</MING> <MAXG>999</MAXG> <MINB>0</MINB> <MAXB>100</MAXB> <ACTION <WIIMOTE> <ID>0</ID> <DURATION>-1</DURATION> </WIIMOTE> <AUDIO> <ID>alert.wav</ID> <LOOP>0</LOOP> </AUDIO> </ACTION> </SECTION> Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis <SECTION> <XMIN>100</XMIN> <XMAX>300</XMAX> <YMIN>150</YMIN> <YMAX>300</YMAX> <MINR>230</MINR> <MAXR>999</MAXR> <MING>230</MING> <MAXG>999</MAXG> <MINB>0</MINB> <MAXB>100</MAXB> <ACTION <WIIMOTE> <ID>0</ID> <DURATION>-1</DURATION> </WIIMOTE> <AUDIO> <ID>alert.wav</ID> <LOOP>0</LOOP> </AUDIO> </ACTION> </SECTION> Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis <SECTION> <XMIN>100</XMIN> <XMAX>300</XMAX> <YMIN>150</YMIN> <YMAX>300</YMAX> <MINR>230</MINR> <MAXR>999</MAXR> <MING>230</MING> <MAXG>999</MAXG> <MINB>0</MINB> <MAXB>100</MAXB> <ACTION <WIIMOTE> <ID>0</ID> <DURATION>-1</DURATION> </WIIMOTE> <AUDIO> <ID>alert.wav</ID> <LOOP>0</LOOP> </AUDIO> </ACTION> </SECTION> Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis <SECTION> <XMIN>100</XMIN> <XMAX>300</XMAX> <YMIN>150</YMIN> <YMAX>300</YMAX> <MINR>230</MINR> <MAXR>999</MAXR> <MING>230</MING> <MAXG>999</MAXG> <MINB>0</MINB> <MAXB>100</MAXB> <ACTION <WIIMOTE> <ID>0</ID> <DURATION>-1</DURATION> </WIIMOTE> <AUDIO> <ID>alert.wav</ID> <LOOP>0</LOOP> </AUDIO> </ACTION> </SECTION> Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis <SECTION> <XMIN>100</XMIN> <XMAX>300</XMAX> <YMIN>150</YMIN> <YMAX>300</YMAX> <MINR>230</MINR> <MAXR>999</MAXR> <MING>230</MING> <MAXG>999</MAXG> <MINB>0</MINB> <MAXB>100</MAXB> <ACTION <WIIMOTE> <ID>0</ID> <DURATION>-1</DURATION> </WIIMOTE> <AUDIO> <ID>alert.wav</ID> <LOOP>0</LOOP> </AUDIO> </ACTION> </SECTION> Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis <SECTION> <XMIN>100</XMIN> <XMAX>300</XMAX> <YMIN>150</YMIN> <YMAX>300</YMAX> <MINR>230</MINR> <MAXR>999</MAXR> <MING>230</MING> <MAXG>999</MAXG> <MINB>0</MINB> <MAXB>100</MAXB> <ACTION <WIIMOTE> <ID>0</ID> <DURATION>-1</DURATION> </WIIMOTE> <AUDIO> <ID>alert.wav</ID> <LOOP>0</LOOP> </AUDIO> </ACTION> </SECTION> Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Real time Video Analysis Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Evaluate How efficient is RTSS? Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Thursday, June 30, 2011
  • Thursday, June 30, 2011
  • User Study Anonymously recruited and tested Evaluated 2 versions of the hurlde game »Regular (Audio + Video) »RTSS ( Audio + Haptic) 28 sighted participants Novice players Switch between versions a*er 3 players 12 people played Regular <-- data loss 16 people played RTSS Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Results version N success time  (seconds) RTSS 12 0.39  (δ=0.26) 28.67  (δ=8.41) Regular 16 0.37  (δ=0.32) 26.76  (δ=4.76) No Significant Difference p>.05 Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Jump Timing Accuracy Significant Difference p<0.05 Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Observations Game Restarts »False Starts ‣Regular: 2 players started too early ‣RTSS: None »Out of Range ‣Regular: None ‣RTSS: 3 players out of range Successful jump depends on pace Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Limitations of RTSS Kinect Sports Kinect Adventures ★ Predefined gestures ★ Different gestures ★ Fixed target ★ Random targets ★ 1 haptic cue ★ (directional) haptic cues? Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Future Work Techniques for keeping players in range and facing the camera. Multiplayer Detecting cues/objects Test with Blind Kids & Measure Energy Expenditure Integrate RTSS in Console OS Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Questions Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011
  • Questions ? Player-Game Interaction Research University of Nevada, RenoThursday, June 30, 2011