Vi bowling

Editor's Notes

1. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
2. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
3. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
4. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
5. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
6. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
7. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
8. Hello, my name is Eelke Folmer and I’m an assistant professor at the University of Nevada in Reno. Thanks the opportunity to present our research on developing exercise games for individuals with visual impairments. Unfortunately my student Tony Morelli who did this research couldn’t be here Let me start my presentation by asking a question to the audience: who likes to exercise? Excellent, ver well that brings me to the question: why do we exercise?
9. Exercising makes us happy as it gets rids of stress that builds up when we don’t exercise
10. But exercise is also a powerful tool in the fight against obesity. Obesity and childhood obesity in particular have reached epidemic proportions. Recent studies show that obesity is reducing the life expectancy of our children as obese children develop diabetes and heart problems at a much younger age as their non obese peers. Especially individuals with visual impairments and children in particular have significant higher levels of obesity as they are less physically active. As a result they more frequently suffer from obesity related health problems such as diabetes, fatigue, deconditioning, heart problems, and depression which are all considered avoidable. What is even more worrying is that
11. ....poor self rated health was recently found to be connected with higher suicide rates among adults with visual impairments.
12. So what is stopping individuals with visual impairments from exercising? Well the physical education researchers that we are collaborating with on this project have identified the following barriers:
13. Reliance on others. To engage in activities such as cycling or running individuals with visual impairments rely upon a sighted guide to help them out, which may not always be available.
14. Self barriers: especially children with visual impairments are self conscious and are afraid of being made fun of while participating in adapted physical activities.
15. But the most important barrier is safety. Individuals with VI have a much higher chance of getting injured while exercising. Fear of injury also leads to psychosocial barriers.
16. For example, parents of blind children often lack an understanding of their child’s ability to be physically active, which leads to overprotective behavior in an attempt to assist them. Although many schools are required to have adapted physical education programs, it is estimated that less than 10% of children with visual impairments actually participate in them. So that’s pretty bad, so here’s our idea:
18. Though video games have been identified to be a contributing factor to obesity, a new type of video game called exercise games has the potential to turn couch potatoes into jumping beans as they use physical activity as input. Exergames have gained tremendous popularity and popular titles include: Dance Dance Revolution and Nintendo Wii Sports. Recent studies show that exergames can get their players into levels of active energy expenditure that are high enough to be considered healthy and as a result exergames are increasingly used in physical education programs across the nation as video games can be powerful motivators especially for children. Exercise games have some attractive properties that may allow individuals with visual impairments to overcome some of the barriers to physical activity; namely: 1) You don’t need any other people to play an exergame you can play against the computer or online against friends 2) Because you exercise in place, the risk of injury is minimal 3) being able to play exergames with friends or family may create new opportunities for socialization. Various studies have shown that when individuals with visual impairments are given equal opportunities to participate in physical activity their health will increase.
19. Unfortunately exergames are not accessible as they rely upon being able to see visual cues that indicate what to do and when. For example wii sports boxing uses visual cues such as the moves of your opponent indicate what to do such as blocking, dodging or punching. Although audio feedback is being provided, this doesn’t contain sufficient cues on what type of input to provide and when.
20. Unfortunately exergames are not accessible as they rely upon being able to see visual cues that indicate what to do and when. For example wii sports boxing uses visual cues such as the moves of your opponent indicate what to do such as blocking, dodging or punching. Although audio feedback is being provided, this doesn’t contain sufficient cues on what type of input to provide and when.
21. So the research question becomes
22. Can we develop exertion interfaces that do not rely upon visual cues?
23. Can we develop exertion interfaces that do not rely upon visual cues?
24. Before we answer this question, lets analyze the problem domain a bit further and sketch out our design space as exergames are played in contexts that are fundamentally different from desktop environments. Exergames are often played in social contexts, e.g. people play these games with friends or family. Exergames also feature music here’s a little sample. Music and being able to socialize puts constraints on being able to use audio. Games are simulations, and exergames simulate physical activities. So what challenges do physical activities involve?
25. Physical activities consist of two types of challenges; namely spatial and temporal challenges: Take for example basketball: - The spatial challenge involves deciding where to shooting the ball - and the temporal challenge deals with when to shoot the ball.
26. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
27. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
28. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
29. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
30. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
31. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
32. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
33. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
34. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
35. A spatial challenge involves any physical activity, whether it be throwing, hitting, kicking that uses some form of aiming at a target located in 3D space around the user. Users typically acquire such targets visually, so we need to develop a way to do this using a different modality.
36. The spatial challenge involved in physical activities is primarily sensorimotor based, and relies primarily upon vision e.g. eye hand coordination.
37. The majority of adapted physical activities for individuals with visual impairments If you look at adapted physical activities for individuals with visual impairments, most of the such as tandem cycling a sighted guide performs the spatial skill. Because there are very few physical activities for individuals with visual impairments to perform spatial skills. Most physical activities for individuals with visual impairments have a sighted guide perform the spatial skills, and adapted physical education researchers have called for the development for physical activities that include spatial skills.
38. An example physical activity that includes a spatial skill is the popular goal ball, which relies on audio-hand coordination. Being able to perform spatial physical activities opens up the opportunity for individuals with visual impairments to play competitive team sports. So lets explore how technology can help perform spatial challenges
39. We’ve established that physical activity involve temporal and spatial challenges and there are constraints on the use of visual and audio feedback. Because this is quite a difficult problem to solve all at once lets solve these challenges independently from each other first. So lets look at temporal challenges.
40. Well we’ve already solved how to make an exergame with a temporal challenge accessible. In previous research we developed VI Tennis which is an tactile/audio exergame which implements the gameplay of Wii sports tennis and which can be played using a low cost motion sensing controller capable of providing vibrotactile feedback. Wii sports tennis does not have a spatial challenge. Vibrotactile cues indicate when the ball bounces and when it is time for the player to return the ball. A comparative analysis between audio and tactile/audio cues showed significant better performance when tactile cues are used and players were able to get into moderate physical activity with this game.
41. So a temporal challenge in an exergame can be solved using tactile feedback, lets focus on a spatial challenge Okay lets go back to exergames, can we find a physical activity that consists solely of a spatial challenge?
42. Bowling is a physical activity that is predominantly a spatial challenge as it is self paced, which is great as it allows us to focus exclusively on the spatial challenge. A further benefit is that the target acquisition is in 2D Bowling has been implemented as an exergame as part of Nintendo Wii Sports
43. This is a screenshot of the game, players play this game using a motion sensing controller.
44. The game basically consists of the steps: aiming and throwing which are repeated over a number of frames. This graph shows the feedback that the game provides, Wii bowling provides feedback in 3 different modalities, Theres audio feedback, such as the sound of the ball rolling and pins being hit. There’s visual information, such as the visualization of the ball and the pins. When the player has successfully thrown their ball they feel a short vibrotactile buzz.We further distinguish primary and secondary cues. Primary cues contain essential information for being able to play the game and secondary cues convey the results of a player action. In order to play a game you must substitute primary cues unless they are encoded in multiple modalities, in which case you can just leave on modality out.
45. So we created our own PC version of Wii Bowling called VI Bowling, which communicates with a wii remote using bluetooth and we implemented the exact same motions as Wii bowling to play the game. This is the sensory substitution graph which maps how particular visual cues have been mapped to either audio or tactile cues. The throwing part only involves secondary cues and since these are encoded in visual as well as audio, VI bowling just implements the same audio cues and the game is still playable, the only thing that we added were speech cues that indicate how many pins had been hit. The aiming part involves primary visual cues, specifically:
46. At the top left you see which pins are still standing and the user uses this information to align a visual marker as to where their ball will go.
47. So how do we make this accessible? We basically want to point the location of the pins to the user and for bowling we can restrict ourselves to a point in a plane to keep things simple. So to play this game we using motion sensing controller to detect motion and this controller also has a vibrotactor implemented in it. Users with visual impairments already locate the location of obstacles using a tactile modality using their cane. So can we do something similar with a wii remote? So similar to a cane a wii remote is longitudinal in design and using two techniques: e.g. proprioception, e.g. the users ability to sense the position and orientation of arm and stereognosis: the users ability to perceive the shape of 3d objects we can use the wii remote to point out a vector to the user that indicates a direction.
48. So how do we make this accessible? We basically want to point the location of the pins to the user and for bowling we can restrict ourselves to a point in a plane to keep things simple. So to play this game we using motion sensing controller to detect motion and this controller also has a vibrotactor implemented in it. Users with visual impairments already locate the location of obstacles using a tactile modality using their cane. So can we do something similar with a wii remote? So similar to a cane a wii remote is longitudinal in design and using two techniques: e.g. proprioception, e.g. the users ability to sense the position and orientation of arm and stereognosis: the users ability to perceive the shape of 3d objects we can use the wii remote to point out a vector to the user that indicates a direction.
49. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
50. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
51. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
52. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
53. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
54. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
55. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
56. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
57. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
58. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
59. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
60. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
61. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
62. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
63. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
64. and we developed a technique called tactile dowsing which has the user wield their wii remote like a dowsing rod and this technique works as follows: 1) the user moves their wii remote from left to right 2) Using an extern infrared transmitter we track the direction the wii remote points using the wii remotes infrared optical sensor 3) we defined a tactile window around the target direction 4) as soon as the wii remote enters the tactile window we pulse the vibrotactor. 5) we decrease the pulse delay the closer the user gets to the target direction, so we use a form of haptification 6) how many pins are being hit depends on how close the user is towards the target vector when the user throws their ball.
70. So we did a user study to evaluate VI Bowling
71. A little video of one of our participants playing VI Bowling
72. A little video of one of our participants playing VI Bowling
73. We recruited six people with visual impairments 4 of which were legally blind and two totally blind.
74. Here is a chart that shows the results. It shows the average dowsing time over 10 frames and the number of pins hit. Data is highly variable and decrease in dowsing time was not found significant. The average aiming error was 9.5 degrees which is good
75. Some of the issues that we are investigating currently are: 1) how effective are multimodal motion instructions? what is hte best way to encode motion instructions using tactile feedback. The wii remote is limited in its ability to provide tactile feedback. 2) how accurate is the motion capturing