Playful Toothbrush CHI 2008 presentation


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • I would like to thank the session chair for the introduction. Also thank everyone here for coming to my talk. My talk is about the playful toothbrush for teaching tooth brushing to kindergarten children
  • Playful Toothbrush CHI 2008 presentation

    1. 1. Playful Toothbrush: Ubicomp Technology for Teaching Tooth Brushing to Kindergarten Children CHI 2008, Florence Italy Yu-Chen Chang, Chao-Ju Huang, Hao Chu , Peggy Chi National Taiwan University Jin-Ling Lo, Nan-Yi Hsu, Hisn-Yen Wang, Ya-Lin Hsieh National Taiwan University Hospital
    2. 2. Child Behavior Training <ul><li>Child behavior training as important but challenging parental responsibility </li></ul><ul><ul><li>Potty training, self-dressing, cleaning room, self-feeding, tooth brushing (this work). </li></ul></ul>
    3. 3. Persuade Children to Change Behaviors <ul><li>The most common form of parental persuasion is verbal persuasion. </li></ul><ul><ul><li>“ If you don’t brush your teeth properly and thoroughly, you are not allowed to go to sleep”. </li></ul></ul><ul><li>Not effective, why not? </li></ul><ul><ul><li>Verbal persuasion alone lacks proper incentive to motivate children </li></ul></ul>
    4. 4. A Case Study: Child Tooth Brushing <ul><li>Target kindergarten children (5-6Y) </li></ul><ul><ul><li>Learn to care for their own oral hygiene </li></ul></ul><ul><ul><li>Average 5Y children brush only 1/4 of teeth (Rugg-Gunn) </li></ul></ul><ul><ul><li>A common scenario: candies -> improper brushing -> cavities -> dental clinic </li></ul></ul>
    5. 5. Play-based Occupational Therapy <ul><li>Pediatric occupational therapy (O.T.) </li></ul><ul><ul><li>Leverage the desire of children to play to induce their behavior change willingly. </li></ul></ul><ul><ul><ul><li>Children may not like to brush teeth, but like to play. </li></ul></ul></ul><ul><ul><ul><li>Add playfulness (game) into the brushing activity </li></ul></ul></ul><ul><ul><li>Effective – full of toys in pediatric O.T. clinic. </li></ul></ul>
    6. 6. Play-Based O.T. Limitations <ul><li>(1) Children treated in clinics during regular office hours (NTU Hospital) </li></ul><ul><ul><li>Many child behavior problems not observable to therapists </li></ul></ul><ul><ul><ul><li>Eating (dinner time), brushing (before bedtime), etc. </li></ul></ul></ul><ul><ul><ul><li>Effective treatment is difficult. </li></ul></ul></ul><ul><li>(2) Train general performance skill rather than specific functional skill </li></ul><ul><ul><li>Hand dexterity skill vs. tooth brushing skill </li></ul></ul><ul><ul><li>Improvement in general performance skills may not translate into the target functional skills </li></ul></ul>
    7. 7. HCI/UBICOMP Opportunities <ul><li>Embed digital technology into a child naturalistic living environment </li></ul><ul><ul><li>Sensing to observe child behavior anytime, anywhere </li></ul></ul><ul><ul><li>Game playing to influence child behavior anytime, anywhere </li></ul></ul><ul><li>Occupational therapist perspective: </li></ul><ul><ul><li>From treatment clinic </li></ul></ul><ul><ul><li>To the child actual living environment (functional behaviors) </li></ul></ul><ul><li>HCI/UBICOMP perspective: </li></ul><ul><ul><li>From sense and track behaviors </li></ul></ul><ul><ul><li>To engage children to change behaviors </li></ul></ul><ul><ul><ul><li>Also called Persuasive technology (by Fogg, King, and others) </li></ul></ul></ul>
    8. 8. Playful Toothbrush Goals <ul><li>(Child) Dislike brushing </li></ul><ul><ul><li>Make brushing more attractive </li></ul></ul><ul><li>(Child) Habitual brushing but not properly </li></ul><ul><ul><li>Teach proper brushing skill </li></ul></ul><ul><li>Not replace adults’ supervision </li></ul><ul><ul><li>Extend their effectiveness </li></ul></ul><ul><ul><li>Used together can overcome young children’s limited physical and cognitive abilities, such that they can successfully learn proper brushing </li></ul></ul>
    9. 9. <ul><li>Sensing </li></ul><ul><ul><li>Camera vision to detect children’s brushing actions </li></ul></ul><ul><ul><li>Brushing actions become game inputs </li></ul></ul><ul><li>Playing </li></ul><ul><ul><li>An interactive game of brushing teeth </li></ul></ul><ul><ul><li>Start with a mirror image of dirty teeth. </li></ul></ul><ul><ul><li>Brushing own teeth maps to cleaning the same virtual teeth in the mirror image. </li></ul></ul><ul><li>Demo video </li></ul>Playful Toothbrush Brush extension Web camera for tracking brushing motions Camera #2 to record videos for later human analysis Brushing game
    10. 10. Play-based Occupational Therapy Model <ul><li>Playful toothbrush is a treatment program / play activity (three steps). </li></ul>Volition Performance Habituation Bring enough enjoyment to attract a child to participate in the target activity. Ensure a child will have a successful experience. Set appropriate level of difficulty. Apply reinforcement to reward good performance, so increase change of repeating desirable behavior. Enough times to become a habit.
    11. 11. Design Considerations <ul><li>Enjoyment </li></ul><ul><ul><li>Bring enough fun to attract children to brushing </li></ul></ul><ul><ul><li>Associate brushing with game playing and having fun </li></ul></ul><ul><li>Engagement </li></ul><ul><ul><li>Simple Interaction (Not all young children can learn to operate complex devices) </li></ul></ul><ul><ul><li>Use their natural brushing actions as game inputs </li></ul></ul><ul><li>Automation </li></ul><ul><ul><li>Help children learn tooth brushing skill and internalize as habit </li></ul></ul><ul><ul><ul><li>In brushing case, internalizing means doing without much thinking) </li></ul></ul></ul><ul><ul><li>Advice adults not to interrupt children and their own motor planning, necessary for internalization of brushing motor skill </li></ul></ul>
    12. 12. Playful Toothbrush Prototype (two main components) Vision-based brushing tracker Brushing game
    13. 13. Vision-based Brushing Tracker <ul><li>Use one camera to detect brushing </li></ul><ul><ul><li>Recognize brushing of 24 different teeth surface areas (or granularity of 2 adjacent teeth surfaces) </li></ul></ul><ul><li>Mouth closed during brushing </li></ul><ul><ul><li>Bristle-teeth contact area not visible to an external camera </li></ul></ul><ul><li>Brush extension </li></ul><ul><ul><li>Each of four faces has unique LED-pattern </li></ul></ul><ul><ul><li>Used as marker to assist camera-vision recognition </li></ul></ul><ul><ul><li>Children told not to block brush </li></ul></ul><ul><ul><li>extension from the camera </li></ul></ul>
    14. 14. How to recognize a brushing stroke on a specific teeth surface area? <ul><li>Take an example (brushing the outer surface of frontal teeth) </li></ul><ul><li>Three features determines this brushing stroke </li></ul><ul><ul><li>Bristle rotated toward the face </li></ul></ul><ul><ul><li>Brush oriented parallel to the face </li></ul></ul><ul><ul><li>Back-and-forward motion vector </li></ul></ul><ul><ul><li>(parallel to the face) </li></ul></ul>
    15. 15. Computer Vision Technique <ul><li>Use the brush extension marker to reconstruct </li></ul><ul><ul><li>Bristle rotation angle (x-axis) </li></ul></ul><ul><ul><li>Brush orientation angle (z-axis) </li></ul></ul><ul><ul><li>Brush back-n-forward motion vector </li></ul></ul><ul><li>Infer the brushing stroke and its target teeth area </li></ul>x (θ roll ) z (θ yaw ) y brush extension
    16. 16. Accuracy Test (vision-based brushing tracker) <ul><li>13 kindergarten children (72-81 months) </li></ul><ul><li>Recorded 48 minutes of their brushing videos </li></ul><ul><li>Compare human-read brushing strokes (ground-truth) and machine-recognized brushing strokes </li></ul><ul><li>Machine-recognized accuracy 90% </li></ul><ul><ul><li>Need not be perfect. Occasional error in games has little effect. </li></ul></ul>
    17. 17. 8.4% 6.8% Average 12.0% 41/343 3.7% 241/6493 13 0% 0/168 2.4% 118/4974 12 17.6% 111/630 10.2% 695/6847 11 8.6% 115/1298 0.9% 32/3532 10 3.3% 23/688 3.4% 239/7104 9 12.2% 206/1288 13.5% 579/4302 8 0% 0/196 5.3% 290/5520 7 0% 0/80 2% 90/4551 6 9.5% 60/634 4.3% 209/4813 5 5.3% 20/380 11.7% 617/5270 4 16% 209/1304 6.5% 331/5130 3 14.8% 49/332 14.8% 880/5958 2 9.7% 47/483 9.2% 602/6521 1 (%) false positive rate (frame) (%) false negative rate (frame) Children
    18. 18. Brushing Game <ul><li>Start with a virtual mirror image of the children’s own uncleaned teeth </li></ul><ul><ul><li>Grouped into 24 teeth surface areas </li></ul></ul><ul><li>“ 1” appears next to the first cleaning target </li></ul><ul><ul><li>7 layers of plaques were drawn, requiring 7 brushing strokes </li></ul></ul><ul><ul><li>Each brushing stroke removes one layer of plaque </li></ul></ul><ul><ul><li>Combine audio feedback: Do-Re-Mi-Fa .. </li></ul></ul><ul><ul><li>No game response for brushing other teeth areas </li></ul></ul>
    19. 19. Brushing Game <ul><li>After 1st area is complete, “2” appears next to the 2 nd target area. </li></ul><ul><li>Applause at the game completion </li></ul><ul><ul><li>Visual-audio feedbacks provide a sense of accomplishment </li></ul></ul><ul><li>Enforce a brushing sequence (Stillman’s brushing method) </li></ul><ul><ul><li>Upper right, upper left, the lower right, and lower left </li></ul></ul><ul><li>Two benefits of sequencing </li></ul><ul><ul><li>Ensure brushing all teeth </li></ul></ul><ul><ul><li>Repeat this sequence enough time -> “do without thinking” </li></ul></ul>2 3 2
    20. 20. User Study <ul><li>Two questions guided our user study </li></ul><ul><ul><li>How effective is the Playful Toothbrush in improving the brushing skills of kindergarten children? </li></ul></ul><ul><ul><li>What aspects of brushing behaviors were affected by Playful Toothbrush? </li></ul></ul>
    21. 21. User Study Test Subjects <ul><li>13 young children (72 – 81 months) from a NTUH-run kindergarten class </li></ul><ul><ul><li>5 girls and 8 boys </li></ul></ul><ul><li>Written informed consent from parents </li></ul><ul><li>Tooth brushing is required after meals or snacks </li></ul><ul><li>Setup </li></ul><ul><ul><li>Install our system at the kindergarten’s restroom </li></ul></ul><ul><ul><li>A video camcorder to record children’s brushing sessions </li></ul></ul>
    22. 22. User Study Procedure (11 days spanned over 3 weeks) <ul><li>Pre-trial (one day) </li></ul><ul><ul><li>Familiarize with the therapists and our system </li></ul></ul><ul><li>Pretest (two days) </li></ul><ul><ul><li>Brushed with their own toothbrushes </li></ul></ul><ul><ul><li>Established a baseline behavior </li></ul></ul><ul><li>Training (five days) </li></ul><ul><ul><li>Brushed with our playful toothbrush </li></ul></ul><ul><ul><li>A trained therapist helps children understand the brushing game </li></ul></ul><ul><li>Posttest (two days) </li></ul><ul><ul><li>Brushed with their own toothbrushes </li></ul></ul><ul><ul><li>Measured behavior improvement from pretest </li></ul></ul><ul><li>One week follow up (one day) </li></ul><ul><ul><li>Brushed with their own toothbrushes </li></ul></ul><ul><ul><li>Measured behavior retention after a week </li></ul></ul>
    23. 23. User Study: Brushing Effect <ul><li>Teeth cleaning effectiveness </li></ul><ul><ul><li>Red plaque disclosing dye before each brushing session (dye attached the plaque) </li></ul></ul><ul><ul><li>Oral exam (counted teeth surfaces with dye before and after brushing) </li></ul></ul><ul><ul><li>Plaque index </li></ul></ul><ul><ul><ul><li>number of teeth surfaces with plaque / total number of teeth surfaces </li></ul></ul></ul><ul><ul><li>Cleaning effect </li></ul></ul><ul><ul><ul><li>Reduction of Plaque Index from before to after brushing </li></ul></ul></ul>
    24. 24. User Study: Brushing Behavior <ul><li>By analyzing/coding brushing videos, measured 3 aspects of brushing behaviors : </li></ul><ul><ul><li>Length of brushing time </li></ul></ul><ul><ul><li>Number of brushing strokes on each of 24 teeth areas </li></ul></ul><ul><ul><li>Total number of brushing strokes </li></ul></ul>
    25. 25. User Study: Cleaning Effect Results 0 .70( 0 .14) 0 .18( 0 .10) 0 .88( 0 .14) Day 11 Follow-up 0.66(0.28) 0.16(0.07) 0.85(0.10) Average 0 .70( 0 .15) 0 .15( 0 .08) 0 .86( 0 .16) Day 10 0 .67( 0 .13) 0 .16( 0 .10) 0 .83( 0 .11) Day 9 Post - test 0.66(0.17) 0.08(0.06) 0.74(0.19) Average 0 .68( 0 .17) 0 .11( 0 .10) 0 .79( 0 .18) Day 8 0 .64( 0 .26) 0 .04( 0 .05) 0 .68( 0 .28) Day 6 0 .67( 0 .15) 0 .09( 0 .10) 0 .76( 0 .14) Day 4 Training 0 .32( 0 .21) 0 .37( 0 .18) 0 .69( 0 .25) Day 3 Pre-test Cleaning effect M ean ( SD ) After brushing M ean ( SD ) Before brushing M ean ( SD )
    26. 26. User Study: Brushing Behavior Results 120(36) 8.31(5.07) 239.62(107.48) Day 11 Follow-up 118(30) 8.46(4.82) 248.00(87.12) Average 99(31) 9.46(5.12) 214.31( 71.91) D ay 10 137(41) 7.46(4.89) 281.69(120.66) D ay 9 Posttest Average Training 76(45) 12.39(4.75) 190.46(138.38) Average 67(46) 13.46(5.35) 168.62(157.03) Day 3 84(53) 11.31(5.25) 212.31(137.77) Day 2 Pretest Brushing time M ean (SD) (sec) Number of unbrushed teeth areas M ean (SD) Number of brushing strokes M ean (SD)
    27. 27. Result Summary <ul><li>(Pretest) Child subject failed to clean 37% of their teeth surfaces </li></ul><ul><li>After 5 training days, improvements in </li></ul><ul><ul><li>Teeth cleaning effectiveness </li></ul></ul><ul><ul><li>Number of brushing strokes </li></ul></ul><ul><ul><li>Length of brushing time </li></ul></ul><ul><ul><li>Coverage of brushed teeth areas </li></ul></ul>
    28. 28. Conclusion <ul><li>Demonstrate a case study of applying HCI/UBICOMP technology in play-based occupational therapy </li></ul><ul><li>For teaching young children proper tooth brushing, user study results were encouraging </li></ul><ul><li>Other similar repetitive development tasks for young children (self-feeding, potty training, cleaning room, etc.) </li></ul><ul><ul><li>This technique can make training attractive and simple for adults/children. </li></ul></ul>
    29. 29. Examples of Other Persuasive Technologies
    30. 30. <ul><li>Sense to recognize behavior </li></ul><ul><ul><li>Weight sensor underneath the tray to sense eating actions </li></ul></ul><ul><ul><li>Eating actions as game input </li></ul></ul><ul><li>Play to engage behavior change </li></ul><ul><ul><li>Interactive games: coloring cartoon character or penguin fishing </li></ul></ul>Case Study: Playful Tray Encourage good eating habit in young children
    31. 31. <ul><li>Sense to recognize behavior </li></ul><ul><ul><li>Tilt sensor to detect drinking actions </li></ul></ul><ul><ul><li>Drinking actions are game inputs </li></ul></ul><ul><li>Play to engage behavior change </li></ul><ul><ul><li>Game metaphor: hydrating/dehydrating body -> watering/drying a tree </li></ul></ul>Case Study: Mug-Tree Encourage healthy habit of drinking fluid regularly
    32. 32. <ul><li>Sense to recognize dress & dressing color </li></ul><ul><ul><li>Camera and Computer Vision </li></ul></ul><ul><li>Play to engage behavior change </li></ul><ul><ul><li>Playful explore & experiment with how different clothing color look on people </li></ul></ul><ul><li>CHI 2008 poster on Tuesday </li></ul>Case Study: ChroMirror Persuade people to explore more colorful dressing
    33. 33. Thanks Q & A
    34. 34. Lesson #4 <ul><li>Mediate, not automate </li></ul><ul><ul><li>Mediation: not to replace children’s efforts but make the experience of performing more enjoyable for children </li></ul></ul><ul><ul><li>Mediation works better than automation in this case. </li></ul></ul>
    35. 35. Lesson #1 <ul><li>Unpredictable behaviors from young children </li></ul><ul><ul><li>Make activity recognition difficult </li></ul></ul><ul><ul><ul><li>Randomly switch from left to right hands, swing brush wildly, head movement brushing the brush … </li></ul></ul></ul><ul><ul><li>Observe child behaviors carefully before designing and programming the system </li></ul></ul>
    36. 36. Lesson #2 <ul><li>Activity grading </li></ul><ul><ul><li>Children have different physical and cognitive capabilities </li></ul></ul><ul><ul><ul><li>Some learned fast (not challenging enough); some learned slowly (frustration) </li></ul></ul></ul><ul><ul><li>Different levels of challenges fitting to each child’s ability </li></ul></ul><ul><ul><li>“ Play-based O.T. is about the experience of performance, or the fitness between the level of challenge in an activity and a child’s physical and cognitive capabilities.” </li></ul></ul>
    37. 37. Lesson #3 <ul><li>Personalization & customization </li></ul><ul><ul><li>Easily personalized and customized to environmental or human factors (preferences of children, changing performance of a child, different deployment environments). </li></ul></ul><ul><ul><li>Left hand vs. right hand </li></ul></ul><ul><ul><li>Lighting condition </li></ul></ul><ul><ul><li>Child height </li></ul></ul><ul><ul><li>Preferences about game characters </li></ul></ul>
    38. 38. Implementation Details <ul><li>Toothbrush extension weight </li></ul><ul><ul><li>Not too heavy for children to cause inconvenience in brushing </li></ul></ul><ul><li>Prevent child from holding the brush extension </li></ul><ul><ul><li>Block the brush extension from the overhead camera </li></ul></ul><ul><ul><li>Solution: a protruding cap as separator </li></ul></ul><ul><li>Camera placement </li></ul><ul><ul><li>Not too high such that camera cannot see the brush extension clearly. </li></ul></ul><ul><ul><li>Not too low such that children can switch the brush extension outside camera view </li></ul></ul>