PBS KIDS iPhone App Study: Findings and Implications


Published on

Presented @ Game Education Summit North America 2010

Published in: Design, Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • PBS tests! There isn’t much information out there, so we’re regularly working with our producers to find opportunities to go into the field.
  • Are apps usable, age-appropriate, engaging for 3-7 year olds in Title I schools? How frequently will kids play? For how long? How appealing is the content and delivery format to kids and parents? How well do the apps foster learning of the target curriculum?
  • 90 iPod Touches were given to groups of 3-7 year old children from two Title 1 Schools (DC and Bloomington, IN) Low-income students are determined by the number of students enrolled in the free and reduced lunch program.
  • · 90 iPod Touches were given to groups of 3-7 year old children from two Title 1 Schools (DC and Bloomington, IN)
  • How PBS works. About the Apps. Alpha Pig, Princess Presto, Wonder Red, and, of course, Super Why, while practicing the alphabet, rhyming, spelling, writing and reading. Super Duper! ALPHA PIG’S LICKETY LETTER HUNT Help Alpha Pig find his way home by identifying the letters he needs to follow the alphabet path. That’s Alphabet Power! PRINCESS PRESTO’S WANDS-UP WRITING Help Princess Presto make objects appear by identifying letter sounds, tracing letters on the touch screen, and writing words. Spectacular Spelling! WONDER RED’S RHYMING TIME It’s time to rhyme! Help Wonder Red find the rhyming words missing from her Wonder Words Basket. SUPER WHY’S STORY SAVER Super Why...to the rescue! Help Super Why save the day by selecting words to complete sentences that fit the story. LEARNING REWARDS Collect virtual stickers along the way and decorate a “sticker book” with over 50 items.
  • ·      
  • Kids has trouble picking apps. Normally parents would have a screen dedicated to kid apps. kids touched apps icons too long so the apps were deleted. No way to disable this on device. Kids used device’s home button instead of arrows within the app
  • Inaudible instructions: In many instances we found that we had to direct children to turn up the volume on their iPods and/or use headphones to better hear the instructions and question prompts within each game. Inability to hear instructions or questions clearly was mentioned by a small set of participants throughout the study, and we suspect some challenges arose when children were playing in settings where there was more ambient noise.
  • How kids hold the device
  • · Played more at beginning, lost interest when no new content · At the beginning, parents reported children playing the ipods for long persiod of time in each sitting but playtime decreased after the first few days, most noteably after older children had played through and mastered each minigame within both Apps, and after the novelty of each game play experience had worn off for younger children
  • Silliness of licking the plate in Chow Time minigame. Younger kids weren’t reward by getting correct answers, but by mere ability to control the device. They jumped from game to game, app to app quickly and randomly. Less deliberate play. Older kids play more deliberately to get answers wrong/right. Martha Speaks satisfying for less deliberate players.
  • not sure if there is learning if there is too much fun. Like the didatic. How do I win.
  • In addition to assessing children’s availability to express their knowledge of various vocabulary words visually, we were also interested in seeing how well the participants were able to express their knowledge verbally. We selected four questions for each assessment form based on vocabulary from Martha Speaks: Dog Party—specifically “Martha Says” and “Doggie Dress-Up.” Efforts were made to ensure a balance between simpler and more complex vocabulary terms. Prompts were offered for clarification purposes on the words related to motion that have multiple meanings (e.g., rest, dart and pace). Up to 31% increase (martha says, doggy dress up) 3 year olds Letter Sounds Alpha Pig Lickety Letters Princess Pesto 4 yr olds 23% increase (identify what letter makes a sound in a word, image accompanied it) Wonder Reds Rhyming Time 3 yr olds: 16% increase Comprehensive: 33% increase overall in rhyming Doggie dress up & chow time 3 yr olds: 25%
  • Possbilities with iPad Usability (thumbs) Family device
  • Parents are guided to record feedback using exZact data collection client: Kids’ usage Usability notes Other observations, feedback exZact apps also used for initial parent survey Also surveying teachers on use in the classroom
  • PBS KIDS iPhone App Study: Findings and Implications

    1. 1. PBS KIDS iPhone App Study Findings and Implications Nina Walia Associate Director, PBS KIDS Interactive
    2. 2. <ul><li>pbskids.org </li></ul>
    3. 4. <ul><li>pbskids.org </li></ul>9 million unique visitors per month
    4. 6. PBS KIDS Research Studies <ul><li>Regular usability testing, educational efficacy testing, new platform testing </li></ul><ul><li>Sprint cell phone study - 2006 </li></ul><ul><li>iPhone app study, Super WHY! And Martha Speaks - 2010 </li></ul>
    5. 7. App Study Overview <ul><li>Participants and Methodology </li></ul><ul><li>Usability and Usage Patterns </li></ul><ul><li>Appeal </li></ul><ul><li>Learning Outcomes </li></ul><ul><li>What’s Next </li></ul>
    6. 8. I. Participants & Methodology <ul><li>90 Participants </li></ul><ul><li>3-7 year old kids (and their parents) </li></ul><ul><ul><li>Washington, DC </li></ul></ul><ul><ul><li>Bloomington, IN </li></ul></ul><ul><li>Recruited from Title I schools </li></ul><ul><li>Evenly split above/below poverty line </li></ul><ul><li>Funded by a Ready to Learn Grant from the US Department of Education. </li></ul><ul><li>Research conducted by Rockman et al. </li></ul>
    7. 9. Participants Participants by Ethnicity Asian: 3% Latino/Hispanic: 6% Black/African American: 16% White/Caucasian: 64% Other: 11%
    8. 10. The Apps <ul><li>Super WHY! </li></ul>Martha Speaks: Martha’s Dog Party
    9. 11. Methodology <ul><ul><li>  The children played two Apps: </li></ul></ul><ul><ul><ul><li>Super Why </li></ul></ul></ul><ul><ul><ul><li>Martha Speaks: Dog Party </li></ul></ul></ul><ul><ul><li>2 week period </li></ul></ul><ul><ul><li>Parents submitted daily observation forms and completed short pre- and post study surveys </li></ul></ul><ul><ul><li>Children given pre and post study tests to assess ability in reading skills and content areas covered in Apps </li></ul></ul><ul><ul><li>Time with Apps was self initiated, not prescribed </li></ul></ul>
    10. 12. II. Usability and Usage Kids were very skilled at using the device, had few problems, learned quickly
    11. 13. Navigation
    12. 14. Usability <ul><li>Pump up the volume ! </li></ul><ul><li>Kids taught us how to play on touch screens. </li></ul><ul><li>Kids all are thumbs . </li></ul>
    13. 15. Kids Are All Thumbs
    14. 16. Usage Patterns <ul><li>Most notable trend in long term play mentioned by children and parents was a decline in use over time. </li></ul><ul><ul><li>No new content; redundancies in content after mastery </li></ul></ul><ul><ul><li>Only Apps on the iPod Touch </li></ul></ul><ul><li>Average number of minutes playing both apps each session: </li></ul><ul><ul><li>about 40% played 5-10 min </li></ul></ul><ul><ul><li>about 50% played 10-20 min </li></ul></ul><ul><li>Approx 5 hrs total time spent on the iPod during the two weeks </li></ul><ul><ul><li>Each child played approx 2.5 hrs on Martha speaks, playing with the app 11 times </li></ul></ul><ul><li>Parents noted that warmer weather and a desire to play outside (spring break!) won out over the iPod in the later days of the study. </li></ul><ul><li>Allure and interest of the iPods and apps remained high and consistent during long flights or road trips. </li></ul>
    15. 17. Parents <ul><li>How much they helped their children : </li></ul><ul><ul><li>73% responded “not a lot” </li></ul></ul><ul><ul><li>amount of help decreased over time </li></ul></ul><ul><li>3 & 4 yr olds benefit more from parents involvement. </li></ul><ul><ul><li>repeating question </li></ul></ul><ul><ul><li>additional prompts </li></ul></ul><ul><ul><li>clues </li></ul></ul><ul><ul><li>scaffolds to help facilitate correct response </li></ul></ul><ul><ul><li>kids reached out for meaning of words </li></ul></ul><ul><li>Intergeneration play often a reflection of parent or child’s preferred game play style than whether kids needed help. </li></ul>
    16. 18. III. Appeal <ul><li>challenges. </li></ul><ul><li>progressively challenging levels </li></ul><ul><li>characters trick players </li></ul><ul><li>learning new words </li></ul><ul><li>evolved interesting patterns of play to increase challenge & fun </li></ul><ul><li>collectable accomplishments. </li></ul><ul><li>games that featured characters of their gender. </li></ul>
    17. 19. III. Appeal silliness. control of the iPod.
    18. 20. III. Appeal Parents did not like if learning objective was not clear. Games seem to have no end points.
    19. 21. IV. Learning Outcomes Kids were primed to learn different concepts at different ages. That said… There were learning gains on content and skills from both apps. Vocabulary improved as much as 31% from playing the Martha Speaks App! Parents reported apps educational and kids are using the words in every day speech. “ All of a sudden he says, “I’m going to dart into the living room,” and proceeds to run off.” “ I like your crimson dress.” Words where we saw the biggest gains include: mauve, teal, floral, nautical, shapes, amble, stroll, sprint, portrait. With the Super Why App, 4 yr olds had 23% increase in identifying Letter Sounds! 33% overall average improvement in rhyming abilities!
    20. 22. Next Steps <ul><li>Update Apps according to what we learned. </li></ul><ul><li>content updates </li></ul><ul><li>usability tweaks </li></ul><ul><li>iPad experiments. </li></ul><ul><ul><ul><li>look ma! no thumbs! </li></ul></ul></ul><ul><ul><ul><li>family device </li></ul></ul></ul>
    21. 23. <ul><li>Nina Walia </li></ul><ul><li>[email_address] </li></ul><ul><li>@ missmodular </li></ul>Thank you! PBS KIDS Mobile Downloads: pbskids.org/mobile PBS KIDS Mobile Technologies & Learning Research: http://pbskids.org/read/research/mobile.html Find these slides on slideshare.net