NASA Etoys In Space Usability Evaluation

Etoys in Space Etoys in Space Heuristic Review of Usability and Design Issues by Mike Lee and the OLPC Learning Club DC May 2009 Draft 2

• OLPC Learning Club Etoys in Space Jam Session • Post Jam Meeting and Further Analysis • Comments on Re-packaging for OLPC Deployments • Commentary on the Individual Activities • Next Steps

Initial Observations of NASA Squeak Web Activities Running on OLPC XO December 2008, Sugar OS 8.2.1, Etoys 3 • Slow load time on some activities • New Etoys menu bar interferes with activity content objects • Text in some text blocks overflows • Animation sprites sometimes run slowly • Layouts for 1024x768 now feel cramped • 1024x768 background images repeat • Some pages on NASA Connect web site don’t fit on XO screen • The Etoys Activities are located here: http://www.pcs.cnu.edu/~rcaton/SqIndex/squeakindex.html

OLPC Learning Club Etoys in Space Jam

Etoys in Space Content Inventory Created December 2008 Posted to: http://www.visuallee.com/etoys

Insights from the Jan. 9, 2009 Conference Call hosted by Nortel • Randy Caton says it’s pretty easy to fix the NASA Squeak Web Activities. He’ll do it based on feedback logged by the Learning Club. • But he would not only like bugs noted, he also wants suggestions for improving overall usability and the interactive learning experience. • He also wants ideas on how the lessons can be redesigned for use in the least developed countries (LDCs) where there is much less classroom support. • He provided a link to a site where the activities had been translated into Spanish. http://www.pcs.cnu.edu/~rcaton/sp/squeakindexSP.html • The Powerpoint for the conference call is posted on Slideshare http://www.slideshare.net/fotogeek/etoys-in-space-conf-call- presentation

Email comments from Kathleen Harness (via Rita) January 12, 2009 Middle/high school? I don't know how much physics and math are assumed so perhaps all the text in the books is needed, but perhaps not. The cartoon characters look like they will appeal to younger students than the content of lessons. Are the lesson goals tightly aligned with specific national standards and can/will teachers use them during class or as an extra credit activity? Sadly, many teachers I talk to about Etoys are so pressed for time and so focused on testing requirements that they won't use projects that are not precisely what they need. Some objects in the projects are locked and others are not. I leave almost everything unlocked in the spirit of exploration and to give visitors as much power as possible; break it and open the project again. This will also help with scale-to-fit/full screen choices if someone can just pick up something that is in the way. The Playing button in the student section led me expect a demo of the lesson. I would change the name of that button to more closely match what students find there. Two suggestions to free up more space on the screen. You can see examples of both in use at www.EtoysIllinois.org • The K-5 Technology Passport lessons incorporate Etoys Help Quick Guides as references in the word doc materials and in a project flap as a shortcut for people who like a project and want to try it on their own. Anyone trying the project has Etoys open and can click the question mark in the Navigator Bar. Referring to the guides by name cuts out a lot of repetition of information in documents and lets people choose the material they need to read and skip the material they know. • Flaps take up less space on the screen than books and leave more room for objects, scripts, and buttons. Create flaps with (alt-shift-w). Flaps have an extensive options menu. Name the flap what ever you wish: Directions, Page 1, Challenge 1. They show the project's organization and content in a way that a book format does not. Flaps can have text and scripted objects. My students prefer making a flap for their projects like puzzles and games when they want to provide rules, advice, or puzzle pieces. Please let me know if I can be of further help, I am very glad your project is going forward. Regards, Kathleen Harness kharness@illinois.edu

Etoys in Space Jam Participants January 17, 2009 Arlington Career Center, VA Participants: • James Crawley • Edward Dale • Lisa Dale • Sarah Elkins • Jeff Elkner • Mike Lee • Pat Paul • Ton van Overbeek

Etoys in Space Jam Session Notes • Eight participants from the club of varied backgrounds, two with some with software testing experience • Realized three hours was not nearly enough time • An initial 45-minute orientation was not enough; Jam participants had difficulty understanding the Etoys interface as well as the rationale of some of the activity content • Participants logged a few bugs • The group had a useful 30-minute conversation on their initial impressions of the Etoys activities • Jakob Nielsen’s Ten Usability Heuristics were used as a guide

Etoys in Space Jam Session Participant Observations • Most commonly observed was the inconsistent content design across the activities • Noted the extensive amount of reading required on the introductory web pages and in the lessons • A home-school parent said she would not use these particular activities due to the amount of advance preparation apparently necessary and unusual Etoys user interface • Book page turning camera sound was annoying to several • One participant was put off by typos in the activity book text • Animation sprites left trails in some cases (on the XO laptop)

Jakob Nielsen’s Ten Usability Heuristics - 1 1 Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. 2 Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order. 3 User control and freedom Users often choose system functions by mistake and will need a clearly marked \"emergency exit\" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo. 4 Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions. 5 Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action. Source: http://www.useit.com/papers/heuristic/heuristic_list.html

Jakob Nielsen’s Ten Usability Heuristics - 2 6 Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. 7 Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions. 8 Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. 9 Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. 10 Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large. Source: http://www.useit.com/papers/heuristic/heuristic_list.html

Post Jam Meeting and Further Analysis

Randy Caton’s Presentation March 21, 2009 Randy was able to attend a Learning Club meeting to present on his projects and gained some insights in conversation with members who participated.

Further Analysis • Quick test of Etoys on small-screen netbooks for layout • Captured a large library of screenshots from web site and the Etoys activities for reference • Created a user experience flow diagram for the GIS activity • Evaluated for consistency of layout and design • Developed wireframe diagrams to show standardized layouts • Suggested additional authoring and development tools

Etoys in Space in on Netbooks • Netbooks have become a major new market category with increased product development for the education market. • Content design and technical tradeoffs will have to be made in further work on the Etoys in Space activities. • Asking the user to move activity objects around to fit a given screen resolution is not ideal.

Netbook Screen Size Chart This chart is NOT actual size. Please load the original from here: http://farm3.static.flickr.com/2476/3549525335_a6cf3c4402_o.png Source: http://www.eeepc.it/en/monitor-dei-netbook-dimensioni-a-confronto/ Also see Wikipedia’s comparison of netbooks: http://en.wikipedia.org/wiki/Comparison_of_netbooks

Student Interaction Flow Diagram • The Student path from the web site through the GIS Challenge activity was flow diagramed (next slide) • Multiple web pages must be traversed by the student to gather the necessary introductory information yet no back and forth navigation is offered (besides the back button in the browser) • Just before the student enters the Etoys activity, it is suggested that the individual make a detour to the Etoys Quickstart Guide • Once inside the activity, the student relies heavily on 17 pages of book content to complete the five challenges in the activity as well as learning the script and how to modify it

GIS Challenge - Student Interaction Flow Etoys Web Quickstart Site Web Activity Guide Web Activities Web Activity Page Index Page Start Page (800x600px) NASA Connect Etoys Tips Video Content Web Activity Page (Students) Launch Activity Etoys Activity Read 17 Book Challenges Challenges Panels Challenges 1-5 Challenges 1-5 Check 1-5 How to Modify Challenge 2 1-5 Browser and the Script Resolution Settings Study Probe Script Quit Activity Post Activity Reflect & Discuss LEGEND Web Page Task Step

Etoys in Space Introductory Web Pages - 1 Click http://www.pcs.cnu.edu/~rcaton/SqIndex/squeakindex.htm l

Etoys in Space Introductory Web Pages - 2 Click http://www.pcs.cnu.edu/~rcaton/ESgps/index.html

Etoys in Space Introductory Web Pages - 3 Click http://www.pcs.cnu.edu/~rcaton/ESgps/start68.html

Etoys in Space Introductory Web Pages - 4 Click Launches Etoy Activity http://www.pcs.cnu.edu/~rcaton/ESgps/students68.html

Inconsistent Layout Across the Different Activities

Layout Change Recommendations • Unify NASA logo, activity title and spot artwork into single dismissible flap panel • Place book in upper left corner of all activities • Standardize to stationary, horizontal and vertical action layouts • Keep the Control and Watcher panels together and always near the action area • Group together function buttons, move others into a flap • Keep the existing flaps along the bottom edge of the screen • Gutter space between block objects and groups of small objects needs to be consistent • Menu bar will stay visible and clear of activity content • Investigate how layouts’ resolution-dependent animation scripts can support various netbook screen sizes all the way up to OLPC XO 1200 x 900px and larger • The wireframe layout diagrams that follow assume that the Etoys in Space activities will be individually accessed via web pages or as clickable file icons

Wireframe 1 - Stationary Action Layout A B O U T Book Stationary Action Area B U T T O N S Controls Watcher Function Buttons This wireframe is illustrative and not pixel-precise.

Wireframe 2 - Horizontal Action Layout A B O U T Controls Watcher Function Book Buttons B U T T O N S Horizontal Action Area This wireframe is illustrative and not pixel-precise.

Wireframe 3 - Vertical Action Layout A B Controls O U T Book Watcher B U T Vertical T O Action Area N S Function Buttons This wireframe is illustrative and not pixel-precise.

Wireframe 4 - About Flap Open NASA Logo --> A B O U Activity Title --> Sundial T Controls Watcher Function Book Challenge Buttons B U T Spot art --> T O N S Lorem ipsum dolor sit amet, consectetur adipiscing elit. Welcome text --> Nunc interdum lacinia Horizontal Action Area malesuada. Donec vitae neque a mauris auctor dapibus. Nunc interdum cursus sem, non elementum leo euismod sed. This wireframe is illustrative and not pixel-precise.

Wireframe 5 - Buttons Flap Open A B O Lorem ipsum dolor sit amet, U consectetur adipiscing elit. Text if needed. --> T Nunc interdum lacinia malesuada. Book Controls Watcher B U T T Function O N Buttons S Horizontal Action Area This wireframe is illustrative and not pixel-precise.

Wireframe 6 - Expanded About Flap Open NASA Logo --> A B O U Activity Title --> Sundial Controls Watcher T Function Book Challenge Buttons B U T Spot art --> T Book containing O N S text from the introductory Lorem ipsum dolor sit amet, Web site pages. consectetur adipiscing elit. Welcome text --> Nunc interdum lacinia Horizontal Action Area malesuada. Donec vitae neque a mauris auctor dapibus. Nunc interdum cursus sem, non elementum leo euismod sed. This wireframe is illustrative and not pixel-precise.

Book Buttons Violate Fitts’ Law The time to acquire a target is a function of the distance to and size of the target. The buttons should be as tall as the Mac or Windows mouse cursor and generally provide ample area for the user to land the cursor and accurately effect a mouse click. Explanations of Fitt’s Law: http://particletree.com/features/visualizing-fittss-law/ http://en.wikipedia.org/wiki/Fitts's_law

Invoke Display of Bigger Book Buttons As Default We recommend that the default navigation buttons for the books be set to the large size. The green text telling the user how to use the arrow buttons is unnecessary.

A Good Book Design Example http://www.etoysillinois.org/library.php?sl=356

Example of Buttons Stored in a Flap http://www.etoysillinois.org/library.php?sl=169

Check Color Contrast To accommodate individuals with color blindness or other visual impairments contrast between adjacent colors should be maintained. For example, some of the flap colors in the Robot Challenge do not have enough contrast. One way to gauge contrast is to print a screen in grayscale to an inkjet or laser printer or convert a screenshot to grayscale.

Suggestions For Generating Fresh Artwork • Re-create Norbert, Zot and props so that they can be photographed in dioramas from any number of angles with a digital camera to create new artwork. Possible materials: – Sculpy clay – LEGO Bricks • Find a student cartoonist or digital artist to re-create the artwork • Better background textures can be photographed with a digital camera or downloaded as free Creative Commons licensed material from Flickr

One Essential Reference Book http://www.amazon.com/About-Face-Essentials-Interaction-Design/dp/0470084111/

Screen Ruler Utilities These on-screen pixel rulers are very useful to check for consistent spacing between user interface elements. For Microsoft Windows: http://www.spadixbd.com/freetools/jruler.htm http://www.microfox.com/ For Mac: http://www.pascal.com/software/freeruler/

Comments on Re-packaging Etoys in Space for OLPC Country Deployment

Re-Packaging Etoys in Space for OLPC Country Deployment We first recommend embarking on an extensive redesign of the current individual Etoys in Space activities using some of the recommendations presented by the Learning Club. • The Etoys in Space activities will need to be bundled as a single .XO file into existing laptop activity collections and/or made available for download online as an .XO file, which is a compressed ZIP file • .PR files for the separate Etoys in Space activities must all be linked to a launcher page with icons for each activity • The content that prefaces each activity on the web site must be edited for brevity and clarity and bundled into the .PR files • We recommend deconstructing OLE Nepal’s excellent E-Paath Etoys Activities to develop content, design and implementation specifications suitable for country deployment http://www.olenepal.org/activities_download.html

OLE Nepal E-Paath Screenshot 1

Commentary on Individual Activities

Commentary on Individual Activities A series of reduced screenshots of the starting state of each of the activities follows. Each has callouts with comments. • In general, the Escape Browser button implemented in the Ratio Challenge and some of the other activities should be deployed throughout. The additional text label isn’t necessary. • It is assumed that a standard workaround for the overlap with the menu bar at the top can be developed. • Do not use bold for the body text in the books. Subheads are missing in some books and should be used consistently.

GIS Challenge - Commentary Why is this the only activity that starts with a script showing? The forest map is too small and is lost on the page. The size seems forced small unnecessarily because it is desired to illustrate the concept of pixels. The tiny probe dot could be replace with a crosshair. This page should use Wireframe 1 - Stationary Action Layout. The visibility of the large script panel should be managed by show/hide buttons in the book.

Sundial Challenge - Commentary The background artwork is simply too busy and because the characters are largely obscured by the foreground panels, does not add value to the user’s experience. Wireframe 1 Stationary Action should be used.

Park and Mail Ball Gravity - Commentary Wireframe 3 - Vertical Action Layout should be used here leaving much more room around Zot the dog. Subtitles should not look like clickable buttons. Subtitles should not look like clickable buttons.

Park and Mail Challenge Mass Spring - Commentary Wireframe 1 should be used here.

Norbesaur Challenge Dino Run - Commentary Wireframe 2 for Horizontal Action should be used here.

Norbesaur Challenge Dino Track - Commentary Wireframe 2 for Horizontal Action should be used here. The background is overpowering. See next slide for a calmer sand texture image.

Norbesaur Challenge Dino Track - Better Sand Texture Flickr sand texture CC: http://www.flickr.com/photos/tactus/483287393/

Rocket Challenge - Better Rocket Art Flickr sand texture CC: http://www.flickr.com/photos/mwboeckmann/2425571382 /

Laser Challenge - Commentary Wireframe 1 for Stationary Action should be used here. Stack the books. The airplane animation is gratuitous.

Robot Challenge Mission 1 - Commentary Wireframe 3 for Vertical Action should be used here. Avoid the use of vertical text.

Robot Challenge Mission 2 - Commentary 30% larger buttons would make them easier to click and provide some space around the button text for better readability. Wireframe 3 for Vertical Action should be used here. Low color contrast between text and flap color.

Rocket Launch Challenge - Commentary Wireframe 3 for Vertical Action should be used here. The clouds, rocket, gantry and blue ocean look hastily drawn. It seems like it would be easy to replace these with public domain imagery from NASA. The computer-rendered rocket above the control center panel looks like it could be part of the simulation. Place it and the title in the About Flap.

Exercise Challenge - Commentary Wireframe 1 for Staionary Action should be used here.

GPS Challenge - Commentary Wireframe 1 for Staionary Action should be used here.

Theorem Challenge - Commentary Wireframe 1 for Stationary Action should be used here.

Freefall Challenge Elevator - Commentary Titles should not look like clickable buttons. Wireframe 3 for Vertical Action should be used here.

Freefall Challenge Freefall - Commentary Titles should not look like clickable buttons. Wireframe 3 for Vertical Action should be used here.

Ratio Challenge - Commentary Wireframe 1 for Stationary Action should be used here.

Cars Challenge - Commentary Wireframe 1 for Stationary Action should be used here with the exception that the book does fit well in the oval raceway.

Pyramid Challenge - Commentary Placing Flap at top breaks from your own style here. Wireframe 1 for Stationary Action should be used here.

Graph Challenge - Commentary Layout looks OK.

Plotting - Commentary Layout looks OK.

Demo - Commentary Layout looks OK. This demo doesn’t seem compelling enough.

Next Steps • Further conversation with Randy Caton • A project roadmap? • Funding?

Contact • Mike Lee curiouslee [at] gmail.com • http://www.olpclearningclub.org

NASA Etoys In Space Usability Evaluation

0 comments

Notes on slide 1

Favorites, Groups & Events