USA TODAY iPad Application Usability Report

Usability Test Final Report USA TODAY’S iPad application By Kathleen Sullivan, Gannett/ For USA TODAY 5.3.2010

Background • Success of USA TODAY mobile applications motivates design and development of native iPad app

USA TODAY iPhone app

AutoPilot iPhone app

Android app

USA TODAY iPad video http://www.youtube.com/watch?v=p5nJVtLygOM&feature=player_embedded#!

Purpose • The purpose of this usability evaluation is to determine how users of the USA TODAY iPad application will perform with the application and to make recommendations for improving the application.

Why do a usability test? • It is necessary to conduct usability testing on the USA TODAY iPad application in the interest of improving the program for future versions of the application. • The usability evaluation of USA TODAY iPad app will identify problems in the user interface that impact readability, ease-of-use and overall user satisfaction.

Core objectives of test: The core objectives for conducting usability tests on the USA TODAY iPad application are to ensure that the application: • Is easy to launch and use. • Allows users to find USA TODAY content they are looking for easily. • Loads quickly at launch and throughout use. • Presents all formats of USA TODAY content clearly. • Allows users to share and save content quickly and efficiently. • Design promotes intuitive navigation among the USA TODAY content sections. • Evaluate the overall user interface in order to determine whether the application was designed to allow users to browse and find content efficiently. • Evaluate the layout of different content formats among sections and navigation options. • Identify opportunities to enhance the user interface design and user experience design. • Measure the time required to complete tasks in order to determine ease of use. • Observe any errors that occur during application use so that improvements can be made.

Article layout Weather layout

Scores layout Photo gallery layout

User proﬁles • Eleven participants Age group Number • 5 male, 6 female 18-24 4 25-30 2 • 5 iPhone/iPod touch 30-40 1 users, 6 non-users 40-50 2 50-60 1 60-70 1

Methodology The usability evaluation for USA TODAY’s Apple iPad application was conducted in early-mid April 2010. Evaluations did not start until after April 3 — the release date for the iPad and the application. The USA TODAY iPad application was tested using: 1. The thinking aloud protocol 2. Timed task completion 3. A debrieﬁng survey that included Likert-scale rating questions and open-ended questions • Eleven volunteers participated in a 10-15 minute hands-on evaluation of the iPad application, during which users were timed for task completion and participated in a thinking aloud protocol. After the test, each participant completed a follow-up survey, which included Likert-scale questions on the application and open ended questions.

Task List Script 1. Read the headline for the top story on the home screen of the application. Select that article to read and navigate to the story page. Scroll through the article as if you were skimming it. 2. Find the most recent sports scores for the last NBA basketball game and read it aloud. 3. Go to the Sports section front. Click to read the first article available. Continue onto the next page of that same article. Then navigate to the next sports article. 4. Navigate to the weather and say the current temperature where you are. 5. Change the weather location to one of the other cities listed. 6. Navigate to the Life section and find the latest movie review. Find the option to share the story and click to share the story via e-mail. Cancel before sharing. 7. Still in the Life section, find the photo gallery for “Day in celebrities” and scroll through the photos. Bring the last photo in the gallery to full resolution and share the photo via e-mail (cancel before sending). Then exit out of the gallery. 8. Navigate to the home screen for the application and click on the ad at the bottom. Cancel out of the ad. 9. Navigate to the Money section from the home screen and click on an article to read it. Change the font size of the text in for that article. Then change it back. 10. Please navigate back to the home screen of the application and refresh the content.

Post-test survey Likert scale survey questions: 1. Please rate your experience with the USA TODAY iPad application. Overall experience with the USA TODAY iPad application was (circle one, with 1 representing the least satisfying experience and 7 representing the most satisfying experience across all categories):

Post-test survey 2. Please rate your reaction to the design of the USA TODAY iPad application. (1 represents the least satisfying design and 7 represents the most satisfying design. Feel free to use comments area to express your opinions on the design of each section (optional).

Results Timed results by participant Owns Age range of Q1: Read top Q2: Find the Q3: Navigate Q4: Find the Q5: Change Q6: Navigate Q7: Find the Q8: Click on Q9: Navigate Q10: iPhone/iPod participant story most recent to the sports weather and the weather to the Life photo gallery the ad on to the Navigate to Touch headline and sports scores section, click read the location to section and for “Day in the home Money the home navigate to for the last on any current one of the find the Celebrities”, screen section, click screen and article page NBA sports temperature other cities latest movie scroll to the on any manually basketball article, go to listed listed review last photo article and refresh the game the next and share it change the content page of that via e-mail font size of article and the text then the next article of the section iPod Touch 50-60 2 6 14 2 60 30 20 39 4 3 iPod Touch 40-50 9 4 12 13 47 8 40 35 10 2 iPhone 30-40 6 14 40 7 26 39 57 42 3 2 none 60-70 28 2 4 15 27 15 44 40 25 15 none 18-24 5 3 20 12 60 6 4 20 13 4 none 18-24 6 58 2 13 60 27 20 32 12 2 none 25-30 2 3 3 2 21 25 15 23 4 2 none 25-30 30 2 3 29 60 31 60 1 41 38 iPhone 18-24 28 10 3 33 60 15 33 4 4 5 iPhone 18-24 3 16 4 40 7 12 27 7 2 1 none 40-50 10 60 60 23 60 29 15 33 25 2 MEAN 11.72727272727 16.18181818182 15 17.18181818182 44.36363636364 21.54545454545 30.45454545455 25.09090909091 13 6.909090909091 MEDIAN 6 6 4 13 60 25 27 32 10 2 Editing the weather location was the most difficult task (took the longest at an average of over 60 seconds) Refreshing the content was the least difficult task (average time to complete: 2 seconds)

Product design problem areas: Home screen navigation Weather feature

Timed Results

Timed Results Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 >>>18-30 year-olds tended to ﬁnish tasks faster than participants who were 30 years old or older.

Timed Results No signiﬁcant difference in timed task completion by gender

Likert scale survey results Overall satisfation • Likert scale overall satisfaction rating of 5.197 • Lower ratings indicated that some found the application to be slightly frustrating and difﬁcult to use compared to the other rating indicators.

Likert scale survey results Feature/page ratings • Likert scale overall features satisfaction rating of 5.327 • Lowest rating was given to the weather feature, which most participants had trouble with when trying to edit the location. Highest rating average was given to the home screen and article screen.

Qualitative Results Open-ended question responses 1. What was your initial impression of the USA TODAY iPad application? 1. ositive. P 2. lean — similar to their paper design. C 3. Nice. 4. ice looking. Better than iPhone app. N 5. unctional; a bit of a learning curve but once familiar very easy to navigate. F 6. news app. A 7. lean, focused on the key/important things users want. C 8. ood, informative, crisp. G 9. thought the app was visually beautiful and well laid out. It made ﬁnding many I news articles very easy, and easy to read. Like a newspaper without the gross ink. 10. Good. 11. Easy to use like the iPhone app.

Qualitative Results Open-ended question responses 2. What did you like about the USA TODAY iPad application? 1. ouch action, ease of turning pages, resolution of photos. T 2. eather, ﬂipping through articles, clean design. W 3. esign. D 4. ood size, nice layout, reminded me of paper, easy to read. G 5. lear, colorful, inviting, organized and balanced (information and ent.) C 6. t was good. I 7. asy to navigate, not cluttered. E 8. ood variety of information — liked tying to different sections. G 9. loved the pictures and the screen was just beautiful. I also liked the way the I articles were laid out on the screen. 10. It was well organized and easy to use. 11.The simplicity was likable about the app.

Qualitative Results Open-ended question responses 3. What did you dislike about the USA TODAY iPad application? 1. Some buttons were not intuitive. 2. All sections not listed? Where is “tech”? Your users are early adopters of the iPad! How could you not have tech? Back button on most features also not intuitive. 3. The weather and the navigation. 4. Navigation between sections impossible — no clear button to punch to get from one section to another. Should be able to punch a sports button to get to sports. Photo gallery thumbnails too small. 5. Weather setup. 6. Not enough direction if I wasn’t asked questions. 7. Weather (took a while to ﬁgure out how to change to another city on the list) 8. Weather section (make more intuitive to change location) 9. I did not like the weather feature – not enough detail for my tastes. I prefer weather.com, which gives hour by hour data. 10. The navigation between different topic pages was confusing. 11. The sports scores could be clickable and have more data.

Qualitative Results Open-ended question responses 4. Do you currently own an iPad or plan to purchase one? Currently own: 0 Plan to purchase:1 Do not plan to purchase: 7 Unsure: 3 5. Has your experience with the USA TODAY iPad application inﬂuenced your decision to purchase or plan to purchase an iPad? Yes: 1 No: 6 — Cost, not needed; No need for one; Just bought a new laptop; It doesn’t make phone calls; Unsure: 4

Qualitative Results Open-ended question responses 6. If you own or plan to own an iPad, do you plan to get the USA TODAY iPad application? Yes: 1 — If I owned one, I would get this app because I ﬁnd the iPhone app fun and easy to use. No: 8 Unsure: 2 7. Did you think there was a function missing in the iPad application? No: 9 Unsure: 1 Yes: 1 — Back button, geo-based ads, stories updates;

Qualitative Results Open-ended question responses 8. If you could change any feature to the USA TODAY iPad app, what would it be and how would this enhance your experience? 1. Even more interactive and customization options. 2. Section front experience and back buttons. 3. N/A 4. Navigating between sections was a bother. 5. Order cities for weather; more categorical organization on the home section. 6. N/A 7. N/A 8. Weather and making it easier to get between locations. 9. I would like more games like the jumble or cross-word available. 10. To have ease to change from sports to life or money a lot faster and easier. 11. Add more sports features.

Qualitative Results Open-ended question responses 9. Did you have any difﬁculty using the USA TODAY iPad application? If so, what did you have difﬁculty with and how might this be improved? No: 3 Yes: 8 — Changing city locations; homepage buttons — should add section buttons to homepage so you know how to navigate; weather and sections;

“ ” Qualitative Results Thinking Aloud Protocol “Hard to exit out of the ad. “X” button not intuitive.” “Weather is definitely not intuitive. I haven’t seen that before. On the iPod Touch the weather is much better.” “The iPad seems fine but I wouldn’t buy the first version of anything.” “I like the brightness of the page and the white space.” “I like the way the pages are laid out. I like the fact that you can turn it sideways. I like the photo galleries and the color.” “My nails got in the way of touching some of the buttons.” “I thought the indicators for the weather buttons would be more obvious.” “The share button looks like a reply button.”

“ ” Qualitative Results Thinking Aloud Protocol “Navigation between sections is nonsense. I can’t ﬁgure it out.” “Need a close button on pop-ups.” “When you’re in the Sports section, it’s not intuitive to press the Sports button to go home or somewhere else.” “Sports scores don’t have last night’s scores, they have today’s schedule — that’s annoying when I want to know the score for last night’s game.” “For changing the weather locations it looks like you just are supposed to pick one. There’s nothing to indicate their in any kind of sequence.” “I didn’t see anything else so I tried that button” (upon ﬁguring out how to navigate between sections using the USA TODAY button) “The weather was frustrating.”

“ ” Qualitative Results Thinking Aloud Protocol “I didn’t think it was a link, just thought it was a picture of the USA TODAY logo.” “They should make the selected city a different color so you know to drag it up.” (on the weather edit locations) “Is there a search function on the application?” “I like that you can scroll through pages.” “I don’t like that the tech section is missing. People who have an iPad will want to ﬁnd tech stories.” “I don’t like how hard it is to get back to the homepage.” “I feel like there should be a back button on a lot of these pages.” “Wish it would pull up the scores for teams I like most – like customizable. Can you do that?”

Recommendations Based on the quantitative and qualitative results for the USA TODAY iPad application, we are recommending the following improvements be made as soon as possible: 1. Add a feature to more clearly indicate how to navigate between sections of USA TODAY 2. Adding language (text) or buttons (obvious visual components) to more clearly indicate how to edit/change the weather location 3. Add a sub-layer of navigation and search to include other sections/topics that USA TODAY is known for (Ex: technology, economy, movies, travel, etc.) (Makes content easier to ﬁnd) 4. Take advantage of the iPad’s interactive features by including more interactive visuals and other features within the application (Ex: cross-word games, market graphs and other interactive graphics/games).

USA TODAY iPad Application Usability Report

by kathleensulli on Jul 24, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 90

Statistics

USA TODAY iPad Application Usability Report — Presentation Transcript