Published on

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this


  1. 1. Sarat KonetiTable of contentsIntroduction Full Product Description Test ObjectivesExecutive SummeryMethod Participants Sessions Tasks Test Facility Participant’s computing Environment 1
  2. 2. Sarat KonetiIntroductionA usability test was carried out on the web application at Kingston University. A usability test isintended to determine the extent an interface (such as a website) facilitates a user’s ability to complete routine tasksand how it would make his life easier. Usability testing has many benefits, including that it improves the ease of use ofa website and enables the designers and developers to learn about things that will improve its performance.In this project, an onsite usability test was conducted using the live version of Product is a free online organizer and planner web app that helps families manage their schedules, shopping lists,household chores and to-do lists – all in one place.The target audience is primarily families. Other users are groups of friends. Types of work supported by the app areshopping lists and to do lists, appointments and activities, chores, and meals.Test Objectives/AimsThe objective of the test is whether the app is user-friendly and whether it serves its purpose.Examples ● Whether or not the site easily and quickly allows the user to create, edit and view his or her schedule ● User’s overall experience with the websiteExecutive SummaryThe test users took part in the usability test at the Penrhyn Road campus at Kingston University inLondon, England on 6th December 2012. The purpose of the test was to assess the usability of the web interfacedesign, and whether routine tasks could be easily achieved.6 users participated in the test. Typically, a larger number of participants are involved in a usability test to ensure morereliable results. However, limitations of the project were the number of participants available. Each individual testlasted approximately 30-40 minutes.Analysing from the questionnaire, all participants found the website to be clear, straightforward and 67%found the website easy to use. The test identified some problems, including: confusion with the terminology and lackof feedback.MethodParticipantsFive participants were students and one was a full-time worker. Six participants were scheduled to take the test overtwo testing days (6 and 7 December 2012). Six out of six participants completed the test. Three participants were maleand three were female. The age range was 22-26 with a mean age of 23.8 years old.All participants were frequent computer users with a good working knowledge of Microsoft Windows. None had usedthe product befoe. Two participants have had experience in online calendar applications and the rest are very familiarwith mobile planners. Professional Computer Product Gender Age Education Occupation / role Experience Experience Experience FemP1 Student ale Fem StudenP2 ale tP3 Fem Online ale marketi ng 2
  3. 3. Sarat Koneti consult ant MP4 Student ale MP5 Student ale MP6 Student aleSessionsParticipants were recruited from the university. Each individual session lasted approximately 30-40 minutes. Duringthe session, the test was explained and the participants were asked to fill out a questionnaire about their background.The tests were carried out without any interruptions.Context of Product Use in the TestTasksIntended context of use: Participants were interviewed with a pre-test questionnaire about their initial impression ofthe website and their experience of using similar websites or calendar applications.Context used for the test: Test participants attempted completion of the following tasks (see Appendix for moredetails):1) Creating Events The participant must create two events on the calendar on the given dates and times2) Family Member The participant must add a family member and create an appointment for him on the given date and time3) Calendar Colour Code The participant must change the colour of the calendar colour code of the family member4) Sharing Calendar The participant must share their calendar with a friend using the specified email address5) Repeated Events The participant must create a repeated event that lasts for two months and set a reminder6) To-Do List The participant must go to the specified to-do list and strike off the tasks given in the test7) Shopping List The participant must plan a meal on the given date, find the recipe from the given selections and add it to the Family Favourites. Then the ingredients must be added to the shopping list, which should be named ‘Cooking’.Test FacilityIntended context of use: home.Context used for the test: Two participants carried out the test in the lecture hall at Kingston University, Penrhyn Roadcampus. The remaining participants carried out the test in their homes. Participants worked without any interruptions,and were recorded using computer software. I sat beside them out of screen-sight.Participant’s Computing EnvironmentIntended context of use: web app is intended for use on any browser. The web version is not intendedfor use on Smart phones and external devices.Context used for the test: All the participants were asked to use my own laptop which has • Windows 7 premium operating system • Pentium T4300 , 4gb RAM • Resolution: 1366 x 748 • Safari Browser • ‘Camtasia’ screen recording softwareMaterials – Test Administrator Tools 3
  4. 4. Sarat KonetiTasks were recorded and timed using Camtasia (trial version). Sessions were filmed (a combined picture of the screenand a view of the participant). At the end of the sessions, participants completed a post-test questionnaire (seeAppendix for details).Manual input devices: Key board & MouseProcedureBefore the test, participants were informed that we would be investigating the usability of during thesession, to find out whether it was easy to use and served its purpose, and how long tasks took. They were told that itwas not a test of their abilities, but a test of the website itself. Participants were informed that they would be recordedduring the session. They were asked to sign a consent form. Participants were asked about themselves before the testtasks: occupation, their experience with online calendars, computer experience.Participants were given instructions before the test, and also told to think aloud during the test. They were asked abouttheir initial impression of the website.After the last task, participants were asked to complete a post-test questionnaire which asked about their overallimpression and experience using the website.I asked the participants if they encountered any difficulties using the website and whether there is anything that theyfelt could be improved.Before beginning the test with a new participant, I have cleared all the data(events and fields) which was created bythe previous participant.Expected ResultsIt is expected that is user-friendly, bug-free and intuitive to use – it is a live product.ResultsMost of the tasks were successfully completed by all the participants. All the participants have spent more time toaccomplish the tasks 2, 4 and 7. Although adding a family member/s is the main purpose of the application, usersfound hard time to add a family member to their account. The mean time taken by all the participants to complete allthe tasks is 25min and 40sec.Performance results:Task 1: Participant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Task 2: 4
  5. 5. Sarat KonetiParticipant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Task 3:Participant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Task 4:Participant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Task 5: 5
  6. 6. Sarat KonetiParticipant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Task 6:Participant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Task 7:Participant # Task Time Task Completion rate Completion rate/time Seek Help 1 5.48 100% 18.2% 0 2 3.41 100% 29.3% 0 3 4.10 75% 24.3% 0 4 3.02 100% 33.1% 0 5 3.12 100% 32.05% 0 6 4.30 100% 23.2% 0 mean 3.385 100% 160.15 0Standard error 0 0 0Findings & Recommendations: 6
  7. 7. Sarat KonetiFindings in task 1: • Six out of six participants did not know instantly how to add an appointment to the calendar. All the participants wasted at least one click before double clicking. It was not clear that they had to double click on a date to make an appointment. • However, All participants(100%) managed to complete the task eventually. One participant partially failed to complete the task in the expected direction. • When signing in, if you get either the username or password wrong, it does not inform you which is wrong. So the user has to type in both username and password again, whereas if it just said, e.g. password was wrong, they would only have to type in the password. One participant did not like this.Two ways of adding appointments to the calendarRecommendations for task 1: • There should be a tooltip or it should be intuitive to double click to add an appointment to the calendar. (‘tooltip’ should be disappeared in 20sec from the time page loads or when clicks to close) • Suggestions to improve include better design to make the user automatically think to double click on the date cell, or there should be an icon in the cell ( add should appear when user mouse on the date cell, if events already exists edit button should appear.‘Tooltip’ indicate the user to double click ‘add & edit’ icon on a date cell 7
  8. 8. Sarat KonetiFindings in task 2: • All participants took a long time to add a family member to the account. One participant had to use the Help screen. • 6 out of six participants (100%) clicked on the family photo if they could find more information. But right now user will only be able upload his/her photo in that section. • Generally the participants did not understand the terminology needed to add a family member, i.e. it was not clear they had to go to “General information about household”. • Though they are at the right screen, 2 out of six participants did not notice that it was the right place to add a family member. • 3 participants felt they could find the family information on the home page.Here are two ways to add a family member 8
  9. 9. Sarat KonetiRecommendations for task 2: • There should be more intuitive ways to add a family member. For example, there could be an item in the menu that says “My Family”. This would be more obvious and direct solution. • Other alternatives could be by clicking on the “family photo” the user would be taken to the whole family’s information. This would be a good solution, as my test shows 67% of participants expected this anyway. • The terminology “General information about household” could be clearer to the user, i.e. it could be “My Family”. Two out of six participants actually arrived at this screen but clicked away because it was not obvious it was the correct place to add a family member. • Another solution could be to have the family information shown on the homepage, for example, a “Welcome” message and then a “Family information” section. • This task proved to be more of the more difficult. However, it could be easily resolved.Recommended changes in navigation to add a family member 9
  10. 10. Sarat KonetiFindings in task 3: • 5 participants were managed to complete this task easily. • One participant thought it is not possible to change the colour code of the family member but finally he managed to complete the task after seeking assistance. The reason for his confusion is participant could not see the family members calendar colour codes in month view. So he thought that, it does not exist. • 4 out of six participants able to complete this task in less than 2min. • 2 participants clicked directly on the small circle icon which was used to represent the family member colour code to change its colour. • 3 participants expected to change colour code from the same page where they have added a family member. • One participant expected that changing the calendar colour code of the family member, will affect the entire colour theme of the calendar e.g. date colours, appointment colours for that specific family member. • One participant suggested that changing the calendar colour code of the family member should change the appointments of that specific family member.No colour code for family members in month view Colour code for family members in day viewRecommendations for task 3: 10
  11. 11. Sarat Koneti • Although five participants completed the task easily, there were some unnecessary clicks (more than required to achieve the task) by most of them. • A suggestion is to have this colour change ability on the calendar itself. Also, as recommended by one the participant’s, the colour change should be used to differentiate between different family member appointments. • For example, if Peter’s calendar colour is red and John’s is blue, when the user looks at the whole family’s calendar, he would see all Peter’s appointments in red and John’s in blue. This change would be time- consuming and require developer input. • Currently, the only significance of the calendar colour change is superficial and not to improve usability. • When the calendar colour is changed, only the heading bar on the calendar changes colour. To be more user- friendly, I would recommend that the whole calendar colour is changed, as one participant expected. • Considering all the suggestions, I recommend that instead keeping button name ‘show’ on the top, It could be the family member name which is active. Also colour change for the appointmentsPresent menu for filtering appointments Recommended menuFindings in task 4: • Most of the participants found problems in sharing the calendar with his/her friend. • All the participants did at least 2 mistakes to share the calendar. • 3 out of six participants tried to share the calendar from the date cell on the calendar, which is wrong. • Some participants felt the terminology used to share the calendar was misleading, “Send URL”- They were not clear about the completion of the task. • One participant was confused with the text area where they had to enter the email address. • Most of the participants wanted an acknowledgement that the calendar had been shared. • One participant wanted to see a history of people with who he had shared the calendar. 11
  12. 12. Sarat KonetiParticipant was not sure the she is doing rightRecommendations for task 4: • This task was very unintuitive. Recommendations are to change the terminology, so instead of “Send URL” it would say “Share Calendar”. Once the calendar had been shared, there would be an acknowledgement that it had been shared. As suggested by one participant, it would also be better if the user could see a history of who had been sent the calendar. • There are currently two ways in which the calendar can be shared- directly from the calendar under ‘Set up’ and then ‘Share appointments’ or in the Settings menu and then ‘Calendar Settings – Send URL’. This is not user-friendly – the terminology should be consistent. Both phrases are misleading. I would recommend “Share calendar” for each one. This is easy to do. • One participant’s suggestion was to see a history of people the calendar had been shared with, which I think is a good idea. However, this would require further development and would be time-consuming.My recommendation for sharing calendar design 12
  13. 13. Sarat KonetiFindings in task 5: • All the participants completed this task with ease except the participant who fails to know that, double clicking the date will allow him/her to add an event, chose different way to complete this task too. Four out of six participants did not like the American date format.Failed participant way to add repeated events General way of adding repeated eventsRecommendations for task 5: • Same recommendations as with task 1 – to double click an appointment should be more intuitive. Under Settings, there should be a way to format your date and time depending on your country. This could be easily resolved.Findings in task 6: • Six out of six participants completed this task with ease.To-Do listRecommendations for task 6: 13
  14. 14. Sarat Koneti • The To-Do list functions well. An alternative idea could be to have a ‘Complete?’ button next to the item, where the user can tick whether they have done it. However, because 100% of the test participants completed this task very easily, this is not needed.Findings in task 7:All the participants took a long time to do this task. There are three parts to this task: ● Some participants chose the calendar page instead of Meals page. ● To add a recipe to the Family’s Favourites – all participants completed this task easily ● To add the list of ingredients needed to the shopping list and name it cooking – all the participants completed this task easilyExsiting Meals Interface(The right side ‘your recipe box’ is neglected by the most of the users)Recommendations for task 7: • I would recommend to reduce the number of steps needed to plan a meal from ‘Cozi Picks’. Instead of going to different pages, all the information would be on one page arranged in columns so the user can easily drag and drop from one column to the date. • It is not intuitive to drag and drop how it is now, I would suggest using a tooltip on the mouse-over. Although all participants completed the task successfully, the times were very long and could be reduced with further thought and design. 14
  15. 15. Sarat KonetiMy recommendation for ‘Cozi picks’ layoutDiscussionMost of the participants enjoyed using the software and found it acceptable to use. However all participants areunlikely to use the website in the future. This is most likely because there are alternative options, such as mobilecalendars, Google calendar etc. However, my test results may not be truly representative because the participants werenot specifically the target audience of the website, i.e. not “busy families”. However, all participants were users ofonline or mobile calendars so the context of the test was appropriate. 15