Usability and User Research (UX2 UCD Day)


Published on

This presentation introduces user research methods and highlights examples from case studies to demonstrate how these methods can be used. A basic guide to usability test techniques is provided including best practice and practical advice.

Published in: Education, Technology, Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Just going to quickly cover some of the basics so everyone is on the same page. Then I'll provide some reasons why its important, these might hopefully help you when trying to make a positive change in your own institutions Usefulness and usability and why usefulness is just as important Then I will go through some of the methods used during our project and the impact it had and lessons we learned. I’ll be skipping pretty quickly through a lot of information so if I’m going too fast or you want to ask a question please ask as we go. If there’s time we’ll have more questions at the end.
  • Often the type of thing I've heard from people over the years. Commonly this is what many service managers believe when they don't feel there is a problem but as we all know, lots of 'little' problems can quickly add up and create a bad user expeience.
  • So why is usability important to all digital services? People will go elsewhere if they have difficulties using a services – already we've seen from our and other's research which points to the fact that students use a variety of resources to fulfil their needs, frequently using external resources such as Google books, scholar among others. Often students are unaware of the benefits or the full features of their university services including digital library because of usability issues such as poor signposting, complicated forms and hidden services which students are unaware of.
  • Steve Krug provides a great example of why its important by illustrating the typical reservoir of goodwill each person has inside them.
  • Small problems can quickly add up
  • These apply across the board, regardless of the type of digital service you provide.
  • Comforts – breadcrumb links, clear navigational buttons.
  • This is the definition of usability from ISO, key things being effectiveness, efficiency and satisfaction.
  • These are all questions you should ask when evaluating your own services and systems. Throughout user research these questions should be addressed.
  • Testing can be conducted with on average 6 users but even as few as 3 if conducted regularly on a monthly basis. With three users and just one or two tasks you can evaluate a system in a couple hours once a month. Also you don't need an expensive test lab to do user research. A laptop with internet access and paper +pencil is sufficient to conduct tasks and take notes. There are also cheap recording software packages available and free trials to record each test and share your findings with others. I’m hoping I might be able to convince you all of these things so that you might go out and conduct your own user research!
  • In addition to learnability, efficiency and satisfaction – utility is also important attribute to be aware of when creating and evaluating digital services. Usefulness is just as important as usability
  • When we started the UX2 project we looked at theoretical frameworks which might help us to evaluate an information system in the most holistic way, ensuring that the research was thorough and robust. TAM was one model we examined. It has been around for many years and will reveal a persons intention to adopt an IS.
  • Interactive Triptych Framework (ITF) was another framework we researched. It is based on TAM among other theoretical frameworks Interaction between 3 components -provides 3 evaluation approaches Interaction is affected equally by content and system characteristics Examines usefulness separately Ability of ITF to thoroughly evaluate success of DLs in a holistic manner So when we began to carry out user research, we tried to evaluate not only the usability of a system but also it’s usefulness to its users. Next are some methods which you can apply at different stages of your research to help design information systems and evaluate existing ones.
  • Personas are part of the data gathering phase of research, before you test anything you ideally want to have an idea of who your typical users are. This is especially important when you are approaching a new service This is an example of the personas we created during our project.
  • Data gathering system where the wide array of data and information is gathered and funnelled down into several personas which represent all the individuals involved in the data gathering. This can be done using several methods including interviews and usability testing. The primary aim it to gather data on users behaviour, goals and attitudes.
  • Number of steps involved in creating personas: 1. collect qualitative data – we chose to interview a number of students and librarians 2. Collect all notes on each interview and present in a single page under different headings. We chose goals & behaviour, attitudes and personal traits 3. take each individuals interview notes and try to summaries each person using a couple lines or several bullet points. Separate out each aspect into different coloured post-it notes. These must provide an efficient summary of that person. 4. next slide...
  • 4. conduct two-by-two comparisons of each interviewee. Plot each person's distinction on a scale – next slide..
  • We then created a scale for each distinction identified during the two-by-two comparison and determined end points. Doing so allowed us to place each participant on the scale and directly compare them.  Most variables can be represented as ranges with two ends. It doesn’t matter whether a participant is a 7 or 7.5 on the scale; but what matters is where they appear relative to other participants. The next slide provides an example of our 12 scales mapped for each of our 17 participants.
  • We used coloured pens and stickers to plot each individual as well as labelling each individual as a number – this made it easier to 'eyeball' each scale and spot individuals who were close on a number of scales. Creating segments from these scales was by far the trickiest part. It's not an exact science and is open to interpretation. We initially had 6 personas which we were able to narrow further once we started designing each persona. At that stage the similarities between two personas became apparent and the argument to merge these personas was strong enough to reduce the number further.
  • When designing personas include a name, photo, personal information such as their background, interests and hobbies as well as their attitudes to information seeking/ digital services etc. You can also include some of the more important scales to provide a quick overview in addition to the more detailed description. You'll get the chance to see a basic persona if you're attending the prototyping workshop this afternoon. We were unable to recruit staff and therefore did not create a persona which represented them.
  • If its the first time you undertake persona development, factor in extra time to read up on the processes and get familiar with what you are required to do. Don't underestimate the amount of notes you need during the interview to use later. During the interview keep in mind the information you are looking for and ensure that you ask the right questions to get an understanding of these things i.e. information seeking behaviour.
  • One-on-one observations of work practice in its naturally occurring context. [Switch to Boon's slides]
  • If we go back to our definition and remind ourselves what we are looking for when we conduct a usability evaluation.
  • Scavenger hunt : typical task scenarios which are useful when testing a particular area of service in the interface. The Reverse Scavenger hunt : show users the answer and ask them to find and purchase it. E.g. Image of clothing to source online. Self-generated tasks : using interview questions beforehand to help participants identify their requirements Part self-generated : ask users to bring data with them to the session and then use it to construct an ad-hoc scenario. ‘ Skin in the game’ tasks : gives users real cash to purchase something as part of the task making it as realistic as possible Troubleshooting tasks : ask the user to solve a problem
  • This is an example of a scenario which we designed for our own usability tests. By adding context you provide a sense of realism. Wherever possible, try to ensure the subject matter is relevant to user (but not always the case, in which case emphasise that the participant must try to imagine themselves in this situation) Adding specific details focuses the task to the area you want to evaluate. In this example we added the detail of the type of information required. This forced the participant to think about how to screen for information published recently – hopefully using the facet navigation to do so.
  • However, sometimes scavenger hunt tasks have limitations – particularly when testing search services and digital libraries.
  • This is what a self generated task might look like in a test script with an example of the type of task a student might come up with.
  • Self generated tasks can provided the greatest realism and therefore generate greater findings which scavenger hunt tasks might not reveal. However it also means you can't directly compare tasks between participants and is therefore not idea if you are looking to gather quantitative data (although it can be used in conjunction with other tasks which specifically collect quantitative data)
  • Word choice questionnaire was a list of standard adjectives, both positive and negative. Participants were asked to tick all the words which summarised their experience and circle the top three. Morae is a usability test software which is typically used to record tests by professionals and it good at helping you edit together a highlights video to share with others. We have several edited highlights from the testing which are available to see as an example.
  • Try not to interrupt participants and give them lots of time to speak – sometimes you find they say something unexpected. Hard not to be quiet and takes practice but certainly an important skill. Ask open questions which aren't closed (one word answers) or too leading. Clarify a participant's point at the end to ensure you've understood their feedback. Remember that what participants say is often different to what they do. Make a note of any discrepancies but it's not necessary to pick up participants on this.
  • Too much help will only affect the results of the testing and its robustness. Tell participants that if they need help or ask a question you will be able to help them but only at the end of the task. I find that once a participant has demonstrated that they cannot complete the task and stated they would give up, you can provide pointers. This is helpful as you can gather feedback on user’s expectations.
  • Guerilla is similar to ‘discount’ usability testing and makes usability testing cheaper and more accessible to everyone to undertake. Characteristics include, real environments, loose recruitment – often on the spot. Shorter test times and smaller incentive (if any) e.g. Free coffee or voucher. Not as rigorous as traditional testing, not always with representative users.
  • Anyone attending IS usability testing training days here at Edinburgh? If so you may remember our tests at the end of the class.
  • Useful and accurate metaphor for mobile internet use compared to desktop use. Is important to bear in mind when designing mobile web services - has impact on the type of services users are likely to want and find useful.
  • During the UX2 project we created a mobile friendly version of the desktop prototype digital library. Wanted to understand how students searched for information on smaller screens and if their needs changed under different circumstances. Here was can see an example of the test environment – student searching on their phone which is hooked up to the recording equipment. Web cam positioned above phone allowed us to capture what was happening on screen – image provides example of the mobile site.
  • Other mobile testing equipment include ELMO-cam which is quite expensive and commonly found in speciality test laboratories. We wanted to lower-tech and cheaper alternative which could do the job equally as good if not better. We used a ‘sled’ system which has been used and documented by others (see resources on last slide). Using this as guidance we built our own sled with the following materials. For more detailed information please read our blog titled Mobile Usability Testing
  • A close-up of the set-up showing the camera clipped onto the Perspex and the phone sitting in the case. A second web cam in the background was used to capture body language in picture in picture (PiP) format.
  • To conclude – When undertaking mobile testing we considered two things in particular: Should we develop a website or app first? Should we ensure testing is as realistic as possible or test in a controlled environment such as a lab or office room? Research revealed that consensus of prioritising mobile websites are best. Research also revealed that realistic mobile testing does not necessarily reveal additional usability issues. Example os realistic testing includes making participants walk around obstacle course while performing tasks or making users go out while conducting tests i.e. Travel on tube train etc. - As this was a preliminary test which was intended to be first of many, the scope of the project determined that realistic testing would be too time consuming to be worthwhile at this stage. Might have been more useful closer to launch or post-launch.
  • Some resources mentioned directly or indirectly during the presentation. Very worthwhile reading! I have copies of some of the mentioned books if you would like to take a look at them feel free to come up at the end/during lunch.
  • Usability and User Research (UX2 UCD Day)

    1. 1. USABILITY AND USER RESEARCH: CASE STUDIES AND CURRENT PRACTICE Lorraine Paterson, Boon Low NeSC, University of Edinburgh User Centred Design Day 17 May 2011
    2. 2. Agenda <ul><li>Why user research is important? </li></ul><ul><li>What is usability? </li></ul><ul><li>Usefulness and usability </li></ul><ul><li>Methods </li></ul><ul><ul><li>Personas </li></ul></ul><ul><ul><li>Contextual Enquiry (Boon) </li></ul></ul><ul><ul><li>‘ Traditional’ Usability Testing </li></ul></ul><ul><ul><li>Guerilla Testing </li></ul></ul><ul><ul><li>Mobile Usability Testing </li></ul></ul>
    3. 3. “ It’s only a little problem...they’ll get over it.”
    4. 4. <ul><li>If its difficult to use, people leave </li></ul><ul><li>If users get lost they leave </li></ul><ul><li>If users don’t understand how to use a service, they leave. </li></ul><ul><li>If users do not understand the value of using a service, they leave </li></ul><ul><li>If it’s hard to read or doesn’t address their key questions, people leave </li></ul>Why is usability necessary?
    5. 5. Why is usability important ? <ul><li>Websites are full of little usability problems (and sometimes big ones). No website is ever perfect! </li></ul><ul><li>Each problem frustrates users a little more reducing their ‘Reservoir of goodwill’ – Steve Krug, Don’t Make Me Think (2000). </li></ul>
    6. 6. Reservoir of goodwill <ul><li>Every little frustration gradually erodes our goodwill. </li></ul><ul><ul><li>Different people have different size reservoirs. </li></ul></ul><ul><ul><li>Sometimes you have more goodwill than others. </li></ul></ul><ul><ul><li>You can refill it even if you make mistakes. </li></ul></ul><ul><ul><li>Sometimes a single mistake can empty it. </li></ul></ul><ul><li>Therefore every small improvement can have an impact. </li></ul>
    7. 7. Things that increase good will <ul><li>Know the main things people want to do on your site and make them obvious and easy </li></ul><ul><ul><li>Top 3 goals important to the success of the site </li></ul></ul><ul><li>Tell me what I want to know </li></ul><ul><ul><li>Be upfront about things. Explain registration procedure clearly </li></ul></ul><ul><li>Save me steps wherever you can </li></ul><ul><ul><li>Registration, short-cut links </li></ul></ul><ul><li>Put effort into it </li></ul><ul><ul><li>The more effort you put into generating information which is presented clearly and is organised in a way that it can be found quickly will allow users to answer questions themselves </li></ul></ul>
    8. 8. Things that increase good will <ul><li>Anticipate the type of questions are likely to be asked </li></ul><ul><ul><li>FAQ are useful when kept up to date and easy to navigate </li></ul></ul><ul><li>Provide me with creature comforts like printer-friendly pages </li></ul><ul><li>Make it easy to recover from errors </li></ul><ul><ul><li>E.g. Suggestions and advice when no search results returned </li></ul></ul><ul><li>When in doubt, apologise </li></ul><ul><ul><li>Let them know you are aware of the issue </li></ul></ul>
    9. 9. Things that diminish good will <ul><li>Hiding information I want: </li></ul><ul><ul><li>Contact details, shipping details or prices </li></ul></ul><ul><li>Punishing me for not doing things your way: </li></ul><ul><ul><li>Formatting data in forms, forgetting to complete a field, visiting the wrong page or conducting a search with no results </li></ul></ul><ul><li>Asking me for information you don’t really need: </li></ul><ul><ul><li>Personal details, marketing questions </li></ul></ul><ul><li>Putting ‘sizzle’ in my way: </li></ul><ul><ul><li>Long Flash intros, bloated pages, marketing or adverts </li></ul></ul><ul><li>Providing an amateur looking site: </li></ul><ul><ul><li>Sloppy, disorganised and unprofessional </li></ul></ul>
    10. 10. Definition <ul><li>Usability definition from International Organisation for Standardisation (ISO) 9241-11: </li></ul><ul><ul><li>“ The extent to which a product can be used by specified users to achieve specified goals with effectiveness , efficiency and satisfaction in a specified context of use.” </li></ul></ul>
    11. 11. Definition <ul><li>Usability components: </li></ul><ul><ul><li>Learnability : How easy is it for users to accomplish tasks the first time? </li></ul></ul><ul><ul><li>Efficiency : Once users have learned the design, how quickly can they perform tasks? </li></ul></ul><ul><ul><li>Memorability : When users return to the design after a period, how easily can they re-establish proficiency? </li></ul></ul><ul><ul><li>Error prevention : How many errors do users make, how easily can they recover? </li></ul></ul><ul><ul><li>Satisfaction : How pleasant is it to use the design? </li></ul></ul>
    12. 12. Ways to improve usability <ul><li>User testing is the best way to get feedback on your website and will always uncover something you weren’t aware of </li></ul><ul><li>Usability professionals mantra: </li></ul><ul><ul><li>“ Test early and often!” </li></ul></ul><ul><li>Contrary to popular beliefs, </li></ul><ul><ul><li>Usability testing can be done quickly </li></ul></ul><ul><ul><li>Can be done cheaply </li></ul></ul><ul><ul><li>Can be done by anyone </li></ul></ul>
    13. 13. <ul><li>Another key attribute is utility </li></ul><ul><li>Does it do what the users need? </li></ul><ul><li>Is it useful while also being usable ? </li></ul>
    14. 14. Usefulness and Usability <ul><li>Technology Acceptance Model (TAM) </li></ul><ul><ul><li>A person’s intention to adopt an information system is affected by two beliefs: perceived ease of use and perceived usefulness </li></ul></ul><ul><li>However, TAM can only identify that a system is not likely to be accepted by users but not offer feedback on how it can be improved (Dillon & Morris, 1999) </li></ul>Dillon, A. and Morris, M. (1999). Power, Perception and Performance: From Usability Engineering to Technology Acceptance with the P3 Model of User Response. Presentation of the 43rd Annual Conference of the Human Factors and Ergonomics Society, Santa Monica, CA: HFES
    15. 15. Interactive Triptych Framework “ Unless users perceive an information system as being useful at first, its ease of use has no effect in the formation of intention” ~ Szajna 1996 Szanja, B. (1996). Empirical Evaluation of the Revised Technology Acceptance Model. Management Science, Vol. 42(1) Tsakonas, G., Papatheodorou, C. (2006). Analysing and evaluating usefulness and usability in electronic information services. Journal of Information Science, 32(5)
    16. 16. Personas
    17. 17. Personas <ul><li>Personas are fictional person who represents a major user group for your site </li></ul><ul><li>Help to visualise your user when developing user interfaces </li></ul><ul><li>Enables library stakeholders to better judge user goals and needs, and to overcome incorrect system assumptions </li></ul><ul><li>How many personas? </li></ul><ul><ul><li>No definitive answer but if you have a lot it’s likely that some will overlap and can be combined </li></ul></ul><ul><ul><li>Don’t always need one persona per demographic group. It’s the behaviour and attitudes that are important, these might be very similar or the same for example librarians and academic staff </li></ul></ul><ul><ul><li>Normally 4-6 is a good indicator </li></ul></ul>
    18. 18. AquaBrowserUX Personas <ul><li>Created a set of library personas for JISC AquaBrowserUX </li></ul><ul><li>Planned to create qualitative personas with quantitative validation </li></ul><ul><li>Only able to create qualitative personas in end </li></ul>Image: Mulder, S. & Yaar, Z. (2007) The User Is Always Right, A practical Guide to Creating and Using Personas for the Web
    19. 19. AquaBrowserUX Personas <ul><ul><li>Conduct qualitative research to reveal insights: interviews with 17 participants </li></ul></ul><ul><ul><li>Summarise participants using coloured post-its: </li></ul></ul><ul><ul><li>Goals </li></ul></ul><ul><ul><li>Information seeking behaviour </li></ul></ul><ul><ul><li>How they relate to library services </li></ul></ul><ul><ul><li>Skills, abilities and interests </li></ul></ul>
    20. 20. AquaBrowserUX Personas <ul><li>Two-by-two comparisons: </li></ul><ul><ul><li>Selecting pairs of randomly participants to identify distinctions and plot them on a distinction scale </li></ul></ul><ul><ul><li>Keep selecting two participants until all have been compared </li></ul></ul>
    21. 21. Positive attitude To library Preference for Digital resources (e-books) Uses digital Library a lot Uses physical Library a lot Use of Classic Use of Internal resources Positive att to AB Positive att to Classic Simple search Negative attitude To library Uses digital Library little Uses physical Library little Use of External resources Use of Aquabrowser Preference for physical Resources (books) Negative att to AB Negative att to Classic Boolean search Plot each participant on every distinction scale
    22. 22. AquaBrowserUX Personas Identify segments, looking for groups with similar traits
    23. 23. AquaBrowserUX Personas <ul><li>Write a persona for each segment you have </li></ul><ul><li>We created 3 student personas and 1 library persona </li></ul><ul><li>Eve the e-book reader: “I like to find excerpts of books online which sometimes can be enough. It saves me from having to buy or borrow the book.” </li></ul><ul><li>Sandra the search specialist: “In a quick-fire environment like ours we need answers quickly” </li></ul><ul><li>Pete the progressive browser: “Aquabrowser and Classic, it’s like night and day” </li></ul><ul><li>Baadal the search butterfly: “Classic is simple and direct but Aquabrowser’s innovative way of browsing is also good for getting inspiration.” </li></ul>
    24. 24. AquaBrowserUX Personas <ul><li>Most difficult step is getting from two-by-two comparisons to identifying segments </li></ul><ul><li>You may find you create more segments than eventually use </li></ul><ul><li>Once you begin writing the persona you may realise there are two personas who are very similar or not different enough to justify two individual personas </li></ul><ul><li>Time consuming process, 17 hours of interviews followed by a week of analysis and persona writing </li></ul><ul><li>Worth thinking about individual characteristics e.g. Goals, interests, skills, information seeking behaviour, attitude to library, before you begin interviews so you are thinking about this throughout </li></ul>
    25. 25. Contextual Enquiry
    26. 26. Usability Testing <ul><li>Usability components: </li></ul><ul><ul><li>Learnability : How easy is it for users to accomplish tasks the first time? </li></ul></ul><ul><ul><li>Efficiency : Once users have learned the design, how quickly can they perform tasks? </li></ul></ul><ul><ul><li>Memorability : When users return to the design after a period, how easily can they re-establish proficiency? </li></ul></ul><ul><ul><li>Error prevention : How many errors do users make, how easily can they recover? </li></ul></ul><ul><ul><li>Satisfaction : How pleasant is it to use the design? </li></ul></ul>
    27. 27. Usability Testing <ul><li>Recruit representative users of your service </li></ul><ul><li>Ask them to perform representative tasks using the service interface </li></ul><ul><li>Observe what the users do, where they succeed, and where they have difficulties with the user interface. </li></ul>
    28. 28. Characteristics <ul><li>Ask participants to verbalise their thoughts by ‘thinking aloud’ </li></ul><ul><li>One-to-one tests, unlike focus groups </li></ul><ul><li>Normally small numbers of participants (5 considered adequate in most situations*) </li></ul><ul><li>Most effective when used iteratively throughout development </li></ul><ul><li>Provides information on how people use an interface </li></ul>*Research by Jacob Nielsen found that 5 participants found 85% of usability issues:
    29. 29. Representative Task Scenarios <ul><li>Task scenarios must be realistic to the user </li></ul><ul><li>Written in the participant’s own words, unambiguous and asks the participant to explore a particular area of interest </li></ul><ul><li>Types of task scenarios: </li></ul><ul><ul><li>Scavenger hunt : tasks with one clear, ideal answer. </li></ul></ul><ul><ul><li>The Reverse Scavenger hunt : show users the answer and ask them to find and purchase it. E.g. Image of clothing to source online. </li></ul></ul><ul><ul><li>Self-generated tasks : using interview questions beforehand to help participants identify their requirements </li></ul></ul><ul><ul><li>Part self-generated : ask users to bring data with them to the session and then use it to construct an ad-hoc scenario. </li></ul></ul><ul><ul><li>‘ Skin in the game’ tasks : gives users real cash to purchase something as part of the task making it as realistic as possible </li></ul></ul><ul><ul><li>Troubleshooting tasks : ask the user to solve a problem </li></ul></ul>
    30. 30. Scavenger Hunt “ As part of your coursework your lecturer has asked you to read a recent presentation on the developments in fusion. Using the prototype can you find a suitable presentation published in the last 2 years?” Subject matter, where possible relevant to user’s knowledge base Task restriction, forces the user to think about narrowing their search. Also task completion indicator Making the scenario realistic so the user can relate to it
    31. 31. Self Generated Task <ul><li>Interview-based tasks are often more suitable for Search Interfaces than pre-defined tasks </li></ul><ul><li>Pre-defined tasks wont feel real to participant and this may affect how they search for information </li></ul><ul><li>Many search interfaces such as digital library catalogues don’t have one or two specific tasks that are more important </li></ul><ul><li>Here you create a task with the input of the participant and agree what task success would be beforehand </li></ul><ul><li>Provides a valuable insight into information seeking behaviour </li></ul>AquaBrowserUX blog post: Realism In Testing Search Interfaces by David Hamill
    32. 32. Self Generated Task <ul><li>‘ What do you study at the University?’ </li></ul><ul><li>‘ Describe the last time you had to search for information as part of your course? What were you looking for?’ </li></ul><ul><li>‘ Let’s try and see if we can search for this information today using.... Can you use the web to show me how you would search for this information?’ </li></ul><ul><li>Example: Task scenario created based on an essay to write on The National identity in the work of Robert Louis Stevenson . </li></ul><ul><ul><li>Led to searches on the architecture in Jekyll and Hyde, Edinburgh's architecture in Scottish literature, opinion on architecture in Stevenson's work and opinion on architecture in national identity </li></ul></ul>AquaBrowserUX blog post: Realism In Testing Search Interfaces by David Hamill
    33. 33. Self Generated Task <ul><li>Downside to such tasks is that it’s difficult to report useful measurements </li></ul><ul><ul><li>Each participant undertakes a different task </li></ul></ul><ul><ul><li>Inappropriate method to provide an overall measure of usability </li></ul></ul><ul><ul><li>Perhaps useful in conjunction with other types of task scenarios i.e. Scavenger hunt tasks. </li></ul></ul>
    34. 34. AquaBrowserUX Testing <ul><li>Recruited students based on established Personas </li></ul><ul><li>Mixture of undergraduate and postgraduates </li></ul><ul><li>12 participants, £15 book voucher incentive </li></ul><ul><li>1 hour sessions carried out over 3 days with external freelance consultant </li></ul><ul><li>Post-test interview to sum up experience </li></ul><ul><li>Word choice questionnaire </li></ul><ul><li>Ran pilot with member of staff, also tested Morae set-up </li></ul><ul><li>Video highlights from AquaBrowserUX tests: </li></ul>
    35. 35. Facilitating user research <ul><li>Do </li></ul><ul><li>Put participants at ease </li></ul><ul><li>Put participants at ease (avoid the word ‘test’) </li></ul><ul><li>Emphasise you’re not testing them, just the design </li></ul><ul><li>Encourage them to think aloud </li></ul><ul><li>Allow them to stop and leave at any time </li></ul><ul><li>Make notes, even when you are recording </li></ul><ul><li>Most important, learn to listen and not lead participants as this can influence participant's answers </li></ul>
    36. 36. Facilitating user research <ul><li>Don’t </li></ul><ul><li>Help participants with hints or leading questions </li></ul><ul><li>Justify the design or content to them: </li></ul><ul><ul><li>Stay objective and encourage constructive criticism </li></ul></ul><ul><li>Be afraid to improvise: </li></ul><ul><ul><li>Have some questions ready to get started but don't stick strictly to a script, everyone’s experience will differ </li></ul></ul>
    37. 37. Informal (Guerilla) Testing <ul><li>Quick and easy to perform </li></ul><ul><li>Conducted in a ‘real’ environment e.g. Library or cafe </li></ul><ul><li>Tests are much shorter (15 mins max) </li></ul><ul><li>Useful when testing a design or answer a conflict </li></ul><ul><li>Relatively inexpensive </li></ul><ul><ul><li>shorter sessions mean you can test half a dozen participants within an afternoon. </li></ul></ul><ul><li>However, bias can be an issue if you are closely involved in project </li></ul><ul><li>Less statistical rigour on quantitative results, appropriate as a user-centred design method </li></ul><ul><li>Difficult to recruit representative users </li></ul>
    38. 38. UX2 Guerilla Testing <ul><li>Piggybacked onto workshop run for university staff twice every 6 months which teaches usability testing methods </li></ul><ul><li>Asked attendees to participate for 10-15 mins at end of workshop </li></ul><ul><li>Tested two different interfaces using Blacklight technology </li></ul><ul><li>Aim: to get initial feedback on both interfaces and identify successful aspects of each for future design changes </li></ul><ul><li>Recruited 7 participants in total over 2 workshops (2 phases) </li></ul><ul><li>Alternated design exposure to reduce familiarity factor </li></ul><ul><li>Able to make changes from findings in phase 1 to test in phase 2 </li></ul>
    39. 39. Scuba Diving Vs Snorkelling Desktop internet is like scuba diving, where the search can be immersive and invites exploration and discovery. Mobile internet use is closer to snorkelling, where shallow dipping in and dipping out of content for quick checking of key content is desired. ~ Hinman et al., 2008 Hinman, R., Spasojevic, M., & Isomursu, P. (2008). They call it surfing for a reason: identifying mobile internet needs through pc internet deprivation. In CHI '08 extended abstracts on Human factors in computing systems (pp. 2195-2208). Florence, Italy: ACM. doi:10.1145/1358628.1358652
    40. 40. UX2 Mobile Usability Testing <ul><li>Testing UX2 prototype digital library </li></ul><ul><li>Aimed at understanding how participants find searching for items on a smaller screen </li></ul><ul><li>How easy is the website to navigate on a mobile? </li></ul><ul><li>How useful are the features/services provided? </li></ul>
    41. 41. UX2 Mobile Usability Testing <ul><li>Various methods for testing mobile interfaces </li></ul><ul><li>UX2 used the ‘sled’ which is tried and tested with other researchers (see resources slide). </li></ul><ul><li>Easy ‘do-it-yourself’ method and relatively cheap </li></ul><ul><li>Requirements: </li></ul><ul><ul><li>Perspex (30cm x 11cm) </li></ul></ul><ul><ul><li>Web cam (x2 where possible) </li></ul></ul><ul><ul><li>Carry case for each type of phone being tested (E.g. iPhone 3G and 4) </li></ul></ul><ul><ul><li>Velcro to attach phone case to Perspex </li></ul></ul><ul><ul><li>Computer with adequate recording software (Morae) </li></ul></ul><ul><li>Total cost (excluding Morae software): approx £80 </li></ul>
    42. 43. UX2 Mobile Usability Testing <ul><li>Mobile website vs. Native applications (Apps) </li></ul><ul><ul><li>General consensus that mobile websites should be prioritised before apps for a number of reasons: </li></ul></ul><ul><ul><ul><li>Fewer compatibility issues, don’t have to design for individual mobile platforms or handsets </li></ul></ul></ul><ul><ul><ul><li>Users don’t have to download anything </li></ul></ul></ul><ul><ul><ul><li>Don’t need approval from app store like Apple to publish </li></ul></ul></ul><ul><li>Realistic mobile testing vs. Laboratory testing </li></ul><ul><ul><li>Review of existing literature revealed that no additional usability issues are revealed when testing in realistic conditions ( ) </li></ul></ul>
    43. 44. Further Reading & Resources <ul><li>JISC Research Project - Usability Foundation Study and Investigation of Usability in JISC Services (2003): </li></ul><ul><li>Book - Rocket Surgery Made Easy: The Do-It Yourself Guide to Finding and Fixing Usability Problems, Steve Krug, New Riders, 2010 </li></ul><ul><li>Book - Don’t Make Me Think (2nd Edition), Steve Krug, New Riders 2006 </li></ul><ul><li>Book - User-Centred Library Websites: Usability Evaluation Methods, Carole A. George, Chandos Publishing, 2008 </li></ul><ul><li>Blog - Usability Ed: Website usability and content management in higher education, by Neil Allison </li></ul><ul><li>Blog – 90 Percent of Everything by Harry Brignull. Test sled blog: </li></ul>
    44. 45. Thank You! <ul><li>UX2.0 </li></ul><ul><li>Website: </li></ul><ul><li>Wiki: </li></ul><ul><li>Twitter ID: @ux2 </li></ul><ul><li>Lorraine Paterson </li></ul><ul><li>Email: [email_address] </li></ul><ul><li>Blog: </li></ul><ul><li>Twitter ID: @lorraine_p </li></ul>