Download to read offline
Uploaded byAllison Bloodworth
Selling userneedsassessment 7-30-07_full
This document summarizes a presentation on conducting user needs assessments. It discusses why user-centered design is important, how to sell needs assessments to project sponsors, and techniques for gathering user information, including surveys, interviews, observations and contextual inquiries. Case studies are provided to illustrate how these techniques were applied to understand users of a webcast study tool. The document concludes with recommendations for analyzing findings and convincing teams to use results.
More Related Content
Similar to Selling userneedsassessment 7-30-07_full
Selling userneedsassessment 7-30-07_full
- 1. Why Your SponsorsKnow Too Much and What to Do About It: Selling and Performing User Needs Assessment Allison Bloodworth, Educational Technology Services Ian Crew, Information Services and Technology Daphne Ogle, Educational Technology Services UC Berkeley © 2006-2007 Regents of the University of California
- 2. Agenda I. Why user-centereddesign and user needs assessment is important II. Selling user needs assessment III. Gathering good information about your users IV. Understanding the information you gathered
- 3. Adapted from UsabilityProfessionals’ Association website, http://www.upassoc.org/usability_resources/about_usability/definitions_of_usability.html I. Why User-Centered Design? • Increased customer satisfaction • Increased user productivity, efficiency, and accuracy • Increased service/site usage and adoption • Decreased support and training costs • Reduced development time and costs – Create only the features users need • Reduced maintenance costs – Do it right the first time
- 4. User Centered Design ProcessOverview 3. Specify user and organizational requirements 2. Specify the context of use 4. Produce design solutions 1. Plan the user centered design process 5. Evaluate designs against user requirements Complete Adapted from ISO 13407: http://www.ucc.ie/hfrg/emmus/methods/iso.html
- 5. How does needs assessmenthelp? • Focuses on understanding: – Who are the users? – What are their goals? • Goals drive a person’s actions • Tasks are things a person does in order to accomplish his goals – What are their pain points? • Uses a variety of observational techniques to gather this data • This understanding drives design
- 6. II. Selling userneeds assessment to project sponsors • This is hard to do • Think like a salesperson: – You need to get them to buy (and buy in) to user needs assessment – Use the sponsor’s terminology and background to shape your arguments – Quantify where possible (e.g. reduced training & support or development costs) – Discuss user’s expectations (Google, Amazon) – Start small and sell up
- 7. Talking your sponsor’s language •UCD processes include activities sponsors may call: – Business Analysis – Requirements Definition and Verification – Information Architecture – Web Design and/or Development – User Testing – Proactive User Support – Other ideas?
- 8. Why sponsors aren’tusers • Sponsors have: – great familiarity with the existing solution – great familiarity with underlying business processes – too much experience receiving and responding to requests for assistance, complaints, or suggestions from more vocal users • Sponsors know (or think they know): – all about the service area that needs improvement – “how it's always been done” – what “can't be changed” – constituencies that use or are impacted by the service – what financial and personnel resources are available to do the work
- 9. Set yourself upfor success • A good project to choose is/has: – One you can get involved with early – Real users you can talk to – A receptive project team – A scope that is the right size for you/your team to handle – A short project allowing you to show quick wins – Improving an existing product • Learn how to say no
- 10. III. Gathering good informationabout your users • Case study: Webcast Study Tool • Selecting representative users • User Needs Assessment techniques
- 11. Case study: Webcaststudy tool • Many Berkeley lectures are webcast • Allow students to apply known effective study techniques to opaque media • Surveyed students & faculty • Interviewed stakeholders • Interviewed & observed students: – in traditional study environments – using webcast to study
- 12. Selecting representative users • Determinewhich characteristics define your target user base • Ensure that each characteristic is represented in (approximately) the right proportion – Don’t test just one end of the spectrum (e.g. vocal users who are calling customer support) • Can use a profile matrix to keep track of all the characteristics that matter
- 13. Sample profile matrix Never had webcast class Has watched webcasts ofclass Studies with class webcasts Has computer at home Works in computer lab Undergrad Student 2 1 1 1, 2 1 Graduate Student 3 3 4 3 3,4 Virtual Student 5,6 5 5,6
- 14. Needs assessment techniques 1. Overalltips 2. Surveys 3. Focus groups 4. Interviews 5. Observation 6. Contextual Inquiries
- 15. 1. User needsassessment tips • Be a good listener • Remain neutral: don’t react • Focus on goals first, tasks second • Don’t limit yourself to a fixed set of questions • Encourage story telling • Distance yourself from the product • Avoid making the user a designer • Categorize notes = easier analysis • Analyze your notes within 48 hours • Ideally should be performed in teams
- 16. From Jakob Nielsen,“Field Studies Done Right: Fast & Observational,” http://www.useit.com/alertbox/20020120.html 1. User needs assessment tips • Don’t use questions that can be answered with “yes” or “no” • Don’t ask leading questions • Don’t use jargon • Don’t draw attention to specific issues that you care about
- 17. Keeping yourself ontrack • It’s not always easy to determine what is relevant • Your problem statement likely describes your project’s focus – A starting perspective, lens, or viewpoint – Is present whether articulated or not • Focus structure document – Clusters of questions or pieces of information you are looking to explore, grouped categorically
- 18. 2. Surveys • Strengths –Getting general feelings about an existing product – Acquiring demographic data on target market – Understanding what users “think” is important (marketing data) • Weaknesses – Survey design is very difficult to get right – Constrained responses – Typically limited time & focus for response – Limited or no ability to follow-up for clarification – Relies on user to self-report accurately • Resource for designing surveys: http://www.ssri.psu.edu/survey/educ.htm
- 19. 3. Focus Groups •Strengths – Can talk to many people at once – Allows users to feed on each others’ ideas – Understanding users' attitudes, beliefs, desires – Getting users' reactions to ideas or to prototypes • Weaknesses – Understanding what people REALLY do with a product – Understanding what features people will really use in a new product – “Group think” drives people toward consensus with the loudest opinion – Relies on user to self-report accurately
- 20. 4. Interviews • Strengths –Understanding how users understand their work – Analyzing goals of work – Ability to follow-up and clarify – Builds relationships • Weaknesses – Relies on user to self-report accurately – Experts often have an inability to describe what has become subconscious (unconscious competence) – More time intensive for facilitator
- 21. “Users are perfectly capableof expressing their latent needs. They just can’t do it verbally. That’s why we do ethnography and empathic research!” -Rich Sheridan, Menlo Innovations 5. Observation • Strengths – Allows you to watch what people do rather than rely what they say (self-report) – More likely to discover unmet user needs – Truly understanding how users get their work done in context – Observing subtleties of work (e.g. post-it notes, cheat sheets, interruptions) – Overcomes experts’ inability to describe what has become subconscious • Weaknesses – Time commitment – Difficult to be “a fly on the wall” – Relies on observers’ interpretation – Hard to know what to pay attention to
- 22. Interview (Process influenced more by designers) Observation (Process influencedmore by end-users) Contextual Inquiry (Process influenced by both designers and end-users) 6. Contextual Inquiries • Combines strengths of interview and observation • Interview in the context of where the work happens • “Show and tell” • Find “pauses” to ask questions; Don’t interrupt their thought processes
- 23. Case study: surveys toldus • Bookmarking and video download are the features that are of greatest interest across the board • Searchable captions, chaptering, and Powerpoint sync are the features most highly rated by webcast.berkeley students. • Annotation is less popular than bookmarking. • Interest in knowledge sharing tools is relatively low. • The general webcast.berkeley audience is the only one highly interested in being notified about posting of video.
- 24. Case study: interviews& observations told us • Greatest pain points are finding specific spots in webcast lectures • Powerpoint slides are often-used reference point for finding spot • There’s administrative overhead in marking down time code for getting to or returning to specific points • Students replay specific segments to aid in understanding, creating study sheets, etc. • Students jot down notes while watching • Students look at more than one webcast in a sitting
- 25. IV. Understanding the informationyou gathered • Who are users and how do they accomplish their goals now? – Personas – Task analysis – Activity Diagrams • What do users need? – Scenarios
- 26. Personas • Research-based userarchetypes representing needs of a set of constituents – Based on patterns from user research • Allow designers & developers to put themselves into the shoes of “real” users • Can help build consensus & commitment to the design • Should be specific, real & memorable – Pictures, posters – Include details about their life—humanize them • Keep us from designing for: – The elastic user – The mirror persona (ourselves) – Design edge cases
- 27. Academic goals: • Getinto Med school • Feel confident walking into exams • Be as efficient as possible Personal goals: • Stay healthy • Have time to spend with friends Example Persona - Webcast Study Project Lisa Ng: Conscientious Student • 2nd year undergraduate • Planning to go to med school, so doesn’t feel she can take risks with classes • Rarely uses webcast as a replacement for class • Relies on computers in lab on campus • Use of webcast is primarily for studying for exams • Good study skills: When studying with text, uses highlighters to mark parts she’ll want to be able to find again & to identify key points or points of confusion. • When doesn’t understand what happened in class, uses webcast to review • Refers to PowerPoint slides when studying
- 28. Modeling what usersdo now • Task Analysis – Decomposes tasks in order to understand procedures better & provide support for these tasks in the interface • Helps ensure necessary features aren’t overlooked – Define the task and the goal of the task and then list the steps involved – Can rate tasks on frequency, importance, difficulty • Tells you what functionality is important • Can help you combine similar personas • Help you choose which tasks to include or emphasize in scenarios • Activity Diagrams – Modeling existing user behavior and interaction with their existing system
- 29. Task analysis matrix Persona/ Task KatieRichardson Harold Jackson Sally McNeil Nina Sanchez Create a Calendar Frequency Importance Frequency Importance Frequency Importance Frequency Importance Design calendar appearance LOW LOW MEDIUM HIGH MEDIUM HIGH LOW MEDIUM Set up calendar on website LOW LOW MEDIUM HIGH MEDIUM HIGH LOW MEDIUM Manage Events Frequency Importance Frequency Importance Frequency Importance Frequency Importance Add Event LOW HIGH HIGH HIGH HIGH HIGH MEDIUM HIGH Edit Event LOW MEDIUM MEDIUM HIGH HIGH HIGH MEDIUM HIGH Delete Event LOW MEDIUM MEDIUM HIGH HIGH HIGH MEDIUM HIGH Approve Events LOW MEDIUM HIGH HIGH HIGH HIGH LOW LOW
- 30.
- 31. Activity diagrams -cont’d
- 32. Scenarios • A designtechnique used to envision future use of a system – Focusing on how users can achieve their goals – Helps designers & developers understand how system will really be used • A story about user interacting with the system • Categorize scenarios as Daily, Necessary, and Edge Use • Can be used for usability testing • Iterate on scenarios
- 33. Case study contextscenario: Studying for exam • Lisa has an exam coming up and wants to create a study sheet she can use for the next week while on the elliptical @ the gym. • She gets out notepaper, her textbook, and her binder with PPT “notes” pages and gets comfy on the couch. • She starts reviewing the powerpoints and notes from the lectures after the last exam. As she does this, she’s making notes (summarizing important topics) on her notepaper. (This will become her study sheet). • As she’s making her way through the slides she decides it would be useful to hear the instructor’s explanation of DNA replication again. • She goes to … a point in the webcast where that ppt slide is, and listens. One sentence he says seems to encapsulate the concept for her, so she tries to get it down word for word. Since her prof talks fast and does not always use lay terms, she relistens several times. • After she feels like she understands, she adds some notes in the study sheet. • She sees that there were a number of segments that she’d highlighted
- 34. Convincing project teamto use needs assessment results • Include project team (sponsors, developers, designers) in needs assessment process: – Have team members attend interviews, observations, user tests – Share interesting results with team while needs assessment or usability tests are ongoing – Share and get feedback on tools, questions, and suggestions you’re developing – Offer introductory training in UCD and needs assessment techniques
- 35. Recommended books • TheInmates are Running the Asylum and About Face 3.0 – Alan Cooper • The Design of Everyday Things and Emotional Design – Don Norman • Don’t Make Me Think – Steve Krug • Usability Engineering – Jakob Nielsen • User Interface Task Analysis - Joann T. Hackos and Janice Redish • Designing for Interaction - Dan Saffer • Other recommendations?
- 36. Recommended websites • URLsin Handout • User Interface Engineering Virtual Seminars (http://www.uie.com/events/virtual_seminars/) • Usability Professionals Association (www.upassoc.org/) • http://usability.gov/ • http://www.useit.com • http://www.usabilityfirst.com/ • http://www.usableweb.com/ • http://usabilitynet.org/ • http://www.stcsig.org/usability/ • ACM SIGCHI (http://sigchi.org/) • UC Berkeley’s Technology Program Office Resources (http://tinyurl.com/2cmx88)
- 37. Thank you! • Questions… •Contact Us – Allison Bloodworth – abloodworth@berkeley.edu – Ian Crew – icrew@berkeley.edu – Daphne Ogle – daphne@media.berkeley.edu
Editor's Notes
- #2 IAN © 2006-2007 Regents of the University of California Permission to use/modify can be obtained by contacting Ian Crew (icrew@berkeley.edu), Allison Bloodworth (abloodworth@berkeley.edu), or Daphne Ogle (daphne@media.berkeley.edu)
- #3 IAN I and II-Ian III-Allison IV-Daphne
- #4 IAN Fewer lawsuits--ADA, etc. Better design=better maintainability POLITICS: Buy-in ･Start with a good design, know what your users need, don't waste time developing unnecessary features/fixing problems in the future ･if a system is not usable or seems to be designed without regard for the needs of the users, users may: ･be less productive ･be less accurate (in terms of data entered) ･be unhappy that they must use inefficient tools that make it more difficult for them to do their jobs
- #5 HIDDEN Focuses on understanding: Who are the users? What are their goals? Goals drive a person’s actions Tasks are things a person does in order to accomplish his goals What are their pain points? This understanding drives design
- #6 IAN Focusing on parts 2 and 3 in this talk
- #7 IAN Goals: Are what the user wants to do, but not how the user achieves them Help you not to make any assumptions about the system interface Can be used to compare different interface design alternatives in a fair way Can be personal, practical, or false (don't focus on false goals!) tasks: Describe the steps necessary to achieve the goals Can vary with the available technology Are broken down into steps for task analysis, and are recombined into sequence of steps for scenario development The main emphasis on our process is to understand who our users are, what there goals are and what their current pain points are. So, who are we building the system for? It’s likely not the designers, developers, testers, support folks or anyone else on the development team. I’ll got into a little more depth in minute about how we learn about our users. We want user goals to drive our system definition…hence goal-directed design. Goals help us think out of the box and truly figure out how we can make users lives easier with technology. For instance, I have a daily goal of getting to the office. My task used to be to drive in traffic to meet that goal. Now most the time I ride my bike to meet my goal. In the future, maybe I’ll be able to simply press a button to get the office. So the goal stays the same but the tasks change with context, with technological capabilities, with innovation… We also want to focus on understanding where users feel the most pain with their current processes? Can technology take on some of the burden? Rather than replicate exactly what they do now in the new system, we need to figure out how we can make things better for them. All this is meant to help us design systems that truly make our users lives easier
- #8 IAN This is admittedly a difficult challenge--we’re still learning how best to do this, and we can’t give you a magic wand, but we can give you a few good tips Think like a salesperson--this is a sales/marketing process--UCD does cost real money, so you need to find ways to sell it in their language Start with a small project “to show concrete results“ Or use a few techniques on a project to get some user input Quantify where possible: Increased productivity Increased usage and adoption Decreased support and training costs Reduced development time and costs Reduced maintenance costs Increased customer satisfaction Adapted from Usability Professionals’ Association website, http://www.upassoc.org/usability_resources/about_usability/definitions_of_usability.html
- #9 IAN Proactive user support instead of reactive user support – solving problems for users before they happen These are our ideas, be thinking as we talk about these, will ask you for ideas
- #10 IAN This knowledge is very useful, and is something you should certainly pay attention to as you do your work, but this all distinguishes the sponsor from a real user and prevents them from being able to accurately represent what the user needs
- #11 IAN Start small! With a small set of users Non-disruptive technology Under the radar (not politically loaded, non-mission-critical) Learn how to say no Choose a few projects you can do really well rather than spreading the team too thin and doing nothing well. Doesn’t have to have all of these things, these are just some suggestions based on our experience. If you can get a project with 3 or 4 of these criteria, you’re on the right track.
- #12 DAPHNE Next we’ll talk about how to gather good information about users. Goal to understand what they do now so you make sure a new system can support their needs Using case study about project done in ETS to demonstrate the activities we describe. Allison will talk briefly about selecting users to study And then we’ll discuss some user needs assessment techniques that we’ve found useful
- #13 DAPHNE
- #14 ALLISON
- #16 ALLISON
- #17 ALLISON Distance: It's often best not to describe why you have a vested interest in the project so the user feels comfortable being frank with you. Interview where the interaction happens Avoid discussions of technology
- #18 ALLISON
- #19 ALLISON While it’s important to explore different topics brought up by the users, it is important make sure you don’t end up with lots of irrelevant data In interview situations, you need to know when to stop users from talking about things they love, but are not pertinent to the your project – this is a judgment call because you may not know for sure whether or not it is relevant, but if your time is limited you likely only have enough time to explore selected lines of questioning/focus In an observation, you need to know what to look for, what behaviors will inform what you are researching Note-taking short-cuts/abbreviations may also help you organize your thoughts OLD Getting information that isn’t relevant Problem statements Who are the users
- #20 ALLISON Have you ever been asked to take a survey and can’t figure out where you fit on the given responses/the response you want to give isn’t a choice?
- #21 ALLISON a traditional market research technique Need a skilled moderator to ensure everyone participates OLD People have trouble accessing what they do and how Good for gauging initial reaction to form of a product, or getting feedback on visual appearance Industrial design
- #22 ALLISON
- #23 HIDDEN May start out with structures and comprehensive set of questions Good goal is to get to where you are using a “guide” of information you’d like to know about rather than a fixed set of questions. Will make it seem more conversational. “Think of yourself as a therapist” and draw the user out without imposing your own ideas or preconceptions. Ask users: What are their overall goals? - Goals remain constant even when the technology changes What tasks are they performing? How often is each task performed? How easy is it to perform each task using the current system? Ask yourself: How critical is it that the system support each task? – Your job is to determine whether the tasks they are performing are really necessary or can be done in a different way to achieve their goals How can the current system be improved? (If possible, observe the user using this design.) Tasks vs. Goals: -If a stated goal involves a particular technology, it’s probably not the real goal--dig deeper Phases: -May not be able to do all phases in a project--that’s OK. -Is also possible to combine first two phases in some cases.
- #24 HIDDEN
- #25 HIDDEN 6. It's often best not to describe why you have a vested interest in the project so the user feels comfortable being frank with you.
- #26 DAPHNE Observations are popular technique Don’t have to rely on users to remember and be able to tell you the details and subtleties of their work Post-it notes and cheat sheets give us clues about where the system could support them better Research shows as people become experts much of their work becomes automatic -- it’s done at a subconscious level. Observations allows you to overcome this Talk about weaknesses One way to overcome interpretation is to ask questions at the end
- #27 DAPHNE Cross between interviews and observation and combines strengths of both With contextual inquiries you still go the user and interview them in their normal context Idea is to interview users in the context of their work and ask them to show you what they describe along the way Tip: Important to look for natural pauses in their answers and as they show their work so you don’t interrupt their natural flow Talk about contuim
- #28 DAPHNE More surfacy, high level information What they are interested in and what’s popular Talk to bullets As mentioned they are good for helping you focus further studies
- #29 DAPHNE Don’t expect you read the entire slide, important points are in bold. Interviews & observations gave us more detailed information. Talk through the bullets
- #30 ALLISON personas and scenarios can (from UIE seminar): 1) bring focus 2) common language [Daphne’s idea] 3) build empathy 4) encourage consensus 5) create efficiency Common language is important--users may know a great deal about their field, but not about computers and therefore may use different terms than you would. Figuring out what they really mean is critical. Ian’s department is currently working on a project to talk to Arts and Humanities scholars and those folks often fall into this category. Getting the vocabulary clear and normalized across users is critical. Assuming all this work will lead to a new system design, it’s worth mentioning, the importance of sharing out what you learn along the way before any design even happens. If you don’t gain common ground around who the users are and what they need, by the time you get to the design stage there is tends to be an assumption that the disagreement is about the specific implementation when it’s more likely about one of the previous two. “Bring the rest of the team along” in your understanding Create models to help understand and analyze the data
- #31 ALLISON Personas can: Determine what a product should do an how it should behave Communicate with stakeholders, developers & other designers Measure the design’s effectiveness Contribute to other product-related efforts such as marketing & sales plans Bullseye model of personas HELPS AVOID THE ELASTIC USER - “the user in the developer’s head often changes to match what can be done, not what the user really needs”--WE’RE ALL VULNERABLE TO THIS Limit number personas--one per important user category Don’t use real people Think about the target audience for your personas Developers will find it useful to know different things about a persona than marketing people would, but both can find personas helpful
- #32 ALLISON Flickr Creative Commons is a good resource for pictures Role-playing - interview a persona Post short versions in the designers’/developers’ workspace & keep the longer descriptions handy (e.g. on a website).
- #33 ALLISON Tasks can later be recombined as part of scenario development DON’T READ Definition - a set of methods for decomposing people's tasks in order to understand the procedures better and to help provide computer support for those tasks. The basic approach is to define the task and the goal of the task and then to list the steps involved. The level of detail in decomposing the steps is determined by how the analysis is going to be used. Task analyses are useful for making time predictions for how long a task will take and for spotting potential errors (steps in the process which are extremely difficult or confusing). Task analyses are also good for spotting areas in a user interface that may have been overlooked or oversimplified. A task analysis is a fundamental part of a cognitive model of user performance, such as GOMS. From http://www.usabilityfirst.com/glossary/term_294.txl
- #34 ALLISON Different users perform tasks with differing levels of frequency & importance May tell you about how to expose functionality in the interface (allowing functional minimalism) Can also create weighted matrix using numbers (0-3) multiplied by the importance of each persona if you need to prioritize features in order to choose which ones to implement
- #35 DAPHNE As Allison mentioned activity diagrams can be a good way to think through what you learn from users The idea is to diagram the work you saw and/or heard about it Again, this from webcast project The highlighted area shows how cumbersome the activity of getting to a particular point in the webcast based on a certain point in time in lecture they want to get back to. This led us to build in automatic time adjustment from the lecture time they wrote down or remembered to right time in the video
- #36 DAPHNE 2nd half of the same diagram Visualizes parallel work Our design allowed them to have all of this at once with the ppts synched to the video and several ways to help them add notes like highlighting parts of the video, adding bookmarks and annotations, and in 1 click making an assignment at segment to come back to.
- #37 DAPHNE Writing scenarios technique to start thinking about how the new system needs to support what you learned about users and their needs. Scenario is a hypothetical chain of events through activities that meet users goals So, if the user’s goal is to give students feedback on their latest assignement, the scenario describes the most effective and efficent way they can do that. Another helpful technique is to categorize scenarios Scenarios can be used throughout the project. When doing user testing you can ask users to walk through a similar scenario to see how well the new system supports them. Another tip is to iterate on scenarios. Next I’ll show you an early scenario.
- #38 HIDDEN
- #39 DAPHNE One of the scenarios from the webcast project. Don’t expect for you read all this, I’ve highlighted some important aspects in the scenario that I’ll talk about in a minute. This is early in the project. At this point we like to describe the work without any implementation details so the focus on is what kind of information the user needs along with the natural flow of the work. By not constraining yourself with how the system will support users at this point, it allows you to think out of the box. Walk through what we learned.
- #40 DAPHNE And lastly, we wanted to briefly touch on the importance of getting buy-in from the rest of the team Besides convincing sponsors to let you do user needs assessment in the first place, you may need to convince your team to use the results. I find this is most often the case when it comes to making decisions about what features are in and out and how much time to spend giving user a particular way to accomplish a task -- even though it might require additional implementation time. “Bringing the team along with you” and gaining buy-in along the way while you are doing the research and creating models will help ensure they respect and understand how the results drive your system design. Some ways to do this are:
- #41 DAPHNE
- #42 DAPHNE TPO site for all the links above, as well as many other resources Accessibility is a related topic; more resources about that on TPO site too