Desirability Testing: Analyzing Emotional Response to a DesignMegan Grocki
In the design process we follow, once we have defined the conceptual direction and content strategy for a given design and refined our approach through user research and iterative usability testing, we start applying visual design. Generally, we take a key screen whose structure and functionality we have finalized—for example, a layout for a home page or a dashboard page—and explore three alternatives for visual style. These three alternative visual designs, or comps, include the same content, but reflect different choices for color palette and imagery. The idea is to present business owners and stakeholders with different visual design options from which they can choose. Sometimes there is a clear favorite among stakeholders or an option that makes the most sense from a brand perspective. However, there can often be disagreements among the members of a project team on which direction to choose. If we’ve done our job right, there are rationales for our various design decisions in the different comps, but even so, there may be disagreement about which rationale is most appropriate for the situation.
As practitioners of user-centered design, it is natural for us to turn to user research to help inform and guide the process of choosing a visual design. But traditional usability testing and related methods don’t seem particularly well suited for assessing visual design for two reasons:
1. When we reach out to users for feedback on visual design options, stakeholders are generally looking for large sample sizes—larger than are typical for a qualitative usability study.
2. The response we are looking for from users is more emotional—that is, less about users’ ability to accomplish tasks and more about their affective response to a given design.
With this in mind, we were very interested in articles we saw on Desirability Testing. In one article, the author posits desirability testing as a mix of quantitative and qualitative methods that allow you to assess users’ attitudes toward aesthetics and visual appeal. Inspired by his overview, we researched desirability studies a bit further and tried a modified version of the techniques on one of our projects. This presentation reviews the variants of desirability testing that we considered and the lessons we learned from a desirability study on visual design options for one of our projects. Interestingly, we found that while desirability testing did help us better understand participant’s self reported emotional response to a visual design, it also helped us identify other key areas of the experience that could be improved.
The document describes the steps involved in making effective decisions: 1) Define the problem, gather relevant information, and establish criteria for evaluating alternatives. 2) Brainstorm alternative solutions. 3) Use the criteria to evaluate alternatives and select the best option. 4) Implement the chosen decision and evaluate the outcome. It provides details on each step, such as identifying stakeholders and creating cause-and-effect diagrams when gathering information.
Tips for better surveys: better questions in your questionnaire, better overall survey process. From UPA2012 in Las Vegas.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Write better survey questions, run a survey from start to finish, survey tips from the expert survey methodologists. Workshop at JBoye Conference, Aarhus, Denmark, 2011.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The first part of a workshop on user experience surveys. Topics: (1) how to improve the questions in surveys and (2) how to assess UX using a survey.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Forms that work: Understanding forms to improve their design by @cjformsCaroline Jarrett
A day-long workshop on forms design, focusing on why businesses need forms and how people interact with them.
Accessibility note: I've tried to make this version of the presentation accessible. If you find that it's not working for you, please let me know and I'll try my best to solve the problems.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Improving Care Experiences Through Human Centered Design - HXR 2016 - Jonatha...Mad*Pow
This document discusses improving patient care experiences through human-centric design and experience strategy. It emphasizes understanding behavioral variances and trigger events, discovering how life integrates with needs, and finding core customer value propositions. It questions why the healthcare system does not learn from mistakes and suggests a network of sharing and evolving prescriptive patient experience standards to address issues.
Meeting at the Intersection of Content Strategy and UX - UXPA Boston 2015 - M...Mad*Pow
Marli Mesibov, the Director of Content Strategy at Mad*Pow, gave a presentation on how content strategy and UX design intersect. She identified six key areas of intersection: branding, user journeys, site maps, pair programming, content templates, and microinteractions. She also provided five tips for easier collaboration between content strategists and UX designers, such as swapping skillsets and tools, using pictionary, dictating sketches, doing group gamestorming, and being willing to compromise.
Desirability Testing: Analyzing Emotional Response to a DesignMegan Grocki
In the design process we follow, once we have defined the conceptual direction and content strategy for a given design and refined our approach through user research and iterative usability testing, we start applying visual design. Generally, we take a key screen whose structure and functionality we have finalized—for example, a layout for a home page or a dashboard page—and explore three alternatives for visual style. These three alternative visual designs, or comps, include the same content, but reflect different choices for color palette and imagery. The idea is to present business owners and stakeholders with different visual design options from which they can choose. Sometimes there is a clear favorite among stakeholders or an option that makes the most sense from a brand perspective. However, there can often be disagreements among the members of a project team on which direction to choose. If we’ve done our job right, there are rationales for our various design decisions in the different comps, but even so, there may be disagreement about which rationale is most appropriate for the situation.
As practitioners of user-centered design, it is natural for us to turn to user research to help inform and guide the process of choosing a visual design. But traditional usability testing and related methods don’t seem particularly well suited for assessing visual design for two reasons:
1. When we reach out to users for feedback on visual design options, stakeholders are generally looking for large sample sizes—larger than are typical for a qualitative usability study.
2. The response we are looking for from users is more emotional—that is, less about users’ ability to accomplish tasks and more about their affective response to a given design.
With this in mind, we were very interested in articles we saw on Desirability Testing. In one article, the author posits desirability testing as a mix of quantitative and qualitative methods that allow you to assess users’ attitudes toward aesthetics and visual appeal. Inspired by his overview, we researched desirability studies a bit further and tried a modified version of the techniques on one of our projects. This presentation reviews the variants of desirability testing that we considered and the lessons we learned from a desirability study on visual design options for one of our projects. Interestingly, we found that while desirability testing did help us better understand participant’s self reported emotional response to a visual design, it also helped us identify other key areas of the experience that could be improved.
The document describes the steps involved in making effective decisions: 1) Define the problem, gather relevant information, and establish criteria for evaluating alternatives. 2) Brainstorm alternative solutions. 3) Use the criteria to evaluate alternatives and select the best option. 4) Implement the chosen decision and evaluate the outcome. It provides details on each step, such as identifying stakeholders and creating cause-and-effect diagrams when gathering information.
Tips for better surveys: better questions in your questionnaire, better overall survey process. From UPA2012 in Las Vegas.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Write better survey questions, run a survey from start to finish, survey tips from the expert survey methodologists. Workshop at JBoye Conference, Aarhus, Denmark, 2011.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The first part of a workshop on user experience surveys. Topics: (1) how to improve the questions in surveys and (2) how to assess UX using a survey.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Forms that work: Understanding forms to improve their design by @cjformsCaroline Jarrett
A day-long workshop on forms design, focusing on why businesses need forms and how people interact with them.
Accessibility note: I've tried to make this version of the presentation accessible. If you find that it's not working for you, please let me know and I'll try my best to solve the problems.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Improving Care Experiences Through Human Centered Design - HXR 2016 - Jonatha...Mad*Pow
This document discusses improving patient care experiences through human-centric design and experience strategy. It emphasizes understanding behavioral variances and trigger events, discovering how life integrates with needs, and finding core customer value propositions. It questions why the healthcare system does not learn from mistakes and suggests a network of sharing and evolving prescriptive patient experience standards to address issues.
Meeting at the Intersection of Content Strategy and UX - UXPA Boston 2015 - M...Mad*Pow
Marli Mesibov, the Director of Content Strategy at Mad*Pow, gave a presentation on how content strategy and UX design intersect. She identified six key areas of intersection: branding, user journeys, site maps, pair programming, content templates, and microinteractions. She also provided five tips for easier collaboration between content strategists and UX designers, such as swapping skillsets and tools, using pictionary, dictating sketches, doing group gamestorming, and being willing to compromise.
How do we talk about healthcare - HXR 2016 - Marli Mesibov & Dana OrtegonMad*Pow
The document summarizes a presentation by Marli Mesibov and Dana Ortégon of Mad*Pow about content strategy for healthcare organizations. They discuss why content strategy is important for healthcare sites, the challenges of healthcare content, and how the content strategy process works. They then take questions from the audience about topics like when to create original content vs use external sources, how to determine metadata and ensure HIPAA compliance, and how to establish an authoritative voice.
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan BerlinMad*Pow
The document provides guidance on best practices for experience research. It discusses understanding research goals, choosing appropriate research methods, gathering qualitative data through tasks, moderator guides, note taking, and organizing findings. The key points are: understand business goals and user needs to define research goals; use a methods chart to evaluate options based on goals, timeline, budget and other constraints; and properly document studies through moderator guides, notes grids, and findings sheets to facilitate analysis.
Don't Fear the Small Numbers - HXR 2016 - Dan Berlin and Susan MercerMad*Pow
The document discusses how small numbers of participants in design research can provide big insights. It argues that 5-8 users are typically needed to find 80-90% of usability problems. Design research is iterative - finding most issues, updating the design, and repeating. Trends often emerge after a few interviews. Outliers provide incremental data. The "right" number depends on project scope, but 12-16 participants are usually sufficient per segment and user group for discovery, and 8-10 for usability studies. Small numbers maximize the ROI on user experience research.
Personas, the Cause of and Solution to All of Life's Problems - UXPA Boston 2...Mad*Pow
Mad*Pow is a design agency that focuses on improving experiences with technology, organizations, and human interaction. The document discusses personas, which are archetypes based on real user behaviors used to represent target users throughout the design process. The document provides guidance on creating effective personas, such as focusing on behaviors over demographics and avoiding ambiguous details.
Structured Data and User Experience - Paul Kahn, 2012Mad*Pow
This document discusses structured data and user experience design. It notes that as the amount of digital content has grown exponentially since 1995, designers' task has been to help readers navigate and understand information online. While early websites lacked the physical context of print, designers adapted visual languages and created navigation systems. Today, users create and share various types of digital content, implicitly manipulating metadata through their interactions even if they do not understand the concept of metadata explicitly. The document examines the design challenges posed by data with no structure, some structure from user-added tags or profiles, and fully structured data in predefined fields, and how to design experiences that leverage existing structured data in a usable way for people.
Harness the power of avatars, case studies - HXR 2014 - Ciara TaylorMad*Pow
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise boosts blood flow and levels of neurotransmitters and endorphins which elevate and stabilize mood.
Prototyping for Early Validation - Michael Hawley, 2009Mad*Pow
This document discusses using Axure for early validation through rapid prototyping. It summarizes that prototyping allows stakeholders to envision solutions early in the design process and provides users the opportunity to provide feedback. Prototyping reduces risks and rework by validating designs before full production. The presentation provides an overview of prototyping benefits and considerations for selecting the right tool. It argues fidelity may not be the most important factor and multiple tools can create publishable, platform-independent prototypes for shared workgroups. The presentation concludes with an invitation for questions and contact information.
Communities of Care, Strategic Social Interaction Design for Healthcare - ide...Mad*Pow
This document discusses using social media in healthcare. It begins by defining social media and outlining how it could help in healthcare by providing more frequent access to information and healthcare providers. Examples of current social media use in healthcare are given, such as patient forums and doctors participating on forums. Opportunities for using social media to connect patients with their care team, track health data, and enable patient-doctor interactions are presented. Risks and barriers like legal issues and lack of positive results are also covered. The document concludes by asking attendees what social media examples they have seen work well and what they hope to see in the future regarding social media in healthcare.
Organizational Parkour, the Negotiation Game - Seattle Infocamp 2013 - Joan V...Mad*Pow
This document discusses using the principles of parkour to navigate organizational barriers. It suggests practicing three techniques: strength training through observation and empathy building; fundamentals of principled negotiation like separating problems from people; and obstacle courses like roleplaying games that simulate project negotiations. The goal is to gain skills in dealing with conflicts and communication issues that are inherent parts of any organizational work.
Naratives in Healthcare, Stories as Drivers of Change - HXR 2016 - Samantha D...Mad*Pow
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise causes chemical changes in the brain that may help boost feelings of calmness, happiness and focus.
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy CuevaMad*Pow
This document provides examples and best practices for search and filter interfaces. It discusses types of searches, considerations for search interface design, examples of different search interfaces, and guidelines for presenting search results and ensuring relevancy. The document is intended as a reference for designing effective search functions on websites and applications.
Research & Design Methods in Healthcare - HDX 2013 - Adam ConnorMad*Pow
The document discusses research and design methods in healthcare. It begins by outlining the challenges of designing for healthcare due to its complex nature with many coordinated parts. It then discusses different research methods like usability testing, interviews, and ethnography to understand users and inform design. Journey mapping is presented as a way to synthesize research into narratives of user experiences over time. The document advocates for design studios with cross-functional teams to quickly generate, discuss, and refine ideas through an iterative process of sketching, presenting, and critiquing concepts. This helps build consensus around solutions.
Designing Design Workshops - Adam Connor, 2016Mad*Pow
The document provides an overview of designing effective workshops. It discusses how workshops can generate ideas, allow sharing of perspectives, and build consensus around solutions. However, effective workshops require more planning than simply putting people in a room. The document then covers exploring and practicing different workshop activities, when to use each type, best practices for planning, and tips for facilitation. Specific activities discussed include challenge mapping to reframe problems and free listing to gather understanding of a topic.
Digital Whips and Other Persuasive Technologies - HXR 2014 - Dustin DitommasoMad*Pow
The document discusses concepts from Michel Foucault and ED Deci & Richard Ryan regarding technologies of power, technologies of the self, and self-determination. It notes that technologies of the self can be technologies of domination, and that one can be as tyrannical toward oneself as others can be through control. The document also briefly outlines behaviorism and its non-motivational account of human behavior based on external environmental factors and reinforcements rather than internal intentionality or motivation.
The Need for Speed, Optimizing the User Experience pt2 - UXPA Boston 2014 - J...Mad*Pow
This document summarizes strategies for optimizing a website's user experience by improving page load speeds. It discusses researching current page speeds and user locations/devices, setting performance goals, optimizing technologies like images, scripts and caching, improving information architecture to reduce page sizes, addressing trends that slow performance, and visual design techniques to decrease file sizes like using illustrations and CSS sprites. The overall message is that digital strategies must prioritize speed optimization to meaningfully improve the user experience.
Building Out a User Experience Team, Making UX Relevant Companywide - UPA 201...Mad*Pow
The document summarizes a panel discussion on building out a user experience team and making UX relevant companywide. The panelists provided their perspectives on establishing UX teams, including the need for executive buy-in, integrating UX into product development processes, hiring qualified practitioners, establishing support structures for UX teams, and considering social psychological factors for team success.
Customer Journey Mapping Illustrating the Big Picture - MIMA Summit 2013 - Me...Mad*Pow
Customer journey mapping involves illustrating the relationship between an individual and an organization over time through various touchpoints and channels. The document discusses how to create customer journey models through gathering research, designing iterations, and socializing the models. Key steps include conducting empathy maps, brainstorming sessions with different lenses, and mapping timelines, touchpoints, and opportunities. Maintaining the models involves periodic research updates and involving stakeholders. The overall goal is to improve customer experiences and strategize engagement across an organization.
HxRefactored - Case Studies - Ciara Taylor - Mad*PowHxRefactored
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive function. Exercise causes chemical changes in the brain that may help protect against mental illness and improve symptoms.
Preference and Desirability Testing: Measuring Emotional Response to Guide De...Paul Doncaster
(From UPA 2011-Atlanta) Usability practitioners have a variety of methods and techniques to inform interaction design and identify usability problems. However, these tools are not as effective at evaluating the visceral and emotional response generated by visual design and aesthetics. This presentation will discuss why studying visual design is important, review considerations for preference and desirability testing and present two alternative approaches to user studies of visual designs in the form of case studies.
Desirability and Preference Testing - UPA International 2011hawleymichael
This document discusses methods for measuring emotional response to guide design, including case studies of their application. Preference and desirability testing can provide insights beyond just asking users which option they prefer. Methods discussed include using product reaction cards to assess impressions of brand attributes, quick exposure memory tests, and physiological measurements. Case studies demonstrate applying these methods to redesign a hospital website and refine the design of WestlawNext based on user feedback. The document advocates using both qualitative and quantitative research to inform design decisions.
How do we talk about healthcare - HXR 2016 - Marli Mesibov & Dana OrtegonMad*Pow
The document summarizes a presentation by Marli Mesibov and Dana Ortégon of Mad*Pow about content strategy for healthcare organizations. They discuss why content strategy is important for healthcare sites, the challenges of healthcare content, and how the content strategy process works. They then take questions from the audience about topics like when to create original content vs use external sources, how to determine metadata and ensure HIPAA compliance, and how to establish an authoritative voice.
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan BerlinMad*Pow
The document provides guidance on best practices for experience research. It discusses understanding research goals, choosing appropriate research methods, gathering qualitative data through tasks, moderator guides, note taking, and organizing findings. The key points are: understand business goals and user needs to define research goals; use a methods chart to evaluate options based on goals, timeline, budget and other constraints; and properly document studies through moderator guides, notes grids, and findings sheets to facilitate analysis.
Don't Fear the Small Numbers - HXR 2016 - Dan Berlin and Susan MercerMad*Pow
The document discusses how small numbers of participants in design research can provide big insights. It argues that 5-8 users are typically needed to find 80-90% of usability problems. Design research is iterative - finding most issues, updating the design, and repeating. Trends often emerge after a few interviews. Outliers provide incremental data. The "right" number depends on project scope, but 12-16 participants are usually sufficient per segment and user group for discovery, and 8-10 for usability studies. Small numbers maximize the ROI on user experience research.
Personas, the Cause of and Solution to All of Life's Problems - UXPA Boston 2...Mad*Pow
Mad*Pow is a design agency that focuses on improving experiences with technology, organizations, and human interaction. The document discusses personas, which are archetypes based on real user behaviors used to represent target users throughout the design process. The document provides guidance on creating effective personas, such as focusing on behaviors over demographics and avoiding ambiguous details.
Structured Data and User Experience - Paul Kahn, 2012Mad*Pow
This document discusses structured data and user experience design. It notes that as the amount of digital content has grown exponentially since 1995, designers' task has been to help readers navigate and understand information online. While early websites lacked the physical context of print, designers adapted visual languages and created navigation systems. Today, users create and share various types of digital content, implicitly manipulating metadata through their interactions even if they do not understand the concept of metadata explicitly. The document examines the design challenges posed by data with no structure, some structure from user-added tags or profiles, and fully structured data in predefined fields, and how to design experiences that leverage existing structured data in a usable way for people.
Harness the power of avatars, case studies - HXR 2014 - Ciara TaylorMad*Pow
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise boosts blood flow and levels of neurotransmitters and endorphins which elevate and stabilize mood.
Prototyping for Early Validation - Michael Hawley, 2009Mad*Pow
This document discusses using Axure for early validation through rapid prototyping. It summarizes that prototyping allows stakeholders to envision solutions early in the design process and provides users the opportunity to provide feedback. Prototyping reduces risks and rework by validating designs before full production. The presentation provides an overview of prototyping benefits and considerations for selecting the right tool. It argues fidelity may not be the most important factor and multiple tools can create publishable, platform-independent prototypes for shared workgroups. The presentation concludes with an invitation for questions and contact information.
Communities of Care, Strategic Social Interaction Design for Healthcare - ide...Mad*Pow
This document discusses using social media in healthcare. It begins by defining social media and outlining how it could help in healthcare by providing more frequent access to information and healthcare providers. Examples of current social media use in healthcare are given, such as patient forums and doctors participating on forums. Opportunities for using social media to connect patients with their care team, track health data, and enable patient-doctor interactions are presented. Risks and barriers like legal issues and lack of positive results are also covered. The document concludes by asking attendees what social media examples they have seen work well and what they hope to see in the future regarding social media in healthcare.
Organizational Parkour, the Negotiation Game - Seattle Infocamp 2013 - Joan V...Mad*Pow
This document discusses using the principles of parkour to navigate organizational barriers. It suggests practicing three techniques: strength training through observation and empathy building; fundamentals of principled negotiation like separating problems from people; and obstacle courses like roleplaying games that simulate project negotiations. The goal is to gain skills in dealing with conflicts and communication issues that are inherent parts of any organizational work.
Naratives in Healthcare, Stories as Drivers of Change - HXR 2016 - Samantha D...Mad*Pow
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise causes chemical changes in the brain that may help boost feelings of calmness, happiness and focus.
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy CuevaMad*Pow
This document provides examples and best practices for search and filter interfaces. It discusses types of searches, considerations for search interface design, examples of different search interfaces, and guidelines for presenting search results and ensuring relevancy. The document is intended as a reference for designing effective search functions on websites and applications.
Research & Design Methods in Healthcare - HDX 2013 - Adam ConnorMad*Pow
The document discusses research and design methods in healthcare. It begins by outlining the challenges of designing for healthcare due to its complex nature with many coordinated parts. It then discusses different research methods like usability testing, interviews, and ethnography to understand users and inform design. Journey mapping is presented as a way to synthesize research into narratives of user experiences over time. The document advocates for design studios with cross-functional teams to quickly generate, discuss, and refine ideas through an iterative process of sketching, presenting, and critiquing concepts. This helps build consensus around solutions.
Designing Design Workshops - Adam Connor, 2016Mad*Pow
The document provides an overview of designing effective workshops. It discusses how workshops can generate ideas, allow sharing of perspectives, and build consensus around solutions. However, effective workshops require more planning than simply putting people in a room. The document then covers exploring and practicing different workshop activities, when to use each type, best practices for planning, and tips for facilitation. Specific activities discussed include challenge mapping to reframe problems and free listing to gather understanding of a topic.
Digital Whips and Other Persuasive Technologies - HXR 2014 - Dustin DitommasoMad*Pow
The document discusses concepts from Michel Foucault and ED Deci & Richard Ryan regarding technologies of power, technologies of the self, and self-determination. It notes that technologies of the self can be technologies of domination, and that one can be as tyrannical toward oneself as others can be through control. The document also briefly outlines behaviorism and its non-motivational account of human behavior based on external environmental factors and reinforcements rather than internal intentionality or motivation.
The Need for Speed, Optimizing the User Experience pt2 - UXPA Boston 2014 - J...Mad*Pow
This document summarizes strategies for optimizing a website's user experience by improving page load speeds. It discusses researching current page speeds and user locations/devices, setting performance goals, optimizing technologies like images, scripts and caching, improving information architecture to reduce page sizes, addressing trends that slow performance, and visual design techniques to decrease file sizes like using illustrations and CSS sprites. The overall message is that digital strategies must prioritize speed optimization to meaningfully improve the user experience.
Building Out a User Experience Team, Making UX Relevant Companywide - UPA 201...Mad*Pow
The document summarizes a panel discussion on building out a user experience team and making UX relevant companywide. The panelists provided their perspectives on establishing UX teams, including the need for executive buy-in, integrating UX into product development processes, hiring qualified practitioners, establishing support structures for UX teams, and considering social psychological factors for team success.
Customer Journey Mapping Illustrating the Big Picture - MIMA Summit 2013 - Me...Mad*Pow
Customer journey mapping involves illustrating the relationship between an individual and an organization over time through various touchpoints and channels. The document discusses how to create customer journey models through gathering research, designing iterations, and socializing the models. Key steps include conducting empathy maps, brainstorming sessions with different lenses, and mapping timelines, touchpoints, and opportunities. Maintaining the models involves periodic research updates and involving stakeholders. The overall goal is to improve customer experiences and strategize engagement across an organization.
HxRefactored - Case Studies - Ciara Taylor - Mad*PowHxRefactored
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive function. Exercise causes chemical changes in the brain that may help protect against mental illness and improve symptoms.
Preference and Desirability Testing: Measuring Emotional Response to Guide De...Paul Doncaster
(From UPA 2011-Atlanta) Usability practitioners have a variety of methods and techniques to inform interaction design and identify usability problems. However, these tools are not as effective at evaluating the visceral and emotional response generated by visual design and aesthetics. This presentation will discuss why studying visual design is important, review considerations for preference and desirability testing and present two alternative approaches to user studies of visual designs in the form of case studies.
Desirability and Preference Testing - UPA International 2011hawleymichael
This document discusses methods for measuring emotional response to guide design, including case studies of their application. Preference and desirability testing can provide insights beyond just asking users which option they prefer. Methods discussed include using product reaction cards to assess impressions of brand attributes, quick exposure memory tests, and physiological measurements. Case studies demonstrate applying these methods to redesign a hospital website and refine the design of WestlawNext based on user feedback. The document advocates using both qualitative and quantitative research to inform design decisions.
Have you always wanted to do more UX research but thought it might cost too much, or take too much time? Learn how a few UX ers, Jodi Bollaert and Megan Schwarz, at Team Detroit (advertising) in Michigan, have used several fast & cheap web-based tools & methodologies to glean valuable user insights for digital automotive projects.
Conjoint analysis is a technique used to understand how people value different attributes or features of a product or service. It involves showing people combinations of attributes and levels and asking them to choose their preferred options. This allows researchers to model the importance of attributes and understand how people trade off different features. The document provides an overview of conjoint analysis, including what problems it can address, how it works, and key considerations in designing a conjoint study. An example study is presented to illustrate attribute selection, task design, and the type of outputs generated, including part-worth utilities and importance scores.
It seems simple, build a system that allows users to effortlessly reach their goals. So why is it a challenge for teams to stay aligned to user needs throughout the entire design process? Distractions crop up, extra design elements are added along the way, and soon you have a user interface that potentially causes user diversion and is not the streamlined interface you envisioned it would be. In this session, the audience follows the development of an app and discovers five simple steps they can take to build UI that avoids distraction and helps users reach their goals.
• Mapping results from research, recognizing and grouping user needs to recognize importance and hierarchy
• Mapping user needs to specific features
• Building a rough information architecture from features
• Prototyping and testing
• Tips for distraction-free visual design
Physical and Online Card Sorts: A Practical Overview and Case StudyBob Thomas
This is a practical case study where I worked with an educational testing and assessment company to help them redesign their information architecture for their corporate intranet, using an open card sort.
So many companies build corporate intranets and then don’t do anything with them, so they just languish and are ignored by employees.
The company clearly saw the benefit of a corporate intranet and wanted to improve the site for its employees. It saw this as a competitive advantage.
Every study starts with a question. This session at CSUN 2014 started by examining the questions that usability testing can answer. Short case studies illustrate how the right technique will help us know not only what is happening but also why it’s happening. It's an overview of usability testing as a research method, and what you can (and can’t) learn from working with real people as they try to use a web site or other product.
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...BrittanyRubinstein
As part of Denver's 2019 Startup Week, Crownpeak's Director of UX, Ari Weissman and Lys Maitland, Experience Research Manager at a national healthcare organization, presented a joint session on "Choosing a direction: Learning how to test ideas and designs."
Ch 6 only 1. Distinguish between a purpose statement, research pMaximaSheffield592
Ch 6 only
1. Distinguish between a purpose statement, research problem, and research questions.
2. What are major ideas that should be included in a qualitative purpose statement?
3. What are the major components of a quantitative purpose statement?
4. What are the major components of a mixed methods purpose statement?
Requirements Engineering (20 points)
In Chapter 4 of Software Engineering. Sommerville, Pearson, 2016 (10th edition), Sommerville discusses ethnography as a method for eliciting requirements.
1. Discuss two advantages and two disadvantages of an ethnographic approach. (5 points)
2. Suggest two contexts where ethnography might be a challenging method of requirements engineering. For each context, how would you recommend that your team elicit requirements? (15 points)
Design (20 points)Design patterns (5 points)
Which of the following statements is (are) true? Explain.
1. StudentsDatabase is the model, StudentsManager is the controller, and WebApplication is the view.
2. StudentsDatabase is the model, StudentsManager is the view, and WebApplication is the controller.
3. StudentsManager is the model, StudentsDatabase is the view, and StudentsManager is the controller.
4. This is not MVC, because StudentsManager must use a listener to be notified when the database changes.
(Credit: EPFL)Design task (15 points)
Suppose you are asked to design a time management and notetaking system to support (1) scheduling meetings; and (2) tracking the documents associated with those meetings (e.g. agendas, presentations, meeting minutes).[footnoteRef:1] The system should accommodate [1: Such a feature seems like an inevitable development in any messaging platform…]
Make reasonable assumptions as needed.
1. Create a use case for “Schedule meeting”. You might follow the style in Sommerville Figure 7.3. (5 points)
2. Identify the objects in your system. Represent them using a structural diagram showing the associations between objects (“Class diagram” – cf. Sommerville Figure 5.9). (5 points)
3. Draw a sequence diagram showing the interactions between objects when a group of people are arranging a meeting (cf. Sommerville Figure 5.15). (5 points)
1. Implementation (20 points)
Consider the software package is-positive.[footnoteRef:2] Examine its source code (see index.js) and its test suite (see test.js), then complete these questions. [2: https://www.npmjs.com/package/is-positive]
1. Describe the API surface of this package. (2 points)
2. Describe how you would test this package. Describe how and why your approach would change if you maintained a similar package in a different programming language of your choice. (2 points)
3. According to npmjs.com, this package receives over 16,000 downloads each month.
a. Why might an engineer choose to use this package? (4 points)
b. Why might an engineer choose not to use this package? (You may find insights from the chapter ab ...
Ch 6 only 1. distinguish between a purpose statement, research pnand15
This document provides guidance and examples for developing different components of a research proposal or study across qualitative, quantitative, and mixed methods approaches. It discusses key elements such as developing a purpose statement, research questions and hypotheses, reviewing literature, using theory, and addressing ethical considerations. Examples are provided for different types of qualitative studies, quantitative studies using surveys and experiments, and mixed methods studies with convergent, explanatory sequential and exploratory sequential designs. Guidance is also given on writing strategies, developing introductions, and structuring different sections of a research proposal.
The document provides an overview of user experience (UX) design for startups. It discusses:
1) What UX design is and how UX designers work to identify problems and find solutions to create a seamless user experience.
2) When to bring in UX designers, which should be early in the process to allow for collaboration with engineers.
3) The importance of user research through methods like focus groups, interviews and usability testing to understand users and inform the design.
4) How to develop personas, functions lists, information architecture and user scenarios/flows to map the optimal website structure and organization.
Designed by Committee: An Analytics and User-Focused Approach to the Overhaul...nealw6971
Neal Wollenberg redesigned the homepage for Kansas State University's Global Campus. He formed a committee, conducted usability testing and analyzed site analytics. The redesign focused on the top pages users accessed based on analytics and addressed issues from usability testing. The new homepage launched in December 2017. Analytics now show improved values for key pages and decreased bounce rate, though the exit rate increased. The redesign process taught Valuable lessons about using data and gathering stakeholder feedback to create user-focused websites.
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)Jennifer Romano Bergstrom
This document discusses best practices for designing usable government forms and surveys. It covers topics like usability testing methods, navigation patterns, input field design, and placement of instructions. Specifically, it recommends:
- Conducting one-on-one sessions, focus groups, and surveys to test designs
- Placing the "Next" button on the left for primary navigation
- Avoiding vertical and horizontal scrolling for open-ended responses
- Using radio buttons and checkboxes according to conventions
- Placing instructions near related questions to reduce cognitive load
SIGIR Tutorial on IR Evaluation: Designing an End-to-End Offline Evaluation P...Jin Young Kim
This tutorial aims to provide attendees with a detailed understanding of end-to-end evaluation pipeline based on human judgments (offline measurement). The tutorial will give an overview of the state of the art methods, techniques, and metrics necessary for each stage of evaluation process. We will mostly focus on evaluating an information retrieval (search) system, but the other tasks such as recommendation and classification will also be discussed. Practical examples will be drawn both from the literature and from real world usage scenarios in industry.
The usability test report summarizes tests conducted on the Ram Page website. Four participants were observed using eye tracking software as they freely navigated the website for 10 minutes. Both positive and negative feedback was gathered. Positively, pictures drew attention and functionality was good. Negatively, small font size made articles hard to read, event descriptions were confusing, and advertisements were ineffective and missed. The tests provided insights on improving reader engagement and navigation of the website.
Qualitative Research vs Quantitative Research - a QuestionPro Academic WebinarQuestionPro
Hosted on October 14, 2020, this QuestionPro Academic focused webinar delved into the differences of Qualitative and Quantitative research and how you can achieve this using the QuestionPro research platform. We spoke about Heatmap and Hotspot analysis, card sorting, online focus groups using video discussions and even a beta feature coming soon, LiveCast that uses NLP to build real-time analytics from video survey questions. Our speaker was Dan Fleetwood, the President for Research and Insights at QuestionPro.
1. The document outlines the steps to conduct a usability test, including deciding what to test, designing the test, conducting the test, and writing up the test findings.
2. Key steps in designing the test involve identifying users and test tasks, developing metrics to evaluate usability, and creating a data collection form.
3. Conducting the test involves playing the role of a new user, completing each task while collecting data, and taking screenshots to support findings.
The Benefits of Survey Standardization in Building Brand | SoGoSurveySogolytics
Every connection with your audience is an opportunity to build your brand, and surveys are no exception. Review these benefits of survey standardization, define your style, and grow brand identity and recognition!
Customer Centric: Product/Service Design for Businessstudiotelon
In 2016 I gave a guest lecture to Business students on customer-centric design in developing products and services. The lecture was part of a one-semester course on digital business.
We get lots of feedback from our customers telling us how we can make our project prioritization and AHP decision-support software even better. This short slide deck will show you what’s new.
Similar to Preference and Desirability Testing, Measuring Emotional Response to Guide Design - UPA 2011 - Michael Hawley & Paul Doncaster (20)
Webinar: What Did I Miss? The Hidden Costs of Depriortizing Diversity in User...Mad*Pow
Characteristics like race, ethnicity, gender, and disability status can have a significant impact on how we experience the world, and how the world experiences us. In UX research, diversity is the first thing to vanish from the recruit when the going gets tough; Megan will talk about what we miss when that happens, and what researchers can do about it in their own practice. This presentation will demonstrate why a diverse recruit is imperative for a strong user research study, provide examples of what we miss when the recruit is homogeneous, and offering tactics for addressing the issue.
Presented by Megan Campos, Experience Research Director, Mad*Pow
Watch the presentation at https://youtu.be/E41q8Nx67Do
Webinar: Intro to Strategic Foresight & Futures ThinkingMad*Pow
Presented by Mad*Pow Experience Strategist, Liz Possee Corthell.
When the future is uncertain, how can organizations design and innovate boldly but responsibly? Futures thinking is an approach to strategic design that considers what is likely to change and what is likely to stay the same in the future, as a means to be more reflective in strategic planning. Considered by some to be more of an art, and by others to be a science, futures thinking gives us a framework to talk about our current world, and how the world may look in the future.
To quote futurist Dr. Sohail Inayatullah, “With futures thinking, we use the future to change the present. “
In this webinar, you’ll learn that futures thinking is not an effort to predict the future, but rather a means to illuminate unexpected implications of present-day issues that empower individuals and organizations to actively design desirable futures. The emphasis isn’t on what will happen, but on what could happen, given various observed drivers.
It’s a way of gaining new perspectives and context for present-day decisions, as well as for navigating the dilemma at the heart of all strategic thinking: the future can’t be predicted, yet we have to make choices based on what is to come.
This presentation will include a few tools you can start using right away, as well as a few activities to get us thinking about the future.
Let’s Get Meta: Applying Service Design To Improve Employee Experiences… and ...Mad*Pow
Love it or hate it, people spend most of their lives working. Those working hours include behaviors, tasks, and, interactions that all add up to… experiences… and how well the employee experience is designed can have far reaching impacts on the delivery of products and services to customers. As the world embraces human centered design and focuses more and more on the importance of thoughtfully designed customer experiences, we must not lose sight of the other humans in our experience ecosystem, (not just the ones paying for a product or service). Employee experience is more than just physical environments and HR benefits – it’s about understanding the unique needs of people who mediate the experiences of others, whether through direct interaction with customers or behind the scenes roles with downstream effects. Thankfully, the very tools that help us design and deliver exceptional experiences for customers also help us understand and support the employees within an organization.
Join this webinar to learn more about service design, and how grounding your customer engagement strategies in service design methods can provide uniquely powerful aids to improve employee experience– retaining talent, scaling operational efficiencies, and ultimately empowering your employees to deliver better customer experiences in turn.
Presented by Jen Briselli, Mad*Pow SVP Experience Strategy & Service Design
Behavior Change Design: A Comprehensive Yet Practical Approach to Improving H...Mad*Pow
We live in an age where most of the pressing health issues we face as a society can be linked directly or indirectly to underlying social and behavioral determinants. These two issues present not only significant challenges to healthcare providers but also to payers seeking cost-effective ways to manage population health and provide value. Supporting people in living healthier lifestyles is, therefore, a fundamental concern for both affected and at-risk populations as well as for healthcare payers, providers, caregivers, and governments.
But how do we best support people in adopting and sustaining health promoting and protective behaviors, and reducing or avoiding health-risk behaviors over the course of a lifetime? The answer, lies of course, in the ever-maturing science of behavior change. The past decade has materialized a renaissance of theory-and-evidence-to-practice approaches that focus not only on identifying ‘what works’ when it comes changing behavior for a given problem, population, and context but also on how these techniques can be used to deploy interventions through any channel to change behavior and achieve meaningful outcomes.
This webinar will present an overview of the essential components of modern, applied behavioral science, and a process model for the design, implementation, and evaluation of effective behavior change interventions.
Communication Strategies to Keep Employees Engaged and Informed During a Chronic Crisis
View the webinar here: https://youtu.be/2frLDn5C_zs
As the new normal continues to evolve, companies are being challenged daily to keep employees engaged and informed while supporting their business operations. Throughout the pandemic, employees have demonstrated their adaptability in the face of remote working, unanticipated childcare needs, furloughs, and isolation. Many employers are realizing that effective employee communication is the key.
Join Mad*Pow Founder and Chief Experience Officer Amy Heymans and Beth Clauss, President, Small Potatoes Communications, to learn how they have helped clients engage their employees, strengthen their company culture and create a unified and informed employee community. The webinar will cover how organizations can create an employee communications strategy that helps employees weather the unique circumstances of a long-term, ongoing crisis, while navigating the treacherous waters of promoting productivity and profits during a pandemic.
Design More Innovative Solutions with a Holistic Understanding of the Chronic...Mad*Pow
This document discusses designing better solutions for chronic health experiences through a holistic understanding.
It presents an experience map that blends aspects of traditional journey maps and ecosystem diagrams. The map depicts the journey of an individual seeking and maintaining health over their lifetime. It shows how their experience is influenced by their social environment, physical environment, self, and interactions with physical and digital objects in the health ecosystem.
The map is intended to support ideating interventions and solutions, build internal alignment, and guide product teams to create consistent experiences across channels for chronic health seekers. It provides an example of how to use the map by focusing on a key health event, identifying influencing factors, and brainstorming related solutions.
Accessibility for Design & Content hosted by VP, Content Strategy, Marli Mesibov & Director, Experience Design, James Christie
Mad*Pow is offering a two hour accessibility workshop for people who design digital products and services. Through a mix of presentations and participatory activities attendees will learn and practice the skills needed to ensure digital sites and services meet the needs of a real-world diverse audience.
Design and content teams have nearly universally embraced user experience, which is wonderful news for their audiences! Unfortunately, too many still lack the knowledge or ability to create accessible, inclusive designs. That means the final experiences are great for some people, but not all.
Standards and guidelines exist, but they can be complicated and long winded. Join us to move past the legalese. You will participate in activities that give you tools to improve your UX work.
This workshop is valuable for any UX designer, content strategist, product manager, or anyone else with an impact on design decision making.
By the end of the workshop, participants will
Understand the various levels of accessibility
Gain a working knowledge of the legal and regulatory frameworks that define and enforce digital accessibility
Practice how to identify and categorize accessibility problems — so you can fix them
Plan and prepare accessible design and content, before it gets to your users.
This document discusses implementing strategic design across an organization. It begins by noting that design is rarely used beyond creative departments. The presenter then outlines the benefits of strategic design, which puts design first in all initiatives. Key objectives of strategic design are discussed, such as establishing a design culture and embracing failure. Finance case studies show how other companies have launched design studios and trained staff. The document concludes by providing steps to implement strategic design, such as treating it like a new business and expanding via collaboration.
The document discusses challenges that companies face in fully adopting design practices and realizing their benefits. It notes that most companies are struggling to affect change at all levels of the organization. A study of over 2,200 companies across 23 industries found that only 5% have fully adopted design practices and realized high benefits, while 41% have low adoption of design. The document advocates for confronting cultural barriers and biases that prevent change, and investing in cultural and emotional innovation as much as product innovation. It provides examples of practices companies can adopt to help shift team behaviors and perspectives.
The document discusses the era of multimodal design and some of the challenges and opportunities presented by multimodal interfaces that integrate voice, touch, and visual elements. It notes that while voice interfaces excel at certain tasks, they have limitations like poor learnability and an inability to present multiple choices effectively. Multimodal interfaces that combine voice, text, and graphics can overcome these limitations by allowing users to interact in a more natural, flexible way similar to the physical world. The document provides tips for designing more effective multimodal experiences that account for interruptions, varying levels of cognitive load, and the need for ongoing refinement.
The document discusses how design culture and individuals can work together perfectly. It explains that establishing trust through consistent communication, a judgement-free zone and shared purpose is important. It also emphasizes the value of open conversation through shared language, making time for questions, and prioritizing discussions. Developing a culture of critique involves candor, an iterative approach and critical thinking. An effective north star consists of a clear mission, customer definition and principles to provide guidance. Cultural norms should support tactical, creative and human aspects of work. Curiosity stems from relentless exploration, comfort with ambiguity and whimsy.
FXD attendees kicked off their experience at a half-day Leadership Forum, 12:30pm -4:30pm on October 24, 2019. This forum was comprised of a diverse, creative, thoughtful group of thinkers and leaders from across the financial ecosystem and they were engaged an intimate and inspiring conversation.
During the forum, Mad*Pow’s Chief Design Officer, Michael Hawley hosted structured networking and workshop-type activities designed to identify and answer key challenges of the financial services industry. By coming together in structured dialog and sharing ideas from a leadership perspective, attendees created opportunities to learn from each other and help us lead our organizations to deliver better experiences. The forum was rich with opportunities for attendees to grow their networks and build new relationships with other leaders in finance.
Specific topics for discussions were driven by the participants in the forum, so they were as relevant as possible. The structure of the event will allowed us to build toward collective insight and inspiration:
“Meet Your Peers” – Facilitated networking and identification of challenges to designing to great experiences in finance
“Solving Challenges” - Idea sharing and relevant experiences, process, and organizational approaches to key challenges
“Imagining the Future” – Learning and finding inspiration from others by collaboratively constructing stories and future experience ideas.
Engaging with People Through Multiple Touchpoints, Channels, and Technologies.
New technologies, device types, and evolving patient expectations place a large burden on service offerings from health organizations. New technologies can be disruptive, but they can also be disrupting, especially if organizations don’t have a strategy on how to deal with the evolving landscape. Virtual reality pain management? Passive low-band telemetry data? Health monitoring? We will discuss approaches that health organizations can take to manage the ever evolving technology landscape and shifting patient dynamic from hospital care to home care.
Facilitator: Jonathan Podolsky, VP Experience Strategy, Mad*Pow
Human-Centered Design and Innovation in Health Organizations.
There is increasing acknowledgement and movement toward human-centered design and design thinking for innovation, service design, and product development. However, evolving and transforming toward these practices in well-established and highly regulated health organizations is a challenge. Organizations have explored Innovation Centers, re-organizing around products and service lines, aligning with functional domains, and expanding design thinking through training. Attendees will share their experiences as we collectively look at how health organizations can evolve to get the most impact from their design transformation efforts.
Facilitator: Adam Connor, VP Design Transformation, Mad*Pow.
Designing for Health Behavior Change.
Beyond use of digital tools and services, health organizations are increasingly considering how they can help people make positive change in their lives. Additionally, there are potential business benefits to changing behaviors to align with the organization's objectives. But designing for behavior change is challenging and has long-term outcome goals that may not be aligned with short-term business incentives for health organizations. Issues of trust and ethics also come into play. With these complex factors in mind, this discussion will explore the strategic options for health organizations to consider related to changing behavior.
Facilitator: Dr. Amy Bucher, Behavior Change Design Director, Mad*Pow.
Aline Holzwarth is an applied behavioral scientist, primarily focusing on digital health research and scientifically informed product design. She is Head of Behavioral Science at Pattern Health, a healthcare technology company that makes it easy to create personalized care plans (patterns) for patients, leveraging behavioral science to help patients stick to these patterns. She also co-founded the Behavior Shop, a behavioral science advisory company, and holds an appointment as Principal of the Center for Advanced Hindsight at Duke University, an applied behavioral science lab that helps people be happier, healthier and wealthier, at home and abroad.
Vanessa is the research director for IFTF's Future 50 Partnership, a network of future-smart organizations that support strategic foresight research into the urgent futures that will shape the next decade across the business, social and civic spheres. Her research and foresight work delivers and scales real-world impact with a focus on health and healthcare, equity and technology.
Prior to Institute for the Future, Vanessa worked in a variety of roles at the intersection of inclusive design, innovation and health, advancing product and business strategy for technology that advances health equity and programs and strategies that foster entrepreneurship among underrepresented populations.
She is a frequent speaker and has been recognized as a 2018 Aspen Ideas Festival Spotlight Health Scholar, 40 Under 40 Tech Diversity Silicon Valley, 2016 New Leaders Council San Francisco Fellow, 200 Black Women in Tech to Follow on Twitter and as a 2016 TEDMED Research Scholar. Vanessa earned her BA in psychology from Yale University and her MPH in global health from Columbia University
Trina Histon, Aubrey Kraft, W. Scott Heisler, Kaiser Permanente Care Manageme...Mad*Pow
This document provides an overview of Project Chamai, which aims to empower Kaiser Permanente members to better manage their emotional health. It discusses developing tools and resources using members' own language to feel more appropriate and authentic, and contextualizing offerings to fit members' real lives and experiences. The project team is led by two co-leads and includes service designers, strategists, and experience designers.
Practical eLearning Makeovers for EveryoneBianca Woods
Welcome to Practical eLearning Makeovers for Everyone. In this presentation, we’ll take a look at a bunch of easy-to-use visual design tips and tricks. And we’ll do this by using them to spruce up some eLearning screens that are in dire need of a new look.
Architectural and constructions management experience since 2003 including 18 years located in UAE.
Coordinate and oversee all technical activities relating to architectural and construction projects,
including directing the design team, reviewing drafts and computer models, and approving design
changes.
Organize and typically develop, and review building plans, ensuring that a project meets all safety and
environmental standards.
Prepare feasibility studies, construction contracts, and tender documents with specifications and
tender analyses.
Consulting with clients, work on formulating equipment and labor cost estimates, ensuring a project
meets environmental, safety, structural, zoning, and aesthetic standards.
Monitoring the progress of a project to assess whether or not it is in compliance with building plans
and project deadlines.
Attention to detail, exceptional time management, and strong problem-solving and communication
skills are required for this role.
Explore the essential graphic design tools and software that can elevate your creative projects. Discover industry favorites and innovative solutions for stunning design results.
Best Digital Marketing Strategy Build Your Online Presence 2024.pptxpavankumarpayexelsol
This presentation provides a comprehensive guide to the best digital marketing strategies for 2024, focusing on enhancing your online presence. Key topics include understanding and targeting your audience, building a user-friendly and mobile-responsive website, leveraging the power of social media platforms, optimizing content for search engines, and using email marketing to foster direct engagement. By adopting these strategies, you can increase brand visibility, drive traffic, generate leads, and ultimately boost sales, ensuring your business thrives in the competitive digital landscape.
Preference and Desirability Testing, Measuring Emotional Response to Guide Design - UPA 2011 - Michael Hawley & Paul Doncaster
1. Michael Hawley
Chief Design Officer, Mad*Pow
@hawleymichael
Paul Doncaster
Senior User Experience Designer, Thomson Reuters
2. Why we should care
Why it’s not always as simple as asking:
“Which option do you prefer?”
Methods to consider
Case Study: Greenwich Hospital
Case Study: WestlawNext
Summary/Comparison
10. “There’s just
something
about it . . .”
“It reminds me
of…”
“I ordinarily
don’t like red,
but for some
reason it works
here . . .”
“It’s nice and
clean.”
“It’s better
than the
other ones.”
11. “We should go with design C over
A and B, because I feel it evokes
the right kind of emotional
response in our audience that is
closer to our most important
brand attributes.”
13. Present three different concepts or ideas to participants, and ask them to
identify how two of them are different from the third and why.
13
14. Broad, experience-based
questionnaires, that also include
questions relating to visual appeal and
aesthetics
• SUS (System Usability Scale)
• QUIS (Questionnaire for User
Interface Satisfaction)
• WAMMI (Website Analysis and
Measurement Inventory)
14
15. Show participants a user
interface for a very brief
moment, then take it away.
Participants recall their first
impression, then moderator
probes for meaning.
• Helpful for layout decisions,
prominence of content, labels
• www.fivesecondtest.com
15
Attention designers:
You have
50 milliseconds
to make a good
first impression
20. Determine intended brand attributes (and their opposites)
20
1. Leverage existing marketing/brand
materials
2. Alternatively, stakeholder brainstorm
to identify key brand
attributes/descriptors using full list of
product reaction cards as a start
3. Tip: “If the brand was a person, how
would it speak to your customers?”
21. Methodology
1. Include 60/40 split of positive and negative words
2. Target 60 words, optimized to test brand
3. Simple question: “Which of the following words do you feel
best describe the site/design/product (please select 5):”
4. One comp per participant, or multiple comps per participant
(no more than 3)
Participants
1. Qualitative: Paired with usability testing
2. Quantitative: Target minimum of 30 per option if possible
21
22. 1. Calculate percentage of
positive and negative
attributes per design
2. Visualize overall
sentiment of feedback
using “word clouds” (see
wordle.net)
22
68%Positive
32%Negative
23. • Align the website with the character of the Hospital
• Update the site after nearly 10 years
• Counter impressions that Greenwich is more than just
maternity and elder care
• Communicate that they are long-standing members of
the community
23
24. • 3 visually designed comps
• 50 people reacted to each comp (quantitative) via survey
• Additional feedback obtained via participant interviews
(qualitative)
24
Hello, I am requesting feedback on a website I am working on. Your answers let
me know if the site is conveying the right feel.
1. What are your initial reactions to the web site?
2. Which of the following words best do you feel best describe the site (select 5):
Survey Questions
29. • Mix of qualitative and quantitative is key
o Qualitative helps provide color to the results
o Quantitative resonates with stakeholders and executives
• Position results as one form of input to decision-making
process, not declaring a “winner”
• Simple, cost-efficient way to assess audience’s emotional
response to a design
29
32. for Phase 1
• Use large sample sizes to establish a design “baseline,”
from which to advance the design direction in subsequent
iterations
• Isolate preference trends for specific page design aspects
• Determine tolerance for manipulation of the site “brand”
• Maintain tight security
33. Sessions were held in 4 cities over 5 days
◦ Seattle
◦ Denver
◦ Memphis
◦ Minneapolis-St. Paul
4 sessions were held per day, with a maximum of 25
participants per session
1.5 hours allotted per study, most participants finished
in less than 1 hour
319 participants successfully completed their sessions
34. Participants completed the study at individual
workstations at their own pace
All workstations included a 20” monitor, at 1024x768
resolution
Memphis, TN, May 2009
35. 1. Brief review of Westlaw critical screens
2. Positive/negative word selection to describe Westlaw
35
36. 1. Each set of Element variations were viewed in full screen
2. Participant selects “top choice” by dragging a thumbnail
image to a drop area
36
38. 1. All options viewed in full screen
2. Participant selects “top choice” by dragging a
thumbnail image to a drop area
39.
40.
41. Visual Weight (6 options)
Use of Imagery (8 options)
Components (4 options)
Search Area (4 options)
Palette (10 options)
42. 1. 19 HP designs viewed in full screen (randomized)
2. All 19 options are presented again; participant assigns a
rating using a 10-point slider.
3. Top 5 and Bottom 2 choices are positioned in order of
rating values on one long, scrollable page.
Next to each design displayed, rates key aspects for
each design on a 5-point scale
43.
44.
45.
46.
47.
48. Repeat the process for Results List design:
• Design Elements
• Column Collapsing (4 options)
• Column Separation (2 options)
• Theme/Color (8 options)
• Design Gallery
• 14 Results Lists designs (randomized)
• Key Aspects Rated
• Color scheme
• Global Header
• Summary and Excerpt (list contents)
• Filters design (left column)
• Overall look and feel
49. Repeat the process for Document Display design:
• Design Elements
• Tabs vs. Links (4 options)
• Background Separation (4 options)
• Margin Width (3 options)
• Font Size (12 options)
• Locate (2 options)
• Design Gallery
• 9 Document Display designs (randomized)
• Key Aspects Rated
• Color scheme
• Layout of content
• Text formatting
• Overall look and feel
50. “Based on the designs I’ve liked most today . . .”
50
51. Results were analyzed across 8 different sample filters
• Job Title
• Age
• Testing Location
• Years of Experience
• Hours per Week Researching
• Organization Size
• Role (decision-maker status)
The top picks were surprisingly consistent across all of the
‘Top 5’ lists analyzed
53. Home Page (19)
◦ HP16 & HP15 designs consistently
placed in the Top 5 across all filters
Results List (14)
◦ RL4 consistently placed in the Top 3
across all sample filters, and was the
#1 choice for 80% of all participants
Document Display (9)
◦ DD3 placed in the Top 5 across all
sample filters and was the #1 choice
for 77% of all participants
54. Note, participants were asked to describe the current Westlaw
before being shown the new designs.
54
Current Westlaw
1. Cluttered
2. Helpful
3. Comfortable
4. Efficient
5. Credible
New Designs
1. Attractive
2. Modern
3. Efficient
4. Helpful
5. Comfortable
55. 5 design themes were derived from post-session discussions
• “New design(s) are better than current Westlaw”
• “Clean and Fresh”
• “Contrast is Important”
• “Prefer Westlaw Blue”
• “No Big Fonts Please”
The study narrowed the list of potential designs, and we
better understood what design elements that Westlaw users
liked and disliked.
57. Goals
• Refine preferences for selected design directions
• Understand users personal reasons for liking their
preferred choices
• Get closure on other design options for online and
printed content
• Sustain tight security
Tool
• Same as in Round 1, with some minor revisions to
accommodate specialized input
58. Method
◦ View, Rate, and Pick Top Choice for
Homepage (3 options)
Result List (2 options)
Document Display (2 options)
“Why?”
◦ Simple preference selection for two unresolved UI design
issues
Citing References: Grid display or List display?
Out of Plan Indication design (6 options)
◦ Type formatting preferences for 3 different content types
Font Face
Font Size
Margin Width
59. Logistics
◦ 3 cities (Philadelphia, Kansas City, Los Angeles)
◦ 1 Day
◦ 226 participants
Analysis
◦ Filters (8 categories) were used to score the designs for each
visual preference
Results
◦ Clear choices for top designs in each of all categories
◦ “Why” feedback shed new light on designs under consideration
and helped focus “homestretch” design activities
60. Home Page (3)
◦ HP3 ranked #1in 94% of filter groups
(54% of total participants)
Results List (2)
◦ RL5 ranked #1in 97% of filter groups
(58% of total participants)
Document Display (2)
◦ DD7 ranked #1in 94% of filter groups
(61% of total participants)
61. The main concerns regarding Homepage Design HP3
◦ Search Box
Too small
How do I do a Terms-and-Connectors search?
◦ Browse Section
How do I specify multiple or specific search content?
Poor organization
Poor label
◦ Need access to “often-used” content
◦ Need better access to help
61
62. Goals
◦ Get feedback on branding options from decision makers and
those who influence purchase of the product
◦ Get closure on final outstanding design issues
Tool
◦ Same as in Rounds 1 & 2, with some minor revisions to
accommodate specialized input
63. Method
◦ Wordmark/Branding
View wordmark color combinations and design elements against
different backgrounds, pick top choice and provide comments
Make a final “Top Choice” from all selections
◦ Simple preference selection for outstanding UI design issues
Header Space: Tile or No Tile?
Notes Design
Location: Inline or Column?
State: Open or Closed?
Headnote Icon design (4 variations)
64. 1
2
3
4
Your Most Liked
Your Least Liked
What color combination do you prefer? Please rank the 4 combinations below according to your preferences.
To rank, click and drag an item from the left to a box on the right.
65. Logistics
◦ 3 cities (Seattle, Denver, Boston)
◦ 1 Day
◦ 214 participants
Analysis
◦ Simple preference, no advanced filters
Results
◦ Decision-makers confirmed that critical brand elements should be
retained
68. Measuring
Emotional Response
to Guide Design
• Quantitative & qualitative data to
identify preference trends
• “Slicing” across identifiable filters
• Emphasis on “gut-level” reactions
• Intolerance for manipulation of
product brand
• Rapid turnaround of data to all
stakeholders
o Executive
o Design
o Development
70. At what cost(s)?
• We held off asking “why” until the second round
• If we had asked why in the first round, we might have
o avoided some of internal design battles
o gotten more granular ammunition for communicating the design
vision to stakeholders
• “Need for speed” attained at the cost of detailed analysis
71. Recommendations for anyone thinking of
undertaking something like this
• Procure a “Matt” to create and administer your tool
• Get a good technical vendor for on-site
• Report results in as close to real-time as possible on a
wiki or other web-page
75. Benedek, Joey and Trish Miner. “Measuring Desirability: New Methods for
Evaluating Desirability in a Usability Lab Setting.” Proceedings of UPA 2002
Conference, Orlando, FL, July 8–12, 2002.
http://www.microsoft.com/usability/uepostings/desirabilitytoolkit.doc
Lindgaard, Gitte, Gary Fernandes, Cathy Dudek, and J. Brown. "Attention Web
Designers: You Have 50 Milliseconds to Make a Good First Impression!"
Behaviour and Information Technology, 2006.
http://www.imagescape.com/library/whitepapers/first-impression.pdf
Rohrer, Christian. “Desirability Studies: Measuring Aesthetic Response to
Visual Designs.” xdStrategy.com, October 28, 2008. Retrieved February 10,
2010. http://www.xdstrategy.com/2008/10/28/desirability_studies
75
76. User Focus. "Measuring satisfaction: Beyond the Usability Questionnaire."
Retrieved February 10, 2010.
http://www.userfocus.co.uk/articles/satisfaction.html
UserEffect. "Guide to Low-Cost Usability Tools." Retrieved May 12, 2010.
http://www.usereffect.com/topic/guide-to-low-cost-usability-tools
Tullis, Thomas and Jacqueline Stetson. “A Comparison of Questionnaires for
Assessing Website Usability.” Usability Professionals’ Association Conference,
2004.
home.comcast.net/~tomtullis/publications/UPA2004TullisStetson.pdf
Westerman, S. J., E. Sutherland, L. Robinson, H. Powell, and G. Tuck. “A
Multi-method Approach to the Assessment of Web Page Designs.”
Proceedings of the 2nd international conference on Affective Computing and
Intelligent Interaction, 2007.
http:// portal.acm.org/citation.cfm?id=1422200
76
77. Five Second Test
http://fivesecondtest.com/
Feedback Army
http://www.feedbackarmy.com
Wordle
http://www.wordle.net
PrEmo
http://www.premo-online.com
77