Benchmarking allows you to track UX progress over time, giving you an indication of how successful digital platform changes have been. It provides a holistic product assessment and requires greater attention to methodology, stakeholders, tasks, protocol, and analysis than a typical feature study does. Our method captures quantitative measures, along with qualitative feedback, for product stakeholders to use to justify and inform their business decisions.
In this session, you’ll get tips for developing a benchmark strategy. You’ll also hear stories about how benchmarks have impacted our organization’s digital strategy.
You will learn:
The business impact of benchmark studies
Designing, running, and analyzing such studies
How to avoid issues with recruiting, study design, execution, reporting
A variety of UX and product professionals, including seasoned researchers, novice designers, and digital product owners can learn and take action from this session.
UXPA2019 I am the LAAW! The Lean Accessibility Audit WorkshopUXPA International
Fitting accessibility into an agile development cycle can be challenging. Often accessibility specialists are spread thin across agile squads and they have to deliver quickly into multiple sprint cycles.
To meet the demand I looked to Lean UX principles and developed The Lean Accessibility Audit Workshop (LAAW). LAAW has two goals 1) detect accessibility problems quickly and 2) spread the accessibility knowledge. This is accomplished by training members of agile squads on accessibility basics, evaluation methods and tools. The training evolves into an accessibility audit as squad members collaboratively capture, share and prioritize findings. The LAWW method compresses a 6-8 week evaluation process into a 2 weeks process while training squad members to detect and avoid accessibility issues in the future. And it can be pretty fun.
This session will explain how to bring LAAW to your organization.
UXPA2019: Voice User Interface Design for Customer Support: Design guidance ...UXPA International
This talk from UXPA 2019 focused on the challenges, lessons learned, and best practices in designing voice user interfaces for customer support reinforced by three recent user studies. It explored both high and low level design issues. It also discussed methodological techniques for testing voice user interfaces including creating flow diagrams for prototypes and conducting usability evaluations of these designs with users. This was an informative session especially for those who are beginning their journey in designing and testing voice user interfaces. The session included audience participation in a VUI design exercise.
UXPA2019 9 Simple Strategies for Designing Inclusive ExperiencesUXPA International
Implementation of Accessibility guidelines by following WCAG or Section 508 might be perceived as a daunting task. WCAG 2.0 pertains 3 levels and 4 principles with 61 guidelines. WCAG 2.1 became effective from 2018 June with 17 more guidelines, totalling 78 guidelines to design experience for all abilities. This presentation will simplify the complex WCAG and 508 standards into 9 simple strategies to design Inclusive Experience.
UXPA2019 Enhancing the User Experience for People with Disabilities: Top 10 ...UXPA International
An estimated 1.3 billion people globally report limitations in their daily activities due to a disability. When it comes to the physical world, businesses have made progress in accommodating customers with disabilities. But in the digital world, websites lack basic accessibility features such as text alternatives describing images, proper heading level structures so individuals who are blind and use screen readers can understand the content on a webpage, or captioning for multimedia content for individuals who are deaf or are hard of hearing – let alone assistive technology for customers who have trouble using mobile devices due to dexterity limitations that arise from a variety of conditions.
In this session, attendees will:
* Understand people with disabilities (PWDs) and how they use the web
* Learn about common barriers, issues and solutions
* Discover the different testing methodologies and their interdependencies
* Uncover ROI
UXPA2019 Optimal AR UX for Complex Purchases — How immersive technology boost...UXPA International
Augmented Reality for eCommerce is everywhere. Major retailers and Shopify have mainstreamed 3D. But so far, nearly all product shoppers do is simply “see this in their room.” For complex, configurable, personalized purchases, this isn’t enough.
This session focuses on effective AR uses that increase user success with planning and decision-making. Think of projects such as a kitchen redesign — design aesthetics, myriad features/options, physical characteristics, and lack of buyer knowledge all stand in the way.
I’ll discuss wide-ranging aspects of AR’s potential and provide a framework for planning product-focused applications. I’ll share lots of examples and insights from recent projects, plus others I’ve found along the way, including UX principles for image-based visualizers and configurators refined over 2 decades. This knowledge with help spur ideas for your own projects.
Going beyond, I’ll align user expectations with present and future capabilities of 3D platforms/engines/hardware, giving you a working knowledge for the next generation of 3D: Mixed- and eXtended-Reality.
UXPA2019 Not Your Average Chatbot: Using Cognitive Intercept to Improve Infor...UXPA International
This presentation from UXPA 2019 will review cognitive intercept as pertains to search, and how it extends to an additional domain (live agent chat). Evidence that it helps users and lowers help desk volumes will be discussed.
The population of the developed world is aging. Most websites, apps, and digital devices are used by adults aged 50+ as well as by younger adults, so they should be designed accordingly. This talk, based on the presenter’s recent book, presents age-related factors that affect older adults’ ability to use digital technology, as well as design guidelines that reflect older adults’ highly varied capabilities, usage patterns, and preferences. Features:
• demographics of users of digital technology, by age,
• age-related factors affecting ability to use computers and online services,
• common design problems that decrease usability for older adults,
• design guidelines that can help designers avoid these common pitfalls.
UXPA2019 I am the LAAW! The Lean Accessibility Audit WorkshopUXPA International
Fitting accessibility into an agile development cycle can be challenging. Often accessibility specialists are spread thin across agile squads and they have to deliver quickly into multiple sprint cycles.
To meet the demand I looked to Lean UX principles and developed The Lean Accessibility Audit Workshop (LAAW). LAAW has two goals 1) detect accessibility problems quickly and 2) spread the accessibility knowledge. This is accomplished by training members of agile squads on accessibility basics, evaluation methods and tools. The training evolves into an accessibility audit as squad members collaboratively capture, share and prioritize findings. The LAWW method compresses a 6-8 week evaluation process into a 2 weeks process while training squad members to detect and avoid accessibility issues in the future. And it can be pretty fun.
This session will explain how to bring LAAW to your organization.
UXPA2019: Voice User Interface Design for Customer Support: Design guidance ...UXPA International
This talk from UXPA 2019 focused on the challenges, lessons learned, and best practices in designing voice user interfaces for customer support reinforced by three recent user studies. It explored both high and low level design issues. It also discussed methodological techniques for testing voice user interfaces including creating flow diagrams for prototypes and conducting usability evaluations of these designs with users. This was an informative session especially for those who are beginning their journey in designing and testing voice user interfaces. The session included audience participation in a VUI design exercise.
UXPA2019 9 Simple Strategies for Designing Inclusive ExperiencesUXPA International
Implementation of Accessibility guidelines by following WCAG or Section 508 might be perceived as a daunting task. WCAG 2.0 pertains 3 levels and 4 principles with 61 guidelines. WCAG 2.1 became effective from 2018 June with 17 more guidelines, totalling 78 guidelines to design experience for all abilities. This presentation will simplify the complex WCAG and 508 standards into 9 simple strategies to design Inclusive Experience.
UXPA2019 Enhancing the User Experience for People with Disabilities: Top 10 ...UXPA International
An estimated 1.3 billion people globally report limitations in their daily activities due to a disability. When it comes to the physical world, businesses have made progress in accommodating customers with disabilities. But in the digital world, websites lack basic accessibility features such as text alternatives describing images, proper heading level structures so individuals who are blind and use screen readers can understand the content on a webpage, or captioning for multimedia content for individuals who are deaf or are hard of hearing – let alone assistive technology for customers who have trouble using mobile devices due to dexterity limitations that arise from a variety of conditions.
In this session, attendees will:
* Understand people with disabilities (PWDs) and how they use the web
* Learn about common barriers, issues and solutions
* Discover the different testing methodologies and their interdependencies
* Uncover ROI
UXPA2019 Optimal AR UX for Complex Purchases — How immersive technology boost...UXPA International
Augmented Reality for eCommerce is everywhere. Major retailers and Shopify have mainstreamed 3D. But so far, nearly all product shoppers do is simply “see this in their room.” For complex, configurable, personalized purchases, this isn’t enough.
This session focuses on effective AR uses that increase user success with planning and decision-making. Think of projects such as a kitchen redesign — design aesthetics, myriad features/options, physical characteristics, and lack of buyer knowledge all stand in the way.
I’ll discuss wide-ranging aspects of AR’s potential and provide a framework for planning product-focused applications. I’ll share lots of examples and insights from recent projects, plus others I’ve found along the way, including UX principles for image-based visualizers and configurators refined over 2 decades. This knowledge with help spur ideas for your own projects.
Going beyond, I’ll align user expectations with present and future capabilities of 3D platforms/engines/hardware, giving you a working knowledge for the next generation of 3D: Mixed- and eXtended-Reality.
UXPA2019 Not Your Average Chatbot: Using Cognitive Intercept to Improve Infor...UXPA International
This presentation from UXPA 2019 will review cognitive intercept as pertains to search, and how it extends to an additional domain (live agent chat). Evidence that it helps users and lowers help desk volumes will be discussed.
The population of the developed world is aging. Most websites, apps, and digital devices are used by adults aged 50+ as well as by younger adults, so they should be designed accordingly. This talk, based on the presenter’s recent book, presents age-related factors that affect older adults’ ability to use digital technology, as well as design guidelines that reflect older adults’ highly varied capabilities, usage patterns, and preferences. Features:
• demographics of users of digital technology, by age,
• age-related factors affecting ability to use computers and online services,
• common design problems that decrease usability for older adults,
• design guidelines that can help designers avoid these common pitfalls.
User Interface Design
User Centred Design and principles, Iterative Design, User research, Building Personas, Design studio method, Prototyping basics and tools, Paper prototyping, Usability testing
Going from Here to There: Transitioning into a UX Careerdpanarelli
A lot of people are curious about transitioning into the field of User Experience Design (UX). In this talk, I talk about a few different ways that you can transition into a UX career, be it grad school, night classes, or the ol' school of hard knocks, backed up by case studies. This talk was given at NoVA UX Meetup in the offices of AddThis, hosted by organizer Jim Lane.
Introduction to user experience design + usability
Describe the field of UX + how it relates to other disciplines
Identify the different roles within UX + the responsibilities of each
UX Process: Traditional [“Waterfall”], Agile, Lean
Learn to conduct UX research
Introduction to user experience documentation + deliverables + software
Learn about personas, user flows, sitemaps, wireframes
Determine when to use which documentation
Discover new tools and frameworks for creating deliverables
Introduction to usable web forms
Identify the different elements of forms and how to use them effectively
Learn what makes a strong user experience with forms
Identify expected outcomes
Curated list of UX resources
Recommended blogs, books, experts to follow, companies of note, local organizations and recommended conferences
The JoomlaChicago Loop sponsored "Joomla & Responsive Design", a presentation focused on the key ingredients and dynamics of making a Joomla website flow and react to the different viewing devices and browser viewport sizes.
Dennis Kmetz (Director of Interactive Media, Taylor Bruce Design Partnership) presented Joomla & Responsive Design on Thursday, March 1, 2012.
Slides from a talk I did at Web Directions South in Sydney Oct 2009.
Outline:
Designing for dynamic web applications and mobile devices poses a new set of challenges. Web designers are increasingly being asked to apply their skills to where the page model no longer applies. We need new ways of exploring the user experience and communicating behaviours involving sub-page changes and movement.
Enter rapid prototyping. Widely acclaimed as one of the best ways to create great user experiences, it isn't without it's own pitfalls. This session will discuss the pros and cons of different prototyping techniques, and introduce a new technique called "screenflows" that focuses on visualising the user experience.
Discover how to combine the best of paper prototyping, wireframes and HTML prototyping into one simple and effective prototyping technique. Learn how using this method can dramatically decrease the need for documentation, while increasing the speed and agility of the development process.
User Experience Basics for Product ManagementRoger Hart
User Experience (UX) has matured as a discipline and radically changed how products are delivered. It touches workflows, usability, customer needs, and of course visual design and UI. Product managers can't ignore it, even if they want to... and if they want to, they're probably wrong. The tools of User Experience can help us get closer to our customers and differentiate our products.
In this three hour workshop I present an introduction to the UCD process, an overview of the basic technologies of the web and a survey of current Mobile Web Design trends.
Designing user experience (ux) for digital productsVijay Morampudi
User experience design isn’t just moving pixels; it’s much bigger than solely the user interface (UI). You should start considering the entire customer experience: the full life-cycle of your customer’s experience across every channel, digital and non-digital. Evaluate every touch point, and redesign each one as necessary to meet your customer’s needs. The theme of this talk is how to define User Experience (UX) for digital products
Key takeaways
• Applying Design Thinking to UX
• From touch points to end-to-end experiences
• User research and Analytics to identify Personas and pain points
• Journey mapping
• Wireframing from lo-fi to hi-fi
• Usability and A/B testing
Basic principles of Usable Interface
Learnability, Efficiency, Memorability, Errors, Satisfaction
Utility - does it what you need?
How to improve Usability
Excelling in the User Experience Economy of Today and TomorrowUserZoom
User Experience is a fast-paced, dynamic, and multi-faceted field. How do you keep up on everything that’s important to your organization, let alone get ahead of the industry curve?
Dean Barker will look at the convergence of and predictions for emerging and likely trends in technology and the UX/Usability field. He’ll discuss what it means for User Experience professionals and the best focus for our careers in the near future.
Usability and User Experience Training Seminarlabecvar
This presentation describes a day-long seminar for giving participants an overview of best practices in usability design and research. Also included are several hand-on exercises to be done throughout the day to solidify participants' understanding of course concepts.
UserZoom Education Series - Research Deep Dive - Advanced - Task-Based TOL (b...UserZoom
Today you will learn about benchmarking studies. Specifically, you will learn how to conduct a competitor benchmarking study on live websites.
After this session you will be able to:
Know when to use benchmarking
Know the type of data you can collect with benchmarking
Create a benchmarking study in UserZoom
Interpret the results of a benchmark
How to Conduct UX Benchmarking Studies Your Own Site Over Time + Competitors ...UserZoom
In this webinar on-demand, Ann Rochanayon, Director of UX/CX Research at UserZoom, will guide you on how to measure and manage the user experience by outlining the essential elements of successful UX Benchmarking.
User Interface Design
User Centred Design and principles, Iterative Design, User research, Building Personas, Design studio method, Prototyping basics and tools, Paper prototyping, Usability testing
Going from Here to There: Transitioning into a UX Careerdpanarelli
A lot of people are curious about transitioning into the field of User Experience Design (UX). In this talk, I talk about a few different ways that you can transition into a UX career, be it grad school, night classes, or the ol' school of hard knocks, backed up by case studies. This talk was given at NoVA UX Meetup in the offices of AddThis, hosted by organizer Jim Lane.
Introduction to user experience design + usability
Describe the field of UX + how it relates to other disciplines
Identify the different roles within UX + the responsibilities of each
UX Process: Traditional [“Waterfall”], Agile, Lean
Learn to conduct UX research
Introduction to user experience documentation + deliverables + software
Learn about personas, user flows, sitemaps, wireframes
Determine when to use which documentation
Discover new tools and frameworks for creating deliverables
Introduction to usable web forms
Identify the different elements of forms and how to use them effectively
Learn what makes a strong user experience with forms
Identify expected outcomes
Curated list of UX resources
Recommended blogs, books, experts to follow, companies of note, local organizations and recommended conferences
The JoomlaChicago Loop sponsored "Joomla & Responsive Design", a presentation focused on the key ingredients and dynamics of making a Joomla website flow and react to the different viewing devices and browser viewport sizes.
Dennis Kmetz (Director of Interactive Media, Taylor Bruce Design Partnership) presented Joomla & Responsive Design on Thursday, March 1, 2012.
Slides from a talk I did at Web Directions South in Sydney Oct 2009.
Outline:
Designing for dynamic web applications and mobile devices poses a new set of challenges. Web designers are increasingly being asked to apply their skills to where the page model no longer applies. We need new ways of exploring the user experience and communicating behaviours involving sub-page changes and movement.
Enter rapid prototyping. Widely acclaimed as one of the best ways to create great user experiences, it isn't without it's own pitfalls. This session will discuss the pros and cons of different prototyping techniques, and introduce a new technique called "screenflows" that focuses on visualising the user experience.
Discover how to combine the best of paper prototyping, wireframes and HTML prototyping into one simple and effective prototyping technique. Learn how using this method can dramatically decrease the need for documentation, while increasing the speed and agility of the development process.
User Experience Basics for Product ManagementRoger Hart
User Experience (UX) has matured as a discipline and radically changed how products are delivered. It touches workflows, usability, customer needs, and of course visual design and UI. Product managers can't ignore it, even if they want to... and if they want to, they're probably wrong. The tools of User Experience can help us get closer to our customers and differentiate our products.
In this three hour workshop I present an introduction to the UCD process, an overview of the basic technologies of the web and a survey of current Mobile Web Design trends.
Designing user experience (ux) for digital productsVijay Morampudi
User experience design isn’t just moving pixels; it’s much bigger than solely the user interface (UI). You should start considering the entire customer experience: the full life-cycle of your customer’s experience across every channel, digital and non-digital. Evaluate every touch point, and redesign each one as necessary to meet your customer’s needs. The theme of this talk is how to define User Experience (UX) for digital products
Key takeaways
• Applying Design Thinking to UX
• From touch points to end-to-end experiences
• User research and Analytics to identify Personas and pain points
• Journey mapping
• Wireframing from lo-fi to hi-fi
• Usability and A/B testing
Basic principles of Usable Interface
Learnability, Efficiency, Memorability, Errors, Satisfaction
Utility - does it what you need?
How to improve Usability
Excelling in the User Experience Economy of Today and TomorrowUserZoom
User Experience is a fast-paced, dynamic, and multi-faceted field. How do you keep up on everything that’s important to your organization, let alone get ahead of the industry curve?
Dean Barker will look at the convergence of and predictions for emerging and likely trends in technology and the UX/Usability field. He’ll discuss what it means for User Experience professionals and the best focus for our careers in the near future.
Usability and User Experience Training Seminarlabecvar
This presentation describes a day-long seminar for giving participants an overview of best practices in usability design and research. Also included are several hand-on exercises to be done throughout the day to solidify participants' understanding of course concepts.
UserZoom Education Series - Research Deep Dive - Advanced - Task-Based TOL (b...UserZoom
Today you will learn about benchmarking studies. Specifically, you will learn how to conduct a competitor benchmarking study on live websites.
After this session you will be able to:
Know when to use benchmarking
Know the type of data you can collect with benchmarking
Create a benchmarking study in UserZoom
Interpret the results of a benchmark
How to Conduct UX Benchmarking Studies Your Own Site Over Time + Competitors ...UserZoom
In this webinar on-demand, Ann Rochanayon, Director of UX/CX Research at UserZoom, will guide you on how to measure and manage the user experience by outlining the essential elements of successful UX Benchmarking.
UserZoom Webinar: How to Conduct Web Customer Experience BenchmarkingUserZoom
You can't manage what you can't measure, so... How do you actually measure user experience?
In this webinar we covered what, why, and how to conduct website user experience & usability benchmarking. We discussed how to effectively measure the quality of a website's user experience across various competitors, within one industry, across time, using an online quantitative research methodology commonly referred to as "unmoderated remote usability testing."
My presentation at 24 hours of UX
Links and special mentions
Medium posts:
https://uxdesign.cc/measuring-the-perceived-usability-of-a-system-using-the-system-usability-scale-3418971dd7a3 - Katerina Maniataki
https://uxdesign.cc/measuring-and-quantifying-user-experience-8f555f07363d - Matej Latin
Books:
Designing with Data - Rochelle King, Elizabeth F. Churchill, Caitlin Tan
Measuring the User Experience - Tom Tullis, Bill Albert
Alfonso de la Nuez's talk, "How to conduct global UX benchmarking", at BigDesign event, about what, why, and how to conduct website user experience & usability benchmarking.
Outranking Your Competition with UX Benchmark Studies | UserZoomUserZoom
Knowing how your competitors’ web and mobile sites are doing is a huge part of running a successful online business. However, gathering competitive intelligence around the quality of User Experience can be quite challenging.
Not anymore! Unmoderated remote UX testing solutions like UserZoom can help you easily benchmark Key Performance Indicators on competitors' websites against your own, all without ever having to set up complicated tracking. It’s as easy as knowing your competitors’ domain name or URL.
Discover:
-WHY and HOW to conduct competitive UX Benchmarking
-WHAT metrics & key performance indicators you can measure
-WHEN to conduct UX benchmark studies over time
-A sneak-peek into UserZoom’s newest UX Benchmarking solution: Xperience360
I taught a class titled "You Don't Know C.R.A.P. about UX/UI" for SkillShare Philadelphia on 8/23/11. For more information on the class visit: http://www.skillshare.com/You-Dont-Know-CRAP-about-UX-UI/1632896614/
User experience (UX) is the basis for all Web activity, and thus underpins everything we do in Web design and development. Successful projects bake UX in from the ground up, from discovery through planning, iteration, testing and deployment. No matter how beautiful our code may be, of what use is it if it’s irrelevant to our users?
Advocating for your users is key to project success. Kirsten Burgard and I show how, even developers can accomplish this via our process and case studies.
How does usability relate to the role of a Business Analyst or a Business Analyst Manager
Trent Mankelow from Optimal Usability looks at the two and asks - "Can't we all be friends?"
Descriptions of UX Design projects and the design thinking behind them. The foundational work includes personas, task analysis, user stories, user goals, and problem statements. Later steps include wireframing, mockups and prototypes constructed in tools like Axure, Balsamiq, Sketch. or Figma. User Research and Usability test reports appear in several projects.
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...UXPA International
Imagine creating experiences for your rookie designers’ first couple years that are rewarding, enriching, and full of learning — without taking all your time or energy to manage. We’ll share techniques any team leader can put into practice using real-life examples from associate programs, apprenticeships, and internships.
Topics include onboarding, varied work challenges, developing multiple capabilities, buddy systems, group sharing, guest speakers, time with executives, and mentorship. We’ll also share how to operationalize learning, soft skills like communication and collaboration, setting boundaries, time management, achieving deep work, and more skills we all wish we were explicitly taught early on.
We’ll focus on modern-day associate programs, but even if you can’t create a full-fledged program, you’ll leave this session with ideas to use with your fledgling professionals. The benefits go beyond efficiency; it’s a foundation for culture, camaraderie, autonomy, and mastery.
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...UXPA International
Digital advances are being made at a rapid-fire pace, yet disability inclusivity continues to fall short of the digital revolution. As the number of people living with disabilities rises, the time to take digital accessibility to the next level is now. Let’s disrupt inaccessibility together! Come hear about a multi-part discovery research and ideation project informing foundational UX designs for our customers. You’ll get insights from our unique study, which are widely applicable across industries, and walk away with tips and inspiration to kick off your own accessibility-focused discovery and ideation. Only YOU can prevent inaccessibility – are you in?
User experience can be drastically elevated by combining data science insights with user-based insights from research. Data analytics on its own can make themes and correlations difficult to explain and to provide accurate recommendations. For example, themes identified via large global surveys and usage data can be better understood with UX insights from focused user research, such as user interviews and/or cognitive walkthroughs. This presentation will highlight the complimentary nature of data science and UX and will focus on the benefits of bringing the two disciplines together. This will be buttressed with practical examples of enterprise projects and applications that combined data and skills from the two disciplines, guidance on how the two disciplines can better work together, and the skills needed to improve as a UX professional when working with data science teams.
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden InsightsUXPA International
Users do not always accurately describe what they mean or feel. There are many reasons for this, ranging from politeness to poor introspection, to lack of sufficient technical vocabulary. Fortunately, UX researchers have tools in their trade to deduce what was really meant. We call this UX Fracking, a mixed methods approach that is optimized for extracting hidden user insights. We will illustrate the dangers of inadequate, superficial research, and how this may lead to outcomes incapable of addressing the users’ core issues. We will explore ways to avoid these pitfalls by leveraging mixed research methods to test hypotheses about the users’ intent and needs. This starts with a thorough understanding of who the user is, their goals, and how they work today, to an approach that combines surveys, interviews, and comment analysis with behavioral observation, and finally, validating the newly discovered user insights with the users themselves.
UXPA 2023: Learn how to get over personas by swiping right on user rolesUXPA International
This session walks through the concept of user roles as an alternative to personas as a means to generate and disseminate user insights for product development teams. We will describe the tools and methods used to create a research database organized by user roles, along with examples and short exercises to help attendees think through user roles within their own context.
By the end of the session, attendees should be aware of tools and approaches for:
Organizing user research information in a database
Disseminating user role information to product and design teams
Managing a user roles database as part of a long term UX Research program
If you’re ready to ditch personas but don’t know how, this session is for you!
We will present a case study that details our approach for replacing user personas with user roles for a multi-national SAAS company. We will take the audience on a journey that starts with an executive request for personas, travels through the tribulations of realizing personas suck, and concludes with convincing others to accept a new and innovative way to understand the people who use the product. Our key message is that personas lack real value for organizations that already understand the importance of empathizing with users. Building user-centered products requires easily accessible and well organized user insights. We will discuss defining users through a process of stakeholder consultation and content review, and structuring data around Jobs to Be Done and product interactions. We will also discuss the dissemination of user roles in our organization using relational databases, interactive dashboards and online wikis. Spoiler alert, our stakeholders loved user roles!
UXPA 2023: Experience Maps - A designer's framework for working in Agile team...UXPA International
Agile Methodology refers to software design and development methodologies centered around the idea of iterative design and development, where requirements and concepts evolve through collaboration between self-organizing cross-functional teams. Thus, Agile enables teams to deliver value faster, with greater quality and predictability, and greater aptitude to respond to change. With evolving product features every design sprint, designers & researchers find it difficult to follow the design process. This sometimes leads to designs delivered in haste or sub-par design artifacts which result in UX debt. UX debt is accumulated when design teams take actions or shortcuts to expedite the delivery of a piece of functionality or a project which later needs to be refactored. It is the result of prioritizing speedy delivery of design to the development team over a perfect experience journey. Experience Maps is a great tool to practice UX in Agile as well as manage UX Debt.
UXPA 2023: UX Enterprise Story: How to apply a UX process to a company withou...UXPA International
How to build a UX Department from scratch, in an environment they think UX people do social media posters and posts! An agile implementation just started, and people are moving from a waterfall and ad-hoc mindset to agility. In this session, I will talk about my Journey to establish a UX Department for a company that is part of a global brand, but this local branch just started the digital transformation movement. Challenges like: spreading awareness and educating people about UX, hiring the right team, defining the right team structure, establishing workflow and day-to-day operations, and applying localization (non-western culture).
UXPA 2023: High-Fives over Zoom: Creating a Remote-First Creative TeamUXPA International
I started my current job in March of 2020. Many of us remember something clearly about the month that COVID started to shut things down. I remember being surprised to hear that my new on-site-only job would be starting in my living room over zoom. How do you lead a design team when none of the team members live near each other and creativity is highly collaborative? Taking from over a decade of working in HR software, I knew whatever I did needed to put people first. That what employees love about a job is often deeper than the work, it’s the culture, the relationships and people they work with. It’s the feeling that their work has value, and their contribution matters. In this talk I will walk though some of the rituals and best practices I have learned over the last two years building a remote-first creative team.
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & ...UXPA International
As humans, we are biased by design. Our intricate and fascinating brains have developed shortcuts through centuries of human evolution. They reduce an unimaginable load of paralyzing decisions, keep us alive, and help us navigate this complex world. Now, these life saving biases affect how we behave with modern technology. Understanding some of the theories and reasons why these biases exist is the key to unlocking their power. In this workshop we will cover some theories around how the brain works. We will review some of our mental shortcuts, take a look at some common biases, and learn how they affect our users, our research, and our designs. Lastly we will review some advantages of biases, and ways to identify and reduce bias. This workshop is targeted for designers who do their own research, and researchers looking to learn more about removing bias from their studies.
UXPA 2023 Poster: Improving the Internal and External User Experience of a Fe...UXPA International
UXPA 2023 Poster: Improving the Internal and External User Experience of a Federal Government Legacy Application Using User Experience and Agile Principles
Are you new to UX management, or thinking of getting into management? Then this talk is for you. After reading countless books, attending countless trainings, mentoring and being menteed, nothing quite prepared me for management like my first year. I’ll share with you what I wish they’d told me. I’ll also share my process for generating team research roadmaps, establishing team values, keeping employees motivated, and not burning out.
UXPA 2023: Redesigning An Automotive Feature from Gasoline to Electric Vehicl...UXPA International
Join us for an interaction design case study from the automotive industry. We created a Human-Machine Interface (HMI) for a vehicle feature that provides household-levels of power in electrical outlets for our customers to use at work and play. This case study will reveal: · Our debate of re-using version 1.0’s HMI vs designing a new user interface for the electric vehicle—when to break with consistency and why? · User research we conducted to guide our early design concept. · Paper prototypes we created to support our usability testing of the concept with vehicle owners. · How we solved internal debate over the interaction design in moving from internal combustion vehicles to electric vehicles. * Advice to help you evangelize user-centered design that is also brand-centered for a new product.
Collapsing Narratives: Exploring Non-Linearity • a micro report by Rosie WellsRosie Wells
Insight: In a landscape where traditional narrative structures are giving way to fragmented and non-linear forms of storytelling, there lies immense potential for creativity and exploration.
'Collapsing Narratives: Exploring Non-Linearity' is a micro report from Rosie Wells.
Rosie Wells is an Arts & Cultural Strategist uniquely positioned at the intersection of grassroots and mainstream storytelling.
Their work is focused on developing meaningful and lasting connections that can drive social change.
Please download this presentation to enjoy the hyperlinks!
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
UXPA2019 Building Your Benchmark: How to Measure UX for Product Impact Over Time
1. Building Your Benchmark
How to Measure UX
for Product Impact Over Time
Jennifer Otto | Paul Sisler | Yina Li Turchetti
UXPA
2 0 1 9
2. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Welcome
Yina Li Turchetti
Senior UX Researcher
Autodesk
Jennifer Otto
Director, UX Research Ops
Fidelity Investments
Paul Sisler
Senior UX Researcher
Pearson Education
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
3. 1. Benchmark Basics
2. Method
3. Cadence
4. Recruiting
5. Study Design
6. Analysis
7. Reporting
Contents
Photo by Kimberly Farmer on Unsplash
4. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
What Is a Benchmark?
Test end-to-end system, site, or app
Get an overall product pulse (not to be
for a deep understanding of one area)
Repeatable and run on a regular
cadence
Measure how one change affects
overall performance
Compare to previous
performance and competitors’
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
*Sample data.
9.0
25.0
29.0
22.9
49.7 50.2 49.0 46.6
-10
0
10
20
30
40
50
60
70
2012 2013 2014 2015 2016 2017 2018 2019
NPS for iOS Mobile App
5. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
2012
2014
2016
2018+
A History of Our Benchmarks
Mobile app
baseline
Standardized
study design
across platforms
Refined metrics
and investigated
automation
opportunities
Expanded scope to
include desktop
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
6. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Our Approach
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
100-200 participants per platform
10 most common tasks per
platform, including some pre-login
and post-login
Participant demographics line up
with site and app visitors
Run at year-end
Quantitative and Qualitative task-
based measures
7. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Benchmark Measurements
Metrics are compared YoY and platform to platform
Task success Ease of use
Time on task SUS
system usability score
NPS
net promotor score
Meets Needs
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
8. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Why We Perform Benchmarks
Product Strategy
Competition
Market Research
Customer Feedback
(attitudinal)
Analytics (behavioral)
Benchmarking
attitudinal & behavioral
Full Website task
success
= 92.4%
Global navigation
changes
Feature testing
went well
Full Website
Benchmark
task success
= 75.9%
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
10. Repeatable
A future team member can execute the
same protocol on a new version
Measurable
Produces quantifiable observations
Comparable
Results in findings suitable for
comparison across rounds
Plays to Team Forte
Relies on familiar approach
Method
Photo by Hans Reniers on Unsplash
11. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Method
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Pilot - Qualitative
Unmoderated online video
Study design, tools issues, wording
Core - Quantitative
Task-based online
Success, time, task ease, SUS/NPS
Insights - Qualitative
Unmoderated online video
Insight and stories to explain core
1.
2.
3.
12. Follow a recurring cycle
that respects your release
schedule and rhythm of
your industry with enough
time to observe change.
Cadence
Photo by Wayne Bishop on Unsplash
13. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Cadence
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Considerations Investing Education
Time to observe changes Once per year Academic year
Unpredictable times of year Tax season Finals and breaks
Team bandwidth November to December Summer breaks
Release schedule Skipped year for redesign Back-to-school
Unexpected events Market changes Educational policy
14. Stay in field for as little
time as possible to limit
effects of unplanned
updates, current events,
and other factors out of
your control.
Time
in Field
Photo by Bruno Martins on Unsplash
15. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Time in Field
Impacts on time
Scope and number of tasks Sample size, complexity, and incident rate
Number of platforms Recruit process (direct, panel)
Changes from previous rounds of study Study requirements (log in to account)
Size and experience of team Technical issues (product, tools, vendors)
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
16. Match your sample to users by
platform using info from
analytics and other sources.
Aim for consistency, and plan
for subtle demographic changes
over time.
Recruiting
Photo by Jacek Dylag on Unsplash
17. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Recruiting
Guidelines Experience
Consistency
Same screener round-to-round
Sample changes as user demographics
changes
Actual users
(not prospects, co-workers, your mom)
Customer panel and a third-party recruiter
Significant results
About 150 for each task per platform and
per segment
About 300 mobile app users on two
platforms and 300 web (on computer)
Clean data
Prepare for cheaters and speeders
Removed 30% of sample after closing
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
18. Test a core of common and
critical tasks likely to remain
important over time.
Cover the breadth of your
information architecture.
What
To Test
Photo by Sherman Yang on Unsplash
19. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
What to Test
Guidelines Experience
Same tasks every time Tasks evolve with team and site/app
Same tasks across platforms Core tasks and platform specific tasks
Same tasks as possible with competitor 2-3 competitors focused on our main services
Shortest reasonable test time
Limit to 10 minutes - Adjust tasks-per-participant
and increase sample size
Randomly assign tasks if you have too many Login and logout are first and last
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
21. Repeatable
A future team member can execute the
same protocol on a new version
Measurable
Produces qutifiable observations
Comparable
Results in findings suitable for
comparison across rounds
Plays to Team Forte
Relies on familiar approach
TitleDefine
Task
Goals
Identify your goals before
creating your tasks.
Photo by Christian Widell on Unsplash
22. Task goal example
How well does our target
audience find and
understand the general
market performance?
Task examples
Option 1:
According to [Platform], is the
NASDAQ up or down today?
Option 2:
According to [Platform], what
is the current value of Dow
Jones?
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
23. Repeatable
A future team member can execute the
same protocol on a new version
Measurable
Produces qutifiable observations
Comparable
Results in findings suitable for
comparison across rounds
Plays to Team Forte
Relies on familiar approach
Title
Be
Sensitive
to Data
Privacy
Have a way for participants to
opt-out of the study if they are
not comfortable continuing, due
to potential sensitivity of their
data.
Photo by Markus Spiske on Unsplash
24. Situations
Your study requires people
to log in to their own
accounts to complete
some tasks which may
make some
uncomfortable
Opt-out Examples
Study level:
Are you willing to log in to
your account in your web
browser on a computer and go
through a few tasks for this
study?
Task level:
Find your account with the
smallest balance. What is its
current value?
Provide ranged answers and
an option of Prefer Not to Say
Your study requires people
to log in to their own
accounts to complete
some tasks which may
make some
uncomfortable
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
25. Repeatable
A future team member can execute the
same protocol on a new version
Measurable
Produces qutifiable observations
Comparable
Results in findings suitable for
comparison across rounds
Plays to Team Forte
Relies on familiar approach
TitleUse
Verifiable
Tasks
Use verifiable answers if you
can.
Self-report scores tend to
introduce noise and produce
inflated results.
Photo by Fleur on Unsplash
26. Type
Self-report –
Likert:
Validation Questions
How confident are you that you
have completed the task successfully?
1 (Not confident at
all) – 7 (Extremely
confident)
According to [Platform], is the NASDAQ up or down today?
Answers
Self-report –
binary:
Semi-verifiable:
Verifiable:
Did you complete this task
successfully?
Is the NASDAQ up or down?
What is NASDAQ at today?
Yes | No | I don’t
know
Up | Down | I don’t
know
[Text entry]
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
27. Repeatable
A future team member can execute the
same protocol on a new version
Measurable
Produces qutifiable observations
Comparable
Results in findings suitable for
comparison across rounds
Plays to Team Forte
Relies on familiar approach
Title
Consider
Measurable
vs Realistic
Tasks
Balance realistic task design and
verifiable answers.
Sometimes, you may have to
choose between a more realistic
task which only allows for self-
reported answers versus a less
realistic task which allows for
measurable results.
Photo by Ali Abdul Rahman on Unsplash
28. Task goal
Can participants
locate news
relevant to a
stock?
Task examples
More realistic:
Find a news article about [company
X]. Were you able to complete this task?
Self-report: Y/N
More measurable:
Find the most recent article on [platform]
about [company X]. What is the first word
of the article?
Verifiable: [write-in]
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
29. Repeatable
A future team member can execute the
same protocol on a new version
Measurable
Produces qutifiable observations
Comparable
Results in findings suitable for
comparison across rounds
Plays to Team Forte
Relies on familiar approach
TitleIncrease
Data
Integrity
Use open-ended answer format
over multiple choice options to
curb cheating behavior.
Photo by Jan Kahánek on Unsplash
30. Task
Find the most
recent price for
Apple stock
(AAPL).
Task examples
Easy to grade:
Below $150
Between $151 and $190
Between $191 and $230
Between $231 and $270
Above $271
I don’t know
Discourage cheating:
[Open ended text answer]
I don’t know
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
39. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Where Are We Now
Creating a measurement
framework similar to
Google’s HEART
Monitoring on an on-
going basis
Planning baseline of
existing products to
precede launch of new
learning platform.
Evaluating scope:
academic subjects, tasks,
number of participants.
Identifying personnel
with skills and bandwidth.
Continuing to run the
benchmark annually
Exploring ways to
automate benchmarks of
experiences
Building a data storage
component in order to
create dashboards
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
40. UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
In Summary
Consider your
research goals
when developing
tasks
Leverage data
from both
qualitative and
quantitative task-
based methods
Take time when
developing your
study design
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
41. Questions?
Keep in touch!
Jennifer Otto: jennifer.otto@fmr.com
Paul Sisler: paul.sisler@pearson.com
Yina Li Turchetti: yina.turchetti@autodesk.com
UXPA
2 0 1 9
UX Benchmarking ■ #UXPA19 ■ #UXBenchmark
Editor's Notes
Ask audience who has experience with running task-based quant studies.
Speak to data storage in the last point.
Changes in business approach, environment, users, released features.
Good to do it even nothing has changed: Data may show your customers have become more familiar with a year old release.
Clearly define success
Don’t change scales (5- to 7-point)
Consistent tasks and answers make it easy.
Binary vs. Likert success
5-point vs. 7-point scale
It maybe different even within the same tool for how you institute the study.
Version control our analysis files and save them in more than one places!
Determine what’s a correct answer.
E.g. position task. Is entering a % a correct answer?
There was some manual labor (Most gain a day)
Potentially incorporate into the Method slide or leave off.