Users do not always accurately describe what they mean or feel. There are many reasons for this, ranging from politeness to poor introspection, to lack of sufficient technical vocabulary. Fortunately, UX researchers have tools in their trade to deduce what was really meant. We call this UX Fracking, a mixed methods approach that is optimized for extracting hidden user insights. We will illustrate the dangers of inadequate, superficial research, and how this may lead to outcomes incapable of addressing the users’ core issues. We will explore ways to avoid these pitfalls by leveraging mixed research methods to test hypotheses about the users’ intent and needs. This starts with a thorough understanding of who the user is, their goals, and how they work today, to an approach that combines surveys, interviews, and comment analysis with behavioral observation, and finally, validating the newly discovered user insights with the users themselves.
User Interface Design: Definitions, Processes and PrinciplesMoodLabs
An introduction to User Interface Design, often called UX / UI. Presented by David Little, User Interface Designer, DDH from King's College London Digital Humanities program.
Requirements Engineering for the HumanitiesShawn Day
This workshop explores how requirements engineering can be employed by digital and non-digital humanities scholars (and others) to conceptualise and communicate a research project.
requirementsEngineeringAs the field of digital humanities has evolved, one of the biggest challenges has been getting the marrying technical expertise with humanities scholarly practice to successfully deliver sustainable and sound digital projects. At its core this is a communications exercise. However, to communicate effectively demands an ability to effectively translate, define and find clarity in your own mind.
Julie Grundy gives an overview of user experience Design, why it's important, guiding principles, UX research overview, and tactics used by UX professionals. November 2015.
User Interface Design: Definitions, Processes and PrinciplesMoodLabs
An introduction to User Interface Design, often called UX / UI. Presented by David Little, User Interface Designer, DDH from King's College London Digital Humanities program.
Requirements Engineering for the HumanitiesShawn Day
This workshop explores how requirements engineering can be employed by digital and non-digital humanities scholars (and others) to conceptualise and communicate a research project.
requirementsEngineeringAs the field of digital humanities has evolved, one of the biggest challenges has been getting the marrying technical expertise with humanities scholarly practice to successfully deliver sustainable and sound digital projects. At its core this is a communications exercise. However, to communicate effectively demands an ability to effectively translate, define and find clarity in your own mind.
Julie Grundy gives an overview of user experience Design, why it's important, guiding principles, UX research overview, and tactics used by UX professionals. November 2015.
An overview of how UX Research is conducted in entrepreneurial Lean UX organizations. Principles and practices of Lean/Agile UX teams in high-tech, mostly Silicon Valley, settings.
Presented by Susan Wilhite to startupUCLA, an accelerator for UCLA students, on June 7, 2012 on the campus. Watch the startupUCLA web site for a video of the live presentation.
Knowing that a problem exists is one thing. Knowing how to solve it efficiently and cost-effectively is another. Discover the core foundational requirements in UX and Design Thinking that are vital to the success of an application that gets optimal buy-in from your users. If you're looking to optimize data visualizations, dashboards, and reports for effective communication of key business metrics, this will put you on the right track.
Mobile Center of Excellence is perfect for organizations looking to ensure the long-term success of their mobile strategies and Applications. It’s built to help you create and define the building blocks of a successful Center of Excellence for Mobile.
Mobile UX COE Strategists will work with your team to understand your current state readiness, build a vision for the Mobile Center of Excellence within your organization, and define the requirements for standing up a Mobile COE. Beyond just the components of a Center of Excellence, helps team creates a realistic roadmap for COE creation based on the people, process, and technology maturity within your business
User Experience Design Fundamentals - Part 2: Talking with UsersLaura B
#2 in a 3-part series on UX Fundamentals: Talking with Users
Understand why you should talk to users to uncover, validate and/or understand their goals.
Learn how and when to talk with your users:
User research methods
Planning
Best practices for interviews
Journey Maps with Legs! Best practices & hot tips for research, design and di...UXPA International
Based on interviews with leading client-side and independent researchers, Jeanne Turner & Julie Francis will share best practices for journey mapping. Their suggestions & stories will cover many facets, including
Kick-off and Discovery: How to structure a productive journey map kickoff
Research: Which research methodologies, questions, & activities reveal the most useful insights
The deliverable: What features make a great journey map?
Dissemination: How to maximize the impact of your journey map
These tips, stories, best practices and case studies will be drawn from expert interviews with researchers, stakeholders & designers with a focus on service design and multi-channel retail. You’ll walk away with practical things you can do to deliver great journey maps that have staying power.
Lecture on Advanced Human Computer Interaction given by Mark Billinghurst on July 28th 2016. This is the first lecture in the COMP 4026 Advanced HCI course.
Lean Startup Circle talk in Austin, Texas - August 20, 2013. Presented by Jon-Eric Steinbomer, Progress UX Principal and Research Director. Topics included: personas, remote research techniques, hardware and mobile research methods, card sorting and diary studies.
Intention of the talk was to provide a general framework of UX research methods and guidance to entrepreneurs and startups so that they could feel empowered to either try their own customer research or better understand how it fits in with the UX ecosystem.
What is User Experience Design?
The Business Case for User Experience Design
What are the UX processes?
How can we measure its effectiveness?
Who needs to be involved?
Slides Ian Multon recently used in his discussion w/ mentees of The Product Mentor.
The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People.
Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community.
http://TheProductMentor.com
There are key things that will give you a much better chance at success. While these are well documented in numerous books, articles, and videos - there are still many stakeholders that don't subscribe to some basic truths, like: product decisions should be based on evidence, or having dedicated UX Designers on product teams.
Jeremy will go over his top ten questions to ask any team to see if they're heading toward launching a great product experience.
This presentation was originally given @ Refresh Dallas on 2/12/15
A tutorial session on UXD hacks I gave at O'Reilly Etech in 2004.
Original context here: http://conferences.oreillynet.com/cs/et2004/view/e_sess/4767
"User-Centered Design and participatory product development are established, proven techniques for making interfaces and information understandable. But how is it possible to use them when your knowledge, the technology, and the possible markets are moving so quickly? Is it possible to create alpha-tech that defines a new market and is a joy to use? UI Design for Alien Cowboys is a three-hour tutorial and workshop that proposes that it is."
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...UXPA International
Imagine creating experiences for your rookie designers’ first couple years that are rewarding, enriching, and full of learning — without taking all your time or energy to manage. We’ll share techniques any team leader can put into practice using real-life examples from associate programs, apprenticeships, and internships.
Topics include onboarding, varied work challenges, developing multiple capabilities, buddy systems, group sharing, guest speakers, time with executives, and mentorship. We’ll also share how to operationalize learning, soft skills like communication and collaboration, setting boundaries, time management, achieving deep work, and more skills we all wish we were explicitly taught early on.
We’ll focus on modern-day associate programs, but even if you can’t create a full-fledged program, you’ll leave this session with ideas to use with your fledgling professionals. The benefits go beyond efficiency; it’s a foundation for culture, camaraderie, autonomy, and mastery.
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...UXPA International
Digital advances are being made at a rapid-fire pace, yet disability inclusivity continues to fall short of the digital revolution. As the number of people living with disabilities rises, the time to take digital accessibility to the next level is now. Let’s disrupt inaccessibility together! Come hear about a multi-part discovery research and ideation project informing foundational UX designs for our customers. You’ll get insights from our unique study, which are widely applicable across industries, and walk away with tips and inspiration to kick off your own accessibility-focused discovery and ideation. Only YOU can prevent inaccessibility – are you in?
More Related Content
Similar to UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
An overview of how UX Research is conducted in entrepreneurial Lean UX organizations. Principles and practices of Lean/Agile UX teams in high-tech, mostly Silicon Valley, settings.
Presented by Susan Wilhite to startupUCLA, an accelerator for UCLA students, on June 7, 2012 on the campus. Watch the startupUCLA web site for a video of the live presentation.
Knowing that a problem exists is one thing. Knowing how to solve it efficiently and cost-effectively is another. Discover the core foundational requirements in UX and Design Thinking that are vital to the success of an application that gets optimal buy-in from your users. If you're looking to optimize data visualizations, dashboards, and reports for effective communication of key business metrics, this will put you on the right track.
Mobile Center of Excellence is perfect for organizations looking to ensure the long-term success of their mobile strategies and Applications. It’s built to help you create and define the building blocks of a successful Center of Excellence for Mobile.
Mobile UX COE Strategists will work with your team to understand your current state readiness, build a vision for the Mobile Center of Excellence within your organization, and define the requirements for standing up a Mobile COE. Beyond just the components of a Center of Excellence, helps team creates a realistic roadmap for COE creation based on the people, process, and technology maturity within your business
User Experience Design Fundamentals - Part 2: Talking with UsersLaura B
#2 in a 3-part series on UX Fundamentals: Talking with Users
Understand why you should talk to users to uncover, validate and/or understand their goals.
Learn how and when to talk with your users:
User research methods
Planning
Best practices for interviews
Journey Maps with Legs! Best practices & hot tips for research, design and di...UXPA International
Based on interviews with leading client-side and independent researchers, Jeanne Turner & Julie Francis will share best practices for journey mapping. Their suggestions & stories will cover many facets, including
Kick-off and Discovery: How to structure a productive journey map kickoff
Research: Which research methodologies, questions, & activities reveal the most useful insights
The deliverable: What features make a great journey map?
Dissemination: How to maximize the impact of your journey map
These tips, stories, best practices and case studies will be drawn from expert interviews with researchers, stakeholders & designers with a focus on service design and multi-channel retail. You’ll walk away with practical things you can do to deliver great journey maps that have staying power.
Lecture on Advanced Human Computer Interaction given by Mark Billinghurst on July 28th 2016. This is the first lecture in the COMP 4026 Advanced HCI course.
Lean Startup Circle talk in Austin, Texas - August 20, 2013. Presented by Jon-Eric Steinbomer, Progress UX Principal and Research Director. Topics included: personas, remote research techniques, hardware and mobile research methods, card sorting and diary studies.
Intention of the talk was to provide a general framework of UX research methods and guidance to entrepreneurs and startups so that they could feel empowered to either try their own customer research or better understand how it fits in with the UX ecosystem.
What is User Experience Design?
The Business Case for User Experience Design
What are the UX processes?
How can we measure its effectiveness?
Who needs to be involved?
Slides Ian Multon recently used in his discussion w/ mentees of The Product Mentor.
The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People.
Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community.
http://TheProductMentor.com
There are key things that will give you a much better chance at success. While these are well documented in numerous books, articles, and videos - there are still many stakeholders that don't subscribe to some basic truths, like: product decisions should be based on evidence, or having dedicated UX Designers on product teams.
Jeremy will go over his top ten questions to ask any team to see if they're heading toward launching a great product experience.
This presentation was originally given @ Refresh Dallas on 2/12/15
A tutorial session on UXD hacks I gave at O'Reilly Etech in 2004.
Original context here: http://conferences.oreillynet.com/cs/et2004/view/e_sess/4767
"User-Centered Design and participatory product development are established, proven techniques for making interfaces and information understandable. But how is it possible to use them when your knowledge, the technology, and the possible markets are moving so quickly? Is it possible to create alpha-tech that defines a new market and is a joy to use? UI Design for Alien Cowboys is a three-hour tutorial and workshop that proposes that it is."
Similar to UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights (20)
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...UXPA International
Imagine creating experiences for your rookie designers’ first couple years that are rewarding, enriching, and full of learning — without taking all your time or energy to manage. We’ll share techniques any team leader can put into practice using real-life examples from associate programs, apprenticeships, and internships.
Topics include onboarding, varied work challenges, developing multiple capabilities, buddy systems, group sharing, guest speakers, time with executives, and mentorship. We’ll also share how to operationalize learning, soft skills like communication and collaboration, setting boundaries, time management, achieving deep work, and more skills we all wish we were explicitly taught early on.
We’ll focus on modern-day associate programs, but even if you can’t create a full-fledged program, you’ll leave this session with ideas to use with your fledgling professionals. The benefits go beyond efficiency; it’s a foundation for culture, camaraderie, autonomy, and mastery.
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...UXPA International
Digital advances are being made at a rapid-fire pace, yet disability inclusivity continues to fall short of the digital revolution. As the number of people living with disabilities rises, the time to take digital accessibility to the next level is now. Let’s disrupt inaccessibility together! Come hear about a multi-part discovery research and ideation project informing foundational UX designs for our customers. You’ll get insights from our unique study, which are widely applicable across industries, and walk away with tips and inspiration to kick off your own accessibility-focused discovery and ideation. Only YOU can prevent inaccessibility – are you in?
User experience can be drastically elevated by combining data science insights with user-based insights from research. Data analytics on its own can make themes and correlations difficult to explain and to provide accurate recommendations. For example, themes identified via large global surveys and usage data can be better understood with UX insights from focused user research, such as user interviews and/or cognitive walkthroughs. This presentation will highlight the complimentary nature of data science and UX and will focus on the benefits of bringing the two disciplines together. This will be buttressed with practical examples of enterprise projects and applications that combined data and skills from the two disciplines, guidance on how the two disciplines can better work together, and the skills needed to improve as a UX professional when working with data science teams.
UXPA 2023: Learn how to get over personas by swiping right on user rolesUXPA International
This session walks through the concept of user roles as an alternative to personas as a means to generate and disseminate user insights for product development teams. We will describe the tools and methods used to create a research database organized by user roles, along with examples and short exercises to help attendees think through user roles within their own context.
By the end of the session, attendees should be aware of tools and approaches for:
Organizing user research information in a database
Disseminating user role information to product and design teams
Managing a user roles database as part of a long term UX Research program
If you’re ready to ditch personas but don’t know how, this session is for you!
We will present a case study that details our approach for replacing user personas with user roles for a multi-national SAAS company. We will take the audience on a journey that starts with an executive request for personas, travels through the tribulations of realizing personas suck, and concludes with convincing others to accept a new and innovative way to understand the people who use the product. Our key message is that personas lack real value for organizations that already understand the importance of empathizing with users. Building user-centered products requires easily accessible and well organized user insights. We will discuss defining users through a process of stakeholder consultation and content review, and structuring data around Jobs to Be Done and product interactions. We will also discuss the dissemination of user roles in our organization using relational databases, interactive dashboards and online wikis. Spoiler alert, our stakeholders loved user roles!
UXPA 2023: Experience Maps - A designer's framework for working in Agile team...UXPA International
Agile Methodology refers to software design and development methodologies centered around the idea of iterative design and development, where requirements and concepts evolve through collaboration between self-organizing cross-functional teams. Thus, Agile enables teams to deliver value faster, with greater quality and predictability, and greater aptitude to respond to change. With evolving product features every design sprint, designers & researchers find it difficult to follow the design process. This sometimes leads to designs delivered in haste or sub-par design artifacts which result in UX debt. UX debt is accumulated when design teams take actions or shortcuts to expedite the delivery of a piece of functionality or a project which later needs to be refactored. It is the result of prioritizing speedy delivery of design to the development team over a perfect experience journey. Experience Maps is a great tool to practice UX in Agile as well as manage UX Debt.
UXPA 2023: UX Enterprise Story: How to apply a UX process to a company withou...UXPA International
How to build a UX Department from scratch, in an environment they think UX people do social media posters and posts! An agile implementation just started, and people are moving from a waterfall and ad-hoc mindset to agility. In this session, I will talk about my Journey to establish a UX Department for a company that is part of a global brand, but this local branch just started the digital transformation movement. Challenges like: spreading awareness and educating people about UX, hiring the right team, defining the right team structure, establishing workflow and day-to-day operations, and applying localization (non-western culture).
UXPA 2023: High-Fives over Zoom: Creating a Remote-First Creative TeamUXPA International
I started my current job in March of 2020. Many of us remember something clearly about the month that COVID started to shut things down. I remember being surprised to hear that my new on-site-only job would be starting in my living room over zoom. How do you lead a design team when none of the team members live near each other and creativity is highly collaborative? Taking from over a decade of working in HR software, I knew whatever I did needed to put people first. That what employees love about a job is often deeper than the work, it’s the culture, the relationships and people they work with. It’s the feeling that their work has value, and their contribution matters. In this talk I will walk though some of the rituals and best practices I have learned over the last two years building a remote-first creative team.
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & ...UXPA International
As humans, we are biased by design. Our intricate and fascinating brains have developed shortcuts through centuries of human evolution. They reduce an unimaginable load of paralyzing decisions, keep us alive, and help us navigate this complex world. Now, these life saving biases affect how we behave with modern technology. Understanding some of the theories and reasons why these biases exist is the key to unlocking their power. In this workshop we will cover some theories around how the brain works. We will review some of our mental shortcuts, take a look at some common biases, and learn how they affect our users, our research, and our designs. Lastly we will review some advantages of biases, and ways to identify and reduce bias. This workshop is targeted for designers who do their own research, and researchers looking to learn more about removing bias from their studies.
UXPA 2023 Poster: Improving the Internal and External User Experience of a Fe...UXPA International
UXPA 2023 Poster: Improving the Internal and External User Experience of a Federal Government Legacy Application Using User Experience and Agile Principles
Are you new to UX management, or thinking of getting into management? Then this talk is for you. After reading countless books, attending countless trainings, mentoring and being menteed, nothing quite prepared me for management like my first year. I’ll share with you what I wish they’d told me. I’ll also share my process for generating team research roadmaps, establishing team values, keeping employees motivated, and not burning out.
UXPA 2023: Redesigning An Automotive Feature from Gasoline to Electric Vehicl...UXPA International
Join us for an interaction design case study from the automotive industry. We created a Human-Machine Interface (HMI) for a vehicle feature that provides household-levels of power in electrical outlets for our customers to use at work and play. This case study will reveal: · Our debate of re-using version 1.0’s HMI vs designing a new user interface for the electric vehicle—when to break with consistency and why? · User research we conducted to guide our early design concept. · Paper prototypes we created to support our usability testing of the concept with vehicle owners. · How we solved internal debate over the interaction design in moving from internal combustion vehicles to electric vehicles. * Advice to help you evangelize user-centered design that is also brand-centered for a new product.
Ensuring the end product is inclusive can be a challenge, but so can also be the process that was used to design it. How do we make sure that design is just and that people and communities are not inadvertently harmed, on the basis of aspects such as age, background, gender, and race, in the design process by the choices we make as designers? How do we do this especially for new innovative technologies, which we might not know much about? In this session, the speaker will review the common pitfalls of typical design research and development. Then, the speaker will walk through a framework for better design work that is more inclusive and minimizes potential social harm.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
The Indian economy is classified into different sectors to simplify the analysis and understanding of economic activities. For Class 10, it's essential to grasp the sectors of the Indian economy, understand their characteristics, and recognize their importance. This guide will provide detailed notes on the Sectors of the Indian Economy Class 10, using specific long-tail keywords to enhance comprehension.
For more information, visit-www.vavaclasses.com
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
1. CIO Design | UXPA 2023, Austin
UX Fracking
Using mixed methods to extract hidden insights
2. Feb 2021
UX Fracking
UXPA 2023, Austin
• UX Fracking:
o Is intentional UX research
o Uses mixed methods
o Tests hypotheses about what users are thinking, doing, or
feeling
o Goes below the surface of token research to crack open
the insights
• “Hidden insights” - because users are notoriously bad at
telling you what they think.
• Understanding what is really going on requires skills, context,
and a scientific, curious outlook.
3. UX Research
Creative Director
Jason Telner, Ph.D. Annette Tassone, Ph.D.
Office of the CIO, CIO Design, IBM
Senior User Experience
Researcher & Designer
Jon Temple, Ph.D.
Design Principal,
UX Design Creative
Director
UX Fracking
UXPA 2023, Austin
5. UX Fracking
UXPA 2023, Austin
The design
cycle –User
research and
design
research
Define
Ideate
Protype
Test
Empathize
Discovery:
User
research
Validation:
Design
research
6. UX Fracking
UXPA 2023, Austin
Design
Research Map
Research Led
Sanders, L. (2008) On modeling an evolving map of design practice and design
research. Interactions 15(6) pp13-17. add t
• Research-led perspective
relies on the data from
qualitative and
quantitative research
methods to inform
design and has the roots
of their methods based in
traditional disciplines
such as applied
psychology,
anthropology, sociology.
• Design-led approaches
are more modern
methods that focus on
the experiences of users
of a product or service.
k.
7. User Experience
Research Map
Scripted (lab-based) use of
product
Natural use of product De-contextualized/not using product
Combination/hybrid
• User experience research
consist of the dimensions of
qualitative vs. quantitative
research approaches, as
well as behavioral vs
attitudinal data.
• Various research methods
lie along the spectrum and
those in the middle consist
of mixed methods and data
sources.
• The methods can also be
categorized by if they
examine users using a
product in their natural
environments, in a lab
based setting, not using the
product or some
combination of approaches
8. Cultural shifts
To save time, we see
general movement
away from users and
their needs and
towards the design.
Move away from
strategic to the
tactical.
User Research :: UX Design
Design Research :: UI Design
UX Fracking
UXPA 2023, Austin
D
e
s
i
g
n
U
s
e
r
9. Do research
faster!
• Commoditization of
UX Research
• No time to acquire
domain knowledge
• Top-down design
(stakeholders, PO)
• UXR adds time
• Do all research
within a
development sprint
(don’t do that!)
10. Design
research vs. UX
research –
compare and
contrast
UX Fracking
UXPA 2023, Austin
What is the
purpose of
the
research?
Design Research
User Experience Research
To inform design of products
services or user interfaces.
To better understand how
users interact and feel about a
product
What
methods
are used?
Use a variety of qualitative
and quantitative methods to
inform the look and feel of a
product or service.
Use a variety of qualitative and
quantitative methods to
understand and improve how a
user interacts and experiences a
product or service.
What are
example
research
activities?
Investigate which design a
user prefers with regards to
how it looks.
Investigate how a user
interacts with a proposed
design and what their
experience is like.
11. Hallmarks of
research
unlikely to
provide deeper
insights
• Focus on preferences – “they
liked A more than B”. (But why?)
• Testing navigation that doesn’t
support real tasks – “they found
it!”
• Looking at just the happy path
• Using narrow tasks that are not
end-to-end
• Sticking to “the script”
• Unsupervised research conducted
by untrained, non-UXR
• Starting research over from scratch
– dismiss “old” data
• Users that never complain or act
negative
• Using personas built in a co-
creation workshop w/o real
users
• Failing to include alternative
outcomes or designs
• Generalizing from a single outlier
• Consider user language to be
technically precise
• Comment analysis: Always
backing into preexisting
categories
• Focusing on extremely rare edge
cases
But may have their
place within the
context of design
research.
12. Dangers of doing superficial research
• Inaccurate insights
• Misinformed design decisions
• Missed opportunities
• Lower user satisfaction
• Higher training costs
• Higher support costs
• Wasted time and money
• High redesign costs
• Failure to innovate and differentiate
Dangers are similar to not doing any UXR; work
products that evaluate without revealing.
13. What is “UX Fracking”?
UX Fracking
UXPA 2023, Austin
14. • Inject “fracturing
fluid”
• Data-driven decisions
• Extremely lucrative
way to get hidden
natural resources
• Controversial
practice (groundwater
contamination,
wasteful, flaming
faucets)
Natural Gas
Fracking
15. UX Fracking
UXPA 2023, Austin
• Metaphor - intentional mixed-
methods research
• Data to decide if big UXR payoff
• Seek hidden UX insights
• Not controversial – except where
companies are not design or research
led
UX Fracking
1
16. Extracting Hidden UX
insights
Great UXR helps us understand what
is in the mind of the user.
Which may or may not be what they
are saying.
UX Fracking is another way to
extract insights from our users.
UX Fracking
UXPA 2023, Austin
17. UX Fracking
UXPA 2023, Austin
UX Fracking in
7 steps
1
1. Listen to what users say (with a critical ear)
2. Watch what users do (with a critical eye)
3. Think end-to-end about the user’s tasks
4. Build testable hypotheses about what was thought or meant
5. Test hypotheses by chaining mixed methods
6. Synthesize findings across methods
7. Review what you think you have learned with users
18. 1. Listen to what users say (with a
critical ear)
1
Always, always talk
to users
Interview them…
Survey them…
Interview them
again!
Words are a clue;
need a lot of
context to interrupt
what is meant.
Users do not use the
same words in the
same way as experts.
Not technically
accurate.
Users are not
designers. Do not
take their
suggestions
literally.
Image from wired.com
19. Role of implicit versus explicit
memory
Human information processing
limitations.
Users may feel or understand
something they cannot
describe.
From: SimplyPsychology
20. “Faster horses”
Henry Ford famously (never) said this:
“If I had asked people what they wanted,
they would have said faster horses.”
Illustrates many users/consumers lack
sufficient creativity and imagination.
Doesn’t mean users do not recognize
value (superiority of recognition over
recall).
21. When users design…
Remember:
• Users Don’t Always Know
What They Want
• Users Are Not Trained
Interaction Designers
• Users Design for Themselves
“Make sure that you do not take
user feedback or requests for
design changes at face value.”
(UX Matters, 2011)
Homer Simpson’s dream car. (Cup holder for beer not
included)
22. Real world example
Long ago, in a usability lab far far
away…
Usability test: Users couldn’t find the
“key” information on a busy
webpage.
Asked them to redesign the page.
I got this:
• Move everything above the fold
• Bold everything that might be
important
• Flaming letters?
23. 2. Watch what users do (with a
critical eye)
1
Methods:
Usability testing –
too late for current
release but rich
quantitative data
for the future.
Disconnect
between what they
do and say is PURE
GOLD!
A-B Tests
Diary studies – watch
users in “the wild”.
Usage metrics
provide more
context.
“Relax, and remember we are testing the
application and not you. You can begin!”
24. Behavioral data requires critical
analysis, too
During WWII, American bombers
shot down.
Naïve analysis: Double up the armor
where there are holes.
Statistician Abraham Wald pointed
out:
• Data from bombers who returned
(Survivor bias)
• "Missing data"
Hidden insight led to improving the
armor where no holes (cockpit,
engines, tail).
Look at those bullet holes! Better double up the armor
there, right? Guess again.
25. 3. Think end-to-end about the user’s
tasks
1
Projects are often
siloed. “In scope”.
But may not fully
reflect what the
user does or need.
Thinking end-to-
end uncovers
hidden pain points,
gaps in
functionality,
missed
opportunities.
For example:
How do they get
started?
How do they know
when they are done?
Methods:
• Journey maps
• Task analysis
• Friction analysis
26. 4. Build testable hypotheses about
what was thought or meant
1
Use “hypothesis” in
a similar manner to
Jeff Gothelf.
Doesn’t need to be
formal or require
significance testing.
Have both a null and
an alternative
hypothesis.
Hypotheses must
be measurable and
capable of being
rejected.
27. • Experience: I’ve seen this
elsewhere – is it true here?
• Violation of a key UX heuristic.
Experience & best practices
2
4
• Stakeholders
• Product Owner
• Development team
• Insightful user comment
Stakeholders &
extended team
• Disconnect between the user model
and the expert or business model
(gaps, workarounds).
• Disconnect between UX methods –
user fails test but reports high
satisfaction.
• Known pain points may lead to
other questions.
User disconnects &
top pain points • Root decisions in research – and
test hypotheses to decide
between alternatives.
• (You don’t need a formal A-B test
to make an A-B decision).
Design decisions
UX Fracking
UXPA 2023, Austin
1
3
Where do
hypotheses come
from?
Anywhere!
28. 5. Test hypotheses by chaining
mixed methods
1
Triangulate on a
problem with a
combination of
methods.
Discovery phase:
fishing, look for
patterns.
Deepening expertise
in a domain: Knowing
what to look for, test
more hypotheses.
Goal is to feel
reasonably confident
you have surfaced an
important insight
that will improve the
solution.
Build out a test
plan, or topological
map, that captures
your journey.
29. 6. Synthesize findings across
methods
1
Proper
methodology is
important; be
prepared to speak
to it.
Record enough for
posterity.
Focus not on
standalone work
products.
The UX insight is the
deliverable.
Focus should be on
the problem, the new
insight.
Helps you understand
the solution better,
what should be done.
30. 7. Review what you think you have
learned with users
1
Review your
findings with users,
give them a chance
to react.
Recognition is
better than recall.
What they could not
articulate, they
might recognize.
If you are wrong, let
the users surprise
you.
“I say: I noticed y'all missed the Submit button
because the scroll bar was hidden. Maybe a pattern
that works great on mobile isn't so great on the
desktop."
31. Story time: How we used mixed
methods to help us understand
the true source of friction
UX Fracking
UXPA 2023, Austin
32. UX Fracking topology
NPS
Feedback
Comments
Walkthrough
of new Device
selection
design
Hypothesis:
Data migration
is the biggest
pain point
Interviews
about end-to-
end device
migration
Friction
analysis
(Analyze +
Quantify)
Survey to
validate
discovery with
larger sample
Compare
to
alternative
design 1
Compare
to
alternative
design 2
Summarize
Longitudinal
UX metrics at
task level
Compare
Monitor
Discovery
Test Hypothesis
Journey map
of As-Is
Evaluate remediation
Validate
33. Business wanted to refresh an old design
Website in need of a refresh
UX Fracking
UXPA 2023, Austin
34. Dunnings – borrowed from collection
agencies
Business wanted
old devices
returned
promptly
UX Fracking
UXPA 2023, Austin
35. Business model
Simone needs
a new laptop
Simone returns
old device
Selects
replacement
device
Receives reminder
emails
Schedules a
return
1. Select a new device
2. Receive reminders to return old device
3. Return the old device
37. Employee quotes
“I am not a technical person and I work 60-80 hours at a client site. When things
don't work out I am stuck. I have no time to do it and it takes hours longer for a
nontechnical person when things don't go correctly (like installing Notes).”
“I cannot spend a whole day setting up a computer.”
“The new machine migration was
hard.”
“If you are not an expert it's really hard. I am getting a lot of requests
from people around to help or guide them in this process. The main
issues is notes data files/location.”
"Migrating data using Crashplan is not practical at all, and with USB ports disabled, it
was very difficult to know what files/settings to migrate outside of my home folder. "
"What directions there were on the WEB were not real clear when
everything did not start automatically. If not for the help desk then it
would have taken much much longer."
"It would help if the DEVICES@IBM could have a list of applications that we need to be installed before
the device is sent out. Some of this information is available via the IBM Standard Asset Manager
Software list."
[Before returning machine] “2 months. When I’ve got an email from IBM to return the laptop.
Otherwise, I would probably have sent it back after about 4 months.”
38. NULL HYPOTHESIS:
The process is straightforward, and everyone knows what to do*. But users
are very busy and do not have time, so they wait as long as they can.
HYPOTHESIS 1:
The process is complex and time-consuming, and most do not know what
to do.
Users take their time to ensure they haven’t “missed” anything that might
later prove important.
UX Fracking begins with a question…
Question: Why are users really taking so long to return an old device?
39. Interviews
3
Understanding the end-to-end process
• Product Owners
• Subject Matter Experts
(SMEs)
• End users
Journey map
4
• High-level
• Main phases, key steps
• End to end
UX Fracking
UXPA 2023, Austin
40. Business versus the user perspective
Simone needs a
new laptop
Simone returns
old device
Selects
replacement
device
Receives reminder
emails
Schedules a
return
Experience from a BUSINESS perspective
Asks manager
about device
refresh rules
Reads policy info Selects
replacement
device
Receives
device
Follows setup
process
Figure out how to
migrate data
Learn how &
then transfer
bookmarks
Manually
migrates data
Manually
installs missing
software
Wipes data /
passwords /
packs up old
device
Receives Dunning
emails
Schedules a
return
Experience from a USER’S perspective
41. Identifying problem areas
What would you say is the biggest pain point in
this area? (n=27)
Data Migration 15 56%
Installing apps 4 15%
Network setup 2 7%
Platform adoption 1 4%
Device setup 2 7%
Dunning Email 3 11%
How confident are you that you got
everything you need? (n=23)
Confident 12 52%
Somewhat confident 5 22%
Not at all confident 5 22%
Unclear 1 4%
How long did you wait to return your
old machine? (n=24)
Returned same day 1 4%
Within 1 week 4 17%
Within 1 month 7 29%
After 1 month 4 17%
Not returned yet 8 33%
UX Fracking
UXPA 2023, Austin
42. As-is experience: The Journey to A New Device
Tyson
Employee
Scenario
Tyson just started a new project requiring a lot of number
crunching and decided to upgrade his aging PC.
Goal
Quickly select a new PC and confidently migrate all of his data and apps
over, without losing a day of work.
1. How do I get a new device? 2. Order new device 3. Migrate data and apps 4. Return old device
My machine is slow. Am
I eligible for a new
machine? Who do I ask?
Not really sure what to
search for, so I ask my
manager, who told me
to go to Devices@IBM.
Found Devices@IBM
link in search. Not sure
whether I am eligible
will request a refresh
anyway just in case.
I need a more powerful
machine now. But
difficult to compare all
of the specs, which
seem buried.
I don’t see the
‘Developer machine’ I
am told is the best. Will
ask my manager how to
request an exception.
Exception was
approved. UPS tells me I
should receive the new
device soon.
Received my new PC
today. There is a link to
setup instructions in the
box.
Confused about how
to move my data
over. Code42?
Migration tool? A
friend says to copy
everything to Box; it’s
really slow, like half a
day.
Installing apps from
the IBM app store
was a breeze!
Missing some key apps. Need to
track them down one by one. One
requires a new Software License.
I can’t get the VPN to
work. Calling the Help
Desk.
I can’t get IBM
Notes to work.
Calling the Help
Desk again!
Keep receiving
“Dunning” emails to
return my machine. But
I am not sure I have all
of my data or apps yet.
My manager says not to
worry.
It’s been 2 months and I
think I have everything.
Time to return my old
clunker (and shut off
those ”Dunnings”).
Manager Devices@IBM Setupmypc or
Help@IBM
Box, Code42, or
Migration tool
IBM App Store SLM catalog,
App Store, other
Help desk ThinkDesk, UPS,
manager, etc
45. Data migration
I have not
completed
the migration
1%
1 - 3 hours
16%
4 - 7 hours
23%
1 - 2 days
27%
3 - 5 days
16%
1 - 3 weeks
12%
1 - 2 months
3%
More than 2
months
2%
Value Percent Count
I have not completed the migration 0.8% 3
1 - 3 hours 16.4% 61
4 - 7 hours 23.3% 87
1 - 2 days 27.1% 101
3 - 5 days 15.8% 59
1 - 3 weeks 11.5% 43
1 - 2 months 3.5% 13
More than 2 months 1.6% 6
Totals 373
Yes
41%
(147 )
No
59% (213)
26.7%
52.1%
27.4%
6.2%
0
10
20
30
40
50
60
From a colleague In person from
Helpdesk,
Deskside support,
or THINKDESK
Over the phone or
chat with
Helpdesk
Other - Write-in
Percent
Did you need help with your migration
(including all data and applications)?
32% took 3 days or more….
How long did it take?
46. Friction analysis: As-Is
5 6
Task analysis, identify
pain points
PURE scoring of As-
Is steps
• Task analysis - detailed analysis
at the step level
• Use to synthesize findings (pain
points) across methods
• Use a spreadsheet
• Optionally, journey map
• PURE (Pragmatic User
Ratings by Expert)
• Multiple scorers
Current (As Is)
UX Fracking
UXPA 2023, Austin
47. PURE Methodology
Pragmatic User Rating by Experts
From NN/g Nielsen Norman Group: https://www.nngroup.com/articles/pure-method/
48. Scoring process
1. Detailed task analysis
(discrete substeps)
2. Map pain points /
quotes against steps
3. Individual raters
assign score
4. Team score
5. Add them up
As-is PURE for Device
management: 69
49. Compare alternative designs
• Full Data migration
• Full app migration
• PURE: 42
• Reduction in friction: 39%
Design alternative 1:
As Is: 69 (PURE)
Design alternative 2:
• Mirror hard drive in cloud
• Re-installing apps still
manual
• PURE: 58
• Reduction in friction: 16%
UX Fracking
UXPA 2023, Austin
50. UX Fracking
UXPA 2023, Austin
What did we
learn? Why does this matter?
• Majority of development for
device selection, had relatively
low friction, and for emails
urging device returns.
• Should prioritize highest points
of friction over lower.
• Improved automation of
migration likely to lower overall
friction than just telling people.
Winner:
H1
Hypothesis confirmed:
Data migration and app
installation are very time
consuming, difficult to do,
and are directly
responsible to the delay in
returning old machines.
The user model around acquiring
new devices:
Sure, device selection design could
be improved.
But my biggest pain points – was
data migration, reinstalling apps.
These are unsupported.
I don't understand how to start, it
takes a long time to do, and I have
low confidence I was successful.
“
“
51. Where did we
end up? In the beginning:
• Seen but not seen: “I know this!”
• Diffusion of responsibility: "Not my app!"
With UX Fracking, drew attention to issues “hidden in plain sight”:
• Dismissed claims: “everyone knows how to do that!”
• Measured the highest points of friction
UX Fracking
UXPA 2023, Austin
52. Story time: How we determined
why “search” sucked and what
should be done about it
UX Fracking
UXPA 2023, Austin
53. Anecdotal:
“Search
sucks!”
• Search
usage
• SEO
• Alternative
engines
Survey
Feedback
Comments
Hypothesis:
Search may
suck – but
users mean
something
else
Intercept
survey: Ask
about search
experience
Discovery
Test Hypothesis
Summarize
Non-UX
Interviews
about search
experience
Analyze
patterns of
success and
failure
Compare
reported
outcome to an
idealized
outcome
Recommend
pivots
UX Fracking topology
54. NULL HYPOTHESIS:
Users are unable to find the solutions provided.
HYPOTHESIS 1:
Users are unable to find solutions that do not exist or do not meet
expectations.
UX Fracking begins with a question…
Question: What do users mean when they say “Search sucks”?
55. NPS Comment analysis (N=387)
Search-specific complaints (35%)
• “The website looks nice, but it's hard to find
what you need.”
• “Search brings up too much data that has very
low relevance to the search. I wasn't able to
find the 2018 holiday calendar for
Poughkeepsie.”
Content is missing or not relevant (18%)
• “It's difficult to find out the solution to your
specific problem”
• “handles main cases, but not rare cases”
• “Too vague, more helpful searching forums in
most cases”
• “Things I can't solve by myself are the more
obscure issues that aren't handled by a FAQ.”
• “I have installed outlook and it was not easy at
all to find related information.”
Interviews with people who identified
themselves:
If I didn’t find it for any reason, I would agree with
the statement: “search sucks.”
57. Intercept survey
Comparing what was sought to what could be found
UX Fracking
UXPA 2023, Austin
86% (n=132) were
looking for content
that did not (yet)
exist.
Evaluated the ease of
locating this
information and
compared this back to
the reasons for search
failure.
“Bunch of garbage text for normies
to not feel anxious but no normal
information on how to resolve a
problem.”
“We need a better system in
place to get direct answer to
many avenues to find the correct
answer.”
Survey respondents
58. UX Fracking
UXPA 2023, Austin
What did we
learn? Why does this matter?
• Correct diagnosis
• Enhancements to the search engine
unlikely to help
• Focus needs to be related to
content: missing content, answering
specific questions, quality
Winner:
H1
Hypothesis confirmed:
Complaints about "search"
were not technically about
"search".
The user model around failed
searches:
I used search but...
I did not find anything that helped.
Missing content or content that didn't
meet expectations.
I used search; I did not find an
answer; so I consider this to be
'search' related.
“
“
59. Where did we
end up? UX Fracking surfaced issues completely hidden:
• Naïve comment analysis by keywords alone would
have missed this key insight
• Listened to what users were really trying to tell us:
“I didn’t find what I needed!”
• Disentangled search from content
UX Fracking
UXPA 2023, Austin
61. UX Fracking
UXPA 2023, Austin
Summary
Remember: If UX is not
answering these questions,
someone not trained in UX
will act as if they know the
answers.
And they will likely get it
wrong.
UX Fracking can be
combined with any UXR
program to get to the
bottom of key user
assumptions or decide
between design approaches.
UX Fracking is a mixed
methods approach that uses
"hypotheses" to help test
out poorly understood or
hidden user issues.
66. User Experience Index (UXI)
Computation
1. Research end-to-end tasks to determine
the actual “friction” for that task.
2. Design the ideal user experience and
estimate the target friction for the task.
Compute the “Change in Friction”.
3. Create a normalized difference score
(enables cross-task comparison) by
converting it to a 100-point scale with
larger numbers being better. Assign grade.
4. To compare the same task against
different targets or goals, use the Change
in Friction (CIF) not the UXI. Multiply by
100%.
Actual Friction Target Friction
-
Actual Friction
Scores range from 0 to 1, with lower scores being better
(0 means actual experience matches ”ideal”)
UXI = 100 * 1 - Change in Friction
Change in
Friction (CIF)
=
Use the CIF score to compare the same task with different targets; use UXI to
compare different tasks over time against a single but realistic target or goal.
67.
68. Task Ease (3Q21) Ease (3Q22)
Sign in and/or authenticate during installations 70% 75%
Install the software and tools that I need 64% 72%
Connect a new device to the IBM network 63% 69%
Find the software and tools that I need 60% 65%
Return an old device 60% 66%
Select the appropriate computer that meets my needs 56% 67%
Purchase an old laptop/desktop for my personal use from IBM 54% 53%
Setup a new device 51% 64%
Contact the help desk to solve problems encountered during setup 48% 59%
Find help information to solve problems encountered during setup 46% 58%
Refresh/upgrade my laptop/desktop 43% 57%
Transfer all existing data from my old device to a new device 42% 55%
Request an emergency replacement for my laptop/desktop 41% 55%
Find the relevant policies and procedures about acquiring devices or accessories 37% 48%
Manage registration - 66%
Long-term monitoring
1 Very difficult --- Neither easy nor difficult --- Very easy 7