Ethnography in Software Design
An Anthropologist’s Point of View
Big Design - Dallas, September 2015
: https://www.flickr.com/photos/101187156@N03/14366224997/
@Kel_Moran#bigd15

Kelly
Moran
Lead Design Researcher
@Kel_Moran
#bigd15

MS Applied Anthropology
University of North Texas
BA Anthropology
California State U. Northridge

US Peace Corps
Uzbekistan
Study Abroad
Saudi Arabia
In
the
Wild

Worked with:

“Uncover user needs,
Design great solutions,
and build out solutions to launch.”

“Uncover user needs,
Design great solutions,
and build out solutions to launch.”

DistillMeaningfromObservation
BuildtheBacklog
CONTACT BUILDING & EVOLVINGFOCUSED INNOVATION
Only a subset of these activities will be appropriate for any given project.
Qualitative Research
Quantitative Research
Affinity Diagramming
Construct themes from qualitative
data.
Analysis & Synthesis Opportunities
Ideation & Iteration
Validation & Evaluation
Approach Planning Design Research Experience Strategy & Strategic Ideation PlanningDefinition
IdentifyChallengesinContext
SettheStage
REVEALING REALITY
Foundational Analysis
Heuristic Evaluation
Identify inital breakdowns and
opportunities
Digital Marketing SWOT
Analysis
Identify strengths, weaknesses,
opportunities & theats.
Stakeholder Interviews
Understand staheholders’ business
goals & strategy.
Technical Organization
Capability Aanalysis
Analyze existing skills and toolsets.
Scenario-based System
Walkthroughs
Demo of existing solution.
Metrics Evaluation
Establish quantitative baseline of
engagement & conversion data.
Content Inventory
Catalog the content of the site or
application.
Competitive Analysis
Evaluate competitors and
comparables on specific axes.
User Experience
Data
Technology
Enterprise Architecture
Capability Assesment
Review existing enterprise
technology infrastructure.
Solution(s) Architecture
Assessment
Review existing application(s)
architecture.
RFP Request
Deliver proposal to defined scope
document and existing
requirements.
Project Approach
Asses a possible project and plan
high level approach.
Align & Assess Workshop
Assess readiness across core
capabilities.
Backlog Grooming
Reprioritize backlog, add new
stories.
Zero Feature Release
Demonstrate CI, automated
testing, core solution setup.
High Level Technical
Architecture
Describe high level architecture,
including packaged components.
Development
Infrastructure
Configuration
Continuous integration setup.
Technical Package
Identification & Evaluation
Perform product evaluations for
package solution components.
Architecture Spikes &
Proofs of Concept
Prove candidate architectures via
top bottom spikes.
Existing Research Review
Market research, website feedback,
corporate strategy, etc.
BringtheSolutionintoFocus
Research Plan
Design activities to meet research
goals.
Contextual Inquiries
Observe & document user in
context & environment.
User Journals & Diaries
User document their experiences
over time.
Participatory Design
Co-creation explorations with
users.
Card Sort
Explore users’ mental models for
content and labeling.
Surveys
Solicit structured feedback from
users.
User Workflow Modeling
Visually document workflows &
work systems.
Ideation Workshops
Immerse stakeholders in data and
brainstorm opportunities.
Persona Development
Create customer types to
document observed behaviors and
values.
Consolidated Workflow
Diagram
Aggregate individual user workflows
into one diagram.
Current Journey Map
Visualize the user’s perspective of
the current experience.
Quantitative Data
Visualization
Present quantitative data visually.
Opportunities Generation &
Evaluation
Opportunities & prototype choice.
Opportunities Matrix
Prioritize in three dimensions,
including user experience impact.
Marketing Opportunities
Strategic planning of owned,
earned & paid online tactics.
Engagement Plan
Develop the strategic and tactical
plan to achieve the client’s goal.
Experience-Driven
Roadmap
Plan how great UX can be achieved
through the design.
Design Principals
Articulate design principals to
guide the design and development
process.
Requirements & User
Stories Definition
Write user stories based on
detailed user scenarios.
Application & Navigation
Framework
Validated navigation and
framework.
Wireframed Key Workflows
Validated wireframes of key
workflows.
Visual Design Language
Visual design language defined.
Information Architecture
Map the product from the users’
point of view.
Storyboards
Illustrate graphical representations
of scenarios.
Qualitative & Quantitative
Data Synthesis
Analyze validation data..
User Scenarios
Write detailed narratives for user
experience flows.
Future Journey Map
Visualize the user’s future,
improved experience.
Concept Validation
Validate design prototypes through
user feedback.
Application & Navigation
Framework Concepts
Create models for the navigation &
framework of the application or
site.
Workflow Concepts
Draw high-level wireframes for key
workflows.
Visual Exploration
Explore different visual treatments
and styles of the application or site.
KANO Feature Prioritization
Prioritize features with users
through KANO analysis.
we make software make sense.
TM

“Design Research directs
the strategy & design for
our software projects”

WHAT DOES
ANTHROPOLOGY
LOOK LIKE?

http://michaelvhurley.com/2013/01/04/my-worthless-degree/

Anthropologist?

CSI Vegas?

Me!

“Anthropology demands the
open-mindedness with which one
must look and listen, record in
astonishment, and wonder that
which one would not have been
able to guess"
– Margaret Mead
@Kel_Moran

@Kel_Moran

@Kel_Moran

WHAT IS
ETHNOGRAPHY?
10 things to know

1
Ethnography is the descriptive
study of people and cultures
@Kel_Moran

2
Ethnographic Research is…
An approach not a specific
method
@Kel_Moran

3
Ethnographic Research…
Favors qualitative over
quantitative - the ethnographer
seeks to provide “thick”
description
@Kel_Moran

;-}
;-/
;-)
Thick Description: Clifford Geertz @Kel_Moran

4
Ethnographic Research is…
Conducted in context, typically
over an extended period of time,
and holistic, seeking the wider
picture
@Kel_Moran

To understand the pen, you must
understand the paper.
@Kel_Moran

5
Ethnographic Research is…
Systematically conducted
but responsive to emerging
trends and themes.
@Kel_Moran

6
Ethnographic Research…
Utilizes key informants who can
act as guides and help provide
access to the community
(Also offer “reality checks”)
@Kel_Moran

7
Ethnographic Research…
Seeks out the insider (emic)
perspective and layers in outsider
(etic) insights and
interpretations
@Kel_Moran

8
Ethnographic Research is…
Generative - done to discover
new information, not to test
existing hypotheses
@Kel_Moran

“Never theorize before you have
data. Invariably you end up
twisting facts to suit theories
instead of theories to suit facts.”
- Sherlock Holmes
(Arthur Conan Doyle)
@Kel_Moran

9
Ethnographic Research…
Seeks to tease out the implicit, not
typically stated, features of a group
@Kel_Moran

“Making the familiar strange
and the strange familiar”
@Kel_Moran
http://www.gizmocrazed.com/2012/04/top-10-theories-on-how-the-world-will-end/

The Nacirema
@Kel_MoranThe Nacirema: Horace Miner

The Nacirema
American
@Kel_MoranThe Nacirema: Horace Miner

Ethnographic Research…
Is inclusive, typically including a
participatory component
10
@Kel_Moran

Texas BONUS!
In 1919 James Edwin
Pearce, chairman of the
Department of
Institutional History at the
University of Texas,
changed its name to the
Department of
Anthropology - making it
one of the first such
departments of its type.
@Kel_Moran

WHAT IS
DESIGN ETHNOGRAPHY?
3 things to know

@Kel_Moran
1
Design Ethnography…
Is a way “…to increase the success
probability of a new product or service
or, more appropriately, to reduce the
probability of failure specifically due to a
lack of understanding the basic
behaviors and frameworks of
consumers.”
Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,”
Design Management Journal (pp. 35-41). p.37

1
Design Ethnography…
Is a way “…to increase the success
probability of a new product or service
or, more appropriately, to reduce the
probability of failure specifically due to a
lack of understanding the basic
behaviors and frameworks of
consumers.”
Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,”
Design Management Journal (pp. 35-41). p.37 @Kel_Moran

75-95%
of
new
products
fail

75-95%
of
new
products
fail

2
Avoids an over-reliance on
self-reported data
(what they say is not always what they do)
@Kel_Moran

What does “green” mean?

3
Qualitative research, done in the context
(environment) of the intended users,
seeking to discover and understand
their problems from their viewpoint,
with the designer’s viewpoint used to
ideate potential solutions.
@Kel_Moran

INTEGRATING
IN-CONTEXT RESEARCH

Discovery Phase
• Designer pairs with a Researcher
• Researcher leads with a background in the social
sciences
• Designer assists
Researcher Designer
@Kel_Moran

Design Phase
• Roles switch, and the Researcher assists the Designer
ResearcherDesigner
@Kel_Moran

After Detailed Design
• Designer typically is embedded into the Development
Phase
• Researcher comes back on board for user testing
Designer
Developer (x N)
Researcher @Kel_Moran

Provides continuity and keeps the
user’s voice present
User Focused Innovation
Discovery Design Build
@Kel_Moran

CASE STUDY

ENTERPRISE SOFTWARE
ACCOUNT RECONCILIATIONS

The Client and the Product
Vendor of accounting software
• Customer feedback of “too many clicks” and “hard to use”
• Sales were lagging
Used in organizations with large, multi-
functional accounting departments
• Needed to be customizable
• Should fit within a suite of other enterprise products
@Kel_Moran

Research Basics
Who
• Nineteen users at six user sites, plus three users inside the client
company
Where
• Seven locations across four states
• 2-4 users at each location
@Kel_Moran

From 22 users:
Around 800 unique
insights and observations
were recorded
@Kel_Moran

IN-CONTEXT OBSERVATION
@Kel_Moran

“We have everything set up
for you in a
conference room.”
@Kel_Moran

What do you see?
@Kel_Moran

Contextual Learnings
• 5 out of 6 of the client-user groups observed used two monitors
• adding machines still in use
• typically in either a cube or an open workspace
• paper everywhere - post its galore!
• User observed walking to a locked room (with a broken ankle)
to look up reference numbers
@Kel_Moran

CONTEXTUAL LESSONS
FROM USERS
@Kel_Moran

Contextual Learnings
With a lot on their minds, Excel is king
• High cognitive load
• High use of Excel quick keys
• Importing data of multiple types into Excel as images
• Highlighting and/or circling important data
http://icons8.com
@Kel_Moran

“It’s better than having it
rejected back.”
@Kel_Moran

Contextual Learnings (continued)
Attachments and cover sheets
• Manager preference for a single attachment
• Printing out documents from multiple sources to scan
them into one attachment
• Making a Custom “cover sheet” in Excel to sum up the
work
@Kel_Moran

“I look at this as
basically
just a holding station.”
@Kel_Moran

Contextual Learnings (continued)
“Roles” as defined in the software did not match the
work-based roles of the users
• In several locations a “reconciler” also held the role of
“administrator,” but juggling these in the software was
cumbersome
• The administrator role could become overwhelming
Assumed User Role with Linear Workflow
Observed Interrupted User/Admin Workflow
Start
Administrative Work
Start End
User Work
End
User WorkUser Work
Interruption
@Kel_Moran

“I have to change gear…
to move to that other role.”
@Kel_Moran

“Soon I won’t have any
accounting work. It will all
be [administrative] work.”
@Kel_Moran

Contextual Learnings (continued)
The problem goes beyond the system
• Lack of technical support
• The “real work” was done outside the system
@Kel_Moran

Now what do we do
with all that data?
@Kel_Moran

Synthesizing the Data - Major Themes
1. Learning, training, &
support
2. Don’t make it hard on us
3. We know who we are and
what we need
4. We need an agile, smart
workflow and tracking
system
5. Don’t add to our cognitive
load
6. Be our partner
7. We have a lot going on
besides what you do for us
8. Collaboration, teamwork,
communication, & working
with others
@Kel_Moran

Users viewed the system
as supplementary, and
part of a larger process.
Organize the
system’s
workstream to
match the user’s
view.
Cover sheets were
created and
standardized by each
accounting group.
Renaming the
“account home
page” the “cover
sheet” and
emphasizing its
similarities to the
Excel-made
coversheets.
High use of Excel.
Bringing in more
Excel patterns.
Observations Lead to Design Ideas
@Kel_Moran

Observations Lead to Design Ideas (continued)
Uploading and viewing
attachments was a
common problem; and it
didn’t match the rest of
the experience.
Making both
uploading and
viewing
attachments
easier and more
consistent with
the rest of the UI.
Inconsistent UI was
jarring to navigate.
Overall more
consistent UI.
Users had to scan the
page to find the most
relevant information on
each screen.
Putting the
information the
users needed
most to keep in
mind in a
prominent, and
persistent, place
on every screen.
@Kel_Moran

Design Ideas get Prioritized

TRANSITIONING
INTO DESIGN
@Kel_Moran

Old

New

VALIDATION TESTING
@Kel_Moran

Three Pages Tested
Two Versions Each
• Dashboard
• List Page - results of search
• Refined filtering
• Account Details - account home; i.e. cover sheet
@Kel_Moran
(revised and re-tested for a total of three testing rounds)

User-Led Validation and Changes
• Most users don’t need the 6-currency view panel
• Changed the default to 3 currencies
• Accountants need to be sure their entries save
• Ties back to the generative research
• Added a “save” confirmation button instead of auto-saving
• Attaching files to the cover page was validated as
preferable to using a separate attachments page
@Kel_Moran

BRINGING ETHNOGRAPHY
TO YOUR PROJECTS

Start Small - Start Anywhere!
Does not need to cover 7 regions across a continent
(seeing a few users makes a difference)
Get into the environment
(try Skype if needed)
Don’t go in with solutions in mind
(don’t be too sure you know the problems already either)
@Kel_Moran

Obtain Informed Consent
@Kel_Moran

How to Observe
The physical environment
- open? cramped? hot? cold?
Lighting, noise
- pleasant? distracting? manipulated by the user?
People
- who interacts with whom?
Artifacts
- equipment, paper notes, binders…
Document it all
- notes, yes, but also photographs and audio/video if permitted.
@Kel_Moran

How to Ask Questions
You are not an expert in their work/play
- even if you think you are
Rephrase what they say and ask if you got it right
- let them correct you even if you know you did
Avoid leading questions
- ask them to describe instead
Take note of their ideas and ask “What problem does this solve?”
- they’re not the experts at finding solutions, but they tried their best
@Kel_Moran

Honor the Idea of Reciprocity
Your users (research participants) are giving
you something, so be sure to give
something back.
@Kel_Moran

RECOMMENDED
READING

: https://www.flickr.com/photos/101187156@N03/14366224997/
Kelly Moran
Lead Design Researcher
Thank You
@Kel_Moran#bigd15

SUGGESTED
RESOURCES

How Tos
Brian A. Hoey. "A Simple Introduction to the Practice of Ethnography and Guide to Ethnographic
Fieldnotes" Marshall University Digital Scholar (2014): 1-10.
Available at: http://works.bepress.com/brian_hoey/12
Methods of Discovery a Guide to Research Writing http://methodsofdiscovery.net/?q=node/19
Ethnography https://www.academia.edu/1022047/
Picken_F._2009_What_is_Ethnography_in_M._Walter_Ed_Social_Research_Methods_Melbourne_OUP
What is Ethnography? http://www.cusag.umd.edu/documents/workingpapers/epiontattrib.pdf
Articles
Horace Miner’s (1956) article “Body Ritual among the Nacirema”
Available at: https://www.msu.edu/~jdowell/miner.html or at: http://www.sfu.ca/~palys/Miner-1956-
BodyRitualAmongTheNacirema.pdf
Salvador, Tony; Genevieve Bell; and Ken Anderson. "Design Ethnography," Design Management Journal. (1999) (pp.
35-41). p.37
Books
Geertz, Clifford. The Interpretation of Cultures: Selected Essays. Basic Books, Inc. (1973) 470pp.