An updated, refreshed version of the talk I gave in Poland, this is the 2015 Big Design slide deck of "Ethnography in Software Design: An Anthropologist's Point of View."

1. Ethnography in Software Design
An Anthropologist’s Point of View
Big Design - Dallas, September 2015
: https://www.flickr.com/photos/101187156@N03/14366224997/
@Kel_Moran#bigd15

2. Kelly
Moran
Lead Design Researcher
@Kel_Moran
#bigd15

3. MS Applied Anthropology
University of North Texas
BA Anthropology
California State U. Northridge

4. US Peace Corps
Uzbekistan
Study Abroad
Saudi Arabia
In
the
Wild

5. Worked with:

6. “Uncover user needs,
Design great solutions,
and build out solutions to launch.”

7. “Uncover user needs,
Design great solutions,
and build out solutions to launch.”

8. DistillMeaningfromObservation
BuildtheBacklog
CONTACT BUILDING & EVOLVINGFOCUSED INNOVATION
Only a subset of these activities will be appropriate for any given project.
Qualitative Research
Quantitative Research
Affinity Diagramming
Construct themes from qualitative
data.
Analysis & Synthesis Opportunities
Ideation & Iteration
Validation & Evaluation
Approach Planning Design Research Experience Strategy & Strategic Ideation PlanningDefinition
IdentifyChallengesinContext
SettheStage
REVEALING REALITY
Foundational Analysis
Heuristic Evaluation
Identify inital breakdowns and
opportunities
Digital Marketing SWOT
Analysis
Identify strengths, weaknesses,
opportunities & theats.
Stakeholder Interviews
Understand staheholders’ business
goals & strategy.
Technical Organization
Capability Aanalysis
Analyze existing skills and toolsets.
Scenario-based System
Walkthroughs
Demo of existing solution.
Metrics Evaluation
Establish quantitative baseline of
engagement & conversion data.
Content Inventory
Catalog the content of the site or
application.
Competitive Analysis
Evaluate competitors and
comparables on specific axes.
User Experience
Data
Technology
Enterprise Architecture
Capability Assesment
Review existing enterprise
technology infrastructure.
Solution(s) Architecture
Assessment
Review existing application(s)
architecture.
RFP Request
Deliver proposal to defined scope
document and existing
requirements.
Project Approach
Asses a possible project and plan
high level approach.
Align & Assess Workshop
Assess readiness across core
capabilities.
Backlog Grooming
Reprioritize backlog, add new
stories.
Zero Feature Release
Demonstrate CI, automated
testing, core solution setup.
High Level Technical
Architecture
Describe high level architecture,
including packaged components.
Development
Infrastructure
Configuration
Continuous integration setup.
Technical Package
Identification & Evaluation
Perform product evaluations for
package solution components.
Architecture Spikes &
Proofs of Concept
Prove candidate architectures via
top bottom spikes.
Existing Research Review
Market research, website feedback,
corporate strategy, etc.
BringtheSolutionintoFocus
Research Plan
Design activities to meet research
goals.
Contextual Inquiries
Observe & document user in
context & environment.
User Journals & Diaries
User document their experiences
over time.
Participatory Design
Co-creation explorations with
users.
Card Sort
Explore users’ mental models for
content and labeling.
Surveys
Solicit structured feedback from
users.
User Workflow Modeling
Visually document workflows &
work systems.
Ideation Workshops
Immerse stakeholders in data and
brainstorm opportunities.
Persona Development
Create customer types to
document observed behaviors and
values.
Consolidated Workflow
Diagram
Aggregate individual user workflows
into one diagram.
Current Journey Map
Visualize the user’s perspective of
the current experience.
Quantitative Data
Visualization
Present quantitative data visually.
Opportunities Generation &
Evaluation
Opportunities & prototype choice.
Opportunities Matrix
Prioritize in three dimensions,
including user experience impact.
Marketing Opportunities
Strategic planning of owned,
earned & paid online tactics.
Engagement Plan
Develop the strategic and tactical
plan to achieve the client’s goal.
Experience-Driven
Roadmap
Plan how great UX can be achieved
through the design.
Design Principals
Articulate design principals to
guide the design and development
process.
Requirements & User
Stories Definition
Write user stories based on
detailed user scenarios.
Application & Navigation
Framework
Validated navigation and
framework.
Wireframed Key Workflows
Validated wireframes of key
workflows.
Visual Design Language
Visual design language defined.
Information Architecture
Map the product from the users’
point of view.
Storyboards
Illustrate graphical representations
of scenarios.
Qualitative & Quantitative
Data Synthesis
Analyze validation data..
User Scenarios
Write detailed narratives for user
experience flows.
Future Journey Map
Visualize the user’s future,
improved experience.
Concept Validation
Validate design prototypes through
user feedback.
Application & Navigation
Framework Concepts
Create models for the navigation &
framework of the application or
site.
Workflow Concepts
Draw high-level wireframes for key
workflows.
Visual Exploration
Explore different visual treatments
and styles of the application or site.
KANO Feature Prioritization
Prioritize features with users
through KANO analysis.
we make software make sense.
TM

9. “Design Research directs
the strategy & design for
our software projects”

11. WHAT DOES
ANTHROPOLOGY
LOOK LIKE?

12. http://michaelvhurley.com/2013/01/04/my-worthless-degree/

13. Anthropologist?

14. CSI Vegas?

15. Me!

16. “Anthropology demands the
open-mindedness with which one
must look and listen, record in
astonishment, and wonder that
which one would not have been
able to guess"
– Margaret Mead
@Kel_Moran

17. @Kel_Moran

18. @Kel_Moran

19. WHAT IS
ETHNOGRAPHY?
10 things to know

20. 1
Ethnography is the descriptive
study of people and cultures
@Kel_Moran

21. 2
Ethnographic Research is…
An approach not a specific
method
@Kel_Moran

22. 3
Ethnographic Research…
Favors qualitative over
quantitative - the ethnographer
seeks to provide “thick”
description
@Kel_Moran

23. ;-}
;-/
;-)
Thick Description: Clifford Geertz @Kel_Moran

24. 4
Ethnographic Research is…
Conducted in context, typically
over an extended period of time,
and holistic, seeking the wider
picture
@Kel_Moran

25. To understand the pen, you must
understand the paper.
@Kel_Moran

26. 5
Ethnographic Research is…
Systematically conducted
but responsive to emerging
trends and themes.
@Kel_Moran

27. 6
Ethnographic Research…
Utilizes key informants who can
act as guides and help provide
access to the community
(Also offer “reality checks”)
@Kel_Moran

28. 7
Ethnographic Research…
Seeks out the insider (emic)
perspective and layers in outsider
(etic) insights and
interpretations
@Kel_Moran

30. 8
Ethnographic Research is…
Generative - done to discover
new information, not to test
existing hypotheses
@Kel_Moran

31. “Never theorize before you have
data. Invariably you end up
twisting facts to suit theories
instead of theories to suit facts.”
- Sherlock Holmes
(Arthur Conan Doyle)
@Kel_Moran

32. 9
Ethnographic Research…
Seeks to tease out the implicit, not
typically stated, features of a group
@Kel_Moran

33. “Making the familiar strange
and the strange familiar”
@Kel_Moran
http://www.gizmocrazed.com/2012/04/top-10-theories-on-how-the-world-will-end/

34. The Nacirema
@Kel_MoranThe Nacirema: Horace Miner

35. The Nacirema
American
@Kel_MoranThe Nacirema: Horace Miner

36. Ethnographic Research…
Is inclusive, typically including a
participatory component
10
@Kel_Moran

37. Texas BONUS!
In 1919 James Edwin
Pearce, chairman of the
Department of
Institutional History at the
University of Texas,
changed its name to the
Department of
Anthropology - making it
one of the first such
departments of its type.
@Kel_Moran

38. WHAT IS
DESIGN ETHNOGRAPHY?
3 things to know

39. @Kel_Moran
1
Design Ethnography…
Is a way “…to increase the success
probability of a new product or service
or, more appropriately, to reduce the
probability of failure specifically due to a
lack of understanding the basic
behaviors and frameworks of
consumers.”
Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,”
Design Management Journal (pp. 35-41). p.37

40. 1
Design Ethnography…
Is a way “…to increase the success
probability of a new product or service
or, more appropriately, to reduce the
probability of failure specifically due to a
lack of understanding the basic
behaviors and frameworks of
consumers.”
Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,”
Design Management Journal (pp. 35-41). p.37 @Kel_Moran

41. 75-95%
of
new
products
fail

42. 75-95%
of
new
products
fail

43. 2
Avoids an over-reliance on
self-reported data
(what they say is not always what they do)
@Kel_Moran

44. What does “green” mean?

47. 3
Qualitative research, done in the context
(environment) of the intended users,
seeking to discover and understand
their problems from their viewpoint,
with the designer’s viewpoint used to
ideate potential solutions.
@Kel_Moran

48. INTEGRATING
IN-CONTEXT RESEARCH

49. Discovery Phase
• Designer pairs with a Researcher
• Researcher leads with a background in the social
sciences
• Designer assists
Researcher Designer
@Kel_Moran

50. Design Phase
• Roles switch, and the Researcher assists the Designer
ResearcherDesigner
@Kel_Moran

51. After Detailed Design
• Designer typically is embedded into the Development
Phase
• Researcher comes back on board for user testing
Designer
Developer (x N)
Researcher @Kel_Moran

52. Provides continuity and keeps the
user’s voice present
User Focused Innovation
Discovery Design Build
@Kel_Moran

54. CASE STUDY

55. ENTERPRISE SOFTWARE
ACCOUNT RECONCILIATIONS

56. The Client and the Product
Vendor of accounting software
• Customer feedback of “too many clicks” and “hard to use”
• Sales were lagging
Used in organizations with large, multi-
functional accounting departments
• Needed to be customizable
• Should fit within a suite of other enterprise products
@Kel_Moran

57. Research Basics
Who
• Nineteen users at six user sites, plus three users inside the client
company
Where
• Seven locations across four states
• 2-4 users at each location
@Kel_Moran

58. From 22 users:
Around 800 unique
insights and observations
were recorded
@Kel_Moran

59. IN-CONTEXT OBSERVATION
@Kel_Moran

60. “We have everything set up
for you in a
conference room.”
@Kel_Moran

61. What do you see?
@Kel_Moran

62. Contextual Learnings
• 5 out of 6 of the client-user groups observed used two monitors
• adding machines still in use
• typically in either a cube or an open workspace
• paper everywhere - post its galore!
• User observed walking to a locked room (with a broken ankle)
to look up reference numbers
@Kel_Moran

63. CONTEXTUAL LESSONS
FROM USERS
@Kel_Moran

64. Contextual Learnings
With a lot on their minds, Excel is king
• High cognitive load
• High use of Excel quick keys
• Importing data of multiple types into Excel as images
• Highlighting and/or circling important data
http://icons8.com
@Kel_Moran

65. “It’s better than having it
rejected back.”
@Kel_Moran

66. Contextual Learnings (continued)
Attachments and cover sheets
• Manager preference for a single attachment
• Printing out documents from multiple sources to scan
them into one attachment
• Making a Custom “cover sheet” in Excel to sum up the
work
@Kel_Moran

67. “I look at this as
basically
just a holding station.”
@Kel_Moran

68. Contextual Learnings (continued)
“Roles” as defined in the software did not match the
work-based roles of the users
• In several locations a “reconciler” also held the role of
“administrator,” but juggling these in the software was
cumbersome
• The administrator role could become overwhelming
Assumed User Role with Linear Workflow
Observed Interrupted User/Admin Workflow
Start
Administrative Work
Start End
User Work
End
User WorkUser Work
Interruption
@Kel_Moran

69. “I have to change gear…
to move to that other role.”
@Kel_Moran

70. “Soon I won’t have any
accounting work. It will all
be [administrative] work.”
@Kel_Moran

71. Contextual Learnings (continued)
The problem goes beyond the system
• Lack of technical support
• The “real work” was done outside the system
@Kel_Moran

72. Now what do we do
with all that data?
@Kel_Moran

74. Synthesizing the Data - Major Themes
1. Learning, training, &
support
2. Don’t make it hard on us
3. We know who we are and
what we need
4. We need an agile, smart
workflow and tracking
system
5. Don’t add to our cognitive
load
6. Be our partner
7. We have a lot going on
besides what you do for us
8. Collaboration, teamwork,
communication, & working
with others
@Kel_Moran

75. Users viewed the system
as supplementary, and
part of a larger process.
Organize the
system’s
workstream to
match the user’s
view.
Cover sheets were
created and
standardized by each
accounting group.
Renaming the
“account home
page” the “cover
sheet” and
emphasizing its
similarities to the
Excel-made
coversheets.
High use of Excel.
Bringing in more
Excel patterns.
Observations Lead to Design Ideas
@Kel_Moran

76. Observations Lead to Design Ideas (continued)
Uploading and viewing
attachments was a
common problem; and it
didn’t match the rest of
the experience.
Making both
uploading and
viewing
attachments
easier and more
consistent with
the rest of the UI.
Inconsistent UI was
jarring to navigate.
Overall more
consistent UI.
Users had to scan the
page to find the most
relevant information on
each screen.
Putting the
information the
users needed
most to keep in
mind in a
prominent, and
persistent, place
on every screen.
@Kel_Moran

77. Design Ideas get Prioritized

78. TRANSITIONING
INTO DESIGN
@Kel_Moran

79. Old

80. New

81. VALIDATION TESTING
@Kel_Moran

82. Three Pages Tested
Two Versions Each
• Dashboard
• List Page - results of search
• Refined filtering
• Account Details - account home; i.e. cover sheet
@Kel_Moran
(revised and re-tested for a total of three testing rounds)

83. User-Led Validation and Changes
• Most users don’t need the 6-currency view panel
• Changed the default to 3 currencies
• Accountants need to be sure their entries save
• Ties back to the generative research
• Added a “save” confirmation button instead of auto-saving
• Attaching files to the cover page was validated as
preferable to using a separate attachments page
@Kel_Moran

84. BRINGING ETHNOGRAPHY
TO YOUR PROJECTS

85. Start Small - Start Anywhere!
Does not need to cover 7 regions across a continent
(seeing a few users makes a difference)
Get into the environment
(try Skype if needed)
Don’t go in with solutions in mind
(don’t be too sure you know the problems already either)
@Kel_Moran

86. Obtain Informed Consent
@Kel_Moran

87. How to Observe
The physical environment
- open? cramped? hot? cold?
Lighting, noise
- pleasant? distracting? manipulated by the user?
People
- who interacts with whom?
Artifacts
- equipment, paper notes, binders…
Document it all
- notes, yes, but also photographs and audio/video if permitted.
@Kel_Moran

88. How to Ask Questions
You are not an expert in their work/play
- even if you think you are
Rephrase what they say and ask if you got it right
- let them correct you even if you know you did
Avoid leading questions
- ask them to describe instead
Take note of their ideas and ask “What problem does this solve?”
- they’re not the experts at finding solutions, but they tried their best
@Kel_Moran

90. Honor the Idea of Reciprocity
Your users (research participants) are giving
you something, so be sure to give
something back.
@Kel_Moran

91. RECOMMENDED
READING

95. : https://www.flickr.com/photos/101187156@N03/14366224997/
Kelly Moran
Lead Design Researcher
Thank You
@Kel_Moran#bigd15

96. SUGGESTED
RESOURCES

97. How Tos
Brian A. Hoey. "A Simple Introduction to the Practice of Ethnography and Guide to Ethnographic
Fieldnotes" Marshall University Digital Scholar (2014): 1-10.
Available at: http://works.bepress.com/brian_hoey/12
Methods of Discovery a Guide to Research Writing http://methodsofdiscovery.net/?q=node/19
Ethnography https://www.academia.edu/1022047/
Picken_F._2009_What_is_Ethnography_in_M._Walter_Ed_Social_Research_Methods_Melbourne_OUP
What is Ethnography? http://www.cusag.umd.edu/documents/workingpapers/epiontattrib.pdf
Articles
Horace Miner’s (1956) article “Body Ritual among the Nacirema”
Available at: https://www.msu.edu/~jdowell/miner.html or at: http://www.sfu.ca/~palys/Miner-1956-
BodyRitualAmongTheNacirema.pdf
Salvador, Tony; Genevieve Bell; and Ken Anderson. "Design Ethnography," Design Management Journal. (1999) (pp.
35-41). p.37
Books
Geertz, Clifford. The Interpretation of Cultures: Selected Essays. Basic Books, Inc. (1973) 470pp.