Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ethnography in Software Design - An Anthropologist's Perspective

2,833 views

Published on

Ethnography claims its roots from the field of anthropology. How can a technique used for such a seemingly exotic purpose be useful in the modern world of software design? Revealing and most importantly understanding user needs requires sensitivity, empathy, and a disciplined approach – all of which can be found within ethnography. This talk outlines the basic components of an ethnographic perspective, explores a case study from a recent engagement between projekt202 and an enterprise software company, and highlights how the impact of this research ripples through the software development process.

Published in: Software

Ethnography in Software Design - An Anthropologist's Perspective

  1. 1. Ethnography in Software Design 
 An Anthropologist’s Point of View ACE! Conference 2015, Krakow Poland : https://www.flickr.com/photos/101187156@N03/14366224997/
  2. 2. Kelly Moran Lead Design Researcher
  3. 3. @Kel_Moran
  4. 4. Education @ CSUN
  5. 5. Early Influences
  6. 6. Uzbekistan
  7. 7. Masters Program @ UNT
  8. 8. Study Abroad @ Saudi Arabia
  9. 9. Worked for:
  10. 10. Worked with:
  11. 11. “Uncover user needs, 
 Design great solutions, 
 and build out solutions to launch.”
  12. 12. “Uncover user needs, 
 Design great solutions, 
 and build out solutions to launch.”
  13. 13. DistillMeaningfromObservation BuildtheBacklog CONTACT BUILDING & EVOLVINGFOCUSED INNOVATION Only a subset of these activities will be appropriate for any given project. Qualitative Research Quantitative Research Affinity Diagramming Construct themes from qualitative data. Analysis & Synthesis Opportunities Ideation & Iteration Validation & Evaluation Approach Planning Design Research Experience Strategy & Strategic Ideation PlanningDefinition IdentifyChallengesinContext SettheStage REVEALING REALITY Foundational Analysis Heuristic Evaluation Identify inital breakdowns and opportunities Digital Marketing SWOT Analysis Identify strengths, weaknesses, opportunities & theats. Stakeholder Interviews Understand staheholders’ business goals & strategy. Technical Organization Capability Aanalysis Analyze existing skills and toolsets. Scenario-based System Walkthroughs Demo of existing solution. Metrics Evaluation Establish quantitative baseline of engagement & conversion data. Content Inventory Catalog the content of the site or application. Competitive Analysis Evaluate competitors and comparables on specific axes. User Experience Data Technology Enterprise Architecture Capability Assesment Review existing enterprise technology infrastructure. Solution(s) Architecture Assessment Review existing application(s) architecture. RFP Request Deliver proposal to defined scope document and existing requirements. Project Approach Asses a possible project and plan high level approach. Align & Assess Workshop Assess readiness across core capabilities. Backlog Grooming Reprioritize backlog, add new stories. Zero Feature Release Demonstrate CI, automated testing, core solution setup. High Level Technical Architecture Describe high level architecture, including packaged components. Development Infrastructure Configuration Continuous integration setup. Technical Package Identification & Evaluation Perform product evaluations for package solution components. Architecture Spikes & Proofs of Concept Prove candidate architectures via top bottom spikes. Existing Research Review Market research, website feedback, corporate strategy, etc. BringtheSolutionintoFocus Research Plan Design activities to meet research goals. Contextual Inquiries Observe & document user in context & environment. User Journals & Diaries User document their experiences over time. Participatory Design Co-creation explorations with users. Card Sort Explore users’ mental models for content and labeling. Surveys Solicit structured feedback from users. User Workflow Modeling Visually document workflows & work systems. Ideation Workshops Immerse stakeholders in data and brainstorm opportunities. Persona Development Create customer types to document observed behaviors and values. Consolidated Workflow Diagram Aggregate individual user workflows into one diagram. Current Journey Map Visualize the user’s perspective of the current experience. Quantitative Data Visualization Present quantitative data visually. Opportunities Generation & Evaluation Opportunities & prototype choice. Opportunities Matrix Prioritize in three dimensions, including user experience impact. Marketing Opportunities Strategic planning of owned, earned & paid online tactics. Engagement Plan Develop the strategic and tactical plan to achieve the client’s goal. Experience-Driven Roadmap Plan how great UX can be achieved through the design. Design Principals Articulate design principals to guide the design and development process. Requirements & User Stories Definition Write user stories based on detailed user scenarios. Application & Navigation Framework Validated navigation and framework. Wireframed Key Workflows Validated wireframes of key workflows. Visual Design Language Visual design language defined. Information Architecture Map the product from the users’ point of view. Storyboards Illustrate graphical representations of scenarios. Qualitative & Quantitative Data Synthesis Analyze validation data.. User Scenarios Write detailed narratives for user experience flows. Future Journey Map Visualize the user’s future, improved experience. Concept Validation Validate design prototypes through user feedback. Application & Navigation Framework Concepts Create models for the navigation & framework of the application or site. Workflow Concepts Draw high-level wireframes for key workflows. Visual Exploration Explore different visual treatments and styles of the application or site. KANO Feature Prioritization Prioritize features with users through KANO analysis. we make software make sense. TM
  14. 14. Heuristic Evaluation Findings and themes from the research synthesis Personas Consolidated Workflow Diagram Journey Map Stakeholder workshop: Findings presentation and Opportu Experience Principles Opportunities generation and consolidation Documentation/Data Review Identify participants, begin coordinating recruiting Scenario-based Product Walkthroughs Stakeholder Interviews SME Consultations Heuristic Evaluation Revealing Reality Continue contextual inquiry (CI) participant scheduling CI protocol development
  15. 15. “Design Research directs the strategy & design for our software projects”
  16. 16. WHAT DOES 
 ANTHROPOLOGY LOOK LIKE?
  17. 17. http://michaelvhurley.com/2013/01/04/my-worthless-degree/
  18. 18. Anthropologist?
  19. 19. CSI Vegas?
  20. 20. Me!
  21. 21. “Anthropology demands the open-mindedness with which one must look and listen, record in astonishment, and wonder that which one would not have been able to guess" – Margaret Mead
  22. 22. WHAT IS 
 ETHNOGRAPHY? 10 things to know
  23. 23. 1 Ethnography is the descriptive study of people and cultures @Kel_Moran
  24. 24. 2 Ethnographic Research is… An approach not a specific method @Kel_Moran
  25. 25. 3 Ethnographic Research… Favors qualitative over quantitative - the ethnographer seeks to provide “thick” description @Kel_Moran
  26. 26. ;-} ;-/ ;-)
  27. 27. 4 Ethnographic Research is… Conducted in context, typically over an extended period of time, and holistic, seeking the wider picture @Kel_Moran
  28. 28. To understand the pen, you must understand the paper.
  29. 29. 5 Ethnographic Research is… Systematically (purposefully) conducted, but is responsive to emerging trends and themes (flexible). @Kel_Moran
  30. 30. 6 Ethnographic Research… Utilizes key informants who can act as guides and help provide access to the community (Also offer “reality checks”) @Kel_Moran
  31. 31. 7 Ethnographic Research… Seeks out the insider (emic) perspective and layers in outsider (etic) insights and interpretations @Kel_Moran
  32. 32. 8 Ethnographic Research is… Generative - done to discover new information, not to test existing hypotheses @Kel_Moran
  33. 33. “Never theorize before you have data. Invariably you end up twisting facts to suit theories instead of theories to suit facts.” - Sherlock Holmes (Arthur Conan Doyle)
  34. 34. 9 Ethnographic Research… Seeks to tease out the implicit, not typically stated, features of a group “Making the familiar strange and the strange familiar” @Kel_Moran
  35. 35. The Nacirema
  36. 36. The Nacirema American
  37. 37. Ethnographic Research… Is inclusive, typically including a participatory component 10 @Kel_Moran
  38. 38. BONUS! Bronislaw Malinowski was a Polish born Anthropologist who pioneered extended fieldwork. @Kel_Moran
  39. 39. WHAT IS 
 DESIGN ETHNOGRAPHY? 3 things to know
  40. 40. 1 Design Ethnography… Is a way “…to increase the success probability of a new product or service or, more appropriately, to reduce the probability of failure specifically due to a lack of understanding the basic behaviors and frameworks of consumers.” Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,” Design Management Journal (pp. 35-41). p.37 @Kel_Moran
  41. 41. 1 Design Ethnography… Is a way “…to increase the success probability of a new product or service or, more appropriately, to reduce the probability of failure specifically due to a lack of understanding the basic behaviors and frameworks of consumers.” Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,” Design Management Journal (pp. 35-41). p.37 @Kel_Moran
  42. 42. 2 Avoids an over-reliance on self-reported data 
 (what they say is not always what they do) @Kel_Moran
  43. 43. 3 Qualitative research, done in the context (environment) of the intended users, seeking to discover and understand their problems from their viewpoint, with the designer’s viewpoint used to ideate potential solutions. @Kel_Moran
  44. 44. INTEGRATING
 IN-CONTEXT RESEARCH .
  45. 45. Discovery Phase • Designer pairs with a Researcher • Researcher leads with a background in the social sciences • Designer assists Researcher Designer
  46. 46. Design Phase • Roles switch, and the Researcher assists the Designer ResearcherDesigner
  47. 47. After Detailed Design • Designer typically is embedded into the Development Phase • Researcher comes back on board for user testing DesignerDeveloper x N Researcher
  48. 48. Discovery Design Build Provides continuity and keeps the user’s voice present User Focused Innovation
  49. 49. CASE STUDY
  50. 50. ENTERPRISE SOFTWARE ACCOUNT RECONCILIATIONS
  51. 51. The Client and the Product Vendor of accounting software • Customer feedback of “too many clicks” and “hard to use” • Sales were lagging Used in organizations with large, multi-functional accounting departments • Needed to be customizable • Should fit within a suite of other enterprise products
  52. 52. Research Basics Who • 19 users at 6 user sites, plus 3 users inside the client company • = 22 total observed users • Both primary reconcilers (doers) and reviewers (managers) Where • 7 locations across 4 states • 2-4 users at each location
  53. 53. From 22 users: Around 800 unique insights and observations were recorded
  54. 54. IN-CONTEXT OBSERVATION
  55. 55. “We have everything set up for you in a conference room.”
  56. 56. What do you see?
  57. 57. Contextual Learnings • 5 out of 6 of the client-user groups observed used two monitors • adding machines (calculators) still in use • typically in either a cube or an open workspace • = lots of noise and movement • users “get into a zone” to focus on their work • paper everywhere • one user had multiple post-it notes with the same number for customer support displayed near the phone • User observed walking to a locked room (with a broken ankle) to look up reference numbers
  58. 58. CONTEXTUAL LESSONS FROM USERS
  59. 59. Contextual Learnings • With a lot on their minds, Excel is king • High cognitive load • High use of Excel quick keys • Importing data of multiple types into Excel as images • Highlighting and/or circling important data http://icons8.com
  60. 60. “It’s better than having it rejected back.”
  61. 61. Contextual Learnings (continued) • Attachments and cover sheets • Manager preference for a single attachment • Printing out documents from multiple sources to scan them into one attachment • Making a Custom “cover sheet” in Excel to sum up the work
  62. 62. “I look at [the product] as basically just a holding station.”
  63. 63. Contextual Learnings (continued) • “Roles” as defined in the software did not match the work-based roles of the users • In several locations a “reconciler” also held the role of “administrator,” but juggling these in the software was cumbersome • The administrator role could become overwhelming Assumed User Role with Linear Workflow Observed Interrupted User/Admin Workflow Start Administrative Work Start End User Work End User WorkUser Work Interruption
  64. 64. “I have to change gear…to move to that other role.”
  65. 65. “Soon I won’t have any accounting work. It will all be [administrative] work.”
  66. 66. Contextual Learnings (continued) • The problem goes beyond the system • Lack of technical support • The “real work” was done outside the system before we arrived.
  67. 67. EXTENDED RESEARCH FOLLOW-UP RETROSPECTIVES
  68. 68. Understanding What We Didn’t See • Visited 4 new groups • Plus an internal check with the client’s accounting department • Increased our overall reach • Used a custom-built research activity • Helped us understand how the users view their work stream, and where the software fits in
  69. 69. Now what do we do with all that data?
  70. 70. Synthesizing the Data - Major Insights 1. Learning, training, & support 2. Don’t make it hard on us 3. We know who we are and what we need 4. We need an agile, smart workflow and tracking system 5. Don’t add to our cognitive load 6. Be our partner 7. We have a lot going on besides what you do for us 8. Collaboration, teamwork, communication, & working with others
  71. 71. Insights Lead to Concepts
  72. 72. TRANSITIONING INTO DESIGN
  73. 73. Concepts Lead to Design • Organizing the system’s workstream to match the user’s view. • Renaming the “account home page” the “cover sheet” and emphasizing its similarities to the Excel-made coversheets. • Bringing in more Excel patterns • Users viewed the system as supplementary, and part of a larger process. • Cover sheets were created and standardized by each accounting group. • Observed high use of Excel.
  74. 74. Concepts Lead to Design (continued) • Making both uploading and viewing attachments easier and more consistent with the rest of the UI. • Overall more consistent UI • Putting the information the users needed most to keep in mind in a prominent, and persistent, place on every screen. • Uploading and viewing attachments was a common problem; and it didn’t match the rest of the experience. • Inconsistent UI was jarring to navigate. • Users had to scan the page to find the most relevant information on each screen.
  75. 75. VALIDATION TESTING
  76. 76. Three Pages Tested - Two Versions Each (revised and re-tested for a total of three testing rounds) • Reconciler Dashboard • List Page - results of search • Refined filtering • Details page - account home; i.e. cover sheet
  77. 77. User-Led Validation and Changes • Most users don’t need the 6-currency view panel • Changed the default to 3 currencies • Accountants need to be sure their entries save • Ties back to the generative research • Added a “save” confirmation button instead of auto-saving • “Build-a-filter” concept performed better than the “expose all filters” option • Attaching files to the cover page was validated as preferable to using a separate attachments page
  78. 78. BRINGING ETHNOGRAPHY
 TO YOUR PROJECTS .
  79. 79. Start Small - Start Anywhere! Does not need to cover 7 regions across a continent (seeing a few users makes a difference) Get into the environment (try Skype if needed) Don’t go in with solutions in mind (don’t be too sure you know the problems already either) @Kel_Moran
  80. 80. How to Observe The physical environment - open? cramped? hot? cold? Lighting, noise - pleasant? distracting? manipulated by the user? People - who interacts with whom? Artifacts - equipment, paper notes, binders… Document it all - notes, yes, but also photographs and audio/video if permitted. @Kel_Moran
  81. 81. How to Ask Questions Be respectful, but don’t be shy - they have a job to do, and so do you You are not an expert in their work/play - even if you think you are Rephrase what they say and ask if you got it right - let them correct you even if you know you did Avoid leading questions - ask them to describe instead Take note of their ideas and ask “What problem does this solve?” - they’re not the experts at finding solutions, but they tried their best @Kel_Moran
  82. 82. Honor the Idea of Reciprocity Your users (research participants) are giving you something, so be sure to give something back. @Kel_Moran
  83. 83. RECOMMENDED
 READING
  84. 84. : https://www.flickr.com/photos/101187156@N03/14366224997/ Kelly Moran Lead Design Researcher Thank You @Kel_Moran
  85. 85. SUGGESTED
 RESOURCES
  86. 86. How Tos Brian A. Hoey. "A Simple Introduction to the Practice of Ethnography and Guide to Ethnographic Fieldnotes" Marshall University Digital Scholar (2014): 1-10.
 Available at: http://works.bepress.com/brian_hoey/12 Methods of Discovery a Guide to Research Writing http://methodsofdiscovery.net/?q=node/19 Ethnography https://www.academia.edu/1022047/ Picken_F._2009_What_is_Ethnography_in_M._Walter_Ed_Social_Research_Methods_Melbourne_OUP What is Ethnography http://www.cusag.umd.edu/documents/workingpapers/epiontattrib.pdf Articles Horace Miner’s (1956) article “Body Ritual among the Nacirema” Available at: https://www.msu.edu/~jdowell/miner.html or at: http://www.sfu.ca/~palys/Miner-1956- BodyRitualAmongTheNacirema.pdf

×