Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Research Beyond

Originally delivered at BayCHI in April 2019, a talk on research projects that go beyond user research to answer large, broad questions

  • Be the first to comment

  • Be the first to like this

Research Beyond

  1. 1. Researching where no one has gone before Cyd Harrell BayCHI April 9, 2019
  2. 2. 2008: How do people really use phones in their cars?
  3. 3. Research methods & skills: - Ride-along interviews - Broadcasting - Observer management - Photography - Video editing
  4. 4. We disappeared
  5. 5. Center for Civic Design
  6. 6. 2012: Field Guides to Ensuring Voter Intent
  7. 7. Research question: What do voters want to know from (local govt) election websites?
  8. 8. Method hypothesis: Remote moderated interviews
  9. 9. Remote moderated interviews: - Recruiting plan - Discussion guide - Recording protocol - Synthesis plan
  10. 10. So. Many. Ways.
  11. 11. Research question: What do voters want to know from election websites? Not-so-side quest: What’s up with election websites anyway?
  12. 12. Local election officials
  13. 13. 50 states 3,144 counties 19,354 cities & towns
  14. 14. Constructing a sample
  15. 15. A structured way to assess homepages
  16. 16. Election site presence check
  17. 17. What voters need to know
  18. 18. Electiontools.org
  19. 19. Research methods & skills: - Remote moderated interviews - Volunteer cohort management - Structured site analysis - Busywork (presence checks) - Sample construction - Conversational synthesis
  20. 20. Synthesis is a conversational practice
  21. 21. Epic Journey of American Voters (privileged version) Dana Chisnell, 2017 https://medium.com/civic-designing/the-epic-journey-of-american-voters-ed07bd0e6c57
  22. 22. Epic Journey of American Voters (burdened version) Dana Chisnell, 2017 https://medium.com/civic-designing/the-epic-journey-of-american-voters-ed07bd0e6c57
  23. 23. 2014: Digital Front Door
  24. 24. Research question: How can we help cities run useful websites?
  25. 25. Method hypothesis: - Citizen survey - City staff interviews - Some kind of analytics
  26. 26. Municipal service use cases: pay a parking ticket, attend a council meeting register for summer camp, find work with the city get a building permit, call an ambulance, street planning renew a business permit, find swimming pools, get a library card report a pothole, find out about parks, report trash on a sidewalk get a parking permit, leave public comment pay your property taxes, apply for a special event permit request a streetlight, find the police non-emergency number take a CPR training from the fire department find your recycling day, call your council member
  27. 27. Municipal sites serve 200-300 infrequent transactions.
  28. 28. 50 states 3,144 counties 19,354 cities & towns
  29. 29. Research questions: - What are city website teams prioritizing & why? - What does the public want? - What does the public expect? - How can we define a purpose for a really general website?
  30. 30. Live dashboard showing page titles
  31. 31. A starting place
  32. 32. A starting place
  33. 33. 1,000 resident survey
  34. 34. 1,000 resident survey
  35. 35. Research questions: - What are city website teams prioritizing & why? - How are city staff running websites & what barriers do they encounter? - What does the public want? - What does the public expect? - How can we define a purpose for a really general website?
  36. 36. Staff web editor interviews
  37. 37. Synthesis with policy & IT folks
  38. 38. Research methods & skills: - Moderated interviews - Survey design - Map & data science - Facilitated synthesis - Web analytics
  39. 39. Municipal service use cases: - Parking - pay a ticket, get a permit - Jobs - find work with the city - Permitting - building, planning, business, event - Recreation - summer camp, swimming pools, libraries - Maintenance - potholes, streetlights, parks, recycling - Democracy - comment, meetings, property taxes - Public safety - police, fire, ambulance
  40. 40. 2016: Best Practices in Digital Transformation
  41. 41. Research question: What are the best practices for government digital transformation?
  42. 42. Method hypothesis: - Interviews?? - ¯_(ツ)_/¯
  43. 43. FASPBS TTS ITC
  44. 44. Research question: What makes modern digital practices stick within a government entity, beyond a single innovation project?
  45. 45. Assumptions: - Transformation is not as risky as government agencies may perceive it to be. - Transformation doesn’t have an endpoint; you work on it continually. - Changing culture to be more open and agile makes government services more user-centered and builds trust. - There’s not one right way to do this work.
  46. 46. Interview questions: - What does digital transformation mean in government? - What is the goal or end-state? How do you know when you’re done? - What are the biggest obstacles to this work? How can we help people get past those obstacles? - Is there one digital transformation process or many? - How do you make the changes last? - What resources would be valuable for other agencies doing this work?
  47. 47. Research methods & skills: - Moderated interviews - Literature review - Policy & regulation review - Cluster recruiting - Tight consent & confidentiality
  48. 48. When companies were going digital New York Times, 1999 https://www.nytimes.com/1999/12/23/business/toys-r-us-falls-behind-on-shipping.html
  49. 49. Cluster recruiting: Who else in the organization had an important role in this moving forward or not?
  50. 50. bit.ly/CydBPDT
  51. 51. 2018: Self-Help in the CA Court System
  52. 52. Research question: What’s the actual experience of people representing themselves in the civil court system?
  53. 53. Non-criminal case types: - Family law - divorce, parentage, custody - Small claims - lawsuits under $10,000 - Many restraining orders - Probate law - estates, guardianship, conservatorship - Civil law - lawsuits, housing, name/gender changes, more
  54. 54. 1 supreme court 6 appellate courts 58 superior courts 254 courthouses
  55. 55. Method hypothesis: Intercept recruiting & interviews at courts
  56. 56. Research questions: - How do judges work with people representing themselves? - Which process steps trip up self-represented litigants? - What language do assisters use to explain legal matters? - What is people’s mental state at court?
  57. 57. Field days at court
  58. 58. Going to court without a lawyer
  59. 59. “Backstage” at court Annotated form packets Tools for court clerks
  60. 60. How to talk to staff
  61. 61. Research methods & skills: - Website “self-help” inventory - Courtroom observation - Shadowing - Intercept recruiting - Informal, unscripted interviews - Taco research - Synthesis dinners - Court day summaries
  62. 62. Work out what to ask the court Tell the court (in writing) Share with others involved Write up get signed & act on This isn’t as easy as it sounds, & help centers (& attorneys) spend quite a bit of time on it. The main step that forms, “self help” assistance, online doc- assembly interviews, etc, are intended to help with. Service: unfamiliar concept for laypeople. Basically the court can’t act until it knows that everyone has the same information. There may be action by the user in this step (appearance, evidence, etc) but the point is the court does something. You, the user, have to write up your own orders - OAH, judgment, etc - you’re NOT DONE when the judge says something. Multiple rounds common here Court decision Agree with or w/o help
  63. 63. Simplified instructions for serving papers
  64. 64. “We’re not just asking people to be their own lawyer. We’re asking them to be a paralegal, legal admin, process server, and scheduler.” - Kyanna Williams
  65. 65. Institutions act both at scale and over time.
  66. 66. Libraries, schools, hospitals
  67. 67. Things are strange in 2019.
  68. 68. In the 21st century, we interact with long-timescale institutions via short-timescale technologies.
  69. 69. Public servants make more design decisions than the entire design industry.
  70. 70. The best thing we can do is open our practice to support everyone who makes design decisions.
  71. 71. Questions?
  72. 72. Thank you so much for coming! @cydharrell any time you want to talk about this

×