Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Too minimal - role of UX research in government MVP


Published on

This interactive talk describes a government MVP project, with a Goldilocks approach to discussing Minimum and Viable - Not enough (too minimal), just right, and too much (not minimal). MVP is new to most government teams - that plays a role.

Published in: Government & Nonprofit
  • Be the first to comment

Too minimal - role of UX research in government MVP

  1. 1. Lisa Fast @lisavation #GOAT2017 Not Too Minimal: UX Research in a Government Minimum Viable Product
  2. 2. 2 Original Minimum Viable Product View – focus on user need
  3. 3. 3 “The MVP Build-Measure-Learn feedback loop is at the core of the Lean Startup model. An experiment is … also a first product.” - Eric Ries
  4. 4. 4 Minimum Valuable Product prototype-and-a-minimum-viable-product-MVP
  5. 5. 5 “A Minimum Viable Product is the smallest thing you can build that delivers customer value (and as a bonus captures some of that value back).” – Ash Maurya
  6. 6. 6
  7. 7. 7 Risky assumption testing requires a level of Agile maturity not present in most of the Government of Canada. In this project, the idea of starting small was a struggle, most expected to get the complete product at launch – despite ongoing mega-project troubles…
  8. 8. 8
  9. 9. 9 Receptive moment in government
  10. 10. 10 UX Research is all about Value & Viability Understanding needs for value Ensuring they can derive that value
  11. 11. 11 The challenge: Email-only Consultations for Gazette Part I Part I: “a final opportunity to review and comment on a proposed regulation at the last stages of the regulation-making process, before it is enacted”
  12. 12. 12 Starting Point: US eRegulations -18F Comment Pilot Test in June 2016
  13. 13. 13 LEAN UX Pilot assumptions: We believe that online regulatory consultations will lead to effective commenting & analysis – teams just fell into emails & spreadsheets as people stopped sending letters This will improve: the quality and efficiency of commenting by Canadians and analysis by government teams We will have demonstrated this when we can measure*: More time to think about comments – less time ’handling’ them: • A decrease in frequency of copy/paste for receivers & analysts and decrease in ‘lost’ comments • A decrease in time from closing date to completed analysis/review • Improved quality and coverage of comments in review and decision process Clearer and more thorough comments from Canadians: • An increase in comments clearly associated with a specific section of the RIAS or proposed regulatory text • An increase in the number of comments overall * Measurement will be ‘loose’ for this pilot since so few submissions are expected for the selected regulation.
  14. 14. 14 Start with prototype before contract as proof of concept - assume that 18F design is good start
  15. 15. 15 UX Research sessions started on prototype, progressed to dev site as features were implemented Exploratory sessions – use and talk
  16. 16. 16 Task: Get started - Orient the commenter PROTOTYPE COMMENT MVP Name and submit was at bottom of page • First participant looked for it at top, wanted an idea of what to expect, what was possible Moved name etc. to top & added ‘Skip to first comment box’ to orient users to where they could enter comments. Add ‘View summary /Submit’ buttons to top of page
  17. 17. 17 Task: Enter comments • Research participants wanted reassurance it was saved P2: “I'm always worried about saving– that’s what drives us back to Word.” • Had been burned by other experiences • P1: “3 times I went to submit and it cleared my input - 3 times!! PROTOTYPE COMMENT MVP • Save button and saved message added • P3: “Good to see that saved message!” • P4: “This is very straightforward”
  18. 18. 18 Task: Review summary before submit PROTOTYPE Participants were clear they need a record of their submission for their members. Some post it on their website. Would prefer a table view of their comments. P1: If there was a table.” P2: “I want to see a table of ALL possible comments with blanks where I missed them” Changed to table view in MVP –for Pilot, must return to main page to edit/enter, long term should build it into here, also want to email a copy of this to themselves P4: “Display empty comments - That makes sense to see what you missed especially if it was very long. I might put ‘read it’ in there for myself” COMMENT MVP
  19. 19. 19 Aha! Associations Perform Same Share-Receive-Analyze Process Just like government, associations gather member comments, review & synthesize the comments into a submission - need support & integration, not add-on copy/paste work Participant 2: “So I could send them a link to see our actual submission- I like that idea as long as it was easy for them to do”. Participant 4: “if we could do this, would we even keep a Word copy? Because it’s tedious… this could streamline our process” Future is participative democracy: invite them in to the system, let them use it with their members & feed their submission in directly Association members Association submission Individual commenter
  20. 20. 20 On-time launch of Comment Pilot in French & English Site launched on-time on March 4th simultaneous with Gazette on-line launch of the proposed amendment
  21. 21. 21 Hold up stickies to vote on 4 UX research sessions – in 6 weeks from contract to launch Too much Too minimal Just right Minimum viable Not enough Not minimal
  22. 22. 22 External Stakeholders really got ‘minimal’ • Result: Only 1 of 5 association stakeholders submitted their comments online – the rest stuck to email and the HC team entered their emailed comments into the system themselves to learn • 18F pilot had slightly more, about half of submissions came in via the pilot, was better integrated but their participants also sought in-progress drafts and mult—session editing Participant 1: “I get that this is much better for the government, then it’s all laid out in a table for them… But it creates more work for me. It’s a second layer – I put it in a table and say hey members, send out a table to them and get them to do it. There’s no save and share option here.”
  23. 23. 23 Make notes on stickies during next bit Not enough Too minimal Just right Too much Not minimal Your noteYour noteYour note
  24. 24. 24 External Commenter Site Internal Analysis Toolkit Now to the other VALUE - the back end (18F MVP did not do this)
  25. 25. 25 UX Research: Kicked off with Discovery workshop to discover various roles, user needs & pain points in current process Prepare Gazette Receive (Reg Management) Worry: lose a comment or lose track of a comment Analyze (Programs) Worry: miss an important comment or issue Review (all and TBS)
  26. 26. 26 Discovery – your vote? Too much Too minimal Just right Minimum viable Not enough Not minimal
  27. 27. 27 Process from Discovery Maps Onto Key Tasks in Toolkit demo
  28. 28. 28 Data Model
  29. 29. 29 Consultation Toolkit MVP co-designed with Health Canada team UX research: Sketching, 4 co-design workshops & 2 usability test sessions with team members
  30. 30. 30 Toolkit: View comments by submission (commentary) or by document section
  31. 31. 31 Notes capture analysis, progress and decisions: - See public notes by other analysts - Assign note a status - Add public and private notes
  32. 32. 32 Co-design process – your vote? Too much Too minimal Just right Minimum viable Not enough Not minimal
  33. 33. 33 18F MVP did not provide analysis toolkit – it was performed by an external consultant. Made their job easier
  34. 34. 34 Analysis toolkit had more discovery time – got farther Got closer to meeting needs of receivers, analysts and reviewers – co-created site with them “I definitely think we are headed in the right direction – Very supportive of proceeding” “I’m supportive of proceeding with changes to terminology, documentation and reporting”. All consultations entered online - team performed analysis & review on the Toolkit Lots of unimplemented bits – preparation of the document for launch was done by hand
  35. 35. 35 eRegs Pilot Consultation System - Design Components Map External Commenter Site Internal Analysis Toolkit Written in Swift
  36. 36. 36 Open Source Code 3500 lines of Swift written for the MVP and 52000 lines of Swift including the open source used. 750 lines of Javascript written on WET/ JQuery base 250 lines of CSS with WET and Bootstrap reuse 2800 lines of web page template code “My experience is that Swift was a productive choice. Good quality, stability, maintainability. Swift and Vapor are in rapid evolution phases and this will require ongoing code maintenance to stay current.” – Steve Hume
  37. 37. 37 No documentation or automated test case development - it’s an MVP - ran out of time and $
  38. 38. 38 “We recognise that it would be a lot for someone to take on the code base as it stands, in such an early prototype state. The limited time to deliver an MVP, leads to the code being less robust than we’d like. However, by sharing the project progress and research we hope it will be useful to any council considering a similar technical tool for their ways into work programme.”
  39. 39. 39 Open source code base – your vote? Too much Too minimal Just right Minimum viable Not enough Not minimal
  40. 40. 40 Minimum Viable Product is a team approach – learn together Learning from 1st MVP experiment dictates what to build for next experiment Each iteration adds & tests new assumptions Product Owner maintains this vision, makes final call on each experiment step, is present throughout the process * We acted as product owner – Government needs to be product owner!
  41. 41. Questions? MVP regulation consultation for commenters – demo link: • Article on LinkedIn: • US 18F pilot results: • Repository @lisavation