Successfully reported this slideshow.
Your SlideShare is downloading. ×

Too minimal - role of UX research in government MVP

Too minimal - role of UX research in government MVP

This interactive talk describes a government MVP project, with a Goldilocks approach to discussing Minimum and Viable - Not enough (too minimal), just right, and too much (not minimal). MVP is new to most government teams - that plays a role.

This interactive talk describes a government MVP project, with a Goldilocks approach to discussing Minimum and Viable - Not enough (too minimal), just right, and too much (not minimal). MVP is new to most government teams - that plays a role.

More Related Content

Too minimal - role of UX research in government MVP

  1. 1. Lisa Fast @lisavation #GOAT2017 Not Too Minimal: UX Research in a Government Minimum Viable Product
  2. 2. 2 Original Minimum Viable Product View – focus on user need http://blog.crisp.se/2016/01/25/henrikkniberg/making-sense-of-mvp
  3. 3. 3 “The MVP Build-Measure-Learn feedback loop is at the core of the Lean Startup model. An experiment is … also a first product.” - Eric Ries
  4. 4. 4 Minimum Valuable Product https://www.quora.com/What-is-the-difference-between-a- prototype-and-a-minimum-viable-product-MVP
  5. 5. 5 “A Minimum Viable Product is the smallest thing you can build that delivers customer value (and as a bonus captures some of that value back).” – Ash Maurya https://blog.leanstack.com/minimum-viable-product-mvp-7e280b0b9418
  6. 6. 6 https://hackernoon.com/the-mvp-is-dead-long-live-the-rat-233d5d16ab02
  7. 7. 7 Risky assumption testing requires a level of Agile maturity not present in most of the Government of Canada. In this project, the idea of starting small was a struggle, most expected to get the complete product at launch – despite ongoing mega-project troubles…
  8. 8. 8
  9. 9. 9 Receptive moment in government
  10. 10. 10 UX Research is all about Value & Viability Understanding needs for value Ensuring they can derive that value
  11. 11. 11 The challenge: Email-only Consultations for Gazette Part I Part I: “a final opportunity to review and comment on a proposed regulation at the last stages of the regulation-making process, before it is enacted”
  12. 12. 12 Starting Point: US eRegulations -18F Comment Pilot Test in June 2016 https://18f.gsa.gov/2016/07/26/new-pilot-aims-to-streamline-notice-and-comment-process/ https://epa-notice.usa.gov/
  13. 13. 13 LEAN UX Pilot assumptions: We believe that online regulatory consultations will lead to effective commenting & analysis – teams just fell into emails & spreadsheets as people stopped sending letters This will improve: the quality and efficiency of commenting by Canadians and analysis by government teams We will have demonstrated this when we can measure*: More time to think about comments – less time ’handling’ them: • A decrease in frequency of copy/paste for receivers & analysts and decrease in ‘lost’ comments • A decrease in time from closing date to completed analysis/review • Improved quality and coverage of comments in review and decision process Clearer and more thorough comments from Canadians: • An increase in comments clearly associated with a specific section of the RIAS or proposed regulatory text • An increase in the number of comments overall * Measurement will be ‘loose’ for this pilot since so few submissions are expected for the selected regulation.
  14. 14. 14 Start with prototype before contract as proof of concept - assume that 18F design is good start
  15. 15. 15 UX Research sessions started on prototype, progressed to dev site as features were implemented Exploratory sessions – use and talk
  16. 16. 16 Task: Get started - Orient the commenter PROTOTYPE COMMENT MVP Name and submit was at bottom of page • First participant looked for it at top, wanted an idea of what to expect, what was possible Moved name etc. to top & added ‘Skip to first comment box’ to orient users to where they could enter comments. Add ‘View summary /Submit’ buttons to top of page
  17. 17. 17 Task: Enter comments • Research participants wanted reassurance it was saved P2: “I'm always worried about saving– that’s what drives us back to Word.” • Had been burned by other experiences • P1: “3 times I went to submit and it cleared my input - 3 times!! PROTOTYPE COMMENT MVP • Save button and saved message added • P3: “Good to see that saved message!” • P4: “This is very straightforward”
  18. 18. 18 Task: Review summary before submit PROTOTYPE Participants were clear they need a record of their submission for their members. Some post it on their website. Would prefer a table view of their comments. P1: If there was a table.” P2: “I want to see a table of ALL possible comments with blanks where I missed them” Changed to table view in MVP –for Pilot, must return to main page to edit/enter, long term should build it into here, also want to email a copy of this to themselves P4: “Display empty comments - That makes sense to see what you missed especially if it was very long. I might put ‘read it’ in there for myself” COMMENT MVP
  19. 19. 19 Aha! Associations Perform Same Share-Receive-Analyze Process Just like government, associations gather member comments, review & synthesize the comments into a submission - need support & integration, not add-on copy/paste work Participant 2: “So I could send them a link to see our actual submission- I like that idea as long as it was easy for them to do”. Participant 4: “if we could do this, would we even keep a Word copy? Because it’s tedious… this could streamline our process” Future is participative democracy: invite them in to the system, let them use it with their members & feed their submission in directly Association members Association submission Individual commenter
  20. 20. 20 On-time launch of Comment Pilot in French & English Site launched on-time on March 4th simultaneous with Gazette on-line launch of the proposed amendment
  21. 21. 21 Hold up stickies to vote on 4 UX research sessions – in 6 weeks from contract to launch Too much Too minimal Just right Minimum viable Not enough Not minimal
  22. 22. 22 External Stakeholders really got ‘minimal’ • Result: Only 1 of 5 association stakeholders submitted their comments online – the rest stuck to email and the HC team entered their emailed comments into the system themselves to learn • 18F pilot had slightly more, about half of submissions came in via the pilot, was better integrated but their participants also sought in-progress drafts and mult—session editing Participant 1: “I get that this is much better for the government, then it’s all laid out in a table for them… But it creates more work for me. It’s a second layer – I put it in a table and say hey members, send out a table to them and get them to do it. There’s no save and share option here.”
  23. 23. 23 Make notes on stickies during next bit Not enough Too minimal Just right Too much Not minimal Your noteYour noteYour note
  24. 24. 24 External Commenter Site Internal Analysis Toolkit Now to the other VALUE - the back end (18F MVP did not do this)
  25. 25. 25 UX Research: Kicked off with Discovery workshop to discover various roles, user needs & pain points in current process Prepare Gazette Receive (Reg Management) Worry: lose a comment or lose track of a comment Analyze (Programs) Worry: miss an important comment or issue Review (all and TBS)
  26. 26. 26 Discovery – your vote? Too much Too minimal Just right Minimum viable Not enough Not minimal
  27. 27. 27 Process from Discovery Maps Onto Key Tasks in Toolkit demo
  28. 28. 28 Data Model
  29. 29. 29 Consultation Toolkit MVP co-designed with Health Canada team UX research: Sketching, 4 co-design workshops & 2 usability test sessions with team members
  30. 30. 30 Toolkit: View comments by submission (commentary) or by document section
  31. 31. 31 Notes capture analysis, progress and decisions: - See public notes by other analysts - Assign note a status - Add public and private notes
  32. 32. 32 Co-design process – your vote? Too much Too minimal Just right Minimum viable Not enough Not minimal
  33. 33. 33 18F MVP did not provide analysis toolkit – it was performed by an external consultant. Made their job easier
  34. 34. 34 Analysis toolkit had more discovery time – got farther Got closer to meeting needs of receivers, analysts and reviewers – co-created site with them “I definitely think we are headed in the right direction – Very supportive of proceeding” “I’m supportive of proceeding with changes to terminology, documentation and reporting”. All consultations entered online - team performed analysis & review on the Toolkit Lots of unimplemented bits – preparation of the document for launch was done by hand
  35. 35. 35 eRegs Pilot Consultation System - Design Components Map External Commenter Site Internal Analysis Toolkit Written in Swift
  36. 36. 36 Open Source Code 3500 lines of Swift written for the MVP and 52000 lines of Swift including the open source used. 750 lines of Javascript written on WET/ JQuery base 250 lines of CSS with WET and Bootstrap reuse 2800 lines of web page template code “My experience is that Swift was a productive choice. Good quality, stability, maintainability. Swift and Vapor are in rapid evolution phases and this will require ongoing code maintenance to stay current.” – Steve Hume
  37. 37. 37 No documentation or automated test case development - it’s an MVP - ran out of time and $
  38. 38. 38 “We recognise that it would be a lot for someone to take on the code base as it stands, in such an early prototype state. The limited time to deliver an MVP, leads to the code being less robust than we’d like. However, by sharing the project progress and research we hope it will be useful to any council considering a similar technical tool for their ways into work programme.” https://blog.wearefuturegov.com/lets-open-source-by-default-fb06f509dd9e
  39. 39. 39 Open source code base – your vote? Too much Too minimal Just right Minimum viable Not enough Not minimal
  40. 40. 40 Minimum Viable Product is a team approach – learn together Learning from 1st MVP experiment dictates what to build for next experiment Each iteration adds & tests new assumptions Product Owner maintains this vision, makes final call on each experiment step, is present throughout the process * We acted as product owner – Government needs to be product owner!
  41. 41. Questions? MVP regulation consultation for commenters – demo link: https://dev.regconsultation.ca:8080/ • Article on LinkedIn: https://www.linkedin.com/pulse/government-minimum-viable-product-learning-from-small-lisa-fast • US 18F pilot results: https://github.com/18F/epa-notice/files/548513/FinalDemo-Phase3UserResearchFindings.pdf • Repository https://github.com/canada-ca/regs-consult-wet @lisavation lisa@vation.ca

Editor's Notes

  • Kniberg calls it earliest testable product

    “The Minimum Viable Product Build-Measure-Learn feedback loop is at the core of the Lean Startup model. An experiment is more than just a theoretical inquiry; it is also a first product.” 
    ― Eric Ries, The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses

    You don’t start building the 2nd MVP until the learning from the first is done. That’s what drives the knowledge of what to add to the next iteration.
  • Really relevant to government because the learning is at all levels
    he skateboard is actually a usable product that helps the customer get from A to B. Not great, but a tiny bit better than nothing. So we tell the customer “don’t worry, the project is not finished, this was just the first of many iterations. We’re still aiming to build a car, but in the meantime please try this and give us feedback“. Think big, but deliver in small functionally viable increments.
    We might learn some really surprising things. Suppose the customer says he hates the skateboard, we ask why, and he says “I hate the color”. We’re like “uh…. the color? That’s all?”. And the customer says “Yeah, make it blue! Other than that, it’s fine!”. You just saved *alot* of money not building the car! Not likely, but who knows?
  • https://www.quora.com/What-is-the-difference-between-a-prototype-and-a-minimum-viable-product-MVP
    Bit of a backlash - focus on value – not a prototype – table with differences between a prototype and product
  • https://hackernoon.com/the-mvp-is-dead-long-live-the-rat-233d5d16ab02

  • https://blog.leanstack.com/minimum-viable-product-mvp-7e280b0b9418
  • Kniberg calls it earliest testable product

    “The Minimum Viable Product Build-Measure-Learn feedback loop is at the core of the Lean Startup model. An experiment is more than just a theoretical inquiry; it is also a first product.” 
    ― Eric Ries, The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses

    You don’t start building the 2nd MVP until the learning from the first is done. That’s what drives the knowledge of what to add to the next iteration.
  • Lots of initial work and pre-publication consultations precede publication in Part I of the Gazette. eRegs can address that as well as it grows.
    This would build up a set of stakeholders for a particular topic who can follow along, contribute online. This is done informally and sometimes formally now, but there is no continuity between the steps in the process, for the GC team or the stakeholders.
    P2 Aviation Association: “By the time it gets to the Gazette, it's almost all over for us - we've given a lot of input”.
  • Relied on XML, unilingual. Evaluation of 18F open source was completed in November – large source code-base with many team members, most of the eRegs code is to re-display various sets of existing regulations in a consistent well-formatted way, small pilot of online commenting with no results reported, commenting pilot relies on the structure of the e-regs ouput (in Canada, the Gazette output for CG1 is not at all consistent or highly structured), no support for multiple languages in the code base - notice focus on individual comments by section, each comment opens on new page (tedious, lose context)
    First step of eRegs is to reformat the regulations into new format – then they layer on the commenting
    https://18f.gsa.gov/2016/07/26/new-pilot-aims-to-streamline-notice-and-comment-process/


  • Architecture of the system will be designed in microservices and APIs, so that different teams/provinces/associations can use some or all of the system

  • Purchased domain name previous week, gave up on Cloud
    Received marked-up Word doc with comment tag markers – in English – then French
    Gazette can’t proceed until it’s released to them – they do the web publishing into HTML & PDF
    Received Gazette-formatted HTML EN & FR on March 3rd
    Reviewed with team for Go-No Go on morning of 4th
    Decision was Go

    Design: Oriented to 18F-style for individual commenters
    Backlog: rethink design in light of stakeholder needs – possibly provide them access to full system for more participatory role. For next MVP, at minimum provide editable Summary View
  • 18 F results had same learning – about half of their submissions came in via the new system, and it was built-in to the process in a way that ours was not
    - Their participants also sought in-progress drafts and multi-session editing
    Gazette page displayed a link with the email address – very hard to notice and required taking that extra step of clicking it to see the pilot page

  • After launch, work began in earnest on the internal analysis toolkit
  • Process from discovery was mapped into the new design
    A commentary is the set of comments submitted by one person
    System was built using Canada’s Web Experience Toolkit (used for Canada.ca) – accessible
    Accounts were created for the HC regulatory team
  • Meetings with team and some research sessions to understand their needs and meet them through the design.
  • Did not build a back end – hired a consultant to build a spreadsheet for them
  • Lots of things in the backlog from discussions with team

×