This document summarizes a presentation on doing user experience (UX) work in the government without losing your mind. It discusses some of the challenges of UX in government such as the Paperwork Reduction Act, terms of service agreements, and IT procurement processes. It provides tips for navigating these challenges like using existing clearances, fast tracking requests, and evaluating UX proposals. The growth of UX in government is also noted, with the establishment of groups like the US Digital Service and more federal usability labs.

1. How to do UX in
government without
losing your freakin
mind
UXPA-DC 2014
Jonathan Rubin
U.S. General Services
Administration
@jonathan_rubin #userfocus2014

2. Agency A
Agency B

4. “The UX State of
our Union is
getting stronger.”
- Not Obama, but still true!

6. ● US Digital Service
Gov.uk Design Rules
CFPB Design Rules
18F @ GSA
Digital Analytics Pro.
7+ Fed Usability Labs
Healthcare.gov unsuck!
(maybe)

7. Has # of UX people changed since last year?

8. How widespread is UX at your agency? - 2013

9. How widespread is UX at your agency? - 2014

10. How widespread is UX at your agency? - 2014
Awwww yeah :)

11. DigitalGov User Experience Program
● Find us on Digitalgov.gov
● Help agencies succeed in UX via support + education
● Remove barriers to UX use
● Manage Federal UX community (525+ members)
● Also:
o Usability Starter Kit
o 25+ Usability Case Studies
o API Usability Evaluations
o Federal Usability Volunteer List

13. Policy Challenges
o Paperwork Reduction Act (PRA)
o Terms of Service
o IT Procurement (quick review!)
o Section 508 / Accessibility
Secrets of Good Gov UX
o Sell it (well)
o Shut up and do it
o Viva UX Team
o Get crafty (resources!)
1
2

14. Paperwork
Reduction
Act (PRA)

15. 1. Paperwork Reduction Act
● Clearance needed to collect info from public

16. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended

17. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms

18. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item

19. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
● Does NOT apply to Feds, most Social Media

20. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
● Does NOT apply to Feds, most Social Media
● Regulates incentives

22. You create: Supporting Statements
● Justify what you’re doing and why
● Target audience
● Cost (if publishing in Fed Register)
● Screenshots (if applicable)
● List of questions (if applicable)
● # of estimated hours + respondents (Burden Hours)

23. Customer Service Generic Clearance
● AKA Umbrella Clearance
● You should already have this! If not: Get it!
o Search your agency on Reginfo.gov + collection
o Talk to PRA Officer
● Worth it if you are doing lots of collections
● Can’t publish results + no statistical info
● With it you can Fast Track

24. NOTE: All workflows that follow
are drafts and are not official
(yet)

25. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
Includes:
Questions / screenshot
Supporting Statements
Law / Authority

26. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments

27. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
Time needed: 120+ days

28. Survey
Focus
Group
Usability
Test
Forms
Regular
PRA
Process
OK. You’re
good to go

29. Usability
Test
Usability
Test
Usability
Test
Usability
Test
Regular
PRA
Process
for initial
package.
Allows
grouping
like items
together, if
it makes
sense.
Generic
“Umbrella”
Clearance
UT 1
UT 2
UT 3
UT 4

31. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
Time needed: 120+ days

32. Option 2. Use existing PRA Generic Clearance,
then get Fast Track
Agency
develops PRA
plan
OMB Reviews
(5 days)
Time needed: 1-3 weeks
You’re good!
Get # to put
on your
survey

33. 120+
days
or….
1-3 weeks!

34. Agency Burden hours
(8,000)
Requested 20 hours
(20 people x 1 hour each)
Requested 500 hours
(5000 people x 6
mins each)
March -
Usability
Test 1 (10
people )
April -
Usability
Test 2 (10
people)
Survey
(5000
people)
Request #1:
Usability Testing
Request #2:
Survey

35. Agency Burden hours
(8,000)
Research Prototype Build Test Release Maintain

36. Agency Burden hours
(8,000)
Fastrack #1:
User interviews
(20 hours)
Fastrack #2:
Paper Prototype
Test (20 hours, 2
tests)
Fastrack #3:
Usability Test
(20 hours, 2
tests)
Research Prototype Build Test Release Maintain

37. More PRA Tips
● Be nice to your PRA officer

38. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire

39. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices

40. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
● Digitalgov.gov > “Usability Starter Kit”

41. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
● Digitalgov.gov > “Usability Starter Kit”
● Find examples from Reginfo.gov

44. Incentives and the PRA
● Their view: Trying to save the gov money
● OMB ceiling is
o $40 an hour general public
o $50-$75 for focus groups
o More $ needs justification in support statements
o Work with PRA officer early + often to increase $
● Contractors can get around this
● Besides money: Goodwill, swag, gift cards

45. Terms of Service

47. One way to get amended TOS signed
Find a tool you
like
Have your
lawyers talk to
their lawyers
Come up with
Amended
Terms of
Service
Use tool /
Dominate
Universe
Review the
terms

48. TOS on DigitalGov.gov
● GSA can help
negotiate if there is
community need
● What is TOS?
● Template for
amending TOS
● Examples of 70+
signed TOS
● Flowchart!

49. Procurement (quick!)

50. Problems with UX in IT contracts
lots of people much smarter than me are working on IT procurement reform - just looking at UX in IT contracting.
some issues:
1. No UX law (unlike 508)
2. One-size fits all contracts don’t apply well to IT
3. COs + PMs not familiar/don’t care about UX
4. CO and PM may not work together from starT
5. Waterfall :(
6. Biz requirements… what about user req?
7. No UX in SOW > Proposals don’t include UX
8. Unclear about contractor UX qualifications
9. Lessons learned not shared

52. Things that might help
things that might help
1. Don’t bury lede in RFP! Clearly state problem, what u want
2. Include UX definitions and glossary
3. Have a UX person look at contract first
4. Connet wireframes to usability testing
5. UX = deliverable / labor category, UAT 508… - right near 508
6. But be broad! Don’t constrain too much
7. Testing as milestone (UAT?), block out test time
8. Do your own user research BEFORE contract... (or get help)
9. Look at SOW Templates in Digitalgov.gov and usability.gov
10. Past performance - screenshots, not text

53. Evaluating UX proposals
● Do RFI first?

54. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX

55. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio

56. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist

57. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?

58. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?
● Screenshots vs. text

59. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?
● Screenshots vs. text
● Oral presos may be faster - real products!

60. Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?
● Screenshots vs. text
● Oral presos may be faster - real products!
● White Paper: Choosing UX vendor

61. Contractor Sub-
Contractor
New UX
Contractor
You
Who should do UX?

62. Measuring UX performance
● Attend ALL UX activities

63. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports

64. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor

65. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase

66. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase
● Create KPIs and measure against them

67. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase
● Create KPIs and measure against them
● Refer to usability best practices

69. Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook

70. Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
● Promotes iterative development (Agile)

71. Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
● Promotes iterative development (Agile)
o Improved manageability + budgetary feasibility
o Reduction of risk
o Frequent delivery of usable capabilities
o Increased flexibility
o Greater visibility into contractor performance

72. Do Agile
and UX,
suckas.

73. More Procurement resources
● Possible Contract Vehicles:
o 541 - Advertising and Marketing
 541-3 - Web Design
 541-4A - Market Research and Analysis
o GSA Schedule 70 - IT Services
● UX education at DigitalGov.gov + calendar
● Check out UX SOWs on DG.gov + Usability.gov
● FBO.gov

74. Section 508

76. What accessible coding looks like
● Alternative text for all graphics and all non-text media

77. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies

78. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content

79. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading

80. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
● Written in Plain Language

81. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
● Written in Plain Language
● Pop-ups are keyboard accessible
● Etc.

82. Hate getting sued? Me too!

83. When to think about accessibility?
Research Prototype Build Test Release Maintain

84. When to think about accessibility?
Yikes!
Research Prototype Build Test Release Maintain

85. When to think about accessibility?
Yikes!
Better than
nothing
(barely)
Research Prototype Build Test Release Maintain

86. When to think about accessibility?
Yikes!
Better than
nothing
(barely)
Now we’re
talking :)
Research Prototype Build Test Release Maintain

87. More 508 resources
● Add into contract, start early!

88. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.

89. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start

90. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones

91. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal

92. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag

93. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
● Role-Based Accessibility in Government (PPT)

94. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
● Role-Based Accessibility in Government (PPT)
● Find accessibility testers + 508 template

95. Policy Challenges
o PRA
o Terms of Service
o Procurement (quick!)
o Section 508
Secrets of Gov UX
o Sell it (well)
o Shut up and do do it
o Viva UX Team!
o Get crafty (resources!)
1
2

96. Sell it
sell it - right
1. Always resistance to new ideas, persistence! be a pain in the ass!
2. use real quotes
3. Explain UX but don’t be a jerk (+ use lingo)
4. Share + Create Usability Case Studies
5. #uxgov hashtag
6. Keep asking “What do the Users think?”
7. Share user failures (esp. big ones)
8. Build on a big mistake
9. Prioritize: What’s the most important thing?
10. Precedent: Digital Gov Strategy

97. Get out of the building!
shut up and get out there!
solo or 1-2 devs or others
1. the strategy is delivery
2. Deliver something and point to it!
3. Get real feedback (no hearsay!)
4. Get early / bad things in front of users ASAP
5. get it into strategic plan or development plan
6. Do easy techniques
o User Interviews
o Prototype tests
o Hallway tests
o User Free Exploration

99. Be crafty - get resources
get crafty - find resources
1. Enlist passionate people on operations side
2. Start with fractions
3. Hallway Testing (also sells!)
4. UX Community Listserve
5. Free usability tools + check with CIO / Comms / procurement to see what tools people are using
6. Create your own agency UX community
7. But best of all….

100. Open Opportunities
● GSA program for inter-agency volunteering
● Digitalgov.gov, search “Open Opportunities”
● Create finite tasks or ongoing 20%er
● Me: 100% success in 24 hours or less
● Ideas
o Facilitator
o Script Writing
o Educator
o Speakers, Etc.

101. Thank you!
Jonathan.rubin@gsa.gov
@jonathan_rubin

102. Thank you!
Jonathan.rubin@gsa.gov
@jonathan_rubin

103. Thank you!
Jonathan.rubin@gsa.gov
@jonathan_rubin
And join our UX listserve!