1.
How to do UX in
government without
losing your freakin
mind
UXPA-DC 2014
Jonathan Rubin
U.S. General Services
Administration
@jonathan_rubin #userfocus2014

2.
Agency A
Agency B

3.
“The UX State of
our Union is
getting stronger.”
- Not Obama, but still true!

4.
● US Digital Service
Gov.uk Design Rules
CFPB Design Rules
18F @ GSA
Digital Analytics Pro.
7+ Fed Usability Labs
Healthcare.gov unsuck!
(maybe)

5.
Has # of UX people changed since last year?

6.
How widespread is UX at your agency? - 2013

7.
How widespread is UX at your agency? - 2014

8.
How widespread is UX at your agency? - 2014
Awwww yeah :)

9.
DigitalGov User Experience Program
● Find us on Digitalgov.gov
● Help agencies succeed in UX via support + education
● Remove barriers to UX use
● Manage Federal UX community (525+ members)
● Also:
o Usability Starter Kit
o 25+ Usability Case Studies
o API Usability Evaluations
o Federal Usability Volunteer List

10.
Policy Challenges
o Paperwork Reduction Act (PRA)
o Terms of Service
o IT Procurement (quick review!)
o Section 508 / Accessibility
Secrets of Good Gov UX
o Sell it (well)
o Shut up and do it
o Viva UX Team
o Get crafty (resources!)
1
2

11.
Paperwork
Reduction
Act (PRA)

12.
1. Paperwork Reduction Act
● Clearance needed to collect info from public

13.
1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended

14.
1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms

15.
1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item

16.
1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
● Does NOT apply to Feds, most Social Media

17.
1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
● Does NOT apply to Feds, most Social Media
● Regulates incentives

18.
You create: Supporting Statements
● Justify what you’re doing and why
● Target audience
● Cost (if publishing in Fed Register)
● Screenshots (if applicable)
● List of questions (if applicable)
● # of estimated hours + respondents (Burden Hours)

19.
Customer Service Generic Clearance
● AKA Umbrella Clearance
● You should already have this! If not: Get it!
o Search your agency on Reginfo.gov + collection
o Talk to PRA Officer
● Worth it if you are doing lots of collections
● Can’t publish results + no statistical info
● With it you can Fast Track

20.
NOTE: All workflows that follow
are drafts and are not official
(yet)

21.
Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
Includes:
Questions / screenshot
Supporting Statements
Law / Authority

22.
Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments

23.
Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
Time needed: 120+ days

24.
Survey
Focus
Group
Usability
Test
Forms
Regular
PRA
Process
OK. You’re
good to go

25.
Usability
Test
Usability
Test
Usability
Test
Usability
Test
Regular
PRA
Process
for initial
package.
Allows
grouping
like items
together, if
it makes
sense.
Generic
“Umbrella”
Clearance
UT 1
UT 2
UT 3
UT 4

26.
Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
Time needed: 120+ days

27.
Option 2. Use existing PRA Generic Clearance,
then get Fast Track
Agency
develops PRA
plan
OMB Reviews
(5 days)
Time needed: 1-3 weeks
You’re good!
Get # to put
on your
survey

28.
120+
days
or….
1-3 weeks!

29.
Agency Burden hours
(8,000)
Requested 20 hours
(20 people x 1 hour each)
Requested 500 hours
(5000 people x 6
mins each)
March -
Usability
Test 1 (10
people )
April -
Usability
Test 2 (10
people)
Survey
(5000
people)
Request #1:
Usability Testing
Request #2:
Survey

30.
Agency Burden hours
(8,000)
Research Prototype Build Test Release Maintain

31.
Agency Burden hours
(8,000)
Fastrack #1:
User interviews
(20 hours)
Fastrack #2:
Paper Prototype
Test (20 hours, 2
tests)
Fastrack #3:
Usability Test
(20 hours, 2
tests)
Research Prototype Build Test Release Maintain

32.
More PRA Tips
● Be nice to your PRA officer

33.
More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire

34.
More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices

35.
More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
● Digitalgov.gov > “Usability Starter Kit”

36.
More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
● Digitalgov.gov > “Usability Starter Kit”
● Find examples from Reginfo.gov

37.
Incentives and the PRA
● Their view: Trying to save the gov money
● OMB ceiling is
o $40 an hour general public
o $50-$75 for focus groups
o More $ needs justification in support statements
o Work with PRA officer early + often to increase $
● Contractors can get around this
● Besides money: Goodwill, swag, gift cards

38.
Terms of Service

39.
One way to get amended TOS signed
Find a tool you
like
Have your
lawyers talk to
their lawyers
Come up with
Amended
Terms of
Service
Use tool /
Dominate
Universe
Review the
terms

40.
TOS on DigitalGov.gov
● GSA can help
negotiate if there is
community need
● What is TOS?
● Template for
amending TOS
● Examples of 70+
signed TOS
● Flowchart!

41.
Procurement (quick!)

42.
Problems with UX in IT contracts
lots of people much smarter than me are working on IT procurement reform - just looking at UX in IT contracting.
some issues:
1. No UX law (unlike 508)
2. One-size fits all contracts don’t apply well to IT
3. COs + PMs not familiar/don’t care about UX
4. CO and PM may not work together from starT
5. Waterfall :(
6. Biz requirements… what about user req?
7. No UX in SOW > Proposals don’t include UX
8. Unclear about contractor UX qualifications
9. Lessons learned not shared

43.
Things that might help
things that might help
1. Don’t bury lede in RFP! Clearly state problem, what u want
2. Include UX definitions and glossary
3. Have a UX person look at contract first
4. Connet wireframes to usability testing
5. UX = deliverable / labor category, UAT 508… - right near 508
6. But be broad! Don’t constrain too much
7. Testing as milestone (UAT?), block out test time
8. Do your own user research BEFORE contract... (or get help)
9. Look at SOW Templates in Digitalgov.gov and usability.gov
10. Past performance - screenshots, not text

44.
Evaluating UX proposals
● Do RFI first?

45.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX

46.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio

47.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist

48.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?

49.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?
● Screenshots vs. text

50.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?
● Screenshots vs. text
● Oral presos may be faster - real products!

51.
Evaluating UX proposals
● Do RFI first?
● Past Performance: Step-by-step of good UX
● UXperience: Certification / Degree / Portfolio
● Key Personnel provision: UX specialist
● Required: Gov exp.? Catch-22?
● Screenshots vs. text
● Oral presos may be faster - real products!
● White Paper: Choosing UX vendor

52.
Contractor Sub-
Contractor
New UX
Contractor
You
Who should do UX?

53.
Measuring UX performance
● Attend ALL UX activities

54.
Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports

55.
Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor

56.
Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase

57.
Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase
● Create KPIs and measure against them

58.
Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase
● Create KPIs and measure against them
● Refer to usability best practices

59.
Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook

60.
Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
● Promotes iterative development (Agile)

61.
Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
● Promotes iterative development (Agile)
o Improved manageability + budgetary feasibility
o Reduction of risk
o Frequent delivery of usable capabilities
o Increased flexibility
o Greater visibility into contractor performance

62.
Do Agile
and UX,
suckas.

63.
More Procurement resources
● Possible Contract Vehicles:
o 541 - Advertising and Marketing
 541-3 - Web Design
 541-4A - Market Research and Analysis
o GSA Schedule 70 - IT Services
● UX education at DigitalGov.gov + calendar
● Check out UX SOWs on DG.gov + Usability.gov
● FBO.gov

64.
Section 508

65.
What accessible coding looks like
● Alternative text for all graphics and all non-text media

66.
What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies

67.
What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content

68.
What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading

69.
What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
● Written in Plain Language

70.
What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
● Written in Plain Language
● Pop-ups are keyboard accessible
● Etc.

71.
Hate getting sued? Me too!

72.
When to think about accessibility?
Research Prototype Build Test Release Maintain

73.
When to think about accessibility?
Yikes!
Research Prototype Build Test Release Maintain

74.
When to think about accessibility?
Yikes!
Better than
nothing
(barely)
Research Prototype Build Test Release Maintain

75.
When to think about accessibility?
Yikes!
Better than
nothing
(barely)
Now we’re
talking :)
Research Prototype Build Test Release Maintain

76.
More 508 resources
● Add into contract, start early!

77.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.

78.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start

79.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones

80.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal

81.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag

82.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
● Role-Based Accessibility in Government (PPT)

83.
More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
● Role-Based Accessibility in Government (PPT)
● Find accessibility testers + 508 template

84.
Policy Challenges
o PRA
o Terms of Service
o Procurement (quick!)
o Section 508
Secrets of Gov UX
o Sell it (well)
o Shut up and do do it
o Viva UX Team!
o Get crafty (resources!)
1
2

85.
Sell it
sell it - right
1. Always resistance to new ideas, persistence! be a pain in the ass!
2. use real quotes
3. Explain UX but don’t be a jerk (+ use lingo)
4. Share + Create Usability Case Studies
5. #uxgov hashtag
6. Keep asking “What do the Users think?”
7. Share user failures (esp. big ones)
8. Build on a big mistake
9. Prioritize: What’s the most important thing?
10. Precedent: Digital Gov Strategy

86.
Get out of the building!
shut up and get out there!
solo or 1-2 devs or others
1. the strategy is delivery
2. Deliver something and point to it!
3. Get real feedback (no hearsay!)
4. Get early / bad things in front of users ASAP
5. get it into strategic plan or development plan
6. Do easy techniques
o User Interviews
o Prototype tests
o Hallway tests
o User Free Exploration

87.
Be crafty - get resources
get crafty - find resources
1. Enlist passionate people on operations side
2. Start with fractions
3. Hallway Testing (also sells!)
4. UX Community Listserve
5. Free usability tools + check with CIO / Comms / procurement to see what tools people are using
6. Create your own agency UX community
7. But best of all….

88.
Open Opportunities
● GSA program for inter-agency volunteering
● Digitalgov.gov, search “Open Opportunities”
● Create finite tasks or ongoing 20%er
● Me: 100% success in 24 hours or less
● Ideas
o Facilitator
o Script Writing
o Educator
o Speakers, Etc.

89.
Thank you!
Jonathan.rubin@gsa.gov
@jonathan_rubin

90.
Thank you!
Jonathan.rubin@gsa.gov
@jonathan_rubin

91.
Thank you!
Jonathan.rubin@gsa.gov
@jonathan_rubin
And join our UX listserve!