Howto.gov/firstfridays
Top User Experience
Challenges in Government
Jonathan Rubin
First Fridays Usability Program
July 10, 2013
Howto.gov/firstfridays
1. Survey
Results: Gov
UX Challenges
Our Survey of UX Activities
1. Survey Monkey with 65 respondents
2. 100% Cabinet agencies
3. Co-created with Jean Fox (+ thanks Axle
Brown)
4. Not scientific or representative
5. Goal:
• Audience research
• Develop web-based and in-person training
• Share aggregate in presentations, etc.
Howto.gov/firstfridaysHowto.gov/firstfridays
What we asked
1. Agency (but anonymous)
2. About you – title, UX education, team
size
3. How widespread is UX
4. Challenges to implementing UX
5. Resources
6. Types of UCD methods
7. What would help?
Howto.gov/firstfridaysHowto.gov/firstfridays
Good news
• Average team: 2 PT and 2 FT
• Wide range of UX methods being used
• #1 surveys, #2 style guide, #3 tests
• Thirst for training, many self-taught
• 91% doing UX on websites, ⅓ mobile
• Using respected tools (20% eyetracking!)
• Future: More testing, services
• Some agencies have UX institutionalized
Howto.gov/firstfridaysHowto.gov/firstfridays
Challenges / Lack of…
• Awareness:
• “[People] refuse to believe the results of
usability tests because they are painful.”
• “We are spending hundreds of thousands of
dollars without one usability test. Web staff
understand the need, but no one is listening.”
• Facilities
• “No facilities” / “Our lab is about 15 years
young”
Howto.gov/firstfridaysHowto.gov/firstfridays
Challenges / Lack of…
• Resources
• “Very old models are used for interaction”
• “Army of one”
• Training
• “Figure it out as we go along”
• Standards / Best Practices
• “Applying standard best practices can be
challenging… many offices and components
have different processes and standards.”
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays
34%
17%15%
15%
6%
6%
5%
2%
What best describes what you do?
Website Mngr
Other
Project Mngr
UX Specialist
Web Designer
IA
Researcher
Web Metrics Mng
# UX staff
Howto.gov/firstfridaysHowto.gov/firstfridays
Part Time Full Time
Had at least 1
person
86% 59%
Median # of
people
2 2
Average # of
people
2 3
Range of
people
0-10 0-30
Howto.gov/firstfridaysHowto.gov/firstfridays
0%
10%
20%
30%
40%
50%
60%
70%
College /
Grad school
Certification Workshops,
webinars
On-the-job
mentorship
Other
Where did your staff get UX training?
Howto.gov/firstfridaysHowto.gov/firstfridays
3%
19%
46%
25%
8%
0% 10% 20% 30% 40% 50
op to Bottom UX
Used frequently
Occasional UX
Know but don't
practice
No one knows
UX
How widespread is UX at your organization?
Howto.gov/firstfridaysHowto.gov/firstfridays
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
What types of UX products do you work on?
Howto.gov/firstfridaysHowto.gov/firstfridays
% Who “Often” or “Always” use
0% 10% 20% 30% 40% 50%
Tests: Mobile
Contextual inquiries
Focus groups
Tests: Remote
Personas
Tests: High Fidelity
User Analysis
Expert Reviews
Task Analysis
Tests: In-person
Surveys
Style guides
Howto.gov/firstfridaysHowto.gov/firstfridays
% Who “Never” use these methods
0% 10% 20% 30% 40% 50% 60%
Personas
Tests: Remote
Context. Inquiries
Tests: High Fidelity
Test: Mobile
Card sorts
Tests: Low Fidelity
Affinity diagrams
Tests:: First-Click
Tests: Eyetracking
Howto.gov/firstfridaysHowto.gov/firstfridays
How often do you perform usability tests?
0
5
0
5
0
5
Never Rarely Sometimes Often Always Don't Know
In person Remote
Asked about UX methods
• Top 3: Surveys, Style guide, Tests
• Bottom 3: Affinity diagrams
Eyetracking, First Click
• 10% never a usability test – bad?
• 16% never a survey
• Task analysis - 8.5% always, 55%
sometimes
Howto.gov/firstfridaysHowto.gov/firstfridays
Training Needs
• “UX Training classes - webinars aren't
enough”
• “Software” - Morae, Surveys, CrazyEgg
• “Any and all”
• “Be more aware of findings”
• “How to get buy-in from higher ups “
• Methods: First-Click Testing, Style Guides,
Card Sorts, Surveys, Task analysis
Howto.gov/firstfridaysHowto.gov/firstfridays
Plans for the future
Howto.gov/firstfridaysHowto.gov/firstfridays
Asked
about
future
plans:
Howto.gov/firstfridaysHowto.gov/firstfridays
Biggest obstacle to using UX?
0% 10% 20% 30% 40% 50% 60% 70% 80%
Tried UX, failed
Other
No UX awareness
Don't get UX ROI
UX not built in
Lack of time
No resources
Howto.gov/firstfridaysHowto.gov/firstfridays0% 10% 20% 30% 40% 50% 60%
One-way mirror
Observation room
Testing room
Other
Multi-purpose rooms
No space for testing
Which do you have? (check all)
Challenges / Lack of…
• Resources - staff and metrics software
• Awareness – Managers not sold on UX
• Training - Staff and Manager
• Standards / Best Practices
• Testing facilities
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/firstfridays
2. Top Gov UX
Problems on
Sites
Two types of gov challenges:
1. Gov UX Strategic and Resource
challenges(Survey)
2. Persistency UX problems on fed
websites (FF test data)
Howto.gov/firstfridaysHowto.gov/firstfridays
FF Test Data
1. 29 products reviewed in 2012
2. 130+ significant problems
3. 32 different usability categories -
problems big and small
Thanks to Cynthia, Molly, Josh, and Ruth!
Howto.gov/firstfridaysHowto.gov/firstfridays
Top 6 Usability Challenges
1. Navigation difficult, confusing or hidden
2. Search feature hidden or doesn’t work
3. Forms difficult to use
4. Purpose of site unclear
5. Difficulty with Top Tasks
6. Lack of Plain Language
Howto.gov/firstfridaysHowto.gov/firstfridays
1. Navigation difficult to use
Howto.gov/firstfridaysHowto.gov/firstfridays
Before After
• Average daily visits up 50%
• Mobile Home visits up 32%
• Help Page visits up 1200%
• Desktop site visits up 3.3%
Howto.gov/firstfridays
2. Poor search functionality
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays
Howto.gov/fiHowto.gov/f
Before: 7 text items to scan
After: 3 text items to scan
3. Forms require too much effort
3. Forms difficult to use
1. Too much effort
2. Poor terminology
3. Malfunction / deleted answers
4. Too many questions
5. Unclear what’s required
3. Forms difficult to use
1. Too much effort
2. Poor terminology
3. Malfunction / deleted answers
4. Too many questions
5. Unclear what’s required
4. Purpose of Site Unclear
5. Difficulty with top tasks
5. Difficulty with top tasks
1. Can’t find info via any means or know
where to begin
5. Difficulty with top tasks
1. Can’t find info via any means or know
where to begin
6. Lack of Plain Language
6. Lack of Plain Language
1. Insider Speak
2. Technical / scientific terms
3. Jargon
4. Alphabet soup
Top 6 Usability Challenges
1. Navigation difficult, confusing or hidden
2. Search feature hidden or doesn’t work
3. Forms difficult to use
4. Purpose of site unclear
5. Can't find what they are looking for
6. Lack of Plain Language
Howto.gov/firstfridaysHowto.gov/firstfridays
Wrap up: Opportunities
• Howto.gov
• Us – howto.gov/firstfridays
• Blog.howto.gov – 3,000+ subscribers
• DigitalGov University - howto.gov/training
• User Experience Community of Practice
• Usability Testing Community of Practice
Share your greatness
• Email us at firstfridays@gsa.gov or let people
know about:
• Labs or conference space to share?
• Great tools or resources (e.g. Expert Eval tool or
Persona template)
• Best Practices
• Testimonials
• ROI stats
• Speakers available
First Fridays Usability Program
Jonathan Rubin
Jonathan.rubin@gsa.gov
Twitter: @govnewmedia
www.Howto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays

Top User Experience Challenges in Government

  • 1.
    Howto.gov/firstfridays Top User Experience Challengesin Government Jonathan Rubin First Fridays Usability Program July 10, 2013
  • 3.
  • 4.
    Our Survey ofUX Activities 1. Survey Monkey with 65 respondents 2. 100% Cabinet agencies 3. Co-created with Jean Fox (+ thanks Axle Brown) 4. Not scientific or representative 5. Goal: • Audience research • Develop web-based and in-person training • Share aggregate in presentations, etc. Howto.gov/firstfridaysHowto.gov/firstfridays
  • 5.
    What we asked 1.Agency (but anonymous) 2. About you – title, UX education, team size 3. How widespread is UX 4. Challenges to implementing UX 5. Resources 6. Types of UCD methods 7. What would help? Howto.gov/firstfridaysHowto.gov/firstfridays
  • 6.
    Good news • Averageteam: 2 PT and 2 FT • Wide range of UX methods being used • #1 surveys, #2 style guide, #3 tests • Thirst for training, many self-taught • 91% doing UX on websites, ⅓ mobile • Using respected tools (20% eyetracking!) • Future: More testing, services • Some agencies have UX institutionalized Howto.gov/firstfridaysHowto.gov/firstfridays
  • 7.
    Challenges / Lackof… • Awareness: • “[People] refuse to believe the results of usability tests because they are painful.” • “We are spending hundreds of thousands of dollars without one usability test. Web staff understand the need, but no one is listening.” • Facilities • “No facilities” / “Our lab is about 15 years young” Howto.gov/firstfridaysHowto.gov/firstfridays
  • 8.
    Challenges / Lackof… • Resources • “Very old models are used for interaction” • “Army of one” • Training • “Figure it out as we go along” • Standards / Best Practices • “Applying standard best practices can be challenging… many offices and components have different processes and standards.” Howto.gov/firstfridaysHowto.gov/firstfridays
  • 9.
  • 10.
    34% 17%15% 15% 6% 6% 5% 2% What best describeswhat you do? Website Mngr Other Project Mngr UX Specialist Web Designer IA Researcher Web Metrics Mng
  • 11.
    # UX staff Howto.gov/firstfridaysHowto.gov/firstfridays PartTime Full Time Had at least 1 person 86% 59% Median # of people 2 2 Average # of people 2 3 Range of people 0-10 0-30
  • 12.
    Howto.gov/firstfridaysHowto.gov/firstfridays 0% 10% 20% 30% 40% 50% 60% 70% College / Grad school CertificationWorkshops, webinars On-the-job mentorship Other Where did your staff get UX training?
  • 13.
    Howto.gov/firstfridaysHowto.gov/firstfridays 3% 19% 46% 25% 8% 0% 10% 20%30% 40% 50 op to Bottom UX Used frequently Occasional UX Know but don't practice No one knows UX How widespread is UX at your organization?
  • 14.
  • 15.
    Howto.gov/firstfridaysHowto.gov/firstfridays % Who “Often”or “Always” use 0% 10% 20% 30% 40% 50% Tests: Mobile Contextual inquiries Focus groups Tests: Remote Personas Tests: High Fidelity User Analysis Expert Reviews Task Analysis Tests: In-person Surveys Style guides
  • 16.
    Howto.gov/firstfridaysHowto.gov/firstfridays % Who “Never”use these methods 0% 10% 20% 30% 40% 50% 60% Personas Tests: Remote Context. Inquiries Tests: High Fidelity Test: Mobile Card sorts Tests: Low Fidelity Affinity diagrams Tests:: First-Click Tests: Eyetracking
  • 17.
    Howto.gov/firstfridaysHowto.gov/firstfridays How often doyou perform usability tests? 0 5 0 5 0 5 Never Rarely Sometimes Often Always Don't Know In person Remote
  • 18.
    Asked about UXmethods • Top 3: Surveys, Style guide, Tests • Bottom 3: Affinity diagrams Eyetracking, First Click • 10% never a usability test – bad? • 16% never a survey • Task analysis - 8.5% always, 55% sometimes Howto.gov/firstfridaysHowto.gov/firstfridays
  • 19.
    Training Needs • “UXTraining classes - webinars aren't enough” • “Software” - Morae, Surveys, CrazyEgg • “Any and all” • “Be more aware of findings” • “How to get buy-in from higher ups “ • Methods: First-Click Testing, Style Guides, Card Sorts, Surveys, Task analysis Howto.gov/firstfridaysHowto.gov/firstfridays
  • 20.
    Plans for thefuture Howto.gov/firstfridaysHowto.gov/firstfridays Asked about future plans:
  • 21.
    Howto.gov/firstfridaysHowto.gov/firstfridays Biggest obstacle tousing UX? 0% 10% 20% 30% 40% 50% 60% 70% 80% Tried UX, failed Other No UX awareness Don't get UX ROI UX not built in Lack of time No resources
  • 22.
    Howto.gov/firstfridaysHowto.gov/firstfridays0% 10% 20%30% 40% 50% 60% One-way mirror Observation room Testing room Other Multi-purpose rooms No space for testing Which do you have? (check all)
  • 23.
    Challenges / Lackof… • Resources - staff and metrics software • Awareness – Managers not sold on UX • Training - Staff and Manager • Standards / Best Practices • Testing facilities Howto.gov/firstfridaysHowto.gov/firstfridays
  • 24.
  • 25.
    Two types ofgov challenges: 1. Gov UX Strategic and Resource challenges(Survey) 2. Persistency UX problems on fed websites (FF test data) Howto.gov/firstfridaysHowto.gov/firstfridays
  • 26.
    FF Test Data 1.29 products reviewed in 2012 2. 130+ significant problems 3. 32 different usability categories - problems big and small Thanks to Cynthia, Molly, Josh, and Ruth! Howto.gov/firstfridaysHowto.gov/firstfridays
  • 27.
    Top 6 UsabilityChallenges 1. Navigation difficult, confusing or hidden 2. Search feature hidden or doesn’t work 3. Forms difficult to use 4. Purpose of site unclear 5. Difficulty with Top Tasks 6. Lack of Plain Language Howto.gov/firstfridaysHowto.gov/firstfridays
  • 28.
    1. Navigation difficultto use Howto.gov/firstfridaysHowto.gov/firstfridays
  • 29.
  • 30.
    • Average dailyvisits up 50% • Mobile Home visits up 32% • Help Page visits up 1200% • Desktop site visits up 3.3% Howto.gov/firstfridays
  • 31.
    2. Poor searchfunctionality Howto.gov/firstfridaysHowto.gov/firstfridays
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
    Howto.gov/fiHowto.gov/f Before: 7 textitems to scan After: 3 text items to scan
  • 37.
    3. Forms requiretoo much effort
  • 38.
    3. Forms difficultto use 1. Too much effort 2. Poor terminology 3. Malfunction / deleted answers 4. Too many questions 5. Unclear what’s required
  • 39.
    3. Forms difficultto use 1. Too much effort 2. Poor terminology 3. Malfunction / deleted answers 4. Too many questions 5. Unclear what’s required
  • 40.
    4. Purpose ofSite Unclear
  • 42.
  • 43.
    5. Difficulty withtop tasks 1. Can’t find info via any means or know where to begin
  • 44.
    5. Difficulty withtop tasks 1. Can’t find info via any means or know where to begin
  • 45.
    6. Lack ofPlain Language
  • 47.
    6. Lack ofPlain Language 1. Insider Speak 2. Technical / scientific terms 3. Jargon 4. Alphabet soup
  • 48.
    Top 6 UsabilityChallenges 1. Navigation difficult, confusing or hidden 2. Search feature hidden or doesn’t work 3. Forms difficult to use 4. Purpose of site unclear 5. Can't find what they are looking for 6. Lack of Plain Language Howto.gov/firstfridaysHowto.gov/firstfridays
  • 49.
    Wrap up: Opportunities •Howto.gov • Us – howto.gov/firstfridays • Blog.howto.gov – 3,000+ subscribers • DigitalGov University - howto.gov/training • User Experience Community of Practice • Usability Testing Community of Practice
  • 51.
    Share your greatness •Email us at firstfridays@gsa.gov or let people know about: • Labs or conference space to share? • Great tools or resources (e.g. Expert Eval tool or Persona template) • Best Practices • Testimonials • ROI stats • Speakers available
  • 52.
    First Fridays UsabilityProgram Jonathan Rubin Jonathan.rubin@gsa.gov Twitter: @govnewmedia www.Howto.gov/firstfridays Howto.gov/firstfridaysHowto.gov/firstfridays