0
Measuring Gov 2.0
(But First, Lessons from Gov 1.0!)




          Andrew Krzmarzick
            Senior Project Coordinato...
AGENDA
 Gov 1.0 Measurement
 Advancing Social Media Survey
 Gov 2.0 Measurement Hallmarks
3
“In many ways,
the Internet
more resembles
an ancient
bazaar…”


                 4
5
Ordinary people…possess information –
serious, expert, fact-based,
scientific information.

[They] are ready and willing t...
GOV 1.0 MEASUREMENT

1. Brookings
2. Forrester
3. OMB’s E-Gov Initiative
4. ForeSee
GOV 1.0 MEASUREMENT
1. Brookings Institution E-Government Study
Since 2000:

   • Reviewed >   1,500 State / Federal Websi...
GOV 1.0 MEASUREMENT
 1. Brookings Institution E-Government Study
                                      4 points
         4...
GOV 1.0 MEASUREMENT
1. Brookings Institution E-Government Study


You got it…
or you don’t.
GOV 1.0 MEASUREMENT
1. Brookings Institution E-Government Study

 Applied to Gov 2.0:
  Should we count
the number of soci...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey
         User Goals       Point Scale
                          ...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey

Value
• Is essential content available where needed?
• Is essen...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey

Navigation
• Are menu categories/sub-categories clear?
• Is wor...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey

Presentation
• Does site use language that’s easy to understand...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey

Trust
• Does the site present privacy and security policies?
• ...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey


                          Trust the
                          ...
GOV 1.0 MEASUREMENT
2. Forrester Website Benchmark Survey

                       Applied to Gov 2.0

                    ...
GOV 1.0 MEASUREMENT
3. OMB E-Government Initiatives

E-Government Act of 2002
“a broad framework of measures

that require...
WEB 1.0 MEASUREMENT
3. OMB E-Gov Initiatives
WEB 1.0 MEASUREMENT
3. OMB E-Gov Initiatives
• Business.gov
• Regulations.gov   As of January 2009:
• USALearning.gov   • ...
WEB 1.0 MEASUREMENT
3. OMB E-Gov Initiatives

d) Improve E-Gov Initiatives Performance Measures
• Adoption/Participation –...
WEB 1.0 MEASUREMENT
3. OMB E-Gov Initiatives


                             Peer
                           Pressure?
GOV 2.0
DASHBOARD?
GOV 2.0 DASHBOARD?
GOV 1.0 MEASUREMENT
4. ForeSee: E-Government Satisfaction Index
GOV 1.0 MEASUREMENT
 4. ForeSee: E-Government Satisfaction Index
E-Gov – What is it?
                                • Key...
GOV 1.0 MEASUREMENT
4. ForeSee: E-Government Satisfaction Index

E-Gov Outcomes as of Q1 2009
  • All-time high: 74.1% sat...
WEB 1.0 MEASUREMENT
3. ForeSee: E-Government Satisfaction Index

E-Gov Top Performers
  • http://www.ssa.gov/estimator (Sc...
WEB 1.0 MEASUREMENT
4. ForeSee: E-Government Satisfaction Index

    The
customers’
  always
  right?
WEB 1.0 MEASUREMENT
  4. ForeSee: E-Government Satisfaction Index
Applied to Gov 2.o

 Add/Alter a series of
 questions re...
“ADVANCING SOCIAL MEDIA”
SURVEY

      • 10 questions
      • 4,000 potential respondents
         • Web Manager’s Forum
 ...
“ADVANCING SOCIAL MEDIA” SURVEY
Questions
1.   What enables or hinders you from using social media?
2.   What social media...
Advancing Social Media in Government


Do you establish metrics prior to implementing any of the
above social media tools ...
“ADVANCING SOCIAL MEDIA” SURVEY

 “…we don't have benchmark data,
   nor do we have measurable objectives
            for ...
Advancing Social Media in Government                                                 

      If you answered YES for #4, w...
GENERAL                                              BLOGS
• Views                                              • Posts
  ...
“ADVANCING SOCIAL MEDIA” SURVEY


“Metrics only tell part of the story...
    it is difficult to directly measure

       ...
40
TRANSPARENCY
ENGAGEMENT
For More Information:
                    Andrew Krzmarzick
                 Senior Project Coordinator
                  ...
RESOURCES
1.    Ari Herzog: http://www.ariwriter.com, @ariherzog
2.    Andrew Krzmarzick: http://generationshift.blogspot....
Web 2.0 Measurement: Open Government Innovations Conference
Web 2.0 Measurement: Open Government Innovations Conference
Upcoming SlideShare
Loading in...5
×

Web 2.0 Measurement: Open Government Innovations Conference

1,706

Published on

Presentation delivered at the Open Government and Innovations (OGI) Conference in Washington, DC, on July 22, 2009. Outlines the ways in which government has measured its web presence in a "1.0" context, including an overview of the measurement activities conducted by Brookings Institution, Foresee, Forrester and the e-Government Act of 2002.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,706
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
40
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Web 2.0 Measurement: Open Government Innovations Conference"

  1. 1. Measuring Gov 2.0 (But First, Lessons from Gov 1.0!) Andrew Krzmarzick Senior Project Coordinator The Graduate School http://www.graduateschool.edu http://generationshift.blogspot.com Twitter: @krazykriz
  2. 2. AGENDA  Gov 1.0 Measurement  Advancing Social Media Survey  Gov 2.0 Measurement Hallmarks
  3. 3. 3
  4. 4. “In many ways, the Internet more resembles an ancient bazaar…” 4
  5. 5. 5
  6. 6. Ordinary people…possess information – serious, expert, fact-based, scientific information. [They] are ready and willing to share that information across geographic, disciplinary, and institutional boundaries. - Beth Noveck, Wiki Government 6
  7. 7. GOV 1.0 MEASUREMENT 1. Brookings 2. Forrester 3. OMB’s E-Gov Initiative 4. ForeSee
  8. 8. GOV 1.0 MEASUREMENT 1. Brookings Institution E-Government Study Since 2000: • Reviewed > 1,500 State / Federal Websites •0 – 100 Point Scale
  9. 9. GOV 1.0 MEASUREMENT 1. Brookings Institution E-Government Study 4 points 4 points +1 4 points 4 points +2 4 points per feature (72 total possible points) + 28 more for frequency
  10. 10. GOV 1.0 MEASUREMENT 1. Brookings Institution E-Government Study You got it… or you don’t.
  11. 11. GOV 1.0 MEASUREMENT 1. Brookings Institution E-Government Study Applied to Gov 2.0: Should we count the number of social media tools an agency deploys?
  12. 12. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey User Goals Point Scale 2 = Strong Pass (Best Practice)  Value 1 = Pass (No problems)  Navigation -1 = Fail (One major or many minor problems) -2 = Strong Fail (2+ Major…)  Presentation  Trust
  13. 13. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey Value • Is essential content available where needed? • Is essential function available where needed? • Are essential content and function given priority in display?
  14. 14. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey Navigation • Are menu categories/sub-categories clear? • Is wording in hyperlinks clear and informative? • Are keyword searches comprehensive and precise?
  15. 15. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey Presentation • Does site use language that’s easy to understand? • Does the site use graphics, icons, and symbols “ “ ? • Is text legible? Text format/layout support easy scanning? • Are form fields and interactive elements placed well?
  16. 16. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey Trust • Does the site present privacy and security policies? • Do location cues orient the user? • Is contextual help available at key points? • Does the site help users avoid and recover from errors?
  17. 17. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey Trust the experts?
  18. 18. GOV 1.0 MEASUREMENT 2. Forrester Website Benchmark Survey Applied to Gov 2.0 Blog Review Tool? Tool Placement?
  19. 19. GOV 1.0 MEASUREMENT 3. OMB E-Government Initiatives E-Government Act of 2002 “a broad framework of measures that require using Internet-based IT to enhance citizen access to Government information and services...”
  20. 20. WEB 1.0 MEASUREMENT 3. OMB E-Gov Initiatives
  21. 21. WEB 1.0 MEASUREMENT 3. OMB E-Gov Initiatives • Business.gov • Regulations.gov As of January 2009: • USALearning.gov • 28/28 agencies had • Grants.gov implementation plans • FedBizOpps.gov • 87% of milestones met • Recreation.gov • USAJOBS.gov
  22. 22. WEB 1.0 MEASUREMENT 3. OMB E-Gov Initiatives d) Improve E-Gov Initiatives Performance Measures • Adoption/Participation – Is the relevant community participating? • Usage – What’s the level of use by the target community? • Customer Satisfaction – Is the community satisfied w/ products/services? • Cost Savings/Avoidance – What’s $ value for government /citizens? • Efficiency - Any decreases in time and/or increases in productivity?
  23. 23. WEB 1.0 MEASUREMENT 3. OMB E-Gov Initiatives Peer Pressure?
  24. 24. GOV 2.0 DASHBOARD?
  25. 25. GOV 2.0 DASHBOARD?
  26. 26. GOV 1.0 MEASUREMENT 4. ForeSee: E-Government Satisfaction Index
  27. 27. GOV 1.0 MEASUREMENT 4. ForeSee: E-Government Satisfaction Index E-Gov – What is it? • Key question: • Measures 94 e-gov sites on: How satisfied are citizens? 1. Navigation • University of Michigan’s 2. Functionality American Customer 3. Search Satisfaction Index (ACSI) 4. Look and Feel
  28. 28. GOV 1.0 MEASUREMENT 4. ForeSee: E-Government Satisfaction Index E-Gov Outcomes as of Q1 2009 • All-time high: 74.1% satisfaction overall • Most successful sites: • Citizens find information quickly and easily • E-Commerce and Transaction functions
  29. 29. WEB 1.0 MEASUREMENT 3. ForeSee: E-Government Satisfaction Index E-Gov Top Performers • http://www.ssa.gov/estimator (Score: 89) • http://medlineplus.gov (Score: 86) • http://www.niams.nih.gov/index.htm (Score: 82) • http://www.cia.gov/employment (Score: 81)
  30. 30. WEB 1.0 MEASUREMENT 4. ForeSee: E-Government Satisfaction Index The customers’ always right?
  31. 31. WEB 1.0 MEASUREMENT 4. ForeSee: E-Government Satisfaction Index Applied to Gov 2.o Add/Alter a series of questions related to social media? Real-time polling data (shared right away)?
  32. 32. “ADVANCING SOCIAL MEDIA” SURVEY • 10 questions • 4,000 potential respondents • Web Manager’s Forum • GovLoop • GovTwit Directory •International Contacts • 105 responses • 7 countries
  33. 33. “ADVANCING SOCIAL MEDIA” SURVEY Questions 1. What enables or hinders you from using social media? 2. What social media tools does your agency use? 3. Rate tools per value/importance in achieving mission. 1. Do you establish metrics prior to implementation? 1. If yes, for which tools and what variables do you measure? 2. Is privacy, security and monitoring social tools important? 3. How often do you use social media in your job? 4. Thoughts re: gov standards w/browsers, software, etc.? 5. Is CTO/CIO actively involved in social media initiatives? 6. Where are you from/what agency do you represent?
  34. 34. Advancing Social Media in Government Do you establish metrics prior to implementing any of the above social media tools at your agency? Answer Options Response Response Frequency Count Yes 43.8% 46 No 56.2% 59 answered question 105 skipped question 0
  35. 35. “ADVANCING SOCIAL MEDIA” SURVEY “…we don't have benchmark data, nor do we have measurable objectives for any of our tools. We basically just put them out there and hope they work. It's kind of annoying.”
  36. 36. Advancing Social Media in Government     If you answered YES for #4, what tools are you measuring?   Answer Options Response Response Frequency Count 1  Blogs 60.4% 29 2  E-mail 58.3% 28 3  Twitter 56.3% 27 4  RSS 43.8% 21 5  YouTube 33.3% 16 6  Facebook 31.3% 15 7  Podcasts 25.0% 12 8  Mobile devices 16.7% 8 9  Wikis 14.6% 7 10  Delicious 8.3% 4 11  Z-other social networking (e.g. Flickr, Govloop, LinkedIn, Ning) 8.3% 4 12  LinkedIn 6.3% 3 13  Z-other video sharing (e.g. Hulu, Vimeo, Viddler) 6.3% 3 14  Z-other social bookmarking (e.g. Digg, StumbleUpon) 4.2% 2 15  Gaming 2.1% 1 16  MySpace 2.1% 1 17  Second Life 0.0% 0 18  Z-other virtual worlds/3D Web (e.g. YooWalk) 0.0% 0   answered question 48
  37. 37. GENERAL BLOGS • Views • Posts  Number of views/page • Views PODCASTS • Comments (quantitative)  Time on site/page • Subscriptions • People • Visitors • Downloads Google • Complaints • Visits  Unique Analytics • Comments (qualitative)  Overall • Subscriptions  Paths to site • Paths • Links elsewhere?  Paths on site WIKIS  Geographic distribution •Users •Adoption rate RSS  Searches • Subscriptions  Satisfaction •Edits  Comments TWITTER •Followers •Growth YOUTUBE/VIMEO •Link click-throughs (when, what types of • Views content) • Comments •Retweets • Downloads •Rankings (Twinfluence, Twitter Grader) • Page placement traffic •Comparison to similar organizations • Ratings •Friends • Click-throughs •Conversations
  38. 38. “ADVANCING SOCIAL MEDIA” SURVEY “Metrics only tell part of the story... it is difficult to directly measure influence…”
  39. 39. 40
  40. 40. TRANSPARENCY
  41. 41. ENGAGEMENT
  42. 42. For More Information: Andrew Krzmarzick Senior Project Coordinator (202) 821-6288 Andrew_Krzmarzick@grad.usda.gov Thank You! GraduateSchool.edu GenerationShift.blogspot.com IAmPublicsService.org Twitter.com/krazykriz LinkedIn.com/in/AndrewKrzmarzick Delicious.com/akrzmarzick Slideshare.net/akrzmarzick 44
  43. 43. RESOURCES 1. Ari Herzog: http://www.ariwriter.com, @ariherzog 2. Andrew Krzmarzick: http://generationshift.blogspot.com, @krazykriz 3. Brookings Institution E-Government Study: http://snurl.com/crpxn 4. Forrester Website Benchmark Survey: http://www.forrester.com/cxpbenchmark 5. ForeSee: E-Government Satisfaction Index: http://snurl.com/crpyu 6. OMB E-Gov Initiative and Report: http://www.whitehouse.gov/omb/e-gov/ 7. Social Media and Government (Jeffrey Levy): http://snurl.com/crq0x 8. Federal Web Manager’s “Putting Citizens First:” http://snurl.com/crndj 9. IBM’s “Leveraging Web 2.0 in Government”: http://snurl.com/crq3e 10. Air Force Blog Assessment: http://is.gd/eAYo
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×