Your SlideShare is downloading. ×
The Trade-Offs of Transitioning to Open Government Data
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Trade-Offs of Transitioning to Open Government Data


Published on

A case study that examines the transition from traditionally published federal spending data to open government-ish federal spending data.

A case study that examines the transition from traditionally published federal spending data to open government-ish federal spending data.

Published in: News & Politics, Technology

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • IntroNational Priorities Project: non-profit, non-partisan federal budget research org in Northampton, MA15 years of data-driven projects, started working with federal spending data in early 2011
  • Why CFFR & represent federal government spending data – some of the most critical data our government generates and publishesAssistance (grants, loans, payments, insurance)Procurement (contracts)Both derived via same-ish data sources (though CFFR is more inclusive)CFFR: Federal Assistance Awards Data System (FAADS) + Federal Procurement Data System + Salaries + non-FAADS FAADS plus + Federal Procurement Data SystemThese similarities make an interesting case study to compare old and new ways of delivering government dataOut of scope: data quality. For more information, see from the Sunlight Foundation
  • CFFR – Functional but not sexyAuthorized in 1983Limited interactive queriesBulk downloadsExtensive documentation: methodology, data dictionary, files for supporting codes100+ page PDF with maps, charts, and analysisArchived dataPublished annuallyAggregated by federal program/agency and geographyCFFRCensus BureauAuthorized in 1982 and reported federal spending data from FY 1983 – FY 2010 ( reports: total funds obligated by geography and total funds actually spent by geographyLaw requires 10 data sources in addition to FAADS and FPDSReports/data due 180 days after close of fiscal year“This program provides for the production of the annual Consolidated Federal Funds Report (CFFR) products, including the CFFR Report, on-line query tool, and Federal Aid to States Report. The CFFR products provide statistics on Federal Government expenditures or obligations in each state, territory, county, and municipality. Coverage of the CFFR includes Federal salaries and wages, procurement, grants, and direct payments, as well as Federal insurance and loan programs. The Congress is especially interested in CFFR data as it illustrates the effects of Federal programs on state and sub-state economies. “ Canceled when the Census terminated the Federal Financial Statistic program in the FY 2012 budget in order to “fund higher priority programs within the Census Bureau and Department.” ( Also see the “Strengthening Federal Statistics” section of the FY 2012 Budget Request’s Analytical Perspectives document: (page 342)
  • OMBFederal Funding Accountability and Transparency Act of 2006 required establishment of a single, searchable website before Jan. 2008 ( for fiscal year 2007 and each fiscal year thereafterAwards > $25,000Name of entity receiving awardAmount of awardAward info including funding agency, transaction type, program source, award description, NAICS or CFDA code, location of award recipient and place of award performanceGrants AND sub-grantsContracts AND sub-contractsUpdated no later than 30 days after award is granted
  • much more modern websiteWhat we gain from USASpending.govGranularity/flexibilityTimelinessAPIDetailed custom searchInteractive graphs/mapsSub-awards
  • Shifting the burden to the end user—we need to talk about that.
  • With more possibilities, comes more work.It’s easier to present accurate data when it’s only scrubbed and presented once a year. Frequent updates to mean that bad data will find its way in. Of course there’s a process for providing corrections, but the user must know how it works.More data, more visualizations, and a robust custom search = a website that is harder to user and information that is harder to interpret. Consumer will have to make more decisions (e.g., aggregate by place of performance or recipient location?)Transaction-level source records from different systems (awards and procurement) = different download formatsDifferent types of awards records are populated differently
  • Unreasonable to expect duplicate data entry foreverHowever, it’s unfortunate to have a gap in our spending data, especially at this time
  • Data dictionary not enforcedZeroes versus blanks versus nullsGeographic inconsistency: for example, two same addresses might have a slightly different zip code
  • USASpending will not accept support calls, only e-mailSite does list the Senior Accountable Official for each agency, but there is no contact information questions about this….How can consumers and the govt. work together to make sure people understand what’s being published?What effect does outsourcing have on the quality of support?
  • In some ways, the switch from published statistical data to open data makes access less democratic—those without tech resources can’t get what they need.
  • Transcript

    • 1. Transitioning to Open Government Data: What Are the Trade-Offs? a federal spending data case study Becky Sweger Director of Data and Technology National Priorities Project
    • 2. Federal Spending Data: Make Way for the New
    • 3. Consolidated Federal Funds Report
    • 4. (FFATA)
    • 5. Shiny
    • 6. Responsibility Shifts to Consumer“change in emphasis from providing governmentinformation to information consumers toproviding information to mediators…Whileinformation consumers typically require someanalysis and simplification, informationmediators can achieve more innovativesolutions with the most raw form of governmentdata.”2nd Principle of Open Government Data from Open Government Data by Josh Tauberer
    • 7. Unavoidable Trade-Offs
    • 8. Unavoidable Trade-OffsImprovement Trade-OffTimeliness AccuracyMore Data Fields Simplicity & UsabilityGranularity Single FormatGranularity Convenient Aggregates
    • 9. Unfortunate Losses
    • 10. Unfortunate Losses: Data SourcesCFFR had:  Federal disability and retirement benefits  Aggregated salaries  Earned Income Tax Credit (refundable portion)
    • 11. Unfortunate Losses: Continuity
    • 12. Unfortunate Losses: Data Scrubbing
    • 13. Unfortunate Losses: Support• USASpending support = no phone number• CFFR: phone number published on front page
    • 14. Can We Mitigate Trade-Offs & Losses?• How can we support data consumers without tech resources?• What’s a good model for promoting dataset literacy and community?• Can open gov types contribute by filling in some of the gaps?