Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Loading in …3
1 of 15

The Trade-Offs of Transitioning to Open Government Data



Download to read offline

A case study that examines the transition from traditionally published federal spending data to open government-ish federal spending data.

Related Books

Free with a 30 day trial from Scribd

See all

The Trade-Offs of Transitioning to Open Government Data

  1. 1. Transitioning to Open Government Data: What Are the Trade-Offs? a federal spending data case study Becky Sweger Director of Data and Technology National Priorities Project
  2. 2. Federal Spending Data: Make Way for the New
  3. 3. Consolidated Federal Funds Report
  4. 4. (FFATA)
  5. 5. Shiny
  6. 6. Responsibility Shifts to Consumer “change in emphasis from providing government information to information consumers to providing information to mediators…While information consumers typically require some analysis and simplification, information mediators can achieve more innovative solutions with the most raw form of government data.” 2nd Principle of Open Government Data from Open Government Data by Josh Tauberer
  7. 7. Unavoidable Trade-Offs
  8. 8. Unavoidable Trade-Offs Improvement Trade-Off Timeliness Accuracy More Data Fields Simplicity & Usability Granularity Single Format Granularity Convenient Aggregates
  9. 9. Unfortunate Losses
  10. 10. Unfortunate Losses: Data Sources CFFR had:  Federal disability and retirement benefits  Aggregated salaries  Earned Income Tax Credit (refundable portion)
  11. 11. Unfortunate Losses: Continuity
  12. 12. Unfortunate Losses: Data Scrubbing
  13. 13. Unfortunate Losses: Support • USASpending support = no phone number • CFFR: phone number published on front page
  14. 14. Can We Mitigate Trade-Offs & Losses? • How can we support data consumers without tech resources? • What’s a good model for promoting dataset literacy and community? • Can open gov types contribute by filling in some of the gaps?

Editor's Notes

  • IntroNational Priorities Project: non-profit, non-partisan federal budget research org in Northampton, MA15 years of data-driven projects, started working with federal spending data in early 2011
  • Why CFFR & represent federal government spending data – some of the most critical data our government generates and publishesAssistance (grants, loans, payments, insurance)Procurement (contracts)Both derived via same-ish data sources (though CFFR is more inclusive)CFFR: Federal Assistance Awards Data System (FAADS) + Federal Procurement Data System + Salaries + non-FAADS FAADS plus + Federal Procurement Data SystemThese similarities make an interesting case study to compare old and new ways of delivering government dataOut of scope: data quality. For more information, see from the Sunlight Foundation
  • CFFR – Functional but not sexyAuthorized in 1983Limited interactive queriesBulk downloadsExtensive documentation: methodology, data dictionary, files for supporting codes100+ page PDF with maps, charts, and analysisArchived dataPublished annuallyAggregated by federal program/agency and geographyCFFRCensus BureauAuthorized in 1982 and reported federal spending data from FY 1983 – FY 2010 ( reports: total funds obligated by geography and total funds actually spent by geographyLaw requires 10 data sources in addition to FAADS and FPDSReports/data due 180 days after close of fiscal year“This program provides for the production of the annual Consolidated Federal Funds Report (CFFR) products, including the CFFR Report, on-line query tool, and Federal Aid to States Report. The CFFR products provide statistics on Federal Government expenditures or obligations in each state, territory, county, and municipality. Coverage of the CFFR includes Federal salaries and wages, procurement, grants, and direct payments, as well as Federal insurance and loan programs. The Congress is especially interested in CFFR data as it illustrates the effects of Federal programs on state and sub-state economies. “ Canceled when the Census terminated the Federal Financial Statistic program in the FY 2012 budget in order to “fund higher priority programs within the Census Bureau and Department.” ( Also see the “Strengthening Federal Statistics” section of the FY 2012 Budget Request’s Analytical Perspectives document: (page 342)
  • OMBFederal Funding Accountability and Transparency Act of 2006 required establishment of a single, searchable website before Jan. 2008 ( for fiscal year 2007 and each fiscal year thereafterAwards > $25,000Name of entity receiving awardAmount of awardAward info including funding agency, transaction type, program source, award description, NAICS or CFDA code, location of award recipient and place of award performanceGrants AND sub-grantsContracts AND sub-contractsUpdated no later than 30 days after award is granted
  • much more modern websiteWhat we gain from USASpending.govGranularity/flexibilityTimelinessAPIDetailed custom searchInteractive graphs/mapsSub-awards
  • Shifting the burden to the end user—we need to talk about that.
  • With more possibilities, comes more work.It’s easier to present accurate data when it’s only scrubbed and presented once a year. Frequent updates to mean that bad data will find its way in. Of course there’s a process for providing corrections, but the user must know how it works.More data, more visualizations, and a robust custom search = a website that is harder to user and information that is harder to interpret. Consumer will have to make more decisions (e.g., aggregate by place of performance or recipient location?)Transaction-level source records from different systems (awards and procurement) = different download formatsDifferent types of awards records are populated differently
  • Unreasonable to expect duplicate data entry foreverHowever, it’s unfortunate to have a gap in our spending data, especially at this time
  • Data dictionary not enforcedZeroes versus blanks versus nullsGeographic inconsistency: for example, two same addresses might have a slightly different zip code
  • USASpending will not accept support calls, only e-mailSite does list the Senior Accountable Official for each agency, but there is no contact information questions about this….How can consumers and the govt. work together to make sure people understand what’s being published?What effect does outsourcing have on the quality of support?
  • In some ways, the switch from published statistical data to open data makes access less democratic—those without tech resources can’t get what they need.
  • ×