Demanding More Federal Agencies’ Data Use to Drive Mission and Meet Mandates


Published on

Study conducted in conjunction with Government Business Council

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Demanding More Federal Agencies’ Data Use to Drive Mission and Meet Mandates

  1. 1. Demanding MoreFederal Agencies’ Data Use to Drive Mission and Meet Mandates Sponsored by:
  2. 2. About The Research Intelligence Division Of Government Executive Media Group Dedicated to Advancing the Business of Government Through analysis, insight and analytical independence Extension of GEMG’s 40 Years of Editorial Standards & Ethical Values GBC studies influential decision-makers to produce intelligence-based analysis 2
  3. 3. Methodology and ScopeStudy Scope• This study aims to assess current data analysis behaviors and efforts in the federal government, identify common causes for why data collection and analytics use may fall short of expectations and identify potential areas where analytics can enhance agency information and decision-making.• Research sought to determine: – the types of data that are most frequently used in agencies and what goals agencies have for their analysis; – current levels of confidence in analytics results and use; – the most common challenges agencies encounter in implementation; – and emerging needs in skills and education and cultural adoption.Focus Group• The Government Business Council (GBC) conducted a focus group of twelve GS/GM-12 through SES- level federal employees in June 2011.• The focus group assessed the attitudes and knowledge of federal employees regarding the use of data in their agencies. Participants offered examples of data use in their agencies, success stories and challenges.Online Survey• The focus group informed the development of a survey of federal managers, drawing from the Government Executive subscriber database. GBC deployed the survey by email in October 2011. A full 209 federal managers completed the survey.• The data and analysis shared in this report reflect the opinions of those federal managers who participated in the focus group or survey.
  4. 4. Executive SummaryData In DemandBudget pressures are compelling federal agencies to increase their use and analysis of data. An August 2011memorandum from Office of Management and Budget Director Jacob Lew called on agencies to request five percentless than their 2011 appropriations when preparing their fiscal year 2013 budgets. Lew instructed agencies to explainhow they will ―acquire, analyze, evaluate, and use data to improve policy and operational decisions.‖In this GBC survey of federal managers, sponsored by Deloitte 1, a full 58 percent say their programs collect data inorder to comply with the in-depth reporting requirements of OMB and other external authorities. Half of managersbelieve cost constraints have ―significantly increased‖ the importance of data.Managers Struggle to Make Data Accessible and ActionableCollecting data and putting it to ―meaningful‖ use is challenging, given the procedural, technical, and human capitalconstraints managers face. Agencies have difficulty collecting and sharing datasets with outside groups and federalorganizations. Nearly half of managers have difficulty making data both accessible and actionable.Serving many audiences, both internal and external, is challenging as agencies must alter their analyses in response todiverse demands. This strain on manager time, amplified by insufficient staffing, renders difficulties for agency leadersas they look to data for planning. Federal managers lack the trusted, data-capable staff necessary to analyze data forinternal decision-making.Data Challenges Result in Hasty Cost-CuttingGiven the challenges of collection, managers are unlikely to use data to determine cost-savings. It is more likely foragencies to eliminate jobs than to take a closer look at their programs and functions. Over half of managers say theiragency or program will downsize the federal workforce in order to achieve cost-savings, while only 23 percent willanalyze data to review the outcomes of programs, reduce improper payments or consolidate their business systems. 1 As used in this document, ―Deloitte‖ means Deloitte LLP and its subsidiaries. Please see for a detailed description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting.
  5. 5. Respondent Profile 5
  6. 6. Survey Respondent Job Grade / Rank Profile SES 2% • The majority of respondents hold GS/GM-15 16% a job grade of GS/GM-11 and GS/GM-14 18% above, and the majority of GS/GM-13 19% respondents are concentrated in GS/GM-12 15% the GS/GM-13 through SES range. GS/GM-11 13% Lt. Col./Commander 1% • Forty percent of respondents are Major/Lt. Commander 1% involved in operations and Other 15% program management. Minorities of the sample are responsible for finance, human capital and Areas of Responsibility information technology. Operations 40% Finance 22% • Respondents from both federal Technical 15% civilian and defense agencies Other 14% participated in the survey. Human capital 11% Agency leadership 11% Information technology 11% Facilities and fleet management 5%Percentage of respondents, n=209 Legislative 3% Other 14% 6
  7. 7. Research Findings
  8. 8. As Budget Impact of Budget Pressures on Data’s ImportancePressures Rise,Importance of Data SignificantlyIncreases less important 2% Somewhat• An escalating focus on budget in less important federal agencies will only increase 1% the importance of data. More than seventy percent of federal mangers think budget pressures No change and cost constraints will make 26% Significantly data more crucial to their more programs. important 50% Somewhat• While most managers believe more data will increase in importance, important one quarter believe there will be 21% no change to how their agencies regard data. Percentage of respondents, n=288 8
  9. 9. ―The data wontreally be any moreimportant than ―Data is what webefore. Only now, use to form budgetsmanagement may for programs. Andnow be forced to data is what we uselook at the data to analyze futuremore closely and funds shortfalls.‖pay attention.‖ 9
  10. 10. Data Can Be Both Characterization of Agency’s DataConfusing andMeaningful 14%• When asked how they would 5% 19% characterize their agency‘s data, 8% both collected and reported, 19 3% percent of managers describe it as ―confusing,‖ yet an equal percentage describe it as 23% ―meaningful‖. 19%• In open-ended responses, 10% managers volunteer descriptions of their data quality, including ―less than stellar‖ to ―Some functions have ―comprehensive and integral.‖ ―Data is fragmented meaningful measures and and incomplete-- use them appropriately. deep in some areas Others collect data in an and almost unsophisticated manner non-existent in that is fundamentallyPercentage of respondents, n=277 others. ― flawed.‖Graphic Source: 10
  11. 11. Programs Collect Reasons Programs Collect DataData to MeetMandates and Compliance with mandates set by 58%Operate OMB or other external authorities Management of day-to-day 57% operations• Federal managers collect data at the program level for four major purposes: to meet external Strategic planning 55% mandates, to manage operations, to measure a program‘s Measurement of performance 54% performance or to plan. Evaluating programs, or establishing 53%• More than half of federal a need for a future program managers believe data aids their Conducting public outreach or to performance and program 27% fulfill transparency requirements evaluation. A full 55 percent collect data for strategic planning. Slightly more than a quarter (27 I dont know why we collect our data 8% percent), collect data to share with the public. Other 6% Percentage of respondents, n=258 11
  12. 12. Operations Are First Uses of Data in Federal AgenciesData Priority, NotAccuracy Day-to-day operations 62% Performance metrics 54% Set benchmarks and goals 50%• Federal managers collect and use Analyzes data from past and present to data for day-to-day operations forecast 45% and performance management, yet many have expanded their Past data to pinpoint areas of weakness 40% capacity to more advanced forms Evaluates present data, or collects in "real time" 35% of analytics. Verifies data for accuracy 33% Levels of Analysis• Forty-five percent of managers say their agencies are engaging in Assesses accuracy or "data quality" 31% 1st Operations predictive analytics, yet fewer Reports on historical data primarily 30% than one third assess the quality 2nd Review of data and only 33 percent verify Analysis to launch full-scale programs or initiatives 29% their data for accuracy. 3rd Forecast Advanced statistical analysis for program- level reporting or strategy 29% None of the above 6% Other 5% Percentage of respondents, n=231 12
  13. 13. ―Im confident aboutthe data we collect,but once it getsrolled up to theagency level, itsmore or lessmeaningless.You can specify thatyou need x dollarsfor your program -but once its rolledup, that identity andaccuracy is lost.‖ 13
  14. 14. Data Less Accurate Confidence in Data Accuracy at Agency andafter Combined at Program LevelsAgency-level 65%• More managers are confident Program 28% 37% 15% 13% 7% about the accuracy of their data at the program-level than data accuracy agency wide.• A full 65 percent of managers at least somewhat agree with the statement, ―I‘m confident in the accuracy of the data my program collects,‖ as opposed to half who Agency 20% 39% 17% 14% 10% agree on the agency-level. Strongly agree Somewhat agree Somewhat disagree Strongly disagree I dont know/would rather not say Percentage of respondents, n=249, 247 14
  15. 15. “I am able to share information with other agencies and haveData Seldom Shared access to relevant information from them when necessary.”acrossOrganizations Somewhat• Agencies have difficulty assessing Disagree data from other agencies, due to Somewhat 18% either restrictions or lack of sharing mechanisms. Agree 34%• When offered the statement, ―I am Strongly able to share information with Strongly other agencies and have access Agree Disagree to relevant information from other 7% 32% agencies when necessary,‖ fifty I don‘t percent of managers ―somewhat‖ know or ―strongly disagree.‖ ―Data is not made available 9% to the rest of the organization, so data gathering is performed by each and every seeker.‖ Percentage of respondents, n=209 15
  16. 16. Agencies Struggle Challenges in Reporting Datato Make DataActionable External Reporting (i.e., OMB, GAO) Internal Reporting (i.e., measuring performance or program evaluation)• Seeing the ―bigger picture‖ behind the data is a challenge when it comes to reporting for both 58% 57% external and internal purposes. 49% 45% 47%• When asked which challenges managers found in their agency‘s 33% reporting process, ―coming to meaningful conclusions or actionable measures from reports‖ is a struggle for managers, in both external (58 percent) and internal (49 percent) reporting. Coming to meaningful Integrating information Crafting high-level conclusions or actionable from multiple sources into analysis from reports measures from reports one report Percentage of respondents, n=210 16
  17. 17. ―We have few useful ―It is frustrating toreporting ―We find have an OMBmechanisms in ourselves analyst sending outplace--cannot doing the data calls for‗burden‘ the field same work information theywith reporting twice.‖ wont understand orusefully detailed have any idea whatdata--and spend they intend to use itmost of our analysis for.‖time verifying oftenmisleading data.‖
  18. 18. Agencies Struggle Challenges in Reporting Datato Make DataAccessible External Reporting (i.e., OMB, GAO) Internal Reporting (i.e., measuring performance or program evaluation)• When asked which challenges managers found in their agency‘s 61% 61% reporting process to external 46% 49% 40% 46% parties such as the OMB or GAO, 37% more than 60 percent have 31% trouble crafting easy to interpret and widely-accessible reports.• More than sixty percent of Regularly Making reports Gaining Customizing managers struggle to regularly conducting reports easy to interpret stakeholder reports to fit the report and deliver to stay in and delivering them and widely- agreement on different compliance with requests from on time accessible reports requirements of OMB, GAO and other external conclusions and different audiences organizations. implications Percentage of respondents, n=210 18
  19. 19. Restrictions and Challenges Preventing Data CollectionInsufficient SystemsPrevent Data Complicated processes or restrictions 39%Collection Insufficient technical and system resources 38% Lack of employees with relevant skills• The challenges that dampen a and training 37% program‘s ability to collect data Budget cuts or insufficient funding for are procedural, technical, and this specific function 33% budgetary. Agencies also lack employees with relevant skills and Not enough time 29% training to collect good data. Difficulty interpreting or find the "meaning" behind data 28%• One third of federal managers Lack of dedicated data analysts 27% believe budget cuts have limited their ability to collect data, yet 39 Not a leadership priority 26% percent identify complicated processes or restrictions. Colleagues are reluctant to share data 24% None of the above 8% Other 4% Percentage of respondents, n=219 19
  20. 20. ―With a staffreduction of two-thirds over the pasttwo years, I nolonger have enoughtrained personnel toaddress mandatory,non-core tasks andcollect and analyzedata in meaningfulways.‖ 20
  21. 21. Lack of Skills and Challenges Preventing Use of Data toStaff Prevent Data- Guide Decision-makingbased Decisions Lack of employees with relevant skills and training 40% Not a leadership priority 34%• Federal managers experience a Lack of dedicated data analysts 33% variety of challenges when Difficulty interpreting or finding the "meaning" behind data 32% attempting to use data for decision-making, yet a need for Colleagues are reluctant to act on information 30% training and dedicated data analysts is common among more Insufficient technical and system resources 29% than one third of managers. Complicated processes and restrictions 28%• Leadership challenges rank as Not enough time 26% central concerns. A full 34 percent Colleagues are reluctant to take on the of federal managers report responsibility of analyzing data 24% leaders do not make data-based Budget cuts or insufficient funding for this specific 21% decision-making a priority. function Colleagues are reluctant to share data 20% None of the above 11% Other 4% Percentage of respondents, n=219 21
  22. 22. Agencies Training Agency Efforts to Strengthen Data CapacityExisting Workforceto StrengthenCapacity Training of current workforce• Many federal managers—42 42% None of the percent—say their agencies or above programs are training current staff 36% to strengthen analytic and evaluation capacity. Contracting• Engaging external resources and skilled workers recruiting is less common for 19% Internship federal managers, and only five programs Tuition percent are instituting interagency 16% reimbursement, transfers to augment their data scholarshipsUniversity 11% recruitment analytics capacity. 10% Interagency Other transfers 6% 5% Percentage of respondents, n=201 22
  23. 23. Focus Group Findings Confirmed Federal managers struggle to both Federal executives Agencies have verify the accuracy strive to share trouble reporting of their data and to information with their data across select the correct colleagues in other multiple outlets, in type of data to fit agencies, but addition to their goals. encounter both tailoring the same Agency leaders cultural and data to fit the are unaware of administrative demands of what kind of data challenges that multiple would be prevent them from audiences. ―meaningful‖ in sharing their data each circumstance or establishing they may standard practices encounter and are for collaboration. unable to plan for each contingency.
  24. 24. Agencies Choose Agency Initiatives to Implement Cost-SavingsDownsizing overData-driven Cost Downsize workforce (federal employees) 56%Cuts Reduce costs or manage expenses 43%• The consequence of data challenges, cost cuts in agencies Consolidate or cut programs 38% may not be strategic. More than half (56 percent) of federal Redesign programs 28% managers say their agency or program will downsize the federal Offer telework or alternative work 27% employee workforce in order to strategies achieve cost-savings. Consolidate business systems 26%• Significantly fewer (23 percent) will look to review the outcomes of Review program outcomes 23% programs, reduce improper payments (22 percent) or Other 7% consolidate their business systems (26 percent) in order None of the above 7% save costs. Percentage of respondents, n=298 24
  25. 25. ContactErin Dian DumbacherAssociate Director of ResearchGovernment Business CouncilTel 202.266.7619edumbacher@govexec.com 25
  26. 26. Sponsored by: ―Data analytics can help agencies get more value out of their existing data and could reveal under-performing programs or fraud, waste, and abuse,‖ notes Brad Eskind, Principal, Deloitte Consulting LLP and Federal Technology and Analytics Leader. ―Analyzing data can help agencies uncover these kinds of insights and realize important savings.‖ Applying data analytics can also help agencies possibly seeLearn more about different outcomes by simulating conditions, constraints, andDeloitte‘s Federal practice at timeframes. These types of scenarios could help assess trade-offs based on different decisions and results. ―Preemptive steps like this can help agencies allocate their limited resources more effectively, make more strategic cuts, and emerge as leaner and more effective organizations,‖ says Eskind. 26
  27. 27. About DeloitteFederal agencies turn to Deloitte for their most meaningful and challenging problems--fromcybersecurity and sustainability to cost reduction and reform. With a mix of public sector experienceand private sector perspective, Deloitte‘s diverse capabilities across consulting, audit and enterpriserisk, financial advisory, and tax services help clients address issues from many dimensions. See howDeloitte works with clients to deliver results today and build a stronger foundation for tomorrow, andhelp government help America. Visit Engage with Deloitte: Find us on Follow us @DeloitteGov 27