Examining the Big Data Frontier


Published on

Brief report on the Examining the Big Data Frontier GovLoop Guide. Best Practices, Common Challenges and much more!

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Examining the Big Data Frontier

  1. 1. March 26, 2014 #gltrain
  2. 2. Welcome Steve Ressler Founder & President GovLoop - Tweets: #gltrain - Slides, Video archive & other resources will be emailed to you later this week. - Check out the GovLoop VIP programhttp://www.govloop.com/TrainingVIPs - Be sure to fill out the evaluation to obtain your 3 CPE’s at the end of the event.
  3. 3. The Data Revolution “As we sit on the cusp of remarkable innovations, we must remember that this time, modern innovations are powered by data.”
  4. 4. Our Goals - GovLoop Research • Our research: empower and educate • Our basic formula: – Survey of GovLoop Audience – Case study from state, local and federal government – Best practices, challenges – Cheat sheet • Our mission: Help you do your job better
  5. 5. What You’ll Find in the Report • A local government spotlight showing how the city of Louisville, KY., has leveraged data to improve services • A federal government case study highlighting the Army’s Enterprise Management Decision Support program. • Industry insights on the current big data landscape. • 8 Strategies and best practices for smart big data adoption and analysis. • GovLoop’s big data cheat sheet.
  6. 6. GovLoop Survey Data
  7. 7. Defining Big Data “Big Data refers to the massive amounts of data that collect over time that are difficult to analyze and handle using common database management tools.” – PC Magazine The Method for an Integrated Knowledge Environment open-source project. The MIKE project argues that big data is not a function of the size of a data set but its complexity. Consequently, it is the high degree of permutations and interactions within a data set that defines big data. The National Institute of Standards and Technology. NIST argues that big data is data which “exceed(s) the capacity or capability of current or conventional methods and systems.” In other words, the notion of “big” is relative to the current standard of computation. Definitions from: The Big Data Conundrum: How to Define It?
  8. 8. A Working Definition “Big data is a term describing the storage and analysis of large and or complex data sets using a series of techniques including, but not limited to: NoSQL, MapReduce and machine learning.” See full study: Undefined By Data: A Survey of Big Data Definitions - Jonathan Stuart Ward and Adam Barker, School of Computer Science, University of St. Andrews, UK.
  9. 9. Do we care? • Leveraging data in new ways to meet mission need. • Unlocking new insights by synthesizing data across your department • Collaborating and sharing resources
  10. 10. Convergence of Technology You can’t talk about big data without thinking about cloud and mobile.
  11. 11. 3 Big Data Case Studies
  12. 12. Social Welfare • The oldest case is the Famine Early Warning Systems Network (FEWS NET) developed by U.S. Agency for International Development in 1986. • The $25 million dollar program helps optimize the distribution of up to $1.5 billion dollars per year in USAID Food for Peace assistance.
  13. 13. Smarter Healthcare • Predicting the likelihood of hospitalization or death within 90 days, the Patient Care Assessment System (PCAS) calculates the Care Assessment Needs (CAN) Score. • This score allows the Veterans Health Administration (VHA) to focus care teams and proactively care for their patients. • The system collects 120 unique elements for 5.25 million patients and is supported by an 80-terabyte corporate data warehouse.
  14. 14. Biometric Intelligence • Since 2003, US armed forces have collected biometric information from non-US citizens in Iraq and Afghanistan. • It identifies enemy combatants and permits access into controlled areas. • The system, known as The Automated Biometric Identification System (ABIS), stores 4.4 million unique identities and has identified over 3,000 enemy combatants, added 190,000 identities to the watch list, and protected the welfare of the United States and its allies.
  15. 15. Big Data’s Challenges – From the Survey • Data governance • Data locked in legacy systems • Unclear mission and goals • Lack of support • Lack of clarity on metrics
  16. 16. Big Data’s Core Challenge How do we find the needle in the haystack…when the haystack keeps getting exponentially bigger?
  17. 17. Our Big Data Best Practices
  18. 18. 1: Executive Leadership
  19. 19. 2: Business Before Treasure
  20. 20. 3: Know Thyself: Define Use Cases
  21. 21. 4: Leverage Existing Resources & Augment
  22. 22. 5: Integrate Legacy Systems
  23. 23. 6: Partner With Industry
  24. 24. 7: Focus on Governance and Data Quality
  25. 25. 8: Have a Deep Bench - Train
  26. 26. Where to begin? • What problem are we trying to solve? • How do we engage the right people? • How do we break down silos? • What kinds of data do we need? • What do I need to be able to do with the data? • Who needs access and when? • Can I leverage existing technology my agency has? • What does success look like?
  27. 27. Before we wrap up… Today, public service offers a unique opportunity Photo license: by Pa1nt, FlickR Creative Commons