1
!Ira A. (Gus) HuntChief Technology OfficerBeyond Big DataRiding theTechnology Wave
Our MissionWe are the nations first line of defense. We accomplishwhat others cannot accomplish and go where otherscannot ...
2	  3	  4 Big Bets–  Acquire, federate, secure and exploit. Grow the haystack, magnify the needles.Revolutionize Big Data ...
2	  3	  6 Key Technology Enablers–  World-class abilities to discover patterns, correlate information, understand plans an...
6It’s aBig DataWorld
Google> 100 PB> 1T indexed URLs> 3 million servers> 7.2B page-views/day7
8FaceBook> 1 billion users> 300PB; +> 500TB/day> 35% of world’s photographs
9YouTube> 1000PB+>72 hours/minute>37 million hours/year> 4 billion views/day
10World Population> 7,057,065,162
11Twitter> 124B tweets/year> 390M/day~4500/sec
12Global Text Messages> 6.1T per year> 193,000 per second> 876 per person per year
13US Cell Calls> 2.2 T minutes/year> 19 minutes / person / day(uncompressed < 1 YouTube/year)
143Driving Forces
Social15MobileCloud
16Big Data+ + =
17+ +Increases the velocity ofinnovation
18Accelerates socialChange+ +
19
20Altered theFlowof Information+ +
3EmergingForces
Nano22BioSensors
23MicrophoneImage3-axis accelerometerTouchLightProximityGeolocationMobile Sensor PlatformCommunicator, Tricorder, Transpor...
24PacemakerBlood sugar testerInsulin controllerHealth monitorExercise coachRemote tune-upsEarly warning systemMobile Healt...
25Identity by 3-axis accelerometerGender (71%)Height--tall or short (80%)Weight--heavy or light (80%)You by your gait (100...
26The inanimate becomessentient+ ++ ++=
27Smarter PlanetCars drive themselvesMachines know your needs+ ++ ++=
28Drive radical efficienciesEnhance social engagementImprove information sharingEnables global reachGreen (automatic routi...
2	  3	  Sensors are Really BigSensors are unbounded1	  Sensors are indiscriminateSensors are promiscuous
2	  3	  The Internet of Things is BiggerEverything is Connected1	  Everything is a SensorEverything Communicates
31That’s theReally Big DataChallenge of the future
32WhyWeCare
33WhyWeCare
34WhyWeCare
35WhyWeCare
2	  3	  Impact of Big DataKnow what we knowDiscover the gaps in our knowledgeMore effective use of expensive or long leadc...
37Implications
2	  3	  4 Rules of Big DataIt’s the data…Power to the peopleContext, context, context1	  4	  Latency breeds contempt- Apol...
39It’s the Data…
Data vs Tools—A HistoryLesson•  Sophisticated tools without the data areuseless•  Mediocre tools with the data are frustra...
2	  3	  Our JobLeverage the Big Data worldFind the Information that MattersConnect the DotsUnderstand the Plans of our Adv...
TheProblem42
2	  3	  Our Problem: Which 5KDon’t know the future value of dataWe cannot connect dots we don’t haveTraditional, requireme...
2	  3	  Characteristics of Big DataMore is always betterSignal to noise only gets worseRequirements are usually hindsight1...
45•  Analysts and operators are not data engineers•  Need insight and understanding•  Ask a question and get a coherent an...
46Power to thePeople
47•  Analytics and tools are hard to use•  Specialists are required to derive value•  Skilled people are in short supply• ...
48New Fields ofExpertiseData ScientistInformation Engineer
Data ScienceData science combines elements from manyfields:MathStatisticsData EngineeringPattern Recognition and LearningA...
50The power of big datacan only be fully realizedwhen it is in the handsof the average userBig Data Democracy Wins
Tomorrow•  Elegant, powerful and easy to use tools andvisualizations•  Machines to do more of the heavy lifting•  Intellig...
52PeoplePlacesOrganizationsTimeEventsConceptsThings7 Universal Constructs forAnalytics
53User Built Recipes
Keep it Simple•  Data Scientists focus on hard problems•  Build reusable components that anyonecan apply—Recipes•  Share t...
55Latency BreedsContempt
Its All About Speed•  Hadoop/Map Reduce—batch•  Flexible, powerful, slow•  Equivalent of Real-Time Map/Reduce•  Flexible, ...
Tectonic Technology ShiftsTraditional ProcessingData on SANMove Data to QuestionBackupVertical scalingCapacity after deman...
New Computing Architectures•  Data close to compute•  Power at the edge•  Optical Computing/Optical Bus•  End of the mothe...
59Context,Context,Context
Everything in Your Frame ofReference•  Widgets—Webtop in context to business•  Schema on Read—Data in context to yourquest...
61ClosingThoughts
62High Noonin theInformation Age
63It is nearly within our graspto compute on all humangenerated information
64FaceBook> 1 billion users> 35% of all photographs
65The inanimate is rapidlybecoming sentientSmarter PlanetCars drive themselvesMachines know your needs
663rd Wave ofComputingCognitive MachinesWatson
67Moving faster than government can keep upThe legal system is woefully behindWhat are your rights? Who owns your data?Dri...
68
Upcoming SlideShare
Loading in...5
×

Central Intelligence Agency - GigaOM 2013

484

Published on

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
484
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Central Intelligence Agency - GigaOM 2013"

  1. 1. 1
  2. 2. !Ira A. (Gus) HuntChief Technology OfficerBeyond Big DataRiding theTechnology Wave
  3. 3. Our MissionWe are the nations first line of defense. We accomplishwhat others cannot accomplish and go where otherscannot go. We carry out our mission by:Collecting information that reveals the plans, intentions andcapabilities of our adversaries and provides the basis fordecision and action.Producing timely analysis that provides insight, warning andopportunity to the President and decisionmakers charged withprotecting and advancing Americas interests.Conducting covert action at the direction of the President topreempt threats or achieve US policy objectives.
  4. 4. 2  3  4 Big Bets–  Acquire, federate, secure and exploit. Grow the haystack, magnify the needles.Revolutionize Big Data ExploitationAccelerate Operational ExcellenceServe CIA by supporting the ICDrive Performance through Talent Management–  Assume a leadership role in IC activities that matter to CIA; Build to share–  Innovate IT operations and run IT like a business.–  Focus on continuous learning and diversity of thought, experience, background1  4  
  5. 5. 2  3  6 Key Technology Enablers–  World-class abilities to discover patterns, correlate information, understand plans and intentions,and find and identify operational targets in a sea of data. Big Data analytics as a serviceAdvanced Mission Analytics—Analytics as a ServiceEnterprise Widgets and ServicesSecurity as a ServiceData Harbor—Data as a Service–  One environment, all data, protected and secure.--ubiquitous encryption, enterpriseauthentication, audit, DRM, secure ID propagation, and Gold Version C&A.–  A customizable, integrated and adaptive webtop that lets analysts, ops officers, andtargeters to “have it their way”. Personalization in context.–  An ultra-high performance data environment that enables CIA missions to acquire,federate, and position and securely exploit huge volumes data. Data in context.1  4  Cloud Computing—Infrastructure as a Service–  Capacity ahead of demand. Large scale, elastic, commodity hosting, storage, and compute5  –  Immediate, secure and appropriate access to people, data and tools from anywhere at anytimeSecure Mobility0  
  6. 6. 6It’s aBig DataWorld
  7. 7. Google> 100 PB> 1T indexed URLs> 3 million servers> 7.2B page-views/day7
  8. 8. 8FaceBook> 1 billion users> 300PB; +> 500TB/day> 35% of world’s photographs
  9. 9. 9YouTube> 1000PB+>72 hours/minute>37 million hours/year> 4 billion views/day
  10. 10. 10World Population> 7,057,065,162
  11. 11. 11Twitter> 124B tweets/year> 390M/day~4500/sec
  12. 12. 12Global Text Messages> 6.1T per year> 193,000 per second> 876 per person per year
  13. 13. 13US Cell Calls> 2.2 T minutes/year> 19 minutes / person / day(uncompressed < 1 YouTube/year)
  14. 14. 143Driving Forces
  15. 15. Social15MobileCloud
  16. 16. 16Big Data+ + =
  17. 17. 17+ +Increases the velocity ofinnovation
  18. 18. 18Accelerates socialChange+ +
  19. 19. 19
  20. 20. 20Altered theFlowof Information+ +
  21. 21. 3EmergingForces
  22. 22. Nano22BioSensors
  23. 23. 23MicrophoneImage3-axis accelerometerTouchLightProximityGeolocationMobile Sensor PlatformCommunicator, Tricorder, Transporter
  24. 24. 24PacemakerBlood sugar testerInsulin controllerHealth monitorExercise coachRemote tune-upsEarly warning systemMobile Health Platform
  25. 25. 25Identity by 3-axis accelerometerGender (71%)Height--tall or short (80%)Weight--heavy or light (80%)You by your gait (100%)Mobile Sensor PlatformActitracker—Android App
  26. 26. 26The inanimate becomessentient+ ++ ++=
  27. 27. 27Smarter PlanetCars drive themselvesMachines know your needs+ ++ ++=
  28. 28. 28Drive radical efficienciesEnhance social engagementImprove information sharingEnables global reachGreen (automatic routing)Improve our healthStop/prevent crime…+ ++ ++=
  29. 29. 2  3  Sensors are Really BigSensors are unbounded1  Sensors are indiscriminateSensors are promiscuous
  30. 30. 2  3  The Internet of Things is BiggerEverything is Connected1  Everything is a SensorEverything Communicates
  31. 31. 31That’s theReally Big DataChallenge of the future
  32. 32. 32WhyWeCare
  33. 33. 33WhyWeCare
  34. 34. 34WhyWeCare
  35. 35. 35WhyWeCare
  36. 36. 2  3  Impact of Big DataKnow what we knowDiscover the gaps in our knowledgeMore effective use of expensive or long leadcollection assets1  4  Focus targeting to fill the gapsBetter global coverage to limit surprise5  Enhance understanding and improve analysis6  
  37. 37. 37Implications
  38. 38. 2  3  4 Rules of Big DataIt’s the data…Power to the peopleContext, context, context1  4  Latency breeds contempt- Apologies to James Carville- Apologies to the Black Panthers- Apologies to Aesop- Apologies to Lord Harold Samuel
  39. 39. 39It’s the Data…
  40. 40. Data vs Tools—A HistoryLesson•  Sophisticated tools without the data areuseless•  Mediocre tools with the data are frustrating•  Analysts will always opt for frustration overfutility, if that is their only option
  41. 41. 2  3  Our JobLeverage the Big Data worldFind the Information that MattersConnect the DotsUnderstand the Plans of our AdversariesSafeguard our national security1  4  
  42. 42. TheProblem42
  43. 43. 2  3  Our Problem: Which 5KDon’t know the future value of dataWe cannot connect dots we don’t haveTraditional, requirements driven, collectionfails in the Big Data world- Can’t task for data you don’t know you do need- The few cannot know the needs of the many- Global Coverage requires Global Data1  
  44. 44. 2  3  Characteristics of Big DataMore is always betterSignal to noise only gets worseRequirements are usually hindsight1  4  Enumeration not modeling
  45. 45. 45•  Analysts and operators are not data engineers•  Need insight and understanding•  Ask a question and get a coherent answer•  Cannot know what data sets containinformation of value to them•  Imbue data services and tools with thosesmarts•  Smart Data, smart tools, smarter intelligenceData as a Service
  46. 46. 46Power to thePeople
  47. 47. 47•  Analytics and tools are hard to use•  Specialists are required to derive value•  Skilled people are in short supply•  Algorithms are dense and arcane•  Require a lot of hand curation•  Built for business not for intelligenceToday
  48. 48. 48New Fields ofExpertiseData ScientistInformation Engineer
  49. 49. Data ScienceData science combines elements from manyfields:MathStatisticsData EngineeringPattern Recognition and LearningAdvanced ComputingVisualizationUncertainty ModelingData WarehousingHigh performance computing* Wikipedia*
  50. 50. 50The power of big datacan only be fully realizedwhen it is in the handsof the average userBig Data Democracy Wins
  51. 51. Tomorrow•  Elegant, powerful and easy to use tools andvisualizations•  Machines to do more of the heavy lifting•  Intelligent systems that learn from the user•  Correlation not search•  “Curiosity layer”– machines that are curious onyour behalf
  52. 52. 52PeoplePlacesOrganizationsTimeEventsConceptsThings7 Universal Constructs forAnalytics
  53. 53. 53User Built Recipes
  54. 54. Keep it Simple•  Data Scientists focus on hard problems•  Build reusable components that anyonecan apply—Recipes•  Share them widely—Apps Store/Apps Mall—Recipe Book•  Let users assemble components their way•  Experiment and fail quickly to succeed faster
  55. 55. 55Latency BreedsContempt
  56. 56. Its All About Speed•  Hadoop/Map Reduce—batch•  Flexible, powerful, slow•  Equivalent of Real-Time Map/Reduce•  Flexible, powerful and fast•  Demel, Caffeine, Impala, Apache Drill, Spanner…•  Recursive Streams processing w/complex analytics•  In-memory—peta-scale RAM architectures•  Distributed, in-memory analytics
  57. 57. Tectonic Technology ShiftsTraditional ProcessingData on SANMove Data to QuestionBackupVertical scalingCapacity after demandDRSize to peak loadTapeSANDiskRAM limitedMass Analytics/Big DataData at processorMove Question to DataReplication managementHorizontal scalingCapacity ahead of demandCOOPDynamic/elastic provisioningSANDiskSSDPeta-scale RAM
  58. 58. New Computing Architectures•  Data close to compute•  Power at the edge•  Optical Computing/Optical Bus•  End of the motherboard—shared pools ofeverything•  Software defined everything—compute,storage, networking, data center•  Network is the bottleneck and constraint
  59. 59. 59Context,Context,Context
  60. 60. Everything in Your Frame ofReference•  Widgets—Webtop in context to business•  Schema on Read—Data in context to yourquestion•  User assembled analytics—answers incontext to your questions•  Elastic computing—computing in contextto your demand
  61. 61. 61ClosingThoughts
  62. 62. 62High Noonin theInformation Age
  63. 63. 63It is nearly within our graspto compute on all humangenerated information
  64. 64. 64FaceBook> 1 billion users> 35% of all photographs
  65. 65. 65The inanimate is rapidlybecoming sentientSmarter PlanetCars drive themselvesMachines know your needs
  66. 66. 663rd Wave ofComputingCognitive MachinesWatson
  67. 67. 67Moving faster than government can keep upThe legal system is woefully behindWhat are your rights? Who owns your data?Driving the pace of social changeExponentially increasing cyber threats+ ++ ++=
  68. 68. 68

×