• Save
Central Intelligence Agency - GigaOM 2013
Upcoming SlideShare
Loading in...5
×
 

Central Intelligence Agency - GigaOM 2013

on

  • 662 views

 

Statistics

Views

Total Views
662
Views on SlideShare
662
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Central Intelligence Agency - GigaOM 2013 Central Intelligence Agency - GigaOM 2013 Presentation Transcript

    • 1
    • !Ira A. (Gus) HuntChief Technology OfficerBeyond Big DataRiding theTechnology Wave
    • Our MissionWe are the nations first line of defense. We accomplishwhat others cannot accomplish and go where otherscannot go. We carry out our mission by:Collecting information that reveals the plans, intentions andcapabilities of our adversaries and provides the basis fordecision and action.Producing timely analysis that provides insight, warning andopportunity to the President and decisionmakers charged withprotecting and advancing Americas interests.Conducting covert action at the direction of the President topreempt threats or achieve US policy objectives.
    • 2  3  4 Big Bets–  Acquire, federate, secure and exploit. Grow the haystack, magnify the needles.Revolutionize Big Data ExploitationAccelerate Operational ExcellenceServe CIA by supporting the ICDrive Performance through Talent Management–  Assume a leadership role in IC activities that matter to CIA; Build to share–  Innovate IT operations and run IT like a business.–  Focus on continuous learning and diversity of thought, experience, background1  4  
    • 2  3  6 Key Technology Enablers–  World-class abilities to discover patterns, correlate information, understand plans and intentions,and find and identify operational targets in a sea of data. Big Data analytics as a serviceAdvanced Mission Analytics—Analytics as a ServiceEnterprise Widgets and ServicesSecurity as a ServiceData Harbor—Data as a Service–  One environment, all data, protected and secure.--ubiquitous encryption, enterpriseauthentication, audit, DRM, secure ID propagation, and Gold Version C&A.–  A customizable, integrated and adaptive webtop that lets analysts, ops officers, andtargeters to “have it their way”. Personalization in context.–  An ultra-high performance data environment that enables CIA missions to acquire,federate, and position and securely exploit huge volumes data. Data in context.1  4  Cloud Computing—Infrastructure as a Service–  Capacity ahead of demand. Large scale, elastic, commodity hosting, storage, and compute5  –  Immediate, secure and appropriate access to people, data and tools from anywhere at anytimeSecure Mobility0  
    • 6It’s aBig DataWorld
    • Google> 100 PB> 1T indexed URLs> 3 million servers> 7.2B page-views/day7
    • 8FaceBook> 1 billion users> 300PB; +> 500TB/day> 35% of world’s photographs
    • 9YouTube> 1000PB+>72 hours/minute>37 million hours/year> 4 billion views/day
    • 10World Population> 7,057,065,162
    • 11Twitter> 124B tweets/year> 390M/day~4500/sec
    • 12Global Text Messages> 6.1T per year> 193,000 per second> 876 per person per year
    • 13US Cell Calls> 2.2 T minutes/year> 19 minutes / person / day(uncompressed < 1 YouTube/year)
    • 143Driving Forces
    • Social15MobileCloud
    • 16Big Data+ + =
    • 17+ +Increases the velocity ofinnovation
    • 18Accelerates socialChange+ +
    • 19
    • 20Altered theFlowof Information+ +
    • 3EmergingForces
    • Nano22BioSensors
    • 23MicrophoneImage3-axis accelerometerTouchLightProximityGeolocationMobile Sensor PlatformCommunicator, Tricorder, Transporter
    • 24PacemakerBlood sugar testerInsulin controllerHealth monitorExercise coachRemote tune-upsEarly warning systemMobile Health Platform
    • 25Identity by 3-axis accelerometerGender (71%)Height--tall or short (80%)Weight--heavy or light (80%)You by your gait (100%)Mobile Sensor PlatformActitracker—Android App
    • 26The inanimate becomessentient+ ++ ++=
    • 27Smarter PlanetCars drive themselvesMachines know your needs+ ++ ++=
    • 28Drive radical efficienciesEnhance social engagementImprove information sharingEnables global reachGreen (automatic routing)Improve our healthStop/prevent crime…+ ++ ++=
    • 2  3  Sensors are Really BigSensors are unbounded1  Sensors are indiscriminateSensors are promiscuous
    • 2  3  The Internet of Things is BiggerEverything is Connected1  Everything is a SensorEverything Communicates
    • 31That’s theReally Big DataChallenge of the future
    • 32WhyWeCare
    • 33WhyWeCare
    • 34WhyWeCare
    • 35WhyWeCare
    • 2  3  Impact of Big DataKnow what we knowDiscover the gaps in our knowledgeMore effective use of expensive or long leadcollection assets1  4  Focus targeting to fill the gapsBetter global coverage to limit surprise5  Enhance understanding and improve analysis6  
    • 37Implications
    • 2  3  4 Rules of Big DataIt’s the data…Power to the peopleContext, context, context1  4  Latency breeds contempt- Apologies to James Carville- Apologies to the Black Panthers- Apologies to Aesop- Apologies to Lord Harold Samuel
    • 39It’s the Data…
    • Data vs Tools—A HistoryLesson•  Sophisticated tools without the data areuseless•  Mediocre tools with the data are frustrating•  Analysts will always opt for frustration overfutility, if that is their only option
    • 2  3  Our JobLeverage the Big Data worldFind the Information that MattersConnect the DotsUnderstand the Plans of our AdversariesSafeguard our national security1  4  
    • TheProblem42
    • 2  3  Our Problem: Which 5KDon’t know the future value of dataWe cannot connect dots we don’t haveTraditional, requirements driven, collectionfails in the Big Data world- Can’t task for data you don’t know you do need- The few cannot know the needs of the many- Global Coverage requires Global Data1  
    • 2  3  Characteristics of Big DataMore is always betterSignal to noise only gets worseRequirements are usually hindsight1  4  Enumeration not modeling
    • 45•  Analysts and operators are not data engineers•  Need insight and understanding•  Ask a question and get a coherent answer•  Cannot know what data sets containinformation of value to them•  Imbue data services and tools with thosesmarts•  Smart Data, smart tools, smarter intelligenceData as a Service
    • 46Power to thePeople
    • 47•  Analytics and tools are hard to use•  Specialists are required to derive value•  Skilled people are in short supply•  Algorithms are dense and arcane•  Require a lot of hand curation•  Built for business not for intelligenceToday
    • 48New Fields ofExpertiseData ScientistInformation Engineer
    • Data ScienceData science combines elements from manyfields:MathStatisticsData EngineeringPattern Recognition and LearningAdvanced ComputingVisualizationUncertainty ModelingData WarehousingHigh performance computing* Wikipedia*
    • 50The power of big datacan only be fully realizedwhen it is in the handsof the average userBig Data Democracy Wins
    • Tomorrow•  Elegant, powerful and easy to use tools andvisualizations•  Machines to do more of the heavy lifting•  Intelligent systems that learn from the user•  Correlation not search•  “Curiosity layer”– machines that are curious onyour behalf
    • 52PeoplePlacesOrganizationsTimeEventsConceptsThings7 Universal Constructs forAnalytics
    • 53User Built Recipes
    • Keep it Simple•  Data Scientists focus on hard problems•  Build reusable components that anyonecan apply—Recipes•  Share them widely—Apps Store/Apps Mall—Recipe Book•  Let users assemble components their way•  Experiment and fail quickly to succeed faster
    • 55Latency BreedsContempt
    • Its All About Speed•  Hadoop/Map Reduce—batch•  Flexible, powerful, slow•  Equivalent of Real-Time Map/Reduce•  Flexible, powerful and fast•  Demel, Caffeine, Impala, Apache Drill, Spanner…•  Recursive Streams processing w/complex analytics•  In-memory—peta-scale RAM architectures•  Distributed, in-memory analytics
    • Tectonic Technology ShiftsTraditional ProcessingData on SANMove Data to QuestionBackupVertical scalingCapacity after demandDRSize to peak loadTapeSANDiskRAM limitedMass Analytics/Big DataData at processorMove Question to DataReplication managementHorizontal scalingCapacity ahead of demandCOOPDynamic/elastic provisioningSANDiskSSDPeta-scale RAM
    • New Computing Architectures•  Data close to compute•  Power at the edge•  Optical Computing/Optical Bus•  End of the motherboard—shared pools ofeverything•  Software defined everything—compute,storage, networking, data center•  Network is the bottleneck and constraint
    • 59Context,Context,Context
    • Everything in Your Frame ofReference•  Widgets—Webtop in context to business•  Schema on Read—Data in context to yourquestion•  User assembled analytics—answers incontext to your questions•  Elastic computing—computing in contextto your demand
    • 61ClosingThoughts
    • 62High Noonin theInformation Age
    • 63It is nearly within our graspto compute on all humangenerated information
    • 64FaceBook> 1 billion users> 35% of all photographs
    • 65The inanimate is rapidlybecoming sentientSmarter PlanetCars drive themselvesMachines know your needs
    • 663rd Wave ofComputingCognitive MachinesWatson
    • 67Moving faster than government can keep upThe legal system is woefully behindWhat are your rights? Who owns your data?Driving the pace of social changeExponentially increasing cyber threats+ ++ ++=
    • 68