Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Defense Intelligence & The Information Challenge


Published on

TRACK 1 A - The Big Data Imperative for Smarter Defence - Tim Paydos.pdf

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Defense Intelligence & The Information Challenge

  1. 1. Defense Intelligence & The Information Challenge Tim Paydos Director, IBM Threat Prediction & Prevention 9 October 2012 1 © 2009 IBM Corporation
  2. 2. “In our ever-changing world, Americas first line of defense is timely, accurate intelligence that is shared, integrated, analyzed and acted upon quickly and effectively.” - President Barack Obama, 1/7/10 This isn’t an Information Sharing Problem. Its an Information Management Problem. Information Sharing Is only one piece of the larger problem General Michael Hayden, Former Director CIA, Former Director NSA It is not just about sharing information. Its about making sense of it and exploiting it once we have it. There is too much information to identify that which is important. We need tools that tell us what’s important, and what is relevant. Juan Zarate, Deputy National Security Adviser for Combating Terrorism2 © 2009 IBM Corporation © 2009 IBM Corporation
  3. 3. Today’s DiscussionIf you only remember four things Intensifying threat and policy pressures have pushed agencies to a 1 tipping point – they know they need to transform Agency leaders embrace this, are defining the new requirements, 2 and are demonstrating success Achieving transformation requires a broad set of capabilities, and a 3 combination of technology & expertise The experience and assets exist to help you define a strategy and 4 a roadmap to guide your transformation3 © 2009 IBM Corporation FOR INTERNAL IBM USE ONLY
  4. 4. Agency Leaders Telling Us What They NeedTrusted Information -- On Multiple Levels Insightful Comprehensive analytics to drive new meaning from your data Can I extract new Insight to make it all worthwhile? Governed & Secure Rules are in place and tools are deployed to limit visibility, secure sensitive information, and protect privacy Can I Trust My Partner and My Own Agency? Accurate & Complete Complex and disparate data transformed, cleansed, reconcile and delivered Can I Trust The Information? © 2009 IBM Corporation
  5. 5. Military & Intelligence Agency Challenges Leaders Often Stuck Between A Rock And A Hard Place Today’s intensifying challenges mandate a fresh approach to Traditional Approaches managing threat information have become obsoleteMultiplication of threat types, and frequency Information is compartmentalized – lack of fullwith which they occur integration obscures visibilityThreats are increasingly asymmetrical Query State limits ability to address complexity of threatsExplosion in complexity of threat identification Inability to manage and search across an expandingMulti cultural nature of citizenry array of unstructured data sourcesFrequency of transaction/interactions Inability to link unstructured content with structured data and manage togetherSocial Media and “Big Data” playing anincreasingly important role Untimely – Sense & Respond vs. Predict & ActTransparency is clouding Out of context – lack of visual analysis, collaboration, and support/guidance once threat isCitizen, Policy Maker and Regulatory identifiedexpectations and pressures are increasing 5 © 2009 IBM Corporation
  6. 6. The Big Data Challenge Is Only Making It HarderBig Data Is An Explosion In Volume, Variety & VelocityIn 2008, we created 40 Exabytes – more than in all recorded history . Now we we create2.5 Quintillion bytes, or 2.5 Exabytes of data each day — so much that 90% of the data inthe world today has been created in the last two years alone. • For Every 5 Minutes In Realtime, 60 Hours Of Video Is Up-Loaded to You Tube • 12 terabytes of Tweets are created each day, providing insight into public sentimentVolume of Digital Data • 80% of all the data created daily is unstructured – videos, images, emails, and social media • Structured data now includes a massive range of sensors, click streams, log files, call records, transactionsVariety of Information • 5 billion financial transactions occur every single day • There are 30,000 commercial air flights, accounting for 1,500,000 air passengers – every single dayVelocity of Decision Making © 2009 IBM Corporation
  7. 7. Smarter Agencies Leading The Way Optimize decision making with actionable insights Aware Obstructed view Aligned Variety Fragmented Inability to predict of insight Lack VolumeInefficient access Volume VelocityVelocity Rigid Agile Variety Inability to predict Volume Velocity Inefficient access Skilled analytics experts Everyone Lack of insight Variety Velocity Distracted Focused Reactive Proactive © 2009 IBM Corporation 7
  8. 8. Leading the Way Government Leaders Moving to Address These ChallengesStreamlined Information Sharing Western National 8 Law Enforcement Registry of Identities, Objects and EventsAcross Fed and Local Agencies Automated content extraction, entity Western National resolution and analysis from seizedChild Predator Investigation Law Enforcement assets US Department of Connect the dots, predict and preventProtecting the homeland Homeland Security threats US Department of Perpetual credentialing and vettingInsider Threat Defense across branches and bases Centralized Proactive enterprise data activity Screening Database monitoring & extrusion preventionInsider Threat © 2009 IBM Corporation
  9. 9. Leading the Way Government Leaders Moving to Address These Challenges State of The Art covert surveillance systemNational Border & Security based on Streams platform Realtime Information Sharing, Discovery &Crime Prediction & Prevention analysisNational Borders & Security: Real time network intrusion detection, sub-Cyber Security millisecond analysis and responseNational Borders & Security: Real time threat detection & prevention through Big Data & WatsonThreat Prediction & Prevention © 2009 IBM Corporation
  10. 10. Leading the WayMaritime Threat Intelligence10 © 2009 IBM Corporation
  11. 11. Leading the Way Who is Talking to WhomStream A Denoising & Social Conversation Pairing Speaker Detection Network Analysis A B Olivier Mihalis talks to talks to C D Ching-Yung UpendraStream B talks to talks toStream C E Deepak After denoising - Just-in-time - Just-in-time - Social networkStream D - Features: Volumetrics - Features: GSM domain - Fusion technique - Very high accuracy - High accuracy - Iterative method - Very low complexity - Moderate complexity - Robust to noise - Robust to noise © 2009 IBM Corporation
  12. 12. Leading the Way Department of Immigration and Citizenship (Australia)Challenges DIAC, Australia has embarked on a major transformation program to fix key identity issues : Siloed databases, little connectivity between sources Gaps and vulnerabilities in their existing implementation Inconsistent user, citizen, and immigrant identity verification techniques Receive 4 million Visa applications annuallySolution Launched a four year program to transform existing systems using a Service Oriented Architecture (SOA) and implementing a client-centric organization by providing: Roles-based portals to case management personnel Single view of a person’ across all client data sources (over 150 million ‘person’ records) SOA transformation and integration of legacy environmentBusiness Benefits United previously fragmented person’ data and improved processing time 20% Improved ‘person’ identity resolution resulted in significant reduction in false negatives and inappropriate actions being taken Dramatic cost savings from streamlining associated business processes 12 © 2009 IBM Corporation
  13. 13. What We Have LearnedBig Data Requires A Different Approach – It Breaks The Traditional Analytics Model Traditional Approach Big Data Approach Business Users IT Determine what question Delivers a platform to enable to ask creative discovery IT Structures the data to Business answer that question Explores what questions could be asked Structured & Repeatable Analytics Iterative & Exploratory Analytics •Query Based -- Questions Drive Data •Autonomic -- Insight Drives Answers •Citizen Surveys – Push VS. •Citizen Sentiment – Pull •Monthly, Weekly, Daily •Persistent & Ad Hoc •Data At Rest •Data In Motion © 2009 IBM Corporation
  14. 14. What We Have LearnedA New Approach Is Required – Smarter Analytics Supporting Big DataAdvanced Analytic Applications – Integrate and manage the full breadth of data types, and at full volume scale – Gain Insight from Data In Motion, Versus Data At Rest – Apply advanced analytics to information in its native form – Visualize all available data for ad-hoc analysis – Development environment for building newBig Data Platform analytic applicationsProcess and analyze any type of data – Workload optimization and scheduling – Security and Governance Accelerators © 2009 IBM Corporation
  15. 15. What We Have LearnedA Complete Set of Capabilities Is Required To Address The Challenge © 2009 IBM Corporation
  16. 16. What’s Required To Bridge The Gap Establish, Govern, Manage & Deliver Information That You Can Trust IBM InfoSphere Information Server Unified Deployment Understand Cleanse Transform Deliver Discover, model, and Standardize, merge, Combine and Synchronize, virtualize govern information and correct information restructure information and move information structure and content for new uses Unified Metadata Management Parallel Processing Rich Connectivity to Applications, Data, and Content © 2009 IBM Corporation 16
  17. 17. What’s Required To Bridge The Gap And Monitor Continuously & In Real Time Traditional Approach: Native Logging• Lives within the silo – limits visibility• Ex Post Facto, insecure, costly, inaccurate• Can’t identify who and when data is accessed• Does not block unauthorized data-extrusion• Severe negative impact on database performance 4th Generation Enterprise Activity Monitoring Discovers and classifies sensitive data Lock down on who has access to what Identifies the specific person (IP address) who touched the data, how and when – tamper proof audit Provides separation of duties – access to DB functions does not grant the individual the right to see all sensitive information © 2009 IBM Corporation
  18. 18. What’s Required To Bridge The GapPersistent Relationship Awareness & Rules of Visibility Queries & Data Flow Through The Same DATA “Smart” Channel Enterprise DATA Limited DATA Alerts sent to analyst proactively Restricted18 © 2009 IBM Corporation
  19. 19. What’s Required To Bridge The GapPersistent Relationship Awareness – IBM Identity InsightA highly-specialized Identity Analytics repository providing real-time detection of obvious and non-obviousrelationships between people, organizations, events, and other identity types Addresses The Thorny Issues Of: •Realtime Discovery •Enterprise Amnesia - Continuous •Enterprise Brain Hemorrhage - Persistent •Complex and Expanding attribution •Degrees of Separation DATA •Extensibility •Complex event processing Linked to Identities •Information pedigree and audit trail of DATA merges/splits •Anonymization DATA19 © 2009 IBM Corporation
  20. 20. What’s Required To Bridge The GapRules of Visibility – IBM InitiateProvides accurate, real-time access to entity, object and event data acrossdisparate sources, systems and networks Relationship & Hierarchy Management Views Collaborative, Visual Data Stewardship Capabilities Data Security & Privacy Enterprise Search Capabilities Highly configurable/custom composite views Open integration options Dynamic implementation models Limited Data security & privacy Reporting and analytics Restricted20 © 2009 IBM Corporation
  21. 21. Taking it to the Next Level with Advanced Business Analytics What is happening What could happen? Simulation How many, how often, where? What if How can we achieve these trends Language & the best outcome? continue? Sentiment What Optimisation Forecasting exactly is the problem? What will How can we achieveStructured Data What Are Made What happen next the best outcome and Extracting People& Unstructured consumable and actions are if? address variability? concepts and Predictive Stochastic Talking AboutContent accessible to relationships needed? & Feeling everyone Modelling Optimisation Descriptive Predictive Prescriptive Content Web Analytics Analytics Analytics Analytics Analytics 21 © 2009 IBM Corporation
  22. 22. Today’s DiscussionIf you only remember four things Intensifying threat and policy pressures have pushed agencies to a 1 tipping point – they know they need to transform Agency leaders embrace this, are defining the new requirements, 2 and are demonstrating success Achieving transformation requires a broad set of capabilities, and a 3 combination of technology & expertise The experience and assets exist to help you define a strategy and 4 a roadmap to guide your transformation22 © 2009 IBM Corporation FOR INTERNAL IBM USE ONLY