Dr. Gary M. Shiffman
President and CEO
Giant Oak
Background:
• Georgetown University
• The Chertoff Group
• L-3
• US Custo...
War Among the People
Complex Threat Vectors
-- but all human
Ahmed Ressam
Mohamed al-Kahtani Mohamed Atta
Data => reaction l action
Challenge = Opportunity
Assessment: hadoop is Inevitable
• HAWHAT?
• Big Data Big Errors
• Cloud ≠ Cloud Analytics
• Share and Control Data
• No “...
Gary M. Shiffman
Giant Oak
• Big data
• Behavioral science
• Bad guys
How Hadoop Enables Analytics for Law Enforcement and National Security
Upcoming SlideShare
Loading in …5
×

How Hadoop Enables Analytics for Law Enforcement and National Security

659 views

Published on

The US National Security enterprise must fight and win the nation’s wars, protect and defend Americans citizens and US friends and allies, at home and abroad, and preserve the US democratic form of government and fee markets. Government leaders, although facing different budgetary and institutional goals and constraints than actors in the commercial world, must come to terms with the volume, velocity, and variety of data in the world and across their enterprises. Hadoop has recently arrived in federal office buildings in a big way, at least as a term to be discussed if not yet a data store deployed. As of June 2013, Hadoop appears destined for widespread adoption in the federal government, although the path forward remains uncertain. This session will provide a strategic view of the US federal government “big data” landscape as a marketplace, and suggest areas for highest return on investment for the national security community. Focusing on analytics—the use cases and the end customer at the “pointy end of the spear” seems likely to illuminate the most efficient way forward. Gary M. Shiffman is an economist, professor at Georgetown University in Washington, DC, and CEO of Giant Oak, a data analytics company focusing on detecting fraud, crime, insurgency, terrorism, and other forms of organized theft and violence.

Published in: Technology, Business, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
659
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Great honor to be among the revolutionaries. I believe. Next year will have 27,000 people here.Hadoop has moved from Proof of Concept and is transitioning toward the enterprise—”crossing the chasm.”I want to talk about MISSION. How do we make the HADOOP ecosystem relevant for operators and analysts. I focus on “bad guys”—national security and law enforcement.My message today:Crime, insurgency, and terrorism are about human behaviorWe know a lot about human behaviorLarge data environments allows us an opportunity for REVOLUTIONARY advances in UNDERSTADNING and INFLUENCING human behavior, and therefore, security.Hadoop (with Accumulo and others) can enable that revolutionary advanceBut we must prove the value in analytics—Hadoop will succeed when in this space when operationallyrelevant to the mission.
  • I’m a Gulf War veteran. We use T-hawks to take out air defenses, then established air superiority, then ground forces. I was on a Tomahawk shooter. We got the Iraqi uniformed soldiers and political leaders to surrender, but NOT ALL THE PEOPLE.Quiz: Identify the “bad guy”Left: Gulf War. Bad guys wearing uniforms and in tanks.Right: combat environment today. Do you see a bad guy at all? If I tell you one is in this photo, can you identify him or her?“Let’s just bomb the entire country if there is another terrorist attack.” is cold war thinking, but does not work today. Problems?Need to know geography and geology to do the bombing campaignNeed to know geography and terrain to roll tanks throughBut we don’t how do find the “bad guys” in the absence of arrayed troops in uniforms. - WE NEED THE MODELS OF THE TERRAIN: ANTHROPOLOGICAL, SOCIAL, FINANCIAL, ETC.Warfare today is personal: we are fighting a war “among” the people. WE NEED TO BE ABLE TO OBSERVE AND UNDERSTAND HUMAN BEHAVIOR IN WAR AMONG THE PEOPLE.
  • For example, a science of human behavior + a data-rich world = Massive increase in observations of human behavior.Apple falling from a tree: reaction modeled with precise measurements; scientists love observations in controlled environments.As a behavioral scientists, the more I can observe an person making choices, the more accurately I will be able to forecast—predict—a person’s reactions the next time facing the same scarcity.I can move from characterizing large demographics: Pakistani Male b/w ages of 18 – 35 (wrong and inefficient) to characterizing Mohammed Atta.What caused the riots taking place in Istanbul’s Taksim Square? How will Prime Minister Erdogan react? You can see how this would be a scientific question, if we could replicate the factors and run experiments thousands of times. Social science, however, is difficult in this respect. But what if we could monitor crowds by following Twitter and Facebook? Videos and photos from inside the square? Welcome to the future.We are in the midst of a “big data” revolution. Data in the world doubling every 12-24 months. We’ve heard about “big data.” we’ve seen focus on the technology of data generation, storage, and processing advance over the last 5 years. Over the next five years, look for revolutionary evolution in the MISSION SPACE: analysis and user interfaces.“Man, up to now at least, has always gone lamentably amiss in his attempts to bridge the gulf that he sees yawning between mind and matter, between the rider and the horse, between the mason and the stone. It would be preposterous to view this failure as a sufficient demonstration of the soundness of a dualistic philosophy. This ignorance splits the realm of knowledge into two separate fields, the realm of external events, commonly called nature, and the realm of human thought and action. Ludwig von Mises, 1957”Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation License. http://commons.wikimedia.org/wiki/File:Tahrir_Square_on_November_18.jpghttp://commons.wikimedia.org/wiki/File:Osama_bin_Laden_portrait_cropped.jpg
  • In the United States, for example, we have tremendous capacity to address threats to public safety and security. A few examples: we have the Department of Homeland Security, with customs and immigration authorities. We have a Treasury, Justice, and Defense Departments, to name a few, each charged with different aspects of national security. We also have state and local law enforcement, and our friends and allies around the world. And of course (in light of PRISM news, etc.) we have the US Intelligence community, comprised of 17 agencies. Cover this latest WIRED, “God of War: For years, U.S. Genreal Keith Alexander has been amassing a secret cyber army. Now it’s ready to attack.” (July 2013)I was going to call this slide “the PRISM of applied data analytics but decided against it.”[click to insert lines]Discussion of incentives to share and not share information.Example: Drug trafficking organization, moving guns, and taking over villages, committing acts of terrorism. CBP, ICE, DEA, ATF, FBI, DOD, Treasury. Within DOD, military services, special operators. Can the Hadoop environment (Accumulo, etc.) enable trust and sharing across these lines? Can it address security? Governance? This is the opportunity, and it is huge. https://commons.wikimedia.org/wiki/File:United_States_Intelligence_Community_Seal_2008.jpg[Insert ICE, TSA, Coast Guard]
  • HAWHAT? You mean a Big Data?Cloud and analytics, but not yet cloud analyticsMost don’t know what Hadoop, Accumulo, Apache, etc.Big Data => Big Errors (law of large numbers?)The mean of the sample, no matter how large, may not converge to the mean of the population.CLOUD ≠ CLOUD ANALYTICSWe can create a distributed file system, and then pull data into a Cube to run analytics. The analytic tools do not yet exist. (Giant Oak team works hard in this area).Share AND Control DataMany people express strong concerns about the potential for unauthorized access to a cluster. Some argue that logical separation can substitute for physical separation; they believe security solutions that have “accumulo” as a foundation will help reduce that risk.No “Easy” button, yetMost also see that Hadoop takes highly skilled java experts to configure and make run and many are calling for solutions that do not require users to go to the IT department. Can we build a system off of Hadoop that operators and analysts can manipulate without Software skills?Hadoop the inevitable: LEA, IC and DoD: millions of people and 100's of organizations, but in general, a growing number of people recognize that the cost savings and scalability of hadoop-centric approaches mean you are going to have at least some hadoop in your organization.  They do know “big data” and the risks they run for not taking action. People WANT to learn about Hadoop. They want data scientists. For example, FBI bid on video analytics, all these functional silos, especially in law enforcement community. Functional solos exist. Nuggets of data. no way to be shared or corresponded. With Hadoop, storage, pieces can be shared across agencies otherwise not connected, to solve cases.Tagging at cell level. E.g. video servers that are proprietary. Based on what happened in Boston. Google Goggles. Can share data many ways. At ingest, or meta-data tagging by location, types, etc. tagging the video on my phone tagged with time and location, brining it all together to help paint the picture.
  • The Future?DescriptionInferenceInfluence. The future may not be the Matrix: a neural interactive simulation. Computer generated dream world to keep us under control. Inside a computer program, mental projections can be real because of electrical signals interpreted by brain. Think about “the internet of things.” All of our devices talk to each other. We have near constant observation of human actions, preferences, constraints. Warfare is about human behavior, and we know a lot about human behavior. We have a science. We can build models.Data provides us with an opportunity for Revolutionary advances in understanding and influencing behavior. In all aspects of our lives, including national security and law enforcement.Hadoop allows us the chance to enable data sharing and analytics. To succeed we must focus on operational relevance.  Great honor to be among the revolutionaries. My message today:Warfare and crime is about human behaviorWe know a lot about human behaviorLarge data environments allows us an opportunity for REVOLUTIONARY advances in UNDERSTADNING and INFLUENCING human behavior, and therefore, security.Hadoop (with Accumulo and others) can enable that revolutionary advanceBut we must prove the value in analytics—Hadoop will succeed when in this space when operationallyrelevant.
  • How Hadoop Enables Analytics for Law Enforcement and National Security

    1. 1. Dr. Gary M. Shiffman President and CEO Giant Oak Background: • Georgetown University • The Chertoff Group • L-3 • US Customs and Border Protection • US Senate • Pentagon and US Navy Gulf War Veteran
    2. 2. War Among the People
    3. 3. Complex Threat Vectors -- but all human Ahmed Ressam Mohamed al-Kahtani Mohamed Atta
    4. 4. Data => reaction l action
    5. 5. Challenge = Opportunity
    6. 6. Assessment: hadoop is Inevitable • HAWHAT? • Big Data Big Errors • Cloud ≠ Cloud Analytics • Share and Control Data • No “Easy” Button, Yet?
    7. 7. Gary M. Shiffman Giant Oak • Big data • Behavioral science • Bad guys

    ×