Your SlideShare is downloading. ×
0
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Geer -  Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Geer - Hutton - Shannon - A Pilot Project On The Use Of Prediction Markets In Information Security

900

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
900
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. A Pilot Project on theUse of Prediction Markets in Information Security Dan Geer, In-Q-Tel Alex Hutton, Verizon Business Greg Shannon, Carnegie Mellon April 20th, 2011 alpha-pilot at securitypredictions dot com
  • 2. Overview  Motivation (dg)  Prediction Market Examples (gs)  What is the pilot; what information will it generate? (gs)  Why is this valuable to the infosec industry? (ah)  How is this helpful to security teams and professionals? (ah) Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 2
  • 3. Motivations  Our Goal: Accelerated aggregation and dissemination of actionable security information from diverse sources  Purpose of this talk: Explain the Pilot Project  Purpose of the pilot: Validate that we can use a market to collect informed opinions from participants that when aggregated and shared is of interest to individuals, organizations and the information security industry.  Excellent overview and references in:   "Using Prediction Markets to Enhance US Intelligence Capabilities," CIA Center for the Study of Intelligence, 2006, v50 n6, PDF 17pp. http://tinyurl.com/6kdqpl Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 3
  • 4. The Art in Prediction  In prediction markets, the art is selecting the questions, i.e., prediction markets are invulnerable to idiots but not to idiotic questions.    Science and practice alike have shown that prediction markets have greater accuracy than surveys and, unlike surveys, can be run continuously.    As the rewards available to market participants rise, the precision of the markets predictions improves. Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 4
  • 5. PrimerSuccessful Public Prediction MarketsGeer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 5
  • 6. A Simple Market Example  http://en.wikipedia.org/wiki/Prediction_market  Will candidate X win election Y? Yes or no?  Three elements: Participants, Contracts, Incentives Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 6
  • 7. PrimerWhat are Prediction Markets? Large groups of people are smarter than an elite few,no matter how brilliant — better at solving problems,fostering innovation, coming to wise decisions,even predicting the future. — James Surowiecki, author of The Wisdom of Crowdsdef. Speculative markets used to make predictions of specificevents. Contracts representing the event, or outcome, arebought and sold resulting in contract price fluctuations. Thecurrent price represents the current group estimate of thelikelihood of the event. April 2011 7
  • 8. How They Work:Reflecting Confidence in Outcomes  Individual answers are anonymous, market aggregates consensus  Participants are incented to express the strength of their confidence  Participants are rewarded based on the accuracy of their contributions  Social collaboration and comments by question, surface root causes April 2011 8
  • 9. How They Work:Revealing Early Warning Indicators  Participants invest in stocks (buy/sell) and thus drive the price up or down. The price reflects the crowd’s confidence in the stated outcome.  Decision-makers receive an analytical, real-time consensus view into the true state of key issues. Project Aries will achieve customer acceptance by 30-Sept-2011. Information contained in dropping confidence April 2011 9
  • 10. Social Analytic Reports &Decision Dashboards Tracking changing trends in consensus opinions Identifying divergent opinions among participants subgroups – Monitor  par*cipa*on   where does the information to  ensure  diversity   reside? April 2011 10
  • 11. Pilot Overview  60-day alpha pilot  Use Consensus Point as the market platform  20-30 hand-picked participants  Internal (market) recognition as the incentive  Binary contracts varying in topic and duration   Written by Geer, Hutton, Shannon  Pilot objectives:   At least 10 contracts open at all times   20 contracts with at least 10 participants,100 trades   Positive survey results from participants at the end   At least 3 unclosed contracts estimating future events   Have a contract payout on an unexpected security event   Gain enough confidence to start a half-year beta Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 11
  • 12. What Do We Want To Know?  What is the collective, anonymous, incented opinion about actionable information security events and states of the world?  How accurate and stable is this opinion/knowledge?  Can this knowledge benefit participants, 3rd parties and the industry to improve information security?  Can a prediction market mitigate the unavailability of detailed operational infosec data? Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 12
  • 13. Criteria For Contracts  A binary question   Good: The market-cap leader in consumer operating systems issues a press-release on a security-critical patch this quarter.   Poor: The number of software vulnerabilities discovered in the most popular consumer operating system increased this quarter over the previous quarter.  A definitive authority on the result   Good: government agency, public company, nationally-recognized institution   Poor: news, an individual, on-line poll, micro-blog traffic  A history of indisputable previous outcomes   Good: Alerts issued, scores published, reports published   Poor: News articles, court documents, non-public sources  Market information is likely actionable   Good: A disruptive OS patch is in the pipeline   Poor: Companies will lose more data this year than last  Morally benign  Difficult for single entities to influence the outcome of the underlying event Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 13
  • 14. Candidate ContractsGeer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 14
  • 15. Other Candidate Sources & Contracts  US-CERT alerts  Botnet species announced  Statistics from data breach reports  Trends in security surveys and indexes  Statistics from software security or controls reports  MITRE CVE reports Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 15
  • 16. Criteria for Alpha Participants  Demonstrated knowledge of information security  At least 5 years of professional experience in such  Diverse across   Sectors: Government, Industry, Academic   Verticals: Civilian Gov’t, Health, Financial, DoD, Telecom, etc.   Layers: hosts, networks, applications, infrastructure, content   Life cycle: creation, installation, operation, incidents, remediation   Specialties: privacy, risk, availability, integrity, etc.   Demographics Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 16
  • 17. Incentive Criteria  Is legal  Is sufficient to entice participants to divulge their knowledge through market activity  Benefits are tangible to all participants   Not just the top performers  Does not encourage market manipulation or spectuation  Scales to 50 active contacts and 1,000 participants Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 17
  • 18. Value to the InfoSec Industry  Opportunity for big-time benefit to the industry. Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 18
  • 19. Value to the InfoSec Industry  A prediction market is a specifically framed piece of knowledge (belief as a probability)  What do you want knowledge about?   Understand trends as they happen (or don’t happen) Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 19
  • 20. Value to the InfoSec Industry Suggested context: Capability to manage (skills, resources, asset landscape decision quality…) impact landscape risk threat landscape controls landscapeGeer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 20
  • 21. Value to the InfoSec Industry  Example: Mobile Malware   % Mobile devices as targeted asset in 2011 DBIR   % Mobile devices as targeted asset in 2012 DBIR   % Mobile devices as targeted asset in 2013 DBIR   The effect of new vulnerability research on the above contracts...   The effect of new security technologies on the above contracts... Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 21
  • 22. Value to the InfoSec Industry Suggested context: Capability to manage (skills, resources, asset landscape decision quality…) impact landscape risk threat landscape controls landscapeGeer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 22
  • 23. Value to InfoSec Teams and Professionals  An internally facing prediction market can be used for decision support   Success/Failure of big dollar security projects   What current projects (both security and non-security) mean to the frequency or impact of security events   Impact of current security events   This breach will cost how much? Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 23
  • 24. Value to InfoSec Teams and Professionals  Calibration   Ability to better qualify the subjective evidence around us  Ability to “mine” changes in “price” for causes Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 24
  • 25. Recap  Our Goal: Accelerated aggregation and dissemination of actionable security information from diverse sources  To follow or join the pilot send e-mail to: alpha-pilot at security predictions dot com Geer Hutton Shannon Pilot Project for an InfoSec Prediction Market April 2011 25
  • 26. On The Use of Prediction Markets inInformation Security (from src-bos program) A tool created to help establish beliefs as probabilities, prediction markets are speculative markets created for the purpose of understand the probability of future events. Not widely used in Information Security, Prediction Markets may have benefits to our industry. Dan Geer, Alex Hutton and Greg Shannon will give a background around what prediction markets are, how they can be used by the information security industry as a whole, and how security departments and professionals can use them as a tool to help defend their environments. Dan Geer is a computer security analyst and risk management specialist and currently the chief information security officer for In-Q-Tel. Alex Hutton is a principal for Research & Intelligence with the Verizon Business RISK Team. Dr. Greg Shannon is the chief scientist for the CERT® Program at Carnegie Mellon University’s Software Engineering Institute. http://www.sourceconference.com/boston/speakers_2011.asp#dgeerGeer Hutton Shannon Pilot Project for an InfoSec Prediction April 2011 26 Market

×