Measurement, Quantitative vs. Qualitative and Other Cool Stuff
Upcoming SlideShare
Loading in...5
×
 

Measurement, Quantitative vs. Qualitative and Other Cool Stuff

on

  • 4,502 views

InfoSec Measurement and Quantitative vs Qualitative Methods ...

InfoSec Measurement and Quantitative vs Qualitative Methods


Recorded Webinar Here:
https://www3.gotomeeting.com/register/604059902

Aliado and Risk Centric Security would like to introduce you to the world of quantitative risk and decision analysis.

Our webinars will provide you with a glimpse of the power and credibility that quantitative methods can bring to the problems that Information Security Professionals face every day

Topics covered include:

What is risk?
Possibility and Probability
What is a measurement and what is it for?
Qualitative vs. Quantitative methods
Static modeling vs. Monte Carlo simulation
Calibration and the power of a calibrated estimate
Modeling Expert Opinion and the RCS BetaPERT calculator

A. Definitions
1. Risk
2. Risk and Opportunity
3. Possibility vs. probability
4. Measurement
5. Precision vs. accuracy
6. Qualitative vs. quantitative methods


Statistics

Views

Total Views
4,502
Views on SlideShare
3,623
Embed Views
879

Actions

Likes
1
Downloads
94
Comments
0

7 Embeds 879

http://www.aliadocorp.com 741
http://www.riskcentricsecurity.com 90
http://www.linkedin.com 23
http://www.eproneur.com 22
url_unknown 1
http://aliadocorp.com 1
https://www.linkedin.com 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Measurement, Quantitative vs. Qualitative and Other Cool Stuff Measurement, Quantitative vs. Qualitative and Other Cool Stuff Presentation Transcript

  • Measurement,Qualitative vs. Quantitative Analysis,and other Cool Stuff
    Presenting: Risk Centric Security, Inc.
    www.riskcentricsecurity.com
    Sponsor: Aliado
    www.aliadocorp.com
    Risk Centric Security, Inc. Confidential and Proprietary .
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
    Risk Analysis for the 21st Century
  • Introductions
     
    Heather Goodnight is an accomplished Global Sales and Business Development Consultant. Over the years, her unique, practical insight into problems of risk and opportunity have provided important guidance for organizations both large and small. She is a cofounder of Risk Centric Security and currently serves as President of the Corporation.
    Patrick Florer has worked in information technology for 30 years. In addition, he worked a parallel track in medical outcomes research, analysis, and the creation of evidence-based guidelines for medical treatment. His roles have included IT operations, programming, and systems analysis. From 1986 until now, he has worked as an independent consultant, helping customers with strategic development, analytics, risk analysis, and decision analysis. He is a cofounder of Risk Centric Security and currently serves as Chief Technology Officer.
    Risk Centric Security, Inc. Confidential and Proprietary .
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Houston, we have a problem …
    When speaking with our customers, we recognized:
    Information Security Professionals are comfortable speaking the technical language of firewalls, logs, threats, vulnerabilities, and exploits.
    Business managers are comfortable speaking the language of return on investment, discounted cash flows, and risk as financial impact.
    Mutual misunderstanding can occur, and it is often a source of frustration for everyone.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • So how do we bridge the language gap?
    By learning to speak about risk in business terms, Information Security Professionals can reach out and bridge the language gap.
    The technical details of sql injection attacks may be important to you, but your business counterparts may not understand, and they usually don’t care.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • So how do we bridge the language gap?
    Instead of talking about threats, vulnerabilities, and controls, talk about risk in terms of financial impact. Tell the business people what a sql injection attack could cost.
    They will understand that!
    (They may not believe you, but they will understand what you are saying!)
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What are we going to talk about today?
    Risk
    Risk and Opportunity
    Possibility vs. probability
    Measurement
    Precision vs. accuracy
    Qualitative vs. quantitative methods
    The “not enough data” syndrome
    Monte Carlo simulation
    Modeling expert opinion and the PERT distribution
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What is risk, anyway?
    From The American Heritage dictionary*:
    The possibility of suffering harm or loss; danger.
    A factor, thing, element, or course involving uncertain danger; a hazard.
    The danger or probability of loss to an insurer.
    The amount that an insurance company stands to lose.
    The variability of returns from an investment.
    The chance of nonpayment of a debt.
    *The American Heritage® Dictionary of the English Language, Fourth Edition. Copyright © 2006, 2000. Houghton Mifflin Company.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What is risk, anyway?
    From ISO 31000:
    1.1 risk - effect of uncertainty on objectives
     
    NOTE 1 An effect is a deviation from the expected —positive and/or negative.
    NOTE 2 Objectives can have different aspects (such as financial, health and safety, and environmental goals) and can apply at different levels (such as strategic, organization-wide, project, product and process).
    NOTE 3 Risk is often characterized by reference to potential events (3.5.1.3) and consequences (3.6.1.3), or a combination of these.
     
    NOTE 4 Risk is often expressed in terms of a combination of the consequences of an event (including changes in circumstances) and the associated likelihood (3.6.1.1) of occurrence.
     
    NOTE 5 Uncertainty is the state, even partial, of deficiency of information related to, understanding or knowledge of, an event, its consequence, or likelihood
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What is risk, anyway?
    In the USA, NIST, Special Publication 800-30 describes risk in the following way:
    Risk is:
    “the net mission impact considering the probability that a particular threat-source will exercise (accidentally trigger or intentionally exploit) a particular information system vulnerability, and the resulting impact if this should occur.”
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What is risk, anyway?
    NIST (The National Institute of Standards and Technology), provides an additional definition of risk in Special Publication 800-39:
    Risk
     
    A measure of the extent to which an entity is threatened by a potential circumstance or event, and typically a function of: (i) the adverse impacts that would arise if the circumstance or event occurs; and (ii) the likelihood of occurrence.
     
    Information system-related security risks are those risks that arise from the loss of confidentiality, integrity, or availability of information or information systems and reflect the potential adverse impacts to organizational operations (including mission, functions, image, or reputation), organizational assets, individuals, other organizations, and the Nation.
    NIST, The National Institute of Standards and Technology, Special Publication 800-39, Appendix B, Page B-7.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What are the common elements here?
    A probability that something will happen
    A probable impact if something does happen
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What, then, is our working definition of risk?
    The probability that something will happen to cause a negative impact in financial terms:For example, a 50% chance that it will cost 50 million dollars if our data are stolen.
    Another way to express this is to multiply the two numbers together and say that:
    Risk = 25 million dollars on an annualized basis
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Risk and Opportunity
    For our discussion today, Risk will be used to indicate loss or harm.
    Opportunity can be viewed as the positive aspect of Risk.
    The techniques that apply to Risk analysis can also be applied to Opportunity analysis.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Possibility and Probability
    Let’s look at tossing a coin:
    What are the possibilities?
    What are the probabilities?
    Does knowing either help us predict what will happen when we toss the coin next time?
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Possibility and Probability
    A possibility is something that is “capable of happening, existing, or being true without contradicting proven facts, laws, or circumstances known to be true. *”
    A probability is "the likelihood that a given event will occur.”*
    *All quotes from The American Heritage® Dictionary of the English Language, Fourth Edition. Copyright © 2006, 2000. Houghton Mifflin Company.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Possibility and Probability
    In statistics, a probability is “a number expressing the likelihood that a specific event will occur, expressed as the ratio of the number of actual occurrences to the number of possible occurrences.“
    Probability is calculated after tossing the coin many times.
    Probability is always a number between 0 and 1, sometimes expressed as:
    *All quotes from The American Heritage® Dictionary of the English Language, Fourth Edition. Copyright © 2006, 2000. Houghton Mifflin Company.
    0 <= P(X) <= 1
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Possibility and Probability
    How can we use this in information security risk analysis?
    The fact that something can happen (possibility) doesn't tell us how likely it is to happen (probability), or how much impact it might have if it does happen (probability).
    Estimating these values helps us prioritize our activities in a rational way.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Measurement
    What is a measurement?
    An observation that “ascertains the dimensions, quantity, or capacity of” an object or process”*
    A set of observations that reduce uncertainty where the result is expressed as a quantity**
    *The American Heritage® Dictionary of the English Language, Fourth Edition. Copyright © 2006, 2000. Houghton Mifflin Company
    ** Hubbard, Douglas W., “How to Measure Anything 2nd Edition”, John Wiley & Sons, New Jersey, 2010, p. 23
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Measurement
    What are the properties of a measurement?
    Validity – does the measurement actually do what you think it does?
    Reproducibility – when repeated, does the measurement give a consistent answer?
    Detail – does the measurement provide a useful level of detail?
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Measurement
    What are some sources of error in measurement?
    Random error – a function of the instrument
    Bias – a function of the measurement taker
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Measurement
    Why do we make measurements?
    Measurements are a way to collect data.
    Making measurements should be about reducing uncertainty.
    A measurement only has to be good enough for the decision at hand.
    Sometimes, you cannot get the data you think you need, so you have to use a proxy.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Precision and Accuracy
    Precision is “the ability of a measurement to be consistently reproduced.”
    Accuracy is “the ability of a measurement to match the actual value of the quantity being measured.”
    *All quotes from The American Heritage® Dictionary of the English Language, Fourth Edition. Copyright © 2006, 2000. Houghton Mifflin Company.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Precision and Accuracy
    Precision: a machine can produce the same part to within 1/1000th mm all day long. This is no guarantee that the part is the correct length, however.
     
    Accuracy: a machine can produce the same part to within +/- 2/1000th mm of the correct length. Although some parts are a bit shorter and some are a bit longer, every part is within spec.
     
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Precision and Accuracy
    Precision: 100.001, or 10.233%
     
    Accuracy: 100 or 10%, or 10.2%
     
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Precision and Accuracy
     
    Prefer Accuracy to Precision.
    Precise Accuracy? – it would be nice!
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative and Quantitative Methods
    Qualitative methods: green, yellow, orange, and red (dashboards) or a scale from 0 – 5 (categorical, nominal, and ordinal).
    Quantitative methods: real numbers (cardinal scale).
    Most of the time, quantitative methods are easier.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Benefits of qualitative methods?
    They are useful in certain scenarios, and can be quick and good enough.
    Problems with qualitative methods?
    Variability between assessors
    Inconsistency of a single assessor
    Arithmetic and statistical operations not possible
    Problems near the boundaries of categories
    Loss of information
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Variability between assessors
    Faced with the same set of facts, different assessors apply a scale differently.
    Two QSA’s apply the PCI standards differently.
    Two risk analysts classify risks differently – one says low, one say medium
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Inconsistency of a single assessor
    Given the same set of facts, an assessor might make different assessments when the only difference is the passage of time.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Difficulty with arithmetic and statistical operations
    From ISO 17999
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Difficulty with arithmetic and statistical operations
    From ISO 17999
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Difficulty with arithmetic and statistical operations
    From ISO 17999
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Difficulty with arithmetic and statistical operations
    Imagine if money worked this way:
    The value of a dollar would be relative to the purchase price of an item.
    The value of a dollar might vary from store to store.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Problems with aggregation and estimates near the boundaries of categories
    Assume that:
    Low = < 1M
    Medium = 1M – 5M
    High = >5M
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    And assume that the following risks have been identified and put into categories:
    $100K, 500K, 800K: all in Low category
    $1M, 3M, 3M, 4M: all in Medium category
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    What happens when you aggregate based upon qualitative scales?
    What is the real difference between a very “high Low” and a very “low Medium”?
    How can we justify and defend category boundaries that are essentially arbitrary?
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Qualitative Methods
    Loss of information
    Most of the time, we get a number in mind.
    Then, we assign it to a category.
    Why not just keep the number?
    Or better yet, create a distribution around a range of estimates to better express our beliefs and confidence?
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Quantitative Methods
    Benefits of quantitative methods?
    The numbers mean what they are (cardinality).
    Arithmetic and statistical methods are possible.
    Problems with quantitative methods?
    Data are required.
    Estimates are estimates – the future hasn’t happened yet.
    Formal training in calibration techniques is very helpful.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • ”We don’t have enough data!”
    They say: there isn’t enough “good” data, so you are just processing “garbage in and garbage out.”
    The reason we need data is to reduce uncertainty in decision-making.
    The decision we need to make will define the data we need – some decisions require very little data, others require quite a bit.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • How much data do we really need?
    A sample can be smaller than you think.
    Parametric vs. non-parametric methods
    Contact us for more information on these topics.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • What is the definition of “good” data?
    We often hear that the data are poor –
    What does this mean?
    Data are just data – some data may be more interesting than other data – it depends on what you are doing.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Interesting data collection projects
    Dan Geer et al.:
    The Index of Cybersecurity
    (http://www.cybersecurityindex.org/)
    Prediction Market Project
    The Beewise Project
    (http://beewise.org/markets/metricon.ctrl)
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • ”We don’t have enough data!” - Sources
    Please refer to the slides at the end of this presentation.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Monte Carlo Simulation
    Monte Carlo simulation is a game changer for information security risk analysis.
    Less sophisticated methods use single-point estimates or even simple ranges of estimates:
    35%, or from 20% - 51%
    Monte Carlo methods sample thousands or tens of thousands of values, and provide a much clearer picture of the possible outcomes.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Minimum:
    What is the least or lowest (best or worst) numerical estimate that you believe to be reasonable? This will be the smallest number you come up with.
     
    Most Likely:
    What is the most likely or most probable numerical estimate in your opinion? This number must fall between the minimum and maximum. It may equal either the minimum or the maximum, but should not equal both
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Maximum:
    What is the greatest or highest (best or worst) numerical estimate that you believe to be reasonable?
     
    Note that “best” or “worst” case estimates could be either minimum or maximum values, depending upon the scenario.
    In a risk / loss exposure scenario, lower is better, so the minimum represents the lowest loss, or best outcome. The maximum represents the highest loss, or worst outcome.
    In a sales or opportunity scenario, it’s the reverse: lower is not better, so the minimum represents the worst case. Higher is better, so the maximum represents the best case.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Confidence:
    On a scale that includes “Very Low”, “Low”, “Average”, “High”, and “Very High”, how confident are you in the accuracy of your estimates?
    This parameter controls the sampling around the most likely value, and thereby also controls the height of the histogram or slope of the cumulative plot.
    For most analyses, using “Average” for the confidence parameter works well. In this instance, “Average” really means having no strong feeling about the matter – being evenly divided between under-confidence and over-confidence.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Percentile Tables
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    PercentileTables
    1% of values are <= 10,044 and 99% are > 10,044
    10% of values are <= 11,120 and 90% are > 11,120
    20% of values are <= 11,658 and 80% are > 11,658
    50% of values are <= 13,025 and 50% are > 13,025
    The 50th percentile has another name - it’s called the Median.
     
    The Median is the mid-point in a list of values - half of the values in the list are less and half are greater than the Median.
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Histogram
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • The Beta Pert Calculator
    Cumulative Plot
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • Thank you !
    Heather Goodnight
    Patrick Florer
    Co-founders
    Risk Centric Security, Inc
    heather@riskcentricsecurity.com
    patrick@riskcentricsecurity.com
    www.riskcentricsecurity.com
    214.405.5789
    Jody Keyser
    Aliado
    jkeyser@aliadocorp.com
    Risk Analysis for the 21st Century
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • ”We don’t have enough data!” - Sources
    Open Security Foundation: datalossdb and osvdbhttp://www.opensecurityfoundation.org/
    Computer Security Institute (CSI): http://gocsi.com/
     
    Office of Inadequate Security: http://www.databreaches.net/
     
    Identity Theft Resource Center: http://www.idtheftcenter.org/
     
    ISACA: www.isaca.org
     
    ISSA: www.issa.org
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • ”We don’t have enough data!” - Sources
    Mitre Corporation: www.mitre.org
    OWASP: http://owasp.com/index.php/Main_Page
    Privacy Rights Clearing House: http://www.privacyrights.org/
     
    SANS: www.sans.org
     
    The Ponemon Institute: www.ponemon.org
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • ”We don’t have enough data!” - Sources
    Conference procedings: Black Hat, RSA, Source Conferences, BSides
     
    Internet tools:
     
    Search engines: Google, Bing, Yahoo, Ask.com
     
    Trend Analyzers:
     
    Google trends: http://www.google.com/trends
    Twitter Trends: www.trendistic.com
     
    Amazon: http://www.metricjunkie.com/
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.
  • ”We don’t have enough data!” - Sources
    Securitymetrics.org – mailing list
    Society of Information Risk Analysts (SIRA)
    Risk Centric Security, Inc. Confidential and Proprietary.
    Copyright © 2011 Risk Centric Security, Inc . All rights reserved.