Information Security Metrics Dashboards and Progress Reports

27,411 views

Published on

Published in: Technology, News & Politics
4 Comments
5 Likes
Statistics
Notes
  • Charles, I did have the opportunity of participating in the Corporate Information Security Working Group (CISWG) works, so obviously I am aware of it. I am glad you liked it, now we just need more people to use these ideas effectively.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I LIKE IT! It is important to address metrics reporting with the understanding that the people receiving the report are not likely to be experts in the subject measured. Too many security metrics presentations, books, articles I have seen either assume the audience has a deep understanding of the subject or, worse, the presenter is too busy illustrating his brilliance to focus on the needs of the audience.
    Are you familiar with the information security elements and related metrics documented by the Corporate Information Security Working Group (CISWG)? It is a little old, but includes some great content and logic.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • James,

    That is exactly my point. Measurements gain meaning from comparisons with previous (your example of trends) or comparable (my example of the rest of the population).

    TALL is not subjective at all if you define a threshold for tallness. In this particular example we are calling TALL for people who are taller than 95% of the population their age. You are right that categorization is important as well, that is an additional context that is relevant in many situations. I didn't got into details in this presentation as I wanted to talk only on how to represent the meaning of metrics (Check O-ISM3 and http://www.ism3.com/?q=node/18 for deeper detail)

    Benchmarking (comparing with external contextual data) provides with additional insight on situational awareness.

    Thanks for your comment!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 'a kids height is 100cm means nothing...'

    I agree, but where you have taken a comparison against other kids, what about the kids height last month, last quarter, last year? or the projected trend on how tall the kid is likely to be next quarter?

    'TALL' is subjective. So, you'd also need to define what TALL means. And, are you comparing against other kids of the same country? ethnicity ? or Globally? Kids from different parts of the world are different heights.

    So, when drawing a comparison, you would need to provide objective data against which that comparison is, and set it in the context of current v past and potentially future situation.

    So, 'A kid's height is 100cm', could become..

    A kids height is currently 100cm, in the previous months it was 98cm, 95cm, 92cm. At current growth rates it is likely to be 102cm next month, and 120cm next year. When compared with other kids of the same age, gender, ethicity, this kid is considered tall.

    Thus something like '98% of workstations are fully patched', could become...

    98% of production workstations are currently patched. In comparison with servers (insert % here), workstations are less/more compliant patched.
    In comparison against previous months this is getting better/worse/same. In comparison with other internal departments this is better/worse/same.
    In comparison with other offices/countries/regions this is
    In comparison with other industry peers this is (unknown). etc... etc.
    Overall 98% of workstations being patched is good/bad/ugly.

    If available, I would recommend taking into consideration external contextual data, but understand it's hard to get that because people don't really want to share their security posture with peers/competitors, which is something understandable for many security related metrics.


    (my two cents...)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
27,411
On SlideShare
0
From Embeds
0
Number of Embeds
11,153
Actions
Shares
0
Downloads
246
Comments
4
Likes
5
Embeds 0
No embeds

No notes for slide

Information Security Metrics Dashboards and Progress Reports

  1. 1. 1 Representing Security Metrics in DashBoards and Progress Reports © Inovement and Vicente Aceituno 2013
  2. 2. 2
  3. 3. 3 Metrics Representation  Metrics are measurements that gain meaning from comparison with previous or equivalent measurements.  For example, “A kid’s height is 100cm” means nothing.  “The height of a kid is 100cm in while the height of more than 95% kids is age is 90cm or less” means he is TALL.
  4. 4. 4 Metrics Representation  We get the most value from Metrics when we investigate the root causes for measurements that deserve our attention.  Correct representation of metrics can make obvious when a measurement deserves investigation.  Unfortunately, many representations of metrics hide meaning instead of highlighting it.
  5. 5. 5 Metrics Representation  There are 15 main metrics for a process or a control.  It is not practical to represent every metric for every control or process in an ISMS when there is a large number of controls.  It is therefore necessary to choose and find a compact way to represent metrics in order to gain situational awareness. Note: The canonical list of security metrics will be published early 2014 in a white paper.
  6. 6. 6 Metrics Representation  The interpretation of a metric always renders one or several of the following meanings:  Current Value:  Normal or Abnormal.  Satisfactory or Unsatisfactory.  Trend:  Better or Worse.  Increase or Decrease.  A good use of color and arrows can represent this in a compact a visually evident way.  Telling issues to investigate from those that require urgent attention evident brings added value to the dasboard.
  7. 7. 7 Metrics Representation  Some metrics correlate with value, some not, for example;  Without value:  Number of drops in a firewall. Fewer drops doesn’t we are not being attacked.  Number of viruses cleaned. More viruses cleaned doesn’t mean systems are cleaner.  With value:  Backups performed. The more backups, the more data can be recovered.  Authorized logins successful. When authorized people can login, they can work.
  8. 8. 8 Metrics Representation  When a metric does not correlate with value we have the following meanings:  Current Value:  Normal or Abnormal.  Trend:  Increase or Decrease.  When a metric correlates with value we have the following meanings:  Current Value:  Satisfactory or Unsatisfactory.  Trend:  Better or Worse.
  9. 9. 9 Metrics Representation  When a metric is not about value it can be represented using a square.  When a metric is about value it can be represented using a circle.
  10. 10. 10 Metrics Representation  Normal / Abnormal is a distinction that can be represented using Blue (Normal), Grey (Abnormal) and Black (Abnormal) for urgent Action.  Satisfactory / Unsatisfactory is a distinction that can be represented using Green (Satisfactory), Yellow (Unsatisfactory) and Red (Unsatisfactory) for urgent Action.
  11. 11. 11  Increase / Decrease trends is a distinction that can be represented using an arrow colored depending if the trend makes the current situation likely to stay.  Better / Worse trends is a distinction that can be represented using an arrow colored depending if the trend makes the current situation likely to stay. Metrics Representation
  12. 12. 12 Metrics Representation  The direction of the arrow indicates the type of change.  The color of the arrow indicates what that means.  A straight up or down arrow indicates the need for urgent action.  Examples:
  13. 13. 13 Metrics Representation  Exercise: Guess what the following mean:
  14. 14. 14 Metrics Representation  Solution: Abnormal, Increasing towards Normal, Urgent Action Abnormal, Decreasing towards Normal Normal, Decreasing Unsatisfactory, Getting better, Urgent Action Satisfactory, Getting worse Unsatisfactory, Getting worse fast, Urgent Action
  15. 15. 15 Metrics Representation  To summarize, any Security Metrics work is incomplete unless the representation of metrics in DashBoards and Progress Reports makes the meaning as obvious as possible.  It is possible to use colors and shapes to highlight meaning in a very compact way.
  16. 16. 16 Information Security that makes Business Sense inovement.es/oism3 Web www.inovement.es Video Blog youtube.com/user/vaceituno Blog ism3.com Twitter twitter.com/vaceituno Presentations slideshare.net/vaceituno/presentations Articles slideshare.net/vaceituno/documents
  17. 17. 17 668862242 learn@inovement.es Calle Loeches, 1, 28008, Madrid, Spain

×