Presented by: Russell Thomas, George Mason University Abstract: Two aspects of cyber security that everyone struggles with are metrics and business impact. How do we measure it to improve and how do we make it meaningful to business decision makers? This gap appeared again recently in the NIST Cyber Security Framework (CSF) process RFI responses. But there is no need to wait for NIST CSF or anything else because there is a viable method available now that you can use to build your own CSF. Namely the “Balanced Scorecard” method. The key idea is to focus on performance against measurable objectives in all critical dimensions that, taken together, will lead to better security, privacy, and resiliency outcomes, even in a dynamic and highly uncertain threat environment. In this presentation, we’ll explain the ten critical dimensions of cyber security performance, explain how they are interrelated and feed off each other, show how to create a performance index in each dimension, and describe how the balanced scorecard can be used to drive executive decisions. This presentation should be valuable to managers and executives in every type of organization in the energy sector, including the supply/service chain. Consultants, regulators, and academics should also find it interesting and useful.

1. How to Build Your Own
Cyber Security Framework
using a Balanced Scorecard"
Russell Cameron Thomas!
EnergySec 9th Annual Security Summit!
September 18, 2013!
Twitter:
@MrMeritology!
Blog:
Exploring Possibility Space!

2. Who here loves frameworks?!

3. Who here loves frameworks?!
NIST Cyber Security Framework?!
Other?!

4. Frameworks can matter (a lot)

5. Frameworks can matter (a lot)
if they are instrumental in
driving new levels of
Cyber Security Performance

6. What the hell is
“Cyber Security Performance”?!

7. Yes, “Cyber”!

8. Yes, “Cyber”!
Confluence of…!
• Information Security!
• Privacy!
• IP Protection!
• Critical Infrastructure Protection & Resilience!
• Digital Rights!
• Homeland & National Security!
• Digital Civil Liberties!

9. What the hell is
“Cyber Security Performance”?!

10. “Cyber security performance” is… "
… systematic improvements in an
organization's dynamic posture
and capabilities relative to its
rapidly-changing and uncertain
adversarial environment.”!

11. “Cyber security performance” is… "
…Management By Objectives!
(Drucker)!

12. “Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!

13. “Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
…Staffing, training, organizing!

14. “Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
…Staffing, training, organizing!
…Organization learning, agility!

15. “Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
…Staffing, training, organizing!
…Organization learning, agility!
… and good practices!

16. “Performance” vs “Practices”!

17. Using the Universal Language of Executives….

18. Using the Universal Language of Executives….

20. Keep
your head
still

21. Keep
your head
still
“Keep
your arm
straight”

22. Keep
your head
still
“Keep
your arm
straight”
“Swing on
one plane”

23. Keep
your head
still
“Keep
your arm
straight”
“Swing on
one plane”
“Swing easy”

24. Keep
your head
still
“Grip it
and rip it!
“Keep
your arm
straight”
“Swing on
one plane”
“Swing easy”

25. Best practices are like golf tips…

26. Best practices are like golf tips…
Golf tips alone don't make good golfers

27. Why Agility?
Why Rapid Innovation?!

29. State of
the Art!
Lagging
InfoSec
Program!

30. Time for some drama!

31. Time for some drama!
Set in the Summer of 2017!

32. “
I
in central Texas.”
t was another
long heat wave

33. Spare generating capacity was dangerously low!

34. You run information security!
at a large industrial company!
that includes several
and cogeneration.!

35. Thanks to
deregulation
and incentives,
microgrids
have taken off,
especially
in Texas
=
10+
microgrids
Microgrid Adoption, 2017

36. In recent days,
instead of selling
its excess power,
your firm was
buying at peak
spot prices.
This was strange.!

37. 18
months
earlier
YouEnergy Ops
Manager
Business
Continuity
Manager

38. Effective Response,
Recovery Resilience

39. Your Microgrid Automation
hosted
auto-configuring
software
reporting/trending!
system config!
diagnostics!
Internet
Microgrid
Supervisory
Controller
12
months
earlier

40. Spot trading was largely automated
via microgrid automation software.
12
months
earlier

41. Optimize Exposure

42. Insiders?
Threat Intelligence
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
24
months
earlier

43. Our New Capability:
Attack-driven Defense
1. Raise cost to attackers
2. Increase odds of detection
3. Iterate defense based on real attack patterns
24
months
earlier
source:
Etsy
h7p://www.slideshare.net/zanelackey/a7ackdriven-­‐defense

44. Insiders?
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
Threat IntelligenceYesterday

45. Effective
Threat Intelligence

46. Sensors Pattern Detection for
Anomalous User Behavior
24
months
earlier
Any
Non-
Tech. Tech.
source:
Etsy
h7p://www.slideshare.net/zanelackey/a7ackdriven-­‐defense
User
Class

47. Insiders?
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
X
Threat Intelligence
X
Yesterday

48. Quality of
Protections Controls

49. Insiders?
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
X
X
Threat IntelligenceYesterday

50. Efficient/Effective
Execution Operations

51. 12
months
earlier

52. Effective
External Relationships

53. The Crime:
ArDficially
Congested
Subsided
Generators
Manipulation of
Wholesale Market
Subsidies
Conges'on
pa+erns,
July
14,
2017

54. Losers: You and hundreds of other microgrids forced to
generate spot market bids during price spikes.
(Botnet-style. Each loses a little $$)
Scam: Generate losing trades in one market
to make money in another market

55. Attack: Compromised Hosted
Auto-Configuration Software
hosted
auto-configuring
software
reporting/trending!
system config!
diagnostics!
Internet
Microgrid
Supervisory
Controller

56. The Attackers
Insider: Contractor
at web application
software company
Outsider: Hedge
fund manager
bribed contractor
with profit sharing

57. Gold Man Hacks Bid Probe
2017
2017
Gold Man Hacks Faces Record Fine Over Energy

58. Over
the
last
24
months
Adap've
Threat
Intelligence
A+ack-­‐
driven
Defense
Expanded
External
Engagement
Expanded
Detec'on
Response
Metrics

59. Effective
Agility Learning

60. Over
the
last
24
months

61. Effective
Design Development

62. Over
the
last
24
months

63. Optimize
Cost of Risk

64. Over
the
last
24
months

65. Accountability
Responsibility

66. The End

67. Summary:
The Ten Dimensions of
Cyber Security Performance!

68. Actors
Systems
The
Organiza7on
Events
Context

69. Actors
Systems
1.
Exposure
Events
Dimension 1:
Optimize Exposure

70. Actors
Systems
1.
Exposure
2.
Threats
Events
Dimension 2:
Effective Threat
Intelligence

71. Actors
Systems
1.
Exposure
3.
Design
Dev.
2.
Threats
Events
Dimension 3:
Effective Design
Development

72. Actors
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
Protec'ons
Controls
Events
Dimension 4:
Quality of Protection
Controls

73. Actors
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
Execu'on
Opera'ons
Events
Dimension 5:
Effective/Efficient
Execution Operations

74. Events
Actors
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
6.
Response,
Recovery
Resilience
Dimension 6:
Effective Response,
Recovery Resilience

75. Opera7onal
Cyber
Security
Dimensions 1 – 6
Measure Core
Performance
Events
Actors
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
6.
Response,
Recovery
Resilience

76. First
Loop
Learning
“First Loop Learning”
is Continuous Improvement
in Daily Operations

77. Events
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
Actors
7.
External
Engagement
The
Organiza7on
Other
Organiza7ons
Government
Law
Enforcement
Dimension 7:
Effective External
Engagement
6.
Response,
Recovery
Resilience

78. Events
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
Actors
7.
External
Engagement
Other
Organiza7ons
Government
Law
Enforcement
8.
Agility
Learning
Dimension 8:
Effective Agility
Learning
6.
Response,
Recovery
Resilience

79. Events
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
Actors
7.
External
Engagement
8.
Agility
Learning
9.
Total
Cost
of
Risk
Other
Organiza7ons
Government
Law
Enforcement
Dimension 9:
Optimize
Total Cost of Risk
6.
Response,
Recovery
Resilience

80. Events
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
Actors
7.
External
Engagement
Total
Cost
of
Risk
10.
Accountability
Responsibility
Stakeholders
9.
Total
Cost
of
Risk
8.
Agility
Learning
Other
Organiza7ons
Government
Law
Enforcement
Dimension 10:
Accountability
Responsibility
6.
Response,
Recovery
Resilience

81. Dynamic
Capabili7es
Dimensions 7 – 10
Measure Systemic
Agility
Events
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
ProtecDons
Controls
5.
ExecuDon
OperaDons
Actors
Total
Cost
of
Risk
10.
Accountability
Responsibility
Stakeholders
9.
Total
Cost
of
Risk
8.
Agility
Learning
Other
Organiza7ons
Government
Law
Enforcement
7.
External
Engagement
6.
Response,
Recovery
Resilience

82. Second
Loop
Learning
“Second Loop Learning”
is Innovation
and Reinvention*
*
Individual
and
CollecDve

83. Events
Systems
1.
Exposure
2.
Threats
3.
Design
Dev.
4.
Protec'ons
Controls
5.
Execu'on
Opera'ons
Actors
7.
External
Engagement
Stakeholders
10.
Accountability
Responsibility
9.
Total
Cost
of
Risk
8.
Agility
Learning
Other
Organiza7ons
Government
Law
Enforcement
Ten Dimensions of
Cyber Security
Performance
6.
Response,
Recovery
Resilience

84. Last thought…!

85. “Can’t you make it simpler?”!

86. “Can’t you make it simpler?”!
“We need a crayon version
for executives and other
business and policy types”!

87. Sure!

88. Sure!
• “Transcendental numbers hurt my head”!

89. Sure!
• “Transcendental numbers hurt my head”!
• Declare π = 3.0!

90. Sure!
• “Transcendental numbers hurt my head”!
• Declare π = 3.0!
• But we lose something essential!
“Circle”

91. russell.thomas@meritology.com
http://exploringpossibilityspace.blogspot.com/
@MrMeritology!