In July 2010, BC Hydro, the electric utility and grid operator of British Columbia began implementation of its AMI program, formally known as the Smart Meter & Infrastructure (SMI) program. The SMI program transformed BC Hydro from a traditional metering utility to a smart metering utility by implementing smart meters on the customer service points. It was the first step in the smart grid transformation. The SMI program required the introduction of many new devices and applications into BC Hydro’s infrastructure. Some of these had never been deployed before anywhere in the world. Many were field deployed, outside of BC Hydro’s physical security perimeter. The SMI Security Delivery Team was formed to deliver on these commitments and to take responsibility for the end to end security of the SMI program. The Team implemented a multi-pronged approach to securing SMI including security risk assessments, security penetration testing by the team, design reviews, whole project risk assessments and third party security penetration testing. A standards based approach was required to ground the test plan both in best practice and in a common set of principles that BC Hydro and its vendors could accept. The Advanced Metering Infrastructure (AMI) Risk Assessment document prepared by the Advanced Metering Infrastructure Security (AMI-SEC) Task Force was used as a basis for the test plan. This document has since been passed to the National Institute of Standards and Technology (NIST) Cyber Security Working Group and was integrated into NIST IR 7628. NIST IR 7628 contains a comprehensive list of possible threats to AMI systems. The program was highly successful. Test results informed BC Hydro’s deployment decisions and allowed the manufacturers to improve their products. Lessons were learned about how best to conduct third party security testing. A full lessons learned section is included in the presentation.

1. Third Party Security Testing
for
Advanced Metering
Infrastructure Projects
Steve Vandenberg
and
Robert Hawk

2. Disclaimer
Please note that the observations and
commentary presented here are based on
industry experience and do not represent
information or positions specific to any project,
utility, its vendors or partners.

3. BC Hydro’s AMI Deployment
• BC is larger than CA, OR and WA put together
• BC Hydro SMI Project - CAD $2 Billion
• 1.8 million meters
• Thousands of Field Area Routers (FAR)
• Itron Meters, Cisco Network (IPv6)
• Started deployment 2011
• Meter to Cash, Energy & Theft Analytics
• Southwest Research Institute (SwRI) chosen as
third party test lab through RFP

4. Example AMI Hacks
• Looking into the eye of the meter, (Optical
Port Hack) Don C. Weber, InGuardians, Defcon, 2012
• Hacking For Privacy Discovery 28C3, (RF Hack)
Dario Carluccio and Stephan Brinkhaus, 28th Chaos
Communication Congress 2011
• Puerto Rico Utility electricity theft via Smart
Meter, (Optical Port Hack) US FBI / Krebs on Security,
12 April 2009
• We are so screwed, (Connected Grid Router
Hack) Larry Pesce, SANS, Defcon 21 Comedy Jam, 2013

5. AMI Projects vs IT Projects
IT
• 3 to 5 Year Lifespan
• 1 to 3 Fiscal Quarters
for Deployment
• 10’s of servers
• 100’s of network
devices
• Thousands of end
nodes
AMI
• 50+ Year Lifespan
• 2 to 5 Year Deployment
• 100’s of servers
• Thousands of network
devices
• Millions of end nodes

6. Security Test Scope
• Home Area Network (HAN) interface
• Smart Meter ZigBee network
• Smart Meters
• Optical Port Interface of the Smart Meter
• RFLAN Range Extenders
• RFLAN Smart Meter mesh network
• Field Area Routers (FAR)
• Smart Meter Data Collection Systems
• Software Releases (Versions & Patches)

7. Security Test Plans
• Common set of principles the Utility, vendors
and service providers would accept
• Standards based approach: AMI-Sec & NIST IR
7628
• Standards were developed using risk
assessment methodology with interfaces and
controls being the primary focal points
• A standards based risk rating modified by use
case for each test case allowed prioritization
of security testing
Note: Be prepared to change test plans and priorities
during test cycle in response to results

8. Develop Test Plans Beforehand
• Create use case documents to shape the test
plan
• Do a security assessment of the product to
shape the test plan
• Tie the test plan back to a recognized standard
e.g. NIST
• Consider all threat vectors – cyber, physical,
social

9. Threat Vectors
• Physical – Magnetic attack, Meter / Router
Tampering
• Cyber – RF Denial of Service, Optical Probe
Hack, Zigbee or DNP3 Hack
• Social – Current meter status info or improper
disconnect from call center
• Combination – HAN device association to
target home/business, SD Card Hack
Note: Turn off the power, mess with billing, know when
people are home and what they’re doing…

10. Before Starting…
• Get results of security testing carried out by
the manufacturer or others
• Eliminate redundant, expensive testing
• Shape the test plans
• Pick up where other testing left off
• Further explore their findings

11. Ensure Production-Like Environment
• You are spending a lot of money on these
results, be sure they are valid
• Production-like equipment is not enough,
Production-like configuration is needed
• Document any differences between
environments and discuss with the vendor
and implementation team how these
differences will impact results
• Who will build the test environment? Who
will troubleshoot it? Who will maintain it?

12. Support Structure for Security Testing
• Third party testing needs substantial support from
the manufacturer
• Test environment construction and maintenance
• Test environment troubleshooting
• Defect triage – vetting of false positives
• Defect resolution
• Information for white box testing
• Engage the manufacturer and test lab beforehand to
form a common understanding
• Make this part of the contract

13. Handling the Results
• Confidentiality vs Availability – results are
there to be used
• How will the information be handled?
• Document Control Systems…
• Encryption? Password Management ? Document
Repositories?
• Who will see it and how?
• Know that you are responsible for the
information. Information leakage could
damage:
• The public, The utility, The manufacturer, Other
users of the product

14. Using the Results
• How will defects be mitigated?
• What conditions if any would be applicable for
deployment?
• What findings would change the use case?
• What will the criteria be for halting
deployment ?

15. Setting an Achievable Goal
• What does done look like?
• All test findings will point to need for more
testing…
• Where to stop?

16. Handling Transition to Production
• After deployment is over, patching and new
versions of applications and devices will occur
• What if some defects survive the project?
How to manage them to closure? Retesting
challenges. What evidence is acceptable for
closure?
• Will the third party test program continue?
• Maintaining and upgrading the security test
environment
• Location of the test environment?

17. Questions ?
Steve Vandenberg
http://www.linkedin.com/in/vandenbergsteve
Robert Hawk
http://www.linkedin.com/in/ironmanrbh