SESSION ID:
#RSAC
Andrew Storms
How Security can be the Next
Force Multiplier in DevOps
ASD-F01
VP, Security Services
New Context
@St0rmz
#RSAC
Make security the reason for DevOps adoption
u  Software development challenges
u  DevOps doesn’t address secure coding challenges
u  Its our duty to affect change in DevOps
u  Security embedded in DevOps, makes DevOps better
u  Don’t fear DevOps – Know the people, processes and tools
u  Find your positive entry points
u  Making a plan
2
#RSAC
Software Development Challenges
3
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
PM Dev QA
Release
Mgmt
Ops
Ops
Security
u  Non DevOps software development environment
u  Everything is separate
Process Step
Owner
#RSAC
Software Development Challenges
4
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
PM Dev QA
Release
Mgmt
Ops
Ops
Security
Time To Market Changing Requirements
Tech Debt Control Costs
Risk ReductionReporting
u  Downward business pressures
Process Step
Owner
#RSAC
Software Development Challenges
5
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
PM Dev QA
Release
Mgmt
Ops
Ops
Security
Time To Market Changing Requirements
Tech Debt Control Costs
Risk Reduction
Threat Mgmt Risk Reduction
Reporting
u  Upward security pressures
Process Step
Owner
#RSAC
Software Development Challenges
6
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
PM Dev QA
Release
Mgmt
Ops
Ops
Security
Governance
Policy
Audit
Compliance
Time To Market Changing Requirements
Tech Debt Control Costs
Risk Reduction
Threat Mgmt Risk Reduction
Reporting
Program Management
Business &
Product
Security &
Compliance
Software Dev
Pressure
Pressure
#RSAC
Software Development Challenges
7
u  External pressures
u  Disjointed
u  Costly
u  Siloed
u  Opaque
u  Complex
u  Always late, out of sync, fragile
#RSAC
Then along came the DevOps
Non DevOps
u  Disjointed
u  Costly
u  Opaque
u  Always late
DevOps
u  Conjoined
u  Lean
u  Transparent
u  Agile
8
#RSAC
Then along came the DevOps
9
Governance
Compliance
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
DevOps
Ops
Security
Policy
Audit
Time To Market Changing Requirements
Tech Debt Control Costs
Risk Reduction
Threat Mgmt Risk Reduction
Reporting
Agile
Green = DevOps
#RSAC
Then along came the DevOps
u  Meets business & product needs
u  On time within budget
u  Meets ops and dev needs
u  Agile, harmonious, consistent
u  Fails to meet security needs
u  No attempt to deliver secure application code
u  Security still left out and left last
10
#RSAC
Then along came the DevOps
11
Governance
Compliance
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
DevOps
Ops
Security
Policy
Audit
Time To Market Changing Requirements
Tech Debt Control Costs
Risk Reduction
Threat Mgmt Risk Reduction
Reporting
Agile
Business &
Product needs
Operational &
Dev needs
#RSAC
Then along came the DevOps
12
Governance
Compliance
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
DevOps
Ops
Security
Policy
Audit
Time To Market Changing Requirements
Tech Debt Control Costs
Risk Reduction
Threat Mgmt Risk Reduction
Reporting
Agile
Business &
Product needs
Operational &
Dev needs
Security, Audit
Compliance
Needs Unmet
#RSAC
Then along came the DevOps
13
Governance
Compliance
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
DevOps
Ops
Security
Policy
Audit
Time To Market Changing Requirements
Tech Debt Control Costs
Risk Reduction
Threat Mgmt Risk Reduction
Reporting
Agile
Pressure
Pressure
#RSAC
How popular is DevOps?
u  Oct 2014 CA Technologies Survey
u  88% respondents already have or plan to adopt DevOps in the next 5
years. (up from 66% on prior year)
u  Top obstacle (28%) to DevOps in their organization were security or
compliance concerns
u  Oct 2014 Rackspace Survey
u  55% already implemented DevOps. 31% planning to implement
DevOps within 3 years.
u  Primary driver for DevOps? Only 2% said audit or compliance
14
http://rewrite.ca.com/us/articles/devops/research-report--devops-the-worst-kept-secret-to-winning-in-the-application-economy.aspx
http://www.rackspace.co.uk/sites/default/files/devops-automation-report.pdf
#RSAC
DevOps Kicks The Security Can Down The Road
15
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
DevOps
Ops
Security
PM Dev QA
Release
Mgmt
Ops
Ops
SecurityOld Way
DevOps Way
Security is still
the last guy
#RSAC
DevOps Is Bad For Security
u  Fast
u  ~50 deploys a day!
u  Faster to production = faster to be pwned
u  Too much complexity
u  Unwieldy
u  Everyone has access to everything
u  Full stack engineers
u  Fewer test cases
u  Deplorable
u  No audit
u  No control points
u  No process
16
#RSAC
DevOps Is Good For Security
u  Increases process insertion points
u  Increases consistency
u  Increases predictability
u  Decreases time to change
u  Increases audit ability
u  Reduces costs
u  Reduces waste
17
Simple	
   Manageable	
  
Automatable	
   Testable	
  
#RSAC
Security Is Good For DevOps
u  Business enabler
u  Transparency
u  Trust
u  Protects privacy
u  Accountability
u  Regulatory & audit
18
Security
DevOps
Let the people focus on their core competencies
#RSAC
Know Your Nemesis
Security Team
u  Compliance
u  Silos
u  Change control
u  FUD masters
DevOps Teams
u  Security != compliance
u  Open
u  Lots of change
u  Data scientists
19
“If you know the enemy and know yourself, you need not fear the result of a hundred battles.
If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If
you know neither the enemy nor yourself, you will succumb in every battle.” – Sun Tzu
#RSAC
How do we get these teams to work together?
(Every DevOps presentation must have random gears image)
20
#RSAC
Action Plan
u  Pipeline
u  Tools
u  Processes
u  Today’s todos
21
Long	
  	
  term	
  
Short	
  term	
  
Know	
  your	
  DevOps	
  
#RSAC
Apply Security Expertise to DevOps Pipeline
22
InstrumentationLogAnalysis
Logging
Functional
Tests
Security
Tests
Other
Tests
statsd
Jenkins
App Code Inf Code Templates
Dev
Git Chef
Stage Prod
#RSAC
Security Makes DevOps Better - Tools
u  Git (Source Code Management)
u  Make it the source of truth for everything
u  Sometimes people use Chef for revision control
u  Separate repositories for each cookbook
u  Branching strategy needs to support isolation, rollback, logging
u  Git Hooks
u  Enforce policy at commit time
u  Commit message, additional logging
23
#RSAC
Security Makes DevOps Better - Tools
u  Chef (IT Automation)
u  Continuous configuration & compliance
u  Write some code!
u  Map security controls to recipes
u  Apply technical controls. Ex: https://cipherli.st/
u  Add logging
u  Reduces complexity and helps out everyone
u  Ensures consistency (dev, stage, prod)
u  Makes audits easier (most of the time)
24
#RSAC
Security Makes DevOps Better - Tools
u  Jenkins (Continuous Integration)
u  Automated code security test suites
u  Gauntlt (Ruby), Mittn (Python), BDD-Security (Java)
u  Infrastructure code too
u  Chefspec, test-kitchen
u  External security systems orchestration
u  Network scanners, fuzzers, sqlmappers
u  Test security policies and controls
u  No pass = no go
25
#RSAC
Security Makes DevOps Better - Tools
u  Instrumentation
u  Business logic metrics also good for security
u  Number failed logins in last 24 hours
u  Site performance & availability
u  How do you measure risk management in DevOps?
u  Benchmarking
u  Security test coverage
u  Time to audit
u  Mean time to remediate
26
#RSAC
Security Makes DevOps Better - Tools
u  Monitoring
u  New Relic, PagerDuty, Boundry, Pingdom
u  Performance & availability
u  Create useful alerts and alert the right people
u  Logging
u  Splunk, SumoLogic
u  Get your app team to log useful events
u  “There was an error”
u  “RabbitMQ tried to write to DB, but got error…”
27
#RSAC
Apply Security Expertise to DevOps Process
28
Plan
Code
Test
Release
Deploy
Operate
#RSAC
Security Makes DevOps Better - Process
u  Policy
u  Does your SDLC include DevOps tools and process?
u  Definition of done
u  How do devs know they are meeting security requirements?
u  Moving security earlier
u  Story review
u  Threat vector analysis
u  Security training
u  Design & architecture
29
Plan
Moving	
  security	
  leA	
  
Plan	
   Code	
   Test	
   Release	
   Deploy	
   Operate	
  
#RSAC
Security Makes DevOps Better - Process
Standards Enforcement
u  Lint checkers
u  Branching strategy
u  Peer review
Get Involved
u  Write code
u  Attend stand ups
u  Peer review
u  Pair programming
30
Code
Security experts can’t expect software experts to be security experts.
#RSAC
Security Makes DevOps Better - Process
Security Tests
u  Behaviors
u  Lock the user out after x failures
u  Must use SHA-256
u  Infrastructure
u  Port scans
u  User accounts
Non Functional Tests
u  Performance (Availability)
u  System readiness
u  Deploying using latest AMI
u  Latest OpenSSL
31
Test
Functional
Tests
Security
Tests
Other
Tests
#RSAC
Security Makes DevOps Better - Process
u  Make tests automated
u  Continuous integration with Jenkins
u  Pick a pluggable framework
u  Use TDD
u  Automate security tests up front
u  Done-Done includes security
u  What’s the definition of done?
32
Test
Functional
Tests
Security
Tests
Other
Tests
#RSAC
Security Makes DevOps Better - Process
Release
u  Separation
u  Systems
u  Duties
u  “Here be dragons”
u  Oversight
u  Approvals
u  2-man rule
Deploy
u  Change control mgmt
u  “Here be more dragons”
u  Convey assurance
u  Convey trust
u  What’s in the change log?
u  What tests were run?
33
Release
Deploy
#RSAC
What You Can Do Today
u  Get acquainted with popular tools
u  Git, Jenkins, Chef, Statsd, New Relic, PagerDuty
u  Read about new concepts
u  Agile, continuous integration, continuous deployment
u  Test driven development
u  Think about metrics
u  What metrics are valuable to both DevOps & Security
u  Get involved
34
#RSAC
u  Security people are secretive
u  DevOps people LOVE to talk and SHARE
u  Watch some videos on YouTube
u  Attend a DevOps conference
u  Read some articles at devops.com
Do Some Industry Research
35
#RSAC
Remember To
u  Be transparent
u  Good security is always transparent. DevOps will amplify opaqueness.
u  Be measurable
u  DevOps breeds automation. Find where you can automate metrics.
u  Embrace feedback loops
u  Attend retrospectives. Request feedback. Adjust as needed.
u  Embrace iterations
u  Nothing is ever 100% done or 100% perfect.
36
#RSAC
Make DevOps Work For You
DevOps Says
u  Collaboration
u  Automation
u  Agile
Security Says
u  Everyone’s responsibility
u  Standards, reporting, benchmarks
u  Risk management
37
Use DevOps to create the next generation information security program.
It might just be your only hope in combating the next cyber threat.
#RSAC
Make DevOps Work For You
Self	
  Study	
  
Discovery	
  
Plan	
  Measure	
  
Feedback	
  
38
Today’s Assignment
Reading: Etsy, NetFlix
Next Week
What tools, people and
processes are in use?
Next Month
How can you impact DevOps
in a positive way?
3 Months
Have you made an impact?
3 Months
What can you do better?
#RSAC
Summary
u  For many, Security is the after thought in DevOps
u  Its your duty to affect change in DevOps
u  Security embedded in DevOps, makes DevOps better
u  Get to know the people, processes and tools
u  Find your positive entry points
u  Make a plan & measure the outcome
39
Gears	
  
More	
  
Needs	
  
#RSAC
Q & A
Andrew Storms
@St0rmz
storms@newcontext.com
Devops.com
40

How Security can be the Next Force Multiplier in DevOps

  • 1.
    SESSION ID: #RSAC Andrew Storms HowSecurity can be the Next Force Multiplier in DevOps ASD-F01 VP, Security Services New Context @St0rmz
  • 2.
    #RSAC Make security thereason for DevOps adoption u  Software development challenges u  DevOps doesn’t address secure coding challenges u  Its our duty to affect change in DevOps u  Security embedded in DevOps, makes DevOps better u  Don’t fear DevOps – Know the people, processes and tools u  Find your positive entry points u  Making a plan 2
  • 3.
    #RSAC Software Development Challenges 3 Plan   Code   Test   Release   Deploy   Operate   PM Dev QA Release Mgmt Ops Ops Security u  Non DevOps software development environment u  Everything is separate Process Step Owner
  • 4.
    #RSAC Software Development Challenges 4 Plan   Code   Test   Release   Deploy   Operate   PM Dev QA Release Mgmt Ops Ops Security Time To Market Changing Requirements Tech Debt Control Costs Risk ReductionReporting u  Downward business pressures Process Step Owner
  • 5.
    #RSAC Software Development Challenges 5 Plan   Code   Test   Release   Deploy   Operate   PM Dev QA Release Mgmt Ops Ops Security Time To Market Changing Requirements Tech Debt Control Costs Risk Reduction Threat Mgmt Risk Reduction Reporting u  Upward security pressures Process Step Owner
  • 6.
    #RSAC Software Development Challenges 6 Plan   Code   Test   Release   Deploy   Operate   PM Dev QA Release Mgmt Ops Ops Security Governance Policy Audit Compliance Time To Market Changing Requirements Tech Debt Control Costs Risk Reduction Threat Mgmt Risk Reduction Reporting Program Management Business & Product Security & Compliance Software Dev Pressure Pressure
  • 7.
    #RSAC Software Development Challenges 7 u External pressures u  Disjointed u  Costly u  Siloed u  Opaque u  Complex u  Always late, out of sync, fragile
  • 8.
    #RSAC Then along camethe DevOps Non DevOps u  Disjointed u  Costly u  Opaque u  Always late DevOps u  Conjoined u  Lean u  Transparent u  Agile 8
  • 9.
    #RSAC Then along camethe DevOps 9 Governance Compliance Plan   Code   Test   Release   Deploy   Operate   DevOps Ops Security Policy Audit Time To Market Changing Requirements Tech Debt Control Costs Risk Reduction Threat Mgmt Risk Reduction Reporting Agile Green = DevOps
  • 10.
    #RSAC Then along camethe DevOps u  Meets business & product needs u  On time within budget u  Meets ops and dev needs u  Agile, harmonious, consistent u  Fails to meet security needs u  No attempt to deliver secure application code u  Security still left out and left last 10
  • 11.
    #RSAC Then along camethe DevOps 11 Governance Compliance Plan   Code   Test   Release   Deploy   Operate   DevOps Ops Security Policy Audit Time To Market Changing Requirements Tech Debt Control Costs Risk Reduction Threat Mgmt Risk Reduction Reporting Agile Business & Product needs Operational & Dev needs
  • 12.
    #RSAC Then along camethe DevOps 12 Governance Compliance Plan   Code   Test   Release   Deploy   Operate   DevOps Ops Security Policy Audit Time To Market Changing Requirements Tech Debt Control Costs Risk Reduction Threat Mgmt Risk Reduction Reporting Agile Business & Product needs Operational & Dev needs Security, Audit Compliance Needs Unmet
  • 13.
    #RSAC Then along camethe DevOps 13 Governance Compliance Plan   Code   Test   Release   Deploy   Operate   DevOps Ops Security Policy Audit Time To Market Changing Requirements Tech Debt Control Costs Risk Reduction Threat Mgmt Risk Reduction Reporting Agile Pressure Pressure
  • 14.
    #RSAC How popular isDevOps? u  Oct 2014 CA Technologies Survey u  88% respondents already have or plan to adopt DevOps in the next 5 years. (up from 66% on prior year) u  Top obstacle (28%) to DevOps in their organization were security or compliance concerns u  Oct 2014 Rackspace Survey u  55% already implemented DevOps. 31% planning to implement DevOps within 3 years. u  Primary driver for DevOps? Only 2% said audit or compliance 14 http://rewrite.ca.com/us/articles/devops/research-report--devops-the-worst-kept-secret-to-winning-in-the-application-economy.aspx http://www.rackspace.co.uk/sites/default/files/devops-automation-report.pdf
  • 15.
    #RSAC DevOps Kicks TheSecurity Can Down The Road 15 Plan   Code   Test   Release   Deploy   Operate   DevOps Ops Security PM Dev QA Release Mgmt Ops Ops SecurityOld Way DevOps Way Security is still the last guy
  • 16.
    #RSAC DevOps Is BadFor Security u  Fast u  ~50 deploys a day! u  Faster to production = faster to be pwned u  Too much complexity u  Unwieldy u  Everyone has access to everything u  Full stack engineers u  Fewer test cases u  Deplorable u  No audit u  No control points u  No process 16
  • 17.
    #RSAC DevOps Is GoodFor Security u  Increases process insertion points u  Increases consistency u  Increases predictability u  Decreases time to change u  Increases audit ability u  Reduces costs u  Reduces waste 17 Simple   Manageable   Automatable   Testable  
  • 18.
    #RSAC Security Is GoodFor DevOps u  Business enabler u  Transparency u  Trust u  Protects privacy u  Accountability u  Regulatory & audit 18 Security DevOps Let the people focus on their core competencies
  • 19.
    #RSAC Know Your Nemesis SecurityTeam u  Compliance u  Silos u  Change control u  FUD masters DevOps Teams u  Security != compliance u  Open u  Lots of change u  Data scientists 19 “If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.” – Sun Tzu
  • 20.
    #RSAC How do weget these teams to work together? (Every DevOps presentation must have random gears image) 20
  • 21.
    #RSAC Action Plan u  Pipeline u Tools u  Processes u  Today’s todos 21 Long    term   Short  term   Know  your  DevOps  
  • 22.
    #RSAC Apply Security Expertiseto DevOps Pipeline 22 InstrumentationLogAnalysis Logging Functional Tests Security Tests Other Tests statsd Jenkins App Code Inf Code Templates Dev Git Chef Stage Prod
  • 23.
    #RSAC Security Makes DevOpsBetter - Tools u  Git (Source Code Management) u  Make it the source of truth for everything u  Sometimes people use Chef for revision control u  Separate repositories for each cookbook u  Branching strategy needs to support isolation, rollback, logging u  Git Hooks u  Enforce policy at commit time u  Commit message, additional logging 23
  • 24.
    #RSAC Security Makes DevOpsBetter - Tools u  Chef (IT Automation) u  Continuous configuration & compliance u  Write some code! u  Map security controls to recipes u  Apply technical controls. Ex: https://cipherli.st/ u  Add logging u  Reduces complexity and helps out everyone u  Ensures consistency (dev, stage, prod) u  Makes audits easier (most of the time) 24
  • 25.
    #RSAC Security Makes DevOpsBetter - Tools u  Jenkins (Continuous Integration) u  Automated code security test suites u  Gauntlt (Ruby), Mittn (Python), BDD-Security (Java) u  Infrastructure code too u  Chefspec, test-kitchen u  External security systems orchestration u  Network scanners, fuzzers, sqlmappers u  Test security policies and controls u  No pass = no go 25
  • 26.
    #RSAC Security Makes DevOpsBetter - Tools u  Instrumentation u  Business logic metrics also good for security u  Number failed logins in last 24 hours u  Site performance & availability u  How do you measure risk management in DevOps? u  Benchmarking u  Security test coverage u  Time to audit u  Mean time to remediate 26
  • 27.
    #RSAC Security Makes DevOpsBetter - Tools u  Monitoring u  New Relic, PagerDuty, Boundry, Pingdom u  Performance & availability u  Create useful alerts and alert the right people u  Logging u  Splunk, SumoLogic u  Get your app team to log useful events u  “There was an error” u  “RabbitMQ tried to write to DB, but got error…” 27
  • 28.
    #RSAC Apply Security Expertiseto DevOps Process 28 Plan Code Test Release Deploy Operate
  • 29.
    #RSAC Security Makes DevOpsBetter - Process u  Policy u  Does your SDLC include DevOps tools and process? u  Definition of done u  How do devs know they are meeting security requirements? u  Moving security earlier u  Story review u  Threat vector analysis u  Security training u  Design & architecture 29 Plan Moving  security  leA   Plan   Code   Test   Release   Deploy   Operate  
  • 30.
    #RSAC Security Makes DevOpsBetter - Process Standards Enforcement u  Lint checkers u  Branching strategy u  Peer review Get Involved u  Write code u  Attend stand ups u  Peer review u  Pair programming 30 Code Security experts can’t expect software experts to be security experts.
  • 31.
    #RSAC Security Makes DevOpsBetter - Process Security Tests u  Behaviors u  Lock the user out after x failures u  Must use SHA-256 u  Infrastructure u  Port scans u  User accounts Non Functional Tests u  Performance (Availability) u  System readiness u  Deploying using latest AMI u  Latest OpenSSL 31 Test Functional Tests Security Tests Other Tests
  • 32.
    #RSAC Security Makes DevOpsBetter - Process u  Make tests automated u  Continuous integration with Jenkins u  Pick a pluggable framework u  Use TDD u  Automate security tests up front u  Done-Done includes security u  What’s the definition of done? 32 Test Functional Tests Security Tests Other Tests
  • 33.
    #RSAC Security Makes DevOpsBetter - Process Release u  Separation u  Systems u  Duties u  “Here be dragons” u  Oversight u  Approvals u  2-man rule Deploy u  Change control mgmt u  “Here be more dragons” u  Convey assurance u  Convey trust u  What’s in the change log? u  What tests were run? 33 Release Deploy
  • 34.
    #RSAC What You CanDo Today u  Get acquainted with popular tools u  Git, Jenkins, Chef, Statsd, New Relic, PagerDuty u  Read about new concepts u  Agile, continuous integration, continuous deployment u  Test driven development u  Think about metrics u  What metrics are valuable to both DevOps & Security u  Get involved 34
  • 35.
    #RSAC u  Security peopleare secretive u  DevOps people LOVE to talk and SHARE u  Watch some videos on YouTube u  Attend a DevOps conference u  Read some articles at devops.com Do Some Industry Research 35
  • 36.
    #RSAC Remember To u  Betransparent u  Good security is always transparent. DevOps will amplify opaqueness. u  Be measurable u  DevOps breeds automation. Find where you can automate metrics. u  Embrace feedback loops u  Attend retrospectives. Request feedback. Adjust as needed. u  Embrace iterations u  Nothing is ever 100% done or 100% perfect. 36
  • 37.
    #RSAC Make DevOps WorkFor You DevOps Says u  Collaboration u  Automation u  Agile Security Says u  Everyone’s responsibility u  Standards, reporting, benchmarks u  Risk management 37 Use DevOps to create the next generation information security program. It might just be your only hope in combating the next cyber threat.
  • 38.
    #RSAC Make DevOps WorkFor You Self  Study   Discovery   Plan  Measure   Feedback   38 Today’s Assignment Reading: Etsy, NetFlix Next Week What tools, people and processes are in use? Next Month How can you impact DevOps in a positive way? 3 Months Have you made an impact? 3 Months What can you do better?
  • 39.
    #RSAC Summary u  For many,Security is the after thought in DevOps u  Its your duty to affect change in DevOps u  Security embedded in DevOps, makes DevOps better u  Get to know the people, processes and tools u  Find your positive entry points u  Make a plan & measure the outcome 39 Gears   More   Needs  
  • 40.
    #RSAC Q & A AndrewStorms @St0rmz storms@newcontext.com Devops.com 40