We will discuss the what, why and the how of running modern security operations. We will take a look at the pain points in a DevOps life cycle and see the benefits of pragmatic security solutions. Attendees will get an idea about where and how to start devsecops for secure devops pipeline. This talk is focused on the what, why and the how of running security operations in the modern world. The way attacks are changing and developers are moving ahead with the next generation technologies is blazingly fast. However, traditional operations still exist. It then becomes imperative to make changes in the way security operations should run to defend against attackers and work with developers and modern businesses. In this talk, we will see what are the real world problems faced by organisations, how we can rapidly adapt to changes by modifying the culture and methodologies while relying on processes, tools and techniques.

1. October 24, 2017
Modern Security Operations
aka
Secure DevOps
Madhu Akula
Automation Ninja @ Appsecco

2. October 24, 2017
About Me
• Automation Ninja at Appsecco
• Interested in Security, DevOps and Cloud
• Speaker & Trainer : Defcon, All Day DevOps, DevSecCon,
c0c0n, null, etc.
• Discovered security vulnerabilities in Google, Microsoft,
Yahoo, Adobe, etc.
• Never ending learner
• Follow me (or) Tweet to me @madhuakula

3. October 24, 2017
Modern Security Operations
• To improve collaboration between Developers, Operations
and Security
• Applying security into each phase of DevOps lifecycle
• Practice of developing and deploying safer software sooner
• Building secure defaults and following best practices
• Proactive monitoring & defence
• Performing redteam activities, before real attacks happen
• Learning & sharing with community

4. October 24, 2017
What is DevOps?
There are many definitions for this term.
I personally follow CAMS by Damon Edwards and John Willis
‘Implementing a culture of sharing between
Development and Operations’
● Culture
● Automation
● Measurement
● Sharing

5. October 24, 2017
DevOps lifecycle
Test MonitorDeployCodePlan

6. October 24, 2017
Let’s talk about some
DevOops highlights

7. October 24, 2017
Security Misconfiguration
Source: https://www.upguard.com/breaches/cloud-leak-accenture

8. October 24, 2017
Components with known security vulnerabilities
Source: https://github.com/blog/2447-a-more-connected-universe

9. October 24, 2017
Insecure Defaults
Source: https://hackernoon.com/capturing-all-the-flags-in-bsidessf-ctf-by-pwning-our-infrastructure-3570b99b4dd0

10. October 24, 2017
Secret keys in public github
Source: https://www.theregister.co.uk/2015/01/06/dev_blunder_shows_github_crawling_with_keyslurping_bots

11. October 24, 2017
Unauthorised access
Source: https://www.shodan.io/report/nlrw9g59

12. October 24, 2017

13. October 24, 2017
DevSecOps

14. October 24, 2017
What is DevSecops?

15. October 24, 2017
DevSecOps moto
“The purpose and intent of DevSecOps is to build on the
mindset that “everyone is responsible for security” with the goal
of safely distributing security decisions at speed and scale to
those who hold the highest level of context without sacrificing
the safety required.”
Source: http://www.devsecops.org/blog/2015/2/15/what-is-devsecops

16. October 24, 2017
Let’s fit security into
DevOps lifecycle

17. October 24, 2017
Thinking about security from the outset
“Companies that consider security from the start assess their
options and make reasonable choices based on the nature of
their business and the sensitivity of the information involved.
Threats to data may transform over time, but the fundamentals
of sound security remain constant.”
- Federal Trade Commision
Source: https://www.ftc.gov/system/files/documents/plain-language/pdf0205-startwithsecurity.pdf

18. October 24, 2017
Planning
● The most important phase of development is planning
● Involve all parties (Dev, Sec, Ops) from the beginning, it will
enable everyone to understand and speed up things without
compromising quality & security
● Build things with the mindset of secure defaults with built-in
security

19. October 24, 2017
Planning
● This applies to all teams
○ Developers need to think about secure coding best practices,
using secure libraries and keeping up to date with latest
vulnerabilities.
○ Operations teams need to be aware of technology specific
security configurations, best practices and hardening
guidelines.
○ Security teams have to understand the workflow, create
suitable standards and apply them throughout the lifecycle

20. October 24, 2017
● Version control gives the power of moving traditional
operations to modern DevOps shops
● Managing things will be super-easy and efficient
● This way everything can be audited, tracked and can be
rolled back if required
Version control

21. October 24, 2017
● Some of the things that can be version controlled include
○ Documentation, knowledge bases, etc
○ Developers’ code
○ Op’s configurations and playbooks
○ Custom scripts and snippets
○ Many more...
Version control

22. October 24, 2017
● Infrastructure as code aims to make operations more
efficient and remove human errors
● By doing this, we can achieve
○ Version controlled and codified versions of secure
infrastructure
○ We can perform continuous integration with the deployment
process
○ We can improve the inventory by building Configuration
Management DataBases (CMDB)
Infrastructure as code

23. October 24, 2017
● This requires process and tools
○ Identifying the all manual repetitive tasks and structuring
them for automation
○ With tools like Ansible, Chef, Terraform, etc.
● We can validate our infrastructure as code against security &
compliance
● We can create security playbooks for hardening & patching
Infrastructure as code

24. October 24, 2017
Ansible playbook snippet for MySQL hardening
- name: Secures the MySQL root user
mysql_user:
user: root
password: "{{ mysql_root_password }}"
host: "{{ item }}"
login_password: "{{ mysql_root_password }}"
login_user: root
with_items:
- 127.0.0.1
- localhost
- ::1
- "{{ ansible_fqdn }}"
- name: Removes the MySQL test database
mysql_db:
db: test
state: absent
login_password: "{{ mysql_root_password }}"
login_user: root

25. October 24, 2017
● The practice of integrating work frequently, which requires
quick verification to process next steps using automated
build processes
● In this phase you include your test cases and security checks,
performing them before going to production
● It allows us to integrate existing tool sets using web hooks
and plugins into the build process
Continuous everything

26. October 24, 2017
● Ensures that the quality of the code and configurations
remains the same by using automated test cases and
validation checks
● This requires defining the steps each team needs to perform
to speed up the delivery process without compromising
security
Continuous everything

27. October 24, 2017
● Deployment is the phase where things are made live; aka
production
● Using a standard baseline-OS and containers, which reduces
the level of security risk
● Hardening configuration and environments with best
practice and against known vulnerabilities (Using your
security playbooks)
Secure deployments

28. October 24, 2017
● Managing secrets and data is a key part while deploying to
production, use secure communication channels and storage
like Vault
● Verify deployments by running security scans against them
for misconfigurations
● Also using modern tool-set like Moby project, LinuxKit, etc.
for docker containers
Secure deployments

29. October 24, 2017
● To make an important decisions (or) to troubleshoot things,
monitoring is the place to start
● Monitoring needs to apply to every phase of the DevOps
lifecycle
● Health checks of applications & infrastructure to know how
things are going
● Security monitoring of applications, servers, network devices
Proactive monitoring & alerting

30. October 24, 2017
● Alerting based on thresholds and attack anomalies
● Fine-tuning and improving the alerting system gives more
control
● Automating actions against known repetitive alerts can be
efficient, but take care
Proactive monitoring & alerting

31. October 24, 2017
● Define baseline security
○ Test against it
○ And run tests continuously
● Drive testing from the DevOps pipeline
● Never deploy sub-standard code
● Requires tests to be passed in order to deploy into
production
● Empower DevOps teams to fix issues
● Apply feedback loops
Test driven security
Source: https://docs.google.com/presentation/d/1H0Ym0bJ4TgVz7BRkoeun7ndGuTIteVyFGDDxguXmff0

32. October 24, 2017
● This requires you to have proactive monitoring in place,
which includes building centralised logging and monitoring
systems
● Build your defences from an offensive mindset and start by
focusing on your critical infrastructure
● Enable DevOps teams to better understand and identify
what security attacks look like by red teaming (we can also
use this log data to train defence systems)
Attack driven defence
Source: https://www.slideshare.net/zanelackey/attackdriven-defense

33. October 24, 2017
● Identify patterns and anomalies for alerting and take action
against them using automated defence
● Apply data science and machine learning techniques for data
sets
● Build defence systems with real attack data and defend like
an attacker
Attack driven defence
Source: https://www.slideshare.net/zanelackey/attackdriven-defense

34. October 24, 2017
● Clear communication enables us to be more productive
● Collaboration between teams makes things faster. It should
start from outset!
● Break requirements into actionable items and assign them
to respective teams
● Eliminate the barriers between Devs, Ops and Security
teams and work towards a DevSecOps approach (everyone
is responsible for security)
Communication & collaboration

35. October 24, 2017
● Use task and project management tools for collaboration,
this will help showcase dependencies between teams
● Spread awareness of different roles and skills by conducting
social events; learning by lunch, etc.
Communication & collaboration

36. October 24, 2017
Training people
Training developers and operations about how attackers work,
by using vulnerable labs and applications, will give them a better
understanding
OWASP Vulnerable Web Applications Directory Project

37. October 24, 2017
● We must learn from each other, the best way to do this is
sharing with others
○ For example, security teams can write a playbook to harden
infrastructure to meet the policies and standards rather
pointing out that it’s an ops issue
● Rather than working as a big teams, we can mix the different
teams into smaller groups and work together to achieve
great results
Culture & innovation

38. October 24, 2017
● Simplicity, documentation and clear communication is a
win-win
● When things go wrong transparency and open contributions
is vital
● Attending conferences and meetups and being part of the
community helps us to know how the world is doing things
differently to us
Culture & innovation

39. October 24, 2017
● Fail fast and early, so there is less cost and damage for the
business
● Maintain secure backups and validate the restore process
● Test for resiliency and recoverability using tools like chaos
monkey and security monkey
● Conduct internal hackathons and bug bounty programs
● Perform redteam activities, simulate how real attacks
happen
Think about failures, before they occur

40. October 24, 2017
Wardly maps for DevSecOps
Source: https://github.com/devsecops/wardley-maps

41. October 24, 2017
Demo Time
Code to Production

42. October 24, 2017
https://www.youtube.com/watch?v=y9Usd0Q2Il0

43. October 24, 2017
What did we see?

44. October 24, 2017
Takeaways
● Everyone is responsible for security (Dev + Sec + Ops)
● Clear communication, active collaboration is key to success
● Build with secure defaults mindset
● Test driven development & Attack driven defence
● Hack your applications, infra, etc. like real attackers
● Keep learning and sharing

45. October 24, 2017
References
● https://www.devsecops.org
● http://www.devseccon.com/devsecops-whitepaper
● https://pages.cloudpassage.com/sans-a-devsecops-playboo
k.html
● https://devops-security-checklist.sqreen.io

46. October 24, 2017
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle

47. October 24, 2017
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle

48. October 24, 2017
bit.ly/addo-slack
Find me on slack, right now!

49. October 24, 2017
Thank You
@madhuakula | @appseccouk