Devops, Secops, OpSecs, Devsec , my 2016 Unicom Devops Summit Brussels.

1. Devops, Secops, Opsec,Devops, Secops, Opsec,
DevSec *ops *.* ?DevSec *ops *.* ?
Kris Buytaert
Brussels , February 2016

2. Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● Even did Security (OSSTM)Even did Security (OSSTM)
● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source
Consultant @inuits.euConsultant @inuits.eu
● Everything is an effing DNS ProblemEverything is an effing DNS Problem
● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore
● Some books, some papers, some blogsSome books, some papers, some blogs
● Too many conferences.Too many conferences.

3. Who is runningWho is running
● DrupalDrupal
● OpenSSLOpenSSL
● BashBash

4. Who is runningWho is running
● Drupal < 7.38Drupal < 7.38
● OpenSSL 1.0.1 → 1.0.1fOpenSSL 1.0.1 → 1.0.1f
● Bash < 4.3...Bash < 4.3...

5. Who has them upgraded over the pastWho has them upgraded over the past
12 months ?12 months ?

6. What's this Devops thing really about ?What's this Devops thing really about ?

7. World , 200X-2009World , 200X-2009
Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, JezzPatrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz
Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, andHumble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and
lots of others ..lots of others ..
Gent , October 2009Gent , October 2009
Mountain View , June 2010Mountain View , June 2010
Hamburg , October 2010Hamburg , October 2010
Boston, March 2011Boston, March 2011
Mountain View, June 2011Mountain View, June 2011
Bangalore, Melbourne,Bangalore, Melbourne,
Goteborg , October 2011Goteborg , October 2011

8. C(L)AMSC(L)AMS
● CultureCulture
● (Lean)(Lean)
● AutomationAutomation
● MeasurementMeasurement
● SecuritySecurity
Damon Edwards and John WillisDamon Edwards and John Willis

9. Frank BreedijkFrank Breedijk
@seccubus@seccubus
● Http → httpsHttp → https
● Imap → imapsImap → imaps
● Pop3 → pop3sPop3 → pop3s
● Devop → devopSDevop → devopS

10. ““DevOps is a cultural andDevOps is a cultural and
professional movement”professional movement”
Adam JacobAdam Jacob

11. How did we get here ?How did we get here ?

12. The Old DaysThe Old Days
● ““Put this Code Live, here's a tarball” NOW!Put this Code Live, here's a tarball” NOW!
● What dependencies ?What dependencies ?
● No machines available ?No machines available ?
● What database ?What database ?
● Security ?Security ?
● High Availability ?High Availability ?
● Scalability ?Scalability ?
● My computer can't install this ?My computer can't install this ?

13. Devs vs OpsDevs vs Ops

14. People hate SysadminsPeople hate Sysadmins
BecauseBecause
•They slow stuff downThey slow stuff down
•The say noThe say no
•They say no againThey say no again
•They refuse to break stuffThey refuse to break stuff
•They care about uptimeThey care about uptime
•They don't care about fancy new featuresThey don't care about fancy new features

16. People hate SecurityPeople hate Security
OfficersOfficers
BecauseBecause
•They slow stuff downThey slow stuff down
•The say noThe say no
•They say no againThey say no again
•They refuse to leave holes openThey refuse to leave holes open
•They care about securityThey care about security
•They don't care about fancy new featuresThey don't care about fancy new features
•Security Officers have an expiry dateSecurity Officers have an expiry date

18. 10 days into operation10 days into operation
● What High Load ? What Memory usage ?What High Load ? What Memory usage ?
● Are these Logs ? Or this is actualy customerAre these Logs ? Or this is actualy customer
data ?data ?
● How many users are there , should they launchHow many users are there , should they launch
100 queries each ?? Oh we're having 10K100 queries each ?? Oh we're having 10K
usersusers
● Why is debugging enabled ?Why is debugging enabled ?
● Who wrote this ?Who wrote this ?

19. 11 days into operations11 days into operations

20. 12 days into operations12 days into operations

21. 13 days into operations13 days into operations

22. We can solve this !We can solve this !
● We are not here toWe are not here to
blockblock
● Some people thinkSome people think
the Security /the Security /
Operations workOperations work
starts on deploymentstarts on deployment
● It starts much earlierIt starts much earlier
● Start talking asapStart talking asap

23. Culture,Culture,
automation,automation,
Measturement,Measturement,
sharingsharing

24. Breaking the SilosBreaking the Silos
Getting AlongGetting AlongOpsOpsDevsDevs

25. ● Who is in charge of security ?Who is in charge of security ?
● What do your developers think about security ?What do your developers think about security ?
● When do you think about securityWhen do you think about security
● The problem with security is it doesn'tThe problem with security is it doesn't
generate revenuegenerate revenue
● Security needs to become part of your DNA.Security needs to become part of your DNA.

26. Build TrustBuild Trust
● ExperimentExperiment
•
DevDev
•
TestTest
● ProdProd
● Automate all theAutomate all the
thingsthings
● Measure successMeasure success
● Measure FailureMeasure Failure

27. With great power ...With great power ...
Your code will go to production..Your code will go to production..
You will be able to fix it ..You will be able to fix it ..
You will have access to the logsYou will have access to the logs
Access to the metrics...Access to the metrics...

30. Devops is a ReorgDevops is a Reorg
● New role for Change ManagementNew role for Change Management
● New role for Security OfficersNew role for Security Officers
● Added roles for TestersAdded roles for Testers

32. Culture,Culture,
Automation,Automation,
Measurement,Measurement,
SharingSharing

33. " Our job as engineers (and ops, dev-ops, QA," Our job as engineers (and ops, dev-ops, QA,
support, everyone in the company actually) is tosupport, everyone in the company actually) is to
enable the business goals. We strongly feel thatenable the business goals. We strongly feel that
in order to do that you must havein order to do that you must have the ability tothe ability to
deploy code quickly and safelydeploy code quickly and safely. Even if the. Even if the
business goals are to deploy strongly QA’d codebusiness goals are to deploy strongly QA’d code
once a month at 3am (it’s not for us, we push allonce a month at 3am (it’s not for us, we push all
the time), having a reliable and easythe time), having a reliable and easy
deployment should bedeployment should be non-negotiablenon-negotiable."."
Etsy Blog upon releasing DeployinatorEtsy Blog upon releasing Deployinator
http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment/http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment/

34. Continuous Delivery is aContinuous Delivery is a
Security RequirementSecurity Requirement

35. How do we get there ?How do we get there ?

36. Use Version ControlUse Version Control
No ExcusesNo Excuses
Also for scripts/config/cookbooks,manifests,etcAlso for scripts/config/cookbooks,manifests,etc

37. CI ToolsCI Tools
● HudsonHudson
● JenkinsJenkins
•A zillion pluginsA zillion plugins
● Make your builds reproducible !Make your builds reproducible !
● Test your (Puppet/Chef/CFengine)Test your (Puppet/Chef/CFengine)

38. Build PipelinesBuild Pipelines

39. Test AutomationTest Automation
● Unit testsUnit tests
● Regression testsRegression tests
● SeleniumSelenium
● CucumberCucumber
● TDDTDD
● BDDBDD

41. What's in your Pipeline ?What's in your Pipeline ?

42. A pipelineA pipeline
● Checkout codeCheckout code
● SyntaxSyntax
● StyleStyle
● Code CoverageCode Coverage
● TestsTests
● BuildBuild
● More TestsMore Tests
● PackagePackage
● Upload to RepoUpload to Repo

43. A pipeline++A pipeline++
● Checkout codeCheckout code
● SyntaxSyntax
● StyleStyle
● Code CoverageCode Coverage
● TestsTests
● BuildBuild
● More TestsMore Tests
● PackagePackage
● Upload to RepoUpload to Repo
● Deploy on TestDeploy on Test
● ……
● Insert SECURITYInsert SECURITY
TESTS !TESTS !

44. Attack yourselve onAttack yourselve on
every buildevery build
● Gauntlt , write security testsGauntlt , write security tests
● Vulnerability scans (Arachni)Vulnerability scans (Arachni)
● Content Scanner (DIRB)Content Scanner (DIRB)
● ……
● https://github.com/garethr/pentesting-playgrouhttps://github.com/garethr/pentesting-playgrou
ndnd

45. Infrastructure as CodeInfrastructure as Code
● Configure 1000 nodes,Configure 1000 nodes,
● Modify 2000 files,Modify 2000 files,
● TogetherTogether
● Think :Think :
•Cfengine,Puppet, ChefCfengine,Puppet, Chef
● Put configs under version controlPut configs under version control
● Please don't roll your own ...Please don't roll your own ...

46. Puppet in ActionPuppet in Action

47. OrchestrationOrchestration
● Fix security issues with 1 commandFix security issues with 1 command
● Mco package bind upgradeMco package bind upgrade
● Write Ansible role to upgradeWrite Ansible role to upgrade

48. Culture,Culture,
Automation,Automation,
Measurement :Measurement :
measure all the thingsmeasure all the things
SharingSharing

49. Logstash in ActionLogstash in Action

52. Security in devops ?Security in devops ?
● Version control => AuditingVersion control => Auditing
● CI => Add security IN the pipelineCI => Add security IN the pipeline
● Configuration MgmtConfiguration Mgmt
•Policy DefinitionPolicy Definition
•Auditing & EnforcingAuditing & Enforcing
● MonitoringMonitoring

53. Debunking the CriticsDebunking the Critics
Security not included ?Security not included ? Everyone is Included:Everyone is Included:
security, dba, devs,security, dba, devs,
ops, designer, analysts,ops, designer, analysts,
We are solving a busines problem,We are solving a busines problem,
Not a technology problemNot a technology problem

54. *ops*ops
*.**.*

55. It's not about the toolsIt's not about the tools
It's about changeIt's about change
It's about the peopleIt's about the people

56. {devops security }{devops security }
is not a product you can buy,is not a product you can buy,
It's a lifestyleIt's a lifestyle

57. ContactContact
Kris Buytaert Kris.Buytaert@inuits.euKris Buytaert Kris.Buytaert@inuits.eu
Further ReadingFurther Reading
@krisbuytaert@krisbuytaert
http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/
http://www.inuits.eu/http://www.inuits.eu/
InuitsInuits
Essensesteenweg 31Essensesteenweg 31
2930 Brasschaat2930 Brasschaat
BelgiumBelgium
891.514.231891.514.231
+32 475 961221+32 475 961221