Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DevSecCon London 2018: Is your supply chain your achille's heel

325 views

Published on

COLIN DOMONEY
The advent of DevOps and large scale automation of software construction and delivery has elevated the software supply chain – and its underpinning delivery pipeline – to mission critical status in any modern enterprise. The increased velocity of modern pipelines and the removal of manual checks and balances has meant that modern pipelines are potential single points of failure in the delivery of secure software.

Automotive and consumer electronics industries have long understood the need for both provenance (understanding the origin of materials) and veracity (ensuring the integrity of their manufacturing processes) in their supply chains; this presentation will address threats to software supply chains and practical approaches to reducing the fragility of your supply chain. Several examples of software supply chain failures will be presented and deconstructed to understand the typical failure modes.

At the most elementary level many pipelines are poorly constructed with low levels of repeatability and poor test coverage, in other organisations there is a lack of governance over the supply chain allowing careless or willingly negligent actors to subvert or bypass controls or testing within the pipeline. There is also no standard mechanism to ensure a ‘chain of custody’ within a pipeline due to a lack common interchange format between tools, or a standard manner to represent the steps within a pipeline build process.

This presentation will cover approaches (using ‘people and process’) in enforcing governance within a supply chain by describing best practices used in large-scale AppSec programmes. Several emerging technology initiatives will be presented: Google’s Grafeas is a means to ensure vulnerability information is represented in a uniform manner across all steps of a pipeline process, while In-Toto is a project to formally enforce the integrity of a pipeline process. A reference secure pipeline will be presented demonstrating both tools working in symphony, along with standard open source and commercial AppSec tools.

Finally the pipeline itself may become the Achille’s Heel in an organisation – many pipelines are not sufficiently hardened and are themselves open to attack by use of vulnerable components and their extensible nature, often along with very wide open permissions. Guidance will be given on hardening of typical pipelines, and a fully secured ephemeral Jenkins pipeline will be demonstrated.

Benefits of this Session: The attendee will gain an increased awareness of the pivotal importance of the software supply chain, and gain an understanding of some common failure modes and weaknesses. Most importantly the attendee will come away with practical guidance on enforcing higher levels of governance on their supply chain without reducing delivery velocity, as well as how to harden the pipeline infrastructure itself.

Published in: Technology
  • Be the first to comment

DevSecCon London 2018: Is your supply chain your achille's heel

  1. 1. LONDON 18-19 OCT 2018 Is your supply chain your Achille's heel ? COLIN DOMONEY
  2. 2. LONDON 18-19 OCT 2018 About the Presenter @colindomoney • Built lots of hardware and software • Done AppSec at scale in large enterprise • Worked in vendor-land in AppSec • Currently a transformation consultant • Veteran DevSecCon presenter • Interested in all things new and shiny
  3. 3. LONDON 18-19 OCT 2018 Thank You’s and Acknowledgements @controlplaneio @lukeb0nd @sublimino Santiago Torres @ In-Toto @torresariass
  4. 4. veracity /vəˈrasɪti/ noun conformity to facts; accuracy.
  5. 5. provenance /ˈprɒv(ə)nəns/ noun the place of origin or earliest known history of something.
  6. 6. LONDON 18-19 OCT 2018 How Do Other Industries Manage Their Supply Chains
  7. 7. LONDON 18-19 OCT 2018 Big Pharmaceuticals Understand Supply Chains Surely ?
  8. 8. LONDON 18-19 OCT 2018 Then I Remembered This …
  9. 9. LONDON 18-19 OCT 2018 Consumer Electronics Understand Supply Chains Surely ?
  10. 10. LONDON 18-19 OCT 2018 And Then This Happened ! “Having a well-done, nation-state- level hardware implant surface would be like witnessing a unicorn jumping over a rainbow” Joe Grand
  11. 11. LONDON 18-19 OCT 2018 Software Supply Chain Failure Modes
  12. 12. LONDON 18-19 OCT 2018 Vulnerable 3rd Party Components
  13. 13. LONDON 18-19 OCT 2018 Typosquatting
  14. 14. LONDON 18-19 OCT 2018 The CCleaner Malware Attack • Malware distributed via official download site • Affected 2.7 million users • Initial entry point via a compromised developer account • Three stage deployment compromising intermediate build machines
  15. 15. LONDON 18-19 OCT 2018 Trust, but Verify • Ensure developers aren’t ‘optimising’ your security testing out of the pipeline • Validate what is scanned is what is deployed • Validate that what you test is representative of the actual application • Hunt for shadow build infrastructure • Get early warnings for new development
  16. 16. LONDON 18-19 OCT 2018 Build Pipelines : Then and Now, and Beyond
  17. 17. LONDON 18-19 OCT 2018 Before DevOps …
  18. 18. LONDON 18-19 OCT 2018 DevSecOps … SecDevOps … CAB
  19. 19. LONDON 18-19 OCT 2018 Making It Go Faster – Just Remove all Security Measures
  20. 20. LONDON 18-19 OCT 2018 Software Supply Chain Basics
  21. 21. LONDON 18-19 OCT 2018 Prescribe a Policy for OSS Use • Prescribe a policy for the use of OSS based on: • Risk appetite • Business criticality • Time to market • Organisational maturity • Provide a recommended architecture of commonly used and pre-approved components • Educate your security team in the use of OSS components and risk determination
  22. 22. LONDON 18-19 OCT 2018 Control Your Repositories • Use a caching binary repository server (such as Nexus) • Maintain a blacklist of known bad (and hence banned) components • Maintain a whitelist of known good (and hence approved) components • Quarantine unknown components until assessed • In extremis disable access to public internet repositories
  23. 23. LONDON 18-19 OCT 2018 Hardening your Build Pipeline
  24. 24. LONDON 18-19 OCT 2018 Using Your Pipeline as a Bitcoin Miner • Exploits CVE-2107-1000353 in Jenkins disclosed in April 2017 • Deploys XMRig miner and a RAT • Over $3 million mined thus far https://www.csoonline.com/article/3256314/security/hackers-exploit-jenkins- servers-make-3-million-by-mining-monero.html
  25. 25. LONDON 18-19 OCT 2018 Harden Your CI/CD Infrastructure • Harden the hosts, ensure patching is rigorously applied • Lock down your tools (Jenkins is wide open by default) • Lock down and harden your config management tools • Ensure that keys, credentials and secrets are protected • Secure access to all repositories • Review and audit your access controls to your pipeline • Treat your pipeline as you would your production infrastructure https://www.oreilly.com/ideas/9-tips-for-a-more-secure-continuous-delivery-pipeline
  26. 26. LONDON 18-19 OCT 2018 In-Depth with In-Toto
  27. 27. LONDON 18-19 OCT 2018 The Update Framework TUF’s primary goals are: • Framework that can be used to secure systems • Minimise the impact of key compromises • Be flexible and easy to integrate Guards against the following attacks: • Replay attacks of same file • Compromised and vulnerable versions • Key compromise in signing files Implemented as Notary by Docker (originally) http://www.eweek.com/security/cncf-brings-in-notary-the-update-framework-to- boost-container-security
  28. 28. LONDON 18-19 OCT 2018 What Is In-Toto Motivation: “Although many frameworks ensuring security in the "last mile" (e.g., software updaters) exist, they may be providing integrity and authentication to a product that is already vulnerable; it is possible that, by the time the package makes it to a software update repository, it has already been compromised.” Goals: “in-toto aims to provide integrity, authentication and auditability to the supply chain as a whole. This means that all the steps within the supply chain are clearly laid out, that the parties involved in carrying out a step are explicitly stated, and that each step carried out meets the requirements specified by the actor responsible for this software product.”
  29. 29. LONDON 18-19 OCT 2018 In-Toto Basic Terminology Materials: the elements used (e.g., files) to perform a step in the supply chain. Product: the result of carrying out a step. Products are recorded as part of link metadata. Link: metadata information gathered while performing a supply chain step or inspection, signed by the functionary that performed the step or the client that performed the inspection Verification: the process by which data and metadata included in the final product is used to ensure its correctness.
  30. 30. LONDON 18-19 OCT 2018 In-Toto Actors Project Owner: Defines the layout of the software supply chain. Functionary: Performs a step in the supply chain and provides a piece of link metadata as a record that such a step was carried out. Client: Performs verification on the final product by checking the provided layout and link metadata.
  31. 31. LONDON 18-19 OCT 2018 In-Toto Layouts - Steps • A recipe for taking materials and producing an output product. • Steps can be chained, and sub-layouts can be specified.
  32. 32. LONDON 18-19 OCT 2018 In-Toto Layouts - Inspect • Executes at the final stage of verification to verify the resultant product matches that specified in the layout. • Takes an input list of expected materials and expected products. • Returns a go/no-go result.
  33. 33. LONDON 18-19 OCT 2018 In-Toto Links • Record information about the execution environment. • Cryptographically signed by the functionary carrying out the action.
  34. 34. LONDON 18-19 OCT 2018 And Finally : In-Toto In Action A Passing Verification: A Failing Verification: https://in-toto.github.io/
  35. 35. LONDON 18-19 OCT 2018 In-Toto in a Jenkins Server stage('Build') { agent { docker { #image name here } } steps { withCredentials([#any credentials here]) { in_toto_wrap(['stepName': 'Build', 'keyPath': "${WORKER_KEY}", 'transport': "redis://${REDIS_ENDPOINT}:6379"]){ #your actual step here } } } } }
  36. 36. LONDON 18-19 OCT 2018 Getting It Right By Design : Cloud Native and Containers
  37. 37. LONDON 18-19 OCT 2018 Point Solutions Are Not Enough
  38. 38. LONDON 18-19 OCT 2018 What Can You Trust ? • Git ensures integrity but not identity • Anyone can pretend to commit as someone else ! • Most people assume Git is a trusted source • Signing and verification are easy • Enterprise key management not so much ! https://mikegerwitz.com/papers/git-horror-story https://medium.com/@pjbgf/spoofing-git-commits-7bef357d72f0
  39. 39. LONDON 18-19 OCT 2018 Security-hardened Container Supply Chain Base Image Code Build Application Image Deploy Controlled base images Hash based addressing Static analysis Dependency analysis Hermetic Reproducible Rootless Vulnerability scanning Configuration scanning Admission control Runtime configurations Docker Hub TUF Notary Grafeas In-Toto Clair Aqua Microscanner Kubernetes Kritis
  40. 40. LONDON 18-19 OCT 2018 Securing Builds with Metadata • Pipeline metadata is rich and varied • Initiating users and/or events • Installed dependencies and their versions • Veracity test data (unit/integration/acceptance tests) • Security test data • Data can be used for: • Recording i.e. audit • Report/enforcing i.e. policy
  41. 41. LONDON 18-19 OCT 2018 Storing Metadata with Google Grafeas • Google’s open-source project to audit and govern the software supply chain • Stores metadata about artefacts and their vulnerabilities • Twistlock, Aqua, JFrog Xray, BlackDuck can send metadata to Grafeas • Possible to query that metadata to gate builds and deployments
  42. 42. LONDON 18-19 OCT 2018 Grafeas in Action
  43. 43. LONDON 18-19 OCT 2018 The Art of the Possible
  44. 44. LONDON 18-19 OCT 2018 In-Toto in a Container SDLC
  45. 45. LONDON 18-19 OCT 2018 DevSecOps … with In-Toto CAB
  46. 46. LONDON 18-19 OCT 2018 Avoid the Horror • Practice basic hygiene • Trust with caution • Trust but verify • Understand your abuse cases • Embrace new ways of working • Backport the best of new technology
  47. 47. LONDON 18-19 OCT 2018 [Last slide for thank you message, links, etc] @colindomoney

×