Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Adding the Sec to Your DevOps Pipelines

276 views

Published on

DevSecOps is the premise that everyone in the software development lifecycle is responsible for security. DevSecOps aims to embed security in every part of the development process. In this *workshop*, participants explore taking a standard CI/CD pipeline and adding security stages to improve security posture. Learn how to use AWS CodeCommit and AWS CodePipeline to build and publish golden AMI images. Also, learn how to modify pipeline flow to add security test cases. You also have to opportunity to perform CVE analysis and code analysis using Amazon Inspector and perform observational container analysis using Amazon GuardDuty.

  • Be the first to comment

  • Be the first to like this

Adding the Sec to Your DevOps Pipelines

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nate Case & Dave Walker Security Geeks Workshop Guide: https://tinyurl.com/yakft2rq DevSecOps on AWS Introduction to DevSecOps
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Setup the initial Account - Run the Cloudformation (This can take a bit, so while we wait..) • Introduction to DevSecOps • Introduction to Pipelines Review of the Pipeline - Correct the code! Or the rule? - Test the Test environment - Amazon Systems Manager - To manual or not to manual? - Finished!
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Introduction to DevSecOps
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is DevOps? Cultural Philosophy Practices Tools
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is DevOps? Break down cultural barriers Work as one team Support business and IT agility Collaborate and communicate Treat infrastructure as code Automate Test, measure, and monitor Culture Process
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Why do organizations adopt DevOps? Faster time to value Agility Quality Speed
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Competing Forces Business Development Operations Build it faster Keep it stable Security Make it secure
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is DevSecOps? DevSecOps is the combination of cultural philosophies, practices, and tools that exploits the advances made in IT automation to achieve a state of production immutability, frequent delivery of business value, and automated enforcement of security policy. DevSecOps is achieved by integrating and automating the enforcement of preventive, detective, and responsive security controls into the pipeline. Security OperationsDevelopment
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tenets of DevSecOps 1. Test security as early as possible to accelerate feedback. 2. Prioritize preventive security controls to stop bad things from happening. 3. When deploying a detective security control, ensure it has a complementary responsive security control to do something about it. 4. Automate, automate, automate.
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Three Major Components to DevSecOps 1. Security OF the pipeline 2. Security IN the pipeline 3. Enforcement of the pipeline
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Security OF the pipeline Use the Core 5 Security Epics from the CAF • Identity and Access Management • Detective Controls • Infrastructure Security • Data Protection • Incident Response Security OF the Pipeline module will provide guidance
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Security IN the pipeline Static analysis • Infrastructure-as-code • Security-as-code Dynamic analysis • Unit tests • Integration tests • System tests
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Enforce the pipeline Establish environments (e.g. Sandbox, Dev, Test, Prod) Humans should have increasingly fewer rights as you progress through environments Only the pipeline should be able to “make changes” to Prod
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is a Pipeline? Build automation Continuous Integration • Deployment automation Test automation Service Orchestration
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CI vs CD Continuous Integration Techniques and tools to implement the continuous process of applying quality control; in general, small pieces of effort, applied frequently, to improve the quality of software, and to reduce the time taken to deliver it. Continuous Deployment Techniques and tools to improve the process of software delivery, resulting in the ability to rapidly, reliably, and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead.
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Promotion Process in Continuous Deployment
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pipelines as Workloads
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pipeline as a workload Securing the application starts with securing the pipeline The CI/CD pipeline is a workload Its purpose is to integrate and deliver other workloads It has users, supporting infrastructure, application, and data components, etc. Those components are typically managed as code …
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Infrastructure as Code is a practice where by traditional infrastructure management techniques are supplemented and often replaced by using code based tools and software development techniques.
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Resources Operating System and Host Configuration Application Configuration Amazon Virtual Private Cloud (VPC) Amazon Elastic Compute Cloud (EC2) AWS Identity and Access Management (IAM) Amazon Relational Database Service (RDS) Amazon Simple Storage Service (S3) AWS CodePipeline … Windows Registry Linux Networking OpenSSH LDAP Centralized logging System Metrics Deployment agents Host monitoring … Application dependencies Application configuration Service registration Management scripts Database credentials … AWS CloudFormation AWS SSM AWS CodeDeploy
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. allOfThis == $Code https://secure.flickr.com/photos/wscullin/3770015991
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Enforcing Least Privilege between pipelines Pipeline can perform a specific job E.g. Jenkins/Spinnaker/CodePipeline is a pipeline factory Pipelines can be limited to blast radius-based functions • Pipeline Factory • AMI Factory • Artifact Factory
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Decompose security requirements for CI/CD Remember that CI/CD is not a “thing” unto itself CI/CD is a collection of microservices In the same way we use the cloud to protect the cloud, use Agile to deploy Agile Start with Security epics and create user stories for security features to include in the pipeline Make the pipeline another reference architecture Don’t forget that the pipeline is a workload!
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Security Epics Identity & Access Management Logging & Monitoring Infrastructure Security Data Protection Incident Response Resilience Compliance Validation Secure CI/CD (DevSecOps) Configuration & Vulnerability Analysis Big Data Analytics Bearded security guy asks, “Who can list the 10 Security Epics of the CAF in order?”
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Top 10 11 IAM Best Practices 0. Users 1. Permissions 2. Groups 3. Conditions 4. Auditing 5. Passwords 6. Rotation 7. MFA 8. Sharing 9. Roles 10.Root Oops, looks like a 0-based code error J
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Some IAM risks for pipelines Anyone can run build jobs Consistent user management across build servers Pipeline role is too permissive Slave node adverse effects on Masters
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Exercise: IAM for pipelines wrap-up Could you write a user story for the DevOps team managing the pipeline to implement? • If not, what is missing? What are the acceptance criteria for your user story? How would you validate your user story?
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Top Detective Controls Best Practices 1. Coordinate time sources (EC2 Time Sync!) 2. Capture all logs 3. Determine auditable events and audit record content 4. Protect the confidentiality and integrity of audit logs (WORM) 5. Detect audit processing failures 6. Determine thresholds for warnings and alerts 7. Respond to warnings and alerts
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Exercise: Detective Controls • What produces logs? • How are logs produced? • Where do logs go? • How do I protect my logs? • What are the items of interest in my logs? • At what threshold are those items interesting? • What should I do when thresholds are exceeded?
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Detective Controls for pipelines Who logged in? What code was committed and by whom? What jobs did they run? Did the jobs succeed/fail? Was static/dynamic analysis enforced? What were the results of the static/dynamic analysis?
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Detective Controls for pipelines wrap-up There are multiple consumers of logs produced by the pipeline. Fast feedback to the log consumers is critical. Results of static/dynamic tests are as important as any other audit trail.
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Top Infrastructure Security Best Practices Provision AWS accounts in accordance with team organization Separate environments by their access levels Create good architectures for pipelines (e.g. dedicated, shared, team) Whitelist the environments, API’s, and services the pipeline is allowed to interact with Use EC2 roles combined with IAM policies (and CloudTrail!) to your advantage Limit the blast radius!
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hopefully you've seen this already… Developer Sandbox Dev Pre-Prod Team/Group accounts Security Core accounts AWS Organizations Shared Services Network Log Archive Prod Team Shared Services Network Path Developer accounts Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Services Security Core accounts AWS Organizations Shared Services Network Log Archive Network Path Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Developers Developer Sandbox Team/Group accounts AWS Organizations Network Path Developer accounts Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Connections to Team Accounts Developer Sandbox Team/Group accounts AWS Organizations Network Path Developer accounts Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake Core accounts Dev Pre-Prod Team/Group accounts Prod Team Shared Services
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Team verses Production Accounts Dev Prod Green Team/Group accounts Prod Blue Dev Pipeline account Network Path Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake Testing Account Prod Pipeline account Network Prod/Group accounts
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Top Infrastructure Security Best Practices Provision AWS accounts in accordance with team organization Separate environments by their access levels Create good architectures for pipelines (e.g. dedicated, shared, team) Whitelist the environments, API’s, and services the pipeline is allowed to interact with Use EC2 roles combined with IAM policies (and CloudTrail!) to your advantage Limit the blast radius!
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Infrastructure Security risks to pipelines Who has access to underlying infrastructure resources? How are pipelines patched and updated? How is least privilege between pipelines enforced? Are my pipelines deploying into approved AWS accounts? Does the pipeline align with organizational responsibility?
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Infrastructure Security for pipelines wrap-up The pipeline is a workload and needs to be treated with the same rigor as other critical infrastructure. Build a pipeline factory to build pipelines from known good configurations. Deploy workloads into known good environments.
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use IAM roles for Amazon EC2 instances Benefits Easy to manage access keys on EC2 instances Automatic key rotation Assign least privilege to the application AWS SDKs fully integrated AWS CLI fully integrated How to get started Create an IAM role Assign permissions to role Launch instances w / role If not using SDKs, sign all requests to AWS services with the role’s temporary credentials
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Top Data Protection Best Practices Control access and permissions to the code repository Trigger builds automatically (time-based or event-based) Use tokenization or dummy data in non-production environments Categorize data and enforce restrictions through pipeline • For example, pipeline is configured to build Dev environment is not allowed to pull Production data from repo
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No more humans in production. Dev Prod Green Team/Group accounts Prod Blue Dev Pipeline account Network Path Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake Testing Account Prod Pipeline account Network Prod/Group accounts
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data Protection risks for pipelines Who can change/commit code? How is production data prevented from being introduced into non-prod environments? How is artifact integrity maintained?
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No more humans in production. Dev Prod Green Team/Group accounts Prod Blue Dev Pipeline account Network Path Data Centre Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake Testing Account Prod Pipeline account Network Prod/Group accounts
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data Protection for pipelines wrap-up Control access and permissions to source repository: artifacts are critical data for your pipeline. Build pipelines that are environment-aware (e.g. prod vs. non-prod). Build artifact handlers to validate integrity across pipelines and environments.
  47. 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DevSecOps Benefits • Confidence that workloads and changes are validated against corporate security policies. • Consistency and repeatability of security validation. • Match the business’ pace of innovation. • Security at scale!
  48. 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Helpful Links https://aws.amazon.com/devops/ https://aws.amazon.com/devops/what-is-devops/ https://stelligent.com/2015/01/20/deployment- pipeline-aws/ https://aws.amazon.com/getting-started/projects/set- up-ci-cd-pipeline/ https://aws.amazon.com/certification/certified-devops- engineer-professional/ https://github.com/awslabs/git-secrets
  49. 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Questions?
  50. 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Core accounts Core Accounts AWS Organizations Master Network Path Data Center Foundational Building Blocks Once per organization Have their own development life cycle (dev/qa/prod)
  51. 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Log archive account Core Accounts AWS Organizations Master Log Archive Network Path Data Center Versioned Amazon S3 bucket Restricted MFA delete CloudTrail logs Security logs Single source of truth Alarm on user login Limited access
  52. 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Security account Core Accounts AWS Organizations Master Log Archive Network Path Data Center Optional data center connectivity Security tools and audit GuardDuty Master Cross-account read/write Automated Tooling Limited access Security
  53. 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Shared services account Security Core Accounts AWS Organizations Master Log Archive Network Path Data Center Connected to DC DNS LDAP Shared Services VPC Deployment tools Golden AMI Pipeline Scanning infrastructure Inactive instances Improper tags Snapshot lifecycle Monitoring Limited access Shared Services
  54. 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network account Security Core Accounts AWS Organizations Master Shared Services Log Archive Network Path Data Center Managed by network team Networking services AWS Direct Connect Limited access Network
  55. 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Developer sandbox Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Network Path No connection to DC Innovation space Fixed spending limit Autonomous Experimentation Developer Sandbox Developer Accounts
  56. 56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Team/group accounts Developer Sandbox Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Network Path Developer Accounts Data Center Based on level of needed isolation Match your development lifecycle Think Small Team/Group Accounts
  57. 57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dev Developer Sandbox Team/Group Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Network Path Developer Accounts Data Center Develop and iterate quickly Collaboration space Stage of SDLCDev
  58. 58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pre-production Developer Sandbox Dev Team/Group Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Network Path Developer Accounts Data Center Connected to DC Production-like Staging Testing Automated Deployment Pre-Prod
  59. 59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Production Developer Sandbox Dev Pre-Prod Team/Group Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Network Path Developer Accounts Data Center Connected to DC Production applications Promoted from Pre-Prod Limited access Automated Deployments Prod
  60. 60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Team shared services Developer Sandbox Dev Pre-Prod Team/Group Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Prod Network Path Developer Accounts Data Center Grows organically Shared to the team Product-specific common services Data lake Common tooling Common services Team Shared Services
  61. 61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Innovation pipeline Developer Accounts Developer Accounts PoC Developer Accounts Developer Accounts Dev Pre-Prod Team/Group Accounts Prod Shared Services PoC New initiatives Experimentation Innovation
  62. 62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Multi-account approach Developer Sandbox Dev Pre-Prod Team/Group Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Prod Team Shared Services Network Path Developer Accounts Data Center Orgs: Account management Log Archive: Security logs Security: Security tools, AWS Config rules Shared services: Directory, limit monitoring Network: Direct Connect Dev Sandbox: Experiments, Learning Dev: Development Pre-Prod: Staging Prod: Production Team SS: Team Shared Services, Data Lake
  63. 63. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Team: Billing tools Developer Sandbox Dev Pre-Prod Billing Tools Team Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Prod Network Path Developer Accounts Data Center Reduces access to Organizations account Billing reports Usage metrics and reporting Usage optimizations and RI management
  64. 64. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Team: Internal audit Developer Sandbox Dev Pre-Prod Internal Audit Team Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Prod Network Path Developer Accounts Data Center Regulatory compliance Read-only access to needed logs Limited access ENT315: Automate and Audit Cloud Governance and Compliance in your Landing Zone
  65. 65. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Team: Amazing new product Developer Sandbox Dev Pre-Prod Amazing New Product Team Accounts Security Core Accounts AWS Organizations Master Shared Services Network Log Archive Prod Network Path Developer Accounts Data Center Match your development lifecycle Think Small

×