Bio IT World 2015 - DevOps Security and Transparency
•Download as PPT, PDF•
1 like•947 views
As more companies adopt DevOps programs and build new infrastructure, the quantity and sensitivity of data being processed outside of the traditional IT stack are growing. Few organizations know where the access points into this information are, or how to secure them. We outline best practices for establishing visibility and control in this new space, drawing real-world examples from environments large and small.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Bio IT World 2015 - DevOps Security and Transparency
Similar to Bio IT World 2015 - DevOps Security and Transparency (20)
Recently uploaded
Recently uploaded (20)
Bio IT World 2015 - DevOps Security and Transparency
- 2. © Confidential Rethinking Cloud Security: You Can’t Control What You Can’t See Kevin Gilpin CTO & Co-Founder, Conjur, Inc. @kegilpin
- 3. © Confidential As more companies adopt DevOps programs and build new infrastructure, the quantity and sensitivity of data being processed outside of the traditional IT stack are growing. Few organizations know where the access points into this information are, or how to secure them. We outline best practices for establishing visibility and control in this new space, drawing real-world examples from environments large and small. Today’s Discussion: Abstract
- 4. © Confidential Technical Co-founder of Conjur Early DevOps and Cloud Implementor and Architect Father of 4 Enterprise software career spanning Automotive, Fin Svcs, ERP, Pharma, Healthcare, … Who is Kevin?
- 5. © Confidential I. Security + DevOps Overview
- 6. © Confidential DevOps is : Continuous Delivery Dev, Test, & Prod Environments Code Review Infrastructure Source Code Infrastructure Code Developer deploy Continuous Build & Unit Test Config, Release, Deployment commit on branch build check approval tests pass
- 7. © Confidential Security& Compliance Concerns Slow The Adoption Of DevOps Source: DevOps: The Worst-Kept Secret to Winning in the Application Economy by CA Technologies, October 2014 (http://rewrite.ca.com/us/~/media/rewrite/pdfs/white-papers/devops-winning-in-application-economy.pdf) These are cultural challenges with a technical component.
- 8. © Confidential We’re All In It Together
- 9. © Confidential How does DevOps work? Magic. How does DevOps work? Magic. DevOps: Powerful, But Hard To Understand
- 10. © Confidential Lack of transparency is the #1 obstacle to compliance
- 11. © Confidential II. Security for DevOps: Status and Challenges
- 12. © Confidential Objective : Continuous Security & Compliance ● Robust security and compliance controls … with ● Full support for automation
- 13. © Confidential SecDevOps 1.0: Where Are We Today? Source Control Automated Build and Test Configuration Management Orchestration Software-Defined Networking Monitoring
- 14. © Confidential Tools Are Being Pushed Beyond Their Intended Function
- 15. © Confidential Anti-Pattern: Production-only Workflows Problem: security controls that developers cannot replicate locally Result: Speed-killer
- 16. © Confidential Anti-Pattern: Human Bottlenecks
- 17. © Confidential Anti-Pattern: Conflation of Concerns
- 18. © Confidential Anti-patterns create “Security Debt” Addressing security bottlenecks and issues are often deferred, until... New Product Feature New Security Feature
- 19. © Confidential Worst-Case Scenario? Full Stop ● Regulated Workloads Aren’t Brought into the DevOps arena ● Security Incident o Breach or unauthorized access because of workflow challenges in getting the job done ● Static Workflow Caps Velocity o Changing is too hard or too risky
- 20. © Confidential III. Security for DevOps: Moving Forward
- 21. © Confidential New Tools: Security Policy As Code dev prod stage Conjur Policy DSL
- 22. © Confidential New Tools: Identity For Machines At Scale ● Each Server (VM), Container (Docker, LXC) and Service needs to have an identity for access control to be meaningful ● Provisioning of these identities needs to be automated and included in SecDevOps workflow ● Establish machine-to- machine trust
- 23. © Confidential New Tools: Identity Management For Robots Machine trust and identity that works for servers, VMs, containers, and IOT. Apply known tools and techniques from traditional identity management to robots Example: Segregation of regulated applications/cloud into distinct application layers using policies that govern each service
- 24. © Confidential ◁ Machines have an identity, presented that to fetch secrets. Easily given and revoked. ◁ Permissions are role-based, applied to layers not hosts ◁ Secrets fetched via authenticated HTTPS call ◁ Full audit log of changes New Tools : Secrets as a Service VM or Container https RESTful API audit log
- 25. © Confidential Problem: Solution: Result: Clear Controls And Processes
- 26. © Confidential Top Takeaways 1) Start conversations with all the stakeholders to address current security and compliance challenges 2) Map security and compliance best practice and principles into continuous delivery 3) Expect this to be iterative and evolving process
- 27. © Confidential IV. Q & A
- 28. © Confidential Additional Questions? Connect with me... ● email: kgilpin@conjur.net ● web: conjur.net ● twitter: @kegilpin @conjurinc Thank You!
Editor's Notes
- I'm Kevin Gilpin CTO of Conjur. Conjur is a venture backed startup out of Waltham Massachusetts focused on security and compliance for cloud infrastructure. We were founded in 2011 and our customer base includes well-known organizations like the Broad Institute, Novartis, Netflix, Open DNS and Rally Software. These organizations are using Contra today to satisfy requirements like HIPAA, SOX, and FISMA in cloud and hybrid infrastructure.
- Cloud infrastructure goes hand-in-hand with DevOps, which is a culture and technology discipline created by its practitioners to implement a new kind of software and operations process called continuous delivery. Continuous delivery offers rapid releases of high-quality software with short feedback cycles. The result is better user experience and better communication within a business and with customers. Release cycle times can be reduced from months down to days and even hours. •High-performing organizations are deploying code 30 times more often with 50% fewer failures •High IT performance correlates with strong business performance, helping to boost 2x an enterprise’s productivity, profitability, and market share https://puppetlabs.com/2014-devops-report
- So why isn't everybody doing DevOps today? DevOps is a new discipline and it's hard to achieve. Who requires culture changes lots of willingness to learn on behalf of the participants involvement from many different parts of the organization and a willingness to build a common understanding and jointly owned processes for delivering software in a rapid, robust, and safe manner.
- Implementing continuous delivery requires everyone to work together. There's a growing recognition that it's not just about developers and operations, but also security compliance and business management functions. Business needs DevOps to compete Security and Compliance need transparency and to participate in building out a safe and secure processes. Dev and Ops need buy-in on the transformative potential of automation of Security and Compliance. --- And the answer is, make DevOps less mysterious. Build DevOps compliance and security on tools that have a great user experience and are multi-functional.How many of you are using configuration management? Who of you aren’t using config mgmt, but you have a good understanding of what it is and how it works? And how many of you aren’t using it, but you understand how config mgmt is secured, audited, and how access control is performed? At retailmenot their user management grew to 30% of their entire puppet codebase and took 90 seconds to converge on each run, on each machine. At Hubspot, the puppet pros who built their platform were consultants. When they left, the system was left to interns to operate (including secrets and users/ssh). I had a client in AWS who bakes the windows admin password into their image, and launched thousands of machines from it. Developers at several customers are rebelling against chef and puppet because the ops/security have locked them out of it. Config Mgmt doesn’t scale for security and compliance.
- Because DevOps today is primarily a technical discipline, and it's continuously evolving, it can seem like magic. It's great when it works but it can also be frustrating and mysterious to those on the "outside". When talking about security and compliance words like magical and mysterious are not welcome. And it's certainly true that in some cases continuous delivery processes are sometimes inadequate from a security and compliance standpoint. But an even bigger issue is lack of transparency in these processes. Going forward this lack of transparency can be addressed by involving security and compliance teams in the DevOps process from the outset. What can specifically be achieved? * Security, Compliance, Developers, and Operations can build personal relationships and mutual understanding. * Differences in language: The way that security, compliance, developers and ops talk about the same problem can be bridged. * And everyone can gain a clear understanding of how things work, and feel personally invested in the success of continuous delivery.
- Lack of transparency is the #1 obstacle to compliance Policies are buried in code Lack of well-defined management tools makes change controls hard to define Little to no visual reporting of access controls and system activity
- In continuous delivery operations can’t say “here are the servers, build your app to run on them”. And dev can’t say “here’s the app, magically create the infrastructure to run it with stability and scale”. Development and operations have to evolve the design together. And when it comes to security and compliance, it’s the same problem. You can’t build an app and then hand it to somebody and say “secure it”, or “and some access management”, or “bolt on some compliance”. And security can’t say “Here are your deployment processes, re-write your app to meet them”. Co-evolution is required.
- This co-evolution has been happening, in some places, for quite some time now. 5-10 years in some cases. And there’s been a steady development and understanding of the best practices and new tooling that are required. Source Control manages the code in a scalable and separable way. Build + Test produces reliable artifacts. Configuration management builds systems to run the code, Orchestration spins up and manages entire systems, and SDN creates the network architecture. All of these things are programmable, the entire system can be operated by a developer from a terminal. Teams of 5 or 6 people can build and operate really big systems. So, we’re done? No. See Slide 6. Business is really concerned about security and compliance. And, there are justifiable reasons for that. If Security and Compliance are coming late to the party, for whatever reason, it’s a good bet that they are not fully represented, and it’s a good bet that there are really important principles and practices that have been a bit sidelined.
- I’ve been talking a lot about culture and communication, but there are tooling issues as well. DevOps teams have forced to try and solve some really challenging security and access management problems with a limited set of tools. At speed, and at scale, and at velocity, which are the most prized achievements of DevOps, things start to break down. Ad-hoc security and compliance doesn’t scale. It’s not fast, and it can’t be rapidly modified or corrected. Some examples of this are: Using source control to store infrastructure passwords and keys Using configuration management servers to control user access to machines Using the build server to push code into production When tools are pushed beyond their intended function are usually lacking in a few areas: * Management tools aren't built to handle the new requirements * The tools aren't built with separation of duties and least privilege access that's easy to manage and clear * Visual reporting of access controls and audit can be missing or nonexistent * Encryption and key management aren't well handled by systems that weren't designed to be security tools
- When Security and Compliance are applied too late in the CD lifecycle, the result is often workflows that can only be applied in production. When this happens, it becomes much harder to make predictable releases, because the production deployment entails “special” processes which are not part of the development/test/stage phases. The CD workflow gets slowed down by annoying, hard-to-troubleshoot problems like network misconfiguration, decryption failures, access control blockages and missing permissions. Delivery becomes a lot less continuous. In short the workflows used in development don't match up to the workflows used for production. The result is a waterfall style handoff with all the efficiencies and delays that can arise from that.
- The operations team is often willing and even enthusiastic to take on all kinds of security and access management responsibilities. But over time, these tasks become a burden. Highly skilled (and highly paid) people are spending their time doing routine permissions changes, key rotation, public key management, etc. And their work is obscure, because they are working with low-level tools and lots of custom scripts that aren’t designed for compliance and reporting. As a result, tech resources are wasted on trivial tasks, unclear organizational ownership of tasks. Continuous delivery throughput suffers, and so does morale. And security and compliance teams find themselves without a clear understanding of how security and access management practices are being implemented.
- DevOps teams build aggressively on the tools that they have, especially source control, continuous integration servers (e.g. Jenkins) and config mgmt (for example, Chef and Puppet). Tools get pressed into service for all kinds of tasks to which they are ill suited. * Systems get built using lots of custom scripts and glue code, which makes them hard to maintain * Tools that were originally built with collaboration in mind, like source control and configuration management, suddenly become security concerns. Access is locked down to a smaller number of trusted personnel, with adverse effects to collaboration and communication. * It becomes very hard to change the architecture. The application becomes locked in to a specific toolset for security and compliance reasons.
- Over time, these practices add more and more inertia to continuous delivery. And like all technical debt, there’s very little glory in cleaning it up. The new feature is the sexy place to be, not the cleanup job to get secrets out of source control, or to separate production from development access.
- In cloud infrastructure there are many types of machines and code; everything from custom scripts, to containers, virtual machines and bare metal servers.
- Tracking and managing all of these assets is essential. And once they're all identified, the need to be governed according to security policies and role-based access control. The ways in which machines are provisioned and join the infrastructure should be very clear and well thought out. Machines should never just be blindly trusted.
- Deploying secrets like database passwords and SSL certificates is no longer the job of a person. All provisioning and configuration is performed by code. Therefore access to secrets by code needs to be very tightly managed and clear. The use of a secrets server, which ties into identity of machines and people, applies role-based access control policies, and provides robust reporting and audit, is essential. Enables human administrators to “delegate” their authority to code and scripts Example: Providing secrets to docker containers. --- ~ building an in-house system for decryption key storage/retrieval is high-risk ~ one decryption key per node when using encrypted databags makes it difficult and tedious to implement Least Privilege
- The result can be clear policies, reporting, and troubleshooting that take the undesirable "magic" out of DevOps, and enable security at scale.
- Security Compliance + DevOps is a holistic problem We describe it as “a culture problem with a technology component”; both aspects are important, and both aspects are challenging. What you’re aiming for is: continuous delivery without sacrificing security or compliance. In fact, your security and compliance stance should emerge much stronger, because of the rigor and automation of the new processes. The biggest security and compliance risks are human errors, things as simple as weak passwords and typing mistakes. CD is inherently much stronger at avoiding these types of problems. Iterate on processes which are: Intuitive Reportable/ Audited Independent of the specific tools in the continuous delivery toolchain