Continuous Delivery with Containers Guide
•Download as PPTX, PDF•
7 likes•1,679 views
Is continuous delivery mainstream? We would all like to think so, but as William Gibson reminds us, “The future is already here—it’s just not evenly distributed.” The large (unicorn) companies have been talking about deploying containerized applications for some time, but the processes, techniques, and technologies involved are not always clear when looking in from the outside. This can make it difficult to emulate their success. Based on his experience building several Docker-based applications that were deployed to a range of orchestration and scheduling platforms, Daniel Bryant demonstrates how to create a scalable build pipeline that takes a series of Java applications, containerizes them, and deploys them to Docker Swarm. Example code will be available to download via GitHub, and the examples can be executed locally via Vagrant.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to Continuous Delivery with Containers Guide
Similar to Continuous Delivery with Containers Guide (20)
More from Daniel Bryant
More from Daniel Bryant (20)
Recently uploaded
Recently uploaded (20)
Continuous Delivery with Containers Guide
- 1. (A Practical Guide to) Continuous Delivery with Containers Daniel Bryant @danielbryantuk
- 2. Setting the scene… • Continuous delivery is a large topic • Focusing on the process and tooling • Rather than each explicit step • My O’Reilly mini-book will provide step-by-step instructions • Assuming basic knowledge of Docker 20/10/2016 @danielbryantuk
- 3. Today… • Continuous Delivery (CD) • The impact of containers on CD • Creating a container pipeline • Migrations: Architectural guidance • Lessons learned the hard way 20/10/2016 @danielbryantuk
- 4. @danielbryantuk • Chief Scientist at OpenCredo, CTO at SpectoLabs • Agile, architecture, CI/CD, DevOps • Java, Go, JS, microservices, cloud, containers • Leading change through the application of technology and teams • London Java Community Associate • InfoQ Editor, DZone MVB, O’Reilly… • Conference regular: Devoxx, JavaOne, QCon… 20/10/2016 @danielbryantuk
- 6. Continuous Delivery • Produce valuable and robust software in short cycles • Optimising for feedback and learning • Not (necessarily) Continuous Deployment 20/10/2016 @danielbryantuk
- 7. Creation of a build pipeline is mandatory for continuous delivery 20/10/2016 @danielbryantuk
- 9. The Impact of Containers on CD 20/10/2016 @danielbryantuk
- 10. Containers: Expectations versus reality 20/10/2016 @danielbryantuk “DevOps”
- 11. Container technology • OS-level virtualisation • cgroups, namespaces, rootfs • Technology to package and execute software • The container image becomes the source of truth • Mechanical sympathy is vital 20/10/2016 @danielbryantuk
- 12. We’ll focus on Docker today • Docker images are built via a Dockerfile • docker build –t danielbryantuk/test:1.4 • Publish images • docker push danielbryantuk/test:1.4 • Download images • docker pull danielbryantuk/test:1.4 • Run an image as a container • docker run –p 80:80 danielbryantuk/test:1.4 20/10/2016 @danielbryantuk
- 15. Quick interuption: Microservices… • Containers and microservices are complementary • Not covering details for deploying microservices today • But if you are interested: • Consumer-based contracts • Service virtualisation • Synthetic transactions and semantic monitoring 20/10/2016 @danielbryantuk https://specto.io/blog/recipe-for-designing-building-testing-microservices.html
- 19. Creating a Pipeline for Containers 20/10/2016 @danielbryantuk
- 21. Make your dev environment like production • Develop locally or copy/code in container • Ensure language runtime/SDK is synced • Must build/test containers locally • Perform (at least) happy path tests before pushing code • All tests should be runnable locally 20/10/2016 @danielbryantuk
- 22. What to put in the Dockerfile • OS choice • Exposed to OS (often implictly?) • Choose lightweight OS if possible e.g. Alpine, Debian Jessie • Configuration • Build artifacts • Exposing ports • Java • JDK vs JRE • Oracle vs OpenJDK • Golang • Statically compiled binary • Python • Virtualenv 20/10/2016 @danielbryantuk
- 23. Please talk to the sysadmin people: Their operational knowledge is invaluable 20/10/2016 @danielbryantuk
- 24. Different dev and test containers? • Test container • Full OS (e.g. Ubuntu) • JDK • Test tools • Test data • Easy to see configuration drift • Interesting ONTEST proposal by Alexi Ledenev 20/10/2016 @danielbryantuk http://blog.terranillius.com/post/docker_testing/
- 26. Building images with Jenkins • Standard Jenkins Java • Gradle or Maven • SonarQube for code quality • (Optionally) push to artifact repo • Nexus and Artifactory support Java artifacts and Docker images • Build Docker Image • Cloudbees Docker Build and Publish Plugin 20/10/2016 @danielbryantuk
- 28. Storing in an image registry (DockerHub) 20/10/2016 @danielbryantuk
- 30. A little context… 20/10/2016 @danielbryantuk
- 31. Introducing Docker Compose 20/10/2016 @danielbryantuk
- 32. Testing: Jenkins Pipeline as Code 20/10/2016 @danielbryantuk
- 33. Testing: Jenkins Pipeline as Code 20/10/2016 @danielbryantuk
- 37. Docker Compose & Jenkins Pipelines 20/10/2016 @danielbryantuk
- 38. Testing: Functional • Automate all the things! • Deploy to realistic environments • API-driven functional • REST-assured • UI-driven functional • Selenium • Serenity BDD • Geb 20/10/2016 @danielbryantuk
- 39. Testing: NFRs • Execution (runtime) • Security • Observability • Evolvability (static) • Testabillity • Maintainability • Scalability • Extensibility 20/10/2016 @danielbryantuk
- 40. Testing: NFRs • Security testing • Findsecbugs • OWASP Dependency check • Bdd-security (OWASP ZAP) / Arachni • Gauntlt / Serverspec • Docker Bench for Security / AQUA • Performance and Load testing • Gatling / Jmeter • Flood.io 20/10/2016 @danielbryantuk
- 41. Special mention: Container security testing 20/10/2016 @danielbryantuk
- 43. Special mention: Fault tolerance testing 20/10/2016 @danielbryantuk
- 44. Fault tolerance 20/10/2016 @danielbryantuk techblog.netflix.com/2016/10/netflix-chaos-monkey-upgraded.html github.com/tomakehurst/saboteur
- 45. Hoverfly • Lightweight Service virtualisation • Open source (Apache 2.0) • Go-based / single binary • Written by @Spectolabs • Flexible API simulation • HTTP / HTTPS • Highly performant 20/10/2016 @danielbryantuk
- 46. 20/10/2016 @danielbryantuk • Middleware • Remove PII • Rate limit • Add headers • Middleware • Fault injection • Chaos monkey
- 48. Deploy • Test environments should represent production (as much as possible) • Fan-in infrastructure pipelines with applications as soon as possible • Ask yourself: Do you really want to create a container platform? 20/10/2016 @danielbryantuk
- 49. Don’t underestimate the value of PaaS… 20/10/2016 @danielbryantuk | @spoole167 49
- 51. When bad things happen, people are at the center 20/10/2016 @danielbryantuk | @oakinger
- 52. Monitoring is vital with continuous delivery • Host monitoring • Container monitoring • Application monitoring 20/10/2016 @danielbryantuk https://github.com /Kentik/docker-monitor
- 53. Migrations: Architectural Guidance 20/10/2016 @danielbryantuk
- 54. Containerise the monolith? • For • We know the monolith well • Allows homogenization of the pipeline and deployment platform • Can be a demonstrable win for tech and the business • Against • Can be difficult (100+ line scripts) • Often not designed for operation within containers, nor cloud native • Putting lipstick on a pig? 20/10/2016 @danielbryantuk
- 55. Key lessons learned • Conduct an architectural review • Architecture for Developers, by Simon Brown • Architecture Interview, by Susan Fowler • Look for data ingress/egress • File system access • Support resource constraints/transience • Optimise for quick startup and shutdown • Evaluate approach to concurrency • Store configuration (secrets) remotely 20/10/2016 @danielbryantuk
- 56. Containers and cloud: Design for failure • Distributed Computing Principles • Jeff Hodges ‘Distributed Systems’ (bit.ly/1FeaVtt) • Scalable Web Architecture (bit.ly/1tt703O) • ‘For young bloods’ (bit.ly/1pKVepz) • Design patterns • Timeouts / retries • Bulkheads / circuit-breakers 20/10/2016 @danielbryantuk
- 57. New design patterns 20/10/2016 @danielbryantuk bit.ly/2efe0TP
- 58. Using containers does not obviate the need for good architecture 20/10/2016 @danielbryantuk
- 60. Lessons Learned the Hard Way 20/10/2016 @danielbryantuk
- 61. Miscellaneous (but vital) • Beware of the ‘latest’ Docker tag • Properly version your containers • Metadata is vital • Labels can be valuable • h/t MicroBadger • www.notonthehighstreet.com case study and learnings • http://bit.ly/1PMlpIL 20/10/2016 @danielbryantuk
- 62. Mechanical sympathy: Docker and Java • Set container memory appropriately • JVM requirements = Heap size (Xmx) + Metaspace + JVM overhead • Account for native thread requirements e.g. thread stack size (Xss) • Default fork/join thread pool sizes (based from host CPU count) • Watch out for ulimits • Entropy • Host entropy can soon be exhausted by crypto operations • –Djava.security.egd=file:/dev/urandom • Be aware of security ramifications 20/10/2016 @danielbryantuk | @spoole167 62
- 64. In summary • Continuous delivery is vitally important in modern architectures/ops • Container images must be the (single) source of truth within pipeline • Mechanical sympathy is important (assert properties in the pipeline) • We’re now bundling more into our artifact (e.g. an OS) • Not all developers are operationally aware • The tooling is now becoming stable/mature • We need to re-apply old CD practices with new technologies/tooling 20/10/2016 @danielbryantuk
- 66. Thanks for listening • Any questions? • Feel free to contact me • @danielbryantuk • daniel.bryant@opencredo.com 20/10/2016 @danielbryantuk
Editor's Notes
- Daniel