Downloaded 66 times
Uploaded byDaniel Bryant
Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly"
The document discusses continuous delivery (CD) using container technology, emphasizing that container images serve as the vital 'single source of truth' within the build pipeline. It highlights the importance of metadata for ensuring system quality and operational effectiveness, as well as the need for developers to understand operational challenges when using containers. Additionally, it outlines lessons learned, best practices for building and testing containers, and the significance of applying existing CD practices to new technologies.
More Related Content
Similar to Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly"
Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly"
- 1. Continuous Delivery withContainers: The Good, the Bad, and the Ugly Daniel Bryant @danielbryantuk
- 2. Containers: Expectations versusreality 08/11/2017 @danielbryantuk “DevOps”
- 3. Setting the scene… •Continuous delivery is a large topic • No business focus today (value stream etc) • PaaS and Serverless are super interesting… • But I’m assuming you’re all-in on containers • Focusing today on the process and tooling • No live coding today • Mini-book contains more details (thanks nginx!) 08/11/2017 @danielbryantuk bit.ly/2jWDSF7
- 4. TL;DR – Containersand CD • Container image becomes the build pipeline ‘single binary’ • Adding metadata to containers images is vital, but challenging • Must validate container constraints on system quality attributes (NFRs) • Cultivate container ‘mechanical sympathy’ 08/11/2017 @danielbryantuk
- 5. @danielbryantuk • Independent TechnicalConsultant, CTO at SpectoLabs • Architecture, DevOps, Java, microservices, cloud, containers • Continuous Delivery (CI/CD) advocate • Leading change through technology and teams 08/11/2017 @danielbryantuk
- 6.
- 7.
- 8. Continuous Delivery • Producevaluable and robust software in short cycles • Optimising for feedback and learning • Not (necessarily) Continuous Deployment 08/11/2017 @danielbryantuk
- 9. Creation of abuild pipeline is mandatory for continuous delivery 08/11/2017 @danielbryantuk
- 10. 08/11/2017 @danielbryantuk Feedback: - Wasour initial hypothesis proven? - How can we improve business, architecture and ops?
- 11. The impact ofcontainers on CD 08/11/2017 @danielbryantuk
- 12. Container technology (andCD) • OS-level virtualisation • cgroups, namespaces, rootfs • Package and execute software • Container image == ‘single binary’ 08/11/2017 @danielbryantuk
- 13.
- 14.
- 15. Microservices multiply thechallenges 08/11/2017 @danielbryantuk
- 16. Creating a pipelinefor containers 08/11/2017 @danielbryantuk
- 17.
- 18. Make your devenvironment like production • Develop locally or copy/code in container • Must build/test containers locally • Perform (at least) happy path tests • Use identical base images from production • With same configuration 08/11/2017 @danielbryantuk
- 19. Lesson learned: Dockerfilecontent is super important • OS choice • Configuration • Build artifacts • Exposing ports • Java • JDK vs JRE and Oracle vs OpenJDK? • Golang • Statically compiled binary in scratch? • Python • Virtualenv? 08/11/2017 @danielbryantuk
- 20. Please talk tothe sysadmin people: Their operational knowledge is invaluable 08/11/2017 @danielbryantuk
- 21. Different test andprod containers? • Create “test” version of container • Full OS (e.g. Ubuntu) • Test tools and data • Easy to see app/configuration drift • Use test sidecar containers instead • ONTEST proposal by Alexi Ledenev 08/11/2017 @danielbryantuk http://blog.terranillius.com/post/docker_testing/
- 22. Docker multi-stage builds 08/11/2017@danielbryantuk http://blog.alexellis.io/mutli-stage-docker-builds/ https://github.com/moby/moby/pull/31257 https://github.com/moby/moby/pull/32063
- 23. Java specific stuff… 08/11/2017@danielbryantuk github.com/oracle/docker-images/tree/master/OracleJava jdk.java.net/9/ea
- 24. Hot off thepress: Modularity • Create minimal runtime images • “jlink delivers a self-contained distribution of your application and the JVM, ready to be shipped.” • Benefits: • Reduced footprint • Performance • Security 08/11/2017 @danielbryantuk
- 25.
- 26. Building images withJenkins • My report covers this • Build as usual… • Build Docker Image • Cloudbees Docker Build and Publish Plugin • Push image to registry 08/11/2017 @danielbryantuk
- 27. Storing in animage registry (DockerHub) 08/11/2017 @danielbryantuk
- 28. Metadata – Bewareof “latest” Docker Tag • Beware of the ‘latest’ Docker tag • “Latest” simply means • the last build/tag that ran without a specific tag/version specified • Ignore “latest” tag • Version your tags, every time • danielbryantuk/test:2.4.1 08/11/2017 @danielbryantuk
- 29. Lesson learned: Metadatais valuable • Application metadata • Version / GIT SHA • Build metadata • Build date • Image name • Vendor • Quality metadata • QA control, signed binaries, ephemeral support • Security profiles (AppArmor), Security audited etc 08/11/2017 @danielbryantuk
- 30. Metadata - AddingLabels at build time • Docker Labels • Add key/value data to image 08/11/2017 @danielbryantuk
- 31. Metadata - AddingLabels at build time • Microscaling Systems’ Makefile • Labelling automated builds on DockerHub (h/t Ross Fairbanks) • Create file ‘/hooks/build’ • label-schema.org • microbadger.com 08/11/2017 @danielbryantuk
- 32. Metadata - AddingLabels at runtime 08/11/2017 @danielbryantuk $ docker run -d --label uk.co.danielbryant.lbname=frontdoor nginx • Can ’docker commit’, but creates new image • Not possible to update running container • Docker Proposal: Update labels #21721
- 33. Liz Rice (andAqua) to the rescue! 08/11/2017 @danielbryantuk github.com/aquasecurity/manifesto
- 34. External registry withmetadata support 08/11/2017 @danielbryantuk
- 35.
- 36.
- 37. Testing: Jenkins Pipeline(as code) 08/11/2017 @danielbryantuk
- 38.
- 39. Testing individual containers 08/11/2017@danielbryantuk
- 40.
- 41. Introducing Docker Compose 08/11/2017@danielbryantuk
- 42. Docker Compose &Jenkins Pipeline 08/11/2017 @danielbryantuk
- 43. Ephemeral Kubernetes Clusters •Kubernaut (WIP) • Manages a pool of clusters • ”Claim” a fresh cluster • Use Helm to install dependencies 08/11/2017 @danielbryantuk
- 44. Testing NFRs inthe build pipeline • Performance and Load testing • Gatling / jmeter • Flood.io • Security testing • Findsecbugs / OWASP Dependency check • Bdd-security (OWASP ZAP) / Arachni • Gauntlt / Serverspec • Docker Bench for Security / CoreOS Clair 08/11/2017 @danielbryantuk
- 45.
- 46.
- 47. Delaying NFRs tothe ‘Last Responsible Moment’ Newsflash! Sometimes the last responsible moment is up-front Modern platforms/architectures don’t necessarily make this easier 08/11/2017 @danielbryantuk
- 48. Mechanical sympathy: Dockerand Java • Watch for JVM cgroup/taskset awareness • getAvailableProcessors() may incorrectly report the number of cpus in Docker (JDK-8140793) • Runtime.availableProcessors() ignores Linux taskset command (JDK-6515172) • Default fork/join thread pool sizes (and others) is based from host CPU count • Set container memory appropriately • JVM requirements = Heap size (Xmx) + Metaspace + JVM overhead • Account for native thread requirements e.g. thread stack size (Xss) • Entropy • Host entropy can soon be exhausted by crypto operations and /dev/random blocks • -Djava.security.egd=file:/dev/./urandom (notes on this) 08/11/2017 @danielbryantuk 48
- 49.
- 50. Observability is coreto continuous delivery 08/11/2017 @danielbryantuk www.infoq.com/articles/monitoring-containers-at-scale
- 51. Containers are nota silver bullet 08/11/2017 @danielbryantuk
- 52. Moving to containers:Going all-in? 08/11/2017 @danielbryantuk OR
- 53. Should I buildmy own container platform? Probably not (Unless you are Google, AWS or IBM) Whatever you decide… push it through a pipeline ASAP! 08/11/2017 @danielbryantuk
- 54. Using containers doesnot get rid of the need for good architectural practices 08/11/2017 @danielbryantuk
- 55.
- 56.
- 57. In summary • Continuousdelivery is vitally important in modern architectures/ops • Container images must be the (single) source of truth within pipeline • And metadata added as appropriate… • Mechanical sympathy is important (assert properties in the pipeline) • Not all developers are operationally aware • The tooling is now becoming stable/mature • We need to re-apply existing CD practices with new technologies/tooling 08/11/2017 @danielbryantuk
- 58.
- 59. Thanks for listening! •Book signing in 10 mins! • Follow me to the O’Reilly table • Feel free to contact me • @danielbryantuk • daniel.bryant@tai-dev.co.uk 08/11/2017 @danielbryantuk bit.ly/2jWDSF7 Coming soon!
- 60. Bonus slides (forextra context) 08/11/2017 @danielbryantuk
- 61. Containerise an existing(monolithic) app? • For • We know the monolith well • Allows homogenization of the pipeline and deployment platform • Can be a demonstrable win for tech and the business • Against • Can be difficult (100+ line scripts) • Often not designed for operation within containers, nor cloud native • Putting lipstick on a pig? 08/11/2017 @danielbryantuk
- 62. Key lessons learned •Conduct an architectural review • Architecture for Developers, by Simon Brown • Architecture Interview, by Susan Fowler • Look for data ingress/egress • File system access • Support resource constraints/transience • Optimise for quick startup and shutdown • Evaluate approach to concurrency • Store configuration (secrets) remotely 08/11/2017 @danielbryantuk
- 63. New design patterns 08/11/2017@danielbryantuk bit.ly/2efe0TP
- 64. Microservices… Containers and microservicesare complementary Testing and deployment change 08/11/2017 @danielbryantuk https://specto.io/blog/recipe-for-designing-building-testing-microservices.html
- 65.
- 66.
- 67.
- 68. Quick Aside: Running*entire* system locally 08/11/2017 @danielbryantuk https://news.ycombinator.com/item?id=13960107 https://opencredo.com/working-locally-with-microservices/ https://www.datawire.io/telepresence/ | https://hoverfly.io/