Successfully reported this slideshow.
Your SlideShare is downloading. ×

Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly"

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 68 Ad

Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly"

Download to read offline

Implementing a continuous delivery (CD) pipeline is not trivial, and the introduction of container technology to the development stack can introduce additional challenges and requirements. In this talk we will look at the high-level steps that are essential for creating an effective pipeline for creating and deploying containerized applications.

Topic covered include:

The impact of containers on Java and Continuous Delivery
Adding metadata to container images
Validating NFR changes imposed by executing Java applications within a container
Lessons learned the hard way (in production)
A supporting O'Reilly report "Containerizing Continuous Delivery in Java" will also be available, and this contains instructions and code for how to create a Jenkins-based continuous delivery pipeline that takes a series of Java applications and containerizes them, ready for functional and nonfunctional testing, and ultimately, deployment.

Implementing a continuous delivery (CD) pipeline is not trivial, and the introduction of container technology to the development stack can introduce additional challenges and requirements. In this talk we will look at the high-level steps that are essential for creating an effective pipeline for creating and deploying containerized applications.

Topic covered include:

The impact of containers on Java and Continuous Delivery
Adding metadata to container images
Validating NFR changes imposed by executing Java applications within a container
Lessons learned the hard way (in production)
A supporting O'Reilly report "Containerizing Continuous Delivery in Java" will also be available, and this contains instructions and code for how to create a Jenkins-based continuous delivery pipeline that takes a series of Java applications and containerizes them, ready for functional and nonfunctional testing, and ultimately, deployment.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly" (20)

Advertisement

More from Daniel Bryant (20)

Advertisement

Devoxx 2017 "Continuous Delivery with Containers: The Good, the Bad, and the Ugly"

  1. 1. Continuous Delivery with Containers: The Good, the Bad, and the Ugly Daniel Bryant @danielbryantuk
  2. 2. Containers: Expectations versus reality 08/11/2017 @danielbryantuk “DevOps”
  3. 3. Setting the scene… • Continuous delivery is a large topic • No business focus today (value stream etc) • PaaS and Serverless are super interesting… • But I’m assuming you’re all-in on containers • Focusing today on the process and tooling • No live coding today • Mini-book contains more details (thanks nginx!) 08/11/2017 @danielbryantuk bit.ly/2jWDSF7
  4. 4. TL;DR – Containers and CD • Container image becomes the build pipeline ‘single binary’ • Adding metadata to containers images is vital, but challenging • Must validate container constraints on system quality attributes (NFRs) • Cultivate container ‘mechanical sympathy’ 08/11/2017 @danielbryantuk
  5. 5. @danielbryantuk • Independent Technical Consultant, CTO at SpectoLabs • Architecture, DevOps, Java, microservices, cloud, containers • Continuous Delivery (CI/CD) advocate • Leading change through technology and teams 08/11/2017 @danielbryantuk
  6. 6. @danielbryantuk 08/11/2017 @danielbryantuk
  7. 7. Continuous Delivery 08/11/2017 @danielbryantuk
  8. 8. Continuous Delivery • Produce valuable and robust software in short cycles • Optimising for feedback and learning • Not (necessarily) Continuous Deployment 08/11/2017 @danielbryantuk
  9. 9. Creation of a build pipeline is mandatory for continuous delivery 08/11/2017 @danielbryantuk
  10. 10. 08/11/2017 @danielbryantuk Feedback: - Was our initial hypothesis proven? - How can we improve business, architecture and ops?
  11. 11. The impact of containers on CD 08/11/2017 @danielbryantuk
  12. 12. Container technology (and CD) • OS-level virtualisation • cgroups, namespaces, rootfs • Package and execute software • Container image == ‘single binary’ 08/11/2017 @danielbryantuk
  13. 13. 08/11/2017 @danielbryantuk
  14. 14. 08/11/2017 @danielbryantuk
  15. 15. Microservices multiply the challenges 08/11/2017 @danielbryantuk
  16. 16. Creating a pipeline for containers 08/11/2017 @danielbryantuk
  17. 17. 08/11/2017 @danielbryantuk
  18. 18. Make your dev environment like production • Develop locally or copy/code in container • Must build/test containers locally • Perform (at least) happy path tests • Use identical base images from production • With same configuration 08/11/2017 @danielbryantuk
  19. 19. Lesson learned: Dockerfile content is super important • OS choice • Configuration • Build artifacts • Exposing ports • Java • JDK vs JRE and Oracle vs OpenJDK? • Golang • Statically compiled binary in scratch? • Python • Virtualenv? 08/11/2017 @danielbryantuk
  20. 20. Please talk to the sysadmin people: Their operational knowledge is invaluable 08/11/2017 @danielbryantuk
  21. 21. Different test and prod containers? • Create “test” version of container • Full OS (e.g. Ubuntu) • Test tools and data • Easy to see app/configuration drift • Use test sidecar containers instead • ONTEST proposal by Alexi Ledenev 08/11/2017 @danielbryantuk http://blog.terranillius.com/post/docker_testing/
  22. 22. Docker multi-stage builds 08/11/2017 @danielbryantuk http://blog.alexellis.io/mutli-stage-docker-builds/ https://github.com/moby/moby/pull/31257 https://github.com/moby/moby/pull/32063
  23. 23. Java specific stuff… 08/11/2017 @danielbryantuk github.com/oracle/docker-images/tree/master/OracleJava jdk.java.net/9/ea
  24. 24. Hot off the press: Modularity • Create minimal runtime images • “jlink delivers a self-contained distribution of your application and the JVM, ready to be shipped.” • Benefits: • Reduced footprint • Performance • Security 08/11/2017 @danielbryantuk
  25. 25. 08/11/2017 @danielbryantuk
  26. 26. Building images with Jenkins • My report covers this • Build as usual… • Build Docker Image • Cloudbees Docker Build and Publish Plugin • Push image to registry 08/11/2017 @danielbryantuk
  27. 27. Storing in an image registry (DockerHub) 08/11/2017 @danielbryantuk
  28. 28. Metadata – Beware of “latest” Docker Tag • Beware of the ‘latest’ Docker tag • “Latest” simply means • the last build/tag that ran without a specific tag/version specified • Ignore “latest” tag • Version your tags, every time • danielbryantuk/test:2.4.1 08/11/2017 @danielbryantuk
  29. 29. Lesson learned: Metadata is valuable • Application metadata • Version / GIT SHA • Build metadata • Build date • Image name • Vendor • Quality metadata • QA control, signed binaries, ephemeral support • Security profiles (AppArmor), Security audited etc 08/11/2017 @danielbryantuk
  30. 30. Metadata - Adding Labels at build time • Docker Labels • Add key/value data to image 08/11/2017 @danielbryantuk
  31. 31. Metadata - Adding Labels at build time • Microscaling Systems’ Makefile • Labelling automated builds on DockerHub (h/t Ross Fairbanks) • Create file ‘/hooks/build’ • label-schema.org • microbadger.com 08/11/2017 @danielbryantuk
  32. 32. Metadata - Adding Labels at runtime 08/11/2017 @danielbryantuk $ docker run -d --label uk.co.danielbryant.lbname=frontdoor nginx • Can ’docker commit’, but creates new image • Not possible to update running container • Docker Proposal: Update labels #21721
  33. 33. Liz Rice (and Aqua) to the rescue! 08/11/2017 @danielbryantuk github.com/aquasecurity/manifesto
  34. 34. External registry with metadata support 08/11/2017 @danielbryantuk
  35. 35. 08/11/2017 @danielbryantuk
  36. 36. Component testing 08/11/2017 @danielbryantuk
  37. 37. Testing: Jenkins Pipeline (as code) 08/11/2017 @danielbryantuk
  38. 38. 08/11/2017 @danielbryantuk
  39. 39. Testing individual containers 08/11/2017 @danielbryantuk
  40. 40. Integration testing 08/11/2017 @danielbryantuk
  41. 41. Introducing Docker Compose 08/11/2017 @danielbryantuk
  42. 42. Docker Compose & Jenkins Pipeline 08/11/2017 @danielbryantuk
  43. 43. Ephemeral Kubernetes Clusters • Kubernaut (WIP) • Manages a pool of clusters • ”Claim” a fresh cluster • Use Helm to install dependencies 08/11/2017 @danielbryantuk
  44. 44. Testing NFRs in the build pipeline • Performance and Load testing • Gatling / jmeter • Flood.io • Security testing • Findsecbugs / OWASP Dependency check • Bdd-security (OWASP ZAP) / Arachni • Gauntlt / Serverspec • Docker Bench for Security / CoreOS Clair 08/11/2017 @danielbryantuk
  45. 45. 08/11/2017 @danielbryantuk www.owasp.org/index.php/OWASP_Dependency_Check
  46. 46. 08/11/2017 @danielbryantuk github.com/arminc/clair-scanner
  47. 47. Delaying NFRs to the ‘Last Responsible Moment’ Newsflash! Sometimes the last responsible moment is up-front Modern platforms/architectures don’t necessarily make this easier 08/11/2017 @danielbryantuk
  48. 48. Mechanical sympathy: Docker and Java • Watch for JVM cgroup/taskset awareness • getAvailableProcessors() may incorrectly report the number of cpus in Docker (JDK-8140793) • Runtime.availableProcessors() ignores Linux taskset command (JDK-6515172) • Default fork/join thread pool sizes (and others) is based from host CPU count • Set container memory appropriately • JVM requirements = Heap size (Xmx) + Metaspace + JVM overhead • Account for native thread requirements e.g. thread stack size (Xss) • Entropy • Host entropy can soon be exhausted by crypto operations and /dev/random blocks • -Djava.security.egd=file:/dev/./urandom (notes on this) 08/11/2017 @danielbryantuk 48
  49. 49. Deployment 08/11/2017 @danielbryantuk skillsmatter.com/skillscasts/10668-looking-forward-to-daniel-bryant-talk docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.deploy-existing-version.html
  50. 50. Observability is core to continuous delivery 08/11/2017 @danielbryantuk www.infoq.com/articles/monitoring-containers-at-scale
  51. 51. Containers are not a silver bullet 08/11/2017 @danielbryantuk
  52. 52. Moving to containers: Going all-in? 08/11/2017 @danielbryantuk OR
  53. 53. Should I build my own container platform? Probably not (Unless you are Google, AWS or IBM) Whatever you decide… push it through a pipeline ASAP! 08/11/2017 @danielbryantuk
  54. 54. Using containers does not get rid of the need for good architectural practices 08/11/2017 @danielbryantuk
  55. 55. 08/11/2017 @danielbryantuk https://speakerdeck.com/caseywest/containercon-north-america-cloud-anti-patterns
  56. 56. Summary 08/11/2017 @danielbryantuk
  57. 57. In summary • Continuous delivery is vitally important in modern architectures/ops • Container images must be the (single) source of truth within pipeline • And metadata added as appropriate… • Mechanical sympathy is important (assert properties in the pipeline) • Not all developers are operationally aware • The tooling is now becoming stable/mature • We need to re-apply existing CD practices with new technologies/tooling 08/11/2017 @danielbryantuk
  58. 58. Bedtime reading 08/11/2017 @danielbryantuk
  59. 59. Thanks for listening! • Book signing in 10 mins! • Follow me to the O’Reilly table • Feel free to contact me • @danielbryantuk • daniel.bryant@tai-dev.co.uk 08/11/2017 @danielbryantuk bit.ly/2jWDSF7 Coming soon!
  60. 60. Bonus slides (for extra context) 08/11/2017 @danielbryantuk
  61. 61. Containerise an existing (monolithic) app? • For • We know the monolith well • Allows homogenization of the pipeline and deployment platform • Can be a demonstrable win for tech and the business • Against • Can be difficult (100+ line scripts) • Often not designed for operation within containers, nor cloud native • Putting lipstick on a pig? 08/11/2017 @danielbryantuk
  62. 62. Key lessons learned • Conduct an architectural review • Architecture for Developers, by Simon Brown • Architecture Interview, by Susan Fowler • Look for data ingress/egress • File system access • Support resource constraints/transience • Optimise for quick startup and shutdown • Evaluate approach to concurrency • Store configuration (secrets) remotely 08/11/2017 @danielbryantuk
  63. 63. New design patterns 08/11/2017 @danielbryantuk bit.ly/2efe0TP
  64. 64. Microservices… Containers and microservices are complementary Testing and deployment change 08/11/2017 @danielbryantuk https://specto.io/blog/recipe-for-designing-building-testing-microservices.html
  65. 65. 08/11/2017 @danielbryantuk
  66. 66. 08/11/2017 @danielbryantuk
  67. 67. 08/11/2017 @danielbryantuk
  68. 68. Quick Aside: Running *entire* system locally 08/11/2017 @danielbryantuk https://news.ycombinator.com/item?id=13960107 https://opencredo.com/working-locally-with-microservices/ https://www.datawire.io/telepresence/ | https://hoverfly.io/

×