Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Chicago JUG / GOTO Meetup

A quick intro to some of the tradeoffs between structured and unstructured cloud approaches, and a demo of a possible use of Mesos/Marathon for desktop-to-cloud development

  • Login to see the comments

  • Be the first to like this

Chicago JUG / GOTO Meetup

  1. 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Full Stack Java Microservices with Mesos/Marathon and Docker How to take an app server and blow it up into a lots of pieces…and s5ll be able to call it a “win”. Edward Burns @edburns ConsulKng Member of Technical Staff April 2017
  2. 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direcKon. It is intended for informaKon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcKonality, and should not be relied upon in making purchasing decisions. The development, release, and Kming of any features or funcKonality described for Oracle’s products remains at the sole discreKon of Oracle.
  3. 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Speaker QualificaKons – Ed Burns •  25 Years SoXware Development Experience – NCSA Mosaic è Marathon and much in between – Spec Lead for Java EE Servlet and JSF •  Informal spare Kme trainer •  Author of four IT-related books for McGraw-Hill
  4. 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Speaker QualificaKons – Ed Burns •  The PerspecKve I offer – On the front lines of the disrupKon caused by the shiX to cloud naKve development – I work on things that have historically enabled “The Monolith” •  Java EE: JSF, Servlet •  App Servers: GlassFish and WebLogic – I understand the need to adapt to changing developer requirements •  I also understand the perils of chasing the shiny new thing •  Blog hdp://ridingthecrest.com/blog/
  5. 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The New Stack •  Economic forces are driving the move to the cloud – More efficiency •  The commodiKzaKon of runKmes is nearly complete – CPU/Storage/Network: all virtual, all billable per-use – Several choices of vendors •  Much of the funcKon of an IT department can be codified into a Service Level Agreement and purchased from a cloud vendor – “SoXware defined data center”
  6. 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The New Stack •  Economic forces are driving the move to the cloud – Beder environment for conKnuous improvement •  At runKme: horizontal scalability •  At innovaKon Kme: conKnuous delivery in pracKce
  7. 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Architecture QuesKons
  8. 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Source hdp://www.artofadventure.net/two-paths- and-the-road-not-taken/ Two Paths for Cloud NaKve Enterprises
  9. 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Two Paths for Cloud NaKve Enterprises •  Structured Plagorms – You buy into their parKcular concept layout and soluKon assignment for each of the boxes •  Unstructured Plagorms – You make up your own boxes and decide how to fill them out, usually with off-the-shelf open source offerings – You have to decide how to provision/host/run/maintain it
  10. 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Two Paths for Cloud NaKve Enterprises •  You’re sKll going to need a lot of support technology in either approach •  Build System •  CI/CD Pipeline – TesKng •  ArKfact management •  Version Control – Source code – Binary code
  11. 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The ExisKng Java EE Stack is SKll Useful •  One of the only two games in town before cloud disrupKon, the other being .NET •  Structured when Structured wasn’t cool •  What is a monolithic app? That usually means “app server”. –  You get a lot of benefits with the app server •  Simplified deployment model: some collecKon of WAR/EAR files •  Easy ORM: JPA database connecKon •  Easy redundancy: clustering support •  Easy load balancing •  Easy TransacKonality: JTA •  Statefulness •  Reliability •  Debugability and logging
  12. 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The ExisKng Java EE Stack is SKll Useful •  But there are also problems – Ops departments may not support WAR/EAR deployment model and prefer GNU/ Linux naKve packages such as RPMs – Best pracKce for app servers seems to have converged on the “one app per app server” model. This is a lot of wasted resources. – Tend to be vulnerable to environment specific problems: not repeatable. – Not as easy to do massive horizontal scaling
  13. 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The ExisKng Java EE Stack is SKll Useful • The good stuff from Java EE – Easy programming model • Java is sKll predy great • Stable, Kme proven technologies • Rich tooling • Root cause analysis is well understood and manageable • Leverage eventual consistency where beneficial
  14. 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | New Stack Risks • It is new – Best pracKces sKll emerging – Hard to stay up to date with them as they emerge – Lots of soluKons are offered as side effects that started out as enabling other businesses • LyX Confidant • Neglix Hystrix • AWS itself • Google cloud • … your needs are not their needs – Open Source as employee retenKon • Employees benefit by building their reputaKon
  15. 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | New Stack Risks • It’s a much more “roll your own” approach – RBAC – Security and AuthorizaKon • Everyone has to be more of a sysadmin • No more single neck to strangle • Subject to the “flavor of the day” syndrome • With containers you need to know exactly what is in your containers – What versions of all the components?
  16. 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | New Stack Rewards •  ContainerizaKon – Package dependencies •  Resiliency – Beder upKme – Beder response Kme •  Enable things such as eventual consistency and CQRS •  Beder embrace iteraKon lifestyle
  17. 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | How to Get Ready •  You have to be more of a generalist •  There is no more “throwing it over the wall”, everyone has to think like a sysadmin •  Study up on Linux Sysadmin skills – Systemd iniKalizaKon tool – Package management and proxy navigaKon – Networking, including the OSI protocol stack – Get some familiarity with Ansible •  Study up on build systems •  Study up on arKfact management systems •  Study up on JVM tuning arguments
  18. 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | No Catchy Name •  Lots of stacks have catchy names – LAMP – SMACK – MEAN – MERN – hdps://en.wikipedia.org/wiki/SoluKon_stack •  I don’t have a catchy name for this stack – Plagorm as a Service on bare metal
  19. 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Conceptual Layout Foundation Layer Support Layer Business Layer Your code goes here But with DevOps, you must understand here… … and here
  20. 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Quick Tour of the Stack: FoundaKon Layer •  OperaKng System – networking •  Node management •  ContainerizaKon •  Scheduling of high level processes •  Load balancing •  Binary arKfact management •  GNU/Linux – SDN •  Apache Mesos •  Docker •  Marathon •  NGINX •  ArKfactory Conceptual Layout One Possible Concrete Implementa5on Layout
  21. 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Quick Tour of the Stack: Support Layer •  Logging •  Telemetry •  Lightweight Persistence •  Heavyweight Persistence •  IdenKty and Secret Management •  ELK •  Prometheus •  Etcd •  NoSQL DB (MongoDB) •  Hashicorp Vault Conceptual Layout One Possible Implementa5on Layout
  22. 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Quick Tour of the Stack: Business Layer •  ContainerizaKon technology •  Interchange technology •  Programming Language •  Docker •  REST (JAX-RS/Jersey) •  Java 8 Conceptual Layout One Possible Implementa5on Layout
  23. 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Aside: Which Container Technology? •  Docker vs Rkt (pronounced rock-it) – hdps://coreos.com/blog/rkt-accepted-into-the-cncf.html – hdp://containerops.org/2014/12/19/docker-vs-rocket-gimme-a-break/ – hdps://medium.com/@adriaandejonge/moving-from-docker-to- rkt-310dc9aec938#.szvh6atkt – Rkt can run Docker images •  Converts container to appc format •  CoreOS (maker of rkt) and Docker – Open Container IniKaKve hdps://www.opencontainers.org/ •  Image SpecificaKon •  RunKme SpecificaKon
  24. 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Installing and Running the Stack •  The challenge is to have the developer environment be as similar as possible to the producKon environment. – How much resides on the local developer machine? – How to integrate with ConKnuous IntegraKon and ConKnuous Delivery?
  25. 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Moving Parts of the Stack: FoundaKon Layer •  Apache Mesos hdp://mesos.apache.org/ – Allows abstracKng a computer as •  CPU/Memory/IO/Storage – Allows starKng and stopping containers: containerizer •  Support for Docker, rkt and appc containers – Runs as a “mesos cluster” •  Three kinds of nodes – Master (at least one) – Slave (at least one) – Zookeeper (cluster) (at least one, but it can run as a cluster)
  26. 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Moving Parts of the Stack: FoundaKon Layer •  Marathon hdps://mesosphere.github.io/marathon/ •  Is a Mesos “Framework” – Handles scale up/scale down – Provides a deployment descriptor concept – Introduces the concept of an “ApplicaKon” – Storage accounKng – Health checks – Metrics – Constraints (mapping applicaKons to racks/nodes, etc)
  27. 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Moving Parts of the Stack: FoundaKon Layer •  Marathon hdps://mesosphere.github.io/marathon/ •  Has a REST API for gezng its work done – hdps://mesosphere.github.io/marathon/docs/rest-api.html – There is a not maintained Java API to this rest client hdps://github.com/mohitsoni/marathon-client •  Can build your own orchestrator on top of it
  28. 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Moving Parts of the Stack: FoundaKon Layer •  Demo: Show foundaKon layer running using docker maven plugin Thanks Manfred Riem <hdp://www.manorrock.com/blog/> –  ReplicaKon of some part of producKon env –  It is also runnable under CI –  Review pom.xml •  address uid/gid permissions issues •  address docker network issues –  Review deployment json –  Need to access logs in a special way •  Can’t simply docker logs –f <hash> •  docker exec into mesos slave •  cd /var/tmp/mesos/slaves/*/frameworks/*/executors/mycontainer8g*/runs/latest and tail -f stderr|stdout
  29. 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Moving Parts of the Stack: Support Layer •  A wide collecKon of off-the-shelf support services, running on top of the foundaKon layer •  Look at Three Specific Examples – NGINX – ELK + FileBeat – Etcd •  Other Services – Services to enable security in the Business Layer
  30. 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Moving Parts of the Stack: Business Layer •  Docker container with Java fat jar inside •  Must keep much in mind – hdps://developers.redhat.com/blog/2017/03/14/java-inside-docker/ – hdp://ridingthecrest.com/blog/2017/02/23/ray-tsang-docker-Kps-notes.html
  31. 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Running the Stack: Maven •  Demo: Show foundaKon layer running using docker maven plugin Thanks Manfred Riem hdp://www.manorrock.com/blog/ •  Plugins with InteresKng Uses – maven-antrun-plugin •  Convey current UID/GID all the way through to the Docker running the workloads •  Assemble the individual service Marathon deployment descriptors into one json file •  Use curl to install the Business Layer app before tesKng and remove it aXer tesKng – maven-resources-plugin •  Template properKes into json files for ports, variables, configuraKon – Fabric8 docker-maven-plugin •  Stand up foundaKon layer, note hard-coded network values
  32. 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Running the Stack: Maven •  Demo: Show foundaKon layer running using docker maven plugin Thanks Manfred Riem hdp://www.manorrock.com/blog/ •  Plugins with the Usual Uses – maven-compiler-plugin – maven-deploy-plugin – maven-failsafe-plugin •  Able to use standard Junit with Jersey Client to assert correctness of Business layer app •  Need to pass system properKes for endpoint locaKon – maven-javadoc-plugin – maven-source-plugin
  33. 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Java EE 8 Will Be Available by JavaOne 2017!
  34. 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 34 Java EE 7 Connector JAXB JSP Debugging Managed Beans JSP Concurrency EE Interceptors JAX-WS WebSocket Bean ValidaKon JASPIC Servlet JMS JTA Deployment Batch JACC Dependency InjecKon JAXR JSTL Management CDI EJB JAX-RPC Web Services JSF Java Persistence JSON-P Common AnnotaKons EL JAX-RS Web Services Metadata JavaMail CDI JSON-B Security Bean ValidaKon JSF JAX-RS JSON-P Servlet Java EE 8 Will Be Available by JavaOne 2017! JSP
  35. 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The preceding is intended to outline our general product direcKon. It is intended for informaKon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcKonality, and should not be relied upon in making purchasing decisions. The development, release, and Kming of any features or funcKonality described for Oracle’s products remains at the sole discreKon of Oracle. 35

×