server to cloud: converting a legacy platform to an open source paas


Published on

This session discusses the process to move legacy applications "into the cloud". It is intended for a diverse audience including developers, architects, and managers. We will discuss techniques, methodologies, and thought processes used to analyze, design, and execute a migration strategy and implementation plan -- from planning through rollout and operational.

An important aspect of this is the necessity for technical staff to effectively communicate to mid-level management how these design decisions and strategies translate into cost, complexity and schedule.

Commonly used migration strategies, cloud technologies, architecture options, and low level technologies will be discussed.

The case will be made that investing in strategic refactoring and decomposition during the migration will reap the benefits of a modern, decoupled and simplified system.

The end game being alignment and adoption of current best practices around PaaS, Saas, SOA, event-driven architectures, and message-oriented middleware, at scale in the cloud, to provide quantifiable business value.

This talk will focus more on the big picture, at times delving into technical architectures and discussion of certain technologies and service providers.

Use of Containers (Docker) is evangelized for decoupling and decomposing legacy systems.

Published in: Software, Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

server to cloud: converting a legacy platform to an open source paas

  1. 1. inBloom, Inc. server to cloud converting a legacy platform to an open source paas todd fritz April 3. 2014
  2. 2. inBloom, Inc. agenda • whoami • goals • change • genesis • decompose • refactor • architecture • ajug • questions • references • appendix APRIL 2014 1
  3. 3. inBloom, Inc. whoami bio: http://www.linkedin/com/in/tfritz • architect at inbloom  opinions contained within this presentation may not represent my employer, but I think they should • evangelist of layered, distributed, message-oriented-middleware • current focus is middleware through caching, nosql data store • exposed to different companies, projects, people and technologies • novice bass player • recent father of a five month old • scuba diver. next adventure:  APRIL 2014 2 No sleep for you!
  4. 4. inBloom, Inc. thanks credit for those who helped (or listened) • altisource  todd nist • inbloom  verlin henderson  paul lawler  vincent mayers  ben morgan  bill siggelkow • red hat  ray ploski APRIL 2014 3
  5. 5. inBloom, Inc. inBloom, Inc. email: twitter: @inBloomDev APRIL 2014 4
  6. 6. inBloom, Inc. goals APRIL 2014 5
  7. 7. inBloom, Inc. objectives for next 45 minutes • to delay you from happy hour and inspire curiosity • discuss the benefits of change • shine a light on a path forward • cover the if, why and how to modernize • define basic cloud migration criteria • explain techniques to decompose legacy apps • virtualization and containerization • evangelize containers as an architecture enabler • discuss migration strategy and architectures • this talk does not (yet) include code examples (coming soon to atlanta java users group) APRIL 2014 6
  8. 8. inBloom, Inc. change APRIL 2014 7
  9. 9. inBloom, Inc. what? change? why? APRIL 2014 8 an unknown manager in the wild, ready to spray a can of dilbert.
  10. 10. inBloom, Inc. may we live in interesting times innovation is impossible without change • ―Software is eating the world.‖ – Marc Andreesson, 2011, Wall Street Journal  ―We are in the middle of a dramatic and broad technological and economic shift in which software companies are poised to take over large swathes of the economy. More and more major businesses and industries are being run on software and delivered as online services— from movies to agriculture to national defense.‖ • ―Every company is a software company -- or at least aspiring to be one. That reality will shake up industries, lead to huge successes and failures and potentially make or break brands.‖ – Larry Dignan • ―…this software revolution in every company will be similar to how enterprise resource planning changed the game for businesses and their processes.‖ – Forrester analyst John McCarthy APRIL 2014 9
  11. 11. inBloom, Inc. innovation economics • a growing economic doctrine that adapts conventional economics theory so that knowledge, technology, entrepreneurship, and innovation are at the center of the model  rather than independent forces that are largely unaffected by policy  good fit for companies that manage software • two fundamental tenets: 1. economic policy should drive productivity through innovation 2. market reliance on resources and price signals alone may not be as effective to create both productivity and economic growth • differs from conventional economic doctrines • companies benefit from innovation in obvious ways APRIL 2014 10
  12. 12. inBloom, Inc. a path forward how does a company that has not traditionally been in the software space become a successful software company? also applies to a software company that is reinventing itself, and startups. 1. protect intellectual property 2. re-think software monetization models  based on how customers want to pay  subscription models 3. automate the entire software, device and entitlement lifecycle  software installation, provisioning, activation  subscription management, software updates and upgrades  software entitlements (features) 4. purpose-built software licensing and entitlement management APRIL 2014 11
  13. 13. inBloom, Inc. status quo gets left behind • today will not be tomorrow • cloud provides competitive advantage for some use cases  amazon is market leader  ongoing competition between cloud providers; amazon and google  encourages use standards and adoption of new tech and patterns • technology advances disrupt computing infrastructure and software to expose opportunity • quantum computing, e.g. d-wave qubit  ok, some controversy, but it is fast  significant advance in compute power  will disrupt software design and how we scale, e.g. compute grids  it is real, maturing, and is not going away  outpaces moore’s law as it scales  APRIL 2014 12
  14. 14. inBloom, Inc. cloud adoption – 36% increase – benefits APRIL 2014 13
  15. 15. inBloom, Inc. good candidates for cloud 1. strong business sponsorship at tip of spear 2. well understood with few dependencies 3. uses common standards and implementations 4. can be decomposed or decoupled 5. already modularized or service oriented 6. already virtualized 7. favorable cost-benefit analysis 8. security model translates to cloud 9. opportunities to improve by modernization APRIL 2014 14
  16. 16. inBloom, Inc. unfavorable candidates – who has one? 1. lacks strong business sponsorship 2. security equation difficult to understand 3. complex architecture and external dependencies 4. technology or deployment lock in 5. latency sensitive (real time apps) 6. not standards based or uses home-grown libraries 7. etl heavy (not parallelized) or long running jobs 8. government regulations, contractual requirements, or certifications (pci) 9. unfavorable economics (cost-benefit/risk analysis) 10. transaction lifecycles not well understood APRIL 2014 15
  17. 17. inBloom, Inc. the go forward decision APRIL 2014 16
  18. 18. inBloom, Inc. genesis APRIL 2014 17
  19. 19. inBloom, Inc. hello world! i am a legacy system APRIL 2014 18
  20. 20. inBloom, Inc. what to do with it? break it apart; analyze employ an iterative methodology  leverage existing knowledge and literature  some analysis before  decompose and modularize (scale cube)  separation of concerns  proof of concepts are your friend deployment environments  private servers  cloud (private, public)  hybrid platform architectures  legacy  paas  xpaas APRIL 2014 19
  21. 21. inBloom, Inc. terms • paas  “Platform as a service (PaaS)…provides a computing platform and a solution stack as a service. Along with software as a service (SaaS) and infrastructure as a service (IaaS), it is a service model of cloud computing… the consumer creates the software using tools…from the provider. The consumer also controls software deployment and configuration... The provider provides the networks, servers, storage, and other services that are required to host the consumer's application” -wikipedia • xpaas  standardization of enterprise paas  create platforms from a catalogue of paas/saas/iaas APRIL 2014 20
  22. 22. inBloom, Inc. paas by segment gartner’s 2012 paas market share chart (from red hat’s xpaas whitepaper). APRIL 2014 21
  23. 23. inBloom, Inc. xpaas topology APRIL 2014 22 • enables specialization, layering, separated concerns, decoupling
  24. 24. inBloom, Inc. decompose APRIL 2014 23
  25. 25. inBloom, Inc. i am the fragments of a legacy system APRIL 2014 24 hear me roar…
  26. 26. inBloom, Inc. what have I gotten myself into? APRIL 2014 25 “legacy spaghetti” the code sucks… i can’t even build it…
  27. 27. inBloom, Inc. getting started • involve devops from the beginning • if your company does not have devops then call pressureManager() • make decisions from quantitative assessments • agile systems analysis and integration modeling • agile modeling best practices • use the afk scale cube  services-for-scale/ • read ―The Art of Scalability‖ by abbott and fisher  • perform functional decomposition and service identification • the cloud is not ―all or nothing‖  phased migrations can realize immediate value  new technologies can coexist with legacy • iterative design ahead • use containers to enable decoupling, architectural flexibility, confine legacy APRIL 2014 26
  28. 28. inBloom, Inc. legacy analysis and design • decompose into layers; functions and separate concerns • design decoupled components and services • isolate technologies within components (insulate lock-in) • conceptualize legacy components and services as ―legos‖ • future state runs alongside current state to provide value • take heed of transaction lifecycle, batch jobs, data retention and use cases • be mindful of customer impact, cost and schedule constraints • consider security at each layer and service APRIL 2014 27
  29. 29. inBloom, Inc. success – value add criteria APRIL 2014 28 “For every complex question there is a simple and wrong solution.” - Albert Einstein avoid “Khan’s paradigm”: a top-down plan led by an overlord of super-humans will fail. my Java kung fu can crush any project. tactical • time, cost, quality strategic • sustainability, relevance, effect
  30. 30. inBloom, Inc. agile modeling best practices APRIL 2014 29
  31. 31. inBloom, Inc. refactor APRIL 2014 30
  32. 32. inBloom, Inc. assembling the new system APRIL 2014 31
  33. 33. inBloom, Inc. we can rebuild it • cynical optics  rather than one bowl of spaghetti; several small, independent plates  investment in analysis, design ahead, separating concerns has tangible benefits  use containers! APRIL 2014 32 before after
  34. 34. inBloom, Inc. putting it together • favor continuous deployment • prefer reusable, modularized components • decoupled services; soa done right (microservice) • message oriented and event driven • parallelize development across business function • select the right tool for each component; polyglot • include automated unit and integration tests • leverage containerization instead of virtualization when possible (see next slide) APRIL 2014 33
  35. 35. inBloom, Inc. why containers? APRIL 2014 34
  36. 36. inBloom, Inc. revisting paas requirements • ―Virtualization vs. Containers to support PaaS‖  by Dua, Raja, Kakadia   Basis of next three slides • paas focuses on developer productivity and abstracts out underlying infrastructure • 3 key paas requirements for the infrastructure 1. network, compute and storage programmatically managed and provisioned 2. h/a infrastructure (e.g. nodes) efficiently utilized 3. ability to bind applications/services to external network (dns, routers) APRIL 2014 35
  37. 37. inBloom, Inc. paas requirements (cont.) 1. network, compute and storage programmatically managed and provisioned 2. h/a infrastructure (e.g. nodes) efficiently utilized 3. ability to bind applications/services to external network (dns, routers)  vms good for #1 & #2  apps can reside within: • vms • containers • vms with containers  containers better for #3 as resources better utilized and light weight APRIL 2014 36
  38. 38. inBloom, Inc. paas requirements (cont.) • containers have weaknesses (for now)  standardization  strong security  os independence  robust monitoring • ―Containers have inherent advantage over VMs for PaaS use case‖. - Dua, Raja, Kakadia APRIL 2014 37
  39. 39. inBloom, Inc. container platform contenders • warden (cloud foundry)  • docker  • google lmctfy (let me contain that for you)   december, 2013 • openvz APRIL 2014 38
  40. 40. inBloom, Inc.APRIL 2014 39 container of the day
  41. 41. inBloom, Inc. about docker docker is a micro container framework for paas • • open-source • easier scalability • lightweight, portable, insulated containers • reusable from dev (―local cloud‖) through production • can run at scale on vms, bare metal, cloud; virtually anywhere • encapsulate any payload (application) • run consistently on and between virtually any server APRIL 2014 40
  42. 42. inBloom, Inc. more about docker APRIL 2014 41
  43. 43. inBloom, Inc. brief overview about how it works • docker builds on lxc which offers system-level virtualization and has existed since linux 2.6.32 (December, 2009) – but use 3.8+ • docker has three parts  docker daemon runs as root to manage containers  docker containers spawn from images, which are tiny and can be versioned  docker repository allows images to be exchanged and versioned like code (public or private) • each container has its own ip address • link exposed ports and variables across containers through configuration using abassador containers to avoid hard coding (svendowideit):  consumer  redis-ambassador  redis • port and pipework to expose containers outside host • can share volumes, multi home, integrate containers into host network, and much more (ajug) • continuous integration can generate versioned docker images, web hooks, repo notifications • supervisor management tool to manage processes within container ( APRIL 2014 42
  44. 44. inBloom, Inc. even more about docker • common use cases:  automate application packaging and deployment  lightweight paas environments  automate testing, continuous integration, and deployment  deploy and scale web apps, databases, backend services • growing adoption since dec 2013 • red hat fast-tracks docker apps for enterprise linux  hat-fast-tracks-docker-apps-enterprise-linux-238122 • production ready deployment planned for december, 2014.  Support services planned for early 2015 APRIL 2014 43
  45. 45. inBloom, Inc. containers available for reuse APRIL 2014 44 a docker container image may already exist
  46. 46. inBloom, Inc. docker image registry – versioning! APRIL 2014 45 artifactory for virtualization images?
  47. 47. inBloom, Inc. New in Docker 0.9 • execution driver api  customize execution environment around container; enables use of other isolation tools • built in execution container - libcontainer  alongside lxc, boosts stability, insulates docker from different versions of lxc APRIL 2014 46
  48. 48. inBloom, Inc. architecture APRIL 2014 47
  49. 49. inBloom, Inc. containers enable architecture • containers enable architecture and design • design, build, or migrate, each layer or module into a container • containers encapsulate technology, isolate lock-in, and are easy to scale • enables use of best tool; go polyglot • easy to upgrade what’s in a container  replace node.js with vert.x  technology portability • understand workload and transaction use cases (data stores, social) • service oriented (done right, not soap) • learn message oriented middleware (mom)  enterprise integration patterns (eip)  apache camel  queues (amqp) • prefer data streams to batch jobs • leverage power of compute grids and distributed caching • pay the piper and dedicate effort to data/domain architecture APRIL 2014 48
  50. 50. inBloom, Inc. ajug APRIL 2014 49
  51. 51. inBloom, Inc. coming soon • v2 of this presentation  with code examples  in-depth architecture ideas  development concepts • to be presented to atlanta java users group (ajug) APRIL 2014 50
  52. 52. inBloom, Inc. questions APRIL 2014 51
  53. 53. inBloom, Inc. references APRIL 2014 52
  54. 54. inBloom, Inc. reference material • • • • • • to-cloud-computing • • • • • • • • • 2012/ • • • • • and-deployment/ APRIL 2014 53
  55. 55. inBloom, Inc. More Reference Material • Redmonk on DB technology: • • Decomposing applications for scalability and deployability  • • the-key-to-business-model-success-.html • virtual-maschines/ • containers/ • APRIL 2014 54
  56. 56. inBloom, Inc. appendix (slides that did not fit within time constraint) APRIL 2014 55
  57. 57. inBloom, Inc. paas -> xpaas an xpaas is composed of multiple, specialized paas systems ―…xPaaS services augment core container functionality with integration, business process management (BPM) and mobile capabilities.‖  integration paas (ipaas) -- simplifies connections, messages, route definitions, and data transformations  bpm paas (bpmpaas) -- process modeling, process engine, simplify definition and evolution of business processes  mobile paas (mpaas) -- push notifications, data synchronizations and back-end integration  … APRIL 2014 56
  58. 58. inBloom, Inc. agile legacy systems analysis and integration modeling APRIL 2014 57 Methodology:
  59. 59. inBloom, Inc. the scale cube APRIL 2014 58
  60. 60. inBloom, Inc. Analysis Considerations • Refine understanding of current and future state solutions  Client-Server (beware of stateful thick clients)  N-Tier  SaaS ready components? • Features, use cases, request lifecycle • Messaging models, usage patterns, volume and velocity – capacity planning • Messaging? Network configuration? • All aspects of security • Identify components and concerns, layers • Impact of latency • Caching, and Master Data Management (MDM) • Identify problems and do not ignore technical debt… • Product Management != Project Management (or scrum master) The unknown may lead to failure, delays or increased cost. APRIL 2014 59
  61. 61. inBloom, Inc. Security First FEBRUARY 2014 60 • Foundational • Assess all concerns and services • Network, OS, Disk (data at rest) • Applications • Encryption • Data • Protecting Personally Identifiable Information (PII) • Securing logged information • Securing in-memory objects and caches • User Authentication and Authorization • Manage users, groups, roles and permissions, SSO • Separate abstractions for Authentication from Authorization (CAS, etc) • OAUTH, SAML2 • Learn about SENDS • Science-Enhanced Networked Domains and Secure Social Spaces • Security needs to be more than a technology solution • behavior/72826/ • Be aware of industry bias toward technical solutions with security. • Can’t control stupid human behaviors such as taping a password to keyboard. • APRIL
  62. 62. inBloom, Inc. What? Perhaps you are wondering… • Why Docker? The next several slides provide background • (Borrowed from Docker’s site.) Keep in mind: • Docker containers encapsulate concerns • Avoids Holy Wars about specific architectures to implement • Great for the dozens of Spring projects..  Run each within its own Container APRIL 2014 61
  63. 63. inBloom, Inc. how a docker works • uses linux kernel for containment (jailing)  chroot: changes root directory of process w/ child  cgroups: control groups • groups processes (to unit of thread) • pseudo filesystem • numerous subsystems functions implemented – CPU sets, etc • linux containers: lxc  file system isolation  network and process isolation  resource allocation APRIL 2014 62
  64. 64. inBloom, Inc. Converting the Legacy App into Docker • Containers enable extreme choice – use what you want. • Container model facilitates separates concerns  Applications (various technologies)  Back End systems including ―Big Data‖  Messaging  Caching (e.g. Hazelcast!)  Microservices. Use Data as glue. APRIL 2014 63
  65. 65. inBloom, Inc.APRIL 2014 64
  66. 66. inBloom, Inc. Docker Ecosystem APRIL 2014 65
  67. 67. inBloom, Inc. Why Developers should Care about Docker APRIL 2014 66
  68. 68. inBloom, Inc. DevOps & Docker APRIL 2014 67
  69. 69. inBloom, Inc. More about Docker • Deis is a framework that caps Docker and Heroku   ―Deis…is an open source PaaS that makes it easy to deploy and scale Docker containers and Chef nodes used to host applications, databases, middleware and other services. Deis leverages Chef, Docker, Heroku Buildpacks to provide a private PaaS that is lightweight and flexible.‖  Supported Languages • Java, Scala, PHP, Ruby, Python, Node.js, Clojure, Play, Perl, Dart, Go. • Deis can deploy anything using Heroku Buildpacks or Dockerfiles.  Supported Providers • Any system including every public cloud, private cloud or bare metal. • Automatic provisioning for EC2, Rackspace, Digital Ocean • Integration testing with Maven and Docker  • Industry timeline from 1995 to Docker  • Decker APRIL 2014 68
  70. 70. inBloom, Inc. Message Oriented Middleware APRIL 2014 69