Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, the Ugly

1,708 views

Published on

CloudNativeCon North America 2017, Austin (Texas, USA): Talk by Josef Adersberger (@adersberger, CTO at QAware)

Abstract:

Running applications on Kubernetes can provide a lot of benefits: more dev speed, lower ops costs, and a higher elasticity & resiliency in production. Kubernetes is the place to be for cloud native apps. But what to do if you’ve no shiny new cloud native apps but a whole bunch of JEE legacy systems? No chance to leverage the advantages of Kubernetes? Yes you can!

We’re facing the challenge of migrating hundreds of JEE legacy applications of a major German insurance company onto a Kubernetes cluster within one year. We're now close to the finish line and it worked pretty well so far.

The talk will be about the lessons we've learned - the best practices and pitfalls we've discovered along our way. We'll provide our answers to life, the universe and a cloud native journey like:

- What technical constraints of Kubernetes can be obstacles for applications and how to tackle these?
- How to architect a landscape of hundreds of containerized applications with their surrounding infrastructure like DBs MQs and IAM and heavy requirements on security?
- How to industrialize and govern the migration process?
- How to leverage the possibilities of a cloud native platform like Kubernetes without challenging the tight timeline?

Published in: Technology
  • Are you literally FEEDING your diabetes putting this one "health" food on your dinner plate? This is important. You must stop eating this food today or you could be doubling the speed at which your diabetes progresses...  https://tinyurl.com/y2956vb5
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Greek Ritual REVERSES Diabetes (do this before bed) Before you go to bed tonight, do this ONE "stupidly simple" Greek ritual to reverse your diabetes...  https://bit.ly/2n5cFHd
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Earn Up To $316/day! Easy Writing Jobs from the comfort of home! ◆◆◆ http://t.cn/AieXSfKU
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, the Ugly

  1. 1. Migrating Hundreds of Legacy Applications to Josef Adersberger, CTO, QAware
 @adersberger
 }THE GOOD, THE BAD, THE UGLY proud CNCF member
  2. 2. HYPERSCALE RESILIENT OPEX SAVINGS SPEED
  3. 3. CIOLet’s bring 
 all our web 
 applications
 into the cloud! I’ve already 
 canceled our
 current data center 
 contract.
  4. 4. Chief Architect Great goal! But we’ll need 
 our time.
  5. 5. Excellent! You’ve got one year.
  6. 6. CIOMy priorities: (1) Security level (2) Time (3) OpEx savings (4) Migration costs
  7. 7. 8 WE WERE BRAVE
  8. 8. AND WE HAD … IMPEDIMENTS
  9. 9. THE GOOD
  10. 10. 11 Visibility
  11. 11. Let’s architect the cloud!
  12. 12. PACKAGED AND DISTRIBUTED AS CONTAINERS BUILD AND COMPOSED AS MICROSERVICES DYNAMICALLY EXECUTED IN THE CLOUD CLOUD NATIVE APPLICATIONS 3KEYPRINCIPLES
  13. 13. OMG - no greenfield
 approach!? Hundreds of legacy systems!?
  14. 14. Questionnaire: Typical Questions And Their Motivation 1. Technology stack (e.g. OS, appserver, jvm) 2. Required resources (memory, CPU cores) 3. Writes to storage (local/remote storage, write mode, data volume) 4. Special requirements (native libs, special hardware) 5. Inbound and outbound protocols (protocol stack, TLS, multicast, dynamic ports) 6. Ability to execute (regression/load tests, business owner, dev knowhow, release cycle, end of life) 7. Client authentication (e.g. SSO, login, certificates) What images to provide? How many applications will be hard/ inefficient to schedule (>3 GB RAM, > 2 cores)? What storage solutions to provide? What applications will be hard to impossible to be containerized? Are there any non cloud-friendly protocols? How risky is the migration? Is the migration maybe not needed? What IAM and security mechanisms have to be ported to the cloud?
  15. 15. 16 QAVALIDATOR SONARQUBE EAM TOOL QUESTIONNNAIRES JIRA XLS STATIC ANALYSIS IBM MIGRATION TOOL … MIGRATION TASKS BASIC TOUR-DE-MIGRATION SYSTEM PROPERTIES dashboard for information radiator freestyle analysis with Tableau MIGRATION DATABASE CLOUDALYZER
  16. 16. 17 Emergent design of software landscapes
  17. 17. OLD APPLICATIONS (~200) MORE MODERN APPLICATIONS (~200) ALL WEB APPLICATIONS (~400) Re-architect to run on k8s on AWS lift & shift VMs to AWS EC2
  18. 18. APPLICATION HTTPD WEB LAYER ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6 DB MQ HOST BATCH FS CLIENTS TLS 1.0+ mostly non-TLS;
 TCP-Binary, WS, REST, C:D, LDAP
 Corba, SMTP, FTP, NAS, … RACF ESB ONPREM DATA CENTER ONPREM DATA CENTER DB MQ HOST BATCH FS RACF ESB KUBERNETES / OPENSHIFT (CAAS) DOCKER JVM 8 ALMIGHTY FRAMEWORK NG INNER APPLICATIONS AWS WEB LAYER AWS CLIENTS TLS 1.2 all TLS 1.2 JEE 7 APPSERVER API GATEWAY OUTER APPLICATIONS all 2-way TLS 1.2
 & OIDC
 identity token
  19. 19. APPLICATION HTTPD WEB LAYER ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6 DB MQ HOST BATCH FS CLIENTS TLS 1.0+ mostly non-TLS;
 TCP-Binary, WS, REST, C:D, LDAP
 Corba, SMTP, FTP, NAS, … RACF ESB ONPREM DATA CENTER ONPREM DATA CENTER DB MQ HOST BATCH FS RACF ESB KUBERNETES / OPENSHIFT (CAAS) DOCKER JVM 8 ALMIGHTY FRAMEWORK NG INNER APPLICATIONS AWS WEB LAYER AWS CLIENTS TLS 1.2 all TLS 1.2 JEE 7 APPSERVER API GATEWAY OUTER APPLICATIONS all 2-way TLS 1.2
 & OIDC
 identity token
  20. 20. 2 Can we evolve existing enterprise applications into the cloud with reasonable effort? Containerization 12-Factor App Principles Microservices Cloud-native Apps Monolithic Deployment Traditional Infrastructure CLOUD FRIENDLY CLOUD NATIVECLOUD ALIEN
  21. 21. 2 Can we evolve existing enterprise applications into the cloud with reasonable effort? Containerization 12-Factor App Principles Microservices Cloud-native Apps Monolithic Deployment Traditional Infrastructure Sweetspot
 time and value (security, opex) Put the monolith into a container … and enhance the
 application according the 12 factors CLOUD FRIENDLY
  22. 22. Edge Service to the Rescue
  23. 23. MONOLITH MONOLITH EDGE SERVICE BACKEND CLIENTS API GATEWAY BACKEND CLIENTS BEFORE AFTER
  24. 24. MONOLITH EDGE SERVICE CLIENTS TOKEN PROVIDER TLS 1.2 + diverse user contexts (SAML, Cookie, Header, Certificate, User/Pwd, …) Change user context to OIDC identity token IAM SYSTEMS API GATEWAY TLS 1.2 + identity token TLS 1.2 + identity token Redirect to (S)SO if user context not set / valid but required user context > 10
  25. 25. Edge Service <<pod>> Edge Service <<container>> Edge Service Chassis <<spring boot app>> Spring Framework Spring Boot Netflix Zuul / Spring Cloud Zuul Edge Service Filters Redis Edge Library Token Service Token Cache SPI Token SPIToken APIToken Handling Filter <<pre>> Static Content Filter <<routing>> Security Filter <<pre>> Auth Plugins Token Provider Web Layer API Gateway
  26. 26. Sidecars to the Rescue
  27. 27. Container Patterns Applied • Log extraction / re-formatting (fluentd) • Scheduling (Quartz) Sidecar: Enhance container behaviour Ambassador: Proxy communication Adapter: Provide standardized interface • Configuration (ConfigMaps & Secrets to files) • TLS tunnel (Stunnel, ghosttunnel) • Circuit Breaking (linkerd) • Request monitoring (linkerd) Pod Application Container Pattern Container Other Container “Design patterns for container-based distributed systems”. Brendan Burns, David Oppenheimer. 2016
  28. 28. Kubernets 
 Constraints Initially we thought we’ll run into k8s restrictions on our infrastructure like: ‣ No support for multicast ‣ No RWX PVC available We did. But cutting these
 application requirements lead to a better architecture in each and every case.
  29. 29. THE BAD
  30. 30. STATE
  31. 31. Databases ONPREM DATA CENTER AWS DATABASES MONOLITH Databases stay in onprem data center • Advantages: • onprem/cloud version of the application can run in parallel • privacy • Disadvantages: • latency (no real problem) Activate TLS for all database connections
 (low effort on application-side but separate project on the database side)
  32. 32. Files File persistence is very restricted to keep persistent state completely out of AWS (privacy). ‣ No file writes allowed into container ‣ No RWX PVC available ‣ Files with application data written to PVC must be deleted after 15mins ‣ No NAS mounts from onprem data centers into containers allowed Migration tasks for affected applications ‣ Store files as BLOB in database or use FTPS
  33. 33. Session State 90% OF THE APPLICATIONS HAVE SESSION STATE 100% OF THE APPLICATIONS HAVE MORE THAN 1 INSTANCE Session Stickyness: not within the cloud! Session Persistence: ‣ Within existing application database: performance impact too high for most ‣ Redis: no transport encryption out-of-the-box and separate infrastructure required Session Synchronization: ‣ Application server: nope, no dynamic peer lookup within k8s ‣ In-memory data grid: Hazelcast. Nope. $$$ for TLS. ‣ In-memory data grid: Apache Ignite. Done. ‣ Embedded within application or standalone in a separate container. ‣ Little bit cumbersome but working k8s peer lookup ‣ Runs into JIT bug on IBM JVM (some methods have to be excluded from JIT)
  34. 34. DIAGNOS-
 ABILITY need a DevOps?
  35. 35. Metrics Events / LogsTraces Diagnosability
  36. 36. Metrics Events / LogsTraces Diagnosability
  37. 37. Metrics Events / LogsTraces • High effort to instrument for valuable insights •Scalability unclear for hundreds of applications •Applications have no time to run their own Prometheus instance •Scalability unclear for hundreds of applications (Jaeger & ZipKin) •Applications have no time to run their own instance •Scalability unclear (a lot of events lost) •Applications have no time to run their own EFK instance •Non-standardized log format requires custom log rewrite adapter but no fluentd DaemonSet Diagnosability
  38. 38. Metrics Events / LogsTraces
  39. 39. SECURITY need a SecDevOps?
  40. 40. We Came Far BACKEND MONOLITH EDGE SERVICE TLS 1.2 + user context API GATEWAY Two-way TLS 1.2 + identity token Two-way TLS 1.2 + identity token TOKEN CHECK TLS 1.2 + user context secrets injected at container startup security filters included CLIENTS CLIENTCERT ACL EGRESS RULES
  41. 41. The Bad: Certificate Management VAR 1: CLOUD NATIVE STYLE CERTIFICATE MANAGEMENT 
 (SPIFFE-BASED, AT SERVICE MESH OR APPLICATION LEVEL) VAR 2: REPLACE BY POLICIES (AT NETWORKING LEVEL) e.g. e.g. SPIRE
  42. 42. THE UGLY String hostRequest = new HostRequest().hostusPokus(message);
  43. 43. 44 CLOUD ENABLING 
 CLOUD ALIENS
  44. 44. TOXIC
 TECH
  45. 45. The Almighty Legacy Framework • “worry-free package framework” from the early 2000s with about 500kLOC and 0% test coverage • Migration tasks • from J2EE 1.4 to JEE 7 and Java 6 to 8 • add identity token check and token relay • modify session handling (synchronization) • modify logging (to STDOUT) • modify configuration (overwrite from ConfigMap) • enforce TLS 1.2 • place circuit breakers • predefined liveness and readiness probes • Strategies: • the hard way: migrate manually and increase coverage • decorate with ambassadors, sidekicks and adapters • do not migrate parts and replace that API within the applications APPLICATION ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6
  46. 46. 47 ~ 100 systems live on target by end of this year (after 8mo) ~ 200 systems live on target by end of first quarter 2018 other ~200 systems migrated by end of first quarter 2018 via virtual lift & shift. They will be migrated onto Kubernetes afterwards
  47. 47. That’s what we’ve learned from migrating hundreds
 of J2EE legacy apps
 onto Kubernetes?

  48. 48. No stupid “as-is into 
 containers” approach!
  49. 49. Getting as close as 
 possible to cloud friendly 
 application principles. Increasing the security 
 level by an order of 
 magnitude!
  50. 50. Q -> A proud member of the CNCF
  51. 51. TWITTER.COM/QAWARE - SLIDESHARE.NET/QAWARE Thank you! josef.adersberger@qaware.de @adersberger
  52. 52. BONUS SLIDES
  53. 53. 54 Industrialization
  54. 54. ARCHITECTURE TEAM (~2 FTE) SUPPORT TEAM (~10 FTE) DOZENS OF MIGRATION PROJECTS RUNNING IN PARALLEL (UP TO ~80) ‣ Training sessions ‣ Support sessions ‣ Guidance (cookbook, sample application) ‣ Development environment (SEU-as-Code) ‣ Standard images ‣ Automated Refactorings (RefactoBot) ‣ Pre-migrated frameworks ‣ Solutions: Edge service, ambassadors INDUSTRIALIZATION TEAM (~6 FTE) ‣ Application blueprint (target architecture, migration rules) ‣ Migration database
  55. 55. NO SPEAK 
 CLOUD
  56. 56. DMS System APPLICATION DMS Proprietary (binary)
 TCP-level on fixed ports
 Non-encrypted
 Multiple target servers DMS ADAPTER Hard-coded DNS names as reference to other servers in response APPLICATION DMS ADAPTER STUNNEL STUNNEL DMS ONPREM KUBERNETES ON AWS POD STUNNEL STUNNEL PODS SVC.HOST1 SVC.HOST2 SVC.HOST3 lbl:host1 lbl:host2 lbl:host3 ONPREM dnshostname1.com.myapp.svc.cluster.local dnshostname1.com
  57. 57. 8 PLAN BUILD RUN GRASP Continuous
 Delivery Continuous
 Feedback
  58. 58. IDE (ECLIPSE, INTELLIJ) ARTIFACTORY Kubernetes Cluster RUNNING TESTED APPLICATIONS INFORMATION RADIATOR GITHUB ENTERPRISE ACCOUNTABILITY PLUGIN JENKINS JARs
 Docker image Docker image deploy Trigger
 Code Code Status

×