Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Microservices Patterns, Issues, Migration Processes

85 views

Published on

Davide Taibi' Colloquium on Microservices at the University of Applied Science in Zurich

Published in: Software
  • Be the first to comment

  • Be the first to like this

Microservices Patterns, Issues, Migration Processes

  1. 1. Microservice migration motivations, patterns and antipatterns Davide Taibi davide.taibi@tut.fi www.taibi.it Davide Taibi - Colloquium Series - SPLAB ZHAW
  2. 2. Davide Taibi - Colloquium Series - SPLAB ZHAW
  3. 3. Main Research Areas • Cloud and Web software Engineering • Microservices, Serverless • Migration Processes • Anti Patterns • Orchestration • Technical Debt Davide Taibi - Colloquium Series - SPLAB ZHAW
  4. 4. Software Architecture Evolution D.Taibi, V.Lenarduzzi, C.Pahl, A.Janes. Microservices Architectural Styles: Agile or not Agile? XP 2017 Davide Taibi - Colloquium Series - SPLAB ZHAW
  5. 5. Software Architecture Evolution D.Taibi, V.Lenarduzzi, C.Pahl, A.Janes. Microservices Architectural Styles: Agile or not Agile? XP 2017 Davide Taibi - Colloquium Series - SPLAB ZHAW
  6. 6. Davide Taibi - Colloquium Series - SPLAB ZHAW
  7. 7. Microservices Migration Processes Davide Taibi - Colloquium Series - SPLAB ZHAW Goal: Microservice Migration • Motivations and Expected Benefits • Processes • Issues 21 companies interviewed D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Processes, Motivations and Issues for Migrating to Microservices Architectures: An Empirical Investigation”, IEEE Cloud Computing Journal, vol. 4, no. 5, 2017.
  8. 8. Davide Taibi - Colloquium Series - SPLAB ZHAW Microservices - KölnMicroservices - Köln Migration Motivation
  9. 9. Migration Process (1/2) Davide Taibi - Colloquium Series - SPLAB ZHAW
  10. 10. Migration Process (2/2) Davide Taibi - Colloquium Series - SPLAB ZHAW
  11. 11. Main Issues Identified Davide Taibi - Colloquium Series - SPLAB ZHAW Technical Issues • Decoupling from the monolithic system • Database migration and data splitting • Communication among services • Service orchestration complexity
  12. 12. Main Issues Identified Davide Taibi - Colloquium Series - SPLAB ZHAW Effort-Related issues • Effort estimation and overhead • effort at least 20% higher • Effort required for the DevOps infrastructure • Effort required for library conversion
  13. 13. Main Issues Identified Davide Taibi - Colloquium Series - SPLAB ZHAW Other issues • People’s minds • ROI achieved in longer time (or never) compared to monolithic systems
  14. 14. Microservices Architectural Patterns Goal Classify the architectural Patterns proposed in the Literature D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Architectural Patterns for Microservices: A Systematic Mapping Study”, in 8th International Conference on Cloud Computing and Services Science, CLOSER , 2018. Davide Taibi - Colloquium Series - SPLAB ZHAW
  15. 15. Paper Selection Searched for (microservice* OR micro-service*) AND (architect* OR migrat* OR modern* OR reengineer* OR re-engineer* OR refactor* OR re-factor* OR rearchitect* OR re-architect* OR evol*) Included non peer-reviewed contributions if their number of citations was higher than those of average peer-reviewed papers Summary 65% of these papers were published at conferences 23% were accepted at workshops only 7% of the papers were published as journal articles nearly 5% (2 papers) are non peer-reviewed websites Davide Taibi - Colloquium Series - SPLAB ZHAW
  16. 16. Paper Selection Davide Taibi - Colloquium Series - SPLAB ZHAW
  17. 17. Pattern Categories • Orchestration and Coordination • Deployment strategies • Data management / Data Storage Davide Taibi - Colloquium Series - SPLAB ZHAW
  18. 18. Shopping Cart Product Catalog Recommender System API Gateway Client- Specific APIs Protocol Translation Davide Taibi - Colloquium Series - SPLAB ZHAW API Gateway PatternOrchestration and Coordination
  19. 19. Shop. Cart Instance 1 Service 1 REST API Registr y Client REST API Registr y Client REST API Registr y Client Shop. Cart Instance 2 Shop. Cart Instance 3 Service Registr y Register Registry -aware client Client-Side Discovery Pattern Davide Taibi - Colloquium Series - SPLAB ZHAW Orchestration and Coordination
  20. 20. Shop. Cart Instance 1 Service 1 Load Balancer REST API Registry Client REST API Registr y Client REST API Registr y Client Shop. Cart Instance 2 Shop. Cart Instance 3 Service Registry Load Balance Register Query Request Server-Side Discovery Pattern Davide Taibi - Colloquium Series - SPLAB ZHAW Orchestration and Coordination
  21. 21. Advantages and Disadvantages of the Identified Patterns Davide Taibi - Colloquium Series - SPLAB ZHAW
  22. 22. Classification of Advantages and Disadvantages of the Identified Patterns Davide Taibi - Colloquium Series - SPLAB ZHAW
  23. 23. Emerging Issues Comparison of SOA and Microservices. • differences not thoroughly investigated. • lack of comparisons (e.g., performance, development effort, maintenance). Microservices Explosion. • When qualities degrade? Negative Results. • In which contexts do microservices turn out to be counterproductive? Are there anti-patterns? Davide Taibi - Colloquium Series - SPLAB ZHAW
  24. 24. DevOps • DevOps pipeline is only partially covered by research work. • Only few papers propose specific techniques, and apply them to small example projects. • Release-specific techniques have not been investigated in our selected works. • No empirical validation have been carried out in the selected works. Davide Taibi - Colloquium Series - SPLAB ZHAW
  25. 25. Microservice Antipatterns and Bad Smells FOCUS: MICROSERVICES 56 IE E E S O F T WA R E | PU BLI S H E D BY T H E I E E E CO M PU T E R S O CI E T Y 074 0 -74 5 9/18 / $ 3 3.0 0 © 2 018 I E E E On the Definition of Microservice Bad Smells Davide Taibi and Valentina Lenarduzzi, Tampere University of Technology // To identify microservice-specific bad smells, researchers collected evidence of bad practices by interviewing developers experienced with microservice-based systems. They then classified the bad practices into 11 microservice bad smells frequently considered harmful by practitioners. // MICROSERVICES ARE CURRENTLY enjoying increasing popularity and diffusion in industrial environments, being adopted by several big players such as Amazon, LinkedIn, Netflix, and SoundCloud. Microservices are relatively small and autonomous ser- vices that work together, are mod- eled around a business capability, and have a single and clearly defined purpose.1,2 Microservices enable independent deployment, allowing small teams to work on separated and focused services by using the most suitable technologies for their job that can be deployed and scaled independently.1,2 Microservices are a newly developed architectural style. Several patterns and platforms such as nginx (www.nginx.org) and Kubernetes (kubernetes.io) exist on the market. During the migration process, practitioners often face com- mon problems, which are due mainly to their lack of knowledge regarding bad practices and patterns.3,4 In this article, we provide a cata- log of bad smells that are specific to systems developed using a micro- service architectural style, together with possible solutions to overcome these smells. To produce this catalog, we surveyed and interviewed 72 ex- perienced developers over the course of two years, focusing on bad prac- tices they found during the develop- ment of microservice-based systems and on how they overcame them. We identified a catalog of 11 microservice- specific bad smells by applying an open and selective coding5 proce- dure to derive the smell catalog from the practitioners’ answers. The goal of this work is to help practitioners avoid these bad prac- tices altogether or deal with them more efficiently when developing or migrating monoliths to microservice- based systems. As with code and architectural smells, which are patterns com- monly considered symptoms of bad design,1,6 we define microservice- specific bad smells (called “micro- service smells” hereafter) as indicators of situations—such as un- desired patterns, antipatterns, or bad practices—that negatively affect software quality attributes such as understandability, testability, extensi- bility, reusability, and maintainability of the system under development. Background Several generic architectural-smell detection tools and practices have been defined in the past years.7–9 Moreover, several microservice- specific architectural patterns have been defined.10 However, to the best of our knowledge, no peer-reviewed work and, in particular, no empirical studies have proposed bad practices, antipatterns, or smells specifically concerning microservices. However, some practitioners have started to discuss bad practices in microservices. In his ebook Micro- services AntiPatterns and Pitfalls, IEEE Software May/June 2018 Davide Taibi - Colloquium Series - SPLAB ZHAW D. Taibi and Lenarduzzi, V. , “On the Definition of Microservice Bad Smells”, IEEE Software , vol. 35, no. 3, 2018.
  26. 26. Microservices Bad Smells Exploratory Survey (72 practitioners) [2] 11 Smells identified • Cyclic Dependencies • Problem of Microservices Explosion • More than 50 connected microservices become unmanageable Davide Taibi - Colloquium Series - SPLAB ZHAW
  27. 27. Davide Taibi - Colloquium Series - SPLAB ZHAW Microservice Bad Smells
  28. 28. Microservice Bad Smells Davide Taibi - Colloquium Series - SPLAB ZHAW
  29. 29. Davide Taibi - Colloquium Series - SPLAB ZHAW Microservice Bad Smells
  30. 30. Migration to Microservices and Technical Debt Davide Taibi - Colloquium Series - SPLAB ZHAW L. Valentina and Davide, T. , “Microservices, Continuous Architecture, and Technical Debt Interest: An Empirical Study”, in Euromicro/SEAA, Prague, 2018.
  31. 31. Case Study - RQs RQ1: Is the TD of a monolithic system higher than the TD of the same system developed with a microservices architectural style? RQ2: Is the TD of a monolithic system growing with the same trend as a microservices-based system? Davide Taibi - Colloquium Series - SPLAB ZHAW
  32. 32. Case Study - Object One Italian Company Document Management System Java Deployed on Microsoft Azure Delivered as a web application +desktop application to support document uploading Davide Taibi - Colloquium Series - SPLAB ZHAW
  33. 33. Case Study - Object Monolithic system 280K lines of code >12 years old Expected Benefits -Ease maintenance -Separate each business process -Reduce the need for synchronization between the two development teams. Davide Taibi - Colloquium Series - SPLAB ZHAW
  34. 34. Case Study - Object 6 business process extracted into 6 MS Two of the six microservices were deeply re- architected after the first release MS1: MySQL replaced by MongoDB MS2: re-written from Java to in Python Davide Taibi - Colloquium Series - SPLAB ZHAW
  35. 35. Case Study – Results Davide Taibi - Colloquium Series - SPLAB ZHAW 0 2000 4000 6000 8000 10000 12000 14000 TechcnicalDebt Microservice1 Microservice2 Microservice4 Microservice3 Mar.2017 Jan.2018 April2017 May2017 June2017 July2017 Aug.2017 Sept.2017 Oct.2017 Nov.2017 Dec.2018
  36. 36. Case Study Results: TD Trend Davide Taibi - Colloquium Series - SPLAB ZHAW TD Trend TD Trend Predicted TD without Migration No Migration Overall System TD Monolithic System Microservice Migration
  37. 37. Case Study Results The TD of the monolithic system is lower than TD after the migration. TD tends to be more stable and to increase slower in each microservice compared to the whole system’s technical debt The Total TD grows faster compared to the TD growth before the migration. Davide Taibi - Colloquium Series - SPLAB ZHAW
  38. 38. Microservices Slicing with Process Mining Can we use process mining techniques to support slicing? Industrial Case Study with a SME • Legacy system developed in 2006 • Document Management System for tax accountants • > 1000 Classes Davide Taibi - Colloquium Series - SPLAB ZHAW
  39. 39. Idea Mining log files to identify business processes Identify potential MS candidate • Low coupling • High Cohesion Davide Taibi - Colloquium Series - SPLAB ZHAW
  40. 40. 5-steps process Davide Taibi - Colloquium Series - SPLAB ZHAW Step 1: Process Mining Step 2: Process Usage Ranking Step 3: Process-driven identification of decomposition solutions Step 4: Circular Dependencies, Cohesion, and Coupling Step 5: Identification of decomposition alternatives
  41. 41. Log Files Example Davide Taibi - Colloquium Series - SPLAB ZHAW ID Start Timestamp Complete Timestamp Activity Class Method 1 02.01.2018 12:00:00:00 02.01.2018 12:00:00:36 Create Invoice MainInvoice createInvoice() 2 02.01.2018 12:00:01:00 02.01.2018 12:00:01:39 Validate Invoice ValidateInvoice validate() 3 02.01.2018 12:00:01:40 02.01.2018 12:00:01:45 Save Invoice MainInvoice save() 4 02.01.2018 12:00:01:45 02.01.2018 12:00:01:55 Save Invoice DB INVOICE 5 02.01.2018 12:00:01:56 02.01.2018 12:00:02:05 Save Invoice DB ACCOUNTANT
  42. 42. Slicing Example Davide Taibi - Colloquium Series - SPLAB ZHAW InvoiceForm.submitButtonClick MainInvoice.createInvoice() ValidateInvoice.validate() MainInvoice.save() MainInvoice.sendToMinistry() SDI Webservice (Ministry of Finance Information System) INVOICE ACCOUNTANT CUSTOMER MainInvoice.updateStatus() AmazonGlacier.storePermanently() DBManager.read() InvoiceForm.getInvoiceDetails() MainInvoice.getInvoice() DBManager.save() DBManager WebApp MainInvoice Microservice 1 Microservice 2
  43. 43. Microservices Adoption Framework • How to decide if migrate to Microservices or Not? • Which measure should be considered? • Literature: Limited comparison between MS and Monothic systems • Limited number of empirical studies on software qualities Davide Taibi - Colloquium Series - SPLAB ZHAW
  44. 44. MS Adoption Framework - Approach • Systematic Mapping Study • Metrics adopted in studies comparing MS and Monolithic systems • Survey (52 practitioners) • Which measures have you considered before migrating? • Which measures were useful? • Which measures should have been considered? • How useful are the metrics adopted in the mapping studies? Davide Taibi - Colloquium Series - SPLAB ZHAW
  45. 45. Preliminary Results (Mapping Study) Davide Taibi - Colloquium Series - SPLAB ZHAW Product related PerformanceAvailability Maintenance CHARACTERISTICS (RQ2) Meantimetorecover Downtime Meantimetofailure CPUutilization ResponseTime Pathlength Usageofcontainers Impactofprogramminglanguage #featurepermicroservice #requestsperminute Complexity Testability MEASURES (RQ3) Process related Cost Developmentindependencebetweenteams Continuousdelivery Reusability Personnel Infrastructure Developmentcost Costperhour Costpermillionofrequests Scalability Waitingtime
  46. 46. Microservices and FaaS Practitioners started migrate to microsercvices and FaaS Mixed Approach (microsercvices + Functions) Open Issues • When and Why Extract a feature as Function or as Microservice? • Which pattern should be adopted Davide Taibi - Colloquium Series - SPLAB ZHAW
  47. 47. Conclusion Microservices - Cagliari 11.07.2018Microservices - Köln 26/06/2017 Microservices - Köln 26/06/2017 Migration Motivation Migration Process (1/2) Microservices - Cagliari 11.07.2018 Microservices Architectural Patterns Goal Classify the architectural Patterns proposed in the Literature D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Architectural Patterns for Microservices: A Systematic Mapping Study”, in 8th International Conference on Cloud Computing and Services Science, CLOSER , 2018. Microservices - Cagliari 11.07.2018 Microservices Bad Smells Exploratory Survey (72 practitioners) [2] 11 Architectural Smells identified • Cyclic Dependencies • Problem of Microservices Explosion • More than 50 connected microservices become unmanageable Microservices - Cagliari 11.07.2018 Microservices Technical Debt 8.9.201848 TD Trend TD Trend Predicted TD without Migration No Migration Overall System TD Monolithic System Microservice Migration Process Mining Slicing Microservice 1 Microservice 2 Microservice Adoption Framework Microservices and FaaS Practitioners started migrate to microsercvices and FaaS Mixed Approach (microsercvices + Functions) Open Issues • When and Why Extract a feature as Function or as Microservice? • Which pattern should be adopted
  48. 48. References • D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Processes, Motivations and Issues for Migrating to Microservices Architectures: An Empirical Investigation”, IEEE Cloud Computing Journal, vol. 4, no. 5, 2017. • D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Architectural Patterns for Microservices: A Systematic Mapping Study”, in 8th International Conference on Cloud Computing and Services Science, CLOSER , 2018 • D. Taibi and Lenarduzzi, V. , “On the Definition of Microservice Bad Smells”, IEEE Software , vol. 35, no. 3, 2018 • P. Rosati, Fowley, F. , Pahl, C. , Taibi, D. , and Lynn, T. , “Making the Cloud work for Software Producers: linking Architecture, Operating Cost and Revenue”, in 8th International Conference on Cloud Computing and Services Science, 2018 • L. Valentina and Davide, T. , “Microservices, Continuous Architecture, and Technical Debt Interest: An Empirical Study”, in Euromicro/SEAA, Prague, 2018 Davide Taibi - Colloquium Series - SPLAB ZHAW

×