10. Main Issues Identified
• Decoupling from the monolithic system
• Architectural Patterns and Anti-Patterns are not clears
• Service orchestration complexity
• Higher effort (at least 20% more)
• Dev-Ops infrastructure
• Library conversion
• …
12. Migration Process (2/2)
D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Processes, Motivations and Issues for
Migrating to Microservices Architectures: An Empirical Investigation”, IEEE Cloud
Computing Journal, vol. 4, no. 5, 2017.
13. Microservices Architectural Patterns
• Goal
• Classify the architectural Patterns proposed in the Literature
• D. Taibi, Lenarduzzi, V. , and Pahl, C. , “Architectural Patterns for Microservices: A Systematic Mapping Study”, in 8th
International Conference on Cloud Computing and Services Science, CLOSER , 2018.
18. Microservices Bad Smells
• Exploratory Survey (72 practitioners) [2]
• 11 Smells identified
• Cyclic Dependencies
• Problem of Microservices Explosion
• More than 50 connected microservices become unmanageable
19. Microservice bad-smells
• Wrong Cuts
• Hard-Coded Endpoints
• Cyclic Dependency
• Shared Persistency
• API Versioning
• ESB Usage
• Not Having an API Gateway
• Inappropriate Service Intimacy
• Shared Libraries
• Too Many Standards
• Microservice Greedy
21. Microservice Decomposition
• Idea
• Mining log files to identify business processes
• Identify potential MS candidate
• Low coupling
• High Cohesion
D.Taibi, K. Systä. From monolithic systems to
microservices: A decomposition framework based
on process mining. CLOSER 2019
25. Open Questions - Microservices
• How to refactor a monolithic system into microservices?
• Which anti-patterns are actually harmful and when?
• How to evaluate coupling and cohesion between Microservices?
• How to efficiently orchestrate a system?
• Is a microservices-based system more understandable, and more maintainable?
• How to measure them?
• Lack of large-scale open source projects
29. O’Reilly SW Architecture Conference 2018
Stop using microservices!
Move to serverless functions as soon as possible!
30. What is Serverless [3]
a cloud-native platform
for
short-running, stateless computation
and
event-driven applications
which
scales up and down instantly and automatically
and
charges for actual usage at a millisecond granularity
What is Serverless?
[3] Baldini I. et al. (2017) Serverless Computing: Current Trends and Open Problems. In: Chaudhary S., Somani G., Buyya
R. (eds) Research Advances in Cloud Computing. Springer, Singapore
31. How does it work
Runs code only on-demand on a
per-request basis
Serverless
deployment &
operations model No servers Just code
Server-less means no servers?
Or worry-less about servers?
@davidetaibi
32. Runscode in response to events
Event-programming
model
What triggers code execution?
33. Current Platforms for Serverless
Azure Functions
AWS Lambda
Kubernetes
Google Functions
Red-Hat
35. Migration Motivations
Companies Already moved to Microservices
• OPS Effort for Microservices
• Get rid of Kubernetes
• No OPS
Companies Migrating from Monolithic systems
• New (hype) technology
• Promising technology
• No initial infrastructural costs (pay as you use)
• Automatic scaling
• Lack of skilled OPS personnel
36. Preliminary Results – Migration Issues
Developers are not used to the event-oriented programming
Very hard to test
Debug almost impossible
Unknown Patterns and antipatterns
Anomalies can generate unexpected costs!
38. Serverless Anti-Patterns Summary
#1 Async Calls
#2 Functions calling other functions
#3 Shared Code
#4 Shared Libraries as Functions
#5 Too many libraries
#6 Too Many technologies
#7 Too many functions
EXTRA: The distributed Monolith
39. Open Questions - Serverless
• When is better to use serverless and when microservices
• How to architect a system based on serverless functions?
• Or to combine functions to create a microservice?
• Architectural Patterns? Anti-Patterns?
• How to prevent anomalies?
40. Microservices and FaaS
• Practitioners started migrate to microservices and FaaS
• Mixed Approach (microservices + Functions)
• Open Issues
• When and Why Extract a feature as Function or as Microservice?
• Which pattern should be adopted
44. Micro-Frontends
• Adopted by several large-companies
• SAP, Zalando, Springer, NewRelic, Ikea, Starbucks, Spotify, DAZN, …
• Increase the team velocity
• But… create duplications in common parts
45. Micro-frontends - Motivations
• Decompose the front-end into individual and semi-independent micro applications.
• In microservice-based systems, the frontend is implemented as a monolithic (or fat)
• Microservice and frontend teams need to synchronize changes
46. Micro-Frontends – How they are implemented
• One micro-frontend per page
• Each team develop a page completely
• Common parts are duplicated (sidebar, header, footers, …)
• Multiple micro-frontends per page
• Client-side composition
• Edge-side composition
• Server-side composition
47. Micro-Frontends – How they are implemented
• One micro-frontend per page
• Each team develop a page completely
• Common parts are duplicated (sidebar, header, footers, …)
• Multiple micro-frontends per page
• Client-side orchestration
• Edge-side orchestration
48. Micro-Frontends – Open Issues
• When and why they should be used?
• Not for every project.
• Are duplications “beneficial”?
• How about coupling between teams
• Increased application complexity due to page composition at run-time.
• Increased observability complexity, due to the increased number of moving parts.
49. New Trends
• Serverless and micro-frontends adoption is increasing
• Companies are trying to move the computation closer to the data è Edge Computing
• New on-premise solutions for edge computing
• Hybrid cloud/on-premise solutions
50. Conclusion
• Serverless and Microservices are very powerful and useful technologies
• Still several open questions
• Developers should carefully consider the “old fashioned” software engineering practices
• Properly design a modular system
• Pay attention to coupling and cohesion
• Think about long-term maintenance
• Avoid the distributed monolith
• Some companies are moving back to on-premise, other are considering hybrid approaches
• Edge computing is coming
53. The Continuous Monitoring Data-Ops Platform
Project
Data
Chronjob / Listener
Code Quality Issues Code Reviews CI/CD Commit Classif. Other
SonarQube
Kiuwan
Coverity
Jira
GitHub
Gerrit
Jenkins
TravisCI
Swanson
ChangeDistiller
Listener
Listening Topics
Tools
Message Queue (Publish/Subscribe) - RabbitMQ
Listen for new events on DevOps tools
(commits, faults, reviews, …)
One worker per project
Refacgtoring
Miner
PySpark or Flink Reasoner
Listen for new events
Create Topics Classifying new events
Containers:
- Message-subscriber listening for new topics
- Save results into Cassandra translating results into Key Value store
SZZ
Jira
Pronto
Fault
Postgres Data
Storage
Scheduler
Worker
(tool)
54. Requirements
•Possibility to add projects with metadata
• SCM, issue tracker, code review, …
•Launch all “Dockerized” tools on all the metadata
• Analyze all SCM commits, all issues, …
•Update the analysis whenever new metadata exists
• New commits, new faults, …
55. Container Example 1
Message-subscriber
Listen for new Topics
Is the topic
interesting to me?
Execute
Sonar-Scanner
Translate SonarQube Data
into Key Value store
Send Message (data)
to Rabbit MQ
Execute SonarQube
export
SonarQube Data
56. The “Extensible” DataSet
PROJECT
COMMIT
FAULT
[2] SZZ FAULT_INDUCING
COMMIT
[1] COMMIT CHANGES
SONAR ISSUESONAR MEASURE SONAR RULES
SONARQUBE
[1] Andreas Mauczka, Florian Brosch, Christian Schanes, and Thomas Grechenig. 2015. Dataset of developer-labeled commit messages. In Proceedings of the 12th Working Conference on Mining Software Repositories (MSR '15).
[2] . Sliwerski, T. Zimmermann, and A. Zeller ”When Do Changes Induce Fixes”. International Workshop on Mining software repositories (MSR’05). pp. 1-5. 2005
[3] Nikolaos Tsantalis, Matin Mansouri, Laleh Eshkevari, Davood Mazinanian, and Danny Dig, "Accurate and Efficient Refactoring Detection in Commit History," ICSE 2018. https://github.com/tsantalis/RefactoringMiner
[3] REFACTORING MINER
ChangeDistiller
Commit-Analysis Tools
REVIEW [4] GERRIT MINER
Review-Analysis Tools
[4] Git Miner
57. Tool Implementation
Scheduler
Worker_PRJ1 (eg. Sonar-scanner)
Worker_PRJ2
Worker_PRJn
LISTENER
Job Queue
In some case, we may want to slow
down or pause the workers to avoid
resource issues
(eg. Too many open files)
Listen to the message queue
If there are new “interesting`” event
(eg. If a tool is listening for new
commits of “Java” projects)
- Get Project Metadata
- Send the list of tasks to the scheduler
together with the metadata
(eg, list of commits to be analyzed)
• Execute the tool as is. Eg. Execute Sonar-Scanner
• Send the results to the message queue as “blob” reporting project ID and commit ID (or Fault ID,
or any other information to identify the analysis) Eg:
Project-id: Ambari
Commit-sha=12344
ToolName=PMD
Results = <xml…. >
• Send the “analysis done” messafge to the queue
Project-id:ambari
Commit-sha=12344
ToolName=PMD
Finished = true