Recommended
More Related Content
What's hot
What's hot (20)
Similar to Microservices Delivery Platform. Tips & Tricks
Similar to Microservices Delivery Platform. Tips & Tricks (20)
Recently uploaded
Recently uploaded (20)
Microservices Delivery Platform. Tips & Tricks
- 1. Microservices Delivery Platform. Tips and Tricks
- 2. Solution Architect @ EPAM Andriy Trubitsyn 17 years of software development; Solution architect @ microservice accelerator in EPAM Speaker @ XP Days Ukraine Speaker @ meetups and events
- 3. Microservices delivery platform = Microservices platform + CI/CD +
- 4. What Microservices Platform is ? https://dzone.com/articles/deploying-microservices-spring-cloud-vs-kubernetes
- 5. One Microservices Platform 7 ECS Cluster ECS Cluster Amazon API Gateway Amazon RDS Amazon DynamoDB Amazon ElastiCache Amazon ES Amazon S3 AWS CloudFormation Amazon CloudWatch Legacy DB Amazon SQS Amazon SNS AWS Lambda for AAA Sumologic OpsGenie Parameter Store Mixed AWS Cloud & OnPrem Platforms relies on Netflix OSS Dozens ECS clusters Hundreds of microservices Heterogenous ecosystem: Java, .Net, NodeJS Synchronous and asynchronous microservices communication
- 6. 8
- 8. CI/CD Pipelines Code Build DevQA INT LIVE ECR registry AWS CloudFormation Image Build CI/CDCI OWASP ZAP Code Build
- 9. CI/CD Pipelines shared library & templates CI/CD pipeline templateShared libraries • Pipeline Building blocks • API for tools • Agents injection • Default CI/CD pipelines • Sample book
- 10. Lessons Learnt 1 C I / C D P I P E L I N E S A R E V E RY S I M I L A R 5 2 3 4 M A K E S S E N S E TO D I F F C I A N D C D P I P E L I N E S T E M P L AT E P I P E L I N E I S H E L P F U L T E M P L AT E P I P E L I N E I S TO B E C U S TO M I Z A B L E T E M P L AT E U P DAT E S D I S T R I B U T I O N I S A D I S A S T E R
- 12. Microservices generator • Integrates with Service Discovery Eureka • Sends tracing meta information to a tracing tool Zipkin. • Sends metrics to a monitoring tool DataDog • Sends logs to SumoLogic • Cloudformation script includes ECS Service and TaskDefinition configuration AWS CloudFormation A buildable java project with CI/CD pipeline which
- 13. Microservices generator • Default scaling strategy by CPU • Can be build and deployed to all environments DEV, INT, LIVE • Configuration for Cost Optimization function • Imported CI/CD pipeline in Jenkins with configured hooks and integrated with SonarQube and Jfrog artifactory AWS CloudFormation A buildable java project with CI/CD pipeline which
- 14. Lessons Learnt 1 G E N E R ATO R S S AV E I N C R E D I B L E A M O U N T O F E F F O R T S 2 3 4 O N E G E N E R ATO R N E E D S P E R E A C H L A N G UA G E G E N E R ATO R S B R I N G V E R S I O N I M P L I C I T C O N F L I C T S V I R T UA L T E A M TO O W N G E N E R ATO R S
- 15. Dedicated repo for DTO objects and Contracts DTO
- 16. Dedicated repo for DTO objects
- 17. Dedicated Artifactory Section for Contracts Catalog microservice Order microservice Search microservice
- 18. Lessons Learnt 1 S I N G L E S O U R C E O F T R U T H F O R A L L M I C R O S E R V I C E S 2 3 4 A R C H I T E C T S G O V E R N A C T I V I T Y V I A N F R S A R C H I T E C T S O W N S R E P O S N E E D S A C H A N N E L O F S C H E M A U P DAT E N O T I F I C AT I O N S
- 20. Naming Strategy my.little.pony Entire System Domain Sub Domain Microservice Microservice Microservice
- 21. Versioning Strategy Microservice Order Endpoint Microservice OrderExtendedSchema Non Backward Compatible Endpoint
- 22. Lessons Learnt 1 S I M P L E B U T I M P O R TA N T O T H E R W I S E TO TA L M E S S 5 2 3 4 A R C H I T E C T S T R A C K A L L N E W R E P O & M I C R O S E R V I C E S A LWAY S L AT E S T I S S I M P L E B U T M U C H D U P L I C AT I O N S T I L L U S E V E R S I O N S F O R S H A R E D T E M P L AT E S / L I B S H A R D TO I N V E N T N E W N A M E S
- 23. 24/7 support
- 25. OpsGenie 27
- 26. Lessons Learnt 1 H A R D TO E S TA B L I S H T H E P R O C E S S 2 3 C O D E Q UA L I T Y B E C O M E S M U C H B E T T E R T E A M V E LO C I T Y S O M E W H AT D E C R E A S E D 4 S O M E P EO P L E R E F U S E W O R K S H I F T S
- 28. Infrastructure Cost Function microservice Amazon S3 Amazon RDS Amazon ElastiCache Tagging in CF: • Domain • SubDomain • Team
- 29. Lessons Learnt 1 TA G YO U R R E S O U R C E S 2 3 T R A C K I N F R A C O S T O F A N Y C H A N G E S C L E A N R E S O U R C E S A U TO M AT I C A L LY 4 D E V E LO P E R S D O E S N O T U S E R E S O U R C E S E F F I C I E N T LY
- 31. Dedicated Platform Team https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/ Team Team Team … Domain/ART Platform Team SA SM PO On Call Support
- 32. Lessons Learnt 1 T H E R E I S W O R K N O T F O R C R O S S F U N C T I O N A L T E A M S 2 3 D I R E C T I N VO LV E M E N T W O R K S C R O S S F U N C T I O N A L T E A M S A R E N O T F U L LY I N D E P E N D E N T
- 34. Architecture Documentation • Entire Domain C&C diagram • Microservice C&C diagrams • Microservice interfaces & batch jobs sequence diagrams
- 35. Architecture Documentation • Microservice cost estimations • Dedicated project with Swagger documentation • NFRs check lists
- 36. Lessons Learnt 1 2 3 D O C U M E N TAT I O N T E M P L AT E S M A N DATO RY T E X T I S N O T U S E F U L , D I A G R A M S I S M O R E H E L P F U L D I F F E R E N T D O C U M E N TAT I O N F O R D I F F E R E N T P U R P O S E S 4 N F R S C H E C K L I S T A R E R EQ U I R E D
- 38. Dedicated Test Projects • Dedicated e2e tests ( nightly run ) - Positive & negative scenarios - Cloud & OnPrem • Dedicated Performance tests ( by demand ) - Scaling - Resource usage - Performance Microservices In Cloud On Prem Apps On Prem DBs together
- 39. Lessons Learnt 1 2 3 4 P E R F O R M A N C E T E S T S A R E Q U I T E E X PA N S I V E D E D I C AT E D A Q A T E A M M AY B E A N A N S W E R A L L K I N D S O F T E S T S D O N ’ T F I T O N E M I C R O S E R V I C E P E R F O R M A N C E T E S T S A R E Q U I T E T I M E C O N S U M I N G
- 40. Conclusion
Editor's Notes
- Briefly discuss about all microservices concerns shown on the slide Tell about attempts to build a microservices reference architecture and platforms then
- 1. Describe this case study microservices platform
- Let’s start talking about practices. What kind of practices I am talking about some things which can not be found in internet
- Many microservices with similar pipelines PoC showed we need a templates
- Jenkins, groovy Describe steps Continuous delivery Three AWS environments Each commit to master goes to LIVE in ~25 minutes Dozens cross-functional teams
- Library and utils to build pipelines Agents to collect logs and metrics Stop deployment if this is a branch
- JS consumes JSON JSONJAVA POJO JFROG – consumed by JAVA
- Hard to find a miroservice at GitHub “Order management” is taken
- 1. Domain driven design
- Can add changes backward compatible Hard to name Always last version Simple to support changes Code duplication
- AWS do 24/7
- Talk to teams Schedule shifts ( with reward ) Create escalation rules
- Why we need to track costs (experiments, tests, inefficient resource usage, ELBs) Why we need to tag resources Untagged to be deleted in 5 minutes AWS gives infrastructure grouping/tag grouping. Because organizational structure does not reflect AWS resources. Some teams use AWS resources in efficient way ( too much cpu, memory, disk. The same for all three envs – serverless RDS, no HA in Dev, etc)
- Even with cross-functional teams we have the classic dev & ops problem.
- SAFe, Spotify, Cross-functional teams – BE+AQA, FE+BE+AQA, (+DBA) , (+DevOps), BIs Need of a platform team for user and roles creation, platform deployment, CI/CD and tools support Problems with the dedicated platform team OnCall support from the platform team
- We have MRA, PoCs.
- Entire domain C&C diagram shows connections between microservices, gives a high level vision (no details) C&C diagrams to understand how many new microservices we need, what microservices to change, integration points, work spread Microservice interface description Sequence diagrams for developers it means they understand steps to implement business logic, sync & async intercommunication
- Cost is to permanently grow. We need to estimate cost before microservice release to plan our infrastructure spending's NFRs check list to ensure a microservice is built according to microservice reference architecture It is useful to have a dedicated project with all swagger documents ( refer to sandox) for external B2B integration
- Tell about different type of tests and why we need them Describe envs to involve, when each of them is to start E2E involve Cloud & Prem TDD & BDD Developers write tests Each commit takes 20 minutes to release Priority queues.
- Problems: 1. Time consuming vs 20 minutes release cycle what to do?
- Microservices delivery platform is not a simple thing, I will have to build it as your scale growths. But now you know my mistakes and can your life a little easier But definitely during this year you and I will have to overcome many challenges at our way So see you next time and we will share our finding and become wiser.