Modeling Microservices

1. @aahoogendoorn MODELING MICROSERVICES Sander Hoogendoorn ditisagile.nl Mentoring ▪ Consulting ▪ Training ▪ Agile ▪ Software architecture ▪ Code

2. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 2 @aahoogendoorn www.ditisagile.nl Sander Hoogendoorn Me  Dad  Mentor, trainer, software architect, programmer  Books and +250 articles  Keynotes and talks at +150 conferences Work  Owner ditisagile.nl  CTO Klaverblad Insurances Web  www.sanderhoogendoorn.com  www.smartusecase.com  www.speedbird9.com  @aahoogendoorn  sander@ditisagile.nl

4. @aahoogendoorn MONOLITHS Hard to deliver, even harder to test and impossible to maintain

6. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 6 @aahoogendoorn www.ditisagile.nl Monoliths Advantages  A single (layered) architecture  A single technology stack  A single code base maintained by multiple teams Disadvantages  All parts are interconnected  Many other systems are connected to your system  Hard to change, hard to maintain  Long time between releases, thereby increasing risks  Slow innovation  Hard to move to newer technologies  Doesn’t scale very well Product Account Order Customer Products Accounts Orders Customers

8. @aahoogendoorn A BRIEF HISTORY OF COMPONENTS AND SERVICES

13. @aahoogendoorn MICROSERVICES Beyond the hype?

16. @aahoogendoorn MICROSERVICES The clear benefits

18. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 18 @aahoogendoorn www.ditisagile.nl Inshort,themicroservicearchitecturalstyleisanapproachtodevelopingasingleapplicationasasuiteofsmallservices, eachrunninginitsownprocessandcommunicatingwithlightweightmechanisms,oftenanHTTPresourceAPI. Theseservicesarebuiltaroundbusinesscapabilitiesandindependentlydeployablebyfullyautomateddeploymentmachinery. Thereisabareminimumofcentralizedmanagementoftheseservices,whichmaybewrittenindifferentprogramminglanguages andusedifferentdatastoragetechnologies. MartinFowler

19. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 19 @aahoogendoorn www.ditisagile.nl Inshort,themicroservicearchitecturalstyleisanapproachtodevelopingasingleapplicationasasuiteofsmallservices, eachrunninginitsownprocessandcommunicatingwithlightweightmechanisms,oftenanHTTPresourceAPI. Theseservicesarebuiltaroundbusinesscapabilitiesandindependentlydeployablebyfullyautomateddeploymentmachinery. Thereisabareminimumofcentralizedmanagementoftheseservices,whichmaybewrittenindifferentprogramminglanguages andusedifferentdatastoragetechnologies. MartinFowler

20. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 20 @aahoogendoorn www.ditisagile.nl Monoliths. Scalability Product Account Order Customer Product Account Order Customer Product Account Order Customer

22. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 22 @aahoogendoorn www.ditisagile.nl Microservices. Scalability Product Account OrderCustomer Product CustomerCustomer Customer Account Account

23. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 23 @aahoogendoorn www.ditisagile.nl Microservices. Scalability & containers Product Account Order Product Customer Account Account CustomerCustomer

24. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 24 @aahoogendoorn www.ditisagile.nl Monoliths. Persistence Product Account Order Customer Products Accounts Orders Customers

25. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 25 @aahoogendoorn www.ditisagile.nl Microservices. Polyglot persistence Product Account OrderCustomer MongoDB Customers MongoDB Orders ActiveDirectory Accounts Oracle Products

26. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 26 @aahoogendoorn www.ditisagile.nl Microservices. Promises  Products not projects  Scalable  Decentralized governance  Replaceable parts  High performance  Technology independent  Polyglot persistence  Easy to build  Easy to test  Easier deployment than monoliths Product Account Order Customer

27. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 27 @aahoogendoorn www.ditisagile.nl Microservices. But…  What is a microservice exactly?  How small is a microservice?  Requirements in a microservice world  Components or services  Who owns a microservice?  What technologies do you use?  What protocols do you apply?  How to define messages  How to test microservices  How to coordinate when business services run across components?  How to build deployment pipelines? Product Account Order Customer

29. @aahoogendoorn ARE MICROSERVICES A STAIRWAY TO HEAVEN?

30. @aahoogendoorn OR A HIGHWAY TO HELL?

31. @aahoogendoorn A REAL WORLD CASE

33. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 33 @aahoogendoorn www.ditisagile.nl A major insurance company We have  Most functionality on an expensive mainframe  A wide variety of large systems written in Java that are hard to maintain and to test, and that are very hard to replace  Individual systems that cover large areas of functionality, usually coupled to departments  Aging technology  No mobile strategy, allowing for new business or new services to clients, and intermediaries We need to  Get rid of the mainframe  Shorten time-to-market  Lower TCO  Uphold a fully secure systems landscape

38. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 38 @aahoogendoorn www.ditisagile.nl FORTHETHINGSWEHAVETOLEARN BEFOREWECANDOTHEM, WELEARNBYDOINGTHEM Aristotle

40. @aahoogendoorn START WITH SOME GUIDING PRINCIPLES

41. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 41 @aahoogendoorn www.ditisagile.nl Questions, questions, questions Communicationarchitecture.Theglue Howdowedefineinterfacesbetweenappsandcomponent?Howdowearrangemessaging? Howdowegluetogetherrapidlychangingappsandcomponent? Applicationarchitecture EnduserfacingDifferentusers,differentfastevolvingneeds Whichtechnologyisthebestforwhichpurpose? Componentarchitecture Componentsandservicesareevolvingrapidly.Howdowedecidewhichcomponentsweneed? Howdowedealwithversioning?Howdowedealwithdistributedprocessesandtransactions?

42. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 42 @aahoogendoorn www.ditisagile.nl Case. Guiding principles We decided to go from here  Client thinks in business processes, so we implement business processes  We move away from the mainframe, to a new systems landscape, consisting of micro-applications and micro- components  Requirements and documentation are modeled rather than written  Applications implement a single (elementary) business process  Components serve a single purpose and offer services  Applications and components all have their own bounded context – a domain model  Applications and components will have an similar internal software architecture to facilitate ease of maintenance and allow for harvesting re-use  Communication between applications and components will use a simple open protocol - REST

43. @aahoogendoorn APPLICATIONS

44. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 44 @aahoogendoorn www.ditisagile.nl Presentation Process Domain Services Outsideworld Pages Grids/Panels,Controls Usecases Flow Domainobjects,Factories/Repositories Enums/Valueobjects/Tupels/Referenceobjects Servicegateways,Serviceclients Infoobjects/Searchobjects ComponentsRelations Dossiers Intermediaries Accounts Rates

45. @aahoogendoorn COMPONENTS

46. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 46 @aahoogendoorn www.ditisagile.nl Serviceinterface Process Domain Data/Services Outsideworld Resources Representations Usecases Flow Domainobjects,Factories/Repositories Enums/Valueobjects/Tupels/Referenceobjects Storagegateways,Storageclients Infoobjects/Searchobjects StorageRelations Dossiers Intermediaries DB2 MongoDB

49. @aahoogendoorn BUSINESS PROCESSES FIRST

53. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 53 @aahoogendoorn www.ditisagile.nl Split into elementary processes - OTOPOP Step3.3.1 Step3.3.2 Step3.3.3 Step3.3.4 Step3.3 Step3.3.5

54. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 54 @aahoogendoorn www.ditisagile.nl Smart use cases Modeling applications  Each elementary process is implemented in a single application  The requirements are modeled using smart use cases  Each application consists of a single sea-level use case and a number of fish-level use cases  Additionally we add the services that are required to implement the applications to the model  Doing so, we can easily do impact mapping on our services  Also, the smart use cases form a strong foundation for integration testing

55. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 55 @aahoogendoorn www.ditisagile.nl Different levels of use cases Traditional use cases Smart use cases Format Textual Visual Granularity Different Unified Estimate Hard Easy Unit of work Lousy Good Reuse Incidental Normal Traceability Possible Normal Testability Poor Good

59. @aahoogendoorn MODULAR DESIGN / DESIGNING MICROSERVICES

60. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 60 @aahoogendoorn www.ditisagile.nl Single responsibility principle (SRP) SOLID  Single Responsibility Principle  Open Closed Principle  Liskov Substituion Principle  Interface Segregation Principle  Dependency Inversion Principle Single Responsibility Principle  Every module should have responsibility over a single part of the functionality provided by the software,  That responsibility should be entirely encapsulated by the class  All its services should be narrowly aligned with that responsibility Therefore  Group together things that change together  Separate things that change for different reason

61. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 61 @aahoogendoorn www.ditisagile.nl Bounded context Domain driven design  The paradigm of designing software based on models of the underlying domain  The domain model helps the business and the developers to reason about the functionality  A model needs to be unified – internally consistent without contradictions Bounded context  The bounded context is a central pattern in domain driven design  When you model larger domains, it becomes progressively harder to create this single unified model  So, instead of creating a single unified model, you create several, all valid within their bounded context

62. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 62 @aahoogendoorn www.ditisagile.nl The single unified domain model Product Vendor Stock Order Client Delivery Payment

63. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 63 @aahoogendoorn www.ditisagile.nl Bounded contexts Product Vendor Stock Order Client Delivery Payment Product

64. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 64 @aahoogendoorn www.ditisagile.nl Bounded contexts Modeling  Modeled in a (single) domain model  Model contains entities, domain objects  Usually evolves around a single main entity (aggregate root) Services  Interrogate the bounded context  Resources usually follow the aggregate root  Services post and put representations  Representations are mapped to the bounded context Validation  Representations can be validated before being mapped  Bounded context can be validated as a whole e.g. before storing  Business rules  Properties and property types Product Vendor Stock

65. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 65 @aahoogendoorn www.ditisagile.nl Properties and property types Basic types  string, integer, date, datetime  Include nullable wrapping Enumerations  Set up at design time, unchangeable at run-time  Genders, Categories Value objects  No specific instances  Isbn, Email, Url, Money References (code tables)  Changeable at run-time, such as contract types Associations  To cached entities such as Country, Nationality  To first level citizens such as Customer, Product

66. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 66 @aahoogendoorn www.ditisagile.nl 66 Every application, worker and component has its own bounded context

67. @aahoogendoorn AND THE REST IS COMMUNICATION (OVER HTTP)

68. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 68 @aahoogendoorn www.ditisagile.nl HTTPRETURNCODESCHEATSHEET 1**.Holdon 2**.Hereyougo 3**.Goaway 4**.Youfuckedup 5**.Ifuckedup

69. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 69 @aahoogendoorn www.ditisagile.nl HTTP GET GET  Retrieve whatever information is identified by the request URI in the form of an entity  The entity is usually a single or a list of objects (of the type provided by the service, often JSON, or XML)  GET is safe (retrieval only) and idempotent  Unfortunately there are many ways of creating GET requests (see examples below)  Returns 200 (found), possibly 400 (bad request) or 404 (not found) Examples  Get an entire collection localhost:8080/countries  Find objects in the collection localhost:8080/countries?name=“stan”  Find an object in the collection by ID localhost:8080/countries/38  Find a sub-object in the collection by ID localhost:8080/countries/38/capital  Find object in the collection by ISO localhost:8080/countries?isocode=“GRC”  Find object in the collection by ISO localhost:8080/countries/isocode/GRC  Find object in the collection by ISO localhost:8080/countries/GRC

70. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 70 @aahoogendoorn www.ditisagile.nl HTTP POST POST  Request that the server accepts the entity enclosed in the request as a new subordinate of the resource identified by the request URI  The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it  Returns the location header for the new resource, possibly with created entity in the body  Returns 201 (created), possibly 500 (server error) Examples  Post to the collection (with entity in body) localhost:8080/countries  Returning location header localhost:8080/countries/38

71. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 71 @aahoogendoorn www.ditisagile.nl HTTP PUT PUT  Requests that the enclosed entity is stored under the supplied request URI  If the request URI refers to an existing resource, the enclosed entity is an modified version of the resource  POST identifies the resource that will handle the enclosed entity  PUT identifies the entity enclosed with the request  Returns 200 or 204 (when resource is modified), possibly 201 (if a resource is created), possibly 500 (server error) Examples  Put with ID (with entity in body) localhost:8080/countries/38  Update capital (with entity in body) localhost:8080/countries/38/capital

72. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 72 @aahoogendoorn www.ditisagile.nl HTTP DELETE DELETE  Requests that the resource identified by the request URI is deleted.  Returns 200 or 204 (when resource is deleted), possibly 500 (server error) Examples  Delete localhost:8080/countries/38

73. @aahoogendoorn MODELING RESOURCES

74. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 74 @aahoogendoorn www.ditisagile.nl Modeling resources Root resources (component) /qa /questionnaires GET Questionnaire Id,Name,Description /qa GET the collection, but only limited to this representation (but with locations likely) /qa/questionnaires /questionnaires GET /{id} Questionnaire Id,Name,Description Question Type,Name,Description Answer Name,Value GET a single item from the collection, but with representation /qa/questionnaires/334532

78. @aahoogendoorn TESTING MICROSERVICES

79. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 79 @aahoogendoorn www.ditisagile.nl A service development lifecycle Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design

80. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 80 @aahoogendoorn www.ditisagile.nl What to test Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design Developers Unittests Developers Q&A Testers Scenario’s&API’s Testers Scenario’s&API’s Productowner Product

82. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 82 @aahoogendoorn www.ditisagile.nl … please automate your tests Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design Developers Unittests Developers Q&A Testers Scenario’s&API’s Testers Scenario’s&API’s Productowner Product

86. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 86 @aahoogendoorn www.ditisagile.nl Presentation Process Domain Services Outsideworld Pages Grids/Panels,Controls Usecases Flow Domainobjects,Factories/Repositories Enums/Valueobjects/Tupels/Referenceobjects Servicegateways,Serviceclients Infoobjects/Searchobjects ComponentsRelations Dossiers Intermediaries Accounts Rates Integrationtests BDD&Selenium Unittests Unittests Unittests

87. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 87 @aahoogendoorn www.ditisagile.nl Serviceinterface Process Domain Data/Services Outsideworld Resources Representations Usecases Flow Domainobjects,Factories/Repositories Enums/Valueobjects/Tupels/Referenceobjects Storagegateways,Storageclients Infoobjects/Searchobjects StorageRelations Dossiers Intermediaries DB2 MongoDB Integrationtests UsingBDD Unittests Unittests Unittests

89. @aahoogendoorn DEPLOYING MICROSERVICES Continuous integration, build pipelines and continuous delivery

90. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 90 @aahoogendoorn www.ditisagile.nl A typical build pipeline Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design

91. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 91 @aahoogendoorn www.ditisagile.nl ProductionAcceptanceIntegrationTestDevelopment A typical build pipeline Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design

93. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 93 @aahoogendoorn www.ditisagile.nl Microservices. Building a deployment pipeline Code DeveloperTest Test AcceptanceTest Acceptance Live Code DeveloperTest Test AcceptanceTest Acceptance Live Code DeveloperTest Test AcceptanceTest Acceptance Live Code DeveloperTest Test AcceptanceTest Acceptance Live

94. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 94 @aahoogendoorn www.ditisagile.nl Microservices. Pipeline hell? Codev.2 DeveloperTestv.2 Testv.2 AcceptanceTestv.2 Acceptancev.2 Code DeveloperTest Test AcceptanceTest Acceptance Live Testv.2 AcceptanceTestv.2 Acceptancev.2 Livev.2 DeveloperTest Test AcceptanceTest Acceptance Live Codev.3 DeveloperTestv.3 Live Codev.2

95. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 95 @aahoogendoorn www.ditisagile.nl Dealing with breaking changes Customer v1 App1 App2 Customer v1 App1 App2 v2 Customer v1 App1 App2 v2 Customer App1 App2 v2

96. @aahoogendoorn DO WE REALLY NEED PROJECTS? From projects to releases to continuous delivery

101. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 101 @aahoogendoorn www.ditisagile.nl Maintenance From projects to continuous delivery? Project MaintenanceMVP MaintenanceContinuousdelivery

103. @aahoogendoorn

104. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 104 @aahoogendoorn www.ditisagile.nl WORDLYWISDOMTEACHESUSTHATITISBETTERFORREPUTATION TOFAILCONVENTIONALLYTHANTOSUCCEEDUNCONVENTIONALLY JohnMaynardKeynes

105. @aahoogendoorn SOME LAST THOUGHTS

106. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 106 @aahoogendoorn www.ditisagile.nl Microservices are not for everyone Whatarewegoingto benefit fromthemost? Polyglotpersistence? Beingableto multiplestacks? Independently deployable? Enforce modulardesign? Continuous Delivery?

109. @aahoogendoorn ALLOW THE TEAM TO LEARN CONTINUOUSLY…

112. @aahoogendoorn … AND HAVE FUN

113. @aahoogendoorn THIS IS AGILE www.createspace.com/4747266 Password: agilescrum Discount code: KGNWKKWG

114. @aahoogendoorn www.sanderhoogendoorn.com www.smartusecase.com www.speedbird9.com sander@ditisagile.nl @aahoogendoorn REFERENCES AND QUESTIONS