JDD 2014 HOW TO REBUILD MONOLITHIC APPLICATION TO MICRO SERVICES ARCHITECTURE ?
•Download as PPTX, PDF•
4 likes•862 views
When creating application based on architecture of micro services, a team needs to face up many challenges. -building easily extensible architecture of micro services. When we creating new service, we always need data from other service. To be able to receive that data, we only need to import light client, which is the interface of another service. -How to use HTTP protocol properly, to change it to our biggest assets, and do not attempt to hide it unnecessarily. -Which functionalities of monolithic application pull out first. How to use domain driven design, to model the business domain specific for each micro service ? -How to manage configuration and deployment of various services ? -implementing Circuit Breaker patter, using Netfix Hystrix library -real time monitoring of micro services -aggregating logs from many servers in one central point, and analysing them -creating documentation, which could be use as a client of each service
Recommended
More Related Content
What's hot
What's hot (20)
Similar to JDD 2014 HOW TO REBUILD MONOLITHIC APPLICATION TO MICRO SERVICES ARCHITECTURE ?
Similar to JDD 2014 HOW TO REBUILD MONOLITHIC APPLICATION TO MICRO SERVICES ARCHITECTURE ? (20)
Recently uploaded
Recently uploaded (20)
JDD 2014 HOW TO REBUILD MONOLITHIC APPLICATION TO MICRO SERVICES ARCHITECTURE ?
- 1. Tomasz Lelek tomekl007 @tomekl007
- 2. How to rebuild monolithic application to micro services architecture?
- 3. Presentation Plan 1. Business context 2. Technology 3. Architecture 4. Configuration management 5. Monitoring 6. Aggregating logs 7. Deploying ( vagrant, docker ) 8. Modern documentation
- 4. Monolith
- 6. One entity – multiple identities
- 7. Same functionality – new architecture
- 9. Url Rewriting #Apache rewrite url, work for post request RewriteEngine On RewriteRule /generate http://reportservice.no/generate [P]
- 10. Needs
- 11. i
- 13. Starting Service ./sbt >container:start [info] Started SelectChannelConnector@0.0.0.0:8090 [success] Total time: 2 s
- 14. How microservice looks like?
- 15. Using via imported client
- 16. Many clients
- 17. Scala java - interoperability
- 18. Rest, json, http
- 19. Architecture
- 25. Services monitoring - status
- 29. Where is service with configuration?
- 30. etcd
- 31. Circuit Breaker
- 32. Hystrix
- 33. Implementation
- 34. Many services = many logs
- 36. + logstash
- 40. Amazon EC2
- 41. vagrant plugin install vagrant-aws vagrant up --provider=aws vagrant ssh <MACHINE NAME>
- 43. Technology stack
- 45. Documentation
- 46. Generated json
- 48. Documentation as a client
- 49. Time for building microservices !
- 50. Thank You, Questions ?
Editor's Notes
- 1.Powiedz o sobie, gdzie pracuejsz, jakie projekty Witam wszystkich: nazywam się Tomasz Lelek, jestem programistą w firmie Schibsted Tech Polska. Pracuje przy projektach zwiazanych z java. Mozecie mnie znalesc na twitterze i githubue Jesli ktos bedzie mial pytania to prosze o zadawanie ich na koniec prezentacji.
- 2.In my presentatnion i’ll try to show, how to build microservices using scala language and scalatra framework
- 1, why we want build microserwice architecture ? 2.Which technologies allow us to build them realtivly simply 3.I’ll create new serivice, implement couple rest methods, write some tests, and show how easily use akka actors in that service 4.How to monitor such microservice
- 4.Why we even want to create architecutre of microserwices ? In a java work, often we see projects, which could be describe by one work : monolith Characteristics of arhitecure such projects is very tight coupling, where change in one place in code, could have big impanct on code in place which is totaly not connected with our change. I am working with application which could be described as Monolith
- Domain of that application is an accounting. Subscirption for newspapers which are ordered by clients, are processed by that system. Subsscripiton are processed via nigtly batch, operations are accounted, and client accounts are charged. In such accounting and banking domains, applications often are monolithic, when one funcitonality ( proccessing data via batch operations ) is in same system as generating reports ( eg. about number of sell newspapers ). In that point, we should see a possibility to pull out functionality of generating reports to separate service, without affecting working system
- Applying domain driven desingn, In monolithic application, we have one model, in which object Subscripton ( subscription for newspaper ) is used in a lot of places, and a lot of contexts. We should notice, that Subscription, when we generate reports, is something entirly different, from subscription, which is ordered by client. ( both subsriptions need to have different fields ). One big model means that we have one big Class - entity Subscritpion, which have all fields needed for those two functionality. When we see something like this in our application, that is a sign, that when we generating reporsts, subscription which all fields needed to generate this report could be pull out to other service. That pulled Subscripiton will have only those fields neccesary for that one functionality of generating reports, there will be no fields connected with ordering newspaper by client. By doing such operation we slim our domain. Subscription have more that one form
- 7 When we made decision to pull out some fucntionality from monolithic app into microserwice, we want to change arhictecture, in a way, which won’t be seen for a end user of our application. We want our application to work in exact same way as previosly, but handling requests of user will be done not in monoilithic application, but in new microservice. We want that change be unnoticable for end user.
- To be able to achive this, we use simple dispacher ( proxy). When from web application ( browser) request for doing some action which was previosuly done in monolithic app is send, that reqeust is captured by server (eg apache ) and redirect to new microserwice, which execture actions, which was previosuly executed by legacy system application.This is unnoticable for end user.
- 9.APACHE MOD REWRITE – it works like this : define ..
- 10.w tym momecie musimy zdecydowac co potrzebujemy, jakich technlogii, do zaimplemenowania takiej architektury. Framework kotry pozowli na latwe deployowanie aplikacji, lekki, w kotrym szybko mozna napisac lekkie serwisy,
- 11. Concepcual scalatra is based on very poprular ruby framework – sinatra. It has couple librareis, embedded container jetty, it is based on servlets, it’s written entirly in scala language. It has integrated couple useful libraries like json mapper
- When building microservices we want, our application to start very quickly. Best solution is using embeded container. Micorservice could not be deployd on a container (like tomcat ) which will be starting long time, we want possiliobty to take jar file with packed service, and run that service in one command. We want that to happen very quickly, becuaes it reduce feedback for a developer, therefore he could work much more effectivly. When you proggraming you service, and you need to wait 40s to see your changes, it disturbs you, you loosing you focus, and flow, therefore tou work much less effectivly. When you could seen you changes in seconds, you reduce you feedback time, you could be much more better in programming, Our service is then like a packed gift, which need only to be unpack ( run )
- 13. Only thing that we need to do to run our service is to Start sbt Then fire command container:start. In that moment our application is started on proper port We see that it took only 2 seconds to run that Very useful tip : ./sbt >container:start ~ ;copy-resources;aux-compile -it’ll be monitoring directory with our project, and if any file will change, it’ll restart our container
- 14. When we know our tool ( more or less :-) we need to think about architecture. We want our service to supply light client for other services. Then that client could be imported by other service, which could easily and transparently speak with that first service, without worring about Sending http reqeust So: Service importing client of other service, use that client like normal library, and client is responsible for sending requests to that other sercie. All that happends transaprently for serivce which import that light client. Our services consist of three componenst : proper srecvie, which works on server, it’s run as embeded container, model wich will be shared beetewen client and server ( it is used as a contract ), And third component is a mentioned light client
- 15. Each request to our sercie throuout client, which could be imported by another microservies. You could speak to service, also via any other http client, like curl, because each service is created according to rest -show how client looks like ( configuration service client, maybe show how i implemented caching )
- One microservice could supply many light clients, each is written in different language ( scala, java, js ), then other proggramer, which want create microservice in other language, or technology, when want to use that service, only need to import client, which is written in languege of his choice.
- I’ll show you how easiliy client written in scala language, work with java code Spid-lib -> ConfServiceFacade //todo slide with code
- To that point you could see that each servie is implemented accoring to rest, using json as exhachange format, and proper use of http protocol
- 18.When building csv-reports-generator and need data form dataservice, we only need to import light client supplied from dataservice. Then we could easily use dataservice from csv generator transparently. We could see that architecture of microservice could easily evolve
- One example of microservice which we our use in production system is configruation service. It stores data about location of all microservices, and other useful things like db config, amazon sqs confiig, whatever we want
- Configuration service is use as a facade abouve etcd nodes. Etcd nodes are responsible for persistance of data and syncrhonization of data beetween them.
- 20. Main purpous behind such service, is to create centralized point with all configuration. Then we could use this as following
- 21.w configuration serwis trzymamy konfiguracje wszyskitch serwisow dla srodowisk test, staging, i production. Cinfiguraiton serwis trzyma informacje ( conneciton string ) o bazach danych Daje nam to mozliwosc dynamicznego podmnieniania tej konfifuracji. Kiedy np. chcemy wygenerowac raport na naszym srodowisku testowym, z danych z bazy produkcyjnej, wystarczy tylko zapytac o dane dla bazy danych production. MOżemy również majac testowy serwis do generowania raportu, zaznaczyc aby produckyjny serwis, dostarczal mu te dane, nie testowy. NIe ma potrzeby redepoloyowania serwisu, jesli chcemy zmienic jego konfigruracje, wystarczy tylko jedna zmaina w confiuguration serwis
- 22.Configuration serwis wystaiwa rowzniez json, ktory pokazuje wszyskie serwisy ktore dzialaja, i informacje, czy podany serwis dziala, dzieki temu mozna bardzo latwo monitorowac wszyskie serwisy. Configuration serwis dziala podobnie jak nagios, pinguje wszyskie serwisy ktore sa w nim zarejetrowane, i pokazuje jaki jest ich status ( czy sa ok, czy nie dzialaja )
- 23.W udostepnionym jsonie, jest rowniez informacja o zaleznosciach pomiedzy wszystkimi serwisami ( ktory serwis, uzywa ktory ). Jeśli ktoś potrzebuje wiecej informacji o serwisach, mozna uzyc do zbierania tych danych biblioteki metrics Ta informacja jest wydobywana w ten sposob, -data serwis pyta o swoja konfiguracje, czyli ma zaleznosc do configuration serwis (dependecies) -report generaotr serwis pyta conf serwis o konfiguracje, nastepnie pyta o to gdzie znajdzuje sie data serwis, gdyz potrzebuje inforacji od niego. W tym momenice cofiguration serwis, zapisuje sobie ta informacje, stad tez wie ze jest zaleznosc poniedzy csv-report- generator-serwis i data serwis.
- 24. It’s a simple visualization of excerpt of achitrecutre of microservices. We see which service is currenly unavailable. Some services could sometimes went down
- 25. Kiedy najedziemy myszka na jeden z serwisow, automatycznie zostaja podswietlone serwisy z ktorymi dany serwis rozmawia
- 26.w tym momencie mozna zauwazyc ze do cofniguratoin serwis wszyskie serwisy beda mialy zaleznosc ( jest to wiec signle point of failure) Jednak ten problem zostal rozwiazany przez nas. Oczywistym roziwazaniem tego problemu jest posiadanie wielu ( kilku rozproszonych configuration serwisow ), //Jest to jednak problem jajka i kury Nasuwa sie pytanie, skad wtedy serwisy ktore pytaja o konfiguracje, ( serwisu z konfguracjami ) beda wiedziec gdzie ten configuration serwis sie znajduje ? Jest to jedyna konfiguracja, ktora jest zawszyta ( hard coded ) w kliencie do konfiguration serwisu. Klient ma liste z url'ami na ktorych dzialaja konfiguration serwisy. jesli request do pierwszego z nich nie powiedzie sie, probuje pytac o konfiguracje nastepny serwis na tej liscie.
- 27. polecam gotowe rozwiazanie: Jesli ktos chce miec za darmo serwis ktory bedzie zarzadzal konfiguracja, i bedzie zajmowal sie replikacja danych ( trzymaniem ich synchronizowanych ) pomiedzy nimi to polecam : https://github.com/coreos/etcd Pisanie narzedzia do zarzadzanie taka rozporoszona konfigracja, jest nieoplacalna inwestycja, dla tego planujemy zintegorwac etcd z naszym konfiguration serwis, tak aby configuraiotn srewis byl tylko na wierzchu jak facada, a etcd podspodem, byl odpowiedzialny za replikacje danych
- In such disrupted environment, failure will occure sonner or later. Idea behind Circuit Breaker Pattern is to quickly find a failure, and do not repeat same action, if it is know that this action will cause failure again. When we want to send request to other service, and that service is broken, we do not want our servie to hang on waiting for response from that other service. If one service will hang on waiting for response from another, that service itself will hang after a while, and it will cause faulire in yet another service, it'll case cascade failures.
- 29.biblioteka ktora pomoga w zaimplementowaniu tego wzorca to Hystrix, napisana przez programisow z firmy Netflix. Uzycie w aplikacji : kiedy wysylamy request do innego serwisu, musimy miec na uwadze ze moze sie nie powiesc, wysylamy go wiec tworzoca nowy obiekt CommandWithFallback Kazdy request opakowaujemy w tak zwany Command (komende) - jest to konstrukt z bilbioteki Netflix
- 30.Obiekt ten jest zaimplememtowany w nast sposob. rozszrzamy obiekt HystrixCommand bedzie on zwracal nam String ( response ) Mozemy pogrupowac te komendy ( hystrixowe ) The command group key is used for grouping together commands such as for reporting, alerting, dashboards or team/library ownership. nastepnie implementujemy dwie metody, pierwsza run wykonuje wlasciwa akcje, wysylania requestu do serwisu, nastepnie jesli ta akcja sie nie powiedzie hystrix wywola metoda getFallback w ktorem mozemy obsluzyc ten blad, poprzez np wyslanie tej infomracji do narzedzia monitorujacego ktore odrazu zawiadomi ze cos jest nie tak
- 31.majac wiele serwisow, jest problem z tym ze jest bardzo duza ilosc logow generowanych na roznych serwerach. Jesli chcemy zdjagnozowac cos, musimy wtedy logowac sie na wiele serwerow i przegladac wiele logowa, nie jest to latwe ani szybkie
- 32.Optymalnym rozwiazaniem bylo by posiadanie jednego miescja w ktorym progarmista moglby wejsc i przegladac te logi ktore go inresuja
- 33.rozwiazaniem jest elastiserch i logstash. dziala to w ten sposob ze na jednym serwerze mamy odpalony elastsearch. nastepnie na kazdym serwerze gdzie mamy nasze serwisy instalujemy i konfigurujemy logstasha Logstah monitoruje plik logow, i jesli wykryje w nim cos nowego przesyla ten teskt do elsaticsearch, elastic indeksuje te pliki.
- 34.konfigurujemy logstash'a tak aby nasluchiwal zmian w odpowiedznim pliku, nastepnie dla kazdego logu z tego serwisu doisujemy keyword type = 'bartch-staging' tak aby pozniej bylo latwo filtrowac po tych typach w elasticsearchu Ustawiamy rowniez host na ktorym jest elasticsearch. Jest to nasz centralny punkt z logami.
- 35.nastepnie udajac sie w miejsce gdzie mamy elasticsearch mozemy przefiltrowac rekordy po wczesniej wyspecyfikowanym typie w konfiguracji logstash. widzimy message z logow tej aplikacji Mamy wiec w centralnym punkcie logi dla wszyskich serwisow
- 37.mozemy uzyc graficznego klienta, aby prezentowac te logi w fajniejeszej formie - kibana - jest to anrzedzie do tego celu, widzimy logi, w czasie rzeczywistym . Mozemy specyfikowac filtry - w tym przykladzie chce wyswietlic logi dla procesy batch, ktore zawieraja w message string excetpion, i odpowiedznim odstepie czasu. Logi sa zideksowane przez full text serach ( elasticsearch ) dzieki czemu, mozemy je filtrowac w kazdy mozliwy sposób Widizmi w przykaldzie ze np. o 12:14 było 30 takich logow. Po klikniecciu na pasek dostaniemy dokaldna liste logow wraz z wiadomosciami.
- We are deploying our service to amazon ec2 instantces.
- You need to pull repo with Vagrant file.
- Docker is a technology which allow to create light containers atop of liunx distrubuition, using kernel of underlying operating system. Need to run atop of linux, because of that, but thanks to that it starts very quickly, and docker images took not much disk space.
- 36.Istotnym roweniez elementem jest stworzenie dobrej dokumentacji webowych metod ( interfejsu ) naszej aplikacji. Doskonalym narzedziem do tego jest swagger. Dokumentacja powinna byc precyzyjna i zawierac tylko to, co potrzeba
- 37.aby stworzyc dokumentcje, musimy odpowiednio annotowac nasza metode http. @ApiOperation - piszemy co ta metoda robi, w notes - dajemy dokladnieszy opis @ApiResponses - mozemy specyficzne dla naszego kontrolera ( lub dospecyfikwane te z protokolu http ) response cody @ApiParam mozemy opisac parametry metody http
- 38.nastepnie swagger wygenruje dokumentacje do ktorej mamy dostep pod kontrollerem w naszej aplikacji. Deokumetnacja jest wygyenreowana w pliku json
- 39.nastepnie pozostaje nam tylko wkleic link do naszego kontrollera z dokumentacja w formacie json do interfejsu webowego swagger - swagger-ui
- 40.efektem tego jest stworzona piekna klikalna dokumencacja, nie dosc ze mamy udokumenotwane metody http naszego resotwego mikroserwsu, to jeszcze mozemy testowac go ( uzywac jako jego klienta ) Po wypelnieniu parametrow, mozemy kliknac na przycisk "try it out" i request zostanie wykonany
- 41.Teraz kiedy mamy juz wszyskie kawalki ukladanki, mozemy zaczac tworzyc architekture malych, komunikujacych sie ze soba za pomoca protokolu http, majacych tylko jedną odpwiedzialnosc mikroserwisow. Mozemy przeniesc zasady SOLDI na poziom architektury ( marko ), z poziomu kodu ( mikro )
- 42. dziekuje, jakies pytania ? link na githubie do przykladowego microserwisu, ktory to kazdy moze sobie uruchomic