Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intro & implementation of a microservices architecture

4,516 views

Published on

Approach to developing a single application as a suite of small services
Designing software applications as suites of independently deployable services.

By PeoplePerHour team
presented by CTO Spyros Lambrinidis & Senior DevOps Panagiotis Moustafellos @ Docker Athens Meetup (The Cube) 18/02/2015

Published in: Technology
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Intro & implementation of a microservices architecture

  1. 1. Micro Services Intro & implementation of a microservices architecture Docker Athens Spyros Lambrinidis @slambrinidis CTO 18 Feb 2014 Panagiotis Moustafellos @pmoust DevOps
  2. 2. What is a MicroService “approach to developing a single application as a suite of small services” “designing software applications as suites of independently deployable services”
  3. 3. What is a MicroService
  4. 4. What is a MicroService ● Each service is loosely coupled / independently deployable ○ Changes made only to this service ● Each service has a bounded context ○ Service should not know about surrounding service (domain driven design)
  5. 5. Characteristics ● Services ● Products vs Projects ● Smart endpoints / dumb pipes ● Decentralized governance ● Decentralised data management ● Automation ● Design for failure ● Evolutionary design
  6. 6. Simple Blog Example ELB WebServer / FE WebServer / FE Comments Service Pages Service Posts Service Redis MySql MySql Mongo Redis SOLR
  7. 7. Advantages ● Small code base / easier to test / maintain ● Easy to scale - clone ● Easy to throw away ● Easy to deploy and track errors ● Freedom to switch tech stack ● Maximise team agility ● Maximise resource utilisation
  8. 8. Disadvantages ● Devops challenge on multiple fronts ● Complexity in messaging and front end ● Most container technologies still new ● Freedom of tech stack not always good news (for the future and for the CTO)
  9. 9. Micro Service and agility ● Modern agile practice can not ignore tech ● No modern tech = no absolute agility ● Micro services enable agility in a special way ○ Enforce team creation ○ Enforce faster deployments / better & easier tests ○ CI / CD ○ Easier communication flow methods (APIs) ○ Each service = small scale product
  10. 10. Container Technology in micros ● Containers assist micro architecture in ○ Visualising services ○ Building / sharing services between coders ○ Deploying services ○ Utilising server resources to run containers irrespective of underlying tech ● Popular container technologies ○ Docker ○ Rocket
  11. 11. Container Management ● Used to maintain and utilize containers / services ○ Make sure all services up and running ○ Make sure server utilisation is maxed out ● Popular container management technologies ○ CoreOS fleet ○ Docker-machine ○ Mesos ○ Kubernetes ○ AWS ECS / Google Container Engine
  12. 12. PPH Specifics ● Architecture history ○ 2008 shared server – outsourced code – raw php ○ 2010 more servers on peer1 and standalone mysql ○ 2010 move to aws ○ 2011 move to yii framework (not 100% complete) ○ Multiple features and optimisation since then ○ 2014 – supertasker.com launch ● Present ○ Monolithic app ○ Multiple db tech (sql, dynamo, mongo, memcache) ○ Search through SOLR ○ Traffic on supertasker.com
  13. 13. PPH Challenges ● Constantly Growing load (data + traffic) ○ 160GB db ○ 1300 rpm avg ○ 35k uniques / day (55k sessions) ● Code complexity from multiple features ● Usage of yii v1.6 – difficult to change. Not exciting for devs to work on it ● Code tightly coupled in many ways ● More products evolving and needing similar features – supertasker.com
  14. 14. PPH Future ● Minimise core db - micros use their own ● Ability to scale to amazing level - scale out ○ design to clone things (clone dbs / web servers) ○ design to split things (one core vs many small ones) ○ design to split similar things (shard / partition) ● Absolute resource utilisation ● Devs playground (use desired tech – within limits)
  15. 15. PPH Future ● Utilisation of communities through container sharing – no need to share apps ● Service sharing between products ● Fully tested / API enabled services ● Gradually decrease power or core db and machines ○ never reach maximum of scaliong up ability
  16. 16. PPH Micro Internals ● Container technology ○ Docker ○ Docker-compose ● Preferred micro language ○ Php using yii v2 which is REST API enabled ● REST API ○ Communication only through well defined APIs ● Full tested Services ○ Each service to have full test coverage (unit & integration & contract)
  17. 17. PPH Micro Internals ● Data ○ Each service to be tied to its own data ● Container Management ○ CoreOS fleet ● Deployment ● CI tool ○ shippable
  18. 18. Workflow: PPH Metrics ● planning the ecosystem ● keeping it simple ● describing the infrastructure plan under version control ● describing the API ● opting for identical D-S-P environments ● enforcing testing (make everything fail) ● deploying seamlessly (as possible) ● applying Continuous Integration & Continuous Delivery
  19. 19. PPH Metrics: Planning Goals: ● Scalable ● Self-contained ● Interoperable ● Proven in production ● Cost effective PPH preference: AWS EC2 AutoScaling + CoreOS (stable channel)
  20. 20. PPH Metrics: defining an API ● Lots of tools (Swagger, RAML, etc) ● Well defined entities - check out schema.org ● Discussion amongst dev team for internal API usage ● Join Athens API Meetup :) We were lazy - no excuses - moving on
  21. 21. PPH Metrics: KISS ● Single node definition ● Single scaling up/down strategy ● Platform agnostic (if possible) ● Human readable Configuration Management and IaC ● DevOps friendly (bridge the gap between Dev & Ops) ● API matters - Preferred Stack does not (or does it?) ● Bring up dev environment in seconds
  22. 22. PPH Metrics: Describing it whole ● Infrastructure as Code Terraform ● Configuration Management CoreOS Cloud-config + Fleet Services ● Linking Containers docker-compose / swarm
  23. 23. PPH Metrics: Terraform Why Terraform? ● Ops make mistakes ● Many environments, many resources, too much room for error, too little time for documenting changes and current state ● Awesome human readable DSL ● Easy to maintain infrastructure state ● Multiple providers (we just needed AWS) ● Developers understand it ● Still very fresh, but well tested ● Created by Hashicorp Version 0.3.7 meets our needs We made small contributions that we needed to have upstream
  24. 24. Terraform DSL - defining an AWS autoscaling group behind a Load Balancer
  25. 25. PPH Metrics: CoreOS Why CoreOS? ● Built with application containers in mind ● Built for scale ● Fleet - manages nodes and services in a cluster ● Etcd - distributed key-value store ● [Unit] - [Service] - [Timer] ● fleetctl - the cli tool to manage Fleet ● etcdctl - the cli tool to manage Etcd
  26. 26. Hands-on example: Cloud-config #cloud-config coreos: etcd: discovery: https://discovery.etcd.io/<YOUR_TOKEN> addr: $private_ipv4:4001 peer-addr: $private_ipv4:7001 fleet: public-ip: $private_ipv4 metadata: role=microservices update: group: stable reboot-strategy: etcd-lock users: - name: pmoust coreos-ssh-import-github: pmoust groups: - sudo - docker ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1…. manage_etc_hosts: localhost units: - name: etcd.service runtime: true command: start - name: fleet.service runtime: true command: start - name: metrics.service command: start content: | [Unit] Description=Metrics Author=pmoust After=docker.service [Service] Restart=always ExecStartPre=/usr/bin/docker kill metrics ExecStartPre=/usr/bin/docker rm metrics ExecStartPre=/usr/bin/docker pull peopleperhour/metrics ExecStart=/usr/bin/docker run --rm --name metrics peopleperhour/metrics ExecStop=/usr/bin/docker stop -t 5 metrics
  27. 27. PPH Metrics: Fleet + services ~ export FLEETCTL_ENDPOINT=http://10.30.2.234:4001 ~ fleetctl list-machines MACHINE IP METADATA 66cc918ae936440b896d201ee47b3877 10.30.2.234 role=microservices bf78eab69d3f4c6f9310c971fd95fd4d 10.30.1.68 role=microservices ~ fleetctl start metrics.service ~ fleetctl list-units -l UNIT MACHINE ACTIVE SUB metrics.service 66cc918ae936440b896d201ee47b3877/10.30.2.234 active running metrics.service bf78eab69d3f4c6f9310c971fd95fd4d/10.30.1.68 active running
  28. 28. PPH Metrics: etcd Used for service discovery and generating configuration files (via confd or other methods) ~ etcdctl ls --recursive / /microservices /microservices/metrics/10.30.2.234:6000 /microservices/metrics/10.30.1.68:6000 /microservices/metrics/version/ffa2eeb4 /microservices/metrics/db/host/metrics.db.peopleperhour.com /microservices/metrics/db/name/metrics /microservices/metrics/db/user/awsdbuser /microservices/metrics/db/pass/youwish /microservices/metrics/aws/access_key/KEY /microservices/metrics/aws/secret_key/SECRET_KEY /microservices/metrics/google/adwords/clientId/CLIENT_ID …...
  29. 29. PPH Metrics: Keeping envs identical The Stack ● MySQL ● Nginx ● PHP (in FastCGI) ● Memcached ● Metrics (our Yii2 framework app (with its dependencies)) In Staging and Production environments most of the stack is out of container scope MySQL -> AWS RDS HTTP LoadBalancing -> AWS ELB Memcached - AWS ElasticCache Etcd holds their endpoints and connection credentials How do we link all these in a Development environment (and remain sane)?
  30. 30. PPH Metrics: docker-compose Fast, isolated development environments using Docker. Simple YAML syntax to link containers, container volumes, expose ports & link hosts.
  31. 31. ● Deployment (strive for seamless updates) ● Continuous Integration ● Continuous Delivery TBD in an upcoming Docker Athens Meetup PPH Metrics: Ship it!
  32. 32. Thank you! Questions?

×