Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CloudConf2017 - Deploy, Scale & Coordinate a microservice oriented application

183 views

Published on

A presentation about deploy, scaling and the coordination problem. We will focus on redis as a coordination system in order to simplify the migration to ETCd as coordination system

Published in: Software
  • Be the first to comment

  • Be the first to like this

CloudConf2017 - Deploy, Scale & Coordinate a microservice oriented application

  1. 1. CloudConf Turin 16th March 2017 Deploy, scale and coordinate a microservice oriented application cloudconf.it
  2. 2. Walter Dal Mut walter.dalmut @ corley.it Solution Architect @ Corley
  3. 3. Microservices
  4. 4. They're coming outta the walls
  5. 5. Docker Swarm
  6. 6. Issue a new process (deploy) $ docker service create --name app registry.walterdalmut.com/app:v1 $ ./app &
  7. 7. Run different processes (scale) $ docker service scale app=10 $ ./app & $ ./app & $ ./app &
  8. 8. What about the locking system? In a distributed system: locks (distributed locks) are the foundations for activity synchronization
  9. 9. What about coordination?
  10. 10. A service have its own con guration at launch I am here with this address, port etc... A service require other service con gurations Where the database is, which password i should use, etc... The service should reports its owns status I am alive and responsive (for healthcheck) other services can check the health report for the maintenance mode or to shortcircuit the service dependency
  11. 11. To expose the coordination problem we create an application
  12. 12. Read the twitter stream #cloudconf2017
  13. 13. users that tweet with this handle create a reserved API service [ JSON over HTTP ] GET /tweet - list my tweets POST /tweet - record a new tweet The database to store tweets is self-contained in the API service
  14. 14. Now every service is exposed with a unique pair address:port in the swarm
  15. 15. every box is a container (service) blinks for activities (publish new tweets)
  16. 16. Multiple services (1) (1) (1..*) (n) (1..*) Stream readerer Distributed queue Service Worker (need a distributed lock) a lock identi es the service deployment progress on missing => service deploy on existing => publish messages Per user container A proxy to list users and redirects requests
  17. 17. HTTP framework > GET /user/walterdalmut HTTP/1.1 > Host: cluster.corsi.walterdalmut.com:30000 > User-Agent: curl/7.47.0 > Accept: */* < HTTP/1.1 302 Found < location: http://cluster.corsi.walterdalmut.com:30002/v1/tweet < vary: origin < cache-control: no-cache < content-length: 0 < Date: Sun, 12 Mar 2017 11:26:50 GMT < Connection: keep-alive
  18. 18. CloudConf2017 Example
  19. 19. Every user have its own network address and port Every user expose its own API How do we connect services together?
  20. 20. DNS as a coordination system DNS is a good solution to point things in a network DNS SRV expose a service address con guration $ dig srv _auth._tcp.walterdalmut.com +short 1 10 8080 1.api.walterdalmut.com 1 10 8080 2.api.walterdalmut.com
  21. 21. And service con gurations? host: db.mynet.local port: 3306 username: root password: root dbname: example
  22. 22. K / V Several coordination systems available Etcd is one of the most interesting coordination system available Consul integrates di erent things together like: DNS, KV, etc... many other: zookeeper, etc...
  23. 23. Redis as a coordination service
  24. 24. distribute con gurations at paths $ cat mydb.conf | redis-cli set /path/to/disk/mydb.conf -
  25. 25. Where is my `ls` command now? $ redis-cli keys /path/* 1) "/path/to/disk/mydb.conf"
  26. 26. Get my con guration back $ redis-cli get /path/to/disk/mydb.conf host: db.mynet.local port: 3306 username: root password: root dbname: example
  27. 27. How to report the application status? Healthchecks
  28. 28. Con gurations can also expires Dead man switch application reports continuously cat mydb.conf | SETEX /path/to/disk/mydb.conf 30 - EXPIRE /path/to/disk/mydb.conf 30 ... sleep 20 EXPIRE /path/to/disk/mydb.conf 30 ...
  29. 29. Services links together Can i watch for con guration changes? refresh my services on updates
  30. 30. Redis Keyspace Noti cations or in your con guration le CONFIG SET notify-keyspace-events AKE
  31. 31. Listen for my con guration changes SUBSCRIBE __keyspace@0__:/path/to/disk/mydb.conf
  32. 32. Here the event $ cat mydb.conf | redis-cli set /path/to/disk/mydb.conf - 1) "message" 2) "__keyspace@0__:/path/to/disk/mydb.conf" 3) "set"
  33. 33. Distributed locks In a single node for redis NX - if not exists PX 30000 - expires in 30000 ms SET /etc/lock/username/.lock {random_value} NX PX 30000
  34. 34. After 30 seconds the lock expires SET /etc/lock/walterdalmut/.lock 3891573 NX PX 30000 OK SET /etc/lock/walterdalmut/.lock 2857152 NX PX 30000 (nil)
  35. 35. How do i release the lock? DEL /etc/lock/walterdalmut/.lock
  36. 36. How do i extend the lock? EXPIRE /etc/lock/walterdalmut/.lock 30
  37. 37. How do i watch for lock release? Lock releases SUBSCRIBE __keyspace@0__:/etc/lock/walterdalmut/.lock 1) "message" 2) "__keyspace@0__:/etc/lock/walterdalmut/.lock" 3) "del" 1) "message" 2) "__keyspace@0__:/etc/lock/walterdalmut/.lock" 3) "expired"
  38. 38. Thank you for listening

×