Micro-service architectures with Gilmour
•
0 likes•3,372 views
Explores common patterns in microservice architectures and how these are addressed in the gilmour library. Discusses async signal-slot as well as synchronous request-response architectures. Introduces unix inspired composition of microservices for more modular and flexible design.
Recommended
Recommended
More Related Content
Similar to Micro-service architectures with Gilmour
Similar to Micro-service architectures with Gilmour (20)
Recently uploaded
Recently uploaded (20)
Micro-service architectures with Gilmour
- 1. Micro-service architectures using Gilmour Aditya Godbole @aagdbl, aa@gdbl.me TechJam
- 2. Services • Process isolation • Polyglot • Potentially remote vs Libraries
- 3. Services • Logical entity - higher level of abstraction • Can be comprised of sub-services • Multiple servers for fault tolerance • Different expectations of reliability vs Servers
- 8. Why?
- 9. Why? • Software problem: Function point scalability
- 10. Why? • Software problem: Function point scalability • People problem: Clean separation of authority and responsibility
- 11. Patterns
- 12. Async - Signals and slots
- 13. Async - Signals and slots Web
- 14. Async - Signals and slots Web User added Signal
- 15. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal
- 16. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal
- 17. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal Push message notifier
- 18. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal user.add.500 Push message notifier
- 19. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal user.add.500 user.add.* user.add.* user.add.* Push message notifier
- 20. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal user.add.500 user.add.* user.add.* user.add.* VIP customer notified user.add.420 Push message notifier
- 21. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal user.add.500 user.add.* user.add.* user.add.* VIP customer notified user.add.420 Push message notifier Push message notifier Push message notifier
- 22. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal user.add.500 user.add.* user.add.* user.add.* VIP customer notified user.add.420 Push message notifier Push message notifier Push message notifier FP scaling
- 23. Async - Signals and slots Web User added Email notifier SMS notifier Slots Signal user.add.500 user.add.* user.add.* user.add.* VIP customer notified user.add.420 Push message notifier Push message notifier Push message notifier FP scaling excl_group: push_msg
- 24. 7 8 server = Server.new 9 gilmour = server.enable_backend('redis') 10 11 waiter = Gilmour::Waiter.new 12 13 waiter.add 14 gilmour.slot "user.added.*" do 15 # This one will send out email notifications 16 logger.info "Sent email notification for -" 17 logger.info request.body 18 waiter.done 19 end 20 21 waiter.add 22 gilmour.slot "user.added.*" do 23 # This one will send out push notifications 24 logger.info "Sent push notification for -" 25 logger.info request.body 26 waiter.done 27 end 28 29 EM.next_tick { 30 gilmour.signal!({ username: 'foo', email: 'foo@bar.com' }, 'user.added.9') 31 } 32 33 waiter.wait
- 25. Signals and slots - Failure Web User added Email notifier SMS notifier Push message notifier Slots Signal
- 26. Signals and slots - Failure Web User added Email notifier SMS notifier Push message notifier Slots Signal Error reporter gilmour.error
- 32. Request Response RPC Hello? Aloha! { data: The actual payload, sender: The origin of the request (unique for each request), code: The response code if this is a response. This uses HTTP error codes } sender_id Wrong number! code
- 34. Signals/Slots vs Request/Reply • Ownership of failure
- 35. Error detection • confirm_subscriber - Are there any active subscribers? • timeout - Client side and service side timeouts
- 36. Errors Hello?
- 41. Errors Hello? Aloha! sender_id Wrong number! code Error reporter ?
- 42. Errors Hello? Aloha! sender_id Wrong number! code Error reporter ? { code: response code (non-‐200) sender: the "sender" from the request topic: the topic to which the request was sent request_data: the request payload userdata: implementation dependent debug infomation backtrace: the backtrace of the error timestamp: the timestamp of the error }
- 44. request_id = Errors + Logs
- 45. Ops - Servers • Log forwarding and aggregation • Error reporting • Monitoring
- 46. Ops - services • What are the services in my system? • Do my services have spare capacity • Do my services respond in time?
- 47. health-bulletin
- 48. Gilmour • Library - no external process • Async - Signals, slots • Sync - Request, Reply • Low level publish subscribe • Scaling - Exclusion groups • Error and health monitoring • Failure detection • No message persistence or failure handing
- 49. For everything else … • Hystrix - Backpressure management, circuit breaking, failure queues, retries
- 51. Services
- 52. Services Micro?
- 53. The wisdom of the masters • Do one thing and do it well • Design programs to be connected to other programs • When a program has nothing surprising to say, it should say nothing. • When you must fail, fail noisily and as soon as possible. • Design for the future, because it will be here sooner than you think.
- 54. Unix philosophy -> micro-services
- 58. stdin, stdout, stderr stdin stderr Error reporter stdout
- 59. Composition • compose - cmd1 | cmd2 | cmd3 • andand - cmd1 && cmd2 && cmd3 • batch - cmd1; cmd2; cmd3 > out OR (cmd1; cmd2; cmd3) > out • parallel & sync
- 60. ((bringup_dc | dc_validate) | (parallel bringup_logger bringup_monitor) | infra_validate)
- 61. 1 def do_bringup(_data, responder) 5 dc_pipeline = responder.compose([ 6 { topic: 'private.customer.datacenter.create', message: data }, 7 { topic: 'private.datacenter.validate' } 8 ]) 9 nodes_pipeline = responder.compose([ 10 responder.parallel([ 11 { topic: 'private.customer.logger.create', message: data }, 12 { topic: 'private.customer.monitor.create', message: data }, 13 ]), 14 ->(d) { d.reduce({}) { |m, out| m.merge(out[:data]) } }, 15 { topic: 'private.dummy.validate' } 16 ]) 17 18 responder.sync(responder.compose([dc_pipeline, nodes_pipeline])) do | resdata, rescode| 19 $stderr.puts rescode, resdata 25 end 26 end
- 62. Gilmour • Library - no external process • Async - Signals, slots • Sync - Request, Reply • Scaling - Exclusion groups • Error and health monitoring • Failure detection • Composition - real micro-services • No message persistence or failure handing
- 63. Languages • Ruby - https://github.com/gilmour-libs/gilmour • Reference implementation. • Most feature complete • Nice • Java / Go - • https://github.com/gilmour-libs/gilmour-e-go and https://github.com/ gilmour-libs/gilmour-e-java • The cleanest API design • Request-reply, signal-slot abstractions • Composition
- 64. Goli - Gilmour cli $: goli request hello-world <(cat data.yml) | echo $: goli request hello-world <(cat data.yml) | goli signal another-hello | echo $: timeout 5 goli request hello-world <(cat data.yml) $: goli status --topic hello-world --topic another- hello $: goli health-check | curl -I “pagerduty….” $: goli tail gilmour.error
- 65. Credits • Ruby community • Piyush Verma @meson10 • Datascale