Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mastering message queues

170 views

Published on

Talk given at DPC 2018 Amsterdam.

Published in: Technology

Mastering message queues

  1. 1. Mastering Message Queues Tobias Nyholm @tobiasnyholm @tobiasnyholm
  2. 2. @tobiasnyholm The data structure Top Bottom Front Back
  3. 3. @tobiasnyholm Why?
  4. 4. @tobiasnyholm Tobias Nyholm • Full stack unicorn on Happyr.com • Certified Symfony developer • Symfony core member • PHP-Stockholm • Open source
  5. 5. @tobiasnyholm Open source PHP-cache HTTPlug Mailgun LinkedIn API clientSwap Stampie BazingaGeocoderBundle PHP-Geocoder FriendsOfApi/boilerplate Guzzle Buzz CacheBundlePSR7 SymfonyBundleTest NSA SimpleBus integrations PSR HTTP clients Neo4j KNP Github API PHP-Translation Puli Assert Backup-manager/symfony php-http/httplug-bundle php-http/multipart-stream php-http/discovery happyr/normal-distribution-bundle nyholm/effective-interest-rate MailgunBundle league/geotools
  6. 6. @tobiasnyholm 2013
  7. 7. @tobiasnyholm
  8. 8. @tobiasnyholm
  9. 9. @tobiasnyholm <?php namespace WebfishCompanyBundleController; use MoneyMoney; use RawlsBaseBundleControllerBaseController; use SensioBundleFrameworkExtraBundleConfigurationRoute; use SensioBundleFrameworkExtraBundleConfigurationTemplate; use SymfonyComponentHttpFoundationRedirectResponse; use SymfonyComponentHttpFoundationRequest; use WebfishCompanyBundleEntityCompany; use WebfishCompanyBundleEntitySaleImport; use WebfishCompanyBundleFormSalesImportType; use WebfishCompanyBundleModelSalesImportModel; class ImportController extends BaseController { /** * @param Request $request * * @Route("/import/sales", name="sales_import") * @Template() * * @return array|RedirectResponse */ public function importAction(Request $request)
  10. 10. –Twitter “Testing your controllers.”
  11. 11. @tobiasnyholm
  12. 12. @tobiasnyholm Matthias Noback
  13. 13. @tobiasnyholm Infrastructure The world outside Web browser Terminal Database Messaging Filesystem (E)mail (Matthias Noback)
  14. 14. @tobiasnyholm Controller Command Creates CommandHandler Command bus
  15. 15. @tobiasnyholm HTTP Request Form Request Controller Patient (entity) W eb port PatientRepository RegisterPatient- Handler RegisterPatient (command) Infrastructure Application Dom ain (Matthias Noback)
  16. 16. @tobiasnyholm Infrastructure The world outside Web browser Terminal Database Messaging Filesystem (E)mail (Matthias Noback)
  17. 17. @tobiasnyholm Hexagonal architecture https://www.youtube.com/watch?v=fgQWnglnGeU
 https://matthiasnoback.nl/2015/01/a-wave-of-command-buses/
 https://martinfowler.com/bliki/CQRS.html
 https://hexagonal-architecture.eu/
  18. 18. @tobiasnyholm What did we win?
  19. 19. @tobiasnyholm Middlewares!
  20. 20. @tobiasnyholm Controller Command Creates CommandHandler Command bus
  21. 21. @tobiasnyholm Controller Command Creates CommandHandler Command bus Queue
  22. 22. @tobiasnyholm Let’s talk about queues
  23. 23. @tobiasnyholm Work queue Producer Consumer A Consumer B
  24. 24. @tobiasnyholm Fanout Producer Consumer A Consumer B Binding Binding Publish/Subscribe
  25. 25. @tobiasnyholm Direct Producer Consumer A Consumer B Foo Baz Bar Routing
  26. 26. @tobiasnyholm Routing Direct Producer Consumer A Consumer B Foo Foo
  27. 27. @tobiasnyholm Topics Topic Producer Consumer A Consumer B *.error app.* firewall.# app.error firewall.notice app.notice
  28. 28. @tobiasnyholm Let’s see some code
  29. 29. @tobiasnyholm $ composer req symfony/messenger
  30. 30. @tobiasnyholm Controller Message Creates Message handler Message bus
  31. 31. @tobiasnyholm<?php namespace AppMessageCommand; class SendNotification { private $message; private $users; public function __construct(string $message, array $users) { $this->message = $message; $this->users = $users; } public function getMessage(): string { return $this->message; } public function getUsers(): array { return $this->users; } }
  32. 32. @tobiasnyholm<?php namespace AppController; use AppMessageCommandSendNotification; use SymfonyComponentMessengerMessageBusInterface; // ... class DefaultController { public function index(MessageBusInterface $bus, Request $request) { $users = ['samuel', 'christelle']; $message = $request->query->get('message', 'Something.'); $bus->dispatch(new SendNotification($message, $users)); return new Response('<html><body>OK.</body></html>'); } }
  33. 33. @tobiasnyholm
  34. 34. @tobiasnyholm <?php namespace AppMessageHandler; use AppMessageCommandSendNotification; class SendNotificationHandler { public function __invoke(SendNotification $message) { foreach ($message->getUsers() as $user) { echo "Send notification to... ".$user."n"; } } }
  35. 35. @tobiasnyholm # config/services.yaml services: AppMessageCommandHandlerSendNotificationHandler: tags: - messenger.message_handler # — — — — — — # config/services.yaml services: AppMessageCommandHandler: resource: ‘../src/Message/CommandHandler/*’ tags: ['messenger.message_handler']
  36. 36. @tobiasnyholm
  37. 37. @tobiasnyholm Message Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Transports
  38. 38. @tobiasnyholm Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Message Transports • Sender • Receiver • Configurable with DSN • Encode/Decode
  39. 39. @tobiasnyholm # config/packages/messenger.yaml framework: messenger: transports: default: '%env(MESSENGER_DSN)%' routing: ‘AppMessageCommandSendNotification’: default # ------------- # .env MESSENGER_DSN=amqp://guest:guest@localhost:5672/%2f/messages
  40. 40. @tobiasnyholm
  41. 41. @tobiasnyholm $ bin/console messenger:consume-messages
  42. 42. @tobiasnyholm
  43. 43. @tobiasnyholm
  44. 44. @tobiasnyholm Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Message
  45. 45. @tobiasnyholm
  46. 46. @tobiasnyholm Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Message
  47. 47. @tobiasnyholm Transport dispatch Transport send Worker receive Message handler handles Message handler dispatch App 1 TransportTransport Worker receive Message handler handles Message handler dispatch App 2 MessageMessage
  48. 48. @tobiasnyholm # config/packages/messenger.yaml framework: messenger: transports: default: ‘%env(MESSENGER_DSN)%' enqueue: '%env(ENQUEUE_DSN)%' routing: ‘AppMessageCommandSendNotification': [default, enqueue] # — — — — — — # .env MESSENGER_DSN=amqp://guest:guest@localhost:5672/%2f/messages ENQUEUE_DSN=enqueue://acme
  49. 49. @tobiasnyholm Direct Producer Consumer A Consumer B Foo Baz Bar TransportTransport Worker receive Message handler handles Message handler dispatch App 2
  50. 50. @tobiasnyholm Multiple busses
  51. 51. @tobiasnyholm Multiple busses Command Command Handler Event Query Query Handler Event SubscriberEvent SubscriberEvent Subscriber
  52. 52. @tobiasnyholm Multiple busses # config/packages/messenger.yaml framework: messenger: # The bus that is going to be injected when injecting MessageBusInterface: default_bus: messenger.bus.commands # Create buses buses: messenger.bus.command: ~ messenger.bus.query: ~ messenger.bus.event: ~
  53. 53. @tobiasnyholm Multiple busses # config/packages/messenger.yaml framework: messenger: # The bus that is going to be injected when injecting MessageBusInterface: default_bus: messenger.bus.commands # Create buses buses: messenger.bus.command: middleware: - messenger.middleware.exactly_one_handler - messenger.middleware.validation - 'AppMiddlewareEventStoreMiddleware' messenger.bus.query: middleware: - messenger.middleware.exactly_one_handler - messenger.middleware.validation messenger.bus.event: middleware: - messenger.middleware.allow_no_handler - messenger.middleware.validation
  54. 54. @tobiasnyholm Multiple busses services: _defaults: # ... bind: $commandBus: '@messenger.bus.command' $queryBus: '@messenger.bus.query' $eventBus: '@messenger.bus.event'
  55. 55. @tobiasnyholm <?php namespace AppMessageQuaryHandler; use AppMessageQuaryGetUserForNotification; class GetUsersForNotificationHandler { public function __invoke(GetUserForNotification $query) { $users = $this->getUsers($query->getSubject()); return $users; } private function getUsers(string $subject) { // TODO Decide upon $subject return ['tobias', ‘maria', 'samuel']; } }
  56. 56. @tobiasnyholm Send events from handler class SendNotificationHandler { private $eventBus; public function __construct(MessageBusInterface $eventBus) { $this->eventBus = $eventBus; } public function __invoke(SendNotification $message) { foreach ($message->getUsers() as $user) { echo "Send notification to... ".$user."n"; } $this->eventBus->dispatch(new NotificationSent()); } }
  57. 57. @tobiasnyholm Handle failure
  58. 58. @tobiasnyholm Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Requeue Message
  59. 59. @tobiasnyholm Error
  60. 60. @tobiasnyholm Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Requeue Message
  61. 61. @tobiasnyholm Create new entities
  62. 62. @tobiasnyholmuse SymfonyComponentValidatorConstraints as Assert; class CreateUser { /** * @AssertUuid() * @AssertNotBlank() */ private $uuid; /** * @AssertNotBlank() * @AssertLength(min=3) */ private $username; public function __construct(string $uuid, string $username) { $this->uuid = $uuid; $this->username = $username; } public function getUuid(): string { return $this->uuid; } public function getUsername(): string {
  63. 63. @tobiasnyholm <?php namespace AppController; use AppMessageSendNotification; use SymfonyComponentMessengerMessageBusInterface; // ... class UserController { public function create(MessageBusInterface $bus, Request $request) { $uuid = (string) Uuid::uuid4(); $bus->dispatch(new CreateUser($uuid, $request->request->get('username', ''))); return new Response('See /api/user/'.$uuid); } }
  64. 64. @tobiasnyholmuse SymfonyComponentValidatorConstraints as Assert; class CreateUser { /** * @AssertUuid() * @AssertNotBlank() */ private $uuid; /** * @AssertNotBlank() * @AssertLength(min=3) */ private $username; public function __construct(string $uuid, string $username) { $this->uuid = $uuid; $this->username = $username; } public function getUuid(): string { return $this->uuid; } public function getUsername(): string {
  65. 65. @tobiasnyholm Questions? https://joind.in/talk/bf257
  66. 66. @tobiasnyholm Envelope Message handler Message bus Message handler dispatch handles Worker dispatch Transport Transport receive send Message
  67. 67. @tobiasnyholm Questions? https://joind.in/talk/bf257

×