Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Messaging in the AWS Cloud

3,721 views

Published on

Amazon Web Services provides multiple messaging options that you can use to create scalable, distributed systems, implement event sourcing to unlock hidden context and utilise CQRS for efficient data access. In this session we will look at various messaging patterns and discuss techniques and use cases for Amazon SQS, Amazon SNS, Amazon Kinesis, Amazon DynamoDB and Amazon Web Services IoT in your application.

Speaker: Stephen Liedig, Solutions Architect, Amazon Web Services

Published in: Technology

Messaging in the AWS Cloud

  1. 1. ©  2016,  Amazon  Web  Services,  Inc.  or  its  Affiliates.  All  rights  reserved. Stephen  Liedig  – Solutions  Architect September  2016 Messaging  in  the  AWS  Cloud Choices  for  Distributed  Systems
  2. 2. “Loosely  coupled  systems” The  looser  they  are  coupled, the  bigger  they  will  scale, the  more  fault  tolerant  they  will  be,   the  less  dependencies  they  will  have,   the  faster  you  will  innovate.
  3. 3. AWS  Messaging  Building  Blocks Amazon  SQS Amazon  SNS Amazon  Kinesis AWS  IoT Amazon   EC2 AWS   Lambda
  4. 4. Amazon  SQS • Message  queue  service  that  store   messages  waiting  to  be  processed   (60  sec -­14  days) • Separate  parts  of  an  application • Perform  tasks  asynchronously • Batch  and  burst  processing
  5. 5. Amazon  SQS Customer  Order   Queue Order   message Amazon   CloudWatch Scale  independently • Queue  length • ApproximateAgeOfOldestMessage (New!) Priority  Order   Queue Client Extend  your  business  requirements without  impacting  your  application  logic Dead  letter   queue Priority  Order   message Amazon RDS Amazon EC2
  6. 6. Amazon  SQS DEMO
  7. 7. Amazon  SNS • Reliable  storage  of  messages  for  immediate   or  delayed  processing • Publish  /  Subscribe  – Direct,  Broadcast,   Targeted  “push”  messaging • Multiple  subscribers  protocols  – SQS,   HTTP/S,  Email,  SMS,  Mobile  Push,  Lambda { "default": "Message body text here.", "email": "Message body text here.", "sms": "Message body text here." }
  8. 8. Amazon  SNS Customer  Order   Queue Order   message Priority  Order   Queue Client Priority  Order   message Amazon RDS Amazon EC2 Violating  Design  Principles!  
  9. 9. Amazon  SNS Client Amazon   Redshift AWS Lambda Amazon SNS Amazon EC2 Order   notification Developers Order   Queue Developer Queue Fan-­out  Pattern Marketing
  10. 10. • Focused  on  high  volume  data   streaming • Multiple  consumers • Re-­playable  (24hrs  up  to  7days) • FIFO  (per  shard) Amazon  Kinesis  Streams
  11. 11. Serverless  Sentiment  Analysis AWS Lambda AWS Lambda Kinesis   StreamShard  /  Partition Multiple  Adapters Amazon  Elasticsearch   Service Social  Media   Sources Producers Consumers Amazon   CloudWatch 5min #food Amazon EC2
  12. 12. Amazon  SNS DEMO
  13. 13. • Ephemeral  publish  /  subscribe • Supports  non-­IoT applications • MQTT  over  Websockets AWS  IoT Message  Boker wss://<endpoint>.iot.<region>.amazonaws.com/mqtt
  14. 14. • Need  to  provision  topic • Only  publishes  to  subscribed   endpoints • Multiple  Protocols  /  Mobile • 64-­256  KB Pub  /  Sub  Messaging  Options  Compared • Ephemeral  topics,  no   provisioning  necessary • Wildcard  subscription   • MQTT  over  Websockets • NodeJs,  iOS,  Android  SDK’s   for  non-­IoT use  cases • 128  KB  
  15. 15. Pub  /  Sub  Messaging  Options  Compared Wildcard Description # Matching  the  current  tree  and  all  subtrees.   Subscribe:  Sensor/#   Publish:          Sensor/ Sensor/temp Sensor/temp/room1 + Matches  exactly  one  item  in  the  topic  hierarchy.   Subscribe: Sensor/+/room1 Publish:          Sensor/temp/room1 Sensor/moisture/room1
  16. 16. • Need  to  provision  topic • Only  publishes  to  subscribed   endpoints • Multiple  Protocols  /  Mobile • 64-­256  KB Pub  /  Sub  Messaging  Options  Compared • Ephemeral  topics,  no   provisioning  necessary • Wildcard  subscription   • MQTT  over  Websockets • NodeJS,  iOS,  Android  SDK’s   for  non-­IoT use  cases • 128  KB  
  17. 17. CQRS Command  Query  Responsibility  Segregation
  18. 18. CQRS • Based  on  CQS  (Command  Query  Separation)  pattern   [Myer] • CQRS  [Young,  Dahan]  extends  CQS  pattern  by  providing   physical  separation  of  Command  and  Queries. • CQRS  is  a  pattern  where  you  use  one  model  to  update   information  and  a  separate  model  to  read  information
  19. 19. CQRS • Benefits • Distribute  and  scale  reads  and  writes • Take  advantage  of  eventual  consistency • Flexibility  in  read  and  write  models  (polyglot  persistence) • Not  a  general  purpose  architecture,  will  add  complexity! • Specific  bounded  context • Designed  for  multi-­user  collaborative  applications.
  20. 20. CQRS Amazon   RDS (3NF  DB) Amazon   DynamoDB (1NF  View  Model) Client Commands Queries Domain  Model Command  Handlers ORM <  Event  > Separate  read  and  write  models (Eventually  Consistent) View  models: • one  table  =  one  page  view • Select  *  from  table • Does  not  require  layered   architecture  and  multiple  data   transformations. Commands: • Asynchronous • Designed  not  to  fail De-­normaliser AWS Lambda Amazon   SQS Thin  Read  Layer <  DTO  >
  21. 21. Event  Sourcing
  22. 22. Event  Sourcing • Rebuild  domain  state  by  re-­playing  event  sequence. • Allows  for  projections  and  what-­if  scenarios. • CQRS  pattern  lays  the  foundations  for  an  event  sourced  application. • Capture  all  changes  to  an  application  domain  state  as  a  sequence  of   events. • As  events  come  in,  they  are  appended  to  an  immutable,  append-­only   log  and  can  trigger  automation  workflows,  much  like  a  accounting   ledger.
  23. 23. Amazon   RDS (3NF  DB) CQRS  +  Event  Sourcing Amazon   DynamoDB (1NF) Client Commands Queries Domain  Model Command  Handlers ORM <  Event  > <  DTO  > Immutable,   append-­only  event   log  /  journal • Files  /  Aggregate • RDS • DynamoDB • Event  Store De-­normaliser Amazon SQS Amazon SNS AWS   Lambda Thin  Read  Layer Amazon   DynamoDB (Event  Log) AWS   Lambda
  24. 24. Summary Use  messaging  to  build  reliable,  scalable,  distributed   applications • Amazon  SQS – simple,  scalable,  decoupling  of  components • Amazon  SNS – reliable  publish  subscribe  scenarios • Amazon  Kinesis – scalable  stream  data  processing • AWS  IoT – ephemeral  event  bus  and  message  broker  
  25. 25. Thank  You!

×