Successfully reported this slideshow.

SVC105 AWS Messaging - AWS re: Invent 2012

2,099 views

Published on

Amazon SNS and Amazon SQS are important systems for sending, managing, and queuing system notifications. This session details the fundamentals on how to use these services, demonstrates the value for application developers, and covers some common use cases and customers where they have solved a critical business problem.

SVC105 AWS Messaging - AWS re: Invent 2012

  1. 1. Queues are everywhere
  2. 2. Notifications are everywhere
  3. 3. Simple QueuingService (SQS)Simple NotificationService (SNS)
  4. 4. Simple QueuingService (SQS)Simple NotificationService (SNS)
  5. 5. Requests
  6. 6. OverloadOverload
  7. 7. ThroughputBacklog
  8. 8. Simple QueuingService (SQS)Simple NotificationService (SNS)
  9. 9. Publish
  10. 10. PublishPublishPublishPublish
  11. 11. Free tier PriceHTTP/S 100k $0.60 / 1mEmail 1k $2 / 100kSMS 100 $0.75 / 100SQS Free delivery to SQS
  12. 12. Amazon SQS + Amazon SNSStore and retrieve any volume of messagesServe any level of throughputWithout any operational burden
  13. 13. Example: cupaNoodle Mobile Food Ordering • Mobile app for food ordering delivery • Promise to customers and to restaurants: “It just works.”
  14. 14. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  15. 15. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  16. 16. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  17. 17. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  18. 18. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  19. 19. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  20. 20. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  21. 21. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  22. 22. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  23. 23. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  24. 24. Turn low availabilityservices into high Data Warehouse Queue Restaurant Payment CC Billingavailability services Queues placed Favorite Meals Queue Social Network Crawling Queue allow non-essential processing to be deferred during Favorite Meals Social Network RDS DB Instance Processors Auto Scaling group Crawlers Auto Scaling group overloads
  25. 25. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  26. 26. Separate queues per restaurant prevents delays in anyone restaurant from affecting deliveries by the others Restaurant 1 Queue Restaurant 2 Queue Restaurant 3 Queue
  27. 27. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  28. 28. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  29. 29. Data Restaurant CC Warehouse Payment Billing Queue Orders Topic Restaurant Notification QueueDurable delivery combined with configurablere-delivery so each order is fully processed
  30. 30. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  31. 31. Seamlessly matches the Data Restaurant CCthroughput of both the Warehouse Payment Billing Queueorder rate and restaurants’delivery rates Restaurant Notification Queue Restaurant 1 QueueAlways catch up, no Restaurant 2matter how large the Restaurant Notification Servers Auto Scaling group Queuebacklog becomes Restaurant 3 Queue
  32. 32. Auto Scaling group Data Restaurant CC Mobile Client Warehouse Payment Billing Queue Orders Elastic Load API Servers Topic BalancingClient Restaurant Notification Queue Restaurant 1 Queue Favorite Social Network Meals Crawling CC Queue Queue Processing Restaurant 2 Restaurant Queue Notification Servers Auto Scaling group Favorite Meals Social Network Processors Crawlers Restaurant 3 RDS DB Instance Auto Scaling group Auto Scaling group Queue Social Networks
  33. 33. Unused queues are free; Data Warehouse Queue Restaurant Payment CC Billingpay only for the requestsyou make Restaurant Notification Queue Restaurant 1 QueueAmazon SQS costs<$0.0001 per cupaNoodle Restaurant Notification Servers Restaurant 2 Queue Auto Scaling grouporder Restaurant 3 Queue
  34. 34. Demo
  35. 35. Best Practices
  36. 36. SendMessage SendMessageBatch
  37. 37. What’s New
  38. 38. prefetching
  39. 39. prefetching
  40. 40. Amazon SQS:Queuing and PullAmazon SNS:Fan-out and Push
  41. 41. http://aws.amazon.com/documentation/sqs/http://aws.amazon.com/documentation/sns/
  42. 42. We are sincerely eager to hear your feedback on thispresentation and on re:Invent. Please fill out an evaluation form when you have a chance.

×