Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Open Source Versions of Amazon's SNS and SQS.pptx

7,762 views

Published on

true

  • Be the first to comment

Open Source Versions of Amazon's SNS and SQS.pptx

  1. 1. CMB An Open Source Alternativefor Queuing and Notification Cloud Services Ryan King Comcast Silicon Valley OpenStack Summit – San Diego 2012
  2. 2. Intro
  3. 3. SummaryA team inside Comcast open sourced acompatible version of Amazon SNS andSQS on top of Cassandra & Redis
  4. 4. Who are we?• Comcast Silicon Valley – Consumer-facing internet products for Comcast – Web, Mobile, TV• Silicon Valley – Product Development & Innovation Center
  5. 5. CMB CMB – Message Bus CQS – Queue ServiceCNS – Notification Service
  6. 6. But, wait.. Why build your own???
  7. 7. Comcast building Private Cloud (… using OpenStack!)
  8. 8. To power next-gen services (… including TV!)
  9. 9. Latency extremely important
  10. 10. Every millisecond counts
  11. 11. Needs to be cost effective (… for 50MM users)
  12. 12. What else is out there?
  13. 13. None of those could meet our requirements 
  14. 14. Requirements• Compatible with AWS• Support Active-Active Multi-Data-Center• Horizontal Scalability• Guaranteed Delivery• Very Low Latency
  15. 15. NOT Required1. Order is NOT guaranteed2. Duplicate messages CAN happen2. Duplicate messages CAN happen
  16. 16. First, the Queue CMB – Message Bus CQS – Queue ServiceCNS – Notification Service
  17. 17. What is Simple Queue Service?(from a Popular Cloud Service Provider)“Simple Queue Service offers a reliable, highlyscalable, hosted queue for storing messages asthey travel between computers”
  18. 18. Simple Queue Service
  19. 19. Main SQS Methods• CreateQueue• SendMessage• ReceiveMessage• DeleteMessage
  20. 20. Our implementation: CQS
  21. 21. Redis ListsCQS API Hashtables CQS API API Payload cache Cassandra
  22. 22. Cassandra Persistence• Queue sharded over N=100 rows –Avoids wide rows (> 500K items) –Reduces “churn” (Tombstones) –Distributes queue among nodes
  23. 23. Redis ListsCQS API Hashtables CQS API API Payload cache
  24. 24. Redis Caching• Message IDs: List• Visibility: Hashtable• Payload cache: Set
  25. 25. CQS Notes• Minimal duplication and good ordering – When things are happy..• Cassandra tombstones degrade message performance – Normally reads => Redis
  26. 26. CQS Performance 90 80 70Latency (ms) 60 50 40 30 20 10 0 10 50 75 100 250 500 1000 Msg/Sec
  27. 27. Why use Cassandra?• very low latency• very high write throughput• extremely high availability (across multi-dc)• Near linear horizontal scalability
  28. 28. Why add Redis?• Queue metadata: – changes frequently – not durable• Needed extremely low latency reads• Reduces “churn” in Cassandra
  29. 29. Next, Notifications CMB – Message Bus CQS – Queue ServiceCNS – Notification Service
  30. 30. What is Simple Notification Service?(from a Popular Cloud Services Provider … inSeattle… )“Simple Notification Service is a web service thatmakes it easy to set up, operate, and sendnotifications from the cloud”
  31. 31. Simple Notification Service
  32. 32. Main SNS Methods•CreateTopic•Subscribe•Publish
  33. 33. Our Implementation: CNS
  34. 34. Scalability• Publish Requests => API servers• # subscribers => Delivery Producers• Publishing to endpoints => Delivery Consumers• Overall => CQS & Cassandra
  35. 35. Throughput Scalability 1200 1000Latency (ms) 800 600 6 workers 400 3 workers 200 0 500 1000 2000 4000 8000 Throughput (Msgs/Sec)
  36. 36. Subscriber Scalability 3000 2500Latency (ms) 2000 1500 3 workers 1000 6 workers 500 0 100 250 500 750 1000 1250 1500 # of Subscribers per Topic
  37. 37. What are we using this for?
  38. 38. Now Open Source! github.com/Comcast/cmb
  39. 39. Next steps• Does this fit into OpenStack?• Get Feedback from the community (you!)• Questions?• ryan@sv.comcast.com• @ryanking• PS - We’re hiring @ Comcast SV!

×