DynamoDB at HasOffers
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
313
On Slideshare
312
From Embeds
1
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 1

https://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. DynamoDB at HasOffers
  • 2. • Attribution for web & mobile marketing • Tons of data • Ad-hoc analysis • Near real-time
  • 3. DynamoDB at HasOffers 18 months in production 207 billion items 87 TB of data
  • 4. Use Cases Tracking sessions Deduplication Attribution
  • 5. Tracking Sessions
  • 6. Use Case: Tracking Sessions Session Data 671c6a9a-bed0-4cec-a465-613cca8c9832 • Impression • Click
  • 7. Use Case: Tracking Sessions • Ideal for NoSQL • KV lookups • Scalability • Availability
  • 8. Deduplication
  • 9. Use Case: Event Deduplication 671c6a9a-bed0-4cec-a465-613cca8c9832 ebe0efef-aa71-471c-b3f8-7b66b458bd79 ebe0efef-aa71-471c-b3f8-7b66b458bd79 d149e52c-d2b3-49bc-b6af-f304b5fab57f DEDUPE! (Idempotency)
  • 10. Use Case: Event Deduplication Hash uuid Range <none> Write capacity = X, Read capacity = 1 Conditional puts
  • 11. Use Case: Event Deduplication def is_duplicate( dynamo_tbl, uuid ): row_data = {‘host’: socket.gethostname(), ‘created’: int(time.time())} row = dynamo_tbl.new_item( hash_key=uuid, attrs=row_data ) DEDUPE! try: row.put( expected_value={‘host’: False} ) except DynamoDBConditionalCheckFailedError: return True # not a duplicate return False
  • 12. Use Case: Event Deduplication • Real-time dedupe • Conditional put • Writes only; no read • Extra debug info {uuid: d149e52c-d2b3-49bc-b6af-f304b5fab57f, host: server03.hasoffers.com, created: 1398884990} DEDUPE!
  • 13. Why NoSQL SaaS?
  • 14. Cluster Resizing • Upsize and downsize trivial in DynamoDB • Infrequent on own cluster = limits operational experience
  • 15. Cluster Software Updates 0.7 1.0 • Production depends on NoSQL solution being available. • Software version updates somewhere between nail-biting and terrifying
  • 16. Dynamo Operations
  • 17. DynamoDB Capacity Per table R/W capacity Capacity units Exceed = throttling
  • 18. DynamoDB Throttling CloudWatch = 5 minutes DynamoDB = 1 second
  • 19. DynamoDB Throttling • Double capacity on table • Root cause from log analysis – capacity usage – key hot-spotting • Fix root cause • Reduce capacity on table
  • 20. Closing Thoughts
  • 21. Closing Thoughts • Compelling offering • Simple operations • Avoid hammer syndrome • Very happy
  • 22. Thank you! niek@hasoffers.com