0
DynamoDB at HasOffers
• Attribution for web & mobile marketing 
• Tons of data 
• Ad-hoc analysis 
• Near real-time
DynamoDB at HasOffers 
18 months in production 
207 billion items 
87 TB of data
Use Cases 
Tracking sessions 
Deduplication 
Attribution
Tracking 
Sessions
Use Case: Tracking Sessions 
Session 
Data 
671c6a9a-bed0-4cec-a465-613cca8c9832 
• Impression 
• Click
Use Case: Tracking Sessions 
• Ideal for NoSQL 
• KV lookups 
• Scalability 
• Availability
Deduplication
Use Case: Event Deduplication 
671c6a9a-bed0-4cec-a465-613cca8c9832 
ebe0efef-aa71-471c-b3f8-7b66b458bd79 
ebe0efef-aa71-4...
Use Case: Event Deduplication 
Hash uuid 
Range <none> 
Write capacity = X, Read capacity = 1 
Conditional puts
Use Case: Event Deduplication 
def is_duplicate( dynamo_tbl, uuid ): 
row_data = {‘host’: socket.gethostname(), 
‘created’...
Use Case: Event Deduplication 
• Real-time dedupe 
• Conditional put 
• Writes only; no read 
• Extra debug info 
{uuid: d...
Why NoSQL 
SaaS?
Cluster Resizing 
• Upsize and downsize trivial in DynamoDB 
• Infrequent on own cluster = limits operational experience
Cluster Software Updates 
0.7 1.0 
• Production depends on NoSQL solution 
being available. 
• Software version updates so...
Dynamo 
Operations
DynamoDB Capacity 
Per table R/W capacity 
Capacity units 
Exceed = throttling
DynamoDB Throttling 
CloudWatch = 5 minutes 
DynamoDB = 1 second
DynamoDB Throttling 
• Double capacity on table 
• Root cause from log analysis 
– capacity usage 
– key hot-spotting 
• F...
Closing 
Thoughts
Closing Thoughts 
• Compelling offering 
• Simple operations 
• Avoid hammer syndrome 
• Very happy
Thank you! 
niek@hasoffers.com
Upcoming SlideShare
Loading in...5
×

DynamoDB at HasOffers

386

Published on

How HasOffers uses DynamoDB

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
386
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "DynamoDB at HasOffers "

  1. 1. DynamoDB at HasOffers
  2. 2. • Attribution for web & mobile marketing • Tons of data • Ad-hoc analysis • Near real-time
  3. 3. DynamoDB at HasOffers 18 months in production 207 billion items 87 TB of data
  4. 4. Use Cases Tracking sessions Deduplication Attribution
  5. 5. Tracking Sessions
  6. 6. Use Case: Tracking Sessions Session Data 671c6a9a-bed0-4cec-a465-613cca8c9832 • Impression • Click
  7. 7. Use Case: Tracking Sessions • Ideal for NoSQL • KV lookups • Scalability • Availability
  8. 8. Deduplication
  9. 9. Use Case: Event Deduplication 671c6a9a-bed0-4cec-a465-613cca8c9832 ebe0efef-aa71-471c-b3f8-7b66b458bd79 ebe0efef-aa71-471c-b3f8-7b66b458bd79 d149e52c-d2b3-49bc-b6af-f304b5fab57f DEDUPE! (Idempotency)
  10. 10. Use Case: Event Deduplication Hash uuid Range <none> Write capacity = X, Read capacity = 1 Conditional puts
  11. 11. Use Case: Event Deduplication def is_duplicate( dynamo_tbl, uuid ): row_data = {‘host’: socket.gethostname(), ‘created’: int(time.time())} row = dynamo_tbl.new_item( hash_key=uuid, attrs=row_data ) DEDUPE! try: row.put( expected_value={‘host’: False} ) except DynamoDBConditionalCheckFailedError: return True # not a duplicate return False
  12. 12. Use Case: Event Deduplication • Real-time dedupe • Conditional put • Writes only; no read • Extra debug info {uuid: d149e52c-d2b3-49bc-b6af-f304b5fab57f, host: server03.hasoffers.com, created: 1398884990} DEDUPE!
  13. 13. Why NoSQL SaaS?
  14. 14. Cluster Resizing • Upsize and downsize trivial in DynamoDB • Infrequent on own cluster = limits operational experience
  15. 15. Cluster Software Updates 0.7 1.0 • Production depends on NoSQL solution being available. • Software version updates somewhere between nail-biting and terrifying
  16. 16. Dynamo Operations
  17. 17. DynamoDB Capacity Per table R/W capacity Capacity units Exceed = throttling
  18. 18. DynamoDB Throttling CloudWatch = 5 minutes DynamoDB = 1 second
  19. 19. DynamoDB Throttling • Double capacity on table • Root cause from log analysis – capacity usage – key hot-spotting • Fix root cause • Reduce capacity on table
  20. 20. Closing Thoughts
  21. 21. Closing Thoughts • Compelling offering • Simple operations • Avoid hammer syndrome • Very happy
  22. 22. Thank you! niek@hasoffers.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×