Upcoming SlideShare
(ADV402) Beating the Speed of Light with Your Infrastructure in AWS | AWS re:Invent 2014
- 1. November 14, 2014 | Las Vegas, NV Valentino Volonghi, CTO, AdRoll Siva Raghupathy, Principal Solutions Architect, AWS
- 2. 60 billion requests/day
- 3. We Must Stay Up 1% downtime = >$1M
- 4. No Infinitely Deep Pockets
- 5. 100ms MAX Latency
- 6. Paris-New York: ~6000km Speed of Light in fiber: 200,000 km/s RTT latency without hops and copper: 60ms Paris-New York: ~6000km Speed of Light in fiber: 200,000 km/s RTT latency without hops and copper: 60ms 6000 km 60 ms c-RTT
- 7. a a a r •Memcache aOpensource aMature aBlazinglyfast rNostrong guarantees •Redis aOpensource rStoragescale rNotreally distributed rOperationallyintense. •Hbase(we still use this) aOpensource aMaturingquickly aGreatscale rReallyhard to operate
- 8. http://techblog.netflix.com/2014/07/revisiting-1-million-writes-per-second.html Cassandra DynamoDB Delta 10/90mix, $/month $287,064 $131,040 219% 50/50 mix, $/month $287,064 $280,800 ~0% 10/90, 3-yrreserved $27,075.6 ($904kupfront) $15,736 ($504k upfront) 180% •10 people Cassandra ops team: $150k/month (fully loaded) •0 DynamoDBops team: $0
- 9. Instance Nginx ErlangBidder Black Box Supervisor ASG Amazon Kinesis Amazon S3 DynamoDB Elastic Load Balancing Instance Nginx ErlangBidder Black Box Supervisor ASG Elastic Load Balancing
- 10. Amazon S3 Amazon Kinesis DynamoDB DynamoDB DynamoDB
- 11. HashKey Value Key {‘segments’: set([‘123’, ‘345’]), ‘timestamp’:‘1234556’, ‘leads’: ‘to-write-contention’} HashKey RangeKey Value (just a single timestamp) Key Segment 1234554343254 Key Segment1 1231231433235
- 12. ArrayListListWriteRequestArrayListWriteRequestKeyAtomicIntegerAtomicIntegerSetWriteRequestCollectionsHashSetWriteRequestStringAmazonDynamoDB RunnableRunnablerun ListWriteRequest BatchWriteItemResultBatchWriteItemRequestImmutableMapListWriteRequest Map
- 13. Not for me!
- 14. Data Collection Data Storage Global Distribution Bid Storage Bidding
- 15. Data Collection Data Storage Global Distribution Bid Storage Bidding
- 16. Data Collection = Batch Layer Bidding = Speed Layer Data Collection Data Storage Global Distribution Bid Storage Bidding
- 17. US East region Availability Zone Availability Zone Elastic Load Balancing instances instances Auto Scaling group Amazon S3 Amazon Kinesis
- 18. US East region Availability Zone Availability Zone Elastic Load Balancing instances instances Auto Scaling group Amazon S3 Amazon Kinesis Apache Storm DynamoDB US West region EU West region DynamoDB DynamoDB
- 19. Data Collection Bidding US East region Availability Zone Availability Zone Elastic Load Balancing instances instances Auto Scaling group Amazon S3 Amazon Kinesis Apache Storm DynamoDB Availability Zone Availability Zone Auto Scaling group Elastic Load Balancing
- 20. Data Collection •Amazon EC2, Elastic Load Balancing, Auto Scaling Store •Amazon S3 + Amazon Kinesis Global Distribution •Apache Storm on Amazon EC2 Bid Store •DynamoDB Bidding •Amazon EC2, Elastic Load Balancing, Auto Scaling Data CollectionBiddingAd Network 2Ad Network 1Auto Scaling GroupAuto Scaling GroupAuto Scaling GroupAuto Scaling GroupAuto Scaling GroupAuto Scaling GroupAuto Scaling GroupAuto Scaling GroupAuto Scaling GroupApache Stormv2V3V3v1v2V3V3v1V2V3V3V1Auto Scaling GroupV3V4Elastic Load BalancingElastic Load BalancingElastic Load BalancingElastic Load BalancingDynamoDBWriteReadReadReadReadReadReadWriteWritesWriteWriteReadV3` Elastic Load BalancingElastic Load BalancingElastic Load BalancingElastic Load BalancingElastic Load BalancingElastic Load BalancingDynamoDBData CollectionBiddingDynamoDBWriteReadReadWriteWriteWriteAmazon S3Amazon Kinesis
- 21. http://bit.ly/awsevals
Login to see the comments