RedisConf17 - Turbo-charge your apps with Amazon Elasticache for Redis
•Download as PPTX, PDF•
3 likes•501 views
This document provides an overview and summary of Amazon ElastiCache for Redis. It discusses the key features of ElastiCache including easy deployment and monitoring, enhanced Redis engine capabilities, high availability, cost effectiveness, and integration with other AWS services. It also covers usage patterns such as database caching, streaming data processing, and building real-time apps. Finally, it discusses best practices for building resilient architectures on ElastiCache including reference architectures, failure scenarios, and open source contributions from AWS.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to RedisConf17 - Turbo-charge your apps with Amazon Elasticache for Redis
Similar to RedisConf17 - Turbo-charge your apps with Amazon Elasticache for Redis (20)
More from Redis Labs
More from Redis Labs (20)
Recently uploaded
Recently uploaded (20)
RedisConf17 - Turbo-charge your apps with Amazon Elasticache for Redis
- 1. Michael Labib, Global Specialist Data Architect Turbo-charge your apps with Amazon ElastiCache for Redis © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon ElastiCache @michaelslabib
- 2. “Generally, the craft of programming is the factoring of a set of requirements into a set of functions and data structures.” GET quoteREDIS:6379> -- Douglas Crockford --
- 3. What to Expect from the Session LRANGE agenda 0 2 1) “Amazon ElastiCache Overview” 2) “Engine Enhancements” 3) “Usage Patterns” REDIS:6379>
- 4. SMEMBERS featuresREDIS:6379> Amazon ElastiCache 1) “Easy to deploy & monitor” 2) “Enhanced Redis Engine“ 3) “Highly Available with fast auto-failover” 4) “Fully managed; Zero admin” 5) “Cost Effective” 6) “Redis and Memcached” 7) “Microsecond performance” 8) “Secured in VPC”
- 5. SMEMBERS features REDIS:6379> Amazon ElastiCache 1) “Easy to deploy & monitor” AWS Config Amazon CloudWatch AWS CloudTrail AWS CloudFormation AWS Management Console AWS CLI and SDKs alarm REDIS:6379> hget feature:details “deploy-monitor” Amazon SNS Email Notification AWS Lambda
- 6. SMEMBERS features REDIS:6379> Amazon ElastiCache REDIS:6379> hget feature:details “enhancements” 2) “Enhanced Redis Engine” Optimized Swap Memory •Mitigate the risk of increased swap usage during syncs and snapshots. Dynamic write throttling •Improved output buffer management when the node’s memory is close to being exhausted. Smoother failovers •Clusters recover faster as replicas avoid flushing their data to do a full re-sync with the primary.
- 7. SMEMBERS ec-team:open-source:contributors Amazon ElastiCache REDIS:6379> https://raw.githubusercontent.com/antirez/redis/2.8/00-RELEASENOTES (from https://github.com/antirez/redis/issues/1814) 1) “Kevin McGehee” 2) “Qu Chen” hgetall ec-team:open-source:contributionsREDIS:6379> 1) PSYNC2 (Redis 4.0) https://raw.githubusercontent.com/antirez/redis/4.0/00-RELEASENOTES 3) BGSAVE (Redis 3.2) https://raw.githubusercontent.com/antirez/redis/3.2/00-RELEASENOTES 5) MIGRATE (Redis 3.0) 7) MASTER TIMEOUT (Redis 2.8) https://raw.githubusercontent.com/antirez/redis/3.0/00-RELEASENOTES https://raw.githubusercontent.com/antirez/redis/2.8/00-RELEASENOTES 9) INCREASE 2Billion+ KEYS in a DATASET (Redis 2.8) 2) 4) 6) 8) 10)
- 8. SMEMBERS features REDIS:6379> Amazon ElastiCache 5) “Cost Effective” REDIS:6379> hget feature:details “cost” • Get Started for Free! - Get 750 hours free per month for an entire year with a micro node • On Demand – Pay as low of 1.7 cents / hour • Reserved Instances – Optimize further by reserving 1 or 3 year capacity and save up to 65% using ElastiCache Ris • No Cross-AZ Data Transfer Costs – No charge associated with multi-AZ cluster communication
- 9. ElastiCache Redis Reference Architecture Availability Zone B Availability Zone CAvailability Zone A REDIS:6379> hget feature:details “ref-arch”
- 10. ElastiCache Redis Reference Architecture Availability Zone B Availability Zone CAvailability Zone A Private SubnetPrivate Subnet Private Subnet REDIS:6379> hget feature:details “ref-arch”
- 11. ElastiCache Redis Reference Architecture Availability Zone B Availability Zone CAvailability Zone A Private SubnetPrivate Subnet Private Subnet REDIS:6379> hget feature:details “ref-arch” security group security group security group
- 12. ElastiCache Redis Reference Architecture Availability Zone B Availability Zone CAvailability Zone A Private SubnetPrivate Subnet Private Subnet REDIS:6379> hget feature:details “ref-arch” security group security group security group Primary Elasticache Redis Cluster Amazon S3 bucket REDIS RDB snapshot
- 13. ElastiCache Redis Reference Architecture Availability Zone B Availability Zone CAvailability Zone A Private SubnetPrivate Subnet Private Subnet REDIS:6379> hget feature:details “ref-arch” security group security group security group Primary Elasticache Redis Cluster security group Public Subnet security group Public Subnet Public Subnet Low Latency Secure Direct Access Amazon S3 bucket REDIS RDB snapshot security group
- 14. ElastiCache for Redis Failure Scenarios “Everything fails, all the time.” GET quote:failureREDIS:6379> -- Werner Vogels, CTO Amazon.com --
- 15. Availability Zone A slots 0 - 5454 slots 5455 – 10909 Redis Cluster slots 10910 – 16363 Availability Zone B Availability Zone C slots 5455 – 10909 slots 5455 – 10909slots 0 - 5454 slots 0 - 5454 slots 10910 – 16363 slots 10910 – 16363 Scenario 1: Single Primary Failure
- 16. Availability Zone A slots 0 - 5454 slots 5455 – 10909 Redis Cluster Scenario 1: Single Primary Failure slots 10910 – 16363 Availability Zone B Availability Zone C slots 5455 – 10909 slots 5455 – 10909slots 0 - 5454 slots 0 - 5454 slots 10910 – 16363 Mitigation: 1. Automatic Failure Detection & Replica Promotion (~15-30s) 2. Repair Failed Node slots 10910 – 16363
- 17. Availability Zone A slots 0 - 5454 slots 5455 – 10909 Redis Cluster Scenario 2: Majority of Primaries Fail slots 10910 – 16363 Availability Zone B Availability Zone C slots 5455 – 10909 slots 5455 – 10909slots 0 - 5454 slots 0 - 5454 slots 10910 – 16363slots 10910 – 16363
- 18. Availability Zone A slots 0 - 5454 slots 5455 – 10909 Redis Cluster slots 10910 – 16363 Availability Zone B Availability Zone C slots 5455 – 10909 slots 5455 – 10909slots 0 - 5454 slots 0 - 5454 Mitigation: Redis enhancements on ElastiCache • Automatic Failure Detection and Replica Promotion • Repair Failed Nodes slots 10910 – 16363slots 10910 – 16363 Scenario 2: Majority of Primaries Fail
- 20. Usage Patterns Session Management Database Caching APIs (HTTP responses) IOT Streaming Data Analytics (Filtering/Aggregation) Pub/Sub Social Media (Sentiment Analysis) Standalone Database (Metadata Store) Leaderboards
- 21. Database Caching Clients Amazon ElastiCache Redis Amazon DynamoDB Elastic Load Balancing Amazon EC2 Amazon RDS AWS Lambda write-through reads/ writes DDB streams mysql.lambda_async Simplified write-through with RDS and DynamoDBreads/ writes https://aws.amazon.com/caching/database-caching/
- 22. Amazon Kinesis Analytics Amazon Kinesis Streams Amazon Kinesis Streams Amazon ElastiCache (Redis) cleansed stream Streaming Data Enrichment / Processing DataSources raw stream Subscribers Lambda Function 1 ContinualData Filtering/Enrichment Real-time Pub/Sub Lambda Function 2
- 23. Big Data Architectures using Redis Amazon Kinesis DataSources AWS Lambda Apache Storm on EMR Spark Streaming on EMR Amazon Kinesis App Amazon EC2 AWS IoT Amazon ElastiCache Collect Store Process Amazon S3 Apache Kafka AWS Lambda Custom App Spark on EMR Analyze
- 24. Event Data AWS Lambda Amazon ElastiCach e Amazon DynamoDB Streams Amazon SES Amazon S3 Amazon Cognito Amazon SNS Amazon CloudWatch (Events/Logs) Amazon Kinesis Streams Amazon API Gateway AWS CloudFormation AWS CodeCommit Amazon Lex Amazon Echo Amazon ElastiCache Redis AWS Lambda ElastiCache as an event store event sources Node.js,Java, Python Dashboards, Apps, etc. Sub Pub
- 25. Rules Engine Amazon ElastiCache Redis AWS Lambda Direct Integration LambdaSNS SQS S3 KinesisDDB AWS IoT Devices AWS IoT Sensor Store IoT powered by ElastiCache https://aws.amazon.com/blogs/database/managing-iot-and-time-series-data-with-amazon-elasticache-for-redis/
- 26. Mobile Apps Powered by ElastiCache Amazon API Gateway AWS Lambda Amazon ElastiCache Redis GEOADD GEORADIUS Search points of interest Update points of interest https://aws.amazon.com/blogs/database/amazon-elasticache-utilizing-redis-geospatial-capabilities/ Amazon DynamoDB DDB streams Amazon EC2
- 27. Chat Apps powered by ElastiCache https://aws.amazon.com/blogs/database/amazon-elasticache-utilizing-redis-geospatial-capabilities/ Clients Chat Apps Application Load Balancer WebSockets Amazon ElastiCache Redis PubSub Serverpersistent connections Elastic Beanstalk
- 28. Amazon ElastiCache Summary Enhanced Redis engine for increased robustness to support your critical workloads Cost-effective solution with various pricing options to optimize your TCO Native support with the AWS ecosystem of services Fully managed with zero administration allowing you to focus on your business
- 29. Thank You!