Cost Savings at High Performance with Redis Labs and AWS
•
1 like•1,802 views
High performance Redis is popular among developers for its incredible performance, versatility and simplicity. The powerful combination of low cost memory and high performance Redis brings to life new next generation analytic uses - such as simultaneous real time transaction and analytics processing. With Redis Labs' RLEC Flash on AWS SSD instances, you can get fantastic performance at up to 70% lower costs. Join this session to learn how next generation Flash from leading memory provider Intel has made significant strides in performance while retaining its cost advantage to memory. Using a combination of AWS' powerful SSD instances, and Redis Labs' RLEC Flash, you can achieve up to 3M ops/sec at sub millisecond latencies, with a combination of RAM and Flash. The session will also feature customer use cases from a large university, a large customer engagement company and a pioneer of online Flash sales. Session sponsored by Redis Labs.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Cost Savings at High Performance with Redis Labs and AWS
Similar to Cost Savings at High Performance with Redis Labs and AWS (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
Cost Savings at High Performance with Redis Labs and AWS
- 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cost Savings at High Performance with Redis Labs and AWS Frank Ober, Data Center Solutions Architect, Intel Yiftach Shoolman, CTO, Redis Labs August 11, 2016
- 2. 2 Open source in-memory database platform for high performance Operational/Analytics/Hybrid use cases
- 3. 3 Redis Tops Database Popularity Rankings ……..#1 NoSQL in User Satisfaction and Market Presence ……..#1 NoSQL among Top 10 Data Stores ……..#1 database on Docker #1 NoSQL database deployed in containers ………#1 in growth among top 3 NoSQL databases ………#1 database in skill demand ………# 1 database in Top Paying Technologies
- 4. 4 Redis Labs – Home of Redis • Founded in 2011 • HQ in Mountain View CA, R&D center in Tel-Aviv IL. • The commercial company behind OSS Redis • Provides enhanced Enterprise-Class Redis: As-a-service – Redis Cloud On-premises – Redis Labs Enterprise Cluster (RLEC)
- 5. 5 Much More than a Key/Value store High Performance Key/Value Store Data-Structure Engine ListsSorted Sets Hashes Hyperlog- logs Geospatial Indexes Bitmaps SetsStrings Bit field
- 6. 6 Data Structure Engine ListsSorted Sets Hashes Hyperlog-logs Geospatial Indexes Bitmaps SetsStrings Bit field • Replication, Data-Persistence, Auto-failover, Scaling • ACID transactions, Pub/Sub, Eviction, Expiration, Lua, 150+ commands • 9 data-structures:
- 7. 7 +
- 8. 8 Analytics & BI Data Sink Spark Architecture Processing Data Source RDD, Data Source, Data Set RDD, Data Source, Data Set Spark SQL & Data Frame
- 9. 9 Spark is Considered Fast But… Read to RDD Deserialization Processing Serialization Write to RDD Analytics & BI 1 2 3 4 5 6 Data SinkData Source
- 10. 10 When Spark Meets Redis Data Source Serving Layer Analytics & BI 1 2 Read filtered/sorted data Write filtered/sorted data Processing
- 11. 11 Spark SQL & Data Frame When Spark Meets Redis Data Source Serving Layer Analytics & BI 1 2 Processing Spark-Redis connector Read filtered/sorted data Write filtered/sorted data
- 12. 12 Accelerating Spark Time-Series with Redis Redis is faster by up to 100 times compared to HDFS and over 45 times compared to Tachyon or Spark
- 13. 13 Support Any Use Case with Redis Modules High Performance Key/Value Store Data-Structure Engine ListsSorted Sets Hashes Hyperlog- logs Geospatial Indexes Bitmaps SetsStrings Bit field In-memory database Platform
- 14. 14 Redis Modules • Any C/C++ (soon Go and Python) program can now run on Redis • Use existing or add new data-structures • Enjoy simplicity, high performance, infinite scalability and high availability • Can be created by anyone
- 15. 15 Available Modules redisearch redabloom topk countminsketch GraphSecondary Indexes Coming soon: Available OSS: JSON + >20 more Machine Learning
- 16. 16 16 3.15 2.40 21.00 8.70 24.57 10.61 0.00 5.00 10.00 15.00 20.00 25.00 30.00 Full text search Prefix search Average Latency (msec) RLEC Elasticsearch Solr 20,045 6,831 690 3,686 621 3,133 0 5,000 10,000 15,000 20,000 25,000 Full text search Prefix search Ops/sec RLEC Elasticsearch Solr 85% higher 32x higher 7.8x faster 4.1x faster How Fast Can a Module Run ? Redisearch - the world’s fastest text search engine
- 17. 17 • A marketplace for OSS and closed source modules • All modules are certified by Redis Labs for these platforms: Module Hub 1M+ instances 350K+ instances 100+ F500 Companies www.redismodules.com
- 18. 18 Is Redis Expensive ?
- 19. 19 Sometimes Redis Can Be Very Expensive #1 200GB #2 200GB #50 200GB 50 x r3.8xlarge instances #51 200GB #51 200GB #100 200GB 1st replica for HA #101 200GB #102 200GB #150 200GB 2nd replica for quorum HA deployment of 10TB Redis dataset on AWS Total cost (reserved instances) = $2,132,250/yr
- 20. 20 Do we really need 2 replicas? #1 200GB #2 200GB #50 200GB 50 x r3.8xlarge instances #51 200GB #51 200GB #100 200GB 1 replica for HA #101 15GB Efficient HA deployment of 10TB in-memory dataset on AWS Total cost (reserved instances) = $1,421,500/yr Savings = $710,750/yr Redis Labs Patent Pending Technology 1 quorum server
- 21. 21 .. Can We Save More?
- 22. 22 Price/Performance of Memory Technology 0.10 1.00 90.00 250.00 0.10 1.00 25.00 500.00 0.10 1.00 10.00 100.00 1,000.00 $0 $1 $2 $3 $4 $5 $6 $7 $8 $9 $10 DRAM NV-DIMM/PM NVMe SSD SATA SSD microsec $/GB 1 GB cost vs. Read/Write Performance 1 GB cost Read latency Write latency $9 $0.4 $1 $2
- 23. 23 Redis on Flash (RoF) • Flash used as a RAM extender (not as a persistent storage) ‒ In RAM: dict & keys & hot values ‒ On Flash: cold values ‒ Values are either in RAM or on Flash • Multi-shard; multi-threaded; asynchronous • Pluggable storage engine (RocksDB and IBM’s ark) • 100% compatible with the OSS Redis • Available on Redis Cloud Private (AWS VPC) • Created by the team who develops and maintains the OSS Redis
- 26. 26 Increase RAM ratio Scale-out Cluster
- 27. 27 10TB Redis Deployment on AWS Redis (on RAM) 2 replicas Redis (on RAM) 1 replica Redis on Flash 1 replica Instance type r3.8xlarge r3.8xlarge i2.8xlarge # of instances 150 100 10 RAM 30TB 20TB 2TB Flash - - 64TB Persistent storage (EBS) 150TB 100TB 80TB 1yr costs (reserved instances) $2,132,250 $1,421,500 $345,310 Yearly savings - $710,750 $1,786,940 Savings % - 33.33% 83.81%
- 28. 28 .. How Fast RoF Can Run ?
- 29. 29 Factor #1 – RAM:Flash Hit Ratio 95% / 85% / 50% 5% / 15% / 50% Higher performance Lower performance
- 30. 30 Factor #2 – Storage Speed (how many IOPS) Single Instance Max Ops/sec @ < msec latency RAM/Flash hit ratio – 50%/50%, Read/Write ratio – 50%/50%, Object size – 100B 21K 59K 126K 175K 0K 20K 40K 60K 80K 100K 120K 140K 160K 180K 200K i2.xlarge R/W IOPS - 35K/35K i2.2xlarge R/W IOPS - 75K/75K i2.4xlarge R/W IOPS - 175K/155K i2.8xlarge R/W IOPS - 365K/315K
- 32. 32 Remember to complete your evaluations!
- 33. Thank you!