Moving Beyond Cache by Yiftach Shoolman - Redis Day Bangalore 2020
•Download as PPTX, PDF•
0 likes•183 views
Moving Beyond Cache with Redis Enterprise by Yiftach Shoolman of Redis Labs
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Moving Beyond Cache by Yiftach Shoolman - Redis Day Bangalore 2020
Similar to Moving Beyond Cache by Yiftach Shoolman - Redis Day Bangalore 2020 (20)
More from Redis Labs
More from Redis Labs (19)
Recently uploaded
Recently uploaded (20)
Moving Beyond Cache by Yiftach Shoolman - Redis Day Bangalore 2020
- 1. Moving Beyond Cache Yiftach Shoolman - @yiftachsh Co-Founder & CTO, Redis Labs
- 2. PRESENTED BY MOST LAUNCHED MOST USED MOST LOVED Sumo Logic, Sept. 2019 Stack Overflow, 2017, 2018, 2019Launches/day: Docker Hub, Jan. 2020 How popular is Redis?
- 3. PRESENTED BY redis.io redislabs.com How popular is Redis in India?
- 5. PRESENTED BY Why Redis as a database? 100ms 50ms 50ms 1ms DatabaseApp servers Humans Apps Machines
- 6. PRESENTED BY Why Redis as a database? 100ms 50ms 50ms 1ms App servers Humans Apps Machines
- 7. PRESENTED BY Moving beyond cache • Database ingredients • Modern data-types • Shared dataset across any deployment: cloud/multi-cloud/hybrid/VMs/Kubernetes
- 8. PRESENTED BY Hashes Bitmaps Strings Bit Field Streams Hyperloglog Sorted Sets Sets Geospatial Search Graph TimeSeries AI JSON Gears Redis core Redis modules BloomFilter Redis Enterprise core Redis Enterprise Lists Linear scalability HA Geo-distributionDurability Backup & restore Tiered-memory access SecurityMulti-tenant
- 9. PRESENTED BY Redis core Redis modules Redis Enterprise core Redis Enterprise Hashes Bitmaps Strings Bit Field Streams Hyperloglog Sorted Sets Sets Geospatial Lists Durability Geo-distributionLinear scalability HA Backup & restore Tiered-memory access SecurityMulti-tenant Search Graph TimeSeries AI JSON Gears BloomFilter
- 11. PRESENTED BY Persistent but NOT Durable DRAM SSD DRAM Persistent AND Durable presented by
- 12. PRESENTED BY DRAMDRAM SSD Persistent but NOT Durable Persistent AND Durable presented by
- 13. PRESENTED BY DRAM SSD DRAMDRAM SSD DATA LOSS! DRAM NO DATA LOSS! Persistent but NOT Durable Persistent AND Durable presented by
- 14. PRESENTED BY Why is this important? 0.5ms Redis Disk-based 0.5ms presented by
- 15. PRESENTED BY ~0.5ms Redis >50ms Disk-based Multiple IOPSA single IOPS ~100x faster A ‘write’ request presented by
- 16. PRESENTED BY Redis >50ms Disk-based Multiple IOPSZero IOPS → 0.1ms ~500x faster A ‘read’ request presented by
- 17. PRESENTED BY Why you need durability and replication Node 1 Node 2 Node 3 presented by
- 18. PRESENTED BY Durability challenges solved by Redis Enterprise presented by 1 Maintaining fairness and high IOPS when multiple instances share the same disk 2 High-performance during AOF rewrite/snapshot 3 Guaranteed SLA for recovering from AOF
- 19. PRESENTED BY Redis Enterprise: fast storage engine presented by Single node Redis Enterprise ACID ops/sec (<1ms) on Dell-EMC VMAX
- 20. PRESENTED BY Redis core Redis modules Redis Enterprise core Redis Enterprise Hashes Bitmaps Strings Bit Field Streams Hyperloglog Sorted Sets Sets Geospatial Lists Geo-distributionLinear scalability HA Durability Backup & restore Tiered-memory access SecurityMulti-tenant Search Graph TimeSeries AI JSON Gears BloomFilter
- 21. PRESENTED BY 21 Disaster recovery Why you need Active-Active Redis
- 22. PRESENTED BY 22 Session managementDisaster recovery Why you need Active-Active Redis
- 23. PRESENTED BY 23 Globally distributed applications ₤$ $ ₱ € Session managementDisaster recovery Why you need Active-Active Redis
- 24. PRESENTED BY • Internet latency → >100ms • Conflict resolution → too complex to program • How to guarantee strong resiliency (multiple region failures)? Active-Active challenges
- 25. PRESENTED BY Redis CRDTs: solving the Active-Active challenges • Local <1ms latency <1ms <1ms <1ms
- 26. PRESENTED BY Redis CRDTs: solving the Active-Active challenges • Local <1ms latency • CRDTs-based conflict resolution Lists BitmapsBit field Sorted SetsSets Counters
- 27. PRESENTED BY Redis CRDTs: solving the Active-Active challenges • Local <1ms latency • CRDTs-based conflict resolution • Strong resiliency based on consensus-free protocol Strong eventual consistency
- 28. PRESENTED BY Active-Active comparison App App App App App App Eventual consistency Strong consistency App App App Strong eventual consistency + Causal consistency 100ms 200ms <1ms Cassandra/DynamoDB Spanner Redis CRDTs
- 29. PRESENTED BY 29 99.9% (<10m4.8s) Active-passive replication Single-AZ 99.99% (<4m23s) Active-passive replication Multi-AZ 99.999% (<26.3s) Active-Active replication Multi-region Redis Enterprise cloud SLA
- 30. PRESENTED BY App App App App AKS EKS GKE PKS OpenShift On Premises Redis Enterprise: multi-cloud, on-premises, and hybrid deployments 30
- 31. PRESENTED BY Announcing Active-Active Redis Streams Enables true geo-distributed apps to be developed A large telecommunication provider Globally distributed call-center tasks
- 32. PRESENTED BY Redis core Redis modules Redis Enterprise core Redis Enterprise Hashes Bitmaps Strings Bit Field Streams Hyperloglog Sorted Sets Sets Geospatial Lists Geo-distributionLinear scalability HA Durability Backup & restore Tiered-memory access SecurityMulti-tenant Search Graph TimeSeries AI JSON Gears BloomFilter
- 33. PRESENTED BY Popular and unpopular data models
- 34. PRESENTED BY Modern data models in Redis Streams RediSearch RedisGraph RedisTimeSeries RedisAIRedisJSONRedisBloom
- 35. PRESENTED BY Main capabilities • Full-text search • Indexing, autocomplete • Aggregation, ranking, stemming • Extendable and modular Use cases • Fast search • Multi-tenant indexing RediSearch 35 RediSearch
- 36. PRESENTED BY How fast is RediSearch? 36
- 37. PRESENTED BY Optimizing shipping operations Announcing RediSearch v1.6 37 1 Index aliasing Movie Index_a Index_b 3 RediSearch as a library for other modules 2 Performance improvement
- 38. PRESENTED BY Main capabilities • Cypher queries • Property graph • Graph as sparse-adjacency matrices • Graph operations through linear algebra expressions (GraphBLAS) Use cases • Social graph • Authentication, network topology • SimRank RedisGraph 38 RedisGraph
- 39. PRESENTED BY 4 1 2 5 7 6 3 1 2 3 4 5 6 7 1 2 3 4 5 6 7 RedisGraph: sparse adjacency matrices
- 40. PRESENTED BY 4 1 2 5 7 6 3 1 2 3 4 5 6 7 1 2 3 4 5 6 7 RedisGraph: graph traversal with linear algebra Source Level 1 Level 2
- 41. PRESENTED BY Graph databases benchmark 0.0 500.0 1,000.0 1,500.0 2,000.0 2,500.0 RedisGraph TigerGraph Neo4j Neptune JanusGraph ArangoDB 1.0 16.2 53.8 34.6 69.7 235.6 1.0 30.1 256.3 361.5 493.5 2,093.4 Normalized query response time (lower is better) graph500 twitter 30x 260x 360x 490x 2100x
- 42. PRESENTED BY Announcing RedisGraph v2.0 Resource management and transaction validation • Broad support for Cypher: MERGE, WITH, UNION, filter patterns • Full-text search using RediSearch • GraphBLAS 3.1.1 parallel execution of algebraic operations • Many more...
- 43. PRESENTED BY RedisGraph v2.0 is even faster
- 44. PRESENTED BY RedisTimeSeries Main capabilities • Downsampling/compaction • Indexing, queries, aggregation • Integration (Grafana, Prometheus) Use cases • Monitoring, filtering • IoT RedisTimeSeries 44 Modern and reliable photon tools for an ever-evolving newsroom
- 45. PRESENTED BY Announcing double-delta compression in RedisTimeSeries v1.2 What is double-delta compression? Why is it important?
- 46. PRESENTED BY RedisAI 46 RedisAI Main capabilities • Serving AI close to the data • Support for both DL and ML • Frameworks: PyTorch, Tensorflow, ONNXRuntime, and more to come • Torch scripts Use cases • Fraud detection • Recommendation engine
- 47. PRESENTED BY RedisAI: performance 47 Serving AI far from the data Serving AI with the data N x {x3 - x10} faster
- 48. PRESENTED BY Infinite programmability options: • Acting on keyspace notification • ‘Write behind’ • Stream processing • And many more Hashes Strings Sets RedisGears: serverless engine in Redis
- 50. PRESENTED BY RedisInsight (previously RDBTools): a standard UI for Redis • Developer-friendly UI • Adds insight to existing and new data structures: Streams, Search, JSON, Graph, TimeSeries • Enhanced memory-analysis and recommendation capabilities • Admin monitoring, commands tracer, CLI interface, bulk operations redislabs.com/redisinsight
- 51. PRESENTED BY Redis core Redis modules Redis Enterprise core Redis Enterprise Hashes Bitmaps Strings Bit Field Streams Hyperloglog Sorted Sets Sets Geospatial Lists Tiered-memory access Multi-tenantGeo-distributionLinear scalability HA Durability Backup & restore Security Search Graph TimeSeries AI JSON Gears BloomFilter
- 52. PRESENTED BY Why multi-tenant architecture? 52 • 64 Redis instances • Total memory: 10GB • Total throughput: 100K ops/sec $1.99/hr $3.27/hr $9.98/hr $0.00/hr $2.00/hr $4.00/hr $6.00/hr $8.00/hr $10.00/hr $12.00/hr Redis Enterprise Cloud (2xc5.xlarge + 1xm5.large)) AWS EKS (4xc5.4xlarge) AWS ElastiCache (64xcache.m4.large)
- 53. PRESENTED BY GBs → TBs → PBs in 53
- 54. PRESENTED BY DRAM = 768 GB SSD = 3,072 GB DRAM = 768 GB On DRAM 1M+ ops/sec, $$$ On Flash/SSD 100K+ ops/sec, $ Pay just +15% for 5x memory capacity Running Redis on multiple memory technologies 54
- 55. PRESENTED BY 1TB replicated dataset (2TB total) @ 100K ops/sec <1ms <1ms <10ms <1ms presented by $725,217.00 $507,652 $418,108 $145,745 $0.00 $100,000.00 $200,000.00 $300,000.00 $400,000.00 $500,000.00 $600,000.00 $700,000.00 $800,000.00 ElastiCache OSS Redis DynamoDB Redis Enterprise (RoF)Redis by cloud companies OSS Redis Other NoSQL Redis on Flash 55
- 56. PRESENTED BY DRAM = 768GB SSD = 3,072GB On Flash/SSD 100K+ ops/sec, $ On DRAM 1M+ ops/sec, $$$ DRAM = 768GB On Persistent Memory 1M+ ops/sec, $$ Redis Enterprise on multiple memory technologies PMEM = 3,840GB / 7680GB 56
- 58. PRESENTED BY Forward looking • Redis 6.0 GA before RedisConf (May 12-14) (https://events.redislabs.com/redisconf20/) • New versions of modules • New stuff around AI and Gears • Redis Enterprise 6.0: external identity integration (e.g. LDAP) • New data-type/data-model available on Active-Active • Easy consumption on cloud/multi-cloud/hybrid cloud and Kubernetes
- 59. Thank You!