Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

RedisConf18 - Scaling Whitepages With Redison Flash

179 views

Published on

Breakout Session

Published in: Technology
  • Be the first to comment

  • Be the first to like this

RedisConf18 - Scaling Whitepages With Redison Flash

  1. 1. Scaling Whitepages With Redis on Flash Heather Wade, Varun Kumar, & Jason Frazier April 25, 2018
  2. 2. Whitepages Overview Global Identity Graph Benchmarking Redis on Flash Key Learnings Agenda
  3. 3. 3 Whitepages // Confidential Two business units, two assets. 50,000,000+ Unique Monthly Visitors 5 billion+ data elements creating 2 billion+ new linkages per month. 2,000+ enterprise customers 250,000+ users
  4. 4. 4 Whitepages // Confidential AP I Web Our B2B solution: Global identity verification data for fraud prevention in the ecommerce and lending industries.
  5. 5. 5WHITEPAGES PRO / pro.whitepages.com / 5WHITEPAGES PRO / pro.whitepages.com / Identity verification market approach Traditional lending Online lending Global payments Travel and hospitality eCommerce Platforms partners Decisioning partners
  6. 6. 6 Whitepages // Confidential Direct-to-consumer background information lookup app. Fast, accurate contact data subscription. Optimized for mobile users. Background report data for more detailed identity insights.
  7. 7. Whitepages Overview Global Identity Graph Benchmarking Redis on Flash Key Learnings Agenda
  8. 8. 9 Whitepages // Confidential Person Business PhoneEmail Address 80 global identity records a month 2B A living local graph Whitepages Identity Graph™
  9. 9. 10 Whitepages // Confidential Global Identity Graph 5B entities 18B links 3 TB data 200,000 requests/sec
  10. 10. 11 Whitepages // Confidential Data Flow API Result Subgraph IGraph Retrieval Service Search Service
  11. 11. 12 Whitepages // Confidential Key Requirements Persistent store, not a cache Latency: Subgraph retrieval latency < 50ms, Data Store < 1ms Throughput: 200K requests/s Stability: Consistent latency, 99.99% uptime HA: Multi AZ, and replica with failover
  12. 12. 13 Whitepages // Confidential IGraph Technology Evolution Start the shop! Simple & stableNeed to grow Growth explosion Whitepages® Whitepages, Inc. All other logos, trademarks and product names are the property of their respective owners, and are not associated or affiliated with Whitepages.
  13. 13. 14 Whitepages // Confidential Flat Storage & Shape Language
  14. 14. Whitepages Overview Global Identity Graph Benchmarking Redis on Flash Key Learnings Agenda
  15. 15. 16 Whitepages // Confidential Cassandra & MongoDB PROS • Great scalability • Great support CONS • Latency Problems • Not well suited for our use case • Learning curve Cassandra Benchmark MongoDB Benchmark *These results are based only on Whitepages’s internal tests and will vary depending on system architecture, type and quantity of data processed, etc. Whitepages does not intend to make any representations about the overall quality of the products mentioned or their suitability to your business.
  16. 16. 17 Whitepages // Confidential Couchbase PROS • Promising latencies • Great support • Good fit for our use case CONS • Minimum RAM requirement • Latency problems at scale • Cost Couchbase Benchmark *These results are based only on Whitepages’s internal tests and will vary depending on system architecture, type and quantity of data processed, etc. Whitepages does not intend to make any representations about the overall quality of the products mentioned or their suitability to your business.
  17. 17. 18 Whitepages // Confidential Elasticache & Self-Hosted Redis CONS • Cost issues • Management • Scale issues PROS • Great performance • Familiar Technology *These results are based only on Whitepages’s internal tests and will vary depending on system architecture, type and quantity of data processed, etc. Whitepages does not intend to make any representations about the overall quality of the products mentioned or their suitability to your business.
  18. 18. Whitepages Overview Global Identity Graph Benchmarking Redis on Flash Key Learnings Agenda
  19. 19. 20 Whitepages // Confidential Key Features Redis on RAM + SSD • Flash storage – NVMe volumes provide highly optimized I/O with RAM-like latencies – Better utilization of node resources • Low cost – Flash storage is much cheaper than RAM • Cut our cost per GB of data by 60% – Number of nodes needed decreased by 40% – Required no change to existing infrastructure
  20. 20. 21 Whitepages // Confidential Why we chose Redis on Flash • Easy integration – Able to put into production right away • Fully managed solution – Required no extra training – Great 24/7 support • Unchanged latencies – ~0.1ms latency during peak traffic
  21. 21. Whitepages Overview Global Identity Graph Benchmarking Redis on Flash Key Learnings Agenda
  22. 22. 23 Whitepages // Confidential Redis (ElastiCache _ vs _Labs _) All Active Nodes Master Replica Master Replica
  23. 23. 24 Whitepages // Confidential System and Optimizations Clustered Deployment with 20 i3.4xlarge node cluster with 1 replica Client: lettuce clustered client Compression : Zstd with Custom Dictionary CPU Utilization: shards
  24. 24. 25 Whitepages // Confidential Redis Labs Dashboard
  25. 25. Questions?

×