Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB
•
0 likes•1,958 views
Expedia Group is a multibillion dollar online travel company. Learn why they decided to move from Apache Cassandra to Scylla to further their corporate growth, what they learned in the migration process, and how Scylla improved their operations.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB
Similar to Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB (20)
More from ScyllaDB
More from ScyllaDB (20)
Recently uploaded
Recently uploaded (20)
Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB
- 1. EG’s Migration Journey to Scylla Singaram Ragunathan & Dilip Kolasani © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 2. Singaram Ragunathan & Dilip Kolasani NoSQL Enthusiasts, Expedia Group Singaram is a seasoned data architect passionate about distributed database systems and the design of highly scalable big data models, data pipeline, and application software. He has fun building database infrastructure as code and enabling application teams with the journey to cloud. Dilip is a database specialist with over 12 years of experience in the IT industry. He has a deep understanding of relational, as well as NoSQL databases and specializes in enabling application transitions to NoSQL databases in the cloud. © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 3. We match global travelers with the best options for their needs, and connect travel partners with the widest range of travelers and technology. *As of December 2019* 200+ Travel Booking Sites in 70+ Countries 43% International Revenue Full Year 2019 Over 40% of Transactions on Consumer OTA Brands Booked via Mobile 500+ Airlines, Dozens of Cruise Lines, and 210K+ Unique Activities. 2.1M+ Vrbo Online Bookable Listings 1.6M Properties on Core Lodging Platform $12.1B Revenue 389M Room Nights $ 107.9B In Gross Bookings © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 4. © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 5. Technical Challenge © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 6. Challenges with Apache Cassandra <250MS Garbage Collection Burst Traffic Infrastructure Costs Infrequent Releases © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 7. 3.0.2 3.0.3 3.0.4 3.0.8 3.0.9 3.0.10 3.0.11 2.3.1 2.3.2 2.3.3 3.11.4 3.11.6 Q3 Q4 Q3 Q4Q1 Q2 2019 20202018 Cassandra ScyllaDB Apache Cassandra Vs ScyllaDB Release Timeline © 2020 Expedia Group. All rights reserved. Expedia Group Confidential 3.11.2 3.11.3 3.11.5 Q1 Q3 3.11.7 3.11.8 3.11.9 4.1.1 4.1.2 4.1.3 4.1.4 4.1.5 4.1.6 4.1.7 4.1.8 4.1.9 3.0.5 3.0.6 3.0.7 Q2
- 8. Code Compatibility Majority of use cases required no code changes from C* Ease of migration Infra provisioning framework and concepts remain unchanged Total cost of ownership Can do more with less $$ Roadmap ScyllaDB has a well-defined roadmap Vendor Support Truly impressive turnaround time via Scylla community slack channel Why ScyllaDB? 8© 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 9. Initial Use Case Geo RealTime Geo Content Geo tools • Web service with REST/Json APIs • Access geography data • Perform geography-based selection • Other geospatial functionality • Geography content and related data are • created • aggregated • managed • Admin Tools • Part of the back-office system © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 10. Infrastructure - Apache Cassandra vs Scylla Current Apache Cassandra Cluster ScyllaDB Cluster • 24 Nodes • 25 TB • I3EN.2XL AWS INSTANCE • Cassandra 3.0.9 • 24 Nodes • 25 TB • I3.2XL AWS INSTANCE • ScyllaDB 4.1.4 © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 11. Performance - Apache Cassandra Vs ScyllaDB © 2020 Expedia Group. All rights reserved. Expedia Group Confidential Throughput Read/Write
- 12. ScyllaDB Benchmark Results Throughput P99 Read Performance Infrastructure Savings 3x <5ms 35% © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 13. Cerebro – Internal NoSQL Automation Framework © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 14. Next Steps • L1 Cache – Redis • L2 Cache – ScyllaDB • ScyllaDB supports Redis API support – Possible replacement of L1 Cache • Evaluate Scylla Alternator as opensource replacement to DynamoDB © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 15. Considerations 1 2 3 4 Logs are piped to syslog Ease of use with Change Data Capture (CDC) Cluster scale out/in are resumable Support for large partitions © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 16. Did you know we’re hiring? Join our global team at Expedia Group! Data, AI, Platforms & Systems is looking for tech rock stars to join our team. Check out the EG Careers Site for more details. (lifeatexpediagroup.com) © 2020 Expedia Group. All rights reserved. Expedia Group Confidential
- 17. YouThank Singaram Ragunathan & Dilip Kolasani © 2020 Expedia, Inc., an Expedia Group company. All rights reserved. Trademarks and logos are property of their respective owners. https://www.linkedin.com/in/singaram-ragunathan-2a11201b/ https://www.linkedin.com/in/dilip-kolasani-13308526/
Editor's Notes
- Hello everyone.....It gives us immense pleasure in sharing our journey towards ScyllaDB, during this ScyllaSummit 2021. As part of this presentation, would take you all through who are we, what we do, why we choose ScyllaDB and finally the outcome.
- I'm Singa, I will be joined by Dilip copresentor for this talk. We are both passionate database engineers at Expedia Group, working with multiple NoSQL technologies and strive to align use cases that makes the best out of the underlying datastore.
- Expedia Group, Inc. is one of world’s largest travel platforms. At ExpediaGroup – Our mission is to bring the world within reach. We firmly believe that travel has the power to change lives!
- We do that through the power of our brands
- Alright lets get into the nitty gritty of why we picked ScyllaDB and how it helped our developer journey. Currently at EG, there are multiple applications built on top of Apache cassandra, which comes with its own set of challenges. We will be going through some of them, throughout this deck.
- Apache Cassandra written in Java, brings in the onus of managing GC & making sure its appropriately tuned for the workload in hand. Though GC is tunable, it takes significant amount of time and effort as well as expertise required to handle/tune GC pause for every specific use case. With burst traffic or sudden peak in the workload, there is significant disturbance to the P99 response time. So, we end up adding buffer nodes to handle this peak capacity, which results in more infrastructure costs. Another significant worry is, based on past 4 years history the number of Apache Cassandra releases has significantly slowed down.
- We would like to compare the open source commits in Cassandra vs ScyllaDB here and highlight the amount of releases that Scylla has gone through the same past 3 year period. As you can see, it gives enough confidence towards ScyllaDB that given an issue/bug with a specific release, it will be soon addressed with a patch. In contrast with Apache Cassandra one might have to wait longer.
- So why did we end up with ScyllaDB? From an Apache Cassandra codebase, its frictionless for developers to switch over to ScyllaDB. For the use cases that we tried, there wasn’t any data model changes necessary and the scylladb driver was entirely compatible and a swap in replacement with Cassandra driver dependency. With a few tweaks to our automation framework that provisions Apache Cassandra cluster, we were able to provision ScyllaDB open source cluster. Thanks to C++ backend of ScyllaDB, we no longer have to worry about stop the world GC pauses. Also we were able to store more data per node and achieve more throughput per node, thereby saving significant $$$ for company. Clear roadmap and support from ScyllaDB slack community comes in very handy.
- The candidate application chosen for this POC, is our geo system that provides information about geographical entities and the relationships between them. It aggregates data from multiple systems like hotel location info, 3rd party data , etc. This rich geography dataset enables different types of data searches, using a simple REST API while guarantying single digit msec P99 read response time. To speed up API responses, we are using multi layered cache with redis as first layer and cassandra as second level. With ScyllaDB as a swap in replacement for Cassandra, I’m handing it over to Dilip for going over the infra setup, benchmark results and next steps.
- Thank you Singa. Our POC cluster in ScyllaDB were to store around 25TB of data exactly like our existing PROD Cassandra cluster. To begin with we provisioned same total number of instances between Cassandra and ScyllaDB but the instance type chosen was I3.2XL which is 35% cheaper than I3EN.2XL.
- The use case demands a high read throughput, while tiny write throughput. As shown in the first graph whether its ScyllaDB or Cassandra the writes are almost negligible or flat line at bottom. While the real winner is on Reads where the Cassandra P99 throughput is flaky as shown by the spikes, while the ScyllaDB P99 read response times are relatively flat. This is of significant advantage to our read heavy application. In terms of throughput comparison as shown in second graph, we were able to push almost double the TPS with ScyllaDB when compared with Cassandra, especially with a flat P99 SLA.
- Here are some of the facts that made ScyllaDB benchmark stand out. We were able to get triple the throughput with flat single digit P99 read response times, at the same time achieve over 35% reduction in total cost of ownership. At this point it was a no brainer to switch towards ScyllaDB for this application production workload.
- Huge shoutout to our automation team which made the provisioning of ScyllaDB cluster a breeze., made possible via our internal tool called Cerebro. We use this same internal tool for managing over 7 different NoSQL technologies with the aim of enabling our application teams to focus on bringing great products to the market, without having to worry about managing databases.
- This application in hand currently uses L1 cache (Redis) before hitting this backend persistent store ScyllaDB. With the advantage of ScyllaDB supporting Redis compatible API and proven P99 improves to be under single digit msec, we are thinking about turning off the in memory cache engine and rely completely on ScyllaDB as only database backend for the application. This will bring in significant additional cost advantage both in terms of infrastructure and application code. Also we recently learned about Scylla Alternator and are currently evaluating if it’s a viable alternative to DynamoDB as advertised.
- Logs are being pushed to syslog and there isn’t a configuration to route them to a customer folder of your choice. The CDC functionality is significantly better compared to Apache Cassandra, so this might entice applications that rely on change streams. A good thing about ScyllaDB node replacements either during scale up or scale down are resumable. Please pay caution while using large partition, the performance might vary depending on how large the partitions are.
- If you are interested in what you heard and want to build great products with us....Please join hands
- Thanks for this opportunity to present to ScyllaDB enthuasists all over the world. We enjoyed every moment of putting this together and hope you did too.