Couchbase_UK_2013_Amadeus_and_Couchbase

2,700 views
2,513 views

Published on

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,700
On SlideShare
0
From Embeds
0
Number of Embeds
1,529
Actions
Shares
0
Downloads
47
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Couchbase_UK_2013_Amadeus_and_Couchbase

  1. 1. Amadeus & Couchbase First steps together Jeremy Meyer © 2013 Amadeus IT Group SA Amadeus Data Management R&D Couchbase [UK] Conference London, 11th of April 20131 Confidential Information – Couchbase [UK] conference 2013
  2. 2. Amadeus mission To be the leading provider of IT solutions that enable success in the travel and tourism industry © 2013 Amadeus IT Group SA2 Confidential Information – Couchbase [UK] conference 2013
  3. 3. Global Distribution System Airlines Travel agencies Global Distribution Car companies System Corporations (B2B) Hotel chains Provider sites (B2C) Rail companies Airports © 2013 Amadeus IT Group SA Providers Distributors3 Confidential Information – Couchbase [UK] conference 2013
  4. 4. Operations 3.7M 1.6MM 45MM Bookings/day Transactions/day DB accesses/day 13PB <0.5s 100+ Of storage Of response time IT changes/day © 2013 Amadeus IT Group SA No service outage permitted4 Confidential Information – Couchbase [UK] conference 2013
  5. 5. Amadeus & Couchbase first steps Technical Context Big (!) yet agile system Continuously undertaking several parallel migrations Thousands of R&D software developers pushing new features NoSQL/NewSQL solutions are welcome Massive memcached deployments in Production since 2009 RDBMs used in star-schema mode, still need to scale further Wishing to get flexibility on ACID and A/CP properties © 2013 Amadeus IT Group SA Testing Couchbase: from what we started? General enthusiasm from R&D community Aim to deeply understand it and to push it to the limits Awareness that there is no silver bullet5 Confidential Information – Couchbase [UK] conference 2013
  6. 6. Agenda 1 Our architecture 2 Our motivations to test Couchbase 3 Two use cases: Amadeus Selling Platform Connect and Amadeus Availability Processing Engine 4 Wrap-up © 2013 Amadeus IT Group SA6 Confidential Information – Couchbase [UK] conference 2013
  7. 7. Service oriented architecture (SOA) External Systems Client Services Distribution Channels Airline out of scope Systems ATO/CTO, Travel Agencies , GDS, Direct Channel Travel Revenue management , Providers accounting , CRM ,data warehouse, … Internet GUI Green Screen EDIFACT/XML Interface Services AMADEUS PLATFORM Services Integrator Altéa Altéa Shared Servers Altea DCS Inventory Reservation Application Services Print Services Logon Business Customer Seating Central Server Rules Management © 2013 Amadeus IT Group SA Fares N Ticketing o Fare Document Baggage Schedule n Servers Server Handling Schedules Availability Reference - E-Ticket Customer Data A Server Server PNR …… i Flight Inventory r SBR Management7 Confidential Information – Couchbase [UK] conference 2013
  8. 8. Our interests in Couchbase A low-latency key/value store Performances Sub-millisecond latency Cross data center replication (XDCR) Up to 50 data centers Elasticity Frequent farm capacity extension Data persistence: boot with data, backup abilities Huge write traffic © 2013 Amadeus IT Group SA Open-source & Memcached proximity A company as a provider to rely on: support, training, quality assurance of the product, functional evolution8 Confidential Information – Couchbase [UK] conference 2013
  9. 9. The CAP theorem Consistency vs Availability Strong consistency mandatory on some of our applications © 2013 Amadeus IT Group SA Total availability mandatory on some others9 Confidential Information – Couchbase [UK] conference 2013
  10. 10. Two of our use cases Selling Platform Connect & Availability Processing Engine © 2013 Amadeus IT Group SA10 Confidential Information – Couchbase [UK] conference 2013
  11. 11. © 2013 Amadeus IT Group SA Confidential Information – Couchbase [UK]11 conference 2013
  12. 12. Amadeus Selling Platform Connect 500 000 terminals 150 000 simultaneous active users User Sessions: 500 GB of data Quality for professionals © 2013 Amadeus IT Group SA12k Confidential Information – Couchbase [UK] conference 2013
  13. 13. Amadeus Selling Platform Connect Motivations to rely on a distributed key/value store Offloading user sessions stored in the JVMs to a distributed scalable and robust system Reason for going to distributed store for the user session: Total Cost Ownership (memory footprint), and scalability SLA: user experience should be as good as with the user sessions part of the Java EE process © 2013 Amadeus IT Group SA13 Confidential Information – Couchbase [UK] conference 2013
  14. 14. Amadeus Selling Platform Connect Tests Simulation of real users Usage of Java Smart Client Application specificity: affinity per node © 2013 Amadeus IT Group SA14 Confidential Information – Couchbase [UK] conference 2013
  15. 15. Amadeus Selling Platform Connect Tests Test of performances: OK Couchbase can be an appropriate solution Network/CPU consumption: OK for Amadeus Selling Platform Connect will allow to sustain same traffic with 3 times Operational tests less Java EE nodes Add cache node: OK Stop/start Java EE: OK Stop/start cache node: manual action needed when more than one node is down to perform failover © 2013 Amadeus IT Group SA time needed for failover to happen too long (30s) => usage of node affinity to improve availability15 Confidential Information – Couchbase [UK] conference 2013
  16. 16. Availability © 2013 Amadeus IT Group SA Processing Engine16 Confidential Information – Couchbase [UK] conference 2013
  17. 17. Looks familiar? © 2013 Amadeus IT Group SA Powered by Amadeus Availability Processing Engine17 Confidential Information – Couchbase [UK] conference 2013
  18. 18. Architecture overview PRICING & SHOPPING PLATFORM AVAILABILITY CALCULATION FLIGHT SELECTION AVAILABILITY DISTRIBUTION COMPLEX 28 RDBMS JOURNEY SERVER AVAILABILITY PROCESSING ENGINE 20TB Schedules 1.9M get/s Polling Avl Mgmt rules Cache 400K set/s Selection rules Objects of 1KB DYNAMIC AVAILABILITY DYNAMIC AVAILABILITY ALTEA INVENTORY PLATFORM © 2013 Amadeus IT Group SA Distribution gateways AVAILABILITY COMPUTATION Inventory dataDynamic Direct Access Ticket less RailAvailability Airlines Access AirlinesAirlines 18 Confidential Information – Couchbase [UK] conference 2013
  19. 19. APE test configuration © 2013 Amadeus IT Group SA 7 Application Servers (Prod: 350) 5 Couchbase nodes: 1TB RAM, PCI-Express SSD storage cards Mirroring of live production traffic No code change, usage of memcached protocol to target Moxi19 Confidential Information – Couchbase [UK] conference 2013
  20. 20. APE throughput test 9% of Prod traffic on a single node, 142k get/s, 33k set/s Thread at 100% CPU © 2013 Amadeus IT Group SA20 Confidential Information – Couchbase [UK] conference 2013
  21. 21. APE data rebalancing testNormal7MB/sSmart7MB/s © 2013 Amadeus IT Group SA40h for1TB21 Confidential Information – Couchbase [UK] conference 2013
  22. 22. APE cold boot test 5 mins of outage 15 mins of service © 2013 Amadeus IT Group SA degradation22 Confidential Information – Couchbase [UK] conference 2013
  23. 23. APE backup and restore test ~30 hours for 1TB 21K docs/s ( vs 1MM docs per node ~ © 2013 Amadeus IT Group SA 12 hours )23 Confidential Information – Couchbase [UK] conference 2013
  24. 24. APE document fragmentation and compaction test © 2013 Amadeus IT Group SA24 Confidential Information – Couchbase [UK] conference 2013
  25. 25. APE cross data center replication test 4K doc/s ( vs 400K sets/s ) © 2013 Amadeus IT Group SA25 Confidential Information – Couchbase [UK] conference 2013
  26. 26. APE tests outcome Couchbase technology looks appropriate solution for APE requirements, and would allow to decommission the 28 RDBMS Before going to Production, follow-up ongoing on a couple of to-be-improved points Main ones: Multithreading of persistency activities, Read replica Right © 2013 Amadeus IT Group SA High expectations on 2.0.2 / 2.1 !26 Confidential Information – Couchbase [UK] conference 2013
  27. 27. Wrap-up © 2013 Amadeus IT Group SA27 Confidential Information – Couchbase [UK] conference 2013
  28. 28. Global compliance with demanding expectations Product Performances Stability Feature set Care for enterprise matters i.e Migrations, API stability Support © 2013 Amadeus IT Group SA Involved and skilled for handling immediate matters during the test period (Thanks Perry) For the information on upcoming needed features/changes28 Confidential Information – Couchbase [UK] conference 2013
  29. 29. Expectations on Couchbase evolutions Core features Tuning of high availability/consistency, multithreading of queues consumption, usage of PCI-E SSD direct access Management features Firecell concept, tunable routing policy for XDCR, several processes per host, support of heterogeneous farms,ability to not invalidate data from a failed-over node © 2013 Amadeus IT Group SA Docs Process/thread models, how data is stored on disk, incident management and troubleshooting guides29 Confidential Information – Couchbase [UK] conference 2013
  30. 30. Amadeus & Couchbase first steps together Conclusion We have stringent data management needs within a demanding industry We assessed Couchbase for a part of our applications having a key/value need, and tested critical ones within an 8 week period Couchbase looked solid during the preliminary tests Couchbase looks good candidate to support our business starting version 2.0.2/2.1 © 2013 Amadeus IT Group SA30 Confidential Information – Couchbase [UK] conference 2013
  31. 31. © 2013 Amadeus IT Group SA31 Confidential Information – Couchbase [UK] conference 2013

×