Couchbase_UK_2013_Amadeus_and_Couchbase
 

Couchbase_UK_2013_Amadeus_and_Couchbase

on

  • 1,785 views

 

Statistics

Views

Total Views
1,785
Slideshare-icon Views on SlideShare
684
Embed Views
1,101

Actions

Likes
2
Downloads
25
Comments
0

2 Embeds 1,101

http://www.couchbase.com 1100
https://www.google.fr 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Couchbase_UK_2013_Amadeus_and_Couchbase Couchbase_UK_2013_Amadeus_and_Couchbase Presentation Transcript

    • Amadeus & Couchbase First steps together Jeremy Meyer © 2013 Amadeus IT Group SA Amadeus Data Management R&D Couchbase [UK] Conference London, 11th of April 20131 Confidential Information – Couchbase [UK] conference 2013
    • Amadeus mission To be the leading provider of IT solutions that enable success in the travel and tourism industry © 2013 Amadeus IT Group SA2 Confidential Information – Couchbase [UK] conference 2013
    • Global Distribution System Airlines Travel agencies Global Distribution Car companies System Corporations (B2B) Hotel chains Provider sites (B2C) Rail companies Airports © 2013 Amadeus IT Group SA Providers Distributors3 Confidential Information – Couchbase [UK] conference 2013
    • Operations 3.7M 1.6MM 45MM Bookings/day Transactions/day DB accesses/day 13PB <0.5s 100+ Of storage Of response time IT changes/day © 2013 Amadeus IT Group SA No service outage permitted4 Confidential Information – Couchbase [UK] conference 2013
    • Amadeus & Couchbase first steps Technical Context Big (!) yet agile system Continuously undertaking several parallel migrations Thousands of R&D software developers pushing new features NoSQL/NewSQL solutions are welcome Massive memcached deployments in Production since 2009 RDBMs used in star-schema mode, still need to scale further Wishing to get flexibility on ACID and A/CP properties © 2013 Amadeus IT Group SA Testing Couchbase: from what we started? General enthusiasm from R&D community Aim to deeply understand it and to push it to the limits Awareness that there is no silver bullet5 Confidential Information – Couchbase [UK] conference 2013
    • Agenda 1 Our architecture 2 Our motivations to test Couchbase 3 Two use cases: Amadeus Selling Platform Connect and Amadeus Availability Processing Engine 4 Wrap-up © 2013 Amadeus IT Group SA6 Confidential Information – Couchbase [UK] conference 2013
    • Service oriented architecture (SOA) External Systems Client Services Distribution Channels Airline out of scope Systems ATO/CTO, Travel Agencies , GDS, Direct Channel Travel Revenue management , Providers accounting , CRM ,data warehouse, … Internet GUI Green Screen EDIFACT/XML Interface Services AMADEUS PLATFORM Services Integrator Altéa Altéa Shared Servers Altea DCS Inventory Reservation Application Services Print Services Logon Business Customer Seating Central Server Rules Management © 2013 Amadeus IT Group SA Fares N Ticketing o Fare Document Baggage Schedule n Servers Server Handling Schedules Availability Reference - E-Ticket Customer Data A Server Server PNR …… i Flight Inventory r SBR Management7 Confidential Information – Couchbase [UK] conference 2013
    • Our interests in Couchbase A low-latency key/value store Performances Sub-millisecond latency Cross data center replication (XDCR) Up to 50 data centers Elasticity Frequent farm capacity extension Data persistence: boot with data, backup abilities Huge write traffic © 2013 Amadeus IT Group SA Open-source & Memcached proximity A company as a provider to rely on: support, training, quality assurance of the product, functional evolution8 Confidential Information – Couchbase [UK] conference 2013
    • The CAP theorem Consistency vs Availability Strong consistency mandatory on some of our applications © 2013 Amadeus IT Group SA Total availability mandatory on some others9 Confidential Information – Couchbase [UK] conference 2013
    • Two of our use cases Selling Platform Connect & Availability Processing Engine © 2013 Amadeus IT Group SA10 Confidential Information – Couchbase [UK] conference 2013
    • © 2013 Amadeus IT Group SA Confidential Information – Couchbase [UK]11 conference 2013
    • Amadeus Selling Platform Connect 500 000 terminals 150 000 simultaneous active users User Sessions: 500 GB of data Quality for professionals © 2013 Amadeus IT Group SA12k Confidential Information – Couchbase [UK] conference 2013
    • Amadeus Selling Platform Connect Motivations to rely on a distributed key/value store Offloading user sessions stored in the JVMs to a distributed scalable and robust system Reason for going to distributed store for the user session: Total Cost Ownership (memory footprint), and scalability SLA: user experience should be as good as with the user sessions part of the Java EE process © 2013 Amadeus IT Group SA13 Confidential Information – Couchbase [UK] conference 2013
    • Amadeus Selling Platform Connect Tests Simulation of real users Usage of Java Smart Client Application specificity: affinity per node © 2013 Amadeus IT Group SA14 Confidential Information – Couchbase [UK] conference 2013
    • Amadeus Selling Platform Connect Tests Test of performances: OK Couchbase can be an appropriate solution Network/CPU consumption: OK for Amadeus Selling Platform Connect will allow to sustain same traffic with 3 times Operational tests less Java EE nodes Add cache node: OK Stop/start Java EE: OK Stop/start cache node: manual action needed when more than one node is down to perform failover © 2013 Amadeus IT Group SA time needed for failover to happen too long (30s) => usage of node affinity to improve availability15 Confidential Information – Couchbase [UK] conference 2013
    • Availability © 2013 Amadeus IT Group SA Processing Engine16 Confidential Information – Couchbase [UK] conference 2013
    • Looks familiar? © 2013 Amadeus IT Group SA Powered by Amadeus Availability Processing Engine17 Confidential Information – Couchbase [UK] conference 2013
    • Architecture overview PRICING & SHOPPING PLATFORM AVAILABILITY CALCULATION FLIGHT SELECTION AVAILABILITY DISTRIBUTION COMPLEX 28 RDBMS JOURNEY SERVER AVAILABILITY PROCESSING ENGINE 20TB Schedules 1.9M get/s Polling Avl Mgmt rules Cache 400K set/s Selection rules Objects of 1KB DYNAMIC AVAILABILITY DYNAMIC AVAILABILITY ALTEA INVENTORY PLATFORM © 2013 Amadeus IT Group SA Distribution gateways AVAILABILITY COMPUTATION Inventory dataDynamic Direct Access Ticket less RailAvailability Airlines Access AirlinesAirlines 18 Confidential Information – Couchbase [UK] conference 2013
    • APE test configuration © 2013 Amadeus IT Group SA 7 Application Servers (Prod: 350) 5 Couchbase nodes: 1TB RAM, PCI-Express SSD storage cards Mirroring of live production traffic No code change, usage of memcached protocol to target Moxi19 Confidential Information – Couchbase [UK] conference 2013
    • APE throughput test 9% of Prod traffic on a single node, 142k get/s, 33k set/s Thread at 100% CPU © 2013 Amadeus IT Group SA20 Confidential Information – Couchbase [UK] conference 2013
    • APE data rebalancing testNormal7MB/sSmart7MB/s © 2013 Amadeus IT Group SA40h for1TB21 Confidential Information – Couchbase [UK] conference 2013
    • APE cold boot test 5 mins of outage 15 mins of service © 2013 Amadeus IT Group SA degradation22 Confidential Information – Couchbase [UK] conference 2013
    • APE backup and restore test ~30 hours for 1TB 21K docs/s ( vs 1MM docs per node ~ © 2013 Amadeus IT Group SA 12 hours )23 Confidential Information – Couchbase [UK] conference 2013
    • APE document fragmentation and compaction test © 2013 Amadeus IT Group SA24 Confidential Information – Couchbase [UK] conference 2013
    • APE cross data center replication test 4K doc/s ( vs 400K sets/s ) © 2013 Amadeus IT Group SA25 Confidential Information – Couchbase [UK] conference 2013
    • APE tests outcome Couchbase technology looks appropriate solution for APE requirements, and would allow to decommission the 28 RDBMS Before going to Production, follow-up ongoing on a couple of to-be-improved points Main ones: Multithreading of persistency activities, Read replica Right © 2013 Amadeus IT Group SA High expectations on 2.0.2 / 2.1 !26 Confidential Information – Couchbase [UK] conference 2013
    • Wrap-up © 2013 Amadeus IT Group SA27 Confidential Information – Couchbase [UK] conference 2013
    • Global compliance with demanding expectations Product Performances Stability Feature set Care for enterprise matters i.e Migrations, API stability Support © 2013 Amadeus IT Group SA Involved and skilled for handling immediate matters during the test period (Thanks Perry) For the information on upcoming needed features/changes28 Confidential Information – Couchbase [UK] conference 2013
    • Expectations on Couchbase evolutions Core features Tuning of high availability/consistency, multithreading of queues consumption, usage of PCI-E SSD direct access Management features Firecell concept, tunable routing policy for XDCR, several processes per host, support of heterogeneous farms,ability to not invalidate data from a failed-over node © 2013 Amadeus IT Group SA Docs Process/thread models, how data is stored on disk, incident management and troubleshooting guides29 Confidential Information – Couchbase [UK] conference 2013
    • Amadeus & Couchbase first steps together Conclusion We have stringent data management needs within a demanding industry We assessed Couchbase for a part of our applications having a key/value need, and tested critical ones within an 8 week period Couchbase looked solid during the preliminary tests Couchbase looks good candidate to support our business starting version 2.0.2/2.1 © 2013 Amadeus IT Group SA30 Confidential Information – Couchbase [UK] conference 2013
    • © 2013 Amadeus IT Group SA31 Confidential Information – Couchbase [UK] conference 2013