Couchbase_UK_2013_Skyscanner_and_Couchbase

1,361 views
1,299 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,361
On SlideShare
0
From Embeds
0
Number of Embeds
714
Actions
Shares
0
Downloads
30
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Couchbase_UK_2013_Skyscanner_and_Couchbase

  1. 1. 25 April, 2013Couchbase at Skyscanner
  2. 2. Who areSkyscanner?
  3. 3. Edinburgh
  4. 4. website appstraditionalmobile siteOTAsbudgetbudget
  5. 5. How doesSkyscannerwork?
  6. 6. budgetLive PricingServiceBrowseCacheService
  7. 7. A route-date specific search
  8. 8. The end result
  9. 9. Powering a live pricing searchPrices +TimetablesData CollectionServices1) Which websitesshould we show?2) What prices dowe already havecached?3) Live updatewhat we stillneed. 4) Clean up andsave the new data5) Return theprices to the user.Live PricingService
  10. 10. Scaling thelive pricingcache
  11. 11. The original solutionPrice CacheLive PricingService
  12. 12. Solution 1 – Split up Reads and WritesRead Only Cache, writing queues of new dataQuote ProcessorsReplicationLive PricingService
  13. 13. Solution 2 – A temporary local cacheLive PricingServiceCached Prices (key/value)
  14. 14. At this scale2 bn itineraries and quotes270 GB table250 GB indices2000 quotes per second
  15. 15. What we really neededConsistencyPersistenceSpeedResilienceHorizontal ScalingElasticitySimplicity
  16. 16. Solution 3 – Introduce CouchbaseLive PricingServiceLegacy read-only cacheLegacy quote Processors
  17. 17. The Documentsstruct TransportQuoteSet {1: string MarketingAgentId,2: common.PassengerGroup PassengerGroup,3: i32 OutboundDate,4: bool OneWay,5: optional i32 InboundDate,6: common.Place OriginPlace,7: common.Place DestinationPlace,8: common.Combinability Combinability,9: bool Comprehensive10: list<TransportQuote> Quotes,11: list<ChildTransportQuoteSet>ChildTransportQuoteSets,12: optional i32 UtcQuoteDateTime13: optional bool Additional14: optional string CabinClass}
  18. 18. The actual cluster
  19. 19. The buckets
  20. 20. The quotes bucket
  21. 21. Some specifics on the setupDedicated 6 servers:SSDs1 replica of dataTaking nodes in and out of the clusterPerformance with multiple bucketsMonitoring and reporting in one place2 x Quad core Intel i7 2.0ghz256GB of 1600mhz RAMDell R620 1U.Centos 6.23 x 600gb MLC SSD each(Raid1 + Hot spare)
  22. 22. Going beyonda cache.
  23. 23. Running a car hire searchPrices + ReferenceDataData CollectionServices1) Which websitesshould we show?2) What prices dowe already havecached?3) Live updatewhat we stillneed. 4) Save and enrichthe new data5) Return theprices to the user.Car Hire LivePricingService
  24. 24. Reference DataSupplier-LocationMap reduce viewspython Couchbase 2.0 client libCar Codes {"website": {"published": true,"id": "affd", ...},"office_id": "1","city_id": "AUHA","raw_data": [...]"address":"closing_time": "00:00","routenodeid": "9618","type": "office“}
  25. 25. Price DataPython Smart Client Isn’t Ready, so…
  26. 26. Cluster Info3 nodesVirtualBenched to1000+ ops/secSASLAuth.
  27. 27. What’s next?
  28. 28. A flexible search
  29. 29. The results.
  30. 30. What hasCouchbasebrought us?
  31. 31. quartermile one15 lauriston placeedinburgh eh3 9entel: +44 (0)131 252 5353fax: +44 (0)131 252 5354twitter: @skyscannerfacebook: skyscannerThank you forlistening.Alistair Hannalistair.hann@skyscanner.net

×