Flight Centre Limited

How MongoDB has empowered the business
  to rapidly respond to market conditions

               Michael Frost
            Web Solution Architect
Flight Centre
   Flight Centre Limited is one of the world's largest travel agency
   groups, with more than 2000 leisure, corporate and wholesale
                       businesses in 11 countries.
     After starting with one shop 30 years ago, we have enjoyed
                      remarkable ongoing growth.

Our rapidly expanding network now extends throughout Australia, New
  Zealand, the United States, Canada, the United Kingdom, South
  Africa, Hong Kong, India, China, Singapore and the United Arab
        Emirates, providing travellers with a complete service.
Flight Centre Online
• 120 Online staff (Web Developers, SEO, Copy Writers,
  SEM, Application Developers, Online Marketers,
  Creative designers and Usability experts)
• 60 Web sites over 6 different countries
• Up to 200,000 pages on a site
• Larger sites have 450,000 views per day
• Multiple booking engines Air, Hotel, Sea, Insurance, Car
• Two data centres for web hosting. Physical in Brisbane
  and Amazon EC2
Brands Implemented
Business Problem
• Product feeds for lead-in pricing (15 feeds
  and growing). 750,000 records
• Content feeds (Hotel, Ship, Cruise Line,
  IATA, Destination). 550,000 records
  – Every feed has unique model
• Need to override content
  – Each brand in each country is a different
    business
• Need to be responsive to market. E.g. new
  cruise ship, or cruise ship sinks
Business Problem
• Content has different models, e.g. cruise
  ship content compared with hotel
• Business constantly demand new model
  types or changes to existing models
  – Hotel
• Majority developers are front end web
Solution Architecture Diagram
FCL MongoDB Environment
• FCL data centre
  RHEL6 64Bit
  2 X CPU
  7G RAM
  100G data partition

• Amazon servers
  c1.xlarge Amazon EC2 instances

• MongoDB V2.0.3

• Replication with one master, and 2 slaves in each datacentre. New
  slaves can be quickly brought up if demand requires

• 5 databases with up to 55 collections in a database
• ~ 600,000 MongoDB requests per day
• 2 mil records
Hosting environment
with Amazon data centre


•Akamai geo-load balanced
data centres
•No single point of failure for
hosting
•Each data centre is
designed to take full load if
one data centre goes down
•Each layer in each stack
can be extended quickly if
unexpected massive load
came through
What we found with MongoDB
• Developers do not need to worry about
  schemas
• Business can come to developers with new
  feed, content type, product, to be loaded in
  – E.g. Cruise Ship, Cruise Line, IATA location data,
    Hotel content, Hotel imaging
  – Define own domain models
• Native way of thinking with Web Dev
  (JSON/REST)
• Powers the majority of our sites
What we found with MongoDB
• Arbiter is only used to provide extra votes
  that the master we want stays as master
• If master goes down slaves will continue
  to allow reads. Writes will not work
  – Writes are not mission critical – Competitions
    and the like
• SLAVE_OK so slaves can perform reads
What we found with MongoDB
• Web solution enhancements become easier
  – Used to use Oracle
• Oracle was homogenous. Had to know domain
  model and data structure up front.
  – With MongoDB we are able to change domain model
    at any point. Even at a record level.
• We do not use sharding. Our data set did not
  easily provide a shard key
  – Has not affected performance while data can fit in
    memory
Example- Cruise

Page comprises of:
•Call to product for lead-in pricing, then
•Call to Ship content
•Call to Cruise Line content
•Call for other sailings

•Page response time (without Akamai):
200ms
Moving forward
• Moved to cloud – Amazon EC2
   – MongoDB makes this much easier and cheaper than
     Oracle
• Multiple data centres catering for local countries
   – Replication model for read only is trivial. Write
     requires a bit of thinking
• Data centre fail over
   – Each data centre will be able cater for all FCL country
     load
• Application Developers changing to elastic design
   – MongoDB is well suited for elastic solutions
Lessons learnt
• New projects are very easy
• Converting heavy relational models takes some time
• Use case of data is important to know upfront
   – Our product model denormalises to 80 fields
   – Indexes are important and require constant review
      • Our developers can hit the models with any permutation of ways
        (id, location, supplier, star rating, keyword, duration)
• Performance for new solutions excellent, performance
  after converting relation solution required more
  hardware. A minor redesign in our solution will solve
  performance issues
   – Thinking about problems in a relational way is inefficient
     with MongoDB. Need to tackle problems with a different
     mind set
Lessons learnt
• Learning curve for Web Developers easy.
  Learning curve for Application Developers a little
  longer (use to relational world)
• Traditional reporting tools do not work easily.
  – Created a data warehouse in Oracle for reporting
• Global write lock. Make sure database size fits in
  memory
Questions



          Michael Frost
      Flight Centre Limited
Michael.Frost@flightcentre.com.au

MongoDB at Flight Centre Ltd

  • 1.
    Flight Centre Limited HowMongoDB has empowered the business to rapidly respond to market conditions Michael Frost Web Solution Architect
  • 2.
    Flight Centre Flight Centre Limited is one of the world's largest travel agency groups, with more than 2000 leisure, corporate and wholesale businesses in 11 countries. After starting with one shop 30 years ago, we have enjoyed remarkable ongoing growth. Our rapidly expanding network now extends throughout Australia, New Zealand, the United States, Canada, the United Kingdom, South Africa, Hong Kong, India, China, Singapore and the United Arab Emirates, providing travellers with a complete service.
  • 3.
    Flight Centre Online •120 Online staff (Web Developers, SEO, Copy Writers, SEM, Application Developers, Online Marketers, Creative designers and Usability experts) • 60 Web sites over 6 different countries • Up to 200,000 pages on a site • Larger sites have 450,000 views per day • Multiple booking engines Air, Hotel, Sea, Insurance, Car • Two data centres for web hosting. Physical in Brisbane and Amazon EC2
  • 4.
  • 5.
    Business Problem • Productfeeds for lead-in pricing (15 feeds and growing). 750,000 records • Content feeds (Hotel, Ship, Cruise Line, IATA, Destination). 550,000 records – Every feed has unique model • Need to override content – Each brand in each country is a different business • Need to be responsive to market. E.g. new cruise ship, or cruise ship sinks
  • 6.
    Business Problem • Contenthas different models, e.g. cruise ship content compared with hotel • Business constantly demand new model types or changes to existing models – Hotel • Majority developers are front end web
  • 7.
  • 8.
    FCL MongoDB Environment •FCL data centre RHEL6 64Bit 2 X CPU 7G RAM 100G data partition • Amazon servers c1.xlarge Amazon EC2 instances • MongoDB V2.0.3 • Replication with one master, and 2 slaves in each datacentre. New slaves can be quickly brought up if demand requires • 5 databases with up to 55 collections in a database • ~ 600,000 MongoDB requests per day • 2 mil records
  • 9.
    Hosting environment with Amazondata centre •Akamai geo-load balanced data centres •No single point of failure for hosting •Each data centre is designed to take full load if one data centre goes down •Each layer in each stack can be extended quickly if unexpected massive load came through
  • 10.
    What we foundwith MongoDB • Developers do not need to worry about schemas • Business can come to developers with new feed, content type, product, to be loaded in – E.g. Cruise Ship, Cruise Line, IATA location data, Hotel content, Hotel imaging – Define own domain models • Native way of thinking with Web Dev (JSON/REST) • Powers the majority of our sites
  • 11.
    What we foundwith MongoDB • Arbiter is only used to provide extra votes that the master we want stays as master • If master goes down slaves will continue to allow reads. Writes will not work – Writes are not mission critical – Competitions and the like • SLAVE_OK so slaves can perform reads
  • 12.
    What we foundwith MongoDB • Web solution enhancements become easier – Used to use Oracle • Oracle was homogenous. Had to know domain model and data structure up front. – With MongoDB we are able to change domain model at any point. Even at a record level. • We do not use sharding. Our data set did not easily provide a shard key – Has not affected performance while data can fit in memory
  • 13.
    Example- Cruise Page comprisesof: •Call to product for lead-in pricing, then •Call to Ship content •Call to Cruise Line content •Call for other sailings •Page response time (without Akamai): 200ms
  • 14.
    Moving forward • Movedto cloud – Amazon EC2 – MongoDB makes this much easier and cheaper than Oracle • Multiple data centres catering for local countries – Replication model for read only is trivial. Write requires a bit of thinking • Data centre fail over – Each data centre will be able cater for all FCL country load • Application Developers changing to elastic design – MongoDB is well suited for elastic solutions
  • 15.
    Lessons learnt • Newprojects are very easy • Converting heavy relational models takes some time • Use case of data is important to know upfront – Our product model denormalises to 80 fields – Indexes are important and require constant review • Our developers can hit the models with any permutation of ways (id, location, supplier, star rating, keyword, duration) • Performance for new solutions excellent, performance after converting relation solution required more hardware. A minor redesign in our solution will solve performance issues – Thinking about problems in a relational way is inefficient with MongoDB. Need to tackle problems with a different mind set
  • 16.
    Lessons learnt • Learningcurve for Web Developers easy. Learning curve for Application Developers a little longer (use to relational world) • Traditional reporting tools do not work easily. – Created a data warehouse in Oracle for reporting • Global write lock. Make sure database size fits in memory
  • 17.
    Questions Michael Frost Flight Centre Limited Michael.Frost@flightcentre.com.au