Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using BigQuery as a main Big Data solution


Published on

My recent slide deck on BigQuery and how we use it in

Published in: Data & Analytics
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ ◀ ◀ ◀ ◀
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Using BigQuery as a main Big Data solution

  1. 1. Nikolay Novozhilov Using BigQuery as a main Big Data solution
  2. 2. About Wego is Asia Pacific and the Middle East’s leading flight/hotel metasearch engine used by millions of travelers. Wego was founded in 2005 in Singapore
  3. 3. Introducing BigQuery Service for interactive analysis of massive datasets (TBs) Query billions of rows: seconds to write, seconds to return Uses a SQL-style query syntax It's a service, accessed by a RESTful API Pay only for what you use Based on internal Google tool - Dremel Column oriented, append only…
  4. 4. Data architecture in Wego ...
  5. 5. Why did we do it? MySQL “Zoo” BigQuery
  6. 6. Why Hadoop is more popular?
  7. 7. My collection of concerns Your data goes to cloud Not open-source, Google can stop the service “Strange” pricing model Hadoop is trending, has bigger community Append only database ???
  8. 8. Costs: storage + cost per query Same fallacy again:  “I want to launch a mom@pop – let’s buy a building”  “I want to build a site – let’s by servers”  “I want big data – let’s build a data-warehouse” Usual concerns:  No realistic estimate upfront  “Fear of running a query”
  9. 9. StackOverflow support 53 minutes!
  10. 10. Append only… Slowly changing dimensions:  daily re-load from MySQL  daily upload from MySQL, keeping history Absolutely necessary updates:  do you really need it?  BigQuery allows to save query to initial table: Your table Query
  11. 11. Actually useful - “Discovery mode”
  12. 12. Actually useful Huge joins REGEXT_MATCH(), … Rich SQL - window functions Nested data
  13. 13. My answer
  14. 14. What is Big Data revolution? There is no difference between big data and small data anymore
  15. 15. Contacts Blog: Email:
  16. 16. “Yes, Sir, I tired to build an ROI case for our BI project - but I couldn’t access any reliable data!”