More Related Content
Similar to Ellie W
Similar to Ellie W (20)
More from Hilary Ip
More from Hilary Ip (20)
Recently uploaded
Recently uploaded (20)
Ellie W
- 1. How does Expedia Group approach it? Enhance your customer experience with Big Data Ellie Warren Director of Account Management ANZ
- 3. What customer needs am I solving?
- 4. WHY IS TECHNOLOGY KEY TO Enhanced Experience? 6.7bn 200-page books Enough data to fill 189bn Rows of Consumer Behaviour Insights With over 100s Data Scientists and Analysts And
- 5. 10 Billion ways to get here…
- 6. ` SIN to TYO 50 flights 1 stop 10,000,000,000 2 stops Nn = trillions+ 3 seconds $490$517
- 7. Best Fare Search sorts through millions of flights in a few seconds and narrows down to the 1000 most relevant ~19,000,000,000,000,000 possible combinations for a single route
- 8. C O N F I D E N T I A L APPLYING THE ALGORITHM FOR BUSINESS TRAVEL BEST FARE OPTION
- 10. THE SCIENTIFIC METHOD IS THE STATE-OF-THE-ART SYSTEM TO DE-RISK INVESTMENT 3. CONSTRUCT A HYPOTHESIS 4. TEST AS QUICKLY AS POSSIBLE 5. ANALYZE /REPORT/ REPEAT 1. ASK A (SIMPLE) QUESTION 2. COLLECT OBSERVATIONS
- 11. Expedia Confidential OBSERVE, QUESTION AND LISTEN LIKE A SCIENTIST
- 12. Skyline Downtown Metropolis Light City HorizonSkyscraper Sun/Sunlight CLASSIFICATION CLUSTERING AND PREDICTION We need to understand the problem before the algorithm can
- 13. USING SMART DATA TO LEARN FAST… VELOCITY MATTERS
- 14. 1500+ 70% FAIL BEING SCIENTIFIC PAYS OFF
- 16. Every once in a while a new technology, an old problem, and a big idea turn into an innovation Dean Kamen
- 17. GETTING A LITTLE BETTER EVERY DAY
- 18. QUESTIONS?
Editor's Notes
- It’s a pleasure to be here talking about a subject that can be an undeniable, overwhelming, but yet very powerful means to driving business performance and making sure your business is delivering what the customer wants. Over the next 30minutes I will share with you a little bit about why data is important in today’s business environment to make the right decisions and reach company's goals, We all aspire to create disruption that constructs new realities for customers and builds a distinct advantage for our organizations. In order to make that leap, we have to look across trends and decide: which variables and trends we need to care about which ones will prevail which ones we invest in
- Key point: Based on a webinar we’ve done last year with webinar Microsoft they’ve mentioned that we should expect an astronomical explosion of data. By 2020, data will reach 44ZB, quadruple what it is today You need to understand that Big data is not applicable to just large companies, any company over the years accumulated tons of data from each transaction. And this data holds a lot of precious information for you as a business The key to true business transformation – or the magic behind it – doesn’t come from any one of these on their own. It comes from the dynamic and fluid intersection Through the infusion of data intelligence into your systems (marketing, finance, CRM, etc), amplifying the insight, market awareness, and strategic thinking, you will accelerate your speed to market, provide truly unique user experiences for your customers, and streamline your operations to remove error from repetitive processes
- At Expedia group, we keep our Customer at the Heart of our Innovation. Be that on the B2B or B2C side We view our innovation from the lens of the consumer – We always ask ourselves: What are the key customer problems? Are we solving the right things? How do we make the inspiration to booking phase as seamless and frictionless as possible? Are we serving a relevant ad to the right customer and clicking them through to the most relevant and personalized landing page? We look at what data we have that will allow to solve these problems and we follow this data to come to the right outcome. We see consumer delight through the lens of delivering personalisation and relevance, leveraging data and new technology to enhance and simplify, but also to inspire and help discover. People don’t travel to go to a place, people travel for an outcome; the complexity of travel is determining the specificity of the outcome.
- Data & Insights we have over 1,500 Terabytes of data. This is equal to over 6.7 Billion 200 page books. With access to over 189 Billion rows of customer behavior insights, our incredibly talented data scientists and analysts focus on machine learning algorithms and building out customer behavior models to identify patterns. This not only ensures we are presenting the customer with the right selection, in the right order, with the required context needed to make a purchase decision, but also that we are presenting the right ad, to the right customer and paying an efficient cost for that ad across our marketing channels. All of this comes down to one main concept: we are analyzing a ton of travel market data, learning from it, and passing the knowledge on to our partners and our customers. we make our data work hard to create a better, personalised, customer experience.
- At Expedia group, we keep our Customer at the Heart of our Innovation. Be that on the B2B or B2C side We view our innovation from the lens of the consumer – We always ask ourselves: What are the key customer problems? Are we solving the right things? How do we make the inspiration to booking phase as seamless and frictionless as possible? Are we serving a relevant ad to the right customer and clicking them through to the most relevant and personalized landing page? We look at what data we have that will allow to solve these problems and we follow this data to come to the right outcome. We see consumer delight through the lens of delivering personalisation and relevance, leveraging data and new technology to enhance and simplify, but also to inspire and help discover. People don’t travel to go to a place, people travel for an outcome; the complexity of travel is determining the specificity of the outcome.
- Let’s look at an example for a min: From SIN to NRT on any given day, there might be 50-100 direct flights. If you add one stop, the number of options increases to 10B+ If you add two stops, the number of options increases to N to the power of N which is in the trillions+ We have to calculate all the options within 3 seconds One price change in any given segment will cause us to recalculate the entire domain of options again. So you can see straight away how complex this can be. Travel represents a complex matching problem, which is made exponentially more difficult in a world where we offer multiple products. How do we match the traveller with the best trip for them in the quickest and easiest way possible? We try to solve this matching problem with data science and machine learning.
- Machine Learning BFS is our complex algo 10 years in the making: it filters from millions of options to 10,000 most likely booked flights, then reduces down to 1000 most relevant options in 3 seconds. The BFS uses Data Science to narrow the options you’re really interested in But how many of these options does the BFS really sift through? The BFS analyzes more than 19 QUADRILLION itineraries when you plan a round trip, say from Singapore to Tokyo. That’s a 19 with 15 zeroes behind it. It can be hard to wrap your head around that number but let’s take a moment to think about what that figure actually means? It’s 27x more than all the hairs on human heads in the world. It’s 175,000x more than all the people who have ever lived. It’s 55,000x more than all the passengers who have ever flown. Now, Expedia does 30 billion searches a year. 30 billion searches TIMES 19.7 quadrillion results comes out to 591 SEPTILLION itineraries … that’s 591 with TWENTY-SIX zeros.
- BFO is the functionality that provides a ‘benchmark’ proposal of flight and price for each Air searches that, as mentioned, takes into account convenience and the resulting cost policy for the Company. Alternative Best Fare Option is the single flight result that by leveraging your policies is the best balance between convenience and price
- But it wasn’t always like that… Before 2005 we (Expedia group) didn’t spend a great deal of money on technology. Our observation was in order to play in the market and be a leader, we knew we needed to be on the crest of the technology wave, but honestly didn’t know exactly where to put our money. We figured it made more sense to invest in an agile process, than to try to anticipate what was coming next. So in 2005 we spent 50Mil on a framework for innovation where we Walk in our customers shoes, come up with a hypothesis and then test our inventions and constantly course correct based on the signals we get in a live environment. Innovation is at the heart of Expedia Group. And we fully recognize THE NEED FOR SPEED!! With today’s rate of change and the accelerated speed of technology adoption, changing customer needs and customers becoming more demanding, we all need to think about how we can innovate and optimize to keep up with this evolving landscape.
- Expedia approaches online innovation and product development in a really scientific way – we call it the ‘test-and-learn’ approach. We adapt and innovate our sites and app as a process of continual improvement. We: Ask a simple question Collect observations Form a hypothesis Test and Learn Analyse results…. We then ’follow the data’ and make incremental changes. At Expedia Test & Learn became part of our DNA a few years ago. In 2010, we began a path to transform our platform and introduced a test & learn approach to innovation rooted in data. We haven’t looked back. So we follow a real scientific approach to our business. Where we ask a simple question, collect the observation, construct a hypothesis, test that out as quickly as possible, then analyse the learning and repeat on that. Our test & learn methodology makes sure our innovation is customer-led and verified by real data & insights. This philosophy has allowed us to optimise quicker and more often using our powerful data and insights to evaluate new ideas and make decisions. Test & Learn changed the way we think about innovation. We moved from four release cycles per year to weekly releases allowing us to continually iterate our site with an agile development mindset. We believe in an incremental approach to product innovation that involves making many small investments to see which gets the best signal before we place a big bet on something. And this is not only about Product and technology. We Test and Learn across everything we do at Expedia. Marketing, Merchandising….even HR tests and learns all the time. But Why Data is critical: the more informed you are across your organization the more valuable your hypothesis is going to be.
- we learn from every visitor to our site: While each individual has a unique experience booking a trip, collectively this data reveals trends that inform future site and app innovation. In this way, Expedia makes the change that really matter to customers, which helps both them and its partners. We have 10+ years of data from Asia revealing travel search, booking and destination patterns / what works on our site, where there is room for improvement, etc.
- The application of Data Science and machine learning in driving optimization has become a huge focus at Expedia. Such is the case that we have set up ourselves an objective of managing the majority of our Marketing dollars globally by computers. Via data science, we have been able to contextually optimize on a real-time basis the content and messages we push to our customers. It has also allowed us to manage risk prediction and churn intervention models to increase the life time value of our customers. META TAGGING When it comes to the use of data in defining marketing and creative execution, we have started implementing a data science approach by meta tagging all of our creatives, in order to understand the driver of ad performance in future campaigns. Using the google video API we are able to detect different features of our ads like the city skyline, or a sunset, and use those tags for classification, clustering and prediction. You can also use those tags to regress against a specific metric to understand how certain features influence performance. The key though, is not to assume that big data automatically solves the problem. That is certainly not the case. (CLICK) We have huge amount of data, but the distribution of observed points can be very sparse especially if the tail of the distribution is long. Most companies KPI’s have these long tails. A simple example of this happens with businesses with an uneven geographical distribution of sales. If a business doesn’t have sufficient observation points from Asia does not mean Asia does not have enormous business potential. So, we can’t blindly apply machine learning. If we don’t understand how the problem is posed to the algorithms we will end up getting the wrong answers leading to bad business outcomes.
- Because it’s all about velocity and learning. Remember it really does not matter how many times a test is a winner or a loser, as long as you are learning from it. That’s why the only tests we kind of do not like are the Neutral or inconclusive tests. Getting it right or wrong, but at speed, will allow you to become incrementally better.
- Customers telling us what’s the product they want, or what are the ads they like the most in small, measurable, incremental tests is as valuable as it gets. But here’s the thing. Out of the 1500+ tests we run, about 70% fail. So you have to be comfortable with the fact that you are going to invest time and effort on something that will be live on your site for a few days or weeks and then will be thrown away. No one likes investing their product or marketing budgets on testing and run the risk of failing. But even if we fail more than half of the time, that equates to hundreds of wins. So at the end of the day being scientific pays off.
- Unless you continuously look at ways of utilizing your data through ongoing, creative and persistent experimentation, it will be difficult to shift the needle. Focus on the how not the what. You need to throw enough stuff on the wall to see what sticks. And most importantly, don't be afraid of being wrong. On the contrary, embrace it. We are wrong most of the time. Our CEO constantly asks us to fail. But the trick is in failing fast and moving on to the next thing.
- As a company, we strive to innovate with ways to make travel planning easier, no matter where you are or what device you're using. The market has seen rapid adoption of these new technologies within the past year and we want to make sure that Expedia customers are able to get the travel information they need, exactly when they need it.
- As long as you get a little better every day, the aggregation of marginal gains will make you exponentially better over time. Think about the potential this has if you apply it to everything you do. Think about the potential if everyone within your company applies it to everything they do. You don’t only become better professionals and create greater companies, you become greater people and contribute to a greater world. So find your purpose and aim for that 1% improvement.