Yahoo!       Web site optimization using a bucket  testing framework built into web analytics     Ashish Vikram     Vice P...
Ashish VikramVice President – Engineering ,User Data and AnalyticsYahoo!Ashish Vikram is Vice President - Engineering, Use...
Big Data Statistics at Yahoo!    500 Peta Bytes of Data    50 Tera Bytes of Data added everyday    700 Million Users  ...
Web Analytics – 7 days trafficYahoo! Presentation, Confidential   4   2/28/2012
Time Spent - BreakdownYahoo! Presentation, Confidential   5   2/28/2012
Overview                                              Test the *effect of product changes on                              ...
Workflow: Media Business CaseThe media team desires to test the effect on user engagement fordifferent layouts of a webpag...
Workflow: Setup, Targeting & Reporting  SETUP  The Editor sets up an experiment using the Experiment Management System (EXP)
Creating a new ExperimentYahoo! Presentation, Confidential   9   2/28/2012
Workflow: Setup, Targeting & Reporting  SETUP  The Editor assigns the experiment from the Publisher Desktop (PD2)
Workflow: Setup, Targeting & Reporting  TARGETING  The Traffic Splitter then creates control and test group from all visit...
Workflow: Setup, Targeting & Reporting  REPORTING  Microstrategy Reports with detailed findings from the test are made ava...
Results SummaryYahoo! Presentation, Confidential   13   2/28/2012
Time Spent – control Vs TestYahoo! Presentation, Confidential   14   2/28/2012
Success Stories : Wider Today Module Headline ImageFront page team desires to test the effect on user engagement on the To...
Success Stories: Wider Search BoxFront page wants to test the effect on user engagement on Search link as aresult of widen...
Thank You!        Ashish Vikram  VP Engineering, UDA
Yahoo! Web site optimization using a buckettesting framework built into web analytics
Upcoming SlideShare
Loading in …5
×

Information excellence 2012feb_yahoo_ashish vikram_website optmzn_bucket testing fw

774 views

Published on

Information Excellence 2012 Spring Summit "DATA DYNAMICS"
Website optimization using bucket testing FW for web analytics, Ashish Vikram, VP, User Data and Analytics (UDA), Yahoo!

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
774
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Information excellence 2012feb_yahoo_ashish vikram_website optmzn_bucket testing fw

  1. 1. Yahoo! Web site optimization using a bucket testing framework built into web analytics Ashish Vikram Vice President – Engineering, User Data and Analytics Yahoo!Information Excellence Summit,February 25, 2012 Bangalorehttp://Informationexcellence.wordpress.com
  2. 2. Ashish VikramVice President – Engineering ,User Data and AnalyticsYahoo!Ashish Vikram is Vice President - Engineering, User Data and Analytics (UDA) atYahoo!. UDA is the group that analyzes Yahoo!’s customer data to provide deep insightsinto user behavior, which is then leveraged to personalize digital content and enhanceYahoo!’s products.Ashish is a software industry veteran with over 20 years of hands-on design anddevelopment experience. He worked with Rational Software for 15 years both in the USand India, a role in which he set up Rational’s Software Development Center in Indiaand grew it into a highly productive lab with complete responsibility for several ofRational’s Software Engineering products, including the flagship Rational Rose. Ashishalso led the team through the transition, subsequent to IBM’s acquisition of RationalSoftware.He went on to play a crucial role in creating highly productive softwareevelopment centers in Bangalore for two more startups, OAT Systemsand Kalido. OAT was a pioneer and leader in the RFID space, whileKalido is a leading provider of data driven governance solutions.Ashish holds a B. Tech. in Electrical and Electronics Engineering fromthe Indian Institute of Technology, Delhi and an M.S. in ComputerScience from Purdue University, Indiana.
  3. 3. Big Data Statistics at Yahoo! 500 Peta Bytes of Data 50 Tera Bytes of Data added everyday 700 Million Users Created Hadoop to help deal with Big Data and open sourced itYahoo! Presentation, Confidential 3 2/28/2012
  4. 4. Web Analytics – 7 days trafficYahoo! Presentation, Confidential 4 2/28/2012
  5. 5. Time Spent - BreakdownYahoo! Presentation, Confidential 5 2/28/2012
  6. 6. Overview Test the *effect of product changes on a sample set of users by before launching to the entire population Measure *reach, engagement and monetization metrics on different versions of the product pages Reduce risk Test on smaller scale Set realistic referral expectations Forecast impact of product changes Forecast impact of product changes Statistical data Statistical data Set realistic expectations Why Experiment? Data substantiated decisions Statistical data Quantify user reaction Test on smaller scale Reduce risk Test on smaller scale Forecast impact of product changes Quantify user reaction
  7. 7. Workflow: Media Business CaseThe media team desires to test the effect on user engagement fordifferent layouts of a webpage before rolling it out to the entireaudience… HYPOTHESIS The new layout is more engaging than the existing design…
  8. 8. Workflow: Setup, Targeting & Reporting SETUP The Editor sets up an experiment using the Experiment Management System (EXP)
  9. 9. Creating a new ExperimentYahoo! Presentation, Confidential 9 2/28/2012
  10. 10. Workflow: Setup, Targeting & Reporting SETUP The Editor assigns the experiment from the Publisher Desktop (PD2)
  11. 11. Workflow: Setup, Targeting & Reporting TARGETING The Traffic Splitter then creates control and test group from all visitors of the page Control Group All visitors Traffic Test Group Splitter Not being tested
  12. 12. Workflow: Setup, Targeting & Reporting REPORTING Microstrategy Reports with detailed findings from the test are made available to the Media team
  13. 13. Results SummaryYahoo! Presentation, Confidential 13 2/28/2012
  14. 14. Time Spent – control Vs TestYahoo! Presentation, Confidential 14 2/28/2012
  15. 15. Success Stories : Wider Today Module Headline ImageFront page team desires to test the effect on user engagement on the Today module content, as aresult of widening the headline imageControl group has a narrow headline image and the test group has a wider headline image Control Bucket Test Bucket The CTR of the Today module increased as = a result of widening the headline image
  16. 16. Success Stories: Wider Search BoxFront page wants to test the effect on user engagement on Search link as aresult of widening the search box.Control group has a regular search box the test group has a wider version of search box Control Bucket A wider and pronounced search box led to an = increase in the CTR of the Test Bucket search link.
  17. 17. Thank You! Ashish Vikram VP Engineering, UDA
  18. 18. Yahoo! Web site optimization using a buckettesting framework built into web analytics

×