Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

RecsysFR: Criteo presentation

1,669 views

Published on

Talk given by Etienne Sanson, Criteo during the RecsysFR meetup on October 6th 2016.

Published in: Internet
  • Be the first to comment

RecsysFR: Criteo presentation

  1. 1. RecSys FR: 4th Session 6th October 2016 Etienne Sanson – manager R&D Engine
  2. 2. About Criteo 1
  3. 3. 3 | Copyright © 2016 Criteo Our mission TARGET THE RIGHT USER AT THE RIGHT TIME WITH THE RIGHT MESSAGE
  4. 4. 4 | Copyright © 2016 Criteo Key Figures 16 000 PUBLISHERS 90% RETENTION RATE2 +130 COUNTRIES LISTED ON THE NASDAQ SINCE OCTOBER 2013 R&D REPRESENTS 21% OF THE WORKFORCE 2000 EMPLOYEES 21 BILLIONS $3 11 000 ADVERTISERS 1.19 bn€1 31 OFFICES 1: REVENUE IN 2015 2: ANNUAL RATE 2015 3: $ OF TURNOVER GENERATED TO OUR CLIENTS - TURNOVER POST-CLICK WW FROM JANUARY TO DECEMBER 2015
  5. 5. 5 | Copyright © 2016 Criteo Revenue Growth 2009 2010 2011 2012 2013 22M$ 86M$ 199M$ 349M$ 589M$ 988M$ 2014 1,3MM$ 2015
  6. 6. 6 | Copyright © 2016 Criteo GENERAL CONCEPT Users visit an advertiser’s website 1 Criteo identifies the users (via cookies) 2 Users leave the advertiser’s website & browse publisher on the Internet 3 Criteo identifies users on these pages (via cookie) 4 Criteo displays an advertising banner, personalized for each user 5 Click through directly to the advertiser’s page 6 @ Retargeting principles
  7. 7. 7 | Copyright © 2016 Criteo Infrastructure Key Figures Sunnyvale 2 PoP 500 kVA 1 559 Servers New York 2 PoP 930 kVA 2 625 Servers Hong Kong 2 PoP 472 kVA 2185 Servers Paris 4 Pop 1 800 kVA 3 625 Servers Amsterdam 2 PoP +2 500 kVA 3 609 Servers Tokyo 2 PoP 455 kVA 2 564 Servers Shanghai 1 PoP 200 kVA 931 Servers World Wide 15 PoP 6,8 MVA 17 098 Servers > 55Gbps + 2.5M req/s Hosting Global Partners :
  8. 8. About ML@Criteo 2
  9. 9. 9 | Copyright © 2016 Criteo Our challenges – Product recommendation • Select the best ~10 products to show to a user • >1B users • Product catalog contains ~1M items, up to 1B • Time constraints: 20ms • Combination of offline/online processing steps • CF • Product embeddings (word2vec -> prod2vec) • CNN for image features What products should we recommend?
  10. 10. 10 | Copyright © 2016 Criteo Our challenges - Bidding How much should we bid for this display? What is the best campaign to display? My company BUY! BUY! BUY! BUY! • Select the best campaign to display and evaluate its value in a few ms • Large scale regression models • >1B daily displays (but few positive examples!) • >1M parameters • Distributed optimization (SGD, L-BFGS) • Feature Engineering • Transfer learning, FFM, Policy learning • Marketplace, game theory, auction theory
  11. 11. 11 | Copyright © 2016 Criteo Our challenges – X-device • Build a huge graph (Billions of nodes/edges): • Nodes = devices • Edge = the 2 devices belong to the same user • How to connect 2 devices? • How to know the ground truth? • How to keep it stable? • At scale & taking care about privacy Who is the user behind the device?
  12. 12. 12 | Copyright © 2016 Criteo Our challenges – Testing • We test everything! • Offline tests / AB Tests • Infrastructure to perform large-scale tests • >100K offline tests / year • >1K AB Tests / year • Dedicated teams • Technical / Business Metrics • Randomization • Counterfactual evaluation
  13. 13. Thank you! …and we’re hiring!

×