李仁杰/ Riot Games Head of Data Science

1. DATA SCIENCE RENJIE LI AT RIOT GAMES 07.16.2016

2. 2009 LAUNCH TEAM ORIENTED ONLINE GAME 120+ CHAMPS LIVE PLAYERS VS. LIVE PLAYERS

3. BASKETBALL LEAGUE OF LEGENDS

4. Players & Data W O R L D W I D E Statistics released Jan 2014 67+ million monthly active players 500+ billion data points per day 26 petabytes data collected since beta

5. DATA SCIENCE AT RIOT GAMES EMPOWER RIOTERS TO MAKE BETTER DATA POWERED PRODUCTS TEAM MISSION

6. DATA SCIENCE AT RIOT GAMES PLAYER FOCUSED DATA INFORMED, NOT DATA DRIVEN TEAM PHILOSOPHIES

7. DATA DRIVEN INFORMED DECISION MAKING DATA DRIVEN DATA INFORMED

8. DATA SCIENCE AT RIOT GAMES Data Science

9. DATA SCIENCE AT RIOT GAMES Data Science RiskMarketing Ecommerce

10. DATA SCIENCE AT RIOT GAMES Data Science Social Play RiskMarketing Ecommerce

11. DATA SCIENCE AT RIOT GAMES Data Science Social Play Risk Match Making Marketing Ecommerce

12. DATA SCIENCE AT RIOT GAMES Data Science Social Play Risk Match Making Player Onboarding Marketing Ecommerce

13. DATA SCIENCE AT RIOT GAMES Data Science Social Play AI Risk R & D Match Making Eco System Player Support Player Onboarding Game Balance Design Marketing Ecommerce

14. CASE STUDY #1 UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR

15. UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR CHAMPION PLAY PATTERNS

16. UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR CHAMPION PLAY PATTERNS

17. UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR CHAMPION PLAY PATTERNS TYPE A CHAMPION TYPE B CHAMPION TYPE C CHAMPION TYPE E CHAMPION TYPE F CHAMPION TYPE G CHAMPION TYPE D CHAMPION 7 KEY TYPES OF CHAMPIONS OUR DESIGN PHILOSOPHY ALIGNS WITH PLAYERS PLAY STYLES

18. UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR 0% 10% 20% 30% 40% 50% 60% Type A Champion Type B Champion Type C Champion Type D Champion Type E Champion Type F Champion Type G Champion Player 1 Player 2 PLAYER SEGMENTATION BASED ON HOW THEY PLAY EACH TYPE OF CHAMPION

19. UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR PLAYER SEGMENTATION BASED ON HOW THEY PLAY EACH TYPE OF CHAMPION 9 DIFFERENT CHAMPION PLAY BEHAVIOR

20. UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR Type A Type B Type C Type D Type E Type F Type G IDEAL VS. CURRENT DISTRIBUTION OF CHAMPION CLUSTERS CURRENT IDEAL OVERSERVEDUNDERREPRESENTED

21. CASE STUDY #1 TAKEAWAYS CERTAIN CHAMPION ARCHETYPES MAY BE UNDERREPRESENTED OR OVERSERVED HOW DIFFERENTLY PLAYERS PLAY OUR CHAMPIONS WHETHER OUR DESIGN PHILOSOPHY ALIGNS WITH PLAYERS’ PLAY STYLES CHAMPION PLAY PATTERN MODEL HELPS US BETTER UNDERSTAND:

22. CASE STUDY #2 UNDERSTANDING PLAYER ENGAGEMENT

23. UNDERSTANDING PLAYER ENGAGEMENT week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10 week 11 week 12 week 13 week 14 week 15 week 16 week 17 week 18 week 19 week 20 week 21 week 22 week 23 week 24 Actual TOTALHOURSPLAYED EVENTB EVENTC EVENTD EVENTE EVENTA

24. UNDERSTANDING PLAYER ENGAGEMENT week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10 week 11 week 12 week 13 week 14 week 15 week 16 week 17 week 18 week 19 week 20 week 21 week 22 week 23 week 24 Predicted TOTALHOURSPLAYED EVENTB EVENTC EVENTD EVENTE EVENTA

25. UNDERSTANDING PLAYER ENGAGEMENT week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10 week 11 week 12 week 13 week 14 week 15 week 16 week 17 week 18 week 19 week 20 week 21 week 22 week 23 week 24 Predicted Actual TOTALHOURSPLAYED EVENTB EVENTC EVENTD EVENTE EVENTA

26. CASE STUDY #2 TAKEAWAYS PROVIDE INSIGHTS FOR STRATEGIC DECISION MAKING AND PLANNING QUANTIFY EFFECT OF EVENTS AND CHANGES IN THE GAME FIND AND UNDERSTAND IMPORTANT FACTORS THAT PLAYERS ARE INTERESTED IN WEEKLY ENGAGEMENT PREDICTION MODEL HELPS US :

27. Production Predictive Diagnostic Descriptive

28. CASE STUDY #3 PERSONALIZED RECOMMENDATION

29. PERSONALIZED RECOMMENDATION MANY ENTERTAINMENT SERVICES AND ECOMMERCE COMPANIES HAVE MADE RECOMMENDER SYSTEM A PROMINENT PART OF THEIR WEBSITES GOOD RECOMMENDATIONS ADD ANOTHER DIMENSION TO THE USER EXPERIENCE AND BOOST CONTENT ENGAGEMENT

30. PERSONALIZED RECOMMENDATION GOOGLE NEWS: RECOMMENDATIONS GENERATE 38% MORE CLICKTHROUGH NETFLIX: 66% OF THE MOVIES WATCHED ARE RECOMMENDED AMAZON: 35% SALES ARE FROM RECOMMENDATIONS

31. PERSONALIZED RECOMMENDATION SIMILAR RECOMMENDATION The recommender system is a predictive model that generates recommendations based on similarities between users and items as well as user-item interaction.

32. PERSONALIZED RECOMMENDATION

35. CASE STUDY #4 INSTANT FEEDBACK SYSTEM

36. TRIBUNAL

37. NATURAL LANGUAGE PROCESSING

38. NATURAL LANGUAGE PROCESSING

39. INSTANT FEEDBACK SYSTEM TRIBUNAL NATURAL LANGUAGE PROCESSING+

40. INSTANT FEEDBACK SYSTEM SUPPORTS 14 DIFFERENT LANGUAGES INCREASES CHATLOG COVERAGE FROM 10% TO 100% DECREASES DETECTION TIME FROM WEEKS to 15 MINS

41. DATA SCIENCE AT RIOT GAMES PLAYER FOCUSED DATA INFORMED, NOT DATA DRIVEN TEAM PHILOSOPHIES EMPOWER RIOTERS TO MAKE BETTER DATA POWERED PRODUCTS TEAM MISSION

42. DATA SCIENCE Renjie Li rli@riotgames.com THANK YOU! TW OFFICE Wayne Lee Julia Su walee@riotgames.com jsu@riotgames.com

43. QUESTIONS?

李仁杰/ Riot Games Head of Data Science

台灣資料科學年會

李仁杰/ Riot Games Head of Data Science