The document discusses Rakuten's application of AI and machine learning technologies across various domains, particularly focusing on data science for product categorization, user behavior analysis, and recommendation systems. It highlights the significant techniques used for automating tasks such as product data structuring, user segmentation, and targeted marketing based on user preferences. Furthermore, it outlines the achievements and impact of these AI applications on revenue generation and customer engagement within Rakuten's services.

1. Actual cases of applying AI related technologies
in Rakuten
Yu Hirate, Dr. Eng.
Rakuten Institute of Technology Tokyo,
Rakuten, Inc.

2. 2
Yu Hirate
 Principal Scientist, Rakuten Institute of Technology
Manager, Intelligence domain research group
 Bio.
• 2005-2008 CS div. graduate school of Science and Engineering, Waseda
University.
• 2006-2009 Research Associate, Media Network Center, Waseda University.
• 2009- current Rakuten Institute of Technology．
Working on projects for extracting knowledge from large scale of data
by utilizing data mining, machine learning technologies.

3. 3
Who is Rakuten?

4. 4
Rakuten Institute of Technology
Masaya Mori
Global head
• Established in 2006.
• Launched R.I.T. NY in 2010.
• Launched R.I.T. Paris in 2014.
• Launched R.I.T. Singapore / Boston in 2015.
Strategic R&D organization for Rakuten Group

5. 5
Rakuten Institute of Technology
+100 researchers in 5 locations
Tokyo NYSingapore
Paris
Boston

6. 6
Research Groups in RIT
3 research groups for adapting with Internet growth
RealityIntelligencePower
• HCI
• AR / VR
• Image Processing
• Distributed Computing
• HPC
• IoT
• Machine Learning
• Deep Learning
• NLP
• Data Mining

7. 7
Mission of Intelligence Domain Group
Assisting Rakuten businesses to boost with AI and Data Mining
Optimizing A/B testing
Item Classification
User Segmentation
AI
Coupon Distribution
Recommender System Economy Prediction / Demand Prediction
Review Analysis
Anomaly Detection / Fraud Detection
Image Recognition

8. 8
1. Product Data Science

9. 9
Product Data in Rakuten
Huge!
Unstructured!
241 million items
Num.ofItems(million)
date

10. 10
Product Data Science
Product Catalog
Organize Large Amounts of Unstructured Information Into Structured Knowledge
Why are we working on this problem? (Key Benefits)
‣ To organize our catalog in accordance with customer
expectations
‣ To precisely search our catalog for products and its variants
‣ To measure and enforce merchant KPI's.
What are we doing? (Key Tasks)
‣ Product Genre Classification
‣ Attribute Extraction from Product Information
‣ Merchant and Item Review Analysis
How are we doing? (Key Technologies)
‣ Large-Scale Gradient Boosted Decision Trees
‣ Deep Learning (RNN's, CNN's, others)
‣ Computing Massive Number of NLP Features
Businesses

11. 11
Generating Structured Product Data
Each product can be assigned a category and attributes. For instance:
+Category Grocery & food
Subcategory Wine
Each (sub)category has a number of relevant attributes with a list of valid values
Challenge: this structured information is not always present or correct
Goal: automatically predict category and attributes from text and/or images

12. 12
Product Category Classification

13. 13
Category Classification based on Deep Learning
Classifier based on
Deep Learning Algorithm (CNN)
Prec@1 92%
Prec@10 99%
Classifier based on
Deep Learning Algorithm(CNN)
Prec@1 57%
Prec@3 75%
Extracting Words
* Tested to Ichiba L3 category (1.5K categories)
* Tested for PriceMinister Image Data
Text Data
• Item Title
• Item Description
Image Data

14. 14
Category Classification from Image Data
Hobby and Entertainment
> Books and Magazine
> Business Electronics
> Audio
> Earphone / Headphone
Electronics
> Smartphone
> AC Adaptor / Battery

15. 15

16. 16
Attribute Extraction from Text and Images
Text : Automatic Rule Generation
with Machine Learning
Images:
Attributes prediction with Deep Learning.
attribute top-1 accuracy # of values
type 74% 94
material 66% 84
shape 83% 5
color 75% 24
PriceMinister furniture category

17. 17
Review Analysis : Informative Item Review Extraction
Informative
Positive Review
Informative
Negative Review
Non-informative
Informative
PositiveNegative
Classify All Item Reviews with two perspective
amount of information and polarity.
Pick up two reviews
Boosted up Ichiba GMS by 15 billion JPY
(as of 2015)

18. 18
Review Analysis : Merchant Review Analysis
Simple moving average
Reviews corresponding to the category
(evidence is highlighted)
category
Positive/negative

19. 19
Review Analysis : Travel Review Analysis
in collaboration with
Our Review Analysis Algorithm has
also been applied to Travel breakfast
festival

20. 20
2. User Behavior Analysis

21. 21
User Behavior Data
Purchase History Data Search Query Log Data
date date
DailyTransactionCount
DailySearchRequests

22. 22
Utilizing User Behavior Analysis
Extracting user preference, and
applying to various functions.
Discount CouponItem Recommender System
Ads
Optimization
Prospective
User Extraction

23. 23
1. Recommender Algorithm based on Distributed Representation
CategoriesMerchants
Users
Items
Services
Similar User
Item Recommender
Coupon Item
Recommender
Category
Prediction
Genre
Prediction
Related Categories
→ Navigations
Category
Navigation
Merchant
Recommender
Similar shops
(competitor)
Related Shops
Transforming {items, users, services, categories}
to distributed representation (vectors), and computing similarities.

24. 24
Recommender Algorithm based on Distributed Representation
Utilizing word2vec, doc2vec algorithms
• Similar words are projected into similar vectors.
• Relationship between words can be expressed as a simple
vector calculation.
• Analogy
• v(“woman”) – v(“man”) + v(“king”) = v(“queen”)
[T.Mikolov et al. NIPS 2013]

25. 25
Sample results of word2vec
query: coffee
• cocoa 0.603515
• robusta 0.565269
• beans 0.565232
• bananas 0.565207
• cinnamon 0.556771
• citrus 0.547495
• espresso 0.542120
• caff 0.542082
• infusions 0.538069
• tea 0.532565
Example 2
(CalTech-101)
Example 1
(English Wikipedia)

26. 26
Applying doc2vec algorithm to recommender system
Document : a sequence of words with context.
User : a sequence of item views with user’s intention.
Set of documents
Vectors for words
Vectors for documents
sim{word, word}
sim{doc, word}
sim{doc, doc}
Set of user behaviors
Vectors for items
Vectors for users
sim{item, item}
sim{user, item}
sim{user, user}

27. 27
Sample Results
Query
RecommendedResults

28. 28
Performance Evaluation
3.42
8.02
18.2
21.04
24.17
0
5
10
15
20
25
30
Item Similarity Matrix
Factorization
Doc2Vec-based Item Vector-
based (from
Word2vec)
Item Vector-
based (from
Doc2vec)
hit-rate(%)
Distributed Representation Based AlgorithmConventional methods
Hit-Rate : Percentage of Users who bought recommended items.

29. 29
Applying actual functions
Discount Coupon
Item Recommender System
Recommender algorithms w/ distributed representations are
testing in the actual services in Ichiba.

30. 30
2. Extracting Potential Customers
• We are providing 70+ services in
Japan that covers human life.
• Each service has its on purchase
history data, browsing history data.
• Those data are connected with a
single ID, Rakuten ID.

31. 31
Extracting Potential Customers
Detect prospective applicants from Ichiba purchasers
by using their purchase trends and demographics
Ichiba Active
Users
Prospective
Applicants
Extract a Fintech service

32. 32
Extracting Potential Customers : Approach
Ichiba
Active Users
Overlap
7,413
Positive Samples
7,417
Negative Samples
About 50% of contractors of the Fintech service were Ichiba Active Users.

33. 33
Extracting Potential Customers : Significant Factors in the customer
model
0 0.1 0.2 0.3
genre_41_100890_ / 花・ガーデン・DIY / DIY・工具
genre_72_111078_ / キッズ・ベビー・マタニティ / キッズ
genre_50_110983_ / 靴 / メンズ靴
Age-05-[35-40]
genre_93_101077_ / スポーツ・アウトドア / ゴルフ
Area-01-Kanto
Area-00-Others
genre_113_101126_ / 車用品・バイク用品 / カー用品
Age-08-[50-*]
Age-03-[25-30]
Age-00-none
gms
Gender-00-none
basket_max_price
frequency
basket_average_price
average_unit_price
Age-02-[20-25]
Gender-02-female
Gender-01-male
Top 20 factors selected from 141 factors
Ichiba Genre /Cars & Motorcycles/Car Accessories
Ichiba Genre /Sports & Outdoors/Golf
Ichiba Genre /Shoes/Men’s Shoes
Ichiba Genre /Kids & Baby/Kids
Ichiba Genre /Gardening & Tools / DIY Tools
average_unit_price
basket_average_price
frequency
basket_max_price
gms
Loyalty
Life Stage

34. 34
Extracting Potential Customers : Evaluation for the model
Prospective Users Control Group
• Randomly Selected
• About 300,000 users
• Score >= 0.8
• About 300,000 users
Send ichiba mail magazine to two groups
Ichiba Mail Magazine

35. 35
Extracting Potential Customers : Evaluation Result
Mail Deliver
Open Mail
Click Contents
(Visit Service
Page)
Click Rate went up by +49.23%
compared with control group
+3.52% +49.23%

36. 36
Actual Cases in Rakuten
Premium Membership Service in Japan Ichiba (E-Commerce Service)
Existing Customers
(~31K users)
Ichiba
Purchase
History Data
User
Demographic
Data
Data used for Customer Model
Conducted Email Marketing to Japan Ichiba Users
RIT
Customer
Model
Normal
Comparison of CVR
+123.30%

37. 37
Other Cases
Potential Home Loan Contractors for Rakuten Bank
Potential mobile service subscribers for Rakuten Mobile.
Potential contents purchasers for showtime (online video streaming service).
Potential Life Insurance contractors for Rakuten Insurance
Advertisement Targeting
Potential customers for offline shops

38. 38
3. Geolocation Based Marketing
Understand user’s preference
>> make appropriate recommendations
Locate users & Explore their behavior patterns
>> provide other options
>> expand usage coverage
>> extract potential cross-service users
>> facilitate cross-use between services
Rakuten Strategy: Fusion of the Internet and the Offline

39. 39
Location Based Recommender System
Which shops should be offered
to users located at this point?
User Viewpoint Shop Viewpoint
Who are potential customers
for this shop?
Issue : How to Identify major areas for each users?

40. 40
Location Based Recommender System
Centroid
Shop coordinates
Boundary
Step 1 : scatter used shops
Step 4 : locate recommended shop
Step 2 : label centroid, build boundary
Step 3 : extract all shops

41. 41
Effectiveness of geolocation based recommender
CVR (%) gms/user (JPY)
×8.6
×1.3
Send emails to make recommendations to users within their major areas
Both CVR and GMS/uu are improved
Geolocation
based
recommender
Geolocation
based
recommender

42. 42
4. Others

43. 43
Janken

44. 44
Logistics Pick Path Optimization

45. 我们真的很有诚意了。
你说我一个老总都亲
自跑了好几趟了。
Speech
recognition
Machine
translation
45
is a Rakuten group company which provides video streaming service.
Volunteers are editing subtitles and translated subtitles.
Machine Translations for TV dramas

46.  Translate from Chinese to English sentences
 Extracted 10,000 Chinese-English sentence-pairs to
evaluate commercial APIs and IBot, e.g.,
 我一个老总都亲自跑了好几趟了
 I’m a director and yet I’ve made so many trips
 Extracted another 2.1 million sentence-pairs to train
IBot’s model
46
Data Preparation

47.  Applying Attentional Recurrent Neural Networks
(RNN)
 Neural Machine Translation by Jointly Learning to
Align and Translate [Bahdanau, Cho & Bengio, ICLR 2015]
 658 citations (Google scholar)
 Train RNN with 2.1 million c
Chinese-English sentence
pairs
47
RIT’s deep learning based MT model

48.  Evaluated on 10,000 Chinese-English sentence pairs
System BLEU (%) METEOR (%)
Google API 12 20
Microsoft API 12 20
IBM Watson API 3 12
RIT (Aug 24) 10 15
RIT (Sep 7) 14 19
RIT (Sep 21) 22 24
RIT (Nov 28) 36 30
48
RIT MT module outperforms in TV drama domain

49. 49