Today almost every website and app collect data about the interactions (clicks, likes, views…) between users and items. The most common use case for these sparse “user-item” matrices is to train and improve different recommendation systems. In my presentation I will introduce how we can use exactly the same matrices together with additional datasets to generate valuable features that can be used to train different regression and classification models. I will start with describing how it was implemented at SimilarWeb, in order to accurately estimate different website metrics like demographics (age and gender) and category and continue with explaining how we can expand the algorithm to solve similar problems in different domains.

1. Shlomi Babluki, Principal Data Scientist
October, 2018
Interaction Based Feature Extraction
How to Convert User Activity into Valuable Features

2. User Activity
Business Proprietary & Confidential | 2
LikeClick
Share Play
Buy

3. Every Website Every Mobile App
Ranking
Engagement
App Store
Category
Keywords
Traffic Metrics
Traffic Sources
Audience
Industry
Content
Provides Digital Insights for 190+ countries
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
3

4. SimilarWeb PRO
4

5. Website Demographics
5

6. Millions of user in almost every country.
Our Data
6
International Panel
Learning Set
Direct measurement data (like Google Analytics) for ~50,000 Websites.

7. Gender Distribution - Standard Solution
● For each website in our panel:
○ Count the number of males
○ Count the number of females
○ Calculate the gender distribution
● Use the learning set to improve the estimation
Our panel is completely anonymous!

8. Website B Gender Distribution
77% - 88% Females
Website B:
● Panel: 90 Users
● Learning set: N/A
Website A - 80 Females and 20 Males
Our Idea - Example
A
B
Website A:
● Panel: 100 Users
● Learning set: 80% Females / 20% Males
8

9. Our Idea
Estimate website gender distribution
based on its user engagement with the other sites in
the learning set.
9

10. ● An indicator matrix of websites (S) and users (U):
P(i, j) = 1 if user j visited website i
P(i, j) = 0 Otherwise
Our Panel Matrix (P)
10
Convert Our Panel Into An Interaction Matrix
● |S| = Millions, |U| = Millions
dim(P) = |S| x |U| → P is a very large sparse matrix!

11. - Principal Component Analysis
- Matrix Factorization (ALS Model)
- Word2Vec
- ...
‫״‬The Curse of Dimensionality‫״‬
We need to reduce the dimension of the panel matrix (P):
Feature Extraction / Dimension Reduction Algorithms:
dim(P) = |S| x |U|
dim(F) = |S| x K
Business Proprietary & Confidential | 11
The standard algorithms didn’t
solve our problem

12. The standard algorithms reduce the dimension without taking into account our problem.
Dimension Reduction - Conclusion
Business Proprietary & Confidential | 12
An algorithm that reduces the dimension in a way that is optimized to solve our problem.
The Problem
The Solution

13. Interaction Based Feature Extraction
13

14. Interaction Based Feature Extraction - Step 1/3
● Convert our learning set (L) into a matrix (D1):
○ Split the gender percentiles into K (=10) “buckets”:
[0.0-0.1, 0.1-0.2, 0.2-0.3, ... 0.9-1.0]
○ Map each value from the learning set into an indicator vector:
Website A, 0.73 → [0, 0, 0, 0, 0, 0, 0, 1, 0, 0]
Website B, 0.26 → [0, 0, 1, 0, 0, 0, 0, 0, 0, 0]
...
dim(D1) = |L| x K
14

15. Interaction Based Feature Extraction - Step 1/3
0.0 0.0 0.0 0.0 ...
0.0 0.0 1.0 0.0 ...
... ... ... ... ...
Learning Set Vector (K)
LearningSetWebsites
D1
15

16. ● Multiply the D2 matrix by D1:
D = D2 * D1
dim(D) = (|U| x |L|) * (|L| x K) → |U|x K
Interaction Based Feature Extraction - Step 2/3
● Create another matrix (D2) of users (U) and only the learning set websites (L):
D2(i, j) = 1 if user i visited learning set website j
D2(i, j) = 0 Otherwise
dim(D2) = |U| x |L|
● Normalize each row (user) in the matrix D to 1.0
16

17. Interaction Based Feature Extraction - Step 2/3
0.0 0.0 1.0 0.0 1.0
1.0 0.0 0.0 0.0 0.0
... ... ... ... ...
X
Learning Set Websites Learning Set Vector (K)
Users
LearningSetWebsites
D2 D1
Normalize Each Row
D =
17

18. Interaction Based Feature Extraction - Step 2/3
● D Matrix - Example:
User 1: [0.0, 0.0, 0.0, 0.0, 0.2, 0.3, 0.1 0.1, 0.2, 0.1]
User 2: [0.1, 0.0, 0.2, 0.1, 0.5, 0.0, 0.1, 0.0, 0.0, 0.0]
Business Proprietary & Confidential | 18

19. ● Multiply the panel matrix (P) by D:
F = P * D
dim(F) = (|S| x |U|) * (|U| x K) → |S|x K
Interaction Based Feature Extraction - Step 3/3
● Normalize each row (website) in the matrix F to 1.0
19

20. Interaction Based Feature Extraction - Step 3/3
0.0 0.0 1.0 0.0 ...
1.0 1.0 0.0 0.0 ...
... ... ... ... ...
X X
Users Learning Set Websites Learning Set Vector (K)
Users
Websites
LearningSetWebsites
P D2 D1
Normalize Each RowNormalize Each Row
F =
20

21. Interaction Based Feature Extraction - Features
21
F1 F3 F4 F5 F6 F7 F8 F9 F10F2

22. Train a Regressor
Male Traffic ShareAllWebsites
F
Features (K=10)
LearningSetWebsites
X Y
22
Train
Predict

23. Random Forest Regression - Results
23

24. Expanding The Algorithm
24

25. ● 27,000 Movies
● 138,000 Users
● ~20M Ratings (Explicit Feedback: 1-5)
● Multiple Genres per Movie
Can We Expand It To Other Domains?
MovieLens Dataset:
25
F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems (TiiS) 5,
4, Article 19 (December 2015), 19 pages. DOI=http://dx.doi.org/10.1145/2827872

26. ● Action
● Adventure
● Animation
● Children
● Comedy
● Crime
● Documentary
● Drama
● Fantasy
Predicting Movie Genres
MovieLens Movie Genres (18):
● Film-Noir
● Horror
● Musical
● Mystery
● Romance
● Sci-Fi
● Thriller
● War
● Western
26

27. Predicting Movie Genres
27
movies.csv
ratings.csv
movieId,title,genres
1,Toy Story (1995),Adventure|Animation|Children
2,Jumanji (1995),Adventure|Children|Fantasy
3,Grumpier Old Men (1995),Comedy|Romance
...
userId,movieId,rating,timestamp
1,2,3.5,1112486027
1,29,3.5,1112484676
1,32,3.5,1112484819
...
International Panel
Learning Set

28. Predicting Movie Genres - Genre Features
3.0 0.0 1.0 4.0 ...
... ... ... ... ...
2.0 0.0 0.0 5.0 ...
... ... ... ... ...
X X
Users Train Set Movies Genre Vector (18)
Users
Movies
TrainSetMovies
P D2 D1
Normalize Each RowNormalize Each Row
F =
1.0 1.0 0.0 0.0 0.0 ...
0.0 0.0 1.0 1.0 0.0 ...
... ... ... ... ... ...
28

29. Predicting Movie Genres - Genre Features
29

30. Genre: Adventure
● Total Accuracy: 86.9% (6559)
● True Positive Rate: 70.0% (682)
● True Negative Rate: 88.9% (5877)
Predicting Movie Genres - Results
30
Genre: Animation
● Total Accuracy: 95.5% (6611)
● True Positive Rate: 93.4% (273)
● True Negative Rate: 95.6% (6338)
* Movies with more than 20 users / votes
Overall Accuracy: ~83%

31. ● High Accuracy
● Very Low Dimension
● Scalable
● Simple & Explainable (not a “black box” solution)
Business Proprietary & Confidential | 31
Interaction Based Feature Extraction

32. Questions?

33. Interaction Based Feature Extraction - SQL Implementation
0.0 0.0 1.0 0.0 ...
1.0 1.0 0.0 0.0 ...
... ... ... ... ...
X X
Users Learning Set Websites Learning Set Vector (K)
Users
Websites
LearningSetWebsites
P D2 D1
Normalize Each RowNormalize Each Row
F =
Business Proprietary & Confidential | 33
0.0 0.0 0.0 0.0 ...
0.0 0.0 1.0 0.0 ...
... ... ... ... ...
0.0 0.0 1.0 0.0 1.0
1.0 0.0 0.0 0.0 0.0
... ... ... ... ...

34. Table D1: <ls_site, k1, k2... k10>
Table D2: <user, ls_site>
Table P: <site, user>
Interaction Based Feature Extraction - SQL Implementation
Business Proprietary & Confidential | 34

35. SELECT
site
AVG(f1) as f1,
...
AVG(f10) as f10
FROM (
SELECT
site,
user,
D.k1 / (D.k1 + ... + D.k10) as f1,
...
D.k10 / (D.k1 + ... + D.k10) as f10
FROM (
SELECT P.site, D.user, D.ls_site, D.k1... D.k10
FROM P
JOIN (
SELECT D2.user, D1.ls_site, D1.k1... D1.k10
FROM D1
JOIN D2
ON D1.ls_site = D2.ls_site
) AS D
ON P.user = D.user AND P.site <> D.ls_site
) as F1
GROUP BY site, user
) as F
GROUP BY site
Interaction Based Feature Extraction - SQL Implementation
Business Proprietary & Confidential | 35

36. SELECT
site
AVG(f1) as f1,
...
AVG(f10) as f10
FROM (
SELECT
site,
user,
D.k1 / (D.k1 + ... + D.k10) as f1,
...
D.k10 / (D.k1 + ... + D.k10) as f10
FROM (
SELECT P.site, D.user, D.ls_site, D.k1... D.k10
FROM P
JOIN (
SELECT D2.user, D1.ls_site, D1.k1... D1.k10
FROM D1
JOIN D2
ON D1.ls_site = D2.ls_site
) AS D
ON P.user = D.user AND P.site <> D.ls_site
) as F1
GROUP BY site, user
) as F
GROUP BY site
Interaction Based Feature Extraction - SQL Implementation
Business Proprietary & Confidential | 36
Avoid Data Leakage

37. Thank You!
shlomib@similarweb.com
@shlomibabluki