Tadeh Alexani, profile picture
Uploaded byTadeh Alexani
PDF, PPTX423 views

BeepTunes Music Recommender System

The document discusses the development of a music recommender system using the Beep Tunes dataset, outlining various approaches such as collaborative filtering and content-based filtering. It emphasizes the importance of user and item profiling, data analysis, and different model evaluation techniques including mean absolute error. The system's evaluation demonstrated significant improvements in recommendation accuracy by optimizing parameters and incorporating additional data features.

Technologyā—¦
Related topics:
Data Mining Insightsā€¢
In this document
Powered by AI

Overview of a presentation on building a Music Recommender System using the Beep Tunes dataset.

Emphasis on the importance of questioning results in data science to ensure thorough analysis.

Introduction to the dataset utilized for building the recommender system: BEEPTUNES.COM.

Different approaches to recommender systems including Manual Curation, Editorial Tagging, and Audio Signals.

Explanation of Content-Based, Collaborative Filtering, Memory-Based, and Model-Based systems.

Details on collaborative filtering distinguishing between memory-based and model-based methods with advantages and disadvantages.

Description of user-based and item-based collaborative filtering approaches.

List of datasets used for Exploratory Data Analysis, relevant to the recommender system.

Analysis of user interactions and their metrics in terms of downloaded tracks, purchased tracks, and likes.

Sample data presentation showing various user actions and timestamps related to track interactions.

Explanation of how the interaction score (Impression) combines likes, downloads, and purchases.

Discusses the time analysis of user actions concerning track publication and user interaction.

Methodology for weighing user actions based on the time of interaction and frequency of actions.

Additional data on user actions, incorporating time parameters for deeper analysis.

Framework for evaluating recommendations using online and offline evaluation methods.

Explanation of how training and test datasets were derived from a large dataset of user actions.

Metrics showing user interactions with tracks, including their impressions, durations, and pricing.

Introduction to evaluating the modelā€™s performance using Mean Absolute Error on the test dataset.

Focus on the tools and methods used for regression evaluation in the recommender system context.

Describes how correlations are computed between user-item matrices to enhance recommendation quality.

Outlines the tools used in memory-based collaborative filtering, focusing on various ML tasks.

Visualization of user-item interactions showing the matrix factorization approach for recommendations.

Discusses the advantages of using matrix factorization techniques in dimensionality reduction.

Introduction to ALS as a matrix factorization algorithm that enhances computational efficiency.

Describes how ALS solves issues related to the scalability and sparseness of user ratings.

Optimization strategy using a grid search to determine the best parameters for model performance.

Evaluation comparing content-based and collaborative filtering recommendations.

Summary of the evaluation results showing improvements in user recommendation metrics.

Recap of the different types of recommender systems including collaborative filtering and content-based.

Details of track collection based on various metrics in exploratory data analysis.

Explains how item and user profiles are constructed for content-based filtering.

Describes the creation of item profiles based on the features of each track.

Explanation of how user profiles are calculated based on interactions with weighted item profiles.

Mathematical formulation used in content-based filtering for calculating recommendations.

Advantages of content-based filtering such as recommending unique items to users.

Challenges of content-based filtering including overspecialization and feature identification difficulties.

Presentation of a sample track used for evaluating the recommender system.

List of tracks found to be similar to the sample track using the content-based filtering model.

Instructions for executing a Spark job relevant to the collaborative filtering model.

Script showcasing how to update track collections for the recommender system.

Overview of scheduled jobs for updating track collections in the system.

Discussion on the cold start problem and its impact on recommender systems.

Details of utilizing hybrid models in recommending systems based on user actions.

Describes the method for extracting similar tracks based on user input.

Acknowledgment of the team members involved in the project.

Final thank you to the audience and conclusion of the presentation.

Embed presentation

Download as PDF, PPTX
Building a Music Recommender System from Scratch on the Beep Tunes Dataset Niloufar Farajpour, Mohamadreza Kiani, Mohamadreza Fereydooni, Tadeh Alexani 1 Rahnema College - Winter 2020
Frank Kane As a data scientist, question the results, because, often there is something you missed. 2
3 Dataset: BEEPTUNES.COM
Approaches ā— Manual Curation by Experts ā— Editorial Tagging ā— Audio Signals ā— Recommender Systems 4
5 Recommender Systems Content Based Collaborative Filtering Memory BasedModel Based
6 CollaborativeFiltering(CF) Model Based Memory Based Find similar users based on cosine similarity or pearson correlation and take weighted avg. of ratings Use machine learning to ļ¬nd user ratings of unrated items. e.g. PCA, SVD, Neural Nets, Matrix Factorization Advantage Easy creation and explainability of results Disadvantage Performance Reduces when data is sparse. So, non scalable Advantage Dimensionality reduction deals with missing/ sparse data Disadvantage Inference is intractable because of hidden/ latent factors
7 User-based vs Item-based Users who are similar to you also liked ā€¦ Users who liked this item also liked ā€¦
8 EDA ā— Given Dataset: ā—‹ album_like.csv ā—‹ album_track_purchase.csv ā—‹ album_artist.info ā—‹ download_album.csv ā—‹ artist_like.csv ā—‹ track_download.csv ā—‹ track_like.csv ā—‹ track_info.csv ā—‹ track_tag.csv ā—‹ track_artist.csv
9 EDA Downloaded Tracks Purchased Tracks Likes Tracks
10 EDA USER_ID TRACK_ID C_DATE PUBLISH_DATE LIKED DOWNLOAD PURCHASE 8681411 553605400.0 2019-11-19 11:11:29 2019-10-23 00:10:13 0.0 1.0 0.0 46170847 550442677.0 2019-06-19 09:06:27 2018-05-22 09:05:20 0.0 1.0 0.0 469026688 486718552.0 2018-06-29 23:06:07 2017-07-23 16:07:31 0.0 1.0 0.0 6644142 6058526.0 2014-03-14 20:03:04 2013-03-21 16:03:15 0.0 0.5 0.0 511266472 509711880.0 2018-09-19 20:09:36 2018-02-20 12:02:32 0.0 0.5 0.0 ActionCollection
11 EDA Impression = (Like*4) + (Download*2) + (Purchase*1) Based on the inverse of like, download, purchase frequency among all actions
12 EDA Downloaded Tracks Purchased Tracks Likes USER_ID TRACK_ID C_DATE PUBLISH_DATE LIKED DOWNLOAD PURCHASE OCCURED_ AFTER 8681411 553605400.0 2019-11-19 11:11:29 2019-10-23 00:10:13 0.0 1.0 0.0 27 days 11:01:16 46170847 550442677.0 2019-06-19 09:06:27 2018-05-22 09:05:20 0.0 1.0 0.0 393 days 00:01:07 469026688 486718552.0 2018-06-29 23:06:07 2017-07-23 16:07:31 0.0 1.0 0.0 341 days 06:58:36 6644142 6058526.0 2014-03-14 20:03:04 2013-03-21 16:03:15 0.0 0.5 0.0 358 days 03:59:49 511266472 509711880.0 2018-09-19 20:09:36 2018-02-20 12:02:32 0.0 0.5 0.0 211 days 08:07:04
13 EDA We considered two time parameters as well. ā— Action-Publish: ā—‹ If it is less than 10 days 1.5 ā—‹ If it is more than 10 days 1 ā— Action-Today: 1 + (Action Year - 2011) / 10 + Action Month * 0.025 Example: 2016-5 : 1 + (2016-2011)/10 + (5*0.025) 1.625
14 EDA USER_ID TRACK_ID C_DATE LIKED DOWNLOAD PURCHASE Action-Today Action-Publish 11128390 6625384 2014-09-23 20:09:18 0 0.5 0 1.375 1 10481467 7014550 2014-08-01 12:08:21 0 0.5 0 1.35 1 38359414 2832153 2015-06-27 17:06:34 0 0 1 1.45 1 102312590 513326911 2018-05-30 19:05:49 0 0.5 0 1.75 1.5 12904873 2855686 2016-06-04 15:06:39 0 1 0 1.55 1
Evaluation: Outline of Online and Offline Impression 15 Model Recommendation List 1- Track A 2- Track B 3- Track C Evaluate 1- Track A 2- Track B 1- Track A 2- Track B Dataset for Recommendation Dataset for Recommendation Model 1- Track A 2- Track B 3- Track C Recommendation List Compare Correct Data OnlineOffline
16 Evaluation ā— Split to train/test data using date (e.g. a year) ā—‹ From 70.000.000 action records from 2011 to 2020: ā–  Train data -> 2019 -> ~ 10.000.000 actions ā–  Test data -> 2020 -> ~ 1.000.000 actions
17 Evaluation User_ID Track_ID Impresion Duration(Normal) Price(Normal) 46170847 6034074 2.7 1.665000 1.160 469026688 6036881 1.3 0.331667 1.245 Implicit Explicit
18 Evaluation ā— Evaluate the model by computing the Mean Absolute Error (MAE) on the test data
19 Evaluation: RegressionEvaluator (pyspark.ml.evaluation)
20 Model: Model Based CF ā— Compute a Correlation Score for every column pair in the matrix ā— This gives us a Correlation Score between every pair of track ā— Too long to compute ā— Sparseness ā— Scalability
21 Model: Memory Based CF MLlib ā— classiļ¬cation: logistic regression, linear SVM, naive bayes ā— regression ā— clustering: k-means ā— collaborative ļ¬ltering: alternating least squares (ALS)
22
23 Model: Matrix Factorization of User-Item Matrix 4.5 2.0 4.0 3.5 5.0 2.0 3.5 4.0 1.0 User Item 1.2 0.8 1.4 0.9 1.5 1.0 1.2 0.8 1.5 1.2 1.0 0.8 1.7 0.6 1.1 0.4 = x User Matrix Item Matrix W X Y Z A B C D W X Y ZA B C D
24 Model: Matrix Factorization of User-Item Matrix ā— Latent factors are the features in the lower dimension latent space projected from user-item interaction matrix. ā— Matrix factorization is one of very eļ¬€ective dimension reduction techniques in machine learning.
25 Model: ALS Alternating Least Square (ALS) is also a matrix factorization algorithm and it runs itself in a parallel fashion.
26 Model: ALS ā— Solve scalability and sparseness of the Ratings data ā— Itā€™s simple and scales well to very large datasets
27 Optimization: Grid Search to Find Model Best Parameters Latent Factors Regularization Max Iterations MAE 50 0.1 10 0.7106074619 50 0.15 10 0.7042538296 50 0.2 10 0.7041747462 50 0.25 10 0.7087336317 100 0.1 10 0.7076845943 100 0.15 10 0.7036464454 100 0.2 10 0.7048896137 100 0.25 10 0.7097821267 150 0.1 10 0.7076660709 150 0.15 10 0.7035191506 150 0.2 10 0.704869857 150 0.25 10 0.7097177331 200 0.1 10 0.7077100207 ā†’ Best Parameters
28 Optimization: Evaluation Content-Based Model Similar Tracks Collaborative Filtering User History Expected Tracks User Recs Compare
29 Optimization: EvaluationOptimization: Evaluation Result ā— Using 2019 data as train ā†’ 10M ā— Using 2020 data as test ā†’ 1M ā— Running on the Old Result: ā—‹ Total Users: 116526 ā—‹ Mean Score: 0.01% ā—‹ Max Score: 40% ā— Using all data ā†’ 70M ā— Add date coeļ¬ƒcients ā— Finding Best Parameters ā— Running on the New Result: ā—‹ Total Users: 577457 ā—‹ Mean Score: 1.1% ā—‹ Max Score: 50% x110 improvement based on new data
30 Recommender Systems Content Based Collaborative Filtering Memory BasedModel Based
31 EDA: Tracks Collection TRACK_ID TIME_CREATED PRICE PUBLISH_DATE ALBUM_ID duration TYPE_KEY_CURATION TYPE_KEY_GENR E TAG_ID_37995085 1 TAG_ID_37995085 2 6034074 12/21/13 18:12 9990 10/23/13 16:10 6032170 232 0 1 0 1 6036881 12/22/13 15:12 1990 10/23/13 16:10 6012439 249 0 1 0 0 6037213 12/22/13 17:12 0 10/23/13 16:10 2828262 192 1 1 0 0 6049227 12/25/13 15:12 0 3/21/13 16:03 6048970 203 0 1 0 0 6059662 12/28/13 15:12 8990 1/1/12 1:01 6059612 549 0 1 1 0
32 Model: Content-Based Filtering ā— Item proļ¬le for each track we should construct a vector based on itā€™s features like tags and artists it has ā— User proļ¬le for each user we need a vector that shows his interests based on ratings or likes and downloads
33 Model: Content-Based Filtering/Item Proļ¬le 0 1 1 0 0 Item vector Feature 1 Feature 2 Feature 3 Feature 4 Feature 5
34 Model: Content-Based Filtering/User proļ¬le ā— User has rated items with proļ¬les i1 , i2, i3, ... , in ā— One approach is weighted average of rated item proļ¬les
35 Model: Content-Based Filtering/User proļ¬le ā— Items are songs, only feature is ā€œtagā€ ā— Item proļ¬le: vector with 0 or 1 for each Actor ā— Suppose user x has downloaded or liked 5 songs ā— 2 songs featuring TAG A ā— 3 songs featuring TAG B ā— User proļ¬le = mean of item proļ¬les ā— Feature Aā€™s weight = 2/5 = 0.4 ā— Feature Bā€™s weight = 3/5 = 0.6
36 Model: Content-Based Filtering/User proļ¬le 0.4 0.6 0 0 ... User vector Tag A Tag B Tag C Tag D ...
37 Model : Content-Based Filtering/Cosine similarity ā— Estimate U(x,i) = cos(Īø) = (x . i)/( |x| |i| ) Īø
38 Pros ā— No need for data on OTHER users ā— Able to recommend to users with unique tastes ā— Able to recommend new & unpopular items ā—‹ No ļ¬rst-rater problem ā— Explanations for recommended items
39 Cons ā— Finding the appropriate features is hard ā— Overspecialization ā—‹ Never recommends items outside userā€™s content proļ¬le
40 Evaluation: Sample Track Homayoun Shajarian & Alireza Ghorbani ā€œAfsane Chashmhayatā€ Genre: Persian Traditional Music
41 Evaluation: Similar Tracks Found by Content-Based Model ā— Homayoun Shajarian - ā€œJana Be Negahiā€ ā— Gholam-Hossein Banan - ā€œShart e Rahā€ Ballad ā— Mohammad-Reza Shajarian - ā€œAh Baranā€ ā— Alireza Eftekhari - ā€œAsiriā€ Ballad ā— Gholam-Hossein Banan - ā€œMeykade Arezooā€ Ballad ā— Salar Aghili - ā€œDaghe Jodaeiā€ ā— Homayoun Shajarian - ā€œBe Tamashaye Negahatā€ Genre: Persian Traditional Music & Ballad + Same Artists in Some Items
Spark Job 42 #!/bin/bash HOME=/home/rc12g2 HADOOP_HOME=/user/rc12g2 source $HOME/.bashrc echo "spark job -> started" hadoop fs -rm -r -f $HADOOP_HOME/final-actions-v3 spark-submit $HOME/beeptunes_recsys/collaborative-filtering/final-action-aggregator.py > $HOME/log/spark.log hadoop fs -getmerge $HADOOP_HOME/user_recs_v3 $HOME/result/user_recs_v3.csv hadoop fs -rm -r -f $HADOOP_HOME/user_recs_v3 sed -i '1USER_ID,RECOMMENDATION_IDS' $HOME/result/user_recs_v3.csv mongoimport --port 28018 -u 4max -p HFmk87Q3DgfEKKgC --type csv -d g2recsys -c user_rec --headerline --drop $HOME/result/user_recs_v3.csv echo "spark run -> complete"
Track Collection Update Job 43 #!/bin/bash HOME=/home/rc12g2 echo "update_track_collection job -> started" cd $HOME/beeptunes_recsys/jobs mongoexport --port 28018 -u 4max -p HFmk87Q3DgfEKKgC --type csv -d g2recsys -c tracks --fieldFile tracks_fields.txt --out ./export.csv > $HOME/log/update_track_collection.log python update_trackCollection.py mongo --port 28018 -u 4max -p HFmk87Q3DgfEKKgC g2recsys --eval 'db.tracks.drop()' rm $HOME/beeptunes_recsys/jobs/export.csv echo "update_track_collection job -> ended"
Crontab 44 rc12g2@BIHADOOP-Master1:~$ crontab -l # m h dom mon dow command 0 2 1 * * /home/beeptunes_recsys/jobs/trackUpdate_job.sh 0 2 1 * * /home/beeptunes_recsys/jobs/spark_job.sh
Cold Start Problem? 45
Hybrid Model 46 /user/<user_id>/recommend USER_ID Entered Recommender System Trends** Collection Collaborative Filtering Content-Based User Exists User Doesnā€™t Exists <16* actions >=16* actions *25% of users have less than 16 actions **Top 30 Tracks with Most Actions in the last 3 months
Hybrid Model 47 /track/<track_id>/similars TRACK_ID Entered Recommender System Trends Collection Content-Based Track Exists Track Doesnā€™t Exists Extract Similars
48
49
Niloufar Farajpour Mohamadreza Fereydooni Mohamadreza Kiani Tadeh Alexani Our Great Team! 50
Thank You! 51

More Related Content

PDF
Recommender Systems
byCarlos Castillo (ChaTo)
Ā 
PPTX
Recommenders Systems
byTariq Hassan
Ā 
PDF
Recommender Systems
byFrancesco Casalegno
Ā 
PDF
IntroductionRecommenderSystems_Petroni.pdf
byAlphaIssaghaDiallo
Ā 
PDF
Demystifying Recommendation Systems
byRumman Chowdhury
Ā 
PDF
ESSIR 2013 Recommender Systems tutorial
byAlexandros Karatzoglou
Ā 
PDF
IRJET- A Personalized Music Recommendation System
byIRJET Journal
Ā 
PDF
A survey of memory based methods for collaborative filtering based techniques
byIAEME Publication
Ā 
Recommender Systems
byCarlos Castillo (ChaTo)
Ā 
Recommenders Systems
byTariq Hassan
Ā 
Recommender Systems
byFrancesco Casalegno
Ā 
IntroductionRecommenderSystems_Petroni.pdf
byAlphaIssaghaDiallo
Ā 
Demystifying Recommendation Systems
byRumman Chowdhury
Ā 
ESSIR 2013 Recommender Systems tutorial
byAlexandros Karatzoglou
Ā 
IRJET- A Personalized Music Recommendation System
byIRJET Journal
Ā 
A survey of memory based methods for collaborative filtering based techniques
byIAEME Publication
Ā 

Similar to BeepTunes Music Recommender System

PDF
A Content Boosted Hybrid Recommendation System
bySeval Ƈapraz
Ā 
PPTX
Recommender systems: Content-based and collaborative filtering
byViet-Trung TRAN
Ā 
PPTX
Recommender systems for E-commerce
byAlexander Konduforov
Ā 
PPT
FCI filtering class in Social on Media.ppt
byamritasharma278295
Ā 
PDF
Recommendation Systems
byCenk Bircanoğlu
Ā 
PDF
Architecting AI Solutions in Azure for Business
byIvo Andreev
Ā 
PPT
Introduction to recommendation system
byAravindharamanan S
Ā 
PDF
Modern Perspectives on Recommender Systems and their Applications in Mendeley
byKris Jack
Ā 
PDF
productionising-recommenders
byLudovik Coba
Ā 
PDF
Recommendation engines
byGeorgian Micsa
Ā 
PPTX
Recommender Systems: Advances in Collaborative Filtering
byChangsung Moon
Ā 
PDF
Real-world News Recommender Systems
bykib_83
Ā 
PPTX
Offer recommendation methodology
byDipesh Patel
Ā 
PPTX
Collaborative Filtering Recommendation System
byMilind Gokhale
Ā 
PDF
Introduction to Recommendation Systems
byZia Babar
Ā 
PDF
Recommendations 101
byEsh Vckay
Ā 
PDF
Recommendation systems
byAnton Ermak
Ā 
PPT
Filtering content bbased crs
byAravindharamanan S
Ā 
PPTX
Offer Recommendation methodology for Vito's Mobile App
byDipesh Patel
Ā 
PDF
Music recommendations @ MLConf 2014
byErik Bernhardsson
Ā 
A Content Boosted Hybrid Recommendation System
bySeval Ƈapraz
Ā 
Recommender systems: Content-based and collaborative filtering
byViet-Trung TRAN
Ā 
Recommender systems for E-commerce
byAlexander Konduforov
Ā 
FCI filtering class in Social on Media.ppt
byamritasharma278295
Ā 
Recommendation Systems
byCenk Bircanoğlu
Ā 
Architecting AI Solutions in Azure for Business
byIvo Andreev
Ā 
Introduction to recommendation system
byAravindharamanan S
Ā 
Modern Perspectives on Recommender Systems and their Applications in Mendeley
byKris Jack
Ā 
productionising-recommenders
byLudovik Coba
Ā 
Recommendation engines
byGeorgian Micsa
Ā 
Recommender Systems: Advances in Collaborative Filtering
byChangsung Moon
Ā 
Real-world News Recommender Systems
bykib_83
Ā 
Offer recommendation methodology
byDipesh Patel
Ā 
Collaborative Filtering Recommendation System
byMilind Gokhale
Ā 
Introduction to Recommendation Systems
byZia Babar
Ā 
Recommendations 101
byEsh Vckay
Ā 
Recommendation systems
byAnton Ermak
Ā 
Filtering content bbased crs
byAravindharamanan S
Ā 
Offer Recommendation methodology for Vito's Mobile App
byDipesh Patel
Ā 
Music recommendations @ MLConf 2014
byErik Bernhardsson
Ā 

Recently uploaded

PDF
KMWorld - KM & AI Bring Collectivity, Nostalgia, & Selectivity
byJonathan Ralton
Ā 
PDF
Agentic Intro and Hands-on: Build your first Coded Agent
byUiPathCommunity
Ā 
PDF
[DevFest Strasbourg 2025] - NodeJs Can do that !!
bymoassiongbon
Ā 
PDF
Cheryl Hung, Vibe Coding Auth Without Melting Down! isaqb Software Architectu...
byCheryl Hung
Ā 
PDF
[BDD 2025 - Full-Stack Development] PHP in AI Age: The Laravel Way. (Rizqy Hi...
byWintari Yasmin
Ā 
PDF
So You Want to Work at Google | DevFest Seattle 2025
byMargaret Maynard-Reid
Ā 
PPTX
Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]
byUiPathCommunity
Ā 
PDF
Transforming Supply Chains with Amazon Bedrock AgentCore (AWS Swiss User Grou...
byChris Bingham
Ā 
PDF
Open Source Post-Quantum Cryptography - Matt Caswell
byAll Things Open
Ā 
PDF
"DISC as GPS for team leaders: how to lead a team from storming to performing...
byFwdays
Ā 
PDF
Integrating AI with Meaningful Human Collaboration
byCadabra Studio
Ā 
PPTX
Leon Brands - Intro to GPU Occlusion (Graphics Programming Conference 2024)
byleonbrands1998
Ā 
PPTX
Guardrails in Action - Ensuring Safe AI with Azure AI Content Safety.pptx
byUdaiappa Ramachandran
Ā 
PDF
Top Crypto Supers 15th Report November 2025
byStephen Perrenod
Ā 
PDF
The partnership effect: Libraries and publishers on collaborating and thrivin...
byBookNet Canada
Ā 
PDF
The Evolving Role of the CEO in the Age of AI
byPipal Tree Services Executive Search Firm
Ā 
PDF
DUBAI IT MODERNIZATION WITH AZURE MANAGED SERVICES.pdf
byLogicEra
Ā 
PDF
Crane Accident Prevention Guide: Key OSHA Regulations for Safer Operations
bySharpEagle Technology
Ā 
PDF
Transcript: The partnership effect: Libraries and publishers on collaborating...
byBookNet Canada
Ā 
PDF
5 Common Supply Chain Attacks and How They Work | CyberPro Magazine
byCyberPro Magazine
Ā 
KMWorld - KM & AI Bring Collectivity, Nostalgia, & Selectivity
byJonathan Ralton
Ā 
Agentic Intro and Hands-on: Build your first Coded Agent
byUiPathCommunity
Ā 
[DevFest Strasbourg 2025] - NodeJs Can do that !!
bymoassiongbon
Ā 
Cheryl Hung, Vibe Coding Auth Without Melting Down! isaqb Software Architectu...
byCheryl Hung
Ā 
[BDD 2025 - Full-Stack Development] PHP in AI Age: The Laravel Way. (Rizqy Hi...
byWintari Yasmin
Ā 
So You Want to Work at Google | DevFest Seattle 2025
byMargaret Maynard-Reid
Ā 
Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]
byUiPathCommunity
Ā 
Transforming Supply Chains with Amazon Bedrock AgentCore (AWS Swiss User Grou...
byChris Bingham
Ā 
Open Source Post-Quantum Cryptography - Matt Caswell
byAll Things Open
Ā 
"DISC as GPS for team leaders: how to lead a team from storming to performing...
byFwdays
Ā 
Integrating AI with Meaningful Human Collaboration
byCadabra Studio
Ā 
Leon Brands - Intro to GPU Occlusion (Graphics Programming Conference 2024)
byleonbrands1998
Ā 
Guardrails in Action - Ensuring Safe AI with Azure AI Content Safety.pptx
byUdaiappa Ramachandran
Ā 
Top Crypto Supers 15th Report November 2025
byStephen Perrenod
Ā 
The partnership effect: Libraries and publishers on collaborating and thrivin...
byBookNet Canada
Ā 
The Evolving Role of the CEO in the Age of AI
byPipal Tree Services Executive Search Firm
Ā 
DUBAI IT MODERNIZATION WITH AZURE MANAGED SERVICES.pdf
byLogicEra
Ā 
Crane Accident Prevention Guide: Key OSHA Regulations for Safer Operations
bySharpEagle Technology
Ā 
Transcript: The partnership effect: Libraries and publishers on collaborating...
byBookNet Canada
Ā 
5 Common Supply Chain Attacks and How They Work | CyberPro Magazine
byCyberPro Magazine
Ā 

BeepTunes Music Recommender System

  • 1.
    Building a Music Recommender Systemfrom Scratch on the Beep Tunes Dataset Niloufar Farajpour, Mohamadreza Kiani, Mohamadreza Fereydooni, Tadeh Alexani 1 Rahnema College - Winter 2020
  • 2.
    Frank Kane As adata scientist, question the results, because, often there is something you missed. 2
  • 3.
  • 4.
    Approaches ā— Manual Curationby Experts ā— Editorial Tagging ā— Audio Signals ā— Recommender Systems 4
  • 5.
    5 Recommender Systems Content BasedCollaborative Filtering Memory BasedModel Based
  • 6.
    6 CollaborativeFiltering(CF) Model Based Memory Based Findsimilar users based on cosine similarity or pearson correlation and take weighted avg. of ratings Use machine learning to ļ¬nd user ratings of unrated items. e.g. PCA, SVD, Neural Nets, Matrix Factorization Advantage Easy creation and explainability of results Disadvantage Performance Reduces when data is sparse. So, non scalable Advantage Dimensionality reduction deals with missing/ sparse data Disadvantage Inference is intractable because of hidden/ latent factors
  • 7.
    7 User-based vs Item-based Userswho are similar to you also liked ā€¦ Users who liked this item also liked ā€¦
  • 8.
    8 EDA ā— GivenDataset: ā—‹ album_like.csv ā—‹ album_track_purchase.csv ā—‹ album_artist.info ā—‹ download_album.csv ā—‹ artist_like.csv ā—‹ track_download.csv ā—‹ track_like.csv ā—‹ track_info.csv ā—‹ track_tag.csv ā—‹ track_artist.csv
  • 9.
  • 10.
    10 EDA USER_ID TRACK_ID C_DATEPUBLISH_DATE LIKED DOWNLOAD PURCHASE 8681411 553605400.0 2019-11-19 11:11:29 2019-10-23 00:10:13 0.0 1.0 0.0 46170847 550442677.0 2019-06-19 09:06:27 2018-05-22 09:05:20 0.0 1.0 0.0 469026688 486718552.0 2018-06-29 23:06:07 2017-07-23 16:07:31 0.0 1.0 0.0 6644142 6058526.0 2014-03-14 20:03:04 2013-03-21 16:03:15 0.0 0.5 0.0 511266472 509711880.0 2018-09-19 20:09:36 2018-02-20 12:02:32 0.0 0.5 0.0 ActionCollection
  • 11.
    11 EDA Impression = (Like*4) +(Download*2) + (Purchase*1) Based on the inverse of like, download, purchase frequency among all actions
  • 12.
    12 EDA Downloaded Tracks Purchased Tracks Likes USER_ID TRACK_ID C_DATEPUBLISH_DATE LIKED DOWNLOAD PURCHASE OCCURED_ AFTER 8681411 553605400.0 2019-11-19 11:11:29 2019-10-23 00:10:13 0.0 1.0 0.0 27 days 11:01:16 46170847 550442677.0 2019-06-19 09:06:27 2018-05-22 09:05:20 0.0 1.0 0.0 393 days 00:01:07 469026688 486718552.0 2018-06-29 23:06:07 2017-07-23 16:07:31 0.0 1.0 0.0 341 days 06:58:36 6644142 6058526.0 2014-03-14 20:03:04 2013-03-21 16:03:15 0.0 0.5 0.0 358 days 03:59:49 511266472 509711880.0 2018-09-19 20:09:36 2018-02-20 12:02:32 0.0 0.5 0.0 211 days 08:07:04
  • 13.
    13 EDA We considered twotime parameters as well. ā— Action-Publish: ā—‹ If it is less than 10 days 1.5 ā—‹ If it is more than 10 days 1 ā— Action-Today: 1 + (Action Year - 2011) / 10 + Action Month * 0.025 Example: 2016-5 : 1 + (2016-2011)/10 + (5*0.025) 1.625
  • 14.
    14 EDA USER_ID TRACK_ID C_DATELIKED DOWNLOAD PURCHASE Action-Today Action-Publish 11128390 6625384 2014-09-23 20:09:18 0 0.5 0 1.375 1 10481467 7014550 2014-08-01 12:08:21 0 0.5 0 1.35 1 38359414 2832153 2015-06-27 17:06:34 0 0 1 1.45 1 102312590 513326911 2018-05-30 19:05:49 0 0.5 0 1.75 1.5 12904873 2855686 2016-06-04 15:06:39 0 1 0 1.55 1
  • 15.
    Evaluation: Outline ofOnline and Offline Impression 15 Model Recommendation List 1- Track A 2- Track B 3- Track C Evaluate 1- Track A 2- Track B 1- Track A 2- Track B Dataset for Recommendation Dataset for Recommendation Model 1- Track A 2- Track B 3- Track C Recommendation List Compare Correct Data OnlineOffline
  • 16.
    16 Evaluation ā— Split totrain/test data using date (e.g. a year) ā—‹ From 70.000.000 action records from 2011 to 2020: ā–  Train data -> 2019 -> ~ 10.000.000 actions ā–  Test data -> 2020 -> ~ 1.000.000 actions
  • 17.
    17 Evaluation User_ID Track_ID ImpresionDuration(Normal) Price(Normal) 46170847 6034074 2.7 1.665000 1.160 469026688 6036881 1.3 0.331667 1.245 Implicit Explicit
  • 18.
    18 Evaluation ā— Evaluate themodel by computing the Mean Absolute Error (MAE) on the test data
  • 19.
  • 20.
    20 Model: Model BasedCF ā— Compute a Correlation Score for every column pair in the matrix ā— This gives us a Correlation Score between every pair of track ā— Too long to compute ā— Sparseness ā— Scalability
  • 21.
    21 Model: Memory BasedCF MLlib ā— classiļ¬cation: logistic regression, linear SVM, naive bayes ā— regression ā— clustering: k-means ā— collaborative ļ¬ltering: alternating least squares (ALS)
  • 22.
  • 23.
    23 Model: Matrix Factorizationof User-Item Matrix 4.5 2.0 4.0 3.5 5.0 2.0 3.5 4.0 1.0 User Item 1.2 0.8 1.4 0.9 1.5 1.0 1.2 0.8 1.5 1.2 1.0 0.8 1.7 0.6 1.1 0.4 = x User Matrix Item Matrix W X Y Z A B C D W X Y ZA B C D
  • 24.
    24 Model: Matrix Factorizationof User-Item Matrix ā— Latent factors are the features in the lower dimension latent space projected from user-item interaction matrix. ā— Matrix factorization is one of very eļ¬€ective dimension reduction techniques in machine learning.
  • 25.
    25 Model: ALS Alternating LeastSquare (ALS) is also a matrix factorization algorithm and it runs itself in a parallel fashion.
  • 26.
    26 Model: ALS ā— Solvescalability and sparseness of the Ratings data ā— Itā€™s simple and scales well to very large datasets
  • 27.
    27 Optimization: Grid Searchto Find Model Best Parameters Latent Factors Regularization Max Iterations MAE 50 0.1 10 0.7106074619 50 0.15 10 0.7042538296 50 0.2 10 0.7041747462 50 0.25 10 0.7087336317 100 0.1 10 0.7076845943 100 0.15 10 0.7036464454 100 0.2 10 0.7048896137 100 0.25 10 0.7097821267 150 0.1 10 0.7076660709 150 0.15 10 0.7035191506 150 0.2 10 0.704869857 150 0.25 10 0.7097177331 200 0.1 10 0.7077100207 ā†’ Best Parameters
  • 28.
    28 Optimization: Evaluation Content-Based ModelSimilar Tracks Collaborative Filtering User History Expected Tracks User Recs Compare
  • 29.
    29 Optimization: EvaluationOptimization: EvaluationResult ā— Using 2019 data as train ā†’ 10M ā— Using 2020 data as test ā†’ 1M ā— Running on the Old Result: ā—‹ Total Users: 116526 ā—‹ Mean Score: 0.01% ā—‹ Max Score: 40% ā— Using all data ā†’ 70M ā— Add date coeļ¬ƒcients ā— Finding Best Parameters ā— Running on the New Result: ā—‹ Total Users: 577457 ā—‹ Mean Score: 1.1% ā—‹ Max Score: 50% x110 improvement based on new data
  • 30.
    30 Recommender Systems Content BasedCollaborative Filtering Memory BasedModel Based
  • 31.
    31 EDA: Tracks Collection TRACK_IDTIME_CREATED PRICE PUBLISH_DATE ALBUM_ID duration TYPE_KEY_CURATION TYPE_KEY_GENR E TAG_ID_37995085 1 TAG_ID_37995085 2 6034074 12/21/13 18:12 9990 10/23/13 16:10 6032170 232 0 1 0 1 6036881 12/22/13 15:12 1990 10/23/13 16:10 6012439 249 0 1 0 0 6037213 12/22/13 17:12 0 10/23/13 16:10 2828262 192 1 1 0 0 6049227 12/25/13 15:12 0 3/21/13 16:03 6048970 203 0 1 0 0 6059662 12/28/13 15:12 8990 1/1/12 1:01 6059612 549 0 1 1 0
  • 32.
    32 Model: Content-Based Filtering ā—Item proļ¬le for each track we should construct a vector based on itā€™s features like tags and artists it has ā— User proļ¬le for each user we need a vector that shows his interests based on ratings or likes and downloads
  • 33.
    33 Model: Content-Based Filtering/ItemProļ¬le 0 1 1 0 0 Item vector Feature 1 Feature 2 Feature 3 Feature 4 Feature 5
  • 34.
    34 Model: Content-Based Filtering/Userproļ¬le ā— User has rated items with proļ¬les i1 , i2, i3, ... , in ā— One approach is weighted average of rated item proļ¬les
  • 35.
    35 Model: Content-Based Filtering/Userproļ¬le ā— Items are songs, only feature is ā€œtagā€ ā— Item proļ¬le: vector with 0 or 1 for each Actor ā— Suppose user x has downloaded or liked 5 songs ā— 2 songs featuring TAG A ā— 3 songs featuring TAG B ā— User proļ¬le = mean of item proļ¬les ā— Feature Aā€™s weight = 2/5 = 0.4 ā— Feature Bā€™s weight = 3/5 = 0.6
  • 36.
    36 Model: Content-Based Filtering/Userproļ¬le 0.4 0.6 0 0 ... User vector Tag A Tag B Tag C Tag D ...
  • 37.
    37 Model : Content-BasedFiltering/Cosine similarity ā— Estimate U(x,i) = cos(Īø) = (x . i)/( |x| |i| ) Īø
  • 38.
    38 Pros ā— No needfor data on OTHER users ā— Able to recommend to users with unique tastes ā— Able to recommend new & unpopular items ā—‹ No ļ¬rst-rater problem ā— Explanations for recommended items
  • 39.
    39 Cons ā— Finding theappropriate features is hard ā— Overspecialization ā—‹ Never recommends items outside userā€™s content proļ¬le
  • 40.
    40 Evaluation: Sample Track HomayounShajarian & Alireza Ghorbani ā€œAfsane Chashmhayatā€ Genre: Persian Traditional Music
  • 41.
    41 Evaluation: Similar TracksFound by Content-Based Model ā— Homayoun Shajarian - ā€œJana Be Negahiā€ ā— Gholam-Hossein Banan - ā€œShart e Rahā€ Ballad ā— Mohammad-Reza Shajarian - ā€œAh Baranā€ ā— Alireza Eftekhari - ā€œAsiriā€ Ballad ā— Gholam-Hossein Banan - ā€œMeykade Arezooā€ Ballad ā— Salar Aghili - ā€œDaghe Jodaeiā€ ā— Homayoun Shajarian - ā€œBe Tamashaye Negahatā€ Genre: Persian Traditional Music & Ballad + Same Artists in Some Items
  • 42.
    Spark Job 42 #!/bin/bash HOME=/home/rc12g2 HADOOP_HOME=/user/rc12g2 source $HOME/.bashrc echo"spark job -> started" hadoop fs -rm -r -f $HADOOP_HOME/final-actions-v3 spark-submit $HOME/beeptunes_recsys/collaborative-filtering/final-action-aggregator.py > $HOME/log/spark.log hadoop fs -getmerge $HADOOP_HOME/user_recs_v3 $HOME/result/user_recs_v3.csv hadoop fs -rm -r -f $HADOOP_HOME/user_recs_v3 sed -i '1USER_ID,RECOMMENDATION_IDS' $HOME/result/user_recs_v3.csv mongoimport --port 28018 -u 4max -p HFmk87Q3DgfEKKgC --type csv -d g2recsys -c user_rec --headerline --drop $HOME/result/user_recs_v3.csv echo "spark run -> complete"
  • 43.
    Track Collection UpdateJob 43 #!/bin/bash HOME=/home/rc12g2 echo "update_track_collection job -> started" cd $HOME/beeptunes_recsys/jobs mongoexport --port 28018 -u 4max -p HFmk87Q3DgfEKKgC --type csv -d g2recsys -c tracks --fieldFile tracks_fields.txt --out ./export.csv > $HOME/log/update_track_collection.log python update_trackCollection.py mongo --port 28018 -u 4max -p HFmk87Q3DgfEKKgC g2recsys --eval 'db.tracks.drop()' rm $HOME/beeptunes_recsys/jobs/export.csv echo "update_track_collection job -> ended"
  • 44.
    Crontab 44 rc12g2@BIHADOOP-Master1:~$ crontab -l #m h dom mon dow command 0 2 1 * * /home/beeptunes_recsys/jobs/trackUpdate_job.sh 0 2 1 * * /home/beeptunes_recsys/jobs/spark_job.sh
  • 45.
  • 46.
    Hybrid Model 46 /user/<user_id>/recommend USER_ID Entered Recommender System Trends** Collection Collaborative Filtering Content-Based UserExists User Doesnā€™t Exists <16* actions >=16* actions *25% of users have less than 16 actions **Top 30 Tracks with Most Actions in the last 3 months
  • 47.
  • 48.
  • 49.
  • 50.
  • 51.