Project in the context of data mining exam at Sapienza university of Rome. Application of softmax regression for movie market purposes

1. Software Suite for Movie Market Analysis
Daniele De Cillis, Dario Molinari, Lorenzo Vitali
Data Mining Project

2. Contact and references
Github :
https://github.com/93lorenzo
/software-suite-movie-market-analysis
SlideShare :
https://www.slideshare.net/dariospin93/soft
ware-suite-for-movie-market-analysis
Our Contacts :
93lorenzo@gmail.com
dariospin93@gmail.com
daniele93d@gmail.com
Pagina 2

3. Pagina 3
Table of contents
1. Introduction
2. Retrieving Data
3. Graphic Trends
4. Data Analysis
5. Model Description
6. Experiments and Results

4. Pagina 4
Introduction - Purpose of the System
• Softwares to help
companies making
money with movies
• Analysis of past trends
to figure out what is the
best commercial move to
do
• Cross-search to find the
best cast
• Machine learning to
predict how much
popular the movie will be

5. Retrieving
data
• IMDb
• BoxOfficeMojo
Graphic
trends
• Gross
• Number
of movies
Analytics
• Best actors
• Best movie for
genre
Prediction
• Classifier
with high
confidence
Pagina 5
Introduction
Genre
Actors
Writers
Directors
Dataset

6. Pagina 6
Retrieving Data

7. Pagina 7
Dataset: retrieving data
• OMDb: web service to obtain movie information through API in
JSON format, linked with IMDb
• Grosses obtained from Box Office Mojo
• IMDb is the most famous and complete database for movies
• The total number of the movies is 16647

8. Pagina 8
Dataset: retrieving data

9. Pagina 9
Analytics - Graphic Trends

10. Time series
• Sequence of discrete time data
• Temporally ordered, usually equally spaced
• In investing, the variation of data points is studied to predict
future values, and understand possible trends
Pagina 10

11. Predicting trends - ARIMA
• Stochastic method to predict time series
• Mean Squared Error to estimate p,d,q
Parameters :
• p: how many past values will affect the output
• d: differencing steps
• q: past values forecast
errors terms in the
prediction equation
Pagina 11

12. Pagina 12
Movie Trends: drama forecast

13. Pagina 13
Movie Trends: drama gross forecast

14. Pagina 14
Movie Trends: thriller forecast

15. Pagina 15
Movie Trends: thriller gross forecast

16. Pagina 16
Analytics - Cast Selection

17. Pagina 17
Cast selection
• Find best actor or most popular movies for a particular genre
• Most popular movies by genre
• Movies selected weighting their user ratings
• Best actor for genre
• Actors selected weighting their ratings
• Rating actor = mean score of all movies heshe took part

18. GUI for Cast Selection
Pagina 18

19. Pagina 19
Prediction

20. Pagina 20
Prediction
The main application of this project is a smart system able to figure out
if the movie given in input will be a box office success or a flop.
The features that leads to a classification are movie information
like cast, directors, production, genre and so on.

21. Pagina 21
Prediction
• The classifier used is a Softmax Regression
• Generalization of Logistic Regression
• It assigns probabilities to every class for every observation
• There are five classes, that can be seen as:
1. Flop
2. Poor
3. Average
4. Good
5. Success

22. Pagina 22
Model Description
Cost Function:
𝐽 𝜃 = −
𝑖=1
𝑚
𝑘=1
𝐾
1 𝑦(𝑖) = 𝑘 log
exp 𝜃 𝑘 𝑇 𝑥(𝑖)
𝑗=1
𝐾
𝑒𝑥𝑝 𝜃 𝑗 𝑇 𝑥(𝑖)
Hypotesis:
𝑃 𝑦 𝑖 = 𝑘 𝑥 𝑖 ; 𝜃 =
exp 𝜃 𝑘 𝑇
𝑥(𝑖)
𝑗=1
𝐾
𝑒𝑥𝑝 𝜃 𝑗 𝑇 𝑥(𝑖)

23. Pagina 23
Experiments

24. Pagina 24
Experiments
• After the training procedure, we obtain an accuracy of about 82%
• When the guess is wrong, the error is often minimal (only 1 class)
• If you want to see a movie, ask us!

25. Pagina 25
Experiments - Confusion matrix

26. Pagina 26
Experiments: Stress tests

27. Pagina 27
Experiments: Azure Machine Learning

28. Pagina 28
Experiments: Comparison

29. Pagina 29
In Conclusion...

30. Pagina 30