The document discusses using data science to predict Oscar winners. It outlines the data science process of framing a question about predicting Best Picture winners, collecting financial and review data on films, cleaning and formatting the data, then using machine learning techniques like decision trees and random forests to make predictions and explore patterns in the data. Basic tools for data science like Jupyter notebooks, NumPy, Pandas and Scikit-learn are introduced. The models were able to accurately predict most Best Picture winners except for 2017. The document also provides information about an online program for learning data science.

1. Predicting the Oscars with Data Science
http://bit.ly/tf-predict-oscars

2. About me
• Jasjit Singh
• Self-taught developer
• Worked in finance & tech
• Co-Founder Hotspot
• Thinkful General Manager

3. About us
Thinkful prepares students for web development &
data science jobs with 1-on-1 mentorship programs

4. What’s your background?
• I have a software background
• I have a math or stats background
• None of the above

5. Data Science Process
• Frame the question.
• Collect the raw data.
• Process the data.
• Explore the data.
• Communicate results.

6. Frame the question
• Who will win the Oscar for Best Picture?

7. Collect the Data
• What kind of data do we need?
• Financial data (Budget, box office…)
• Reviews, ratings and scores.
• Awards and nominations.

8. Process the data
• How’s the data “dirty” and how can we fix it?
• User input, redundancies, missing data…
• Formatting: adapt the data to meet certain
specifications.
• Cleaning: detecting and correcting
corrupt or inaccurate records.

9. Explore the data
• What are the meaningful patterns in the
data?
• How meaningful is each data point for our
predictions?

10. Goals
• Introduction to a data scientist's tools and
methods:
• Jupyter notebooks, numpy, pandas,
sklearn…
• Overview of basic machine learning
concepts:
• Data formatting and cleaning, Decision
trees, Overfitting, Random Forests…

11. Jupyter Notebooks
• One of data scientist’s everyday tools.
• Find the links in our classroom tool.
• Contains cells with code.

12. NumPy
• The fundamental package for scientific
computing with Python.
• Provides powerful multi-dimensional array
objects.
• Many methods for fast operations on arrays.

13. Pandas
• Fundamental high-level building block for
doing practical, real world data analysis in
Python.
• Built on top of NumPy.
• Offers data structures and operations for
manipulating numerical tables and time
series.

14. Scikit-learn
• Python module for machine learning.
• Provides a large menu of libraries for
scientific computation, such as integration,
interpolation, signal processing, linear
algebra, statistics, etc.

15. Initial imports and loading data with Pandas

16. Understanding your data
• .head(n) method: Returns first n rows.
• .value_counts() method: Returns the counts
of unique values in the DataFrame.

17. Formatting your Data

18. Formatting your Data
• Rate values in a non-numeric format. Thus,
we will need to assign each rate a unique
integer so that Python can handle the
information.
• With the .ix method you create a subset of
rows and assign a value to a certain variable
of that subset of observations.

19. Cleaning your Data

20. Decision Trees
• It breaks down a dataset into smaller and
smaller subsets.
• The final result is a model with a tree
structure that has:
• Decision nodes: ask a question and have
two or more branches.
• Leaf nodes: represent a classification or
decision.

22. Classification vs Regression
• Classification — Predict categories.
• Identifying group membership.
• Regression — Predict values.
• Involves estimating or predicting a
response.

23. Classification

24. Classification
?

25. Creating your first Decision Tree
You will use the scikit-learn and numpy
libraries to build your first decision tree. We
will need the following to build a decision tree
• target: A one-dimensional numpy array
containing the target from the train data.
• features: A multidimensional numpy array
containing the features/predictors from the
train data.

26. Creating your first Decision Tree

27. Importances and Score
• .feature_importances_ attribute: tells us
how important the features are for the final
result.
• .score() method: returns the mean accuracy
of our fitting.

28. Importances and Score

29. Predicting

30. Pretty bad results :(
Let’s improve it!

31. Let’s improve it!

32. Modify the feature list

33. Run the prediction again

34. Overfitting
• Resulting model too tied to the training set.
• It doesn’t generalize to new data, which is
the point of prediction.

35. Random Forest Classifier
• Random Forest Classifiers use many
Decision Trees to build a classifier.
• We introduce a bit of randomness.
• Each Tree can give a different answer (a
vote). The final classification is the most
common amongst the Trees.

36. Random Forest Classifier

37. Creating your first Decision Tree

38. Importances and Score

39. Predicting with Random Forest Classifiers

40. Results

41. 1976
Rocky

42. 1984
Amadeus

43. 1996
The English Patient

44. 2009
The Hurt Locker

45. And the Oscar goes to…

46. La La Land!!

49. The End
Nothing happened after that.
Right?? RIGHT??

50. We can predict the Oscars
Except for 2017 ¯_(ツ)_/¯

52. More about Thinkful
• Anyone who’s committed can learn to code
• 1-on-1 mentorship is the best way to learn
• Flexibility! Learn anywhere, anytime, & at your
own pace

53. Our Program
You’ll learn concepts, practice with drills, and build
capstone projects — all guided by a personal mentor

54. Our Mentors
Mentors have, on average, 10+ years of experience

55. Data Science Syllabus
• Managing data with SQL and Python
• Modeling with both supervised and unsupervised
models
• Data visualization and communicating with data
• Technical interviews + Career prep

56. Our Results
Job Titles after GraduationMonths until Employed

57. Special Introductory Offer
• Prep course for 50% off — $250 instead of $500
• Covers math, stats, Python, and data science toolkit
• Option to continue into full program
• Talk to me (or email me) if you’re interested

58. October 2015
Questions?
jas@thinkful.com
schedule a call through thinkful.com