Introduction to mining massive datasets

1. Mining massive datasets
(based on Standford CS246)
Viet-Trung TRAN
Viet-­‐Trung
Tran
1

2. Credits
• Jure Leskovec, Anand Rajaraman, Jeff
Ullman - Stanford University
• http://web.stanford.edu/class/cs246/
• http://mmds.org/
Viet-­‐Trung
Tran
2

3. What is data mining?
• Knowledge discovery from data
Viet-­‐Trung
Tran
3

4. Viet-­‐Trung
Tran
4

5. Data contains value and
knowledge
Viet-­‐Trung
Tran
5

6. Data mining
• Store
• Manage
• Analyzed
Data
mining
~
Big
Data
~
Predic5ve
Analysis
~
Data
science
Viet-­‐Trung
Tran
6

7. Demand for data mining (US)
Viet-­‐Trung
Tran
7

8. What is data mining
• Given lots of data
• Discover patterns and make predictions that
are
– Valid
– Useful
– Unexpected
– Understandable
Viet-­‐Trung
Tran
8

9. Data mining tasks
• Descriptive methods
– Find human-interpretable patterns that describe
data
• Clustering
• Predictive methods
– Use some variables to predict the unknown or
future values of other variables
• Recommender systems
Viet-­‐Trung
Tran
9

10. Meaningfulness of analytic
answers
• Risk of "data mining" is that the discover is
meaningless
• Bonferroni's principle
– An algorithm or method we think is useful for
finding a particular set of data actually returns
more false positives
Viet-­‐Trung
Tran
10

11. Dealing with data?
Viet-­‐Trung
Tran
11

12. Data mining cultures
• Overlap with
– Database: large scale data, simple queries
– Machine learning: Small data, complex models
– CS theory: (Randomized) algorithms
• Different cultures
– To DB guys: extreme form of analytic
processing
– To ML guys: inference of models (A conclusion
reached on the basis of evidence and
reasoning)
Viet-­‐Trung
Tran
12

13. What will be learn
• Mine different types of data
– High dimensional
– Graph
– Infinite/never-ending
– Labeled
• Use different models of computation
– Batch processing
– Stream
Viet-­‐Trung
Tran
13

14. To solve real-world problems
Viet-­‐Trung
Tran
14