This document discusses using machine learning and data mining techniques to gain knowledge from big data. It defines key terms like data, databases, and big data. It explains that machine learning and data mining can help solve the problem of "data overloading" by discovering patterns and making predictions. The document also introduces social network analysis and crowdsourcing as collective intelligence approaches for learning from data. It provides examples like Amazon Mechanical Turk, which utilizes crowdsourcing for various tasks.

1. d ge
Data le
o w
Kn
Learning
& Crowd
Mining
Prof. Wei-Guang Teng ( 鄧維光 )
Dept. of Engineering Science
National Cheng Kung University
2013/3/18

2. Let’s Start with “Data”
• Data
– Known facts that can be recorded and have an
implicit meaning
• Database
– A collection of related data
Data
2

3. Big
What Are
?
Data

4. Big Data
• Exabyte (EB or 1018 bytes)
– 1 EB = 1,000 PB = 1,000,000 TB
Data
• See Big Data - Big Opportunity
– http://www.youtube.com/watch?v=e6K0VHz6FVc
4

5. Problem of Data Overloading
• We are drowning in data, but starving for knowledge!
Data
5

6. How to Solve this Problem?

7. Possible Solutions!
Data
Learning
& 7
Mining

8. Machine Learning ( 機器學習 )
• Machine learning, a branch of artificial intelligence, is
about the construction and study of systems that can
learn from data
• Example 1: OCR (optical character recognition)
– Printed characters are recognized automatically based on
previous examples
• Example 2: Spam Detection
– A machine learning system could be trained on email messages
to learn to distinguish between spam and non-spam messages.
After learning, it can then be used to classify new email
Data messages into spam and non-spam folders
Learning
& 8
Mining

9. Data Mining ( 資料探勘 )
• To extract interesting patterns from large databases
– Interesting: non-trivial, implicit, previously unknown & potentially useful
– Patterns: association rules, classification, clustering, ...
Data
Learning
& 9
Mining

10. ML vs. DM
• Machine learning focuses • Data mining focuses on
on prediction, based on the discovery of
known properties learned (previously) unknown
from the training data properties on the data
Data
Learning
& 10
Mining

11. To Work Together
• See What is Predictive Analytics / Data Mining?
– http://www.youtube.com/watch?v=BjznLJcgSFI
Data
Learning
& 11
Mining

12. Why Collective Intelligence,
not Personalized ?

13. Social Network
• A social structure made up of a set of actors and the
dyadic ties between these actors
– Social network analysis is to identify local and global patterns,
locate influential entities, and examine network dynamics
Data
Learning
& Crowd 13
Mining

14. Media, Management & Marketing ...
• See A Day in the Life of Social Media
– http://www.youtube.com/watch?v=iReY3W9ZkLU
• See The Social Media Revolution 2012-13
– http://www.youtube.com/watch?v=0eUeL3n7fDs
Data
Learning
& Crowd 14
Mining

15. What’s Next ?

16. Crowdsourcing ( 群眾外包 )
• See Crowdsourcing Evolution
– http://www.youtube.com/watch?v=OKMOI9PSQwo
e
dg
Data le
ow
Kn
Learning
& Crowd 16
Mining

17. Amazon Mechanical Turk
e
dg
Data le
ow
Kn
Learning
& Crowd 17
Mining

18. Types of Tasks
e
dg
Data le
ow
Kn
Learning
& Crowd 18
Mining

19. Questions?