VU University Amsterdam - The Social Web 2016 - Lecture 4

Social Web
2016
Lecture 4: How do we MINE, ANALYSE &
VISUALISE the Social Web?
Davide Ceolin (credits to: Lora Aroyo)
The Network Institute
VU University Amsterdam

Announcements
• Results of Assignment 1 are out: well
done
• Assignment 2 is out: due on 01/03!
• Next deadlines:
• Wednesday 23:59: final project update
• Friday 10:00: post your question
• Friday 17:00: vote your question

Announcements
• This Thursday (lab session):
• F153 – S345 – S329
• Anca & Niels will be there in person
• I will join via hangout
(davide.ceolin@gmail.com)
• Next Monday: guest lecture

• 200 billion tweets on Twitter in 2015, by
1.3 billion registered users
• 4.5 billion likes generated on Facebook in
2015, by 1.55 billion different users
• 300 hours of videos uploaded to
YouTube every minute
• 60.7 million photos uploaded to flickr per
month
The Age of BIG Data
Social Web 2016, Davide Ceolin

Science with BIG Data

BIG Data Challenges

Big Data vs. Deep Data
• Social Web data often follow a long tail
distribution
Big Deep

enormous wealth of data = lots of insights
• insights in users’ daily lives and activities
• insights in history
• insights in politics
• insights in communities
• insights in trends
• insights in businesses & brands
Why?

enormous wealth of data = lots of insights
• who uploads/talks? (age, gender, nationality,
community, etc.)
• what are the trending topics? when?
• what else do these users like? on which
platform?
• who are the most/least active users?
• ..…
Why?

Web Source Criticism?
Source criticism checklist
(https://en.wikipedia.org/wiki/Source_criticism)
• Who is the author and what are the qualifications of the
author in regard to the topic that is discussed?
• When was the information published?
• What is the reputation of the publisher?
• Does the source show a particular cultural or political bias?
• Does the source contain a bibliography?
• Has the material been reviewed by a group of peers, or has
it been edited?
• …
How does this apply to Web sources?

Image:
http://www.co.olmsted.mn.us/prl/propertyr
ecords/RecordingDocuments/PublishingI
mages/forms.jpg
This doesn’t work

How about this?

Web of Trust
https://www.mywot.com/en/scorecard/pulse.seattlechi

Who uses it?

Politicians
Governmental
institutions

Whole society

Whole society
repurposing
data
danger of
second order
effect

Whole society
Repurposing data
discoveries & correlations
Web-Scale Pharmacovigilance: Listening to Signals from the Crowd, R.W. White et al (20

Scientists
Bibliometrics

Culture
History

Culture
Bill Howe, University of Washington

Entertainment

You?
https://klout.com/#/measure

Companies

Who does it?

The Rise of the Data Scientist
Data Geeks Skills:
Statistics & Math
Data munging
Visualisation

http://radar.oreilly.com/2010/06/what-is-data-science.html
The Rise of the Data Scientist

• Data Science enables the creation of data
products
• Data products are applications that acquire their
value from the data, and create more data as a
result.
• Users are in a feedback loop: they constantly
provide information about the products they use,
which gets used in the data product.
Data Science

Data Science Venn Diagram
Drew Conway

Data Science Venn Diagram

Popular Data Products
Data Science is about
building products
not just answering questions

empower the others
to use the data
empower the
others to their
own analysis

http://www.metacog.com/resources/banner3.jp

(Inspired by George Tziralis’ FOSS Conf’09, John Elder IV’s Salford Systems
Data Mining Conf. and Toon Calders’ slides)
Data mining is the exploration & analysis of
large quantities of data
in order to discover valid, novel, potentially useful,
& ultimately understandable patterns in data
http://www.freefoto.com/images/33/12/33_12_7---Pebbles_web.jp
Data Mining 101

Database
s
Statistics/
Numerical
methods
Artificial
Intelligenc
e
Data Mining 101
• Data input &
exploration
• Preprocessing
• Data mining
algorithms
• Evaluation &
Interpretation

• What data do I
need to answer
question X?
• What variables
are in the data?
• Basic stats of
my data?
Data Input & Exploration
“LikeMiner”

• Cleanup!
• Choose a suitable data model
• What happens if you integrate data from multiple sources?
• Reformat your data
Preprocessing
“LikeMiner”

• Classification: Generalising a known structure
& apply to new data
• Association: Finding relationships between
variables
• Clustering: Discovering groups and structures
in data
Data Mining Algorithms

• Filter users by interests
• Construct user graphs
• PageRank on graphs to
mine representativeness
• Result: set of influential
users
• Compare page topics to
user interests to find
pages most representative
for topics
Mining in “LikeMiner”

Evaluation & Interpretation
What does the pattern I found mean?
• Pitfalls:
• Meaningless Discoveries
• Implication ≠ Causality (Intensive care -> death)
• Simpson’s paradox
• Data Dredging
• Redundancy
• No New Information
• Overfitting
• Bad Experimental Setup

Data Mining is not easy

Popular ML –
Deep learning
http://www.kdnuggets.com/wp-
content/uploads/deep-learning-
small-big-data.jpg
http://scyfer.nl/wp-
content/uploads/2014/05/Deep_Neu
ral_Network.png

Deep learning
frameworks
https://code.facebook.com/posts/16878615181260
48/facebook-to-open-source-ai-hardware-design/

Data Journalism

source: http://kunau.us/wp-
content/uploads/2011/02/Screen-shot-
2011-02-09-at-9.03.46-PM-w600-
h900.png
Mining Social Web Data

Source:
http://infosthetics.com/archives/2011/12/all_the_information_facebook_knows_about_you.html
See also: http://www.youtube.com/watch?feature=player_embedded&v=kJvAUqs3Ofg
Single Person

http://www.brandrants.com/brandrants/obama/
Populations

Brand Sentiment via Twitter
http://flowingdata.com/2011/07/25/brand-sentiment-showdown/

Sentiment Analysis as Service
http://www.crowdflower.com/type-sentiment-analysis

http://text-processing.com/demo/sentiment/

http://www.cs.cornell.edu/home/kleinber/networks-book/networks-book.pdf
Recommended Reading

http://www.actmedia.eu/media/img/text_zones/English/small_38421.j
Assignment 2: Semantic Markup• Part I: enrich/create a Web page with semantic markup
• Step 1: Mark up two different Web pages with the appropriate markup describing
properties of at least people, relationships to other people, locations, some temporally
related data and some multimedia. You can also try out tools such as Google Markup
Helper
• Step 2: Validate your semantic markup. Use existing validator.
• Step 3: Explain why you chose particular markups. Compare the advantages and
disadvantages of the different markups. Include screenshots from validators.
• Part II: analyse other team’s Web page markup - as a consumer & as a publisher
• Step 1: Perform evaluation and report your findings (consider findability or content
extraction)
• Step 2: Support your critique with examples of how the semantic markup could be
improved.
• In introductory section explain what semantic markup is, what it is for, what it looks like
etc.
• Support your choices and explanations with appropriate literature references.
• 5 pages (excluding screen shots).
• Other group’s evaluation details in appendix.
• Deadline: 1 March 23:59

image source:
http://www.flickr.com/photos/bionicteaching/1375254387/
Hands-on Teaser
• Build your own recommender system 101
• Recommend pages on del.icio.us
• Recommend pages to your Facebook friends

VU University Amsterdam - The Social Web 2016 - Lecture 4

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to VU University Amsterdam - The Social Web 2016 - Lecture 4

Similar to VU University Amsterdam - The Social Web 2016 - Lecture 4 (20)

More from Davide Ceolin

More from Davide Ceolin (6)

Recently uploaded

Recently uploaded (20)

VU University Amsterdam - The Social Web 2016 - Lecture 4

Editor's Notes