Twarql (Presentation at I-SEMANTICS 2010)

TwarqlTapping Into the Wisdom of the Crowd
Pablo N. Mendes, PavanKapanipathi, Alexandre Passant
I-SEMANTICS
Graz, Austria
September 2nd, 2010

Outline
Introduction
Motivation
Contributions
Use Cases
IPad Scenario
Location, Sentiment, Recommendations, Competitors
System
Demo
Architecture
Activity Flow
Annotation Pipeline
Conclusion

Tap into the Wisdom of the Crowd?
“taking into account the collective opinion of a group of individuals rather than a single expert to answer a question” (Wikipedia)
Has been done successfully
box-office revenue prediction for movies (CoRR’10)
earthquake detection (WWW’10)
Can be useful in many scenarios

Social Media: Motivation

Social Media: Motivation (contd.)
Information Overload!

Twarql Contributions
Expressive description of an information need
Beyond keywords (uses SPARQL)
Flexibility on the point of view
Ability to “slice and dice” data in several dimensions: thematic, spatial, temporal, sentiment, etc.
Streaming data + background knowledge
Enables automatic evolution and serendipity
Scalable real time delivery
Using sparqlPuSH(SFSW’10)

Use Cases (IPad Scenario)
Location
Retrieve stream of locations where my product is being mentioned right now.
Consumer sentiment
Retrieve all people that have said negative things about my product.
Content suggestion
Retrieve all URLs that people recommend with relation to my product.
Related entities
What competitors are being mentioned with my product?

Use Case 1: Location (query)
Retrieve a stream of locations where my product is being mentioned right now.
SELECT ? location
WHERE {
?tweet moat:taggedWithdbpedia:IPad .
?presence opo:currentLocation ?location .
?presence opo:customMessage ?tweet .
}

Use Case 1: Location
Incoming microposts…
@anonymized
@anonymized
@anonymized
Loremipsumblabla this is an example tweet
opo:currentLocation
?presence
?location
SELECT ? location
WHERE {
? tweet moat : taggedWithdbpedia : IPad .
? presence opo: currentLocation ?
location .
? presence opo: customMessage ? tweet .
}
opo:customMessage
moat:taggedWith
dbpedia:IPad
?tweet

Use Case 1: Location
@anonymized
@anonymized
@anonymized
Update view if micropost matches contraints
opo:currentLocation
?presence
?location
SELECT ? location
WHERE {
? tweet moat : taggedWithdbpedia : IPad .
? presence opo: currentLocation ?
location .
? presence opo: customMessage ? tweet .
}
opo:customMessage
moat:taggedWith
dbpedia:IPad
?tweet

Use Case 2: Consumer Sentiment
Retrieve all people that have said negative things about my product.
SELECT ? user
WHERE {
? tweet sioc:has_creator ? user .
? tweet moat:taggedWithdbpedia:IPad .
? tweet twarql:sentimenttwarql:Negative .
}

Use Case 2: Consumer sentiment
@anonymized
twarql:sentiment
?user
:Negative
sioc:has_creator
moat:taggedWith
dbpedia:IPad
?tweet

Use Case 2: Consumer sentiment
Invite users for testing our new launch:
@pablomendes
@terraces
@anonymized
@pavankaps
Trigger action if micropost matches contraints
@anotheruser
twarql:sentiment
?user
:Negative
sioc:has_creator
Update view
moat:taggedWith
dbpedia:IPad
?tweet

Use Case 3: Content suggestion
Retrieve all URLs that people recommend with relation to my product
SELECT ?url
WHERE {
? tweet moat:taggedWithdbpedia:IPad .
? tweet sioc:links_to ?url .
}
Note: Twarql extracts links and resolves shortened URIs before annotating the tweet

Use Case 3: Content Suggestion
@anonymized
?url
sioc:links_to
moat:taggedWith
dbpedia:IPad
?tweet

Use Case 3: Content Suggestion
My IPad Journal
@anonymized
If micropost matches contraints,
accumulate information and update view
?url
sioc:links_to
moat:taggedWith
dbpedia:IPad
?tweet

Use Case 4: Competitors (query)
What competitors of my product are being mentioned?
SELECT ? competitor
WHERE {
dbpedia:IPadskos:subject ?category .
?competitor skos:subject ?category .
?tweet moat:taggedWith ?competitor .
}

Use Case 4: Competitors (query)
What competitors of my product are being mentioned with my product?
- comparative opinion!
SELECT ? competitor
WHERE {
dbpedia:IPadskos:subject ?category .
?competitor skos:subject ?category .
?tweet moat:taggedWith ?competitor .
}
?tweet moat:taggedWithdbpedia:Ipad .

Use Case 4: Competitors
Background Knowledge (e.g. DBpedia)
@anonymized
dbpedia:IPad
skos:subject
?category
?category
?competitor
skos:subject
skos:subject
moat:taggedWith
?tweet

@anonymized
category:Wi-Fi
dbpedia:IPad
category:Touchscreen
skos:subject
?category
?category
?competitor
skos:subject
skos:subject
moat:taggedWith
Background knowledge is dynamically “brought into” microposts.
?tweet

@anonymized
category:Wi-Fi
dbpedia:IPad
category:Touchscreen
skos:subject
?category
?category
?competitor
skos:subject
skos:subject
moat:taggedWith
?tweet
Trigger action if micropost matches constraints.

Use Case 4: Competitors (contd.)
Highlights
When a new competitor “appears” in the KB, no change is needed in the query => Automatic Evolution
We found interesting products that we didn’t initially consider as competitors of IPad(e.g. IPhone)=> Serendipity

Demonstration
Cuebee
query formulation
Twarql
information extraction
stream querying
sparqlPuSH
real time delivery
Demo link: http://bit.ly/twarql

Architecture
Mendes, Passant, Kapanipathi, Sheth. Linked Open Social Signals, Web Intelligence 2010

Twarql Streaming Activity Diagram
DIST. HUB
Web Client
APP SERVER
(SEMANTIC)
PUBLISHER
SOCIAL SENSOR
Twitter API
SETUP
keywords
FORMULATE QUERY
LISTEN(tweet)
STREAM(tweet)
query, #id
/register
/subscribe
STREAM(query, #id)
ANNOTATE(tweet)
REGISTER(query,
new hubURL())
PUBLISH(tweet)
/publish
hubURL
FILTER(tweet, for all query)
#id
REQUEST(#id)
PULL(hubURL, req)
/feed
STORE(tweet)
feed update
UPDATE INTERFACE
RDF store
UPDATE(hubURL,rssTweet)
UPDATE(tweet)
#id
POLL
cache
CACHE(tweet)
QUERY(#id, query)
PUSH(tweet, subscriber)
RELAY
QUERY
/sparql
/sparql
http://www.slideshare.net/pablomendes/streaming-annotatedtweets

Annotation
URL extraction
Regex based, short URL resolution via http redirects
Hashtag extraction
Regex based, “resolution” via TagDef and Tagal.us
Entity mention extraction
“Spotting” via string matching (prefix tree) based on a dictionary (DBpedia)
Disambiguation on the way! (via DBpedia Spotlight)
Conversion to RDF triples
using SIOC, FOAF, MOAT, etc.

Conclusion
Flexibility and expressiveness in managing real time streams of information!
Triples generated for the IPad scenario
From June 3rd to June 8th
511,147 tweets
4,479,631 triples
… and counting!
You can generate triples too: http://twarql.sf.net
53,237 positive;
6,739 negative;
451,171 neutral

Thank you
Connect with us:
@pablomendes
@terraces
@pavankaps
Collaborate:
http://twarql.sf.net
http://wiki.knoesis.org/index.php/Twarql

Twarql (Presentation at I-SEMANTICS 2010)

by Pablo Mendes on May 05, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Twarql (Presentation at I-SEMANTICS 2010) Presentation Transcript