M. Platakis, D. Kotsakos and D. Gunopulos: "Discovering Hot Topics in the Blogosphere", Proceedings of the 2nd Panhellenic Scientific Student Conference on Informatics, Related Technologies and Applications EUREKA 2008, pp. 122-132. Best Paper Award.

1. M. Platakis1, D. Kotsakos1, D. Gunopulos1
1Department of Informatics and Telecommunications, University of Athens
{platakis, d.kotsakos, dg}@di.uoa.gr

2.  Introduction
 Related Work
 Our Approach
 Experimental Results
 Future Work

3.  What is a blog?
 Why research on blogs?
◦ 200K new blogs every day
◦ 120M English blogs
Blogosphere

4.  What is a blog?
 Why research on blogs?
◦ 200K new blogs every day
◦ 120M English blogs
Blogosphere

5.  What is a blog?
 Why research on blogs?
◦ 200K new blogs every day
◦ 120M English blogs
Blogosphere

6.  The Problem: Searching in blogs
 Blogs are an instant medium. When something
important happens bloggers write about it
 Need for an automated searching/analyzing
mechanism

7.  Each blog post has a timestamp
◦ Not present in traditional web content
◦ Not taken into account by etc.
 Popularity of a term may increase in time
 This is a burst!
 In our work we detect bursty terms and
correlations between them…
…or at least we try to!

8. This is a burst!

9.  Introduction
 Related Work
 Our Approach
 Experimental Results
 Future Work

10.  J. Kleinberg’s attempt to find meaningful
structure in e-mail streams
 State transitions mark the bursts in an infinite
state probabilistic automaton
 Costs assigned to state transitions eliminate
short bursts and fragmentation of longer
ones

11.  HMM Theory
 Given an event stream a minimum-cost state
sequence is found (dynamic programming)

12.  [Bansal et al. (2007)] present Blogscope.net – a
system that identifies keyword bursts
 A standard normal curve, formed by the popularity
values of a term for the last few days, is used to
find outliers
 Analysis performed on the body of blog posts

13.  [Vlachos et al. (2004)]: finding similarities between
time-series
 [Kumar et al. (2005)], [Chi et al. (2007)]: further
analysis of the Blogosphere
 [Wang et al. (2007)]: mining of correlated bursty
topic patterns
 [Zhang et al. (2007)]: detecting bursts
 [Agarwal (2008)]: identifying influential bloggers
 [Cheng et al. (2008)]: discovering bloggers’
interests
 [Ding et al. (2008)]: opinion mining

14.  Introduction
 Related Work
 Our Approach
 Experimental Results
 Future Work

15. 1. Application on blog data
2. Bursty terms discovery
3. Accuracy evaluation: matching the results
with real-life events (if possible)

16. 4. Correlated keywords usually form a topic;
our goal is to identify such topics
5. Burstiness = when + how much
Similar burstiness  potential term correlation
6. Accuracy evaluation: matching obtained groups
of keywords with real-life topics (if possible)

17.  Why use Kleinberg’s algorithm?
◦ Sophisticated notion of temporal bursts
◦ Blogscope’s team claim it would be too slow
 We experiment on sole blog posts’ titles
◦ To determine if titles are suffice to extract
meaningful bursts
◦ To gain some computational time
 Titles had to be parsed through the
permalinks of the blog posts

18.  Introduction
 Related Work
 Our Approach
 Experimental Results
 Future Work

19.  Bursty keywords examples in comparison
with Blogscope.net
 Dataset from the first ten days of April 2008
◦ 30,000 blog post titles
 Data pre-processing
◦ stop words removal, downcase, no stemming
 Automaton’s output (comes in real time!):
“Keyword, weight, start_day, end_day”

20.  “Fool” comes out as a bursty term
 Output: “fool,17.343,1,2”
 Thus, “fool” was bursty on April’s first two days

21.  “Fool” comes out as a bursty term
 Output: “fool,17.343,1,2”
 Thus, “fool” was bursty on April’s first two days

22.  “preprocessor” comes out as a bursty term
 Output: “preprocessor,85.947,6,6”
 Thus, “preprocessor” was bursty on April’s 6th
day

23.  “preprocessor” comes out as a bursty term
 Output: “preprocessor,85.947,6,6”
 Thus, “preprocessor” was bursty on April’s 6th
day
 Numerous spam blog posts were uploaded on
April’s 6th day
 All contained the term “preprocessor”

24.  Searching through the results to identify
clusters of correlated keywords
 An example: Keyword Weight Start day End day
heston 12.213 5 5
charlton 9.929 5 5
actor 4.302 4 5
dead 3.388 5 5
84 3.08 5 5
 Bloggers wrote about:

25.  Introduction
 Related Work
 Our Approach
 Experimental Results
 Future Work

26.  Tags + Titles + Multimedia metadata
 Development of an automated method to
discover correlated keywords
 Extraction of popularity curve per keyword
 Employment of an appropriate distance metric
(time series’ analysis) to compare such curves
 Application in micro-blogging and continuous
news’ streams