Your SlideShare is downloading. ×
VIStology: Mining the Malaysian Sopo Blogosphere
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

VIStology: Mining the Malaysian Sopo Blogosphere

426
views

Published on

Presentation at Harvard Political Networks Conference, June 13, 2009. Brian Ulicny (presenter). …

Presentation at Harvard Political Networks Conference, June 13, 2009. Brian Ulicny (presenter).
Mieczyslaw Kokar (VIStology, Northeastern), Chris Matheus (coauthors)

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
426
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Raja Petra Kamarudin, editor of M2day
  • 2K Blogs 1,059 Blogspot Blogs, 801 Wordpress, 16 Typepad 272 .org, 45 .gov.my Of the Blogspot blogs (structured profiles) 659 Malaysia, 166 other, 234 unspecified 122 Kuala Lumpur 88 Selangor (State) 31 Johor (State), 10 Subang Jaya, 11 Klang, 6 Johor Bahr 10 list Koran as favorite book; 4 Bible Colette, M. 2006. Blogging Phenomenon Sweeps Asia. MSN Press Release. November 27, 2006.
  • Tan, Jun-E and Ibrahim, Zawawi. 2008. Blogging and Democratization in Malaysia: A New Civil Society in the Making. Strategic Information and Research Development Centre. Petaling Jaya, Malaysia.
  • http://www.iht.com/bin/printfriendly.php?id=8278702 http://www.jeffooi.com/2007/11/how_they_painted_it_yellow.php
  • 31 documents from our Bersih rally corpus that cite one or more of 10 popular news article about the rally.
  • Transcript

    • 1. MINING THE MALAYSIAN ‘SOPO’ BLOGOSPHERE June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 2. IBlogs Introduction
      • Our intention in this project was to:
      • Provide an automated framework for analysts to monitor an unfamiliar blogosphere
      • Do so in a way that reflects the specific nature and structure of blogs
      • In particular, social/political blogging
      • Use Malaysian ‘sopo’ blogosphere as case study
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 3. Outline of Talk
      • Background: Malaysian Sopo Blogosphere
      • Background: Blogs and Blog Metrics
      • New Metrics for Sopo Blog:
        • Relevance,
        • Credibility,
        • Timeliness and
        • Specificity
      • Conclusions and Future Work
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 4. Typically, a blogosphere is modeled as a directed network of blog-blog links June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA Image from Case Western Reserve’s Blog Miner Project
    • 5. A (Sopo) Blogosphere is a directed multi-relational network June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA Links are all directed (hrefs); Direction not shown. Plus ‘chrome’
    • 6. Why Malaysian Sopo Blogs?
      • Controlled Press; Less control of Web.
        • RSF: 124 th of 169 in Press Freedom Index.
      • Great deal of activity in political blogosphere, leading to first mass rallies in many years.
      • Government pushback: arrests; libel suits; harassment of bloggers; “Malaysia Today” shut down temporarily, publisher jailed under Security Act.
      • Bersih Rally pushing for electoral reform precedes unseating of ruling party supermajority in 2008. Several prominent bloggers elected.
      • Blogs mostly in Malaysian English; Bahasa Melayu (1/5); small number in Chinese (which we had to ignore)
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 7. Bersih Rally, November 10, 2007 June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 8. Profile Mining: How Big is MY SoPo Blogosphere? 385 blogs in crawl seed yields 220.3K urls, 4693 Sites @ depth 4 Requires identifying blog profile URL on blog homepage and named entity recognition. June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA Sopo Crawl Rally Bloggers 2K Total Early Nov ‘07. 1K Blogspot 800 Wordpress 16 Typepad, etc. On Blogspot, 659 Malaysians, 122 others, 234 unspecified. 122 Kuala Lumpur
      • 1271 blog posts in 7 days after.
      • 878 unique bloggers.
      • 408 Blogspot.
      • 228 Wordpress.
      • 9 LiveJournal.
      • 2 Typepad.
        • Only 144 in common with those identified via Sopo crawl.
        • ~ Size of Kuala Lumpur bloggers.
        • 324 if rally bloggers + their blogrolls included
        • Arrests for participation.
        • Potentially faced 1-2 years prison. (None were.)
      MSN Study (2006): implied 6 million Malaysian sopo blogs (potentially)
    • 9. Comparison of Profile Information Disclosed in Malaysian Sopo and Random Malaysian Blogs June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 10. Less Polarization in Malaysian Sopo Blogosphere than US Malaysian Sopo Blogosphere US Political Blogosphere (Adamic & Glance, 2005) June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 11. Blogs and Blog Metrics June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 12. Blogs and Blog Metrics: Google For each Term in Post Text: Term Frequency * Inverse Document Frequency (plus incoming link text) IDF: the more posts on X, the less each post is about X. Google PageRank (Eigenvector centrality, determined by links from other websites) June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 13. Blogs and Blog Metrics: Technorati For each Term in Post Text: Term Frequency * Inverse Document Frequency Technorati Authority # of inbound links from other blogs over past 6 months (Indegree centrality) June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 14. Blogs and Blog Metrics: VIStology For each Term in Post Text: (Term Frequency * Inverse Document Frequency) + Outlink Proper Name Frequency. Credibility (Authority + Engagement + Accountability) Specificity Timeliness June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 15. Credibility = Authority + Engagement + Accountability
      • Google PageRank and
      • Technorati Authority determine quality of a
      • blog (post) purely in
      • terms of centrality as determined by
      • links from
      • other websites,
      • in particular, other blogs.
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 16. Comment Counts ≈ Engagement June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 17. Inlinks and Comments not strongly correlated
      • Surprisingly, at least in the Malaysian sopo blogosphere, inlinks and comment counts are not very strongly correlated. Some blogs receive many comments but few links; other receive many links but few comments.
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 18. Accountability ≈ Inverse of Anonymity
      • Full Name?
      • City/State?
      • Country?
      • Industry/Occupation? Email/IM Address?
      • Profile Picture?
      • Affiliation(s)?
      • Blog longevity?
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 19. Validation of Metric: Top Sopo Bloggers from Tan and Ibrahim June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 20. Top Malaysian Sopo Bloggers by Credibility Score June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 21. Post + Outlink Dyad Determines Post Topic Aboutness Determined by Term Frequency in Post Text plus Linked Text Requires: news outlink identification and named entity recognition . VIStology, Inc. | Harvard Networks in Political Science Conference Post + Citations Jeff Ooi Post 6/13/08 Elections, electoral+ reform, Pak+Lah, Coalition for Clean and Fair Elections, (PM) Abdullah (Badawi) Better representation of what post is about:
    • 22. Relevance: Post and Outlink Term Documents Characterize Set of Posts than Post Terms alone June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 23. Specificity Comparison of Blog Post 1 and 2 June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 24. Time distribution of Bersih posts
      • Timeliness is determined by difference between blog post time and news article publication time to whith it links.
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 25. Sopo Bloggers are more engaged with news than Random Bloggers
      • Outlinking properties of 77 Malaysian sopo and 50 random Malaysian bloggers compared
      June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 26. Conclusions
      • We have developed an automated system implementing novel metrics for Blog Topical Relevance (outlink sensitive) and Quality (Credibility, Timeliness, Specificity) appropriate to monitoring a social/political blogosphere.
      • We have validated these metrics against current standards:
        • Credibility vs. Technorati Authority, Google PageRank
        • Outlink-informed Relevance vs. Post-only Tf*Idf
        • Specificity vs. Length
        • Timeliness vs. Recency
      June 13, 2009s VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 27. Thank You Questions? Brian Ulicny: bulicny@vistology.com
    • 28. IBlogs Screenshot June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 29. IBlogs Architecture June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 30. Blog Reception Metrics Compared June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 31. Political Affiliations by Blog Profile Count June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA
    • 32. Distribution of News Sources in Bersih Blog Posts June 13, 2009 VIStology - Harvard Political Networks '09 - Cambridge, MA