Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Socialcom2011 discussionactivityprediction

IEEE International Conference on Social Computing, Boston, USA
(http://www.iisocialcom.org/conference/socialcom2011/)
In this event, the OU team presented their work for anticipating
discussion activity on community forums. This work tried to address
two main research questions: which features are key to stimulating
discussions? And, how do these features influence discussion length?
This analysis offers policy makers the opportunity to focus on posts
that are bound to generate a higher attention from the public.

  • Be the first to comment

  • Be the first to like this

Socialcom2011 discussionactivityprediction

  1. 1. Anticipating Discussion Activity on Community Forums Matthew Rowe, Sofia Angeletou and Harith Alani Knowledge Media Institute, The Open University, Milton Keynes, United Kingdom The Third IEEE International Conference on Social Computing. MIT, Boston, USA. 2011
  2. 2. Community Content• Online communities are now used to: – Ask questions – Post opinions and ideas – Discuss events and current issues• Content analysis in online communities is attractive for: – Market analysis – Brand consensus and product opinion• Social network analytics in the US is predicted to reach $1 billion by 2014 (Forrester 2009)• Masses of data is now being published in online communities: – Facebook has more than 60 million status updates per day (Facebook statistics 2010)Anticipating Discussion Activity on Community Forums 1
  3. 3. Anticipating Discussion Activity on Community Forums 2
  4. 4. The Need for Analysis• Analysts need to know which piece of content will generate the most activity – i.e. the most auspicious or influential – Helps focus the attention of human and computerised analysts • What to track?• Need to understand the effect features (community and content) have on attention to content• Enable content creators to shape their content in order to maximise impact – E.g. promoters, government policy makersRQ1: Which features are key to stimulating discussions?RQ2: How do these features influence discussion length?Anticipating Discussion Activity on Community Forums 3
  5. 5. Outline• Anticipating Discussion Activity: Approach Overview – Identifying Seed Posts – Predicting Discussion Activity• Features• Dataset – Community Message Board: Boards.ie• 1. Identifying Seed Posts• 2. Predicting Discussion Activity• Findings• ConclusionsAnticipating Discussion Activity on Community Forums 4
  6. 6. Approach Overview• Two-stage approach to predict discussion activity in online communities: 1. Identify seed posts • i.e. Thread starters that yield a reply • Will a given post start a discussion? • What are the properties that seed posts exhibit? – What parameters tend to trigger a discussion? 2. Predict discussion activity levels • From the identified seed posts • What is the level of discussion that a seed post will generate? • What features correlate with heightened discussion activity?Anticipating Discussion Activity on Community Forums 5
  7. 7. Features• For each post, model: a) the author, b) the content and c) the topical concentration of the author• F1: User Features – In-degree, out-degree: social network properties of the author – Post count, age, post rate: participation information of the author• F2: Content Features – Post length, referral count, time in day: surface features of the post – Complexity: cumulative entropy of terms in the post – Readability: Gunning Fog index of the post – Informativeness: TF-IDF measure of terms within the post – Polarity: average sentiment of terms in the postAnticipating Discussion Activity on Community Forums 6
  8. 8. Features (2)• F3: Focus Features – Topic entropy: the concentration of the author across community forums • Higher entropy indicates a wider spread of forum activity • More random distribution, less concentrated – Topic Likelihood: the likelihood that a user posts in a specific forum given his post history • Measures the affinity that a user has with a given forum • Lower likelihood indicates a user posting on an unfamiliar topicAnticipating Discussion Activity on Community Forums 7
  9. 9. Dataset: Boards.ie• Irish community message board that was established in 1998• Covers a wide array of topics and themes in forums – E.g. World of Warcraft, Japanese Culture, Rugby• We were provided with the complete dataset spanning 1998- 2008 of all posts and forum information – Focussed on 2006 due to the scale of entire dataset• No explicit social connections exist in the dataset – Social network features were built from the reply-to graph• 6-month window prior to the post date was used to build the user and focus featuresAnticipating Discussion Activity on Community Forums 8
  10. 10. 1. Identifying Seed Posts• Will a given post start a discussion?• What are the properties that seed posts exhibit?• Experiment Setup: – Used all thread starter posts from Boards.ie in 2006 – Training/validation/testing sets using a 70/20/10% random split – Binary classification task: Is this a seed post or not? – Measures: precision, recall, f-measure, area under ROC curve• Performed 2 experiments: – a) Model Selection • Tested individual feature sets (user, content, focus) and combinations – b) Feature Assessment • Dropping 1 feature at a time, record reduction in f-measureAnticipating Discussion Activity on Community Forums 9
  11. 11. 1.a) Model SelectionAnticipating Discussion Activity on Community Forums 10
  12. 12. 1.b) Feature AssessmentAnticipating Discussion Activity on Community Forums 11
  13. 13. 1.b) Feature AssessmentAnticipating Discussion Activity on Community Forums 12
  14. 14. 2. Predicting Discussion Activity• What is the level of discussion that a seed post will generate?• What features correlate with heightened discussion activity?• Experiment Setup: – Train: seed posts in 70% training split – Test: seed posts in 20% validation split – Measure: Normalised Discounted Cumulative Gain (nDCG) • Look at varying rank positions: nDCG@k, k=1,2,5,10,20,50,100• Performed 2 experiments – a) Model Selection • Regression models: Linear, Isotonic, Support Vector Regression • Tested individual feature sets (user, content, focus) and combinations – b) Feature Contributions • Assess the features in the best performing model from a)Anticipating Discussion Activity on Community Forums 13
  15. 15. 2.a) Model SelectionAnticipating Discussion Activity on Community Forums 14
  16. 16. 2.a) Model Selection Linear Isotonic Support Vector RegressionAnticipating Discussion Activity on Community Forums 15
  17. 17. 2.b) Feature Contributions• What features correlate with heightened discussion activity?Anticipating Discussion Activity on Community Forums 16
  18. 18. FindingsRQ1:Which features are key to stimulating discussions?• Having many URLs in a post can negatively impact discussion activity – Could associate the post with spam content• Seed posts are associated with greater forum likelihood• Lower informativeness is associated with seed posts – i.e. seeds use language that is familiar to the communityRQ2: How do these features influence discussion length?• Lower forum entropy = heightened discussion activity• Greater complexity = heightened discussion activity – i.e. include more diverse language in the post• Increased activity can be expected from an increase in forum likelihood coupled with a decrease in forum entropy• Negative sentiment posts generate more activityAnticipating Discussion Activity on Community Forums 17
  19. 19. Conclusions and Future Work• The two-stage approach is able to: – Identify seed posts to a high degree of accuracy • F-measure: 0.792 – Predict discussion activity levels • nDCG@1: 0.89 (linear regression model) • Content and focus features yield best performing model – Average nDCG@k: 0.756• Findings inform: – Market Analysts to track high activity posts from the outset – Content creators to shape content in order to maximise impact• Currently applying approach over different platforms: – How can we predict activity on a given social web system? – How do social web systems differ in generate activity?Anticipating Discussion Activity on Community Forums 18
  20. 20. Questions?Web: http://people.kmi.open.ac.uk/roweEmail: m.c.rowe@open.ac.ukTwitter: @mattroweshowAnticipating Discussion Activity on Community Forums 19

×