a Study on Users' Voting Tendency in Daum View


Published on

Presentation by Kanghak Kim (KAIST Social Computing Lab) in Social Media session

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

a Study on Users' Voting Tendency in Daum View

  1. 1. 01 Participatory Journalism and Social News Services -Issues in social news services Issues Participatory Journalism? Participatory journalism: The act of a citizen or group of citizens, playing an citizen, citizens active role in the process of collecting, reporting, analyzing and disseminating news and information. The intent of this participation is to provide independent, reliable, accurate, wide-ranging and relevant information that a democracy requires. Participatory Journalism in Social News Services Users Collecting & Analyzing Disseminating Reporting 1 2 3… Aggregated News Ranked News Social News Services
  2. 2. 01 Participatory Journalism and Social News Services (Cont’d) -Issues in social news services Issues Journalistic values, such as accuracy, credibility, reliability, are considered important p 1997 2009 Popularity Media Credibility Slashdot’s Meta-moderation(1999) Daum’s Open Editing Algorithm (2008)
  3. 3. 02 Problems in Voting-Based News Ranking System Fundamental problems: Credibility of news media is determined by news contents, but votes cannot guarantee values of news contents Problems in each algorithms of representative social news services bl i h l ih f i i l i Voting Service Description Limitations Method Simple S Digg Once a submission has earned a critical O Group voting G Popularity Voting mass of Diggs, it is promoted to “popular” problems (Digg > and “most popular.” Mafia, Digg Credibility “Bury” for spam filtering Bury Brigade) Reddit Submission score is determined by the Group voting number of Upvotes minus the number of problems downvotes (Reddit downmod squad) News Credit Article / Discredit Article Group voting Cred problems “credibility?” Weighted Slashdot Moderation-Metamoderation system: Credibility of metamoderators Voting metamoderators are randomly chosen in are not considered the system and rate moderators Rating News An rating instrument is given for users to Increased user burden Based Trust rate news articles based on news values Voting
  4. 4. 03 Researches on Journalistic Media Credibility Elements in Traditional Media Credibility Measures – Trustworthiness, expertise, dynamism ( , p , y (Infante, 1980) , ) – Trustworthiness, fairness, bias, completeness, respect for privacy, representation of individual interests, accuracy, concern for community well-being, separation of fact and opinion, concern for public interest, factual foundations of information published, qualifications of reporters (Rimmer and Weaver, 198 ) bli h d lifi i f ( i d 1987) – Believability, fairness, accuracy, depth of information (Johnson and Kaye, 1998, 2000) Elements in Online News Credibility Measures – factual/opinionated, unfair/fair, accurate/inaccurate, untrustworthy/trustworthy, balan ced/unbalanced, biased/unbiased, reliable/unreliable, thorough/not thotough, informative/not informative (Ognianova, 1998) – Ki i (1999) measured online news credibility by asking whether online news is Kiousis d li dibili b ki h h li i factual, concerned with making profits, concerned with community well- being, invades people’s privacy with 5-point Likert-type scale
  5. 5. 03 Researches on Journalistic Media Credibility (Cont’d) Related debates concerning news value elements Related R l t d works k Researcher R h S. Bowman and C. Willis, We Media 2003 We the Media D. Gilmore, 2006 Blog, Blogging, Journalism & Credibility: Battleground and Common Participatory R MacKinnon, 2005 Ground Journalism Online News Credibility: An Examination of the Perceptions of W. P. Cassidy, 2007 Newspaper Journalists Restating News Values: Contemporary Criteria for Selecting the J. Mcgregor, 2007 News
  6. 6. 04 Research Questions & Process Assumption: “There will be a group of people who vote for journalistically valuable news stories” Q Q1. How to measure j journalistic value of social news contents? – What are the attributes that valuable news stories have? – How to measure values of a number of news contents? – Should each news value elements have different weight for online social news contents? Q2. Who Q2 Wh are voters th t tent to vote for journalistically valuable news?(Value Voters) t that t t t t f j li ti ll l bl ?(V l V t ) – How to derive value voters? – Do they show consistent tendency in voting behavior? 1. Measuring 2. Finding Journalistic Value Alpha Voters 3. Evaluation News Set 2 News Set 1 voted by Value voters with Value Scores Value Voters News Set 2 With Value Scores News Raters 1 News Set 2 voted by general voters News R t N Raters 2 General Voters
  7. 7. 05 Measuring Journalistic Value of Social News Contents C. Lampe and R. Kelly Garrett’s Methodology – Compare performances of several rating instruments (normative, descriptive, full, mini review) – Subjects rated original and degraded news story with 4 rating instruments. – We adopt normative review because it shows the highest level of accuracy in that the result shows similar news values as those from experts. - Representative News Value Elements - Accuracy of ratings by instrument and story quality Attribute Description Instrument Err2 | Err | sd r Accuracy Full .80 .69 .06 .285** Credibility Descriptive 1.39 .94 .07 .234** Normative Fairness Normative .64 .62 .05 .218* Informativeness Mini 2.90 1.43 .09 .326** Originality Balance y Clarity Context Descriptive Diversity Evidence Objectivity Transparency
  8. 8. 05 Measuring Journalistic Value of Social News Contents (Cont’d) 369 well-educated people measure values of top 100 weekly best current news stories of Daum View with normative review instrument (Accuracy, Credibility, Fairness, Informativeness, Originality) To derive importance of each element in online news, our survey asked subjects to rate them with 5-point Likert scale, and we rescaled them so that the news value score ranges from 1 to 5 Derived i h f D i d weights for each elements are listed as follows h l li d f ll - Weights for each rating items Accuracy Credibility Fairness Informativeness Originality 0.225 0.227 0.198 0.202 0.148
  9. 9. 05 Measuring Journalistic Value of Social News Contents (Cont’d) Finally, we derive the value scores of top 100 current news of Daum Blogger News Examples of survey result Previous Rank URL Value 89 http://v.daum.net/link/3918133 http://v daum net/link/3918133 4.109859 4 109859 93 http://v.daum.net/link/3930134 3.996898 76 http://v.daum.net/link/3931410 3.882244 45 http://v.daum.net/link/3912458 3.856029 22 http://v.daum.net/link/3931027 3.807791 47 http://v.daum.net/link/3912861 3.777746 67 http://v.daum.net/link/3926142 3.705755 29 http://v.daum.net/link/3913246 3.703777 87 http://v.daum.net/link/3905833 3.646875 33 http://v.daum.net/link/3948105 http://v daum net/link/3948105 3.642477 3 642477 36 http://v.daum.net/link/3915024 3.633902 18 http://v.daum.net/link/3902557 3.625941 30 http://v.daum.net/link/3902704 3.622996 59 http://v.daum.net/link/3922091 3.591387 70 http://v.daum.net/link/3900538 3.584172 83 http://v.daum.net/link/3913077 3.580856 57 http://v.daum.net/link/3911474 3.578499 43 http://v.daum.net/link/3928202 3.552268 64 http://v.daum.net/link/3925490 http://v daum net/link/3925490 3.521333 3 521333 25 http://v.daum.net/link/3913131 3.519752
  10. 10. 06 Finding Value Voters We consider news stories whose value scores are over 3.5 as valuable news and votes for the articles valid. – In 5-point Likert Sclae, score 3 is a neutral point – We assume that news scores from 3.5 to 5 are positive points. We i l W simply measure each users’ valid voting rate (number of valid votes divided h ’ lid ti t ( b f lid t di id d by total number of votes) Voters who made less than 5 votes are excluded from value voter list. list – Users with less than 5 votes can have high valid voting rate by chance We flexibly adjust the lowest limit of valid voting rate from 0.3 to 0.5 y j g
  11. 11. 07 Evaluation 232 well-educated people participated in the second survey and evaluate another 100 current news stories of Daum View We compared the value scores of top 50 news contents with the votes made by alpha voters The result is as follows – Previous votes does not show any significant correlation with news value scores. – Votes from value voters shows significant correlation with news value scores – Selected with the valid voting rate over 0.4, alpha voters’ votes show significant correlation with news value scores 70 # voters News Value Score 60 Previous N/A Pearson Correlation -.063 Votes Sig. (2-tailed) .665 50 Votes Pearson Correlation .288* 40 VVR > 273 0.5 Sig. (2-tailed) .043 V 30 VVR > 620 Pearson Correlation .326* 20 0.4 Sig. (2-tailed) .021 10 VVR > 914 Pearson Correlation .287* 0.3 Sig. (2-tailed) .043 0 2 2.5 3 3.5 4 News Value Score
  12. 12. 08 Works in Progress… This paper is on the initial stage of deriving value voters and adapting the votes to news articles We will conduct researches concerning… – Diversifying weights on the valuable stories (e.g. (e g based on news values of articles or excluding malicious votes) – Finding correlation between news values and news stories’ attributes. ( g (e.g. correlation between news values of articles and number of trackback or comments) ) – Developing a news ranking model based on news values Calculating Voters’ Reputation value Voters Seed Articles Valuable News Ranking System based on Alpha Voters Votes
  13. 13. Thank You