CSCW2012 talk to doctoral colloquium. Building a standpoints web to support decision-making in Wikipedia.

1. Digital Enterprise Research Institute www.deri.ie
Building a Standpoints Web to Support
Decision-Making in Wikipedia
Jodi Schneider
Doctoral Colloquium at CSCW 2012
2012-02-12
Seattle, Washington
© Copyright 2010 Digital Enterprise Research Institute. All rights reserved.

2. What I’m looking for
Digital Enterprise Research Institute www.deri.ie
 Scoping & focus
 Detailed mentoring on CSCW/HCC methodologies
 Interviewing
 Qualitative Research
 Statistics
 Suggestions for evaluating my work
2

3. Digital Enterprise Research Institute www.deri.ie
3

4. Should we delete this article?
Digital Enterprise Research Institute www.deri.ie
4

5. Improving deletion discussions
Digital Enterprise Research Institute www.deri.ie
 Main problems:
 Newcomers who don’t know how to argue
 Overwhelm of long discussions
 Discussions that happen over and over again
 Deletion as quality control
 Large number of discussions - ~500/week
5

6. Deletion argument
Digital Enterprise Research Institute www.deri.ie
[Delete the article]...hasn't
played since 2008. His
66-73 record is far from
stellar and, in my opinion,
does not merit an article.
>>He pitched last month and
plays for the Venezuelan
League. This meets our
article criteria.
6

7. Goals
Digital Enterprise Research Institute www.deri.ie
 Newcomers who don’t know how to argue
 Characterize the “good” and “bad” arguments
 Develop argument templates
 Provide guidance and support for new users in properly
structuring arguments according to Wikipedia’s rhetorical
standards
 Overwhelm of long discussions
 Develop a claims/argument explorer
 Discussions that happen over and over again
 Prototype an argument bot
 Populate argument maps with mixed-initiative claims
extraction
7

8. Overview
Digital Enterprise Research Institute www.deri.ie
 Corpus:
 All Wikipedia deletion discussions from January 29, 2011
 Perspectives/approaches:
 Argumentation
 CSCW/HCC
 Text analytics
 Ontologies/Social Semantic Web
8

9. Current work
Digital Enterprise Research Institute www.deri.ie
 Analysis of the corpus
 Argument schemes (e.g. expert opinion)
 Factors (e.g. notability, uniqueness)
 Newcomer’s arguments
 Interviews
 Administrators
 Experienced users
 Argument exploration
 Text mining cue words (‘however’, ‘therefore’,…)
 Architecture
 Ontology development
 “Standpoints Web”
9

10. Standpoint
Digital Enterprise Research Institute www.deri.ie
[Delete the article]...hasn't
played since 2008. His
66-73 record is far from
stellar and, in my opinion,
does not merit an article.
Proposition: does not merit
an article
Justification: hasn’t played
since 2008, bad record
10

11. Opposing standpoint
Digital Enterprise Research Institute www.deri.ie
>>He pitched last month and
plays for the Venezuelan
League. This meets our
article criteria.
Proposition: keep the article
Justification: meets our article
criteria
11

13. Possible applications
Digital Enterprise Research Institute www.deri.ie
 Visualize decision-making
 Highlight controversies
 Query opinions and arguments
 Discuss arguments interactively with a bot
 Calculate the “best” options
 Analyze, extract, and represent disagreement
13

14. Thanks!
Digital Enterprise Research Institute www.deri.ie
jodi.schneider@deri.org
http://jodischneider.com/jodi.html
@jschneider

15. Acknowledgments
Digital Enterprise Research Institute www.deri.ie
 Thanks to our collaborators!
 Katie Atkinson, Trevor Bench-Capon, Adam Wyner (Liverpool)
 DERI Social Software Unit
 Rhetorical Structure, W3C Health Care and Life Sciences
 Funding
 Science Foundation Ireland Grant No. SFI/08/CE/I1380
(Líon-2)
 Short-term scientific mission (STSM 1868) from the COST
Action ICO801 on Agreement Technologies
15

16. Digital Enterprise Research Institute www.deri.ie

17. “ELIZA for arguments”
Digital Enterprise Research Institute www.deri.ie
Snaith, Lawrence, & Reed, “Mixed initiative argument in public deliberation,” ODET 2010
17

18. Highlight Controversies
Digital Enterprise Research Institute www.deri.ie
Ennals, R., Trushkowsky, B., & Agosta, J. M. (2010). Highlighting Disputed
Claims on the Web. In WICOW at WWW 2010.

19. Transform Debates into
Argument Frameworks
Digital Enterprise Research Institute www.deri.ie
(1) Households should pay tax for
their garbage.
(4) (1)
Paying tax for garbage increases
recycling, so households should Arrow: premise
pay.
(3) (1)
Wyner, van Engers, & Bahreini.
Recycling more is good, so people From Policy-making Statements
to First-order Logic.
should pay tax for their garbage. EGOV 2010
19

20. Calculate best options
(non-contradictory opinions)
Digital Enterprise Research Institute www.deri.ie
Wyner, van Engers, & Bahreini.
From Policy-making Statements to First-order Logic.
EGOV 2010
20

21. Claims Extraction
Digital Enterprise Research Institute www.deri.ie
 Cue words (“Hence Jaffa Cakes are cakes.”) [Marcu]
 Rhetorical Structure Theory

22. Claims Extraction

23. Important Relationships
Digital Enterprise Research Institute www.deri.ie
 Attacks
 Supports
23

25. Case Study
Digital Enterprise Research Institute www.deri.ie
 Understand:
interviews, observation, and content analysis
 Intervene:
Implement & test the Standpoints Web architecture
on Wikipedia deletion discussions
 Evaluation:
Community feedback
Ontology fitness-for-purpose
Precision & recall?
25

26. Digital Enterprise Research Institute www.deri.ie
 Problem
 Possible Uses of a Knowledge Representation
 Concrete Examples
 Some Current Directions
26

27. The Problem
Digital Enterprise Research Institute www.deri.ie
 The Web is full of opinions & commentary.
 A lot of it disagrees.
 How do we learn from other people, when they
disagree?
27

28. My Approach
Digital Enterprise Research Institute www.deri.ie
 Identify peoples’ views
 Collect the explanations people give
 Create a hypertext web of these views & explanations
28

29. Two Persuasive Messages
Digital Enterprise Research Institute www.deri.ie
29

30. Why: Walton’s dialogue types
Digital Enterprise Research Institute www.deri.ie
30

31. Knowledge-based Claims Vary
Digital Enterprise Research Institute www.deri.ie
 Use of statistics & impersonal information
31

32. Versus personal appeals…
Digital Enterprise Research Institute www.deri.ie
 Where opinions and personal values are explicit
32

33. Why: Knowledge, Emotion, Values
as a Proxy
Digital Enterprise Research Institute www.deri.ie
33

34. Purpose-related keywords
Digital Enterprise Research Institute www.deri.ie
 Knowledge
 statistics
 Values
 truth
 secret
 Rhetoric
 you can thank
 Judgment/Opinion
 eradicate
 tough
 rejecting
34

35. Purpose matters
Digital Enterprise Research Institute www.deri.ie
 Knowledge-oriented discussions are straightforward
to reuse
 Opinion-oriented discussion types may require
caveating or balancing
 emotion makes a discussion more interesting
 can also indicate the potential for bias.
35

36. Twitter: Standpoint
Digital Enterprise Research Institute www.deri.ie
Difference between cakes and biscuits? When stale,
cakes go hard, biscuits go soft. Hence Jaffa Cakes
are cakes. (Was official EU ruling).
View: Jaffa Cakes are cakes
Justification: official EU ruling; go hard when stale
36

37. Visualize
Digital Enterprise Research Institute www.deri.ie
bCisive