Invited presentation to the Hypotheses, Evidence & Relationships Workshop, Elsevier Disruptive Technologies Lab, May 2009: http://hyp-er.wik.is

1. HypER Workshop: Hypotheses, Evidence and Relationships
11-12 May 2009, Elsevier, Amsterdam
The Hypermedia Discourse Project
Tools for Annotating, Visualizing & Navigating
Literature as Discourse Networks
Simon Buckingham Shum
Knowledge Media Institute
The Open University
Milton Keynes, UK
http://people.kmi.open.ac.uk/sbs
http://projects.kmi.open.ac.uk/hyperdiscourse
1

2. http://projects.kmi.open.ac.uk/hyperdiscourse
Compendium Institute
2

3. http://projects.kmi.open.ac.uk/hyperdiscourse
Compendium Institute
3

4. questions
4

5. 1665 throws a long shadow
From: To…?
Chaomei Chen, 2006: Citation analysis
Le Journal des Sçavans
January 1665
Philosophical Transactions
of the Royal Society of
London
March 1665 Buckingham Shum et al, 2003: lineage analysis
Buckingham Shum, S. (2007). Digital Research Discourse? Computational Thinking Seminar Series, School of Informatics, 5
University of Edinburgh, 25 Apr. 2007. http://kmi.open.ac.uk/projects/hyperdiscourse/docs/Simon-Edin-CompThink.pdf

6. The question we used to ask in 2001 at
the start of the ScholOnto project
 In 2010, will we still be publishing scientific results
primarily as prose papers, or will a complementary
infrastructure emerge that exploits the power of
the social, semantic web to model the literature as
a network of claims and arguments?
6

7. The question we used to ask in 2001 at
the start of the ScholOnto project
20xx?
 In 2010, will we still be publishing scientific results
primarily as prose papers, or will a complementary
infrastructure emerge that exploits the power of
the social, semantic web to model the literature as
a network of claims and arguments?
7

8. Questions the next generation scientific
infrastructure should help answer
• “What is the evidence for this claim?”
• “Was this prediction accurate?”
• “What are the conceptual foundations for this idea?”
• “Who’s built on this idea? How?”
• “Who’s challenged this idea? Why? How?”
• “Are there distinctive perspectives on this problem?”
• “Are there inconsistencies within this school of thought?”
8

9. assumptions
9

10.  Researchers read meanings into texts that are not
there, and with which the author might disagree
 so we will always require manual annotation tools
 we need ways to make connections to connections
 extremely complex connections may remain the province of human sensemaking
(e.g. is analogous to)
 Good user interfaces will be needed
 to view, edit and navigate HypERnets, whether manually or automatically constructed
 Scientific discourse is a social process
 we take huge care in our writing about how we position ourselves in relation to our
peers — will we trust unsupervised machines to extract and position our more
complex claims?
10

11. modelling
schemes: IBIS
11

12. Rittel’s IBIS: Issue-Based Information System
12

13. Compendium: customisable, collaborative,
hypermedia IBIS mapping
Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley, C. and Nuseibeh, B. (2006). Hypermedia Support for Argumentation-Based
Rationale: 15 Years on from gIBIS and QOC. In: Rationale Management in Software Engineering (Eds.) A.H. Dutoit, R. McCall, I. Mistrik, and B. Paech.
13
Springer-Verlag: Berlin

14. IBIS mapping of Iraq debate
Buckingham Shum, S., and A. Okada. 2008. Knowledge cartography for controversies: The Iraq debate. In Knowledge cartography: 14
Software tools and mapping techniques, ed. A. Okada, S. Buckingham Shum, and T. Sherborne, 249–66. London: Springer.

15. Mapping a nuclear power debate on a blog

16. Mapping a nuclear power debate on a blog

17. Mapping a nuclear power debate on a blog

18. modelling
schemes:
ScholOnto
18

19. ScholOnto schema
Connecting freeform tags with naturalistic connections (“dialects”)
grounded in a formal set of relations (from semiotics and coherence relations)
Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. 19
International Journal of Human Computer Studies, 64, (11), pp.1154-1171. [PrePrint: http://oro.open.ac.uk/6441]

20. Scholarly discourse as CKS…
Beyond document citations… Making formal connections
These annotations are between ideas creates a
freeform summaries of an semantic citation network —>
idea, as one would also find novel literature navigation,
in researchers’ journals, querying and visualization
fieldnotes, lit. review notes “People try to maximise
or blog entries their rate of gaining
“Information scent
information” models”
Method
“Web User Flow by
applies
Theory “Information
Information Scent foraging
(WUFIS)” Claim theory”
?
Paper: “The Scent of a Site: A System for
Analyzing and Predicting Information Scent,
Usage, and Usability of a Web Site”
Paper: “Information
foraging” 20

21. topic maps and subject
centric federation
21

22. Schematic: Documents, Subjects,and
Relations
Relations between
subjects
Topic Map of
documents and their
Subjects in
subjects
documents
Occurrence
links
Document

23. Federated Subjects

24. interaction design
24

25. Interaction design for literature
visualization: pilot study: paper-based literature modelling
S. Buckingham Shum, V. Uren, G. Li, B. Sereno, and C. Mancini. Computational Modelling of Naturalistic Argumentation in Research 25
Literatures: Representation and Interaction Design Issues. International Journal of Intelligent Systems, 22(1):17–47, 2006

26. Interaction design for lit. visualization
From paper prototype to semiformal mapping tool
 The ClaiMapper tool
Starting from paper-based modelling,
move from literature sketches…
…to formal argument maps
Evaluated in: V. Uren, S. Buckingham Shum, G. Li, and M. Bachler. Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User 26
Evaluation. International Journal of Human Computer Studies, 64(5):420–445, 2006

27. Interaction design for doc. annotation
Pilot study: paper-based annotation
Pilot study reported in: B. Sereno, S. Buckingham Shum, and E. Motta. (2005). ClaimSpotter: an Environment to Support 27
Sensemaking with Knowledge Triples. Proc. Int. Conf. Intelligent User Interfaces, pages 199–206, ACM

28. The ClaimSpotter annotation tool
 Web 2.0-style tagging with optional community/system tag
recommendations
Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design:
Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction 28
of
Structured Knowledge, 16th Int. World Wide Web Conference, Banff, Canada; 8-12 May 2007.

29. Lessons Learnt & Design Principles
 Untrained users can do it: in their first hour they created
coherent claims. UI design validated to this degree.
—future work: longitudinal evaluation at scale
 New users attend to what is highlighted for them (matching
tags; primary doct.), and generally don’t click down a level
—next version combines visualizations and document-centric features
 Support incremental formalization
—cf. use of is-about as a placeholder link; provide an Other… category and try
to map automatically to the ontology
 Users’ strategies vary — don’t assume a strong workflow
a paper-based pilot study can provide insights into this
 Web 2.0 UI simplicity: good design needed to provide high
functionality, walk-up-and-use tools
—we overwhelmed some users with overlaid suggestions for tags
Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design:
Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction 29
of
Structured Knowledge, 16th Int. World Wide Web Conference, Banff, Canada; 8-12 May 2007.

30. Cohere: from tag clouds to idea webs
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on
Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421] 30

31. Cohere: embedding an Idea or Map in
another website (a blog post)
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on
Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421] 31

32. Cohere: a mashup visualization merging
different connections around a common Idea
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on
Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421] 32

33. Cohere: semantically filtering a focal
Idea by “contrasting” connections
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on
Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421] 33

34. Cohere: semantically filtering a focal
Idea by “contrasting” connections
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on
Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421] 34

35. “What papers contrast with this paper?”
1. Extract concepts for this document
2. Trace concepts on which they build
3. Trace concepts challenging this set
4. Show root documents
Evaluated in: V. Uren, S. Buckingham Shum, G. Li, and M. Bachler. Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User 35
Evaluation. International Journal of Human Computer Studies, 64(5):420–445, 2006

36. “What is the lineage of this idea?”
Buckingham Shum, S.J., Uren, V., Li, G., Sereno, B. and
Mancini, C. (2007).Modelling Naturalistic Argumentation in
Research Literatures: Representation and Interaction Design
Issues. International Journal of Intelligent Systems, (Special
Issue on Computational Models of Natural Argument, Eds: C.
Reed and F. Grasso, 22, (1), pp.17-47. [PrePrint: http://
oro.open.ac.uk/6463] 36

37. Current projects: scientific collective
intelligence through discourse
 OLnet: Open Learning Network to connect the open
educational resource movement’s discourse/
evidence base: http://olnet.org
 ESSENCE: e-Science/Sensemaking/Climate Change
testing and integrating Web argumentation tools:
http://events.kmi.open.ac.uk/essence
 SocialLearn: Web 3.0 social learning/sensemaking
platform with semantic discourse connections
(launches end of year)
37

38. Marriage made in heaven?
Human Machine
annotation annotation
38

39. Workshop Qs:
 Corpus you are working on; community, type of
content (abstracts, full-text, book..)full text:
 scholarly/scientific, blogs, newspapers, real time
discussions (and video of it), mission doctrine/
policy
 Granularity of knowledge element you are
identifying
 arbitrary: statements, single words, paragraphs
 Relationships between knowledge elements you
have identified
 IBIS: relational types + node types
 ScholOnto: relations + roles...
 Cohere 39

40. Workshop Qs:
 Type of annotation: automatic, manual, combination
 manual annotation
 partial automatic highlighting of text based on
Simone Teufel's work on Argumentative Zoning
 Size of corpus you have annotated so far
 40 pages of blog debate
 12 hours of video
 distill 2 cm of policy docts into IBIS maps
 several books in a literature
 10-30 papers in a sample literature
 30 articles on Iraq
 5 days workshop discussions 40

41. Workshop Qs:
 Data standards, outline of architecture of system
built (if relevant)
 Compendium: XML DTD; SQL
 Cohere API: RDF; XML; JSON
 TopicSpaces: XML Topic Map; RDF; OWL
 Visualisations
 Compendium/ClaiMapper manual maps
 ClaiMaker/Cohere/TopicSpaces generated maps
41

42. Workshop Qs:
 User studies: yes, focusing on interaction design and
usage patterns in both field trials and lab studies
 IUI 2005: evaluation of ClaimSpotter
 IJHCS 2006: evaluation of ClaiMaker
 WWW'07 CKC: evaluation of ClaimSpotter
 IJRME 2008: evaluation of Compendium for mapping
climate change arguments
 Space Ex. Conf 2005: NASA Ames field trials
 DIAC 2008: evaluation of Compendium for mapping
planning discourse
 HCI (under review): evaluation of Compendium
mapping for hostage recovery
42