Simon Buckingham Shum, profile picture
Uploaded bySimon Buckingham Shum
1,278 views

Towards Contested Collective Intelligence

The document discusses the concept of contested collective intelligence in the context of wicked problems, emphasizing the importance of disagreement as a vital part of the discourse process. It explores the design space for tools that facilitate hypermedia discourse, highlighting challenges and potential solutions for synthesizing contributions in collaborative settings. The document also addresses the need for effective visualization and structuring of deliberations to enhance collective decision-making and participation.

Embed presentation

Download to read offline
1 / 111
2 / 111
3 / 111
4 / 111
5 / 111
6 / 111
7 / 111
8 / 111
9 / 111
10 / 111
11 / 111
12 / 111
13 / 111
14 / 111
15 / 111
16 / 111
17 / 111
18 / 111
19 / 111
20 / 111
21 / 111
22 / 111
23 / 111
24 / 111
25 / 111
26 / 111
27 / 111
28 / 111
29 / 111
30 / 111
31 / 111
32 / 111
33 / 111
34 / 111
35 / 111
36 / 111
37 / 111
38 / 111
39 / 111
40 / 111
41 / 111
42 / 111
43 / 111
44 / 111
45 / 111
46 / 111
47 / 111
48 / 111
49 / 111
50 / 111
51 / 111
52 / 111
53 / 111
54 / 111
55 / 111
56 / 111
57 / 111
58 / 111
59 / 111
60 / 111
61 / 111
62 / 111
63 / 111
64 / 111
65 / 111
66 / 111
67 / 111
68 / 111
69 / 111
70 / 111
71 / 111
72 / 111
73 / 111
74 / 111
75 / 111
76 / 111
77 / 111
78 / 111
79 / 111
80 / 111
81 / 111
82 / 111
83 / 111
84 / 111
85 / 111
86 / 111
87 / 111
88 / 111
89 / 111
90 / 111
91 / 111
92 / 111
93 / 111
94 / 111
95 / 111
96 / 111
97 / 111
98 / 111
99 / 111
100 / 111
101 / 111
102 / 111
103 / 111
104 / 111
105 / 111
106 / 111
107 / 111
108 / 111
109 / 111
110 / 111
111 / 111
Simon Buckingham Shum Connected Intelligence Centre • University of Technology Sydney @sbuckshum • http://utscic.edu.au • http://Simon.BuckinghamShum.net Towards Contested Collective Intelligence or… A tour of the CI design space for Hypermedia Discourse University of Melbourne • SWARM Project, 12th Sept. 2017
Contested Collective Intelligence... In wicked problems, there is no master worldview, ontology or logic So disagreement is a necessary process and vital ingredient We can disagree well or badly CI tools should scaffold and improve this proess (e.g. amplify awareness of how stakeholders are framing the problem, reading the signals, seeing connections, and judging success) 2 De Liddo, A., Sándor, Á. and Buckingham Shum, S. (2012). Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work, 21, (4-5), pp. 417-448. http://doi.org/10.1007/s10606-011-9155-x
Dilemmas and (partial) Solutions
Dilemma If everyone just talks with no structure, it’s hard to synthesise CI
© Simon Buckingham Shum 5 Hypermedia Discourse the modelling of discourse / the discourse of modelling …reading and writing networks of documents, concepts, issues, ideas and arguments Buckingham Shum, S. (2006). Sensemaking on the Pragmatic Web: A Hypermedia Discourse Perspective. In: 1st International Conference on the Pragmatic Web, 21-22 Sept 2006, Stuttgart, Germany. ePrint: http://oro.open.ac.uk/6442
© Simon Buckingham Shum 6 Discourse § Dialogue § Deliberation § Argumentation § Reflection (Online & F-F Meetings)
© Simon Buckingham Shum 7 Hypermedia § Modelling discourse relations § Expressing different perspectives on a conceptual space § Supporting the incremental formalization of ideas § Rendering structural visualizations § Connecting heterogeneous content
© Simon Buckingham Shum 8 Discourse Model Key ingredients of a Hypermedia Discourse approach
© Simon Buckingham Shum 9 Notation / Visualisation Discourse Model Key ingredients of a Hypermedia Discourse approach
© Simon Buckingham Shum 10 Notation / Visualisation User Interface Discourse Model Key ingredients of a Hypermedia Discourse approach
© Simon Buckingham Shum 11 Notation / Visualisation User Interface Computational Services Discourse Model Key ingredients of a Hypermedia Discourse approach
© Simon Buckingham Shum 12 Notation / Visualisation User Interface Computational Services Literacy/ Fluency Discourse Model Key ingredients of a Hypermedia Discourse approach
Dilemma If users are required to structure their contributions to a CI repository, the effort must provide tangible benefit (not just potential benefits to future stakeholders)
Solution (in small synchronous settings) A skilled mapper resolves the cost-benefit tradeoff, adding immediate value to the sensemaking
Issue Mapping (or in a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme Buckingham Shum, S. (2003). The roots of computer supported argument visualization. In P. Kirschner, S. Buckingham Shum, & C. Carr (Eds.), Visualizing Argumentation (pp. 3–24). London: Springer. ePrint: http://bit.ly/VizArgRoots
http://compendiuminstitute.net Issue Mapping (or in a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
https://www.youtube.com/watch?v=pxS5wUljfjE Issue Mapping (or in a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
this simple set of moves — combined with hypertext, and mapping fluency — goes a long way… UK Research Excellence Framework (REF) 2014 Impact Case
Compendium software (open source) visual hypermedia for managing the connections between ideas flexibly Deep acknowledgements: Jeff Conklin CogNexus Institute Al Selvin & Maarten Sierhuis NYNEX Science & Technology —> Bell Atlantic —> Verizon —> NASA http://compendiuminstitute.net
20 Structure management in Compendium § Associative linking nodes in a shared context connected by graphical Map links § Categorical membership nodes in different contexts connected by common attributes via metadata Tags § Hypertextual Transclusion reuse of the same node in different views § Templates reuse of the same structure in different views § HTML, XML and RDF data exports for interoperability § Java and SQL interfaces to add services
Compendium Institute: international community http://CompendiumInstitute.net (now archived)
Global Sensemaking Network (2008~2012) http://GlobalSensemaking.net
CogNexus consulting: Issue/Dialogue Mapping http://cognexus.org • http://cognexusgroup.com
Groupaya+CogNexus consulting: Issue/Dialogue Mapping http://delta.groupaya.net
Seven Sigma consulting: Issue/Dialogue Mapping http://www.sevensigma.com.au/what-we- do/sensemaking.html
“Knowledge Artistry” (Al Selvin) Selvin, S. & Buckingham Shum, S. (2015). Constructing Knowledge Art: An Experiential Perspective on Crafting Participatory Representations. Morgan Claypool. http://doi.org/10.2200/S00593ED1V01Y201408HCI023 Hypermedia Discourse fluency at a high level
27 Mapping with IBIS Issue-templates to harvest the firm’s collective intelligence on Y2K contingencies Selvin, A.M. and Buckingham Shum, S.J. (2002). Rapid Knowledge Construction: A Case Study in Corporate Contingency Planning Using Collaborative Hypermedia. Knowledge and Process Management, 9, (2), pp.119-128.
28 Modelling organisational processes in Compendium using a Template
29 Completing a Compendium template
30 Generating Custom Documents and Diagrams from Compendium Templates Selvin, A.M. and Buckingham Shum, S.J. (2002). Rapid Knowledge Construction: A Case Study in Corporate Contingency Planning Using Collaborative Hypermedia. Knowledge and Process Management, 9, (2), pp.119-128.
31 Using Compendium for personnel recovery operations planning Conversational Modelling: real time dialogue mapping combined with model driven templates (AI+IA) DARPA Co-OPR Project (PI: Austin Tate, AIAI, U. Edinburgh) http://www.aiai.ed.ac.uk/project/co-opr
© Simon Buckingham Shum 32 Mission Briefing: Intent template Answers to template issues provided in the JTFC Briefing. Answers may be constrained by predefined options, as specified in the XML schema
© Simon Buckingham Shum 33 Capturing political deliberation/rationale Dialogue Map capturing the planners’ discussion of this option
© Simon Buckingham Shum 34 Planning Engine input to Compendium Issues on which the I-X planning engine provided candidate Options
35 Mapping with IBIS to build a NASA science team’s collective intelligence for planetary geological exploration Clancey, William J.; Sierhuis, Maarten; Alena, Richard L.; Graham, Jeffrey S.; Tyree, Kim S.; Hirsh, Robert L.; Garry, W. Brent; Semple, Abigail; Buckingham Shum, Simon J.; Shadbolt, Nigel and Rupert, Shannon M. (2007). Automating CapCom Using Mobile Agents and Robotic Assistants. In: 1st Space Exploration Conference: Continuing the Voyage ofDiscovery, 30 Jan-1 Feb 2005 , Orlando, FL, US. http://eprints.aktors.org/375
NASA: Mars Habitat field trials in Utah desert
NASA remote science team tools Scientist (Mars) Scientist (Earth) Scientist (Earth) Scientist (Mars) Scientist (Earth) Software Agent Architecture (Mars) Compendium used as a collaboration medium at all intersections: humans+agents reading+writing IBIS maps
Geology dialogue map between Earth-based scientists and ‘Mars’ Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used without permission
Compendium activity plans for surface exploration, constructed by scientists, interpreted by software agents
Compendium science data map, generated by software agents, for interpretation by Mars+Earth scientists
Meeting Replay tool: Earth scientists can browse a (simulated) Mars crew’s planning meeting using Compendium
this simple set of moves — combined with hypertext and mapping fluency — goes a long way… BUT…
Dilemma While co-located mapping is fine for ‘micro-CI’, can we scale this to support asynch. ‘macro-CI’?
Solution Web-based IBIS mapping
Numerous IBIS-based web apps http://oystr.co http://debatemapper.net http://evidence-hub.net http://litemap.net http://cci.mit.edu/klein/deliberatorium.html
Where our tools fit… Given a wealth of documents… 46
Where our tools fit… and tools to detect and render potentially significant patterns… 47
Where our tools fit… and tools to detect and render potentially significant patterns… 48
Where our tools fit: we need ways to express interpretations 49
50 interpretation interpretation interpretation interpretation Where our tools fit: we need ways to express interpretations
51 interpretation interpretationinterpretation interpretation interpretation (a hunch – no grounding evidence yet) interpretation Where our tools fit: we need ways to express interpretations
…and optionally make meaningful connections 52 predictscauses interpretation interpretationinterpretation interpretation interpretation (a hunch – no grounding evidence yet) interpretation Is pre-requisite for
53 prevents predictscauses interpretation interpretationinterpretation interpretation interpretation (a hunch – no grounding evidence yet) Is inconsistent with interpretation challenges Is pre-requisite for …and optionally make meaningful connections
Potentially moving towards stories that make sense of the evidence… i.e. plausible narratives / arguments 54 Question Answer Supporting Argument… Challenging Argument… challengessupports responds to Assumption motivates
Potentially moving towards stories that make sense of the evidence… i.e. plausible narratives / arguments 55 Question Answer Supporting Argument… Challenging Argument… challengessupports responds to Hunch motivates
56 Question Answer Supporting Argument… Challenging Argument… challengessupports responds to Data motivates Potentially moving towards stories that make sense of the evidence… i.e. plausible narratives / arguments
57 Convergence of… web annotation social bookmarking concept mapping structured debate a prototype platform for collective intelligence Opening demo 2:30-10:30: https://www.youtube.com/watch?v=hxI5jPGScoU
Cohere demo (2011): web annotations with discourse connections
Structured deliberation and debate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…) 59
60 Structured deliberation and debate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
— web annotation of document (Firefox extension)
User/community-defined visual language 62
63 Structured deliberation and debate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
Comparison of one’s own ideas to others De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-Centric Learning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff Does the learner compare his/her own ideas to that of peers, and if so, in what ways?
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. (2011). Discourse-centric learning analytics. 1st Int. Conf. Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr). ACM: New York. Eprint: http://oro.open.ac.uk/25829 What epistemic contributions are learners making in the community? 65 Rebecca is playing the role of broker, connecting different peers’ contributions in meaningful ways We now have the basis for recommending that you engage with people NOT like you…
Evidence Many users can make reasonable contributions to IBIS web apps, without training BUT…
Dilemma Asynchronous online mapping is tougher to curate: no on-the- spot sensemaking from a mapper
Solution Familiar looking web interfaces that guide users on how to contribute good IBIS
Evidence Hub: structured storytelling for students, practitioners and researchers Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net A wizard guides the user through the submission of a structured story: • What’s the Issue? • What claim are you making/addressing? • What kind of evidence supports/challenges this? • Link it to papers/data • Index it against the core themes
Evidence Hub: Argument Maps Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net The wizard then generates a structured IBIS tree showing evidence-based claims (and disagreements)
Evidence Hub: professional development http://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning Issue Potential Solution Supporting Evidence (practitioner story)
Dilemma: Unstructured deliberation platforms provide no scaleable assistance in making sense of the collective’s progress
Pain Points in Social Innovation Platforms Catalyst Project Deliverable: http://catalyst-fp7.eu/wp-content/uploads/2014/02/CATALYST-Analysis-of-pain-points-and-user-feedback.pdf
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Effective visualisation of concepts, new ideas and deliberations is essential for shared understanding, but suffers both from a lack of efficient tools to create them and from a lack of ways to reuse them across platforms and debates “As a user, visualisation is my biggest problem. It is often difficult to get into the discussion at the beginning. As a manager of these platforms, showing people what is going on is the biggest pain point.”
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Participants struggle to get a good overview of what is unfolding in an online community debate. Only the most motivated participants will commit a lot of time to reading the debate in order to identify the key members, the most relevant discussions, etc. The majority of participants tend to respond unsystematically to stimulus messages, and do not digest earlier contributions before they make their own contribution to the debate, such is the cognitive overhead and limited time.
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Bringing motivated audiences to commit to action is difficult. Enthusiasts, those who have an interest in a subject but have yet to commit to taking action, are left behind. Need to prompt action in community members Reaching a consensus was considered less important than being enabled to act.
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Motivating participants with widely differing levels of commitment, expertise and availability to contribute to an online debate is challenging and often unproductive. Sustaining participation is more important than enlarging participation. “It is better to have quality input from a small group than a lot of members but very little content”.
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Open innovation systems tend to generate a large number of relatively shallow ideas. Poor collaborative refinement of ideas that could allow the development of more refined, deeply considered contributions. No easy way to see which problem facets remain under- covered. Very partial coverage of the solution space.
Pain Points prioritised by orgs who run social innovation platforms Hard to visualise the debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Patchy evaluation of ideas Poor quality justification for ideas. Hard to see why ratings have been given. Unclear which rationales are evidence based.
Solution Activity analytics + IBIS semantics permit automated checking of the ‘health’ of a conversation
CI in Organisations (CSCW journal special issue) See article by Mark Klein on attention metrics
Crowd-scale deliberation quality metrics + alerts Lead: Mark Klein (MIT/Zurich) https://www.youtube.com/watch?v=UZMJ9mti8h0
Problem-Goal-Exception (PGE) analysis using IBIS syntax checking for potential weaknesses in reasoning http://catalyst-fp7.eu/wp-content/uploads/2016/01/CATALYST_WP4_D4.2b.pdf
Integrating deliberation metrics in the CI- dashboard http://catalyst-fp7.eu/wp-content/uploads/2016/01/CATALYST_WP4_D4.2b.pdf
Integrating deliberation metrics in DebateHub http://catalyst-fp7.eu/wp-content/uploads/2016/01/CATALYST_WP4_D4.2b.pdf
87 “Semantic Google Scholar” — ClaimFinder Victoria Uren, Simon Buckingham Shum, Michelle Bachler, Gary Li, (2006) Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, Vol.64, 5, (420-445).
88 ClaiMaker returns a Lineage tree (the roots of a concept)
Dilemma: Deliberation schemas focus attention on cold rationality, at the expense of social warmth
Solution Addition of social channels in an IBIS mapping web app can restore a sense of connectedness
L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067 Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
Solution Addition of social channels in an IBIS mapping web app can restore a sense of connectedness BUT…
But the group using a Ning discussion forum still outperforms Social-IBIS and Plain-IBIS Mutual Understanding Perceived Effectiveness of Communication L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067 Debate Dashboard socially augmented Cohere mapping Ning discussion forum Cohere
But the group using a Ning discussion forum still outperforms Social-IBIS and Plain-IBIS Accuracy of Prediction (commodity prices)Perceived Ease of Use L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067
Writing is endlessly expressive and hard to improve on as a medium for collective reflection/argumentation (also a social process)
Dilemma: But we would still like the machine to do some work for us in making sense of the state of the CI process or product
Solution NLP could move us beyond simple forum metrics, and help make sense of the quality of contribution
Academic Writing Analytics: feedback on analytical/argumentative or reflective writing Info https://utscic.edu.au/tools/awa
101 Highlighted sentences are colour- coded according to their broad type Sentences with Function Keys have more precise functions (e.g. Novelty) CIC’s automated feedback tool: analytical writing
CIC’s automated feedback tool: reflective writing An early paragraph which is simply setting the scene:
CIC’s automated feedback tool: reflective writing A concluding paragraph moving into professional reflection:
1 CIC’s Text Analytics Pipeline (TAP) A set of linguistic analysis modules + AWA UI —> OSS release Preparation of texts: text cleaning –> de-identification –> indexing –> metadata management Analysis of texts: • Metrics: lengths of words, sentences, paragraphs, and statistics of these • Syllables: metrics at the word level based on syllables • Named Entities: e.g. names of People, Places • Statistics: e.g. noun-verb ratio • Vocabulary: compound words, occurrences at sentence, paragraph and document leve • Expressions: epistemic, self-critique and affective compound words • Spelling: feedback on spelling and basic grammar • Rhetorical moves: in analytical and reflective writing • Complexity: measures of the complexity of words, sentences and paragraphs
Disputational talk characterised by disagreement and individualised decision making. Few attempts to pool resources, to offer constructive criticism or make suggestions. Disputational talk also has some characteristic discourse features - short exchanges consisting of assertions and challenges or counter assertions ('Yes, it is.' 'No it's not!'). Cumulative talk in which speakers build positively but uncritically on what the others have said. Partners use talk to construct a 'common knowledge' by accumulation. Cumulative discourse is characterised by repetitions, confirmations and elaborations. Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168. Disputational/Cumulative/Exploratory talk
Exploratory talk • Partners engage critically but constructively with each other's ideas. • Statements and suggestions are offered for joint consideration. • These may be challenged and counter-challenged, but challenges are justified and alternative hypotheses are offered. • Partners all actively participate and opinions are sought and considered before decisions are jointly made. • Compared with the other two types, in Exploratory talk knowledge is made more publicly accountable and reasoning is more visible in the talk. Disputational/Cumulative/Exploratory talk Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168.
-60 -40 -20 0 20 40 60 80 9:28 9:32 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:13 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:48 11:52 11:54 12:00 12:03 12:04 12:05 Average Exploratory… Discourse analytics on webinar textchat Sheffield, UK not as sunny as yesterday - still warm Greetings from Hong Kong Morning from Wiltshire, sunny here! See you! bye for now! bye, and thank you Bye all for now Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar… Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
-60 -40 -20 0 20 40 60 80 9:28 9:32 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:13 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:48 11:52 11:54 12:00 12:03 12:04 12:05 Average Exploratory… Discourse analytics on webinar textchat …but if we zoom in on a peak… Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
Discourse analytics on webinar textchat -100 -50 0 50 100 9:28 9:40 9:50 10:00 10:07 10:17 10:31 10:45 11:04 11:17 11:26 11:32 11:38 11:44 11:52 12:03 Classified as “exploratory talk” (more substantive for learning) “non- exploratory” …language is used in a manner more akin to “Exploratory Talk” (Neil Mercer) Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
© Simon Buckingham Shum 110 Notation / Visualisation User Interface Computational Services Literacy/ Fluency Discourse Model So, this is the Hypermedia Discourse design space…
Practitioner Fluency Modelling Frameworks Computing Platform Learning Curve Mastery Domain Services Interoperability Discourse Interaction Design Effectiveness Experience Helpful evaluation criteria for CI platforms? Consolidation of the previous elements into 3 classes of evaluation criteria How does the Hypermedia Discourse design space and its tradeoffs compare to the SWARM platform? What can we learn from each other?

More Related Content

PDF
Designing at the Intersection of HCI & AI: Misinformation & Crowdsourced Anno...
byMatthew Lease
 
PDF
Teaching, Assessment and Learning Analytics: Time to Question Assumptions
bySimon Buckingham Shum
 
PDF
Adventures in Crowdsourcing : Toward Safer Content Moderation & Better Suppor...
byMatthew Lease
 
PDF
But Who Protects the Moderators?
byMatthew Lease
 
PDF
UTS CIC2 Briefing, 17 June 2016
bySimon Buckingham Shum
 
PDF
Black Box Learning Analytics? Beyond Algorithmic Transparency
bySimon Buckingham Shum
 
PDF
Learning Analytics vs Cognitive Automation
bySimon Buckingham Shum
 
PDF
Future of Journalism - civil discourse technologies
bySimon Buckingham Shum
 
Designing at the Intersection of HCI & AI: Misinformation & Crowdsourced Anno...
byMatthew Lease
 
Teaching, Assessment and Learning Analytics: Time to Question Assumptions
bySimon Buckingham Shum
 
Adventures in Crowdsourcing : Toward Safer Content Moderation & Better Suppor...
byMatthew Lease
 
But Who Protects the Moderators?
byMatthew Lease
 
UTS CIC2 Briefing, 17 June 2016
bySimon Buckingham Shum
 
Black Box Learning Analytics? Beyond Algorithmic Transparency
bySimon Buckingham Shum
 
Learning Analytics vs Cognitive Automation
bySimon Buckingham Shum
 
Future of Journalism - civil discourse technologies
bySimon Buckingham Shum
 

What's hot

PDF
Analíticas del aprendizaje: una perspectiva crítica
byCENT
 
PDF
Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collectio...
byMatthew Lease
 
PPT
Living in Tech City: 50+ Technology Trends and Innovations Transforming Workp...
bycjbonk
 
PDF
Explainable Fact Checking with Humans in-the-loop
byMatthew Lease
 
PPTX
SOLARSPELL: THE SOLAR POWERED EDUCATIONAL LEARNING LIBRARY - EXPERIENTIAL LEA...
byMicah Altman
 
PDF
Artificial Intelligence AI in Libraries Training for Innovation Webinar
bySaid Ali Said
 
PPT
On Beyond OWL: challenges for ontologies on the Web
byJames Hendler
 
PDF
Trust Management: A Tutorial
byArtificial Intelligence Institute at UofSC
 
PDF
Learning Informatics: AI • Analytics • Accountability • Agency
bySimon Buckingham Shum
 
PPTX
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
byAmit Sheth
 
PDF
Key Challenges in Moderating Social Media: Accuracy, Cost, Scalability, and S...
byMatthew Lease
 
PPTX
A Blind Date With (Big) Data: Student Data in (Higher) Education
byUniversity of South Africa (Unisa)
 
PDF
AI & Work, with Transparency & the Crowd
byMatthew Lease
 
PPTX
Interactive visualization and exploration of network data with gephi
byBernhard Rieder
 
PDF
Designing Human-AI Partnerships to Combat Misinfomation
byMatthew Lease
 
PDF
Teaching the Technologies learning area using a thinking skills approach
byJason Zagami
 
PPTX
Reputation Management for Early Career Researchers
byMicah Altman
 
PPTX
Citizen Sensor Data Mining, Social Media Analytics and Applications
byAmit Sheth
 
PPTX
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
bySimon Knight
 
PDF
Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact...
byMatthew Lease
 
Analíticas del aprendizaje: una perspectiva crítica
byCENT
 
Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collectio...
byMatthew Lease
 
Living in Tech City: 50+ Technology Trends and Innovations Transforming Workp...
bycjbonk
 
Explainable Fact Checking with Humans in-the-loop
byMatthew Lease
 
SOLARSPELL: THE SOLAR POWERED EDUCATIONAL LEARNING LIBRARY - EXPERIENTIAL LEA...
byMicah Altman
 
Artificial Intelligence AI in Libraries Training for Innovation Webinar
bySaid Ali Said
 
On Beyond OWL: challenges for ontologies on the Web
byJames Hendler
 
Trust Management: A Tutorial
byArtificial Intelligence Institute at UofSC
 
Learning Informatics: AI • Analytics • Accountability • Agency
bySimon Buckingham Shum
 
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
byAmit Sheth
 
Key Challenges in Moderating Social Media: Accuracy, Cost, Scalability, and S...
byMatthew Lease
 
A Blind Date With (Big) Data: Student Data in (Higher) Education
byUniversity of South Africa (Unisa)
 
AI & Work, with Transparency & the Crowd
byMatthew Lease
 
Interactive visualization and exploration of network data with gephi
byBernhard Rieder
 
Designing Human-AI Partnerships to Combat Misinfomation
byMatthew Lease
 
Teaching the Technologies learning area using a thinking skills approach
byJason Zagami
 
Reputation Management for Early Career Researchers
byMicah Altman
 
Citizen Sensor Data Mining, Social Media Analytics and Applications
byAmit Sheth
 
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
bySimon Knight
 
Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact...
byMatthew Lease
 

Similar to Towards Contested Collective Intelligence

PDF
From gIBIS to MEMETIC: DCC 2006
bySimon Buckingham Shum
 
PDF
Making Conversations Visible
bySimon Buckingham Shum
 
PPT
Knowledge Mapping for Open Sensemaking Communities
bySimon Buckingham Shum
 
PPT
Knowledge/Argument/Dialogue Mapping and Social Software
bySimon Buckingham Shum
 
PDF
BCII 2016 - Visualizing Complexity
bySimon Buckingham Shum
 
PPTX
Computing for Human Experience [v3, Aug-Oct 2010]
byArtificial Intelligence Institute at UofSC
 
PDF
OD2010- DeLiddo Presentation
byAnna De Liddo
 
PDF
Software Agents in Support of Human Argument Mapping
bySimon Buckingham Shum
 
PPT
The Horizon Report 2008 Presentation
byBo Chamberlain
 
PPT
Social and Collaborative Construction of Structured Knowledge WWW2007
bySimon Buckingham Shum
 
PDF
Supporting Sensemaking by Modelling Discourse as Hypermedia Networks
bySimon Buckingham Shum
 
PPT
Ual2007
bycplp
 
PDF
Gic2011 aula10-ingles
byMarielba-Mayeya Zacarias
 
PDF
Research as Hypermedia Narrative
bySimon Buckingham Shum
 
PDF
Hypermedia Discourse: Theory & Technology for the Pragmatic Web?
bySimon Buckingham Shum
 
PDF
Making Thinking Visible in Complex Times
bySimon Buckingham Shum
 
PDF
Resistance is Futile: The dynamics of the Science Collective
byJudy O'Connell
 
PPT
Generic improvements in communication technology to enhance socio-cognitive g...
byStian Håklev
 
PDF
An Emerging Suite of Social Learning Technologies
bySimon Buckingham Shum
 
PPTX
Collaborative Systems
byPat Parslow
 
From gIBIS to MEMETIC: DCC 2006
bySimon Buckingham Shum
 
Making Conversations Visible
bySimon Buckingham Shum
 
Knowledge Mapping for Open Sensemaking Communities
bySimon Buckingham Shum
 
Knowledge/Argument/Dialogue Mapping and Social Software
bySimon Buckingham Shum
 
BCII 2016 - Visualizing Complexity
bySimon Buckingham Shum
 
Computing for Human Experience [v3, Aug-Oct 2010]
byArtificial Intelligence Institute at UofSC
 
OD2010- DeLiddo Presentation
byAnna De Liddo
 
Software Agents in Support of Human Argument Mapping
bySimon Buckingham Shum
 
The Horizon Report 2008 Presentation
byBo Chamberlain
 
Social and Collaborative Construction of Structured Knowledge WWW2007
bySimon Buckingham Shum
 
Supporting Sensemaking by Modelling Discourse as Hypermedia Networks
bySimon Buckingham Shum
 
Ual2007
bycplp
 
Gic2011 aula10-ingles
byMarielba-Mayeya Zacarias
 
Research as Hypermedia Narrative
bySimon Buckingham Shum
 
Hypermedia Discourse: Theory & Technology for the Pragmatic Web?
bySimon Buckingham Shum
 
Making Thinking Visible in Complex Times
bySimon Buckingham Shum
 
Resistance is Futile: The dynamics of the Science Collective
byJudy O'Connell
 
Generic improvements in communication technology to enhance socio-cognitive g...
byStian Håklev
 
An Emerging Suite of Social Learning Technologies
bySimon Buckingham Shum
 
Collaborative Systems
byPat Parslow
 

More from Simon Buckingham Shum

PDF
ICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
bySimon Buckingham Shum
 
PDF
The Generative AI System Shock, and some thoughts on Collective Intelligence ...
bySimon Buckingham Shum
 
PDF
Deliberative Democracy as a strategy for co-designing university ethics aro...
bySimon Buckingham Shum
 
PDF
AI/Data Analytics (AIDA): Key concepts, examples & risks
bySimon Buckingham Shum
 
PDF
Embracing Imperfection in Learning Analytics
bySimon Buckingham Shum
 
PDF
Conversational, generative AI as a cognitive tool for critical thinking
bySimon Buckingham Shum
 
PDF
Knowledge Art or… “Participatory Improvisational DVN”
bySimon Buckingham Shum
 
PDF
Learning Analytics as Educational Knowledge Infrastructure
bySimon Buckingham Shum
 
PDF
Transitioning Education’s Knowledge Infrastructure ICLS 2018
bySimon Buckingham Shum
 
PDF
Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data
bySimon Buckingham Shum
 
PDF
24/7 Instant Feedback on Writing: Integrating AcaWriter into your Teaching
bySimon Buckingham Shum
 
PDF
SBS_ISLS2022.pdf
bySimon Buckingham Shum
 
PDF
Argumentation 101 for Learning Analytics PhDs!
bySimon Buckingham Shum
 
PDF
March 2021 • 24/7 Instant Feedback on Writing: Integrating AcaWriter into yo...
bySimon Buckingham Shum
 
PDF
Knowledge Art - MDSI Guest Lecture - 1st May 2019
bySimon Buckingham Shum
 
PDF
Could Generative AI Augment Reflection, Deliberation and Argumentation?
bySimon Buckingham Shum
 
PDF
UX/LX for PLSA: Workshop Welcome
bySimon Buckingham Shum
 
PDF
On the Design of a Writing App offering 24/7 Formative Feedback on Reflective...
bySimon Buckingham Shum
 
PDF
Is “The Matter With Things” also what’s the matter with Learning Analytics?
bySimon Buckingham Shum
 
PDF
Educational Data Scientists: A Scarce Breed
bySimon Buckingham Shum
 
ICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
bySimon Buckingham Shum
 
The Generative AI System Shock, and some thoughts on Collective Intelligence ...
bySimon Buckingham Shum
 
Deliberative Democracy as a strategy for co-designing university ethics aro...
bySimon Buckingham Shum
 
AI/Data Analytics (AIDA): Key concepts, examples & risks
bySimon Buckingham Shum
 
Embracing Imperfection in Learning Analytics
bySimon Buckingham Shum
 
Conversational, generative AI as a cognitive tool for critical thinking
bySimon Buckingham Shum
 
Knowledge Art or… “Participatory Improvisational DVN”
bySimon Buckingham Shum
 
Learning Analytics as Educational Knowledge Infrastructure
bySimon Buckingham Shum
 
Transitioning Education’s Knowledge Infrastructure ICLS 2018
bySimon Buckingham Shum
 
Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data
bySimon Buckingham Shum
 
24/7 Instant Feedback on Writing: Integrating AcaWriter into your Teaching
bySimon Buckingham Shum
 
SBS_ISLS2022.pdf
bySimon Buckingham Shum
 
Argumentation 101 for Learning Analytics PhDs!
bySimon Buckingham Shum
 
March 2021 • 24/7 Instant Feedback on Writing: Integrating AcaWriter into yo...
bySimon Buckingham Shum
 
Knowledge Art - MDSI Guest Lecture - 1st May 2019
bySimon Buckingham Shum
 
Could Generative AI Augment Reflection, Deliberation and Argumentation?
bySimon Buckingham Shum
 
UX/LX for PLSA: Workshop Welcome
bySimon Buckingham Shum
 
On the Design of a Writing App offering 24/7 Formative Feedback on Reflective...
bySimon Buckingham Shum
 
Is “The Matter With Things” also what’s the matter with Learning Analytics?
bySimon Buckingham Shum
 
Educational Data Scientists: A Scarce Breed
bySimon Buckingham Shum
 

Recently uploaded

PPTX
Lateral Thinking in - Managerial Skills
bycscprabh
 
PPTX
RESTRICTION ENDONUCLEASE ,TYPES , MECHANISM , APPLICATION
byRemya M S
 
PDF
Cognitive Architectures: Information Compression, Meaning-Making, and Drift i...
byCognitive Drift Institute
 
PPTX
What is Blockchain Security is managed in order to protect some vital informa...
bycscprabh
 
PDF
Enhancing the Role of Knowledge Holders in the UN CBD: Pioneering Capacity-Bu...
bypensoftservices
 
PPTX
Typical_Menstrual_Cycle introductiom .pptx
bysofiamansoor4
 
PPT
Electron Impact Scattering r-matrix method.ppt
byVinayakSavarkar2
 
PPTX
Biodiversity-Chapter-5-Topic-1-Diversity-Indices.pptx
byannep5208
 
PPTX
the achaemenid history from Cyrus till Darius III.
bymaki maki
 
PPTX
Biodiversity and stability.pptx for grade 10
byROLANARIBATO3
 
PDF
Biopsychology 10th Edition PDF by John P. J. Pinel
bykonivon907
 
PDF
Cloud microphysical processes: graduate lecture slides on Cloud Physics UNR C...
byDDM
 
PDF
Class - 10 Science Book Full Concept Explanation With Last Five Years Solved ...
byDr. Juber Hawaldar
 
PPTX
Copy of quiz bee questions for ISKOWIZARD-2019.pptx
byMaryJoyGEvangelista
 
PPTX
Epigenetics (Protein translation, Genetic Code, Steps involved in Translation...
byRashmiMG2
 
PPTX
CLASSIFICATION OF LIVING ORGANISMS: THE THREE DOMAINS OF LIFE
byHOME
 
PDF
Ch3-phys701-Maram optics for 1234566.pdf
byssuserd90459
 
PDF
ZTF25abjmnps (AT2025ulz) and S250818k: A Candidate Superkilonova from a Subth...
bySérgio Sacani
 
PPTX
Quality of irrigation water and brackish water management.pptx
bySaurabh Upadhyay
 
PDF
Chapter-6-Fundamentals-of-Convection.pdf
bysabaam466
 
Lateral Thinking in - Managerial Skills
bycscprabh
 
RESTRICTION ENDONUCLEASE ,TYPES , MECHANISM , APPLICATION
byRemya M S
 
Cognitive Architectures: Information Compression, Meaning-Making, and Drift i...
byCognitive Drift Institute
 
What is Blockchain Security is managed in order to protect some vital informa...
bycscprabh
 
Enhancing the Role of Knowledge Holders in the UN CBD: Pioneering Capacity-Bu...
bypensoftservices
 
Typical_Menstrual_Cycle introductiom .pptx
bysofiamansoor4
 
Electron Impact Scattering r-matrix method.ppt
byVinayakSavarkar2
 
Biodiversity-Chapter-5-Topic-1-Diversity-Indices.pptx
byannep5208
 
the achaemenid history from Cyrus till Darius III.
bymaki maki
 
Biodiversity and stability.pptx for grade 10
byROLANARIBATO3
 
Biopsychology 10th Edition PDF by John P. J. Pinel
bykonivon907
 
Cloud microphysical processes: graduate lecture slides on Cloud Physics UNR C...
byDDM
 
Class - 10 Science Book Full Concept Explanation With Last Five Years Solved ...
byDr. Juber Hawaldar
 
Copy of quiz bee questions for ISKOWIZARD-2019.pptx
byMaryJoyGEvangelista
 
Epigenetics (Protein translation, Genetic Code, Steps involved in Translation...
byRashmiMG2
 
CLASSIFICATION OF LIVING ORGANISMS: THE THREE DOMAINS OF LIFE
byHOME
 
Ch3-phys701-Maram optics for 1234566.pdf
byssuserd90459
 
ZTF25abjmnps (AT2025ulz) and S250818k: A Candidate Superkilonova from a Subth...
bySérgio Sacani
 
Quality of irrigation water and brackish water management.pptx
bySaurabh Upadhyay
 
Chapter-6-Fundamentals-of-Convection.pdf
bysabaam466
 

Towards Contested Collective Intelligence

  • 1.
    Simon Buckingham Shum ConnectedIntelligence Centre • University of Technology Sydney @sbuckshum • http://utscic.edu.au • http://Simon.BuckinghamShum.net Towards Contested Collective Intelligence or… A tour of the CI design space for Hypermedia Discourse University of Melbourne • SWARM Project, 12th Sept. 2017
  • 2.
    Contested Collective Intelligence... Inwicked problems, there is no master worldview, ontology or logic So disagreement is a necessary process and vital ingredient We can disagree well or badly CI tools should scaffold and improve this proess (e.g. amplify awareness of how stakeholders are framing the problem, reading the signals, seeing connections, and judging success) 2 De Liddo, A., Sándor, Á. and Buckingham Shum, S. (2012). Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work, 21, (4-5), pp. 417-448. http://doi.org/10.1007/s10606-011-9155-x
  • 3.
  • 4.
    Dilemma If everyone justtalks with no structure, it’s hard to synthesise CI
  • 5.
    © Simon BuckinghamShum 5 Hypermedia Discourse the modelling of discourse / the discourse of modelling …reading and writing networks of documents, concepts, issues, ideas and arguments Buckingham Shum, S. (2006). Sensemaking on the Pragmatic Web: A Hypermedia Discourse Perspective. In: 1st International Conference on the Pragmatic Web, 21-22 Sept 2006, Stuttgart, Germany. ePrint: http://oro.open.ac.uk/6442
  • 6.
    © Simon BuckinghamShum 6 Discourse § Dialogue § Deliberation § Argumentation § Reflection (Online & F-F Meetings)
  • 7.
    © Simon BuckinghamShum 7 Hypermedia § Modelling discourse relations § Expressing different perspectives on a conceptual space § Supporting the incremental formalization of ideas § Rendering structural visualizations § Connecting heterogeneous content
  • 8.
    © Simon BuckinghamShum 8 Discourse Model Key ingredients of a Hypermedia Discourse approach
  • 9.
    © Simon BuckinghamShum 9 Notation / Visualisation Discourse Model Key ingredients of a Hypermedia Discourse approach
  • 10.
    © Simon BuckinghamShum 10 Notation / Visualisation User Interface Discourse Model Key ingredients of a Hypermedia Discourse approach
  • 11.
    © Simon BuckinghamShum 11 Notation / Visualisation User Interface Computational Services Discourse Model Key ingredients of a Hypermedia Discourse approach
  • 12.
    © Simon BuckinghamShum 12 Notation / Visualisation User Interface Computational Services Literacy/ Fluency Discourse Model Key ingredients of a Hypermedia Discourse approach
  • 13.
    Dilemma If users arerequired to structure their contributions to a CI repository, the effort must provide tangible benefit (not just potential benefits to future stakeholders)
  • 14.
    Solution (in small synchronoussettings) A skilled mapper resolves the cost-benefit tradeoff, adding immediate value to the sensemaking
  • 15.
    Issue Mapping (orin a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme Buckingham Shum, S. (2003). The roots of computer supported argument visualization. In P. Kirschner, S. Buckingham Shum, & C. Carr (Eds.), Visualizing Argumentation (pp. 3–24). London: Springer. ePrint: http://bit.ly/VizArgRoots
  • 16.
    http://compendiuminstitute.net Issue Mapping (orin a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
  • 17.
    https://www.youtube.com/watch?v=pxS5wUljfjE Issue Mapping (orin a meeting real-time: Dialogue Mapping) based on Horst Rittel’s IBIS scheme
  • 18.
    this simple setof moves — combined with hypertext, and mapping fluency — goes a long way… UK Research Excellence Framework (REF) 2014 Impact Case
  • 19.
    Compendium software (opensource) visual hypermedia for managing the connections between ideas flexibly Deep acknowledgements: Jeff Conklin CogNexus Institute Al Selvin & Maarten Sierhuis NYNEX Science & Technology —> Bell Atlantic —> Verizon —> NASA http://compendiuminstitute.net
  • 20.
    20 Structure management inCompendium § Associative linking nodes in a shared context connected by graphical Map links § Categorical membership nodes in different contexts connected by common attributes via metadata Tags § Hypertextual Transclusion reuse of the same node in different views § Templates reuse of the same structure in different views § HTML, XML and RDF data exports for interoperability § Java and SQL interfaces to add services
  • 21.
    Compendium Institute: internationalcommunity http://CompendiumInstitute.net (now archived)
  • 22.
  • 23.
    CogNexus consulting: Issue/DialogueMapping http://cognexus.org • http://cognexusgroup.com
  • 24.
  • 25.
    Seven Sigma consulting: Issue/DialogueMapping http://www.sevensigma.com.au/what-we- do/sensemaking.html
  • 26.
    “Knowledge Artistry” (AlSelvin) Selvin, S. & Buckingham Shum, S. (2015). Constructing Knowledge Art: An Experiential Perspective on Crafting Participatory Representations. Morgan Claypool. http://doi.org/10.2200/S00593ED1V01Y201408HCI023 Hypermedia Discourse fluency at a high level
  • 27.
    27 Mapping with IBISIssue-templates to harvest the firm’s collective intelligence on Y2K contingencies Selvin, A.M. and Buckingham Shum, S.J. (2002). Rapid Knowledge Construction: A Case Study in Corporate Contingency Planning Using Collaborative Hypermedia. Knowledge and Process Management, 9, (2), pp.119-128.
  • 28.
    28 Modelling organisational processesin Compendium using a Template
  • 29.
  • 30.
    30 Generating Custom Documents andDiagrams from Compendium Templates Selvin, A.M. and Buckingham Shum, S.J. (2002). Rapid Knowledge Construction: A Case Study in Corporate Contingency Planning Using Collaborative Hypermedia. Knowledge and Process Management, 9, (2), pp.119-128.
  • 31.
    31 Using Compendium forpersonnel recovery operations planning Conversational Modelling: real time dialogue mapping combined with model driven templates (AI+IA) DARPA Co-OPR Project (PI: Austin Tate, AIAI, U. Edinburgh) http://www.aiai.ed.ac.uk/project/co-opr
  • 32.
    © Simon BuckinghamShum 32 Mission Briefing: Intent template Answers to template issues provided in the JTFC Briefing. Answers may be constrained by predefined options, as specified in the XML schema
  • 33.
    © Simon BuckinghamShum 33 Capturing political deliberation/rationale Dialogue Map capturing the planners’ discussion of this option
  • 34.
    © Simon BuckinghamShum 34 Planning Engine input to Compendium Issues on which the I-X planning engine provided candidate Options
  • 35.
    35 Mapping with IBISto build a NASA science team’s collective intelligence for planetary geological exploration Clancey, William J.; Sierhuis, Maarten; Alena, Richard L.; Graham, Jeffrey S.; Tyree, Kim S.; Hirsh, Robert L.; Garry, W. Brent; Semple, Abigail; Buckingham Shum, Simon J.; Shadbolt, Nigel and Rupert, Shannon M. (2007). Automating CapCom Using Mobile Agents and Robotic Assistants. In: 1st Space Exploration Conference: Continuing the Voyage ofDiscovery, 30 Jan-1 Feb 2005 , Orlando, FL, US. http://eprints.aktors.org/375
  • 36.
    NASA: Mars Habitatfield trials in Utah desert
  • 37.
    NASA remote scienceteam tools Scientist (Mars) Scientist (Earth) Scientist (Earth) Scientist (Mars) Scientist (Earth) Software Agent Architecture (Mars) Compendium used as a collaboration medium at all intersections: humans+agents reading+writing IBIS maps
  • 38.
    Geology dialogue mapbetween Earth-based scientists and ‘Mars’ Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used without permission
  • 39.
    Compendium activity plansfor surface exploration, constructed by scientists, interpreted by software agents
  • 40.
    Compendium science datamap, generated by software agents, for interpretation by Mars+Earth scientists
  • 41.
    Meeting Replay tool: Earth scientistscan browse a (simulated) Mars crew’s planning meeting using Compendium
  • 42.
    this simple setof moves — combined with hypertext and mapping fluency — goes a long way… BUT…
  • 43.
    Dilemma While co-located mappingis fine for ‘micro-CI’, can we scale this to support asynch. ‘macro-CI’?
  • 44.
  • 45.
    Numerous IBIS-based webapps http://oystr.co http://debatemapper.net http://evidence-hub.net http://litemap.net http://cci.mit.edu/klein/deliberatorium.html
  • 46.
    Where our toolsfit… Given a wealth of documents… 46
  • 47.
    Where our toolsfit… and tools to detect and render potentially significant patterns… 47
  • 48.
    Where our toolsfit… and tools to detect and render potentially significant patterns… 48
  • 49.
    Where our toolsfit: we need ways to express interpretations 49
  • 50.
  • 51.
    51 interpretation interpretationinterpretation interpretation interpretation (a hunch –no grounding evidence yet) interpretation Where our tools fit: we need ways to express interpretations
  • 52.
    …and optionally makemeaningful connections 52 predictscauses interpretation interpretationinterpretation interpretation interpretation (a hunch – no grounding evidence yet) interpretation Is pre-requisite for
  • 53.
    53 prevents predictscauses interpretation interpretationinterpretation interpretation interpretation (a hunch –no grounding evidence yet) Is inconsistent with interpretation challenges Is pre-requisite for …and optionally make meaningful connections
  • 54.
    Potentially moving towardsstories that make sense of the evidence… i.e. plausible narratives / arguments 54 Question Answer Supporting Argument… Challenging Argument… challengessupports responds to Assumption motivates
  • 55.
    Potentially moving towardsstories that make sense of the evidence… i.e. plausible narratives / arguments 55 Question Answer Supporting Argument… Challenging Argument… challengessupports responds to Hunch motivates
  • 56.
    56 Question Answer Supporting Argument… Challenging Argument… challengessupports responds to Data motivates Potentially movingtowards stories that make sense of the evidence… i.e. plausible narratives / arguments
  • 57.
    57 Convergence of… web annotation socialbookmarking concept mapping structured debate a prototype platform for collective intelligence Opening demo 2:30-10:30: https://www.youtube.com/watch?v=hxI5jPGScoU
  • 58.
    Cohere demo (2011):web annotations with discourse connections
  • 59.
    Structured deliberation anddebate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…) 59
  • 60.
    60 Structured deliberation anddebate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
  • 61.
    — web annotationof document (Firefox extension)
  • 62.
  • 63.
    63 Structured deliberation anddebate in which Questions, Evidence and Connections are first class entities (linkable, addressable, embeddable, contestable…)
  • 64.
    Comparison of one’sown ideas to others De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-Centric Learning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff Does the learner compare his/her own ideas to that of peers, and if so, in what ways?
  • 65.
    De Liddo, A.,Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. (2011). Discourse-centric learning analytics. 1st Int. Conf. Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr). ACM: New York. Eprint: http://oro.open.ac.uk/25829 What epistemic contributions are learners making in the community? 65 Rebecca is playing the role of broker, connecting different peers’ contributions in meaningful ways We now have the basis for recommending that you engage with people NOT like you…
  • 66.
    Evidence Many users canmake reasonable contributions to IBIS web apps, without training BUT…
  • 67.
    Dilemma Asynchronous online mappingis tougher to curate: no on-the- spot sensemaking from a mapper
  • 68.
    Solution Familiar looking webinterfaces that guide users on how to contribute good IBIS
  • 69.
    Evidence Hub: structuredstorytelling for students, practitioners and researchers Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net A wizard guides the user through the submission of a structured story: • What’s the Issue? • What claim are you making/addressing? • What kind of evidence supports/challenges this? • Link it to papers/data • Index it against the core themes
  • 70.
    Evidence Hub: Argument Maps SystemsLearning & Leadership Evidence Hub: http://sysll.evidence-hub.net The wizard then generates a structured IBIS tree showing evidence-based claims (and disagreements)
  • 71.
    Evidence Hub: professionaldevelopment http://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning Issue Potential Solution Supporting Evidence (practitioner story)
  • 72.
    Dilemma: Unstructured deliberation platforms provideno scaleable assistance in making sense of the collective’s progress
  • 73.
  • 74.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation
  • 75.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Effective visualisation of concepts, new ideas and deliberations is essential for shared understanding, but suffers both from a lack of efficient tools to create them and from a lack of ways to reuse them across platforms and debates “As a user, visualisation is my biggest problem. It is often difficult to get into the discussion at the beginning. As a manager of these platforms, showing people what is going on is the biggest pain point.”
  • 76.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Participants struggle to get a good overview of what is unfolding in an online community debate. Only the most motivated participants will commit a lot of time to reading the debate in order to identify the key members, the most relevant discussions, etc. The majority of participants tend to respond unsystematically to stimulus messages, and do not digest earlier contributions before they make their own contribution to the debate, such is the cognitive overhead and limited time.
  • 77.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Bringing motivated audiences to commit to action is difficult. Enthusiasts, those who have an interest in a subject but have yet to commit to taking action, are left behind. Need to prompt action in community members Reaching a consensus was considered less important than being enabled to act.
  • 78.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Motivating participants with widely differing levels of commitment, expertise and availability to contribute to an online debate is challenging and often unproductive. Sustaining participation is more important than enlarging participation. “It is better to have quality input from a small group than a lot of members but very little content”.
  • 79.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Open innovation systems tend to generate a large number of relatively shallow ideas. Poor collaborative refinement of ideas that could allow the development of more refined, deeply considered contributions. No easy way to see which problem facets remain under- covered. Very partial coverage of the solution space.
  • 80.
    Pain Points prioritised by orgs who run social innovation platforms Hard to visualisethe debate Poor summarisation Poor commitment to action Sustaining participation Shallow contributions and unsystematic coverage Poor idea evaluation Patchy evaluation of ideas Poor quality justification for ideas. Hard to see why ratings have been given. Unclear which rationales are evidence based.
  • 81.
    Solution Activity analytics +IBIS semantics permit automated checking of the ‘health’ of a conversation
  • 82.
    CI in Organisations(CSCW journal special issue) See article by Mark Klein on attention metrics
  • 83.
    Crowd-scale deliberation qualitymetrics + alerts Lead: Mark Klein (MIT/Zurich) https://www.youtube.com/watch?v=UZMJ9mti8h0
  • 84.
    Problem-Goal-Exception (PGE) analysisusing IBIS syntax checking for potential weaknesses in reasoning http://catalyst-fp7.eu/wp-content/uploads/2016/01/CATALYST_WP4_D4.2b.pdf
  • 85.
  • 86.
  • 87.
    87 “Semantic Google Scholar”— ClaimFinder Victoria Uren, Simon Buckingham Shum, Michelle Bachler, Gary Li, (2006) Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, Vol.64, 5, (420-445).
  • 88.
    88 ClaiMaker returns aLineage tree (the roots of a concept)
  • 89.
    Dilemma: Deliberation schemas focus attentionon cold rationality, at the expense of social warmth
  • 90.
    Solution Addition of socialchannels in an IBIS mapping web app can restore a sense of connectedness
  • 91.
    L. Iandoli, I.Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067 Async online IBIS Mapping + Social Cues is better than IBIS alone in some respects
  • 92.
    Async online IBISMapping + Social Cues is better than IBIS alone in some respects
  • 93.
    Async online IBISMapping + Social Cues is better than IBIS alone in some respects
  • 94.
    Solution Addition of socialchannels in an IBIS mapping web app can restore a sense of connectedness BUT…
  • 95.
    But the groupusing a Ning discussion forum still outperforms Social-IBIS and Plain-IBIS Mutual Understanding Perceived Effectiveness of Communication L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067 Debate Dashboard socially augmented Cohere mapping Ning discussion forum Cohere
  • 96.
    But the groupusing a Ning discussion forum still outperforms Social-IBIS and Plain-IBIS Accuracy of Prediction (commodity prices)Perceived Ease of Use L. Iandoli, I. Quinto, S. Buckingham Shum, A. De Liddo (2015), On Online Collaboration and Construction of Shared Knowledge: Assessing Mediation Capability in Computer Supported Argument Visualization Tools, Journal of the Association for Information Science and Technology, 75 (5), pp.1052-1067
  • 97.
    Writing is endlesslyexpressive and hard to improve on as a medium for collective reflection/argumentation (also a social process)
  • 98.
    Dilemma: But we wouldstill like the machine to do some work for us in making sense of the state of the CI process or product
  • 99.
    Solution NLP could moveus beyond simple forum metrics, and help make sense of the quality of contribution
  • 100.
    Academic Writing Analytics:feedback on analytical/argumentative or reflective writing Info https://utscic.edu.au/tools/awa
  • 101.
    101 Highlighted sentences arecolour- coded according to their broad type Sentences with Function Keys have more precise functions (e.g. Novelty) CIC’s automated feedback tool: analytical writing
  • 102.
    CIC’s automated feedbacktool: reflective writing An early paragraph which is simply setting the scene:
  • 103.
    CIC’s automated feedbacktool: reflective writing A concluding paragraph moving into professional reflection:
  • 104.
    1 CIC’s Text AnalyticsPipeline (TAP) A set of linguistic analysis modules + AWA UI —> OSS release Preparation of texts: text cleaning –> de-identification –> indexing –> metadata management Analysis of texts: • Metrics: lengths of words, sentences, paragraphs, and statistics of these • Syllables: metrics at the word level based on syllables • Named Entities: e.g. names of People, Places • Statistics: e.g. noun-verb ratio • Vocabulary: compound words, occurrences at sentence, paragraph and document leve • Expressions: epistemic, self-critique and affective compound words • Spelling: feedback on spelling and basic grammar • Rhetorical moves: in analytical and reflective writing • Complexity: measures of the complexity of words, sentences and paragraphs
  • 105.
    Disputational talk characterised by disagreement and individualiseddecision making. Few attempts to pool resources, to offer constructive criticism or make suggestions. Disputational talk also has some characteristic discourse features - short exchanges consisting of assertions and challenges or counter assertions ('Yes, it is.' 'No it's not!'). Cumulative talk in which speakers build positively but uncritically on what the others have said. Partners use talk to construct a 'common knowledge' by accumulation. Cumulative discourse is characterised by repetitions, confirmations and elaborations. Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168. Disputational/Cumulative/Exploratory talk
  • 106.
    Exploratory talk • Partners engage critically but constructively with each other's ideas. • Statements and suggestions are offered for joint consideration. •These may be challenged and counter-challenged, but challenges are justified and alternative hypotheses are offered. • Partners all actively participate and opinions are sought and considered before decisions are jointly made. • Compared with the other two types, in Exploratory talk knowledge is made more publicly accountable and reasoning is more visible in the talk. Disputational/Cumulative/Exploratory talk Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168.
  • 107.
    -60 -40 -20 0 20 40 60 80 9:28 9:32 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:13 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:48 11:52 11:54 12:00 12:03 12:04 12:05 Average Exploratory… Discourse analyticson webinar textchat Sheffield, UK not as sunny as yesterday - still warm Greetings from Hong Kong Morning from Wiltshire, sunny here! See you! bye for now! bye, and thank you Bye all for now Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar… Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
  • 108.
    -60 -40 -20 0 20 40 60 80 9:28 9:32 9:36 9:40 9:41 9:46 9:50 9:53 9:56 10:00 10:05 10:07 10:07 10:09 10:13 10:17 10:23 10:27 10:31 10:35 10:40 10:45 10:52 10:55 11:04 11:08 11:11 11:17 11:20 11:24 11:26 11:28 11:31 11:32 11:35 11:36 11:38 11:39 11:41 11:44 11:46 11:48 11:52 11:54 12:00 12:03 12:04 12:05 Average Exploratory… Discourse analyticson webinar textchat …but if we zoom in on a peak… Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
  • 109.
    Discourse analytics onwebinar textchat -100 -50 0 50 100 9:28 9:40 9:50 10:00 10:07 10:17 10:31 10:45 11:04 11:17 11:26 11:32 11:38 11:44 11:52 12:03 Classified as “exploratory talk” (more substantive for learning) “non- exploratory” …language is used in a manner more akin to “Exploratory Talk” (Neil Mercer) Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
  • 110.
    © Simon BuckinghamShum 110 Notation / Visualisation User Interface Computational Services Literacy/ Fluency Discourse Model So, this is the Hypermedia Discourse design space…
  • 111.
    Practitioner Fluency Modelling Frameworks Computing Platform Learning Curve Mastery Domain Services Interoperability Discourse Interaction Design Effectiveness Experience Helpful evaluation criteriafor CI platforms? Consolidation of the previous elements into 3 classes of evaluation criteria How does the Hypermedia Discourse design space and its tradeoffs compare to the SWARM platform? What can we learn from each other?