PARC, Apr 1st 2011Contested Collective IntelligenceResilience, Complexity & SensemakingSimon Buckingham Shum & Anna De Lid...
Acknowledgements  Open Learning Network project (2009-12): olnet.org   funded by the William & Flora Hewlett Foundation   ...
Your team, organization, school, professionalnetwork, community...                                                3
Your team, organization, school, professionalnetwork, community...                                                4
Your team, organization, school, professionalnetwork, community...                                                5
How do we augment this system’s capacity tosense, respond to, and shape its environment?§  Through the lens of complex   ...
How do we augment this system’s capacity tosense, respond to, and shape its environment?§  Through the lens of complex   ...
Resilience§  Walker, et al. (2004) define resilience as  “the capacity of a system to absorb    disturbance and reorganiz...
Resilience Platformshttp://www.futureofed.org/driver/Platforms-for-Resilience.aspx                                        ...
Resilience Platformshttp://www.futureofed.org/driver/Platforms-for-Resilience.aspxCreating flexibility and innovation amid...
Resilience in knowledge-intensive ecosystemsWhen knowledge and understanding arekey variables in the system, resiliencedep...
How does this help?Working hypothesis:  Confronted by overwhelming complexity...    (e.g. incomplete, ambiguous data, comp...
Augmenting human intellect (ack. Engelbart)Phenomenon                                                                  Rol...
Designing CI to embody resilience principlesResilience principle                      Role for CI infrastructure?       bu...
How do we augment this system’s capacity tosense, respond to, and shape its environment?§  Through the lens of    sensema...
Sensemaking: the search for plausible, narrativeconnections§  In their review of sensemaking, Klein, et al. conclude:   §...
SensemakingWeick proposes that:§  “Sensemaking is about such things as    placement of items into frameworks,    comprehe...
SensemakingWeick:§  “The point we want to make here is that    sensemaking is about plausibility, coherence,    and reaso...
contested collective intelligence...conversations are critical to sensemaking      there is no master worldview  we need C...
Where our tools fit… Given a wealth ofdocuments…                                         20
Where our tools fit… Given a wealth ofdocuments, and tools to detect and renderpotentially significant patterns…          ...
Where our tools fit… Given a wealth ofdocuments, and tools to detect and renderpotentially significant patterns…          ...
Where our tools fit: making meaningfulconnections between information elements…                                           ...
Where our tools fit: making meaningfulconnections between interpretations       interpretation                        inte...
Where our tools fit: making meaningfulconnections between interpretations             interpretation                      ...
Where our tools fit: making meaningfulconnections between information elements             interpretation                 ...
Where our tools fit: making meaningfulconnections between information elements             interpretation                 ...
Where our tools fit: building the story that makessense of the evidence… i.e. plausible arguments                         ...
Where our tools fit: building the story that makessense of the evidence… i.e. plausible arguments                         ...
Where our tools fit: building the story that makessense of the evidence… i.e. plausible arguments                         ...
a prototype infrastructure forcollective intelligence/social learning                http://cohere.open.ac.ukConvergence o...
Structured deliberation and debate in whichQuestions, Evidence and Connections arefirst class entities (linkable, addressa...
Structured deliberation and debate in whichQuestions, Evidence and Connections arefirst class entities (linkable, addressa...
— web annotation of OER (Firefox extension)
User/community-defined visual language                                         35
Structured deliberation and debate in whichQuestions, Evidence and Connections arefirst class entities (linkable, addressa...
Concept                SocialNetwork               Network            Social          Discourse           Network
Cohere analytics By looking at the post type table it is possible to evaluate learner’s performance connecting the discour...
Cohere analytics                   Legend:                             Neutral link type                             Posit...
Comparing usage of connections
Comparison of one’s own ideas to others                                                                       Does the lea...
Link broker: connecting other people’s ideas                                                                          Does...
seeing the connections people make asthey annotate the web using Cohere    Visualizing all the connections that a         ...
— semantic filtering of connections                                                                                       ...
— web annotation for sensemaking
OLnet	  is	  	                                                            searching	  out	  the	                          ...
Moving from document annotation toconnection-making for sensemaking                                     !
Discourse analysis with Xerox Incremental ParserDetection of salient sentences based on rhetorical markers:BACKGROUND KNOW...
XIP annotation to Cohere
XIP annotations in Cohere’s Firefox Ideas sidebar
XIP annotations in Cohere’s FirefoxConnections sidebar
XIP annotations visualized in Cohere (ack: prefuse)
XIP/Cohere integration: conclusions fromanalysis of the corpus (ack: Ágnes Sándor, XRCE)  §  Machine annotation can effec...
OpenEd Evidence Hub: ci.olnet.organ alpha release                                    54
OpenEd Evidence Hub: ci.olnet.organ alpha release                                    55
OpenEd Evidence Hub: ci.olnet.organ alpha release                                    56
olnet.orgci.olnet.org               57
Upcoming SlideShare
Loading in...5
×

Contested Collective Intelligence: Resilience, Complexity & Sensemaking

6,506

Published on

PARC: Apr 1, 2011

Contested Collective Intelligence: Resilience, Complexity & Sensemaking

Simon Buckingham Shum & Anna De Liddo

Knowledge Media Institute, Open Learning Network Project
Open University UK
http://people.kmi.open.ac.uk/sbs
http://people.kmi.open.ac.uk/anna

ABSTRACT

To thrive, organizational entities (learning communities; teams of analysts; formal companies) must make sense of a complex, changing environment. Our interest is in how sociotechnical “collective intelligence” infrastructures may augment this capacity. We are seeking conceptual lenses that illuminate this challenge, and draw ideas from resilience thinking, sensemaking, and complexity science. We propose that these motivate the concept of Contested Collective Intelligence (CCI), and give examples of how the Cohere platform is being designed in response to these requirements. This is a social/semantic web annotation and knowledge mapping environment, with tools for monitoring networks of ideas and generating novel analytics. We also report experimental integration with the Xerox Incremental Parser, in order to evaluate human+machine annotation of knowledge-level claims expressed through rhetorical moves in documents.

Simon Buckingham Shum is a Senior Lecturer and Associate Director (Technology) at the UK Open University’s Knowledge Media Institute (KMi), where he leads the Hypermedia Discourse Group. Following a PhD at U. York in HCI/Hypertext/Design Rationale (sponsored by Xerox EuroPARC) he has developed a human-centered computing perspective to the challenge of computer-supported sensemaking, reflected in the books Visualizing Argumentation and Knowledge Cartography. He co-founded the Compendium Institute and LearningEmergence.net. http://people.kmi.open.ac.uk/sbs

Anna De Liddo is a Research Associate in KMi, where she works with Simon on the Open Learning Network project (olnet.org), focusing on the design and development of a Collective Intelligence infrastructure for the Open Education Resources movement. She gained her PhD at Polytechnic of Bari, investigating ICT for Participatory Planning and Deliberation, after which she held a postdoctoral position in KMi evaluating human-centred argument mapping for Climate Change. http://people.kmi.open.ac.uk/anna

Published in: Business

Contested Collective Intelligence: Resilience, Complexity & Sensemaking

  1. 1. PARC, Apr 1st 2011Contested Collective IntelligenceResilience, Complexity & SensemakingSimon Buckingham Shum & Anna De LiddoKnowledge Media Institute, Open Learning Network ProjectOpen University UKhttp://people.kmi.open.ac.uk/sbshttp://people.kmi.open.ac.uk/anna 1
  2. 2. Acknowledgements Open Learning Network project (2009-12): olnet.org funded by the William & Flora Hewlett Foundation OLnet Collective Intelligence workstream: http://olnet.org/collective-intelligence Developing conceptual foundations and infrastructure (people+proceses+tools) for Contested Collective Intelligence on the open social web. Example: Open Education Evidence Hub http://ci.olnet.org (alpha) 2
  3. 3. Your team, organization, school, professionalnetwork, community... 3
  4. 4. Your team, organization, school, professionalnetwork, community... 4
  5. 5. Your team, organization, school, professionalnetwork, community... 5
  6. 6. How do we augment this system’s capacity tosense, respond to, and shape its environment?§  Through the lens of complex adaptive systems, resilience and network science...§  Through the lens of sensemaking and HCI... §  Hypermedia Discourse: social- semantic web + models of discourse 6
  7. 7. How do we augment this system’s capacity tosense, respond to, and shape its environment?§  Through the lens of complex adaptive systems, resilience and network science... §  many interacting agents (human and software) §  many weak signals that can build up unexpectedly §  diversity and redundancy §  feedback loops §  visual analytics to reveal emergent patterns and network properties §  ability to withstand change and shock to the system 7
  8. 8. Resilience§  Walker, et al. (2004) define resilience as “the capacity of a system to absorb disturbance and reorganize while undergoing change, so as to still retain essentially the same function, structure, identity, and feedbacks” 8
  9. 9. Resilience Platformshttp://www.futureofed.org/driver/Platforms-for-Resilience.aspx 9
  10. 10. Resilience Platformshttp://www.futureofed.org/driver/Platforms-for-Resilience.aspxCreating flexibility and innovation amid system failures “Platforms for resilience - enabling responsive flexibility, distributed collaboration, and transparency - will allow institutions to meet such challenges through innovation, adaptation, and openness.” 10
  11. 11. Resilience in knowledge-intensive ecosystemsWhen knowledge and understanding arekey variables in the system, resiliencedepends on the capacity for learninge.g. awareness of discrepant evidence,critical practice, reflection and dialoguewhen confronted by challenges or shocksto the system. 11
  12. 12. How does this help?Working hypothesis: Confronted by overwhelming complexity... (e.g. incomplete, ambiguous data, complex adaptive systems, diverse perspectives, technical/social/political dimensions, time pressure…) …Personal and Collective Cognition break down in particular ways… We need Theories, Tools and Practices in order to create CI for tackling such dilemmas (and we need ways to teach these, both to our children, and the current workforce) 12
  13. 13. Augmenting human intellect (ack. Engelbart)Phenomenon Role for CI infrastructure?Dangers of entrained thinking from experts who •  Pay particular attention to exceptionsfail to recognise a novel phenomenon •  Computer-supported argumentation •  Make the system open to diverse perspectives ontologically, and in usabilityComplex systems only seem to make sense •  Stories and coherent pathways areretrospectively: narrative is an appropriately importantcomplex form of knowledge sharing and •  Reflection and overlaying of interpretation(s)reflection for such domains is criticalPatterns are emergent •  Generate gestalt views from the data evidenced in the platform, not from preconceptionsMuch of the relevant knowledge is tacit, shared •  Scaffold the formation of significant inter-through discourse, not formal codifications personal, learning relationshipsMany small signals can build over time into a •  Enable individuals to highlight importantsignificant force/change events and connections à aggregate •  Recommend connections based on different kinds of significant relationship 13Sources include: Weick (1995); Kurtz & Snowden (2003); Browning, L. and Boudès, T. (2005); Hagel et al (2010)
  14. 14. Designing CI to embody resilience principlesResilience principle Role for CI infrastructure? build in the potential for diversity •  manage diversity of worldviews, and the tensions this sets up make tight feedback loops •  shared awareness of dis/agreement amongst peers promote building of trust/social capital •  using social media to build learning particularly for learning and relationships: trust, affirmation, challenge sensemaking enable experimentation •  effective dissemination of findings in relation to key issues and what is already known use a decentralised, modular •  both technically (enabling innovation, architecture interoperability and mashups) but also in how we represent interpretations (ideas as networks, not big chunks of text)a stable state – however temporary – in •  model key coherence relations; explore epistemic terms is a plausible narrative narrative indexing 14
  15. 15. How do we augment this system’s capacity tosense, respond to, and shape its environment?§  Through the lens of sensemaking and HCI... §  many plausible narratives: what was, is, or might be going on?... • cri tical t §  many representational artifacts • arg hinkin being shared and annotated ument g • rhe ation §  attention to the quality of torica conversation: how well are • ass l mov agents listening to each other umpti es and what kinds of contributions • ana ons logica do they make? • ca u l thin §  informal interaction mixed with sality king • jux stronger public claims taposi §  many connections being made, • “ki tions nda r both formal and fuzzy elated ...” 15
  16. 16. Sensemaking: the search for plausible, narrativeconnections§  In their review of sensemaking, Klein, et al. conclude: §  “By sensemaking, modern researchers seem to mean something different from creativity, comprehension, curiosity, mental modeling, explanation, or situational awareness, although all these factors or phenomena can be involved in or related to sensemaking. Sensemaking is a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively. […] A frame functions as a hypothesis about the connections among data.” 16
  17. 17. SensemakingWeick proposes that:§  “Sensemaking is about such things as placement of items into frameworks, comprehending, redressing surprise, constructing meaning, interacting in pursuit of mutual understanding, and patterning.” (Weick, [23], p.6) 17
  18. 18. SensemakingWeick:§  “The point we want to make here is that sensemaking is about plausibility, coherence, and reasonableness. Sensemaking is about accounts that are socially acceptable and credible” ([23] p.61) 18
  19. 19. contested collective intelligence...conversations are critical to sensemaking there is no master worldview we need CI infrastructures to pool awareness of how people are reading small signals, and amplify important connections 19
  20. 20. Where our tools fit… Given a wealth ofdocuments… 20
  21. 21. Where our tools fit… Given a wealth ofdocuments, and tools to detect and renderpotentially significant patterns… 21
  22. 22. Where our tools fit… Given a wealth ofdocuments, and tools to detect and renderpotentially significant patterns… 22
  23. 23. Where our tools fit: making meaningfulconnections between information elements… 23
  24. 24. Where our tools fit: making meaningfulconnections between interpretations interpretation interpretation interpretation interpretation 24
  25. 25. Where our tools fit: making meaningfulconnections between interpretations interpretation interpretation interpretation(a hunch – no grounding evidence yet) interpretation interpretation interpretation 25
  26. 26. Where our tools fit: making meaningfulconnections between information elements interpretation Is pre-requisite for interpretation interpretation(a hunch – no grounding evidence yet) causes predicts interpretation interpretation interpretation 26
  27. 27. Where our tools fit: making meaningfulconnections between information elements interpretation Is pre-requisite for prevents interpretation interpretation(a hunch – no grounding Is inconsistent with evidence yet) causes predicts challenges interpretation interpretation interpretation 27
  28. 28. Where our tools fit: building the story that makessense of the evidence… i.e. plausible arguments Question responds to motivates Answer Assumption supports challenges Supporting Challenging Argument… Argument… 28
  29. 29. Where our tools fit: building the story that makessense of the evidence… i.e. plausible arguments Question responds to motivates Answer Hunch supports challenges Supporting Challenging Argument… Argument… 29
  30. 30. Where our tools fit: building the story that makessense of the evidence… i.e. plausible arguments Question responds to motivates Answer Data supports challenges Supporting Challenging Argument… Argument… 30
  31. 31. a prototype infrastructure forcollective intelligence/social learning http://cohere.open.ac.ukConvergence of…web annotationsocial bookmarkingconcept mappingstructured debate 31
  32. 32. Structured deliberation and debate in whichQuestions, Evidence and Connections arefirst class entities (linkable, addressable, embeddable, contestable…) 32
  33. 33. Structured deliberation and debate in whichQuestions, Evidence and Connections arefirst class entities (linkable, addressable, embeddable, contestable…) 33
  34. 34. — web annotation of OER (Firefox extension)
  35. 35. User/community-defined visual language 35
  36. 36. Structured deliberation and debate in whichQuestions, Evidence and Connections arefirst class entities (linkable, addressable, embeddable, contestable…) 36
  37. 37. Concept SocialNetwork Network Social Discourse Network
  38. 38. Cohere analytics By looking at the post type table it is possible to evaluate learner’s performance connecting the discourse outcomes with the specific learning goal.
  39. 39. Cohere analytics Legend: Neutral link type Positive link type Negative link type
  40. 40. Comparing usage of connections
  41. 41. Comparison of one’s own ideas to others Does the learner compare his/her own ideas to that of peers, and if so, in what ways?De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-CentricLearning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff
  42. 42. Link broker: connecting other people’s ideas Does the learner act as a broker, connecting the ideas of his/her peers, and if so, in what ways?De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L.(2011). Discourse-CentricLearning Analytics. Proc. 1st Int. Conf. Learning Analytics & Knowledge. Feb. 27-Mar 1, 2011, Banff
  43. 43. seeing the connections people make asthey annotate the web using Cohere Visualizing all the connections that a set of analysts have made — but unfiltered, this may not be very helpful
  44. 44. — semantic filtering of connections Visualizing multiple learners’ interpretations of global warming sources Connections have been filtered by a set of semantic relationships grouped as ConsistencyDe Liddo, A. and Buckingham Shum, S. (2010). Cohere: A prototype for contested collective intelligence. In: ACM Computer Supported Cooperative Work(CSCW 2010) - Workshop: Collective Intelligence In Organizations, February 6-10, 2010, Savannah, Georgia, USA. http://oro.open.ac.uk/19554
  45. 45. — web annotation for sensemaking
  46. 46. OLnet  is     searching  out  the   evidence  for    http://www.flickr.com/photos/bartelomeus/4184705426/ effec4ve  OER,   and  building  an     Evidence  Hub     —  a  living    map     by,  of  and  for   the  OER  movement  —   and  those  we  need   to  impact  
  47. 47. Moving from document annotation toconnection-making for sensemaking !
  48. 48. Discourse analysis with Xerox Incremental ParserDetection of salient sentences based on rhetorical markers:BACKGROUND KNOWLEDGE: NOVELTY: OPEN QUESTION:Recent studies indicate … ... new insights provide direct … little is known … evidence ... … role … has been elusive… the previously proposed … ... we suggest a new ... approach ... Current data is insufficient …… is universally accepted ... ... results define a novel role ...CONRASTING IDEAS: SIGNIFICANCE: SUMMARIZING:… unorthodox view resolves … studies ... have provided The goal of this study ...paradoxes … important advances Here, we show ...In contrast with previous Knowledge ... is crucial for ... Altogether, our results ...hypotheses ... understanding indicate... inconsistent with past valuable information ... fromfindings ... studiesGENERALIZING: SURPRISE:... emerging as a promising We have recently observed ...approach surprisinglyOur understanding ... has grown We have identified ... unusual Ágnes Sándor & OLnet Project: http://olnet.org/node/512exponentially ... The recent discovery ... suggests... growing recognition of the intriguing rolesimportance ...
  49. 49. XIP annotation to Cohere
  50. 50. XIP annotations in Cohere’s Firefox Ideas sidebar
  51. 51. XIP annotations in Cohere’s FirefoxConnections sidebar
  52. 52. XIP annotations visualized in Cohere (ack: prefuse)
  53. 53. XIP/Cohere integration: conclusions fromanalysis of the corpus (ack: Ágnes Sándor, XRCE) §  Machine annotation can effectively draw attention to key issues and contrasting ideas, in a cost effective and timely manner §  Human annotation adds higher-level cognitive activities such as abstracting, contextualizing and summarizing. An appropriate combination of both machine and human annotation can augment and enhance both human and machine analysis.
  54. 54. OpenEd Evidence Hub: ci.olnet.organ alpha release 54
  55. 55. OpenEd Evidence Hub: ci.olnet.organ alpha release 55
  56. 56. OpenEd Evidence Hub: ci.olnet.organ alpha release 56
  57. 57. olnet.orgci.olnet.org 57
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×