Hypermedia Discourse: Theory & Technology for the Pragmatic Web?

3,340 views

Published on

Seminar to the AI Centre, SRI International, CA

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,340
On SlideShare
0
From Embeds
0
Number of Embeds
656
Actions
Shares
0
Downloads
75
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Hypermedia Discourse: Theory & Technology for the Pragmatic Web?

  1. 1. SRI International, AI Centre, 5 May 2007 Hypermedia Discourse: Theory & Technology for the Pragmatic Web? Simon Buckingham Shum Knowledge Media Institute & Computing Research Centre The Open University, Milton Keynes, UK kmi.open.ac.uk/people/sbs sbs@acm.org Web Pragmatics Hypermedia Discourse 1 © Simon Buckingham Shum
  2. 2. Acknowledgements Compendium Project: Scholarly Ontologies Project: Al Selvin (Verizon/Open U.) Victoria Uren Maarten Sierhuis (NASA) Gangmin Li Jeff Conklin (CogNexus Inst.) Clara Mancini Michelle Bachler (Open U.) Neil Benn Bertrand Sereno John Domingue Enrico Motta Funding gratefully acknowledged: 2 © Simon Buckingham Shum
  3. 3. web pragmatics? the Pragmatic Web? pragmatic webs? “THE PRAGMATIC WEB CONFERENCE is a unique forum to envision and debate how the emerging social, semantic, multimedia Web mediates the ways in which we construct shared meaning. While there is much research and development into topics relevant to this challenge such as collaboration, usability, knowledge representation, and social informatics, the Pragmatic Web conference provides common ground for dialogue at the nexus of these topics.” 3 © Simon Buckingham Shum
  4. 4. CONTEXT CONVERSATION When contexts change, This is how trust is built, and meanings change in things get done. How do we conversations, documents, and do this fluently over the web? models of the world. How does this shape our use for formal semantics on the Web? What is the interest in the “Pragmatic Web”? COMMITMENT / ACTION How do we support the expression, and coordination, of commitments over the Web? 4 © Simon Buckingham Shum
  5. 5. Pragmatic Web research challenges —from the ICPW’07 CfP: www.PragmaticWeb.info  How can we better understand the usefulness, and limitations, of a concept such as quot;Web Pragmatics”  What pragmatic design principles improve websites where trust and commitment to action are central?  What are the tradeoffs for users of more structured Web collaboration media? (e.g. in learnability, scaleability, intelligibility)  How can participatory work practices and collaboration tools be orchestrated in the design of the standards, data models and ontologies that underpin data-driven Web applications? 5 © Simon Buckingham Shum
  6. 6. Pragmatic Web research challenges  What role does pragmatics play in the design of personalised information and personalised actions channelled through the Web?  What impact (intended or unintended, productive or disruptive) do different levels of computational infrastructure have on Web pragmatics?  How can we clarify our understandings of increasingly important concepts on the Web such as quot;social tiesquot;, quot;metadataquot;, quot;knowledge representationquot;, and quot;transactionquot;?  If quot;contextquot; is pivotal in making human interaction meaningful, how can we take context into account to improve Web applications? 6 © Simon Buckingham Shum
  7. 7. The essence of this talk: Pragmatics Web Hypermedia Discourse 7 © Simon Buckingham Shum
  8. 8. The essence of this talk: Web Pragmatics :-) Hypermedia Discourse 8 © Simon Buckingham Shum
  9. 9. Modelling ~ Discourse The discourse of modelling: How can we support the discourse that drives the development and contextualisation of the models underpinning interactive systems? Modelling discourse: Can we usefully model (structure) sensemaking discourse, without in the process obstructing it? 9 © Simon Buckingham Shum
  10. 10. Discourse Verbal and written workplace communication Discourse communities: “making and taking perspectives” Dialogue Argumentation Claim making Analytical narrative Meetings 10 © Simon Buckingham Shum
  11. 11. Hypermedia Modelling discourse relations Expressing different perspectives on a conceptual space Supporting the incremental formalization of ideas Rendering structural visualizations Connecting heterogeneous content 11 © Simon Buckingham Shum
  12. 12. Characteristics of Hypermedia Discourse Discourse Ontology Literacy/ Notation(s) Fluency Intuitive Computational User Interface Services 12 © Simon Buckingham Shum
  13. 13. 2 examples of Hypermedia Discourse approach and tool support • personal or group concept mapping • real time meeting Compendium capture • participatory modelling • discourse as semantic hypertext • Web publishing of Scholarly scholarly claims and Ontologies argumentation • discourse as semantic Project hypertext 13 © Simon Buckingham Shum
  14. 14. • personal or group concept mapping • real time meeting Compendium capture • participatory modelling • discourse as semantic hypertext 14 © Simon Buckingham Shum
  15. 15. Key elements of Compendium • Shared visual display • Simple notation • Template patterns • Node transclusions Knowledge • Tagging • Hypermedia Media • Interoperability with other data, services and user interfaces Modelling Practitioner skills Frameworks e.g. • Cognitive skills to chunk and link ideas e.g. (Buckingham Shum) • IBIS • Dialogue Mapping (Conklin) • CommonKADS • Conversational Modelling (Sierhuis & Selvin) • World Modelling • Participatory Hypermedia Construction • Critical Systems Heuristics (Selvin) 15 © Simon Buckingham Shum
  16. 16. Discourse grounded in Horst Rittel’s IBIS: Issue-Based Information System 16 © Simon Buckingham Shum
  17. 17. Compendium: hypertext discourse mapping/conceptual modelling 17 © Simon Buckingham Shum
  18. 18. Compendium: Descendent of gIBIS 18 © Simon Buckingham Shum
  19. 19. Structure management in Compendium Associative linking  nodes in a shared context connected by graphical Map links Categorical membership  nodes in different contexts connected by common attributes via metadata Tags Hypertextual Transclusion  reuse of the same node in different views Templates  reuse of the same structure in different views HTML, XML and RDF data exports for interoperability  Java and SQL interfaces to add services  19 © Simon Buckingham Shum
  20. 20. Compendium as the technical and intellectual ‘glue’ Java/RDF Advanced Knowledge Technologies project: www.aktors.org 20 © Simon Buckingham Shum
  21. 21. Compendium as sensemaking hub for emergency response semantic web tools Advanced Knowledge Technologies project: www.aktors.org 21 © Simon Buckingham Shum
  22. 22. Modelling using Issue-templates 22 © Simon Buckingham Shum
  23. 23. Modelling organisational processes in Compendium using a Template 23 © Simon Buckingham Shum
  24. 24. Completing a Compendium template 24 © Simon Buckingham Shum
  25. 25. Generating Custom Documents and Diagrams from Compendium Templates Field Integrated/ Deviations/ Specific Installation Assignable Approvals Revised Changes Assignments Details/ Inventory Requirements (Engr Sched) /Assignment Specs/NDO Notice (E1) List Build Assignable Inventory Assignable Inventory 25 © Simon Buckingham Shum
  26. 26. Collaborative Ontology Design and Merging with Compendium Buckingham Shum, S., Motta, E. and Domingue, J. (2002). Augmenting Design Deliberation with Compendium: Motta, Domingue, The Case of Collaborative Ontology Design. in Workshop on Facilitating Hypertext-Augmented Collaborative Modelling, ACM Hypertext Conference. [PrePrint: http://cognexus.org/ht02]. Modelling, Conference. [PrePrint: http://cognexus.org/ht02]. 26 © Simon Buckingham Shum
  27. 27. Setup for collaborative ontology design and rationale capture 27 © Simon Buckingham Shum
  28. 28. Result of initial requirements discussion 28 © Simon Buckingham Shum
  29. 29. Dialogue Mapping informal discussion 29 © Simon Buckingham Shum
  30. 30. ‘Home window’ of the emerging ontology, linking to design discussions 30 © Simon Buckingham Shum
  31. 31. Template-driven Conversational Modelling of ontology class/slot structure 31 © Simon Buckingham Shum
  32. 32. Using Compendium for personnel recovery planning Co-OPR Project (with Austin Tate): http://www.aiai.ed.ac.uk/project/co-opr 32 © Simon Buckingham Shum
  33. 33. Co-OPR – Sample Screens Left screen: Compendium Middle screen: I-X Process Right screen: I-X Process intelligence database and Panels showing current Panels communicating discussion/rationale state of plan execution and with the external world, capture. situation map e.g. Isolated Personnel 33 © Simon Buckingham Shum
  34. 34. Co-OPR: Operational Sensemaking Situation analysis COA refinement COA generation Operational Sensemaking support for JPRC COA Other comparison/evaluation Tools + Detailed Planning 34 © Simon Buckingham Shum
  35. 35. Co-OPR Scenario Information 35 © Simon Buckingham Shum
  36. 36. PR Doctrine for Situation Analysis extracted as an Issue Template Link to the source doctrine document Issues that require attention (as specified in the doctrine document) Relevant extract from doctrine publication inside the node for reference 36 © Simon Buckingham Shum
  37. 37. Completed Issue Template (1/2) 37 © Simon Buckingham Shum
  38. 38. Completed Issue Template (2/2) Compendium knowledge elements (nodes) can be overlaid on any image, and can point to any media file 38 © Simon Buckingham Shum
  39. 39. Example ONA database maps Maps of  multimedia data from the ONA db Response Mechanism Force data converted to Issue template by drag+drop from Excel spreadsheet 39 © Simon Buckingham Shum
  40. 40. Imported database on Blue Forces 40 © Simon Buckingham Shum
  41. 41. JPRC Compendium Homepage  Sensemaking workflow: Links start from JTFC Briefing, to use of ONA database, to initial COA exploration, to worksheet analysis, to a COA Comparison worksheet for final briefing back to JTFC 41 © Simon Buckingham Shum
  42. 42. Issue Templates for JTFC Briefing The JTFC’s Briefing is  captured in a set of issue templates For each category (menu  item on left) there are a number of issues awaiting answers 42 © Simon Buckingham Shum
  43. 43. JTFC Briefing: Intent template Answers to template issues provided in the JTFC Briefing. Answers may be constrained by predefined options, as specified in the XML schema 43 © Simon Buckingham Shum
  44. 44. Crisis Action Planning template built in an hour Following distribution of the Crisis Action Planning process which was to be followed, a CAP template was created at short notice to support the process 44 © Simon Buckingham Shum
  45. 45. COA Wargame Analysis Worksheet Links to the Situation Analysis and COA Key Issues from PR Comparison Doctrine are listed here worksheet Constraints raised by the JTFC are ‘docked’ here as visual reminders Actors Rows for recording critical events which provoke discussion 45 © Simon Buckingham Shum
  46. 46. PMESII and Relative Superiority Issue Palettes The high level Issues can be expanded if required to see the sub-Issues raised by PR doctrine 46 © Simon Buckingham Shum
  47. 47. Compendium Wargame Analysis Worksheet An argument map is dragged from the template when some aspect of the COA provokes discussion Input from PMESII analysts The discussion map integrated into the discussion map ‘behind’ the cell is then built on the fly 47 © Simon Buckingham Shum
  48. 48. COA-2.1 Wargaming Worksheet (2/2) Dialogue Map capturing the planners’ discussion of this option 48 © Simon Buckingham Shum
  49. 49. COA Comparison Worksheet Summary of how COAs trade off against each other Constraints Doctrinal Issues can also be displayed if required 49 © Simon Buckingham Shum
  50. 50. Grey Matter and Silicon I-X Inputs to Compendium • Issues and Responses • Activity Options • Constraints/Maps • Annotations/Notes 50 © Simon Buckingham Shum
  51. 51. Independent evaluation of Co-OPR Evaluation ratings from six members of the planning cell who were supported by Co-OPR tools in the personnel recovery simulation. (Numbers indicate the number of planners assigning the rating.) 51 © Simon Buckingham Shum
  52. 52. Independent evaluation of Co-OPR In the subsequent vignette of the scenario (in which Co-OPR was not due to participate) Compendium was requested to replace (the usual) PowerPoint as the information management tool. 52 © Simon Buckingham Shum
  53. 53. Large scale NASA e-science field trials: Interoperability with other databases, software agents and collaboration tools Clancey, W.J., Sierhuis, M., Alena, R., Berrios, D., Dowding, J., Graham, J.S., Tyree, K.S., Hirsh, R.L., Garry, W.B., Semple, A., Buckingham Shum, S.J., Shadbolt, N. and Rupert, S. (2005). “Automating CapCom Using Mobile Agents and Robotic Assistants.” 1st Space Exploration Conference, American Institute of Aeronautics and Astronautics, 31 Jan- 1 Feb, 2005, Orlando, FL. Available from: AIAA Meeting Papers on Disc [CD-ROM]: Reston, VA, and as Advanced Knowledge Technologies ePrint 375: http://eprints.aktors.org/375 53 © Simon Buckingham Shum
  54. 54. 54 © Simon Buckingham Shum Image Credits--- Mars: NASA/JPL/MSSS; Earth: NASA/JSC; Composite: MSSS
  55. 55. NASA e-science field trials (2004 and 2005) Distributed Mars-Earth planning and data analysis tools for Mars Habitat field trial in Utah desert, supported from US+UK 55 © Simon Buckingham Shum
  56. 56. NASA Mobile Agents Architecture 56 © Simon Buckingham Shum
  57. 57. Collaboration Configuration Compendium used as a collaboration medium at all intersections: humans+agents, reading+writing maps Software Agent Scientist Architecture (Earth) (Mars) Scientist Scientist Scientist Scientist (Earth) (Mars) (Mars) (Earth) 57 © Simon Buckingham Shum
  58. 58. NASA testbed: Compendium activity plans for surface exploration, constructed by scientists on ‘Earth’, interpreted by software agents on ‘Mars’ Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used without permission The Compendium nodes and relationships in this plan were interpreted by Brahms software agents for monitoring and coordinating astronaut and robot activity during surface explorations. 58 © Simon Buckingham Shum
  59. 59. CoAKTinG NASA testbed: Compendium science data map, generated by software agents, for interpretation by Mars+Earth scientists Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used without permission The Compendium maps were autonomously created and populated with science data by Brahms software agents that use models of the mission plan, work process, data flow and science data relationships to create the maps. 59 © Simon Buckingham Shum
  60. 60. CoAKTinG NASA testbed: Compendium-based photo analysis by geologists on ‘Mars’ Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used 60 without permission © Simon Buckingham Shum
  61. 61. NASA testbed: Compendium scientific feedback map from Earth scientists to Mars colleagues Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used 61 without permission © Simon Buckingham Shum
  62. 62. Importing an Argumentation Scheme as an IBIS template compendium.open.ac.uk/openlearn 62 © Simon Buckingham Shum
  63. 63. Importing an Argumentation Scheme as an IBIS template compendium.open.ac.uk/openlearn 63 © Simon Buckingham Shum
  64. 64. Using Compendium to map and automatically index replayable video conferences CoAKTinG Project: www.aktors.org/coakting Memetic Project: www.memetic-vre.net 64 © Simon Buckingham Shum
  65. 65. Collaborative sensemaking in e-Science: Meeting Replay tool for Earth scientists, synchronising video of Mars crew’s discussion as they annotate their mission plans Copyright, 2004, RIACS/NASA Ames, Open University, Southampton University Not to be used without permission © Simon MR Clip: 00:50 NASA Buckingham Shum 65
  66. 66. Memetic Meeting Replay The CoAKTinG project’s results are now mainstreamed in the project’ Access Grid by the JISC Memetic VRE project 66 © Simon Buckingham Shum
  67. 67. Compendium ‘literacy’? …understanding how to write, read, talk and think in hypermedia IBIS …approaches from consultancy in the field, and video analysis in the lab… 67 © Simon Buckingham Shum
  68. 68. Literacy: significant user community www.CompendiumInstitute.org www.CompendiumInstitute.org 68 © Simon Buckingham Shum
  69. 69. Literacy: Cognitive task analysis  Cognitive tasks involved in using a graphical argumentation scheme (Buckingham Shum 1996)  Affordances of graphical DR for coordinating group design (Buckingham Shum et al 1997) 69 © Simon Buckingham Shum
  70. 70. Literacy: the craft skill of IBIS mapping in meetings: “Dialogue Mapping” Jeff Conklin: CogNexus Institute: www.CogNexus.org 70 © Simon Buckingham Shum
  71. 71. Literacy: expertise analysis (Albert Selvin) What is the nature of expert human performance in  creating and modifying real time conceptual structures for groups?  The NASA knowledge mapper role: Conventional  Listening and interpreting facilitation  Intervening in ‘normal’ conversation flow skills  Getting validation for captured material  Building hypertext representations on Knowledge the fly media  Interrelating data and objects facilitation  Adding metadata skills  Software-specific skills Aesthetic and Ethical Implications of Participatory Hypermedia Practice: First Year Report Selvin, A. (2005), Technical Report KMI-05-17, Knowledge Media Institute, Open University, UK 71 © Simon Buckingham Shum
  72. 72. Compendium literacy: expertise analysis Selvin 2005 Practitioner stances  The position of the practitioner with regard to the current activity:  Knowledge Navigator  Facilitator  Participant  Technical Expert  Editor 72 © Simon Buckingham Shum
  73. 73. • Web publishing of Scholarly scholarly claims and Ontologies argumentation • discourse as semantic Project hypertext Will scientific publishing in 2020 still depend solely on the reading, writing, and discovery of written texts? What might a more network-centric complement look like? 73 © Simon Buckingham Shum
  74. 74. In Gutenberg’s shadow (or standing on his shoulders) Newspapers + Invisible Colleges = Scholarly Journals Le Journal des Philosophical Transactions of Sçavans the Royal Society of London January 1665 March 1665 74 © Simon Buckingham Shum
  75. 75. What if we could get search results like this?… “What is the Turing Debate?” One of seven maps in the Mapping Great Debates: Can Computers Think? Series. MacroVU Press. www.macrovu.com (Horn, 2003; Yoshimi, 2006) 75 © Simon Buckingham Shum
  76. 76. Horn (zoomed in) MacroVU Press. www.macrovu.com 76 © Simon Buckingham Shum
  77. 77. Going beyond citations… People try to maximise their rate of gaining Information scent information models extends Web User Flow by Information Information Scent foraging (WUFIS) theory ? “The Scent of a Site: A System for Analyzing and Predicting Information Scent, Usage, and Usability of a Web Site” “Information foraging” 77 © Simon Buckingham Shum
  78. 78. Combining formal relations with the expressive freedom of ‘folksonomies’ Relational classes and dialects (KMi Scholarly Ontologies project) 78 © Simon Buckingham Shum
  79. 79. If we model concepts in a literature as concept maps… (KMi’s ClaiMapper, built on Compendium) KMi’ ClaiMapper, 79 © Simon Buckingham Shum
  80. 80. “Semantic del.icio.us”: KMi’s ClaimSpotter assigning and linking freeform tags Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design: Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction of Structured Knowledge, 16th Int. World Wide Web Conference (WWW 2007), Banff, 8-12 May 2007. http://www2007.org/workshops/paper_30.pdf 80 © Simon Buckingham Shum
  81. 81. “Semantic Google Scholar” KMi’s ClaimFinder 81 © Simon Buckingham Shum
  82. 82. Visualising claims and arguments When multiple analysts annotate web documents via a server, they can generate a shared view of how they see the field, and where they agree/disagree ClaimFinder prototype: claimfinder.open.ac.uk 82 © Simon Buckingham Shum
  83. 83. “What papers contrast with this paper?” 1. Extract concepts for this document 2. Trace concepts on which they build 3. Trace concepts challenging this set 4. Show root documents 83 © Simon Buckingham Shum
  84. 84. Focusing on a concept incoming+outgoing links 84 © Simon Buckingham Shum
  85. 85. “Semantic Google Scholar” KMi’s ClaimFinder 85 © Simon Buckingham Shum
  86. 86. Lineage tree (the roots of a concept) 86 © Simon Buckingham Shum
  87. 87. Indicators of ClaiMaker literacy? expert user makes more extensive use of Claimaker’s semantic structures in interrogating the network than novices Victoria Uren, Simon Buckingham Shum, Michelle Bachler, Gary Li, (2006) Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, Vol.64, 5, (420-445). 87 © Simon Buckingham Shum
  88. 88. Some answers to our questions… Web Pragmatics The discourse of modelling: Hypermedia Discourse Compendium+IBIS can support many forms of discourse and rationale capture throughout the design lifecycle, from early requirements through to maintenance Modelling discourse: It is possible to co-evolve discourse schemes, tools and literacy to mediate sensemaking discourse. Strong evidence for Compendium, emerging evidence for ClaiMaker 88 © Simon Buckingham Shum
  89. 89. Hypermedia Discourse tools: Discourse Ontology Literacy/ Notation(s) Fluency Intuitive Computational User Interface Services 89 © Simon Buckingham Shum
  90. 90. Common Ground between Pragmatic Web and Hypermedia Discourse perspectives Discourse Ontology negotiating Notation(s) meaning Literacy/ Fluency supporting co-evolving new human-human skillsets with & human-agent new tools communication Intuitive User Interface Computational empowering Services augmenting stakeholders collective to negotiate sensemaking meaning 90 © Simon Buckingham Shum
  91. 91. Ongoing work… 91 © Simon Buckingham Shum
  92. 92. Social bookmarking as semiosis Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. International Journal of Human Computer Studies, 64, (11), pp.1154-1171 92 © Simon Buckingham Shum
  93. 93. Annotation as semiosis in the Scholarly Ontologies project Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. International Journal of Human Computer Studies, 64, (11), pp.1154-1171 93 © Simon Buckingham Shum
  94. 94. Making primary and secondary claims as semiotic and discourse moves Secondary claim: a discourse connective Primary claim: a semiotic move Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. International Journal of Human Computer Studies, 64, (11), pp.1154-1171 94 © Simon Buckingham Shum
  95. 95. Combining formal relations with the expressive freedom of ‘folksonomies’ Relational classes and dialects (KMi Scholarly Ontologies project) 95 © Simon Buckingham Shum
  96. 96. polarity source type comparativeness Cognitive Coherence Relations Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. International Journal of Human Computer Studies, 64, (11), pp.1154-1171 96 © Simon Buckingham Shum
  97. 97. Using CCR-based Coherence Patterns to detect candidate “schools of thought” Doctoral work by Neil Benn, KMi 97 © Simon Buckingham Shum
  98. 98. Using CCR-based Coherence Patterns to detect candidate “schools of thought” Doctoral work by Neil Benn, KMi 98 © Simon Buckingham Shum
  99. 99. Using CCR-based Coherence Patterns to detect candidate “schools of thought” Doctoral work by Neil Benn, KMi 99 © Simon Buckingham Shum
  100. 100. Using CCR-based Coherence Patterns to detect candidate “schools of thought” Doctoral work by Neil Benn, KMi 100 © Simon Buckingham Shum
  101. 101. Hypermedia Discourse Project: http://kmi.open.ac.uk/projects/hyperdiscourse 101 © Simon Buckingham Shum

×