Extending Sense-Making Models with Ideas from Cognition and Learning Theories

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    5 Favorites

    Extending Sense-Making Models with Ideas from Cognition and Learning Theories - Presentation Transcript

    1. Extending Sense-Making Models with Ideas from Cognition and Learning Theories Pengyi Zhang Dagobert Soergel Judith L. Klavans Douglas W. Oard
    2. Outline
      • What is sense-making?
      • Theoretical framework
      • A comprehensive sense-making model
      • Pilot testing
      • Conclusions
      • Future work
    3. Sense-making
      • Sense-making is
      • the process of creating an understanding of a problem or task
      • so that further actions may be taken in an informed manner (Stefik et al., 1999).
    4. Sense-making
        • Sense-making is a pre-requisite for many other tasks such as decision making and problem solving;
        • An important part of sense-making involves making clear the interrelated concepts and their relationships in a problem or task space.
    5. A Sample Sense-making Scenario
      • Task description
      • An intelligence analyst is tasked to gather, analyze, and synthesize information related to Al-Bashir and to make recommendations for action in the form of a report.
      • Background
      • The Darfur conflict is a crisis in the Darfur region of western Sudan. Al-Bashir, the Sudanese president, is one of the key players in the area who is believed to have significant responsibility for continuous conflicts in the region. As part of an effort to resolve these armed conflicts, the administration needs to know as much as possible about al-Bashir in order to better negotiate with the involved parties and strategize its efforts.
    6. Theoretical Framework
      • Sense-making models
        • Generic sense-making models
        • Sense-making models of intelligence analysis
        • Conducting research as sense-making
        • Organizational sense-making
      • Learning
        • Schema theory
        • Assimilation theory
        • Generative Learning Theory
        • Structural knowledge acquisition
      • Cognition
        • Types of conceptual changes
        • Cognitive mechanisms
        • Cognitive structures of knowledge
      • Design of related tools
        • Information extraction
        • Representation and visualization tools
        • Note-taking, analysis tools
        • Task-specific and task-independent tools
    7. Sense-making Elements Processes Sensing Making sense Acti vities Mechanisms Accretion Tuning Restructuring
      • Inductive, data-driven
      • Key item extraction
      • Comparison
      • Schema induction
      • Generalization
      • Deductive, structure-driven
      • Definition
      • Specification
      • Elimination
      • Explanation
      • Inference
      Outcomes Identification of gaps Search Exploratory search Focused search for data for structure Building structure Instantiating structure Consuming instantiated structure
      • Other
      • Metaphor
      • Classification
      • Semantic fit
      • Socratic dialogues
    8. An Iterative Sense-making Model Data gap Structure gap Search: exploratory / focused Outcomes Updated knowledge Structure loop Data loop Identification of Gaps Searching for data Instantiating structure Accretion: Instantiated structure Tuning: Adapted structure Re-structuring: New structure Searching for structure Building structure Existing Knowledge Structures and their instantiations with data The iterations proceed from exploratory to focused search and sense-making. Task / Problem Decision / Solution / Task completion
    9. Pilot Test
      • Participants
      • Data collection
      • Data analysis
      • Findings
    10. Participants
      • 6 information science students as surrogate subjects for intelligence analysts
      • Tasked with synthesizing data from a variety of sources, assessing the credibility of information, and evaluating claims based on supporting evidence
      • Trained in using Rosetta - a multilingual, multimedia news retrieval system
    11. Data Collection
      • 6 2-hour task sessions between March 2007 and May 2007, with 2 to 5 participants attending each
      • Data consists of 17 think-aloud protocols – participants were instructed to verbalize their thoughts as they work on the tasks
      • Sessions were logged and participants were interviewed after each session.
    12. A Sample Task Scenario
      • Task T1: al-Bashir (Abridged Version)
      • Omar Hasan Ahmad al-Bashir is a Sudanese military leader, dictator, and current president of Sudan. Your task is to produce a report identifying information to assess the influence of al-Bashir as a basis for policy decisions and diplomatic actions. Requested information includes:
      • key figures, organizations, and countries who have been associated with al-Bashir;
      • his rise to power; and
      • groups who have resisted him and the level of success in their resistance.
    13. Data Analysis – Coding Scheme A Processes Search Sense-making Exploratory search Gap identification Exploratory search for data Data gap Exploratory search for structure Structural gap Focused search Building structures Focused search for data Using automatically extracted results Focused search for structure Extracting relationships manually Instantiating structures Updating knowledge B Conceptual Changes Sense-making success Sense-making failure Accretion Unable to fit data into structure Tuning Re-structuring Unable to build new structure
    14. Data Analysis – Coding Scheme C Cognitive Mechanisms Inductive mechanisms Deductive mechanisms Key item extraction Definition Comparison Specification Similarity Explanation-based mechanisms Differentiation or discrimination Elimination Analogy and metaphor Semantic fit Classification Socratic dialogues Schema induction Inference Generalization D Emerging Codes Added During Analysis Reasons starting a new loop Resolution of conflicts New lead Disregard conflicting evidence Sense-making success Compromise Sense-making failure Accept new evidence Search failure Confusion
    15. Sample Think-aloud Protocol with Coding Protocol (Energy Security Task) Loops Processes Conceptual Changes Cognitive Mechanisms Okay that was actually a very useful search. So let’s still take this query and look at Algeria, ‘cause obviously Algeria and Nigeria are very close… Loop 4 Focused Search for data Comparison I understand some of the keywords in the article but I don’t understand what the article… Sense-making Failure Key item extraction Okay this has to do with Algeria, southern Algeria. The minister of energy… OPEC meeting… so I am going to see what their connections are with OPEC. Building structures Key item extraction … with all the violence in Nigeria, I was expecting to find the same types of political outrage in Algeria… Instantiating structures Comparison and analogy and I’m not seeing any notice of that at all. Updating of knowledge Re-structuring
    16. Findings – Search and Sense-making Loops
      • The overall search and sense-making loops followed four stages:
        • Task analysis
        • exploratory stage
        • focused stage
        • updates of knowledge representation.
      • Reasons for starting a new loop of search and sense-making:
        • Success of previous sense-making
        • Failure of previous sense-making
        • New lead
        • Failure of search
    17. Findings – Cognitive Mechanisms
      • Participants used a two-way approach: data-driven (bottom-up) 80% structure-driven (top-down) 20%
      • Key item extraction and comparison were used most often.
    18. Findings – Dealing with Conflicts
      • Disregard – “… I wanted that article to say something else. I have to disregard it.”
      • Compromise – “…it is not my understanding that this has anything to do with oil. But okay, this is a different take.”
      • Acceptance – “…I thought these countries had similar serious problems. But [in fact, they] seem to be relatively stable and put a lot of effort into establishing their economy.”
      • Confusion – “…Obviously I have no idea what this is about...”
    19. Findings – Role of Instantiated Structures
      • Entities (represented as names) and key concepts, (represented as keywords) were often the basis for relevance judgments.
      • The relationships embedded in new information and between the new information and participants’ previous knowledge seemed to play an important role in structure building and data fitting.
      • Both concepts and relationships seemed to be crucial for updating knowledge.
    20. Conclusions
      • The empirical think-aloud data seem to be consistent with the iterative sense-making model
      • Users used mostly data-driven cognitive mechanisms
      • Instantiated structure elements play an important role in sense-making
    21. Implications and Future Work
      • Theoretical
        • Better understanding of sense-making processes by extending the existing models with ideas from cognition and learning
        • Further testing and refinement.
      • Empirical
        • Better system design based on findings from user studies.
        • Making design suggestions to support sense-making, not just search.
      • Pengyi Zhang
      • [email_address]
      • terpconnect.umd.edu/~pengyi/sensemaking/

    + PVC.ASISTPVC.ASIST, 2 years ago

    custom

    1061 views, 5 favs, 1 embeds more stats

    by Pengyi Zhang
    Dagobert Soergel
    Judith L. Klavan more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1061
      • 1047 on SlideShare
      • 14 from embeds
    • Comments 0
    • Favorites 5
    • Downloads 0
    Most viewed embeds
    • 14 views on http://asispvc.blogspot.com

    more

    All embeds
    • 14 views on http://asispvc.blogspot.com

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories