Successfully reported this slideshow.

Cognitive Task Analysis of Teams (Team CTA)


Published on

Published in: Technology, Business
  • Outstanding demonstration. Really clear along with useful
    Are you sure you want to  Yes  No
    Your message goes here

Cognitive Task Analysis of Teams (Team CTA)

  1. 2. Cognitive Task Analysis of Teams <ul><li>Yogesh Tadwalkar </li></ul><ul><li>March 22, 2004 </li></ul>
  2. 3. Traditional Task Analysis <ul><li>Task Analysis aims to optimize work performance by matching tasks with human capabilities and limitations </li></ul><ul><li>Human work is comprised of both physical and cognitive activities </li></ul><ul><li>Physical tasks are observable (Overt) </li></ul><ul><li>However, cognitive processes that lead to physical actions are not (Covert) </li></ul>
  3. 4. Cognitive Task Analysis (CTA) <ul><li>Extension of traditional task analysis techniques to uncover information about: </li></ul><ul><ul><li>- Work domain, mental demands on operator </li></ul></ul><ul><ul><li>- Goal Structures </li></ul></ul><ul><ul><li>- Knowledge and Strategies </li></ul></ul><ul><ul><li>- Thought Processes and Decision Making </li></ul></ul><ul><li>Thus, providing a holistic, complete view of human work </li></ul>
  4. 5. Goals of CTA <ul><li>Performance improvement through: </li></ul><ul><ul><li>Better Task and Interface Design </li></ul></ul><ul><ul><li>Ideal Personnel Selection </li></ul></ul><ul><ul><li>Comprehensive Training and Instructions </li></ul></ul><ul><ul><li>Systematic Performance Assessment </li></ul></ul><ul><ul><li>Improved Planning </li></ul></ul>
  5. 6. Cognitive Task Analysis of Teams
  6. 7. Why Conduct Team CTA? <ul><li>Because 2 + 2 ≠ 4 </li></ul><ul><ul><li>Individuals are often required to perform in teams </li></ul></ul><ul><ul><li>Optimum individual performances may not lead to final mission accomplishment if team co-ordination is unsatisfactory </li></ul></ul><ul><li>Thus, individual CTA would not be sufficient for designing optimum task performance </li></ul>
  7. 8. Team CTA <ul><li>As defined by Klein: </li></ul><ul><ul><li>A reflection and representation of the team mind / Knowledge </li></ul></ul><ul><ul><li>Description of the cognitive skills required for effective team performance </li></ul></ul><ul><ul><li>Elaboration of the team decision requirements </li></ul></ul><ul><ul><li>Determination of types of expertise found in effective teams </li></ul></ul>
  8. 9. Objectives of Team CTA <ul><li>Team performance enhancement through: </li></ul><ul><ul><li>Restructuring and Resizing the team </li></ul></ul><ul><ul><li>Designing better: </li></ul></ul><ul><ul><ul><li>Information technology </li></ul></ul></ul><ul><ul><ul><li>Information management strategies </li></ul></ul></ul><ul><ul><ul><li>Human-computer interfaces </li></ul></ul></ul><ul><ul><ul><li>Decision support systems and Communication </li></ul></ul></ul><ul><ul><li>Developing methods for team training and selection </li></ul></ul>
  9. 10. Team Environment
  10. 11. Team Environment <ul><li>Tasks are decomposed and assigned to different team members </li></ul><ul><li>Team members may or may not have direct contact with each other (co-located vs. dispersed teams) </li></ul><ul><li>Tasks can be performed serially or in parallel or overlapping fashion </li></ul><ul><li>Team members possess, acquire and process information, make decisions, solve problems and make plans </li></ul>
  11. 12. Team Environment <ul><li>Teams are intelligent entities </li></ul><ul><li>(Thordsen and Klein, 1989) </li></ul><ul><ul><li>Team members possess, acquire and process </li></ul></ul><ul><ul><li>information </li></ul></ul><ul><ul><li>Communicate with each other </li></ul></ul><ul><ul><li>Make decisions </li></ul></ul><ul><ul><li>Solve problems and </li></ul></ul><ul><ul><li>Make plans </li></ul></ul>
  12. 13. Types of Teams <ul><li>Planning Teams </li></ul><ul><ul><li>Military Command </li></ul></ul><ul><ul><li>Emergency Response at nuclear plants </li></ul></ul><ul><li>Action Teams </li></ul><ul><ul><li>Air Traffic Control </li></ul></ul><ul><ul><li>SWAT </li></ul></ul>
  13. 14. Cognitive Processes of Teams What to measure through Team CTA methods
  14. 15. Cognitive Processes of Teams <ul><li>Methods for team CTA should focus at capturing these 5 processes (Klein): </li></ul><ul><ul><li>1. Shared mental models </li></ul></ul><ul><ul><ul><li>- Team Goals, team member roles </li></ul></ul></ul><ul><ul><li>2. Control of attention </li></ul></ul><ul><ul><ul><li>- Information seeking, filtering, communication </li></ul></ul></ul><ul><ul><li>3. Shared situation awareness </li></ul></ul><ul><ul><ul><li>- Assessment of dynamism, prediction of future </li></ul></ul></ul><ul><ul><li>4. Application of strategies </li></ul></ul><ul><ul><ul><li>- To make decisions, solve problems and plan </li></ul></ul></ul><ul><ul><li>5. Metacognition </li></ul></ul><ul><ul><li>- Self-monitoring, sensing trouble, making alternative plans </li></ul></ul>
  15. 16. A Model for Team Cognition Knowledge Mental Models Attitudes Expectations Skills Member A Member C Member B Team Performance <ul><li>Pre-task Knowledge </li></ul><ul><li>Shared Mental Models </li></ul><ul><li>Strategies </li></ul><ul><li>Dynamic Knowledge </li></ul><ul><li>Control of Attention </li></ul><ul><li>Shared Situation Awareness </li></ul><ul><li>Metacognition </li></ul>(Stout, Cannon-Bowers, and Salas, 1996)
  16. 17. 1. Shared Mental Models <ul><li>The degree to which team members have the same understanding of key processes (Cannon-Bowers, Sales, & Converse, 1992) </li></ul><ul><ul><li>- Knowledge about overall team goals and mission objectives e.g. Football game </li></ul></ul><ul><ul><li>- Roles and responsibility of each team member </li></ul></ul><ul><ul><li>- Team-mate characteristics and preferences </li></ul></ul><ul><ul><li>- Task Procedures, sequences and timing </li></ul></ul><ul><ul><li>- Use and relationship of equipment </li></ul></ul>
  17. 18. Shared Mental Models and Team Performance <ul><li>Shared mental models affect team performance significantly (Oransu, 1990) : </li></ul><ul><ul><li>Anticipating team member behaviours and information requirements </li></ul></ul><ul><ul><li>Performing tasks from a common frame of reference </li></ul></ul><ul><ul><li>Attending to, interpreting, communicating about, and responding to the world more similarly than individuals with discrepant knowledge (Rentsch & Hall, 1994) </li></ul></ul>
  18. 19. Methods to Elicit and Represent Shared Mental Models
  19. 20. Variation of Goal-Directed Analysis (Woods & Hollnagel, 1987) To solicit knowledge of Team Goals and Task Objectives <ul><ul><li>Document Analysis and Observation </li></ul></ul><ul><ul><li>Separate interviews with each team member, followed by group interview </li></ul></ul><ul><ul><li>Representation: Agreement Metric </li></ul></ul>Task Goal Sub-Goal 1 Sub-Goal 2 Sub-Goal 3 Task 1 Agreement Metric Team Goals For Each Member For Whole Team Task 2
  20. 21. Variation of Task-Action Mapping (Coury et al., 1991) To solicit knowledge of Task Procedures, Sequence and Timing <ul><ul><li>Document Analysis and Observation </li></ul></ul><ul><ul><li>Ask each team member to describe key team concepts, associated tasks and specific actions </li></ul></ul><ul><ul><li>Representation: Concept Maps </li></ul></ul>Description Task 1 Task 2 Task 3 <ul><li>Action 1 </li></ul><ul><li>What (Steps) </li></ul><ul><li>Who </li></ul><ul><li>When </li></ul>Concept 1 e.g. Evacuation <ul><li>Action 1 </li></ul><ul><li>What (Steps) </li></ul><ul><li>Who </li></ul><ul><li>When </li></ul>
  21. 22. Bootstrapping (Hoffman, Shadbolt, Burton & Klein 1995) To solicit knowledge of Roles and Responsibilities <ul><ul><li> Bootstrapping </li></ul></ul>Documentation Individual Interviews Observation Group Interviews May not adequately capture dynamic, contextual data like misinterpretations, omissions due to environmental factors
  22. 23. Adapted Critical Incident Method (Flanagan, 1954; Critical Decision Method by Klein, 1989) To solicit knowledge of Roles and Responsibilities, Strategies, Attitudes and Personal Characteristics <ul><ul><li>Document Analysis and Recall of a critical past event e.g. fire, accident, high-stake mission. </li></ul></ul><ul><ul><li>Ask each team member to describe various decisions taken, action triggers, who did what vs. who was supposed to do what </li></ul></ul><ul><ul><li>Describe how each teammate reacted to the situation (perception, prediction, response time, stress, flexibility in co-ordination, etc.) </li></ul></ul>
  23. 24. 2. Control of Attention <ul><li>The way a team engages in information management </li></ul><ul><ul><ul><ul><ul><li>- Information seeking </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>- Filtering irrelevant information </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>- Allocating attention to important functions </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>- Properly distributing messages </li></ul></ul></ul></ul></ul><ul><li>Refers to the team’s working memory </li></ul><ul><ul><li>The way it uses limited resources for processing simultaneous messages (Klein) </li></ul></ul>
  24. 25. Methods to Elicit and Represent Control of Attention
  25. 26. Scenario based Questionnaires (Blickensderfer, 1997) To solicit knowledge of Information Management Strategies, and Communication Skills <ul><ul><li>Document Analysis, observations and interviews to create a questionnaire covering a team work scenario, given to each team member </li></ul></ul><ul><ul><li>Answers to outline relevant information protocol for the scenario, missing information, cues for identifying missing data, strategies to acquire and distribute information, and possible communication bottlenecks </li></ul></ul>
  26. 27. 3. Shared Situation Awareness <ul><li>Extent to which team members have the same interpretation of ongoing events (Cannon-Bowers, Sales, & Converse, 1992) </li></ul><ul><li>Is giving full data to each member a remedy for discrepant situation Awareness? </li></ul><ul><ul><li>How different team members perceive dynamic situations and events </li></ul></ul><ul><ul><li>How they form divergent impressions e.g. police in a riot situation </li></ul></ul><ul><ul><li>How they update each other </li></ul></ul>
  27. 28. Methods to Elicit and Represent Shared Situation Awareness
  28. 29. Adapted Situation Awareness Global Assessment Technique (SAGAT) (Endsley, 1995) To solicit knowledge of compatible situation awareness <ul><ul><li>Bootstrapping to create a mission simulation </li></ul></ul><ul><ul><li>Asking team members to perform mission </li></ul></ul><ul><ul><li>Freezing mission after specific intervals </li></ul></ul><ul><ul><li>Asking each team member to analyze the situation and predict outcome and strategize </li></ul></ul><ul><ul><li>Compare results for all members and assess compatibility and reasons for discrepancy </li></ul></ul>
  29. 30. Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (Muniz et al. , 1998) <ul><li>To solicit knowledge of compatible situation awareness </li></ul><ul><ul><li>1. Delineate behaviours theoretically linked to team situations (24 behaviours, 5 clusters). </li></ul></ul><ul><ul><ul><li>Being able to predict next event </li></ul></ul></ul><ul><ul><ul><li>Managing resources (technical, system , internal, team) </li></ul></ul></ul><ul><ul><ul><li>Managing time </li></ul></ul></ul><ul><ul><ul><li>Feeling of being in control; able to implement elegant solutions </li></ul></ul></ul><ul><ul><ul><li>Taking the right decision at the best moment </li></ul></ul></ul><ul><ul><ul><li>Detecting mismatches, etc. </li></ul></ul></ul><ul><ul><li>2. Development of task scenarios </li></ul></ul><ul><ul><li>3. Identification of specific, observable responses for each scenario </li></ul></ul><ul><ul><li>4. Development of a script (to be administered to a team member) </li></ul></ul><ul><ul><li>5. Observation form: Presence or Absence of response behavior </li></ul></ul>
  30. 31. 4. Application of Strategies <ul><li>Every team has a prepared list of steps, and routines for decision making, problem solving and planning </li></ul><ul><li>However, a team also develops shortcuts and workarounds not codified in procedures, as it gains experience </li></ul><ul><li>A skillful team knows its: </li></ul><ul><ul><li>Key decision makers (to be supported) </li></ul></ul><ul><ul><li>Key sources of expertise (to be consulted) </li></ul></ul><ul><li>Time is critical in strategy execution </li></ul>
  31. 32. Methods to Elicit and Represent Team Strategies
  32. 33. Retrospective Protocol Analysis (Means, 1993) To solicit knowledge of agreement on strategies <ul><ul><li>Based on non-interruption </li></ul></ul><ul><ul><li>Use video-tapes of similar mission or actual team performance recorded previously </li></ul></ul><ul><ul><li>Asking each team member to analyze the situation and justify strategies taken </li></ul></ul><ul><ul><li>Analyze team agreement </li></ul></ul>
  33. 34. 5. Metacognition <ul><li>A team needs to monitor itself during ongoing task performance </li></ul><ul><li>To determine whether and when it is running into difficulty </li></ul><ul><li>Predict consequences of the difficulty </li></ul><ul><li>Determine when and how it needs to shift its strategies e.g. Police at riot situation </li></ul>
  34. 35. Methods to Elicit and Represent Metacognition
  35. 36. Mission Simulation (Klein) To solicit knowledge of self-monitoring or metacognition <ul><ul><li>Bootstrapping to create a mission simulation </li></ul></ul><ul><ul><li>Asking team members to perform mission </li></ul></ul><ul><ul><li>Observing how team members recognize and collect more information about critical incidents </li></ul></ul><ul><ul><li>Conducting individual interviews with each team member to analyze the video-recording </li></ul></ul><ul><ul><li>Analyzing gaps in team agreement </li></ul></ul>
  36. 37. Representing Team CTA Results
  37. 38. Methods for Representing Team CTA Results <ul><li>Dealing with the ‘Envisioned World Problem’ (Woods, in press) </li></ul><ul><ul><li>Creating prototypes and simulations to gauge how team CTA would impact the future </li></ul></ul><ul><ul><li>Iterative testing </li></ul></ul>World Team CTA Prototypes of Changed World
  38. 39. Limitations / Challenges in Applying Team CTA Methods
  39. 40. Challenges in Applying Team CTA Methods <ul><li>Knowledge elicitation </li></ul><ul><ul><li>Knowledge is internalized and even experts find it hard to verbalize it </li></ul></ul><ul><ul><li>Retrospective account of work may differ from reality </li></ul></ul><ul><li>Capturing dynamism </li></ul><ul><ul><li>In many cases, team strategies differ vastly from one situation to another </li></ul></ul><ul><ul><li>Generalizing based on a few situations may be inappropriate </li></ul></ul>
  40. 41. Summary <ul><li>Team CTA is critical to obtain a holistic (both overt and covert tasks) view of human work </li></ul><ul><li>Teams are intelligent entities with 5 key cognitive processes (Klein) </li></ul><ul><ul><li>Shared mental models </li></ul></ul><ul><ul><li>Control of attention </li></ul></ul><ul><ul><li>Shared situation awareness </li></ul></ul><ul><ul><li>Application of strategies to make decisions, solve problems and plan </li></ul></ul><ul><ul><li>Metacognition </li></ul></ul>
  41. 42. Summary <ul><li>The 5 key processes can be spread over 2 types of team knowledge </li></ul><ul><ul><li>Pre-task knowledge </li></ul></ul><ul><ul><li>Dynamic Knowledge </li></ul></ul><ul><li>Team CTA Methods should aim at eliciting team knowledge of these 5 processes </li></ul>
  42. 43. Summary <ul><li>Team CTA can yield significant performance improvement of teams by: </li></ul><ul><ul><li>Restructuring and Resizing the team </li></ul></ul><ul><ul><li>Designing better: </li></ul></ul><ul><ul><ul><li>Information technology </li></ul></ul></ul><ul><ul><ul><li>Information management strategies </li></ul></ul></ul><ul><ul><ul><li>Human-computer interfaces </li></ul></ul></ul><ul><ul><ul><li>Decision support systems and Communication </li></ul></ul></ul><ul><ul><li>Developing methods for team training and selection </li></ul></ul>
  43. 44. References <ul><li>Gary Klein </li></ul><ul><ul><li>Cognitive Task Analysis of Teams (1995) </li></ul></ul><ul><li>Cannon-Bowers, Sales, Baker </li></ul><ul><ul><li>Analyzing knowledge requirements in team tasks </li></ul></ul><ul><ul><li>Shared Mental Models in Expert Team Decision Making (1993) </li></ul></ul><ul><li>Hoffman, R.R. </li></ul><ul><ul><li>Use of Critical Incidents to Elicit Knowledge (1995) </li></ul></ul><ul><li>Oransu, J. </li></ul><ul><ul><li>Shared Mental Models and Crew Performance (1990) </li></ul></ul><ul><li>Coury, B.G., Motte,S., & Seiford, L.M. </li></ul><ul><ul><li>Capturing and representing decision processes in the design of an information system </li></ul></ul><ul><li>Muniz, E.J., Stout, R.J., Bowers & Salas </li></ul><ul><ul><li>A methodology for measuring team situation awareness: Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (1998) </li></ul></ul><ul><li>Woods, D.D. & Hollnagel, E. </li></ul><ul><ul><li>Mapping cognitive demands in complex problem solving worlds (1987) </li></ul></ul>
  44. 45. Questions ?
  45. 46. Cognitive Task Analysis of Teams Thank You !