Cognitive Task Analysis of Teams (Team CTA)

8,227 views

Published on

Published in: Technology, Business
1 Comment
9 Likes
Statistics
Notes
  • Outstanding demonstration. Really clear along with useful
    Teisha
    http://dashinghealth.com http://healthimplants.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
8,227
On SlideShare
0
From Embeds
0
Number of Embeds
1,848
Actions
Shares
0
Downloads
507
Comments
1
Likes
9
Embeds 0
No embeds

No notes for slide

Cognitive Task Analysis of Teams (Team CTA)

  1. 2. Cognitive Task Analysis of Teams <ul><li>Yogesh Tadwalkar </li></ul><ul><li>March 22, 2004 </li></ul>
  2. 3. Traditional Task Analysis <ul><li>Task Analysis aims to optimize work performance by matching tasks with human capabilities and limitations </li></ul><ul><li>Human work is comprised of both physical and cognitive activities </li></ul><ul><li>Physical tasks are observable (Overt) </li></ul><ul><li>However, cognitive processes that lead to physical actions are not (Covert) </li></ul>
  3. 4. Cognitive Task Analysis (CTA) <ul><li>Extension of traditional task analysis techniques to uncover information about: </li></ul><ul><ul><li>- Work domain, mental demands on operator </li></ul></ul><ul><ul><li>- Goal Structures </li></ul></ul><ul><ul><li>- Knowledge and Strategies </li></ul></ul><ul><ul><li>- Thought Processes and Decision Making </li></ul></ul><ul><li>Thus, providing a holistic, complete view of human work </li></ul>
  4. 5. Goals of CTA <ul><li>Performance improvement through: </li></ul><ul><ul><li>Better Task and Interface Design </li></ul></ul><ul><ul><li>Ideal Personnel Selection </li></ul></ul><ul><ul><li>Comprehensive Training and Instructions </li></ul></ul><ul><ul><li>Systematic Performance Assessment </li></ul></ul><ul><ul><li>Improved Planning </li></ul></ul>
  5. 6. Cognitive Task Analysis of Teams
  6. 7. Why Conduct Team CTA? <ul><li>Because 2 + 2 ≠ 4 </li></ul><ul><ul><li>Individuals are often required to perform in teams </li></ul></ul><ul><ul><li>Optimum individual performances may not lead to final mission accomplishment if team co-ordination is unsatisfactory </li></ul></ul><ul><li>Thus, individual CTA would not be sufficient for designing optimum task performance </li></ul>
  7. 8. Team CTA <ul><li>As defined by Klein: </li></ul><ul><ul><li>A reflection and representation of the team mind / Knowledge </li></ul></ul><ul><ul><li>Description of the cognitive skills required for effective team performance </li></ul></ul><ul><ul><li>Elaboration of the team decision requirements </li></ul></ul><ul><ul><li>Determination of types of expertise found in effective teams </li></ul></ul>
  8. 9. Objectives of Team CTA <ul><li>Team performance enhancement through: </li></ul><ul><ul><li>Restructuring and Resizing the team </li></ul></ul><ul><ul><li>Designing better: </li></ul></ul><ul><ul><ul><li>Information technology </li></ul></ul></ul><ul><ul><ul><li>Information management strategies </li></ul></ul></ul><ul><ul><ul><li>Human-computer interfaces </li></ul></ul></ul><ul><ul><ul><li>Decision support systems and Communication </li></ul></ul></ul><ul><ul><li>Developing methods for team training and selection </li></ul></ul>
  9. 10. Team Environment
  10. 11. Team Environment <ul><li>Tasks are decomposed and assigned to different team members </li></ul><ul><li>Team members may or may not have direct contact with each other (co-located vs. dispersed teams) </li></ul><ul><li>Tasks can be performed serially or in parallel or overlapping fashion </li></ul><ul><li>Team members possess, acquire and process information, make decisions, solve problems and make plans </li></ul>
  11. 12. Team Environment <ul><li>Teams are intelligent entities </li></ul><ul><li>(Thordsen and Klein, 1989) </li></ul><ul><ul><li>Team members possess, acquire and process </li></ul></ul><ul><ul><li>information </li></ul></ul><ul><ul><li>Communicate with each other </li></ul></ul><ul><ul><li>Make decisions </li></ul></ul><ul><ul><li>Solve problems and </li></ul></ul><ul><ul><li>Make plans </li></ul></ul>
  12. 13. Types of Teams <ul><li>Planning Teams </li></ul><ul><ul><li>Military Command </li></ul></ul><ul><ul><li>Emergency Response at nuclear plants </li></ul></ul><ul><li>Action Teams </li></ul><ul><ul><li>Air Traffic Control </li></ul></ul><ul><ul><li>SWAT </li></ul></ul>
  13. 14. Cognitive Processes of Teams What to measure through Team CTA methods
  14. 15. Cognitive Processes of Teams <ul><li>Methods for team CTA should focus at capturing these 5 processes (Klein): </li></ul><ul><ul><li>1. Shared mental models </li></ul></ul><ul><ul><ul><li>- Team Goals, team member roles </li></ul></ul></ul><ul><ul><li>2. Control of attention </li></ul></ul><ul><ul><ul><li>- Information seeking, filtering, communication </li></ul></ul></ul><ul><ul><li>3. Shared situation awareness </li></ul></ul><ul><ul><ul><li>- Assessment of dynamism, prediction of future </li></ul></ul></ul><ul><ul><li>4. Application of strategies </li></ul></ul><ul><ul><ul><li>- To make decisions, solve problems and plan </li></ul></ul></ul><ul><ul><li>5. Metacognition </li></ul></ul><ul><ul><li>- Self-monitoring, sensing trouble, making alternative plans </li></ul></ul>
  15. 16. A Model for Team Cognition Knowledge Mental Models Attitudes Expectations Skills Member A Member C Member B Team Performance <ul><li>Pre-task Knowledge </li></ul><ul><li>Shared Mental Models </li></ul><ul><li>Strategies </li></ul><ul><li>Dynamic Knowledge </li></ul><ul><li>Control of Attention </li></ul><ul><li>Shared Situation Awareness </li></ul><ul><li>Metacognition </li></ul>(Stout, Cannon-Bowers, and Salas, 1996)
  16. 17. 1. Shared Mental Models <ul><li>The degree to which team members have the same understanding of key processes (Cannon-Bowers, Sales, & Converse, 1992) </li></ul><ul><ul><li>- Knowledge about overall team goals and mission objectives e.g. Football game </li></ul></ul><ul><ul><li>- Roles and responsibility of each team member </li></ul></ul><ul><ul><li>- Team-mate characteristics and preferences </li></ul></ul><ul><ul><li>- Task Procedures, sequences and timing </li></ul></ul><ul><ul><li>- Use and relationship of equipment </li></ul></ul>
  17. 18. Shared Mental Models and Team Performance <ul><li>Shared mental models affect team performance significantly (Oransu, 1990) : </li></ul><ul><ul><li>Anticipating team member behaviours and information requirements </li></ul></ul><ul><ul><li>Performing tasks from a common frame of reference </li></ul></ul><ul><ul><li>Attending to, interpreting, communicating about, and responding to the world more similarly than individuals with discrepant knowledge (Rentsch & Hall, 1994) </li></ul></ul>
  18. 19. Methods to Elicit and Represent Shared Mental Models
  19. 20. Variation of Goal-Directed Analysis (Woods & Hollnagel, 1987) To solicit knowledge of Team Goals and Task Objectives <ul><ul><li>Document Analysis and Observation </li></ul></ul><ul><ul><li>Separate interviews with each team member, followed by group interview </li></ul></ul><ul><ul><li>Representation: Agreement Metric </li></ul></ul>Task Goal Sub-Goal 1 Sub-Goal 2 Sub-Goal 3 Task 1 Agreement Metric Team Goals For Each Member For Whole Team Task 2
  20. 21. Variation of Task-Action Mapping (Coury et al., 1991) To solicit knowledge of Task Procedures, Sequence and Timing <ul><ul><li>Document Analysis and Observation </li></ul></ul><ul><ul><li>Ask each team member to describe key team concepts, associated tasks and specific actions </li></ul></ul><ul><ul><li>Representation: Concept Maps </li></ul></ul>Description Task 1 Task 2 Task 3 <ul><li>Action 1 </li></ul><ul><li>What (Steps) </li></ul><ul><li>Who </li></ul><ul><li>When </li></ul>Concept 1 e.g. Evacuation <ul><li>Action 1 </li></ul><ul><li>What (Steps) </li></ul><ul><li>Who </li></ul><ul><li>When </li></ul>
  21. 22. Bootstrapping (Hoffman, Shadbolt, Burton & Klein 1995) To solicit knowledge of Roles and Responsibilities <ul><ul><li> Bootstrapping </li></ul></ul>Documentation Individual Interviews Observation Group Interviews May not adequately capture dynamic, contextual data like misinterpretations, omissions due to environmental factors
  22. 23. Adapted Critical Incident Method (Flanagan, 1954; Critical Decision Method by Klein, 1989) To solicit knowledge of Roles and Responsibilities, Strategies, Attitudes and Personal Characteristics <ul><ul><li>Document Analysis and Recall of a critical past event e.g. fire, accident, high-stake mission. </li></ul></ul><ul><ul><li>Ask each team member to describe various decisions taken, action triggers, who did what vs. who was supposed to do what </li></ul></ul><ul><ul><li>Describe how each teammate reacted to the situation (perception, prediction, response time, stress, flexibility in co-ordination, etc.) </li></ul></ul>
  23. 24. 2. Control of Attention <ul><li>The way a team engages in information management </li></ul><ul><ul><ul><ul><ul><li>- Information seeking </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>- Filtering irrelevant information </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>- Allocating attention to important functions </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>- Properly distributing messages </li></ul></ul></ul></ul></ul><ul><li>Refers to the team’s working memory </li></ul><ul><ul><li>The way it uses limited resources for processing simultaneous messages (Klein) </li></ul></ul>
  24. 25. Methods to Elicit and Represent Control of Attention
  25. 26. Scenario based Questionnaires (Blickensderfer, 1997) To solicit knowledge of Information Management Strategies, and Communication Skills <ul><ul><li>Document Analysis, observations and interviews to create a questionnaire covering a team work scenario, given to each team member </li></ul></ul><ul><ul><li>Answers to outline relevant information protocol for the scenario, missing information, cues for identifying missing data, strategies to acquire and distribute information, and possible communication bottlenecks </li></ul></ul>
  26. 27. 3. Shared Situation Awareness <ul><li>Extent to which team members have the same interpretation of ongoing events (Cannon-Bowers, Sales, & Converse, 1992) </li></ul><ul><li>Is giving full data to each member a remedy for discrepant situation Awareness? </li></ul><ul><ul><li>How different team members perceive dynamic situations and events </li></ul></ul><ul><ul><li>How they form divergent impressions e.g. police in a riot situation </li></ul></ul><ul><ul><li>How they update each other </li></ul></ul>
  27. 28. Methods to Elicit and Represent Shared Situation Awareness
  28. 29. Adapted Situation Awareness Global Assessment Technique (SAGAT) (Endsley, 1995) To solicit knowledge of compatible situation awareness <ul><ul><li>Bootstrapping to create a mission simulation </li></ul></ul><ul><ul><li>Asking team members to perform mission </li></ul></ul><ul><ul><li>Freezing mission after specific intervals </li></ul></ul><ul><ul><li>Asking each team member to analyze the situation and predict outcome and strategize </li></ul></ul><ul><ul><li>Compare results for all members and assess compatibility and reasons for discrepancy </li></ul></ul>
  29. 30. Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (Muniz et al. , 1998) <ul><li>To solicit knowledge of compatible situation awareness </li></ul><ul><ul><li>1. Delineate behaviours theoretically linked to team situations (24 behaviours, 5 clusters). </li></ul></ul><ul><ul><ul><li>Being able to predict next event </li></ul></ul></ul><ul><ul><ul><li>Managing resources (technical, system , internal, team) </li></ul></ul></ul><ul><ul><ul><li>Managing time </li></ul></ul></ul><ul><ul><ul><li>Feeling of being in control; able to implement elegant solutions </li></ul></ul></ul><ul><ul><ul><li>Taking the right decision at the best moment </li></ul></ul></ul><ul><ul><ul><li>Detecting mismatches, etc. </li></ul></ul></ul><ul><ul><li>2. Development of task scenarios </li></ul></ul><ul><ul><li>3. Identification of specific, observable responses for each scenario </li></ul></ul><ul><ul><li>4. Development of a script (to be administered to a team member) </li></ul></ul><ul><ul><li>5. Observation form: Presence or Absence of response behavior </li></ul></ul>
  30. 31. 4. Application of Strategies <ul><li>Every team has a prepared list of steps, and routines for decision making, problem solving and planning </li></ul><ul><li>However, a team also develops shortcuts and workarounds not codified in procedures, as it gains experience </li></ul><ul><li>A skillful team knows its: </li></ul><ul><ul><li>Key decision makers (to be supported) </li></ul></ul><ul><ul><li>Key sources of expertise (to be consulted) </li></ul></ul><ul><li>Time is critical in strategy execution </li></ul>
  31. 32. Methods to Elicit and Represent Team Strategies
  32. 33. Retrospective Protocol Analysis (Means, 1993) To solicit knowledge of agreement on strategies <ul><ul><li>Based on non-interruption </li></ul></ul><ul><ul><li>Use video-tapes of similar mission or actual team performance recorded previously </li></ul></ul><ul><ul><li>Asking each team member to analyze the situation and justify strategies taken </li></ul></ul><ul><ul><li>Analyze team agreement </li></ul></ul>
  33. 34. 5. Metacognition <ul><li>A team needs to monitor itself during ongoing task performance </li></ul><ul><li>To determine whether and when it is running into difficulty </li></ul><ul><li>Predict consequences of the difficulty </li></ul><ul><li>Determine when and how it needs to shift its strategies e.g. Police at riot situation </li></ul>
  34. 35. Methods to Elicit and Represent Metacognition
  35. 36. Mission Simulation (Klein) To solicit knowledge of self-monitoring or metacognition <ul><ul><li>Bootstrapping to create a mission simulation </li></ul></ul><ul><ul><li>Asking team members to perform mission </li></ul></ul><ul><ul><li>Observing how team members recognize and collect more information about critical incidents </li></ul></ul><ul><ul><li>Conducting individual interviews with each team member to analyze the video-recording </li></ul></ul><ul><ul><li>Analyzing gaps in team agreement </li></ul></ul>
  36. 37. Representing Team CTA Results
  37. 38. Methods for Representing Team CTA Results <ul><li>Dealing with the ‘Envisioned World Problem’ (Woods, in press) </li></ul><ul><ul><li>Creating prototypes and simulations to gauge how team CTA would impact the future </li></ul></ul><ul><ul><li>Iterative testing </li></ul></ul>World Team CTA Prototypes of Changed World
  38. 39. Limitations / Challenges in Applying Team CTA Methods
  39. 40. Challenges in Applying Team CTA Methods <ul><li>Knowledge elicitation </li></ul><ul><ul><li>Knowledge is internalized and even experts find it hard to verbalize it </li></ul></ul><ul><ul><li>Retrospective account of work may differ from reality </li></ul></ul><ul><li>Capturing dynamism </li></ul><ul><ul><li>In many cases, team strategies differ vastly from one situation to another </li></ul></ul><ul><ul><li>Generalizing based on a few situations may be inappropriate </li></ul></ul>
  40. 41. Summary <ul><li>Team CTA is critical to obtain a holistic (both overt and covert tasks) view of human work </li></ul><ul><li>Teams are intelligent entities with 5 key cognitive processes (Klein) </li></ul><ul><ul><li>Shared mental models </li></ul></ul><ul><ul><li>Control of attention </li></ul></ul><ul><ul><li>Shared situation awareness </li></ul></ul><ul><ul><li>Application of strategies to make decisions, solve problems and plan </li></ul></ul><ul><ul><li>Metacognition </li></ul></ul>
  41. 42. Summary <ul><li>The 5 key processes can be spread over 2 types of team knowledge </li></ul><ul><ul><li>Pre-task knowledge </li></ul></ul><ul><ul><li>Dynamic Knowledge </li></ul></ul><ul><li>Team CTA Methods should aim at eliciting team knowledge of these 5 processes </li></ul>
  42. 43. Summary <ul><li>Team CTA can yield significant performance improvement of teams by: </li></ul><ul><ul><li>Restructuring and Resizing the team </li></ul></ul><ul><ul><li>Designing better: </li></ul></ul><ul><ul><ul><li>Information technology </li></ul></ul></ul><ul><ul><ul><li>Information management strategies </li></ul></ul></ul><ul><ul><ul><li>Human-computer interfaces </li></ul></ul></ul><ul><ul><ul><li>Decision support systems and Communication </li></ul></ul></ul><ul><ul><li>Developing methods for team training and selection </li></ul></ul>
  43. 44. References <ul><li>Gary Klein </li></ul><ul><ul><li>Cognitive Task Analysis of Teams (1995) </li></ul></ul><ul><li>Cannon-Bowers, Sales, Baker </li></ul><ul><ul><li>Analyzing knowledge requirements in team tasks </li></ul></ul><ul><ul><li>Shared Mental Models in Expert Team Decision Making (1993) </li></ul></ul><ul><li>Hoffman, R.R. </li></ul><ul><ul><li>Use of Critical Incidents to Elicit Knowledge (1995) </li></ul></ul><ul><li>Oransu, J. </li></ul><ul><ul><li>Shared Mental Models and Crew Performance (1990) </li></ul></ul><ul><li>Coury, B.G., Motte,S., & Seiford, L.M. </li></ul><ul><ul><li>Capturing and representing decision processes in the design of an information system </li></ul></ul><ul><li>Muniz, E.J., Stout, R.J., Bowers & Salas </li></ul><ul><ul><li>A methodology for measuring team situation awareness: Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (1998) </li></ul></ul><ul><li>Woods, D.D. & Hollnagel, E. </li></ul><ul><ul><li>Mapping cognitive demands in complex problem solving worlds (1987) </li></ul></ul>
  44. 45. Questions ?
  45. 46. Cognitive Task Analysis of Teams Thank You !

×