Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Method For Measuring Tsa


Published on

Ketut Sulistyawati

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Method For Measuring Tsa

  1. 1. Methods for Measuring Team Situation Awareness Ketut Sulistyawati M6603 class presentation 22 March 2006
  2. 2. Situation Awareness (SA) <ul><li>SA is “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (Endsley, 1995) </li></ul>
  3. 3. Team <ul><li>two or more people </li></ul><ul><li>interact dynamically, interdependently </li></ul><ul><li>a common and valued goal/objective/ mission </li></ul><ul><li>specific roles or functions </li></ul><ul><li>limited life span of membership </li></ul><ul><li>(Salas et al., 1992) </li></ul>
  4. 4. Team Situation Awareness <ul><li>mental model of situation (i.e., SA) that is partly shared and partly distributed between team members, which are integrated and coordinated to anticipate important status in the near future (Artman & Garbis, 1998) </li></ul><ul><li>Consideration for measurement: </li></ul><ul><ul><li>Individual or team level ? </li></ul></ul><ul><ul><li>If individual: method of aggregation? </li></ul></ul><ul><ul><ul><li> Highest/ lowest/ average/ similarity? </li></ul></ul></ul>
  5. 5. Measuring TSA: Considerations Individual A’s SA Individual B’s SA Individual C’s SA Team process: e.g., communication, coordination, feedback Team Situation Awareness
  6. 6. Measuring TSA: Considerations Individual A’s SA Individual B’s SA Individual C’s SA Team Situation Awareness Aggregation Method ???
  7. 7. This presentation <ul><li>Coordinated Awareness of Situation by Teams (CAST) (Gorman et al., 2005) </li></ul><ul><li> team process </li></ul><ul><li>Questionnaires for Distributed Assessment of Team Mutual Awareness (MacMillan et al., 2004) </li></ul><ul><li> individual level </li></ul>
  8. 8. CAST - overview <ul><li>Introduce roadblocks at some points during team performance </li></ul><ul><li>Listen to team communication during the glitch period </li></ul><ul><li>Check appropriate boxes on a CAST score sheet </li></ul>
  9. 9. CAST score sheet
  10. 10. CAST - method <ul><li>For each component (perceived firsthand, coordinated perception, coordinated action) </li></ul><ul><li>Higher PF along with lower CP & CA  highest team performance </li></ul><ul><li>CAST scores = checked boxes </li></ul><ul><li> total boxes </li></ul>
  11. 11. CAST - review <ul><li>Advantages: </li></ul><ul><ul><li>Provide understanding on the dynamic process of TSA development </li></ul></ul><ul><ul><li>Can be used regardless of team size and distribution of expertise of a team </li></ul></ul><ul><li>Limitations: </li></ul><ul><ul><li>Validity & reliability (??) </li></ul></ul><ul><ul><li>Relatively time-consuming </li></ul></ul><ul><ul><li>May not account for implicit coordination </li></ul></ul>
  12. 12. Questionnaires for Distributed Assessment of Team Mutual Awareness <ul><li>Using three concepts: </li></ul><ul><li>Measuring team mutual awareness  congruence in team members’ perception </li></ul>Mutual Awareness Workload Awareness Taskwork Awareness Teamwork Awareness
  13. 13. QDATMA – Task Awareness <ul><li>e.g., Think back to when elements of the team had just completed taking the North Beach. </li></ul><ul><ul><li>What task were you performing at that time? </li></ul></ul><ul><ul><li>What tasks did you believe each of the other positions in your team were performing? </li></ul></ul><ul><li>Asks team members to assess the most important task that each member is performing at selected point in time </li></ul>
  14. 14. QDATMA – Workload Awareness <ul><li>Part 2: estimate overall workload experienced by each of other team members </li></ul><ul><li>(WL A – WL A’ ) + (WL B – WL B’ ) </li></ul><ul><li> 2 </li></ul><ul><li>Part 1: rate own awareness using traditional NASA TLX items (i.e., mental, temporal, performance, effort, frustration </li></ul>√ 2 2
  15. 15. QDATMA – Teamwork Awareness <ul><li>e.g., To what extent was team’s behavior coordinated? </li></ul><ul><li>| | | | | | | </li></ul><ul><li>2 3 4 5 6 7 </li></ul><ul><li>poor good </li></ul><ul><li>Asks team members to rate four dimensions of teamwork behaviors: </li></ul><ul><ul><li>Coordination & information management </li></ul></ul><ul><ul><li>Leadership/team orientation </li></ul></ul><ul><ul><li>Communication </li></ul></ul><ul><ul><li>Back-up </li></ul></ul>
  16. 16. QDATMA - review <ul><li>Advantages: </li></ul><ul><ul><li>Quick and easy to complete </li></ul></ul><ul><ul><li>Inexpensive </li></ul></ul><ul><ul><li>Require little training </li></ul></ul><ul><li>Limitations: </li></ul><ul><ul><li>Validity & reliability (??) </li></ul></ul><ul><ul><li>Subjective </li></ul></ul><ul><ul><li>Congruence ≠ accuracy </li></ul></ul><ul><ul><li>In some circumstances, SA does not necessarily be shared by all members </li></ul></ul>
  17. 17. Conclusion <ul><li>There are still many unresolved issues regarding TSA measurement methods </li></ul><ul><li>Validation studies required </li></ul><ul><li>Each method has its advantages and limitations </li></ul><ul><li>Need to use more than one methods and compare the results </li></ul>
  18. 18. References <ul><li>Artman, H., & Garbis, C. (1998). Situation Awareness as Distributed Cognition. In T.Green, L.Bannon, C.Warren & J.Buckley (Eds.), Cognition and cooperation. Proceedings of 9th Conference of Cognitive Ergonomics. (pp. 151-156). Limerick: Ireland. </li></ul><ul><li>Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37 , 32-64. </li></ul><ul><li>Gorman, J. C., Cooke, N. J., Pederson, H. K., Connor, O. O., & DeJoode, J. A. (2005). Coordinated awareness of situation by teams (CAST): Measuring team situation awareness of a communication glitch. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting . </li></ul>
  19. 19. References <ul><li>MacMillan, J., Paley, M. J., Entin, E. B., & Entin, E. E. (2004). Questionnaires for Distributed Assessment of Team Mutual Awareness. In E. Salas (Ed.) Distributed Assessment in The Handbook of Human Factors and Ergonomic Methods , Taylor and Francis. </li></ul><ul><li>Salas, E., Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I. (1992). Toward an understanding of team performance and training. In R. W. Swezey & E. Salas (Eds.), Teams: Their Training and Performance (pp. 3-29). Norwood, NJ: Abex. </li></ul>
  20. 20. Thank you!! Any question??