This document summarizes the evaluation of the Team Climate Inventory (TCI) questionnaire using structural equation modeling. It discusses prior analyses that validated the six-factor structure of the TCI based on aggregated data, which ignores the nested data structure. The current study aims to validate the TCI for use as an independent variable by verifying its six-dimension construct using exploratory factor analysis and structural equation modeling on individual-level data from two time periods. Results found some dimensions may have more than one factor and that items models fit better than scores models. Future work includes validating the models on additional data and modifying item models based on exploratory factor analysis results.

1. Evaluating the TCI
questionnaire using SEM
Evan Kontopantelis
Stephen Campbell
David Reeves

2. Overview
• A warning…
• Part one
– Organizational (Team)
Climate
– Team climate in primary
care
– Team climate concerns
– Team Climate Inventory
– Data structure of the
TCI
– Validation of the TCI
– So, what’s the problem?
• Part two
– Data collection
– What we hope to achieve
& how
– Structured Equation
Modelling
• A few notes
• Results: Items models
• Results: Scores models
– Exploratory Factor
Analysis
– Conclusions
– Future work

3. A warning…
• "Oh, people can come
up with statistics to
prove anything, Ken.
14% of people know
that“

4. Part one
Formulation of the TCI
questionnaire and prior analyses

5. Organizational (Team) Climate
• Cognitive schema approach: “…individuals’
cognitive representation of proximal
environments…expressed in terms of
psychological meaning and significance to the
individual…” (James & Sells 1981)
• Shared perceptions approach: “…the shared
perception of the way things are around here”
(Reichers & Schneider 1990)
• These approaches are compatible, in principle

6. Team climate in primary care
• Healthcare professionals work in integrated
teams in order to improve task effectiveness,
morale and team viability
• Structural changes may only be translated to
positive outcomes if team-level processes are
effective
• Team climate is a concept that may be of
relevance to team processes, hence we
“measure” and relate it to performance
(Bower, Campbell, Bojke, Sibbald, 2003)

7. Team climate concerns
• Can TCI be considered to quantify team
climate (and hence become a useful
measure/predictor of performance)?
– A large variability of responses within a
practice would indicate that the “shared
perception” theory cannot really be applied
– Can there ever really be only one team
climate in a big multi-professional general
practice?

8. Team Climate Inventory
• 65 item measure with six subscales that
attempts to quantify the team climate within a
practice, based on the shared perceptions
approach (Anderson & West, 1994)
• All items on a scale of 1 to 5
• The six subscales (factors) are:
Participation Task orientation
Support for innovation Reflexivity
Clarity of objectives Teamworking

9. Data structure of the TCI
• Although it’s a 3-level structure
( items, respondents &
practices), only two level
constructs can be examined (R:P)
• We either use all the items for
each respondent (items models)
or the average scores of the
subscales (scores models)
• Averaging the responses within a
practice, for each item, data is
condensed to single level

10. Validation of the TCI
• Initially a four-factor theory (Anderson & West, 1994)
• Exploratory and Confirmatory Factor Analysis
indicated that a five-factor construct was
more suitable but…
• Although “…examining item statistics at the
individual level avoids additional problems of
dealing with summed data at the team level…”
analyses use group level sums for each item
(Anderson & West 1998)
• The five factor construct was again verified
with Norwegian data using group level sums
(Mathisen et al 2004)

11. So, what’s the problem?
• Exploratory factor analyses have been
used on unaggregated data, ignoring the
nested structure
• Confirmatory analyses using SEM have
only been used with group averages,
ignoring various statistical issues
• No published analyses on the six
subscale measure which is based on the
earlier versions

12. Part two
NPCRDC data and preliminary
analyses

13. Data collection
• The questionnaire was distributed to all
clinical, nursing and administration staff
working in a sample of 60 practices in 1998
and 42 of the same practices in 2003
• Response rates varied greatly by practice
1998 2003
number of practices 60 42
average respondents per pract 9.5 12.2
average resp rate per pract 63.1% 65.1%

14. What we hope to achieve & how
• What?
– Validate the usage of TCI as an independent
variable in QuIP analyses
• How?
– verify the validity of the 6 dimension construct on
which the questionnaire was based taking into
account the nested structure of the data.
– compare the full-item structure with the score
one, in terms of complexity and information
provided (+)
– use Exploratory Factor Analysis to see if there is
only one factor per dimension

15. A few notes on Structured
Equation Modelling
• Hybrid technique that encompasses aspects of
confirmatory factor analysis, and regression
• It encourages confirmatory rather than exploratory
modelling, hence it is better suited for theory testing
rather than theory development
Observed variables
Latent variables
Causality (reg equations)
Error terms
Correlations
Estimated parameters

16. Results - Scores models
• We estimated the
performance of a
single-factor and a
two-factor model
• The 6 aggregated
variables seem to
comprise a single
latent variable,
which we call “team
climate”

17. Results - Items models
• We estimated the performance
of models with 1, 2, 6 and 7
factors
• Some path weights (2 for reflex
& 5 for work) are close to zero,
hence they don’t contribute to
the calculated factors.
• This is an indication that each
of these two dimensions should
be probably described by more
than one factor.
• Fit was exceptional if error-
term correlations were included
(an indication that something is
amiss)

18. Results - EFA (a)
• For each TCI dimension, Exploratory Factor
Analysis was executed
• Both datasets agreed on the number of
factors we needed to construct, for each
dimension.
• The factor loadings verified that the
datasets agreed not only on which questions
“made up” which factors, but also on the
amount that each question contributed to the
total variance.

19. Results - EFA (b)
• Reflex’s factors seem to be
discussion & consideration of
methods and actual changes
taking place.
• Work’s factors are likely to
be team evaluation, personal
evaluation in relation to the
team and interdependence.
Dimension
# of
factors
Part 1
Supinv 1
Obj 1
Task 1
Reflex 2
Work 3

20. Conclusions
• The Items models appear to perform better
than the Scores ones, although there isn’t a
solid measure available that can compare
models that are that different.
• Exploratory factor analysis indicated that
there may be more than one factor in certain
dimensions. This finding was verified by the
Items SE Models. Using a single aggregate
variable for each of those dimensions is
unsuitable.

21. Future work
• Data collected for 2005 with which the
models will be validated
• Different model structure if information on
respondent IDs becomes available
• Alternative Items’ models will be created,
taking into account the results of EFA (3 and
2 latent variables for work and reflex
respectively).
• Overall EFA performed (not for each
subgroup separately)