1. The document describes the methodology used in a comparative study of principals in Australia and America. Both quantitative and qualitative methods were used, including questionnaires and interviews. 2. Principals were sampled randomly from different school types and sectors in both countries. 81 American and 39 Australian principals participated. 3. Interviews followed a structured format with open-ended questions about research knowledge and policy decisions. Questionnaires collected supplementary multiple choice data. 4. Responses were coded and analyzed statistically to identify response distributions, mean differences between countries, and correlations between predictive and response variables. Both separate country analyses and comparative analyses were reported.

1. Chapter 3
A Design for Investigation
“As I survey the work of the National Research Council, it is
poignantly clear that research has not had the kind of impact
on education that is visible in medical practice, space
exploration, energy, and many other fields” (Biddle & Saha,
2005, p. 37)
Thomas F. Pettigrew, 1985

2. History & Design
• Focus on Principals
• Both quantitative and qualitative methods
used
• Comparative project: Two Countries

3. History and Design
Results
Method
Instruments
Principals –Impact
of Research
Knowledge
Quantitative
Questionnaire
Responses
Qualitative
Interview
Responses

4. History and Design
Qualitative Research Method

5. History and Design
Quantitative Research Method

6. History and Design
Comparative Design

7. History and Design

8. History and Design
Better Decisions for Schools

9. History and Design
Constraints related to
beginning the study

10. Sampling Techniques
Given the constraints, sampling was pulled from:
1. Major types of schools- primary/secondary
2. Three major sectors of edcuation- public,
parochial, and independent
3. Community types
4. Random gender selection
5. At least 1 year of served principalship

11. Sampling Techniques
American Sample:
Public Parochial Independent Total
Primary 17 20 0 37
Secondary 19 17 8 44
Totals 36 37 8 81

12. Sampling Techniques
Australian Sample:
Public Parochial Independent Total
Primary 11 4 1 16
Secondary 17 3 3 23
Totals 28 7 4 39

13. Instruments and Procedures
Process:
1. Letter sent to potential respondents followed
with a phone call
2. Structured and scheduled interviews took
place
3. Questionnaires

14. Instruments and Procedures
The Interview: (6 Sections/Open-Ended Questions)
1. Research Knowledge (Useful)
2. Innovative Resistance
3. Policy Decisions
4. Knowledge-Acquisition
5. Familiarity with research-knowledge topics
6. Opinions about research-knowledge

15. Instruments and Procedures
The Questionnaire: (Closed-ended Questions)
• Supplementary
• Multiple-Choice
– Environments
– Job histories
– Background
– Hobbies
– Career goals
– Characteristics of their schools

16. Data Analysis
Turning the data into quantitative evidence by coding

17. Data Analysis
Coding for Interview Responses:
• Each of the 6 sections had its own manual
• Issues related to coding with interview responses
• Derivative Variables
Coding for Questionnaires:
• Manuals
• Issues related to coding with the questionnaire
• Environmental Variables

18. Data Analysis
Data Treatment:
• Concentrate on 3 quantitative effects-
1. Effects associated with response distributions for
variables, for which the researchers constructed
visual displays
2. Effects concerned with differences between
Australian and American responses for which
calculations of means and mean differences were to
be made
3. Effects focused on predictive conditions and
responses given by principals, for which calculations
of product-moment correlations would be made

19. Data Analysis
Data reports:
• Graphs had to be constructed only with
meaningful data
• Separate analyses reported for both countries
• Graphs had to be represented in proportions

20. The Analysis Model and Our Matching
Strategy
Other issues associated with decision to use
product-moment correlations to represent relations
between predictive and responsive variables:
1. Unexamined assumptions
2. Evidence offered about strengths of relations
between predictive and response variables would
represent only “first order” effects
3. Arbitrary decisions about which variables should be
thought of as representing predictive conditions and
response conditions

21. The Analysis Model and Our Matching
Strategy
Final words: Quantitative vs. Qualitative