2. An Overview
• What is the purpose of your investigation / study /
research?
– Rationale, key issues (focus), reference to the
work of others (keeping track!!).
• Will data be prestructured or evolve? Is the
approach to be tightly or loosely structured?
• Do you need a hypothesis?
Not essential and will depend on type of study,
approach etc. What is more important is ‘whether
you have thought about what is, and what is not,
worth investigating and how the investigation will
be conducted’
3. An Overview continued . . .
• Who is the intended audience?
• What outcomes are intended – short term/long term?
• Which methodological approach will you take?
– Will this draw on/be supported by the work of others?
• Which research instruments?
– Will this draw on/be supported by the work of others?
• How will you support such things as: rigour, the need to be
systematic, the need to operate ethically, reliability,
validity, etc?
• How will you select a sample, gain access to the
associated location, obtain consent, ensure
confidentiality/anonymity, involve your participants
actively etc?
4. An Overview continued
McNiff and Whitehead (2005 p.1) provide the following list of issues for
consideration:
– What is my concern?
– Why am I concerned?
– What kind of evidence can I produce to show why I am concerned?
– What can I do about it?
– What will I do about it?
– What kind of evidence will I produce to show how the
situation is unfolding?
– How do I make sure that any conclusions I come to are reasonably
fair and accurate?
– How do I modify my practice in the light of my evaluations?
5. Some key Issues:
Once you are sure that observation as a research method (instrument) is
fit for purpose (a sound research question(s) – methods fit), consider:
• Strengths:
– Observation gives direct access to social interactions.
– “include the opportunity to record information as it occurs in a
setting” (Cresswell, p.221)
– Observation can give permanent and systematic records of social
interaction (by way of field notes, numerical data that may be nominal,
continuous or ordinal, video, audio etc.)
– Observation can enrich and supplement data gathered by other
techniques/approaches (in a mixed approach it may, for example
supplement results drawn from the analysis of questionnaire
responses).
– Observation techniques are extremely varied (producing qualitative
and or quantitative data, can be used in different contexts/locations,
to answer different types of research question).
• Weaknesses:
– It demands a great deal of time.
– It is susceptible to bias.
– Limits you to situations where you can gain access
6. Validity & Reliability:
‘Validity is about whether or not the researcher has
recorded, or measured, what was intended. Three
techniques in particular will enhance the validity’:
– Use more than one source of evidence and or more
than one data collection technique. (methodological
triangulation).
– Get those who provide you with data to comment on
your interpretations.
(respondent validation).
– Be aware of your own impact … monitor this
carefully and make this explicit in your submission
(reflexivity).
7. Reliability is the idea that another evaluator, or
the same evaluator on another occasion, should
come up with the same results. There are two
dimensions of reliability:
– Inter-judge reliability – which is about
whether two evaluators studying the same
event come up with the same conclusions;
– Intra-judge reliability – which is about the
consistency of the same evaluator’s findings
about the same things on different occasions.
8. Validity and reliability can be supported by:
– a very detailed account of your approach
(rationale, procedures, techniques, management
of data analysis and the drawing of conclusions
etc – sometimes referred to as an audit trail)
that is open to scrutiny;
– This includes a very rigorous/systematic
approach to data collection and analysis.
Based on Craft, A. (2000 pp. 109-110) Continuing Professional Development A Practical Guide for
Teachers and Schools (2nd Edition), London, RoutledgeFalmer
9. How?
• Who/What do you need to observe? Individuals, groups,
on the basis of gender, ability, age etc?
• Setting the Scene (Access, Ethics - including consent,
Building a Rapport etc.)
• When, where and how will you observe?
– Participant Observation (e.g. linked to Action
Research).
– Partial Participant Observation (nearby, but not
taking an active role)
– Non Participation Observation. (e.g. Experimental
work).
– Third Party Observation (e.g. a colleague observing
you).
10. • How will data be collected? (How often, over what length of time etc?)
– Tape (provides detail that can be analysed during repeated listening – a
useful back up to in-situ note taking etc. Quality of audio, doesn’t record
everything (non-verbal language, impact on participants, time consuming
etc. Transcript offers greatest level of depth)
– Video (excellent visual and audio record, can be viewed repeatedly and
will provide good support to in situ note taking, but what about camera
position, affect on participants etc? Transcript offers greatest level of
depth.)
– Note Taking – Descriptive / Narrative (Anecdotal) (Immediacy,
economic use of time, but what might get missed, is recoding superficial,
how might this approach impact on participants?)
Descriptive: Some pre-set categories but more flexibly applied, allowing the
researcher to ‘consider the context of the behaviours, their sequences, their
meanings etc.
Narrative: No structure or restrictions, based on general guiding questions –
evolving focus allowing for the collection of very rich data. Categories emerge
from data analysis.
11. Sample:
Here you need to consider the extent to which your
‘sample’ – a relatively small number of individuals drawn
from the population – is representative of that same
population. This links to the notion of ‘generalisation’.
A general rule of thumb is that a sample smaller than 30 is
unlikely to allow you to make useful comparisons,
particularly of a statistical nature.
Please note that the selection noted below is by no
means definitive.
12. • Random - This approach gives each eligible
element/unit an equal chance of being selected in
the sample; random procedures are employed to
select a sample e.g. names out of a hat, every
tenth name on a list etc.
• Stratified - This takes account of different sub-
groups – men/women, 0-5s/6-11s/12-18s,
different ethnic or religious groupings etc.
• Cluster - This involves successive sampling of
units (or clusters); the units sampled progress
from larger ones to smaller ones (e.g. health
authority/health board, trust, senior managers).
13. • Convenience – Sometimes called ‘opportunistic
sampling’ a strategy that uses the most easily
accessible participants for a study, selected
quite arbitrarily. The selection is unstructured.
• Purposive - Purposive/purposeful sampling: a
strategy in which the researcher selects
participants (or has them selected) who are
considered to effectively represent a particular
population. For example, a selection of able pupils,
or a mixed ability group. This is sometimes also
called judgmental sampling.
However your sample is selected, you must
provide a justification for your approach as part
of a methodology section
14. Research question:
What might non-verbal language tell us
about power relations between adult
interviewers and young children?
Data from a video recording
17. Event Sampling: Duration
AWJ: Arguing without justifying a personal position
RVN: responding very negatively
UBL: using bad language
Participant Categories
AWJ RVN UBL Row Totals
Student A
0 (secs) 10 0
10
Student B
14 8 4
26
Student C
21 0 21
42
Student D
59 12 37
108
Column Totals 94 30 62 186
18. Time/Interval Sampling:
• Simpson and Tuson (2003) indicate that the time
interval adopted by professional researchers lies
between 5 seconds and 5 minutes, with pre-data
collection trials used to determine the most
appropriate time interval.
• Select a set of indicators/categories associated with
a particular area of focus (in the example shown on
the next slide - Non-talk pupil activity) and, during
regular time intervals (every two minutes in this
example), occurrences of these indicators are noted.
19.
20. Time/Interval Sampling (cont’d):
• Recording is based on a simple yes (tick) to indicate that
a category has been observed at some time during the
interval. No more than one tick is recorded, irrespective
of how many times the behaviour may be seen.
• This can be useful in providing an indication of how forms
of behaviour change over time. For example one of the
categories might occur frequently at the start of a
lesson, not be seen whilst children are working on their
task, but feature again during the plenary session.
• However, this form of recording would not provide the
same data as a tally sheet linked to the frequency of
particular occurrences/events.
21. References
• Bell, J. (1999) Doing Your Research Project – A Guide for first time
researchers in education and social-sciences (3rd Edition),
Buckingham, Open university Press.
• Craft, A. (2000 pp. 109-110) Continuing Professional Development A
Practical Guide for Teachers and Schools (2nd Edition), London,
RoutledgeFalmer
• Cresswell, J. (2008) Educational Research. (3rd Edition) New Jersey:
Pearson International
• McNiff, J. and Whitehead, J. (2005) Action Research for Teachers:
A Practical Guide, London, David Fulton Publishers
• Simpson, M and Tuson, J. (2003 pp. 16-20) Using Observation in
Small-Scale Research : A Beginners Guide (Revised Edition), Glasgow,
SCRE
• The Open University (2001 p.207) Masters Programme in Education:
Research Methods in Education Handbook, Milton Keynes, The Open
University
