1© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
See the PDE booklet,
...
2© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Checking in…
– Who ha...
3© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Participant observati...
4© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Quantitative vs. qual...
5© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Observation…
Involves...
6© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Use in program evalua...
7© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Observations
Advantag...
8© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Observation – Ethical...
9© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
What are the implicat...
10© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Participant observat...
11© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
What to observe
• Pe...
12© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Types of observation...
13© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Practice:
Structured...
14© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Example – Observing ...
15© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Recording your obser...
16© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Sample Observation G...
17© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Structured observati...
18© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Notebooks to record ...
19© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Who does the observa...
20© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Training –
preparati...
21© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
How well do you obse...
22© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
23© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Becoming a skilled o...
24© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Data analysis and in...
25© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Practice
your observ...
26© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
Steps in planning fo...
Upcoming SlideShare
Loading in …5
×

Introduction to Participant Observation as a Data Collection Method in Program Evaluation

17,628 views

Published on

Published in: Technology, Education
2 Comments
4 Likes
Statistics
Notes
No Downloads
Views
Total views
17,628
On SlideShare
0
From Embeds
0
Number of Embeds
84
Actions
Shares
0
Downloads
405
Comments
2
Likes
4
Embeds 0
No embeds

No notes for slide
  • Take 15 seconds and look at the picture below. Move to the next slide or turn away from the computer screen and write down everything you observed. Then, come back to the photo and see what you missed (or thought was there and isn’t!).
  • Introduction to Participant Observation as a Data Collection Method in Program Evaluation

    1. 1. 1© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation See the PDE booklet, Collecting evaluation data: Direct observations http://learningstore.uwex.edu/pdf/G3658-05.pdf Using observation to collect evaluation data
    2. 2. 2© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Checking in… – Who has used observation as a data collection method? – What are your experiences with it? – Who is hoping to use it – How?
    3. 3. 3© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Participant observation: What it is
    4. 4. 4© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Quantitative vs. qualitative methods Quantitative Qualitative Survey Observation Tests Interview Questionnaire Focus group
    5. 5. 5© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Observation… Involves all 5 senses: sight, hearing, smell, touch, and taste
    6. 6. 6© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Use in program evaluation • When you want direct information • When you are trying to understand an ongoing behavior, process, unfolding situation, or event • When there is physical evidence, products, or outcomes that can be readily seen • When written or other data collection methods seem inappropriate
    7. 7. 7© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Observations Advantages – Provides direct information; not dependent upon someone’s response – Unobtrusive – See things in natural context – Flexible; discovery oriented Disadvantages – Observer’s presence may create artificial situation – Potential for bias – Time consuming – Requires diligence, preparation – Challenging to collect data while participating
    8. 8. 8© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Observation – Ethical issues • Unobtrusiveness is its greatest strength; also potential for abuse in invasion of privacy • You can venture into places and gather data almost anywhere so questions re. what is ethical – Overt vs. covert • Remember our Human Subjects Protection guidelines – Consent form for participating in an observational study Consider cultural appropriateness of using observation
    9. 9. 9© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation What are the implications for using observation as an evaluation data collection method?
    10. 10. 10© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Participant observation: How to do it?
    11. 11. 11© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation What to observe • People (individuals, groups, communities) – Characteristics – Interactions – Behaviors – Reactions • Physical settings • Environmental features • Products/physical artifacts Observing what does not happen may be as important as observing what does happen
    12. 12. 12© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Types of observation Structured Unstructured Looking for vs. Looking at Sometimes we have something specific we want to observe – leadership skills; level of participation; etc. We use a structured, preset guide of what to observe or a checklist. Sometimes we want to see what is naturally occurring or exists without predetermined ideas. We use have an open-ended approach to observation and record all that we observe
    13. 13. 13© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Practice: Structured/unstructured observations Imagine you are sitting in a room where ten youth are sitting at computers learning about Web 2.0 applications. 1) If you want to assess to what extent students are interested and learning, what specifically would you look (listen) for? 2) If you aren’t sure what specifically indicates student interest or learning and you want to see what is going on during the demonstration, how would you proceed?
    14. 14. 14© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Example – Observing participation in an after school program • Who you will observe: youth attending the program • What you will observe: – Age, gender – Length of time student stays in the program – Involvement in activities: which activities • Level of involvement – Interactions with other youth; with staff • When you will observe: all hours the program is open for one week each month during 2007
    15. 15. 15© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Recording your observations It is not good enough to just observe, you need to systematically record your observations. You might use: – Observation guide – Recording sheet – Checklist – Field note – Picture – Combination of the above
    16. 16. 16© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Sample Observation Guides Guide for structured observations Guide for unstructured observations
    17. 17. 17© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Structured observation guide used for pre and post program evaluation
    18. 18. 18© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Notebooks to record and collect observations as they occur
    19. 19. 19© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Who does the observations? • You – program staff • Participants - Youth • Parents • Teachers • Volunteers • Other stakeholders • Colleagues
    20. 20. 20© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Training – preparation/orientation may be necessary – To learn what to look for – To learn how to record observations – To practice – To ensure that observations across sites are consistent: observers use the same methods, rate an observation in same way
    21. 21. 21© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation How well do you observe?
    22. 22. 22© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
    23. 23. 23© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Becoming a skilled observer includes… • Learning to pay attention, see what there is to see, and hear what there is to hear; • Practice in writing descriptively; • Acquiring discipline in recording notes; • Knowing how to separate detail from trivia; • Using rigorous methods to validate and triangulate observations; • Reporting the strengths and limitations of one’s own perspective » M.Q. Patton, 2002. Qualitative Research and Evaluation Methods. Sage, pg 260
    24. 24. 24© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Data analysis and interpretation Qualitative data = qualitative data analysis – Standard content analysis • Get to know your data • Focus the analysis • Categorize information • Identify patterns and connections • Interpret – bring it all together PDE booklet: Analyzing Qualitative Data http://learningstore.uwex.edu/Assets/pdfs/G3658-12.pdf
    25. 25. 25© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Practice your observation skills everyday in everyway! And, add observation to your data collection toolbox.
    26. 26. 26© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Steps in planning for observation • Determine who/what will be observed. • Determine aspects that will be observed (characteristics, attributes, behaviors, etc.). • Determine where and when observations will be made. • Develop the observation guide • Pilot test the observation guide • Train the observers and have them practice. • Conduct the observations • Analyze and interpret the collected information. • Write up and use your findings.

    ×