L. O'Keefe Writing Sample - Qualitative Research Narrative

John G. Shedd Aquarium
Learning Planning and Evaluation
Prepared by: Lindsay O’Keefe, LPE Intern, August 2014 ©Shedd Aquarium
1
RESEARCH NARRATIVE
Qualitative Methods for Evaluating Informal Science Learning
Introduction
Evaluators use many different methods in order to evaluate programs, broadly the two main
methods are: quantitative methods and qualitative methods. Quantitative methods are defined as,
“consist[ing] of research in which the data can be analyzed in terms of numbers” (Best & Kahn,
2006, p. 79). Qualitative methods can, “describe events and persons scientifically without the use
of numerical data” (Best & Kahn, 2006, p. 79). Neither one of these methods is better than the
other. They both provide important information and insights in different forms and for different
purposes.
Within the field of evaluation many people have seen the value of using both quantitative and
qualitative methods, for these methods help to “improve the validity of their measurements[s]”
(Fitzpatrick, Sanders & Worthen, 2004, p. 305). By employing both quantitate and qualitative
methods, an evaluator can better understand what is really happening within a program. By using
both quantitative and qualitative methods they are able to fill in each other’s gaps and make a
more rounded study. This use of mixed method analysis is also known as triangulation
(Fitzpatrick, et al 2004). By using triangulation, hopefully everything of interest to the evaluator
can be covered and understood. By looking more deeply into the value and use of qualitative
methods, more in-depth ways of evaluating a learning program can be achieved. The purpose of
this narrative is to inform the Learning Group staff about various qualitative methods that can be
used to evaluate the learning programs here at Shedd. The most popular and most common
qualitative methods have been included here for reference.
What are Qualitative Methods?
There are many subcategories within qualitative methods, however, they all have certain
ideologies in common that interconnect them. The basic three forms of qualitative data collection
are, “(1) in-depth, open-ended interviews; (2) direct observation; and (3) written documents”
(Best & Kahn, 2006, p. 247). The many subtypes of qualitative methods help to create specific
ways of understanding the general data that can be gathered through non-numerical methods.
Some of these specific methods include: interviews, observations, document/content analysis,
ethnography, case studies, and focus groups. Qualitative methods are incredibly helpful to getting
insights into what a child involved in a learning program is actually thinking they are learning, as
it may be different than the intended learning outcome. These thoughts become easily visible
through the use of qualitative methods.
Why Qualitative Methods?
The fields of anthropology and sociology are fields that have long used and long perfected the use
of qualitative methods. Sociology also employs the use of some quantitative methods like the use
of surveys; however, more than anything the use of qualitative methods are incredibly important
insights into a culture or society which is why anthropologists and sociologists use them. Within
the field of evaluation qualitative methods can be incredibly helpful for smaller programs that do
not have a large enough sample size and as a way to better understand exactly how a student
understands the material being presented. Qualitative methods create better insight into exactly
how a student interacts with their material. Some advantages to using qualitative methods have to

2
RESEARCH NARRATIVE
do with the fact that they tend to be more flexible than quantitative methods. The evaluator is able
to ask open-ended questions and can better probe the participant (Family Health International).
For these reasons understanding and utilizing qualitative methods can be incredibly helpful for
the field of evaluation and here at Shedd.
INTERVIEWS
Description
Interviews have always been a very important aspect of qualitative methodology. “Qualitative
interviews are used for learning the perspectives, attitudes, behaviors, and experiences of others”
(Fitzpatrick, et al, 2004, p. 347-8). An evaluator has the ability to ask follow up questions to help
participants explain their answer or to get at an idea that may not have been thought to ask in a
survey. Interviews allow an evaluator to understand the perspective of the participant. Many more
specific forms of qualitative methods include the use of interviews, for example, ethnography and
case studies. However, interviews can be a good tool to use by itself without the need to include
other methods.
There are two forms of interviews, structured and in-depth. When doing a structured interview
the interviewer has a strict set of questions that they ask every participant and are instructed not
to probe the participants (Frechtling, 1997). This is good when wanting standardization or when
there are multiple interviewers and there is a want or need to keep interviews as similar as
possible. In-depth interviews are more of a dialogue between the interviewer and participant.
Here is where the interviewer wants to probe and ask for more answers to the questions they are
asking (Frechtling, 1997). This is good when the evaluator wants a more rounded look into a
participant’s thoughts and experiences. Both of these methods have their pros and cons and are
good for different purposes. The goal of any interview according to Diamond (1999) “is to elicit a
subject’s responses in ways that avoid the imposition of bias on the part of the interviewer” (1999,
p. 83). This is incredibly important to remember.
When/Where is it Useful
Structured interviews are useful as a way to explore a set of thoughts or ideas using “a carefully
worded questionnaire” (Fretchling, 2010, p. 60). This type of interviewing is more interested in
“obtaining answers to carefully phrased questions (Fretchling, 2010, p. 60). Whereas in-depth
interviews explores a visitors thought process and encourages the meaningfulness of their
experiences (Fretchling, 2010). The ability to capture an experience through the words of a visitor
is one of the great reasons to use in-depth interviews. Interviews, generally, shed light onto
experiences that may not be captured otherwise. An evaluator will also receive detailed
information and understand more complex matters through the use of interviews (Fretchling,
2010).
Whenever there is a need to discuss a complex matter, detailed information or highly sensitive
matters in-depth interviews are very useful (Fretchling, 2010). Patton suggests the use of
interviews when answering questions like,

3
RESEARCH NARRATIVE
 What does the program look and feel like to the participants? To other stakeholders?
 What do stakeholders know about the project?
 What thoughts do stakeholders knowledgeable about the program have concerning
program operations, processes, and outcomes?
 What are participants’ and stakeholders’ expectations?
 What features of the project are most salient to the participants?
 What changes do participants perceive in themselves as a result of their involvement in the
project? (Fretchling, 2010 p. 60 (Patton, 1990)).
Pros and Cons
Pros Cons
Usually yield richest data, details, new insights Expensive and time consuming
Permit face-to-face contact with respondents Need well-qualified, highly trained
interviewers
Provide opportunity to explore topics in depth Interviewee may distort information through
recall error, selective perceptions, desire to
please interviewer
Allow interviewer to experience the affective
as well as cognitive aspects of responses
Flexibility can result in inconsistencies across
interviews
Allow interviewer to explain or help clarify
questions increasing the likelihood of useful
responses
Volume of information very large, may be
difficult to transcribe and reduce data
Allow interviewer to be flexible in
administering interview to particular
individuals or in participant circumstance
(Frechtling, 2010, p. 61).
Analysis
Interview data can be gathered through recordings or through written notes, aka “field notes.”
When recording an interview someone needs to then transcribe that data after it has been
collected. Transcription is the process of going through the interview and writing down
everything that was asked and said. This process requires a lot of time and is important to put in
tone of voice when necessary and pauses to questions (Frechtling, 2010). Direct quotes from
recorded interviews can often be the most effective way to convey a visitor’s experience
(Diamond, 1999). Some training may be necessary to transcribe interviews.
When using written notes to document an interview, it can be helpful to have multiple
interviewers, one to take notes and the other to ask questions. This way no one is overwhelmed
with too many tasks. Note taking is not always as accurate as recordings; however, it takes less
time in the long run as there is no need to transcribe the material. This method of note taking is
best when there is less time available to collect data (Frechtling, 2010). Regardless of which
method is chosen it is incredibly important to use some form of documentation and not simply
rely on memory.

4
RESEARCH NARRATIVE
When going over field notes it may also be necessary to code these notes to make them easier to
see what categories have been found throughout the interviewing process. It also pulls out all the
information that has been gathered. Coding material is one of the unique strengths of qualitative
methods. Coding specifically refers to a way of understanding and classifying something so that it
is easier to identify commonalities and differences as well as trends. The are two coding methods,
emergent coding, which categories items after some preliminary analysis of the data and a priori
coding which is establishing categorizes before analyzing the data (Stemler, 2001). For emergent
coding,
“two people independently review the material and come up with a set of features that
form a checklist. Second, the [evaluators] compare notes and reconcile any differences that
show up on their initial checklists. Third the [evaluators] use a consolidated checklist to
independently apply coding. Forth, the [evaluators] check the reliability of the coding”
(Stemler, 2001).
To do a priori coding evaluators agree on categories and go on to code the data and revisions are
made when necessary. Categories are tightened and refined as needed (Stemler, 2001). A priori
coding is more strict and planned than emergent coding is. There are programs that help to both
transcribe and code field notes, please see An Overview of Content Analysis1 for more information
on specific coding software options.
Examples in Museums
Many museums have taken advantage of the interview, often in conjunction with a survey, or
observations to see what the visitors got out of an exhibit and what their thoughts were on said
exhibit. Most examples included later in the narrative have some interview component. One
specific example is the Summative Evaluation: Mammoth Discovery!2 In-depth interviews were
used as a way to understand the visitors experience within the exhibit (Korn, 2012). This is just
one of many examples of the use of interviews within evaluation.
OBSERVATIONS
Description
“Observational techniques are methods by which an individual or individuals gather firsthand
data on programs, processes, or behaviors being studied” (Frechtling, 1997). Observations can be
used both alone and in conjunction with other qualitative methods. In fact, many qualitative
methods have some sort of observational component. Observations usually use strict guidelines to
record the behaviors of participants (Frechtling, 1997). Observations are valuable because they
allow the evaluator to watch their participants and see first-hand what participants are connecting
to without the participants translating this information.
Using observations during the formative or summative phase is a great way to see whether or not
the project is being understood appropriately (Frechtling, 1997). These observations give great
1 http://pareonline.net/getvn.asp?V=7&n=17
2 http://informalscience.org/evaluation/ic-000-000-003-553/Summative_Evaluation_Mammoth_Discovery_

5
RESEARCH NARRATIVE
insights into how these projects are being understood by the participants. Shedd has a large
number of learning programs and by observing various classrooms there will be a better
understanding of the similarities and differences existent within them.
Pros and Cons
Pros Cons
Provide direct information about behavior of
individuals and groups
Expensive and time consuming
Permit evaluator to enter into and understand
situation/context
Need well-qualified, highly trained observers;
may need to be content experts
Provide good opportunities for identifying
unanticipated outcomes
May affect behavior of participants
Exist in natural, unstructured, and flexible
setting
Selective perception of observer may distort
data
Investigator has little control over situation
Behavior or set of behaviors observed may be
atypical
(Frechtling, 1997).
Analysis
There are certain steps that need to be observed in order to ensure that an observation is done
well. An evaluator needs to describe the setting in which they are observing. They need to identify
the people being observed. They need to document the interactions. Then describe and assess
what is going on. These “field notes” are incredibly important to understanding what exactly took
place during the observation period (Frechtling, 1997). It can be helpful to have multiple people
observing at the same time in order to increase the quality of the data and make sure there is
consistency within the observations (Frechtling, 1997). Learning this skill may take some time and
may require some further study.
Another ways of tracking what is being observed is by creating an ethogram: “An ethogram is a list
of the major categories of behavior of which a species is capable” (Diamond, 1999, p. 66). If an
evaluator is looking for specific behaviors, for example, students interacting with the animals in
the learning labs, this is a great way to make sure those behaviors are being tracked. Creating an
ethogram allows the observer to easily document what behaviors are being observed and what
they want to observe (Diamond, 1999).
Examples in Museums
Observations are often used in conjunction with other qualitative methods. The Frye Art Museum
used observations in their Summative Evaluation of the Gallery Guide Program3. These
observations allowed evaluators to observe how visitors experienced the tours at the museum.
These observations were used along with post-tour surveys.
3 http://informalscience.org/evaluation/ic-000-000-007-
452/Summative_Evaluation_Of_The_Gallery_Guide_Program_At_The_Frye_Art_Museum

6
RESEARCH NARRATIVE
DOCUMENT/CONTENT ANALYSIS
Description
Document/content analysis provides good insights into a setting or group of people that cannot be
observed in any other way (Frechtling, 2010). By using document/content analysis an evaluator is
able to view thoughts and ideas that would not be revealed through an interview or observation.
For evaluation there are many items that can be used for document/content analysis, for example,
mission statements, student transcripts, minutes of meetings, official correspondence,
presentations, and descriptions of programs among others (Frechtling, 2010). There are also first
person documents that can be used, such as diaries, portfolios, photographs, artwork, scrapbooks
etc. (Frechtling, 2010).
Document/content analysis is useful in understanding interpersonal thoughts of a person that
might not be seen otherwise. It also provides crucial insights into the inner thoughts and ideas of a
person (Frechtling, 2010). They are also incredibly helpful when trying to make comparisons
between participants in a particular program (Frechtling, 2010). Document/content analysis is
also useful as the resources needed for the analysis already exist, since the documents have
already been created so there is no need to gather more data (Frechtling, 1997).
Pros and Cons
Pros Cons
Available locally May be incomplete
Inexpensive May be inaccurate or of questionable
authenticity
Grounded in setting and language in which
they occur
Locating suitable documents may pose
challenges
Useful for determining value, interest,
positions, political climate, public attitudes
Analysis may be time consuming and access
may be difficult
Provide information on historical trends or
sequences
Provide opportunity for study of trends over
time
Unobtrusive
(Fretchling, 2010, p. 71).
Analysis
Many people understand document analysis as doing a word-frequency count (Stemler, 2001).
This is not the case, however, it is a part of the process but definitely not all of it. Although it may
seem straight forward to do a word-frequency count an evaluator needs to recognize synonyms
being used for stylistic reasons or short hand (Stemler, 2001 (Weber, 1990)). Diaries, for example,
are not necessarily written for other people to read so the writer may not write in a way intended
for analysis, making some information possibly confusing to an evaluator. Coding is also
incrediably important in document/context analysis, please see interview analysis section for a
more in-depth description of coding.

7
RESEARCH NARRATIVE
Examples in Museums
One online learning website, WolfQuest,4 used document/content analysis to help evaluate what
their participants were learning and discussing in these online forums. “The goal of the forum
analysis was to investigate the considerable amount of social interaction around the site, looking
specifically at whether there was evidence of scientific habits of mind being reinforced by the
game” (Goldman, Koepfler & Yocco, 2009, p. 9). They specifically looked at topics being discussed.
This was a useful way to see whether the players were actually learning science like the producers
indented or if they just believed it was a fun and interesting game.
ETHNOGRAPHY
Description
The qualitative method of ethnography is a method that is not easily described as there are many
different ways it is utilized. One definition is that ethnography “aims to be holistic by studying
‘naturally occurring human behavior’ through observation” (Roberts, 2009, p. 291). Ethnography
also encompasses the terms field work and participant observation as well. Often in literature
these three terms are used interchangeably, with ethnography being a more encompassing term
tying them all together . It is also described as a “qualitative research approach developed by
anthropologists with the purpose of describing an aspect of culture, but is also aimed at learning
about the culture factor being studied” (Roberts, 2009, p. 291). With these definitions in mind it is
easy to see that ethnography does not fit into a strict box but is open to some interpretation.
Ethnography utilizes many different qualitative methods within this larger study, including
observations, interviews, and document/content analysis. Within ethnography observing is
defined as, “‘individuals, not in isolation, but in relation to their organizations, communities,
customs and culture’” (Roberts, 2009, p. 292 (Clark, 2000, p. 44)). The type of observations done
within an ethnographic context is similar to just regular observations; however there are certain
distinctions that are made. The two types of observations within ethnography are participant and
non-participant (Roberts, 2009). “In participant observation the researcher tries to become part of
the culture by being in the ‘field,’ rather than from a detached stance as in non-participant
observation” (Roberts, 2009, p. 292 (Denscombe, 2003)). “The aim of the non-participant
observer, therefore, is to record observations without any interactions within the setting that is
being studied” (Roberts, 2009, p 292). Neither one of these methods are better than another they
just provide different types of information.
When/Where it is Useful
Ethnographies are incredibly useful when wanting to understand how people experience the
world they are a part of (Roberts, 2009). For example, the environment that is created within the
teen learning labs here at Shedd could be examined through the use of ethnography.
Ethnographies are used as a way to “produce knowledge,” and this can be about a native tribe as
much as a group of teens (Roberts, 2009). It is also a way of being able to make sense of peoples
actions through these observations and experiences (Roberts, 2009). It would also be useful when
looking for a qualitative method that encompasses many of the other methods.
4 http://informalscience.org/evaluation/ic-000-000-003-244/WolfQuest_Summative_Report

8
RESEARCH NARRATIVE
Pros and Cons
Pros for using ethnography include the ability to look at the whole experience of a child in a
learning lab (Roberts, 2009). It is able to provide very rich information about the group that is
being studied (Roberts, 2009).
Cons include the amount of time it takes to do a well-developed ethnography. It is necessary to
have staff with some training in this area. There are some limitations to being able to apply one
students experience with another’s (Roberts, 2009). It can also be difficult not to become too
embedded within what an ethnographer is studying. Ethnographers tend to connect strongly with
their subjects (Roberts, 2009).
Participant Observation Non-Participant Observation
Pros Researcher is a part of the situation and is
more aware of morale, apathy, and
goodwill. May have more credibility.
Can follow a plan of observation.
Cons Might have difficulty recording
observations. Might be seen as a threat.
Might be conspicuous, which can affect
what is being observed.
(Roberts, 2009).
Analysis
When doing either participant or non-participant observations, the evaluator will be writing down
field notes. To do this please see the observations analysis section for a more in-depth explanation
on good observational notes. When it comes to any interviews conducted, they are also analyzed
in the same way as a normal interview. Please see the interview analysis section for an in-depth
description.
Examples in Museums
Ethnography was used by the National Science Foundation to evaluate The Cosmic Serpent Story5.
For this ethnography they specifically used participant observation. “At least one of the three
evaluators– and usually all three evaluators– participated in these modes of interaction with the
project team” (Stein, 2011, p. 5). They also made sure to take advantage of field work and used this
as another way to observe their subjects (Stein, 2011, p. 5).
CASE STUDIES
Description
The case study is another helpful methodology because it allows the evaluator to do an in-depth
analysis of a single unit (Gerring, 2004). John Gerring (2004) wrote that “even among its
defenders there is confusion over the virtues and vices of this ambiguous research design” (2004,
p. 341). Because case studies are not understood with a universal definition, they can be difficult
to define. However, Lynn Davey says that “rather than using large samples and following a rigid
protocol to examine a limited numbers of variables, case study methods involve an in-depth,
452/The_Cosmic_Serpent_Story_Summative_Evaluation_for_Final_Report

9
RESEARCH NARRATIVE
longitudinal examination of a single instance or event” (1991). This is the “case” aspect of the
“case study.” This is helpful at Shedd to follow the flow and change of a specific learning lab. A full
case study will use many different data gathering techniques like document/content analysis,
observations, interviews, and surveys among others (Best and Kahn, 2006). A case study tends to
present new information and not just back up existing knowledge (Gerring, 2004).
There are a couple of different types of case studies that could be used at Shedd. There is the
Exploratory Case Study which is used when uncertainty exists about the case being studied (Davey,
1991). It is helpful when trying to explore what the goals, operations, and results of a program are
not known yet (Davey, 1991).
Program Implementation Case Studies help to weed out if a program's intent matches the
implementation (Davey, 1991). “Extensive, longitudinal reports of what has happened over time
can set a context for interpreting a finding of implementation variability” (Davey, 1991). This type
of case study would be very helpful when it comes to new learning programs at Shedd.
Finally there is the Cumulative Case Study which uses and collects data from various different
times (Davey, 1991). These case studies build upon past case studies as opposed to starting over
with a new one each time (Davey, 1991). It can also use case studies done in the past as well as
conducting new case studies. “Retrospective cumulation allows generalization without cost and
time of conducting numerous new case studies; prospective cumulation also allows generalization
without unmanageably large numbers of cases in process at any one time” (Davey, 1991). This is a
good one to have as a long term goal to track the growth and change of a learning program.
Each one of these case studies is valuable to Shedd, they simply have different uses and purposes.
For a full explanation of each of these methods, and a few others, check out The Application of Case
Study Evaluations.6
Case Studies are useful when trying to understand the whole of a unit. “Case studies are
particularly useful when the purpose of the evaluation is to describe something – a case – in depth.
Very often, evaluations are concerned with exploring the “hows” and “whys” of a program or
policy” (Fitzpatrick, 2004, p. 307). For example, answering the question: How does this class work
and run? By deciding to do a case study on the Icy Icy Adaptations class you are able to understand
specifically how every instructor develops and teaches that specific class. The students and
instructors may change, however, the lesson and the learning outcomes should not, so a case study
could be a great way to understand how these specific classes run even with a changing variable.
Each specific type of case study has a slightly different use as mentioned above.
6 http://pareonline.net/getvn.asp?v=2&n=9

10
RESEARCH NARRATIVE
Pros and Cons
Pros Cons
Provide a rich picture of what is happening, as
seen through the eyes of many individuals
Require a sophisticated and well-trained data
collection and reporting team
Allow a through exploration of interactions
between treatment and contextual factors
Can be costly in terms of the demands on time
and resources
Can help explain changes or facilitating factors
that might otherwise not emerge from the data
Individual cases may be over interpreted or
overgeneralized
(Fretchling, 2010, p. 73).
Analysis
Case studies are a combination of multiple qualitative methods and possibly some quantitative
methods, these methods include interviews, observation, document/content analysis, and surveys
among others. Please see the interview, observation and document/content analysis sections for
an in-depth understanding of how to analyze these methods.
Examples in Museums
A case study method was used to analyze the Animal Eyes7 exhibit in the Museum of
Ophthalmology in San Francisco. They wanted to look at this specific exhibit as their unit of
analysis. The museum used the case study to help analyze how their visitors interacted with the
exhibit to help make improvements for the future. They used observation, open-ended interview
questions, and pre/post tests to see visitor knowledge.
FOCUS GROUPS
Description
Focus groups are defined as, “a research technique that collects data through group interaction on
a topic determined by the researcher” (Morgan, 1996, p. 130). Focus groups consist of 8 to 12
people who have some connection to what is being evaluated, for example, being a part of a
specific learning program at Shedd (Frechtling, 1997). The unique part of a focus group is that a
focus group relies on group dynamics allowing the generation of data that would not otherwise
occur without group dynamics and discussion (Frechtling, 1997).
Focus groups, as other qualitative methods, combine multiple methods of data gathering,
including interviews and participant observation (Frechtling , 1997). Focus groups are useful in
answering the same types of questions as with in-depth interviews except within a social context
(Frechtling, 1997). Some places where focus groups are incredibly helpful are:
 Identifying and defining problems in project implementation
 Identifying project strengths, weaknesses, and recommendations
 Assisting with interpretation of quantitative findings
 Obtaining perceptions of project outcomes and impacts
7 http://www.exhibitfiles.org/animal_eyes

11
RESEARCH NARRATIVE
 Generating new ideas
(Frechtling, 1997).
At Shedd focus groups would allow evaluators to understand what students liked and disliked
about a certain program and what thoughts and ideas they have to improve them. This is a great
way to see the many different viewpoints that each student might have about a program with
rebuttals and conversation among the group. The evaluator gets answers to their questions as well
as observing how the group interacts with each other.
Pros and Cons
Pros include the ability to explore thoughts and insights people have along with their behaviors
(Morgan, 1996). There is a synergy existent within focus groups that are not existent in other
qualitative methods. Focus groups are more efficient. Conducting three focus groups of 10 people
each is less time consuming than conducting 30 individual interviews (Morgan, 1996).
Cons for the focus group are associated with many of the pros. There are some findings that the
moderator can disrupt the flow of a focus group when they ask the participants to explain their
train of thought or idea (Morgan, 1996). The moderator has a lot of influence over the productivity
of the focus group and must keep the discussion flowing and also make sure no one dominates the
discussion (Frechtling, 1997). For these reasons the moderator needs to have some training in
leading a focus group or group discussion in order to make them effective. Also only certain topics
are acceptable for group discussion since group interaction is essential for a focus group (Morgan,
1996). This may eliminate sensitive/personal topics from a focus group discussion.
Analysis
Similarly for interviews an evaluator can either record a focus group session or they can take
down notes. Again it is helpful to have two people if you are planning on taking notes so as not to
split attention from the moderator to be required to both ask questions and record the data
(Fretchling, 1997).
A focus group is not the same as an interview; however they do use the same method for analyzing
the data collected. Please see the interview analysis section for a more in-depth understanding of
this type of analysis.
Examples in Museums
The Cosmic Serpent Story8 not only utilized ethnography, as mentioned previously, but also
utilized focus groups as well. This was done to help the evaluators gather feedback from their
participants to see how they felt about their experience (Stein, 2011). They wanted to understand
exactly what their participants got out of the experience, so the use of a focus group was very
helpful for these evaluation efforts.
452/The_Cosmic_Serpent_Story_Summative_Evaluation_for_Final_Report

12
RESEARCH NARRATIVE
References
Best, J.W., & Kahn, J.V. (2006). Research in Education 10th Edition. Boston: Pearson Education Inc.
Bogdan, R.C. & Bilken, S.K. (2003). Qualitative Research for Education An Introduction To Theorie and Models
Davey, L., (1991). The application of case study evaluations. Practical Assessment, Research & Evaluation,
2(9).
Diamond, J., (1999). The Pratical Evaluation Guide: Tools for Museums & Other Informal Educational Settings.
Lanham, MD: AltaMira Press.
Fitzpatrick, J.L., Sanders, J.R. & Worthen, B.R. (2004). Program Evaluation: Alternative Approaches and
Practical Guidelines 3rd Edition. Boston: Pearson Education Inc.
Frechtling, J., Sharp, L. & Wesat, (Ed.). (1997). User-Friendly Handbook for Mixed Method Evaluations.
National Science Foundation.
Frechtling, J., Westat (2010). The 2010 User-Friendly Handbook for Project Evaluation. National Science
Foundation.
Gerring, J., (2004). What Is a Case Study and What Is It Good for?. American Political Science Review, 98(2)
341-354.
Goldman, K.H., Koepfler, J. & Yocco, V. (2009). WolfQuest Summative Evaluation Full Summative Report.
Minnesota Zoo.
Korn, Randi and Associates. (2012). Summative Evaluation: Mammoth Discovery!. San Jose, CA: Children’s
Discovery Museum.
Morgan, D.L., (1996). Focus Groups. Annual Review of Sociology, 22 129-152.
Randall, et al., (2007). Ethnography and How to Do it. Fieldwork for Design (pp. 169–197). Springer.
Roberts, T., (2009). Understanding ethnography. British Journal of Midwifery, 17(5) 291-293.
Rosen, M., Kinsely, R.P., & Nelson, P. (2012), Formative/Summative Evaluation of the Gallery Guide Program
At The Frye Art Museum. New Directions Partner Institutions.
Stein, J., Valdez, S., & Jones, E. (2011). The Cosmic Serpent Story: Summative Evaluation for Final Report.
Stemler, S., (2001). An Overview of Content Analysis. Pratical Assessment, Research & Evaluation. 7(71).

L. O'Keefe Writing Sample - Qualitative Research Narrative

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (17)

Similar to L. O'Keefe Writing Sample - Qualitative Research Narrative

Similar to L. O'Keefe Writing Sample - Qualitative Research Narrative (20)

L. O'Keefe Writing Sample - Qualitative Research Narrative