Conducting, Analyzing, and Reporting  In-Depth Interviews for Program             Evaluation                  David Filibe...
Developing an In-Depth Interview Guide                              2
What is a Structured In-Depth Interview (IDI)? Questions are the same for each respondent Directly linked to evaluation ...
‘Five Ws and one H’(Who, What, Why, When, Where and How)• What is the specific purpose of the interviews?• What informatio...
Steps to Developing the Structured Guide•   Decide what information is needed•   Search for existing questions•   Draft ne...
GeneralQuestion Order• Opening question non-threatening, broadly worded• Early questions do not restrict what respondents ...
Conducting an In-Depth InterviewTo improve consistency• Always read the instructions to each   participant as stated• Init...
Summary of the interview process• Become familiar with the instruments - study the interview  guide• Revise guide if neces...
AnalysisInherently Qualitative – carried out without statistical methods  - detailed descriptions, direct quotations, and ...
Overview of Qualitative Analysis                 Transcription            Preliminary and Primary                    Analy...
Coding: ExampleQ: Please describe the current program approach?A: We have an upstream delivery method, primarily to   reta...
Coding: ExampleA: We have an upstream delivery method, primarily to retailers, a3rd party vendor keeps track and sends the...
Analysis of IDI Data1. Primary review – take an interview and develop categories   or themes.2. Go back to the complete se...
Writing-up Results: Forming the Narrative• Determine the central message - know your  audience• Remember the narrative is ...
A Note or Two on VerbatimNot to do:• Try not to use quotes to make your points.• Try not to overuse colorful quotes or exa...
General Rules When Writing• Empathize: put yourself in the client’s shoes.• Clarity: avoid jargon, define the unfamiliar.•...
Thank You  David Filibertodmf22@cornell.edu
Upcoming SlideShare
Loading in...5
×

Conducting, analyzing and reporting in depth interviews slideshare 0213 dmf

7,166

Published on

0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,166
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide
  • As stated earlier the narrative is defined as a story that will lead the reader through the data, helping them understand the key issues and leading them to appropriate conclusions. Within the context of the coding and content analysis from the previous steps, utilizing the background youhave of the program now tryto tell a story and paint a picture – as one senior project manager stated this is a creative process, more art than science. Looking at the individual buckets, which has more verbatims, this could be an important bucket prominent. In which bucket do you create themes, weave a story is the art. Generally, you will report qualitative data in terms of "common themes" or "a number of people said…."
  • Not to soDo not use quotes to make your points. Quoting without providing a prior interpretation, leaves readers with the impression that the author is incapable of analysis or even biased. Make the point and then use the quote as description to illustrate it.Try not to overuse colorful quotes or examples. Avoid using the same quote or description over and over. Changing quotes. Try not to edit or revise quotes. This may change the meaning and lead to distortions of findings. TO do:It is also recommended to note the recording time (number of minutes into the interview) as key points arise to support review and analysis later on.   This is so crucial and probably the best way to flag good verbatimFind a balance between the amount of quoting. Qualitative research should be filled with rich and thick description – detailed accounts. Thick description connects individual cases to larger issues in the program and serves as a linkage between the two. By describing the phenomenon, in sufficient detail, youcan begin to evaluate the extent to which the conclusions are transferable to other settings. Quotes and descriptions help the reader understand that conclusions and recommendations have been reached appropriately and not just pulled out of thin air.
  • Conducting, analyzing and reporting in depth interviews slideshare 0213 dmf

    1. 1. Conducting, Analyzing, and Reporting In-Depth Interviews for Program Evaluation David Filiberto, Ph.D. 2013
    2. 2. Developing an In-Depth Interview Guide 2
    3. 3. What is a Structured In-Depth Interview (IDI)? Questions are the same for each respondent Directly linked to evaluation goals Written accounts are created of respondents answers Questions asked usually in same order with careful consideration given to the wording and order Ample opportunity for discussion 3
    4. 4. ‘Five Ws and one H’(Who, What, Why, When, Where and How)• What is the specific purpose of the interviews?• What information is the client interested in exploring?• Who needs this information, and what are they going to do with it?Hint: Scope of Work will provide many of the answers to the questions above 4
    5. 5. Steps to Developing the Structured Guide• Decide what information is needed• Search for existing questions• Draft new questions• Order questions effectively• Pre-test and pilot the questions• Revise and pre-test until deemed acceptable 5
    6. 6. GeneralQuestion Order• Opening question non-threatening, broadly worded• Early questions do not restrict what respondents feel they can say later• Structure from general to specific• Ask about present before the past or future• Group questions according to thematic clusters (work plan for guidance)• Before asking about controversial matters, first ask about some facts• Last questions allow respondents to provide any other information - impressions of the interview Specific 6
    7. 7. Conducting an In-Depth InterviewTo improve consistency• Always read the instructions to each participant as stated• Initially ask every question as stated in the order from the guideTo improve neutrality:• Avoid agreeing or disagreeing with a respondent• Avoid indicating that a participant’s answer is ‘right,’ ‘wrong,’ ‘good,’ ‘poor,’ or ‘interesting.’• Avoid suggesting an answer or interpreting a question for a respondent• Avoid giving opinions during the interview 7
    8. 8. Summary of the interview process• Become familiar with the instruments - study the interview guide• Revise guide if necessary based on pre-test• Organize and label all data documentation materials• Test recording equipment• Greet the respondent in a friendly manner to begin establishing positive rapport.• Briefly describe the steps of the interview• Conduct the interview according to the interview guide• Give the respondent the opportunity to ask questions. Thank them• Expand your notes and transcribe interview within 24 hours if possible 8
    9. 9. AnalysisInherently Qualitative – carried out without statistical methods - detailed descriptions, direct quotations, and observations from the interview are provided as the foundation of the analysisSeveral types of non-statistical or qualitative analysis exist. – content analysis – analysis of case studies – generation of typologies (classification by common traits) – and logical analysis to name a few.Content analysis involves a process designed to condense raw data into categories or themes based on valid inference and interpretation. 9
    10. 10. Overview of Qualitative Analysis Transcription Preliminary and Primary Analysis Coding Content Analysis Reporting 10
    11. 11. Coding: ExampleQ: Please describe the current program approach?A: We have an upstream delivery method, primarily to retailers, a 3rd party vendor keeps track and sends the bulbs out. The idea is to lower the price on shelves. The major constraint is its difficult to know our customer- hard to evaluate who is buying the bulb and what sockets they are putting them into. However there are some advantages. We can control consumer choice, working with retail partners, we have dominant displays of incentivized bulbs. Yet we think EISA is going to really impact future savings… 11
    12. 12. Coding: ExampleA: We have an upstream delivery method, primarily to retailers, a3rd party vendor keeps track and sends the bulbs out. The idea isto lower the price on shelves. The major constraint is its difficult toknow our customer- hard to evaluate who is buying the bulb andwhat sockets they are putting them into. However there are someadvantages. We can control consumer choice, working with retailpartners, we have dominant displays of incentivized bulbs. Yet wethink EISA is going to really impact future savings…Respondent Excerpt Code Themes or Categories 1 Upstream delivery method UP Type of Program Constraint is its difficult to know our Weaknesses of 1 customers NEG Program 1 Control consumer choice POS Strengths of Program EISA is going to really impact future 1 savings EISA Impact of EISA 12
    13. 13. Analysis of IDI Data1. Primary review – take an interview and develop categories or themes.2. Go back to the complete set of interview data. Try out the preliminary organizing scheme to see if new categories and codes emerge.3. Find the most descriptive wording for the topics and turn them into further categories (or nodes if using NVivo).4. Reduction - look to reduce the total list by grouping related topics, illustrating interrelationships.5. Assemble the data material belonging to each category, or node, together looking for apparent overarching themes to abstract to a final report. 13
    14. 14. Writing-up Results: Forming the Narrative• Determine the central message - know your audience• Remember the narrative is defined as a story that will lead the reader through the data, helping them understand the key issues and leading them to appropriate conclusions• Preface report with a summary• Utilizing the background you have of the program now try to tell a story and paint a picture• Have a plan: know your material and budget adequate time to write, review, revise and edit 14
    15. 15. A Note or Two on VerbatimNot to do:• Try not to use quotes to make your points.• Try not to overuse colorful quotes or examples.• Changing quotes. Try not to edit or revise quotes.To do:• Note the recording time (number of minutes into the interview) as key points arise.• Find a balance between the amount of quoting.• Use quotes to emphasize or as examples that support your analysis.• Try to ID quotes which do not require edits or revisions. 15
    16. 16. General Rules When Writing• Empathize: put yourself in the client’s shoes.• Clarity: avoid jargon, define the unfamiliar.• Brevity: use words efficiently, most important facts first, remove redundancy.• Simplicity: balance detail with audience needs for clarity— significance is more important.• Word Choice: avoid needless complexity, avoid ambiguity.• Active Voice: Technical writers want to communicate as efficiently as possible and active voice is more straight forward and is stronger than passive voice.• Committing to Writing as a Process: requires planning, drafting, rereading, revising, and editing, self- review, peer-review, subject-matter expert feedback, and 16 practice.
    17. 17. Thank You David Filibertodmf22@cornell.edu

    ×