As stated earlier the narrative is defined as a story that will lead the reader through the data, helping them understand the key issues and leading them to appropriate conclusions. Within the context of the coding and content analysis from the previous steps, utilizing the background youhave of the program now tryto tell a story and paint a picture – as one senior project manager stated this is a creative process, more art than science. Looking at the individual buckets, which has more verbatims, this could be an important bucket prominent. In which bucket do you create themes, weave a story is the art. Generally, you will report qualitative data in terms of "common themes" or "a number of people said…."
Not to soDo not use quotes to make your points. Quoting without providing a prior interpretation, leaves readers with the impression that the author is incapable of analysis or even biased. Make the point and then use the quote as description to illustrate it.Try not to overuse colorful quotes or examples. Avoid using the same quote or description over and over. Changing quotes. Try not to edit or revise quotes. This may change the meaning and lead to distortions of findings. TO do:It is also recommended to note the recording time (number of minutes into the interview) as key points arise to support review and analysis later on. This is so crucial and probably the best way to flag good verbatimFind a balance between the amount of quoting. Qualitative research should be filled with rich and thick description – detailed accounts. Thick description connects individual cases to larger issues in the program and serves as a linkage between the two. By describing the phenomenon, in sufficient detail, youcan begin to evaluate the extent to which the conclusions are transferable to other settings. Quotes and descriptions help the reader understand that conclusions and recommendations have been reached appropriately and not just pulled out of thin air.
Conducting, Analyzing, and Reporting In-Depth Interviews for Program Evaluation David Filiberto, Ph.D. 2013
What is a Structured In-Depth Interview (IDI)? Questions are the same for each respondent Directly linked to evaluation goals Written accounts are created of respondents answers Questions asked usually in same order with careful consideration given to the wording and order Ample opportunity for discussion 3
‘Five Ws and one H’(Who, What, Why, When, Where and How)• What is the specific purpose of the interviews?• What information is the client interested in exploring?• Who needs this information, and what are they going to do with it?Hint: Scope of Work will provide many of the answers to the questions above 4
Steps to Developing the Structured Guide• Decide what information is needed• Search for existing questions• Draft new questions• Order questions effectively• Pre-test and pilot the questions• Revise and pre-test until deemed acceptable 5
GeneralQuestion Order• Opening question non-threatening, broadly worded• Early questions do not restrict what respondents feel they can say later• Structure from general to specific• Ask about present before the past or future• Group questions according to thematic clusters (work plan for guidance)• Before asking about controversial matters, first ask about some facts• Last questions allow respondents to provide any other information - impressions of the interview Specific 6
Conducting an In-Depth InterviewTo improve consistency• Always read the instructions to each participant as stated• Initially ask every question as stated in the order from the guideTo improve neutrality:• Avoid agreeing or disagreeing with a respondent• Avoid indicating that a participant’s answer is ‘right,’ ‘wrong,’ ‘good,’ ‘poor,’ or ‘interesting.’• Avoid suggesting an answer or interpreting a question for a respondent• Avoid giving opinions during the interview 7
Summary of the interview process• Become familiar with the instruments - study the interview guide• Revise guide if necessary based on pre-test• Organize and label all data documentation materials• Test recording equipment• Greet the respondent in a friendly manner to begin establishing positive rapport.• Briefly describe the steps of the interview• Conduct the interview according to the interview guide• Give the respondent the opportunity to ask questions. Thank them• Expand your notes and transcribe interview within 24 hours if possible 8
AnalysisInherently Qualitative – carried out without statistical methods - detailed descriptions, direct quotations, and observations from the interview are provided as the foundation of the analysisSeveral types of non-statistical or qualitative analysis exist. – content analysis – analysis of case studies – generation of typologies (classification by common traits) – and logical analysis to name a few.Content analysis involves a process designed to condense raw data into categories or themes based on valid inference and interpretation. 9
Overview of Qualitative Analysis Transcription Preliminary and Primary Analysis Coding Content Analysis Reporting 10
Coding: ExampleQ: Please describe the current program approach?A: We have an upstream delivery method, primarily to retailers, a 3rd party vendor keeps track and sends the bulbs out. The idea is to lower the price on shelves. The major constraint is its difficult to know our customer- hard to evaluate who is buying the bulb and what sockets they are putting them into. However there are some advantages. We can control consumer choice, working with retail partners, we have dominant displays of incentivized bulbs. Yet we think EISA is going to really impact future savings… 11
Coding: ExampleA: We have an upstream delivery method, primarily to retailers, a3rd party vendor keeps track and sends the bulbs out. The idea isto lower the price on shelves. The major constraint is its difficult toknow our customer- hard to evaluate who is buying the bulb andwhat sockets they are putting them into. However there are someadvantages. We can control consumer choice, working with retailpartners, we have dominant displays of incentivized bulbs. Yet wethink EISA is going to really impact future savings…Respondent Excerpt Code Themes or Categories 1 Upstream delivery method UP Type of Program Constraint is its difficult to know our Weaknesses of 1 customers NEG Program 1 Control consumer choice POS Strengths of Program EISA is going to really impact future 1 savings EISA Impact of EISA 12
Analysis of IDI Data1. Primary review – take an interview and develop categories or themes.2. Go back to the complete set of interview data. Try out the preliminary organizing scheme to see if new categories and codes emerge.3. Find the most descriptive wording for the topics and turn them into further categories (or nodes if using NVivo).4. Reduction - look to reduce the total list by grouping related topics, illustrating interrelationships.5. Assemble the data material belonging to each category, or node, together looking for apparent overarching themes to abstract to a final report. 13
Writing-up Results: Forming the Narrative• Determine the central message - know your audience• Remember the narrative is defined as a story that will lead the reader through the data, helping them understand the key issues and leading them to appropriate conclusions• Preface report with a summary• Utilizing the background you have of the program now try to tell a story and paint a picture• Have a plan: know your material and budget adequate time to write, review, revise and edit 14
A Note or Two on VerbatimNot to do:• Try not to use quotes to make your points.• Try not to overuse colorful quotes or examples.• Changing quotes. Try not to edit or revise quotes.To do:• Note the recording time (number of minutes into the interview) as key points arise.• Find a balance between the amount of quoting.• Use quotes to emphasize or as examples that support your analysis.• Try to ID quotes which do not require edits or revisions. 15
General Rules When Writing• Empathize: put yourself in the client’s shoes.• Clarity: avoid jargon, define the unfamiliar.• Brevity: use words efficiently, most important facts first, remove redundancy.• Simplicity: balance detail with audience needs for clarity— significance is more important.• Word Choice: avoid needless complexity, avoid ambiguity.• Active Voice: Technical writers want to communicate as efficiently as possible and active voice is more straight forward and is stronger than passive voice.• Committing to Writing as a Process: requires planning, drafting, rereading, revising, and editing, self- review, peer-review, subject-matter expert feedback, and 16 practice.