Assignment submission for SHU Year 2 - Portfolio of Research Skills

1. Qualitative Data Analysis
‘Student L’
Lee James Cox
B0049872
MA in Technical Communication
Module: 14-7104-OOY-B-20123 – Portfolio of Research Skills (A-2012/3)
1

2. This presentation aims to show:
How I would go about analysing and presenting
interview data
based on example of:
Transcripts of 5 text chat interviews with
‘Student L’ in 2006/7
for a research question of:
How well a University module supported them
in developing research skills
2

3. Considerations for Research Analysis
Collate
Analyse
Present
Reliability
Investigation that the source data is
complete and reliable.
Coding
Assigning codes to simplify and
standardize data for analytical
purposes.
Agree purpose and format
Clear scope and question. Ensure
format is suitable for audience.
Reshaping
Ordering of chaotic data into a
standard usable format.
Annotating
Drawing attention to what you
consider to be most significant.
Validity
Describing potential bias, methods
followed and any subjective
conclusions.
Reducing
Removing ‘noise’ not relevant to the
research.
Labelling
Pigeon-holing data so it can be easily
grouped together.
Interpretation
Explaining what the analysis may
mean and themes.
Transforming
Moving data into a format that is
easier to manipulate.
Selection
Choosing of interesting, significant or
representative items to illustrate an
argument.
Conclusions
Answering the research question and
making recommendations
Summarising
Choosing one or more examples as a
synopsis of whole data set.
(Adapted from check-lists in ‘How to Research’, 2010 [1])
Note that the research question, sampling, methodology and actual collection
have already been performed and are out of scope for this presentation.
3

4. How would I
Collate
the findings?
1. Check source data reliability
–
–
–
Screen grabs rather than open documents that can be manipulated
Researcher and subject confirmation that comments are complete and not edited
Filter biased or leading questions
2. Remove personal details by ‘blacking out’ things we shouldn’t know. E.g.
–
–
Student is called L. Goldwin, has 2 children – one born in 2006, the other in 2008.
Was recently living in Israel (08/09) working in Tech Publishing for a software company
3. Remove ‘chit-chat’ dialogue that helped to build a relationship even if they may have had
some influence on answers
4. Transfer the data into a format that reduces it, and makes it easier to manipulate, e.g.
using Excel with columns headings:
5. Add in context to comments – to make them understandable and to make it easy to draw
4
attention to important comments later

5. How would I Analyse
the findings?
1. Categorise the data – to create new data points that can be extracted from the free form
comments. This helps the proving or disproving of theories in any presentation of conclusions.
E.g. If the subject area was difficult (according to questionnaire feedback) did it get most negative statements or the most
positive suggestions? (See Attitude System [2] for more detailed examples)
2. Care with coding – categories must be
clearly defined allowing any 3rd party to
independently conclude the same choice.
3. Counting - statistically counting and
correlating data sets can reveal interesting
trends and relationships between data.
E.g. was the subject simply in a good mood on 8/12/6
4. Map it – social network analysis tools may describe potential relationship between comments.
Does a positive on one statement affect the next?
5

6. How would I Present
the findings?
1. Agree a suitable format for the audience. Report with technical data or presentation with handouts?
2. Establish why the research is representative – findings should be valid & reliable
–
–
Bias and leading questions from researcher declared
Without breaking privacy or other confidentialities, establish why listen to this students’ comments
3. Directly answer the research question as an executive summary:
–
–
E.g. ‘Our research indicated that it is likely to be the most difficult topics
such as Paradigms that helps students to develop better research skills.’
Give weight to the summary by establishing a baseline (even if only assumed) around
predefined measures of what ‘good’ looks like, e.g. Mijksenaar’s gauge [4]
4. Share the key learnings (both positive and negative) from the data analysis
–
–
Extract direct quotes from subject to give weight to conclusions
Use of graphs and tables to summarise ‘workings’ and support conclusions
5. Validate my conclusions and ask for feedback from the
original researcher – Florence Dujardin
9
8
7
6
5
4
3
2
1
0
“I see the interactions
as the greatest asset of
this form of studying. ”
Realis affect
Irrealis affect
Judgements
Appreciations

7. Weaknesses & Strengths of
Qualitative Data Analysis
Weaknesses
Strengths
Bias of researcher and their positional power
over subject: means data may have inherent
reliability issues
Provides depth and detail. A single insight can be
elevated to prominence in the presentation rather
than ‘lost’ in the data.
Content analysis is complex: associating
occurrences of specific words in open dialog of
what is relevant
Even a small data set makes data analysis
possible.
Mistakes can occur in categorisation of data.
Detailed knowledge of the subject is sometimes
required to follow the flow of a conversation.
Describes people's individual experiences: a
picture can be built up about why people act in
certain ways and their feelings about these actions
Privacy more difficult to protect when full
disclosure of transcripts are required to show
reliability of source data.
More open: data can lead to expansion of subjects
responses to new topic areas not initially
considered
Key data points can be lost in the transfer the
source text in to format that can be analysed.
Likely to avoids pre-judgements
7

8. Conclusions
The sample contained a mix of methods – open interview questions as well as a specific questionnaire that
used a fixed rating system.
The richness of the content is likely to reveal interesting points for further cross-tabulation and investigation
which will make interesting reading. However the analysis phase will take some considerable time because
of the intertwined nature of natural dialog and the requirement to understand the subject matter to ‘code’
responses.
Meyer & Avery [2] succinctly describe the challenge of Qualitative Analysis:
‘Researchers are faced with an overwhelming amount of data’..they are often
interested in precisely the connections and nuances that are frequently lost when
filtering and synthesizing.’
Unfortunately there is also likely to be some question over the validity of any findings because:
i. There is only one research subject. A different subject might have completely opposite views*
ii. The interviewer has positional influence over the interviewee
*Nielsen [6] suggests a minimum for 3-5 is needed for heuristic evaluation whereas Baker & Edwards [7]
between 15 and 40.
8

9. References
[1] Blaxter, L., Hughes C. and Tight M. (2010). How to Research. McGraw Hill. 211-277.
[2] Meyer, D., Avery, L. (2009) Excel as a Qualitative Data Analysis Tool, Field Methods, 21 (1): 91112
[3] Lipovsky, C. (2008) Attitude System in 'Constructing affiliation and solidarity in job interviews',
Discourse & Communication, 2(4): 413-416
[4] Mijksenaar P. (1997) Visual Function: an Introduction to Information Design, 010 Publishers
[5] Nielsen, J. (2005) How to conduct a heuristic evaluation. Nielsen Norman Group. Online. Last
accessed 11/12/13 at http://www.useit.com/papers/heuristic/heuristic_evaluation.html
[6] Baker, S. and Edwards, R. (2013). How many qualitative interviews is enough? National Centre
for Research Methods Review Paper. Online. Last accessed 14/12/13 at
http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf
9