Coding Your Results
QUALITATIVE METHODS IN THE LIBRARY, PART 3
JANUARY 2017
CELIA EMMELHAINZ – ANTHROPOLOGY LIBRARIAN – UNIVERSITY OF CALIFORNIA BERKELEY
Image: contextualresearch.files.wordpress.com/2012/07/the-ethnographic-research-cycle.png
Stages of qualitative research:
Develop an anthropological question
Lit review and conversations for focus
Choose a method
Choose a sampling strategy
Collect data ethically
Analyze data by coding for themes
Share results and apply in your communities
Isaacs (2014) “An Overview of Qualitative Research Methodology for Public Health Researchers,” p. 318-21
Outline:
1. Assess the purpose you’re coding towards
2. Options for coding
3. Practice hand-coding
4. Preview of computerized coding
Think of your goal: Assessment
If your goal is assessment, you want to be able to report on the
outcomes and impact of your work processes:
Students get 25% better at searching after flipped instruction;
Donors are more engaged when we feature student stories;
Readers can’t find the VPN in drop-down menus.
Think of your goal: Description
Most library research results fall at the level of describing a
group or perception of a situation:
Users avoid the maps library because of narrow rooms;
Patrons feel sense of community at Ethnic Studies events;
Librarians use nonverbal negotiation to avoid unbalanced task-
loads on committees.
We usually stop here.
Research goal: Uncover relationships
Use textual data to explore how connected two factors are:
The size and rarity of library collections is linked to
administrative perception of library value.
Personable emails from library leadership increase library
engagement, if related to librarians’ core motivations.
Uncover relations between factors in context
Research goal: Type or concept-building
Extend or challenge existing theories, develop new typologies,
connect concepts, or build a whole new framework:
Surface acting leads to higher burnout among front-line staff,
due to less recovery time ‘off-stage.’
Students perceive the library as a) a focused study space, b) a
site of suffering, and c) a social space.
What’s Your Goal?
Note whether your goal is assessment,
description, relationship, or concept building,
and what you hope to uncover.
(three minutes)
Coding is a:
“Systematic way
to condense extensive data
into smaller analyzable units
through the creation of categories & concepts
derived from the data.”
– Lockyer 2004
“
“
Two broad approaches:
Deductive: test a clear hypothesis or question
◦“Which factors build student confidence in accessing
archives?
Inductive: generate ideas from exploring data
◦ Exploratory survey of Kazakh librarians – themes emerged of
“falling into librarianship” and “librarianship as calling”
In the grounded theory approach, code for:
■ Relevant text
■ Repeating ideas
■ Themes you notice
■ Theoretical constructs you’ve seen in literature
■ Your research concerns
Auerbach and Silverstein (2003, p. 35)
Options for Coding
#1. Coding on paper
Image: from Summer Starling – QDA with MaxQDA presentation for the D-Lab, UC Berkeley
2. #hashtags in a document
#3. excel columns
Image: Pappas, Seale, and Emmelhainz – forthcoming chapter on emotional labor in librarianship
#4. qualitative software
Try it! Hand-coding
Hand-code the interview or the survey for
themes or topics.
(five minutes)
Follow-up group discussion
Based on the survey or interview responses,
what additional questions would you ask of
these people?
(five minutes together)
Walkthrough of Atlas.ti Software
When to use CAQDAS software:
Reasons to use
•Build complex codes
•Test relationships
•Handle large data
•Good for teams
Reasons not to use
•Cost
•Learning curve
•Simple or few
interviews
Qualitative analysis software can:
Use text, PDF,
image, video
View codes in
margin
Import
demographics
Export coded
quotations
Add memos
Run searches
(X and Y, near)
Using the free Atlas.ti 8 trial
http://atlasti.com/free-trial-version/
No limits on time, limits on size
10 primary documents of any size
100 quotations
50 codes, 30 memos, 10 network views
Steps in analysis
a. Memos: Start with a close reading and note interesting
points or starter ideas.
b. Coding: Mark texts systemically with the topics you
observe, moving into specific concepts.
c. Explore: your codes in relation to each other or to
respondent demographics or situations.
d. Share: Use evocative quotations to illustrate your
findings
Kuckartz and McWhertor (2014) Qualitative Text Analysis, p.5, 9
1. Reflect with memos
At the project level:
Capture changing thoughts
Note issues and your decisions
Memo ideas to follow up on
On documents, codes, quotes:
Reflect on an idea
Note connection to related ideas
Coding: tag recurring topics or concepts
Highlight and drag code over OR right-click to add a code
Grouping codes
into families
Exploring relationships with queries
Look at quotations in relation to multiple codes
◦ job market OR relocation (broadens)
◦ job market AND relocation (narrows)
◦ job market NOT relocation
◦ student debt WITHIN cost of MLS
◦ successful hire FOLLOWS experience (within a paragraph)
Use Scope to search only some documents
Save resulting quotations under a new Super Code
AND / OR / ONE OF (not both) / NOT
Part 3: Questions & Feedback

Coding Your Results

  • 1.
    Coding Your Results QUALITATIVEMETHODS IN THE LIBRARY, PART 3 JANUARY 2017 CELIA EMMELHAINZ – ANTHROPOLOGY LIBRARIAN – UNIVERSITY OF CALIFORNIA BERKELEY
  • 2.
  • 3.
    Stages of qualitativeresearch: Develop an anthropological question Lit review and conversations for focus Choose a method Choose a sampling strategy Collect data ethically Analyze data by coding for themes Share results and apply in your communities Isaacs (2014) “An Overview of Qualitative Research Methodology for Public Health Researchers,” p. 318-21
  • 4.
    Outline: 1. Assess thepurpose you’re coding towards 2. Options for coding 3. Practice hand-coding 4. Preview of computerized coding
  • 5.
    Think of yourgoal: Assessment If your goal is assessment, you want to be able to report on the outcomes and impact of your work processes: Students get 25% better at searching after flipped instruction; Donors are more engaged when we feature student stories; Readers can’t find the VPN in drop-down menus.
  • 6.
    Think of yourgoal: Description Most library research results fall at the level of describing a group or perception of a situation: Users avoid the maps library because of narrow rooms; Patrons feel sense of community at Ethnic Studies events; Librarians use nonverbal negotiation to avoid unbalanced task- loads on committees. We usually stop here.
  • 7.
    Research goal: Uncoverrelationships Use textual data to explore how connected two factors are: The size and rarity of library collections is linked to administrative perception of library value. Personable emails from library leadership increase library engagement, if related to librarians’ core motivations. Uncover relations between factors in context
  • 8.
    Research goal: Typeor concept-building Extend or challenge existing theories, develop new typologies, connect concepts, or build a whole new framework: Surface acting leads to higher burnout among front-line staff, due to less recovery time ‘off-stage.’ Students perceive the library as a) a focused study space, b) a site of suffering, and c) a social space.
  • 9.
    What’s Your Goal? Notewhether your goal is assessment, description, relationship, or concept building, and what you hope to uncover. (three minutes)
  • 10.
    Coding is a: “Systematicway to condense extensive data into smaller analyzable units through the creation of categories & concepts derived from the data.” – Lockyer 2004 “ “
  • 11.
    Two broad approaches: Deductive:test a clear hypothesis or question ◦“Which factors build student confidence in accessing archives? Inductive: generate ideas from exploring data ◦ Exploratory survey of Kazakh librarians – themes emerged of “falling into librarianship” and “librarianship as calling”
  • 12.
    In the groundedtheory approach, code for: ■ Relevant text ■ Repeating ideas ■ Themes you notice ■ Theoretical constructs you’ve seen in literature ■ Your research concerns Auerbach and Silverstein (2003, p. 35)
  • 13.
  • 14.
    #1. Coding onpaper Image: from Summer Starling – QDA with MaxQDA presentation for the D-Lab, UC Berkeley
  • 15.
    2. #hashtags ina document
  • 16.
    #3. excel columns Image:Pappas, Seale, and Emmelhainz – forthcoming chapter on emotional labor in librarianship
  • 17.
  • 18.
    Try it! Hand-coding Hand-codethe interview or the survey for themes or topics. (five minutes)
  • 19.
    Follow-up group discussion Basedon the survey or interview responses, what additional questions would you ask of these people? (five minutes together)
  • 20.
  • 21.
    When to useCAQDAS software: Reasons to use •Build complex codes •Test relationships •Handle large data •Good for teams Reasons not to use •Cost •Learning curve •Simple or few interviews
  • 22.
    Qualitative analysis softwarecan: Use text, PDF, image, video View codes in margin Import demographics Export coded quotations Add memos Run searches (X and Y, near)
  • 23.
    Using the freeAtlas.ti 8 trial http://atlasti.com/free-trial-version/ No limits on time, limits on size 10 primary documents of any size 100 quotations 50 codes, 30 memos, 10 network views
  • 24.
    Steps in analysis a.Memos: Start with a close reading and note interesting points or starter ideas. b. Coding: Mark texts systemically with the topics you observe, moving into specific concepts. c. Explore: your codes in relation to each other or to respondent demographics or situations. d. Share: Use evocative quotations to illustrate your findings Kuckartz and McWhertor (2014) Qualitative Text Analysis, p.5, 9
  • 25.
    1. Reflect withmemos At the project level: Capture changing thoughts Note issues and your decisions Memo ideas to follow up on On documents, codes, quotes: Reflect on an idea Note connection to related ideas
  • 26.
    Coding: tag recurringtopics or concepts Highlight and drag code over OR right-click to add a code
  • 27.
  • 28.
    Exploring relationships withqueries Look at quotations in relation to multiple codes ◦ job market OR relocation (broadens) ◦ job market AND relocation (narrows) ◦ job market NOT relocation ◦ student debt WITHIN cost of MLS ◦ successful hire FOLLOWS experience (within a paragraph) Use Scope to search only some documents Save resulting quotations under a new Super Code
  • 29.
    AND / OR/ ONE OF (not both) / NOT
  • 30.

Editor's Notes

  • #2 17 people
  • #4 We’re going over the ones in red today, and others in next sessions
  • #6 Before you start coding – what’s your purpose or goal in this research!
  • #8 Note that causation much harder to prove!
  • #9 Conclusions change framework or understanding in libraries: 2/3 Example: 6 years to qualification and opportunity cost of lower status profession weeds out minority candidates before the MLS. Often a long term result of research
  • #13 Move beyond content analysis and “goes beyond merely counting words or extracting objective content from text to examine meanings, themes, and patterns” Wildemuth (2009, 309)
  • #21 CAN SEND TO YOU
  • #22 ALSO RQDA http://rqda.r-forge.r-project.org/
  • #23 BUT not fully open source.
  • #24 Atlas.ti 8 is new and documentation is still being developed. Questions to ask yourself (project plan single user download from Friese) How do you want to set up your project? Which data format do you want to use? Think of the naming convention for your data files. Where should the HU file and your data be stored? How do you want to develop your coding system (inductive, deductive, a mix of both)? Is inter-coder reliability an issue? How can the project be transferred between your two computers? How do you want to save / backup your project? What needs to be considered if you want to edit / modify your data material, especially with regard to transferring the project between computers?
  • #25 Memo – also note choices you make in data management and analysis, starter ideas Coding – two level Explore – user documents groups / families Kuckartz 5 – cites Flick ea 2004 p 9’s characteristics of qualitative research: many methods, used as appropriate, oriented to the everyday and framed by context, taking participants’ views and researcher’s reflection, goal of openness and understanding, analysis starts with individual cases, builds to construct reality, focus on text, discovery, and theory formation… 9 – studies can be ‘exploratory, descriptive, hypothesis-testing, and evaluative” - cites Diekmann 2007 p. 33-40 61 – open coding as ‘identifying and/or naming concepts” 68 – type building is good, but different goal than description or hypothesis testing 88 – “evaluative qualitative text analysis” means not only identifying concepts or codes, but assessing whether respondence evince low/med/high on those codes e.g. “self-confidence” –and then you can compare variables with cross-tabs to see if high self-confidence means low care for environment etc. [weird mix of qual/quant but useful]/
  • #26 Under-used by important [Next Note impressions and broader ideas with memos
  • #27 Can code at levels (high, medium, low) Demo: Add new codes to code manager Right click and select 1+ from list Drag and drop from side menu “Unlink” to remove codes from quote “in vivo” coding uses respondents’ wording
  • #29 o see code overlap w/each other, use co-occurrence explorer Code Family to compare many terms Adds query at top; may need to flip topmost code NOT for only one code or family Look in middle for what’s actually being searched Use 2nd AND to join layers eg AND(NOT "divine assistance" & ("conversation" & "encounter")) Semantic (SUB groups, UP level, SIB same level) require network view links first Friese 157 - WITHIN – if coded long section as childhood, and friendship within that – this will find all instances of that (useful!) FOLLOWS – adjacency operator default is paragraphs for text – so within 10 pps CO-OCCURRENCE – doesn’t matter which order within / overlapped Start with code you want to read; other just narrows down that code
  • #30 Ethics AND government = 1 result Ethics OR government = 48 results
  • #31 Thank you, I look forward to your questions.