Slides from the CAQDAS Networking Project's webinar on 1st September 2023: Artificial Intelligence in Qualitative Data Analysis - Hoo-ha or Step-Change? During 2023 there’s been increasing discussion about the use of artificial intelligence (AI) in qualitative research, spurred by widespread access to generative-AI technologies such as ChatGPT developed by OpenAI. In this webinar Christina first recounts the history of AI in qualitative data analysis, outlining developments that far pre-date the current upsurge; including Qualrus, Discovertext, WordStat and QDA Miner, and Leximancer. She’ll then outline how generative-AI is being used in qualitative data analysis at the moment, discussing three uses: chat bots alongside other analytic tools; integrations of OpenAI technology into already established Qualitative Software; and the rise of new generative-AI applications designed specifically for qualitative data analysis tasks. Christina will open discussion about the implications of these developments for the practice of qualitative research. When are these tools appropriate? What do we need to know about them? What are the ethics of using them? What should we be cautious and excited about? How can the qualitative community shape their development? Whether you’re an advocate of the use of AI in qualitative data analysis or a sceptic, these technologies are here, they have already impacted the field of qualitative research and they will continue to do so. Join Christina to be part of the conversation, find out what’s happening, share your experiences and experimentations, your fears and hopes. Let the developers know how you want to see these technologies harnessed.

1. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Christina Silver, PhD, SFHEA, FAcSS
Associate Professor (Teaching), University of Surrey
Director, CAQDAS Networking Project (CNP)
Founder, Qualitative Data Analysis Services (QDAS)
https://linktr.ee/Christina_QDAS
Overview
Opinions
History
Developments
Implications
Discussion
https://www.qdaservices.co.uk/blog/categories/
ai-in-qualitative-research
1
2

2. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Other events on Qual-AI
• Exploring the future: Qualitative Data Analysis Supported by AI, led
by Dr Susanne Friese, Qeludra, 12 & 13 September (online workshop).
https://www.dortmunder-methoden-werkstatt.de/exploring-the-future-with-ai
• Symposium on AI in Qualitative Analysis. Two-part series, in
partnership with the Social Research Association (SRA). Part 1: 24th
Nov, Part 2: 1st Dec, 2pm London UK time (GMT). FREE online
sessions.
More details soon https://www.linkedin.com/company/caqdas-
networking-project
AI in Qualitative Software (CAQDAS) actually has a
long history
30+ years of CAQDAS – debate since the outset
•See Jackson et al (2018)
3
4

3. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Qualrus: the first “intelligent CAQDAS”
(no longer available )
• Prof. Ed Brent (University of Missouri & Ideaworks Inc. USA)
https://sociology.missouri.edu/people/brent
• Available from 2002 (15 years into CAQDAS)
• Code suggestions based on patterns in qualitative data
• case-based reasoning, natural language understanding, machine learning
and semantic networks
• suggestions could be accepted or rejected &the program learnt based
on those decisions which were used to inform subsequent suggested
codes
Suggesting codes in Qualrus
5
6

4. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
WordStat & QDA Miner: suite of text
analytics tools https://provalisresearch.com/
• Normand Péladeau & Provalis Research (Canada)
• Unsupervised machine learning tools
• Topic extraction using clustering (WordStat since 1999)
• Clustered coding (QDA Miner since 2011)
• Topic modelling (WordStat since 2014)
• Supervised machine learning tools
• Automatic document classification (WordStat since 2005)
• Query-by-example (QDA Miner since 2007)
• Code similarity searching (QDA Miner since 2011)
Many resources on Provalis tools
https://provalisresearch.com/?s=machine+learning&lang=en
CAQDAS webinar
The black box of sentiment analysis: What's in it, and
how to do it better. Normand Péladeau, President and
CEO of Provalis Research. 19 May 2021.
8
9

5. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Leximancer: automatic content analysis
and concept mapping https://www.leximancer.com/
• Andrew Smith & Michael Humphreys (University of Queensland
& Leximancer Pty Ltd, Australia)
• Text mining and content analysis software
• Models textual data to produce high-level concept maps
very quickly
• Finds concepts in context – natural language processing
CAQDAS WEBINAR: Mapping patterns of meaning latent in text: the role of objective modelling. Dr Andrew Smith,
developer of Leximancer, Melbourne Australia. 9 November 2022
10
11

6. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Discovertext: balancing what humans &
computers do best https://discovertext.com/
• Dr Stu Shulman (US National Science foundation funding, Texifter,
LLC, USA)
• Available since 2009 – CAT was predecessor (2007-2020)
• active learning loop between researcher and computer – human choices
customize and improve text processing algorithms.
• initial human coding (undertaken collaboratively by‘peers’) that is adjudicated by
humans, and then used to train a machine classifier that then codes further data
• topic modeling, sentiment detection, duplicate detection, near-duplicate
clustering, and other information retrieval and natural language technologies
Free to academics
CAQDAS WEBINAR: Humans and machines learning together. Stuart W Shulman, PhD, Founder & CEO of Texifter
and inventor of DiscoverText, USA. 14 October 2020.
12
13

7. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Current developments
Automated transcription
Standalone products
• Numerous options
• Can be formatted for CAQDAS import
CAQDAS products
• NVivo transcribe (separate)
https://lumivero.com/solutions/aggregate/tr
anscription-native-language-processing/
• Quirkos transcribe (integrated)
via https://openai.com/research/whisper
• Transana (integrated)
via https://www.speechmatics.com/
Generative AI
Genres of newer“Qualitative-AI”tools
• Use of chatbot tools alongside other
tools (CAQDAS / manual)
• Integration of generative AI into
established CAQDAS programs
• Development of new Aps designed to
harness generative AI capabilities
Use of Chatbots for QDA
• ChatGPT, Bard, etc…
• Use of LLMs for analysing open-ended
Survey responses (Mellon et al)
https://ssrn.com/abstract=4310154
• ChatGPT as summarization tool
• Philip Adu – precursor to analysis
https://www.youtube.com/watch?v=-GzjQszTe30
• Multiple uses of ChatGPT
• Andreas Muller – generating ideas for generating coding frameworks and
developing code definitions as well as summarization
https://www.youtube.com/watch?v=9yN2bmf6BNE
14
15

8. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Generative AI integrations into established
CAQDAS programs
• AI‘open’coding
• AI summaries
• AI code suggestions (web version)
https://atlasti.com/atlas-ti-ai-lab-
accelerating-innovation-for-data-analysis
• Summarize coded
segments
• Summarize text passages
• Suggest sub-codes
https://www.maxqda.com/products
/ai-assist
New Aps harnessing generative AI
• Plethora of options
• many developed in business sector that could be adapted for
academic and applied research purposes
• some developed specifically for qualitative research
• CoLoop (developed by Genei.io, London UK) https://www.coloop.ai
• AILYZE (developed by researchers at MIT (Massachusetts Institute of
Technology, USA) https://www.ailyze.com/
• Cody (developed by researchers at the KIT (Karlsruhe Institute of
Technology, Germany).
16
17

9. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
CoLoop https://www.coloop.ai
https://www.qdaservices.co.uk/post/the-ai-copilot-from-coloop-what-s-it-good-for
•Automated transcription
•Auto summarization
•Analysis grid
•AI chat
What are generative-AI tools good for?
https://www.qdaservices.co.uk/blog/categories/ai-in-qualitative-research
Familiarisation – quick overview to inform
• More useful for certain types of data
• More useful when focused on subsets
• Evaluate the role of automatic coding and
summaries for familiarisation
Codebook development - ideas we may miss
• Code suggestions – upfront and as coding
can add rigour
18
19

10. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Implications
https://www.qdaservices.co.uk/post/navigating-perspectives-on-qualitative-ai-tools-intelligence-and-ethics
Evaluating opinions on Qualitative-AI
•different types of qualitative research
•the relevant units of data for generating insight
•the importance of considering when assistance happens
•the kind of ‘intelligence’we need in qualitative analysis.
•how transparent are the AI tools
•the ethics and potential dangers of AI in qualitative analysis
20
21

11. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
What type of qualitative research
are you doing?
•QR and QDA are not homogenous
• Diversity in purposes, materials, analytic techniques,
outcome requirements
• Understood in terms of methodological spectrum
https://www.youtube.com/watch?v=wFabvZJUymI
22
23

12. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Relevant units of data for generating insight
•Unit of data =“size of text chunk” (word/sentence/paragraph)
•Used to determine“what’s going on”by the AI
•What is the“coherent”unit in the data you’re using?
•The data units AI tools can handle may not be meaningful
• Especially for‘conversational’ types of material when the size of the
meaningful data unit changes – blanket choices for such data =
unlikely useful
When does assistance happen?
Assistance first, human correction after
• ATLAS.ti and CoLoop
• the program does its thing (whether it’s generating codes or summaries) and then
we, the human interpreter, look at what it’s done, and adjust it as required according
to our needs.
Human coding first, AI summaries after
• MAXQDA
• initially implemented it’s AI Assist tool with the AI summarizing what has already
been coded
Oscillating sequencing
• DiscoverText
• Teams of humans code and their coding is adjudicated. Then the machine does it’s
thing, looking at the coding that’s been done by the humans, and using that to code
further data.
• This is then reviewed by humans, adjudicated, and sent back to the machine, which
learns from the human, and so on.
24
25

13. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
What kind of“intelligence”do we want?
•Types of intelligence…
•“interpretive”intelligence…relevant along the
methodological spectrum
•replacement or Assistance (ref CAQDAS)
How transparent are AI tools?
• What’s under the hood?
• How AI is implemented
• How it works
• Questions to ask yourself
• 3rd party or bespoke tools?
• Do developers explain clearly and in detail how they work?
• How much input do you have? Can you teach the tool and
continually adjust it?
• Can you use your RQs to inform/focus the AI?
26
27

14. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
Ethics & potential dangers of AI in QDA
Technical
• How are the language models that underlie AI developed?
• Model collapse and the complete loss of interpretation
• Machines taking over? The risk of qualitative-deepfake
Methodological
• Do you have the right to upload participant data?
• Our responsibilities to our research participants
• Is it valid if you use AI?
Don’t forget other events
• Exploring the future: Qualitative Data Analysis Supported by
AI, led by Dr Susanne Friese, Qeludra, 12 & 13 September
(online workshop). https://www.dortmunder-methoden-
werkstatt.de/exploring-the-future-with-ai
• Symposium on AI in Qualitative Analysis. Two-part series, in
partnership with the Social Research Association (SRA). Part 1:
24th Nov, Part 2: 1st Dec, 2pm London UK time (GMT). FREE
online sessions.
More details soon https://www.linkedin.com/company/caqdas-
networking-project
28
30

15. CAQDAS Networking Project webinar 01/09/2023
Christina Silver, PhD, Associate Professor (Teaching)
Dept Sociology, University of Surrey c.silver@surrey.ac.uk
References
• Jackson, K., Paulus, T., & Woolf, N. H. (2018). The Walking Dead Genealogy: Unsubstantiated Criticisms of
Qualitative Data Analysis Software (QDAS) & the Failure to Put Them to Rest. The Qualitative Report, 23(13), 74-91
https://nsuworks.nova.edu/tqr/vol23/iss13/6/?utm_source=nsuworks.nova.edu%2Ftqr%2Fvol23%2Fiss13%2F6&ut
m_medium=PDF&utm_campaign=PDFCoverPages
• Lewins A & Silver C (2020) Leximancer: Distinguishing features. CAQDAS Networking Project Software Review.
https://www.surrey.ac.uk/sites/default/files/2020-12/cnp-leximancer-5-review.pdf
• Mellon, J, Bailey, J, Scott, R, Breckwoldt, J, Miori, M & Schmedeman, P, Do AIs Know What the Most Important Issue
is? Using Language Models to Code Open-Text Social Survey Responses At Scale (August 27, 2023). Available at
SSRN: https://ssrn.com/abstract=4310154
• Rietz T & Maedche A (2021) Cody: An AI-Based System to Semi-Automate Coding for Qualitative Research.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing
Machinery https://dl.acm.org/doi/abs/10.1145/3411764.3445591
• Silver, C & Lewins A (2010) Qualrus: Distinguishing features and functions. CAQDAS Networking Project Software
Review. https://www.surrey.ac.uk/sites/default/files/QualrusdistinguishingfeaturesFINAL.pdf
• Silver C (2019) DiscoverText: Distinguishing features. CAQDAS Networking Project Software Review
https://www.surrey.ac.uk/sites/default/files/2019-12/discovertext-distinguishing-features.pdf
• Silver C & Lewins A (2020) QDA Miner: Distinguishing features. CAQDAS Networking Project Software Review.
https://www.surrey.ac.uk/sites/default/files/2020-11/qda-miner-distinguishing-features.pdf
31