The webinar discusses the future of verbatim coding using ChatGPT, focusing on its effectiveness in analyzing open-ended comments with varying degrees of accuracy. While ChatGPT demonstrates significant efficiency in generating themes and interactivity, it requires human oversight for better precision. The discussion highlights the importance of custom machine learning models which outperform ChatGPT in certain aspects of coding tasks.

1. Exploring the future of verbatim
coding with ChatGPT
NewMR Webinar June 2023
Tim Brandwood & Damien Gouriet
Digital Taxonomy

2. Founded
in 2016
Verbatim
Coding Software
Efficiency through
cutting-edge tech
WHO WE ARE….

3. THE PURPOSE OF CODING
Open-ended Comments
I like the orange taste,
but it’s too sweet
The sweetness is a bit much
I think the orange colour is
too pale and artificial looking
I don’t like the bits of
orange in it
I don’t like the idea of
artificial sweeteners
Analysable Quantitative Measures
Taste (871) 87%
Too Sweet (632) 63%
Tastes Artificial (421) 42%
Doesn’t taste like real orange (237) 24%
Appearance (423) 42%
Colour – too light/pale (302) 30%
Colour – too artificial looking (150) 15%
Colour – too dark (100) 10%
Colour – other comments (40) 4%
Ingredients (408) 41%
Artificial Sweeteners (300) 30%
Real Orange (150) 15%
Orange bits (50) 5%
Packaging (387) 39%
Bottle too big (223) 22%
Difficult to hold (150) 15%
Label hard to read (87) 8%
Dislike bottle shape (60) 6%
Requires: Accuracy and Precision

4. Extracting Themes
Complete Coding:
Generates rich, human-like themes
Performs extremely well:
CAN CHATGPT DO CODING THEN?
Quantifying Themes
“Chat” interactivity allows you to ask follow-up questions,
and dig deeper
Complete Coding:
Incomplete coverage
Struggles to accurately and precisely quantify themes. Issues include:
Miscoding
Repeatability
20% verbatims coded
80% accuracy

5. PEOPLE STILL NEED TO BE
INVOLVED IN THE PROCESS
Go further, quicker …but you still need to pedal!
ChatGPT is an electric bike for your mind.
-Richard Bowman (https://prompt.mba)

6. CODEFRAME
BUILDER

10. CUSTOM MACHINE LEARNING
Autocoding Accuracy
ChatGPT 20% 80%
Custom ML Model 60% 90%
Still has a role to play in the coding Use Case

11. GPT4, Claude, Bard etc…
Interactivity
Fine-Tuning LLMs
FUTURE DEVELOPMENTS
4

12. ChatGPT is a big step forward – but only truly useful for coding up to a point
Human oversight and quality tools needed
Custom ML models outperform ChatGPT
CONCLUSION
Together these developments point towards better, more enriched research

13. Q & A
Ray Poynter
NewMR
Tim Brandwood
Digital Taxonomy
Damien Gouriet
Digital Taxonomy

14. Sponsors