Presented by Tim Brandwood and Damien Gouriet. Codeit have been involved in Verbatim Coding and Machine Learning for the last 7 years. In this presentation, they share their research on the use of Generative AI for coding market research verbatim responses, and their experiments using Generative AI to automate the coding process. They also discuss the strengths and limitations of this as a coding tool, the role of human oversight in the process, and the potential for customized machine learning models working alongside it. The highlights the importance of human expertise and the use of GPT as a labour-saving tool rather than a complete replacement for humans in the market research process. Access the presentation recording via NewMR.org/Play-Again.

1. Exploring the future of verbatim
coding with ChatGPT
NewMR Webinar June 2023
Tim Brandwood & Damien Gouriet
Digital Taxonomy

2. Founded
in 2016
Verbatim
Coding Software
Efficiency through
cutting-edge tech
WHO WE ARE….

3. THE PURPOSE OF CODING
Open-ended Comments
I like the orange taste,
but it’s too sweet
The sweetness is a bit much
I think the orange colour is
too pale and artificial looking
I don’t like the bits of
orange in it
I don’t like the idea of
artificial sweeteners
Analysable Quantitative Measures
Taste (871) 87%
Too Sweet (632) 63%
Tastes Artificial (421) 42%
Doesn’t taste like real orange (237) 24%
Appearance (423) 42%
Colour – too light/pale (302) 30%
Colour – too artificial looking (150) 15%
Colour – too dark (100) 10%
Colour – other comments (40) 4%
Ingredients (408) 41%
Artificial Sweeteners (300) 30%
Real Orange (150) 15%
Orange bits (50) 5%
Packaging (387) 39%
Bottle too big (223) 22%
Difficult to hold (150) 15%
Label hard to read (87) 8%
Dislike bottle shape (60) 6%
Requires: Accuracy and Precision

4. Extracting Themes
Complete Coding:
Generates rich, human-like themes
Performs extremely well:
CAN CHATGPT DO CODING THEN?
Quantifying Themes
“Chat” interactivity allows you to ask follow-up questions,
and dig deeper
Complete Coding:
Incomplete coverage
Struggles to accurately and precisely quantify themes. Issues include:
Miscoding
Repeatability
20% verbatims coded
80% accuracy

5. PEOPLE STILL NEED TO BE
INVOLVED IN THE PROCESS
Go further, quicker …but you still need to pedal!
ChatGPT is an electric bike for your mind.
-Richard Bowman (https://prompt.mba)

6. CODEFRAME
BUILDER

10. CUSTOM MACHINE LEARNING
Autocoding Accuracy
ChatGPT 20% 80%
Custom ML Model 60% 90%
Still has a role to play in the coding Use Case

11. GPT4, Claude, Bard etc…
Interactivity
Fine-Tuning LLMs
FUTURE DEVELOPMENTS
4

12. ChatGPT is a big step forward – but only truly useful for coding up to a point
Human oversight and quality tools needed
Custom ML models outperform ChatGPT
CONCLUSION
Together these developments point towards better, more enriched research

13. Q & A
Ray Poynter
NewMR
Tim Brandwood
Digital Taxonomy
Damien Gouriet
Digital Taxonomy

14. Sponsors