SlideShare a Scribd company logo
1
Neuro-Symbolic Creative
Artificial Intelligence for Humor
Thomas Winters
Public PhD defense
Slides: thomaswinters.be/defense
2
One you brush and rake,
What is the difference between
leaves and a car?
the other you rush and brake.
3
I asked my
computer to tell me
a Python joke,
but it just hissed
and gave me a byte.
4
Computer
+ humor
= paradox?
5
Humor =
intrinsically human
6
Purpose of Humor = Displaying intelligence?
huh?
aha!
that’s
funny
Laugh: displaying “play”
Humor: Brain rewards noticing
incongruities & successfully resolving
+ linguistic skills
+ hard to fake personal values display
= Evolutionary advantage!
h
7
Incongruity-Resolution Theory
Based on: Ritchie, G. (1999). Developing the incongruity-resolution theory.
Obvious
Interpretation
Hidden
Interpretation
Two fish are in a tank.
Says one to the other:
Setup
Punchline
“Do you know how to
drive this thing?”
8
Human-focused definition!
Machine should not only spot
two mental images
Obvious
Interpretation
Hidden
Interpretation
But also that it is
not too hard or too easy for a human!
9
How can computers
write jokes?
10
Symbolic generator: JAPE
What’s <CharacteristicNP>
and <Characteristic1> ?
A <Word1> <Word2>.
Noun Phrase
Word1 Word2
Homophone1
Characteristic1 CharacteristicNP
What’s green and
bounces?
A spring cabbage.
spring (season)
to bounce
spring (elastic body)
cabbage
green
spring cabbage
Binsted, K., & Ritchie, G. (1994). An implemented model of punning riddles
11
MopjesBot
Het is een <Beschrijving1>
en <Beschrijving2> ?
<EersteDeel> <Rijmwoord>.
Naam
EersteDeel LaatsteDeel
Beschrijving2
Beschrijving1 Rijmwoord
Het is een Belgische politica
en komt tot net boven de
enkel?
Maggie De Sok.
Maggie De
Belgische politica
Block
Komt tot net
boven de enkel
Maggie De Block
Sok
Winters, T. (2019). Generating Dutch Punning Riddles about Current Affairs.
12
How do autoregressive language models work?
1. Open smartphone keyboard
2. Press any autocomplete word 20x
3. You’ve just generated a sentence using
an AI trained to sound similar to you!
Autocomplete counted how often you
used certain words after other words
I have been trying
to get hold of my
client since the
last few days
13
Professor Canon Law & Ex-rector KU Leuven
14
1. Counted in tweets how often Rik Torfs uses one
word after others
2. Start with two real starting words of Rik, and take
random next words
“gevolgd door”
4: een
2: zijn
1: iemand
1: acht
Beste,
15
16
≈ autocomplete
on steroids
Autoregressive models like GPT
Probability of next token given
previous tokens
17
GPT training
Whole textual internet
Training process of several million $
Best public language model
18
Simple prompt
Same 25 variations of jokes 90% of the time
Jentzsch, S., & Kersting, K. (2023). ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models
19
Role
Provoke
Chain-of-
thought
Multiple
examples
with
reasoning
Input
20
21
Neuro-Symbolic Creative
Artificial Intelligence for Humor
22
Research question
How to use & integrate for creative AI & Dutch humor?
Neural AI
Symbolic AI
Neuro-
Symbolic AI
23
Publications
Journal: 2 total, 1 first author
• Computers learning humor is no joke
T. Winters (HDSR 2021)
• RobBERTje: a distilled Dutch BERT model
P. Delobelle, T. Winters, B. Berendt (CLIN journal 2021)
Conferences: 15 total, 9 first author
• Automatic joke generation: Learning humor from examples
T. Winters, V. Nys, D. De Schreye (HCII 2018)
• Towards a general framework for humor generation from rated examples
T. Winters, V. Nys, D. De Schreye (ICCC 2019)
• Generating philosophical statements using interpolated markov models and dynamic
templates
T. Winters (ESSLLI 2019)
• Modelling mutually interactive fictional character conversational agents
T. Winters (BNAIC 2019)
• Discovering textual structures: Generative grammar induction using template trees
T. Winters, L. De Raedt (ICCC 2020)
• Dutch humor detection by generating negative examples
T. Winters, P. Delobelle (Benelearn 2020)
• Automatically generating engaging presentation slide decks
T. Winters, K. W. Mathewson (EvoMusArt 2019)
• Survival of the Wittiest: Evolving Satire with Language Models
T. Winters, P. Delobelle (ICCC 2021)
• DeepStochLog: Neural Stochastic Logic Programming
T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (AAAI 2022)
• RobBERT: a Dutch RoBERTa-based Language Model
P. Delobelle, T. Winters, B. Berendt (EMNLP 2020 findings)
• Playable experiences at the 15th AAAI conference on AIIDE
R. Liu, C. Christopher, C. Martens, S. Ontañón, P. Mirowski, K. W. Mathewson, T.
Winters, S. Farrugia (AIIDE 2019)
• Rosetta code: Improv in any language
P. Mirowski, K. Mathewson, B. Branch, T. Winters, B. Verhoeven, J. Elfving (ICCC
2020)
• SandSlide: Automatic slideshow normalization
S. Bocklandt, G. Verbruggen, T. Winters (ICDAR 2021)
• Shape inference and grammar induction for example-based procedural generation
G. Hermans, T. Winters, L. De Raedt (ICCC 2021)
• Learning to Rank Generated Portmanteau
L. Pollet, T. Winters, P. Delobelle (ICCC 2021)
Preprints: 1 total
• RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language
Use
P. Delobelle, T. Winters, B. Berendt
Abstracts: 6 total, 4 first author
• Generating Dutch punning riddles about current affairs
T. Winters (CLIN 2019)
• TorfsBot Or Not? Evaluating User Perception on Imitative Text Generation
T. Winters (CLIN 2023)
• DeepStochLog: Neural stochastic logic programming (Extended Abstract)
T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (ILP 2022)
• DeepStochLog: Neural stochastic logic programming (Extended Abstract)
T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (NeSy workshop 2022
• RobBERT: a Dutch RoBERTa-based Language Model (Extended Abstract)
P. Delobelle, T. Winters, and B. Berendt (BeneLearn 2021)
• RobBERTje: A Distilled Dutch BERT Model
P. Delobelle, T. Winters, and B. Berendt (CLIN 2021).
Total: 24 | First author: 14
Citations: 300+ | h-index: 7 | i10-index: 6
24
Contribution 2:
Dutch Language
Models for
Computational Humor
Contribution 3:
Neural Definite
Clause Grammars
Main contributions
Contribution 1:
Symbolic Creative AI
Extensions &
Applications
25
Contribution 1:
Symbolic Creative AI
Extensions &
Applications
How can symbolic text generation techniques be tailored
and learned for imitative and humorous applications?
26
Dynamic Template
Base text
Are there also atheists who do not believe in atheism?
They see the fact that the former God (…). Norse popes.
Context text
Generated Text
Are there also popes who do not believe in
God?
VBP
900
EX
200
RB
100
NNS
5
WP
500
VBP
400
VB
20
IN
300
NN
3
PRP
900
VBP
100
DT
100
NN
50
IN
500
DT
100
JJ
50
NN
20
JJ
5
NNS
10
Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019)
RB
800
28
1. Count in Rik Torfs tweets & columns how often a word
follows the previous 2-4 (late normalized interpolated)
2. Start with two real starting words of Rik, and take
random next words
“gevolgd door”
4: een
2: zijn
1: iemand
1: acht
Beste,
Interpolated Markov Chain
29
Late Normalized Interpolated Markov Chains
5-grams
good coherence
but high plagiarism
3-grams
more random but
low coherence
LNIMC: Multiply frequencies of n-grams with weight, then sum and normalize
 Keeps text on track for rare sequence, and more random for frequent 3-grams
Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019)
30
Evaluation: Local cohesion > Global form
Markov chain 35% more interactions
than dynamic template
(17K interactions on 8K tweets over from June 2017 to June 2023)
Winters, T. (2019).Generating Philosophical Statements using Interpolated Markov Models and Dynamic Templates.
31
Tweets daily poll with random tweet from either Rik Torfs or TorfsBot
32
47K votes over 602 polls
(avg 79/poll)
71% correct votes
87% correct majority
68 TorfsBot success
12 Rik Torfs fails
33
Slightly positively
correlated
(0.11 for log(interactions) with %votes)
Little difference between
algorithms
34
Templates + Grammar + Functions = Babbly
Programming language for text generation with grammars
import firstname.words
food = pasta|pizza|sushi
main = {
3: <firstname> loves <food.uppercase>!
1: <firstname> (quite|reasonably|fairly) likes <food>. Oo{1,3}h, I hope they join!
1: <firstname:protagonist> is not (quite){.5} fond of <food:>.
<firstname:protagonist> will thus not go to the <food:> (restaurant|place).
}
Generates sentences like:
• Bob loves PIZZA!
• Thomas is not quite fond of sushi. Thomas will thus not go to a sushi place.
Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
35
Mutually interactive bots
Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
36
Talk Generator
Generates humorous PowerPoints about any given topic
for presenters to improvise on
Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019)
Available on talkgenerator.com
37
Slide Generator
Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019)
Internally uses lots of handwritten grammars, combined
with search engines and knowledge graphs
38
Presentation schema
Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019)
Available on talkgenerator.com
39
Evaluation
40
GITTA: Template Trees for extracting templates
1. Join closest strings
2. Merge similar template slots
3. Iteratively simplify until convergence
A: I like my <B> and my <B>
| <G> the <B> is <F>
B: chicken | cat | dog
F: walking | jumping
G: Alice | Bob | Cathy
Winters, T. & De Raedt, L. (2020). Discovering Textual Structures: Generative Grammar Induction usng Template Trees. (ICCC 2020)
Input
41
Large Language Models
42
Contribution 2:
Dutch Language
Models for
Computational Humor
How to create powerful general Dutch language models,
and how to apply to humor?
43
RobBERT: Dutch RobBERTa model
Phase 1: Pretraining
• Pre-trained using unlabeled web-scraped Dutch
corpus for reusable base model: RobBERT model
• Distilled into smaller RobBERTje model
• Extended & further trained: RobBERT-2022
Phase 2: Finetuning
• Smaller labeled training data for finetuning task
• Outperforms other models on most Dutch NLP
tasks
 1M+ downloads as of August 2023
Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020)
Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021)
Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
44
RobBERT results
• Dutch tokenizer (v2) > English tokenizer (v1) & competitors
• Merging > other distillation for longer texts (SA)
• Further pre-training on recent data both improves & worsens performance
Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020)
Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021)
Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
45
Dutch Humor Detection
No Dutch joke dataset  Scrape Dutch jokes
But what “non-jokes”?
Previously: Proverbs & News
 But is this truly detecting humor?
Mihalcea, R., & Strapparava, C. (2005). Making computers laugh: Investigations in automatic humor
recognition.
46
Jokes are fragile!
Two fish are in a tank. Says one to the other:
“Do you know how to drive this thing?”
men bar
Dynamic Template algorithm:
Generate non-jokes by replacing keywords from other jokes!
Word-based features won’t work anymore!
Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
47
Examples of generated Dutch non-jokes
Het is groen en het is een mummie?
Kermit de Waterkant
Wat is het toppunt van principe?
1) Wachten totdat een Nederlander gaat twijfelen
2) Een Zuster met een autoladder
3) Een brandwacht brandmeester met een brandmeester
van 9 maanden
“Ober, kunt u die schrik uit mijn politieman halen? Want
ik eet liever alleen.”
"Mijn hond is heel vreselijk: Hij schreeuwt mij iedere zus
de broer.“
"Maar dat is toch niet zo heel vreselijk?“
"Jawel, want ik heb geen rapport!"
Wat staat er midden in het bos?
De kapper.
Er loopt een super vriendelijk blondje langs een armband.
Last er een toonbank: “zo, waargaan die mooie mannen
heen?” Blondje: “naar de barkeeper als er niets tussen
komt…”
Hoe heet de vrouw van Sinterklaas?
Keukentafel.
"Twee tanden zwemmen in de zee en ze zien een
stamgast op een stamgast. De ene raad zegt tegen de
andere raad: 'Hé kijk! Ons eten op een bord!'"
48
51%
60%
50%
94% 94%
47%
94% 94%
47%
99% 96%
89%
Jokes vs News Jokes vs Proverbs Jokes vs Generated Jokes
Binary classification of jokes versus texts from other domains
Naive Bayes LSTM CNN RobBERT
Much more challenging dataset!
More truthful humor detection?
Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
49
GALMET: Generating satire with two RoBERTa heads
Satire MLM
Masked Language Model
for Headlines & Satire
Randomly substitutes & adds
tokens
Dog <mask> bites man
 Dog accidentally bites man
Satire Regression
Predicts funniness of headline
Fitness function
"Most Americans Want Congress To Investigate Michael Flynn“
→ Predicted score: 0.01
News: The Perils Of Climate Change: A Rise In Sea Level Of Just One
Foot Would Be Devastating For The Man Tied To This Pier
→ Predicted score: 1.05
Hey, I’ll just alter the headline
one word at a time!
Alright! I’ll check if that edit
actually makes it funnier!
Winters T., Delobelle P. (2021). Survival of the Wittiest: Evolving Satire with Language Models. (ICCC2021)
50
Neuro-Symbolic AI
51
Contribution 3:
Neural Definite
Clause Grammars
How to integrate neural networks with symbolic AI to perform tasks that
require both symbolic reasoning and subsymbolic processing?
52
CFG: Context-Free Grammar
E --> N
E --> E, P, N
P --> [“+”]
N --> [“0”]
N --> [“1”]
…
N --> [“9”]
2 + 3 + 8
N
E
E
P N
E
P N
Is sequence in language?
Which part-of-speech?
Generate all language elements
53
PCFG: Probabilistic Context-Free Grammar
0.5 :: E --> N
0.5 :: E --> E, P, N
1 :: P --> [“+”]
0.1 :: N --> [“0”]
0.1 :: N --> [“1”]
…
0.1 :: N --> [“9”]
2 + 3 + 8
N
E
E
P N
E
P N
What is the probability of generating sequence?
Which parse is more likely?
0.5
0.5
1
1
0.5
0.1 0.1 0.1
54
DCG: Definite Clause Grammar
e(N) --> n(N).
e(N) --> e(N1), p, n(N2),
{N is N1 + N2}.
p --> [“+”].
n(0) --> [“0”].
n(1) --> [“1”].
…
n(9) --> [“9”].
2 + 3 + 8
n(2)
e(2)
e(5)
p n(3)
e(13)
p n(8)
Constrain language  Context-sensitive
Inputs & outputs through unification
55
SDCG: Stochastic Definite Clause Grammar
0.5 :: e(N) --> n(N).
0.5 :: e(N) --> e(N1), p, n(N2),
{N is N1 + N2}.
1.0 :: p --> [“+”].
0.1 :: n(0) --> [“0”].
0.1 :: n(1) --> [“1”].
…
0.1 :: n(9) --> [“9”].
2 + 3 + 8
n(2)
e(2)
e(5)
p n(3)
e(13)
p n(8)
0.5
0.1
1
1
0.1
0.1
0.5
0.5
Probability of this parse = 0.5*0.5*0.5*0.1*1*0.1*1*0.1 = 0.000125
Always
sums
to
1
per
non-terminal
Disambiguating possible parses
56
DeepStochLog
0.5 :: e(N) --> n(N).
0.5 :: e(N) --> e(N1), p, n(N2),
{N is N1 + N2}.
1.0 :: p --> [“+”].
nn(number_nn,[X],[Y],[digit]) :: n(Y) --> [X].
digit(Y) :- member(Y,[0,1,2,3,4,5,6,7,8,9]).
Probability of this parse =
0.5*0.5*0.5*pnumber_nn( =2)*1*pnumber_nn( =3)*1*pnumber_nn( =8)
+ +
Symbolic + subsymbolic sequence processing
Learning rule probabilities with neural networks
Matches SOTA results on all tested NeSy tasks, but improved scaling!
n(2)
e(2)
e(5)
p n(3)
e(13)
p n(8)
0.5
1
1
0.5
0.5
pnumber_nn( =2)
pnumber_nn( =3)
pnumber_nn( =8)
T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (2019). DeepStochLog: Neural Stochastic Logic Programming. (AAAI2022)
57
Conclusion
Contribution 1: Symbolic Creative AI Extensions & Applications
• Extended text generators for imitative text generation
• Created symbolic AI humor generators (bots & talkgenerator)
• Regular generative grammar induction method
Contribution 2: Dutch Language Models for Computational Humor
• SOTA Dutch RobBERT language models
• Dutch humor detector & improved dataset creation
• RoBERTa + genetic algorithm for humor generation
Contribution 3: Neural Definite Clause Grammars
• DeepStochLog as scalable neuro-symbolic framework
58
Neuro-Symbolic Creative
Artificial Intelligence for Humor
Thomas Winters
Public PhD defense
Slides: thomaswinters.be/defense

More Related Content

Similar to Neuro-Symbolic Creative Artificial Intelligence for Humor (PhD Defense)

Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...
Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...
Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...
Takashi Iba
 
The NLP Muppets revolution!
The NLP Muppets revolution!The NLP Muppets revolution!
The NLP Muppets revolution!
Fabio Petroni, PhD
 
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
AI Frontiers
 
The Future of Authorship: AI Text Generation
The Future of Authorship: AI Text GenerationThe Future of Authorship: AI Text Generation
The Future of Authorship: AI Text Generation
Leah Henrickson
 
The linguistic future of internet
The linguistic future of internetThe linguistic future of internet
The linguistic future of internet
Maury Martinez
 
Data Day Seattle, From NLP to AI
Data Day Seattle, From NLP to AIData Day Seattle, From NLP to AI
Data Day Seattle, From NLP to AI
Jonathan Mugan
 
AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018
AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018
AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018
CodeOps Technologies LLP
 
Binary System by Bray Avila
Binary System by Bray AvilaBinary System by Bray Avila
Binary System by Bray Avila
avilab
 
From Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial IntelligenceFrom Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial Intelligence
Jonathan Mugan
 
Artificial Intelligence and Intuition
Artificial  Intelligence  and  IntuitionArtificial  Intelligence  and  Intuition
Artificial Intelligence and Intuition
Viktor Dörfler
 
Lecture 2: Language
Lecture 2: LanguageLecture 2: Language
Lecture 2: Language
David Evans
 
Word2vec: From intuition to practice using gensim
Word2vec: From intuition to practice using gensimWord2vec: From intuition to practice using gensim
Word2vec: From intuition to practice using gensim
Edgar Marca
 
Neural Text Embeddings for Information Retrieval (WSDM 2017)
Neural Text Embeddings for Information Retrieval (WSDM 2017)Neural Text Embeddings for Information Retrieval (WSDM 2017)
Neural Text Embeddings for Information Retrieval (WSDM 2017)
Bhaskar Mitra
 
Natural Language Generation: Breaking the Hermeneutic Contract
Natural Language Generation: Breaking the Hermeneutic ContractNatural Language Generation: Breaking the Hermeneutic Contract
Natural Language Generation: Breaking the Hermeneutic Contract
Leah Henrickson
 
History of F#, and the ML family of languages.
History of F#, and the ML family of languages. History of F#, and the ML family of languages.
History of F#, and the ML family of languages.
Rachel Reese
 
NLP Introduction.ppt machine learning presentation
NLP  Introduction.ppt machine learning presentationNLP  Introduction.ppt machine learning presentation
NLP Introduction.ppt machine learning presentation
PriyankaRamavath3
 
AI based language learning tools
AI based language learning toolsAI based language learning tools
AI based language learning tools
Rakuten Group, Inc.
 
Deep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word EmbeddingsDeep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word Embeddings
Roelof Pieters
 
Social media & sentiment analysis splunk conf2012
Social media & sentiment analysis   splunk conf2012Social media & sentiment analysis   splunk conf2012
Social media & sentiment analysis splunk conf2012
Michael Wilde
 
2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...
2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...
2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...
asahiushio1
 

Similar to Neuro-Symbolic Creative Artificial Intelligence for Humor (PhD Defense) (20)

Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...
Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...
Evolution of Pattern Languages: Designing Human Actions, Dialogue, & Films (P...
 
The NLP Muppets revolution!
The NLP Muppets revolution!The NLP Muppets revolution!
The NLP Muppets revolution!
 
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
 
The Future of Authorship: AI Text Generation
The Future of Authorship: AI Text GenerationThe Future of Authorship: AI Text Generation
The Future of Authorship: AI Text Generation
 
The linguistic future of internet
The linguistic future of internetThe linguistic future of internet
The linguistic future of internet
 
Data Day Seattle, From NLP to AI
Data Day Seattle, From NLP to AIData Day Seattle, From NLP to AI
Data Day Seattle, From NLP to AI
 
AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018
AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018
AI: Feats, Limits and Caveats - Monojit - Opening Keynote AI Dev Days 2018
 
Binary System by Bray Avila
Binary System by Bray AvilaBinary System by Bray Avila
Binary System by Bray Avila
 
From Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial IntelligenceFrom Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial Intelligence
 
Artificial Intelligence and Intuition
Artificial  Intelligence  and  IntuitionArtificial  Intelligence  and  Intuition
Artificial Intelligence and Intuition
 
Lecture 2: Language
Lecture 2: LanguageLecture 2: Language
Lecture 2: Language
 
Word2vec: From intuition to practice using gensim
Word2vec: From intuition to practice using gensimWord2vec: From intuition to practice using gensim
Word2vec: From intuition to practice using gensim
 
Neural Text Embeddings for Information Retrieval (WSDM 2017)
Neural Text Embeddings for Information Retrieval (WSDM 2017)Neural Text Embeddings for Information Retrieval (WSDM 2017)
Neural Text Embeddings for Information Retrieval (WSDM 2017)
 
Natural Language Generation: Breaking the Hermeneutic Contract
Natural Language Generation: Breaking the Hermeneutic ContractNatural Language Generation: Breaking the Hermeneutic Contract
Natural Language Generation: Breaking the Hermeneutic Contract
 
History of F#, and the ML family of languages.
History of F#, and the ML family of languages. History of F#, and the ML family of languages.
History of F#, and the ML family of languages.
 
NLP Introduction.ppt machine learning presentation
NLP  Introduction.ppt machine learning presentationNLP  Introduction.ppt machine learning presentation
NLP Introduction.ppt machine learning presentation
 
AI based language learning tools
AI based language learning toolsAI based language learning tools
AI based language learning tools
 
Deep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word EmbeddingsDeep Learning for Natural Language Processing: Word Embeddings
Deep Learning for Natural Language Processing: Word Embeddings
 
Social media & sentiment analysis splunk conf2012
Social media & sentiment analysis   splunk conf2012Social media & sentiment analysis   splunk conf2012
Social media & sentiment analysis splunk conf2012
 
2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...
2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...
2021-05, ACL, BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language ...
 

More from Thomas Winters

Prompt engineering: De kunst van het leren communiceren met AI
Prompt engineering: De kunst van het leren communiceren met AIPrompt engineering: De kunst van het leren communiceren met AI
Prompt engineering: De kunst van het leren communiceren met AI
Thomas Winters
 
Wetenschapscommunicatie on steroids
Wetenschapscommunicatie on steroidsWetenschapscommunicatie on steroids
Wetenschapscommunicatie on steroids
Thomas Winters
 
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
Thomas Winters
 
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
Thomas Winters
 
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
Thomas Winters
 
Pret met Creatieve Computers
Pret met Creatieve ComputersPret met Creatieve Computers
Pret met Creatieve Computers
Thomas Winters
 
Hoe leer je computers humor?
Hoe leer je computers humor?Hoe leer je computers humor?
Hoe leer je computers humor?
Thomas Winters
 
Hoe schrijven computers zelf tekst? (Kinderlezing)
Hoe schrijven computers zelf tekst? (Kinderlezing)Hoe schrijven computers zelf tekst? (Kinderlezing)
Hoe schrijven computers zelf tekst? (Kinderlezing)
Thomas Winters
 
AI als creatieve partner
AI als creatieve partnerAI als creatieve partner
AI als creatieve partner
Thomas Winters
 
De magie achter afbeeldingsgenerators
De magie achter afbeeldingsgeneratorsDe magie achter afbeeldingsgenerators
De magie achter afbeeldingsgenerators
Thomas Winters
 
How to Attract & Survive Media Attention as PhD
How to Attract & Survive Media Attention as PhDHow to Attract & Survive Media Attention as PhD
How to Attract & Survive Media Attention as PhD
Thomas Winters
 
How can AI be a creative partner for PR & marketing?
How can AI be a creative partner for PR & marketing?How can AI be a creative partner for PR & marketing?
How can AI be a creative partner for PR & marketing?
Thomas Winters
 
Beter leren praten met Artificiële Intelligentie
Beter leren praten met Artificiële IntelligentieBeter leren praten met Artificiële Intelligentie
Beter leren praten met Artificiële Intelligentie
Thomas Winters
 
TorfsBotOrNot @ Nerdland Festival
TorfsBotOrNot @ Nerdland FestivalTorfsBotOrNot @ Nerdland Festival
TorfsBotOrNot @ Nerdland Festival
Thomas Winters
 
Creative AI for Improv Theatre
Creative AI for Improv TheatreCreative AI for Improv Theatre
Creative AI for Improv Theatre
Thomas Winters
 
Humor Workshop: Hoe schrijf je satire? (KU Leugen)
Humor Workshop: Hoe schrijf je satire? (KU Leugen)Humor Workshop: Hoe schrijf je satire? (KU Leugen)
Humor Workshop: Hoe schrijf je satire? (KU Leugen)
Thomas Winters
 
Survival of the Wittiest: Evolving Satire with Language Models
Survival of the Wittiest: Evolving Satire with Language ModelsSurvival of the Wittiest: Evolving Satire with Language Models
Survival of the Wittiest: Evolving Satire with Language Models
Thomas Winters
 
Discovering Textual Structures: Generative Grammar Induction using Template T...
Discovering Textual Structures: Generative Grammar Induction using Template T...Discovering Textual Structures: Generative Grammar Induction using Template T...
Discovering Textual Structures: Generative Grammar Induction using Template T...
Thomas Winters
 
Modelling Mutually Interactive Fictional Character Conversational Agents
Modelling Mutually Interactive Fictional Character Conversational AgentsModelling Mutually Interactive Fictional Character Conversational Agents
Modelling Mutually Interactive Fictional Character Conversational Agents
Thomas Winters
 
Generating Philosophical Statements using Interpolated Markov Models and Dyna...
Generating Philosophical Statements using Interpolated Markov Models and Dyna...Generating Philosophical Statements using Interpolated Markov Models and Dyna...
Generating Philosophical Statements using Interpolated Markov Models and Dyna...
Thomas Winters
 

More from Thomas Winters (20)

Prompt engineering: De kunst van het leren communiceren met AI
Prompt engineering: De kunst van het leren communiceren met AIPrompt engineering: De kunst van het leren communiceren met AI
Prompt engineering: De kunst van het leren communiceren met AI
 
Wetenschapscommunicatie on steroids
Wetenschapscommunicatie on steroidsWetenschapscommunicatie on steroids
Wetenschapscommunicatie on steroids
 
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
 
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
 
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
 
Pret met Creatieve Computers
Pret met Creatieve ComputersPret met Creatieve Computers
Pret met Creatieve Computers
 
Hoe leer je computers humor?
Hoe leer je computers humor?Hoe leer je computers humor?
Hoe leer je computers humor?
 
Hoe schrijven computers zelf tekst? (Kinderlezing)
Hoe schrijven computers zelf tekst? (Kinderlezing)Hoe schrijven computers zelf tekst? (Kinderlezing)
Hoe schrijven computers zelf tekst? (Kinderlezing)
 
AI als creatieve partner
AI als creatieve partnerAI als creatieve partner
AI als creatieve partner
 
De magie achter afbeeldingsgenerators
De magie achter afbeeldingsgeneratorsDe magie achter afbeeldingsgenerators
De magie achter afbeeldingsgenerators
 
How to Attract & Survive Media Attention as PhD
How to Attract & Survive Media Attention as PhDHow to Attract & Survive Media Attention as PhD
How to Attract & Survive Media Attention as PhD
 
How can AI be a creative partner for PR & marketing?
How can AI be a creative partner for PR & marketing?How can AI be a creative partner for PR & marketing?
How can AI be a creative partner for PR & marketing?
 
Beter leren praten met Artificiële Intelligentie
Beter leren praten met Artificiële IntelligentieBeter leren praten met Artificiële Intelligentie
Beter leren praten met Artificiële Intelligentie
 
TorfsBotOrNot @ Nerdland Festival
TorfsBotOrNot @ Nerdland FestivalTorfsBotOrNot @ Nerdland Festival
TorfsBotOrNot @ Nerdland Festival
 
Creative AI for Improv Theatre
Creative AI for Improv TheatreCreative AI for Improv Theatre
Creative AI for Improv Theatre
 
Humor Workshop: Hoe schrijf je satire? (KU Leugen)
Humor Workshop: Hoe schrijf je satire? (KU Leugen)Humor Workshop: Hoe schrijf je satire? (KU Leugen)
Humor Workshop: Hoe schrijf je satire? (KU Leugen)
 
Survival of the Wittiest: Evolving Satire with Language Models
Survival of the Wittiest: Evolving Satire with Language ModelsSurvival of the Wittiest: Evolving Satire with Language Models
Survival of the Wittiest: Evolving Satire with Language Models
 
Discovering Textual Structures: Generative Grammar Induction using Template T...
Discovering Textual Structures: Generative Grammar Induction using Template T...Discovering Textual Structures: Generative Grammar Induction using Template T...
Discovering Textual Structures: Generative Grammar Induction using Template T...
 
Modelling Mutually Interactive Fictional Character Conversational Agents
Modelling Mutually Interactive Fictional Character Conversational AgentsModelling Mutually Interactive Fictional Character Conversational Agents
Modelling Mutually Interactive Fictional Character Conversational Agents
 
Generating Philosophical Statements using Interpolated Markov Models and Dyna...
Generating Philosophical Statements using Interpolated Markov Models and Dyna...Generating Philosophical Statements using Interpolated Markov Models and Dyna...
Generating Philosophical Statements using Interpolated Markov Models and Dyna...
 

Recently uploaded

原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样
原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样
原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样
7lkkjxt
 
快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样
快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样
快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样
ryxqoswi
 
Lifecycle of a GME Trader: From Newbie to Diamond Hands
Lifecycle of a GME Trader: From Newbie to Diamond HandsLifecycle of a GME Trader: From Newbie to Diamond Hands
Lifecycle of a GME Trader: From Newbie to Diamond Hands
mediavestfzllc
 
UR BHATTI ACADEMY AND ONLINE COURSES.pdf
UR BHATTI ACADEMY AND ONLINE COURSES.pdfUR BHATTI ACADEMY AND ONLINE COURSES.pdf
UR BHATTI ACADEMY AND ONLINE COURSES.pdf
urbhattiacademy
 
STUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISM
STUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISMSTUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISM
STUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISM
AJHSSR Journal
 
Your LinkedIn Success Starts Here.......
Your LinkedIn Success Starts Here.......Your LinkedIn Success Starts Here.......
Your LinkedIn Success Starts Here.......
SocioCosmos
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAMLORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAM
lorraineandreiamcidl
 
The Evolution of SEO: Insights from a Leading Digital Marketing Agency
The Evolution of SEO: Insights from a Leading Digital Marketing AgencyThe Evolution of SEO: Insights from a Leading Digital Marketing Agency
The Evolution of SEO: Insights from a Leading Digital Marketing Agency
Digital Marketing Lab
 
HOW TO USE THREADS an Instagram App_ by Clarissa Credito
HOW TO USE THREADS an Instagram App_ by Clarissa CreditoHOW TO USE THREADS an Instagram App_ by Clarissa Credito
HOW TO USE THREADS an Instagram App_ by Clarissa Credito
ClarissaAlanoCredito
 
EASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANEEASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANE
Febless Hernane
 
Dominate Reddit Discussions.............
Dominate Reddit Discussions.............Dominate Reddit Discussions.............
Dominate Reddit Discussions.............
SocioCosmos
 
Project Serenity — 33% Life-time Commissions.docx
Project Serenity — 33% Life-time Commissions.docxProject Serenity — 33% Life-time Commissions.docx
Project Serenity — 33% Life-time Commissions.docx
zeqirielmedina8
 
HMS Facebook Stories All V1 06092024.docx
HMS Facebook Stories All V1 06092024.docxHMS Facebook Stories All V1 06092024.docx
HMS Facebook Stories All V1 06092024.docx
Charles Bayless
 
HOW TO USE FACEBOOK _ by Clarissa Credito
HOW TO USE FACEBOOK _ by Clarissa CreditoHOW TO USE FACEBOOK _ by Clarissa Credito
HOW TO USE FACEBOOK _ by Clarissa Credito
ClarissaAlanoCredito
 
EASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANEEASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANE
Febless Hernane
 

Recently uploaded (15)

原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样
原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样
原版制作(Hull毕业证书)赫尔大学毕业证Offer一模一样
 
快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样
快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样
快速办理(BCR毕业证书)加州大学河滨分校毕业证文凭证书一模一样
 
Lifecycle of a GME Trader: From Newbie to Diamond Hands
Lifecycle of a GME Trader: From Newbie to Diamond HandsLifecycle of a GME Trader: From Newbie to Diamond Hands
Lifecycle of a GME Trader: From Newbie to Diamond Hands
 
UR BHATTI ACADEMY AND ONLINE COURSES.pdf
UR BHATTI ACADEMY AND ONLINE COURSES.pdfUR BHATTI ACADEMY AND ONLINE COURSES.pdf
UR BHATTI ACADEMY AND ONLINE COURSES.pdf
 
STUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISM
STUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISMSTUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISM
STUDY ON THE DEVELOPMENT STRATEGY OF HUZHOU TOURISM
 
Your LinkedIn Success Starts Here.......
Your LinkedIn Success Starts Here.......Your LinkedIn Success Starts Here.......
Your LinkedIn Success Starts Here.......
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAMLORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE TELEGRAM
 
The Evolution of SEO: Insights from a Leading Digital Marketing Agency
The Evolution of SEO: Insights from a Leading Digital Marketing AgencyThe Evolution of SEO: Insights from a Leading Digital Marketing Agency
The Evolution of SEO: Insights from a Leading Digital Marketing Agency
 
HOW TO USE THREADS an Instagram App_ by Clarissa Credito
HOW TO USE THREADS an Instagram App_ by Clarissa CreditoHOW TO USE THREADS an Instagram App_ by Clarissa Credito
HOW TO USE THREADS an Instagram App_ by Clarissa Credito
 
EASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANEEASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE REMINI BY: FEBLESS HERNANE
 
Dominate Reddit Discussions.............
Dominate Reddit Discussions.............Dominate Reddit Discussions.............
Dominate Reddit Discussions.............
 
Project Serenity — 33% Life-time Commissions.docx
Project Serenity — 33% Life-time Commissions.docxProject Serenity — 33% Life-time Commissions.docx
Project Serenity — 33% Life-time Commissions.docx
 
HMS Facebook Stories All V1 06092024.docx
HMS Facebook Stories All V1 06092024.docxHMS Facebook Stories All V1 06092024.docx
HMS Facebook Stories All V1 06092024.docx
 
HOW TO USE FACEBOOK _ by Clarissa Credito
HOW TO USE FACEBOOK _ by Clarissa CreditoHOW TO USE FACEBOOK _ by Clarissa Credito
HOW TO USE FACEBOOK _ by Clarissa Credito
 
EASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANEEASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANE
EASY TUTORIAL OF HOW TO USE G-TEAMS BY: FEBLESS HERNANE
 

Neuro-Symbolic Creative Artificial Intelligence for Humor (PhD Defense)

  • 1. 1 Neuro-Symbolic Creative Artificial Intelligence for Humor Thomas Winters Public PhD defense Slides: thomaswinters.be/defense
  • 2. 2 One you brush and rake, What is the difference between leaves and a car? the other you rush and brake.
  • 3. 3 I asked my computer to tell me a Python joke, but it just hissed and gave me a byte.
  • 6. 6 Purpose of Humor = Displaying intelligence? huh? aha! that’s funny Laugh: displaying “play” Humor: Brain rewards noticing incongruities & successfully resolving + linguistic skills + hard to fake personal values display = Evolutionary advantage! h
  • 7. 7 Incongruity-Resolution Theory Based on: Ritchie, G. (1999). Developing the incongruity-resolution theory. Obvious Interpretation Hidden Interpretation Two fish are in a tank. Says one to the other: Setup Punchline “Do you know how to drive this thing?”
  • 8. 8 Human-focused definition! Machine should not only spot two mental images Obvious Interpretation Hidden Interpretation But also that it is not too hard or too easy for a human!
  • 10. 10 Symbolic generator: JAPE What’s <CharacteristicNP> and <Characteristic1> ? A <Word1> <Word2>. Noun Phrase Word1 Word2 Homophone1 Characteristic1 CharacteristicNP What’s green and bounces? A spring cabbage. spring (season) to bounce spring (elastic body) cabbage green spring cabbage Binsted, K., & Ritchie, G. (1994). An implemented model of punning riddles
  • 11. 11 MopjesBot Het is een <Beschrijving1> en <Beschrijving2> ? <EersteDeel> <Rijmwoord>. Naam EersteDeel LaatsteDeel Beschrijving2 Beschrijving1 Rijmwoord Het is een Belgische politica en komt tot net boven de enkel? Maggie De Sok. Maggie De Belgische politica Block Komt tot net boven de enkel Maggie De Block Sok Winters, T. (2019). Generating Dutch Punning Riddles about Current Affairs.
  • 12. 12 How do autoregressive language models work? 1. Open smartphone keyboard 2. Press any autocomplete word 20x 3. You’ve just generated a sentence using an AI trained to sound similar to you! Autocomplete counted how often you used certain words after other words I have been trying to get hold of my client since the last few days
  • 13. 13 Professor Canon Law & Ex-rector KU Leuven
  • 14. 14 1. Counted in tweets how often Rik Torfs uses one word after others 2. Start with two real starting words of Rik, and take random next words “gevolgd door” 4: een 2: zijn 1: iemand 1: acht Beste,
  • 15. 15
  • 16. 16 ≈ autocomplete on steroids Autoregressive models like GPT Probability of next token given previous tokens
  • 17. 17 GPT training Whole textual internet Training process of several million $ Best public language model
  • 18. 18 Simple prompt Same 25 variations of jokes 90% of the time Jentzsch, S., & Kersting, K. (2023). ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models
  • 20. 20
  • 22. 22 Research question How to use & integrate for creative AI & Dutch humor? Neural AI Symbolic AI Neuro- Symbolic AI
  • 23. 23 Publications Journal: 2 total, 1 first author • Computers learning humor is no joke T. Winters (HDSR 2021) • RobBERTje: a distilled Dutch BERT model P. Delobelle, T. Winters, B. Berendt (CLIN journal 2021) Conferences: 15 total, 9 first author • Automatic joke generation: Learning humor from examples T. Winters, V. Nys, D. De Schreye (HCII 2018) • Towards a general framework for humor generation from rated examples T. Winters, V. Nys, D. De Schreye (ICCC 2019) • Generating philosophical statements using interpolated markov models and dynamic templates T. Winters (ESSLLI 2019) • Modelling mutually interactive fictional character conversational agents T. Winters (BNAIC 2019) • Discovering textual structures: Generative grammar induction using template trees T. Winters, L. De Raedt (ICCC 2020) • Dutch humor detection by generating negative examples T. Winters, P. Delobelle (Benelearn 2020) • Automatically generating engaging presentation slide decks T. Winters, K. W. Mathewson (EvoMusArt 2019) • Survival of the Wittiest: Evolving Satire with Language Models T. Winters, P. Delobelle (ICCC 2021) • DeepStochLog: Neural Stochastic Logic Programming T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (AAAI 2022) • RobBERT: a Dutch RoBERTa-based Language Model P. Delobelle, T. Winters, B. Berendt (EMNLP 2020 findings) • Playable experiences at the 15th AAAI conference on AIIDE R. Liu, C. Christopher, C. Martens, S. Ontañón, P. Mirowski, K. W. Mathewson, T. Winters, S. Farrugia (AIIDE 2019) • Rosetta code: Improv in any language P. Mirowski, K. Mathewson, B. Branch, T. Winters, B. Verhoeven, J. Elfving (ICCC 2020) • SandSlide: Automatic slideshow normalization S. Bocklandt, G. Verbruggen, T. Winters (ICDAR 2021) • Shape inference and grammar induction for example-based procedural generation G. Hermans, T. Winters, L. De Raedt (ICCC 2021) • Learning to Rank Generated Portmanteau L. Pollet, T. Winters, P. Delobelle (ICCC 2021) Preprints: 1 total • RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use P. Delobelle, T. Winters, B. Berendt Abstracts: 6 total, 4 first author • Generating Dutch punning riddles about current affairs T. Winters (CLIN 2019) • TorfsBot Or Not? Evaluating User Perception on Imitative Text Generation T. Winters (CLIN 2023) • DeepStochLog: Neural stochastic logic programming (Extended Abstract) T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (ILP 2022) • DeepStochLog: Neural stochastic logic programming (Extended Abstract) T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (NeSy workshop 2022 • RobBERT: a Dutch RoBERTa-based Language Model (Extended Abstract) P. Delobelle, T. Winters, and B. Berendt (BeneLearn 2021) • RobBERTje: A Distilled Dutch BERT Model P. Delobelle, T. Winters, and B. Berendt (CLIN 2021). Total: 24 | First author: 14 Citations: 300+ | h-index: 7 | i10-index: 6
  • 24. 24 Contribution 2: Dutch Language Models for Computational Humor Contribution 3: Neural Definite Clause Grammars Main contributions Contribution 1: Symbolic Creative AI Extensions & Applications
  • 25. 25 Contribution 1: Symbolic Creative AI Extensions & Applications How can symbolic text generation techniques be tailored and learned for imitative and humorous applications?
  • 26. 26 Dynamic Template Base text Are there also atheists who do not believe in atheism? They see the fact that the former God (…). Norse popes. Context text Generated Text Are there also popes who do not believe in God? VBP 900 EX 200 RB 100 NNS 5 WP 500 VBP 400 VB 20 IN 300 NN 3 PRP 900 VBP 100 DT 100 NN 50 IN 500 DT 100 JJ 50 NN 20 JJ 5 NNS 10 Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019) RB 800
  • 27. 28 1. Count in Rik Torfs tweets & columns how often a word follows the previous 2-4 (late normalized interpolated) 2. Start with two real starting words of Rik, and take random next words “gevolgd door” 4: een 2: zijn 1: iemand 1: acht Beste, Interpolated Markov Chain
  • 28. 29 Late Normalized Interpolated Markov Chains 5-grams good coherence but high plagiarism 3-grams more random but low coherence LNIMC: Multiply frequencies of n-grams with weight, then sum and normalize  Keeps text on track for rare sequence, and more random for frequent 3-grams Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019)
  • 29. 30 Evaluation: Local cohesion > Global form Markov chain 35% more interactions than dynamic template (17K interactions on 8K tweets over from June 2017 to June 2023) Winters, T. (2019).Generating Philosophical Statements using Interpolated Markov Models and Dynamic Templates.
  • 30. 31 Tweets daily poll with random tweet from either Rik Torfs or TorfsBot
  • 31. 32 47K votes over 602 polls (avg 79/poll) 71% correct votes 87% correct majority 68 TorfsBot success 12 Rik Torfs fails
  • 32. 33 Slightly positively correlated (0.11 for log(interactions) with %votes) Little difference between algorithms
  • 33. 34 Templates + Grammar + Functions = Babbly Programming language for text generation with grammars import firstname.words food = pasta|pizza|sushi main = { 3: <firstname> loves <food.uppercase>! 1: <firstname> (quite|reasonably|fairly) likes <food>. Oo{1,3}h, I hope they join! 1: <firstname:protagonist> is not (quite){.5} fond of <food:>. <firstname:protagonist> will thus not go to the <food:> (restaurant|place). } Generates sentences like: • Bob loves PIZZA! • Thomas is not quite fond of sushi. Thomas will thus not go to a sushi place. Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
  • 34. 35 Mutually interactive bots Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
  • 35. 36 Talk Generator Generates humorous PowerPoints about any given topic for presenters to improvise on Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Available on talkgenerator.com
  • 36. 37 Slide Generator Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Internally uses lots of handwritten grammars, combined with search engines and knowledge graphs
  • 37. 38 Presentation schema Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Available on talkgenerator.com
  • 39. 40 GITTA: Template Trees for extracting templates 1. Join closest strings 2. Merge similar template slots 3. Iteratively simplify until convergence A: I like my <B> and my <B> | <G> the <B> is <F> B: chicken | cat | dog F: walking | jumping G: Alice | Bob | Cathy Winters, T. & De Raedt, L. (2020). Discovering Textual Structures: Generative Grammar Induction usng Template Trees. (ICCC 2020) Input
  • 41. 42 Contribution 2: Dutch Language Models for Computational Humor How to create powerful general Dutch language models, and how to apply to humor?
  • 42. 43 RobBERT: Dutch RobBERTa model Phase 1: Pretraining • Pre-trained using unlabeled web-scraped Dutch corpus for reusable base model: RobBERT model • Distilled into smaller RobBERTje model • Extended & further trained: RobBERT-2022 Phase 2: Finetuning • Smaller labeled training data for finetuning task • Outperforms other models on most Dutch NLP tasks  1M+ downloads as of August 2023 Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
  • 43. 44 RobBERT results • Dutch tokenizer (v2) > English tokenizer (v1) & competitors • Merging > other distillation for longer texts (SA) • Further pre-training on recent data both improves & worsens performance Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
  • 44. 45 Dutch Humor Detection No Dutch joke dataset  Scrape Dutch jokes But what “non-jokes”? Previously: Proverbs & News  But is this truly detecting humor? Mihalcea, R., & Strapparava, C. (2005). Making computers laugh: Investigations in automatic humor recognition.
  • 45. 46 Jokes are fragile! Two fish are in a tank. Says one to the other: “Do you know how to drive this thing?” men bar Dynamic Template algorithm: Generate non-jokes by replacing keywords from other jokes! Word-based features won’t work anymore! Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
  • 46. 47 Examples of generated Dutch non-jokes Het is groen en het is een mummie? Kermit de Waterkant Wat is het toppunt van principe? 1) Wachten totdat een Nederlander gaat twijfelen 2) Een Zuster met een autoladder 3) Een brandwacht brandmeester met een brandmeester van 9 maanden “Ober, kunt u die schrik uit mijn politieman halen? Want ik eet liever alleen.” "Mijn hond is heel vreselijk: Hij schreeuwt mij iedere zus de broer.“ "Maar dat is toch niet zo heel vreselijk?“ "Jawel, want ik heb geen rapport!" Wat staat er midden in het bos? De kapper. Er loopt een super vriendelijk blondje langs een armband. Last er een toonbank: “zo, waargaan die mooie mannen heen?” Blondje: “naar de barkeeper als er niets tussen komt…” Hoe heet de vrouw van Sinterklaas? Keukentafel. "Twee tanden zwemmen in de zee en ze zien een stamgast op een stamgast. De ene raad zegt tegen de andere raad: 'Hé kijk! Ons eten op een bord!'"
  • 47. 48 51% 60% 50% 94% 94% 47% 94% 94% 47% 99% 96% 89% Jokes vs News Jokes vs Proverbs Jokes vs Generated Jokes Binary classification of jokes versus texts from other domains Naive Bayes LSTM CNN RobBERT Much more challenging dataset! More truthful humor detection? Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
  • 48. 49 GALMET: Generating satire with two RoBERTa heads Satire MLM Masked Language Model for Headlines & Satire Randomly substitutes & adds tokens Dog <mask> bites man  Dog accidentally bites man Satire Regression Predicts funniness of headline Fitness function "Most Americans Want Congress To Investigate Michael Flynn“ → Predicted score: 0.01 News: The Perils Of Climate Change: A Rise In Sea Level Of Just One Foot Would Be Devastating For The Man Tied To This Pier → Predicted score: 1.05 Hey, I’ll just alter the headline one word at a time! Alright! I’ll check if that edit actually makes it funnier! Winters T., Delobelle P. (2021). Survival of the Wittiest: Evolving Satire with Language Models. (ICCC2021)
  • 50. 51 Contribution 3: Neural Definite Clause Grammars How to integrate neural networks with symbolic AI to perform tasks that require both symbolic reasoning and subsymbolic processing?
  • 51. 52 CFG: Context-Free Grammar E --> N E --> E, P, N P --> [“+”] N --> [“0”] N --> [“1”] … N --> [“9”] 2 + 3 + 8 N E E P N E P N Is sequence in language? Which part-of-speech? Generate all language elements
  • 52. 53 PCFG: Probabilistic Context-Free Grammar 0.5 :: E --> N 0.5 :: E --> E, P, N 1 :: P --> [“+”] 0.1 :: N --> [“0”] 0.1 :: N --> [“1”] … 0.1 :: N --> [“9”] 2 + 3 + 8 N E E P N E P N What is the probability of generating sequence? Which parse is more likely? 0.5 0.5 1 1 0.5 0.1 0.1 0.1
  • 53. 54 DCG: Definite Clause Grammar e(N) --> n(N). e(N) --> e(N1), p, n(N2), {N is N1 + N2}. p --> [“+”]. n(0) --> [“0”]. n(1) --> [“1”]. … n(9) --> [“9”]. 2 + 3 + 8 n(2) e(2) e(5) p n(3) e(13) p n(8) Constrain language  Context-sensitive Inputs & outputs through unification
  • 54. 55 SDCG: Stochastic Definite Clause Grammar 0.5 :: e(N) --> n(N). 0.5 :: e(N) --> e(N1), p, n(N2), {N is N1 + N2}. 1.0 :: p --> [“+”]. 0.1 :: n(0) --> [“0”]. 0.1 :: n(1) --> [“1”]. … 0.1 :: n(9) --> [“9”]. 2 + 3 + 8 n(2) e(2) e(5) p n(3) e(13) p n(8) 0.5 0.1 1 1 0.1 0.1 0.5 0.5 Probability of this parse = 0.5*0.5*0.5*0.1*1*0.1*1*0.1 = 0.000125 Always sums to 1 per non-terminal Disambiguating possible parses
  • 55. 56 DeepStochLog 0.5 :: e(N) --> n(N). 0.5 :: e(N) --> e(N1), p, n(N2), {N is N1 + N2}. 1.0 :: p --> [“+”]. nn(number_nn,[X],[Y],[digit]) :: n(Y) --> [X]. digit(Y) :- member(Y,[0,1,2,3,4,5,6,7,8,9]). Probability of this parse = 0.5*0.5*0.5*pnumber_nn( =2)*1*pnumber_nn( =3)*1*pnumber_nn( =8) + + Symbolic + subsymbolic sequence processing Learning rule probabilities with neural networks Matches SOTA results on all tested NeSy tasks, but improved scaling! n(2) e(2) e(5) p n(3) e(13) p n(8) 0.5 1 1 0.5 0.5 pnumber_nn( =2) pnumber_nn( =3) pnumber_nn( =8) T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (2019). DeepStochLog: Neural Stochastic Logic Programming. (AAAI2022)
  • 56. 57 Conclusion Contribution 1: Symbolic Creative AI Extensions & Applications • Extended text generators for imitative text generation • Created symbolic AI humor generators (bots & talkgenerator) • Regular generative grammar induction method Contribution 2: Dutch Language Models for Computational Humor • SOTA Dutch RobBERT language models • Dutch humor detector & improved dataset creation • RoBERTa + genetic algorithm for humor generation Contribution 3: Neural Definite Clause Grammars • DeepStochLog as scalable neuro-symbolic framework
  • 57. 58 Neuro-Symbolic Creative Artificial Intelligence for Humor Thomas Winters Public PhD defense Slides: thomaswinters.be/defense

Editor's Notes

  1. JAPE joke from 30 years ago
  2. Humor intrinsically human - All civilizations perform humor - humans only animal that make jokes
  3. Haha and Aha are very similar
  4. Het is een Belgisch politica en wordt om de taille gedragen? Maggie De Rok! Het is een Belgisch politica en is een deel van de Aziatische keuken? Maggie De Wok! Het is een Belgisch politica en vertelt een relatief onschuldige leugen? Maggie De Jok! Het is een Belgisch politica en bereidt voedsel tot een maaltijd? Maggie De Kok!
  5. The research question of this thesis relates to the main paradigms in AI. First, most AI was symbolic AI. These are controllable and interpretable methods for reasoning over a space to find solutions. Nowadays, neural networks are the dominant AI paradigm, with their incredible powerful methods for learning from complex, subsymbolic data such as images and text meaning. However, they are often seen as black boxes as they are hard to interpret. There is a new trend called Neuro-symbolic AI that aims to integrate the two approaches. In this thesis, we demonstrate how we can use these paradigms for creative AI and Dutch humor, and how to integrate the two.
  6. Grouped several related publications into the following three main contributions