TW
Uploaded byThomas Winters
PPTX, PDF111 views

Neuro-Symbolic Creative Artificial Intelligence for Humor (PhD Defense)

This document discusses neuro-symbolic creative artificial intelligence for generating humor, focusing on the integration of symbolic and neural approaches. It demonstrates how computers can be trained to understand and create humor through incongruity-resolution theories and models that learn from examples. Key contributions include the development of Dutch language models for humor detection and automated joke generation techniques.

Embed presentation

Download to read offline
1 Neuro-Symbolic Creative Artificial Intelligence for Humor Thomas Winters Public PhD defense Slides: thomaswinters.be/defense
2 One you brush and rake, What is the difference between leaves and a car? the other you rush and brake.
3 I asked my computer to tell me a Python joke, but it just hissed and gave me a byte.
4 Computer + humor = paradox?
5 Humor = intrinsically human
6 Purpose of Humor = Displaying intelligence? huh? aha! that’s funny Laugh: displaying “play” Humor: Brain rewards noticing incongruities & successfully resolving + linguistic skills + hard to fake personal values display = Evolutionary advantage! h
7 Incongruity-Resolution Theory Based on: Ritchie, G. (1999). Developing the incongruity-resolution theory. Obvious Interpretation Hidden Interpretation Two fish are in a tank. Says one to the other: Setup Punchline “Do you know how to drive this thing?”
8 Human-focused definition! Machine should not only spot two mental images Obvious Interpretation Hidden Interpretation But also that it is not too hard or too easy for a human!
9 How can computers write jokes?
10 Symbolic generator: JAPE What’s <CharacteristicNP> and <Characteristic1> ? A <Word1> <Word2>. Noun Phrase Word1 Word2 Homophone1 Characteristic1 CharacteristicNP What’s green and bounces? A spring cabbage. spring (season) to bounce spring (elastic body) cabbage green spring cabbage Binsted, K., & Ritchie, G. (1994). An implemented model of punning riddles
11 MopjesBot Het is een <Beschrijving1> en <Beschrijving2> ? <EersteDeel> <Rijmwoord>. Naam EersteDeel LaatsteDeel Beschrijving2 Beschrijving1 Rijmwoord Het is een Belgische politica en komt tot net boven de enkel? Maggie De Sok. Maggie De Belgische politica Block Komt tot net boven de enkel Maggie De Block Sok Winters, T. (2019). Generating Dutch Punning Riddles about Current Affairs.
12 How do autoregressive language models work? 1. Open smartphone keyboard 2. Press any autocomplete word 20x 3. You’ve just generated a sentence using an AI trained to sound similar to you! Autocomplete counted how often you used certain words after other words I have been trying to get hold of my client since the last few days
13 Professor Canon Law & Ex-rector KU Leuven
14 1. Counted in tweets how often Rik Torfs uses one word after others 2. Start with two real starting words of Rik, and take random next words “gevolgd door” 4: een 2: zijn 1: iemand 1: acht Beste,
15
16 ≈ autocomplete on steroids Autoregressive models like GPT Probability of next token given previous tokens
17 GPT training Whole textual internet Training process of several million $ Best public language model
18 Simple prompt Same 25 variations of jokes 90% of the time Jentzsch, S., & Kersting, K. (2023). ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models
19 Role Provoke Chain-of- thought Multiple examples with reasoning Input
20
21 Neuro-Symbolic Creative Artificial Intelligence for Humor
22 Research question How to use & integrate for creative AI & Dutch humor? Neural AI Symbolic AI Neuro- Symbolic AI
23 Publications Journal: 2 total, 1 first author • Computers learning humor is no joke T. Winters (HDSR 2021) • RobBERTje: a distilled Dutch BERT model P. Delobelle, T. Winters, B. Berendt (CLIN journal 2021) Conferences: 15 total, 9 first author • Automatic joke generation: Learning humor from examples T. Winters, V. Nys, D. De Schreye (HCII 2018) • Towards a general framework for humor generation from rated examples T. Winters, V. Nys, D. De Schreye (ICCC 2019) • Generating philosophical statements using interpolated markov models and dynamic templates T. Winters (ESSLLI 2019) • Modelling mutually interactive fictional character conversational agents T. Winters (BNAIC 2019) • Discovering textual structures: Generative grammar induction using template trees T. Winters, L. De Raedt (ICCC 2020) • Dutch humor detection by generating negative examples T. Winters, P. Delobelle (Benelearn 2020) • Automatically generating engaging presentation slide decks T. Winters, K. W. Mathewson (EvoMusArt 2019) • Survival of the Wittiest: Evolving Satire with Language Models T. Winters, P. Delobelle (ICCC 2021) • DeepStochLog: Neural Stochastic Logic Programming T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (AAAI 2022) • RobBERT: a Dutch RoBERTa-based Language Model P. Delobelle, T. Winters, B. Berendt (EMNLP 2020 findings) • Playable experiences at the 15th AAAI conference on AIIDE R. Liu, C. Christopher, C. Martens, S. Ontañón, P. Mirowski, K. W. Mathewson, T. Winters, S. Farrugia (AIIDE 2019) • Rosetta code: Improv in any language P. Mirowski, K. Mathewson, B. Branch, T. Winters, B. Verhoeven, J. Elfving (ICCC 2020) • SandSlide: Automatic slideshow normalization S. Bocklandt, G. Verbruggen, T. Winters (ICDAR 2021) • Shape inference and grammar induction for example-based procedural generation G. Hermans, T. Winters, L. De Raedt (ICCC 2021) • Learning to Rank Generated Portmanteau L. Pollet, T. Winters, P. Delobelle (ICCC 2021) Preprints: 1 total • RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use P. Delobelle, T. Winters, B. Berendt Abstracts: 6 total, 4 first author • Generating Dutch punning riddles about current affairs T. Winters (CLIN 2019) • TorfsBot Or Not? Evaluating User Perception on Imitative Text Generation T. Winters (CLIN 2023) • DeepStochLog: Neural stochastic logic programming (Extended Abstract) T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (ILP 2022) • DeepStochLog: Neural stochastic logic programming (Extended Abstract) T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (NeSy workshop 2022 • RobBERT: a Dutch RoBERTa-based Language Model (Extended Abstract) P. Delobelle, T. Winters, and B. Berendt (BeneLearn 2021) • RobBERTje: A Distilled Dutch BERT Model P. Delobelle, T. Winters, and B. Berendt (CLIN 2021). Total: 24 | First author: 14 Citations: 300+ | h-index: 7 | i10-index: 6
24 Contribution 2: Dutch Language Models for Computational Humor Contribution 3: Neural Definite Clause Grammars Main contributions Contribution 1: Symbolic Creative AI Extensions & Applications
25 Contribution 1: Symbolic Creative AI Extensions & Applications How can symbolic text generation techniques be tailored and learned for imitative and humorous applications?
26 Dynamic Template Base text Are there also atheists who do not believe in atheism? They see the fact that the former God (…). Norse popes. Context text Generated Text Are there also popes who do not believe in God? VBP 900 EX 200 RB 100 NNS 5 WP 500 VBP 400 VB 20 IN 300 NN 3 PRP 900 VBP 100 DT 100 NN 50 IN 500 DT 100 JJ 50 NN 20 JJ 5 NNS 10 Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019) RB 800
28 1. Count in Rik Torfs tweets & columns how often a word follows the previous 2-4 (late normalized interpolated) 2. Start with two real starting words of Rik, and take random next words “gevolgd door” 4: een 2: zijn 1: iemand 1: acht Beste, Interpolated Markov Chain
29 Late Normalized Interpolated Markov Chains 5-grams good coherence but high plagiarism 3-grams more random but low coherence LNIMC: Multiply frequencies of n-grams with weight, then sum and normalize  Keeps text on track for rare sequence, and more random for frequent 3-grams Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019)
30 Evaluation: Local cohesion > Global form Markov chain 35% more interactions than dynamic template (17K interactions on 8K tweets over from June 2017 to June 2023) Winters, T. (2019).Generating Philosophical Statements using Interpolated Markov Models and Dynamic Templates.
31 Tweets daily poll with random tweet from either Rik Torfs or TorfsBot
32 47K votes over 602 polls (avg 79/poll) 71% correct votes 87% correct majority 68 TorfsBot success 12 Rik Torfs fails
33 Slightly positively correlated (0.11 for log(interactions) with %votes) Little difference between algorithms
34 Templates + Grammar + Functions = Babbly Programming language for text generation with grammars import firstname.words food = pasta|pizza|sushi main = { 3: <firstname> loves <food.uppercase>! 1: <firstname> (quite|reasonably|fairly) likes <food>. Oo{1,3}h, I hope they join! 1: <firstname:protagonist> is not (quite){.5} fond of <food:>. <firstname:protagonist> will thus not go to the <food:> (restaurant|place). } Generates sentences like: • Bob loves PIZZA! • Thomas is not quite fond of sushi. Thomas will thus not go to a sushi place. Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
35 Mutually interactive bots Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
36 Talk Generator Generates humorous PowerPoints about any given topic for presenters to improvise on Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Available on talkgenerator.com
37 Slide Generator Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Internally uses lots of handwritten grammars, combined with search engines and knowledge graphs
38 Presentation schema Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Available on talkgenerator.com
39 Evaluation
40 GITTA: Template Trees for extracting templates 1. Join closest strings 2. Merge similar template slots 3. Iteratively simplify until convergence A: I like my <B> and my <B> | <G> the <B> is <F> B: chicken | cat | dog F: walking | jumping G: Alice | Bob | Cathy Winters, T. & De Raedt, L. (2020). Discovering Textual Structures: Generative Grammar Induction usng Template Trees. (ICCC 2020) Input
41 Large Language Models
42 Contribution 2: Dutch Language Models for Computational Humor How to create powerful general Dutch language models, and how to apply to humor?
43 RobBERT: Dutch RobBERTa model Phase 1: Pretraining • Pre-trained using unlabeled web-scraped Dutch corpus for reusable base model: RobBERT model • Distilled into smaller RobBERTje model • Extended & further trained: RobBERT-2022 Phase 2: Finetuning • Smaller labeled training data for finetuning task • Outperforms other models on most Dutch NLP tasks  1M+ downloads as of August 2023 Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
44 RobBERT results • Dutch tokenizer (v2) > English tokenizer (v1) & competitors • Merging > other distillation for longer texts (SA) • Further pre-training on recent data both improves & worsens performance Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
45 Dutch Humor Detection No Dutch joke dataset  Scrape Dutch jokes But what “non-jokes”? Previously: Proverbs & News  But is this truly detecting humor? Mihalcea, R., & Strapparava, C. (2005). Making computers laugh: Investigations in automatic humor recognition.
46 Jokes are fragile! Two fish are in a tank. Says one to the other: “Do you know how to drive this thing?” men bar Dynamic Template algorithm: Generate non-jokes by replacing keywords from other jokes! Word-based features won’t work anymore! Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
47 Examples of generated Dutch non-jokes Het is groen en het is een mummie? Kermit de Waterkant Wat is het toppunt van principe? 1) Wachten totdat een Nederlander gaat twijfelen 2) Een Zuster met een autoladder 3) Een brandwacht brandmeester met een brandmeester van 9 maanden “Ober, kunt u die schrik uit mijn politieman halen? Want ik eet liever alleen.” "Mijn hond is heel vreselijk: Hij schreeuwt mij iedere zus de broer.“ "Maar dat is toch niet zo heel vreselijk?“ "Jawel, want ik heb geen rapport!" Wat staat er midden in het bos? De kapper. Er loopt een super vriendelijk blondje langs een armband. Last er een toonbank: “zo, waargaan die mooie mannen heen?” Blondje: “naar de barkeeper als er niets tussen komt…” Hoe heet de vrouw van Sinterklaas? Keukentafel. "Twee tanden zwemmen in de zee en ze zien een stamgast op een stamgast. De ene raad zegt tegen de andere raad: 'Hé kijk! Ons eten op een bord!'"
48 51% 60% 50% 94% 94% 47% 94% 94% 47% 99% 96% 89% Jokes vs News Jokes vs Proverbs Jokes vs Generated Jokes Binary classification of jokes versus texts from other domains Naive Bayes LSTM CNN RobBERT Much more challenging dataset! More truthful humor detection? Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
49 GALMET: Generating satire with two RoBERTa heads Satire MLM Masked Language Model for Headlines & Satire Randomly substitutes & adds tokens Dog <mask> bites man  Dog accidentally bites man Satire Regression Predicts funniness of headline Fitness function "Most Americans Want Congress To Investigate Michael Flynn“ → Predicted score: 0.01 News: The Perils Of Climate Change: A Rise In Sea Level Of Just One Foot Would Be Devastating For The Man Tied To This Pier → Predicted score: 1.05 Hey, I’ll just alter the headline one word at a time! Alright! I’ll check if that edit actually makes it funnier! Winters T., Delobelle P. (2021). Survival of the Wittiest: Evolving Satire with Language Models. (ICCC2021)
50 Neuro-Symbolic AI
51 Contribution 3: Neural Definite Clause Grammars How to integrate neural networks with symbolic AI to perform tasks that require both symbolic reasoning and subsymbolic processing?
52 CFG: Context-Free Grammar E --> N E --> E, P, N P --> [“+”] N --> [“0”] N --> [“1”] … N --> [“9”] 2 + 3 + 8 N E E P N E P N Is sequence in language? Which part-of-speech? Generate all language elements
53 PCFG: Probabilistic Context-Free Grammar 0.5 :: E --> N 0.5 :: E --> E, P, N 1 :: P --> [“+”] 0.1 :: N --> [“0”] 0.1 :: N --> [“1”] … 0.1 :: N --> [“9”] 2 + 3 + 8 N E E P N E P N What is the probability of generating sequence? Which parse is more likely? 0.5 0.5 1 1 0.5 0.1 0.1 0.1
54 DCG: Definite Clause Grammar e(N) --> n(N). e(N) --> e(N1), p, n(N2), {N is N1 + N2}. p --> [“+”]. n(0) --> [“0”]. n(1) --> [“1”]. … n(9) --> [“9”]. 2 + 3 + 8 n(2) e(2) e(5) p n(3) e(13) p n(8) Constrain language  Context-sensitive Inputs & outputs through unification
55 SDCG: Stochastic Definite Clause Grammar 0.5 :: e(N) --> n(N). 0.5 :: e(N) --> e(N1), p, n(N2), {N is N1 + N2}. 1.0 :: p --> [“+”]. 0.1 :: n(0) --> [“0”]. 0.1 :: n(1) --> [“1”]. … 0.1 :: n(9) --> [“9”]. 2 + 3 + 8 n(2) e(2) e(5) p n(3) e(13) p n(8) 0.5 0.1 1 1 0.1 0.1 0.5 0.5 Probability of this parse = 0.5*0.5*0.5*0.1*1*0.1*1*0.1 = 0.000125 Always sums to 1 per non-terminal Disambiguating possible parses
56 DeepStochLog 0.5 :: e(N) --> n(N). 0.5 :: e(N) --> e(N1), p, n(N2), {N is N1 + N2}. 1.0 :: p --> [“+”]. nn(number_nn,[X],[Y],[digit]) :: n(Y) --> [X]. digit(Y) :- member(Y,[0,1,2,3,4,5,6,7,8,9]). Probability of this parse = 0.5*0.5*0.5*pnumber_nn( =2)*1*pnumber_nn( =3)*1*pnumber_nn( =8) + + Symbolic + subsymbolic sequence processing Learning rule probabilities with neural networks Matches SOTA results on all tested NeSy tasks, but improved scaling! n(2) e(2) e(5) p n(3) e(13) p n(8) 0.5 1 1 0.5 0.5 pnumber_nn( =2) pnumber_nn( =3) pnumber_nn( =8) T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (2019). DeepStochLog: Neural Stochastic Logic Programming. (AAAI2022)
57 Conclusion Contribution 1: Symbolic Creative AI Extensions & Applications • Extended text generators for imitative text generation • Created symbolic AI humor generators (bots & talkgenerator) • Regular generative grammar induction method Contribution 2: Dutch Language Models for Computational Humor • SOTA Dutch RobBERT language models • Dutch humor detector & improved dataset creation • RoBERTa + genetic algorithm for humor generation Contribution 3: Neural Definite Clause Grammars • DeepStochLog as scalable neuro-symbolic framework
58 Neuro-Symbolic Creative Artificial Intelligence for Humor Thomas Winters Public PhD defense Slides: thomaswinters.be/defense

More Related Content

PPTX
Computational Humor: Can a machine have a sense of humor? (2020)
byThomas Winters
 
PPTX
Computational Humor: Can a machine have a sense of humor (2022)
byThomas Winters
 
PDF
The World of ChatGPT.pdf
byRia Morgan
 
PPTX
Writing Machines: Detection and Stylometric Profiling
byGeorgeMikros3
 
PPTX
OpenAI Chatgpt.pptx
byNawroz University
 
PPTX
Computational Humor: Can a machine have a sense of humor (December 2022)
byThomas Winters
 
PPTX
How do you teach computers humor + Text Generators as Creative Partners (May ...
byThomas Winters
 
PDF
CHATBOTS IN A BOTNET WORLD
byIJCI JOURNAL
 
Computational Humor: Can a machine have a sense of humor? (2020)
byThomas Winters
 
Computational Humor: Can a machine have a sense of humor (2022)
byThomas Winters
 
The World of ChatGPT.pdf
byRia Morgan
 
Writing Machines: Detection and Stylometric Profiling
byGeorgeMikros3
 
OpenAI Chatgpt.pptx
byNawroz University
 
Computational Humor: Can a machine have a sense of humor (December 2022)
byThomas Winters
 
How do you teach computers humor + Text Generators as Creative Partners (May ...
byThomas Winters
 
CHATBOTS IN A BOTNET WORLD
byIJCI JOURNAL
 

Similar to Neuro-Symbolic Creative Artificial Intelligence for Humor (PhD Defense)

PDF
And then there were ... Large Language Models
byLeon Dohmen
 
PDF
INTEGRATING EXTRACTED INFORMATION FROM BERT AND MULTIPLE EMBEDDING METHODS WI...
bykevig
 
PDF
Generative Models and ChatGPT
byLoic Merckel
 
PDF
How do OpenAI GPT Models Work - Misconceptions and Tips for Developers
byIvo Andreev
 
PDF
ITB 2023 - Chatgpt Box! AI All The Things - Scott Steinbeck.pdf
byOrtus Solutions, Corp
 
PDF
ChatGPT and OpenAI.pdf
bySonal Tiwari
 
PDF
How to Build an App with ChatGPT.pdf
byStephenAmell4
 
PPTX
A Comprehensive Review of Large Language Models for.pptx
bySaiPragnaKancheti
 
PDF
How to Build an App with ChatGPT.pdf
byMatthewHaws4
 
PDF
acomprehensivereviewoflargelanguagemodelsfor-230515063139-1fc27b64.pdf
byYaserAli40
 
PDF
ChatGPT- E-Learning.pdf
byMaddy Baig
 
PPTX
Creative AI for Improv Theatre
byThomas Winters
 
PPTX
A brief primer on OpenAI's GPT-3
byIshan Jain
 
PPTX
Smart Health Chatbot health care for patients.pptx
bysakthi13vel08
 
PPTX
‘Big models’: the success and pitfalls of Transformer models in natural langu...
byLeiden University
 
PPTX
Top 8 Large Language Models and Their Superpowers
byMark Yarbrough
 
PPTX
WiNLP2020 Keynote "Challenges for Conversational AI: Reflections on Gender Is...
byVerena Rieser
 
PDF
ITB_2023_Chatgpt_Box_Scott_Steinbeck.pdf
byOrtus Solutions, Corp
 
PDF
NLP Meetup 2023
byGabiMaeztu
 
PDF
DataChat_FinalPaper
byUrjit Patel
 
And then there were ... Large Language Models
byLeon Dohmen
 
INTEGRATING EXTRACTED INFORMATION FROM BERT AND MULTIPLE EMBEDDING METHODS WI...
bykevig
 
Generative Models and ChatGPT
byLoic Merckel
 
How do OpenAI GPT Models Work - Misconceptions and Tips for Developers
byIvo Andreev
 
ITB 2023 - Chatgpt Box! AI All The Things - Scott Steinbeck.pdf
byOrtus Solutions, Corp
 
ChatGPT and OpenAI.pdf
bySonal Tiwari
 
How to Build an App with ChatGPT.pdf
byStephenAmell4
 
A Comprehensive Review of Large Language Models for.pptx
bySaiPragnaKancheti
 
How to Build an App with ChatGPT.pdf
byMatthewHaws4
 
acomprehensivereviewoflargelanguagemodelsfor-230515063139-1fc27b64.pdf
byYaserAli40
 
ChatGPT- E-Learning.pdf
byMaddy Baig
 
Creative AI for Improv Theatre
byThomas Winters
 
A brief primer on OpenAI's GPT-3
byIshan Jain
 
Smart Health Chatbot health care for patients.pptx
bysakthi13vel08
 
‘Big models’: the success and pitfalls of Transformer models in natural langu...
byLeiden University
 
Top 8 Large Language Models and Their Superpowers
byMark Yarbrough
 
WiNLP2020 Keynote "Challenges for Conversational AI: Reflections on Gender Is...
byVerena Rieser
 
ITB_2023_Chatgpt_Box_Scott_Steinbeck.pdf
byOrtus Solutions, Corp
 
NLP Meetup 2023
byGabiMaeztu
 
DataChat_FinalPaper
byUrjit Patel
 

More from Thomas Winters

PPTX
Hoe leer je computers humor?
byThomas Winters
 
PPTX
Dutch Humor Detection by Generating Negative Examples
byThomas Winters
 
PDF
Beter leren praten met Artificiële Intelligentie
byThomas Winters
 
PDF
How can AI be a creative partner for PR & marketing?
byThomas Winters
 
PPTX
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
byThomas Winters
 
PDF
Pret met Creatieve Computers
byThomas Winters
 
PPTX
AI als creatieve partner
byThomas Winters
 
PPTX
Humor Workshop: Hoe schrijf je satire? (KU Leugen)
byThomas Winters
 
PPTX
Survival of the Wittiest: Evolving Satire with Language Models
byThomas Winters
 
PPTX
Wetenschapscommunicatie on steroids
byThomas Winters
 
PPTX
Prompt engineering: De kunst van het leren communiceren met AI
byThomas Winters
 
PPTX
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
byThomas Winters
 
PPTX
How to Attract & Survive Media Attention as PhD
byThomas Winters
 
PPTX
TorfsBotOrNot @ Nerdland Festival
byThomas Winters
 
PPTX
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
byThomas Winters
 
PDF
De magie achter afbeeldingsgenerators
byThomas Winters
 
PPTX
Modelling Mutually Interactive Fictional Character Conversational Agents
byThomas Winters
 
PPTX
Generating Philosophical Statements using Interpolated Markov Models and Dyna...
byThomas Winters
 
PPTX
Discovering Textual Structures: Generative Grammar Induction using Template T...
byThomas Winters
 
PDF
Hoe schrijven computers zelf tekst? (Kinderlezing)
byThomas Winters
 
Hoe leer je computers humor?
byThomas Winters
 
Dutch Humor Detection by Generating Negative Examples
byThomas Winters
 
Beter leren praten met Artificiële Intelligentie
byThomas Winters
 
How can AI be a creative partner for PR & marketing?
byThomas Winters
 
Prompt engineering: de kunst van het leren communiceren met AI (Juni 2023)
byThomas Winters
 
Pret met Creatieve Computers
byThomas Winters
 
AI als creatieve partner
byThomas Winters
 
Humor Workshop: Hoe schrijf je satire? (KU Leugen)
byThomas Winters
 
Survival of the Wittiest: Evolving Satire with Language Models
byThomas Winters
 
Wetenschapscommunicatie on steroids
byThomas Winters
 
Prompt engineering: De kunst van het leren communiceren met AI
byThomas Winters
 
TorfsBot or Not? Evaluating User Perception on Imitative Text Generation (CLI...
byThomas Winters
 
How to Attract & Survive Media Attention as PhD
byThomas Winters
 
TorfsBotOrNot @ Nerdland Festival
byThomas Winters
 
Hoe werken tekstgenerators? (Special Guest in Lieven Scheire's AI voorstelling)
byThomas Winters
 
De magie achter afbeeldingsgenerators
byThomas Winters
 
Modelling Mutually Interactive Fictional Character Conversational Agents
byThomas Winters
 
Generating Philosophical Statements using Interpolated Markov Models and Dyna...
byThomas Winters
 
Discovering Textual Structures: Generative Grammar Induction using Template T...
byThomas Winters
 
Hoe schrijven computers zelf tekst? (Kinderlezing)
byThomas Winters
 

Recently uploaded

PDF
Monique_Herbert_Biography_FINAL Compressed.pdf
byniquaaaa5
 
PDF
My Jirapat Profile and Archievement update 2026
byJirapat Chutithadakul
 
PDF
How Google Search Works: From Crawling to Search Results
byramyasreethota939
 
PDF
Helen Bevan graphics posted 2025 (2).pdf
byHelen Bevan
 
DOCX
High-Quality Buy TikTok Accounts With Verified Status.docx
byBuy Old Gmail Accounts
 
DOCX
Buy Facebook Meta Ads Manager in the New Year.docx
byUsaallhub is a Marketer Self-Employed
 
DOCX
Buy LinkedIn Accounts Online in the New Year.docx
byBusiness & Marketing form usaallhub
 
PPT
Toward a Political Economy of Digital Culture
byJeffrey Hart
 
PDF
Insurance in Transition: A Social Media Cheat Sheet for Insurers in Kenya
byMoses Kemibaro
 
DOCX
Level Up Your Campaigns_ Buy Verified Old Facebook Accounts Profiles.docx
byBuy Old Facebook Accounts
 
PDF
The Role of the International Organization of Francophonie on Cultural Divers...
byAJHSSR Journal
 
PDF
Instagram 2026: All Features Unlocked Explore Now
byTurrboo
 
PDF
Climate Change Communication in Ghana: A Systematic Review of a Decade of Res...
byAJHSSR Journal
 
Monique_Herbert_Biography_FINAL Compressed.pdf
byniquaaaa5
 
My Jirapat Profile and Archievement update 2026
byJirapat Chutithadakul
 
How Google Search Works: From Crawling to Search Results
byramyasreethota939
 
Helen Bevan graphics posted 2025 (2).pdf
byHelen Bevan
 
High-Quality Buy TikTok Accounts With Verified Status.docx
byBuy Old Gmail Accounts
 
Buy Facebook Meta Ads Manager in the New Year.docx
byUsaallhub is a Marketer Self-Employed
 
Buy LinkedIn Accounts Online in the New Year.docx
byBusiness & Marketing form usaallhub
 
Toward a Political Economy of Digital Culture
byJeffrey Hart
 
Insurance in Transition: A Social Media Cheat Sheet for Insurers in Kenya
byMoses Kemibaro
 
Level Up Your Campaigns_ Buy Verified Old Facebook Accounts Profiles.docx
byBuy Old Facebook Accounts
 
The Role of the International Organization of Francophonie on Cultural Divers...
byAJHSSR Journal
 
Instagram 2026: All Features Unlocked Explore Now
byTurrboo
 
Climate Change Communication in Ghana: A Systematic Review of a Decade of Res...
byAJHSSR Journal
 

Neuro-Symbolic Creative Artificial Intelligence for Humor (PhD Defense)

  • 1.
    1 Neuro-Symbolic Creative Artificial Intelligencefor Humor Thomas Winters Public PhD defense Slides: thomaswinters.be/defense
  • 2.
    2 One you brushand rake, What is the difference between leaves and a car? the other you rush and brake.
  • 3.
    3 I asked my computerto tell me a Python joke, but it just hissed and gave me a byte.
  • 4.
  • 5.
  • 6.
    6 Purpose of Humor= Displaying intelligence? huh? aha! that’s funny Laugh: displaying “play” Humor: Brain rewards noticing incongruities & successfully resolving + linguistic skills + hard to fake personal values display = Evolutionary advantage! h
  • 7.
    7 Incongruity-Resolution Theory Based on:Ritchie, G. (1999). Developing the incongruity-resolution theory. Obvious Interpretation Hidden Interpretation Two fish are in a tank. Says one to the other: Setup Punchline “Do you know how to drive this thing?”
  • 8.
    8 Human-focused definition! Machine shouldnot only spot two mental images Obvious Interpretation Hidden Interpretation But also that it is not too hard or too easy for a human!
  • 9.
  • 10.
    10 Symbolic generator: JAPE What’s<CharacteristicNP> and <Characteristic1> ? A <Word1> <Word2>. Noun Phrase Word1 Word2 Homophone1 Characteristic1 CharacteristicNP What’s green and bounces? A spring cabbage. spring (season) to bounce spring (elastic body) cabbage green spring cabbage Binsted, K., & Ritchie, G. (1994). An implemented model of punning riddles
  • 11.
    11 MopjesBot Het is een<Beschrijving1> en <Beschrijving2> ? <EersteDeel> <Rijmwoord>. Naam EersteDeel LaatsteDeel Beschrijving2 Beschrijving1 Rijmwoord Het is een Belgische politica en komt tot net boven de enkel? Maggie De Sok. Maggie De Belgische politica Block Komt tot net boven de enkel Maggie De Block Sok Winters, T. (2019). Generating Dutch Punning Riddles about Current Affairs.
  • 12.
    12 How do autoregressivelanguage models work? 1. Open smartphone keyboard 2. Press any autocomplete word 20x 3. You’ve just generated a sentence using an AI trained to sound similar to you! Autocomplete counted how often you used certain words after other words I have been trying to get hold of my client since the last few days
  • 13.
    13 Professor Canon Law& Ex-rector KU Leuven
  • 14.
    14 1. Counted intweets how often Rik Torfs uses one word after others 2. Start with two real starting words of Rik, and take random next words “gevolgd door” 4: een 2: zijn 1: iemand 1: acht Beste,
  • 15.
  • 16.
    16 ≈ autocomplete on steroids Autoregressivemodels like GPT Probability of next token given previous tokens
  • 17.
    17 GPT training Whole textualinternet Training process of several million $ Best public language model
  • 18.
    18 Simple prompt Same 25variations of jokes 90% of the time Jentzsch, S., & Kersting, K. (2023). ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models
  • 19.
  • 20.
  • 21.
  • 22.
    22 Research question How touse & integrate for creative AI & Dutch humor? Neural AI Symbolic AI Neuro- Symbolic AI
  • 23.
    23 Publications Journal: 2 total,1 first author • Computers learning humor is no joke T. Winters (HDSR 2021) • RobBERTje: a distilled Dutch BERT model P. Delobelle, T. Winters, B. Berendt (CLIN journal 2021) Conferences: 15 total, 9 first author • Automatic joke generation: Learning humor from examples T. Winters, V. Nys, D. De Schreye (HCII 2018) • Towards a general framework for humor generation from rated examples T. Winters, V. Nys, D. De Schreye (ICCC 2019) • Generating philosophical statements using interpolated markov models and dynamic templates T. Winters (ESSLLI 2019) • Modelling mutually interactive fictional character conversational agents T. Winters (BNAIC 2019) • Discovering textual structures: Generative grammar induction using template trees T. Winters, L. De Raedt (ICCC 2020) • Dutch humor detection by generating negative examples T. Winters, P. Delobelle (Benelearn 2020) • Automatically generating engaging presentation slide decks T. Winters, K. W. Mathewson (EvoMusArt 2019) • Survival of the Wittiest: Evolving Satire with Language Models T. Winters, P. Delobelle (ICCC 2021) • DeepStochLog: Neural Stochastic Logic Programming T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (AAAI 2022) • RobBERT: a Dutch RoBERTa-based Language Model P. Delobelle, T. Winters, B. Berendt (EMNLP 2020 findings) • Playable experiences at the 15th AAAI conference on AIIDE R. Liu, C. Christopher, C. Martens, S. Ontañón, P. Mirowski, K. W. Mathewson, T. Winters, S. Farrugia (AIIDE 2019) • Rosetta code: Improv in any language P. Mirowski, K. Mathewson, B. Branch, T. Winters, B. Verhoeven, J. Elfving (ICCC 2020) • SandSlide: Automatic slideshow normalization S. Bocklandt, G. Verbruggen, T. Winters (ICDAR 2021) • Shape inference and grammar induction for example-based procedural generation G. Hermans, T. Winters, L. De Raedt (ICCC 2021) • Learning to Rank Generated Portmanteau L. Pollet, T. Winters, P. Delobelle (ICCC 2021) Preprints: 1 total • RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use P. Delobelle, T. Winters, B. Berendt Abstracts: 6 total, 4 first author • Generating Dutch punning riddles about current affairs T. Winters (CLIN 2019) • TorfsBot Or Not? Evaluating User Perception on Imitative Text Generation T. Winters (CLIN 2023) • DeepStochLog: Neural stochastic logic programming (Extended Abstract) T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (ILP 2022) • DeepStochLog: Neural stochastic logic programming (Extended Abstract) T. Winters*, G. Marra*, R. Manhaeve, and L. De Raedt (NeSy workshop 2022 • RobBERT: a Dutch RoBERTa-based Language Model (Extended Abstract) P. Delobelle, T. Winters, and B. Berendt (BeneLearn 2021) • RobBERTje: A Distilled Dutch BERT Model P. Delobelle, T. Winters, and B. Berendt (CLIN 2021). Total: 24 | First author: 14 Citations: 300+ | h-index: 7 | i10-index: 6
  • 24.
    24 Contribution 2: Dutch Language Modelsfor Computational Humor Contribution 3: Neural Definite Clause Grammars Main contributions Contribution 1: Symbolic Creative AI Extensions & Applications
  • 25.
    25 Contribution 1: Symbolic CreativeAI Extensions & Applications How can symbolic text generation techniques be tailored and learned for imitative and humorous applications?
  • 26.
    26 Dynamic Template Base text Arethere also atheists who do not believe in atheism? They see the fact that the former God (…). Norse popes. Context text Generated Text Are there also popes who do not believe in God? VBP 900 EX 200 RB 100 NNS 5 WP 500 VBP 400 VB 20 IN 300 NN 3 PRP 900 VBP 100 DT 100 NN 50 IN 500 DT 100 JJ 50 NN 20 JJ 5 NNS 10 Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019) RB 800
  • 27.
    28 1. Count inRik Torfs tweets & columns how often a word follows the previous 2-4 (late normalized interpolated) 2. Start with two real starting words of Rik, and take random next words “gevolgd door” 4: een 2: zijn 1: iemand 1: acht Beste, Interpolated Markov Chain
  • 28.
    29 Late Normalized InterpolatedMarkov Chains 5-grams good coherence but high plagiarism 3-grams more random but low coherence LNIMC: Multiply frequencies of n-grams with weight, then sum and normalize  Keeps text on track for rare sequence, and more random for frequent 3-grams Winters, T. (2019). Generating philosophical statements using interpolated markov models and dynamic templates. (ESSLLI 2019)
  • 29.
    30 Evaluation: Local cohesion> Global form Markov chain 35% more interactions than dynamic template (17K interactions on 8K tweets over from June 2017 to June 2023) Winters, T. (2019).Generating Philosophical Statements using Interpolated Markov Models and Dynamic Templates.
  • 30.
    31 Tweets daily pollwith random tweet from either Rik Torfs or TorfsBot
  • 31.
    32 47K votes over602 polls (avg 79/poll) 71% correct votes 87% correct majority 68 TorfsBot success 12 Rik Torfs fails
  • 32.
    33 Slightly positively correlated (0.11 forlog(interactions) with %votes) Little difference between algorithms
  • 33.
    34 Templates + Grammar+ Functions = Babbly Programming language for text generation with grammars import firstname.words food = pasta|pizza|sushi main = { 3: <firstname> loves <food.uppercase>! 1: <firstname> (quite|reasonably|fairly) likes <food>. Oo{1,3}h, I hope they join! 1: <firstname:protagonist> is not (quite){.5} fond of <food:>. <firstname:protagonist> will thus not go to the <food:> (restaurant|place). } Generates sentences like: • Bob loves PIZZA! • Thomas is not quite fond of sushi. Thomas will thus not go to a sushi place. Winters, T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
  • 34.
    35 Mutually interactive bots Winters,T. (2019). Modelling Mutually Interactive Fictional Character Conversational Agents. (BNAIC 2019)
  • 35.
    36 Talk Generator Generates humorousPowerPoints about any given topic for presenters to improvise on Winters T., Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Available on talkgenerator.com
  • 36.
    37 Slide Generator Winters T.,Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Internally uses lots of handwritten grammars, combined with search engines and knowledge graphs
  • 37.
    38 Presentation schema Winters T.,Mathewson K. (2019). Automatically Generating Engaging Presentation Slide Decks. (EvoMusArt 2019) Available on talkgenerator.com
  • 38.
  • 39.
    40 GITTA: Template Treesfor extracting templates 1. Join closest strings 2. Merge similar template slots 3. Iteratively simplify until convergence A: I like my <B> and my <B> | <G> the <B> is <F> B: chicken | cat | dog F: walking | jumping G: Alice | Bob | Cathy Winters, T. & De Raedt, L. (2020). Discovering Textual Structures: Generative Grammar Induction usng Template Trees. (ICCC 2020) Input
  • 40.
  • 41.
    42 Contribution 2: Dutch Language Modelsfor Computational Humor How to create powerful general Dutch language models, and how to apply to humor?
  • 42.
    43 RobBERT: Dutch RobBERTamodel Phase 1: Pretraining • Pre-trained using unlabeled web-scraped Dutch corpus for reusable base model: RobBERT model • Distilled into smaller RobBERTje model • Extended & further trained: RobBERT-2022 Phase 2: Finetuning • Smaller labeled training data for finetuning task • Outperforms other models on most Dutch NLP tasks  1M+ downloads as of August 2023 Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
  • 43.
    44 RobBERT results • Dutchtokenizer (v2) > English tokenizer (v1) & competitors • Merging > other distillation for longer texts (SA) • Further pre-training on recent data both improves & worsens performance Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT: a Dutch RoBERTa-based language model. (EMNLP Findings 2020) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERTje: A Distilled Dutch BERT model. (CLIN journal 2021) Delobelle, P., Winters, T., & Berendt, B. (2020). RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use (Preprint)
  • 44.
    45 Dutch Humor Detection NoDutch joke dataset  Scrape Dutch jokes But what “non-jokes”? Previously: Proverbs & News  But is this truly detecting humor? Mihalcea, R., & Strapparava, C. (2005). Making computers laugh: Investigations in automatic humor recognition.
  • 45.
    46 Jokes are fragile! Twofish are in a tank. Says one to the other: “Do you know how to drive this thing?” men bar Dynamic Template algorithm: Generate non-jokes by replacing keywords from other jokes! Word-based features won’t work anymore! Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
  • 46.
    47 Examples of generatedDutch non-jokes Het is groen en het is een mummie? Kermit de Waterkant Wat is het toppunt van principe? 1) Wachten totdat een Nederlander gaat twijfelen 2) Een Zuster met een autoladder 3) Een brandwacht brandmeester met een brandmeester van 9 maanden “Ober, kunt u die schrik uit mijn politieman halen? Want ik eet liever alleen.” "Mijn hond is heel vreselijk: Hij schreeuwt mij iedere zus de broer.“ "Maar dat is toch niet zo heel vreselijk?“ "Jawel, want ik heb geen rapport!" Wat staat er midden in het bos? De kapper. Er loopt een super vriendelijk blondje langs een armband. Last er een toonbank: “zo, waargaan die mooie mannen heen?” Blondje: “naar de barkeeper als er niets tussen komt…” Hoe heet de vrouw van Sinterklaas? Keukentafel. "Twee tanden zwemmen in de zee en ze zien een stamgast op een stamgast. De ene raad zegt tegen de andere raad: 'Hé kijk! Ons eten op een bord!'"
  • 47.
    48 51% 60% 50% 94% 94% 47% 94% 94% 47% 99%96% 89% Jokes vs News Jokes vs Proverbs Jokes vs Generated Jokes Binary classification of jokes versus texts from other domains Naive Bayes LSTM CNN RobBERT Much more challenging dataset! More truthful humor detection? Winters T., Delobelle P. (2020). Dutch humor detection by generating negative examples. (BNAIC/Benelearn 2020)
  • 48.
    49 GALMET: Generating satirewith two RoBERTa heads Satire MLM Masked Language Model for Headlines & Satire Randomly substitutes & adds tokens Dog <mask> bites man  Dog accidentally bites man Satire Regression Predicts funniness of headline Fitness function "Most Americans Want Congress To Investigate Michael Flynn“ → Predicted score: 0.01 News: The Perils Of Climate Change: A Rise In Sea Level Of Just One Foot Would Be Devastating For The Man Tied To This Pier → Predicted score: 1.05 Hey, I’ll just alter the headline one word at a time! Alright! I’ll check if that edit actually makes it funnier! Winters T., Delobelle P. (2021). Survival of the Wittiest: Evolving Satire with Language Models. (ICCC2021)
  • 49.
  • 50.
    51 Contribution 3: Neural Definite ClauseGrammars How to integrate neural networks with symbolic AI to perform tasks that require both symbolic reasoning and subsymbolic processing?
  • 51.
    52 CFG: Context-Free Grammar E--> N E --> E, P, N P --> [“+”] N --> [“0”] N --> [“1”] … N --> [“9”] 2 + 3 + 8 N E E P N E P N Is sequence in language? Which part-of-speech? Generate all language elements
  • 52.
    53 PCFG: Probabilistic Context-FreeGrammar 0.5 :: E --> N 0.5 :: E --> E, P, N 1 :: P --> [“+”] 0.1 :: N --> [“0”] 0.1 :: N --> [“1”] … 0.1 :: N --> [“9”] 2 + 3 + 8 N E E P N E P N What is the probability of generating sequence? Which parse is more likely? 0.5 0.5 1 1 0.5 0.1 0.1 0.1
  • 53.
    54 DCG: Definite ClauseGrammar e(N) --> n(N). e(N) --> e(N1), p, n(N2), {N is N1 + N2}. p --> [“+”]. n(0) --> [“0”]. n(1) --> [“1”]. … n(9) --> [“9”]. 2 + 3 + 8 n(2) e(2) e(5) p n(3) e(13) p n(8) Constrain language  Context-sensitive Inputs & outputs through unification
  • 54.
    55 SDCG: Stochastic DefiniteClause Grammar 0.5 :: e(N) --> n(N). 0.5 :: e(N) --> e(N1), p, n(N2), {N is N1 + N2}. 1.0 :: p --> [“+”]. 0.1 :: n(0) --> [“0”]. 0.1 :: n(1) --> [“1”]. … 0.1 :: n(9) --> [“9”]. 2 + 3 + 8 n(2) e(2) e(5) p n(3) e(13) p n(8) 0.5 0.1 1 1 0.1 0.1 0.5 0.5 Probability of this parse = 0.5*0.5*0.5*0.1*1*0.1*1*0.1 = 0.000125 Always sums to 1 per non-terminal Disambiguating possible parses
  • 55.
    56 DeepStochLog 0.5 :: e(N)--> n(N). 0.5 :: e(N) --> e(N1), p, n(N2), {N is N1 + N2}. 1.0 :: p --> [“+”]. nn(number_nn,[X],[Y],[digit]) :: n(Y) --> [X]. digit(Y) :- member(Y,[0,1,2,3,4,5,6,7,8,9]). Probability of this parse = 0.5*0.5*0.5*pnumber_nn( =2)*1*pnumber_nn( =3)*1*pnumber_nn( =8) + + Symbolic + subsymbolic sequence processing Learning rule probabilities with neural networks Matches SOTA results on all tested NeSy tasks, but improved scaling! n(2) e(2) e(5) p n(3) e(13) p n(8) 0.5 1 1 0.5 0.5 pnumber_nn( =2) pnumber_nn( =3) pnumber_nn( =8) T. Winters*, G. Marra*, R. Manhaeve, L. De Raedt (2019). DeepStochLog: Neural Stochastic Logic Programming. (AAAI2022)
  • 56.
    57 Conclusion Contribution 1: SymbolicCreative AI Extensions & Applications • Extended text generators for imitative text generation • Created symbolic AI humor generators (bots & talkgenerator) • Regular generative grammar induction method Contribution 2: Dutch Language Models for Computational Humor • SOTA Dutch RobBERT language models • Dutch humor detector & improved dataset creation • RoBERTa + genetic algorithm for humor generation Contribution 3: Neural Definite Clause Grammars • DeepStochLog as scalable neuro-symbolic framework
  • 57.
    58 Neuro-Symbolic Creative Artificial Intelligencefor Humor Thomas Winters Public PhD defense Slides: thomaswinters.be/defense

Editor's Notes

  • #3 JAPE joke from 30 years ago
  • #6 Humor intrinsically human - All civilizations perform humor - humans only animal that make jokes
  • #7 Haha and Aha are very similar
  • #12 Het is een Belgisch politica en wordt om de taille gedragen? Maggie De Rok! Het is een Belgisch politica en is een deel van de Aziatische keuken? Maggie De Wok! Het is een Belgisch politica en vertelt een relatief onschuldige leugen? Maggie De Jok! Het is een Belgisch politica en bereidt voedsel tot een maaltijd? Maggie De Kok!
  • #23 The research question of this thesis relates to the main paradigms in AI. First, most AI was symbolic AI. These are controllable and interpretable methods for reasoning over a space to find solutions. Nowadays, neural networks are the dominant AI paradigm, with their incredible powerful methods for learning from complex, subsymbolic data such as images and text meaning. However, they are often seen as black boxes as they are hard to interpret. There is a new trend called Neuro-symbolic AI that aims to integrate the two approaches. In this thesis, we demonstrate how we can use these paradigms for creative AI and Dutch humor, and how to integrate the two.
  • #25 Grouped several related publications into the following three main contributions