1. AUTOMATIC JOKE GENERATION:
LEARNING HUMOUR FROM EXAMPLES
ir. Thomas Winters, prof. dr. Danny De Schreye, ir. Vincent Nys
KU Leuven
winters.th@gmail.com
@thomas_wint

2. CONTRIBUTIONS
2

3. BACKGROUND
3

4. JAPE: TEMPLATES & SCHEMAS
4
What’s <CharacteristicNP>
and <Characteristic1> ?
A <Word1> <Word2>.
Noun Phrase
Word1 Word2
Homophone1
Characteristic1 CharacteristicNP
What’s green and bounces?
A spring cabbage.

5. PETROVIĆ & MATTHEWS ANALOGY GENERATOR
5
X Y
Z
I like my <X> like I
like my <Y>: <Z>
I like my coffee like I
like my war: cold
Maximise adjective dissimilarity
Maximise co-occurrence Maximise co-occurrence
Maximise # definitions
Minimize commonness

6. STATE OF THE ART PROBLEMS
Individual
Dimension
6
No learning from
previous
generations
Temporal
Dimension
Hardcoded
Templates &
Schemas

7. INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

8. Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

9. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

10. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Obvious
interpretation
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

11. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

12. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

13. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
Obviousness
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

14. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
Conflict
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

15. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
Compatibility
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

16. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
Comparison
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

17. Punchline
Setup
INCONGRUITY RESOLUTION THEORY
Two fish are in a tank. One
turns to the other and says:
“Do you know how to drive
one of these things?”
7
Hidden
interpretation
Obvious
interpretation
Inappropriateness
(Based on: Ritchie, Graeme. Developing the incongruity-resolution theory. 1999.)

18. GENERALISED
ANALOGY
GENERATOR
18

19. HUMOUR METRICS IDENTIFICATION
19
Obviousness
Conflict
Compatibility
Comparison
Inappropriateness
Word Frequency (1-grams)
Word Combination Frequency (N-grams)
Number of meanings (WordNet)
Adjective Vector Difference (N-grams)
Word Sexiness (Corpus 1-gram & 2-gram)

20. SCHEMA GENERALISATION
20
Metric aggregator
Constraint-
based
Schema
Probabilistic
Schema
Values Generator
Metric
functions
Humour
Classification as aggregator
-> Learning component!

21. GAG PIPELINE
11
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

22. GAG PIPELINE
11
“XlikeY
,Z
”jokes
I like my women like I like my flowers: plastic
I like my enemies like I like my amputees: unarmed
I like my men like I like my tea: hot and British
I like my logic like I like my logic: circular and like my logic
I like my men like I like my coffee: I fucking hate coffee
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

23. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes
I like my women like I like my flowers: plastic: [2,3,3,3,4,4,4,4,4]
I like my enemies like I like my amputees: unarmed: [1,2,2,2,2,3,4,4,4,4,4,5,5,5,5]
I like my men like I like my tea: hot and British: [1,2,2,2,3,3,3,3,4,4,4,4,4,4,5,5,5,5]
I like my logic like I like my logic: circular and like my logic: [1,1,2,3,3,4,4,4,4,4,4,5,5,5,5,5,5,5,5]
I like my men like I like my coffee: I fucking hate coffee: [1,1,1,2,2,2,2,3,3,3,3,3,3,4,4,5,5,5]
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

24. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
women, flowers, plastic : [2,3,3,3,4,4,4,4,4]
enemies, amputees, unarmed: [1,2,2,2,2,3,4,4,4,4,4,5,5,5,5]
men, tea, hot: [1,2,2,2,3,3,3,3,4,4,4,4,4,4,5,5,5,5]
men, tea, British: [1,2,2,2,3,3,3,3,4,4,4,4,4,4,5,5,5,5]
logic, logic, circular: [1,1,2,3,3,4,4,4,4,4,4,5,5,5,5,5,5,5,5]
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

25. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
X Y Z Scores FreqX FreqY FreqZ FreqXY …
women flowers plastic [2,3,…,4] 9 929
965
4 050
904
488 4757 0 …
enemies amputee
s
unarmed [1,2,…,5] 784 065 9 211
826
2 010 173 0 …
… … … … … … … … ….
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

26. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
(T
R
A
IN
IN
GD
A
T
A
)
Aggregate Scores:
Average or Mode
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

27. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
(T
R
A
IN
IN
GD
A
T
A
)
S
E
E
D
W
O
R
D
coffee
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

28. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
(T
R
A
IN
IN
GD
A
T
A
)
G
enerated
tem
platevalues
S
E
E
D
W
O
R
D
coffee, women, final
coffee, women, black
coffee, women, popular
coffee, women, rich
coffee, women, Irish
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

29. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
(T
R
A
IN
IN
GD
A
T
A
)
G
enerated
tem
platevalues
T
em
platevalues
withfeaturevalues
S
E
E
D
W
O
R
D
X Y Z Score FreqX FreqY FreqZ FreqXY …
coffee women final ? 784 065 9 929
965
3 070
183
0 …
coffee women black ? 784 065 9 929
965
5 707
260
0 …
coffee women popular ? 784 065 9 929
965
2 311
546
0 ….
… … … ? … … … … …
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

30. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
(T
R
A
IN
IN
GD
A
T
A
)
G
enerated
tem
platevalues
T
em
platevalues
withfeaturevalues
F
iltereds
etof
tem
platevalues
S
E
E
D
W
O
R
D
coffee, women, final | 3
coffee, women, black | 5
coffee, women, popular | 2
coffee, women, rich | 2
coffee, women, Irish | 4
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

31. GAG PIPELINE
11
“XlikeY
,Z
”jokes R
atedjokes S
etofrated
tem
platevalues
R
atedtem
platevalues
withfeaturevalues
(T
R
A
IN
IN
GD
A
T
A
)
G
enerated
tem
platevalues
T
em
platevalues
withfeaturevalues
F
iltereds
etof
tem
platevalues
(A
P
PL
YT
E
MPL
A
T
E
) “XL
IK
EY
,Z
”
J
O
K
EA
B
O
U
T
S
E
E
DW
O
R
D
S
E
E
D
W
O
R
D
I like my coffee like I like my women: black
I like my coffee like I like my women, Irish
J
O
K
E
J
U
D
G
E
R
W
E
B
S
IT
E
T
E
M
P
L
A
T
E
P
R
O
C
E
S
S
O
R
C
L
A
S
S
IF
IE
R
/R
E
G
R
E
S
S
IO
N
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
2-G
R
A
M-B
A
S
E
DG
E
N
E
R
A
T
O
R

32. JOKEJUDGER
32

33. JOKEJUDGER: COLLECTED DATA
33
203 users 9034 ratings
418 markings
424 human jokes
100 generated
jokes
41,9% average
agreement on most
frequent rating

34. CLASSIFICATION & REGRESSION
5 star jokes
= Small number & High variety
-> Hard to predict
34
Classification
most frequent rating
Regression
Average rating
Humour appreciation varies a lot
-> Average score close to centre

35. RANDOM FOREST
Random Trees: subsets of features & data
-> Reduces impact of noisy features
35
frequency_0 frequency_1 frequency_2 word_senses_0 word_senses_1 word_senses_2
Classification
61.5% correctly classified
Regression
RRSE: 84%
…

36. EXAMPLES OF GENERATED JOKES
I like my coffee like I like my myself: hot.
I like my coffee like I like my sleep: strong.
I like my men like I like my graves: nameless.
I like my men like I like my society: heartless.
I like my women like I like my buildings: tall.
I like my men like I like my fire: growing.
I like my women like I like my tests: independent.
I like my women like I like my bills: remaining.
I like my men like I like my rain: gentle.
I like my men like I like my turkey: roast.
I like my partners like I like my universe: early.
I like my computer like I like my affection: personal.
36

37. EVALUATION
37
2,01%
6,44%
35,70%
31,54%
12,89%
7,25%
4,16%
1,39%
7,91%
40,36%
28,99%
11,23%
6,38%
3,74%
1,12%
3,11%
20,86%
25,49%
22,04%
17,85%
9,53%
0,99%
3,39%
23,42%
27,51%
22,08%
15,19%
7,42%
0,73%
3,49%
26,02%
29,65%
19,04%
13,37%
7,70%
OFFENSIVE DON'T GET
IT
1★ 2★ 3★ 4★ 5★
RELATIVE NUMBER OF RATINGS PER RATING TYPE FOR
DIFFERENT SOURCES AND CATEGORIES OF JOKES
DURING THE EVALUATION PHASE
Classifier Regression
Human-created Human-created on JokeJudger
Human-created single-word jokes

38. FUTURE RESEARCH
& EXTENSIONS
38

39. GOOFER: GENERATOR OF ONE-LINERS FROM
EXAMPLES WITH RATINGS
40
(T
R
A
IN
IN
GD
A
T
A
)
H
U
MA
NE
V
A
L
U
A
T
IO
N T
E
MP
L
A
T
EE
X
T
R
A
C
T
O
R C
L
A
S
S
IF
IE
R
ME
T
R
IC
-B
A
S
E
DR
A
T
E
R
V
A
L
U
E
SG
E
N
E
R
A
T
O
R
IN
P
U
T
J
O
K
E
S
S
E
E
D
W
O
R
D
R
atedjokes
(T
E
MP
L
A
T
E
S
)
R
atedtem
platevalues
withfeaturevalues
S
etofratedtem
plate
valuespertem
plate
W
ord+
T
em
plate
G
enerated
tem
platevalues
T
em
platevalues
withfeaturevalues
F
iltereds
etof
tem
platevalues
T
E
MP
L
A
T
E
S
T
O
R
E
(A
P
P
L
YT
E
M
P
L
A
T
E
)
J
O
K
E
SA
B
O
U
T
S
E
E
DININ
P
U
T
J
O
K
E
SS
TY
L
E
Multiple word template values
Template detection & extraction
More metrics: e.g. from CH recognisers
->Preciser estimation of IR-theory
parameters
Larger and diverser joke set

40. CONCLUSION
41

41. PROGRAM CONTRIBUTIONS
42
& more NLP/CH
Tools on
github.com/TWinters

42. CONCLUSION
43
>
11,41%
22,61% 21,08%
GAG: Classifier Human-created on
JokeJudger
Human-created
single-word jokes
4+ RATINGS FREQUENCY

43. Thank you!
44
Automatic Joke Generation:
Learning Humour From Examples
Thomas Winters, Danny De Schreye, Vincent Nys
winters.th@gmail.com
@thomas_wint
Many images of this presentation were based on the works of dooder on freepik.com