Over the spring, I programmed an AI which can write its own poetry! (CS and poetry, the two loves of my life. :) These slides provide more details.

1. What is my project?
Objective: Code a word-level recurrent neural
network which can write!
Training Dataset: Pride and Prejudice, King Midas,
Blithedale Romance, Star of India, Only a Girl’s Love, a
random poem which I wrote
How to Run It: Give it a word, and then watch it
generate 32 more words after that

2. What is a recurrent neural network?
A feed-forward neural network restarts its thinking from scratch every
second. This is not how humans work!
An RNN remembers what happened last, and uses its memory to
make an informed decision. They have loops.

3. How does mine work?
“word”
http://lstm.seas.harvard.edu/
Output
i.e.
“blue”
LSTM

4. Challenges
Repetition! At first when I trained it on large
datasets (like ones where the word “and” appears
19,571 times) it just gave me pages and pages of the
word “and.” I literally have like 7 pages just of the
word “the.”
overfitting

5. How I fixed it
Implementing a dropout layer! Dropout is when you randomly drop
some of the outputs of the neurons.
Why does a dropout layer help with overfitting?
It forces more neurons to learn the characteristics of the network. For
example, suppose we have two neurons. If we overfit, we’re relying
entirely on the prediction of Neuron A. Dropout forces both Neuron A
and Neuron B to learn the function.

6. i skateboard the forest
ravens to her i was me
dream of the tell of forest
from the forest of those we her steel
i steel of nests was i was a me deny her nothing!
yellow and and the and to the climax and that the then
the instance; and and the passion-song and the owe.
singing and and the warnings and the anxiety: and
the fright.so and the battle—there and and the and the
romance,”

7. and the yellow and the wild and the he?
painful of the world.
the formulated and the and the tremble. and the the
music— and the climax and the “peace! the
sweetest and the girl and the tempted
“david and the and the and the sparkled, and and the
me;” and the curse and the and the convulsed and
the and walking and and the heather.”
beautiful and the the sunken. and the and the
you?”“yes,” and the meant.“happy!”