A version of my OpenVisConf talk "Bones of a Bestseller" that gives more detail on topic analysis plus adds python code. Blog post and ipynb code here: http://blogger.ghostweather.com/2013/08/pydata-boston-2013-more-on-fiction.html
THEVIDEO OF THAT TALK:
BASED ON A PREVIOUS
This talk focuses on some more technical details and more on topic analysis.
The IPython notebook of code samples for this lives here:
Text Classification (Commonly)
§ “Bag of words” – each document is considered
a collection of words, independent of order
§ Frequencies of certain words are used to
identify the texts
Seems like this should work with sex scenes,
right? Only so many body parts and behaviors,
Estdsgfd fdsatreatret dfds
Dsrdsf drerear ewrewtrew
Reret retdrtd rewrewrtew
New data in the wild
Sex Scene Detection First Steps
1. Buy 50 Shades on Amazon, unlock text in
Calibre, save as TXT ﬁle.
2. Cut up a doc into 500 “word” chunks using
“Would you like to sit?” He waves me toward an L-shaped white leather couch.
His ofﬁce is way too big for just one man. In front of the ﬂoor-to-ceiling windows, there’s a
modern dark wood desk that six people could comfortably eat around. It matches the
coffee table by the couch. Everything else is white—ceiling, ﬂoors, and walls, except for the
wall by the door, where a mosaic of small paintings hang, thirty-six of them arranged in a
square.They are exquisite—a series of mundane, forgotten objects painted in such precise
detail they look like photographs. Displayed together, they are breathtaking.
“A local artist.Trouton,” says Grey when he catches my gaze.
“They’re lovely. Raising the ordinary to extraordinary,” I murmur, distracted both by him
and the paintings. He cocks his head to one side and regards me intently.
“I couldn’t agree more, Miss Steele,” he replies, his voice soft, and for some inexplicable
reason I ﬁnd myself blushing.
Sample of 50 Shades of Grey
On to the learning algorithm…
So, the training data:
- The text chunks
- The score the raters gave it (averaged) as “truth”
I started with Python’s NLTK (Natural Language
Toolkit) and Naïve Bayes for classifying (working
in an ipython notebook).
Resources on NLTK Naïve Bayes
§ The NLTK book chapter:
§ Jacob Perkins’ example of sentiment analysis
Perkins’ NLTK code for this…
from nltk.classify import NaiveBayesClassifier
from nltk.corpus import movie_reviews
return dict([(word, True) for word in words])
negids = movie_reviews.fileids('neg')
posids = movie_reviews.fileids('pos')
negfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'neg') for f in negids]
posfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'pos') for f in posids]
negcutoff = len(negfeats)*3/4
poscutoff = len(posfeats)*3/4
trainfeats = negfeats[:negcutoff] + posfeats[:poscutoff]
testfeats = negfeats[negcutoff:] + posfeats[poscutoff:]
print 'train on %d instances, test on %d instances' % (len(trainfeats),
classifier = NaiveBayesClassifier.train(trainfeats)
print 'accuracy:', nltk.classify.util.accuracy(classifier, testfeats)
His movie sentiment output
72% accuracy, trained on 1500 inputs.
My results on 50 Shades sex
82 % accuracy!
Previously with less “pos” data: not so
great at 68%
“packet” (they use a lot of condoms)
Python’s sklearn (scikit-learn)
Lots of classiﬁers
for sparse data like
Using a lemmatizer step in the pipeline (to strip endings off words, since some ﬁction in my
later samples was in present tense)
Pipelines in sklearn makes it incredibly easy to run lots of experiments.
Fit the model, using training data and “target” answers (in this case,“50 Shades of Grey”)
Test the model on new data (in this case,“50 Shades Darker”). Check how it did against the
Interpreting the results…
Let’s make a tool!
Really amazing P.S. here…
I paid for coding of a bunch of fan-ﬁction for sex
scenes too, and fed them in to the sklearn SGD
(Note that 50 Shades started life as Twilight
*cross-validating with entire set, not just 50 Shades books.
97% accuracy achieved!*
Almost naked, Silas hurled his pale body down the staircase. He knew he
had been betrayed, but by whom? When he reached the foyer, more
officers were surging through the front door. Silas turned the other way
and dashed deeper into the residence hall.The women's entrance. Every
Opus Dei building has one.Winding down narrow hallways, Silas snaked
through a kitchen, past terrified workers, who left to avoid the naked
albino as he knocked over bowls and silverware, bursting into a dark
hallway near the boiler room. He now saw the door he sought, an exit light
gleaming at the end.
Running full speed through the door out into the rain, Silas leapt off the
low landing, not seeing the officer coming the other way until it was too
late.The two men collided, Silas's broad, naked shoulder grinding into the
man's sternum with crushing force. He drove the officer backward onto the
pavement, landing hard on top of him.The officer's gun clattered away.
Silas could hear men running down the hall shouting. Rolling, he grabbed
the loose gun just as the officers emerged. A shot rang out on the stairs,
and Silas felt a searing pain below his ribs. Filled with rage, he opened
fire at all three officers, their blood spraying.
A dark shadow loomed behind, coming out of nowhere.The angry
hands that grabbed at his bare shoulders felt as if they were infused with
the power of the devil himself.The man roared in his ear. SILAS, NO!
Silas spun and fired.Their eyes met. Silas was already screaming in
Resources for Topic Analysis
§ David Mimno’s java Mallet is “the one everyone
- The R mallet package is rather nice, too:
- This is a GUI wrapper for mallet that outputs nice csv
and html pages:
§ Some pure python (and C) implementations (toy
code, primarily) are listed on Blei’s website:
Pros/Cons vs CMD-Line Mallet
§ Allows stopword ﬁle
§ Produces csv and html
output in a near dir
§ Has a GUI (simpler to just
§ Runs with defaults, so no
optimize-interval or other
cmd line options
§ No diagnostic output (a
§ Not super-well doc’d
Tutorial on cmd line usage:
Maybe I need One More Tool. Any word relations of interest?
Let’s try another hairball…
Filtered to only the
DaVinci Code topics to
“Excitement” rating color scale
avg by chapter, ordered
Topics (48ish) per
Chapter 1… to Chapter 108
Ah, but since it’s svg/d3…
var chart = chart.append("g").attr("translate","0," +
y).attr("transform","rotate(90 600 600)");
But, maybe I need chapter
summaries…. So I can relate
them to the topics?
Add some topic-tooltips
But what did this
Some topics are just neither exciting nor
dull – topic clustering (as I did it) had little
to do with action scenes. It’s slightly helpful
for topics, though J
These nodes are shaded from
gray (dull) to red (exciting)
Color words in texts by topic assigment, to help
tune the stopwords and set up next steps:
• Pre-process text for just the verbs?
• Clean out a class of proper names
• Extract sentences containing the topic words
to help describe the topics/texts better
§ Python is great for the data munging and
§ Some analysis needs serious vis support
Python before you get into js
§ D3 is a great tool for iterative interactive
exploration of your analysis results
My thanks to….
Luminosity for help with Dan Brown summaries, JimVallandingham (@vlandham)
for network parameter and coffeescript help.
Hey, I am a consultant for data analysis and visualization. Look me up!
A Few More References
§ Applied Machine Learning with Scikit-Learn:
§ Naïve Bayes for text in Scikit-Learn:
§ Stochastic Gradient Descent in Scikit-Learn: http://scikit-learn.org/0.13/modules/sgd.html
§ Nice tutorial overview of working with text data:
§ Bearcart by Rob Story – Rickshaw timeseries graphs from python pandas datastructure in 4
§ LDA topic modeling tool with UI - https://code.google.com/p/topic-modeling-tool/
§ Scott Weingart’s nice overview of LDA Topic Modeling in Digital Humanities:
§ Elijah Meeks’ lovely set of articles on LDA & Digital Humanties vis:
§ JimVallandingham’s tooltip code and a great demo/tutorial:
§ Rickshaw for timeseries graphs: https://github.com/shutterstock/rickshaw