EMNLP 2013 - D13-1100

1. DanielPreoţiuc-Pietro,TrevorCohn
DepartmentofComputerScience
{daniel,t.cohn}@dcs.shef.ac.uk
AtemporalmodeloftextperiodicitiesusingGaussianProcesses
AIMS:givenhashtagtimeseries,useGPsto:findperiodicities,grouptimeseries,
forecastfuturevalues,textclassificationusingperiodicityinfo
-givenamodel,computeprobabilityofthedataintegrating
overtheparameterspacei.e.Bayesian‘evidence’
-conditionedonkernelparameters,theevidencecanbe
computedanalytically
-balancesdatafitandmodelcomplexity(Occam‘sRazor)
-complexmodelswhichcanaccountformanydatasets
achievelowevidence
-useNegativelogMarginalLikelihood(ML-II)formodelse-useNegativelogMarginalLikelihood(ML-II)formodelse-
lection,givinganimplicitclassificationoftimeseries
model dataset kernelparameters
ModelSelection
Likelihoodfor#goodmorningwithPSkernel
#raw
#snow#fail
#fyi
#brb
#coffee
#facebook
#facepalm
#funny
#love
##rock
#running
#xbox
#youtube
#breakfast
#eastenders
#ff
#followfriday
#goodnight
#jobs
#news#news
#tgif
#thegame
#ww
#funny
#lego
#likeaboss
#money
#nbd
#nf
#notetose#notetoself
#priorities
#social
#true
#2011
#backintheday
#confessionhour
#februarywish
#haiti
#makeachange
#questionsidont#questionsidontlike
#savelibraries
#snow
#snowday
-trainon1month,predict1monthinthefuture
-performancecomparedtomeanprediction(=GP-Const)
-GP+performsmodelselection
-Lag+ARmodelthatusestheGPdeterminedperiod
Forecasting
7.29%
3.99% -34.5%
7.37%
0.22%
9.22%
GaussianProcesses(GP)
METHOD
-Bayesiannon-parametricframework
-regardedasstate-of-the-artforregression
-assumesalatentfunctiondrawnfrom aGPprior:
m -mean,k-kernel
-thepredictiveposteriorcanbecomputedanalytically
-GPsexplicitlyincorporateuncertainty-GPsexplicitlyincorporateuncertainty
KERNELS
-extrapolationsetting:muchharderthaninterpolation
-GPisfullydefinedbyitskernel(assuming0mean)
-kernelinducessimilaritiesintheresponsebetweenpairsof
datapoints
-intuitively:
smoothfunction->closerpoints,highcovariance
periodicfunction->pointsatperiodlength,highcovariance
-forextrapolation,kernelchoiceisparamount
Gram matrix kernelevalsbetweentestandall trainingpoints
Data
1176hashtagstimeseries
from 1.1.2011-28.2.2011
~6.5mildeduplicatedtweets
~9.55voc.tokens/tweet
aproxyfortopicsonTwitter
Task:Predicthashtagbasedontweettext
UseGPforecastaspriorforNaiveBayes
TextClassification