Can tweets help predict a stock's price movements?
26 | The Journal of the CFA Society of the UK | www.cfauk.orgFeature | Professional InvestorWeb users’ interactions and commentaries in the social mediaspace can reflect current opinions, views and experiences, andtherefore contain helpful information for market research.Consumers who profess stronger positive affinity with a certainbrand are likely to have a higher customer lifetime value, apredictor of the net present value of profits from a customerover the entire future relationship with him/her. The evidenceso far is too little to demonstrate consistent results. However,this new avenue demands further investigation with advancedstatistical analysis and larger scale application.INTRODUCTIONIn the early days when internet search algorithms were beingdeveloped, who could have imagined that search data could beused to predict the future? Yet here we are in 2012 witnessingit. Many organisations have found that data extracted fromspecific searches can predict – or at least model the future.The Bank of England (BoE) is just one organisation that isconvinced that appropriately interpreted search data can act asan indicator of future economic trends. In June 2011, a teamof researchers from the BoE released a report illustrating howFernan Flores asks whether the analysis of tweets or other social mediapostings could be a useful predictor of market movements, as it has beendemonstrated in the case of Google search data.Tweet: “@cfauk – is it true that tweetscan help predict a stock price movement?”
The Journal of the CFA Society of the UK | www.cfauk.org | 27Professional Investor | Featureresults extracted from Google search data could predict changesin unemployment and even house prices.Being not a fan of social media sites, I had never usedTwitter, a micro blogging site, until I read an article describingit as the new Google. As a market research analyst andconsequently a fan of Google, I was intrigued and registered forTwitter to see what the buzz was about.Twitter has indeed a search function that allows anyone tobrowse through tweets, postings or status updates, sometimes inreal time. In fact, the research results by seeking out key wordsfrom tweets proved to be very useful when I undertook somecompetitive intelligence work for a client to check about itscompetitor’s customer service. This was quite a revelation.Web users’ interactions and commentaries in the social mediaspace can reflect current opinions, views and experiences andtherefore contain helpful information for market research.Could the analysis of tweets or other social media postings bea useful predictor of market movements though, as it has beendemonstrated in the case of Google search data?Derwent Capital, a company which was originally establishedas a hedge fund that used consumer tweets in its trading strategybut has now repositioned itself as a technology provider givingtraders and investor access to its proprietary platform, said in anarticle published in August 2011 that based on its research andtesting of randomly selected unstructured data from Twitter thatits algorithm, which helps classify a tweet into a sentiment (e.g.alert, vital, happy), helped predict movements in liquid stocks.A similar strategy was replicated by the University ofManchester and Indiana University in a research paper(Bollen, Mao, and Zeng, 2010), showing that Twitter dataanalysed for sentiment predicted around 87.6% of themovements in the Dow Jones industrial average. The studywas based on an assumption used in behavioural finance,which states that “financial decisions are significantly drivenby emotion and mood… therefore, [it is] reasonable to assumethat the public mood and sentiment can drive stock marketvalues as much as news.”ANALYSISIn order to explain unstructured tweets, many social mediamonitoring and analytics (SMMA) firms like Derwent Capitalhave developed algorithms that categorise tweets (or any socialmedia postings) as positive, neutral or negative. The tweets arefurther classified so that words that express stronger emotionsare classified at the extreme ends of a Likert scale such as theones illustrated in Chart 1 above.The hypothesis that social media can be a strong indicator offinancial performance is based on the principle that consumerswho profess stronger positive affinity with a certain brand willhave a higher customer lifetime value, a predictor of the netpresent value of profits from a customer over the entire futurerelationship with him/her. If a brand or organisation has morecustomers with stronger positive (or less negative) affinity, itshould have a positive financial outlook, which is reflectedthrough a strong stock performance.To prove this relationship at a basic level, I plotted theproportion of positive and negative sentiments againstthe closing stock price of Apple (see Figures 1 and 2) andMicrosoft (see Figures 3 and 4). Because of the volatility ofthe data, especially the sentiments, I used the data’s three-daymoving average standardised with z-scores in order tocompare the movements in the stock price and the sentimentsmore evenly.Chart 1: Likert Scale“Financial decisions are significantlydriven by emotion and mood…therefore,it is reasonable to assume that the publicmood and sentiment can drive stockmarket values as much as news.”Apple annoys me!I will never buy aniPhone again.My iPhone isgettingproblematic.My iPhone isworking ok.I enjoy usingmy iPhone.I love my newiPhone! I stronglyrecommend thateveryone buys one too!1 2 3 4 5Positive sentimentsNegative sentiments
28 | The Journal of the CFA Society of the UK | www.cfauk.orgApple (Jan - Dec 2011)Microsoft (Jan - Dec* 2011)Microsoft (Jan - Dec* 2011)Apple (Dec 2011 - Jan 2012)Feature | Professional InvestorAs can be seen in Figures 1 and 2, the correlation coefficientbetween the stock price and sentiments is very weak for Appleand actually counter-intuitive as the positive sentiments trendis negatively correlated with the stock price.For Microsoft, a relationship seems to exist especially for positivesentiments. As highlighted in Figures 3 and 4, there are days wheneither the positive or negative sentiments clearly moved along withthe changes in stock price (as highlighted by the blue vertical lines).While the accuracy of the technology developed by SMMAfirms in data mining has considerably improved over the years,removing spam or filtering only relevant information remains achallenge with the best technology achieving only an accuracylevel of between 75%-85% and the majority achieving anaccuracy level of between 50%-60%.A deep-dive analysis of Apple verbatims reveals that aconsiderable number of statements analysed refers to eitherapple, the fruit, or apple juice. It is therefore not surprisingthat the relationship between the sentiments and Apple’s stockprice hardly exists at all.In contrast, the data mining technology has more accuratelyanalysed Microsoft given the uniqueness of the brand asa term. The resulting correlation for Microsoft over a year,however, remains weak. It could be possible that whileverbatims for Apple include irrelevant information, analysisfor Microsoft may have excluded tweets that refer to Microsoftbut have been omitted because consumers may have used theirown jargon when spelling the brand or have unintentionallymisspelled it (e.g. MS, Macrosoft, Mikrosoft, Microsof).Figures 3 and 4:Microsoft full year 2011 (positive and negative sentiments)Apple (Jan - Dec 2011)Apple (Jan - Dec 2011)Microsoft (Jan - Dec* 2011)Microsoft (Jan - Dec* 2011)Apple (Dec 2011 - Jan 2012)Source: Yahoo! Finance and TwitterFigures 3 and 4. Z-scores of Microsoft’s closing stock price in NASDAQ versus z-scores of positive sentiments (Figure 3) andnegative sentiments (Figure 4) from Twitter. (Note that z-scores of negative sentiments are shown in reverse order as a decreasein negative sentiment is expected to have a positive impact while an increase in negative sentiment is expected to have a negativeimpact on a stock’s performance.)Figures 1 and 2:Apple full year 2011 (positive and negative sentiments)Apple (Jan - Dec 2011)Apple (Jan - Dec 2011)Microsoft (Jan - Dec* 2011)Microsoft (Jan - Dec* 2011)Apple (Dec 2011 - Jan 2012)Source: Yahoo! Finance and TwitterFigures 1 and 2. Z-scores of Apple’s closing stock price in NASDAQ versus z-scores of positive sentiments (Figure 1) and negativesentiments (Figure 2) from Twitter. (Note that z-scores of negative sentiments are shown in reverse order as a decrease in negativesentiment is expected to have a positive impact while an increase in negative sentiment is expected to have a negative impact on astock’s performance.)Apple (Jan - Dec 2011)Microsoft (Jan - Dec* 2011)Microsoft (Jan - Dec* 2011)Apple (Dec 2011 - Jan 2012)r = -0.26r = 0.39r = 0.14r = -0.26
I conducted regression analysis and made variouscombinations of analysis accounting for potential lag,comparing the weighted average score of all sentiments (i.e.rating extremely positive statements a 5, a relatively positivestatement a 4, a neutral statement a 3, a relatively negativestatement a 2 and an extremely negative statement a 1) andcomparing the net sentiment (i.e. the resulting proportion ofsentiments when negative is deducted from positive) but noneof the resulting analysis proved that the sentiments have astrong relationship with a brand’s stock price.With some effort, I manually cleaned hundreds of Appletweets (i.e. removing tweets that refer to apple, the fruit, orapple juice) from December 2011 until January 2012. Theresulting comparison as shown in Figure 5 illustrates thattweets that are more accurately filtered can potentially bemore effective in predicting a brand’s stock price, achieving acorrelation coefficient of 0.85.CONCLUSIONWhile manually cleaned Twitter sentiments, at least for Applein this example, shows that consumer sentiment movementsmovements can have a strong correlation to a company’sstock price movements, the evidence so far is too little todemonstrate consistent results. Clearly, this new avenueconsisting of exploitating Twitter or other social media websitesdemands further investigation with advanced statistical analysisand application on a larger scale to ascertain the relationshipbetween the two data sets.With the rapid progress of technology in this field, especiallywith search algorithms becoming more and more clever, itis likely that the capability to demonstrate a correlation willimprove across time.Can this work for non-consumer brands (e.g. BHP Billiton)?Can sentiments on brands really have an impact on the stockprice of the company that owns them (e.g. PG tips, Bovriland Persil owned by Unilever)? Can tweets from non-Englishspeaking countries and consumers, which are continuouslyincreasing in share as a proportion of total global tweets,weaken or strengthen the relationship between sentiments andstock price? These are just a few of the questions that we havenot even begun to address. Yet as technology develops, this willspread into other compatible areas, geographies and cultures.Given these issues, using tweets or any social media datafor trading strategy needs further exploration to strengthenthe case for it. But perhaps, based on Everett Rogers’ theoryof “Diffusion of Innovation” this may not be necessary forinnovators and early adopters – the consumer segmentswhich adopt technology ahead of the rest of the population.Given the speed of technological innovation in data mining,combined with advanced statistical analysis, I am confidentthat using social media as a highly reliable predictor of stockprice movements can be achieved much sooner than expected.However, when this point happens and when everyone elsestarts to use insights from tweet sentiments for trading, thenthe opportunity for arbitrage will have disappeared. ■The Journal of the CFA Society of the UK | www.cfauk.org | 29Professional Investor | FeatureProfileFernan FloresFernan Flores is a freelance market researchanalyst and director at Zapienza, a CanaryWharf-based market research consultingfirm that specialises in the technology andfinance sectors, which he established aftercompleting his MBA degree from theCambridge Judge Business School. Apartfrom the technology and finance sectors, healso does a considerable amount of work inthe not-for-profit sector and specialises in thedeployment of technology to solve healthcareissues in developing markets. He has passedthe Level I exam of the CFA Program andis a member of the CFA UK marketing andcommunications committee.Source: Yahoo! Finance and TwitterFigure 5:Apple 2 months December 2011 - January 2012Microsoft (Jan - Dec* 2011)Microsoft (Jan - Dec* 2011)Apple (Dec 2011 - Jan 2012)Source: Yahoo! Finance and TwitterFigure 5. Z-scores of Apple’s closing stock price inNASDAQ versus z-scores of positive sentiments usingdata that are further filtered manually.r = -0.85