The presentation is part of a conference conducted by QuantInsti Quantitative Learning Pvt. Ltd. along with a multinational investment banking firm that engages in global investment banking, securities, investment management, and other financial services.
News is the prime factor which affects prices of financial assets, everything else is secondary. However, owing to the huge volume of news information continuously released by modern electronic communication, it becomes increasingly difficult to process all the information in a timely manner.
You can find a detailed version of this presentation on our blog - http://www.quantinsti.com/blog/quantifying-news-for-automated-trading-methodology-and-profitability/
Connect with us:
Facebook - http://facebook.com/quantinsti
Twitter - http://twitter.com/quantinsti
Youtube - http://youtube.com/quantinsti
6. Historical Perspective - I
Rothschild:
family network spread
across Europe
→
financial information
obtained before peers
e.g.
Knowledge of Battle of
Waterloo result
→
one full day earlier
Methodology → Profitability → QA
10. Historical Perspective - III
Methodology → Profitability → QA
March 27
$2.4 million
March 13
$1-2 million
April 1
< $1 million
11. What is Quantitative News Trading?
News is the first order factor that affects prices, volume,
volatility of stocks, currencies, commodities, etc
Methodology → Profitability → QA
12. What is Quantitative News Trading?
Computer programs that scan news articles & quantify them :
Methodology → Profitability → QA
13. What is Quantitative News Trading?
Computer programs that scan news articles & quantify them :
Methodology → Profitability → QA
14. What is Quantitative News Trading?
News is the prime factor that affects prices, volume, volatility of
stocks, currencies, commodities, etc
Computer programs that scan news articles & quantify them
-> can respond to price moving factors faster than humans
-> can monitor a vaster amount of news reports than humans
Methodology → Profitability → QA
15. What is Quantitative News Trading?
News is the prime factor that affects prices, volume, volatility of
stocks, currencies, commodities, etc
Computer programs that scan news articles & quantify them
-> can respond to price moving factors faster
-> can monitor a vaster amount of news reports
This field is known as ‘Quantitative News Trading’
‘‘During the 200 milliseconds a human is reading the latest news headline, a
trading program will have downloaded the entire article, analyzed its
meaning, & traded based on the content”
Methodology → Profitability → QA
16. What is Quantitative News Trading?
News is the prime factor that affects prices, volume, volatility of
stocks, currencies, commodities, etc
Computer programs that scan news articles & quantify them
-> can respond to price moving factors faster
-> can monitor a vaster amount of news reports
This field is known as ‘Quantitative News Trading’
‘‘During the 200 milliseconds a human is reading the latest news headline, a
trading program will have downloaded the entire article, analyzed its
meaning, & traded based on the content”
Methodology → Profitability → QA
17. What is Quantitative News Trading?
News is the prime factor that affects prices, volume, volatility of
stocks, currencies, commodities, etc
Computer programs that scan news articles & quantify them
-> can respond to price moving factors faster
-> can monitor a vaster amount of news reports
This field is known as ‘Quantitative News Trading’
‘‘During the 200 milliseconds a human is reading the latest news headline, a
trading program will have downloaded the entire article, analyzed its
meaning, & traded based on the content”
Methodology → Profitability → QA
18. What is Quantitative News Trading?
News is the prime factor that affects prices, volume, volatility of
stocks, currencies, commodities, etc
Computer programs that scan news articles & quantify them
-> can respond to price moving factors faster
-> can monitor a vaster amount of news reports
This field is known as ‘Quantitative News Trading’
‘‘During the 200 milliseconds a human is reading the latest news headline, a
trading program will have downloaded the entire article, analyzed its
meaning, & traded based on the content”
How do you quantify news reports and articles ?
Methodology → Profitability → QA
19. What is Quantitative News Trading?
• Sample output of a News Analytics feed: News
represented by numbers
Methodology → Profitability → QA
21. Quantifying News - 1. Sentiment
News articles are assigned a score called ‘sentiment’
Sentiment says whether the article has a positive / negative or
neutral tone
(Sale of Apple iPhones drop = -ve sentiment)
Sentiment at document level is different from sentiment at
entity level
(Samsung beats Apple in smart phone sales = -ve sentiment for
entity named Apple, +ve sentiment for Samsung)
Methodology → Profitability → QA
22. Quantifying News - 1. Sentiment
How is ‘sentiment’ scored ?
Methodology → Profitability → QA
23. Quantifying News - 1. Sentiment
How is ‘sentiment’ scored ?
• Naive parser: based on word count of –ve / +ve keywords
• Discriminated parser: weighted word count
• Grammatical parser: which verbs work on which objects.
check linguistic semantics
• Machine Learning: From the data and the answers, try to find
the factors
Methodology → Profitability → QA
24. Quantifying News - 1. Sentiment
Scoring sentiments: grammatical parsing issues
• Linguistic structures like negation, double negation, sarcasm,
intensification, hanging lemma
(negation: Company X did not become the best in the world
double negation: Company X did not do bad
sarcasm: With such an attitude, X is sure to become the best firm
intensification: Company X did terribly well
hanging lemma: Company X loses lawsuit against company Y. They will
have to pay $1billion USD )
• Word Sense Disambiguation - same word, different meanings
– Company X received a fine
– X is doing fine
– X sells fine grained sand, etc
Methodology → Profitability → QA
25. Quantifying News - Factor 2
Is Sentiment good enough to quantify a news report?
Methodology → Profitability → QA
26. Quantifying News - 2. Relevance
Is Sentiment good enough to quantify a news report?
A news article might:
• be predominantly about a company
• mention that company and others as well
• mention that company in passing in the article
• ‘Relevance’ measures how relevant a news article is for a
particular company
Methodology → Profitability → QA
27. Quantifying News - 2. Relevance
How is relevance scored ?
Methodology → Profitability → QA
28. Quantifying News - 2. Relevance
How is relevance scored ?
Methodology → Profitability → QA
29. Quantifying News - 2. Relevance
How is relevance scored ?
• How many companies are mentioned in the news article
• Is the company mentioned in the headline as the
subject/object
(‘Headline:UBS downgrades HSBC’ is not relevant to UBS)
• In which sentence number is the company first mentioned
• Length of the article & how many times is the firm mentioned
• Number of sentiment words & total words in article
• Two firms mentioned in a news article can both have a
relevance of 1.0 (HP & Compaq announce merger)
Methodology → Profitability → QA
32. Quantifying News - 2. Relevance
Issues with calculating relevance
• Requires synonym database:
– IBM
– International Business Machines
– I.B.M.
– Big Blue
– BAML
– Bank of America
– Merrill Lynch
– Bank of America Merrill Lynch
– Merrill
– BoA
Methodology → Profitability → QA
34. Quantifying News - 3. Novelty
• Often the news article is not reported in its entirety, but in
multiple spurts
– Alert
– News Article
– Update
– Append
Methodology → Profitability → QA
35. Quantifying News - 3. Novelty
• Often the news article is not reported in its entirety, but in
multiple spurts
– Alert
– News Article
– Update
– Append
• Moreover, multiple news
sources report same news
Methodology → Profitability → QA
36. Quantifying News - 3. Novelty
• Often the news article is not reported in its entirety, but in
multiple spurts
– Alert
– News Article
– Update
– Append
• Moreover, multiple news
sources report same news
• News also cause price
changes which themselves
become news
Methodology → Profitability → QA
37. Quantifying News - 3. Novelty
• If we do not keep track & respond to repeated instances of
the same news => we will end up repeating our actions
manifold for the same event
• Therefore every news article should be checked for newness
or ‘novelty’ before responding
Methodology → Profitability → QA
38. Quantifying News - 3. Novelty
How is novelty measured ?
Methodology → Profitability → QA
39. Quantifying News - 3. Novelty
How is novelty measured ?
• The keywords in the current news article are compared to
historical articles about that company for similarity of digital
fingerprints
• A linked articles count is generated
• Novelty is reported for
– Within same news feed novelty (i.e. all Bloomberg news articles only)
– Across all news feeds novelty (i.e. across Reuters, Dow Jones,
Bloomberg articles)
Methodology → Profitability → QA
41. Quantifying News - 4. Market Impact
• Different types of news articles have different impacts on the
price of the asset
• Another aspect of relevance is the likely market impact of the
news article
• Market Impact is therefore a function of the type of news
Methodology → Profitability → QA
44. Quantifying News - News Types
Types of news based on time of news report
• Asynchronous / unexpected
• Synchronous / fixed releases
Methodology → Profitability → QA
45. Quantifying News - Key Factors
While the following are the four key inputs:
• Sentiment
• Relevance
• Novelty
• Market Impact
Some news analytics based strategies use other factors as well…
Methodology → Profitability → QA
46. Quantifying News - 5.i. Volume
The number of news articles on the same topic can be a useful
input to validate the impact
• Volume of news in Social Media also checked sometimes
Methodology → Profitability → QA
55. Methodology - the science behind quantifying news
Profitability - does it really make money
Q&A
Agenda
Methodology → Profitability → QA
56. Is it profitable ?
Source: ThomsonReuters
Methodology → Profitability → QA
57. Where Quantified news work
Machines are faster at responding to events than humans
Low latency event based trading (first to respond)
Machines can process a much vaster amount of information
without any fatigue
Analyze broad spectrum of news to formulate broad views
Methodology → Profitability → QA
58. Where Quantified news work
Analyze broad spectrum of news to formulate broad views
Source: ThomsonReuters
Methodology → Profitability → QA
59. Where Quantified news work
Analyze broad spectrum of news to formulate broad views
Source: ThomsonReuters
Methodology → Profitability → QA
60. Where Quantified news work
Low latency event based trading (first to respond)
Methodology → Profitability → QA
61. Where Quantified news work
Low latency event based trading (first to respond)
For synchronous (fixed releases) expected events (earnings
releases/ economic figures)
• Company figures provided in xml format instead of text
Source: ThomsonReuters
Methodology → Profitability → QA
62. Where Quantified news work
Low latency event based trading (first to respond)
For synchronous (fixed releases) expected events (earnings
releases/ economic figures)
• Company figures provided in xml format instead of text
• Economic figures provided in binary format instead of textual
news articles
Source: ThomsonReuters
Methodology → Profitability → QA
63. Where Quantified news work
Low latency event based trading (first to respond)
For synchronous (fixed releases) expected events (earnings
releases/ economic figures)
• Company figures provided in xml format instead of text
• Economic figures provided in binary format instead of textual
news articles
For asynchronous / unexpected news
• Are quantification algorithms robust enough to calculate
trust-worthy sentiment, relevance, novelty scores ?
Methodology → Profitability → QA
64. Opportunities : initial under-reaction
Quantified news driven trades work even when the trade is done
at the end of the day
(under-reaction to news immediately. Tetlock, et al)
Source: More Than Words: Quantifying Language to Measure Firms’ Fundamentals Tetlock,Saar-Tsechansky &
Macskassy
Methodology → Profitability → QA
65. Late endofday response also profitable
Trading the news immediately = very profitable
At a broad level there is underreaction to news => entering into
trades at the end of the day also makes profits
Source: ThomsonReuters
Methodology → Profitability → QA
66. Long short strategy returns
Source: ThomsonReuters
Methodology → Profitability → QA
67. Filtering sentiments increase profits
Increasing threshold from 90 to
95 percentile increases returns
from 55 to 138 bps in 3 days
Source: ThomsonReuters
Methodology → Profitability → QA
68. Certain sectors more profitable
Moving from Non-Cyclicals to
Financials increased the profit
from 135BP to 147BP
Source: ThomsonReuters
Methodology → Profitability → QA
69. Sectors like Pharma, Defense, Auto, Energy, Banking more sensitive to news
Sensitivity of different sectors
Source: Trading Strategies to Exploit News Sentiment – Wenbin Zhang & Steven Skiena
Methodology → Profitability → QA
70. Small cap firms more profitable
Smaller Cap firms show greater response to extreme sentiment
news event
(bigger firms have greater scrutiny)
Source: Leinweber & ThomsonReuters
Methodology → Profitability → QA
71. Filter & trade fewer stocks
• More is not better. Quality over quantity
• Trading only stocks with very high sentiment/relevance is
better
Source: Trading Strategies to Exploit News Sentiment – Wenbin Zhang & Steven Skiena
Methodology → Profitability → QA
72. Hedged (market-neutral) is better
• Long +ve sentiment stocks only
OR
Short -ve sentiment stocks only. Will fail in different regimes
• Being long +ve sentiment stocks & short -ve sentiment stocks
at the same time gives consistent returns
Source: Trading Strategies to Exploit News Sentiment – Wenbin Zhang & Steven Skiena
Methodology → Profitability → QA
73. Volatile vs stable Economic regimes
• In more volatile markets people tend to react less strongly to
positive news and react more strongly to negative news
Volatility regimes and news
Source: RavenPack, IBES, Macquarie Research, September 2012
Methodology → Profitability → QA
74. Bigger moves happen when there is news in
• Stocks with low beta (i.e. surprises happen to sleepy stocks)
Surprises are more profitable
Source: ThomsonReuters
Methodology → Profitability → QA
75. Bigger moves happen when there is news in
• Stocks with low beta (i.e. surprises happen to sleepy stocks)
• VIX is low (i.e. surprises during calm times)
Surprises are more profitable
Source: ThomsonReuters
Methodology → Profitability → QA
76. Bigger moves happen when there is news in
• Stocks with low beta (i.e. surprises happen to sleepy stocks)
• VIX is low (i.e. surprises during calm times)
• When markets are improving (i.e. surprise to mostly long
position holders)
Surprises are more profitable
Source: ThomsonReuters
Methodology → Profitability → QA
77. Bigger moves happen when there is news in
• Stocks with low beta (i.e. surprises happen to sleepy stocks)
• VIX is low (i.e. surprises during calm times)
• When markets are improving (i.e. surprise to mostly long
position holders)
Surprises are more profitable
Source: ThomsonReuters
Methodology → Profitability → QA
78. Strategy variation - sentiment changes
• Instead of absolute sentiment scores, look at changes in
sentiment scores of firms
• Bought stocks with highest increase in sentiment
• Shorted stocks with highest decrease in sentiment
Source: JP Morgan
Methodology → Profitability → QA
79. Strategy variation - bottom fishing
• Bottom - fishing / turnaround stories
• Buying stocks with reversal in sentiment from grossly
negative (a lot of the stocks turned out to be buybacks)
Source: JP Morgan
Methodology → Profitability → QA
80. Generating Alpha
• Soft (opinion based) vs. Hard (fact based) news
Hard news has a stronger short term reaction than soft news
Source: RavenPack, FactSet, Macquarie Research, September 2012
Methodology → Profitability → QA
81. • Scheduled/expected vs. Unscheduled/unexpected
Investors react more strongly to unscheduled/ unexpected
news than scheduled/ expected
Generating Alpha
Source: RavenPack, FactSet, Macquarie Research, September 2012
Methodology → Profitability → QA
82. • News type Event Study Results
Generating Alpha
Source: RavenPack, FactSet, Macquarie Research, September 2012
Methodology → Profitability → QA
83. News Analytics works best with
• Small cap stocks
• Sectors like pharma, banking, etc
• Stocks with low beta
• When VIX is low
• When markets are improving
• Hard news (vis-a-vis Soft news)
• Unscheduled news events (vis-a-vis scheduled news events)
• Being market-neutral
• Doing fewer stocks, but those with stronger signals
To summarize
Methodology → Profitability → QA
84. Quantifying News - Where it fails?
• News analytics were taught that ‘Osama-Bin-Laden’, and
‘killed’ had -ve sentiments for the markets
Methodology → Profitability → QA
85. Quantifying News - Where it fails?
• News analytics were taught that ‘Osama-Bin-Laden’, and
‘killed’ had -ve sentiments for the markets
• On May 2 2012 when news reporting “Osama Bin-Landen
killed” were published, news bots treated this as a negative
news article and sold stocks
Methodology → Profitability → QA
86. Quantifying News - Where it fails ?
• On Sep. 7, 2008
Google’s newsbots
picked up an old 2002
story about United
Airlines possibly filing
for bankruptcy
• UAL stock dived
immediately
Source: Google Finance
Methodology → Profitability → QA
87. Quantifying News - Where it fails?
Methodology → Profitability → QA
• Dow Jones dropped 0.8% on 23 Apr 2013
– Reasons:
• Twitter account of news publisher hacked – false news
of White house explosion
• News Analytics based automated traders reacted to it
88. Quantifying News – challenges
• Languages like Chinese and Japanese with large number of
alphabetic symbols and complex grammar
However, there is a lot of development in this domain already
• The ever increasing volume of news articles from increased
news sources, and from increased volumes in social media
Methodology → Profitability → QA
89. Methodology - the science behind quantifying news
Profitability - does it really make money
Q&A
Agenda
Methodology → Profitability → QA
90. Contacts
For 4-month Executive Program in Algorithmic Trading:
contact@quantinsti.com
E-PAT: 4 month weekend online program (3hrs every Sat + Sun)
• Statistics
• Quant Strategies
• Technology (programming on algorithmic trading platform)
For algorithmic trading advisory: contact@iragecapital.com
To reach me directly: rajib.borah@iragecapital.com
Methodology → Profitability → QA