Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas

Copyright ® 2009, SAS Institute Inc. All right s reserved.

Financial Data Mining with Algorithmic Trading
Robert Golan
DBmind Technologies, Inc.
Please Note: This is the view of DBmind only which may not
pertain to DBmind’s Client Views

Financial Data Mining with Algorithmic Trading
Algorithmic Trading has changed the world the way the
Traders trade and Trade Support supports. There is a
Brave New World happening with the "hands on" Trading
evolving into "hands off" Algo Trading. Not all trades need
to be made in ultra low latency timing. Future trading will
rely on a broader set of data which will be mined for
relevance. An important series of XBRL Financial
Reporting events are happening throughout the world and
especially in the USA. A critical mass of financial data will be ready for mining which will be a boon for
transparent "low touch" fundamental style algorithmic trading. Also, "low touch" trading such as program trading & direct market access (DMA)
will evolve into advanced Algo Trading strategies. Stock and economic indicators combined with XBRL will add value for Algo Trading. This is
about a well thought out strategic high latency trading strategy with data mining discovering the governing rules while adding the expert rules
with validation. Yes, the trader is still the key to making this all happen. Both fundamental and technical trading rules need to be combined with
the expert rules, the data mined rules, and most importantly the regulatory environment rules. RegNMS in the USA and MiFID in Europe have
indirectly helped the adoption of electronic trading and it is important to integrate the GRC related rules in an agile way. Agility is the key and
thus the rules need to be placed into a rules engine and managed by the experts for proper compliance, risk management, and governance.
Japan, China, and the Netherlands with regards to XBRL are ready to be data mined with Algo Trading now. A XBRL US survey is indicating at
least 340 of the estimated 500 public companies that the SEC requires to begin filing in XBRL format in June 2009, have already converted
their financial statements into XBRL. XBRL US, is the non-profit XML standard setter that developed and maintains the US GAAP taxonomy
used by filers to comply with the SEC mandate. Almost $7 trillion in market capitalization will be represented by this XBRL financial data which
is over 50% of the total market cap for all publicly traded companies reporting to the SEC. As this XBRL Financial Data ripens, a wonderful
harvest awaits us data miners which will enhance the current Algo Trading strategies which use this data.

Algorithmic trading
 http://en.wikipedia.org/wiki/Algorithmic_trading
• In electronic financial markets, algorithmic trading, also known as algo, automated, black
box, or robo trading, is the use of computer programs for entering trading orders with the
computer algorithm deciding on certain aspects of the order such as the timing, price, or type
(market vs. limit, or buy vs. sell) of the order. It is widely used by pension funds, mutual
funds, and other institutional traders to divide up a large trade into several smaller trades in
order to avoid market impact costs or otherwise reduce transaction costs. It is also used by
hedge funds and similar traders to make the decision to initiate orders based on information
that is received electronically, before human traders are even aware of the information.
• Algorithmic trading may be used in any market strategy, including market making,
intermarket spreading, arbitrage, or pure speculation (including trend following) to make the
complete decision on entering trades and electronically executing the trade with no human
intervention, other than in writing the computer program.
• In 2006 at the London Stock Exchange, over 40% of all orders were entered by algo traders,
with 60% predicted for 2007. American markets and equity markets generally have a higher
proportion of algo trades than other markets, and estimates for 2008 range as high as an
80% proportion in some markets. Foreign exchange markets also have active algo trading
(about 25% of orders in 2006). Futures and options markets are considered to be fairly easily
integrated into algorithmic trading, and bond markets are moving toward more access to
algorithmic traders.

 Citadel Investment Group
 Interactive Brokers
 Credit Suisse
 Deutsche Bank
 Goldman Sachs
 Lehman Bros.
 Morgan Stanley
 Susquehanna Investment Group
 UBS
Some of the Algo Players

Statistical Arbitrage in the U.S. Equities Market
Marco Avellaneda† and Jeong-Hyun Lee
First draft: July 11, 2008
This version: June 15, 2009
Abstract
We study model-driven statistical arbitrage in U.S. equities. The trading signals are generated in two
ways: using Principal Component Analysis and using sector ETFs. In both cases, we consider the
residuals, or idiosyncratic components of stock returns, and model them as mean-reverting
processes.
This leads naturally to “contrarian” trading signals.The main contribution of the paper is the construction, back-testing
and comparison of market-neutral PCA- and ETF- based strategies applied to the broad universe of U.S. stocks. Back-testing shows that, after
accounting for transaction costs, PCA-based strategies have an average annual Sharpe ratio of 1.44 over the period 1997 to 2007, with
much stronger performances prior to 2003. During 2003-2007, the averageSharpe ratio of PCA-based strategies was only 0.9. Strategies based
on ETFs achieved a Sharpe ratio of 1.1 from 1997 to 2007, experiencing a similar degradation after 2002. We also introduce a method to
account for daily trading volume information in the signals (which is akin to using “trading time” as opposed to calendar time), and observe
significant improvement in performance in the case of ETF-based signals. ETF strategies which use volume information
achieve a Sharpe ratio of 1.51 from 2003 to 2007. The paper also relates the performance of mean-reversion statistical arbitrage strategies with
the stock market cycle. In particular, we study in detail the performance of the strategies during the liquidity crisis of the
summer of 2007. We obtain results which are consistent with Khandani and Lo (2007) and validate their “unwinding” theory for the quant fund
drawdown of August 2007.
Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, N.Y. 10012
USA
†Finance Concepts, 49-51 Avenue Victor-Hugo, 75116 Paris, France.

XBRL Basics
 XBRL is XML
 It is Extensible
 There is an XBRL specification – tells you how to
use XBRL
 Hinges on taxonomies – the dictionary of terms
for business reporting – which includes financial
statements

XBRL Taxonomy
Created by XBRL
Consortium
Consumed
Rendered
XBRL
Creation
XBRL Document
Created by Preparer
TAGGING

Validation
Standardization
Calculation
Cash = Currency +
Deposits
Calculation
Cash = Currency +
Deposits
Formulas
Cash ≥ 0
Formulas
Cash ≥ 0
Contexts
US $
FY2004
Budgeted
Contexts
US $
FY2004
Budgeted
Label
cashCashEquivalentsAn
dShortTermInvestments
Label
cashCashEquivalentsAn
dShortTermInvestments
References
GAAP I.2.(a)
Instructions
Ad Hoc disclosures
References
GAAP I.2.(a)
Instructions
Ad Hoc disclosures
Presentation
Cash & Cash Equivalents
Presentation
Cash & Cash Equivalents
XBRL
Item
XBRL
Item
XML
Item
XML
Item
XBRL
Item
XBRL
Item
Presentation
Comptant et Comptant
Equivalents
Presentation
Comptant et Comptant
Equivalents
Presentation
Geld & Geld nahe Mittel
Presentation
Geld & Geld nahe Mittel
Presentation
Kas en Geldmiddelen
Presentation
Kas en Geldmiddelen
Presentation
现金与现金等价物
Presentation
现金与现金等价物
Presentation
現金及び現金等価物
Presentation
現金及び現金等価物
Presentation
Деньги и их эквиваленты
Presentation
Деньги и их эквиваленты
Presentation
Гроші та їх еквіваленти
Presentation
Гроші та їх еквіваленти

The Business Reporting Supply Chain
External
Financial
Reporting
Business
Operations
Internal
Financial
Reporting
Investment,
Lending, and
Regulation
Processes
Participants
Auditors
Trading
Partners
Investors
Financial
Publishers
and Data
Aggregators
Regulators
Software VendorsSoftware Vendors
Management
Accountants
Companies
XBRL XBRLXBRL
XBRL
Financial StatementsXBRL-GL
The
Journal
Standard
Transaction
Standards
Collaboration is
KEY!!!

USUS
UKUK
JPJPESES
SESE
CNCN
ZAZA
AUAU
DEDEDKDK
Financial
Banking Regulators Pilot
Pilot Committe
d
Committe
d
KRKR
SGSG
FRFR
NZNZ
NLNL
LuXLuX
Portuga
l
Portuga
l
BEBE
EU CEBSEU CEBS

XBRL Jurisdictions
UK
CA
SPUS
AU
NZ
IR
JP
KR
BE
VZ
CO
BR
AR
PT
RU
SG
HK
NO
SE
PL
FI
IT
CN
IN
LB
CZ
UA
LU
IAS
B
AE
NL
TR
GR
MT
CH
FR
SI
HU
AT
Established
Jurisdictions
Provisional
Jurisdictions
in Construction &
in Project
DE
DK
ZA

CACA UKUK
IEIE
AUAU
NONO
JPJP
NZNZ
NLNL DEDE
CNCN
Tax Authorities Pilot
Pilot Committe
d
Committe
d
Tax XML Technical Committee
recommends use of XBRL
(Oasis-OECD)
29 Tax authorities

Exchanges
& Equity Regulators
Sao Paulo
Sao Paulo
NZSE
NZSE
ASX
ASXJohannesburg
Johannesburg
Shenzen
Shenzen
EuroNext
EuroNext
KOSDAQ
KOSDAQ
Tokyo
Tokyo
Singapore
Singapore
SWX
SWXLux
Lux
Pilot LiveEval
TSX
TSX
OBX
OBX
LSE
LSE
CSE
CSE
Deutsche
Börse
Deutsche
Börse
Taipei
SEC
SEC
Kor
ea
Kor
ea
Shanghai
Shanghai

Grant Boyd, CA
gboyd@aicpa.org
Technical Manager – XBRL, AICPA
•http://www.icgfm.org/XBRLPresentations.htm

Data Mining Approaches
 Three Basic Approaches to Data Mining
• Mathematical-based methods,
• Distance-based methods, and
• Logic-based methods
 Methods may use supervised or unsupervised variable
• Supervised – induction rules for predefined
classifications
• Unsupervised – rules and classifications determined by
data mining method

Mathematical-based Methods
 Neural Network
• Network of nodes modeled after a neuron or neural circuit
• Supervised learning
• Weighted values at different nodes
• Mimics the processing of the human brain
• Form of Artificial Intelligence

Mathematical-based Methods
 Discriminant Analysis
• Similar to multiple regression analysis uses a non-
continuous dependent variable
• Approach identifies the variables (features or cases)
that best explain the classification
• Supervisory learning approach
• Loses effectiveness with large complex data sets

Logic-Based Approach
 Tree and Rule Induction
• Supervised Learning
− Uses an algorithm to induce a decision tree from a file of
individual cases
− Case has set of attributes and the class to which it belongs
• Decision tree can be converted to a rule-based view.
• Major advantage is ability to communicate and understand
information derived from this approach.
• Prior research addressed audit areas of:
− bankruptcy, bank failure, and credit risk

Distance-Based Method
 Clustering
• Data mining approach that partitions large sets of data
objects into homogeneous groups
• Uses unsupervised classification where little manual
pre-screening of data is necessary –
− useful in situations where there is no predefined
knowledge of categories
• Classifications based on an object’s attributes
• Most commonly used in field of marketing but could be
used in auditing

Selecting Data Mining Approach
 Criteria:
• Scalability - how well data mining method works
regardless of data set size
• Accuracy - how well information extracted remains
stable and constant beyond the boundaries of the data
from which it was extracted, or trained
• Robustness - how well the data mining method works
in a wide variety of domains
• Interpretability - how well data mining method provides
understandable information and valuable insight to user

Continuous Auditing, XBRL and Data Mining
Presenters:
Jennifer Moore, Lumsden & McCormick, LLP
Karina Barton, Canisius College
Dr. Joseph O’Donnell, Canisius College
New York State Society of Certified Public Accountants
Technology Assurance Committee
June 15, 2004

Prime Services
 http://en.wikipedia.org/wiki/Prime_brokeragehttp://en.wikipedia.org/wiki/Prime_brokerage
• Prime Brokerage is the generic name for a bundled package of
services offered by investment banks to hedge funds. The business
advantage to a hedge fund of using a Prime Broker is that the Prime
Broker provides a centralized securities clearing facility for the hedge
fund, and the hedge fund's collateral requirements are netted across
all deals handled by the Prime Broker. The Prime Broker benefits by
earning fees ("spreads") on financing the client's long and short cash
and security positions, and by charging, in some cases, fees for
clearing and/or other services.
• The following "core services" are typically bundled into the Prime
Brokerage package:
− Global custody (including clearing, custody, and asset servicing)
− Securities lending
− Financing (to facilitate leverage of client assets)
− Customized Technology (provide hedge fund managers with
portfolio reporting needed to effectively manage money)
− Operational Support (prime brokers act as a hedge fund's primary
operations contact with all other broker dealers)

Hedge Fund
Hedge Fund
Hedge Fund
Exchange Clearing Settlement
Trading
Position
keeping
Clearing
And
Settlement
Prime
Services

Hedge Fund
Mutual Fund
Hedge Fund
Exchange Exchange Exchange
Algorithmic
Trading
System
Prime Service Provider
Ice Berg
VWAP
TWAP

Client
Client
Client
Exchanges Clearing Settlement
Equities
Derivatives
Fixed
Income
Reference
Data
Validation
and
Enrichment
Risk
Management
Financial
Control
Clearing
and
Settlement

Date: 15 May 2007
Produced by: Chris Swan

Overview of stock exchanges
The main stock exchanges in the world include:
 America
• American Stock Exchange
• NASDAQ
• New York Stock Exchange
• São Paulo Stock Exchange
 Europe
• Euronext
• Frankfurt Stock Exchange
• London Stock Exchange
• Madrid Stock Exchange
• Milan Stock Exchange
• Zurich Stock Exchange
• Stockholm Stock Exchange

Listing requirements
 LSE — main market has requirements for a minimum market
capitalization of £700,000, three years of audited financial statements,
minimum public float of 25 % and sufficient working capital for at least 12
months from the date of listing
 NASDAQ — to be listed a company must have issued at least 1.25
million shares of stock worth at least $70 million and must have earned
more than $11 million over the last three years
 NYSE — a company must have issued at least a million shares of stock
worth $100 million and must have earned more than $10 million over the
last three years

Overview of stock exchanges
 Australia/Asia/Africa
• Australian Stock Exchange
• Bombay Stock Exchange
• Hong Kong Stock Exchange
• Johannesburg Securities Exchange
• Korea Stock Exchange
• Shanghai Stock Exchange
• Taiwan Stock Exchange
• Tokyo Stock Exchange
• Toronto Stock Exchange

Participants
 Broker — an individual or firm which operates between a buyer and a seller and usually
charge a commission. For most products a licence is required.
 Dealer — an individual or firm which buys and sells for its own account.
 Broker/dealer — an individual or firm buying and selling for itself and others. A
registration is required.
 Principal — a role of broker/dealer when buying or selling securities for its own account.
 Market maker — a brokerage or bank that maintains a firm bid and ask price in a given
security by standing ready, willing, and able to buy or sell at publicly quoted prices (called
making a market). These firms display bid and offer prices for specific numbers of specific
securities, and if these prices are met, they will immediately buy for or sell from their own
accounts.
 Specialist — a stock exchange member who makes a market for certain exchange-traded
securities, maintaining an inventory of those securities and standing ready to buy and sell
shares as necessary to maintain an orderly market for those shares. Can be an individual,
partnership, corporation or group of firms.

Prototypical trading systems
 Call (periodic) auction — selling stocks by bid at intervals throughout
the day. The orders are stored for execution at a single market clearing price.
 Continuous auction — buyers enter competitive bids and sellers place
competitive offers simultaneously. Continuous, since orders are executed upon
arrival.
 Dealership market — trading occur between principals buying and
selling to their own accounts. Firm price quotations are available prior to order
submission.
 Auction markets are concentrated and order-driven
 Dealership markets are fragmented and quote-driven

Examples
 NYSE — opens with a periodic auction market and then
switches to a continuous auction. Same for Tokyo Stock
Exchange.
 NASDAQ and International Stock Exchange
(London) are quote-driven systems (continuous dealership
market).

Examples
 Euronext Paris — the market is segmented into a number of
different groups of stocks based on size and liquidity. The trading
mechanisms vary depending on the segment.
 Euronext 100, Next 150 ,CAC40 indices and stocks which have
more than 2,500 order book transactions per year — continuous
auction.
 Other stocks — call auction twice a day.

Order types
 Market order — immediate execution at the best price
available when the order reaches the marketplace
 Limit order — to execute a transaction only at a specified price
(the limit) or better
 Stop order
 Good till cancelled
 Fill-or-kill
 All or None
 Day order

Algorithmic Trading: An Overview ofAlgorithmic Trading: An Overview of
Applications And Models.Applications And Models.
Ekaterina Kochieva
Gautam Mitra
Cormac Lucas

Summary
 Definitions and examples have been given about XBRL &
Financial Data Mining.
 Key components to make this work are the global adoption
of a Business Reporting language. This includes
consistent standards being set by both XBRL US & XBRL
International.
 All companies need to be held responsible for their
reporting with the current Reg rules adjusted. A global
certification process is needed with the proper GRC
engagement model followed. The good news is that there
will be plenty of data to be mined.
 How we mine and integrate this into our Trading Strategies
is up to us. This is the "special sauce" which will make or
break how our Trader trades & Trade Support supports.

Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas

Recommended

Recommended

More Related Content

Similar to Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas

Similar to Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas (20)

Recently uploaded

Recently uploaded (20)

Financial Data Mining and Algo Trading presented at the SAS Data Mining Conference in Las Vegas

Editor's Notes