Presentation given on September 18, 2012 at the 'Hadoop in Finance Day' conference held in Chicago and organized by Fountainhead Lab at Microsoft's offices.

1. Big Data at CME Group:
Challenges and Opportunities
Rick Fath & Slim Baltagi
9/18/2012

2. © 2012 CME Group. All rights reserved
Agenda
1. CME Group Overview
2. Big Data at CME Group
3. Big Data Use Cases at CME Group
4. Big Data Challenges and Opportunities at CME Group
5. Big Data Key Learning’s at CME Group
6. Questions

3. 1. CME Group Overview
3

4. © 2012 CME Group. All rights reserved 4
CME Group Overview
• #1 futures exchange in the
U.S. and globally by 2011
volume
• 2011 Revenue of $3.3bn
• 11.8 million contracts per
day
• Strong record of growth,
both organically and
through acquisitions
– Dow Jones Indexes
– S&P Indexes
– BM&FBOVESPA
– CME Clearing Europe
– CME Europe Ltd
Combination is greater
than the sum of its parts

5. © 2012 CME Group. All rights reserved 5
Forging Partnerships to Expand Distribution,
Build 24-Hour Liquidity, and Add New Customers
CBOT Black
Sea Wheat
Partnerships include:
• Equity investments
• Trade matching services
• Joint product development
• Order routing linkages
• Product licensing
• Joint marketing
• European clearing services
• Developing capabilities globally
• Expanding upon global benchmark products
• Positioned well within key strategic closed markets
•Recently announced application to
FSA for CME Europe Ltd. –
expected launch mid-2013

6. Big Data: Transactions + Interactions + Observations
2. Big Data at CME Group

7. © 2012 CME Group. All rights reserved
When you Data gets too Big!!
7

8. © 2012 CME Group. All rights reserved
Big Data and CME Group: a natural fit
Definition (Strategy) of Big Data at CME Group
• CME Group is a Big Data factory! 11 million
contracts a day…
• Data is not merely a byproduct, but an
essential deliverable
• Daily market data generated by the exchange
and also historical data
• Growing partnerships around the world
• Meeting new regulatory requirements (CFTC,
SEC)
• Trading shift from floor to electronic
• Algorithmic trading improvements-high
volumes, low latency, and historical trends

9. © 2012 CME Group. All rights reserved
Solving Big Data at CME Group
• Leverage legacy tools vs. adoption of new Big Data technologies
• Transactional data (trades, market data, order data, etc.)
• Missed opportunities with existing Legacy Big Data applications:
Not all data is being stored and quality is lacking. Learn from this
lesson
• Raw data being sold without analytics
• Risk and Maturity of new Big Data Technologies
• Quality of Historical data: different protocol, format....
9

10. 3. Big Data Use Cases at CME Group
10

11. © 2012 CME Group. All rights reserved
Use Case 1: CME Group's BI solution
Description: Provide complex reporting and data analysis for internal Business
teams.
Challenges:
• Performance Needed: Reports and Analysis are time sensitive. (Existing queries
taking 18+ hours)
• Established tools and partnerships (Informatica, Business Objects, and
established business users)
• Mission critical queries, timely analytics, complex aggregation required
• Risk adverse business customers
11

12. © 2012 CME Group. All rights reserved
Use Case 1: CME Group's BI solution
Opportunities:
• Structured RDBMS
• Minimize integration impact
• Enhance customer analytics
• Deliver faster time to market of new Market Regulation requirements
• Improve and increase fact based decision making
• Reduce batch processes execution duration
Solution:
12

13. © 2012 CME Group. All rights reserved
Why we chose a Data Warehouse Appliance?
• No need to re-engineer the existing Oracle Database applications
• Based on CME evaluation of Data Warehouse Appliances, Exadata is
best suited to CME environment
• DWA performs consistently better than our DB production environment.
• Product maturity
• Faster queries compared to Oracle DB
13

14. © 2012 CME Group. All rights reserved
Description: Reduce reliance on SAN while increasing query performance.
Challenges:
• Expensive storage cost.
• Oracle queries cannot keep up with inserts and do not meet SLAs
Opportunities:
• Reduce storage cost with non specialized hardware. “not commodity”
• Fast Parallel queries
Solution:
14
Use Case 2: CME Group’s Rapid Data Platform

15. © 2012 CME Group. All rights reserved
Description: Provide historical Market Data to exchange customers who want to
understand market trends, conduct market analysis, and buildtest trading
algorithms.
Challenges:
• Expensive storage cost.
• Legacy downstream application with limited support
• Historical data from acquisitions and mergers…”Don’t look for problems
because you will find them”
• 100TB data and growing….
• Data Redundancy and Quality (limited awareness)
• Business users dependent on Technology staff to use the data.
15
Use Case 3: CME Group’s Historical Data Platform

16. © 2012 CME Group. All rights reserved
Opportunities:
• Reduce storage cost with non specialized hardware. “not
commodity”
• Reduce duplication of data (derive data on demand fast)
• Improve Data Quality with better interrogation
• Grow the business (new datasets, mash-ups, analytics)
• Separate Delivery from Storage: Store everything, deliver what you
need.
• Enable Business users (Ad Hoc queries, define new datasets)
• Reduce TCO (support, improve reliability)
Solution:
16
Use Case 3: CME Group’s Historical Data Platform

17. © 2012 CME Group. All rights reserved
Why we chose Hadoop?
• Solution built for scale and growth
• Structure of data doesn’t matter. Data is the Data.
• Hadoop as ETL solution
• Performance POC with Oracle – Beat Oracle queries and
removes duplication
• Decouple Storage and Delivery. Store raw data, deliver
enhanced data.
• Ecosystem reduces development time with proven solutions to
common problems
• Structure is fluid, perfect for historical data
17

18. 4. Big Data Challenges and
Opportunities at CME Group
18

19. © 2012 CME Group. All rights reserved
Enterprise Challenges
• Open Source Paradigm
• Enterprise Maturity- built for failure
• Operations Readiness
• Framework can mask problems
• Educate the enterprise about Hadoop
• Evolution of Hadoop
19

20. © 2012 CME Group. All rights reserved
Deployment Challenges
• Selection of Hadoop distribution
• Growing cluster equals growing support fees. How to lower
maintenance fees
• Future Project candidates (super cluster vs. individual clusters)
• Distributions support and alignment
• Maximize investment in IT assets (Informatica, BusinessObjects,
ExaData)
• DR and Backup Solutions
20

21. © 2012 CME Group. All rights reserved
Potential Hadoop Use Cases for Future
21
• Compliance and regulatory reporting
• Trade surveillance
• Abnormal trading pattern analysis
• Integrate disparate datasets and unlock the value at the intersection
of data.
• Identify new Big Data projects as a new source of Revenue
• Reduce enterprise data redundancy (ETL, storage, analytics),
• Use as alternative for costly CEP solutions where applicable

22. 5. Big Data Key Learning’s at CME
Group
22

23. © 2012 CME Group. All rights reserved
What we Learned
• Embrace open source- “We are not the first ones to solve this
problem!!”
• Enterprise readiness: DR, Backup, Security, HA
• Solving operational maturity (server to admin ratio)
• Leverage existing IT investments
• Adapt standards to make cluster more supportable
• Tackling the learning curve: keep close to community
• Leverage the Hadoop Ecosystem…
• Capture everything (More capture, support for structured and semi-
structured
• Capture data first and then figure out new opportunities (schema-
less)
• Hadoop can be complimentary to existing IT assets.
23

24. 6. Questions
24

25. © 2012 CME Group. All rights reserved 25
Futures trading is not suitable for all investors, and involves the risk of loss. Futures are a
leveraged investment, and because only a percentage of a contract’s value is required to trade,
it is possible to lose more than the amount of money deposited for a futures position. Therefore,
traders should only use funds that they can afford to lose without affecting their lifestyles. And
only a portion of those funds should be devoted to any one trade because they cannot expect to
profit on every trade.
The Globe Logo, CME®, Chicago Mercantile Exchange®, and Globex® are trademarks of
Chicago Mercantile Exchange Inc. CBOT® and the Chicago Board of Trade® are trademarks of
the Board of Trade of the City of Chicago. NYMEX, New York Mercantile Exchange, and
ClearPort are trademarks of New York Mercantile Exchange, Inc. COMEX is a trademark of
Commodity Exchange, Inc. CME Group is a trademark of CME Group Inc. All other trademarks
are the property of their respective owners.
The information within this presentation has been compiled by CME Group for general purposes
only. CME Group assumes no responsibility for any errors or omissions. Although every attempt
has been made to ensure the accuracy of the information within this presentation, CME Group
assumes no responsibility for any errors or omissions. Additionally, all examples in this
presentation are hypothetical situations, used for explanation purposes only, and should not be
considered investment advice or the results of actual market experience.
All matters pertaining to rules and specifications herein are made subject to and are superseded
by official CME, CBOT, NYMEX and CME Group rules. Current rules should be consulted in all
cases concerning contract specifications.