BigData and Beyond


Published on

John Avery Presentation from NYC Data Analytics Summit November 18, 2010

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

BigData and Beyond

  1. 1. Opportunities in #BigData & Beyond John Avery Partner SunGard Global Services Nov 18, 2010
  2. 2. Why am I here? Professional • SunGard #2 in 2010 FINTECH 100 • Products & Services across the FS spectrum • Exploring impacts of Big Data approach for past 18 months Personal • 13+ years of development on Wall Street • Intersection of Global Services Advanced Technology & Information Management Practices • Development + Databases + Data Warehousing + Trading + Risk Management
  3. 3. What is #BigData?  An approach to storing, processing & analyzing data that  Can be “Internet” scale - think Google, Facebook, LinkedIn, Yahoo, Twitter  “Not Just Relational” - augments and/or supplements traditional relational databases and approaches  Economically leverages commodity or cloud-based hardware using the more flexible & scalable NoSQL ecosystem  Enables analysis & insight on data & problems typically left out to pasture Parallel Distributed Analysis Parallel Distributed Processing Parallel Distributed Storage Machine 1 Machine 10 Machine 100 Machine N
  4. 4. Why is #BigData Important? Data is Growing Faster than “Moore’s Law” • Americans consuming on average 34GB/day and uploading 15x more data in 2009 than 2006 • 281 exabytes of online data in 2009 • Business cycle across technology sector very dependent on 40+ years of Moore’s Law assumptions In Financial Services the quest for global trading, risk & transparency creates an insatiable need for data • Regulatory Reform / Systemic Risk Oversight • Hi-Fi trading is 50-75% of cash trading market activity today • The Rise of Unstructured Data & Social Network Analysis
  5. 5. The #BigData Opportunity in Financial Services Take hint from “Internet Scale” B2C players who manage hundreds of terabytes more than what financial services require • Reduce the effort & time to aggregate & analyze data • Economically enable new classes of analysis As an industry, there is no “killer app” for Big Data yet, firms are exploring & learning on their own • Explore new analyses of current & historical system data • Use new programming languages & tools to create solutions Get continued leverage on legacy systems by exposing their data • Expose vertical system data in its raw form to the NoSQL technology ecosystem - in parallel with any other data management initiatives • Years & Years of Proprietary Data History - Can this data be packaged up & resold?
  6. 6. What I Find Interesting About @AsterData 5+ Years of Big Data and Advanced Analytics Commercial, Not Open Source Creating a bridge to a world of SQL-trained Developers & Analysts Exposing data to a world of xDBC database access code (Java, .NET, etc…) Providing Big Data analysis abstractions on top of SQL (e.g.,: SQL-MapReduce) Baked-in “Graph” analytics (more on this in forthcoming slides)
  7. 7. Will #BigData evolve into SmartData (or #BigGraph)? Freedom from perceived (or real) relational constraints via BigData • Regulatory Transparency • Requires consistency & connectedness – globally • Unstructured Data • News, Social Media, Machine learning, linguistics, etc… • Mobile & Tablet Interfaces • Create a new and rapidly growing need for integrated aggregated data in digestible & relevant form • Social Networks • Remind us of hundreds of years of graph theory • Recent Google Infrastructure Overhaul • Elevated importance of graph-based processing
  8. 8. Graph Analytics in Retail Banking
  9. 9. Graph Gallery 1 – MBS Market Structure Image credits - @ValidsKrebs on Twitter
  10. 10. Graph Gallery 2 – Counterparty Networks Image credit - @ValidsKrebs on Twitter
  11. 11. Immediate Opportunities for Graph Analytics Wealth & Investment Management • Portfolio analysis & construction Trading • Pre-trade analysis • Unstructured data input into algo trading + backtesting Risk • Counterparty Credit Risk Network Analysis Regulatory & Compliance • Systemic Risk Transparency • Surveillance & Fraud Detection • Behavioral Analysis
  12. 12. Will Richer Semantics enable more Graph Analytics? Can These Be Input into A Whole New Class of Relationship Based Graph Analyses? Counterparty Relationships Financial Product & Portfolio Relationships Fundamental Balance Sheet Analysis Market Structure Analysis Structured Data Formats du jour XBRL SWIFT FPML Proprietary
  13. 13. BigGraph Opportunity #1 – Pre-Trade Analysis  Commodities Markets  Thousands of relationships upon relationships  Freight <-> Oil  Oil, Gas, Coal <-> Power  Base metals <-> Industrials  Agriculture <-> Fuels  Agriculture <-> Foods
  14. 14. BigGraph Opportunity #2 – Risk Management For A Single Entity • Counterparty relationships intersecting with financial instruments Across Entities • Counterparty relationships intersecting with financial instruments Systemic Risk Oversight (US FSOC & Office of Financial Research) Image credit - @ValidsKrebs on Twitter
  15. 15. A Call to Action – Wholesale Financial Services The tools are here! We need more exploratory application of the technology to new or unanswered problems in the institutional world We need more Advocacy of Exploration We need more People interested and beginning to Explore We need more POCs
  16. 16. Let’s Chat  John Avery, SunGard Global Services   On Twitter  @john_avery  Hashtags  #BigData  #NoSQL  #GraphDB
  17. 17. and Confidential. Not to be distributed or reproduced without permission Copyright © 2010 by SunGard Data Systems (or its subsidiaries, “SunGard”). All rights reserved. No parts of this document may be reproduced, transmitted or stored electronically without SunGard’s prior written permission. 17 COPYRIGHT STATEMENT