SlideShare a Scribd company logo
www.sungard.com
Opportunities in #BigData & Beyond
John Avery
Partner
SunGard Global Services
Nov 18, 2010
Why am I here?
Professional
• SunGard #2 in 2010 FINTECH 100
• Products & Services across the FS spectrum
• Exploring impacts of Big Data approach for past
18 months
Personal
• 13+ years of development on Wall Street
• Intersection of Global Services Advanced
Technology & Information Management Practices
• Development + Databases + Data Warehousing
+ Trading + Risk Management
What is #BigData?
 An approach to storing, processing & analyzing
data that
 Can be “Internet” scale - think Google, Facebook, LinkedIn,
Yahoo, Twitter
 “Not Just Relational” - augments and/or supplements
traditional relational databases and approaches
 Economically leverages commodity or cloud-based hardware
using the more flexible & scalable NoSQL ecosystem
 Enables analysis & insight on data & problems typically left out
to pasture
Parallel Distributed Analysis
Parallel Distributed Processing
Parallel Distributed Storage
Machine 1 Machine 10 Machine 100 Machine N
Why is #BigData Important?
Data is Growing Faster
than “Moore’s Law”
• Americans consuming on average
34GB/day and uploading 15x more
data in 2009 than 2006
• 281 exabytes of online data in 2009
• Business cycle across technology
sector very dependent on 40+ years
of Moore’s Law assumptions
In Financial Services the
quest for global trading,
risk & transparency creates
an insatiable need for data
• Regulatory Reform / Systemic Risk
Oversight
• Hi-Fi trading is 50-75% of cash
trading market activity today
• The Rise of Unstructured Data &
Social Network Analysis
The #BigData Opportunity in Financial Services
Take hint from “Internet Scale” B2C players who manage
hundreds of terabytes more than what financial services
require
• Reduce the effort & time to aggregate & analyze data
• Economically enable new classes of analysis
As an industry, there is no “killer app” for Big Data yet,
firms are exploring & learning on their own
• Explore new analyses of current & historical system data
• Use new programming languages & tools to create solutions
Get continued leverage on legacy systems by exposing
their data
• Expose vertical system data in its raw form to the NoSQL technology
ecosystem - in parallel with any other data management initiatives
• Years & Years of Proprietary Data History - Can this data be packaged up &
resold?
What I Find Interesting About @AsterData
5+ Years of Big Data and Advanced Analytics
Commercial, Not Open Source
Creating a bridge to a world of SQL-trained Developers & Analysts
Exposing data to a world of xDBC database access code (Java, .NET,
etc…)
Providing Big Data analysis abstractions on top of SQL
(e.g.,: SQL-MapReduce)
Baked-in “Graph” analytics (more on this in forthcoming slides)
Will #BigData evolve into SmartData (or #BigGraph)?
Freedom from perceived (or real) relational constraints via
BigData
• Regulatory Transparency
• Requires consistency & connectedness – globally
• Unstructured Data
• News, Social Media, Machine learning, linguistics,
etc…
• Mobile & Tablet Interfaces
• Create a new and rapidly growing need for integrated
aggregated data in digestible & relevant form
• Social Networks
• Remind us of hundreds of years of graph theory
• Recent Google Infrastructure Overhaul
• Elevated importance of graph-based processing
Graph Analytics in Retail Banking
Graph Gallery 1 – MBS Market Structure
Image credits - @ValidsKrebs on Twitter
Graph Gallery 2 – Counterparty Networks
Image credit - @ValidsKrebs on Twitter
Immediate Opportunities for Graph Analytics
Wealth & Investment
Management
• Portfolio analysis &
construction
Trading
• Pre-trade analysis
• Unstructured data input into
algo trading + backtesting
Risk
• Counterparty Credit Risk
Network Analysis
Regulatory &
Compliance
• Systemic Risk Transparency
• Surveillance & Fraud
Detection
• Behavioral Analysis
Will Richer Semantics enable more Graph Analytics?
Can These Be Input into A Whole New Class of
Relationship Based Graph Analyses?
Counterparty
Relationships
Financial Product
& Portfolio
Relationships
Fundamental
Balance Sheet
Analysis
Market Structure
Analysis
Structured Data Formats du jour
XBRL SWIFT FPML Proprietary
BigGraph Opportunity #1 – Pre-Trade Analysis
 Commodities Markets
 Thousands of relationships
upon relationships
 Freight <-> Oil
 Oil, Gas, Coal <-> Power
 Base metals <-> Industrials
 Agriculture <-> Fuels
 Agriculture <-> Foods
BigGraph Opportunity #2 – Risk Management
For A Single
Entity
• Counterparty
relationships
intersecting with
financial instruments
Across Entities
• Counterparty
relationships
intersecting with
financial
instruments
Systemic Risk
Oversight (US
FSOC & Office
of Financial
Research)
Image credit - @ValidsKrebs on Twitter
A Call to Action – Wholesale Financial Services
The tools are here!
We need more
exploratory application
of the technology to new
or unanswered problems
in the institutional world
We need more
Advocacy of Exploration
We need more People
interested and beginning
to Explore
We need more POCs
Let’s Chat
 John Avery, SunGard Global Services
 John.Avery@SunGard.com
 On Twitter
 @john_avery
 Hashtags
 #BigData
 #NoSQL
 #GraphDB
www.sungard.com/consultingProprietary and Confidential. Not to be distributed or reproduced without permission
Copyright © 2010 by SunGard Data Systems (or its subsidiaries, “SunGard”). All rights reserved.
No parts of this document may be reproduced, transmitted or stored electronically without
SunGard’s prior written permission.
17
COPYRIGHT STATEMENT

More Related Content

What's hot

What's hot (20)

Data Science Innovations : Democratisation of Data and Data Science
Data Science Innovations : Democratisation of Data and Data Science  Data Science Innovations : Democratisation of Data and Data Science
Data Science Innovations : Democratisation of Data and Data Science
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Lars Lyberg, Inizio: Rapport från konferensen BigSurv18
Lars Lyberg, Inizio: Rapport från konferensen BigSurv18Lars Lyberg, Inizio: Rapport från konferensen BigSurv18
Lars Lyberg, Inizio: Rapport från konferensen BigSurv18
 
Big data characteristics, value chain and challenges
Big data characteristics, value chain and challengesBig data characteristics, value chain and challenges
Big data characteristics, value chain and challenges
 
Introduction to Big Data & Analytics
Introduction to Big Data & AnalyticsIntroduction to Big Data & Analytics
Introduction to Big Data & Analytics
 
Big data
Big dataBig data
Big data
 
BIG DATA & DATA ANALYTICS
BIG  DATA & DATA  ANALYTICSBIG  DATA & DATA  ANALYTICS
BIG DATA & DATA ANALYTICS
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
Big Data
Big DataBig Data
Big Data
 
Big data.
Big data.Big data.
Big data.
 
Big data Presentation
Big data PresentationBig data Presentation
Big data Presentation
 
Big data and Internet
Big data and InternetBig data and Internet
Big data and Internet
 
Big Data Landscape 2018
Big Data Landscape 2018Big Data Landscape 2018
Big Data Landscape 2018
 
Big data
Big dataBig data
Big data
 
Big data
Big dataBig data
Big data
 
Real time analytics of big data
Real time analytics of big dataReal time analytics of big data
Real time analytics of big data
 
big data Presentation
big data Presentationbig data Presentation
big data Presentation
 
Data science
Data scienceData science
Data science
 
Big data
Big dataBig data
Big data
 
Bigdatacooltools
BigdatacooltoolsBigdatacooltools
Bigdatacooltools
 

Viewers also liked (6)

Presentation to TAP UG/DTU on ICT4D by Dr. Romeo Bertolini, July 2005
Presentation to TAP UG/DTU on ICT4D  by Dr. Romeo Bertolini, July 2005Presentation to TAP UG/DTU on ICT4D  by Dr. Romeo Bertolini, July 2005
Presentation to TAP UG/DTU on ICT4D by Dr. Romeo Bertolini, July 2005
 
Bristol University
Bristol UniversityBristol University
Bristol University
 
Printversion ice summer school 1 7-2013.key
Printversion ice summer school 1 7-2013.keyPrintversion ice summer school 1 7-2013.key
Printversion ice summer school 1 7-2013.key
 
Break The Rules of Social Media
Break The Rules of Social MediaBreak The Rules of Social Media
Break The Rules of Social Media
 
Elisha: Picking up the Mantle
Elisha: Picking up the MantleElisha: Picking up the Mantle
Elisha: Picking up the Mantle
 
4 chap
4 chap4 chap
4 chap
 

Similar to BigData and Beyond

¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
Denodo
 
final oracle presentation
final oracle presentationfinal oracle presentation
final oracle presentation
Priyesh Patel
 
Presentation1 (1).pptx
Presentation1 (1).pptxPresentation1 (1).pptx
Presentation1 (1).pptx
Dat Trinh
 
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Denodo
 

Similar to BigData and Beyond (20)

01-introduction.ppt the paper that you can unless you want to join me because...
01-introduction.ppt the paper that you can unless you want to join me because...01-introduction.ppt the paper that you can unless you want to join me because...
01-introduction.ppt the paper that you can unless you want to join me because...
 
Big data Analytics
Big data AnalyticsBig data Analytics
Big data Analytics
 
Big data analytic market opportunity
Big data analytic market opportunityBig data analytic market opportunity
Big data analytic market opportunity
 
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
 
final oracle presentation
final oracle presentationfinal oracle presentation
final oracle presentation
 
BIG DATA CHAPTER 2 IN DSS.pptx
BIG DATA CHAPTER 2 IN DSS.pptxBIG DATA CHAPTER 2 IN DSS.pptx
BIG DATA CHAPTER 2 IN DSS.pptx
 
uae views on big data
  uae views on  big data  uae views on  big data
uae views on big data
 
Thilga
ThilgaThilga
Thilga
 
Big Data Evolution
Big Data EvolutionBig Data Evolution
Big Data Evolution
 
Data Lake Architecture – Modern Strategies & Approaches
Data Lake Architecture – Modern Strategies & ApproachesData Lake Architecture – Modern Strategies & Approaches
Data Lake Architecture – Modern Strategies & Approaches
 
Architecting for Big Data: Trends, Tips, and Deployment Options
Architecting for Big Data: Trends, Tips, and Deployment OptionsArchitecting for Big Data: Trends, Tips, and Deployment Options
Architecting for Big Data: Trends, Tips, and Deployment Options
 
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
ADV Slides: What the Aspiring or New Data Scientist Needs to Know About the E...
 
Presentation1 (1).pptx
Presentation1 (1).pptxPresentation1 (1).pptx
Presentation1 (1).pptx
 
Kartikey tripathi
Kartikey tripathiKartikey tripathi
Kartikey tripathi
 
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
 
When and How Data Lakes Fit into a Modern Data Architecture
When and How Data Lakes Fit into a Modern Data ArchitectureWhen and How Data Lakes Fit into a Modern Data Architecture
When and How Data Lakes Fit into a Modern Data Architecture
 
Intro big data analytics
Intro big data analyticsIntro big data analytics
Intro big data analytics
 
Big_Data.pptx
Big_Data.pptxBig_Data.pptx
Big_Data.pptx
 
Bigdata (1) converted
Bigdata (1) convertedBigdata (1) converted
Bigdata (1) converted
 
DAS Slides: Graph Databases — Practical Use Cases
DAS Slides: Graph Databases — Practical Use CasesDAS Slides: Graph Databases — Practical Use Cases
DAS Slides: Graph Databases — Practical Use Cases
 

Recently uploaded

Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 

Recently uploaded (20)

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
The architecture of Generative AI for enterprises.pdf
The architecture of Generative AI for enterprises.pdfThe architecture of Generative AI for enterprises.pdf
The architecture of Generative AI for enterprises.pdf
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
In-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT ProfessionalsIn-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT Professionals
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 

BigData and Beyond

  • 1. www.sungard.com Opportunities in #BigData & Beyond John Avery Partner SunGard Global Services Nov 18, 2010
  • 2. Why am I here? Professional • SunGard #2 in 2010 FINTECH 100 • Products & Services across the FS spectrum • Exploring impacts of Big Data approach for past 18 months Personal • 13+ years of development on Wall Street • Intersection of Global Services Advanced Technology & Information Management Practices • Development + Databases + Data Warehousing + Trading + Risk Management
  • 3. What is #BigData?  An approach to storing, processing & analyzing data that  Can be “Internet” scale - think Google, Facebook, LinkedIn, Yahoo, Twitter  “Not Just Relational” - augments and/or supplements traditional relational databases and approaches  Economically leverages commodity or cloud-based hardware using the more flexible & scalable NoSQL ecosystem  Enables analysis & insight on data & problems typically left out to pasture Parallel Distributed Analysis Parallel Distributed Processing Parallel Distributed Storage Machine 1 Machine 10 Machine 100 Machine N
  • 4. Why is #BigData Important? Data is Growing Faster than “Moore’s Law” • Americans consuming on average 34GB/day and uploading 15x more data in 2009 than 2006 • 281 exabytes of online data in 2009 • Business cycle across technology sector very dependent on 40+ years of Moore’s Law assumptions In Financial Services the quest for global trading, risk & transparency creates an insatiable need for data • Regulatory Reform / Systemic Risk Oversight • Hi-Fi trading is 50-75% of cash trading market activity today • The Rise of Unstructured Data & Social Network Analysis
  • 5. The #BigData Opportunity in Financial Services Take hint from “Internet Scale” B2C players who manage hundreds of terabytes more than what financial services require • Reduce the effort & time to aggregate & analyze data • Economically enable new classes of analysis As an industry, there is no “killer app” for Big Data yet, firms are exploring & learning on their own • Explore new analyses of current & historical system data • Use new programming languages & tools to create solutions Get continued leverage on legacy systems by exposing their data • Expose vertical system data in its raw form to the NoSQL technology ecosystem - in parallel with any other data management initiatives • Years & Years of Proprietary Data History - Can this data be packaged up & resold?
  • 6. What I Find Interesting About @AsterData 5+ Years of Big Data and Advanced Analytics Commercial, Not Open Source Creating a bridge to a world of SQL-trained Developers & Analysts Exposing data to a world of xDBC database access code (Java, .NET, etc…) Providing Big Data analysis abstractions on top of SQL (e.g.,: SQL-MapReduce) Baked-in “Graph” analytics (more on this in forthcoming slides)
  • 7. Will #BigData evolve into SmartData (or #BigGraph)? Freedom from perceived (or real) relational constraints via BigData • Regulatory Transparency • Requires consistency & connectedness – globally • Unstructured Data • News, Social Media, Machine learning, linguistics, etc… • Mobile & Tablet Interfaces • Create a new and rapidly growing need for integrated aggregated data in digestible & relevant form • Social Networks • Remind us of hundreds of years of graph theory • Recent Google Infrastructure Overhaul • Elevated importance of graph-based processing
  • 8. Graph Analytics in Retail Banking
  • 9. Graph Gallery 1 – MBS Market Structure Image credits - @ValidsKrebs on Twitter
  • 10. Graph Gallery 2 – Counterparty Networks Image credit - @ValidsKrebs on Twitter
  • 11. Immediate Opportunities for Graph Analytics Wealth & Investment Management • Portfolio analysis & construction Trading • Pre-trade analysis • Unstructured data input into algo trading + backtesting Risk • Counterparty Credit Risk Network Analysis Regulatory & Compliance • Systemic Risk Transparency • Surveillance & Fraud Detection • Behavioral Analysis
  • 12. Will Richer Semantics enable more Graph Analytics? Can These Be Input into A Whole New Class of Relationship Based Graph Analyses? Counterparty Relationships Financial Product & Portfolio Relationships Fundamental Balance Sheet Analysis Market Structure Analysis Structured Data Formats du jour XBRL SWIFT FPML Proprietary
  • 13. BigGraph Opportunity #1 – Pre-Trade Analysis  Commodities Markets  Thousands of relationships upon relationships  Freight <-> Oil  Oil, Gas, Coal <-> Power  Base metals <-> Industrials  Agriculture <-> Fuels  Agriculture <-> Foods
  • 14. BigGraph Opportunity #2 – Risk Management For A Single Entity • Counterparty relationships intersecting with financial instruments Across Entities • Counterparty relationships intersecting with financial instruments Systemic Risk Oversight (US FSOC & Office of Financial Research) Image credit - @ValidsKrebs on Twitter
  • 15. A Call to Action – Wholesale Financial Services The tools are here! We need more exploratory application of the technology to new or unanswered problems in the institutional world We need more Advocacy of Exploration We need more People interested and beginning to Explore We need more POCs
  • 16. Let’s Chat  John Avery, SunGard Global Services  John.Avery@SunGard.com  On Twitter  @john_avery  Hashtags  #BigData  #NoSQL  #GraphDB
  • 17. www.sungard.com/consultingProprietary and Confidential. Not to be distributed or reproduced without permission Copyright © 2010 by SunGard Data Systems (or its subsidiaries, “SunGard”). All rights reserved. No parts of this document may be reproduced, transmitted or stored electronically without SunGard’s prior written permission. 17 COPYRIGHT STATEMENT