Financial services trihug


Published on

Hadoop based applications are becoming critical in the financial services arena for the analysis and correlation of large volumes of structured and unstructured data. In addition, the Dodd-Frank Act signifies the largest US financial regulatory change in several decades and requires much greater transparency on financial data. In this session, we will answer common questions and demonstrate use cases in how Hadoop and Datameer help with asset management and risk management, fraud detection and data security.
Leave this session knowing about:
Financial data and Hadoop. What data lends itself to Hadoop? What doesn’t?
Benchmarks from real-world uses of Hadoop in finance
How to effectively migrate, manage, and analyze financial data using Hadoop

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • There are 3 main pain points + strong economic pressure ++ to make more money companies need to analyze more data ++ IT backlog, self service + large amount of Data + new kind of data + this applies also to verticals like financial services, telco, internet + ebay collects with each page view 300 data points + ebay from 5 to 20 to 40 PB this year. + The Gap use case
  • Problem Description (1 of 3) -- Financial Institutions often hold large positions that they want to trade in ways that don ’ t affect public markets -- Dark pools are private venues that enable trading for this practice -- Dark Pool trading grants advantages but is still costly -- It ’ s a big data problem: large volumes, high velocity -- The lack of visibility is the true nature of the problem: visibility into who the players are and what they ’ re really offering
  • -- Banks “ advertise ” intentions with FIX messages -- they can be structured and nested -- messages are “ offers ” not contracts
  • -- group FIX messages into order families -- mine the order family data for insight -- look at the patterns and identify questionable behavior -- time to review the terms of use with a particular party? -- time to review the relationship with certain vendors?
  • -- negotiation is often mechanical and automated -- doesn ’ t always result in a deal -- key is to find the patterns where volume is wasted -- heavy negotiating resulting in cancellations is very costly
  • Financial services trihug

    1. 1. Hadoop in Financial Services Adam Gugliciello, Solutions Engineer February, 2012
    2. 2. A Different Approach… What is the problem? History of the data... What is the pain in the market place. Real life example.. Walmart or GAP Cost pain point there is 5 main paintpoinst + more competition + to make more mney you need to analyze more money + what is the pain. THIS slide replace with old vs new slow expansive expertice one platform fast economic self service Data Warehouse Expensive ETL Slow Business Intelligence Expertise Hadoop Economical Raw Load Fast Drag and Drop, Spreadsheets Self Service
    3. 3. Internal Asset Movement <ul><li>Focus: Institutional Risk </li></ul><ul><li>Identify departmental asset movement </li></ul><ul><ul><li>Actual “Rogue Trader” identified using Datameer </li></ul></ul><ul><li>Craft an early-warning system for future detection </li></ul><ul><li>Aggregate sources and correlate </li></ul><ul><li>Net result: reduced risk </li></ul>
    4. 4. Client Risk Profiling <ul><li>Focus: Bank Assets </li></ul><ul><ul><li>Identify profile-scoring features in loans and mortgages </li></ul></ul><ul><ul><li>Build qualifying models with Datameer </li></ul></ul><ul><ul><li>Test models against existing data </li></ul></ul><ul><ul><li>Provide recurring reports </li></ul></ul><ul><li>Focus: Client Assets </li></ul><ul><ul><li>Monitor risk across product offerings for all client assets </li></ul></ul><ul><ul><li>Develop common base of client characteristics for models </li></ul></ul><ul><ul><li>Analyze risk types: default, liquidity, attrition risk </li></ul></ul><ul><ul><li>Monitor: transactions, web, email, chats and call logs (all of the different ways clients interact with the bank) </li></ul></ul><ul><li>Net result: correlation and increases agility </li></ul>
    5. 5. SLA Analytics <ul><li>Focus: Infrastructure Performance </li></ul><ul><li>Companies are using Datameer to aggregate all logs to Hadoop </li></ul><ul><li>Transform log-data into actionable insight into IT systems performance </li></ul><ul><li>Measure conformance to existing SLA’s </li></ul><ul><li>Reduce the batch window </li></ul><ul><li>Net result: reduced costs </li></ul>
    6. 6. Security & Fraud <ul><li>Focus: Client protection </li></ul><ul><li>Run statistical models against customer profiles </li></ul><ul><li>Detect anomalies with pattern-recognition algorithms </li></ul><ul><li>Identify and investigate identity theft </li></ul><ul><li>Detect and prevent fraud </li></ul><ul><li>Protect customer assets </li></ul><ul><li>Net result: protected customers </li></ul>
    7. 7. Data Quality & Profiling <ul><li>Focus: Profiling </li></ul><ul><li>Use profiling techniques to assess and score of client accounts </li></ul><ul><li>Run complex data quality rules and models </li></ul><ul><li>Build exception reporting </li></ul><ul><li>Focus: Data Quality </li></ul><ul><li>Enable data stewards to run ad hoc analysis and dashboards </li></ul><ul><li>Drive iterative processes to refine data </li></ul><ul><li>Net result: reduce risk and cost </li></ul>
    8. 8. Security Infrastructure <ul><li>Focus: infrastructure protection </li></ul><ul><li>Monitor external intrusions </li></ul><ul><li>Detect suspicious traffic </li></ul><ul><li>View security aggregates </li></ul><ul><li>Correlate disparate attack sources </li></ul><ul><li>Net result: increased security </li></ul>
    9. 9. Dark Pools and Cost Savings <ul><li>Focus: Trade and Partner Analyses </li></ul><ul><li>Financial Institutions often hold large positions that they want to trade in ways that don't affect public markets </li></ul><ul><li>Dark pools are private venues that enable trading for this practice </li></ul><ul><li>It ’ s a big data problem: </li></ul><ul><ul><li>Large volumes </li></ul></ul><ul><ul><li>High velocity </li></ul></ul><ul><ul><li>Low visibility </li></ul></ul>
    10. 10. All talk, little action indication of interest, action (quote/buy/sell/fulfill), “ symbol ” , “ quantity ” , “ price ” Footer Header Financial Information eXchange (FIX)
    11. 11. Analyze Order Families Order Family Analysis sell cancel Time Instrument Vendor buy
    12. 12. A Second in the life…
    13. 13. Cut through the noise
    14. 14. Questions and Answers….