Real World MongoDB: Use Cases from Financial Services by Daniel Roberts


Published on

Huge upheaval in the finance industry has led to a major strain on existing IT infrastructure and systems. New finance industry regulation has meant increased volume, velocity and variability of data. This coupled with cost pressures from the business has led these institutions to seek alternatives. In this session learn how FS companies are using MongoDB to solve their problems. The use cases are specific to FS but the patterns of usage - agility, scale, global distribution - will be applicable across many industries.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Increased regulation means increased reporting. Increased IT effort spend and complexity. Increase volumes of data. 3 VVV volume velocity and variability. Need to keep regulators happy .
  • There is a move to new technologies to solve these problems in FS… and the starting point is how we manage these large volumes of data at velocity with continued variablity and increasing volume. What kind of area are we working on?
  • 117883, 69461, 102862, 73277, 65134
  • Real World MongoDB: Use Cases from Financial Services by Daniel Roberts

    1. 1. 3• Introduction• We feel your pain• Traditional Solutions not working• What are your peers doing?• How MongoDB has helped them• Current Use Cases• Future Opportunities
    2. 2. 4
    3. 3. 5Traditional SolutionsNot Working
    4. 4. 6• Tick Data Capture & Analysis• Reference Data Management• Risk Analysis & Reporting• Trade Repository• Portfolio / Position Reporting• Caching Tiers• Mainframe offloading• Know Your Client
    5. 5. 9Use Case:– Collect and aggregate risk data– Calculate risk / exposures, potentially real-timeWhy MongoDB?•Collect data from a single or multiple sources•Support for polymorphic data formats.•Documents used to create ‘pre-aggregated’ reports•Aggregation Framework or Map Reduce•Horizontal Scale
    6. 6. 10ReportingHighVolumeData Feeds
    7. 7. 12• Capture real-time market data (multi-asset, top ofbook, depth of book, even news)• Load historical data• Aggregate data into bars, daily, monthly intervals• Enable queries & analysis on raw ticks oraggregates• Drive backtesting or automated signals
    8. 8. 13Trades/metricsFeed HandlerFeed HandlerExchanges/Markets/BrokersExchanges/Markets/BrokersCapturingApplicationCapturingApplicationLow LatencyApplicationsLow LatencyApplicationsHigher LatencyTradingApplicationsHigher LatencyTradingApplicationsBacktesting andAnalysisApplicationsBacktesting andAnalysisApplicationsMarket DataCached Static &Aggregated DataNews & socialnetworkingsourcesNews & socialnetworkingsourcesOrdersOrdersData Types•Top of book•Depth of book•Multi-asset•Derivatives•News (text, video)•Social NetworkingData Types•Top of book•Depth of book•Multi-asset•Derivatives•News (text, video)•Social Networking
    9. 9. 14{"_id" : ObjectId(“…"),“ask" : 1.30028,"bid" : 1.3002,"ts" :ISODate("2012-02-16T12:48:00Z")}
    10. 10. 16• How do you globally distribute reference data?– Polymorphic data• Price / Products / Securities Master• Counterparty information - KYC• Corporate Actions• Golden / Single source truth– Often changing in structure,• e.g. new products– Often High volume• How is this typically solved today?
    11. 11. 17• What do reference data solutions look like today?• Storage– Relational Database or Caching Technologies• Replication– ETL or Messaging• Complex, Costly and Brittle– Maintenance• schema changes• infrastructure– Multiple technologies
    12. 12. 18• What features in MongoDB are ideally suited forGlobal replicated reference data systems?1. Dynamic and flexible schema
    13. 13. 19IssID IssuerName PVCurrency117883 DWS Vietnam Fund USD69461 Independence III Cdo Ltd USD102862 Zamano Plc EUR73277 Green Way BMD65134 First European Growth Inc. CHFSecID EventID Company_Meeting IssID762288 407341 AGM 11788381198 243459 SDCHG 69461422999 410626 AGM 102862422999 243440 SDCHG 10286275128 20056 ISCHG 65134
    14. 14. 20Relational MongoDB{"IssID" : 65134,"IssuerName" : "First EuropeanGrowth Inc.","actions" : [{"Company_Meeting" :"ISCHG","EventID" : 20056,"SecID" : 75128},{"Company_Meeting" :"LSTAT","EventID" : 2716296,"SecID" : 75128}]}
    15. 15. 21• What features in MongoDB are ideally suited forGlobally replicated reference data systems?1. Dynamic and flexible schema2. Built in replication and high availability
    16. 16. 22
    17. 17. 23• What features in MongoDB are ideally suited forGlobally replicated reference data systems?1. Dynamic and flexible schema2. Built in replication and high availability3. Tag Aware Sharding (Geo)
    18. 18. 24NAEMEAAPAC
    19. 19. 25Feeds & Batch data•Pricing•Accounts•Securities Master•Corporate actionsSourceMaster Data(RDBMS)SourceMaster Data(RDBMS)ETLETL ETLETLETLETLETLDestinationData(RDBMS)DestinationData(RDBMS)Each represents•People $•Hardware $•License $•Reg penalty $•& other downstreamproblems
    20. 20. 26Feeds & Batch data•Pricing•Accounts•Securities Master•Corporate actionsReal-timeReal-time Real-timeReal-timeReal-timeReal-timeReal-timeEach represents•No people $•Less hardware $•Less license $•No penalty $•& many lessproblemsMongoDBSecondariesMongoDBPrimary
    21. 21. 27Distribute reference data globally in real-time forfast local accessing and queryingProblem Why MongoDB Results• Delays up to 20 hours indistributing data via ETL• Had to manage 20distributed systems withsame data• Incurring regulatorypenalties from missingSLAs• Stale data causedoperational issues• Dynamic schemamanagement: updateimmediately & in oneplace• Auto-replication: datadistributed in real-time• Both cache and database:cache always up-to-date• Simple data modeling &analysis: easy changesand understanding• Will save about$40,000,000 in costs andpenalties over 5 years• Greater throughput meanscharging more to internalgroups• Network and disk speed isthe bottleneck, notsoftware and applications
    22. 22. 29• Not Just Capital Markets• Insurance / Consumer banking• Inspiration from other Industries– Telco– Retail• Opportunity to consolidate and utilise data ininteresting ways
    23. 23. 32Resource LocationMongoDB Downloads Online Training education.10gen.comWebinars and Events Papers Studies docs.mongodb.orgAdditional Info info@10gen.comResource Location