TIBCO StreamBase: Real-Time Analytics: The Last Mile in Big Data


Published on

TIBCO StreamBase: Real-Time Analytics: The Last Mile in Big Data
Richard Tibbetts, CTO, TIBCO StreamBase
June 25, 2013 The Langham Hotel Boston, MA

Published in: Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

TIBCO StreamBase: Real-Time Analytics: The Last Mile in Big Data

  1. 1. 1Enabling Real Time Analytics in the Capital MarketsRichard TibbettsCTO, TIBCO StreamBaseReal Time Analytics:The Last Mile inBig Data
  2. 2. 2Real Time Visibility: The Last Mile in Big DataThe Last Mile:Delivery tousage site• Expensive• Purpose Built• Non-Optional
  3. 3. 3Big Data: Everyone Talks about VolumeIn the past 10 years, the amount ofdata to be processed has exploded.Mobile devices, sensors, marketdata, video, audio, and social mediaare among the sources of thisexplosion of data.
  4. 4. 4There can be an overwhelming volume
  5. 5. 5Most solutions focus on accumulating and analyzing
  6. 6. 6Especially analyzing things which already happened
  7. 7. 7But how will you access the data when you need it?
  8. 8. 8Closing the loop – creating value with big data Accumulate – Capture and store the data• Use the right technology to capture data as it happens (e.g. Flume)• Load it into appropriate price/performance storage (e.g. HDFS) Analyze – Form and test hypotheses• Discover mathematical relations and patterns (e.g. R)• Crunch enormous volumes of data to validate and tune (e.g. Hadoop) Access – Use the knowledge at the point of decision• Get the data to the people who need it, when they need it• Present it in the way that they can consume it Automate – Optimize even small decisions with data• Control systems with a closed feedback loop• Monitor and focus on the exceptions, automate the common actions
  9. 9. 9Real Time AnalyticsStreamBase Event Processing Platform
  10. 10. 10StreamBase EventFlow - Integration, Analytics, Reaction Visual programming integrates components developed by teammembers with different skills• Developers, Quants, Data Scientists, Traders, Business AnalystsConnectivityDataInfrastructureQuantitativeAnalyticsAuditLoggingBusinessStrategyOff the ShelfComponents
  11. 11. 11ApplicationsIntegrationNormalizationAggregationCorrelationRulesAlertsAutomationAdaptersandHandlersAdaptersandHandlersStreamBase Server(s)StreamBase Studio forDeveloping EventFlow ApplicationsData ManagementPersistence StoresLogsMarketDataTrade DataStatic DataSystemsDataPerformanceBenchmarksAutomationDesktopAlertsStreamBase Platform for Complex Event ProcessingInputs Outputs
  12. 12. 12StreamBase Studio: Eclipse Visual DevelopmentVisual operatorsAlgorithm librarySchemas, threading, etc.Event Application Code
  13. 13. 13StreamBase Studio: Agile Development, Mixed TeamsFeed SimulationRefactoringSeamless Java & visualEventFlow DebuggerVisual Diff/MergeUnit TestingJava Operators &Function Wizards
  14. 14. 14API’sC++, Java, .NET, PythonMessagingTIBCO RV & EMS, JMS, MQ Series, RMDS, Solace, Tervela, Wombat, 29WestAdditional General Purpose AdaptersFIX, STAMP, JDBC, POP3, IRC, HTTP, SMTP, IM, XML, RSS, TwitterStreamBase Connectivity and Visualization OptionsMarket Data/Venue Samples VisualizationEclipse RCPAdobe FlexJava Swing.NETMS ExcelSL RTViewDatabasesTR Velocity
  15. 15. 15Kosmos: StreamBase for High Frequency Trading Who is Kosmos?• One of Brazil’s leading hedge fundswhich employs both quantitative andqualitative investment strategies StreamBase for HFT• Connectivity to wide selection ofequities and FX data sources• Building next generation tradingsystem• Equities, FX and derivatives• Selected StreamBase over other CEPproviders Why StreamBase?• Rapid time-to-market• Flexibility• Developer empowerment
  16. 16. 16SUNGARD: Pre-Trade Risk Solution Business Driver:• Fast Response to Regulatory Change Project:• Enhancement of Assent Liquidity Solutions andBrass Order Management System Asset classes:• Equities• Derivatives• Fixed income Why StreamBase?• Maturity of StreamBase’s platform and success inthe CEP market• Graphical event-flow language and visualenvironment that enable developers and businessusers to work more efficiently• Superior performance to reduce latency issues“CEP... is very good for rapiddevelopment of applicationswhich are deployed in high-volume, low-latency typeenvironments.The products offered by Assentare constantly being challengedwith lots of market data, ordersand execution updates. Wewant to use StreamBase toaugment and enhance some ofthe products that we have.”Chris Lees, VP Liquidity Services,SunGard
  17. 17. 17BNY ConvergEx: StreamBase for Monitoring BNY ConvergEx: The Convergence of Brokerageand Technology• Top 10 NYSE liquidity provider• Vortex, an electronic ATS / dark pool• A leader in algorithmic trading and OMS technology• Deployed StreamBase for monitoring of U.S. tradingoperations BNY ConvergEx Deploys StreamBase for U.S.Trading Operations• Intelligent monitoring of trading operations,including its dark pool Vortex• Real-time market interaction monitoring• Real-time credit risk correlation• System health monitoring• Real-time analysis of market feeds for latency andresponse time“The implementation ofStreamBase allows us tomonitor our risk… andassess the health of ourentire U.S. tradingoperations in real-time.”Joseph Weisbord, ManagingDirector, BNY ConvergExGroup
  18. 18. 18CityIndex: StreamBase for Options Pricing and Monitoring Who is CityIndex?• Leading global provider of retail tradingservices, including CFDs, margined FXand spread betting StreamBase for Pricing and Monitoring• Real-time access to prices• Rapid application development anddeployment Why StreamBase?• Rapid time-to-market• High performance• Ease-of-useWatch what Toby Young has to sayabout StreamBase(Requires 2D barcode scanner or click to view)
  19. 19. 19InternetHadoopMRMRMRMRMRMRMRMRMRMRMRMRMap ReduceStreamBase CEPMonitoringAutomationFilteringAlertingReal-Time ApplicationsData WarehouseHadoop FlumeMySQLHDFSAnalyticAnalyticAnalyticAnalyticOperational MonitoringLiveViewHistorical AnalyticsAnalystsOperational StaffFraud AnalystsFraud detectionFraud, SecurityManagement ControlSite MetricsBusinessIntelligenceReal Time Big Data: Virtual Goods Phishing and Fraud
  20. 20. 20Operational Visibility for Real Time AnalyticsStreamBase LiveView
  21. 21. 21Example End User ExperienceReal-Time GraphsActive TablesAd-hoc queryNotificationsEnd-user configurability
  22. 22. 22StreamBase LV Use Case - Real-Time Risk ManagementReal-time exposure by asset classReal-timetransaction detailReal-time exposure by desk
  23. 23. 23EventFlow and R: Pre-Deal Real Time VAR ChecksNormalizeIntegrate with RSimulationsLoop each tradethrough all scenariosCalculate per-trade,per-scenario RiskCalculate Risk:95%,99%,X%Accept/Rejectproposed trade,minimize risk tocounterpartyAggregateConfigure parametersfor back testingConnect
  24. 24. 24StreamBase LiveView Use Case - Real-Time TCAReal-Time TCA AlertsLive Orders, TCA, FIXWhich orders areslipping?
  25. 25. 25LiveView Server – Data Source and Query ClientsMessage BusMarket DataNewsOrders &ExecutionsDataLiveView ServerSnapshotResultsContinuous Query ProcessorQueryContinuousAlertingActive Tables Active TablesContinuousUpdatesSources Clients Maintain Data Image Respond to ContinuousQueries Materialize analysis Pre-aggregate data Manage alerts
  26. 26. 26Clients: Desktop, Web, Excel, Mobile, Custom .NET, etc
  27. 27. 27GAIN Capital: FX Pricing, Risk, Surveillance and Compliance GAIN Capital• Improved risk management, surveillance and compliancesystem• Rapid applications design, development and deployment• Faster execution GAIN’s developers• Intuitive development environment• Library of application examples and help files• Responsive support team GAIN’s Forex.com clients• Tighter spreads for customers through improved pricing andmarket making StreamBase/GAIN Institutional Clients• Rapid FX applications design, development and deployment• Allow developers to focus on strategies and not the plumbing• High frequency trading connectivity and order execution toall FX liquidity providers including tier 1 and tier 2 banks“StreamBase Event Flowprogramming language is50% faster than traditionalprogramming techniques.”Andrew Haines,CIO, GAIN CapitalWatch what Andrew Haines has to sayabout StreamBase on TABB Forum(Requires 2D barcode scanner or click)
  28. 28. 28Summary and Next Steps Once you have accumulated, analyzed, and accessed big datayou will have insight Operationalizing insight on real time data is Event Processing StreamBase’s Event Processing Platform is best of breed• Natural and powerful EventFlow authoring environment• Extensible integrations with analytics• High performance deployment server• Rich portfolio of connectivity with focus on capital markets• LiveView for operational monitoring and alerting Download and try out http://streambase.tibco.comQuestions?