Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Sql Stream Intro
Similar to Sql Stream Intro (20)
Sql Stream Intro
- 1. Relational Streaming, Big Data, and “What is” not “What was” The leading standards-based platform for streaming Big Data applications Compelling new technology for OEMs, SIs, and ISVs. “Query the Future” Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 2. SQLstream Corporate Facts » Corporate » Founded 2003 » Experienced team of industry and IT professionals » Headquarters in San Francisco, CA » The Pioneer in Streaming Big Data » 8 relational streaming patents (4 granted, 4 pending) » Market launch (GA) in January 2009 » Numerous partners and customers » Only streaming platform based on ISO/ANSI SQL 2 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 3. Our Mission Transforming real-time data into real-time value Transform sensor, system & service data into real-time intelligence Instantly stream out real-time answers to ad-hoc queries Analyze & react to infinite data flows using familiar, standard SQL “SQLstream helps to translate high volumes of streaming data into information that can actually be used and acted upon by mere business mortals.” Michael Dortch, BI in Action 3 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 4. The Real-time Data Challenge Data Explosion Too costly to analyze voluminous real-time data Business Agility Too slow to respond to new requirements Efficiency Too difficult to build & maintain real-time apps 4 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 5. The Real-time Data Challenge Data Explosion Too costly to analyze voluminous real-time data. SQLstream slashes costs of real-time analysis. Business Agility Too slow to respond to new requirements. SQLstream can be pushed new apps without downtime. Efficiency Too difficult to build & maintain real-time apps. SQLstream is appropriate for “mere business mortals” 5 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 6. Examples of Real-time Data Intelligence Headaches… Content Service-level Transaction Performance Security Monitoring Monitoring Monitoring Monitoring Monitoring … Costing Businesses Time, Money, Complexity, … Monitoring == “What happened?” 6 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 7. Relational Streaming using SQLstream delivers “the NOW” Content Service-level Transaction Performance Security Alerting Alerting Alerting Alerting Altering, … Streaming Analytics Steaming Event Streaming Continuous and Aggregation Correlation Alerts & Alarms ETL Alerting == “What is happening?!” 7 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 8. Tuple Processing: Relational Streaming complements Hadoop Hadoop: data chunking coarse-grained dataflow “What happened?” SQLstream: Directed Acyclic Graphs of fine-grained dataflow “What’s happening now?!” 8 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 9. SQLstream Products SQLstream s-Server Core streaming SQL platform and integration layer SQLstream s-Cloud Cloud deployments with integration to Big Data and Hadoop environments (e.g., Google BigQuery, HBase) SQLstream s-Analyzer Integrated reporting and graphical query builder 9 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 10. SQLstream – No new language to learn » SQLstream uses standards-based SQL » Proven scalability » Rapid application development with readily available SQL skills » Apply to existing data warehouse & SQL investments » ANSI/ISO Compliant Streaming SQL example CREATE VIEW compliant_orders AS SELECT STREAM * Generates a FROM orders OVER sla JOIN shipments continuous stream of ON orders.id = shipments.orderid “New York” orders WHERE city = 'New York' shipping within a WINDOW sla AS (RANGE INTERVAL '1' HOUR PRECEDING) service level of 1 hour 10 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 11. SQLstream and S3 Data (Sensor x System x Service) » Sensor Data examples: » Vehicle, GPS and transportation sensors » M2M sensor networks » Smart Energy sensors » System Data examples: » Log file processing for real-time Security, Compliance, Fraud » Cloud performance monitoring » Service Level Monitoring » Service Data examples: » SMS analysis, CDRs for billing, Fraud detection » Real-time pricing and promotion for eCommerce » Active Internet (real-time context-dependent content) 11 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 12. Customer Example: Mozilla – Powered by SQLstream » Mozilla Firefox 4 – Real-time Download Monitor » Continuous processing of download requests » Real-time integration with Hadoop and HBase 12 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 13. Real-world Example: Real-time Traffic Analytics 13 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 14. Summary » SQLstream provides relational streaming » Breakthrough in data driven parallel processing » Extends SQL beyond analytics into ETL, alerting and monitoring » Shares and reuses data streams across apps in real-time. » We are the only ANSI/ISO SQL streaming platform » Mature and proven technology » Truly SQL standards compliant, not “SQL-like” » We own the key streaming patents, both granted and pending. » Why do our customers care? » We unlock the potential of real-time service, system and sensor data » The technology is installed, working, and shipping » We deliver the ability to answer “What is happening NOW” 14 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 15. Partnering with SQLstream – How to get started Completed Operational Pilot Deployment Application » Product & Services delivered in the context of a pilot project » Solution & Design Consulting » Application Development & Consulting » Support through your application development cycle » SQLstream engineers allocated at appropriate cadence and levels » Operational Support » SQLstream provides comprehensive partner knowledge base » Operational support to back up your “level 3” questions and issues “We think SQLstream's focus differentiates it from behemoths, including IBM, that are now moving into the real-time-analytic fray. SQL developers like it – a different audience to others with real-time BI wares” Krishna Roy - 451 Group - Market Insight Service 15 Copyright © 2012 – Proprietary and Confidential Information of SQLstream Inc.
- 16. For further information: Chris Clabaugh VP Business Development ccc@sqlstream.com +1 650 762 5143
Editor's Notes
- This is an introductory presentation to the notion of Relational Streaming, its tie to “Big Data”, and how SQLstream changes the “What happened?” results of Big Data, to “What is happening ?!” through the power to “Query the Future”.
- SQLstream was founded in 2003 with a vision of transforming the way streaming service, system and sensor data are processed.SQLstream is headquartered in San Francisco with agents and partners in EMEA and APAC.Fontinalis Partners led our most recent funding round; Fontinalis is a Venture Capital firm with Private Equity roots and limited partners comprising some of the largest industrial families around the world.The company left stealth mode in 2008 after 1.2M lines of code to deliver the world’s first SQL standards compliant stream-to-business platform.We have 8 patents of which 4 are already granted, and 4 are pending.
- Put simply, there is a new real-time data challenge facing many enterprises today. The common solution is to “store then query”- archive everything, then hope your queries can identify crucial trends in a timeframe of relevance.The data volumes are exploding exponentially – making it too costly to analyze with conventional technologies where you have to store all of the data, even if most of the data might have a very limited ‘shelf life’. The costs come from needing specialized data warehousing systems to handle such large and ever growing data volumes, and those license fees are almost always based on the volume of data stored. So why store everything if you are only really concerned with the results of analyses? At the same time, businesses are having to become nimbler and more agile. They need to consume and analyze data faster than their competitors, and fast enough to hold the attention of their customers while they are interacting with their product, service, systems or personnel.
- Using SQLstream, an organization can now perform low-latency queries on data in flight, delivering immediate low-latency resultsNew queries or modified queries can be pushed into the system without service outageSQLstream can also act as a real-time filter to reduce the volume of obtuse data stored in costly data archivesAnd by the use of standard SQL statements, enables your development staff or partners to create value at a significantly reduced development cost
- SQLstream eliminates this pain by combining the features of relational databases with enterprise service bus concepts – in a low-latency, in-memory implementation.SQLstream’s stream-to-business platform allows you to easily and incrementally plug-in new sources of data and similarly to plug-in new destinations for results. Each consuming application creates a relational VIEW of the data that it needs to see. This VIEW is turned into a continuously executing real-time SQL query acting upon the living streaming data sources (including sources that represent the results of other streaming SQL queries) in order to deliver the required stream of results that the business requires. These SQL queries are based on 2003 ANSI and ISO SQL standards. They are ACTIVE queries in that they executive against the live streaming data while the data are still in flight – WITHOUT HAVING TO STORE THE DATA FIRST!SQLstream provides an ACTIVE relational database where hundreds of SQL VIEWS and QUERIES are continuously combing, indexing, aggregating and correlating massive volumes of data from hundreds or thousands of streaming sources ALL WITHOUT STORING THE DATA FIRST. SQLstream assembles the record streams that each application has requested, so that all of the streaming source data can be reused and repurposed according to need - in real-time.
- SQLstream compliments Hadoop / Map-Reduce solutions to answer both “What happened?” and “What is happening?”.Hadoop, with its phased approached, handles a lot of superscalar executions of 64MB chunks of data. When each phase is complete, and all the records are assembled and sorted from the chunks of records feeding upstream, the next phase can commence.SQLstream implements the Relational Streaming model. This can be visualized as a Directed Acyclic Graph of relational operators operating on streams of tuples synchronized around (normally) timestamps.This is very similar to an electronic logic circuit. The relational operators are the logic gates. The tuples are the binary data signals. Both propagate answers as soon as the results are available with minimal latency. Both utilize dafaflow execution, time synchronization where appropriate, and both offer both pipelining parallelism and superscalar parallelism.
- SQL was developed to elegant process massive quantities of stored data. Using SQLstream, it works just as well in processing massive volumes of streaming data.It has proven scalability and sophisticated query optimizers.It enables rapid application development – a few SQL rules have immense power – and the SQL skills are readily available in the marketplace.It allows easy migration of SQL queries and logic to and from databases and data warehouses and SQLstream.This query example shows how one would use SQLstream to find orders from New York that ship within one hour. The keyword STREAM is used to maintain standards compatibility as without it the query would return a table not a stream of results-- results that could now continue ad infinitum.
- SQLstream operates on streams of data, which we term “S3 Data” – for Sensors, System and Service Data.The Sensor Data category includes Vehicular sensor data, GPS data, RFID data, transportation data, engine data. Machine-to-machine networks, smart energy, manufacturing sensors and so forth all emit this type of sensor data.The System Data category – some call this Machine Data – covers log files emitted from applications and server, and can be used for real-time security, fraud and compliance detection. Also cloud computing monitoring, service level monitoring, and so forth.Finally the Service Data category includes all manner of service data, ranging from SMS Text messages, Call Detail Records (CDRs), fraud logic alerts, real-time pricing and promotion for the “Active Internet” – context-dependent content streamed from low-latency relational streaming output.
- We are one of only two closed source solutions within Mozilla. We power Mozilla’s real-time analytics. Note the ”Powered by SQLstream” logo on the bottom right of their web display. See http://gigaom.com/cloud/dataflow-sqlstream/ for more informationThe SQLstream application processes all of the log-files from Mozilla download servers in real-time, parsing the files, streaming the data, mapping IP addresses to Longitude and Latitude, finding the nearest town, city or village and performing a range of analytics on the streams to feed a Hadoop cluster (Hbase) for displaying historical information complemented by SQLstream’s real-time analytics.
- Another SQLstream customer is a division of the Australian Government. SQLstream monitors the vehicular traffic on their road systems by processing vehicle and road sensors data streams, and in real-time determining congestion, instantaneous average speeds, and much more.The application monitors road traffic flows down to a granularity of 10 meter segments. The results are then displayed on Google Earth and Google Maps underlays.The customer has many more applications in planning stages, including real-time metrics, and integration of data from all modes of transportation – continuously and in real-time, using SQLstream as the relational streaming engine.
- In conclusion, SQLstream offers a fundamental breakthrough in real-time data management.Customers and partners can create complex applications processing massive data volumes and easily reuse the results across multiple applications and share streaming data sources in real-time across the enterprise.The delivers new insights into their existing voluminous but previously untamed data, with dramatically lower development and ownership costs.SQLstream is a key enabler for the real-time enterprise- or as we say “Query the Future”
- Customers and Partners typically start with a small pilot and rapidly move on to develop and deploy key streaming applicationsGiven the entire application is written in standards-based SQL, it is normally easy for the customer or the partner’s developers to take ownershipSQLstream or SQLstream partners are comfortable working with fixed price deliverables and projects where that makes senseWe have a 5 day certification course for Partners, and then generally mentor a Partner through the first project