REAL-TIME AND BIG DATA! 
The perfect union for large-scale sensor 
network management 
Sensors Expo 2013: Solutions For Ma...
About SQLstream 
Distributed 
stream 
processing 
plaSorm 
for 
streaming 
analy7cs 
and 
opera7onal 
intelligence 
from 
...
Big Data Explosion | CHALLENGES! 
Analytic processing at massive scale 
§ COLLECTION: Massive numbers of sensors 
3Copyri...
The “Internet-of-Things” Explosion | SOURCES 
Everything is becoming instrumented and connected 
Environmental Monitoring ...
Real-Time Applications | INDICATORS 
§ Data arriving continuously or in irregular intervals 
§ Sensors 
& 
GPS 
§ Log 
...
Real-time Sensor Applications 
§ Real-time health diagnostics, monitoring and alerting 
§ Asset tracking and location-ba...
Query-then-Store vs. Store-then-Query 
(Stream processing vs. Stored data processing) 
Capability 
Real-­‐Time 
System 
Tr...
Real-Time Analytics vs. Historical Analytics 
CATEGORY 
REAL-­‐TIME 
OPERATIONAL 
INTELLIGENCE 
BUSINESS 
INTELLIGENCE 
Ve...
Real-Time Integrated Vehicle Health Management 
§ Capabilities 
§ Health monitoring of key vehicle systems 
§ Frequent ...
Vehicle Telematics & Health Monitoring 
§ Real-time vehicle diagnostics and alerting 
§ Pre-emptive vehicle service aler...
Insurance & Sensor Monitoring 
§ Insurance Programs 
§ Breaks on insurance for good drivers 
§ Immediate “journey” repo...
Generating Operational Intelligence | Process 
Internet 
of 
Things 
& 
Sensors 
§ Smart 
City 
§ Transporta7on 
§ Indu...
Streaming SQL Example: Bollinger Bands 
SELECT STREAM ROWTIME, VIN, numErrorsLastMinute 
FROM ( SELECT STREAM ROWTIME, VIN...
Summary 
§ Complementary to existing Enterprise systems 
§ Does not replace existing Enterprise and business intelligenc...
Stream Processing as the Enabler for the Internet of Things | Sensors Expo
Upcoming SlideShare
Loading in...5
×

Stream Processing as the Enabler for the Internet of Things | Sensors Expo

4,009

Published on

SQLstream Blaze (http://www.sqlstream.com) is a stream processing platform for streaming analytics and real-time actions for operational intelligence and the Internet of Things. This presentation, presented as sensors expo, discusses the increase in data from sources such as the industrial internet, smart city, intelligent transportation, home automation and smart energy, and how stream processing turns real-time sensor data into actionable intelligence before the data are stored.

Sensors Expo is the leading industry event for the types of intelligent sensor-integrated systems that are driving the next generation Internet of Everything, Industrial Internet, telematics and Machine-to-Machine services. As a key sponsor in the Big Data and Wireless Systems pavilion, and speaker in the Big Data track, the event is a litmus test for the current state of the sensor-related systems and the Internet of Things.
Drinking from the Sensor Firehose

In addition, the talk discusses stream processing for the Intelligent Edge - and how the the cost of sensor transmission backhaul requires a more sophisticated architecture. SQLstream's agent framework enables intelligent data collection agents to be deployed as close to the data source as possible. Intelligence Agents offer basic filtering, aggregation and analytics capability at the edge of the network. Depending on the information content in the data, the volume and therefore cost of real-time transmission can be reduced significantly. The architecture supports two logical connectivity networks, one for the data transmission, and a command and control for remote agent management.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,009
On Slideshare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Stream Processing as the Enabler for the Internet of Things | Sensors Expo "

  1. 1. REAL-TIME AND BIG DATA! The perfect union for large-scale sensor network management Sensors Expo 2013: Solutions For Massive, Real-time, Continuous Big Data Applications Copyright © 2014 – Proprietary and Confiden7al Informa7on of SQLstream Inc.
  2. 2. About SQLstream Distributed stream processing plaSorm for streaming analy7cs and opera7onal intelligence from high-­‐velocity log and sensor machine data. facts § Launched 2009 § Over 1.5M lines of code § Funded by Fon7nalis Partners capabili7es § Process unstructured and structured machine data § Accelerate and extend Hadoop & RDBMS § Open, standards-­‐based plaSorm based on SQL 2Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc. differen7ators § Only true streaming data management plaSorm § Only true standard SQL streaming engine § Covered by 7 broad patents for stream processing
  3. 3. Big Data Explosion | CHALLENGES! Analytic processing at massive scale § COLLECTION: Massive numbers of sensors 3Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc. § Thousands, tens-of-thousands, hundreds-of-thousands, millions+ of sources § Creation of multiple siloes § SCALE: Continuous “fire-hose” § Massive data volumes and velocities § Information overload § PROCESSING: Actionable intelligence § Data “decision” value can drop exponentially with passing seconds § Data velocity and rate-of-change critical for decisions § Lack of real-time processing BIG DATA MACHINE DATA HIGH-­‐VELOCITY, LOW-­‐LATENCY INTELLIGENCE
  4. 4. The “Internet-of-Things” Explosion | SOURCES Everything is becoming instrumented and connected Environmental Monitoring Location-based services Industrial Internet Smart Grid Telematics Home Automation 4Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  5. 5. Real-Time Applications | INDICATORS § Data arriving continuously or in irregular intervals § Sensors & GPS § Log file data § Telema7cs and M2M from cars, machinery, hardware, people etc. § Anything that requires immediate attention § Alerts that must be acted on now, not in minutes, hours or days § Any process that struggles with batch window latency § Service level agreements (SLAs) with narrow 7me-­‐ac7on windows § Batch queries that are con7nuously re-­‐executed § Data cleansing, transforma7on and loading (ETL) § Massive scale § Data flows too large to store -­‐ must be processed con7nuously in-­‐flight before storing 5Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  6. 6. Real-time Sensor Applications § Real-time health diagnostics, monitoring and alerting § Asset tracking and location-based services § Location- and activity-based advertising § Safety, regulatory and compliance § Customer loyalty solutions § Fleet management solutions § Usage-based applications: insurance, tolling, traffic, etc. § Emergency services and responder alerting 6Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  7. 7. Query-then-Store vs. Store-then-Query (Stream processing vs. Stored data processing) Capability Real-­‐Time System Tradi3onal Database / Hadoop Analy3c Processing § Con7nuous queries § Incremental calcula7ons 7Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc. § Batch queries § Expensive, batch recalcula7ons Latency from event to ac3on § Milliseconds to seconds § Tens-­‐of-­‐minutes, hours, days § Significant latency Rate of Change § Monitor data velocity and complex rates-­‐of-­‐change § Sta7c snapshot of point-­‐in-­‐7me Scalability & Hardware Requirements § Millions to tens-­‐of-­‐millions of events/second § Minimal hardware requirements § Massive data inflow and con7nual recalcula7on limits analy7c depth § Significant hardware requirements Itera3ve Analy3cs § Cascading analy7cs within and across events -­‐ pipelining § Requires myriad views, temporary tables and custom applica7on code
  8. 8. Real-Time Analytics vs. Historical Analytics CATEGORY REAL-­‐TIME OPERATIONAL INTELLIGENCE BUSINESS INTELLIGENCE Vehicle Diagnos3cs Is my vehicle about to fail? How many vehicles failed in the last 6 months in 8Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc. the state of New York? Sensors ALERT -­‐ The airbag just deployed! What is the current loca7on, vehicle and driver informa7on? How many vehicles had airbags that deployed in the past 2 weeks by vehicle type? Telema3cs & Asset Tracking Where is the vehicle right now? Where has it been and how is it moving on current journey? What was the average speed of all vehicles on various road segments on prior journeys? Adver3sing What is a relevant ad to place based on current loca7on, current ac7vity and historical paherns? What is a relevant ad to place based on historical paherns? Security Is an intrusion ahempt underway? Where have intrusions happened in the past and from what sources? Fraud/risk Is the current transac7on fraudulent? How many fraudulent transac7ons occurred at online electronics retailers during the past 3 months?
  9. 9. Real-Time Integrated Vehicle Health Management § Capabilities § Health monitoring of key vehicle systems § Frequent monitoring when exception system events occur § Real-time panic alerts § Objectives § Reduce vehicle walkaway events by 50% § Improve customer satisfaction § Reduce warranty costs § Future: Potential “paid for” service 9Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  10. 10. Vehicle Telematics & Health Monitoring § Real-time vehicle diagnostics and alerting § Pre-emptive vehicle service alerts § Critical event monitoring and alerting § Asset tracking for fleet management § Track vehicles, shipments and transit ETA § Real-time driving log § Safety, compliance and alerting § Dynamic road tolling § Assess tolls based on weight, miles, roads used 10Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  11. 11. Insurance & Sensor Monitoring § Insurance Programs § Breaks on insurance for good drivers § Immediate “journey” reports to smartphones § Real-time analytics and alerting for safety, compliance and monitoring § Geo-fencing § Alerts to smartphones when drivers travel outside prescribed boundaries 11Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  12. 12. Generating Operational Intelligence | Process Internet of Things & Sensors § Smart City § Transporta7on § Industrial Internet § Telema7cs § Smart Energy Opera3onal Intelligence & Logs § Security Intelligence § Servers & Applica7ons § Networks & Services Streaming Enterprise Hadoop and Data Warehouse integra7on for joining streaming and stored trend data Machine Data Social Media & TwiQer 12Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc. Real-­‐3me Ac3ons Real-­‐3me Dashboards Con3nuous Data Warehouse Updates Automated Ac3ons
  13. 13. Streaming SQL Example: Bollinger Bands SELECT STREAM ROWTIME, VIN, numErrorsLastMinute FROM ( SELECT STREAM ROWTIME, VIN, numErrorsLastMinute, AVG(numErrorsLastMinute) OVER lastMinute AS avgErrorsPerMinute, STDDEV(numErrorsLastMinute) OVER lastMinute AS stdDevErrorsPerMinute FROM ServiceRequestsPerMinute WINDOW lastMinute AS (PARTITION BY VIN RANGE INTERVAL ‘1’ MINUTE PRECEDING) ) AS S WHERE S.numErrorsLastMinute > S.avgErrorsPerMinute + 2 * S.stdDevErrorsPerMinute; 13Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  14. 14. Summary § Complementary to existing Enterprise systems § Does not replace existing Enterprise and business intelligence systems, but provides powerful, complementary real-time operational intelligence capabilities § Massive scalability for high volume, low latency § Ability to provide streaming analytics and real-time alerting on massive volumes of sensor data with minimal hardware infrastructure requirements § Rapid time to value § Utilize existing SQL and data analytics skills for real-time data applications 14Copyright © 2014 – Proprietary and Confidential Information of SQLstream Inc.
  15. 15. Download SQLstream Blaze for free!! www.sqlstream.com/downloads Contact us: Email: inquiries@sqlstream.com Call: +1 877-571-5775 Copyright © 2014 – Proprietary and Confiden7al Informa7on of SQLstream Inc.

×