The document discusses Azure Stream Analytics, a fully managed stream processing service. It describes how Stream Analytics allows processing of streaming data using simple SQL queries. Examples are provided of using temporal windows like tumbling, hopping and sliding windows to analyze streaming data from scenarios like toll stations. Components of a Stream Analytics application are also outlined.

1. 22 de Janeiro Microsoft Lisbon Experience
The Power of Now!
Rui Quintino | DevScope
22 Jan 2015
rui.quintino@devscope.net
rquintino.wordpress.com
twitter.com/rquintino
Azure Stream Analytics

2. The traditional data warehouse

3. Modern data warehouse

4. What are customers wanting to do?

5. Canonical scenarios for real-time processing

6. KUKA Systems Group created an IoT-powered plant that
centrally connects and monitors hundreds of robots.

7. Introducing Azure Stream Analytics

9. Procure HW Infrastructure and setup
Code for ingress, processing and egress
Plan for resiliency, such as HW failures
Design solution
Build Monitoring and Troubleshooting

11. • Built-in monitoring
– View your system’s performance
at a glance
– Help you find the cost-optimal
way of deployment

12. 3 lines of code in Stream Analytics
Thousand lines of code in other solutions
• Only SQL queries needed
– Developers uses declarative
SQL commands
– Some functions take several
lines of code versus thousands
from other solutions

13. • Implement temporal functions
• Manage out-of-order events
• Manage actions on late events

14. End-to-end stream processing architecture

16. Sample Scenario – Toll Station
TollId EntryTime
License
Plate
State Make Model Type Weight
1 2014-10-25T19:33:30.0000000Z JNB7001 NY Honda CRV 1 3010
1 2014-10-25T19:33:31.0000000Z YXZ 1001 NY Toyota Camry 2 3020
3 2014-10-25T19:33:32.0000000Z ABC 1004 CT Ford Taurus 2 3800
2 2014-10-25T19:33:33.0000000Z XYZ 1003 CT Toyota Corolla 2 2900
1 2014-10-25T19:33:34.0000000Z BNJ 1007 NY Honda CRV 1 3400
2 2014-10-25T19:33:35.0000000Z CDE 1007 NJ Toyota 4x4 1 3800
… … … … … … … …
EntryStream - Data about vehicles entering toll stations
TollId ExitTime LicensePlate
1 2014-10-25T19:33:40.0000000Z JNB7001
1 2014-10-25T19:33:41.0000000Z YXZ 1001
3 2014-10-25T19:33:42.0000000Z ABC 1004
2 2014-10-25T19:33:43.0000000Z XYZ 1003
… … …
ExitStream - Data about cars leaving toll stations
LicensePlate RegistartionId Expired
SVT 6023 285429838 1
XLZ 3463 362715656 0
QMZ 1273 876133137 1
RIV 8632 992711956 0
… … ….
ReferenceData - Commercial vehicle registration data

17. Application Components
Components of an Azure Stream Analytics Application
Azure SQL DB
Azure Event Hubs
Azure Blob StorageAzure Blob Storage
Azure Event Hubs
Reference Data
Query runs continuously against incoming stream of events
Events
Have a defined schema and
are temporal (sequenced in
time)

18. Temporal Windows
• Tumbling Windows
– Repeating, non-overlapping, fixed interval windows
• Hopping Windows
– Generic window, overlapping, fixed size
• Sliding Windows
– Slides by an epsilon and produces output at the occurrence of an event

19. Tumbling Window
SELECT System.TimeStamp AS OutTime, TollId,
COUNT (*)
FROM Input TIMESTAMP BY EntryTime
GROUP BY TollId, TumblingWindow(minute,5)

20. Hopping Windows
SELECT System.TimeStamp AS OutTime, TollId,
COUNT (*)
FROM Input TIMESTAMP BY EntryTime
GROUP BY TollId, HoppingWindow(minute, 10 , 5)

21. Sliding Windows
SELECT System.TimeStamp AS OutTime, TollId, COUNT (*)
FROM Input TIMESTAMP BY EntryTime
GROUP BY TollId, SlidingWindow(minute, 3)
HAVING Count(*) > 3
Finds all toll booths which have served more than 3 vehicle in the last 3 minutes

23. Explore the latest in SQL Server
Visit www.microsoft.com/sql to learn more about our latest innovations
Evaluate SQL Server 2014
Get hands on – visit Microsoft’s TechNet evaluation center and
evaluate SQL Server 2014 today!
Evaluate Windows Server 2012 R2
Get hands on – visit Microsoft’s TechNet evaluation center and
evaluate Windows Server 2102 R2 today!Microsoft Azure