The total amount of data in the world almost doubles every 2 years. Storing data for offline processing is no longer a viable business model. In the past few years, new technologies for real-time data processing emerged. Microsoft Azure offers a comprehensive set of tools to ingest and process data in motion. In this presentation we will go over and learn how to collect data from devices, how to process data in real time using Azure Stream Analytic jobs, and how to produce and handle actionable insights.

1. Getting started with Azure Event Hubs
and Stream Analytic ServicesVladimir Bychkov

2. Who We Are
CUSTOM SOFTWARE DEVELOPERS
200 employees (180 engineers) Certified Amazon - AWS Technology Partner
Scientists, mathematicians, & engineers Certified Azure/Microsoft Partner of the Year 2014
Open Architecture Washington, DC Headquarters
Business Intelligence & Big
Data
Cloud Solutions, IoT &
DevOps
Identity & Access
Management
System Integration &
CRM Applications
Content Management &
Enterprise Portals
Custom Mobile Solutions

3. Join our Team www.eastbanctech.com/careers/vacancies

5. Processing streaming data – use cases
• Automotive - fleet tracking, operations telemetry
• Medical - vital signs, fitness and health monitoring
• Meteorology - weather and environmental data
• Software - applications performance and instrumentation data
• Home automation – energy efficiency, home security
• Agriculture/farming – crops and cattle monitoring

6. Connected cow

7. End-to-end stream processing architecture

8. Producing sensor data with Raspberry Pi

9. Azure Service Bus
Source: https://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307

10. Azure Event Hub
Cloud Services
Storage &
Analytics
Custom Code
& 3rd Party
Services
Web/Mobile
User
Interfaces
Integration
Services
Event Hub
- Hyper Scale -
- Fully Managed -
- Interoperable -
- Secure -
- Cost Effective -

11. Azure Event Hub (cont.)

12. DEMO
• Create Azure Event Hub
• Read events using console windows app
• Send events using Python script on Raspberry Pi

13. Azure Stream Analytics
• Managed service in Azure Cloud
• Build for hyper-scale
• Cost effective
• Real-time data processing
• Developer productivity

14. Azure Stream Analytics: Basic Job Topology

15. Inputs
• Data Stream
• Reference data
• Supported data formats:
• JSON
• CSV
• Apache Avro (binary JSON)

16. Outputs
• Azure Tables or Blob
• SQL Database
• Event Hub
• Service Bus Queue/Topic
• PowerBI dataset
• DocumentDB

17. Query - SQL like language with built in temporal semantics

18. Azure Stream Analytics: Grouping data - Windowing
Tumbling window
Aggregate per time interval
Hopping window
Schedule overlapping windows
Sliding window
Window constantly re-evaluated

19. DEMO
• Create Azure Stream Analytics job
• Add input/output to job topology
• Write pass-through query to archive events to Azure Table
• Write aggregate query to save temp averages only to Azure Table
• Web-dashboard demo

20. ANATOMY OF A H4CK3R ATTACK

21. DEMO
• Review CurcuitBreaker app
• Review CurcuitBreaker ASA query
• Run real-time Dashboard
• Hack into Steel Mountain!

22. Q&A

23. THANK YOU
Vladimir Bychkov
Technical Team Lead
vbychkov@eastbanctech.com
eastbanctech.com