Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting started with Azure Event Hubs and Stream Analytics services

131 views

Published on

The total amount of data in the world almost doubles every 2 years. Storing data for offline processing is no longer a viable business model. In the past few years, new technologies for real-time data processing emerged. Microsoft Azure offers a comprehensive set of tools to ingest and process data in motion. In this presentation we will go over and learn how to collect data from devices, how to process data in real time using Azure Stream Analytic jobs, and how to produce and handle actionable insights.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Getting started with Azure Event Hubs and Stream Analytics services

  1. 1. Getting started with Azure Event Hubs and Stream Analytic ServicesVladimir Bychkov
  2. 2. Who We Are CUSTOM SOFTWARE DEVELOPERS 200 employees (180 engineers) Certified Amazon - AWS Technology Partner Scientists, mathematicians, & engineers Certified Azure/Microsoft Partner of the Year 2014 Open Architecture Washington, DC Headquarters Business Intelligence & Big Data Cloud Solutions, IoT & DevOps Identity & Access Management System Integration & CRM Applications Content Management & Enterprise Portals Custom Mobile Solutions
  3. 3. Join our Team www.eastbanctech.com/careers/vacancies
  4. 4. Processing streaming data – use cases • Automotive - fleet tracking, operations telemetry • Medical - vital signs, fitness and health monitoring • Meteorology - weather and environmental data • Software - applications performance and instrumentation data • Home automation – energy efficiency, home security • Agriculture/farming – crops and cattle monitoring
  5. 5. Connected cow
  6. 6. End-to-end stream processing architecture
  7. 7. Producing sensor data with Raspberry Pi
  8. 8. Azure Service Bus Source: https://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307
  9. 9. Azure Event Hub Cloud Services Storage & Analytics Custom Code & 3rd Party Services Web/Mobile User Interfaces Integration Services Event Hub - Hyper Scale - - Fully Managed - - Interoperable - - Secure - - Cost Effective -
  10. 10. Azure Event Hub (cont.)
  11. 11. DEMO • Create Azure Event Hub • Read events using console windows app • Send events using Python script on Raspberry Pi
  12. 12. Azure Stream Analytics • Managed service in Azure Cloud • Build for hyper-scale • Cost effective • Real-time data processing • Developer productivity
  13. 13. Azure Stream Analytics: Basic Job Topology
  14. 14. Inputs • Data Stream • Reference data • Supported data formats: • JSON • CSV • Apache Avro (binary JSON)
  15. 15. Outputs • Azure Tables or Blob • SQL Database • Event Hub • Service Bus Queue/Topic • PowerBI dataset • DocumentDB
  16. 16. Query - SQL like language with built in temporal semantics
  17. 17. Azure Stream Analytics: Grouping data - Windowing Tumbling window Aggregate per time interval Hopping window Schedule overlapping windows Sliding window Window constantly re-evaluated
  18. 18. DEMO • Create Azure Stream Analytics job • Add input/output to job topology • Write pass-through query to archive events to Azure Table • Write aggregate query to save temp averages only to Azure Table • Web-dashboard demo
  19. 19. ANATOMY OF A H4CK3R ATTACK
  20. 20. DEMO • Review CurcuitBreaker app • Review CurcuitBreaker ASA query • Run real-time Dashboard • Hack into Steel Mountain!
  21. 21. Q&A
  22. 22. THANK YOU Vladimir Bychkov Technical Team Lead vbychkov@eastbanctech.com eastbanctech.com

×