SlideShare a Scribd company logo
1 of 21
Getting started with Azure Event Hubs
and Stream Analytic Services
Processing streaming data – use cases
• Automotive - fleet tracking, operations telemetry
• Medical - vital signs, fitness and health monitoring
• Meteorology - weather and environmental data
• Software - applications performance and instrumentation data
• Home automation – energy efficiency, home security
• Agriculture/farming – crops and cattle monitoring
Connected cow
End-to-end stream processing architecture
Producing sensor data with Raspberry Pi
Azure Service Bus
Source: https://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307
Azure Event Hub
Cloud Services
Storage &
Analytics
Custom Code
& 3rd Party
Services
Web/Mobile
User
Interfaces
Integration
Services
Event Hub
- Hyper Scale -
- Fully Managed -
- Interoperable -
- Secure -
- Cost Effective -
Azure Event Hub (cont.)
DEMO
• Create Azure Event Hub
• Read events using console windows app
• Send events using Python script on Raspberry Pi
Azure Stream Analytics
• Managed service in Azure Cloud
• Build for hyper-scale
• Cost effective
• Real-time data processing
• Developer productivity
Azure Stream Analytics: Basic Job Topology
Inputs
• Data Stream
• Reference data
• Supported data formats:
• JSON
• CSV
• Apache Avro (binary JSON)
Outputs
• Azure Tables or Blob
• SQL Database
• Event Hub
• Service Bus Queue/Topic
• PowerBI dataset
• DocumentDB
Query - SQL like language with built in temporal semantics
Azure Stream Analytics: Grouping data - Windowing
Tumbling window
Aggregate per time interval
Hopping window
Schedule overlapping windows
Sliding window
Window constantly re-evaluated
DEMO
• Create Azure Stream Analytics job
• Add input/output to job topology
• Write pass-through query to archive events to Azure Table
• Write aggregate query to save temp averages only to Azure Table
• Web-dashboard demo
ANATOMY OF A H4CK3R ATTACK
DEMO
• Review CurcuitBreaker app
• Review CurcuitBreaker ASA query
• Run real-time Dashboard
• Hack into Steel Mountain!
Q&A
THANK YOU
Vladimir Bychkov
Technical Team Lead
vbychkov@eastbanctech.com
eastbanctech.com

More Related Content

What's hot

Microsoft Azure Active Directory
Microsoft Azure Active DirectoryMicrosoft Azure Active Directory
Microsoft Azure Active Directory
David J Rosenthal
 

What's hot (20)

App Modernization with Microsoft Azure
App Modernization with Microsoft AzureApp Modernization with Microsoft Azure
App Modernization with Microsoft Azure
 
Event Hub & Azure Stream Analytics
Event Hub & Azure Stream AnalyticsEvent Hub & Azure Stream Analytics
Event Hub & Azure Stream Analytics
 
Azure Messaging Services #1
Azure Messaging Services #1Azure Messaging Services #1
Azure Messaging Services #1
 
Azure App Service
Azure App ServiceAzure App Service
Azure App Service
 
Microsoft Azure Technical Overview
Microsoft Azure Technical OverviewMicrosoft Azure Technical Overview
Microsoft Azure Technical Overview
 
Azure App Service Deep Dive
Azure App Service Deep DiveAzure App Service Deep Dive
Azure App Service Deep Dive
 
Azure Active Directory | Microsoft Azure Tutorial for Beginners | Azure 70-53...
Azure Active Directory | Microsoft Azure Tutorial for Beginners | Azure 70-53...Azure Active Directory | Microsoft Azure Tutorial for Beginners | Azure 70-53...
Azure Active Directory | Microsoft Azure Tutorial for Beginners | Azure 70-53...
 
Introduction to Azure IaaS
Introduction to Azure IaaSIntroduction to Azure IaaS
Introduction to Azure IaaS
 
Azure web apps
Azure web appsAzure web apps
Azure web apps
 
Azure 101
Azure 101Azure 101
Azure 101
 
Azure Active Directory - An Introduction
Azure Active Directory  - An IntroductionAzure Active Directory  - An Introduction
Azure Active Directory - An Introduction
 
Understanding Azure Disaster Recovery
Understanding Azure Disaster RecoveryUnderstanding Azure Disaster Recovery
Understanding Azure Disaster Recovery
 
Best Practices with Azure Kubernetes Services
Best Practices with Azure Kubernetes ServicesBest Practices with Azure Kubernetes Services
Best Practices with Azure Kubernetes Services
 
Azure Functions & Serverless Computing
Azure Functions & Serverless ComputingAzure Functions & Serverless Computing
Azure Functions & Serverless Computing
 
Azure Storage Services - Part 01
Azure Storage Services - Part 01Azure Storage Services - Part 01
Azure Storage Services - Part 01
 
Architect your app modernization journey with containers on Microsoft Azure
Architect your app modernization journey with containers on Microsoft AzureArchitect your app modernization journey with containers on Microsoft Azure
Architect your app modernization journey with containers on Microsoft Azure
 
Microsoft Azure Overview Infographic
Microsoft Azure Overview InfographicMicrosoft Azure Overview Infographic
Microsoft Azure Overview Infographic
 
Introducing Azure SQL Database
Introducing Azure SQL DatabaseIntroducing Azure SQL Database
Introducing Azure SQL Database
 
Azure App Service Architecture. Web Apps.
Azure App Service Architecture. Web Apps.Azure App Service Architecture. Web Apps.
Azure App Service Architecture. Web Apps.
 
Microsoft Azure Active Directory
Microsoft Azure Active DirectoryMicrosoft Azure Active Directory
Microsoft Azure Active Directory
 

Viewers also liked

Viewers also liked (20)

Stream Analytics Service in Azure
Stream Analytics Service in AzureStream Analytics Service in Azure
Stream Analytics Service in Azure
 
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
 
Azure api app métricas com application insights
Azure api app métricas com application insightsAzure api app métricas com application insights
Azure api app métricas com application insights
 
Enterprise Data Workflows with Cascading and Windows Azure HDInsight
Enterprise Data Workflows with Cascading and Windows Azure HDInsightEnterprise Data Workflows with Cascading and Windows Azure HDInsight
Enterprise Data Workflows with Cascading and Windows Azure HDInsight
 
Go Serverless with Azure Functions
Go Serverless with Azure FunctionsGo Serverless with Azure Functions
Go Serverless with Azure Functions
 
Fraud Detection using Hadoop
Fraud Detection using HadoopFraud Detection using Hadoop
Fraud Detection using Hadoop
 
Microsoft NYC 14
Microsoft NYC 14Microsoft NYC 14
Microsoft NYC 14
 
Big data streaming with Apache Spark on Azure
Big data streaming with Apache Spark on AzureBig data streaming with Apache Spark on Azure
Big data streaming with Apache Spark on Azure
 
Azure IOT
Azure IOTAzure IOT
Azure IOT
 
Belgian Windows Server 2012 Launch windows azure insights for the enterprise ...
Belgian Windows Server 2012 Launch windows azure insights for the enterprise ...Belgian Windows Server 2012 Launch windows azure insights for the enterprise ...
Belgian Windows Server 2012 Launch windows azure insights for the enterprise ...
 
Azure Stream Analytics : Analyse Data in Motion
Azure Stream Analytics  : Analyse Data in MotionAzure Stream Analytics  : Analyse Data in Motion
Azure Stream Analytics : Analyse Data in Motion
 
Software scope
Software scopeSoftware scope
Software scope
 
Going serverless
Going serverlessGoing serverless
Going serverless
 
2016-08-25 TechExeter - going serverless with Azure
2016-08-25 TechExeter - going serverless with Azure2016-08-25 TechExeter - going serverless with Azure
2016-08-25 TechExeter - going serverless with Azure
 
Azure HDInsight
Azure HDInsightAzure HDInsight
Azure HDInsight
 
Azure IoT Hub on a Toradex Colibri VF61 – Part 1 - Sending data to the cloud
Azure IoT Hub on a Toradex Colibri VF61 – Part 1 - Sending data to the cloudAzure IoT Hub on a Toradex Colibri VF61 – Part 1 - Sending data to the cloud
Azure IoT Hub on a Toradex Colibri VF61 – Part 1 - Sending data to the cloud
 
Azure functions
Azure functionsAzure functions
Azure functions
 
Open up to a better learning ecosystem
Open up to a better learning ecosystemOpen up to a better learning ecosystem
Open up to a better learning ecosystem
 
Spark on Azure HDInsight - spark meetup seattle
Spark on Azure HDInsight - spark meetup seattleSpark on Azure HDInsight - spark meetup seattle
Spark on Azure HDInsight - spark meetup seattle
 
Building big data solutions on azure
Building big data solutions on azureBuilding big data solutions on azure
Building big data solutions on azure
 

Similar to Getting started with azure event hubs and stream analytics services

Azure Overview Csco
Azure Overview CscoAzure Overview Csco
Azure Overview Csco
rajramab
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming Visualisation
Guido Schmutz
 

Similar to Getting started with azure event hubs and stream analytics services (20)

Getting started with Azure Event Hubs and Stream Analytics services
Getting started with Azure Event Hubs and Stream Analytics servicesGetting started with Azure Event Hubs and Stream Analytics services
Getting started with Azure Event Hubs and Stream Analytics services
 
Getting started with Serverless applications on Microsoft Azure
Getting started with Serverless applications on Microsoft AzureGetting started with Serverless applications on Microsoft Azure
Getting started with Serverless applications on Microsoft Azure
 
Best Practices for Streaming IoT Data with MQTT and Apache Kafka
Best Practices for Streaming IoT Data with MQTT and Apache KafkaBest Practices for Streaming IoT Data with MQTT and Apache Kafka
Best Practices for Streaming IoT Data with MQTT and Apache Kafka
 
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...
 
IBM Cloud Pak for Integration with Confluent Platform powered by Apache Kafka
IBM Cloud Pak for Integration with Confluent Platform powered by Apache KafkaIBM Cloud Pak for Integration with Confluent Platform powered by Apache Kafka
IBM Cloud Pak for Integration with Confluent Platform powered by Apache Kafka
 
Azure Overview Csco
Azure Overview CscoAzure Overview Csco
Azure Overview Csco
 
Azure Event Grid: Glue for the Internet
Azure Event Grid: Glue for the InternetAzure Event Grid: Glue for the Internet
Azure Event Grid: Glue for the Internet
 
Integrierte Anwendungsfälle - Von der einzelnen Nachricht, über zeitnahe Anal...
Integrierte Anwendungsfälle - Von der einzelnen Nachricht, über zeitnahe Anal...Integrierte Anwendungsfälle - Von der einzelnen Nachricht, über zeitnahe Anal...
Integrierte Anwendungsfälle - Von der einzelnen Nachricht, über zeitnahe Anal...
 
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
Best Practices for Streaming IoT Data with MQTT and Apache Kafka®
 
Azure IoT Suite
Azure IoT Suite Azure IoT Suite
Azure IoT Suite
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming Visualisation
 
Apache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesApache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice Architectures
 
Azure IoT Summary
Azure IoT SummaryAzure IoT Summary
Azure IoT Summary
 
Azure によるスピードレイヤの分析アーキテクチャ
Azure によるスピードレイヤの分析アーキテクチャAzure によるスピードレイヤの分析アーキテクチャ
Azure によるスピードレイヤの分析アーキテクチャ
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
UiPath 23.4 Product Release Updates
UiPath 23.4 Product Release UpdatesUiPath 23.4 Product Release Updates
UiPath 23.4 Product Release Updates
 
Streaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkStreaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache Spark
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Apidays Paris 2023 - How to Master the Lifecycle of your APIs, Ivan Frain, Su...
Apidays Paris 2023 - How to Master the Lifecycle of your APIs, Ivan Frain, Su...Apidays Paris 2023 - How to Master the Lifecycle of your APIs, Ivan Frain, Su...
Apidays Paris 2023 - How to Master the Lifecycle of your APIs, Ivan Frain, Su...
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache Kafka
 

More from EastBanc Tachnologies

EastBanc Technologies SharePoint Portfolio
EastBanc Technologies SharePoint PortfolioEastBanc Technologies SharePoint Portfolio
EastBanc Technologies SharePoint Portfolio
EastBanc Tachnologies
 
EastBanc Technologies Data Visualization/BI Portfolio
EastBanc Technologies Data Visualization/BI PortfolioEastBanc Technologies Data Visualization/BI Portfolio
EastBanc Technologies Data Visualization/BI Portfolio
EastBanc Tachnologies
 

More from EastBanc Tachnologies (14)

Unpacking .NET Core | EastBanc Technologies
Unpacking .NET Core | EastBanc TechnologiesUnpacking .NET Core | EastBanc Technologies
Unpacking .NET Core | EastBanc Technologies
 
Azure and/or AWS: How to Choose the best cloud platform for your project
Azure and/or AWS: How to Choose the best cloud platform for your projectAzure and/or AWS: How to Choose the best cloud platform for your project
Azure and/or AWS: How to Choose the best cloud platform for your project
 
Functional Programming with C#
Functional Programming with C#Functional Programming with C#
Functional Programming with C#
 
DevOps with Kubernetes
DevOps with KubernetesDevOps with Kubernetes
DevOps with Kubernetes
 
Developing Cross-Platform Web Apps with ASP.NET Core1.0
Developing Cross-Platform Web Apps with ASP.NET Core1.0Developing Cross-Platform Web Apps with ASP.NET Core1.0
Developing Cross-Platform Web Apps with ASP.NET Core1.0
 
Highlights from MS build\\2016 Conference
Highlights from MS build\\2016 ConferenceHighlights from MS build\\2016 Conference
Highlights from MS build\\2016 Conference
 
Introduction to Kotlin Language and its application to Android platform
Introduction to Kotlin Language and its application to Android platformIntroduction to Kotlin Language and its application to Android platform
Introduction to Kotlin Language and its application to Android platform
 
Estimating for Fixed Price Projects
Estimating for Fixed Price ProjectsEstimating for Fixed Price Projects
Estimating for Fixed Price Projects
 
Async Programming with C#5: Basics and Pitfalls
Async Programming with C#5: Basics and PitfallsAsync Programming with C#5: Basics and Pitfalls
Async Programming with C#5: Basics and Pitfalls
 
EastBanc Technologies US-Russian Collaboration and Innovation
EastBanc Technologies US-Russian Collaboration and InnovationEastBanc Technologies US-Russian Collaboration and Innovation
EastBanc Technologies US-Russian Collaboration and Innovation
 
EastBanc Technologies SharePoint Portfolio
EastBanc Technologies SharePoint PortfolioEastBanc Technologies SharePoint Portfolio
EastBanc Technologies SharePoint Portfolio
 
EastBanc Technologies Data Visualization/BI Portfolio
EastBanc Technologies Data Visualization/BI PortfolioEastBanc Technologies Data Visualization/BI Portfolio
EastBanc Technologies Data Visualization/BI Portfolio
 
EastBanc Technologies Portals and CMS Portfolio
EastBanc Technologies Portals and CMS PortfolioEastBanc Technologies Portals and CMS Portfolio
EastBanc Technologies Portals and CMS Portfolio
 
Cross Platform Mobile Application Development Using Xamarin and C#
Cross Platform Mobile Application Development Using Xamarin and C#Cross Platform Mobile Application Development Using Xamarin and C#
Cross Platform Mobile Application Development Using Xamarin and C#
 

Recently uploaded

Recently uploaded (20)

WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 

Getting started with azure event hubs and stream analytics services

  • 1. Getting started with Azure Event Hubs and Stream Analytic Services
  • 2.
  • 3. Processing streaming data – use cases • Automotive - fleet tracking, operations telemetry • Medical - vital signs, fitness and health monitoring • Meteorology - weather and environmental data • Software - applications performance and instrumentation data • Home automation – energy efficiency, home security • Agriculture/farming – crops and cattle monitoring
  • 6. Producing sensor data with Raspberry Pi
  • 7. Azure Service Bus Source: https://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307
  • 8. Azure Event Hub Cloud Services Storage & Analytics Custom Code & 3rd Party Services Web/Mobile User Interfaces Integration Services Event Hub - Hyper Scale - - Fully Managed - - Interoperable - - Secure - - Cost Effective -
  • 9. Azure Event Hub (cont.)
  • 10. DEMO • Create Azure Event Hub • Read events using console windows app • Send events using Python script on Raspberry Pi
  • 11. Azure Stream Analytics • Managed service in Azure Cloud • Build for hyper-scale • Cost effective • Real-time data processing • Developer productivity
  • 12. Azure Stream Analytics: Basic Job Topology
  • 13. Inputs • Data Stream • Reference data • Supported data formats: • JSON • CSV • Apache Avro (binary JSON)
  • 14. Outputs • Azure Tables or Blob • SQL Database • Event Hub • Service Bus Queue/Topic • PowerBI dataset • DocumentDB
  • 15. Query - SQL like language with built in temporal semantics
  • 16. Azure Stream Analytics: Grouping data - Windowing Tumbling window Aggregate per time interval Hopping window Schedule overlapping windows Sliding window Window constantly re-evaluated
  • 17. DEMO • Create Azure Stream Analytics job • Add input/output to job topology • Write pass-through query to archive events to Azure Table • Write aggregate query to save temp averages only to Azure Table • Web-dashboard demo
  • 18. ANATOMY OF A H4CK3R ATTACK
  • 19. DEMO • Review CurcuitBreaker app • Review CurcuitBreaker ASA query • Run real-time Dashboard • Hack into Steel Mountain!
  • 20. Q&A
  • 21. THANK YOU Vladimir Bychkov Technical Team Lead vbychkov@eastbanctech.com eastbanctech.com

Editor's Notes

  1. Multiple sources analyzed via ETL process
  2. HD Insight – Managed Hadoop service on Azure
  3. Event Hub – partitioned event ingestion
  4. Elliot