1.
Azure IOT:
EVENT HUB, STREAM
ANALYTICS & POWER BI
By Chourouk Hjaiej
February 2016

2.
IOT with Azure Services Chourouk HJAIEJ
2
Table of content
Who should read this book?
Introduction
Azure event hubs, Stream Analytics & Power BI
Understanding the solution
Solution Overview with Practical Cases
Setting Things Up
Azure Event Hub
Azure Stream Analytics
Power BI
Examples of dashboard
Conclusion
Keywords
References

3.
IOT with Azure Services Chourouk HJAIEJ
3
Who should read this book?
This paper is a practical look at IOT and different services in Azure which help to solve these problems.
It assumes the reader has a basic understanding of C#.
It is written for both developers and operations professionals who want to improve their team’s or
organization’s ability to deliver robust, highly available applications successfully. One of the biggest
challenges IOT projects face is how to reliably and securely connect devices to the solution backend.
Introduction
The Internet of Things (IOT) starts with your things: the things that matter most to your business. It`s
the Internet of your Things, it means that you are able to connect your device to a solution without
any difficulties, find and rely on a comprehensive set of technologies to connect to and analyze data,
or even build new intelligent devices.
One of the biggest challenges IOT projects face is how to reliably and securely connect devices to the
solution backend. In this paper, we will demonstrate how Azure is able to ensure these functionalities.
At WPC 2014, Microsoft announced the preview of Azure Event Hubs which will enable the collection
of event streams at high throughput which makes it perfect for IOT, data processing, scenarios. In this
paper, I'll start with a simple example to give you a first hands-on experience with Event Hubs.
We'll be building a simple example that publishes events to an Event Hub which can be processed by
one or more worker processed. I'm not fond of these examples, but in this case this could really be
useful. Think of all those applications deployed on hundreds or even thousands of machines. Some of
those applications are probably deployed on laptops, POS devices, … that don't have a direct link with
the main office, so being related to SQL Server is probably not an option. Wouldn't it be useful if all
these applications could send their information (performance…) to a single 'hub' instead of local files
or the event log, even if they're not in the main office? And what if we could then process all those
events for reporting, performance, …? These are the scenarios that are made possible by Event Hubs.
IOT raises important questions which we will address in this paper within the context of a set of
technologies. The most important one is Microsoft Azure, named by Gartner as a leading cloud
platform (2015 Magic Quadrant).
This paper will discuss the topic which is the real-time branch of the Lambda architecture and we will
implement the following components:
 An Event Hub
 A Stream Analytics with Power BI as an output.
 We will use a program in C # to simulate sending information in real time.
Regards,
Chourouk HJAIEJ, DX Technical Evangelist Intern

4.
IOT with Azure Services Chourouk HJAIEJ
4
Azure event hubs, Stream Analytics & Power BI
Understanding the solution
A complete IOT solution is made of several parts. First of all, we need to ingest all the events and data
sent from devices and it’s a such big problem because in the Internet of Things, we have to think at
“scale” with hundreds, thousands, million and billions of devices. In this scenario, we have to
use an ingestion system that is able to scale and handle incoming data very quickly without slowing
down the entire process.
After acquiring data, the ingestion system must provide them to the business engine of our solution
that could have a path for analyzing data as a stream in real time. This path could expose
these informations to the end user that can monitor the devices and what they are “saying” from the
real world. The same information is very useful as input for a machine learning system that can help
us for predictive analysis to understand how data could evolve in the future based on current incoming
values and take preventive actions.
Solution Overview with Practical Cases
The previous section described how to manage these informations with the Microsoft Azure by
using its services.
This section illustrates those concepts in a real world scenario and demonstrates the usefulness of
Microsoft Azure services.
Among these services are event hub which include ….. and stream analytics.
The aim here is to show how the combination of an event hub and Stream analytics provides an
efficient way tomanage the huge number of events.
The figure below shows the scenario which we will use in this part.

5.
IOT with Azure Services Chourouk HJAIEJ
5
The scenario we will use in this post builds upon previous posts that I have written so far:
We are simulating potentially millions of devices sending telemetry data. The data is sent to an Event
Hub, and processed in parallel by 3 different processes:
Instead of chaining processes together, making one process wait on the completion of a previous
process, we can scale our solution by having many independent processors work with the data in
parallel. This is incredibly powerful.
We will focus in this paper on using Stream Analytics to process streaming event data from Azure
Event Hubs at scale. We will show some pieces of code, but this code sample will show all the project.
You can build a Universal App with service bus (event hub) on Azure using AMQP. This sample explain
it very well.
Event
Source
 Use Azure Event Hub with Visual Studio
 Device Simulator for Event Hubs
Collect  Use Azure Event Hubs with Visual Studio
Filter /
Analyze /
Aggregate
 Scaling Azure Event Hubs Processing
 Use the Azure Stream Analytics
Dashboard  Use Power BI in order to have significant results

6.
IOT with Azure Services Chourouk HJAIEJ
6
Setting Things Up
Before we start using Stream Analytics, we will need an Event Hub, something that sends data to the
Event Hub. Creating the Event Hub is very easy using the Azure Management Portal. The figure below
describes the steps to create an event hub.
Azure Event Hub
Provide a name and region and a namespace. Provide a partition count and message retention period.

7.
IOT with Azure Services Chourouk HJAIEJ
7
As you can see, this is the list of event hubs created by me and this is the last one.
I will create “DevicePolicy” :
You can see the primary and the secondary key of this shared access policy:
Our Event hub is ready. We will now create and configure a service which will consume post events in
our Event Hub. This service is Azure Stream Analytics.
Azure Stream Analytics
From the Azure portal, click the "+" button at the bottom of the window. Select "Data Services",
"Stream Analytics," "Quick Create". Fill in the different fields and click "Create Stream Analytics job".

8.
IOT with Azure Services Chourouk HJAIEJ
8
After a few seconds, my Azure Stream Analytics service will be created.
Setting the Azure Service Stream Analytics
Now that our Azure Stream Analytics service was created, we'll set it to receive information from our
Event Hub (then possibly turn), then send them to a Power BI portal.
Click the name of your Azure Stream Analytics ("streamanalyticsjob" in this example). Then click
"INPUTS". Click on "ADD AN INPUT".
We will select "Events Hub" (Event Hub).

9.
IOT with Azure Services Chourouk HJAIEJ
9
In the Alias Entry field, enter the name you want to give your input in the Azure Stream Analytics.
For this demonstration, we'll select "Use Event Hub from current subscription."

10.
IOT with Azure Services Chourouk HJAIEJ
10
Finally, we will choose the serialization settings. We have the choice between "Avro", "CSV" or "JSON".
We will select JSON because the program which I will use to send the information in the JSON format.
You will have an inputs for this Stream Analytics.
Setting a query
Now that we have defined an entry, we will define a query to retrieve the data. This query can be a
simple "select" or a more complex query to transform data or perform operations on time windows
(to average every 5 seconds).
Click on "Query" to define the data processing request. Note that the language used is very similar to
T-SQL. For this example, I will realize just a select all the fields that are present in the entrance to my
service:

11.
IOT with Azure Services Chourouk HJAIEJ
11
SELECT * FROM Inputs
Click "Save" and confirm registration.
Setting an output
We will now set the output of our service to a Power BI portal.

12.
IOT with Azure Services Chourouk HJAIEJ
12
Set in the "Add a Microsoft Power output BI" the name of the database and the table name. If the table
already exists Power BI side, it will be replaced with the new one. Confirm your changes.
You must obtain a new outlet for your Azure Stream Analytics service.

13.
IOT with Azure Services Chourouk HJAIEJ
13
It remains for us to start the service. At the bottom of the page, click the Start button.
Simulating data injection
Now, we have to interest in how it sends data.
To do this, I used a small program in C # that can simulate sending data, but above all it shows how
to connect to an Event Hub precisely to send the data.
First, in my "Program.cs" file, don`t forget to replace the values with that of your Event Hub.
When running the program, here is an example of the data sent to the Event Hub:

14.
IOT with Azure Services Chourouk HJAIEJ
14
Power BI
Now log in to the portal via the BI Power http://app.powerbi.com address.
In the left pane, in the "DataSets", a new data source, "DeviceDataset" has just appeared. It is
recognizable with the small green asterisk.

15.
IOT with Azure Services Chourouk HJAIEJ
15
Examples of Dashboard

16.
IOT with Azure Services Chourouk HJAIEJ
16
Azure Stream Analytics Monitoring
The Azure Stream Analytics also provides a monitoring dashboard as well as the possibility to refer to
the events related to the service logs. This can be very useful to understand what happens (especially
when it does not work).
In your Azure Stream Analytics service, click "Dashboard" to see the events related to your service (you
can also click on "Monitor").
In the dashboard, you default service supervisory common indicators and access to the operations log.
Click on the link "operations logs" to open the Services management window.

17.
IOT with Azure Services Chourouk HJAIEJ
17
Keywords:
Azure Event Hub, Azure Stream Analytics, Scalability, Power BI, Dashboard, Cloud, Query, Azure,
References
Azure
> What is Microsoft Azure?
https://azure.microsoft.com/en-us/overview/what-is-azure/
> Microsoft – the only vendor named a leader in Gartner Magic Quadrants for IaaS,
Application PaaS and Cloud Storage
https://azure.microsoft.com/en-us/blog/microsoft-the-only-vendor-named-a-leader-in-gartner-
magic-quadrants-for-iaas-application-paas-cloud-storage-and-hybrid/
> Azure Preview Portal
https://azure.microsoft.com/en-us/features/azure-portal/
Azure Event Hub
> What is Azure Event hub?
https://azure.microsoft.com/en-gb/documentation/articles/event-hubs-overview/
Azure Stream Analytics
> What is Azure Stream analytics?
https://azure.microsoft.com/en-us/services/stream-analytics/
Power BI
> What is Power BI?
https://powerbi.microsoft.com/fr/
Azure IOT vs Amazon aws IOT
> What is the differences between Azure IOT and Amazon aws IOT?
https://paolopatierno.wordpress.com/2015/10/13/an-iot-platforms-match-microsoft-azure-iot-vs-
amazon-aws-iot/

18.
IOT with Azure Services Chourouk HJAIEJ
18
Microsoft Azure