1 © Hortonworks Inc. 2011–2018. All rights reserved.
© Hortonworks, Inc. 2011-2018. All rights reserved. | Hortonworks confidential and proprietary information.
IoT Edge Processing with
Apache NiFi and MiniFi and
Apache MXNet for IoT NY 2018
Timothy Spann, Senior Solutions Engineer
Hortonworks @PaaSDev

2 © Hortonworks Inc. 2011–2018. All rights reserved.
Disclaimer
• This document may contain product features and technology directions that are under
development, may be under development in the future or may ultimately not be
developed.
• Technical feasibility, market demand, user feedback, and the Apache Software
Foundation community development process can all effect timing and final delivery.
• This document’s description of these features and technology directions does not
represent a contractual commitment, promise or obligation from Hortonworks to deliver
these features in any generally available product.
• Product features and technology directions are subject to change, and must not be
included in contracts, purchase orders, or sales agreements of any kind.
• Since this document contains an outline of general product development plans,
customers should not rely upon it when making a purchase decision.

3 © Hortonworks Inc. 2011–2018. All rights reserved.
What do we want to do?
• MiniFi ingests camera images and
sensor data
• MiniFi executes Apache MXNet at
the edge
• Run Apache MXNet Inception to
recognize objects in image
• Apache NiFi stores images,
metadata and enriched data in
Hadoop

4 © Hortonworks Inc. 2011–2018. All rights reserved.
Aggregate all data from sensors, drones, cameras, logs, geo-location
devices, images, machines and social feeds.
Collect: Bring Together
Mediate point-to-point and bi-directional data flows, delivering data
reliably to Files, Apache Kafka, Apache HBase, Apache Hive, HDFS, Slack
and Email.
Conduct: Mediate the Data Flow
Parse, filter, join, transform, fork, query, sort, dissect; enrich with weather,
location, image recognition and object detection with TensorFlow and
Apache MXNet.
Curate: Gain Insights

5 © Hortonworks Inc. 2011–2018. All rights reserved.
{
"imagefilename" : "/opt/demo/images/2018-04-
17_1127.jpg",
"yaw" : 100.0,
"host" : "sensehatmovidius",
"top3" : "n06874185 traffic light, traffic signal,
stoplight",
"top5" : "n03773504 missile",
"humidity" : 31.2,
"uuid" : "mxnet_uuid_json_20180417152727.json",
"ipaddress" : 192.168.1.104,
"top2" : "n04286575 spotlight, spot",
"top3pct" : "6.199999898672104",
"top2pct" : "10.199999809265137",
"cputemp2" : 56.92,
"z" : 1.0,
"diskfree" : "4152.5 MB",
"top1pct" : "13.79999965429306",
"currenttime" : "2018-04-17 15:27:37",
"label2" : "n04592741 wing",
"pitch" : 360.0,
"pressure" : 1026.2,
"roll" : 1.0,
"label1" : "n04286575 spotlight, spot",
"top5pct" : "4.30000014603138",
"label4" : "n06874185 traffic light, traffic signal, stoplight",
"y" : 0.0,
"label3" : "n04009552 projector",
"cputemp" : 58,
"top1" : "n02930766 cab, hack, taxi, taxicab",
"top4pct" : "5.000000074505806",
"tempf" : 75.81,
"memory" : 56.5,
"top4" : "n03345487 fire engine, fire truck",
"starttime" : "2018-04-17 15:27:25",
"runtime" : "12",
"label5" : "n09229709 bubble",
"temp" : 35.45,
"x" : 0.0
}
Example Data

6 © Hortonworks Inc. 2011–2018. All rights reserved.
IoT For Enterprise
Multiple devices, protocols, frameworks, languages, data types, sensors and networks
Protocols
• MQTT
• HTTPS / SSL (REST/JSON)
• OPC UA
• CoAP
• AMQP
• TCP/IP
• UDP
• Zigbee
• BLE
• LoRaWAN
• JSON
• XML
• CSV
• Raw Text
• Images (JPEG, PNG)
• Raw Data Streams
• Videos
• Log Files
• Geo Data
• Audio
Data Types Sensors
• Cameras
• Temperature/Humidity
• IR
• Proximity
• Motion Sensors
• GPS
• LIDAR
• Gas
• RFID
Protocols
• NVidia Jetson TX1
• Raspberry Pi
• Arduino
• TS-7800 V2
• ESP8266
• DragonBoard 410c
• BeagleBone Black
• NanoPi
• Google AIY
• Tinkerboard
Devices

7 © Hortonworks Inc. 2011–2018. All rights reserved.
IIoT Processing
Data Acquisition
Edge Processing
Real Time Stream Analytics
Rapid Application Development
IoT
ANALYTICS
CLOUD
Acquire Move
Route
&
Filter
Deliver Parse Enrich Analyze
Aggregate
Model

8 © Hortonworks Inc. 2011–2018. All rights reserved.
Run everywhere
Orchestration
Queueing
Simple Event Processing
Deep Learning Framework
Enabling Record Processing
Schema Management
Open Source Components
Edge Ingest
Data Buffering
Device Integration

9 © Hortonworks Inc. 2011–2018. All rights reserved.
HDF Data-In-Motion Platform – with HDF 3.1

10 © Hortonworks Inc. 2011–2018. All rights reserved.
Apache MiNiFi
• NiFi lives in the data center. Give it an
enterprise server or a cluster of them.
• MiNiFi lives as close to where data is born
and is a guest on that device or system
“Let me get the key parts of NiFi close to where data begins and provide
bidirectional data transfer"

11 © Hortonworks Inc. 2011–2018. All rights reserved.
Edge Intelligence with Apache MiNiFi
à Guaranteed delivery
à Data buffering
‒ Backpressure
‒ Pressure release
à Prioritized queuing
à Flow specific QoS
‒ Latency vs. throughput
‒ Loss tolerance
à Data provenance
à Recovery / recording a rolling log
of fine-grained history
à Designed for extension
Different from Apache NiFi
à Design and Deploy
à Warm re-deploys
Key Features

12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Java Agent C++ Agent
Multi Cloud HDP Clusters
HBase Druid
YARN
HDFS
Hive
Weather
Location
Sensors

13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
http://mxnet.incubator.apache.org/
• Cloud ready
• Experienced team (XGBoost)
• AWS, Microsoft, NVIDIA, Baidu, Intel backing
• Apache Incubator Project
• Run distributed on YARN
• In my early tests, faster than TensorFlow.
• Runs on Raspberry PI, NVidia Jetson TX1
and other constrained devices
• Great documentation
• Gluon
• Great Python Interaction
• Model Server Available
• ONNX Support
• Now in Version 1.1!
• Great Model Zoo
https://mxnet.incubator.apache.org/how_to/cloud.html
https://github.com/apache/incubator-mxnet/tree/1.1.0/example

14 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Apache MXNet Pre-Built Models
• CaffeNet
• SqueezeNet v1.1
• Inception v3
• Single Shot Detection (SSD)
• VGG19
• ResidualNet 152
• LSTM
http://mxnet.incubator.apache.org/model_zoo/index.html
https://github.com/dmlc/mxnet-model-gallery

15 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Apache MXNet Raspberry Pi Setup
pip install --upgrade pip
pip install scikit-image
git clone https://github.com/tspannhw/mxnet_rpi.git
sudo apt-get update -y
sudo apt-get install python-pip python-opencv python-scipy python-picamera -y
sudo apt-get -y install git cmake build-essential g++-4.8 c++-4.8 liblapack* libblas* libopencv*
git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet --branch 1.1.0
cd incubator-mxnet
export USE_OPENCV = 0
make
pip install mxnet==1.1.0
https://mxnet.incubator.apache.org/tutorials/embedded/wine_detector.html

16 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Apache MXNet Raspberry Pi Setup
curl --header 'Host: data.mxnet.io' --header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11;
rv:45.0) Gecko/20100101 Firefox/45.0' --header 'Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en-
US,en;q=0.5' --header 'Referer: http://data.mxnet.io/models/imagenet/' --header 'Connection: keep-
alive' 'http://data.mxnet.io/models/imagenet/inception-bn.tar.gz' -o 'inception-bn.tar.gz' -L
tar -xvzf inception-bn.tar.gz
cp Inception-BN-0126.params Inception-BN-0000.params

17 © Hortonworks Inc. 2011–2018. All rights reserved.
Apache MXNet Running on IoT Devices
https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html
https://github.com/tspannhw/mxnet_rpi
https://community.hortonworks.com/articles/146704/edge-analytics-with-nvidia-jetson-tx1-
running-apac.html

18 © Hortonworks Inc. 2011–2018. All rights reserved.
GPU Powered Nodes with NVidia Jetson TX1 Running Apache MXNet
https://community.hortonworks.com/articles/146704/edge-analytics-with-nvidia-jetson-tx1-running-apac.html

19 © Hortonworks Inc. 2011–2018. All rights reserved.
Using Apache MXNet on Devices with Sensors and Intel Movidius
https://community.hortonworks.com/articles/176932/apache-deep-learning-101-using-apache-mxnet-on-the.html
https://community.hortonworks.com/articles/155475/powering-apache-minifi-flows-with-a-movidius-neura.html
https://community.hortonworks.com/articles/183151/enterprise-iiot-edge-processing-with-apache-nifi-m.html

20 © Hortonworks Inc. 2011–2018. All rights reserved.
TensorFlow with MiniFi
https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html
https://community.hortonworks.com/articles/183151/enterprise-iiot-edge-processing-with-apache-nifi-m.html
https://community.hortonworks.com/articles/130814/sensors-and-image-capture-and-deep-learning-analys.html
https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html

21 © Hortonworks Inc. 2011–2018. All rights reserved.
Human Integration
https://community.hortonworks.com/articles/121916/controlling-big-data-flows-with-gestures-minifi-ni.html
• Voice
• Touch
• Gestures
https://community.hortonworks.com/articles/167196/ingesting-data-from-the-matrix-creator-with-minifi.html
https://community.hortonworks.com/articles/155519/voice-controlled-data-flows-with-google-aiy-voice.html

22 © Hortonworks Inc. 2011–2018. All rights reserved.
Geolocation
https://community.hortonworks.com/articles/101679/iot-ingesting-gps-data-from-raspberry-pi-zero-wire.html
https://community.hortonworks.com/articles/155606/iot-ingesting-gps-data-from-odroid-xu4-devices-wit.html
https://community.hortonworks.com/articles/177137/ingesting-flight-data-ads-b-usb-receiver-with-apac.html
https://community.hortonworks.com/articles/178196/integrating-lucene-geo-gazetteer-for-geo-parsing-w.html

23 © Hortonworks Inc. 2011–2018. All rights reserved.
Weather Integration
https://community.hortonworks.com/articles/101904/part-2-iot-augmenting-gps-data-with-weather.html

24 © Hortonworks Inc. 2011–2018. All rights reserved.
Image Analytics
https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html
https://community.hortonworks.com/articles/155604/iot-ingesting-camera-data-from-nanopi-duo-devices.html
https://community.hortonworks.com/articles/182984/vision-thing-part-2-processing-capturing-and-displ.html
https://community.hortonworks.com/articles/182850/vision-thing.html
https://community.hortonworks.com/articles/77988/ingest-remote-camera-images-from-raspberry-pi-via.html

25 © Hortonworks Inc. 2011–2018. All rights reserved.
Bluetooth Low Energy (BLE)
https://community.hortonworks.com/articles/108947/minifi-for-ble-bluetooth-low-energy-beacon-data-in.html
Beacons
IBeacon
NFC
https://community.hortonworks.com/articles/99861/ingesting-ibeacon-data-via-ble-to-mqtt-wifi-gatewa.html

26 © Hortonworks Inc. 2011–2018. All rights reserved.
Sensors via Python API
{"tempf": 75.14, "temp": 35.08,
"pitch": 1.0,
"diskfree": "1211.8 MB", "yaw": 55.0,
"cputemp": 52.08,
"ts": "2017-06-16 17:39:08",
"humidity": 41.5, "pressure": 0.0,
"memory": 23.0, "y": 0.0, "x": -1.0, "z": 1.0,
"roll": 1.0}
https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html
https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html
https://community.hortonworks.com/articles/155326/monitoring-energy-usage-utilizing-apache-nifi-pyth.html

27 © Hortonworks Inc. 2011–2018. All rights reserved.
Text Processing
https://community.hortonworks.com/articles/178510/integration-apache-opennlp-184-into-apache-nifi-15.html
• Dates
• Locations
• Names
• Money
• Organizations
https://community.hortonworks.com/articles/52415/processing-social-media-feeds-in-stream-with-apach.html
Sentiment AnalysisNamed Entity Extraction
https://community.hortonworks.com/articles/76935/using-sentiment-analysis-and-nlp-tools-with-hdp-25.html
https://community.hortonworks.com/articles/163776/parsing-any-document-with-apache-nifi-15-with-apac.html
https://community.hortonworks.com/articles/81222/adding-stanford-corenlp-to-big-data-pipelines-apac.html
https://community.hortonworks.com/articles/81270/adding-stanford-corenlp-to-big-data-pipelines-apac-1.html

28 © Hortonworks Inc. 2011–2018. All rights reserved.
MQTT
https://community.hortonworks.com/articles/167199/sending-messages-and-displaying-them-on-an-oled-sc.html
https://community.hortonworks.com/articles/183474/iot-using-minifi-java-agent-to-send-mqtt-messages.html
https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html
https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html
https://community.hortonworks.com/articles/72420/ingesting
-remote-sensor-feeds-into-apache-phoenix.html

29 © Hortonworks Inc. 2011–2018. All rights reserved.
Questions?

30 © Hortonworks Inc. 2011–2018. All rights reserved.
Contact
https://github.com/tspannhw/ApacheDeepLearning101
https://community.hortonworks.com/users/9304/tspann.html
https://dzone.com/users/297029/bunkertor.html
https://www.meetup.com/futureofdata-princeton/
https://twitter.com/PaaSDev
@PaasDev

31 © Hortonworks Inc. 2011–2018. All rights reserved.
Hortonworks Community Connection
Read access for everyone, join to participate and be recognized
• Full Q&A Platform (like StackOverflow)
• Knowledge Base Articles
• Code Samples and Repositories

32 © Hortonworks Inc. 2011–2018. All rights reserved.
Community Engagement
Participate now at: community.hortonworks.com© Hortonworks Inc. 2011 – 2015. All Rights Reserved
4,000+
Registered Users
10,000+
Answers
15,000+
Technical Assets
One Website!