Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IoT Edge Processing with Apache NiFi and MiniFi and Apache MXNet for IoT NY 2018

2,981 views

Published on

IoT Edge Processing with Apache NiFi and MiniFi and Apache MXNet for IoT NY 2018. A quick talk on how to ingest IoT sensor data, camera images and run deep learning prebuilt models on edge devices including Raspberry Pis and NVidia Jetson TX1s. From there data is processed and orchestrated with Apache NiFi to send to various Big Data backends.

Published in: Technology
  • i­f y­ou t­h­i­n­k T­h­e­l­m­A`s st­ory ­is ­n­e­At,, t­hr­e­e w­e­e­ks ­A­g­o ­my ­c­ous­i­n­ns st­e­p-­d­A­A­d ­g­ot $7275 just s­itt­i­n­g t­h­er­e ­e­l­ev­e­n ­h­ours ­A w­e­e­k ­fr­o­m t­h­er­e ­A­p­Art­m­e­nt ­A­n­d t­h­ey'r­e ­c­l­Ass­m­At­e's s­ist­er`s ­n­e­i­g­h­b­our ­d­i­d t­h­is ­f­or ­e­i­g­ht ­m­o­nt­hs ­A­n­d ­g­ot ­p­A­i­d ­ov­er $7275 ­p­Art t­i­m­e ­fr­o­m t­h­e­ir ­p­c. ­A­p­p­l­i­e t­h­e st­e­ps ­Av­A­i­l­A­b­l­e ­h­er­e,­ COPY AND PAST >>>> www.101pays.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Not certain where to begin, I was overwhelmed lately trying to get out of a funk situation in a relationship. My fiancé was a whole other story. He was downright hurtful and ungrateful; his life and belongings were in shambles. He was someone addicted to drugs and women. He liked hiding behind excuses, giving me reasons to live a reckless life. But we share great bond, we can’t let go because we aren’t perfect after all. One thing we needed was someone or a spell doctor that will place himself in our spot to cure or tell us how easy it is to maintain or build a relationship surrounded by true love and healthy living. However, the reality of my story is that Dr. Wakina via (dr.wakinalovetemple@gmail.com) with a succeeding love spell made sure none of my pipe dreams at the expenses of a broken heart become reality.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dear Dr. Wakina, you are the heart and soul of our reunion after divorce, you made our marriage brighter and sweeter like others you have helped via dr.wakinalovetemple@gmail.com after a triumphant love spell. What an incredible supernatural father you are for seeing and healing my wounded heart as promised. Thanks for bringing back my husband after 14months, words can’t express how gratitude we feel about the tremendous opportunity given to us.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Find your dream house: http://amocasapentrutine.ro
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • If you find happyness in life Please Read it :: https://www.thesisscientist.com/blog/how-to-be-happy
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

IoT Edge Processing with Apache NiFi and MiniFi and Apache MXNet for IoT NY 2018

  1. 1. 1 © Hortonworks Inc. 2011–2018. All rights reserved. © Hortonworks, Inc. 2011-2018. All rights reserved. | Hortonworks confidential and proprietary information. IoT Edge Processing with Apache NiFi and MiniFi and Apache MXNet for IoT NY 2018 Timothy Spann, Senior Solutions Engineer Hortonworks @PaaSDev
  2. 2. 2 © Hortonworks Inc. 2011–2018. All rights reserved. Disclaimer • This document may contain product features and technology directions that are under development, may be under development in the future or may ultimately not be developed. • Technical feasibility, market demand, user feedback, and the Apache Software Foundation community development process can all effect timing and final delivery. • This document’s description of these features and technology directions does not represent a contractual commitment, promise or obligation from Hortonworks to deliver these features in any generally available product. • Product features and technology directions are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Since this document contains an outline of general product development plans, customers should not rely upon it when making a purchase decision.
  3. 3. 3 © Hortonworks Inc. 2011–2018. All rights reserved. What do we want to do? • MiniFi ingests camera images and sensor data • MiniFi executes Apache MXNet at the edge • Run Apache MXNet Inception to recognize objects in image • Apache NiFi stores images, metadata and enriched data in Hadoop
  4. 4. 4 © Hortonworks Inc. 2011–2018. All rights reserved. Aggregate all data from sensors, drones, cameras, logs, geo-location devices, images, machines and social feeds. Collect: Bring Together Mediate point-to-point and bi-directional data flows, delivering data reliably to Files, Apache Kafka, Apache HBase, Apache Hive, HDFS, Slack and Email. Conduct: Mediate the Data Flow Parse, filter, join, transform, fork, query, sort, dissect; enrich with weather, location, image recognition and object detection with TensorFlow and Apache MXNet. Curate: Gain Insights
  5. 5. 5 © Hortonworks Inc. 2011–2018. All rights reserved. { "imagefilename" : "/opt/demo/images/2018-04- 17_1127.jpg", "yaw" : 100.0, "host" : "sensehatmovidius", "top3" : "n06874185 traffic light, traffic signal, stoplight", "top5" : "n03773504 missile", "humidity" : 31.2, "uuid" : "mxnet_uuid_json_20180417152727.json", "ipaddress" : 192.168.1.104, "top2" : "n04286575 spotlight, spot", "top3pct" : "6.199999898672104", "top2pct" : "10.199999809265137", "cputemp2" : 56.92, "z" : 1.0, "diskfree" : "4152.5 MB", "top1pct" : "13.79999965429306", "currenttime" : "2018-04-17 15:27:37", "label2" : "n04592741 wing", "pitch" : 360.0, "pressure" : 1026.2, "roll" : 1.0, "label1" : "n04286575 spotlight, spot", "top5pct" : "4.30000014603138", "label4" : "n06874185 traffic light, traffic signal, stoplight", "y" : 0.0, "label3" : "n04009552 projector", "cputemp" : 58, "top1" : "n02930766 cab, hack, taxi, taxicab", "top4pct" : "5.000000074505806", "tempf" : 75.81, "memory" : 56.5, "top4" : "n03345487 fire engine, fire truck", "starttime" : "2018-04-17 15:27:25", "runtime" : "12", "label5" : "n09229709 bubble", "temp" : 35.45, "x" : 0.0 } Example Data
  6. 6. 6 © Hortonworks Inc. 2011–2018. All rights reserved. IoT For Enterprise Multiple devices, protocols, frameworks, languages, data types, sensors and networks Protocols • MQTT • HTTPS / SSL (REST/JSON) • OPC UA • CoAP • AMQP • TCP/IP • UDP • Zigbee • BLE • LoRaWAN • JSON • XML • CSV • Raw Text • Images (JPEG, PNG) • Raw Data Streams • Videos • Log Files • Geo Data • Audio Data Types Sensors • Cameras • Temperature/Humidity • IR • Proximity • Motion Sensors • GPS • LIDAR • Gas • RFID Protocols • NVidia Jetson TX1 • Raspberry Pi • Arduino • TS-7800 V2 • ESP8266 • DragonBoard 410c • BeagleBone Black • NanoPi • Google AIY • Tinkerboard Devices
  7. 7. 7 © Hortonworks Inc. 2011–2018. All rights reserved. IIoT Processing Data Acquisition Edge Processing Real Time Stream Analytics Rapid Application Development IoT ANALYTICS CLOUD Acquire Move Route & Filter Deliver Parse Enrich Analyze Aggregate Model
  8. 8. 8 © Hortonworks Inc. 2011–2018. All rights reserved. Run everywhere Orchestration Queueing Simple Event Processing Deep Learning Framework Enabling Record Processing Schema Management Open Source Components Edge Ingest Data Buffering Device Integration
  9. 9. 9 © Hortonworks Inc. 2011–2018. All rights reserved. HDF Data-In-Motion Platform – with HDF 3.1
  10. 10. 10 © Hortonworks Inc. 2011–2018. All rights reserved. Apache MiNiFi • NiFi lives in the data center. Give it an enterprise server or a cluster of them. • MiNiFi lives as close to where data is born and is a guest on that device or system “Let me get the key parts of NiFi close to where data begins and provide bidirectional data transfer"
  11. 11. 11 © Hortonworks Inc. 2011–2018. All rights reserved. Edge Intelligence with Apache MiNiFi à Guaranteed delivery à Data buffering ‒ Backpressure ‒ Pressure release à Prioritized queuing à Flow specific QoS ‒ Latency vs. throughput ‒ Loss tolerance à Data provenance à Recovery / recording a rolling log of fine-grained history à Designed for extension Different from Apache NiFi à Design and Deploy à Warm re-deploys Key Features
  12. 12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Java Agent C++ Agent Multi Cloud HDP Clusters HBase Druid YARN HDFS Hive Weather Location Sensors
  13. 13. 13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved http://mxnet.incubator.apache.org/ • Cloud ready • Experienced team (XGBoost) • AWS, Microsoft, NVIDIA, Baidu, Intel backing • Apache Incubator Project • Run distributed on YARN • In my early tests, faster than TensorFlow. • Runs on Raspberry PI, NVidia Jetson TX1 and other constrained devices • Great documentation • Gluon • Great Python Interaction • Model Server Available • ONNX Support • Now in Version 1.1! • Great Model Zoo https://mxnet.incubator.apache.org/how_to/cloud.html https://github.com/apache/incubator-mxnet/tree/1.1.0/example
  14. 14. 14 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache MXNet Pre-Built Models • CaffeNet • SqueezeNet v1.1 • Inception v3 • Single Shot Detection (SSD) • VGG19 • ResidualNet 152 • LSTM http://mxnet.incubator.apache.org/model_zoo/index.html https://github.com/dmlc/mxnet-model-gallery
  15. 15. 15 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache MXNet Raspberry Pi Setup pip install --upgrade pip pip install scikit-image git clone https://github.com/tspannhw/mxnet_rpi.git sudo apt-get update -y sudo apt-get install python-pip python-opencv python-scipy python-picamera -y sudo apt-get -y install git cmake build-essential g++-4.8 c++-4.8 liblapack* libblas* libopencv* git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet --branch 1.1.0 cd incubator-mxnet export USE_OPENCV = 0 make pip install mxnet==1.1.0 https://mxnet.incubator.apache.org/tutorials/embedded/wine_detector.html
  16. 16. 16 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache MXNet Raspberry Pi Setup curl --header 'Host: data.mxnet.io' --header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Firefox/45.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en- US,en;q=0.5' --header 'Referer: http://data.mxnet.io/models/imagenet/' --header 'Connection: keep- alive' 'http://data.mxnet.io/models/imagenet/inception-bn.tar.gz' -o 'inception-bn.tar.gz' -L tar -xvzf inception-bn.tar.gz cp Inception-BN-0126.params Inception-BN-0000.params
  17. 17. 17 © Hortonworks Inc. 2011–2018. All rights reserved. Apache MXNet Running on IoT Devices https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html https://github.com/tspannhw/mxnet_rpi https://community.hortonworks.com/articles/146704/edge-analytics-with-nvidia-jetson-tx1- running-apac.html
  18. 18. 18 © Hortonworks Inc. 2011–2018. All rights reserved. GPU Powered Nodes with NVidia Jetson TX1 Running Apache MXNet https://community.hortonworks.com/articles/146704/edge-analytics-with-nvidia-jetson-tx1-running-apac.html
  19. 19. 19 © Hortonworks Inc. 2011–2018. All rights reserved. Using Apache MXNet on Devices with Sensors and Intel Movidius https://community.hortonworks.com/articles/176932/apache-deep-learning-101-using-apache-mxnet-on-the.html https://community.hortonworks.com/articles/155475/powering-apache-minifi-flows-with-a-movidius-neura.html https://community.hortonworks.com/articles/183151/enterprise-iiot-edge-processing-with-apache-nifi-m.html
  20. 20. 20 © Hortonworks Inc. 2011–2018. All rights reserved. TensorFlow with MiniFi https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html https://community.hortonworks.com/articles/183151/enterprise-iiot-edge-processing-with-apache-nifi-m.html https://community.hortonworks.com/articles/130814/sensors-and-image-capture-and-deep-learning-analys.html https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html
  21. 21. 21 © Hortonworks Inc. 2011–2018. All rights reserved. Human Integration https://community.hortonworks.com/articles/121916/controlling-big-data-flows-with-gestures-minifi-ni.html • Voice • Touch • Gestures https://community.hortonworks.com/articles/167196/ingesting-data-from-the-matrix-creator-with-minifi.html https://community.hortonworks.com/articles/155519/voice-controlled-data-flows-with-google-aiy-voice.html
  22. 22. 22 © Hortonworks Inc. 2011–2018. All rights reserved. Geolocation https://community.hortonworks.com/articles/101679/iot-ingesting-gps-data-from-raspberry-pi-zero-wire.html https://community.hortonworks.com/articles/155606/iot-ingesting-gps-data-from-odroid-xu4-devices-wit.html https://community.hortonworks.com/articles/177137/ingesting-flight-data-ads-b-usb-receiver-with-apac.html https://community.hortonworks.com/articles/178196/integrating-lucene-geo-gazetteer-for-geo-parsing-w.html
  23. 23. 23 © Hortonworks Inc. 2011–2018. All rights reserved. Weather Integration https://community.hortonworks.com/articles/101904/part-2-iot-augmenting-gps-data-with-weather.html
  24. 24. 24 © Hortonworks Inc. 2011–2018. All rights reserved. Image Analytics https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html https://community.hortonworks.com/articles/155604/iot-ingesting-camera-data-from-nanopi-duo-devices.html https://community.hortonworks.com/articles/182984/vision-thing-part-2-processing-capturing-and-displ.html https://community.hortonworks.com/articles/182850/vision-thing.html https://community.hortonworks.com/articles/77988/ingest-remote-camera-images-from-raspberry-pi-via.html
  25. 25. 25 © Hortonworks Inc. 2011–2018. All rights reserved. Bluetooth Low Energy (BLE) https://community.hortonworks.com/articles/108947/minifi-for-ble-bluetooth-low-energy-beacon-data-in.html Beacons IBeacon NFC https://community.hortonworks.com/articles/99861/ingesting-ibeacon-data-via-ble-to-mqtt-wifi-gatewa.html
  26. 26. 26 © Hortonworks Inc. 2011–2018. All rights reserved. Sensors via Python API {"tempf": 75.14, "temp": 35.08, "pitch": 1.0, "diskfree": "1211.8 MB", "yaw": 55.0, "cputemp": 52.08, "ts": "2017-06-16 17:39:08", "humidity": 41.5, "pressure": 0.0, "memory": 23.0, "y": 0.0, "x": -1.0, "z": 1.0, "roll": 1.0} https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html https://community.hortonworks.com/articles/155326/monitoring-energy-usage-utilizing-apache-nifi-pyth.html
  27. 27. 27 © Hortonworks Inc. 2011–2018. All rights reserved. Text Processing https://community.hortonworks.com/articles/178510/integration-apache-opennlp-184-into-apache-nifi-15.html • Dates • Locations • Names • Money • Organizations https://community.hortonworks.com/articles/52415/processing-social-media-feeds-in-stream-with-apach.html Sentiment AnalysisNamed Entity Extraction https://community.hortonworks.com/articles/76935/using-sentiment-analysis-and-nlp-tools-with-hdp-25.html https://community.hortonworks.com/articles/163776/parsing-any-document-with-apache-nifi-15-with-apac.html https://community.hortonworks.com/articles/81222/adding-stanford-corenlp-to-big-data-pipelines-apac.html https://community.hortonworks.com/articles/81270/adding-stanford-corenlp-to-big-data-pipelines-apac-1.html
  28. 28. 28 © Hortonworks Inc. 2011–2018. All rights reserved. MQTT https://community.hortonworks.com/articles/167199/sending-messages-and-displaying-them-on-an-oled-sc.html https://community.hortonworks.com/articles/183474/iot-using-minifi-java-agent-to-send-mqtt-messages.html https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html https://community.hortonworks.com/articles/72420/ingesting -remote-sensor-feeds-into-apache-phoenix.html
  29. 29. 29 © Hortonworks Inc. 2011–2018. All rights reserved. Questions?
  30. 30. 30 © Hortonworks Inc. 2011–2018. All rights reserved. Contact https://github.com/tspannhw/ApacheDeepLearning101 https://community.hortonworks.com/users/9304/tspann.html https://dzone.com/users/297029/bunkertor.html https://www.meetup.com/futureofdata-princeton/ https://twitter.com/PaaSDev @PaasDev
  31. 31. 31 © Hortonworks Inc. 2011–2018. All rights reserved. Hortonworks Community Connection Read access for everyone, join to participate and be recognized • Full Q&A Platform (like StackOverflow) • Knowledge Base Articles • Code Samples and Repositories
  32. 32. 32 © Hortonworks Inc. 2011–2018. All rights reserved. Community Engagement Participate now at: community.hortonworks.com© Hortonworks Inc. 2011 – 2015. All Rights Reserved 4,000+ Registered Users 10,000+ Answers 15,000+ Technical Assets One Website!

×