Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hands-On Deep Dive with MiniFi and Apache MXNet

201 views

Published on

Deep Learning on The Edge, a hands-on a approach to running deep learning work loads on the edge for IoT as well as in Apache NiFi and in Hadoop 3.1 YARN as dockerized work loads.

Published in: Technology
  • Be the first to comment

Hands-On Deep Dive with MiniFi and Apache MXNet

  1. 1. 1 © Hortonworks Inc. 2011–2018. All rights reserved. A Hands-on deep dive on using Apache NiFi with MiniFi, Apache MXNet and other Deep Learning Libraries on the Edge Timothy Spann, Senior Solutions Engineer Hortonworks @PaaSDev Deep Learning On The Edge
  2. 2. 2 © Hortonworks Inc. 2011–2018. All rights reserved. Disclaimer • This document may contain product features and technology directions that are under development, may be under development in the future or may ultimately not be developed. • Technical feasibility, market demand, user feedback, and the Apache Software Foundation community development process can all effect timing and final delivery. • This document’s description of these features and technology directions does not represent a contractual commitment, promise or obligation from Hortonworks to deliver these features in any generally available product. • Product features and technology directions are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Since this document contains an outline of general product development plans, customers should not rely upon it when making a purchase decision.
  3. 3. 3 © Hortonworks Inc. 2011–2018. All rights reserved. IoT Multiple devices, protocols, frameworks, languages, data types, sensors and networks Protocols • MQTT • HTTPS / SSL (REST/JSON) • OPC UA • CoAP • AMQP • TCP/IP • UDP • JSON • XML • CSV • Raw Text • Images (JPEG, PNG) • Raw Data Streams Data Types Sensors • Cameras • Temperature/Humidity • IR • Proximity • Motion Sensors • GPS Protocols • NVidia Jetson TX1 • Raspberry Pi • Arduino • TS-7800 V2 • ESP8266 • DragonBoard 410c • BeagleBone Black • Sino:bit
  4. 4. 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved http://mxnet.incubator.apache.org/ • Cloud ready • Experienced team (XGBoost) • AWS, Microsoft, NVIDIA, Baidu, Intel backing • Apache Incubator Project • Run distributed on YARN • In my early tests, faster than TensorFlow. • Runs on Raspberry PI, NVidia Jetson TX1 and other constrained devices • Great documentation • Gluon • Great Python Interaction • Model Server Available • ONNX Support • Now in Version 1.2.1! • Great Model Zoo https://mxnet.incubator.apache.org/how_to/cloud.html https://github.com/apache/incubator-mxnet/tree/1.2.1/example
  5. 5. 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache MXNet 1.2.1 Pre-Built Models • CaffeNet • SqueezeNet v1.1 • Inception v3 • Single Shot Detection (SSD) • VGG19 • ResidualNet 152 • LSTM http://mxnet.incubator.apache.org/model_zoo/index.html https://github.com/dmlc/mxnet-model-gallery https://github.com/onnx/models https://mxnet.incubator.apache.org/api/python/gluon/model_zoo.html
  6. 6. 6 © Hortonworks Inc. 2011–2018. All rights reserved. Using Apache MXNet on Devices with Sensors and Intel Movidius https://community.hortonworks.com/articles/176932/apache-deep-learning-101-using-apache-mxnet-on- the.html
  7. 7. 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved • TensorFlow (C++, Python, Java) via ExecuteStreamCommand • TensorFlow NiFi Java Custom Processor • TensorFlow Running on Edge Devices (MiniFi C++ Built-in) • TensorFlow Running on Edge Devices (MiniFi Java) • TensorFlow Running in Spark • TensorFlow Running Through Livy to PySpark • TensorFlow Running on HDP 3.0 in Dockerized Containers Apache NiFi Integration with TensorFlow Options
  8. 8. 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved TensorFlow Python ExecuteStreamCommand NiFi https://community.hortonworks.com/articles/58265/analyzing-images-in-hdf-20-using-tensorflow.html
  9. 9. 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Run TensorFlow on YARN 3.0 https://community.hortonworks.com/articles/83872/data-lake-30-containerization-erasure-coding-gpu-p.html
  10. 10. 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Why TensorFlow? Also Apache MXNet, PyTorch and DL4J. • Google • Multiple platform support • Hadoop integration • Spark integration • Keras • Large Community • Python and Java APIs • GPU Support • Mobile Support • Inception v3 • Clustering • Fully functional demos • Open Source • Apache Licensed • Large Model Library • Buzz • Extensive Documentation • Raspberry Pi Support
  11. 11. 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved TensorFlow Java Processor in NiFi https://community.hortonworks.com/content/kbentry/116803/building-a-custom-processor-in-apache-nifi-12-for.html https://github.com/tspannhw/nifi-tensorflow-processor
  12. 12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved TensorFlow Running on Edge Nodes (MiniFi)
  13. 13. 13 © Hortonworks Inc. 2011–2018. All rights reserved. Sensors via Python API {"tempf": 75.14, "temp": 35.08, "pitch": 1.0, "diskfree": "1211.8 MB", "yaw": 55.0, "cputemp": 52.08, "ts": "2017-06-16 17:39:08", "humidity": 41.5, "pressure": 0.0, "memory": 23.0, "y": 0.0, "x": -1.0, "z": 1.0, "roll": 1.0} https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html https://community.hortonworks.com/articles/155326/monitoring-energy-usage-utilizing-apache-nifi-pyth.html
  14. 14. 14 © Hortonworks Inc. 2011–2018. All rights reserved. MQTT https://community.hortonworks.com/articles/167199/sending-messages-and-displaying-them-on-an-oled-sc.html https://community.hortonworks.com/articles/183474/iot-using-minifi-java-agent-to-send-mqtt-messages.html https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html https://community.hortonworks.com/articles/72420/ingesting-remote-sensor-feeds-into-apache-phoenix.html
  15. 15. 15 © Hortonworks Inc. 2011–2018. All rights reserved. Run everywhere Orchestration Queueing Simple Event Processing Deep Learning Framework Enabling Record Processing Schema Management Open Source Components Edge Ingest Data Buffering Device Integration
  16. 16. 16 © Hortonworks Inc. 2011–2018. All rights reserved. Open Source Image Analytical Components Streaming Analytics Manager Image Ingest Distributed queue Buffering Process decoupling Routing and Pre-Processing Orchestration Queueing Simple Event Processing Image Capture Image Processing
  17. 17. 17 © Hortonworks Inc. 2011–2018. All rights reserved. Streaming Analytics Manager Part of MiniFi C++ Agent Detect metadata and data Extract metadata and data Content Analysis Deep Learning Framework Open Source Image Analytical Components Enabling Record Processing Schema Management
  18. 18. 18 © Hortonworks Inc. 2011–2018. All rights reserved. Edge Intelligence with MiNiFi à Guaranteed delivery à Data buffering ‒ Backpressure ‒ Pressure release à Prioritized queuing à Flow specific QoS ‒ Latency vs. throughput ‒ Loss tolerance à Data provenance à Recovery / recording a rolling log of fine-grained history à Designed for extension Different from Apache NiFi à Design and Deploy à Warm re-deploys Key Features
  19. 19. 19 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Java Agent C++ Agent Multi Cloud HDP Clusters HBase Druid YARN HDFS Hive Weather Location Sensors
  20. 20. 20 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache NiFi: The Ecosystem Enter the MiNiFi Command & Control – Provide tooling to map the UX of interactive command and control in NiFi to the design and deploy approach of MiNiFi But more instances complicate my operational management! https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Command+and+Control
  21. 21. 21 © Hortonworks Inc. 2011–2018. All rights reserved. What do we want to do? • Python collects camera images and sensor data • MiniFi executes Apache MXNet and/or Tensorflow at the edge • Run Inception to classify objects in image • Apache NiFi stores sensor readings, gps, images, metadata and enriched data in Hadoop • One or more Heterogenous Devices in distributed locations • One or more Heterogenous Data Storage Platforms
  22. 22. 22 © Hortonworks Inc. 2011–2018. All rights reserved. Overview • Build Physical IoT Devices (Raspberry Pi, Movidius Neural Stick, Webcam) • Choose MiniFi C++ or MiniFi Java Agent • For MiniFi Java Agent, Install JDK 8 • Design Flow For MiniFi • Install config.yml on Device • Connect to Sensors and Equipment via Python, Go, C++ or Java • Add Shell Script if needed • Start MiniFi Java Agent as a Service • MiniFi Calls Apache NiFi IoT Gateway via S2S or MQTT • Build Data Stream Real-Time in Apache NiFi as Data Arrives • Complete Simple Event Processing • Store Data at Scale in HBase, Druid, HDFS, S3 or Hive LLAP • Add additional processing with Apache Spark, Apache Storm or Apache Beam via Apache Kafka
  23. 23. 23 © Hortonworks Inc. 2011–2018. All rights reserved. • https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_installing- hdf/content/ch_install-ambari.html • docker pull hortonworks/nifi • Apache NiFi Configuration for IoT • https://community.hortonworks.com/articles/67756/ingesting-log-data-using-minifi- nifi.html • nifi.remote.input.host and nifi.remote.input.socket.port Installation of Apache NiFi or HDF 3.x Pre Step 1
  24. 24. 24 © Hortonworks Inc. 2011–2018. All rights reserved. • Download MiniFi (https://nifi.apache.org/minifi/download.html) You can choose Java or C++. For your first usage, I recommend the Java edition unless your device is too small. You can also install on a RHEL or Debian Linux machine or OSX. • Download MiniFi Toolkit (https://nifi.apache.org/minifi/minifi-toolkit.html) • https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Versioncpp-0.5.0 • https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Version0.5.0 • https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_release- notes/content/ch_hdf_relnotes.html#centos7 • https://community.hortonworks.com/articles/108947/minifi-for-ble-bluetooth-low-energy-beacon- data-in.html • https://community.hortonworks.com/content/kbentry/107379/minifi-for-image-capture-and- ingestion-from-raspbe.html MiniFi Java Agent Installation (Latest version 0.5) Pre Step 2
  25. 25. 25 © Hortonworks Inc. 2011–2018. All rights reserved. • You need to do some extra work here compiling for your platform • This is better for smaller devices. MiniFi C++ Agent Installation Pre Step 3 (Optional) https://community.hortonworks.com/articles/167193/building-and-running-minifi-cpp-in-orangepi-zero.html https://nifi.apache.org/minifi/getting-started.html
  26. 26. 26 © Hortonworks Inc. 2011–2018. All rights reserved. • https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processo r=CPU • Install build tools and build from scratch • Walk through install: https://community.hortonworks.com/articles/176932/apache-deep-learning- 101-using-apache-mxnet-on-the.html Install Apache MXNet Pre Step 4
  27. 27. 27 © Hortonworks Inc. 2011–2018. All rights reserved. • Soon Using MiniFi Offline Designer • Design a flow in Apache NiFi UI MiniFi Flow Design Step 1 https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html
  28. 28. 28 © Hortonworks Inc. 2011–2018. All rights reserved. • Save to NiFi Registry • Export Template MiniFi Version Step 2
  29. 29. 29 © Hortonworks Inc. 2011–2018. All rights reserved. MiniFi Toolkit • config.sh transform yourtemplate.xml config.yml MiniFi Convert XML Template to Config YAML Step 3 https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html cd minifi-toolkit-0.5.0 bin/config.sh transform /somedir/16749-rainbowforminifi.xml config.yml Java home: /Library/Java/Home MiNiFi Toolkit home: /somedir/minifi-toolkit-0.5.0 No validation errors found in converted configuration.
  30. 30. 30 © Hortonworks Inc. 2011–2018. All rights reserved. • Copy config.yml to Device’s MiniFi conf directory (SCP or C2 Server) • Restart MiniFi Agent • Check logs • Check status • ./bin/minifi.sh flowStatus systemdiagnostics:heap,processorstats,contentrepositoryusage,flowfilerepositoryusage, garbagecollection MiniFi Deployment Step 4 https://nifi.apache.org/minifi/minifi-toolkit.html https://nifi.apache.org/minifi/download.html https://community.hortonworks.com/articles/167193/building -and-running-minifi-cpp-in-orangepi-zero.html
  31. 31. 31 © Hortonworks Inc. 2011–2018. All rights reserved. MiniFi Deployment – C2 Server Step 4b – Cutting Edge Option https://medium.freecodecamp.org/building-an-iiot-system-using-apache-nifi-mqtt-and-raspberry-pi- ce1d6ed565bc • Add PullHttpChangeIngestor to MiniFi’s Local conf/bootstrap.conf https://github.com/apache/nifi-minifi/tree/master/minifi-c2 https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Command+and+Control
  32. 32. 32 © Hortonworks Inc. 2011–2018. All rights reserved. • Make sure remote host is set • If changing settings, restart • Add a Input Port to the Workspace • Green Means Go! Apache NiFi Server Receive Step 5
  33. 33. 33 © Hortonworks Inc. 2011–2018. All rights reserved. • Store in HDFS Securely • Build or Insert into Hive Tables • Store in Hbase or Phoenix • Query, Report, Analyze • Analyze data with Spark ML Data Stored Final
  34. 34. 34 © Hortonworks Inc. 2011–2018. All rights reserved. References v https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html v https://community.hortonworks.com/articles/80339/iot-capturing-photos-and-analyzing-the-image-with.html v https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html v https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html v https://community.hortonworks.com/articles/155475/powering-apache-minifi-flows-with-a-movidius-neura.html v https://community.hortonworks.com/articles/155606/iot-ingesting-gps-data-from-odroid-xu4-devices-wit.html v https://community.hortonworks.com/articles/155604/iot-ingesting-camera-data-from-nanopi-duo-devices.html v https://community.hortonworks.com/articles/198855/executing-tensorflow-classifications-from-apache-n.html
  35. 35. 35 © Hortonworks Inc. 2011–2018. All rights reserved. Contact https://community.hortonworks.com/users/9304/tspann.html https://dzone.com/users/297029/bunkertor.html https://www.meetup.com/futureofdata-princeton/ https://twitter.com/PaaSDev @PaasDev https://github.com/tspannhw/StrataNYC2018
  36. 36. 36 © Hortonworks Inc. 2011–2018. All rights reserved. Hortonworks Community Connection Read access for everyone, join to participate and be recognized • Full Q&A Platform (like StackOverflow) • Knowledge Base Articles • Code Samples and Repositories
  37. 37. 37 © Hortonworks Inc. 2011–2018. All rights reserved. Community Engagement Participate now at: community.hortonworks.com© Hortonworks Inc. 2011 – 2015. All Rights Reserved 4,000+ Registered Users 10,000+ Answers 15,000+ Technical Assets One Website!

×