Successfully reported this slideshow.
Your SlideShare is downloading. ×

Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 46 Ad

Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021

Download to read offline

Using FLaNK with InfluxDB for EdgeAI IoT at Scale
Timothy from StreamNative take you on a hands-on deep-dive on using Pulsar, Apache NiFi + Edge Flow Manager + MiniFi Agents with Apache MXNet, OpenVino, TensorFlow Lite, and other Deep Learning Libraries on the actual edge devices including Raspberry Pi with Movidius 2, Google Coral TPU and NVidia Jetson Nano. The team run deep learning models on the edge devices and send images, and capture real-time GPS and sensor data. Their low-coding IoT applications provide easy edge routing, transformation, data acquisition and alerting before they decide what data to stream real-time to their data space. These edge applications classify images and sensor readings real-time at the edge and then send Deep Learning results to Flink SQL and Apache NiFi for transformation, parsing, enrichment, querying, filtering and merging data to InfluxDB.

Using FLaNK with InfluxDB for EdgeAI IoT at Scale
Timothy from StreamNative take you on a hands-on deep-dive on using Pulsar, Apache NiFi + Edge Flow Manager + MiniFi Agents with Apache MXNet, OpenVino, TensorFlow Lite, and other Deep Learning Libraries on the actual edge devices including Raspberry Pi with Movidius 2, Google Coral TPU and NVidia Jetson Nano. The team run deep learning models on the edge devices and send images, and capture real-time GPS and sensor data. Their low-coding IoT applications provide easy edge routing, transformation, data acquisition and alerting before they decide what data to stream real-time to their data space. These edge applications classify images and sensor readings real-time at the edge and then send Deep Learning results to Flink SQL and Apache NiFi for transformation, parsing, enrichment, querying, filtering and merging data to InfluxDB.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021 (20)

Advertisement

More from InfluxData (20)

Recently uploaded (20)

Advertisement

Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021

  1. 1. Using FLaNK with InfluxDB for EdgeAI IoT at Scale Tim Spann | Developer Advocate
  2. 2. streamnative.io Tim Spann, Developer Advocate DZone Zone Leader and Big Data MVB Data DJay
  3. 3. streamnative.io Founded by the original developers of Apache Pulsar and Apache BookKeeper, StreamNative builds a cloud-native event streaming platform that enables enterprises to easily access data as real-time event streams.
  4. 4. streamnative.io Apache is an open source, cloud-native distributed messaging and streaming platform.
  5. 5. streamnative.io Powered by Apache Pulsar, StreamNative provides a cloud-native, real-time messaging and streaming platform to support multi-cloud and hybrid cloud strategies. Built for Containers Cloud Native Flink SQL StreamNative Cloud
  6. 6. © 2021  InfluxData Inc. All Rights Reserved. ● Apache Flink ● Apache Pulsar ● StreamNative's Flink Connector for Pulsar ● Apache NiFi ● Apache +++ FLiP(N) Stack
  7. 7. © 2021  InfluxData Inc. All Rights Reserved. Apache Pulsar InfluxDB Sink https:/ /pulsar.apache.org/docs/en/io-influxdb-sink/
  8. 8. © 2021  InfluxData Inc. All Rights Reserved. Apache Pulsar InfluxDB Sink bin/pulsar-admin sink stop --name influxdb-sink-jetson --namespace default --tenant public bin/pulsar-admin sinks delete --tenant public --namespace default --name influxdb-sink-jetson bin/pulsar-admin sinks create --archive ./connectors/pulsar-io-influxdb-2.8.0.nar --tenant public --namespace default --name influxdb-sink-jetson --sink-config-file conf/influxcloud.yml --inputs jetsoninflux --parallelism 1 bin/pulsar-admin sinks get --tenant public --namespace default --name influxdb-sink-jetson bin/pulsar-admin sinks status --tenant public --namespace default --name influxdb-sink-jetson https:/ /pulsar.apache.org/docs/en/io-influxdb-sink/
  9. 9. © 2021  InfluxData Inc. All Rights Reserved. InfluxDB Telegraf MQTT https:/ /www.influxdata.com/integration/mqtt-monitoring/ https:/ /github.com/streamnative/mop MQTT on Pulsar (MoP)
  10. 10. © 2021  InfluxData Inc. All Rights Reserved. InfluxDB Telegraf RabbitMQ https:/ /www.influxdata.com/integration/mqtt-monitoring/ https:/ /github.com/streamnative/aop AMQP on Pulsar (AoP) https:/ /hub.streamnative.io/connectors/amqp-1-0-sink/ https:/ /hub.streamnative.io/connectors/amqp-1-0-source
  11. 11. streamnative.io • Guaranteed delivery • Data buffering - Backpressure - Pressure release • Prioritized queuing • Flow specific QoS - Latency vs. throughput - Loss tolerance • Data provenance • Supports push and pull models • Hundreds of processors • Visual command and control • Over a 300 sources • Flow templates • Pluggable/multi-role security • Designed for extension • Clustering • Version Control Why Apache NiFi?
  12. 12. © 2021  InfluxData Inc. All Rights Reserved. InfluxDB via Apache NiFi https:/ /www.influxdata.com/integration/mqtt-monitoring/ https:/ /github.com/streamnative/mop https://github.com/influxdata/nifi-influxdb-bundle https://www.influxdata.com/blog/building-a-data-stream-for-iot-with-nifi-and-influxdb/
  13. 13. streamnative.io ● Unified computing engine ● Batch processing is a special case of stream processing ● Stateful processing ● Massive Scalability ● Flink SQL for queries, inserts against Pulsar Topics ● Streaming Analytics ● Continuous SQL ● Continuous ETL ● Complex Event Processing ● Standard SQL Powered by Apache Calcite Why Apache Flink?
  14. 14. streamnative.io https://flink.apache.org/2019/05/03/pulsar-flink.html https://github.com/streamnative/pulsar-flink https://streamnative.io/en/blog/release/2021-04-20-flink-sql-on-st reamnative-cloud Flink + Pulsar
  15. 15. StreamNative Hub StreamNative Cloud Unified Batch and Stream COMPUTING Batch (Batch + Stream) Unified Batch and Stream STORAGE Offload (Queuing + Streaming) Apache Flink - Apache Pulsar - Apache NiFi <-> Devices <-> influxdb Tiered Storage Pulsar --- KoP --- MoP --- Websocket --- HTTP Pulsar Sink Pulsar Sink Streaming Edge Gateway Protocols End-to-End Streaming Edge App
  16. 16. DEMO Using EdgeAI Devices
  17. 17. NVIDIA Device
  18. 18. Apache NiFi
  19. 19. Apache NiFi Data to Enhance IoT Stream
  20. 20. Apache NiFi
  21. 21. Apache NiFi Influxdb properties Controller properties
  22. 22. Apache NiFi
  23. 23. Apache NiFi
  24. 24. Apache NiFi
  25. 25. Apache Pulsar
  26. 26. StreamNative Cloud
  27. 27. StreamNative Cloud
  28. 28. StreamNative Cloud
  29. 29. StreamNative Cloud
  30. 30. StreamNative Flink SQL
  31. 31. StreamNative Flink SQL
  32. 32. StreamNative Flink SQL
  33. 33. StreamNative Flink SQL
  34. 34. Influx Data Explorer
  35. 35. Influx Data Explorer
  36. 36. Influx Data Explorer
  37. 37. Influx Data Explorer
  38. 38. Influx Data Explorer
  39. 39. Influx Data Explorer
  40. 40. Influx Data Explorer
  41. 41. Influx Data Explorer
  42. 42. ● https://github.com/tspannhw/FLiP-InfluxDB ● https://pulsar.apache.org/docs/en/io-influxdb-sink/ ● https://pulsar.apache.org/docs/en/io-overview/ ● https://github.com/tspannhw/Flip-solr ● https://github.com/tspannhw/Flip-transit ● https://github.com/tspannhw/Flip-iot ● https://www.datainmotion.dev/2020/10/flank-strea ming-edgeai-on-new-nvidia.html ● https://github.com/tspannhw/minifi-jetson-nano ● https://github.com/tspannhw/SpeakerProfile/ ● https://hub.streamnative.io/connectors/influxdb-sin k/2.5.1/ ● https://github.com/tspannhw/minifi-xaviernx/
  43. 43. Let’s Keep in Touch! Speaker Name Speaker title @PaasDev https://www.linkedin.com/in/timothyspann https://github.com/tspannhw
  44. 44. Questions?
  45. 45. Thank You

×