We walk through a design of an open source Predictive Analytics Pipeline with MiniFi on a Raspberry Pi running Python and ingesting SenseHat sensor readings, a webcam image and running Inception classification on that image. MiniFi sends the resulting JSON and Image along with data provenance to an Apache NiFi server for preprocessing. It is then routed, converted, queried and stored as an Apache Hive table in Apache ORC format.
Apache NiFi, MiniFi, Apache Spark, HDP 3.0, HDF 3.1.2, Hortonworks Schema Registry Raspberry Pi with Intel Movidius running TensorFlow Python JSON Apache Avro Apache Hive HDFS
We also touched on integration with Blockchain, Ethereum and accessing REST API and Websocket interfaces offered by online blockchain explorers like Etherdelta and Etherscan.
This talk was June 28th, 2018 in Hamilton, NJ as part of the Future of Data Princeton and NJ Blockchain/Big Data joint meetup.