In this talk, Edgar Orendain walks through a modern real-time streaming application serving as a reference framework for developing a big data pipeline, complete with a broad range of use cases and powerful reusable core components.
Modern applications can ingest data and leverage analytics in real-time. These analytics are based on machine learning models typically built using historical big data. This reference application provides examples of connecting data-in-motion analytics to your application based on Big Data.
We review code, best practices and considerations involved when integrating different components into a complete data platform. From IoT sensor data collection, to flow management, real-time stream processing and analytics, through to machine learning and prediction, this reference project aims to help developers seed their own open source solutions – fast.