Be the first to like this
Mining large streams of real time data has recently grown to one of the key challenges for Big Data community in both, industry and academia. At the same time the concept of Smart City has gained significant acclaim by providing user-oriented services in the modern cities. One of the key part of the smart city is Intelligent Transportation System. Usual consideration of ITS implementation assumes the large amount of various sensor, both stationary and mobile, being deployed in the city area. Such sensors might in particular include video cameras, traffic detectors, or devices attached to various public transport means. The studies proved the usefulness of this data streams for incident detection, traffic monitoring and congestion detection, optimization of public transport schedules and operations. In this speech we present the architecture of complex system developed within EU-H2020 project VaVeL and deployed in a cluster operating in the City of Warsaw. We will discuss in details the core components of the stream and batch processing pipelines, with particular focus on its implementation, cluster-based deployment and integration, fault detection and maintenance.