SlideShare a Scribd company logo

Kappa Architecture, IoT of the cars - LibreCon 2016

•
•
LibreCon
LibreCon

Juan Tomás García, Open Sistemas

Internet
1 of 23
Download to read offline
Kappa Architecture: IoT of the cars Juantomás García
Who am I • Data Solutions Manager at Open Sistemas • Co-author of the first spanish book about free software “La pastilla roja” • Former president of Hispalinux (Spanish Linux User Group). • Co-organizer of Machine Learning Spain Meetup
What’s Kappa Architecture July 2, 2014 Jay Kreps coined the term Kappa Architecture in an article for O’reilly Radar
Who is Jay Kreps Jay has been involved in lots of projects: Author of the essay: The Log: What every software engineer should know about real-time data's unifying abstraction (12/16/2013) https://engineering.linkedin.com/distributed-systems/log-what-every-software- engineer-should-know-about-real-time-datas-unifying
Who is Jay Kreps Author of the book: I ♥ Logs
Who is Jay Kreps Involved with projects as: Apache Kafka Apache Samza Voldemort Azkaban Ex-Linkedin Now co-founder and CEO of Confluent
Usual data flow
Kappa Architecture Way
Tools we use to implement Kappa Architecture
Tools we use to implement Kappa Architecture
Tools we use to implement Kappa Architecture • Our favorite kafka features. • Easy to use • Robust & Resilient • Kafka connectors • Kafka streams • Compact logs • Very fast writes and reads
Tools we use to implement Kappa Architecture
Some favorite Spark features • If you have an schema spark SQL is perfect. • Spark streaming work very fine with spark and almost all streaming sources. • Structured queries will be a huge advance. • We love scala and scala is the spirit of spark.
Some favorite Spark features • We love code like this:
A real case • One of our clients want to monitor all the cars informations via OBD II • OBD II is a car interface with the car electronics. • Client develop an app to read all the car information throw a ODB II with bluetooth
A real case
First Problems • We need to scale the rest interfaces too many requests. • MySQL don’t scale • Client wants to make realtime expensive queries.
Some Numbers
Architecture V2
Architecture KappaV3
Now we’re more flexible • We can have queries like: • What are the drivers that are not client of the X gas brand, has a few gas and are near of gas station of the brand X and if true, send a notification with a discount coupon and a link with the route.
Takeaways • Kappa architecture is not a silver bullet but helps in a lot of solutions. • Kafka + spark streaming is our favorite tools • There a lots of improvements: • OLAP like Apache Druid • Graph databases like neo4j • Kafka streams and compacts logs
¿Questions? • My contact info: • Juantomás García • juantomas@opensistemas.com • @juantomas

Recommended

Building Reactive Real-time Data Pipeline
Building Reactive Real-time Data PipelineBuilding Reactive Real-time Data Pipeline
Building Reactive Real-time Data PipelineTrieu Nguyen
 
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021InfluxData
 
Lambda Architecture and open source technology stack for real time big data
Lambda Architecture and open source technology stack for real time big dataLambda Architecture and open source technology stack for real time big data
Lambda Architecture and open source technology stack for real time big dataTrieu Nguyen
 
Lambda Architecture 2.0 for Reactive AB Testing
Lambda Architecture 2.0 for Reactive AB TestingLambda Architecture 2.0 for Reactive AB Testing
Lambda Architecture 2.0 for Reactive AB TestingTrieu Nguyen
 
Big Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David GiardBig Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David GiardITCamp
 
Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...
Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...
Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...Big Data Spain
 
Gain Deep Visibility into APIs and Integrations with Anypoint Monitoring
Gain Deep Visibility into APIs and Integrations with Anypoint MonitoringGain Deep Visibility into APIs and Integrations with Anypoint Monitoring
Gain Deep Visibility into APIs and Integrations with Anypoint MonitoringInfluxData
 
From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...
From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...
From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...Databricks
 

Recommended

Building Reactive Real-time Data Pipeline
Building Reactive Real-time Data PipelineBuilding Reactive Real-time Data Pipeline
Building Reactive Real-time Data PipelineTrieu Nguyen
 
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021InfluxData
 
Lambda Architecture and open source technology stack for real time big data
Lambda Architecture and open source technology stack for real time big dataLambda Architecture and open source technology stack for real time big data
Lambda Architecture and open source technology stack for real time big dataTrieu Nguyen
 
Lambda Architecture 2.0 for Reactive AB Testing
Lambda Architecture 2.0 for Reactive AB TestingLambda Architecture 2.0 for Reactive AB Testing
Lambda Architecture 2.0 for Reactive AB TestingTrieu Nguyen
 
Big Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David GiardBig Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David GiardITCamp
 
Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...
Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...
Solving the Industry 4.0 challenges on the logistics domain using Apache Meso...Big Data Spain
 
Gain Deep Visibility into APIs and Integrations with Anypoint Monitoring
Gain Deep Visibility into APIs and Integrations with Anypoint MonitoringGain Deep Visibility into APIs and Integrations with Anypoint Monitoring
Gain Deep Visibility into APIs and Integrations with Anypoint MonitoringInfluxData
 
From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...
From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...
From “All-at-Once, Once-a-Day” to “A-Little-Each-Time, All-the-Time” with Ema...Databricks
 
DataXDay - Real-Time Access log analysis
DataXDay - Real-Time Access log analysis DataXDay - Real-Time Access log analysis
DataXDay - Real-Time Access log analysis DataXDay Conference by Xebia
 
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...Databricks
 
MongoDB .local London 2019: The Tech Behind Connected Car
MongoDB .local London 2019: The Tech Behind Connected CarMongoDB .local London 2019: The Tech Behind Connected Car
MongoDB .local London 2019: The Tech Behind Connected CarMongoDB
 
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDBScylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDBScyllaDB
 
RealTime Recommendations @Netflix - Spark
RealTime Recommendations @Netflix - SparkRealTime Recommendations @Netflix - Spark
RealTime Recommendations @Netflix - SparkNitin S
 
[WSO2Con EU 2018] Tooling for Observability
[WSO2Con EU 2018] Tooling for Observability[WSO2Con EU 2018] Tooling for Observability
[WSO2Con EU 2018] Tooling for ObservabilityWSO2
 
Ingesting IoT data in Food Processing
Ingesting IoT data in Food ProcessingIngesting IoT data in Food Processing
Ingesting IoT data in Food Processingconfluent
 
OVH Analytics Data Compute - Apache Spark Cluster as a Service
OVH Analytics Data Compute - Apache Spark Cluster as a ServiceOVH Analytics Data Compute - Apache Spark Cluster as a Service
OVH Analytics Data Compute - Apache Spark Cluster as a ServiceOVHcloud
 
Ovh analytics data compute with apache spark as a service meetup ovh bordeaux
Ovh analytics data compute with apache spark as a service meetup ovh bordeauxOvh analytics data compute with apache spark as a service meetup ovh bordeaux
Ovh analytics data compute with apache spark as a service meetup ovh bordeauxMojtaba Imani
 
Avoid SPOF in Cloud-native Apps
Avoid SPOF in Cloud-native AppsAvoid SPOF in Cloud-native Apps
Avoid SPOF in Cloud-native AppsThang Chung
 
Building Notebook-based AI Pipelines with Elyra and Kubeflow
Building Notebook-based AI Pipelines with Elyra and KubeflowBuilding Notebook-based AI Pipelines with Elyra and Kubeflow
Building Notebook-based AI Pipelines with Elyra and KubeflowDatabricks
 
SAIS2018 - Fact Store At Netflix Scale
SAIS2018 - Fact Store At Netflix ScaleSAIS2018 - Fact Store At Netflix Scale
SAIS2018 - Fact Store At Netflix ScaleNitin S
 
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...InfluxData
 
How to Build Cloud-based Microservice Environments with Docker and VoltDB
How to Build Cloud-based Microservice Environments with Docker and VoltDBHow to Build Cloud-based Microservice Environments with Docker and VoltDB
How to Build Cloud-based Microservice Environments with Docker and VoltDBVoltDB
 
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of SwaggerWSO2
 
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...HostedbyConfluent
 
The Expert Guide to Fast Data
The Expert Guide to Fast Data The Expert Guide to Fast Data
The Expert Guide to Fast Data VoltDB
 
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli Spark Summit
 
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...apidays
 
Mike Stonebraker on Designing An Architecture For Real-time Event Processing
Mike Stonebraker on Designing An Architecture For Real-time Event ProcessingMike Stonebraker on Designing An Architecture For Real-time Event Processing
Mike Stonebraker on Designing An Architecture For Real-time Event ProcessingVoltDB
 
Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Anton Nazaruk
 
Librecon 2016 bilbao: kappa architecture IoT of the cars
Librecon 2016 bilbao: kappa architecture IoT of the carsLibrecon 2016 bilbao: kappa architecture IoT of the cars
Librecon 2016 bilbao: kappa architecture IoT of the carsJuantomás García Molina
 

More Related Content

What's hot

Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...Databricks
 
MongoDB .local London 2019: The Tech Behind Connected Car
MongoDB .local London 2019: The Tech Behind Connected CarMongoDB .local London 2019: The Tech Behind Connected Car
MongoDB .local London 2019: The Tech Behind Connected CarMongoDB
 
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDBScylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDBScyllaDB
 
RealTime Recommendations @Netflix - Spark
RealTime Recommendations @Netflix - SparkRealTime Recommendations @Netflix - Spark
RealTime Recommendations @Netflix - SparkNitin S
 
[WSO2Con EU 2018] Tooling for Observability
[WSO2Con EU 2018] Tooling for Observability[WSO2Con EU 2018] Tooling for Observability
[WSO2Con EU 2018] Tooling for ObservabilityWSO2
 
Ingesting IoT data in Food Processing
Ingesting IoT data in Food ProcessingIngesting IoT data in Food Processing
Ingesting IoT data in Food Processingconfluent
 
OVH Analytics Data Compute - Apache Spark Cluster as a Service
OVH Analytics Data Compute - Apache Spark Cluster as a ServiceOVH Analytics Data Compute - Apache Spark Cluster as a Service
OVH Analytics Data Compute - Apache Spark Cluster as a ServiceOVHcloud
 
Ovh analytics data compute with apache spark as a service meetup ovh bordeaux
Ovh analytics data compute with apache spark as a service meetup ovh bordeauxOvh analytics data compute with apache spark as a service meetup ovh bordeaux
Ovh analytics data compute with apache spark as a service meetup ovh bordeauxMojtaba Imani
 
Avoid SPOF in Cloud-native Apps
Avoid SPOF in Cloud-native AppsAvoid SPOF in Cloud-native Apps
Avoid SPOF in Cloud-native AppsThang Chung
 
Building Notebook-based AI Pipelines with Elyra and Kubeflow
Building Notebook-based AI Pipelines with Elyra and KubeflowBuilding Notebook-based AI Pipelines with Elyra and Kubeflow
Building Notebook-based AI Pipelines with Elyra and KubeflowDatabricks
 
SAIS2018 - Fact Store At Netflix Scale
SAIS2018 - Fact Store At Netflix ScaleSAIS2018 - Fact Store At Netflix Scale
SAIS2018 - Fact Store At Netflix ScaleNitin S
 
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...InfluxData
 
How to Build Cloud-based Microservice Environments with Docker and VoltDB
How to Build Cloud-based Microservice Environments with Docker and VoltDBHow to Build Cloud-based Microservice Environments with Docker and VoltDB
How to Build Cloud-based Microservice Environments with Docker and VoltDBVoltDB
 
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of SwaggerWSO2
 
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...HostedbyConfluent
 
The Expert Guide to Fast Data
The Expert Guide to Fast Data The Expert Guide to Fast Data
The Expert Guide to Fast Data VoltDB
 
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli Spark Summit
 
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...apidays
 
Mike Stonebraker on Designing An Architecture For Real-time Event Processing
Mike Stonebraker on Designing An Architecture For Real-time Event ProcessingMike Stonebraker on Designing An Architecture For Real-time Event Processing
Mike Stonebraker on Designing An Architecture For Real-time Event ProcessingVoltDB
 

What's hot (20)

DataXDay - Real-Time Access log analysis
DataXDay - Real-Time Access log analysis DataXDay - Real-Time Access log analysis
DataXDay - Real-Time Access log analysis
 
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Qualit...
 
MongoDB .local London 2019: The Tech Behind Connected Car
MongoDB .local London 2019: The Tech Behind Connected CarMongoDB .local London 2019: The Tech Behind Connected Car
MongoDB .local London 2019: The Tech Behind Connected Car
 
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDBScylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
Scylla Summit 2022: Scalable and Sustainable Supply Chains with DLT and ScyllaDB
 
RealTime Recommendations @Netflix - Spark
RealTime Recommendations @Netflix - SparkRealTime Recommendations @Netflix - Spark
RealTime Recommendations @Netflix - Spark
 
[WSO2Con EU 2018] Tooling for Observability
[WSO2Con EU 2018] Tooling for Observability[WSO2Con EU 2018] Tooling for Observability
[WSO2Con EU 2018] Tooling for Observability
 
Ingesting IoT data in Food Processing
Ingesting IoT data in Food ProcessingIngesting IoT data in Food Processing
Ingesting IoT data in Food Processing
 
OVH Analytics Data Compute - Apache Spark Cluster as a Service
OVH Analytics Data Compute - Apache Spark Cluster as a ServiceOVH Analytics Data Compute - Apache Spark Cluster as a Service
OVH Analytics Data Compute - Apache Spark Cluster as a Service
 
Ovh analytics data compute with apache spark as a service meetup ovh bordeaux
Ovh analytics data compute with apache spark as a service meetup ovh bordeauxOvh analytics data compute with apache spark as a service meetup ovh bordeaux
Ovh analytics data compute with apache spark as a service meetup ovh bordeaux
 
Avoid SPOF in Cloud-native Apps
Avoid SPOF in Cloud-native AppsAvoid SPOF in Cloud-native Apps
Avoid SPOF in Cloud-native Apps
 
Building Notebook-based AI Pipelines with Elyra and Kubeflow
Building Notebook-based AI Pipelines with Elyra and KubeflowBuilding Notebook-based AI Pipelines with Elyra and Kubeflow
Building Notebook-based AI Pipelines with Elyra and Kubeflow
 
SAIS2018 - Fact Store At Netflix Scale
SAIS2018 - Fact Store At Netflix ScaleSAIS2018 - Fact Store At Netflix Scale
SAIS2018 - Fact Store At Netflix Scale
 
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
 
How to Build Cloud-based Microservice Environments with Docker and VoltDB
How to Build Cloud-based Microservice Environments with Docker and VoltDBHow to Build Cloud-based Microservice Environments with Docker and VoltDB
How to Build Cloud-based Microservice Environments with Docker and VoltDB
 
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
[WSO2Con EU 2018] OpenAPI Specification 3 - The Evolution of Swagger
 
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...
 
The Expert Guide to Fast Data
The Expert Guide to Fast Data The Expert Guide to Fast Data
The Expert Guide to Fast Data
 
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
 
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
apidays LIVE Paris 2021 - Stargate.io, An OSS Api Layer for your Cassandra by...
 
Mike Stonebraker on Designing An Architecture For Real-time Event Processing
Mike Stonebraker on Designing An Architecture For Real-time Event ProcessingMike Stonebraker on Designing An Architecture For Real-time Event Processing
Mike Stonebraker on Designing An Architecture For Real-time Event Processing
 

Viewers also liked

Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Anton Nazaruk
 
Librecon 2016 bilbao: kappa architecture IoT of the cars
Librecon 2016 bilbao: kappa architecture IoT of the carsLibrecon 2016 bilbao: kappa architecture IoT of the cars
Librecon 2016 bilbao: kappa architecture IoT of the carsJuantomás García Molina
 
Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...
Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...
Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...Mathieu Dumoulin
 
February 2017 HUG: Exactly-once end-to-end processing with Apache Apex
February 2017 HUG: Exactly-once end-to-end processing with Apache ApexFebruary 2017 HUG: Exactly-once end-to-end processing with Apache Apex
February 2017 HUG: Exactly-once end-to-end processing with Apache ApexYahoo Developer Network
 
Data Pipelines with Apache Kafka
Data Pipelines with Apache KafkaData Pipelines with Apache Kafka
Data Pipelines with Apache KafkaBen Stopford
 
Big Data Architecture
Big Data ArchitectureBig Data Architecture
Big Data ArchitectureGuido Schmutz
 
El futuro Data Driven en e-Learning y RR.HH.
El futuro Data Driven en e-Learning y RR.HH.El futuro Data Driven en e-Learning y RR.HH.
El futuro Data Driven en e-Learning y RR.HH.OpenSistemas
 
El software como acción humana
El software como acción humanaEl software como acción humana
El software como acción humanaOpenSistemas
 
DWS16 - Future Networks forum - Anna Krzyzanowska European Commission
DWS16 - Future Networks forum - Anna Krzyzanowska European CommissionDWS16 - Future Networks forum - Anna Krzyzanowska European Commission
DWS16 - Future Networks forum - Anna Krzyzanowska European CommissionIDATE DigiWorld
 
Connected Cars Quickly Becoming Part of the Internet of Things (IoT)
Connected Cars Quickly Becoming Part of the Internet of Things (IoT)Connected Cars Quickly Becoming Part of the Internet of Things (IoT)
Connected Cars Quickly Becoming Part of the Internet of Things (IoT)ParthaS
 
Getting The Deal Through: Telecoms & Media Market Intelligence 2016
Getting The Deal Through: Telecoms & Media Market Intelligence 2016Getting The Deal Through: Telecoms & Media Market Intelligence 2016
Getting The Deal Through: Telecoms & Media Market Intelligence 2016Matheson Law Firm
 
Отражение цифровизации в решениях Cisco
Отражение цифровизации в решениях CiscoОтражение цифровизации в решениях Cisco
Отражение цифровизации в решениях CiscoCisco Russia
 
Apache spark y cómo lo usamos en nuestros proyectos
Apache spark y cómo lo usamos en nuestros proyectosApache spark y cómo lo usamos en nuestros proyectos
Apache spark y cómo lo usamos en nuestros proyectosOpenSistemas
 
Cloudera - Enabling the IoT Revolution Driving Insights in a Connected World
Cloudera - Enabling the IoT Revolution Driving Insights in a Connected WorldCloudera - Enabling the IoT Revolution Driving Insights in a Connected World
Cloudera - Enabling the IoT Revolution Driving Insights in a Connected Worldandreas kuncoro
 
Car talk - The Conversational Future of Automobiles
Car talk - The Conversational Future of AutomobilesCar talk - The Conversational Future of Automobiles
Car talk - The Conversational Future of AutomobilesSudha Jamthe
 
Marsden CNRS European net neutrality law & Guidelines 12092016
Marsden CNRS European net neutrality law & Guidelines 12092016Marsden CNRS European net neutrality law & Guidelines 12092016
Marsden CNRS European net neutrality law & Guidelines 12092016Chris Marsden
 

Viewers also liked (20)

Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?
 
Librecon 2016 bilbao: kappa architecture IoT of the cars
Librecon 2016 bilbao: kappa architecture IoT of the carsLibrecon 2016 bilbao: kappa architecture IoT of the cars
Librecon 2016 bilbao: kappa architecture IoT of the cars
 
ASPgems - kappa architecture
ASPgems - kappa architectureASPgems - kappa architecture
ASPgems - kappa architecture
 
Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...
Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...
Streaming Architecture to Connect Everything (Including Hybrid Cloud) - Strat...
 
February 2017 HUG: Exactly-once end-to-end processing with Apache Apex
February 2017 HUG: Exactly-once end-to-end processing with Apache ApexFebruary 2017 HUG: Exactly-once end-to-end processing with Apache Apex
February 2017 HUG: Exactly-once end-to-end processing with Apache Apex
 
Data Pipelines with Apache Kafka
Data Pipelines with Apache KafkaData Pipelines with Apache Kafka
Data Pipelines with Apache Kafka
 
Big Data Architecture
Big Data ArchitectureBig Data Architecture
Big Data Architecture
 
El futuro Data Driven en e-Learning y RR.HH.
El futuro Data Driven en e-Learning y RR.HH.El futuro Data Driven en e-Learning y RR.HH.
El futuro Data Driven en e-Learning y RR.HH.
 
El software como acción humana
El software como acción humanaEl software como acción humana
El software como acción humana
 
TRAINING1
TRAINING1TRAINING1
TRAINING1
 
DWS16 - Future Networks forum - Anna Krzyzanowska European Commission
DWS16 - Future Networks forum - Anna Krzyzanowska European CommissionDWS16 - Future Networks forum - Anna Krzyzanowska European Commission
DWS16 - Future Networks forum - Anna Krzyzanowska European Commission
 
Connected Cars Quickly Becoming Part of the Internet of Things (IoT)
Connected Cars Quickly Becoming Part of the Internet of Things (IoT)Connected Cars Quickly Becoming Part of the Internet of Things (IoT)
Connected Cars Quickly Becoming Part of the Internet of Things (IoT)
 
Getting The Deal Through: Telecoms & Media Market Intelligence 2016
Getting The Deal Through: Telecoms & Media Market Intelligence 2016Getting The Deal Through: Telecoms & Media Market Intelligence 2016
Getting The Deal Through: Telecoms & Media Market Intelligence 2016
 
Отражение цифровизации в решениях Cisco
Отражение цифровизации в решениях CiscoОтражение цифровизации в решениях Cisco
Отражение цифровизации в решениях Cisco
 
Apache spark y cómo lo usamos en nuestros proyectos
Apache spark y cómo lo usamos en nuestros proyectosApache spark y cómo lo usamos en nuestros proyectos
Apache spark y cómo lo usamos en nuestros proyectos
 
Cloudera - Enabling the IoT Revolution Driving Insights in a Connected World
Cloudera - Enabling the IoT Revolution Driving Insights in a Connected WorldCloudera - Enabling the IoT Revolution Driving Insights in a Connected World
Cloudera - Enabling the IoT Revolution Driving Insights in a Connected World
 
Global mobile trends
Global mobile trendsGlobal mobile trends
Global mobile trends
 
Car talk - The Conversational Future of Automobiles
Car talk - The Conversational Future of AutomobilesCar talk - The Conversational Future of Automobiles
Car talk - The Conversational Future of Automobiles
 
Marsden CNRS European net neutrality law & Guidelines 12092016
Marsden CNRS European net neutrality law & Guidelines 12092016Marsden CNRS European net neutrality law & Guidelines 12092016
Marsden CNRS European net neutrality law & Guidelines 12092016
 
Claire Bury - A New Telecoms Regulatory Framework
Claire Bury - A New Telecoms Regulatory FrameworkClaire Bury - A New Telecoms Regulatory Framework
Claire Bury - A New Telecoms Regulatory Framework
 

Similar to Kappa Architecture, IoT of the cars - LibreCon 2016

Meetup big data developers 2017 madrid - spark real use cases
Meetup big data developers 2017 madrid - spark real use casesMeetup big data developers 2017 madrid - spark real use cases
Meetup big data developers 2017 madrid - spark real use casesJuantomás García Molina
 
Gdg cloud london 2017 kappa architecture 2.0 copia
Gdg cloud london 2017 kappa architecture 2.0 copiaGdg cloud london 2017 kappa architecture 2.0 copia
Gdg cloud london 2017 kappa architecture 2.0 copiaJuantomás García Molina
 
Codemotion madrid 2017 Arquitectura kappa 2.0
Codemotion madrid 2017 Arquitectura kappa 2.0Codemotion madrid 2017 Arquitectura kappa 2.0
Codemotion madrid 2017 Arquitectura kappa 2.0Juantomás García Molina
 
Datascience lab 2017 odessa kappa architecture 2.0
Datascience lab 2017 odessa kappa architecture 2.0Datascience lab 2017 odessa kappa architecture 2.0
Datascience lab 2017 odessa kappa architecture 2.0Juantomás García Molina
 
DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...
DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...
DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...GeeksLab Odessa
 
Lambda architecture for real time big data
Lambda architecture for real time big dataLambda architecture for real time big data
Lambda architecture for real time big dataTrieu Nguyen
 
Data Science at Scale: Using Apache Spark for Data Science at Bitly
Data Science at Scale: Using Apache Spark for Data Science at BitlyData Science at Scale: Using Apache Spark for Data Science at Bitly
Data Science at Scale: Using Apache Spark for Data Science at BitlySarah Guido
 
Global Open Source Development 2011-2014 Review and 2015 Forecast
Global Open Source Development 2011-2014 Review and 2015 ForecastGlobal Open Source Development 2011-2014 Review and 2015 Forecast
Global Open Source Development 2011-2014 Review and 2015 ForecastSammy Fung
 
Liferay and Big Data
Liferay and Big DataLiferay and Big Data
Liferay and Big DataMiguel Pastor
 
Web APIs: The future of software
Web APIs: The future of softwareWeb APIs: The future of software
Web APIs: The future of softwareReuven Lerner
 
Citizen Developer Tools @ Valo Solutions / Blue Meteorite Monday session
Citizen Developer Tools @ Valo Solutions / Blue Meteorite Monday sessionCitizen Developer Tools @ Valo Solutions / Blue Meteorite Monday session
Citizen Developer Tools @ Valo Solutions / Blue Meteorite Monday sessionAntti Koskela
 
Atlanta hadoop users group july 2013
Atlanta hadoop users group july 2013Atlanta hadoop users group july 2013
Atlanta hadoop users group july 2013Christopher Curtin
 
Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018Antti Koskela
 
Serverless brewbox
Serverless brewboxServerless brewbox
Serverless brewboxLino Telera
 
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...Antti Koskela
 
Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...Antti Koskela
 
Big Graph Analytics on Neo4j with Apache Spark
Big Graph Analytics on Neo4j with Apache SparkBig Graph Analytics on Neo4j with Apache Spark
Big Graph Analytics on Neo4j with Apache SparkKenny Bastani
 

Similar to Kappa Architecture, IoT of the cars - LibreCon 2016 (20)

Meetup big data developers 2017 madrid - spark real use cases
Meetup big data developers 2017 madrid - spark real use casesMeetup big data developers 2017 madrid - spark real use cases
Meetup big data developers 2017 madrid - spark real use cases
 
Scalaua 2017 kyev kappa architecture 2.0
Scalaua 2017 kyev kappa architecture 2.0Scalaua 2017 kyev kappa architecture 2.0
Scalaua 2017 kyev kappa architecture 2.0
 
Gdg cloud london 2017 kappa architecture 2.0 copia
Gdg cloud london 2017 kappa architecture 2.0 copiaGdg cloud london 2017 kappa architecture 2.0 copia
Gdg cloud london 2017 kappa architecture 2.0 copia
 
JBCN barcelona 2017 kappa architecture 2.0
JBCN barcelona 2017 kappa architecture 2.0JBCN barcelona 2017 kappa architecture 2.0
JBCN barcelona 2017 kappa architecture 2.0
 
Codemotion madrid 2017 Arquitectura kappa 2.0
Codemotion madrid 2017 Arquitectura kappa 2.0Codemotion madrid 2017 Arquitectura kappa 2.0
Codemotion madrid 2017 Arquitectura kappa 2.0
 
Datascience lab 2017 odessa kappa architecture 2.0
Datascience lab 2017 odessa kappa architecture 2.0Datascience lab 2017 odessa kappa architecture 2.0
Datascience lab 2017 odessa kappa architecture 2.0
 
DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...
DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...
DataScience Lab 2017_Kappa Architecture: How to implement a real-time streami...
 
Lambda architecture for real time big data
Lambda architecture for real time big dataLambda architecture for real time big data
Lambda architecture for real time big data
 
Data Science at Scale: Using Apache Spark for Data Science at Bitly
Data Science at Scale: Using Apache Spark for Data Science at BitlyData Science at Scale: Using Apache Spark for Data Science at Bitly
Data Science at Scale: Using Apache Spark for Data Science at Bitly
 
Global Open Source Development 2011-2014 Review and 2015 Forecast
Global Open Source Development 2011-2014 Review and 2015 ForecastGlobal Open Source Development 2011-2014 Review and 2015 Forecast
Global Open Source Development 2011-2014 Review and 2015 Forecast
 
Liferay and Big Data
Liferay and Big DataLiferay and Big Data
Liferay and Big Data
 
Web APIs: The future of software
Web APIs: The future of softwareWeb APIs: The future of software
Web APIs: The future of software
 
Citizen Developer Tools @ Valo Solutions / Blue Meteorite Monday session
Citizen Developer Tools @ Valo Solutions / Blue Meteorite Monday sessionCitizen Developer Tools @ Valo Solutions / Blue Meteorite Monday session
Citizen Developer Tools @ Valo Solutions / Blue Meteorite Monday session
 
Atlanta hadoop users group july 2013
Atlanta hadoop users group july 2013Atlanta hadoop users group july 2013
Atlanta hadoop users group july 2013
 
Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018
 
Serverless brewbox
Serverless brewboxServerless brewbox
Serverless brewbox
 
Architecting Your First Big Data Implementation
Architecting Your First Big Data ImplementationArchitecting Your First Big Data Implementation
Architecting Your First Big Data Implementation
 
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
 
Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...
 
Big Graph Analytics on Neo4j with Apache Spark
Big Graph Analytics on Neo4j with Apache SparkBig Graph Analytics on Neo4j with Apache Spark
Big Graph Analytics on Neo4j with Apache Spark
 

More from LibreCon

Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...
Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...
Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...LibreCon
 
Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...
Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...
Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...LibreCon
 
La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016
La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016
La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016LibreCon
 
Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016
Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016
Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016LibreCon
 
Samsung y su firme apuesta por el Software Libre - LibreCon 2016
Samsung y su firme apuesta por el Software Libre - LibreCon 2016Samsung y su firme apuesta por el Software Libre - LibreCon 2016
Samsung y su firme apuesta por el Software Libre - LibreCon 2016LibreCon
 
Autonomous intelligence for the Industrial Internet - LibreCon 2016
Autonomous intelligence for the Industrial Internet - LibreCon 2016Autonomous intelligence for the Industrial Internet - LibreCon 2016
Autonomous intelligence for the Industrial Internet - LibreCon 2016LibreCon
 
Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...
Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...
Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...LibreCon
 
Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016
Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016
Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016LibreCon
 
Software libre como habilitador en la Industrua 4.0 - LibreCon 2016
Software libre como habilitador en la Industrua 4.0 - LibreCon 2016Software libre como habilitador en la Industrua 4.0 - LibreCon 2016
Software libre como habilitador en la Industrua 4.0 - LibreCon 2016LibreCon
 
BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016
BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016
BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016LibreCon
 
Open Source in robotics and its business - LibreCon 2016
Open Source in robotics and its business - LibreCon 2016Open Source in robotics and its business - LibreCon 2016
Open Source in robotics and its business - LibreCon 2016LibreCon
 
Fabricación aditiva de componentes industriales - LibreCon 2016
Fabricación aditiva de componentes industriales - LibreCon 2016Fabricación aditiva de componentes industriales - LibreCon 2016
Fabricación aditiva de componentes industriales - LibreCon 2016LibreCon
 
Software Libre por los aires, el caso de AENA
Software Libre por los aires, el caso de AENASoftware Libre por los aires, el caso de AENA
Software Libre por los aires, el caso de AENALibreCon
 
Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016
Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016
Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016LibreCon
 
A sustainable economic model through contributors to Libre/Free Software comm...
A sustainable economic model through contributors to Libre/Free Software comm...A sustainable economic model through contributors to Libre/Free Software comm...
A sustainable economic model through contributors to Libre/Free Software comm...LibreCon
 
Un modelo económicamente sostenible a través de la contribución a comunidades...
Un modelo económicamente sostenible a través de la contribución a comunidades...Un modelo económicamente sostenible a través de la contribución a comunidades...
Un modelo económicamente sostenible a través de la contribución a comunidades...LibreCon
 
¿En qué se parecen una piedra y un pollo? - LibreCon 2016
¿En qué se parecen una piedra y un pollo? - LibreCon 2016¿En qué se parecen una piedra y un pollo? - LibreCon 2016
¿En qué se parecen una piedra y un pollo? - LibreCon 2016LibreCon
 
Integración Bigdata: punto de entrada al IoT - LibreCon 2016
Integración Bigdata: punto de entrada al IoT - LibreCon 2016Integración Bigdata: punto de entrada al IoT - LibreCon 2016
Integración Bigdata: punto de entrada al IoT - LibreCon 2016LibreCon
 
Software libre en la industria de equipamiento médico y tecnología sanitaria ...
Software libre en la industria de equipamiento médico y tecnología sanitaria ...Software libre en la industria de equipamiento médico y tecnología sanitaria ...
Software libre en la industria de equipamiento médico y tecnología sanitaria ...LibreCon
 
Banking 4.0 - LibreCon 2016
Banking 4.0 - LibreCon 2016Banking 4.0 - LibreCon 2016
Banking 4.0 - LibreCon 2016LibreCon
 

More from LibreCon (20)

Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...
Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...
Inteligencia del dato aplicada al negocio de las telecomunicaciones - LibreCo...
 
Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...
Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...
Cómo aumentar los ratios de conversión de tu negocio en el canal telefónico -...
 
La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016
La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016
La plataforma cloud Opensource abierta a toda la industria - LibreCon 2016
 
Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016
Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016
Privacy shield, la nueva realidad en la industria 4.0 - LibreCon 2016
 
Samsung y su firme apuesta por el Software Libre - LibreCon 2016
Samsung y su firme apuesta por el Software Libre - LibreCon 2016Samsung y su firme apuesta por el Software Libre - LibreCon 2016
Samsung y su firme apuesta por el Software Libre - LibreCon 2016
 
Autonomous intelligence for the Industrial Internet - LibreCon 2016
Autonomous intelligence for the Industrial Internet - LibreCon 2016Autonomous intelligence for the Industrial Internet - LibreCon 2016
Autonomous intelligence for the Industrial Internet - LibreCon 2016
 
Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...
Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...
Cómo usar inteligencia competitiva para subirse a la ola de las tecnologías e...
 
Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016
Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016
Beneficios de aplicar la tecnología open a tu proyecto digital - LibreCon 2016
 
Software libre como habilitador en la Industrua 4.0 - LibreCon 2016
Software libre como habilitador en la Industrua 4.0 - LibreCon 2016Software libre como habilitador en la Industrua 4.0 - LibreCon 2016
Software libre como habilitador en la Industrua 4.0 - LibreCon 2016
 
BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016
BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016
BATEAN, caminando hacia la Industria 4.0 - LibreCon 2016
 
Open Source in robotics and its business - LibreCon 2016
Open Source in robotics and its business - LibreCon 2016Open Source in robotics and its business - LibreCon 2016
Open Source in robotics and its business - LibreCon 2016
 
Fabricación aditiva de componentes industriales - LibreCon 2016
Fabricación aditiva de componentes industriales - LibreCon 2016Fabricación aditiva de componentes industriales - LibreCon 2016
Fabricación aditiva de componentes industriales - LibreCon 2016
 
Software Libre por los aires, el caso de AENA
Software Libre por los aires, el caso de AENASoftware Libre por los aires, el caso de AENA
Software Libre por los aires, el caso de AENA
 
Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016
Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016
Industry 4.0 y Big Data. ¿Quién está accediendo a tus datos? - LibreCon 2016
 
A sustainable economic model through contributors to Libre/Free Software comm...
A sustainable economic model through contributors to Libre/Free Software comm...A sustainable economic model through contributors to Libre/Free Software comm...
A sustainable economic model through contributors to Libre/Free Software comm...
 
Un modelo económicamente sostenible a través de la contribución a comunidades...
Un modelo económicamente sostenible a través de la contribución a comunidades...Un modelo económicamente sostenible a través de la contribución a comunidades...
Un modelo económicamente sostenible a través de la contribución a comunidades...
 
¿En qué se parecen una piedra y un pollo? - LibreCon 2016
¿En qué se parecen una piedra y un pollo? - LibreCon 2016¿En qué se parecen una piedra y un pollo? - LibreCon 2016
¿En qué se parecen una piedra y un pollo? - LibreCon 2016
 
Integración Bigdata: punto de entrada al IoT - LibreCon 2016
Integración Bigdata: punto de entrada al IoT - LibreCon 2016Integración Bigdata: punto de entrada al IoT - LibreCon 2016
Integración Bigdata: punto de entrada al IoT - LibreCon 2016
 
Software libre en la industria de equipamiento médico y tecnología sanitaria ...
Software libre en la industria de equipamiento médico y tecnología sanitaria ...Software libre en la industria de equipamiento médico y tecnología sanitaria ...
Software libre en la industria de equipamiento médico y tecnología sanitaria ...
 
Banking 4.0 - LibreCon 2016
Banking 4.0 - LibreCon 2016Banking 4.0 - LibreCon 2016
Banking 4.0 - LibreCon 2016
 

Recently uploaded

Production 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptxProduction 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptxChloeMeadows1
 
Statistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfStatistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfOndejSur
 
Cyber Security Services Unveiled: Strategies to Secure Your Digital Presence
Cyber Security Services Unveiled: Strategies to Secure Your Digital PresenceCyber Security Services Unveiled: Strategies to Secure Your Digital Presence
Cyber Security Services Unveiled: Strategies to Secure Your Digital PresencePC Doctors NET
 
Premier Mobile App Development Agency in USA.pdf
Premier Mobile App Development Agency in USA.pdfPremier Mobile App Development Agency in USA.pdf
Premier Mobile App Development Agency in USA.pdfappinfoedgeca
 
Development Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appsDevelopment Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appscristianmanaila2
 
How Do I Begin the Linksys Velop Setup Process?
How Do I Begin the Linksys Velop Setup Process?How Do I Begin the Linksys Velop Setup Process?
How Do I Begin the Linksys Velop Setup Process?Linksys Velop Login
 
Pvtaan Social media marketing proposal.pdf
Pvtaan Social media marketing proposal.pdfPvtaan Social media marketing proposal.pdf
Pvtaan Social media marketing proposal.pdfPvtaan
 
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.Tortogel
 
The Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyThe Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyDamar Juniarto
 
Reggie miller choke t shirtsReggie miller choke t shirts
Reggie miller choke t shirtsReggie miller choke t shirtsReggie miller choke t shirtsReggie miller choke t shirts
Reggie miller choke t shirtsReggie miller choke t shirtsrahman018755
 
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkkaudience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkklolsDocherty
 
Thank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsThank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsrahman018755
 
Topology of the Network class 8 .ppt pdf
Topology of the Network class 8 .ppt pdfTopology of the Network class 8 .ppt pdf
Topology of the Network class 8 .ppt pdfAnushkaTripathi61
 
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebiThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebJie Liau
 
Bug Bounty Blueprint : A Beginner's Guide
Bug Bounty Blueprint : A Beginner's GuideBug Bounty Blueprint : A Beginner's Guide
Bug Bounty Blueprint : A Beginner's GuideVarun Mithran
 
I’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 ShirtI’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 Shirtrahman018755
 

Recently uploaded (16)

Production 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptxProduction 2024 sunderland culture final - Copy.pptx
Production 2024 sunderland culture final - Copy.pptx
 
Statistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdfStatistical Analysis of DNS Latencies.pdf
Statistical Analysis of DNS Latencies.pdf
 
Cyber Security Services Unveiled: Strategies to Secure Your Digital Presence
Cyber Security Services Unveiled: Strategies to Secure Your Digital PresenceCyber Security Services Unveiled: Strategies to Secure Your Digital Presence
Cyber Security Services Unveiled: Strategies to Secure Your Digital Presence
 
Premier Mobile App Development Agency in USA.pdf
Premier Mobile App Development Agency in USA.pdfPremier Mobile App Development Agency in USA.pdf
Premier Mobile App Development Agency in USA.pdf
 
Development Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of appsDevelopment Lifecycle.pptx for the secure development of apps
Development Lifecycle.pptx for the secure development of apps
 
How Do I Begin the Linksys Velop Setup Process?
How Do I Begin the Linksys Velop Setup Process?How Do I Begin the Linksys Velop Setup Process?
How Do I Begin the Linksys Velop Setup Process?
 
Pvtaan Social media marketing proposal.pdf
Pvtaan Social media marketing proposal.pdfPvtaan Social media marketing proposal.pdf
Pvtaan Social media marketing proposal.pdf
 
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
TORTOGEL TELAH MENJADI SALAH SATU PLATFORM PERMAINAN PALING FAVORIT.
 
The Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyThe Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case Study
 
Reggie miller choke t shirtsReggie miller choke t shirts
Reggie miller choke t shirtsReggie miller choke t shirtsReggie miller choke t shirtsReggie miller choke t shirts
Reggie miller choke t shirtsReggie miller choke t shirts
 
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkkaudience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
audience research (emma) 1.pptxkkkkkkkkkkkkkkkkk
 
Thank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirtsThank You Luv I’ll Never Walk Alone Again T shirts
Thank You Luv I’ll Never Walk Alone Again T shirts
 
Topology of the Network class 8 .ppt pdf
Topology of the Network class 8 .ppt pdfTopology of the Network class 8 .ppt pdf
Topology of the Network class 8 .ppt pdf
 
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWebiThome_CYBERSEC2024_Drive_Into_the_DarkWeb
iThome_CYBERSEC2024_Drive_Into_the_DarkWeb
 
Bug Bounty Blueprint : A Beginner's Guide
Bug Bounty Blueprint : A Beginner's GuideBug Bounty Blueprint : A Beginner's Guide
Bug Bounty Blueprint : A Beginner's Guide
 
I’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 ShirtI’ll See Y’All Motherfuckers In Game 7 Shirt
I’ll See Y’All Motherfuckers In Game 7 Shirt
 

Kappa Architecture, IoT of the cars - LibreCon 2016

  • 1. Kappa Architecture: IoT of the cars Juantomás García
  • 2. Who am I • Data Solutions Manager at Open Sistemas • Co-author of the first spanish book about free software “La pastilla roja” • Former president of Hispalinux (Spanish Linux User Group). • Co-organizer of Machine Learning Spain Meetup
  • 3. What’s Kappa Architecture July 2, 2014 Jay Kreps coined the term Kappa Architecture in an article for O’reilly Radar
  • 4. Who is Jay Kreps Jay has been involved in lots of projects: Author of the essay: The Log: What every software engineer should know about real-time data's unifying abstraction (12/16/2013) https://engineering.linkedin.com/distributed-systems/log-what-every-software- engineer-should-know-about-real-time-datas-unifying
  • 5. Who is Jay Kreps Author of the book: I ♥ Logs
  • 6. Who is Jay Kreps Involved with projects as: Apache Kafka Apache Samza Voldemort Azkaban Ex-Linkedin Now co-founder and CEO of Confluent
  • 9. Tools we use to implement Kappa Architecture
  • 10. Tools we use to implement Kappa Architecture
  • 11. Tools we use to implement Kappa Architecture • Our favorite kafka features. • Easy to use • Robust & Resilient • Kafka connectors • Kafka streams • Compact logs • Very fast writes and reads
  • 12. Tools we use to implement Kappa Architecture
  • 13. Some favorite Spark features • If you have an schema spark SQL is perfect. • Spark streaming work very fine with spark and almost all streaming sources. • Structured queries will be a huge advance. • We love scala and scala is the spirit of spark.
  • 14. Some favorite Spark features • We love code like this:
  • 15. A real case • One of our clients want to monitor all the cars informations via OBD II • OBD II is a car interface with the car electronics. • Client develop an app to read all the car information throw a ODB II with bluetooth
  • 17. First Problems • We need to scale the rest interfaces too many requests. • MySQL don’t scale • Client wants to make realtime expensive queries.
  • 21. Now we’re more flexible • We can have queries like: • What are the drivers that are not client of the X gas brand, has a few gas and are near of gas station of the brand X and if true, send a notification with a discount coupon and a link with the route.
  • 22. Takeaways • Kappa architecture is not a silver bullet but helps in a lot of solutions. • Kafka + spark streaming is our favorite tools • There a lots of improvements: • OLAP like Apache Druid • Graph databases like neo4j • Kafka streams and compacts logs
  • 23. ¿Questions? • My contact info: • Juantomás García • juantomas@opensistemas.com • @juantomas