Download as PDF, PPTX
Uploaded byImply
Apache Druid®: A Dance of Distributed Processes
This document summarizes the key components and collaborations in Apache Druid. It describes Zookeeper's role in coordination, the Overlord's role in task management, the Broker's role in query routing, and the Middle Manager's role in ingestion and indexing. It provides diagrams illustrating how these components work together to ingest and store distributed data, and answer queries in a scalable way.
In this document
Powered by AI
Presentation ownership by Apache Software Foundation and speaker's rich experience in software architecture and IT.
Focus on tasks and principles of collaboration within the system including wrap-ups and question prompts.
Details on Druid's SQL execution, data ingestion, replication, and distribution processes.
Details on Druid's SQL execution, data ingestion, replication, and distribution processes.
Links to ArchiMate tool and resources for Apache Druid community engagement and support.
Focus on tasks and principles of collaboration within the system including wrap-ups and question prompts.
Detailed breakdown of Druid components like Zookeeper, Overlord, Middle Manager, and Deep Store.
Focus on tasks and principles of collaboration within the system including wrap-ups and question prompts.
Links to ArchiMate tool and resources for Apache Druid community engagement and support.
Details on Druid's SQL execution, data ingestion, replication, and distribution processes.
Links to ArchiMate tool and resources for Apache Druid community engagement and support.
Focus on tasks and principles of collaboration within the system including wrap-ups and question prompts.
More Related Content
Similar to Apache Druid®: A Dance of Distributed Processes
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Apache Druid®: A Dance of Distributed Processes
- 1. Apache®, Apache Druid®,Druid®, and the Druid logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.
- 2. peter.marshall@imply.io 20 years inEnterprise Architecture CRM, EDRM, ERP, EIP, Digital Services, Security, BI, RI, and MDM BA Theology (!) and Computer Studies TOGAF certified Book collector & A/V buyer Prime Timeline = proper timeline #werk petermarshall.io
- 3. What the collaborationsdo Some principles Each collaboration Wrap-up Questions!
- 4. Query Distributed execution ofSQL / Druid Native queries on the cluster Ingestion Ingestion tasks that bring data into Druid from storage and delivery services Distribution Replication and distribution of the ingested data according to rules
- 6. ● A jobto do ● Compute to do the ingestion ● A place to store optimised data ● A question to answer ● Data to process! ● Compute to answer queries ● Somewhere to put the data that’s near to the query process ● Some rules to follow Query Distributed execution of SQL / Druid Native queries on the cluster Ingestion Ingestion tasks that bring data into Druid from storage and delivery services Distribution Replication and distribution of the ingested data according to rules
- 7.
- 8. What the collaborationsdo Some principles Each collaboration Wrap-up Questions!
- 9. Zookeeper Overlord Ingest Spec Task Log S3, GCS, ABS,HFDS, HTTP, Local, MySql, Postgres, Druid Kafka Kinesis Middle Manager (Indexer)
- 10.
- 11. Deep Store Zookeeper Overlord Middle Manager (Indexer)columnarise index & encode time-shardDeep Store
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 23. What the collaborationsdo Some principles Each collaboration Wrap-up Questions!
- 28.
- 29. Zookeeper Coordinator Overlord Broker Query Distributed execution ofSQL / Druid Native queries on the cluster Ingestion Ingestion tasks that bring data into Druid from storage and delivery services Distribution Replication and distribution of the ingested data according to rules
- 30. ★ A jobto do ★ Compute to do the ingestion ★ A place to store optimised data ★ A question to answer ★ Data to process! ★ Compute to answer queries ★ Somewhere to put the data that’s near to the query process ★ Some rules to follow Query Distributed execution of SQL / Druid Native queries on the cluster Ingestion Ingestion tasks that bring data into Druid from storage and delivery services Distribution Replication and distribution of the ingested data according to rules
- 31. http://druid.apache.org Imply Distribution https://imply.io/get-started @druidio Add ApacheDruid as a skill Apache Distribution https://github.com/apache/druid ASF Slack #druid Druid Community https://druid.apache.org/community/ Meetup Groups https://www.meetup.com/pro/apache-druid/ Google Groups Druid User Forum https://groups.google.com/
- 32. What the collaborationsdo Some principles Each collaboration Wrap-up Questions!