Be the first to like this
Apache Pulsar is a distributed and open-source pub-sub messaging system. It offers many advantages over Kafka, such as multi-tenant, geo-replication, decoupled storage or even SQL and FaaS directly integrated. The only thing missing for wide adoption is support for the de-facto standard for streaming: Kafka. And this is how our story begins.
In this talk, Sijie Guo from StreamNative and Pierre Zemb from OVHcloud will share the journey on building Kafka-on-Pulsar (KoP) to bring native Kafka protocol support to Pulsar. Before joining the force on building KoP, OVHcloud implemented a Kafka proxy in Rust capable of transforming the Kafka protocol to that Pulsar on the fly and encountered some challenges. After realizing that StreamNative was working on bringing the Kafka protocol natively to Pulsar broker via a pluggable protocol handler mechanism. OVHCloud joined forces with StreamNative to work on brining Kafka protocol support to Pulsar brokers.
At the end of this talk, you will know more about the inner workings of Kafka and Pulsar. You'll also get feedback from both companies from their initial proofs of concepts and the current implementation.