Apache Pulsar is a cloud-native messaging and event-streaming platform originally developed at Yahoo, now an open-source project with a rich ecosystem and extensive features such as durability, scalability, and multi-tenancy. It aims to reduce architectural complexity and improve efficiency for high-scale microservices, demonstrated by its application in a major Chinese payment provider where it streamlined operations and enhanced fraud detection capabilities. With over 10 million downloads and a growing community, Pulsar supports both streaming and batch processing, making it a versatile choice for modern data architectures.

1. Apache Pulsar
Unifies
Streaming and
Messaging for
Real-Time Data

2. ● Apache Pulsar Committer | Author of Pulsar In Action
● Former Principal Software Engineer on Splunk’s messaging
team that is responsible for Splunk’s internal
Pulsar-as-a-Service platform.
● Former Director of Solution Architecture at Streamlio.
David
Kjerrumgaard
Developer Advocate

3. Tim Spann
Developer Advocate
Tim Spann, Developer Advocate at StreamNative
● FLiP(N) Stack = Flink, Pulsar and NiFI Stack
● Streaming Systems & Data Architecture Expert
● Experience:
○ 15+ years of experience with streaming technologies including Pulsar,
Flink, Spark, NiFi, Big Data, Cloud, MXNet, IoT, Python and more.
○ Today, he helps to grow the Pulsar community sharing rich technical
knowledge and experience at both global conferences and through
individual conversations.

4. Sijie Guo
ASF Member
Pulsar/BookKeeper PMC
Founder and CEO
Jia Zhai
Pulsar/BookKeeper PMC
Co-Founder
✓ Data veterans with
extensive industry
experience
✓ Original creators of Apache
Pulsar & BookKeeper
✓ Operated the largest
Pulsar/BookKeeper cluster
Matteo Merli
Pulsar PMC Chair,
BookKeeper PMC
CTO
StreamNative Executive Team

5. Apache Pulsar is a Cloud-Native
Messaging and Event-Streaming Platform.

6. CREATED
Originally
developed inside
Yahoo! as Cloud
Messaging
Service
GROWTH
10x Contributors
10MM+ Downloads
Ecosystem Expands
Kafka on Pulsar
AMQ on Pulsar
Functions
. . .
2012 2016 2018 TODAY
APACHE TLP
Pulsar
becomes
Apache top
level project.
OPEN SOURCE
Pulsar
committed
to open source.
Apache Pulsar Timeline

7. Evolution of Pulsar Growth

8. Pulsar Has a Built-in Super Set of OSS
Features
Durability
Scalability Geo-Replication
Multi-Tenancy
Unified Messaging
Model
Reduced Vendor Dependency
Functions
Open-Source Features

9. Multi-Tenancy Model
Tenants
(Data Services)
Namespace
(Microservices)
Pulsar Cluster
Tenants
(Marketing)
Tenants
(Compliance)
Namespace
(ETL)
Namespace
(Campaigns)
Namespace
(ETL)
Namespace
(Risk Assessment)
Topic-1
(Cust Auth)
Topic-1
(Location Resolution)
Topic-2
(Demographics)
Topic-1
(Budgeted Spend)
Topic-1
(Acct History)
Topic-2
(Risk Detection)

10. Fintech Powered by Pulsar
10
Low latency
Geo-replication
Data integrity
High availability
Durability
Multi-tenancy
Multiple data
consumers:
Transactions,
payment
processing, alerts,
analytics, fraud
detection with ML
Large data
volumes, high
scalability
Financial
event
messaging
Many topics,
producers,
consumers

11. 11
Designed for
teams, with
built in
multi-tenancy
Power and
flexibility,
w/ support for
simultaneous
streaming and
messaging use
cases
Ideal for
high-scale,
mission
critical
microservices
Easy to use,
with a simple
pub/sub API
Asynchronous APIs Empower All

12. Ideal for app and data tiers
Less sprawl and better
utilization
Cloud-native scalability
Build globally without
the complexity
Cost effective long-term
storage
Pulsar across the
organization

13. Joining Streams in SQL
Perform in Real-Time
Ordering and Arrival
Concurrent Consumers
Change Data Capture
Data Streaming

14. Streaming
Consumer
Consumer
Consumer
Subscription
Shared
Failover
Consumer
Consumer
Subscription
In case of failure in
Consumer B-0
Consumer
Consumer
Subscription
Exclusive
X
Consumer
Consumer
Key-Shared
Subscription
Pulsar
Topic/Partition
Messaging

17. Background
● The third-largest payment
provider in China behind
Alipay and WeChat
Payment
● 500 million registered users
and 41.9 million active users
● Need to improve the
efficiency of fraud detection
for mobile payments
● Current lambda architecture
of Kafka + Hive is complex
and difficult to maintain
Benefits
● Reduce complexity by 33%
(clusters reduced from six to
four)
● Improve production
efficiency by 11 times
● Higher stability due to the
unified architecture
Why Pulsar
● Cloud-native architecture
and segment-centric
storage
● Pulsar is able to do both
streaming and batch
processing
● Able to build a unified
data processing stack
with Pulsar and Spark,
streamlining messy
operations problems

18. Apps
Building Real-Time Requires a Team

19. DEMO

20. Scan the QR code
to learn more about
Apache Pulsar and
StreamNative.

21. Scan the QR code
to build your own
apps today.

22. Apache Pulsar
Apache BookKeeper
Broker 0
Producer
Consumer - Kafka
Broker 1 Broker 2
Bookie 0 Bookie 1 Bookie 2 Bookie 3 Bookie 4
T
1
T
2
T
3
T
4
T
0
Consumer - Pulsar