What's New in Apache Pulsar 2.9- Pulsar Summit Asia 2021

1. New features in Apache Pulsar 2.9
Take a deep look at the new features in Apache Pulsar 2.9

2. About Me
● Apache Pulsar PMC Member
● StreamNative Staff Engineer

3. Pluggable metadata interface
● Unified metadata interface
● Support for diverse implementations of metadata backend
● Local KV for Standalone

4. Transactions enhancements
● Transaction logs
● Transaction buffer persistence
● Transaction buffer max read position and snapshot
● Pending ack state persistence
● Pending ack snapshot

5. Message payload processor for Pulsar client
● Originally introduced by KoP
● Support consume Kafka format data by the Pulsar client with custom processor
● Without broker-side format conversion, reduce the burden on brokers

6. Replicate schemas across clusters
● Avoid centralized schema registry
● Reduce the complexity of schema registry operation and maintenance
● Rewrite the schema version during the message replication
● To support global topic policy

7. Produce messages with REST API
Proposal: https://github.com/apache/pulsar/wiki/PIP-64%3A-Introduce-REST-endpoints-for-producing%2C-consuming-and-reading-messages

8. Smart Listener Selection with Multiple
Bind Addresses

9. Message consumption with pooled buffer
● Reduce the client heap memory overhead
● Release the buffer manually

10. Next: 2.10
● PIP 84: Message redelivery epoch
● PIP 104: Add new consumer type: TableView
● PIP 106: Negative acknowledgment backoff
● PIP 110: Topic customized metadata support
● PIP 118: Do not restart brokers when ZooKeeper session expires
● PIP 121: Pulsar cluster level auto failover
● PIP 123: Pulsar metadata CLI tool
● Metadata service batch operations
● RocksDB metadata service backend
● Etcd metadata service backend
● Ack timeout redelivery backoff policy
● Global topic policies

11. THANK YOU FOR WATCHING