[DoKDayNA2022] - Architecting Your First Event Driven Serverless Streaming Applications on K8 https://github.com/tspannhw/FLiPN-DoKDayNA2022 Once you have built a topic in Apache Pulsar, you will quickly see the need to build event-driven applications. This can require a lot of decisions on what framework to use, where to run it, how to deploy it, and how to manage these applications on Kubernetes cloud natively. I will walk you through step-by-step in building Pulsar Functions which is the easy way to design, test, develop, integrate, deploy, monitor, and manage serverless streaming applications in Java and Python. Together we will build a full application as an Apache Pulsar function and enjoy the power of running it in the cloud for IoT events and add any routing, transformation, or machine learning that we need to accomplish our business requirements. Through FunctionMesh we run on Kubernetes natively. In this talk, you will deploy ML functions to transform real-time data on Kubernetes.

1. Tim Spann
“Serverless
Streaming”
“Architecting Your First Event Driven Serverless
Streaming Application on K8”

2. Tim Spann
“Serverless
Streaming” Tim Spann DoK Day North America 2022 @ KubeCon
“Serverless
Streaming”
Timothy Spann
Developer Advocate
FLiP(N) Stack = Flink, Pulsar and NiFi Stack
Streaming Systems & Data Architecture Expert
Experience:
15+ years of experience with streaming technologies including Pulsar, Flink, Spark,
NiFi, Kafka, Big Data, Cloud, MXNet, IoT and more.
Today, he helps to grow the Pulsar community sharing rich technical knowledge and
experience at both global conferences and through individual conversations.

3. Tim Spann
“Serverless
Streaming” Tim Spann DoK Day North America 2022 @ KubeCon
“Serverless
Streaming”
FLiP Stack Weekly
This week in Apache Flink, Apache Pulsar, Apache
NiFi, Apache Spark, Elasticsearch and open source
friends.
https://bit.ly/32dAJft

4. Tim Spann
“Serverless
Streaming” Tim Spann
“Serverless
Streaming”
Apache Pulsar is a Cloud-Native Messaging
and Event-Streaming Platform.
DoK Day North America 2022 @ KubeCon

5. Tim Spann
“Serverless
Streaming”
Apache Pulsar adoption is
being driven by
organizations seeking
cloud-native architectures
and new uses cases.
Microservices
Apache Pulsar - Built for Containers / Modern Cloud
Cloud Native
Hybrid & Multi-Cloud Containers
DoK Day North America 2022 @ KubeCon

6. Tim Spann
“Serverless
Streaming”
Key Pulsar Concepts: Messaging vs
Streaming
Message Queueing - Queueing
systems are ideal for work queues
that do not require tasks to be
performed in a particular order.
Streaming - Streaming works
best in situations where the
order of messages is important.
DoK Day North America 2022 @ KubeCon

7. Tim Spann
“Serverless
Streaming”
Pulsar Cluster
CLUSTER
Global ZooKeeper
Configuration Store
ZK3
ZK2
ZK1
Local
ZooKeeper
Quorum
Bookie 0 Bookie 1 Bookie 2
BookKeeper
Bookie
Ensemble
Pulsar Broker 0 Pulsar Broker 1 Pulsar Broker 2
DoK Day North America 2022 @ KubeCon

8. Tim Spann
“Serverless
Streaming”
● Consume messages from one
or more Pulsar topics.
● Apply user-supplied
processing logic to each
message.
● Publish the results of the
computation to another topic.
● Support multiple
programming languages
(Java, Python, Go)
● Can leverage 3rd-party
libraries
Pulsar Functions
DoK Day North America 2022 @ KubeCon

9. Tim Spann
“Serverless
Streaming”
import java.util.function.Function;
public class MyFunction implements Function<String, String> {
public String apply(String input) {
return doBusinessLogic(input);
}
}
Entire Function
Why Pulsar
Functions?

10. Tim Spann
“Serverless
Streaming”
from pulsar import Function
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
import json
class Chat(Function):
def __init__(self):
pass
def process(self, input, context):
fields = json.loads(input)
sid = SentimentIntensityAnalyzer()
ss = sid.polarity_scores(fields["comment"])
row = { }
row['id'] = str(msg_id)
if ss['compound'] < 0.00:
row['sentiment'] = 'Negative'
else:
row['sentiment'] = 'Positive'
row['comment'] = str(fields["comment"])
json_string = json.dumps(row)
return json_string
Entire Function
Pulsar Python
NLP Function
https://github.com/tspannhw/pulsar-pychat-function

11. Tim Spann
“Serverless
Streaming”
Function Mesh
Pulsar Functions, along with Pulsar IO/Connectors, provide a powerful API for ingesting,
transforming, and outputting data.
Function Mesh, another StreamNative project, makes it easier for developers to create entire
applications built from sources, functions, and sinks all through a declarative API.

12. Tim Spann
“Serverless
Streaming”
K8 Deploy

13. Tim Spann
“Serverless
Streaming”
Function Execution
DoK Day North America 2022 @ KubeCon

14. Tim Spann
“Serverless
Streaming”
Apache Pulsar Kafka K8
https://docs.streamnative.io/platform/v1.3.0/quickstart
DoK Day North America 2022 @ KubeCon

15. Tim Spann
“Serverless
Streaming”
Webinar Series:
Building Microservices
with Pulsar
Watch now Read more
Learn how Pulsar Functions, can be
leveraged to build a message bus for
event-driven microservices
DoK Day North America 2022 @ KubeCon

16. Tim Spann
“Serverless
Streaming”
Apache Pulsar Resources
● https://pulsar.apache.org/docs/en/kubernetes-helm/
● https://pulsar.apache.org/charts
● https://streamnative.io/streamnativecloud/
● https://medium.com/@_oleksii_/how-to-deploy-apache-puls
ar-cluster-in-kubernetes-808ecdfd87
Academy.StreamNative.io

17. Tim Spann
“Serverless
Streaming”
Thank you!
DoK Day North America 2022 @ KubeCon
Tim Spann
“Serverless
Streaming”