My report on Pulsar Summit SF 2022
https://github.com/tspannhw/FLipStackWeekly
The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at all. The atmosphere was open source and community. The hallway talks and discussions between different open source projects and communities were awesome. It was amazing to see everyone come together for one event. From now on this is what I strive for in every event always.
I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved and three things that weren’t perfect.
Three Things I Loved
The event felt and was driven by, for, and of the community. It had all the right content, character, and feel that you want in a conference. It was fun, full of great content, and a great experience. I want events to be like this every time.
It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving awesome talks it would not have been great. The talks just got better and better. We saw amazing new features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short, but it worked.
The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I work within the community, new people, and friends from other projects and platforms.
Three Things That Were Not The Best
Flying to the event. Travelling has become much more difficult with so many cancellations, covid and delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more.
The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring, Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and ApacheCon.
The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker! Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new M1 with 32GB RAM.
Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight, since this report is already more than 20 pages.
2. I am publishing this as a PDF since the first version of this was written natively in LinkedIn and was lost in a weird
loading glitch.
The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run
meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at
all. The atmosphere was open source and community. The hallway talks and discussions between different open
source projects and communities were awesome. It was amazing to see everyone come together for one event.
From now on this is what I strive for in every event always.
I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s
Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved
and three things that weren’t perfect.
3. Three Things I Loved
● The event felt and was driven by, for, and of the community. It had all the right content, character, and feel
that you want in a conference. It was fun, full of great content, and a great experience. I want events to be
like this every time.
● It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving
awesome talks it would not have been great. The talks just got better and better. We saw amazing new
features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short,
but it worked.
● The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees
that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I
work within the community, new people, and friends from other projects and platforms.
Three Things That Were Not The Best
● Flying to the event. Travelling has become much more difficult with so many cancellations, covid and
delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to
Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more.
● The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring,
Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and
ApacheCon.
● The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for
getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker!
Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new
M1 with 32GB RAM.
Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight,
since this report is already more than 20 pages. Everyone who came was awesome especially all the people
working to mange, build, promote, run and make this event happen. The entire StreamNative team put in so many
hours to make this run smooth. Thank you Karin, Carolyn, Sally, Addison, James, Sijie, Matteo, Alice, Sara, Doug
4. and so many more. I left out so many, add yourself in the comments. It was great talking to James and Vivek from
Optum, I so want to do a meetup in the Boston area.
Lari (DataStax) did a great talk and has the Reactive framework to add to the Spring - Pulsar module.
https://github.com/lhotari/reactive-pulsar-in-5-minutes
Michael (Cloudera) came and we got to discuss upcoming Cloudera DataFlow (Apache NiFi) work with Apache
Pulsar as well as SQL Stream Builder (Apache Flink) with Pulsar. We should have some cool events, articles,
demos, and content soon.
https://community.cloudera.com/t5/Community-Articles/Using-Apache-NiFi-with-Apache-Pulsar-for-Streaming/ta-p/33
7891
https://community.cloudera.com/t5/Cloudera-Stream-Processing/Using-Apache-Pulsar-with-SQL-Stream-Builder/m-p/
349917
Marko and Dominiki (Memgraph) had a great discussion with me about some things that Memgraph can do with
Apache Pulsar. I see some interesting use cases coming out of this.
https://memgraph.com/docs/gqlalchemy/how-to-guides/streams/manage-pulsar-streams
Robert (Altinity/Clickhouse) had some great discussions on fast data and is a great source for data knowledge. I
can’t wait for the Altinity conference on open source data.
https://altinity.com/blog/the-open-source-analytics-conference-2022-is-on-15-november
Soby Chacko (VMWare / Pivotal) came for discussions on his new experimental Pulsar module for Spring and great
discussions were all around. I am located near him so we will work on bringing a Spring-Pulsar event to Philly, New
Jersey, and/or NYC. Stay tuned!
● https://github.com/spring-projects-experimental/spring-pulsar
● https://spring.io/blog/2022/08/16/introducing-experimental-spring-support-for-apache-pulsar
● https://github.com/spring-projects/spring-integration/issues/3771
7. Ecosystem Track (I hosted this one)
Nick (Databricks) did a cool talk on DeltaLake plus Apache Pulsar.
● https://pulsar-summit.org/event/san-francisco-2022/sessions/building-reliable-lakehouses-with-apache-pulsa
r-and-delta-lake
● https://github.com/tspannhw/FLiP-Pi-DeltaLake-Thermal
8. Addison (StreamNative) and Alexey (OneHouse) did a cool talk on Apache Hudi and Apache Pulsar.
● https://pulsar-summit.org/event/san-francisco-2022/sessions/unlocking-the-power-of-lakehouse-architecture
s-with-apache-pulsar-and-apache-hudi
● https://github.com/tspannhw/pulsar-io-lakehouse
9. Caito (Ververica) did an awesome talk on Apache Flink, Apache Flink SQL, and using Flink with Apache Pulsar. As
this is the majority of the FLiP Stack this is my favorite stuff. There was a debate on when to use Flink vs Pulsar
SQL (Presto/Trino) vs Pulsar PFSQL vs Spark SQL. For massively scalable continuous SQL with joins,
aggregations, and stateful streaming you should use Flink. Caito also liked my FLiP Stack Cat sticker.!
10. Ricardo (AWS) did a jaw-dropping talk. I did not believe someone would do a talk that was as good as Caito.
Ricardo’s was an amazing all-demo talk fully documented on Github. This was amazing to see him walk through
such a complex usage of Pulsar’s KoP to run as a full-fledged Kafka cluster allowing for KSQLDB apps, CDC, and
Kafka microservices. We are planning to do an open source streaming meetup/round table at a Brazilian
Churrascaria.
● https://github.com/riferrei/is-using-kop-a-good-idea
● https://github.com/streamnative/kop
● https://talks.riferrei.com/LgUiHy/is-using-kop-kafka-on-pulsar-a-good-idea
11. Neng (StreamNative) had the newest and shiniest tech to offer and demo. Pulsar Functions utilizing SQL instead of
Java, Python, or Golang.
14. For continuous updates on slides, articles, photos, code, and more:
https://github.com/tspannhw/PulsarSummitSF2022
Check out the Twitter stream for #PulsarSummmitSF
https://twitter.com/hashtag/PulsarSummitSF?src=hashtag_click&f=live
Check out Scylla’s awesome Pulsar Summit Roundup
https://twitter.com/i/spaces/1dRKZlaBvwbJB?s=20
Twitter
https://twitter.com/PulsarSummit
Website
https://pulsar-summit.org/
The Pre-Report
https://www.linkedin.com/pulse/pulsar-summit-pre-report-2022-tim-spann-/
Unboxing
https://twitter.com/PaaSDev/status/1561438246403440640?s=20&t=2xXb0gOYPQaNbsx-YZTdnA
16. Come see me speak at JCONF.DEV in Chicago, Sept 25-28. This is a community-run #Java #JVM #Cloud #BigData
conference done right. No sales/product pitches here. Just amazing content. Use this link for a $100 discount on any 2 or
3-day pass cevents.io/JC22100
● September 8, 2022: Comcast Labs Connect PHLAI in Philadelphia
● Sept 13-16, 2022: OSS Summit (Virtual)
● Oct 3-6, 2022: Apache Con, New Orleans, LA.
● Oct 4-5, 2022: Current: Apache Kafka Summit. Austin, TX.
● Oct 25, 2022: KubeCon: Detroit
● Oct 25 - Nov 3: AI DevWorld (Virtual)
● Nov 23, 2022: Big Data EU (Virtual)
I hope to see my streaming friends at Current in Austin.
https://2022.currentevent.io/website/39543/welcome/
As well as all my streaming and data friends at ApacheCon in New Orleans.
https://apachecon.com/acna2022/schedule.html
OTHER ITEMS THIS WEEK
● https://www.comcastlabsconnect.com/2022-phlai
● https://www.youtube.com/watch?v=hPZS1ocmWhM