SlideShare a Scribd company logo
1 of 17
Download to read offline
Pulsar Summit SF 2022 Report
I am publishing this as a PDF since the first version of this was written natively in LinkedIn and was lost in a weird
loading glitch.
The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run
meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at
all. The atmosphere was open source and community. The hallway talks and discussions between different open
source projects and communities were awesome. It was amazing to see everyone come together for one event.
From now on this is what I strive for in every event always.
I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s
Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved
and three things that weren’t perfect.
Three Things I Loved
● The event felt and was driven by, for, and of the community. It had all the right content, character, and feel
that you want in a conference. It was fun, full of great content, and a great experience. I want events to be
like this every time.
● It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving
awesome talks it would not have been great. The talks just got better and better. We saw amazing new
features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short,
but it worked.
● The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees
that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I
work within the community, new people, and friends from other projects and platforms.
Three Things That Were Not The Best
● Flying to the event. Travelling has become much more difficult with so many cancellations, covid and
delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to
Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more.
● The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring,
Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and
ApacheCon.
● The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for
getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker!
Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new
M1 with 32GB RAM.
Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight,
since this report is already more than 20 pages. Everyone who came was awesome especially all the people
working to mange, build, promote, run and make this event happen. The entire StreamNative team put in so many
hours to make this run smooth. Thank you Karin, Carolyn, Sally, Addison, James, Sijie, Matteo, Alice, Sara, Doug
and so many more. I left out so many, add yourself in the comments. It was great talking to James and Vivek from
Optum, I so want to do a meetup in the Boston area.
Lari (DataStax) did a great talk and has the Reactive framework to add to the Spring - Pulsar module.
https://github.com/lhotari/reactive-pulsar-in-5-minutes
Michael (Cloudera) came and we got to discuss upcoming Cloudera DataFlow (Apache NiFi) work with Apache
Pulsar as well as SQL Stream Builder (Apache Flink) with Pulsar. We should have some cool events, articles,
demos, and content soon.
https://community.cloudera.com/t5/Community-Articles/Using-Apache-NiFi-with-Apache-Pulsar-for-Streaming/ta-p/33
7891
https://community.cloudera.com/t5/Cloudera-Stream-Processing/Using-Apache-Pulsar-with-SQL-Stream-Builder/m-p/
349917
Marko and Dominiki (Memgraph) had a great discussion with me about some things that Memgraph can do with
Apache Pulsar. I see some interesting use cases coming out of this.
https://memgraph.com/docs/gqlalchemy/how-to-guides/streams/manage-pulsar-streams
Robert (Altinity/Clickhouse) had some great discussions on fast data and is a great source for data knowledge. I
can’t wait for the Altinity conference on open source data.
https://altinity.com/blog/the-open-source-analytics-conference-2022-is-on-15-november
Soby Chacko (VMWare / Pivotal) came for discussions on his new experimental Pulsar module for Spring and great
discussions were all around. I am located near him so we will work on bringing a Spring-Pulsar event to Philly, New
Jersey, and/or NYC. Stay tuned!
● https://github.com/spring-projects-experimental/spring-pulsar
● https://spring.io/blog/2022/08/16/introducing-experimental-spring-support-for-apache-pulsar
● https://github.com/spring-projects/spring-integration/issues/3771
● https://github.com/spring-projects-experimental/spring-pulsar/tree/main/spring-pulsar-sample-apps
● https://docs.spring.io/spring-pulsar/docs/current-SNAPSHOT/reference/html/
David (StreamNative) was there and he signed his awesome Pulsar book. It’s amazing to have a co-worker like
David answer the most obscure questions ever.
https://github.com/david-streamlio/table-view-demo
https://streamnative.io/download/manning-ebook-apache-pulsar-in-action
Decodable sponsored food and were around to chat which is awesome. They have great articles.
https://www.decodable.co/blog/cleansing-osquery-logs-for-apache-pinot
Schedule of Talks
https://pulsar-summit.org/event/san-francisco-2022/schedule
I missed some great talks by Matteo, David, Lari, Heesung, Kai, and Zach. I can’t wait for the videos and slides to
come out.
Ecosystem Track (I hosted this one)
Nick (Databricks) did a cool talk on DeltaLake plus Apache Pulsar.
● https://pulsar-summit.org/event/san-francisco-2022/sessions/building-reliable-lakehouses-with-apache-pulsa
r-and-delta-lake
● https://github.com/tspannhw/FLiP-Pi-DeltaLake-Thermal
Addison (StreamNative) and Alexey (OneHouse) did a cool talk on Apache Hudi and Apache Pulsar.
● https://pulsar-summit.org/event/san-francisco-2022/sessions/unlocking-the-power-of-lakehouse-architecture
s-with-apache-pulsar-and-apache-hudi
● https://github.com/tspannhw/pulsar-io-lakehouse
Caito (Ververica) did an awesome talk on Apache Flink, Apache Flink SQL, and using Flink with Apache Pulsar. As
this is the majority of the FLiP Stack this is my favorite stuff. There was a debate on when to use Flink vs Pulsar
SQL (Presto/Trino) vs Pulsar PFSQL vs Spark SQL. For massively scalable continuous SQL with joins,
aggregations, and stateful streaming you should use Flink. Caito also liked my FLiP Stack Cat sticker.!
Ricardo (AWS) did a jaw-dropping talk. I did not believe someone would do a talk that was as good as Caito.
Ricardo’s was an amazing all-demo talk fully documented on Github. This was amazing to see him walk through
such a complex usage of Pulsar’s KoP to run as a full-fledged Kafka cluster allowing for KSQLDB apps, CDC, and
Kafka microservices. We are planning to do an open source streaming meetup/round table at a Brazilian
Churrascaria.
● https://github.com/riferrei/is-using-kop-a-good-idea
● https://github.com/streamnative/kop
● https://talks.riferrei.com/LgUiHy/is-using-kop-kafka-on-pulsar-a-good-idea
Neng (StreamNative) had the newest and shiniest tech to offer and demo. Pulsar Functions utilizing SQL instead of
Java, Python, or Golang.
https://pulsar-summit.org/event/san-francisco-2022/sessions/simplify-pulsar-functions-development-with-sql
Peter (Scylla) is the consummate professional and every talk he does is a masterclass in data. Every time he
speaks it’s a great learning experience. I can’t wait until we do another masterclass or a round table. He wrapped
up the ecosystem track with an awesome talk on the future of all things data. Where do you stand on the Data
Engineer vs Data Scientist architectures?
https://pulsar-summit.org/event/san-francisco-2022/sessions/distributed-database-design-decisions-to-support-high-p
erformance-event-streaming
For continuous updates on slides, articles, photos, code, and more:
https://github.com/tspannhw/PulsarSummitSF2022
Check out the Twitter stream for #PulsarSummmitSF
https://twitter.com/hashtag/PulsarSummitSF?src=hashtag_click&f=live
Check out Scylla’s awesome Pulsar Summit Roundup
https://twitter.com/i/spaces/1dRKZlaBvwbJB?s=20
Twitter
https://twitter.com/PulsarSummit
Website
https://pulsar-summit.org/
The Pre-Report
https://www.linkedin.com/pulse/pulsar-summit-pre-report-2022-tim-spann-/
Unboxing
https://twitter.com/PaaSDev/status/1561438246403440640?s=20&t=2xXb0gOYPQaNbsx-YZTdnA
UPCOMING
https://github.com/tspannhw/FLiPStackWeekly
Come see me speak at JCONF.DEV in Chicago, Sept 25-28. This is a community-run #Java #JVM #Cloud #BigData
conference done right. No sales/product pitches here. Just amazing content. Use this link for a $100 discount on any 2 or
3-day pass cevents.io/JC22100
● September 8, 2022: Comcast Labs Connect PHLAI in Philadelphia
● Sept 13-16, 2022: OSS Summit (Virtual)
● Oct 3-6, 2022: Apache Con, New Orleans, LA.
● Oct 4-5, 2022: Current: Apache Kafka Summit. Austin, TX.
● Oct 25, 2022: KubeCon: Detroit
● Oct 25 - Nov 3: AI DevWorld (Virtual)
● Nov 23, 2022: Big Data EU (Virtual)
I hope to see my streaming friends at Current in Austin.
https://2022.currentevent.io/website/39543/welcome/
As well as all my streaming and data friends at ApacheCon in New Orleans.
https://apachecon.com/acna2022/schedule.html
OTHER ITEMS THIS WEEK
● https://www.comcastlabsconnect.com/2022-phlai
● https://www.youtube.com/watch?v=hPZS1ocmWhM
● https://www.singularity-data.com/blog/tutorial-pulsar-risingwave-for-fast-twitter-events-processing
● https://www.youtube.com/watch?v=V6u9NsIS0yY
● https://www.youtube.com/watch?v=4KQ6KTkk2no
● https://www.youtube.com/watch?v=ShjDrvV3MNE
● https://www.youtube.com/watch?v=8xADk1UBd4Q
● https://www.youtube.com/watch?v=QTnYVlmyCOw&t=6s
● https://www.youtube.com/watch?v=eGA5LRoqGJ8
● https://www.youtube.com/watch?v=tRChbhHC5fs
● https://www.youtube.com/watch?v=M3R-jBji0g4
● https://www.youtube.com/watch?v=Vfg091JLp90
● https://www.youtube.com/watch?v=l-fWXRfREB0
● https://www.youtube.com/watch?v=5ydrp0WsKnM
● https://github.com/ananthdurai/schemata
● https://www.youtube.com/watch?v=x5OFvx_Ot5o

More Related Content

More from Timothy Spann

2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...Timothy Spann
 
Conf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42-Python-Building Apache NiFi 2.0 Python ProcessorsConf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42-Python-Building Apache NiFi 2.0 Python ProcessorsTimothy Spann
 
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...Timothy Spann
 
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI PipelinesTimothy Spann
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkTimothy Spann
 
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...Timothy Spann
 
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
OSACon 2023_ Unlocking Financial Data with Real-Time PipelinesOSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
OSACon 2023_ Unlocking Financial Data with Real-Time PipelinesTimothy Spann
 
Building Real-Time Travel Alerts
Building Real-Time Travel AlertsBuilding Real-Time Travel Alerts
Building Real-Time Travel AlertsTimothy Spann
 
JConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and FlinkJConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and FlinkTimothy Spann
 
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
[EN]DSS23_tspann_Integrating LLM with Streaming Data PipelinesTimothy Spann
 
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines DemoEvolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines DemoTimothy Spann
 
AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101Timothy Spann
 
26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup
26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup
26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC MeetupTimothy Spann
 
CoC23_ Looking at the New Features of Apache NiFi
CoC23_ Looking at the New Features of Apache NiFiCoC23_ Looking at the New Features of Apache NiFi
CoC23_ Looking at the New Features of Apache NiFiTimothy Spann
 
CoC23_ Let’s Monitor The Conditions at the Conference
CoC23_ Let’s Monitor The Conditions at the ConferenceCoC23_ Let’s Monitor The Conditions at the Conference
CoC23_ Let’s Monitor The Conditions at the ConferenceTimothy Spann
 
OSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdf
OSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdfOSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdf
OSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdfTimothy Spann
 
CoC23_Utilizing Real-Time Transit Data for Travel Optimization
CoC23_Utilizing Real-Time Transit Data for Travel OptimizationCoC23_Utilizing Real-Time Transit Data for Travel Optimization
CoC23_Utilizing Real-Time Transit Data for Travel OptimizationTimothy Spann
 
The Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingThe Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingTimothy Spann
 
Meetup - Brasil - Data In Motion - 2023 September 19
Meetup - Brasil - Data In Motion - 2023 September 19Meetup - Brasil - Data In Motion - 2023 September 19
Meetup - Brasil - Data In Motion - 2023 September 19Timothy Spann
 
PartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC SolutionPartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC SolutionTimothy Spann
 

More from Timothy Spann (20)

2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
 
Conf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42-Python-Building Apache NiFi 2.0 Python ProcessorsConf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42-Python-Building Apache NiFi 2.0 Python Processors
 
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
 
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
 
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
 
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
OSACon 2023_ Unlocking Financial Data with Real-Time PipelinesOSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
 
Building Real-Time Travel Alerts
Building Real-Time Travel AlertsBuilding Real-Time Travel Alerts
Building Real-Time Travel Alerts
 
JConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and FlinkJConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and Flink
 
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines
 
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines DemoEvolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
Evolve 2023 NYC - Integrating AI Into Realtime Data Pipelines Demo
 
AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101
 
26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup
26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup
26Oct2023_Adding Generative AI to Real-Time Streaming Pipelines_ NYC Meetup
 
CoC23_ Looking at the New Features of Apache NiFi
CoC23_ Looking at the New Features of Apache NiFiCoC23_ Looking at the New Features of Apache NiFi
CoC23_ Looking at the New Features of Apache NiFi
 
CoC23_ Let’s Monitor The Conditions at the Conference
CoC23_ Let’s Monitor The Conditions at the ConferenceCoC23_ Let’s Monitor The Conditions at the Conference
CoC23_ Let’s Monitor The Conditions at the Conference
 
OSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdf
OSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdfOSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdf
OSSFinance_UnlockingFinancialDatawithReal-TimePipelines.pdf
 
CoC23_Utilizing Real-Time Transit Data for Travel Optimization
CoC23_Utilizing Real-Time Transit Data for Travel OptimizationCoC23_Utilizing Real-Time Transit Data for Travel Optimization
CoC23_Utilizing Real-Time Transit Data for Travel Optimization
 
The Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingThe Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and Streaming
 
Meetup - Brasil - Data In Motion - 2023 September 19
Meetup - Brasil - Data In Motion - 2023 September 19Meetup - Brasil - Data In Motion - 2023 September 19
Meetup - Brasil - Data In Motion - 2023 September 19
 
PartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC SolutionPartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC Solution
 

Pulsar Summit SF 2022 Report

  • 1. Pulsar Summit SF 2022 Report
  • 2. I am publishing this as a PDF since the first version of this was written natively in LinkedIn and was lost in a weird loading glitch. The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at all. The atmosphere was open source and community. The hallway talks and discussions between different open source projects and communities were awesome. It was amazing to see everyone come together for one event. From now on this is what I strive for in every event always. I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved and three things that weren’t perfect.
  • 3. Three Things I Loved ● The event felt and was driven by, for, and of the community. It had all the right content, character, and feel that you want in a conference. It was fun, full of great content, and a great experience. I want events to be like this every time. ● It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving awesome talks it would not have been great. The talks just got better and better. We saw amazing new features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short, but it worked. ● The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I work within the community, new people, and friends from other projects and platforms. Three Things That Were Not The Best ● Flying to the event. Travelling has become much more difficult with so many cancellations, covid and delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more. ● The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring, Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and ApacheCon. ● The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker! Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new M1 with 32GB RAM. Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight, since this report is already more than 20 pages. Everyone who came was awesome especially all the people working to mange, build, promote, run and make this event happen. The entire StreamNative team put in so many hours to make this run smooth. Thank you Karin, Carolyn, Sally, Addison, James, Sijie, Matteo, Alice, Sara, Doug
  • 4. and so many more. I left out so many, add yourself in the comments. It was great talking to James and Vivek from Optum, I so want to do a meetup in the Boston area. Lari (DataStax) did a great talk and has the Reactive framework to add to the Spring - Pulsar module. https://github.com/lhotari/reactive-pulsar-in-5-minutes Michael (Cloudera) came and we got to discuss upcoming Cloudera DataFlow (Apache NiFi) work with Apache Pulsar as well as SQL Stream Builder (Apache Flink) with Pulsar. We should have some cool events, articles, demos, and content soon. https://community.cloudera.com/t5/Community-Articles/Using-Apache-NiFi-with-Apache-Pulsar-for-Streaming/ta-p/33 7891 https://community.cloudera.com/t5/Cloudera-Stream-Processing/Using-Apache-Pulsar-with-SQL-Stream-Builder/m-p/ 349917 Marko and Dominiki (Memgraph) had a great discussion with me about some things that Memgraph can do with Apache Pulsar. I see some interesting use cases coming out of this. https://memgraph.com/docs/gqlalchemy/how-to-guides/streams/manage-pulsar-streams Robert (Altinity/Clickhouse) had some great discussions on fast data and is a great source for data knowledge. I can’t wait for the Altinity conference on open source data. https://altinity.com/blog/the-open-source-analytics-conference-2022-is-on-15-november Soby Chacko (VMWare / Pivotal) came for discussions on his new experimental Pulsar module for Spring and great discussions were all around. I am located near him so we will work on bringing a Spring-Pulsar event to Philly, New Jersey, and/or NYC. Stay tuned! ● https://github.com/spring-projects-experimental/spring-pulsar ● https://spring.io/blog/2022/08/16/introducing-experimental-spring-support-for-apache-pulsar ● https://github.com/spring-projects/spring-integration/issues/3771
  • 5. ● https://github.com/spring-projects-experimental/spring-pulsar/tree/main/spring-pulsar-sample-apps ● https://docs.spring.io/spring-pulsar/docs/current-SNAPSHOT/reference/html/ David (StreamNative) was there and he signed his awesome Pulsar book. It’s amazing to have a co-worker like David answer the most obscure questions ever. https://github.com/david-streamlio/table-view-demo https://streamnative.io/download/manning-ebook-apache-pulsar-in-action Decodable sponsored food and were around to chat which is awesome. They have great articles. https://www.decodable.co/blog/cleansing-osquery-logs-for-apache-pinot
  • 6. Schedule of Talks https://pulsar-summit.org/event/san-francisco-2022/schedule I missed some great talks by Matteo, David, Lari, Heesung, Kai, and Zach. I can’t wait for the videos and slides to come out.
  • 7. Ecosystem Track (I hosted this one) Nick (Databricks) did a cool talk on DeltaLake plus Apache Pulsar. ● https://pulsar-summit.org/event/san-francisco-2022/sessions/building-reliable-lakehouses-with-apache-pulsa r-and-delta-lake ● https://github.com/tspannhw/FLiP-Pi-DeltaLake-Thermal
  • 8. Addison (StreamNative) and Alexey (OneHouse) did a cool talk on Apache Hudi and Apache Pulsar. ● https://pulsar-summit.org/event/san-francisco-2022/sessions/unlocking-the-power-of-lakehouse-architecture s-with-apache-pulsar-and-apache-hudi ● https://github.com/tspannhw/pulsar-io-lakehouse
  • 9. Caito (Ververica) did an awesome talk on Apache Flink, Apache Flink SQL, and using Flink with Apache Pulsar. As this is the majority of the FLiP Stack this is my favorite stuff. There was a debate on when to use Flink vs Pulsar SQL (Presto/Trino) vs Pulsar PFSQL vs Spark SQL. For massively scalable continuous SQL with joins, aggregations, and stateful streaming you should use Flink. Caito also liked my FLiP Stack Cat sticker.!
  • 10. Ricardo (AWS) did a jaw-dropping talk. I did not believe someone would do a talk that was as good as Caito. Ricardo’s was an amazing all-demo talk fully documented on Github. This was amazing to see him walk through such a complex usage of Pulsar’s KoP to run as a full-fledged Kafka cluster allowing for KSQLDB apps, CDC, and Kafka microservices. We are planning to do an open source streaming meetup/round table at a Brazilian Churrascaria. ● https://github.com/riferrei/is-using-kop-a-good-idea ● https://github.com/streamnative/kop ● https://talks.riferrei.com/LgUiHy/is-using-kop-kafka-on-pulsar-a-good-idea
  • 11. Neng (StreamNative) had the newest and shiniest tech to offer and demo. Pulsar Functions utilizing SQL instead of Java, Python, or Golang.
  • 12. https://pulsar-summit.org/event/san-francisco-2022/sessions/simplify-pulsar-functions-development-with-sql Peter (Scylla) is the consummate professional and every talk he does is a masterclass in data. Every time he speaks it’s a great learning experience. I can’t wait until we do another masterclass or a round table. He wrapped up the ecosystem track with an awesome talk on the future of all things data. Where do you stand on the Data Engineer vs Data Scientist architectures?
  • 14. For continuous updates on slides, articles, photos, code, and more: https://github.com/tspannhw/PulsarSummitSF2022 Check out the Twitter stream for #PulsarSummmitSF https://twitter.com/hashtag/PulsarSummitSF?src=hashtag_click&f=live Check out Scylla’s awesome Pulsar Summit Roundup https://twitter.com/i/spaces/1dRKZlaBvwbJB?s=20 Twitter https://twitter.com/PulsarSummit Website https://pulsar-summit.org/ The Pre-Report https://www.linkedin.com/pulse/pulsar-summit-pre-report-2022-tim-spann-/ Unboxing https://twitter.com/PaaSDev/status/1561438246403440640?s=20&t=2xXb0gOYPQaNbsx-YZTdnA
  • 16. Come see me speak at JCONF.DEV in Chicago, Sept 25-28. This is a community-run #Java #JVM #Cloud #BigData conference done right. No sales/product pitches here. Just amazing content. Use this link for a $100 discount on any 2 or 3-day pass cevents.io/JC22100 ● September 8, 2022: Comcast Labs Connect PHLAI in Philadelphia ● Sept 13-16, 2022: OSS Summit (Virtual) ● Oct 3-6, 2022: Apache Con, New Orleans, LA. ● Oct 4-5, 2022: Current: Apache Kafka Summit. Austin, TX. ● Oct 25, 2022: KubeCon: Detroit ● Oct 25 - Nov 3: AI DevWorld (Virtual) ● Nov 23, 2022: Big Data EU (Virtual) I hope to see my streaming friends at Current in Austin. https://2022.currentevent.io/website/39543/welcome/ As well as all my streaming and data friends at ApacheCon in New Orleans. https://apachecon.com/acna2022/schedule.html OTHER ITEMS THIS WEEK ● https://www.comcastlabsconnect.com/2022-phlai ● https://www.youtube.com/watch?v=hPZS1ocmWhM
  • 17. ● https://www.singularity-data.com/blog/tutorial-pulsar-risingwave-for-fast-twitter-events-processing ● https://www.youtube.com/watch?v=V6u9NsIS0yY ● https://www.youtube.com/watch?v=4KQ6KTkk2no ● https://www.youtube.com/watch?v=ShjDrvV3MNE ● https://www.youtube.com/watch?v=8xADk1UBd4Q ● https://www.youtube.com/watch?v=QTnYVlmyCOw&t=6s ● https://www.youtube.com/watch?v=eGA5LRoqGJ8 ● https://www.youtube.com/watch?v=tRChbhHC5fs ● https://www.youtube.com/watch?v=M3R-jBji0g4 ● https://www.youtube.com/watch?v=Vfg091JLp90 ● https://www.youtube.com/watch?v=l-fWXRfREB0 ● https://www.youtube.com/watch?v=5ydrp0WsKnM ● https://github.com/ananthdurai/schemata ● https://www.youtube.com/watch?v=x5OFvx_Ot5o