Pulsar Summit SF 2022 Report
My report on Pulsar Summit SF 2022 https://github.com/tspannhw/FLipStackWeekly The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at all. The atmosphere was open source and community. The hallway talks and discussions between different open source projects and communities were awesome. It was amazing to see everyone come together for one event. From now on this is what I strive for in every event always. I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved and three things that weren’t perfect. Three Things I Loved The event felt and was driven by, for, and of the community. It had all the right content, character, and feel that you want in a conference. It was fun, full of great content, and a great experience. I want events to be like this every time. It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving awesome talks it would not have been great. The talks just got better and better. We saw amazing new features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short, but it worked. The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I work within the community, new people, and friends from other projects and platforms. Three Things That Were Not The Best Flying to the event. Travelling has become much more difficult with so many cancellations, covid and delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more. The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring, Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and ApacheCon. The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker! Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new M1 with 32GB RAM. Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight, since this report is already more than 20 pages.
Pulsar Summit SF 2022 Report
My report on Pulsar Summit SF 2022 https://github.com/tspannhw/FLipStackWeekly The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at all. The atmosphere was open source and community. The hallway talks and discussions between different open source projects and communities were awesome. It was amazing to see everyone come together for one event. From now on this is what I strive for in every event always. I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved and three things that weren’t perfect. Three Things I Loved The event felt and was driven by, for, and of the community. It had all the right content, character, and feel that you want in a conference. It was fun, full of great content, and a great experience. I want events to be like this every time. It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving awesome talks it would not have been great. The talks just got better and better. We saw amazing new features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short, but it worked. The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I work within the community, new people, and friends from other projects and platforms. Three Things That Were Not The Best Flying to the event. Travelling has become much more difficult with so many cancellations, covid and delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more. The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring, Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and ApacheCon. The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker! Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new M1 with 32GB RAM. Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight, since this report is already more than 20 pages.
Recommended
![[EN]DSS23_tspann_Integrating LLM with Streaming Data Pipelines](https://cdn.slidesharecdn.com/ss_thumbnails/endss23tspannintegratingllmwithstreamingdatapipelines-231115161007-49510320-thumbnail.jpg?width=384&fit=bounds)
More Related Content
More from Timothy Spann
More from Timothy Spann(20)
Pulsar Summit SF 2022 Report
- 1. Pulsar Summit SF 2022 Report
- 2. I am publishing this as a PDF since the first version of this was written natively in LinkedIn and was lost in a weird loading glitch. The event was a lot of fun, it was so impressive to see how well it came together. The event felt like a giant well-run meetup but with professionalism and gourmet food and surroundings. What I mean is that it did not feel corporate at all. The atmosphere was open source and community. The hallway talks and discussions between different open source projects and communities were awesome. It was amazing to see everyone come together for one event. From now on this is what I strive for in every event always. I follow an RV review series because the host is awesome and I was looking for a travel trailer for a while. Matt’s Reviews follows a formula I want to follow for my review of this awesome event. He does three things that he loved and three things that weren’t perfect.
- 3. Three Things I Loved ● The event felt and was driven by, for, and of the community. It had all the right content, character, and feel that you want in a conference. It was fun, full of great content, and a great experience. I want events to be like this every time. ● It’s great that things were so well organized and fun, but if there weren’t the amazing speakers giving awesome talks it would not have been great. The talks just got better and better. We saw amazing new features, future items, great use cases, and amazing demos. I was afraid 30-minute talks were too short, but it worked. ● The main thing that made it amazing was the people. So many great speakers, sponsors, and attendees that every minute was filled with great content, interactions, and fun. It was so cool to see so many people I work within the community, new people, and friends from other projects and platforms. Three Things That Were Not The Best ● Flying to the event. Travelling has become much more difficult with so many cancellations, covid and delays. Virtual was so awesome for avoiding this time and money sink. The only sinks I like send data to Scylla, Elastic, Delta Lake, Apache Hudi, Clickhouse, and more. ● The event was only one day! It was like just a taste, I need more Pulsar. I need more Scylla, Spring, Flink, NiFi, Clickhouse, Hudi, Delta Lake, Spring, and FLiPN. I can’t wait for longer events @ Current and ApacheCon. ● The talks were only 30 minutes long. I want so much more, but it does seem to be the sweet spot for getting the crux of the topic. Ricardo crushed it with an all-live Pulsar - KoP - KSQLDB demo in docker! Wow! M1 on Docker is not the best though. Fix that Apple! Though I am not complaining about my new M1 with 32GB RAM. Now to expand on one of the great parts of the event, the People! There are a lot of people that I did not highlight, since this report is already more than 20 pages. Everyone who came was awesome especially all the people working to mange, build, promote, run and make this event happen. The entire StreamNative team put in so many hours to make this run smooth. Thank you Karin, Carolyn, Sally, Addison, James, Sijie, Matteo, Alice, Sara, Doug
- 4. and so many more. I left out so many, add yourself in the comments. It was great talking to James and Vivek from Optum, I so want to do a meetup in the Boston area. Lari (DataStax) did a great talk and has the Reactive framework to add to the Spring - Pulsar module. https://github.com/lhotari/reactive-pulsar-in-5-minutes Michael (Cloudera) came and we got to discuss upcoming Cloudera DataFlow (Apache NiFi) work with Apache Pulsar as well as SQL Stream Builder (Apache Flink) with Pulsar. We should have some cool events, articles, demos, and content soon. https://community.cloudera.com/t5/Community-Articles/Using-Apache-NiFi-with-Apache-Pulsar-for-Streaming/ta-p/33 7891 https://community.cloudera.com/t5/Cloudera-Stream-Processing/Using-Apache-Pulsar-with-SQL-Stream-Builder/m-p/ 349917 Marko and Dominiki (Memgraph) had a great discussion with me about some things that Memgraph can do with Apache Pulsar. I see some interesting use cases coming out of this. https://memgraph.com/docs/gqlalchemy/how-to-guides/streams/manage-pulsar-streams Robert (Altinity/Clickhouse) had some great discussions on fast data and is a great source for data knowledge. I can’t wait for the Altinity conference on open source data. https://altinity.com/blog/the-open-source-analytics-conference-2022-is-on-15-november Soby Chacko (VMWare / Pivotal) came for discussions on his new experimental Pulsar module for Spring and great discussions were all around. I am located near him so we will work on bringing a Spring-Pulsar event to Philly, New Jersey, and/or NYC. Stay tuned! ● https://github.com/spring-projects-experimental/spring-pulsar ● https://spring.io/blog/2022/08/16/introducing-experimental-spring-support-for-apache-pulsar ● https://github.com/spring-projects/spring-integration/issues/3771
- 5. ● https://github.com/spring-projects-experimental/spring-pulsar/tree/main/spring-pulsar-sample-apps ● https://docs.spring.io/spring-pulsar/docs/current-SNAPSHOT/reference/html/ David (StreamNative) was there and he signed his awesome Pulsar book. It’s amazing to have a co-worker like David answer the most obscure questions ever. https://github.com/david-streamlio/table-view-demo https://streamnative.io/download/manning-ebook-apache-pulsar-in-action Decodable sponsored food and were around to chat which is awesome. They have great articles. https://www.decodable.co/blog/cleansing-osquery-logs-for-apache-pinot
- 6. Schedule of Talks https://pulsar-summit.org/event/san-francisco-2022/schedule I missed some great talks by Matteo, David, Lari, Heesung, Kai, and Zach. I can’t wait for the videos and slides to come out.
- 7. Ecosystem Track (I hosted this one) Nick (Databricks) did a cool talk on DeltaLake plus Apache Pulsar. ● https://pulsar-summit.org/event/san-francisco-2022/sessions/building-reliable-lakehouses-with-apache-pulsa r-and-delta-lake ● https://github.com/tspannhw/FLiP-Pi-DeltaLake-Thermal
- 8. Addison (StreamNative) and Alexey (OneHouse) did a cool talk on Apache Hudi and Apache Pulsar. ● https://pulsar-summit.org/event/san-francisco-2022/sessions/unlocking-the-power-of-lakehouse-architecture s-with-apache-pulsar-and-apache-hudi ● https://github.com/tspannhw/pulsar-io-lakehouse
- 9. Caito (Ververica) did an awesome talk on Apache Flink, Apache Flink SQL, and using Flink with Apache Pulsar. As this is the majority of the FLiP Stack this is my favorite stuff. There was a debate on when to use Flink vs Pulsar SQL (Presto/Trino) vs Pulsar PFSQL vs Spark SQL. For massively scalable continuous SQL with joins, aggregations, and stateful streaming you should use Flink. Caito also liked my FLiP Stack Cat sticker.!
- 10. Ricardo (AWS) did a jaw-dropping talk. I did not believe someone would do a talk that was as good as Caito. Ricardo’s was an amazing all-demo talk fully documented on Github. This was amazing to see him walk through such a complex usage of Pulsar’s KoP to run as a full-fledged Kafka cluster allowing for KSQLDB apps, CDC, and Kafka microservices. We are planning to do an open source streaming meetup/round table at a Brazilian Churrascaria. ● https://github.com/riferrei/is-using-kop-a-good-idea ● https://github.com/streamnative/kop ● https://talks.riferrei.com/LgUiHy/is-using-kop-kafka-on-pulsar-a-good-idea
- 11. Neng (StreamNative) had the newest and shiniest tech to offer and demo. Pulsar Functions utilizing SQL instead of Java, Python, or Golang.
- 12. https://pulsar-summit.org/event/san-francisco-2022/sessions/simplify-pulsar-functions-development-with-sql Peter (Scylla) is the consummate professional and every talk he does is a masterclass in data. Every time he speaks it’s a great learning experience. I can’t wait until we do another masterclass or a round table. He wrapped up the ecosystem track with an awesome talk on the future of all things data. Where do you stand on the Data Engineer vs Data Scientist architectures?
- 14. For continuous updates on slides, articles, photos, code, and more: https://github.com/tspannhw/PulsarSummitSF2022 Check out the Twitter stream for #PulsarSummmitSF https://twitter.com/hashtag/PulsarSummitSF?src=hashtag_click&f=live Check out Scylla’s awesome Pulsar Summit Roundup https://twitter.com/i/spaces/1dRKZlaBvwbJB?s=20 Twitter https://twitter.com/PulsarSummit Website https://pulsar-summit.org/ The Pre-Report https://www.linkedin.com/pulse/pulsar-summit-pre-report-2022-tim-spann-/ Unboxing https://twitter.com/PaaSDev/status/1561438246403440640?s=20&t=2xXb0gOYPQaNbsx-YZTdnA
- 16. Come see me speak at JCONF.DEV in Chicago, Sept 25-28. This is a community-run #Java #JVM #Cloud #BigData conference done right. No sales/product pitches here. Just amazing content. Use this link for a $100 discount on any 2 or 3-day pass cevents.io/JC22100 ● September 8, 2022: Comcast Labs Connect PHLAI in Philadelphia ● Sept 13-16, 2022: OSS Summit (Virtual) ● Oct 3-6, 2022: Apache Con, New Orleans, LA. ● Oct 4-5, 2022: Current: Apache Kafka Summit. Austin, TX. ● Oct 25, 2022: KubeCon: Detroit ● Oct 25 - Nov 3: AI DevWorld (Virtual) ● Nov 23, 2022: Big Data EU (Virtual) I hope to see my streaming friends at Current in Austin. https://2022.currentevent.io/website/39543/welcome/ As well as all my streaming and data friends at ApacheCon in New Orleans. https://apachecon.com/acna2022/schedule.html OTHER ITEMS THIS WEEK ● https://www.comcastlabsconnect.com/2022-phlai ● https://www.youtube.com/watch?v=hPZS1ocmWhM
- 17. ● https://www.singularity-data.com/blog/tutorial-pulsar-risingwave-for-fast-twitter-events-processing ● https://www.youtube.com/watch?v=V6u9NsIS0yY ● https://www.youtube.com/watch?v=4KQ6KTkk2no ● https://www.youtube.com/watch?v=ShjDrvV3MNE ● https://www.youtube.com/watch?v=8xADk1UBd4Q ● https://www.youtube.com/watch?v=QTnYVlmyCOw&t=6s ● https://www.youtube.com/watch?v=eGA5LRoqGJ8 ● https://www.youtube.com/watch?v=tRChbhHC5fs ● https://www.youtube.com/watch?v=M3R-jBji0g4 ● https://www.youtube.com/watch?v=Vfg091JLp90 ● https://www.youtube.com/watch?v=l-fWXRfREB0 ● https://www.youtube.com/watch?v=5ydrp0WsKnM ● https://github.com/ananthdurai/schemata ● https://www.youtube.com/watch?v=x5OFvx_Ot5o