SpringOne Tour 2023: Let&#x27;s Get Streaming! A Guide to Orchestrating Spring Cloud Stream Apps with Glenn Renfro

Date: April 25, 2023 Speaker: Glenn Renfro, Engineer, VMware

Software

Glenn Renfro
Twitter: @cppwfs
Mastodon: @cppwfs@fosstodon.org
Let's Get Streaming!
A Guide to Orchestrating Spring
Cloud Stream Apps
Copyright © 2022 VMware, Inc. or its aﬃliates.

Cover w/ Image
Agenda
● A Function
● Let’s make it an App
● Let’s Orchestrate a Stream with
Data Flow
● Let’s Mutate that Stream with Data
Flow
● Let’s use that data!

Tools
Spring Boot
Spring Cloud Stream
Spring Cloud Data Flow
Spring Cloud Task
Spring Batch

Spring Cloud Data Flow
Spring Cloud Data Flow provides tools to create complex topologies
for streaming and batch data pipelines. The data pipelines consist of
Spring Boot apps, built using the Spring Cloud Stream or Spring
Cloud Task microservice frameworks.

Orchestrating Streams
Create Streams
Deploy Streams
Upgrade Apps in the stream
Mutate the Stream
Undeploy Streams

Java 8 Function:
@Bean
Function<String, String> checkCoordinate() {
return s -> (
StringUtils.commaDelimitedListToStringArray(s).length == 2) ?
"Yep" : "Nope";
}

Thank You
https://github.com/cppwfs/checkcoordinate https://github.com/cppwfs/letsplay

10
Minute
Break
Mario Gray
Native, Serverless μServices in the
Stream

Are you interested in learning how to schedule batch jobs in container runtimes? Maybe you’re wondering how to apply continuous delivery in practice for data-intensive applications? Perhaps you’re looking for an orchestration tool for data pipelines? Questions like these are common, so rest assured that you’re not alone. In this webinar, we’ll cover the recent feature improvements in Spring Cloud Data Flow. More specifically, we’ll discuss data processing use cases and how they simplify the overall orchestration experience in cloud runtimes like Cloud Foundry and Kubernetes. Please join us and be part of the community discussion! Presenters : Sabby Anandan, Product Manager Mark Pollack, Software Engineer, Pivotal

Spring Framework

nomykk

Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...

Data Engineer's Lunch #56: Spring Cloud Data Flow with Cassandra

Anant Corporation

In Data Engineer's Lunch #55 we will be going over how to integrate Spring Cloud Data Flow with Cassandra. Accompanying Blog: Coming Soon! Accompanying YouTube: Coming Soon! Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday: https://www.meetup.com/Data-Wranglers-DC/events/ Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cassandra Email: solutions@anant.us LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-1072927283 Facebook: https://www.facebook.com/AnantCorp/ Join The Anant Team: https://www.careers.anant.us

Microservices with kubernetes @190316

Jupil Hwang

SpringBoot and Spring Cloud Service for MSA

Oracle Korea

How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...

Lightbend

Developing real-time data pipelines with Spring and Kafka

marius_bogoevici

Talk given at the Apache Kafka NYC Meetup, October 20, 2015. http://www.meetup.com/Apache-Kafka-NYC/events/225697500/ Kafka has emerged as a clear choice for a high-throughput, low latency messaging system that addresses the needs of high-performance streaming applications. The Spring Framework has been, in the last decade, the de-facto standard for developing enterprise Java applications, providing a simple and powerful programming model that allows developers to focus on the business needs, leaving the boilerplate and middleware integration to the framework itself. In fact, it has evolved into a rich and powerful ecosystem, with projects focusing on specific aspects of enterprise software development - like Spring Boot, Spring Data, Spring Integration, Spring XD, Spring Cloud Stream/Data Flow to name just a few. In this presentation, Marius Bogoevici from the Spring team will take the perspective of the Kafka user, and show, with live demos, how the various projects in the Spring ecosystem address their needs: - how to build simple data integration applications using Spring Integration Kafka; - how to build sophisticated data pipelines with Spring XD and Kafka; - how to build cloud native message-driven microservices using Spring Cloud Stream and Kafka, and how to orchestrate them using Spring Cloud Data Flow;

Pivoting Spring XD to Spring Cloud Data Flow: A microservice based architecture for stream processing Microservice based architectures are not just for distributed web applications! They are also a powerful approach for creating distributed stream processing applications. Spring Cloud Data Flow enables you to create and orchestrate standalone executable applications that communicate over messaging middleware such as Kafka and RabbitMQ that when run together, form a distributed stream processing application. This allows you to scale, version and operationalize stream processing applications following microservice based patterns and practices on a variety of runtime platforms such as Cloud Foundry, Apache YARN and others. About Sabby Anandan Sabby Anandan is a Product Manager at Pivotal. Sabby is focused on building products that eliminate the barriers between application development, cloud, and big data.

Best of Microsoft Dev Camp 2015

Bluegrass Digital

Scheduling Apps in the Cloud - Glenn Renfro & Roy Clarkson

Scheduling Apps in the Cloud - Glenn Renfro & Roy Clarkson

Spring and Pivotal Application Service - SpringOne Tour Dallas

Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...

SpringOne Platform 2017 Gilbert Lau, Data Stax; Wayne Lund, Pivotal "Spring Cloud Data Flow satisfies all of the demands of modern streaming and task workloads. A growing number of customers are viewing Pivotal Cloud Foundry as an ideal runtime for these types of workloads to take advantage of all of the microservice architecture features of Spring Boot apps leveraging Spring Cloud Services. This is only half of the equation. Once the streaming data is persisted on their database, our customers want to generate actionable insights to provide the best customer experience to stay on top of the competitive marketplace. DataStax Enterprise (DSE) is a single and unified big data platform with Apache Cassandra NoSQL database at its core. Integrated within each node of DSE is powerful indexing, search through Apache Solr, analytics through Apache Spark, and a enterprise-ready graph functionality. It is by far the only operational data platform which can scale linearly in excess of 1,000 nodes, with no single point of failure, and is capable of providing real-time active-everywhere replication across many datacenters and cloud providers. In this presentation and demo we will take a common social data set and show SCDF advantages on PCF for microservice scaling and pipelining data into a DataStax Enterprise Cassandra NoSQL database. Then followed by extracting meaningful information through DataStax Enterprise Search, DataStax Enterprise Analytics, and DataStax Cassandra Service Broker Tile for PCF using a Spring Boot Dashboard application."

Prudential Insurance ExpAnkit Chohan

Spring Boot

HongSeong Jeon

Play Support in Cloud Foundry

rajdeep

AWS re:Invent 2016: Building a Platform for Collaborative Scientific Research...

This session discusses the architecture, formation, and usage of a collaborative HPC/big data scientific research and analysis environment on AWS. The pharmaceutical industry trend toward joint ventures and collaborations has created a need for new platforms in which to work together. We'll dive into architectural decisions for building collaborative systems. Examples include how such a platform allowed Human Longevity, Inc. to accelerate software deployment to production in a fast-paced research environment, and how Celgene uses AWS for research collaboration with outside universities and foundations.

V fabric overviewPronam Chatterjee

App Service Web

Lisa Muthukumar

AWS July Webinar Series: Introducing AWS OpsWorks for Windows Server

When managing your Windows infrastructure on AWS, you can automate operational tasks, like service discovery, software installation and configuration management. Using AWS OpsWorks, you can deploy, manage, and scale your applications running on Windows Server. This webinar will show you how to use Chef recipes with OpsWorks to install or update software packages and security updates. You will also learn how to use OpsWorks to react to lifecycle events, such as running launch and shutdown scripts, or adapting your application's configurations. Learning Objectives: Understand the benefits of using OpsWorks on Windows Server applications

AWS re:Invent 2016: Chalk Talk: Succeeding at Infrastructure-as-Code (GPSCT312)

The days of manually managing infrastructure tasks are quickly coming to an end; businesses increasingly need their infrastructure teams to react with the same agility of their development teams. In this session, we discuss various approaches to infrastructure-as-code utilizing AWS solutions across the areas of templated infrastructure provisioning, configuration management, and policy as code. We invite you to bring your questions and join AWS Solutions Architects as we dive deeper into the concepts and best practices behind infrastructure-as-code.

Multi client Development with Spring

Joshua Long

Handling Asynchronous Workloads With OpenShift and Iron.io

Ivan Dwyer

Cloud web applications: the new perspective of sproutcoreDavid Saitta

_Python Ireland Meetup - Serverless ML - Dowling.pdf

Jim Dowling

Spring into AI presented by Dan Vega 5/14

What AI Means For Your Product Strategy And What To Do About It

Similar to SpringOne Tour 2023: Let's Get Streaming! A Guide to Orchestrating Spring Cloud Stream Apps with Glenn Renfro

Spring Boot Whirlwind Tour

Data Pipeline for The Big Data/Data Science OKC

Mark Smith

Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan

PivotalOpenSourceHub

Best of Microsoft Dev Camp 2015

Bluegrass Digital

Scheduling Apps in the Cloud - Glenn Renfro & Roy Clarkson

Scheduling Apps in the Cloud - Glenn Renfro & Roy Clarkson

Spring and Pivotal Application Service - SpringOne Tour Dallas

Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...

Prudential Insurance ExpAnkit Chohan

Spring Boot

HongSeong Jeon

Play Support in Cloud Foundry

rajdeep

AWS re:Invent 2016: Building a Platform for Collaborative Scientific Research...

V fabric overviewPronam Chatterjee

App Service Web

Lisa Muthukumar

AWS July Webinar Series: Introducing AWS OpsWorks for Windows Server

AWS re:Invent 2016: Chalk Talk: Succeeding at Infrastructure-as-Code (GPSCT312)

Multi client Development with Spring

Joshua Long

Handling Asynchronous Workloads With OpenShift and Iron.io

Ivan Dwyer

Cloud web applications: the new perspective of sproutcoreDavid Saitta

_Python Ireland Meetup - Serverless ML - Dowling.pdf

Jim Dowling

Similar to SpringOne Tour 2023: Let's Get Streaming! A Guide to Orchestrating Spring Cloud Stream Apps with Glenn Renfro (20)

Spring Boot Whirlwind Tour

Data Pipeline for The Big Data/Data Science OKC

Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan

Best of Microsoft Dev Camp 2015

Scheduling Apps in the Cloud - Glenn Renfro & Roy Clarkson

Spring and Pivotal Application Service - SpringOne Tour Dallas

Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...

Prudential Insurance Exp

Spring Boot

Play Support in Cloud Foundry

AWS re:Invent 2016: Building a Platform for Collaborative Scientific Research...

V fabric overview

App Service Web

AWS July Webinar Series: Introducing AWS OpsWorks for Windows Server

AWS re:Invent 2016: Chalk Talk: Succeeding at Infrastructure-as-Code (GPSCT312)

Multi client Development with Spring

Handling Asynchronous Workloads With OpenShift and Iron.io

Cloud web applications: the new perspective of sproutcore

_Python Ireland Meetup - Serverless ML - Dowling.pdf

Recently uploaded

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...

Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.

Prosigns: Transforming Business with Tailored Technology Solutions

Prosigns

Unlocking Business Potential: Tailored Technology Solutions by Prosigns Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support. Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth. Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices. AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making. Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency. DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration. Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly. Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business. Join us on a journey of innovation and growth. Let's partner for success with Prosigns.

May Marketo Masterclass, London MUG May 22 2024.pdf

Adele Miller

Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...

Mind IT Systems

Graphic Design Crash Course for beginners

e20449

How Recreation Management Software Can Streamline Your Operations.pptx

wottaspaceseo

Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.

Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...

Anthony Dahanne

Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ? Venez le découvrir lors de cette session ignite

Developing Distributed High-performance Computing Capabilities of an Open Sci...

COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.

Globus Connect Server Deep Dive - GlobusWorld 2024

How to Position Your Globus Data Portal for Success Ten Good Practices

Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.

Providing Globus Services to Users of JASMIN for Environmental Data Analysis