Walking Through Spring
Cloud Data Flow
September 2–3, 2020
springone.io
Safe Harbor Statement
The following is intended to outline the general direction of VMware's offerings. It is intended for information
purposes only and may not be incorporated into any contract. Any information regarding pre-release of
VMware offerings, future updates or other planned modifications is subject to ongoing evaluation by
VMware and is subject to change. This information is provided without warranty or any kind, express or
implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied
upon in making purchasing decisions regarding VMware's offerings. These purchasing decisions should
only be based on features currently available. The development, release, and timing of any features or
functionality described for VMware's offerings in this presentation remain at the sole discretion of Pivotal.
Pivotal has no obligation to update forward looking information in this presentation.
2
3
Glenn Renfro
Twitter: @cppwfs
Github: github.com/cppwfs
Ilayaperumal Gopinathan
Twitter: @ilayaperumalg
Github: github.com/ilayaperumalg
Data In the Enterprise
?
8
Application Development
Spring Cloud Stream
Event-driven, long-lived
streaming apps
Spring Cloud Task
Ephemeral, batch apps
Continuous
Integration
Kubernetes / Cloud Foundry
Message Broker
/ / / / /
Application Deployment
Streaming applications
Task (batch) applications
Monitor
Performance
Track
Lifecycle
Deploy
Streams
Orchestration using
Spring Cloud Data Flow
Launch
Jobs
Demo Time!
1
2
Sample Demo Data
{
"sku":"102452676",
"amount":"19.99",
"quantity":"1",
"mode":"debit",
"clicks":"10"
}
6
Demo!
What’s New!
Single Step Batch Job Auto Configuration
@EnableBatchProcessing
@EnableTask
@Configuration
public class MyBatchConfiguration {
…
@Bean
public Job myBatchJob(Step step) {
return jobBuilderFactory.get(this.properties.getJobName()).
…
}
…
@Bean
public ItemReader<Map<Object, Object>> itemReader() {
return new JdbcCurserItemReaderBuilder().
…
}
@Bean
public ItemWriter<Map<Object,Object>> itemWriter(DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<Map<Object, Object>>().
…
}
What’s New!
Single Step Batch Job Auto Configuration
spring.batch.job.jobName=job
spring.batch.job.stepName=step1
spring.batch.job.chunkSize=5
spring.batch.job.jdbccursorreader.name=fooReader
spring.batch.job.jdbccursorreader.sql=select item_name from item_sample
spring.batch.job.jdbcwriter.name=jdbcWriter
spring.batch.job.jdbcwriter.sql=INSERT INTO item (item_name) VALUES (:ITEM_NAME)
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-single-step-batch-job</artifactId>
</dependency>
What’s new!
Spring Cloud Data Flow OSS
• 2.6.1 RELEASE
● Task Improvements
● OOTB Functional applications
● Wavefront monitoring support
● Multi platform scheduling support
● Bitnami helm charts for SCDF
Commercial Offering
Spring Cloud Data Flow For Kubernetes
• Built on top of SCDF OSS
• SCDF Pro Server and Pro Dashboard UI
• Packaging
• Certified Container Images
• Image Relocation
• Provisioning of SCDF components
• Multi IO Stream DSL and Editor
What’s Up Next!
Spring Cloud Data Flow OSS
• 2.7.0 in the works
○ Dashboard redesign using Clarity
○ Observability support
○ Single Step Batch Job support
What’s Up Next!
Spring Cloud Data Flow for Kubernetes (SCDF4K8s)
• Container Registry Integration
• Single Step Batch Job Auto Config
• Multi IO support enhancements
Want to learn more?
#springone@s1p
SCDF microsite
SCDF for Kubernetes
Stay Connected.
https://springone.io/2020/sessions/iot-scale-event-stream-processing-for-
predictive-maintenance-at-penske
#springone@s1p
IoT Scale Event-Stream Processing for
Predictive Maintenance at Penske

Walking Through Spring Cloud Data Flow

  • 1.
    Walking Through Spring CloudData Flow September 2–3, 2020 springone.io
  • 2.
    Safe Harbor Statement Thefollowing is intended to outline the general direction of VMware's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of VMware offerings, future updates or other planned modifications is subject to ongoing evaluation by VMware and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding VMware's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for VMware's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 2
  • 3.
    3 Glenn Renfro Twitter: @cppwfs Github:github.com/cppwfs Ilayaperumal Gopinathan Twitter: @ilayaperumalg Github: github.com/ilayaperumalg
  • 4.
    Data In theEnterprise
  • 7.
  • 8.
  • 9.
    Application Development Spring CloudStream Event-driven, long-lived streaming apps Spring Cloud Task Ephemeral, batch apps Continuous Integration Kubernetes / Cloud Foundry Message Broker / / / / / Application Deployment Streaming applications Task (batch) applications Monitor Performance Track Lifecycle Deploy Streams Orchestration using Spring Cloud Data Flow Launch Jobs
  • 10.
  • 12.
  • 13.
  • 14.
    What’s New! Single StepBatch Job Auto Configuration
  • 15.
    @EnableBatchProcessing @EnableTask @Configuration public class MyBatchConfiguration{ … @Bean public Job myBatchJob(Step step) { return jobBuilderFactory.get(this.properties.getJobName()). … } … @Bean public ItemReader<Map<Object, Object>> itemReader() { return new JdbcCurserItemReaderBuilder(). … } @Bean public ItemWriter<Map<Object,Object>> itemWriter(DataSource dataSource) { return new JdbcBatchItemWriterBuilder<Map<Object, Object>>(). … }
  • 16.
    What’s New! Single StepBatch Job Auto Configuration spring.batch.job.jobName=job spring.batch.job.stepName=step1 spring.batch.job.chunkSize=5 spring.batch.job.jdbccursorreader.name=fooReader spring.batch.job.jdbccursorreader.sql=select item_name from item_sample spring.batch.job.jdbcwriter.name=jdbcWriter spring.batch.job.jdbcwriter.sql=INSERT INTO item (item_name) VALUES (:ITEM_NAME) <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-single-step-batch-job</artifactId> </dependency>
  • 17.
    What’s new! Spring CloudData Flow OSS • 2.6.1 RELEASE ● Task Improvements ● OOTB Functional applications ● Wavefront monitoring support ● Multi platform scheduling support ● Bitnami helm charts for SCDF
  • 18.
    Commercial Offering Spring CloudData Flow For Kubernetes • Built on top of SCDF OSS • SCDF Pro Server and Pro Dashboard UI • Packaging • Certified Container Images • Image Relocation • Provisioning of SCDF components • Multi IO Stream DSL and Editor
  • 21.
    What’s Up Next! SpringCloud Data Flow OSS • 2.7.0 in the works ○ Dashboard redesign using Clarity ○ Observability support ○ Single Step Batch Job support
  • 22.
    What’s Up Next! SpringCloud Data Flow for Kubernetes (SCDF4K8s) • Container Registry Integration • Single Step Batch Job Auto Config • Multi IO support enhancements
  • 23.
    Want to learnmore? #springone@s1p SCDF microsite SCDF for Kubernetes
  • 24.