Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

What’s New in Spring Batch?

Download to read offline

SpringOne 2020
What’s New in Spring Batch?

Mahmoud Ben Hassine, Principal Software Engineer at VMware

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

What’s New in Spring Batch?

  1. 1. What’s new in Spring Batch 4.3? Mahmoud Ben Hassine September 2–3, 2020 springone.io
  2. 2. Safe Harbor Statement The following is intended to outline the general direction of VMware's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of VMware offerings, future updates or other planned modifications is subject to ongoing evaluation by VMware and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding VMware's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for VMware's offerings in this presentation remain at the sole discretion of VMware. VMware has no obligation to update forward looking information in this presentation. 2
  3. 3. About me 3 • Principal Software Engineer at VMware • Spring Batch Co-Lead • Open Source enthusiast Github: @benas Twitter: @b_e_n_a_s Mahmoud Ben Hassine
  4. 4. Agenda 4 • Current status • What’s new in Spring Batch 4.3? • New features • Performance improvements • Dependencies updates • Deprecations and technical debt • What’s next? • Q+A
  5. 5. Current status (Sep 2020)
  6. 6. History of Spring Batch (1/2) 6 • Step scope • Chunk-oriented processing • Remote chunking/partitioning • Java 5 • Spring Framework 3 v2.0 Apr 11, 2009 • Initial APIs • Item-oriented processing • XML configuration • Java 1.4 • Spring Framework 2.5 • Job scope • JSR-352 support • SQLite support • Spring Batch Integration • Spring Boot support • Builders for readers • Builders for writers • Java 8 • Spring Framework 5 v3.0 May 22, 2014 v1.0 Mar 28, 2008 v4.0 Dec 1, 2017
  7. 7. History of Spring Batch (2/2) 7 • @SpringBatchTest annotation • @EnableBatchIntegration annotation • JSR 380 Bean Validation support • Json reader/writer • JSR 305 support v4.1 Oct 29, 2018 • Apache Avro support • Apache Kafka support • Micrometer support • Performance improvements • Java 14 records support • GraalVM support • Kafka support refinement • Performance improvements • Deprecations • Builders for readers • Builders for writers • Java 8 • Spring Framework 5 v4.3 Oct XX, 2020 v4.2 Oct 02, 2019 v4.0 Dec 1, 2017 v5.0 XXX XX, 20XX • Spring Framework 6 • Java XX • TBD
  8. 8. History of Spring Batch (2/2) 8 • @SpringBatchTest annotation • @EnableBatchIntegration annotation • JSR 380 Bean Validation support • Json reader/writer • JSR 305 support v4.1 Oct 29, 2018 • Apache Avro support • Apache Kafka support • Micrometer support • Performance improvements • Java 14 records support • GraalVM support • Kafka support refinement • Performance improvements • Deprecations • Builders for readers • Builders for writers • Java 8 • Spring Framework 5 v4.3 Oct XX, 2020 v4.2 Oct 02, 2019 v4.0 Dec 1, 2017 v5.0 XXX XX, 20XX • Spring Framework 6 • Java XX • TBD
  9. 9. What’s new in v4.3? (currently in M2)
  10. 10. New features
  11. 11. Java 14 records support 11 • Use records as items in chunk-oriented steps • Leverage Spring Framework support for records id,name 1,foo 2,bar persons.csv FlatFileItemReader<Person> reader = new FlatFileItemReaderBuilder<Person>() .name("personItemReader") .resource(new FileSystemResource("persons.csv")) .delimited() .names("id", "name") .targetType(Person.class) .build(); public class Person { private int id; Private String name; // constructors // getters and setters // equals and hashcode // toString } public record Person(int id, String name) {}
  12. 12. GraalVM support 12 • Leverage Spring Framework support for GraalVM • Spring Batch jobs run correctly on GraalVM • Reflection • Proxies • etc Faster startup time + Better memory usage
  13. 13. Kafka support refinement 13 • Ability to start reading from a custom offset in a given partition • Ability to start reading from the offset stored in Kafka Image source: http://kafka.apache.org/intro v4.3: read from custom offsetv4.2: read from the beginning
  14. 14. Support for annotation-based job execution listeners 14 class MyListener implements JobExecutionListener { @Override public void beforeJob(JobExecution jobExecution) { // do something before job execution } @Override public void afterJob(JobExecution jobExecution) { // do something after job execution } } class MyListener { @BeforeJob public void beforeJob(JobExecution jobExecution) { // do something before job execution } @AfterJob public void afterJob(JobExecution jobExecution) { // do something after job execution } } v4.2: interface-based listeners v4.3: interface-based + annotation-based listeners
  15. 15. Add SynchronizedItemStreamWriter 15 @Bean public StaxEventItemWriter<Person> itemWriter() { Jaxb2Marshaller marchaller = new Jaxb2Marshaller(); marchaller.setClassesToBeBound(Person.class); return new StaxEventItemWriterBuilder<Person>() .name("personItemWriter") .resource(new FileSystemResource("persons.xml")) .marshaller(marchaller) .rootTagName("persons") .build(); } Expected output • Some item writers are not safe to use in a multi-threaded step (StaxEventItemWriter, JsonItemWriter, etc) <persons> <person> <id>1</id> <name>foo</name> </person> <person> <id>2</id> <name>foo</name> </person> </persons> Actual output <persons> <name>foo</name> <person> <id>1</id> </person> <id>2<person> <name>foo</name> </id> </person> </persons> Item writer bean definition
  16. 16. Add SynchronizedItemStreamWriter 16 @Bean public SynchronizedItemStreamWriter<Person> itemWriter() { Jaxb2Marshaller marchaller = new Jaxb2Marshaller(); marchaller.setClassesToBeBound(Person.class); StaxEventItemWriter<Person> personWriter = new StaxEventItemWriterBuilder<Person>() .name("personItemWriter") .resource(new FileSystemResource("persons.xml")) .marshaller(marchaller) .rootTagName("persons") .build(); SynchronizedItemStreamWriter<Person> synchronizedPersonWriter = new SynchronizedItemStreamWriter<>(); synchronizedPersonWriter.setDelegate(personWriter); return synchronizedPersonWriter; } • Some item writers are not safe to use in a multi-threaded step (StaxEventItemWriter, JsonItemWriter, etc) => SynchronizedItemStreamWriter to the rescue!
  17. 17. Other new features 17 • Add JpaNamedQueryProvider • Add encoding parameter in StaxEventItemReader • Micrometer support improvements • Add DataFieldMaxValueJobParametersIncrementer • etc https://spring.io/blog/2020/08/13/spring-batch-4-3-0-m2-is-out https://spring.io/blog/2020/06/26/spring-batch-4-3-0-m1-is-released-now
  18. 18. Performance improvements
  19. 19. for(item in chunk) { mongoTemplate.save(item); } v4.2 (pseudo code) BulkOperations bulkOperations = mongoTemplate.bulkOps(); for(item in chunk) { bulkOperation.add(item); } bulkOperations.execute(); v4.3 (pseudo code) Use bulk operations in MongoItemWriter 19 https://github.com/benas/spring-batch-lab/tree/master/issues/gh3713
  20. 20. for(item in chunk) { repository.save(item); } v4.2 (pseudo code) repository.saveAll(items); v4.3 (pseudo code) Use bulk operations in RepositoryItemWriter 20 https://github.com/benas/spring-batch-lab/tree/master/issues/gh3720
  21. 21. int getStepExecutionCount(JobInstance jobInstance, String stepName) { int count = 0; List<JobExecution> jobExecutions = findJobExecutions(jobInstance); for (JobExecution jobExecution : jobExecutions) { List<StepExecution> stepExecutions = findStepExecutions(jobExecution); count += countStepExecutions(stepExecutions, stepName); } return count; } v4.2 (pseudo code) v4.3 (pseudo code) int getStepExecutionCount(JobInstance jobInstance, String stepName) { String query = “select count(id) from ... where ...”; return jdbcTemplate.queryForObject( query, new Object[] {jobInstance, stepName}, Integer.class); } Job/Step start time improvement 21 https://github.com/benas/spring-batch-lab/tree/master/issues/gh3657
  22. 22. Dependencies updates
  23. 23. Dependencies updates 23 • Spring Framework 5.3 • Spring Integration 5.4 • Spring Data 2020.0 • Spring AMQP 2.3.0 • Spring for Apache Kafka 2.6.0 • Micrometer 1.5.3
  24. 24. Deprecations and technical debt
  25. 25. Deprecations and technical debt 25 • API deprecation in preparation for v5 • Improve build process • Improve test infrastructure • Docker-based tests with testcontainers • Junit 5 • Improve test coverage • etc
  26. 26. What’s next?
  27. 27. Release plan for 2020+ 27 • Spring Batch 4.3.0-RC1 (Sep 16, 2020) and 4.3.0 (Oct, 2020) • Bug fix releases for Spring Batch v4.2.x (Same EOL as Spring Boot v2.3) • Bug fix releases for Spring Batch v4.3.x (Same EOL as Spring Boot v2.4) • Spring Batch v5 (Spring Framework v6)
  28. 28. Thank you! Q+A springone.slack.com: #session-whats-new-in-spring-batch #springone@springon e

SpringOne 2020 What’s New in Spring Batch? Mahmoud Ben Hassine, Principal Software Engineer at VMware

Views

Total views

246

On Slideshare

0

From embeds

0

Number of embeds

15

Actions

Downloads

19

Shares

0

Comments

0

Likes

0

×