Exposición s

1. Spring Batch Groovy & Gradle integrationSpring Batch Groovy & Gradle integration
Because we like it…Because we like it…
because we have fun with itbecause we have fun with it

2. About me:About me:
Born with C and Java
Github:
http://github.com/amr390
(mostly drops, and tests)
Working with Java and Javascript,
take any opportunity to use
groovy and python
(I love for both of them)
Grown up with Java
since 2000
Twitter: 390amr
(not much into it)

3. Spring Batch. Groovy and GradleSpring Batch. Groovy and Gradle
What is Spring BatchWhat is Spring Batch
Why integrating spring-batch with Groovy and GradleWhy integrating spring-batch with Groovy and Gradle
RequirementsRequirements
➢ DependenciesDependencies
➢ Gradle tasksGradle tasks
What is Spring BatchWhat is Spring Batch
➢Configuring Jobs, Tasks, Steps and stufConfiguring Jobs, Tasks, Steps and stuf
➢Running and Handling JobsRunning and Handling Jobs
Improving Spring Batch performanceImproving Spring Batch performance
➢Multi-threadingMulti-threading
➢Master-Slave approachMaster-Slave approach
➢Parallelizing Step executionsParallelizing Step executions

4. What is Spring Batch I
Batch Framework
Technical services
Functionality to process large volumes of records
ItemReader, ItemWriter, ItemProcessor
Stateless or Stateful step executions
High-volume
High-performance
High-scalability
Trough optimization
Partitioning techniques

5. What is Spring Batch IIWhat is Spring Batch II

6. What is Spring Batch IIIWhat is Spring Batch III

7. What is Spring Batch IVWhat is Spring Batch IV
STEP:
➔ Custom tasklet
➔ Restartable
➔ Skippable
➔ Retry
➔ Rollback
➔ Listeners Before/After
✗ Reader
✗ Processor
✗ Writer
✗ Chunk
✗ Step
➔ Step flow (on status)

8. What is Spring Batch: exampleWhat is Spring Batch: example

9. Philae Lander dataPhilae Lander data
Philae Lander is a spacecraft that sends analysis data from comet 67P/Churyumov–
Gerasimenko (https://en.wikipedia.org/wiki/Philae_(spacecraft))
Sends tones data in files encrypted in Klingon or Murciano.
Files are zipped, and organized with a naming rule of file1, file2 etc ...

10. It is always fun using groovy
Can be nicely coupled to other platforms Grifon, Grails...
Power of Gradle
Homogeneous Architecture
Why integrating Spring-batch withWhy integrating Spring-batch with
groovy and Gradlegroovy and Gradle

11. Plug insPlug insRepositoriesRepositories
DependenciesDependencies
TasksTasks

12. Requeriments IIRequeriments II
Repositories
Plugins

13. Dependencies IIIDependencies III
Dependencies

14. Requirements IVRequirements IV
Gradle tasks
MainClassName: Sets the project main class (because it is a global definition)
run: sets the required arguments for run task, in this case the job configuration class and the
Job to be launched.

15. @Configuration
@EnableBatchProcessing
Injects
a JobRepository
a JobLauncher
a JobRegistry
a PlatformTransactionManager
a JobBuilderFactory
a StepBuilderFactory
Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Job definition configurations, steps, tasklets and stuf

16. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Running and Handling Jobs
Spring batch admin

17. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Running and Handling Jobs
Job Launchers
org.springframework.batch.core.launch
Interface JobLauncher
All Known Implementing Classes:
SimpleJobLauncher
JobExecution run(Job job,
JobParameters jobParameters)
throws JobExecutionAlreadyRunningException,
JobRestartException,
JobInstanceAlreadyCompleteException,
JobParametersInvalidException

18. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Running and Handling Jobs
Job Launchers
org.springframework.batch.core.launch.support
Class CommandLineJobRunner

19. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Performance improvement
There different approached in order to improve the job execution performance
Multi-threading:
Using a
ThreadPoolTaskExecutor
Remote chunking:
Master executes read
phase of step slaves are
distributed in nodes and
execute processing and
writing from a
chunkProvider.
Partitioning:
Clone Job
execution step
and run steps in
parallel
Remote
Partitioning:
Instead of running
the steps clone
locally steps are
distributed
Parallel steps:
step execution is splitted to
execute several steps in
parallel flows.

20. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Multi
threading

21. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Multi
threading
Using task namespace
<task:executor id="taskExecutor"
pool-size="10" />
By java configuration method:
ThreadPoolTaskExecutor taskExecutor
...
Steps must be thread safe

22. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Master-Slave
approach

23. PartitionHandler: Interface defining the responsabilities of
controlling the execution of partitioned step
Partitioner: Interface to determine the way of creating
partitions (key-range, file-chunks...)
Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Master-Slave
approach

24. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Parallelizing Step executions

25. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
Parallelizing Step executions
@Bean
public SimpleFlow mainFlow() {
SimpleFlow splitFlow = new FlowBuilder<SimpleFlow>("Split Flow")
.split(new SimpleAsyncTaskExecutor())
.add(flow2(), flow3())
.build();
return new FlowBuilder<SimpleFlow>("Main Flow")
.start(flow1())
.next(splitFlow)
.end();
}
<batch:step id="step1" next="">...</batch:step>
<batch:split id="splitSteps" next="step3">
<batch:flow> <batch:step … /></batch:flow>
<batch:flow> <batch:step … /></batch:flow>
</batch:split>
<batch:step id="step3">
<batch:tasklet ref="tasklet" />
</batch:step>

26. Demo Github: https://github.com/amr390/groovy-springbatch-demo
Spring batch reference page: http://docs.spring.io/spring-batch/trunk/reference/
Spring batch in action: http://www.manning.com/templier/
Spring batch horizontal-vertical scaling:
http://www.ontheserverside.com//blog/2014/07/23/horizontal-and-vertical-scaling-strategies-for-batch-applications
Spring batch admin: http://docs.spring.io/spring-batch-admin/
Spring Integration reference page: http://docs.spring.io/spring-integration/refer
Groovy: http://www.groovy-lang.org/documentation.html
Gradle: https://docs.gradle.org/current/release-notes
Spring Batch Groovy & GradleSpring Batch Groovy & Gradle
integrationintegration
References