The document discusses batch processing concepts in Java including Spring Batch and Java Batch. It covers batch domain language for defining batch jobs, chunk versus batchlet implementations of steps, partitioning steps for parallel processing, checkpointing for long-running jobs, listeners for job lifecycle events, and integrating batch jobs within Java applications and servers. Examples are provided of defining batch jobs using XML configuration and implementing readers, writers, processors, and other batch components in Java code.

1. Globalcode
–
Open4education
Batch Processing:
Processamento em Lotes no
Mundo Corporativo
Rodrigo Cândido da Silva
@rcandidosilva

2. Globalcode
–
Open4education
Agenda
! Conceitos
! Batch Domain Language
! Chunk vs. Batchlet
! Partitioned Step
! Flow, Split e Decision
! Listeners e Exceptions
! Execution
! Integration

3. Globalcode
–
Open4education
Porque Batch?
! É muito comum em aplicações
! Várias soluções “personalizadas”
! Produtos começaram a surgir
! Spring Batch
! WebSphere Compute Grid
! Ideal para sistemas ETL

4. Globalcode
–
Open4education
Batch API
! Chunk / Batchlet
! Implementação de um Step
! Contexts
! Job e Step at runtime
! Persistência de metadados
! Listeners
! Callback lifecycle events
! Partitioning
! Processamento paralelo

5. Globalcode
–
Open4education
Batch Domain Language
! Batch job XML definition
! Descreve os steps como um agrupamento de
batch artifacts

6. Globalcode
–
Open4education
Batch Domain Language
<job id="adressJob” version="1.0">
<listeners>
<listener ref="MyJobListener"/>
</listeners>
<step id="buildingData" next="adressStep">
<batchlet ref="GenerateDataBatchlet" />
</step>
<step id="adressStep">
<listeners>
<listener ref="MyStepListener"/>
</listeners>
<chunk item-count="10">
<reader ref="adressItemReader" />
<processor ref="adressItemProcessor" />
<writer ref="adressItemWriter" />
</chunk>
</step>
</job>

7. Globalcode
–
Open4education
Chunk vs. Batchlet
! Implementam step dentro do job
! Chunk
! Encapsula padrão ETL
! Single Reader, Processor e Writer
! Executado por pedaços dados (chunk)
! Chunk output é escrito unitariamente
! Batchlet
! Promove a execução de um único e simples processo
! Executado até o fim produzindo um código de retorno

8. Globalcode
–
Open4education
Chunk vs. Batchlet
! Chunk ! Batchlet

9. Globalcode
–
Open4education
Batchlet
@Named
public class MyBatchlet {
@Process
public String process() throws Exception {..}
@Stop
public void stopMe() throws Exception {..}
}
//For a job
<step id=”step1”>
<batchlet ref=“MyBatchlet”/>
</step>
public class MyBatchlet implements Batchlet {..}

10. Globalcode
–
Open4education
Chunk
! Step Job
//For a job
<step id=”sendStatements”>
<chunk reader=”accountReader”
processor=”accountProcessor”
writer=”emailWriter”
item-count=”10” />
</step>
@Named(“accountReader")
...implements ItemReader... {
public Account readItem() {
// read account using JPA
@Named(“accountProcessor")
...implements ItemProcessor... {
public Statement processItems(Account account) {
// read Account, return Statement
@Named(“emailWriter")
...implements ItemWriter... {
public void writeItems(List<Statements> statements) {
// use JavaMail to send email

11. Globalcode
–
Open4education
Chunk
public interface ItemReader<T> {
public void open(Externalizable checkpoint);
public T readItem();
public Externalizable checkpointInfo();
public void close();
}
public interface ItemWriter<T> {
public void open(Externalizable checkpoint);
public void writeItems(List<T> items);
public Externalizable checkpointInfo();
public void close();
}
public interface ItemProcessor<T, R> {
public R processItem(T item);
}

12. Globalcode
–
Open4education
Checkpoint
! Para tarefas intensivas, longo período de tempo
! Checkpoint/restart é um bastante utilizado
! Basicamente…
! Armazena estado do ItemReader, ItemWriter
! Métodos chamados
! reader.checkpointInfo()!
! writer.checkpointInfo()
public interface ItemReader<T> {
public void open(Externalizable checkpoint);
public Externalizable checkpointInfo();
}
public interface ItemWriter<T> {
public void open(Externalizable checkpoint);
public Externalizable checkpointInfo();
}
<chunk checkpoint-policy="item"
commit-interval="10" item-count="10">

13. Globalcode
–
Open4education
Partitioned Step
! Step pode rodar particionado
! [N] instâncias do mesmo step em [N] Threads
! Uma partição por Thread
<step id="step1" >
<chunk ...>
<partition>
<plan partitions=“10" threads="2"/>
<reducer .../>
</partition>
</chunk>
</step>

14. Globalcode
–
Open4education
Partitioned Step
! Partition Mapper
! Decide dinamicamente o número de partições
! Partition Plan
! Partition Reducer
! Demarca a unidade lógica de trabalho
! Partition Collector
! Enviar resultados de processamento das partições
! Partition Analyzer
! Ponto de controle e análise dos resultados enviados

15. Globalcode
–
Open4education
Flow, Split e Decision
Flow
Step I
Task
Step II
Chunk
ItemReader
ItemWriter
Step III
Chunk
Deci-
sion
ItemReader
ItemWriter
Step IV
Chunk
ItemReader
ItemWriter
EndStart
ItemProces
sor
ItemProces
sor
ItemProces
sor

16. Globalcode
–
Open4education
Flow
! Define a lista de steps a ser executado (unitário)
<flow id=”flow-1" next=“{flow, step, decision}-id” >
<step id=“flow_1_step_1”>
</step>
<step id=“flow_1_step_2”>
</step>
</flow>

17. Globalcode
–
Open4education
Split
! Define a lista de flows a serem executados (paralelo)
! Coletores e analisadores para monitoramento
<split …>
<flow …./> <!– each flow runs on a separate thread -->
<flow …./>
</split>

18. Globalcode
–
Open4education
Decision
! Possibilita a implementação de workflows

19. Globalcode
–
Open4education
Decision
@Named
public class Decider {
public String decide(BatchContext context)
throws Exception {
String exit = context.getExitStatus();
if (“SUCCESS”.equals(exit)) {
return “SKIP”;
}
return exit;
}
}
//For a job
<step id=”step1”>
<decision id=“decision1” ref=“Decider”>
<next on=“SKIP” to=“step3”/>
<next on=“*” to=“step2”/>
</decision>
</step>
<step id=“step2” next=“step3”/>
<step id=“step3”/>

20. Globalcode
–
Open4education
Lifecycle
STOPPED
20
STARTING STARTED COMPLETED
FAILED
STOPPING
ABANDONED
stop()
start()
abandon()
abandon()
abandon()
restart()
restart()

21. Globalcode
–
Open4education
Listeners
! Step
! StepListener, ItemReadListener, ItemProcessListener, ItemWriterListener, ChunkListener,
RetryReadListener, RetryProcessListener, RetryWriteListener, SkipReadListener,
SkipProcessListener, SkipWriteListener
! Job
! JobListener
@Named
public class StepListener {
@BatchContext
StepContext context;
@BeforeStep
public void beforeStep() {..}
@AfterStep
public void afterStep() {..}
}
//For a job
<step id=”step1”>
<listeners>
<listener
ref=“StepListener”/>
</listeners>
</step>

22. Globalcode
–
Open4education
Exceptions
<job id=...>
...
<chunk skip-limit=”5” retry-limit=”5”>
<skippable-exception-classes>
<include class="java.lang.Exception"/>
<exclude class="java.io.FileNotFoundException"/>
</skippable-exception-classes>
<retryable-exception-classes>
...
</retryable-exception-classes>
<no-rollback-exception-classes>
...
</no-rollback-exception-classes>
</chunk>
...
</job>

23. Globalcode
–
Open4education
JobOperator e Repository
! JobOperator
! Runtime interface para gerenciamento
! start, stop, restart
! JobRepository interface commands
! JobRepository
! Contém informações sobre os jobs
! Completos e em execução

24. Globalcode
–
Open4education
Execution
! JobInstance
! Representação lógica
de um job runtime
! JobExecution
! Tentativa de executar
um JobInstance
! StepExecution
! Tentativa de rodar um
step de um job

25. Globalcode
–
Open4education
Integration
! Suporte ao Java SE
! Application Server Runtime
! Suporte clustering, segurança, gerenciamento de
recursos
! Dependency Injection com CDI
! XML descriptors
! META-INF/batch-jobs/myJob.xml
! Empacotamento
! JAR, WAR, EJB

26. Globalcode
–
Open4education
DEMO
http://goo.gl/wst7M7

27. Globalcode
–
Open4education
Perguntas
?

28. Globalcode
–
Open4education
Referências
! https://jcp.org/en/jsr/detail?id=352
! https://java.net/projects/jbatch
! http://projects.spring.io/spring-batch/
! http://docs.oracle.com/javaee/7/tutorial/doc/batch-processing.htm
! http://www.oracle.com/technetwork/articles/java/batch-1965499.html
! https://github.com/javaee-samples/javaee7-samples/
! http://blog.arungupta.me/2014/07/schedule-javaee7-batch-jobs-
techtip36/
! http://www.planetjones.co.uk/blog/25-05-2013/introducing-jsr-352-java-
batch-ee-7.html

29. Globalcode
–
Open4education
Obrigado!
@rcandidosilva
rodrigocandido.me