Spring batch

768 views
631 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
768
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Spring batch

  1. 1. Spring Batch
  2. 2. Domain• Benefit Management Application– Ledger information
  3. 3. Goal• Input CSV• The header  System ID,Account ID,PlanNumber,Payee ID,Check Type,Check Date,TransactionCreate Date,Check Number,Benefit Type,Pay Code,PaySource 1,Pay Source 2,Pay Source 3,Pay Source 4,PaySource 5,Pay Source 6,Pay Source 7,Pay Source 8,PaySource 9,Deduction 1,Deduction 2,Deduction3,Deduction 4,Deduction 5,Deduction 6,Deduction7,Deduction 8,Deduction 9,Deduction 10,PayableID,Customer ID,Category of Distribution,Customer LastName,Customer First Name,Customer Middle Initial,…etc…
  4. 4. Output• 600+ bytes to send to Paying Agent• payCode => %-1.1s,• paySource5 => %011d,• taxWithholdingElection => %-1.1s,• taxableMaritalStatusElectionCode => %-1.1s,• deduction10 => %011d,• systemID => %-1.1s,• paySource1 => %011d,• checkType => %-1.1s,• taxableAllowanceElection => %-2.2s,• financialInstitutionCity => %-16.16s,• checkCity => %-16.16s,• paySource9 => %011d,• planNumber => %-6.6s,• payeeDateofDeath => %-8.8s,• aCHRoutingNumber => %-9.9s,• check3rdStreetAddress => %-32.32s,• deduction1 => %011d,• transactionCreateDate => %-8.8s,• deduction2 => %011d,• deduction5 => %011d,• checkNumber => %-10.10s,• checkDate => %-8.8s,• financialInstitutionPostalCode => %-9.9s,• deduction8 => %011d,• categoryofDistribution => %-2.2s,• customerFirstName => %-12.12s,• check1stStreetAddress => %-32.32s,
  5. 5. Tools• Spring Source Tools IDE /Eclipse• Apache Maven• Perl• Bash  Cygwin (Windows)• Spring Batch legacy code that currentlysupports 1.6 spec but needs to upgrade to 1.9spec
  6. 6. Spring Batch Project• Start with the simplest hello-world project• Build more complexity on top
  7. 7. Spring Batch Application Context• <?xml version="1.0" encoding="UTF-8"?>• <beans xmlns="http://www.springframework.org/schema/beans"• xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"• xsi:schemaLocation="http://www.springframework.org/schema/beans• http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">• <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />• <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">• <property name="transactionManager" ref="transactionManager"/>• </bean>• <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">• <property name="jobRepository" ref="jobRepository" />• </bean>•• <!--• <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>• -->• </beans>
  8. 8. Maven POM• <name>Spring Batch Hello World</name>• <properties>• <spring.version>2.5.4</spring.version>• <spring-batch.version>2.0.3.RELEASE</spring-batch.version>• </properties>• <dependencies>• <dependency>• <groupId>org.springframework.batch</groupId>• <artifactId>spring-batch-core</artifactId>• <version>${spring-batch.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-aop</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-tx</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-core</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-beans</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-context</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>org.springframework.test</artifactId>• <version>3.0.4.RELEASE</version>• </dependency>
  9. 9. Domain Class
  10. 10. Getter and setter(s)…
  11. 11. Tokenizer• <!-- Tokenizer - Converts a delimited stringinto a Set of Fields -->• <bean name="defaultTokenizer"class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"/>
  12. 12. Field Set Mapper• <bean name="pbtItemFieldSetMapper"class="aspire.mapper.PBTItemFieldSetMapper" />
  13. 13. Line Mapper• <bean name="pbtItemLineMapper"class="org.springframework.batch.item.file.mapping.DefaultLineMapper">• <property name="lineTokenizer"ref="defaultTokenizer"/>• <property name="fieldSetMapper"ref="pbtItemFieldSetMapper"/>• </bean>
  14. 14. Flat File Item Reader• <bean name="pbtItemReader"class="org.springframework.batch.item.file.FlatFileItemReader">• <property name="lineMapper"ref="pbtItemLineMapper"/>• <!-- use spring integrations for the following, but fornow filename is hard coded -->• <property name="resource" value="classpath:pbt-input-4.txt"/>• </bean>
  15. 15. Input File(s)
  16. 16. Item Processor• <bean name="pbtItemProcessor"class="aspire.processor.PBTItemProcessor">• </bean>
  17. 17. Item Writer• <bean id="pbtItemWriter"• class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">• <property name="resource" value="file:target/output-pbt-4.det"/>• <property name="lineAggregator">• <bean• class="org.springframework.batch.item.file.transform.FormatterLineAggregator">• <property name="fieldExtractor">• <bean• class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">• <property name="names"• value="systemID,accountID,planNumber,payeeID,checkType,checkDate,transactionCreateDate,checkNumber,benefitType,payCode,paySource1,paySource2,paySource3,paySource4,paySource5,paySource6,paySource7,paySource8,paySource9,deduction1,deduction2,deduction3,deduction4,deduction5,deduction6,deduction7,deduction8,……,financialInstitutionfinancialInstitutionAccountNumber,financialInstitutionAccountType,financialInstitutionStreet1,financialInstitutionCity,financialInstitutionStateCode,financialInstitutionProvince,financialInstitutionPostalCode,financialInstitutionIRSCountryCode,taxWithholdingElection,taxableMaritalStatusElectionCode,taxableAllowanceElection" />• </bean>• </property>• <!--• %[argument_index$][flags][width][.precision]conversion• -->• <property name="format" value="%-1.1s%-5.5s%-6.6s%-9.9s%-1.1s%-8.8s%-8.8s%-10.10s%-3.3s%-1.1s%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%-10.10s%-30.30s%-2.2s%-20.20s%-12.12s%-1.1s%-8.8s%-8.8s%-32.32s%-32.32s%-32.32s%-16.16s%-2.2s%-9.9s%-2.2s%-16.16s%-1.1s%-9.9s%-17.17s%-32.32s%-32.32s%-2.2s%-32.32s%-16.16s%-2.2s%-16.16s%-9.9s%-2.2s%-1.1s%-1.1s%-2.2s" />• </bean>• </property>• <property name="footerCallback" ref="pbtfootercallback" />• </bean>• <bean id="pbtfootercallback" class="aspire.core.PBTFooterCallback">• <property name="delegate" ref="pbtItemWriter"></property>• </bean>
  18. 18. Field Set Mapper…
  19. 19. Batch Job• <batch:job id="pbgcPBTIncomingJob">• <!-- batch:step id="step1" next="step2">• <batch:tasklet ref="helloWorldTasklet" />• </batch:step-->• <batch:step id="step4">• <batch:tasklet>• <batch:chunk reader="pbtItemReader" processor="pbtItemProcessor"writer="pbtfootercallback" commit-interval="1">• <batch:streams>• <batch:stream ref="pbtItemReader" />• <batch:stream ref="pbtItemWriter"/>• </batch:streams>• </batch:chunk>• <batch:listeners>• <batch:listener ref="pbtfootercallback"/>• </batch:listeners>• </batch:tasklet>• </batch:step>• </batch:job>
  20. 20. Footer Callback
  21. 21. Footer callback …(2)
  22. 22. How to test/run• Command line– mvn exec….– jUnit test case
  23. 23. Junit Test Case• import org.springframework.batch.core.repository.JobRestartException;• import org.springframework.beans.factory.annotation.Autowired;• import org.springframework.beans.factory.annotation.Qualifier;• import org.springframework.test.context.ContextConfiguration;• import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;• import org.springframework.util.StopWatch;• @ContextConfiguration("classpath:simpleJob.xml")• @RunWith(SpringJUnit4ClassRunner.class)• public class HelloTestCase {• private static Log logger = LogFactory.getLog(HelloTestCase.class);• @Autowired• @Qualifier("pbgcPBTIncomingJob")• private Job job;• @Autowired• private JobLauncher jobLauncher;• @Test• public void test() {• //fail("Not yet implemented");• JobExecution execution = null;• try {• logger.debug("hello world");• logger.debug("Job name: " + job.getName());• execution = jobLauncher.run(job, new JobParameters());• //execution = jobLauncher.run(job,• //new JobParametersBuilder()• //.addString("inputFile", "ack-test-new.xlsx")• //.addString("outputFile", "1002").toJobParameters());• } catch (JobExecutionAlreadyRunningException e)
  24. 24. jUnit Eclipse Plug-in
  25. 25. Sample output…• Abbreviated Input• 12:47:43,019 DEBUG main PBTItemFieldSetMapper:19 - [1, AGENCY1, ….., …….., 61, RA3, R, 45677,7, 6, 5, 4, 3, 2, 1, 0, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1,…….., 7, SMITH, JOE, F, 123456, *, 742 ANY STREET,, *,*, ANY CITY, OH, ……, US, *, A, ………….., *, *, *, *, *, *, *, *, *, N, *, 0]• Abbreviated Output• 12:47:43,019 DEBUG main PBTItemProcessor:21 - after processing [SystemID = 1 AccountID =AGENCY1 PlanNumber ……… BenefitType = RA3 PayCode = R PaySource1 = 45677 PaySource2 = 7PaySource3 = 6 PaySource4 = 5 PaySource5 = 4 PaySource6 = 3 PaySource7 = 2 PaySource8 = 1PaySource9 = 0 Deduction1 = 10 Deduction2 = 9 Deduction3 = 8 Deduction4 = 7 Deduction5 = 6Deduction6 = 5 Deduction7 = 4 Deduction8 = 3 Deduction9 = 2 Deduction10 = 1……….Check2ndStreetAddress = * Check3rdStreetAddress = ……CheckIRSCountryCode = USCheckProvince = * PaymentDestinationType = A ………… FinancialInstitutionPostalCode = *FinancialInstitutionIRSCountryCode = * TaxWithholdingElection = NTaxableMaritalStatusElectionCode = * TaxableAllowanceElection = 0]• 12:47:43,019 DEBUG main PBTFooterCallback:44 - write...callback...• 12:47:43,019 DEBUG main PBTFooterCallback:65 - 8• ….• 12:47:43,035 DEBUG main PBTFooterCallback:31 - FooterCallback....• 12:47:43,035 INFO main SimpleJobLauncher:111 - Job: [FlowJob: [name=pbgcPBTIncomingJob]]completed with the following parameters: [{}] and the following status: [COMPLETED]
  26. 26. Counting the bytes• Fires up cygwin• user@PC123456> /cygdrive/c/dev/spring-batch/batch-dir/Spring-Batch-Hello-World/target– $ head -n1 output-pbt-4.det | wc -c– 669
  27. 27. Be aware of newlines char• Bash/gnu wc utility• userID@PC123456 /cygdrive/c/dev/spring-batch/batch-dir/Spring-Batch-Hello-World/target– $ head -n1 output-pbt-4.det | perl -ne printsubstr($_,667,2); | od -c– 0000000 r n– 0000002• So technically we only 667 bytes instead of 669as reported by wc

×