Spring Batch Introduction Tadaya Tsuyukubo twitter: @ttddyy http://bit.ly/ttddyy_spring-batch-intro
long running process
onetime, daily, monthly, yearly, ...
"The lack of a standard, reusable batch architecture has resulted in the proliferation of many one-off, in-house solutions developed within client enterprise IT functions." - spring batch documentation
Accenture + SpringSource
Accenture : industry knowledge & experience
SpringSource : Tech, Spring programming model
Batch Process Infrastructure
transaction management, skip, repeat, job execution, etc.
why not hadoop or other grid frameworks?
framework vs runtime environment
- no need to have special environment - flexibility of storage and algorithm - reuse of existing java library embedded to existing app HDFS, HBase, MapReduce ... standalone process (possible to distribute) efficient to distribute process Light Weight Heavy Weight (doesn't mean good/bad)
Architecture decision, case by case
pre-process job to hadoop.
ex: transform log and push to hdfs. (flume?)
data & computation
small to medium : spring batch
very large : hadoop or grid frameworks
* "spring batch in action [MEAP] - manning"
retrieve input data from datasource(file, database, queue, etc.)