Gone are the days where you can wait until tonight to run your scheduled job. Your customer has just stepped off the plane and wants to know now if they've achieved Preferred Status. And what exactly is "tonight" anyway when you're running worldwide 24 hours a day?
If you've ever: a) iterated over a list of appointments to send reminder texts, b) aggregated usage data to calculate a monthly fee, or c) collected information from various sources to create a daily digest email, come see how you can do so without running it in the dead of night when you hope no one is watching.
Code: https://github.com/andreasohlund/DeathToTheBatchJob
Blog post: http://particular.net/blog/death-to-the-batch-job
N+1 problem, but hey, it’s a batch job, the DBA won’t notice
Examples:
Generating reports/statements
Interest calculations
UI Tests
For each appointment where date > today and < 3 days from now
Send reminder
Don’t forget to check for duplicates
Others?
Why do we have batch jobs?
- Run at night when computers aren’t busy
What if it fails?
What if you become successful?
No night = no non-peak hours
More customers = longer batch processing
Just landed and checks their elite status
The person just made a purchase and wants to make another
- You’re losing business by making them wait
If you aren’t real-time, someone else will be
Business has been trained in batch jobs. They expect them.
“We’re adding a preferred customer program. Can you update everyone’s status each night?”
“Will the user need to see their current status? Yes? Is it okay if they see it the next day or right away?” “Immediately, obviously”
We’ve trained the business to think in this way
“We want a process that converts customers to preferred every night”
“Will they be able to see their status online”
“Is it okay if they don’t see their updated status right away?”
Instead of focusing on the past, let’s predict the future
Jay orders something for $100. That counts towards his preferred status now. At what point in the future does it no longer count?
Easy! 365 days!
Let’s keep a running total of his purchases and set a reminder to ourselves to deduct $100 from it in 365 days.
.NET timer won’t cut it => durable timeout (i.e. alarm clock)
Scheduled message delivery is natively supported by some queuing systems (e.g. ActiveMQ and Azure ServiceBus) but not others (Rabbit and MSMQ). A good service bus technology will support durable timeouts across the board
Requires a shift in thinking toward messaging
Sagas => coined in the DB community when they had a need for long-running business processes
Two things to handle: when a customer becomes preferred, when they lose the privilege
Locking for a few months = not good
Sagas => compensating action to accommodate failure
With Sagas, time becomes a first-class citizen and it opens new possibilities
This isn’t robust: Will keep spamming user even if already preferred
Will mark as regular even if they weren’t preferred to start