Death of the batch job - NServiceBus Sagas

Dennis van der Stelt
all your batch jobs are belong to us
Software Architect
http://dennis.bloggingabout.net/
dennis@bloggingabout.net
Particular Software engineer
death of the batch job
@dvdstelt
#nservicebus

NServiceBus
In Particular

NServiceBus
It’s not like WCF, which does RPC
But closes to WCF than to BizTalk

BUS TOPOLOGY

Messaging
What is it and why do I need it?

SpatialTemporalPlatform
coupling aspects

PLATFORM
Also known as ‘interoperability’
http, json, xml, xsd, etc…

TEMPORAL
Store Front End Shipping Service

TEMPORAL
Store Front End Shipping ServiceShipping Service

TEMPORAL
Store Front End Shipping ServiceOrder Queue

Messaging
 Reduces spatial coupling
 XML/JSON for platform coupling
 Asynchronous for temporal coupling

PERFORMANCE
RPC versus Messaging

NServiceBus Sagas
A pattern by relation database community

Sagas Recap
“What have you done” – Within Temptation

HANDLING MESSAGES
Behavior like normal message handlers
class MySaga : IHandleMessages<MyMessage>
{
public void Handle(MyMessage message)
{
…
}
}

STARTING SAGAS
Extends the default IHandleMessages<T>
class MySaga : IAmStartedByMessages<MyMessage>
{
{
…
}
}

STORING STATE
Extends the default IHandleMessages<T>
class MySaga : Saga<MySagaData>,
IAmStartedByMessages<MyMessage>
{
{
this.Saga.Data.MyStateProperty = Message.MyProperty;
}
}

CORRELATING MESSAGES TO SAGA INSTANCE
{
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper)
{
mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);
}
{
}
}

REQUESTING TIMEOUTS
Reminders to the Saga itself
IAmStartedByMessages<MyMessage>,
IHandleTimeouts<MyTimeout>
{
{
RequestTimeout<MyTimeout>(TimeSpan.FromSeconds(10));
}
public void Timeout(MyTimeout state)
{
…
}
}

SENDING MESSAGES
Reminders to the Saga itself
{
{
this.Bus.Send(new MyCommand());
this.Bus.Publish(new MyEvent());
}
}

SAGA STATE
Memento
class MySagaData : IContainSagaData
{
public virtual Guid Id { get; set; }
public virtual string Originator { get; set; }
public virtual string OriginalMessageId { get; set; }
[Unique]
public virtual Guid MySagaId { get; set; }
}
ALTER TABLE [dbo].[MySagaData]
ADD UNIQUE NONCLUSTERED
([MySagaId] ASC) ON [PRIMARY]

SAGA STATE
Memento
{
[Unique]
public virtual IList<Product> Products { get; set; }
}
public class Product
{
public virtual Guid ProductId { get; set; }
}

SAGA STATE
Memento
{
[Unique]
public virtual IList<Product> Products { get; set; }
}
public class Product
{
public virtual Guid ProductId MyUniqueId { get; set; }
}

Death to the batch job
Because we don’t want to depend on operations ;-)

scheduled tasks
Your CEO had insomnia and was using the system in the
middle of the night. The batch job failed somewhere in
the middle of updating 74 million records…
You need to figure out which row it failed on (how?),
why it failed, correct the issue, then start the job again
from where it left off, because if you have to start from
the beginning, it won't get done before peak hours in
the morning.
because that sounds better than batch job

BATCH JOB
All customers that ordered $5000 in the last year, get preferred status
DateTime cutoff = DateTime.Today.AddDays(-365);
foreach(var customer in customers)
{
var orderTotal = customer.Orders
.Where(o => o.OrderDate > cutoff)
.Sum(order => order.OrderValue);
customer.Prefered = orderTotal > 5000;
}

Tromsø, Norway

what if
we can see things before they happen?

customer preferred status
Dev: Let's say Steve orders something
for $100. At what point does that
amount no longer count toward Steve's
preferred status?
BA: That's easy, after 365 days!

-$300-$100
+$300
DURABLE TIMEOUTS
Our way to predict the future
2015 2016
+$100

DURABLE TIMEOUTS
public void Handle(OrderPlaced message)
{
if(this.Data.CustomerId == 0)
this.Data.CustomerId = message.CustomerId;
this.Data.RunningTotal += message.Amount;
this.RequestTimeout<OrderExpired>(TimeSpan.FromDays(365),
timeout => timeout.Amount = message.Amount);
CheckForPreferredStatusChange();
}
public void Handle(OrderExpired message)
{
this.Data.RunningTotal -= message.Amount;
CheckForPreferredStatusChange();
}

DURABLE TIMEOUTS
private void CheckForPreferredStatusChange()
{
if(this.Data.PreferredStatus == false && this.Data.RunningTotal >= 5000)
{
this.Bus.Publish<CustomerHasBecomePreferred>(
evt => evt.CustomerId = this.Data.CustomerId);
}
else if(this.Data.PreferredStatus == true && this.Data.RunningTotal < 5000)
{
this.Bus.Publish<CustomerHasBecomeNonPreferred(
evt => evt.CustomerId = this.Data.CustomerId);
}
}

Best Practices
The silver bullets?

Rule #1 : Don’t query data
Never ever, ever, ever query data
- From the saga to another data source
- Owned by saga with a 3rd party tool

Business eventsTimeoutsStarting sagas
out of order

ASYNCHRONOUS COMMUNICATION
Order of message delivery
OrderAccepted
OrderAccepted
OrderBilled

ASYNCHRONOUS COMMUNICATION
Order of message delivery
OrderAccepted
OrderAccepted
OrderBilled
OrderCancelled
How to solve?

STARTING SAGAS
What business events can start a Saga?
IAmStartedByMessages<YourMessage>,
IAmStartedByMessages<AnotherMesssage>
{
{
…
if (VerifyState())
MarkAsComplete();
}
}

404 ON SAGAS
What if the Saga was already MarkedAsComplete
public class SagaNotFoundHandler : IHandleSagaNotFound
{
IBus bus;
public SagaNotFoundHandler(IBus bus)
{
this.bus = bus;
}
public void Handle(object message)
{
bus.Reply(new SagaDisappearedMessage());
}
}

AUTO CORRELATION
Avoids the need for mapping responses
IHandleMessages<Response>
{
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper)
{
mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);
// Mapping for Respose not needed!
}
}
public class RequestHandler : IHandleMessages<Request> {
public void Handle(Request message) {
Bus.Reply(new Response());
}
}

AUTO CORRELATION
Avoids the need for mapping responses
IHandleMessages<MyTimeout>
{
public void Timeout(MyTimeout state)
{
if (state.RasiedAtUtc == …
}
}
public class MyTimeout
{
public DateTime RaisedAtUtc { get; set; } = DateTime.UtcNow;
}
Can be a POCO
No `virtual` as it’s serialized to string format

CONCURRENCY
[LockMode(LockModes.Read)]
class ShoppingCartSagaData : IContainSagaData
{
…
[RowVersion]
public virtual int RowVersion { get; set; }
[Unique]
public virtual Guid CartId { get; set; }
}

CONCURRENCY
DELETE FROM ShoppingCartSagaData
WHERE Id = '1C669378-3508-4A20-8EFD-A5C60168893E'
AND Originator is null
AND OriginalMessageId = 'd6436b7-eb17-4de5-8b92-a5c60168888c'
AND CartId = '49784AF0-0854-4FF9-B8CD-E57CC8EE6CFC‘
AND …
AND …
AND …
DELETE FROM ShoppingCartSagaData
WHERE Id = '1C669378-3508-4A20-8EFD-A5C60168893E'
AND RowVersion = 1

Please…
stay in touch!
http://dennis.bloggingabout.net
dennis@bloggingabout.net

Death of the batch job - NServiceBus Sagas

Recommended

Recommended

More Related Content

Similar to Death of the batch job - NServiceBus Sagas

Similar to Death of the batch job - NServiceBus Sagas (9)

More from Dennis van der Stelt

More from Dennis van der Stelt (10)

Recently uploaded

Recently uploaded (20)

Death of the batch job - NServiceBus Sagas