Dennis van der Stelt
of distributed systems
Dennis van der Stelt
Software Architect
http://dennis.bloggingabout.net/
dennis@bloggingabout.net
NServiceBus Community Champ
PRINCIPLES
@dvdstelt
#sdnevent
Dennis van der Stelt
YOUR PRESENTER
Dennis van der Stelt
http://dennis.bloggingabout.net/
Software Architect
Professional Services
Provides coaching & training
About Dennis
Innovative software architect
Almost 20 years of experience in development of
distributed systems. Has a continuous drive to
learn and improve knowledge in different
architectural styles, including quality in software
development. Highly motivated to share his
knowledge via articles, presentations and his
blog.
Awards
NServiceBus Champ
Typemock MVP
Certified Trainer
• Presentations
• Software & Architecture audits
• SOA & Microservices advice
• Unit Testing training
• NServiceBus training
Presentations
October 15th at Blaak Selectie / Betabit
Reliable systems using messaging
Dennis van der Stelt
AGENDA
Dennis van der Stelt
A monolithic design is characterized by such tight coupling
among modules that they really have no independent existence.
monolithic
Dennis van der Stelt
MONOLITHIC DESIGN
How does my clean code become a big ball of mud?
Database
Dennis van der Stelt
MONOLITHIC DESIGN
How does my clean code become a big ball of mud?
Database
Dennis van der Stelt
Dennis van der Stelt
What’s your problem, dude?
Dennis van der Stelt
PROBLEM STATEMENT
Layering
Problem #1
Dennis van der Stelt
Dennis van der Stelt
lightweight servicebus
Dennis van der Stelt
PROBLEM STATEMENT
“little computers
passing messages”
Problem #2
alan kay http://bit.ly/alankay-oo
reduce
coupling
Dennis van der Stelt
SpatialTemporalPlatform
coupling aspects
Dennis van der Stelt
TEMPORAL
Store Front End Shipping Service
Dennis van der Stelt
TEMPORAL
Store Front End Shipping ServiceShipping Service
Dennis van der Stelt
TEMPORAL
Store Front End Shipping ServiceOrder Queue
Dennis van der Stelt
TEMPORAL
Store Front End Shipping ServiceOrder Queue
Dennis van der Stelt
TEMPORAL
Store Front End Shipping ServiceOrder Queue
Dennis van der Stelt
Messaging
 Reduces spatial coupling
 Solves platform coupling
 Asynchronous for temporal coupling
Dennis van der Stelt
CAP Theorem
Eric Brewer, PODC Conference 2000
Dennis van der Stelt
CAP THEOREM
You can only pick 2
Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability
Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability
Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability
distributed
system
partition tolerantconsistencyavailability
Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability
distributed
system
partition tolerantconsistencyavailability
when there’s network partition,
which do you sacrifice?
Dennis van der Stelt
Match the business perspective
Dennis van der Stelt
Dennis van der Stelt
Dennis van der Stelt
Dennis van der Stelt
But I can’t drop consistency!
Dennis van der Stelt
Basically Available
BASE
What is BASE?
Soft state
Eventually consistent
Dennis van der Stelt
Eventual Consistency
Because 100% consistency isn’t the only option
Dennis van der Stelt
Dennis van der Stelt
Eventual Consistency
Dennis van der Stelt
Eventual Consistency
Dennis van der Stelt
Eventual Consistency
your “enterprise” is already eventual consistent with reality
Dennis van der Stelt
Eventual Consistency
your “enterprise” is already eventual consistent with reality
Dennis van der Stelt
“Allow things to be inconsistent and find ways to
compensate for mistakes, versus trying to prevent
mistakes altogether.”
EVENTUAL CONSISTENCY
Eric Brewer
VP of Infrastructure at Google, Professor at UC Berkeley
Dennis van der Stelt
Some ideas on messaging
Dennis van der Stelt
100% synchronized
that‘s eventual consistency.
Udi Dahanin 2010
Dennis van der Stelt
Distributed Transactions
Are they as evil as you think? Perhaps even more?
Dennis van der Stelt
Who uses distributed transactions?
Dennis van der Stelt
using (var scope = new TransactionScope()))
{
var queue = new
MessageQueue(@".Private$Customers");
var message = queue.Receive();
var con = new SqlConnection();
var cmd = new SqlCommand("update customers ...");
cmd.ExecuteNonQuery();
}
DISTRIBUTED TRANSACTIONS
Microsoft Distributed Transaction Coordinator (MSDTC)
Code
Dennis van der Stelt
using (var scope = new TransactionScope()))
{
var queue = new
MessageQueue(@".Private$Customers");
var message = queue.Receive();
var con = new SqlConnection();
var cmd = new SqlCommand("update customers ...");
cmd.ExecuteNonQuery();
}
DISTRIBUTED TRANSACTIONS
Microsoft Distributed Transaction Coordinator (MSDTC)
Code
MSDTC
Dennis van der Stelt
Dennis van der Stelt
Distributed Transactions
Dennis van der Stelt49
Idempotency
Once and only once delivery is hard!
Dennis van der Stelt
Dennis van der Stelt
IDEM POTENCY
Solving the distributed transactionissues
Sender
{0C836F44-6587-416E-B97A-5615615600D5}
Dennis van der Stelt
IDEM POTENCY
Solving the distributed transactionissues
Sender
{0C836F44-6587-416E-B97A-5615615600D5}
Dennis van der Stelt
IDEM POTENCY
Solving the distributed transactionissues
Sender
Event
Subscriber
{5EDC4993-AB01-4F17-A238-71C4521F750F}
{0C836F44-6587-416E-B97A-5615615600D5}
Dennis van der Stelt
IDEM POTENCY
Solving the distributed transactionissues
Sender
Event
Subscriber
{5EDC4993-AB01-4F17-A238-71C4521F750F}
{0C836F44-6587-416E-B97A-5615615600D5}
Dennis van der Stelt
Dennis van der Stelt
find me.
http://dennis.bloggingabout.net
dvdstelt@outlook.com

Distributed Systems principles

  • 1.
    Dennis van derStelt of distributed systems Dennis van der Stelt Software Architect http://dennis.bloggingabout.net/ dennis@bloggingabout.net NServiceBus Community Champ PRINCIPLES @dvdstelt #sdnevent
  • 2.
    Dennis van derStelt YOUR PRESENTER Dennis van der Stelt http://dennis.bloggingabout.net/ Software Architect Professional Services Provides coaching & training About Dennis Innovative software architect Almost 20 years of experience in development of distributed systems. Has a continuous drive to learn and improve knowledge in different architectural styles, including quality in software development. Highly motivated to share his knowledge via articles, presentations and his blog. Awards NServiceBus Champ Typemock MVP Certified Trainer • Presentations • Software & Architecture audits • SOA & Microservices advice • Unit Testing training • NServiceBus training Presentations October 15th at Blaak Selectie / Betabit Reliable systems using messaging
  • 3.
    Dennis van derStelt AGENDA
  • 4.
    Dennis van derStelt A monolithic design is characterized by such tight coupling among modules that they really have no independent existence. monolithic
  • 5.
    Dennis van derStelt MONOLITHIC DESIGN How does my clean code become a big ball of mud? Database
  • 6.
    Dennis van derStelt MONOLITHIC DESIGN How does my clean code become a big ball of mud? Database
  • 7.
  • 8.
    Dennis van derStelt What’s your problem, dude?
  • 9.
    Dennis van derStelt PROBLEM STATEMENT Layering Problem #1
  • 10.
  • 11.
    Dennis van derStelt lightweight servicebus
  • 12.
    Dennis van derStelt PROBLEM STATEMENT “little computers passing messages” Problem #2 alan kay http://bit.ly/alankay-oo
  • 13.
  • 14.
    Dennis van derStelt SpatialTemporalPlatform coupling aspects
  • 15.
    Dennis van derStelt TEMPORAL Store Front End Shipping Service
  • 16.
    Dennis van derStelt TEMPORAL Store Front End Shipping ServiceShipping Service
  • 17.
    Dennis van derStelt TEMPORAL Store Front End Shipping ServiceOrder Queue
  • 18.
    Dennis van derStelt TEMPORAL Store Front End Shipping ServiceOrder Queue
  • 19.
    Dennis van derStelt TEMPORAL Store Front End Shipping ServiceOrder Queue
  • 20.
    Dennis van derStelt Messaging  Reduces spatial coupling  Solves platform coupling  Asynchronous for temporal coupling
  • 21.
    Dennis van derStelt CAP Theorem Eric Brewer, PODC Conference 2000
  • 22.
    Dennis van derStelt CAP THEOREM You can only pick 2
  • 23.
    Dennis van derStelt CAP THEOREM You can only pick 2 centralized system partition tolerantconsistencyavailability
  • 24.
    Dennis van derStelt CAP THEOREM You can only pick 2 centralized system partition tolerantconsistencyavailability
  • 25.
    Dennis van derStelt CAP THEOREM You can only pick 2 centralized system partition tolerantconsistencyavailability distributed system partition tolerantconsistencyavailability
  • 26.
    Dennis van derStelt CAP THEOREM You can only pick 2 centralized system partition tolerantconsistencyavailability distributed system partition tolerantconsistencyavailability when there’s network partition, which do you sacrifice?
  • 27.
    Dennis van derStelt Match the business perspective
  • 28.
  • 29.
  • 30.
  • 31.
    Dennis van derStelt But I can’t drop consistency!
  • 32.
    Dennis van derStelt Basically Available BASE What is BASE? Soft state Eventually consistent
  • 33.
    Dennis van derStelt Eventual Consistency Because 100% consistency isn’t the only option
  • 34.
  • 35.
    Dennis van derStelt Eventual Consistency
  • 36.
    Dennis van derStelt Eventual Consistency
  • 37.
    Dennis van derStelt Eventual Consistency your “enterprise” is already eventual consistent with reality
  • 38.
    Dennis van derStelt Eventual Consistency your “enterprise” is already eventual consistent with reality
  • 39.
    Dennis van derStelt “Allow things to be inconsistent and find ways to compensate for mistakes, versus trying to prevent mistakes altogether.” EVENTUAL CONSISTENCY Eric Brewer VP of Infrastructure at Google, Professor at UC Berkeley
  • 40.
    Dennis van derStelt Some ideas on messaging
  • 41.
    Dennis van derStelt 100% synchronized that‘s eventual consistency. Udi Dahanin 2010
  • 42.
    Dennis van derStelt Distributed Transactions Are they as evil as you think? Perhaps even more?
  • 43.
    Dennis van derStelt Who uses distributed transactions?
  • 44.
    Dennis van derStelt using (var scope = new TransactionScope())) { var queue = new MessageQueue(@".Private$Customers"); var message = queue.Receive(); var con = new SqlConnection(); var cmd = new SqlCommand("update customers ..."); cmd.ExecuteNonQuery(); } DISTRIBUTED TRANSACTIONS Microsoft Distributed Transaction Coordinator (MSDTC) Code
  • 45.
    Dennis van derStelt using (var scope = new TransactionScope())) { var queue = new MessageQueue(@".Private$Customers"); var message = queue.Receive(); var con = new SqlConnection(); var cmd = new SqlCommand("update customers ..."); cmd.ExecuteNonQuery(); } DISTRIBUTED TRANSACTIONS Microsoft Distributed Transaction Coordinator (MSDTC) Code MSDTC
  • 46.
  • 47.
    Dennis van derStelt Distributed Transactions
  • 48.
    Dennis van derStelt49 Idempotency Once and only once delivery is hard!
  • 49.
  • 50.
    Dennis van derStelt IDEM POTENCY Solving the distributed transactionissues Sender {0C836F44-6587-416E-B97A-5615615600D5}
  • 51.
    Dennis van derStelt IDEM POTENCY Solving the distributed transactionissues Sender {0C836F44-6587-416E-B97A-5615615600D5}
  • 52.
    Dennis van derStelt IDEM POTENCY Solving the distributed transactionissues Sender Event Subscriber {5EDC4993-AB01-4F17-A238-71C4521F750F} {0C836F44-6587-416E-B97A-5615615600D5}
  • 53.
    Dennis van derStelt IDEM POTENCY Solving the distributed transactionissues Sender Event Subscriber {5EDC4993-AB01-4F17-A238-71C4521F750F} {0C836F44-6587-416E-B97A-5615615600D5}
  • 54.
  • 55.
    Dennis van derStelt find me. http://dennis.bloggingabout.net dvdstelt@outlook.com