Your SlideShare is downloading. ×
Dropping
Building Scalable Systems That Work



       Chris Patterson
Chris Patterson


@P h atB o yG
Agenda



Scalability 2010

Availability 2010

Consistency (circa 1983)

Consistency (eventually 2010)
Scalable
Scalable



A system that

  Can handle growth without failure

  Can be expanded to handle even greater growth
Scale Up
Scale Up



Replace existing system with larger, faster system

  At some point, growth exceeds capacity

  Leads to vendo...
Scale Out
Scale Out


Functional Partition

  Increase overall capacity by separating bounded
  contexts

Data Partitioning

  Incre...
Scale Out, Industry Trends


 Cloud

   Elastic Expansion

 Rich Internet Applications

   HTML5/JS, as well as Flex/Silve...
That Work
Available



A system that

  Is in an operable state

  Can provide expected functionality when needed
Living in the Clouds



Available

  24/7, online, anywhere

Response time is a dimension of availability

  95th percenti...
Partial Availability


Data Partitioning

  A single database failure only impacts data on that
  server

  System can be ...
2
PICK
Project Triangle

Cost              Scope




       Schedule
Dating Triangle

Intelligent         Attractive




          Emotionally
            Stable
Brewer’s Theorem

                  Consistency                      Availability




                                    ...
Scalable Systems That



A system that is

  Available

  Scalable

    Partition Tolerant
Consistency


Changes to values are consistent in a system

  An updated address

  A changed telephone number

  An addit...
Atomic
 ACID
Atomic
m ic
A to
Atomic
Atomic

         All or Nothing
Atomic
         Maintain referential
         integrity

         Rollback incomplete
         transactions

         Ensu...
Atomic
         Modifications are not
         accessible to concurrent
         transactions

         Lack of isolation c...
Atomic
         Completed transactions
         survive system failures
ACID Defined, circa 1983



State of the art system

  Single machine

  Vertically scaled up

  DB/2 (or even ISAM)
Distributed Transactional


2PC

  Two-phase commit protocol

Averages availability of involved systems

  System A (99.9%...
Temporal Inconsistency


If I pick up a book at the bookstore, and walk around
with it for an hour, another customer is un...
Eventual Consistency


Relax temporal consistency

We already do this

  IsolationLevel.ReadUncommited

  memcached

  CDN
Eventual Consistency


Make temporal consistency windows explicit

Command-Query Responsibility Segregation

  (it’s just ...
Keep Transactions Internal



 Transactions should not cross service boundaries

   Avoid adding remote resources to the D...
Protocols


Create protocols for cross-functional transactions

  Maintain the state of each exchange in each
  participat...
Simple Transaction



 Created   Completed
Created   Do Work   Completed
Created   Do Work   Completed


           FAIL!      Failed
Pick Up Book    Buy Book       Sale




               Replace Book   No Sale
Pick Up Book    Buy Book       Sale




               Replace Book   No Sale




               Damage Book     Loss
Protocols

Long-lived transaction (saga, workflow)

Orchestration of correlated

  Commands

  Events

Handling of compensa...
Sagas

When(RequestReceived)

 .Then((saga, request) =>

 
 {

 
 
 saga.OrderId = request.OrderId;

 
 
 saga.CustomerId ...
Sagas

During(WaitingForResponse,

 When(ResponseReceived)

 
 .Then((saga, response) =>

 
 
 {

 
 
 
 saga.OrderCreated...
Sagas




State Initial { get; set; }
State WaitingForResponse { get; set; }
State Completed { get; set; }
Sagas




Event<RetrieveOrderDetails> RequestReceived { get; set; }
Event<OrderDetailsResponse> ResponseReceived { get; se...
Sagas





   
   public   virtual   string CustomerId { get; set; }

   
   public   virtual   string OrderId { get; set;...
Durable Messaging


Asynchronous

Temporal decoupling increases availability

Supports eventual consistency

  Given no fu...
Idempotent




Uniquely identify every command

  Allows once-and-only-once completion
Search Keyword: BASE



Basically Available

So--state

Eventually Consistent
Questions?
Contact



 chris [@phatboyg] .com
  http://blog.phatboyg.com
http://phatboyg.lostechies.com
Dropping ACID - Building Scalable Systems That Work
Dropping ACID - Building Scalable Systems That Work
Upcoming SlideShare
Loading in...5
×

Dropping ACID - Building Scalable Systems That Work

882

Published on

Presentation given at DevLink 2010

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
882
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide



























  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure
  • Atomic - All or Nothing
    Consistent - from one consistent state to another with each transaction
    Isolation - data in a transaction in inaccessible until the transaction is complete
    Durable - recoverable in case of a system failure























  • Transcript of "Dropping ACID - Building Scalable Systems That Work"

    1. 1. Dropping Building Scalable Systems That Work Chris Patterson
    2. 2. Chris Patterson @P h atB o yG
    3. 3. Agenda Scalability 2010 Availability 2010 Consistency (circa 1983) Consistency (eventually 2010)
    4. 4. Scalable
    5. 5. Scalable A system that Can handle growth without failure Can be expanded to handle even greater growth
    6. 6. Scale Up
    7. 7. Scale Up Replace existing system with larger, faster system At some point, growth exceeds capacity Leads to vendor lock-in
    8. 8. Scale Out
    9. 9. Scale Out Functional Partition Increase overall capacity by separating bounded contexts Data Partitioning Increase capacity within a functional partition
    10. 10. Scale Out, Industry Trends Cloud Elastic Expansion Rich Internet Applications HTML5/JS, as well as Flex/Silverlight So-ware as a Service
    11. 11. That Work
    12. 12. Available A system that Is in an operable state Can provide expected functionality when needed
    13. 13. Living in the Clouds Available 24/7, online, anywhere Response time is a dimension of availability 95th percentile is key latency metric
    14. 14. Partial Availability Data Partitioning A single database failure only impacts data on that server System can be partially available for a subset of customers
    15. 15. 2 PICK
    16. 16. Project Triangle Cost Scope Schedule
    17. 17. Dating Triangle Intelligent Attractive Emotionally Stable
    18. 18. Brewer’s Theorem Consistency Availability Partition Tolerance Credited to Professor Eric A. Brewer
    19. 19. Scalable Systems That A system that is Available Scalable Partition Tolerant
    20. 20. Consistency Changes to values are consistent in a system An updated address A changed telephone number An additional family member The inventory level of an item
    21. 21. Atomic ACID
    22. 22. Atomic
    23. 23. m ic A to
    24. 24. Atomic
    25. 25. Atomic All or Nothing
    26. 26. Atomic Maintain referential integrity Rollback incomplete transactions Ensure consistent state
    27. 27. Atomic Modifications are not accessible to concurrent transactions Lack of isolation can lead to inconsistent state
    28. 28. Atomic Completed transactions survive system failures
    29. 29. ACID Defined, circa 1983 State of the art system Single machine Vertically scaled up DB/2 (or even ISAM)
    30. 30. Distributed Transactional 2PC Two-phase commit protocol Averages availability of involved systems System A (99.9%, 43m) x System B (99.9%, 43m) Results in combined availability of 99.8% (86m)
    31. 31. Temporal Inconsistency If I pick up a book at the bookstore, and walk around with it for an hour, another customer is unable to purchase that book. Once I decide to buy in on Amazon instead, a bookstore employee will gladly replace the book on the shelf I go on enjoying my coffee knowing I saved $4.20
    32. 32. Eventual Consistency Relax temporal consistency We already do this IsolationLevel.ReadUncommited memcached CDN
    33. 33. Eventual Consistency Make temporal consistency windows explicit Command-Query Responsibility Segregation (it’s just command query separation) www.msteched.com/2010/NorthAmerica/ARC302 Persistent View Model
    34. 34. Keep Transactions Internal Transactions should not cross service boundaries Avoid adding remote resources to the DTC Avoid cross-database transactions
    35. 35. Protocols Create protocols for cross-functional transactions Maintain the state of each exchange in each participating functional partition Example in action Magnum.StateMachine Magnum.Channels
    36. 36. Simple Transaction Created Completed
    37. 37. Created Do Work Completed
    38. 38. Created Do Work Completed FAIL! Failed
    39. 39. Pick Up Book Buy Book Sale Replace Book No Sale
    40. 40. Pick Up Book Buy Book Sale Replace Book No Sale Damage Book Loss
    41. 41. Protocols Long-lived transaction (saga, workflow) Orchestration of correlated Commands Events Handling of compensations Some things can’t be compensated
    42. 42. Sagas When(RequestReceived) .Then((saga, request) => { saga.OrderId = request.OrderId; saga.CustomerId = request.CustomerId; }) .Publish(saga => new SendOrderDetailsRequest { RequestId = saga.CorrelationId, CustomerId = saga.CustomerId, OrderId = saga.OrderId, }) .TransitionTo(WaitingForResponse));
    43. 43. Sagas During(WaitingForResponse, When(ResponseReceived) .Then((saga, response) => { saga.OrderCreated = response.Created; saga.OrderStatus = response.Status; }) .Publish(saga => new OrderDetails { CustomerId = saga.CustomerId, OrderId = saga.OrderId, Created = saga.OrderCreated.Value, Status = saga.OrderStatus, }) .TransitionTo(Completed));
    44. 44. Sagas State Initial { get; set; } State WaitingForResponse { get; set; } State Completed { get; set; }
    45. 45. Sagas Event<RetrieveOrderDetails> RequestReceived { get; set; } Event<OrderDetailsResponse> ResponseReceived { get; set; } Event<OrderDetailsRequestFailed> RequestFailed { get; set; }
    46. 46. Sagas public virtual string CustomerId { get; set; } public virtual string OrderId { get; set; } public virtual OrderStatus OrderStatus { get; set; } public virtual DateTime? OrderCreated { get; set; }
    47. 47. Durable Messaging Asynchronous Temporal decoupling increases availability Supports eventual consistency Given no further changes in state, the system will become consistent
    48. 48. Idempotent Uniquely identify every command Allows once-and-only-once completion
    49. 49. Search Keyword: BASE Basically Available So--state Eventually Consistent
    50. 50. Questions?
    51. 51. Contact chris [@phatboyg] .com http://blog.phatboyg.com http://phatboyg.lostechies.com

    ×