• Save
Dropping ACID - Building Scalable Systems That Work
Upcoming SlideShare
Loading in...5
×
 

Dropping ACID - Building Scalable Systems That Work

on

  • 1,096 views

Presentation given at DevLink 2010

Presentation given at DevLink 2010

Statistics

Views

Total Views
1,096
Views on SlideShare
1,095
Embed Views
1

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • Atomic - All or Nothing <br /> Consistent - from one consistent state to another with each transaction <br /> Isolation - data in a transaction in inaccessible until the transaction is complete <br /> Durable - recoverable in case of a system failure <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Dropping ACID - Building Scalable Systems That Work Dropping ACID - Building Scalable Systems That Work Presentation Transcript

  • Dropping Building Scalable Systems That Work Chris Patterson
  • Chris Patterson @P h atB o yG
  • Agenda Scalability 2010 Availability 2010 Consistency (circa 1983) Consistency (eventually 2010) View slide
  • Scalable View slide
  • Scalable A system that Can handle growth without failure Can be expanded to handle even greater growth
  • Scale Up
  • Scale Up Replace existing system with larger, faster system At some point, growth exceeds capacity Leads to vendor lock-in
  • Scale Out
  • Scale Out Functional Partition Increase overall capacity by separating bounded contexts Data Partitioning Increase capacity within a functional partition
  • Scale Out, Industry Trends Cloud Elastic Expansion Rich Internet Applications HTML5/JS, as well as Flex/Silverlight So-ware as a Service
  • That Work
  • Available A system that Is in an operable state Can provide expected functionality when needed
  • Living in the Clouds Available 24/7, online, anywhere Response time is a dimension of availability 95th percentile is key latency metric
  • Partial Availability Data Partitioning A single database failure only impacts data on that server System can be partially available for a subset of customers
  • 2 PICK
  • Project Triangle Cost Scope Schedule
  • Dating Triangle Intelligent Attractive Emotionally Stable
  • Brewer’s Theorem Consistency Availability Partition Tolerance Credited to Professor Eric A. Brewer
  • Scalable Systems That A system that is Available Scalable Partition Tolerant
  • Consistency Changes to values are consistent in a system An updated address A changed telephone number An additional family member The inventory level of an item
  • Atomic ACID
  • Atomic
  • m ic A to
  • Atomic
  • Atomic All or Nothing
  • Atomic Maintain referential integrity Rollback incomplete transactions Ensure consistent state
  • Atomic Modifications are not accessible to concurrent transactions Lack of isolation can lead to inconsistent state
  • Atomic Completed transactions survive system failures
  • ACID Defined, circa 1983 State of the art system Single machine Vertically scaled up DB/2 (or even ISAM)
  • Distributed Transactional 2PC Two-phase commit protocol Averages availability of involved systems System A (99.9%, 43m) x System B (99.9%, 43m) Results in combined availability of 99.8% (86m)
  • Temporal Inconsistency If I pick up a book at the bookstore, and walk around with it for an hour, another customer is unable to purchase that book. Once I decide to buy in on Amazon instead, a bookstore employee will gladly replace the book on the shelf I go on enjoying my coffee knowing I saved $4.20
  • Eventual Consistency Relax temporal consistency We already do this IsolationLevel.ReadUncommited memcached CDN
  • Eventual Consistency Make temporal consistency windows explicit Command-Query Responsibility Segregation (it’s just command query separation) www.msteched.com/2010/NorthAmerica/ARC302 Persistent View Model
  • Keep Transactions Internal Transactions should not cross service boundaries Avoid adding remote resources to the DTC Avoid cross-database transactions
  • Protocols Create protocols for cross-functional transactions Maintain the state of each exchange in each participating functional partition Example in action Magnum.StateMachine Magnum.Channels
  • Simple Transaction Created Completed
  • Created Do Work Completed
  • Created Do Work Completed FAIL! Failed
  • Pick Up Book Buy Book Sale Replace Book No Sale
  • Pick Up Book Buy Book Sale Replace Book No Sale Damage Book Loss
  • Protocols Long-lived transaction (saga, workflow) Orchestration of correlated Commands Events Handling of compensations Some things can’t be compensated
  • Sagas When(RequestReceived) .Then((saga, request) => { saga.OrderId = request.OrderId; saga.CustomerId = request.CustomerId; }) .Publish(saga => new SendOrderDetailsRequest { RequestId = saga.CorrelationId, CustomerId = saga.CustomerId, OrderId = saga.OrderId, }) .TransitionTo(WaitingForResponse));
  • Sagas During(WaitingForResponse, When(ResponseReceived) .Then((saga, response) => { saga.OrderCreated = response.Created; saga.OrderStatus = response.Status; }) .Publish(saga => new OrderDetails { CustomerId = saga.CustomerId, OrderId = saga.OrderId, Created = saga.OrderCreated.Value, Status = saga.OrderStatus, }) .TransitionTo(Completed));
  • Sagas State Initial { get; set; } State WaitingForResponse { get; set; } State Completed { get; set; }
  • Sagas Event<RetrieveOrderDetails> RequestReceived { get; set; } Event<OrderDetailsResponse> ResponseReceived { get; set; } Event<OrderDetailsRequestFailed> RequestFailed { get; set; }
  • Sagas public virtual string CustomerId { get; set; } public virtual string OrderId { get; set; } public virtual OrderStatus OrderStatus { get; set; } public virtual DateTime? OrderCreated { get; set; }
  • Durable Messaging Asynchronous Temporal decoupling increases availability Supports eventual consistency Given no further changes in state, the system will become consistent
  • Idempotent Uniquely identify every command Allows once-and-only-once completion
  • Search Keyword: BASE Basically Available So--state Eventually Consistent
  • Questions?
  • Contact chris [@phatboyg] .com http://blog.phatboyg.com http://phatboyg.lostechies.com