Using chained transactions for maximum concurrency under load (QCONSF 2010)

Thoughts on Transactions
Chained transactions for scaling ACID
@billynewport
@billynewport
1

Agenda
Explain scenario
Implement using a single database
Explain concurrency issues under load
Implement using a sharded database
Implement using WebSphereeXtreme Scale and chained transactions
@billynewport
2

Scenario
Large eCommerce web site
Problem is order checkout
We track inventory levels for each SKU
Orders during checkout need to adjust available inventory.
@billynewport
3

Shopping cart metrics
Millions of SKUs
Cart size of 5 items for electronics/big ticket items
Cart size of 20 items for clothing
Expect concurrent load of 2500 checkouts per second
@billynewport
4

Database
Begin
For each item in cart
Select for update where sku = item.sku
Decrement available sku level
If not available then rollback…
Update level where sku = item.sku
Commit
Cart items randomly distributed amongst all 2m items, lots of concurrency.
Simple enough, right? All is good?
@billynewport
5

Problem: cabbage patch dolls
Cabbage patch dolls are popular this fall…
@billynewport
6

Database killers!
The dolls cause major concurrency problems
Lots of row level locks
Contention on doll rows
Possible table lock escalation
App server thread issues
Connection pools empty
Then DEATH!
They aren’t sweet and cuddly any more…
@billynewport
7

Database killers
We need a way to get locks to decrement inventory
But, we don’t want to hold the lock for very long
Bigger carts make the problem worse, all the locks held for longer
Ideally, hold locks for constant time
Any contentious items make problem worse
@billynewport
8

Solution
Hold lock on inventory rows for as short a time as possible
Decouple this from size of cart.
How?
@billynewport
9

Chained transactions
Programmers think of transactions in synchronous terms.
Begin / Do Work / Hold locks / Commit
Easy to program, bad for concurrency.
@billynewport
10

Inspiration
Microsoft had COM objects with apartment model threading.
Modern Actor support is similar. Some state with a mailbox.
BPEL supports flows with compensation
Data meets actors is a good analogy
Send a message (cart) to a group of actors identified using their keys with a compensator
@billynewport
11

Alternative
We need to think asynchronously in terms of flows with compensation
Map of <SKU Key/SKU Amount>
Brick:
Do{ code to reduce inventory level for SKU }
Undo{ code to increase level inventory for SKU }
Provide Map with do/undo bricks
Easy to program, great concurrency.
@billynewport
12

Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
@billynewport
13

@billynewport
14

@billynewport
15

Implementation
1PC only required
Data store supporting
Row locks
Row oriented data
Integrated FIFO messaging
IBM WebSphereeXtreme Scale provides these capabilities.
@billynewport
16

Implementation
Application makes map and code bricks
Submits transaction as an asynchronous job.
Uses a Future to check on job outcome.
Do blocks can trigger flow reversal if a problem occurs.
Invoke undo block for each completed step
@billynewport
17

Mechanism
Loop
Receive message for actor key
Process it
Send modified cart to next ‘sku’ using local ‘transmit q’
Commit transaction
Background thread pushes messages in transmit q to the destination shards using exactly once semantics.
@billynewport
18

Performance
A single logical transaction will be slower than a 1PC DBMS implementation.
However, under concurrent load then it will deliver:
Higher throughput
Better response times
Thru better contention management
Each ‘SKU’ only locked for a very short period
@billynewport
19

Generalization
This could be thought of as a workflow engine.
But, a big difference here is that a workflow engine usually talks with a remote store.
Here:
the flow state is the MESSAGE
It moves around to where the data is for the next step
Using a MESSAGE for flow state rather than a database means it scales linearly.
The message ‘store’ is integrated and scales with the data store.
@billynewport
20

Architecture Comparison
Conventional
Message oriented
@billynewport
21
BP
Engine
BP
Engine
BP
Engine
Integrated
Msg/Data store
Integrated
Msg/Data store
Integrated
Msg/Data store
Flow Edge = Msg
Flow
State
Flow
DB
Appl
DB
Msg
Store
Write behind
Appl
DB

Sample implementation
Coming soon.
Running in lab
Working with several eCommerce customers looking to implement soon.
Soon to be published on github as sample code.
@billynewport
22

Using chained transactions for maximum concurrency under load (QCONSF 2010)

by Billy Newport on Nov 05, 2010

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Using chained transactions for maximum concurrency under load (QCONSF 2010) Presentation Transcript