Concurrent Programming Using the Disruptor
Upcoming SlideShare
Loading in...5
×
 

Concurrent Programming Using the Disruptor

on

  • 14,769 views

Presented to the London Java Community at Skillsmatter on 1st March 2012.

Presented to the London Java Community at Skillsmatter on 1st March 2012.

Full presentation can be viewed here: http://skillsmatter.com/podcast/home/the-disruptor/js-3798

Statistics

Views

Total Views
14,769
Views on SlideShare
6,738
Embed Views
8,031

Actions

Likes
26
Downloads
414
Comments
1

47 Embeds 8,031

http://mechanitis.blogspot.co.uk 2092
http://mechanitis.blogspot.com 1929
http://www.smallake.kr 1094
http://smallake.kr 941
http://mechanitis.blogspot.de 249
http://mechanitis.blogspot.in 197
http://mechanitis.blogspot.com.au 132
http://mechanitis.blogspot.se 115
http://mechanitis.blogspot.fr 114
http://mechanitis.blogspot.hk 99
http://mechanitis.blogspot.jp 94
http://mechanitis.blogspot.ca 94
http://mechanitis.blogspot.com.es 83
http://mechanitis.blogspot.kr 61
http://mechanitis.blogspot.ch 55
http://mechanitis.blogspot.com.br 54
http://mechanitis.blogspot.ru 53
http://mechanitis.blogspot.hu 52
http://mechanitis.blogspot.sg 48
http://mechanitis.blogspot.tw 44
http://mechanitis.blogspot.nl 42
http://mechanitis.blogspot.co.nz 37
http://mechanitis.blogspot.be 37
http://mechanitis.blogspot.it 34
http://mechanitis.blogspot.co.at 33
http://mechanitis.blogspot.ro 32
http://mechanitis.blogspot.dk 29
http://smallake.hosting.paran.com 27
http://mechanitis.blogspot.ie 26
http://mechanitis.blogspot.fi 19
http://mechanitis.blogspot.co.il 18
http://mechanitis.blogspot.cz 15
http://mechanitis.blogspot.mx 10
https://www.linkedin.com 9
http://www.mechanitis.blogspot.co.uk 9
http://www.linkedin.com 8
http://www.thoughtworks.com 8
http://mechanitis.blogspot.com.ar 7
http://mechanitis.blogspot.pt 7
http://www.codesorceresses.eu 5
http://mechanitis.blogspot.sk 4
http://mechanitis.blogspot.gr 4
http://mechanitis.blogspot.no 4
http://translate.googleusercontent.com 3
http://webcache.googleusercontent.com 2
http://mechanitis.blogspot.com.tr 1
http://65.55.108.4 1
More...

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • user guide
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Caveats\n - Not much code - will be available\n
  • Who has heard of it?\nWho has seen other presentations or read blogs?\nWho has played with it?\n Open Source\n Message Passing\n
  • What it does, and how it works\nThe really cool stuff\n
  • Uncontended data structure * except multiple producers\nWay to pass data between threads very fast\n\n
  • That’s just words\n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • It is not, in fact, magic\nArray-backed ring buffer\nSequence numbers\nNothing magic\n not going to touch on: how the ring buffer doesn’t wrap; \n
  • Generics to type it\n
  • Garbage friendly\nReuse the events\n
  • Garbage friendly - event factory\n\n
  • \n
  • Producer, writer...\nBlack & White classes are provided by the Disruptor\nColoured ones are your own\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Disruptor provides a BatchEventProcessor \nfor heavy lifting\n
  • Again, coloured classes for you\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Fast, uncontended replacement for a queue\nvery garbage friendly (or can be)\n
  • Uncontended means things can really work in parallel\n\nHas anyone seen this before?\n
  • \n
  • Imagine the world’s most rubbish car\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Tada!\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Sequence barriers\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Maybe putting engine in takes a while\nMaybe it needs to go to a database to get the “engine”\nPassenger seat might be retrieved from another webservice\n\n
  • \n
  • Dude can put engines in more than one car\nDude will write engines into different cars in the ring buffer\nThis is the only thing that can write this field\n
  • Maybe the seats are retrieved from another service\n
  • \n
  • 3 cars\n
  • 2 cars\n
  • 4 cars\n
  • \n
  • Bonnet is dead fast. Maybe it’s calculating the result based on what’s already in the ringbuffer\n\n
  • Bonnet handler writes results into 33\n
  • Front door handler is sloooow. Maybe it got caught waiting for something else, or it’s sub-optimal\nAll door handlers ask their sequence barrier for the next available sequence\nthis is the lowest of the sequence numbers that barrier is tracking\nin that case, it’s 35 - rear seat handler\n
  • In this case its 35\n
  • front door handler is waaaaay behind\nhe’s going to have to put his door into a bunch of cars\n
  • \n
  • \n
  • \n
  • \n
  • Not going to show writing to the buffer\n
  • Wheels blocked because they’re already at 27\nRingBuffer is blocked cos it can’t write any more\nBack-pressure\n
  • \n
  • \n
  • \n
  • \n
  • Can’t run the car demo at full speed on a machine with two cores\nFields have to be volatile\nStill low contention, but you are contending upon CPU\nYour ring buffer should be bigger than 12\n
  • \n
  • Martin Fowler talks about DDD this morning. Aggregates.\nTake care of separate concerns in separate processes\nSharding\nJoe Armstrong spoke about handling errors outside of your process. Can have processes outside of your business logic\n\n\n
  • \n
  • Feedback - any interest in a workshop?\nWhat else do you want to know about?\n.net version\n

Concurrent Programming Using the Disruptor Concurrent Programming Using the Disruptor Presentation Transcript

  • Concurrent Programming Using The Disruptor Trisha Gee, Developer at LMAX @trisha_gee mechanitis.blogspot.com
  • The Disruptor?
  • What I’m covering• Overview of the Disruptor• Create your own!• Turn it up to Eleven• Q&A
  • What is it?• Data structure and work flow with no contention.• Very fast message passing.• Allows you to go truly parallel.
  • So...?
  • The Magic RingBuffer
  • The Magic RingBuffer
  • The Magic RingBuffer
  • The Magic RingBuffer
  • The Magic RingBuffer
  • The Magic RingBuffer
  • The Magic RingBuffer
  • Creating a RingBufferfinal RingBuffer<SimpleEvent> ringBuffer = new RingBuffer<SimpleEvent>(SimpleEvent.EVENT_FACTORY, RING_BUFFER_SIZE);
  • The Events are Buckets
  • Great! I want one!public class SimpleEvent { public static final EventFactory<SimpleEvent> EVENT_FACTORY = new SimpleEventFactory(); private volatile String value; private static class SimpleEventFactory implements EventFactory<SimpleEvent> { @Override public SimpleEvent newInstance() { return new SimpleEvent(); } }}
  • I’ve got a RingBuffer!• Erm.... how do I poke things into it?
  • The Publisher
  • What do I do?public class SimpleEventTranslator implements EventTranslator<SimpleEvent>SimpleEventTranslator translator = new SimpleEventTranslator();EventPublisher<SimpleEvent> publisher = new EventPublisher<SimpleEvent>(ringBuffer);// poke your translator here// ...and when you’re done...publisher.publishEvent(translator);
  • ...so now I want to read• The Disruptor provides nice batching behaviour for free
  • ...and all you need is...public class SimpleEventHandler implements EventHandler<SimpleEvent>{ @Override public void onEvent(final SimpleEvent event, final long sequence, final boolean endOfBatch) throws Exception { // do stuff }}
  • Shiny. So what?
  • Let’s go parallel
  • And now for something different...
  • Remember Henry Ford?
  • Complex workflow...
  • What on Earth has this got to do with RingBuffers?!
  • Don’t wrap the buffer!ringBuffer.setGatingSequences(finalEventProcessor.getSequence());
  • Caveats
  • Is that it?• Wait and claim strategies• Multiple publishers• Different EventHandlers• The Wizard
  • You get...• A framework the encourages you to model your domain• The ability to run in parallel but single- threaded• Reliable ordering• ...and it can be very fast
  • More Information• Google Code Site, including Wiki http://code.google.com/p/disruptor/• Blogs, e.g. mine: mechanitis.blogspot.com• Presentations• Google Group
  • Q&A• We are hiring!• careers@lmax.com