Dynamic Routing at 1 Million Messages per Second with Spring Integration
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Dynamic Routing at 1 Million Messages per Second with Spring Integration

on

  • 3,428 views

This is the deck for the presentation that John Davies (@jtdavies / john.davies@incept5.com) and I gave for a webinar ...

This is the deck for the presentation that John Davies (@jtdavies / john.davies@incept5.com) and I gave for a webinar (http://spring.io/blog/2013/10/16/nov-19th-webinar-dynamic-routing-at-1-million-per-second-with-spring-integration) on the topic of using Spring Integration and C24-technology to process highly specialized data such as that typical of financial services solutions at high speed.

Statistics

Views

Total Views
3,428
Views on SlideShare
3,418
Embed Views
10

Actions

Likes
8
Downloads
79
Comments
0

3 Embeds 10

http://www.linkedin.com 5
https://twitter.com 4
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Dynamic Routing at 1 Million Messages per Second with Spring Integration Presentation Transcript

  • 1. Dynamic Routing at 1 Million Messages per Second with Spring Integration John Davies, CEO, Incept 5 @jtdavies ! Josh Long, Spring Developer Advocate, Pivotal @starbuxman
  • 2. S P R I N G I N T E G R AT I O N 1 0 1 !2
  • 3. what is “EAI”? “ EAI encompasses approaches, methodologies, standards, and technologies allowing very diverse but important systems to share information, processes, and behavior in support of the core business. -David Linthicum Copyright © 2013 Incept5 Ltd. http://www.incept5.com ”
  • 4. what is “EAI”? § file transfer § shared database § remote procedure call § messaging Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 5. what is “EAI”? Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 6. Spring Integration n (n-1) 2 § At the core, an embedded Message Bus § Also, an Application Integration Framework § Support Enterprise Integration Patterns* via the Spring programming model § Build on the Spring Platform B C Copyright © 2013 Incept5 Ltd. http://www.incept5.com A D
  • 7. Spring Integration § Payload can be any object § Header values are stored in a read-only java.util.Map<K,V> § Similar to JMS messages, email mime envelopes, XMPP messages, etc. public interface Message<T> { MessageHeaders getHeaders(); T getPayload(); } Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 8. Spring Integration § Decouples producers from consumers § provide extension points for interceptors § messages may be queued and buffered { ! ! ! point to point ! { ! ! ! publish-subscribe ! Copyright © 2013 Incept5 Ltd. http://www.incept5.com producer send(Message) message channel receive(Message) consumer <channel id="async-p2p"> <dispatcher task-executor=”simpleAsyncThreadPool" /> <queue capacity=“10” /> </channel> consumer producer send(Message) message channel consumer receive(Message) <publish-subscribe-channel id="async-pubsub” task-executor="someThreadPool" /> consumer
  • 9. Spring Integration transformers: convert payloads and modify headers filter: discard messages based on test router: determine next channel based on message and test splitter: generate multiple messages from one aggregator: assemble a single message from multiple messages service activator: delegates processing to a backend service call Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 10. Spring Integration <channel id = “customerChannel” /> ! <jms:outbound-channel-adapter channel = “customerChannel” destination-name=“stocks” connection-factory = “connectionFactory” /> ! Copyright © 2013 Incept5 Ltd. http://www.incept5.com jms:outbound-adapter producer send(Message) jmsTemplate.send(javax.jms.Message) ActiveMQ
  • 11. Spring Integration <channel id = “fileChannel” /> ! <file:inbound-channel-adapter channel = "fileChannel " directory= "#{systemProperties['user.home']}/Desktop/in "> <int:poller fixed-rate="1000"/> </file:inbound-channel-adapter> Document Document Document new File(directory).list() Copyright © 2013 Incept5 Ltd. http://www.incept5.com file:inbound-channel-adapter ! Message<File> mf = fileChannel.receive() consumer
  • 12. Spring Integration ! MessageChannel channel = ... Message<Customer> customerMessage = MessageBuilder.withPayload(customer) .setHeader("customer-id", 10) .build(); channel.send(customerMessage); Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 13. DEFINING PERFORMANCE !13
  • 14. We need a good safety margin • If we can process 200k/sec and we get a peek of 400k for an hour (e.g. Christmas, Thanksgiving, breaking news etc.) • We have a backlog of 200k/sec for 3600 seconds, if the volume drops to 150k/sec it will take 4 hours to catch up • And we’ll need 360 GB of RAM to queue it! And that’s just 500 bytes/msg ! • RabbitMQ was used at a large “fruit” vendor • Queue sizes were critical in the planing of the deployment ! • Even working at 500k/sec and a peak of 550k for 30 minutes dropping to 400k • 15 minute’s delay and 45 GB of RAM ! • We need “REALTIME” Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 15. Where’s the problem? • In order to do any work on a message we need to parse it • For XML that’s a SAX parser, DOM or Java Binding (JAXB, JIBX etc.) ! • If you’re going to store, search or deliver messages these are the basic steps… • Parse the message or part of the message you need • Query and route the message or… • Query to search the message • - Possibly serialise and de-serialise the message • - Possibly enrich or transform the message • Write the message out (if it’s been bound to Java) ! • All this takes time :-( Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 16. Java Binding • Keeping the example simple a CSV generates something like this…
 
 public class Row extends ComplexDataObject {! 
 private static final long serialVersionUID = 1L;! 
 private String name;! Name,Card Number,Expiry Date,Amount,Currency,Transaction Date,Commission,Vendor ID,Country! 
 private String cardNumber;! Stephen Hawkins,4325-6486-3757-2674,10/06,100,GBP,26-09-2006,2,14988603,UK! private String expiryDate;! Tim Berners-Lee,4724-7345-4725-7833,11/07,258,USD,21-12-2006,5,15688632,UK! Bill Clinton,4924-7264-1264-8532,04/09,1250.6,USD,13-09-2006,15,66846035,US! 
 private double amount;! Angela Merkel,4457-4356-0087-0107,05/08,350,EUR,13-11-2006,7,93440252,DE! Richard Branson,4724-7345-4725-7833,11/06,250,USD,22-02-2006,5,14988103,UK! 
 private boolean isamountSet;! Tim Cook,4924-7264-1264-8532,04/09,12250,USD,16-09-2006,1.3,67434435,US! Alan Turing,4325-6486-3757-2674,10/06,50,GBP,23-02-2006,1,14119663,UK private String currency;! 
 private Date transactionDate;! 
 private double commission;! 
 private boolean iscommissionSet;! private long vendorID;! 
 private boolean isvendorIDSet;! private String country; Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 17. Message Parsing • Taking a message with about 50-60 attributes and parsing them so that they can be available to Spring Integration/Batch is not exactly complex ! • But providing the ability to manage the changes and dozens of standards starts to add complexity • Things change over time, we need to manage that change too • We’d also like a consistent API so Java-Binding is still ideal ! • Two relatively simple examples from the Telco and Financial Services industry are RADIUS (rfc-2865) and FIX (from FPL) • Others include ASN.1 & ISO-8583 Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 18. Radius (rfc-2865) • Remote Authentication Dial In User Service (RADIUS) • 77 pages of binary spec...
 
 
 
 
 
 
 
 
 
 
 Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 19. Modelling in C24 first •The RADIUS standard...
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 20. What we can now do • The spec may say that bit 13 of a 32 bit field represents the presence of a field ABC (later in the message) • Programatically we can test bit 5 with a mask 0x00002000 • Using the generated code we can simply call isAbcSet() ! • It may say that bits 4-6 represent the version ID • Programatically we can mask it and then shift it... • mask 0x00000070 • shift >> 4 • Using the generated code we can simply call getVersionId() ! • We now have a nice API that hides the complexity of the binary implementation Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 21. Bits and Bytes to… • We can now go from this… ! jd-server:Radius TestData jdavies$ 00000000 01 02 00 74 ea d5 7c 62 00000010 35 25 e5 8c 1a 17 00 00 00000020 35 37 30 36 32 37 38 38 00000030 31 35 39 30 36 32 35 38 00000040 33 34 32 31 30 32 30 39 00000050 34 37 30 30 34 31 38 38 00000060 af 08 07 32 33 34 33 35 00000070 4e 97 57 a8 00000074 hexdump -C radius.dat 1f d0 f6 fe a3 bf 36 4c 28 af 01 11 32 33 34 34 35 33 36 01 11 32 33 34 38 35 33 36 1f 11 33 35 34 35 35 36 38 5e 0e 34 36 37 33 1a 0d 00 00 28 06 06 00 00 00 02 37 06 ! • To being able to use it in Spring • Or Mule, Fuse, Camel etc… Copyright © 2013 Incept5 Ltd. http://www.incept5.com |...t..|b......6L| |5%......(...2344| |57062788536..234| |159062588536..35| |3421020945568^.4| |47004188673....(| |...23435......7.| |N.W.|
  • 22. To Spring Integration... • Test bits and binary data in native Spring… ! • <filter input-channel="filter-message-channel" outputchannel="process-message-channel" ref=”payload” method=”isAbcSet"/> ! • <filter input-channel="filter-message-channel" outputchannel="process-message-channel" expression="payload.versionId == 5"/> Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 23. A good start but… • We now have a pretty cool API and can use Spring but it’s still pretty slow as it generates LOTS of objects ! • With our telco standards binary to Java binding was “slow” due to the number of objects, we got around 20k/sec/core • Fine for most purposes and with an 8 core machine potentially good for 100k/sec but we wanted better ! • We needed a better solution, it wasn’t the parsing but the complexity of the objects we were creating and parsing into Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 24. ByteBuffer • Java 1.4 added java.nio.ByteBuffer • Basically a wrapper for a byte[]
 
 
 
 
 
 
 
 
 
 Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 25. Lazy parsing • Rather than parse all of the elements from the message every time we retrieve the elements only when we need them • Similar to comparing a DOM with a SAX parser • We assume that we will only need to filter/sort/query on a limited number of fields so we save a lot of redundant parsing ! • Performance goes from ~20k/sec (50µs per message) to ~1 million/sec (1µS per message), some 50 times faster • Even parsing the entire message is significantly faster Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 26. Here’s where it gets interesting... • So we now have a Java object with a ByteBuffer holding the message data and dozens of get() methods to get the content • Message call = new Message(data); • call.getDuration(); • This works pretty fast, plus the JIT compiler kicks in after 10,000 iterations and optimises the method bottleneck moves to the SpEL queries • Now theinput-channel="filter-message-channel" output-channel="process-message-channel" <filter • expression="payload.duration lt 0.1"/> Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 27. Reflection • What’s happening under the hood is reflection, very powerful but sadly still rather “slow” • expression="payload.duration lt 0.1" • Turns into something like... ! ! ! • This adds about 700nS to each message • A few of these and we’ve more than halved the performance Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 28. The Java Compiler to the rescue! • New from Java 1.6: ToolProvider.getSystemJavaCompiler(); • We can create a generic accessor for double values... ! ! • Can now write a class on the fly that implements this method for the getter we want e.g. getDuration()
 
 
 
 Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 29. Java on the fly... Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 30. Compile it... Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 31. Running it... • Load up the compiled class (or byte-code) and run it ! ! • Note however that the first two lines (above) are done only once, outside of the loop ! • The result is “native” performance as if it was code • Which of course it is Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 32. Back to spring... • SpEL expressions are currently interpreted ! • There is some basic caching • java.lang.reflect.Field • java.lang.reflect.Method objects are cached once discovered and reused on subsequent evaluations ! • But overall they are “slow” (for what we need) • We need to look at avoiding reflection by compiling the parsed expression into a class Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 33. SpEL compiler • Currently proposed by Andy Clement (Spring guru) is a SpEL compiler • With the proposed changes SpEL has a Mixed Mode Interpreter (MMI) system in place • Mixed means it mixes the current interpreter with a real expression compiler • When an expression is evaluated X number of times SpEL compiles it to byte-code and uses that going forward • User does nothing, evaluations just accelerate Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 34. Does it make a difference? • Property access:foo.bar.boo
 
 
 
 
 
 
 
 
 
 
 Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 35. Does it make a difference? • Method invocation:hello()
 
 
 
 
 
 ! ! Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 36. Advantages - Performance • We can now parse, compare and filter in around 1 µSec (1 million/sec) per core • Make decisions on what you want early on and drop unwanted messages • Provide “on-the-fly” aggregation • Around 90-95% less CPU per message - $€£¥₹ ! • Disadvantages • Less error checking • Hardware vendors make less money • Machine rooms become uncomfortably cold • Power bills go down Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 37. Other Advantages • Messages in memory are binary encoded in ByteBuffers and so occupy a fraction of the size of a bound Java object • Typically around 1/12 of the size, using only 8% of the memory ! • We can queue around 10 times more messages per unit of memory ! • We get around 1000% capacity advantage for in-memory grids - GigaSpaces, Hazelcast, GemFire, Coherence ! • We can provide custom serialisation of the ByteBuffer for all of the above - boosting node to node replication Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 38. So - In a nutshell • We’ve gone from 20k per second to around 800k • 40 times faster, sometimes better ! • We’ve gone reduced memory footprint by around 12 times • Meaning you can store 12 times more data in your grid • Or use 1/12th of the hardware/memory ! • You can route this with Spring at “native” JIT-compiled Java code speeds • So far no other framework can offer this level of efficiency, dynamic flexibility and performance Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 39. The Future • SpEL to native Java compilation to be included in Spring release • A few issue remain to be resolved in the short term but early versions look good ! • Full support for standards like ASN.1 • This is a standard wire-level implementation option for ISO-20022 • ASN.1 supports multiple encoding rules, Basic, XML, Canonical, Packed etc. ! • XML messages encoded as packed ASN.1 (PER) would be almost as performant as raw binary • Complex XML as packed binary in a ByteBuffer Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 40. More Spring • Read more on the tools we used... ! • Spring XD - http://projects.spring.io/spring-xd/ ! • Spring Integration - http://projects.spring.io/spring-integration/ ! • Spring Batch - http://projects.spring.io/spring-batch/ ! • Spring AMQP - http://projects.spring.io/spring-amqp/ Copyright © 2013 Incept5 Ltd. http://www.incept5.com
  • 41. Thank you! + Josh Long John Davies Twitter: @starbuxman Twitter: @jtdavies GitHub: http:// github.com/joshlong LinkedIn: http://linkedin.com/in/jdavies/ E-Mail: jlong@gopivotal.com E-Mail: John.Davies@C24.biz Spring & Spring Integration: 
 http://spring.io C24: http://www.c24.biz Copyright © 2013 Incept5 Ltd. http://www.incept5.com