Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Asynchronous design with Spring and RTI: 1M events per second

1,813 views

Published on

An application designer usually has to choose where to trade flexibility for specificity (and thus usually performance); knowing when and where to do so is an art and requires experience. This talk will share over a decades worth of experience making these decisions and the learnings from developing Pivotal's successful Real Time Intelligence (RTI) product using the latest versions of Spring projects: Integration, Data, Boot, MVC/REST and XD. A walk through the RTI architecture will provide the base for an explanation about how Spring performs at hundreds (and millions) of events/operations per second and the techniques that you can use right now in your own Spring applications to minimise resource utilisation and gain performance.

Published in: Software
  • Be the first to comment

Asynchronous design with Spring and RTI: 1M events per second

  1. 1. Asynchronous Design: 1M events per second – with Spring By Stuart Williams © 2014 SpringOne 2GX. All rights reserved. Do not distribute without permission.
  2. 2. Bio 2 Stuart Williams • Software Engineer at Pivotal – RTI project lead @pidster
  3. 3. What is this all about? • We built a product using Pivotal products • Learned some lessons • We found a few limitations & some room for improvement… • … but we addressed them & now things go faster. A lot faster. 3
  4. 4. Dogfood • Built with Spring IO Platform • Boot, Data, Integration, Reactor, AMQP, SpEL, Shell (and a little Groovy) • GemFire • RabbitMQ 4 Spring Framework Spring Data Spring Integration Spring Reactor Spring AMQP Spring Hateoas Groovy Spring Boot
  5. 5. Questions for you • Heard of Spring Integration? • Tried it? • In production? • Heard of Reactor? • Tried it? • In production? 5
  6. 6. RTI 6
  7. 7. What is RTI? • RTI == ‘Real Time Intelligence’ • Stream processing application • Location based services • Analytics (e.g. network health) • Telecom network data • ‘Control plane’ is meta data • ‘User plane’ is actual data (30x more) • Rich data model 7
  8. 8. Input Data Rates RTI* • 100k/s average • 120k/s daily peak • 1M/s annual peak 8 *Control-plane only, user-plane is 20x
  9. 9. Input Data Rates RTI* • 100k/s baseline • >120k/s daily peak • >1M/s annual peak 10 *Control-plane only, user-plane is 20x Twitter** • 6k/s average • 9k/s daily peak • 30k/s large events **Source @catehstn twitter.com/catehstn/status/494918021358813184
  10. 10. Load Characteristics • Low numbers of inbound connections • High rates, micro-bursts • Occasional peaks of nearly 2x, rare peaks of 10x • Variable payload size (200B – 300KB) • Internal fan-outs multiple event rates 11
  11. 11. More statistics… • 100k/s order of magnitude • 8,640,000,000 (per day) • An Integer based counter will ‘roll over’ in ~2 days • 400Mbps of raw data (‘control plane’) • 10Gbps NICs required to support traffic peaks • Logging! Any verbose errors can blow a disk away • Queues backing up == #fail 12
  12. 12. Architecture 13
  13. 13. Architecture 14 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
  14. 14. Architectural Challenges • Ingest • Responsibility • Micro-bursts • Infrastructure considerations • Compute • Memory • Disk • Network 15
  15. 15. Architecture 16 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
  16. 16. Ingester Architecture 17 Ingester • Spring Integration • TCP Server • Transformer • Filters • Reactor Stream • GemFire client • Single process • Multiple protocols – different rates & sizes
  17. 17. Architecture 18 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
  18. 18. Analytics Architecture 19 Analytics • Reactor • SpEL evaluation • Hundreds of expression calculations per event • Collate across nodes • Output to File or AMQP
  19. 19. Architecture 20 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP
  20. 20. Architecture 21 Distribution • Spring Integration • Enrichers • Filters • Reactor Stream • Output to File / AMQP / JDBC • Membership checks • LBS, opt-in’s
  21. 21. First Ingester solution 22
  22. 22. Solution #1 – ‘Naïve’ proof of concept • Build codecs • More on this in John Davies’ “Big Data In Memory” talk later today… • Spring Integration (SI) pipeline • TCP Inbound Adapter • Transformer • Filters • Outbound adapter 23
  23. 23. Solution #1 – ‘Naïve’ proof of concept 24
  24. 24. Solution #1 results • Message UUID generation was slow! • SpEL-based method resolution was slow! • Standard Channels added overhead! • Single event output was slow! 25
  25. 25. Ingester Mark 2 26
  26. 26. Solution #2 – Use interfaces 27
  27. 27. Solution #2 – Use interfaces • Use the IdGenerator interface • Use specific endpoint interfaces • … we’ll come back to SpEL … • Use a Chain • Use an Aggregator to build a batch 28
  28. 28. Solution #2 results • IdGenerator helped a lot • Specific interfaces not recognised! • Using <int:chain helped • Aggregator helped, but is too slow • <int:tcp-inbound-adapter is too slow 29
  29. 29. Many whiteboards later… 30
  30. 30. Many whiteboards later… 31
  31. 31. Working version 32
  32. 32. Solution N 33 Message-only Filters Batcher Reference Data Filters GB IUPS IUCS A Radius/Diamete r 4G
  33. 33. Working version • Netty / Reactor TCP • IdGenerator • Specific endpoint interfaces • Chain • Reactor Stream based batching • + many improvements & enhancements 34
  34. 34. Roundup 35
  35. 35. Spring Improvements • Performance • Spring Integration • SpEL • Reactor • Spring XD benefits from these upgrades 36
  36. 36. Spring Integration 37
  37. 37. Spring Integration • UUID generator • MessageBuilderFactory & MutableMessage • Dispatcher optimisation • SpEL parser caching • Counters are ‘long’ • Interfaces used directly – if you’re specific SI respects that 38
  38. 38. Spring Expression Language • Compilation of expressions • Configuration options • SI just re-evaluates expressions • Trade-offs & limitations • Much, much faster 39
  39. 39. SpEL demo 40
  40. 40. Reactor • Drop-in replacements • Thread pools, dispatchers • TCP/UDP Server & Client • Much faster – lower resource utilisation • Stream API • Batching and other functionality • More about Reactor • Thu, 8.30am “Building Reactive Applications…” 41
  41. 41. Batching endpoint code 42
  42. 42. Summary • Spring Integration is much faster • Good performance means better resource utilisation • For cloud applications cost savings can be dramatic • Batching payloads makes a big difference • Many applications wait on network IO • Trade-off risk of data loss for performance • Reactor FTW 43
  43. 43. Questions 44
  44. 44. 45 Learn More. Stay Connected Tweet #rti #s2gx if you’d like to go faster @pidster “Big Data in Memory” John Davis – Trinity 1-2 4.30pm @springcentral | spring.io/video

×