Asynchronous design with Spring and RTI: 1M events per second

2. Bio 2 Stuart Williams • Software Engineer at Pivotal – RTI project lead @pidster

3. What is this all about? • We built a product using Pivotal products • Learned some lessons • We found a few limitations & some room for improvement… • … but we addressed them & now things go faster. A lot faster. 3

4. Dogfood • Built with Spring IO Platform • Boot, Data, Integration, Reactor, AMQP, SpEL, Shell (and a little Groovy) • GemFire • RabbitMQ 4 Spring Framework Spring Data Spring Integration Spring Reactor Spring AMQP Spring Hateoas Groovy Spring Boot

5. Questions for you • Heard of Spring Integration? • Tried it? • In production? • Heard of Reactor? • Tried it? • In production? 5

6. RTI 6

7. What is RTI? • RTI == ‘Real Time Intelligence’ • Stream processing application • Location based services • Analytics (e.g. network health) • Telecom network data • ‘Control plane’ is meta data • ‘User plane’ is actual data (30x more) • Rich data model 7

8. Input Data Rates RTI* • 100k/s average • 120k/s daily peak • 1M/s annual peak 8 *Control-plane only, user-plane is 20x

9. Input Data Rates RTI* • 100k/s baseline • >120k/s daily peak • >1M/s annual peak 10 *Control-plane only, user-plane is 20x Twitter** • 6k/s average • 9k/s daily peak • 30k/s large events **Source @catehstn twitter.com/catehstn/status/494918021358813184

10. Load Characteristics • Low numbers of inbound connections • High rates, micro-bursts • Occasional peaks of nearly 2x, rare peaks of 10x • Variable payload size (200B – 300KB) • Internal fan-outs multiple event rates 11

11. More statistics… • 100k/s order of magnitude • 8,640,000,000 (per day) • An Integer based counter will ‘roll over’ in ~2 days • 400Mbps of raw data (‘control plane’) • 10Gbps NICs required to support traffic peaks • Logging! Any verbose errors can blow a disk away • Queues backing up == #fail 12

12. Architecture 13

13. Architecture 14 Analytics WAN Queue Ingester Ingest Grid Distribution Metrics Firehose AMQP HTTP HTTP

14. Architectural Challenges • Ingest • Responsibility • Micro-bursts • Infrastructure considerations • Compute • Memory • Disk • Network 15

16. Ingester Architecture 17 Ingester • Spring Integration • TCP Server • Transformer • Filters • Reactor Stream • GemFire client • Single process • Multiple protocols – different rates & sizes

18. Analytics Architecture 19 Analytics • Reactor • SpEL evaluation • Hundreds of expression calculations per event • Collate across nodes • Output to File or AMQP

20. Architecture 21 Distribution • Spring Integration • Enrichers • Filters • Reactor Stream • Output to File / AMQP / JDBC • Membership checks • LBS, opt-in’s

21. First Ingester solution 22

22. Solution #1 – ‘Naïve’ proof of concept • Build codecs • More on this in John Davies’ “Big Data In Memory” talk later today… • Spring Integration (SI) pipeline • TCP Inbound Adapter • Transformer • Filters • Outbound adapter 23

23. Solution #1 – ‘Naïve’ proof of concept 24

24. Solution #1 results • Message UUID generation was slow! • SpEL-based method resolution was slow! • Standard Channels added overhead! • Single event output was slow! 25

25. Ingester Mark 2 26

26. Solution #2 – Use interfaces 27

27. Solution #2 – Use interfaces • Use the IdGenerator interface • Use specific endpoint interfaces • … we’ll come back to SpEL … • Use a Chain • Use an Aggregator to build a batch 28

28. Solution #2 results • IdGenerator helped a lot • Specific interfaces not recognised! • Using <int:chain helped • Aggregator helped, but is too slow • <int:tcp-inbound-adapter is too slow 29

29. Many whiteboards later… 30

30. Many whiteboards later… 31

31. Working version 32

32. Solution N 33 Message-only Filters Batcher Reference Data Filters GB IUPS IUCS A Radius/Diamete r 4G

33. Working version • Netty / Reactor TCP • IdGenerator • Specific endpoint interfaces • Chain • Reactor Stream based batching • + many improvements & enhancements 34

34. Roundup 35

35. Spring Improvements • Performance • Spring Integration • SpEL • Reactor • Spring XD benefits from these upgrades 36

36. Spring Integration 37

37. Spring Integration • UUID generator • MessageBuilderFactory & MutableMessage • Dispatcher optimisation • SpEL parser caching • Counters are ‘long’ • Interfaces used directly – if you’re specific SI respects that 38

38. Spring Expression Language • Compilation of expressions • Configuration options • SI just re-evaluates expressions • Trade-offs & limitations • Much, much faster 39

39. SpEL demo 40

40. Reactor • Drop-in replacements • Thread pools, dispatchers • TCP/UDP Server & Client • Much faster – lower resource utilisation • Stream API • Batching and other functionality • More about Reactor • Thu, 8.30am “Building Reactive Applications…” 41

41. Batching endpoint code 42

42. Summary • Spring Integration is much faster • Good performance means better resource utilisation • For cloud applications cost savings can be dramatic • Batching payloads makes a big difference • Many applications wait on network IO • Trade-off risk of data loss for performance • Reactor FTW 43

43. Questions 44

44. 45 Learn More. Stay Connected Tweet #rti #s2gx if you’d like to go faster @pidster “Big Data in Memory” John Davis – Trinity 1-2 4.30pm @springcentral | spring.io/video

Asynchronous design with Spring and RTI: 1M events per second

You are reading a preview.

Check these out next

Asynchronous design with Spring and RTI: 1M events per second

More Related Content

Slideshows for you (20)

Viewers also liked (20)

Similar to Asynchronous design with Spring and RTI: 1M events per second (20)

Recently uploaded (20)