This document discusses various techniques for optimizing application performance, including reducing latency and increasing throughput. It covers strategies like using data structures like linked lists, bloom filters, and Merkle trees efficiently. Other topics include removing contention through approaches like the disruptor pattern, optimizing for network performance, and leveraging the reactor pattern. Performance of transports like XML/JSON and SOAP/REST is also evaluated. Monitoring tools like Java Flight Recorder are also mentioned.

1. Make it fast for everyone
Performance and Middleware design
S. Suhothayan
Technical Lead, WSO2 Inc.

2. Optimising Applications
for
Performance

3. Latency vs Throughput
Latency measures the end-to-end time processing time. In a
messaging environment, teams determine latency by
measuring the time between sending a request and receiving
the response. Latency is measured from the client machine
and includes the network overhead as well.
Throughput measures the amount of messages that a server
processes during a specific time interval (e.g. per second).
Throughput is calculated by measuring the time taken to
processes a set of messages.

4. Latency vs Throughput
http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/

5. WSO2 CEP

6. Data structures
o Linked List vs Array List vs Map
CEP window processor :
o Average temperature of last 10 min.
o Inmemory
o Average temperature of last 24 hour.
o Disc Based implementation

7. Sactificing Accuracy for
Performance (Bloom filters)
http://lkozma.net/blog/sketching-data-structures/
Initial configuration : 0000000000
Insert("Sally") : 0100000001
# h1("Sally") = 2, h2("Sally") = 10
Insert("Jane") : 1110000001
# h1("Jane") = 1, h2("Jane") = 3
Insert("Mary") : 1110100001
# h1("Mary") = 5, h2("Mary") = 2 [collision]
Query("Sally")
# bits 2 and 10 are set,
# return HIT
Query("John")
# h1("John") = 10 set, but h2("John") = 4 not set
# return MISS
Query("Bob")
# h1("Bob") = 5 set, h2("Bob") = 1 set
# return HIT (false positive)

8. Syncing Nodes (Merkle tree)
Working with untrusted peer to peer networks (Torrents,
Cassandra ring)
hash 0 = hash( hash 0-0 + hash 0-1 )
http://en.wikipedia.org/wiki/Merkle_tree

9. Improve performance by
Multi threading
Use correct number of threads too much or too
little is not good.
Scaling
Vertical : Improve Throughput reduce Latency
Horizontal : Improve both Throughput & Latency
Commonly used data structure :
ArrayBlockingQueue

10. Removing contingency (disruptor)
https://lmax-exchange.github.io/disruptor/

11. Removing contingency (disruptor)
o Minimise thread contention
o Maximise the use of single threaded algorithms and data
structures.
https://lmax-exchange.github.io/disruptor/

12. Optimising Performance
over Network

13. Blocking vs Non Blocking
o Handling of thousands of connections
simultaneously (significant number of
connections may be in idle state as well)
o Handling high latency connections
o Request/response handling needs to be
decoupled
o Minimize latency, maximize throughput and
avoiding unnecessary CPU cycles

14. WSO2 ESB
Routing: Header based, content based, rule-based and priority-
based routing
Mediation: EIPs (including scatter/gather, message filters,
recipient list, dead-letter channels, guaranteed delivery and
message enrichment), database integration, event publishing,
logging & auditing, validation
Transformation: XSLT 1.0/2.0, XPath, XQuery, Smooks

15. Reactor Pattern
Uses Readiness selection: It is the ability to choose a socket that will not block when
read or written. This is a very efficient way of handling thousands of concurrent
clients and scales well.
http://wso2.com/library/articles/2013/12/demystifying-wso2-esb-pass-through-transport-part-i/

16. Message Flow

17. NHTTP

18. Message Relay
Why we need to read the whole message if the
necessary info is only at the message headers?
Read only the headers and stream the message
as bytes !

19. Passthrough Transport
Going beyond design patterns & conventions

20. Performance Comparison

21. XML vs JSON & SOAP vs REST
XML vs JSON
o Serialization and Deserialization Performance
o Data Storage
o Data Transfer
o Interoperability
SOAP vs REST
o REST has minimal overhead on top of HTTP
o REST easy to scale don't have any server side sessions
o REST Provides easier interface for client
o SOAP is better when it comes to security
o SOAP has better tools for schema, code generation and
validation

22. Binary Transports
(Thrift, Protocol buffers)
http://ganges.usc.edu/pgroupW/images/a/a9/Serializarion_Framework.pdf

23. Binary Transports
(Thrift, Protocol buffers)
http://ganges.usc.edu/pgroupW/images/a/a9/Serializarion_Framework.pdf

24. Monitor all times (Java Flight Recorder)

25. Monitor all times (Java Flight Recorder)

26. Monitor all times (Java Flight Recorder)

27. Contact us !