[API Word 2021] - Quantum Duality of “API as a Business and a Technology”
Maximize Messaging and Performance and Lowering Infrastructure Footprint
1. Maximize Messaging
Performance and
Lowering Infrastructure
Footprint
Senior Software Architect, WSO2 Inc.
Visiting Faculty, University of Moratuwa
Member, Apache Software Foundation
Research Scientist, Lanka Software Foundation
2. Outline
Goals: Optimizing the deployments
o Message performance
o Reduce footprint
Design and Capacity Planning
Design for Performance and Minimal footprint
o Select right transport
o Minimize footprint and maximize sharing
- MT
- Using Carbon
o Tuning
Wrapping Up
5. Outline on Capacity planning
We discussed this in the Asanka’s webinar last
week
Rough task list
o Design an architecture
o Estimate expected capacity of the who system
o Find the capacity for each type of node
o Calculate the number of nodes required
o Build a POC and verify
7. Our Goal
Building smaller systems saves lot of money
and effort
o Maximizing the performance
o minimizing the footprint
This webinar explains how to capitalize WSO2
products to that end
8. How?
WSO2 Platform provides several techniques to
optimize deployments
Some of the techniques
o Selecting the right transport
o New performance improvements to ESB
o Use carbon to create minimal servers
o Use Multi-tenancy to maximize sharing
o Tuning your setup
9. Select the right Message Transport
WSO2 platform provide several choices for your
system to communicate
o SOAP
o HTTP/JSON
o Thrift
o AMQP
o Others
- XMPP
- TCP
- Mail/ SMTP
- FTP
- Domain specific formats like FIX
10. Message Transports
Performance Interopera Reliability Security Asynchron
(2 core, 4G bility ous
with 4-10K messaging
messages )
TPS
SOAP 2K Very good Has WS- ?
transactions Security/SSL
JSON+HTT 5k Good X SSL X
P
Thrift 20K-100K OK, but not X SSL X
widely
adopted
AMQP 1K OK within Has SSL Yes
JMS transactions
systems and
persistent
messaging
11. Select the right Message Transport
Several concerns
o Performance
o Interoperability
o Support for Add-On features like transactions,
reliability, security
o Ease of use
o Asynchronous nature
Use what is enough
o E.g. internal communication can use thrift while
external communication can use HTTP/JSON for
SOAP
13. Data Bridge
Highly optimized event collection framework
o Many agents can emit events using the data bridge
o They are collected and processed as receivers
o Use by WSO2 CEP and WSO2 BAM
Very Fast
o Can do few 100k events per second
o Uses Apache thrift
o Asynchronous
o Support buffering and batch delivery
o Data formats are predefined and transferred as
tuples
14. Minimize the Product Footprint
What do we mean by foot print?
o Product size (disk space, download)
o Memory foot print
o Unused functionality
Two main ideas
o Remove what is not needed
o Maximize sharing
WSO2 platform provides 2 choices to minimize
the footprint
o Use carbon to build a minimal Product
o Use Multi-tenancy to maximize sharing
15. WSO2 Carbon
OSGI based
components
runtimes
Products are
created by
composing
components
o Read “Carbon: towards a server building
framework for SOA platform” for more details.
16. Build Minimal Product
So you can customize and create a minimal
product by adding and removing components
E.g. We were able to make WSO2 IS with 64MB heap this
way for a given scenario.
In smaller load cases, you can combine
multiple products to a one product
o E.g. Add service hosting to BPS
17. What is Multi-tenancy ?
Many Parties share the same set of resources,
while giving each one his own space
18. Why Multi-tenancy? 1. Increased sharing
• Cloud shares
resources across a
large pool of users.
• Now sharing
happens in the
application level as
oppose to sharing at “There is no delight in
owning anything unshared.”
OS level for multiple Seneca (Roman philosopher, mid-1st
century AD)
processes and
sharing at HW level
with VMs.
• That can bring
greater savings
photo by Ben Gray on Flickr,
http://www.flickr.com/photos/ben_grey/4582294721/, Licensed under CC
19. Why Multi-tenancy? 2. Provide “pay for what you
use”
• Often there will be many accounts
in a PaaS or a SaaS, but only a
fraction of them will be in use.
• We cannot allocate runtime
resource per account (disk may be
ok, as it is cheap). For example,
we cannot run a VM per account.
• By sharing the same server with
many users, Multi-tenancy provides
much reduced runtime cost per
server.
20. How does it Help?
• All WSO2 products support
• multi-tenancy
• Running in the cloud in pay as you go fashion
• So you can share same physical server across
many (e.g. departments) logically
• Or you can outsource some parts of the design
to a PaaS (e.g. Stratos Live)
24. ESB Passthrough and Streaming XSLT
ESB request = message + headers
If mediation logic only depends on headers, we
skip message building
o Binary relay – we copy the message as bytes from
input side to output side
o Pass-through – we copy messages at NIO buffer
level
If mediation logic does not edit the message,
but read the message, we build the message,
but write the incoming bytes out as is to avoid
serialization.
26. Complex Event Processor 2.0
Now CEP server 2.0 comes with a new runtime
called Siddhi
Very fast
o About 2.5M events/sec with java events
o About 250k events/sec over data bridge
Supports Distributed Cache and Persistence for
HA
More details from
o http://wso2.com/products/complex-event-
processor/
27. Tuning
OS Level Tuning
o For example,
http://www.lognormal.com/blog/2012/09/27/linu
x-tcpip-tuning/
o Generally available with the product
o It depends on OS etc.
Product Tuning
o Thread pools sizes mainly, there may be others
Application Tuning
o This is your application, it is good idea to do a
profile and see.
29. What we expect from Tuning
Goals
o Max Throughput
o Within Given Latency
How do we know we are at Max?
o Does resource consumption is high?
o What are the resources
- CPU
- Memory
- IO (Network and Disk)
o At last one should max out
o E.g. if you have too many locks, all above 3 may be
low and you get sub-optimal throughput
We fix problems via profiling
o Detective time!!
30. Profiling
First step is connect through JConsole, and get
basic idea.
Also monitor the load average, CPU usage, and
IO operations (via top or Sar)
Then you can switch to tools like JProfiler,
Yourkit
Important views
o CPU views (bottlenecks)
o Thread views and monitor views
o Memory views (GC, allocations, and memory leaks)
o IO and JDBC tracing
33. Conclusion
Goals: Optimizing the deployments
o Message performance
o Reduce footprint
Design and Capacity Planning
Design for Performance and Minimal footprint
o Select right transport
o Minimize footprint and maximize sharing
- MT
- Using Carbon
o Tuning
We also discussed some of the new
performance improvements.