High Volume Web API Managementwith WSO2 ESB              Paul Fremantle            CTO and Co-Founder            Hiranya J...
WSO2 Offerings• WSO2 Carbon  •   On premise product platform  •   Consistent products sharing the same OSGi core• WSO2 Str...
ForresterWave™ ESB Q2 2011 - Leader
WSO2 ESB 4.0               © WSO2 2011
WSO2 ESB Features•   The WSO2 Enterprise Service Bus offers:    •   Easy configuration through an intuitive graphical inte...
API Management Concerns•   Slow clients, IP denial of service, traffic shaping•   Security, authentication, authorization•...
API Management Concerns•   Slow clients, IP denial of service, traffic shaping    •   Non-blocking transport, priority    ...
eBay Case Study – High Volume•   Requirements:    •   “The solutions we were using no longer met our needs, so we consider...
Memory Usage at eBay                       © WSO2 2011
Deployment             © WSO2 2011
Configuring for High Volume•   Operating System•   JVM•   ESB•   Management                              © WSO2 2011
Tuning the OS• Optimize core network settings for high  throughput network activity  •   /etc/sysctl.conf• Increase the nu...
Tuning the JVM• Allocate sufficient memory for the heap  •   -Xms256m -Xmx2048m -XX:MaxPermSize=256m• Consider reducing th...
Tuning the ESB• Configure transport thread pools  •   Configured through nhttp.properties file  •   IO dispatcher threads ...
Message Relay Mode• WSO2 ESB uses Apache AXIOM and the StAX  API for processing XML  •   XML payloads are streamed through...
Enabling Message Relay•   Enable the binary relay builder (axis2.xml)    •   Instructs the ESB to stream the incoming mess...
HTTP Relay Transport• Brand new in WSO2 ESB 4.0 release• Non blocking HTTP transport implementation  specially designed fo...
Performance              © WSO2 2011
Error Handling• In a large scale, high throughput  deployment, errors are not unusual – Expect the  unexpected• Configure ...
More on Error Handling• Configure sequences to clearly log errors and if  needed notify system administrators• In case the...
Monitoring• In general keep an eye on:  •   CPU usage  •   Memory usage  •   Thread counts  •   Fault counts  •   Latency/...
Monitoring Tools• Utilities Provided by OS  •   top  •   netstat• Mediation statistics• JMX Clients  •   Jconsole
Connections              © WSO2 2011
Endpoints            © WSO2 2011
Latency          © WSO2 2011
Listeners            © WSO2 2011
Memory and Garbage Collection                                © WSO2 2011
S2S Latency              © WSO2 2011
Threads          © WSO2 2011
Questions?             http://www.flickr.com/photos/oberazzi/                                                      30
Resources•   http://wso2.com/landing/ebay-the-esb-casestudy/•   WSO2 ESB 4.0 Download and Library    • http://wso2.org/lib...
Upcoming SlideShare
Loading in …5
×

High Volume Web API Management with WSO2 ESB

1,788
-1

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,788
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
72
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

High Volume Web API Management with WSO2 ESB

  1. 1. High Volume Web API Managementwith WSO2 ESB Paul Fremantle CTO and Co-Founder Hiranya Jayathilaka Associate Technical Lead
  2. 2. WSO2 Offerings• WSO2 Carbon • On premise product platform • Consistent products sharing the same OSGi core• WSO2 Stratos • A “Cloud-Enabled Application Platform” (CEAP) • Multi-tenant, Elastic, Metered/Billed deployment • An extension of the Carbon platform • Same SVN code tree• WSO2 StratosLive • http://stratoslive.wso2.com • A running Platform-as-a-Service (PaaS) • An instance of Stratos © WSO2 2011
  3. 3. ForresterWave™ ESB Q2 2011 - Leader
  4. 4. WSO2 ESB 4.0 © WSO2 2011
  5. 5. WSO2 ESB Features• The WSO2 Enterprise Service Bus offers: • Easy configuration through an intuitive graphical interface • Lean, high-performance design conserves resources through maximum efficiency. • Built on the WSO2 Carbon OSGi platform: adding new capabilities as you need them is a snap. • Integrates with your existing systems, with support for EDI, CSV, Files, FIX, HL7, SAP, JMS, etc• Asynchronous architecture supports very high loads with fixed resources• Routing, mediation and transformation are modular, extensible but provide most use-cases simply by sequencing built-in mediators• Full support for SOAP/WS-* and REST © WSO2 2011
  6. 6. API Management Concerns• Slow clients, IP denial of service, traffic shaping• Security, authentication, authorization• Monitoring, analytics and metrics• Mediation, format translation, micro-orchestration• Load-balancing, failover, routing• Unification of multiple stacks © WSO2 2011
  7. 7. API Management Concerns• Slow clients, IP denial of service, traffic shaping • Non-blocking transport, priority execution, throttle/cache, Message Relay Transport• Security, authentication, authorization • Authn/Authz, SAML2, OAuth, XACML PEP• Monitoring, analytics and metrics • Built-in metrics, JMX, WSO2 BAM Event Publisher• Mediation, format translation, micro-orchestration • JSON support, SOAP<->JSON, Service chaining• Load-balancing, failover, routing • Built in elastic load balancing, session-aware, failover• Unification of multiple stacks • Lightweight proxy layer, CSG support © WSO2 2011
  8. 8. eBay Case Study – High Volume• Requirements: • “The solutions we were using no longer met our needs, so we considered whether to build a new internal system or to adopt third-party technology”, Abhinav Kumar, eBay • Needed enhanced service mediation and orchestration capabilities • Scalability and performance to sustain increasing loads• Evaluation: • eBay evaluated several industry leading hardware and software ESB solutions, looking at open source, commercially licensed ESBs and hardware • Test production during 2009 holiday season proved that our system was rock solid • 1m messages/day with zero downtime, zero memory growth• Results • Shared-nothing architecture on RedHat Linux on x86 hardware • Currently running at more than 1 billion messages / day • Low resource utilization and flat memory usage• The benefits • “Using the WSO2 ESBs, weve been able to provide customers and partners with the quality experience they expect on eBay, even as our global customer base has grown,” Abhinav Kumar © WSO2 2011
  9. 9. Memory Usage at eBay © WSO2 2011
  10. 10. Deployment © WSO2 2011
  11. 11. Configuring for High Volume• Operating System• JVM• ESB• Management © WSO2 2011
  12. 12. Tuning the OS• Optimize core network settings for high throughput network activity • /etc/sysctl.conf• Increase the number of open file descriptors allowed by the OS • /etc/security/limits.conf
  13. 13. Tuning the JVM• Allocate sufficient memory for the heap • -Xms256m -Xmx2048m -XX:MaxPermSize=256m• Consider reducing the new ratio • -XX:NewRatio=n• Consider using the concurrent mark and sweep collector • -XX:+UseConcMarkSweepGC• Read more on JVM and GC tuning • http://wso2.org/library/articles/2010/11/taming- java-garbage-collector
  14. 14. Tuning the ESB• Configure transport thread pools • Configured through nhttp.properties file • IO dispatcher threads carry out network IO at the wire level • Recommended to have one IO dispatcher per CPU core • Server workers and client workers mediate the messages • More the merrier (But keep memory usage and load in mind)
  15. 15. Message Relay Mode• WSO2 ESB uses Apache AXIOM and the StAX API for processing XML • XML payloads are streamed through the ESB • Pull parsing model• But for pure routing, load-balancing and header- only mediation we can avoid even this step• Message relay mode works around this glitch and enables 100% pure streaming of messages • Works regardless of size and format of messages
  16. 16. Enabling Message Relay• Enable the binary relay builder (axis2.xml) • Instructs the ESB to stream the incoming messages through without touching their payloads• Enable the expanding message formatter (axis2.xml) • Allows the ESB to send messages that were received through the binary relay builder<messageBuilder contentType="application/xml" class="org.wso2.carbon.relay.BinaryRelayBuilder"/><messageFormatter contentType="application/xml" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>• Above configuration tells the ESB to process all application/xml messages in message relay mode.
  17. 17. HTTP Relay Transport• Brand new in WSO2 ESB 4.0 release• Non blocking HTTP transport implementation specially designed for streaming messages• Doesn’t require the binary relay builder and expanding formatter• To enable, simply uncomment the relevant entries in the axis2.xml file
  18. 18. Performance © WSO2 2011
  19. 19. Error Handling• In a large scale, high throughput deployment, errors are not unusual – Expect the unexpected• Configure endpoints to gracefully handle the common errors • Connection timeouts • Connection close/reset • Connection refused • HTTP protocol violations!!!!
  20. 20. More on Error Handling• Configure sequences to clearly log errors and if needed notify system administrators• In case the back end server fails to respond, send detailed fault responses to clients – Makes your application more appealing to the customers• Pay attention to HTTP error codes
  21. 21. Monitoring• In general keep an eye on: • CPU usage • Memory usage • Thread counts • Fault counts • Latency/Response time • Active connections
  22. 22. Monitoring Tools• Utilities Provided by OS • top • netstat• Mediation statistics• JMX Clients • Jconsole
  23. 23. Connections © WSO2 2011
  24. 24. Endpoints © WSO2 2011
  25. 25. Latency © WSO2 2011
  26. 26. Listeners © WSO2 2011
  27. 27. Memory and Garbage Collection © WSO2 2011
  28. 28. S2S Latency © WSO2 2011
  29. 29. Threads © WSO2 2011
  30. 30. Questions? http://www.flickr.com/photos/oberazzi/ 30
  31. 31. Resources• http://wso2.com/landing/ebay-the-esb-casestudy/• WSO2 ESB 4.0 Download and Library • http://wso2.org/library/esb• Message Relay Mode • http://wso2.org/project/esb/java/4.0.0/docs/messag e_relay.html• Twitter: @pzfreo• Blogs: • http://pzf.fremantle.org • http://techfeast-hiranya.blogspot.com/• Email: paul@wso2.com hiranya@wso2.com © WSO2 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×