High Volume Web API Management with the WSO2 ESB


Published on

In this webinar Paul Fremantle and Hiranya Jayathilaka describe how to use the WSO2 ESB to manage high volume Web APIs including REST and JSON

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

High Volume Web API Management with the WSO2 ESB

  1. 1. lean enterprise middleware<br />High Volume Web API Management<br />with WSO2 ESB<br />Paul Fremantle, CTO and Co-Founder<br />HiranyaJayathilaka, Associate Technical Lead<br />© WSO2 2011. Not for redistribution. <br />
  2. 2. WSO2 Offerings<br />WSO2 Carbon<br />On premise product platform<br />Consistent products sharing the same OSGi core<br />WSO2 Stratos<br />A “Cloud-Enabled Application Platform” (CEAP)<br />Multi-tenant, Elastic, Metered/Billed deployment<br />An extension of the Carbon platform <br />Same SVN code tree<br />WSO2 StratosLive<br />http://cloud.wso2.com<br />A running Platform-as-a-Service (PaaS)<br />An instance of Stratos<br />© WSO2 2011<br />
  3. 3. ForresterWaveTM ESB Q2 2011 - Leader<br />
  4. 4. © WSO2 2011<br />WSO2 ESB 4.0<br />
  5. 5. WSO2 ESB Features<br />The WSO2 Enterprise Service Bus offers:<br />Easy configuration through an intuitive graphical interface <br />Lean, high-performance design conserves resources through maximum efficiency.<br />Built on the WSO2 Carbon OSGi platform: adding new capabilities as you need them is a snap.<br />Integrates with your existing systems, with support for EDI, CSV, Files, FIX, HL7, SAP, JMS, etc<br />Asynchronous architecture supports very high loads with fixed resources<br /> Routing, mediation and transformation are modular, extensible but provide most use-cases simply by sequencing built-in mediators<br />Full support for SOAP/WS-* and REST<br />© WSO2 2011<br />
  6. 6. Key Features for High Volume Web API <br />Load Balancing and Failover built-in<br />Pluggable models can include affinity based LB<br />Full REST Support (XML and JSON)<br />Especially used in Mobile environments<br />Can bridge into SOAP or other enterprise systems<br />Message Relay Mode and Relay Transport<br />Excellent performance for 100% streaming<br />Priority Execution, Cache and Throttling<br />Manage workloads within the ESB as well as throttle backend<br />Cache content locally if appropriate<br />Service Chaining<br />Asynchronous lightweight orchestration<br />© WSO2 2011<br />
  7. 7. eBay Case Study – High Volume<br />Requirements:<br />“The solutions we were using no longer met our needs, so we considered whether to build a new internal system or to adopt third-party technology”, Abhinav Kumar, eBay<br />Needed enhanced service mediation and orchestration capabilities <br />Scalability and performance to sustain increasing loads<br />Evaluation:<br />eBay evaluated several industry leading hardware and software ESB solutions, looking at open source, commercially licensed ESBs and hardware<br />Test production during 2009 holiday season proved that our system was rock solid<br />1m messages/day with zero downtime, zero memory growth<br />Results<br />Shared-nothing architecture on RedHat Linux on x86 hardware<br />Currently running at more than 1 billion messages / day <br />Low resource utilization and flat memory usage<br />The benefits<br />“Using the WSO2 ESBs, we've been able to provide customers and partners with the quality experience they expect on eBay, even as our global customer base has grown,” Abhinav Kumar<br />© WSO2 2011<br />
  8. 8. Memory Usage at eBay<br />© WSO2 2011<br />
  9. 9. Deployment<br />© WSO2 2011<br />
  10. 10. Configuring for High Volume<br />Operating System <br />JVM<br />ESB<br />Management<br />© WSO2 2011<br />
  11. 11. Tuning the OS<br /><ul><li>Optimize core network settings for high throughput network activity
  12. 12. /etc/sysctl.conf
  13. 13. Increase the number of open file descriptors allowed by the OS
  14. 14. /etc/security/limits.conf</li></li></ul><li>Tuning the JVM<br /><ul><li>Allocate sufficient memory for the heap
  15. 15. -Xms256m -Xmx2048m -XX:MaxPermSize=256m
  16. 16. Consider reducing the new ratio
  17. 17. -XX:NewRatio=n
  18. 18. Consider using the concurrent mark and sweep collector
  19. 19. -XX:+UseConcMarkSweepGC
  20. 20. Read more on JVM and GC tuning
  21. 21. http://wso2.org/library/articles/2010/11/taming-java-garbage-collector</li></li></ul><li>Tuning the ESB<br /><ul><li>Configure transport thread pools
  22. 22. Configured through nhttp.properties file
  23. 23. IO dispatcher threads carry out network IO at the wire level
  24. 24. Recommended to have one IO dispatcher per CPU core
  25. 25. Server workers and client workers mediate the messages
  26. 26. More the merrier (But keep memory usage and load in mind)</li></li></ul><li>Message Relay Mode<br /><ul><li>WSO2 ESB uses Apache AXIOM and the StAX API for processing XML
  27. 27. XML payloads are streamed through the ESB
  28. 28. Pull parsing model
  29. 29. But for pure routing, load-balancing and header-only mediation we can avoid even this step
  30. 30. Message relay mode works around this glitch and enables 100% pure streaming of messages
  31. 31. Works regardless of size and format of messages</li></li></ul><li>Enabling Message Relay<br /><ul><li>Enable the binary relay builder (axis2.xml)
  32. 32. Instructs the ESB to stream the incoming messages through without touching their payloads
  33. 33. Enable the expanding message formatter (axis2.xml)
  34. 34. Allows the ESB to send messages that were received through the binary relay builder</li></ul><messageBuildercontentType="application/xml" class="org.wso2.carbon.relay.BinaryRelayBuilder"/><br /><messageFormattercontentType="application/xml" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/><br /><ul><li>Above configuration tells the ESB to process all application/xml messages in message relay mode.</li></li></ul><li>HTTP Relay Transport<br /><ul><li>Brand new in WSO2 ESB 4.0 release
  35. 35. Non blocking HTTP transport implementation specially designed for streaming messages
  36. 36. Doesn’t require the binary relay builder and expanding formatter
  37. 37. To enable, simply uncomment the relevant entries in the axis2.xml file</li></li></ul><li>Error Handling<br /><ul><li>In a large scale, high throughput deployment, errors are not unusual – Expect the unexpected
  38. 38. Configure endpoints to gracefully handle the common errors
  39. 39. Connection timeouts
  40. 40. Connection close/reset
  41. 41. Connection refused
  42. 42. HTTP protocol violations!!!!</li></li></ul><li>More on Error Handling<br /><ul><li>Configure sequences to clearly log errors and if needed notify system administrators
  43. 43. In case the back end server fails to respond, send detailed fault responses to clients – Makes your application more appealing to the customers
  44. 44. Pay attention to HTTP error codes</li></li></ul><li>Monitoring<br /><ul><li>In general keep an eye on:
  45. 45. CPU usage
  46. 46. Memory usage
  47. 47. Thread counts
  48. 48. Fault counts
  49. 49. Latency/Response time
  50. 50. Active connections</li></li></ul><li>Monitoring Tools<br /><ul><li>Utilities Provided by OS
  51. 51. top
  52. 52. netstat
  53. 53. Mediation statistics
  54. 54. JMX Clients
  55. 55. Jconsole</li></li></ul><li>Connections<br />© WSO2 2011<br />
  56. 56. Endpoints<br />© WSO2 2011<br />
  57. 57. Latency<br />© WSO2 2011<br />
  58. 58. Listeners<br />© WSO2 2011<br />
  59. 59. Memory and Garbage Collection<br />© WSO2 2011<br />
  60. 60. S2S Latency<br />© WSO2 2011<br />
  61. 61. Threads<br />© WSO2 2011<br />
  62. 62. Questions?<br />28<br />http://www.flickr.com/photos/oberazzi/<br />
  63. 63. Resources<br />http://wso2.com/landing/ebay-the-esb-casestudy/<br />WSO2 ESB 4.0 Download and Library<br />http://wso2.org/library/esb<br />Message Relay Mode<br />http://wso2.org/project/esb/java/4.0.0/docs/message_relay.html<br />Twitter: @pzfreo<br />Email: paul@wso2.comhiranya@wso2.com<br />© WSO2 2011<br />