This document summarizes deployment patterns for scalably deploying WSO2 API Manager. It discusses components like the API Store, Gateway, Publisher, and Key Manager. It then covers single-server, distributed, zonal, and partially distributed deployment patterns. Specific patterns discussed include integrating with a workflow engine, enabling statistics collection, high availability of statistics, and multi-data center deployments. The document also discusses horizontally and vertically scaling deployments, an API facade pattern, and deploying on Amazon EC2. It concludes with recommendations for tuning performance.

1. Scalable Deployment Patterns in
API Manager
Sanjeewa Malalgoda
Senior Software Engineer
Uvindra Dias Jayasinha
Senior Software Engineer
Tuesday, February 11, 2014

2. *
About the Presenter
๏ Sanjeewa joined WSO2 in September
2010. He is senior software engineer,
focusing on WSO2 API Manager. He is
core contributor for API Manager, tenant
aware load balancer and stratos.
๏ Uvindra joined WSO2 in November 2013.
He is a Senior Software Engineer in the
WSO2 API Manager Team.

3. *
About WSO2
๏ Global enterprise, founded in 2005 by
acknowledged leaders in XML, web
services technologies, standards and
open source
๏ Provides only open source platform-as-a-
service for private, public and hybrid
cloud deployments
๏ All WSO2 products are 100% open
source and released under the Apache
License Version 2.0.
๏ Is an Active Member of OASIS, Cloud
Security Alliance, OSGi Alliance, AMQP
Working Group, OpenID Foundation
and W3C.
๏ Driven by Innovation
๏ Launched first open source API
Management solution in 2012
๏ Launched App Factory in 2Q 2013
๏ Launched Enterprise Store and
first open source Mobile solution
in 4Q 2013
๏

4. *
What WSO2 delivers

5. *
Deployment Patterns
○ Understanding key components and data model.
○ Single-server deployment
○ Distributed deployment.
How to deploy across zones.
Message flows of sample deployment.
○ Integration with workflow engine.
○ Enabling statistics for API Manager.
○ Multi data center deployment.
○ Deployment with online/offline monitoring.
○ How to scale up and scale out.
○ API Façade deployment pattern.
○ Amazon EC2 deployment.
○ Fine tune

6. Understanding Key Components
*
API Store
This component provides a space for consumers to self-register, discover
API functionality, subscribe to APIs, evaluate them, and interact with API
publishers.
API Key Manager Server
This component is responsible for all security and key-related operations.
When an API call is sent to the Gateway, it calls the Key Manager server and
verifies the validity of the token provided with the API call.

7. Understanding Key Components
*
API Gateway
This component is responsible for securing, protecting, managing, and
scaling API calls. The API gateway is a simple API proxy that intercepts API
requests and applies policies such as throttling and security checks.
API Publisher
This component enables API providers to easily create, publish their APIs,
share documentation, provision API keys, and gather feedback on API
features, quality, and usage.

8. *
Data Storages
User Manager Database
Stores information related to users and user roles. This information is
shared among the Key Manager Server, Store, and Publisher.
API Manager Database
Stores information related to the APIs along with the API subscription
details.
Registry Database
Used to store API metadata and other information. Shares information
between the Publisher and Store.
Default carbon database embedded with API Manager binary distribution
will contain all tables necessary for registry and user management.

9. *
Single-server deployment
Key Manager Key Manager Store Publisher
WSO2CARBON_DB WSO2AM_DB
9443
9763
8280
8243

10. *
Configurations
○ To edit API manager related configurations you need to edit api-manager.
xml file located at /repository/conf directory of product
distribution. To learn about configurations refer this document.
○ To configure data sources and database related configurations you
need to edit master-datasources.xml file available at
/repository/conf/datasources directory.
01. WSO2_CARBON_DB,
02. WSO2AM_DB,
03. WSO2AM_STATS_DB
○ Configurations related to user management are available at user-mgt.
xml file in /repository/conf directory(add, edit user store etc.).
○ Configurations related to registry are available at registry.xml file in
/repository/conf directory(add, edit user store etc.).

11. *
Distributed Deployment

12. *
Zonal Deployment
Deployment Across zones
DMZ

13. *
Deployment Pattern #2 - Data Flow
BAM Key
API Creation
Publisher +
Store
Publisher +
Store
Gateway
Manager
Gateway
Worker
Manager
Gateway
Worker
Key
Manager
Internal
ELB
External
ELB

14. Deployment Pattern #2 - Data Flow
*
API Invocation
Gateway
Worker
Gateway
Worker
Gateway
Manager
Publisher +
Store
Publisher +
Store
External
ELB
Internal
ELB
Key
Manager
Key
Manager
BAM

15. Deployment Pattern #2 - Data Flow
*
Login to publisher/ store
Publisher +
Store
Publisher +
Store
Gateway
Worker
Gateway
Worker
Gateway
Manager
External
ELB
Internal
ELB
Key
Manager
Key
Manager
BAM

16. Deployment Pattern #2 - Data Flow
*
Gateway
Manager
Gateway
Worker
Statistics collection and presentation
Gateway
Worker
Publisher +
Store
Publisher +
Store
External
ELB
Internal
ELB
Key
Manager
Key
Manager
BAM

17. *
Partially Distributed
Deployment

18. *
Partially Distributed
Deployment
Minimum fully distributed API Manager deployment will have 1
gateway, 1 key manager, 1 store and 1 publisher(all together 4 nodes).
But when you need high availability for each server type we might need
2 or more instance per one server type.
2 nodes for combined API store/publisher(api store and publisher
should run on same instance). These 2 nodes should be clustered for
single cluster domain. And load balancer should route all requests
comes with /store or /publisher contexts to these nodes(9443 and
9763).
2 nodes for combined API gateway/key manager(API gateway and /key
manager should run on same instance).

19. Pattern #3 - Workflow Integration
*

20. *
Pattern #4 - Enabling Statistics

21. *
Pattern #5 - HA for Statistics

22. *
Pattern #6 - HA for Statistics -
Multiple Data Centers

23. *
Pattern #7 - Online/Offline
Statistics

24. *
Scaling
○ Most of the time we need to scale gateway and key manager as it hits
large number of requests
○ To scale horizontally means to add more nodes to a system, such as
adding a new node to a gateway cluster.
○ To scale vertically means to add resources to a single node in a system,
typically involving the addition of more computing power.

25. *
Scale Horizontally
○ To scale horizontally means to add more nodes to a system, such as
adding a new node to a gateway cluster.
○ If we take gateway as an example, we can add more nodes to gateway
cluster on demand.
○ Then load balancer should be detect newly added nodes and route
requests to them.
○ This is the most common and widely used pattern of scaling API
Manager deployments .
○ We can apply this pattern to gateway, key manager, store and
publisher.

26. *
Scale Horizontally
500 requests 500 X 4 requests
Scale out

27. *
Scale Vertically
○ Scale vertically means add more resources to a single node in a system,
typically involving the addition of more computing power, RAM and
etc.
○ Implementation of WSO2 API Façade facilitate by using WSO2 API
Manager to build Demand and Façade layer, WSO2 ESB to build
mediation layer and connect to the existing services.
○ In API Manager we can add mediation logic, service chaining, message
formatting to API definition.
○ With API Façade pattern mediation, transformation logic will handled
by ESB layer and it will add additional processing power to API
management layer.

28. *
Scale Vertically
Scale up
API authentication
Message mediation
Service chaining
API authentication
Message mediation
Service chaining
Processing power 3
unit per request
Processing power
1 unit per request
Processing power 2
unit per request

29. *
Pattern #8 - API Façade pattern

30. *
Deploying on Amazon EC2
Using pre-built AMI
○ Switch to the US East (N. Virginia) region Use AMI ID: ami-db182fb2.
○ An instance with 1.7 Gig of memory is enough for testing purposes.
Otherwise use at least a Medium size instance.
○ Create a security group opening ports 22 (SSH), 9443, 9763 (tcp, API
Mgr admin console), 8280 (tcp, API GW ports).
○ Start an instance and attach security group to it.

31. *
Deploying on Amazon EC2
Standard deployment with binary distribution
M1-Medium instance to run one Carbon instance.
Note : based on the I/O performance of instance, it is recommended to
run multiple instances in a Large instance (m1.large or m1.xlarge).
Deploy API manager on EC2 instances. If you have enough resources you
can run multiple API manager instances with port offset.
If you need to create DMZ for gateway and store there are several
options for that. One method is create VPC to isolate deployment also
you can create security group and apply same security group to all nodes
in same zone(here DMZ).

32. *
Tune up
Enable response caching.
Enable caching at API Gateway or Key Manager Server
(repository/conf/api-manager.xml).
<EnableGatewayKeyCache>false</EnableGatewayKeyCache>
<EnableKeyMgtValidationInfoCache>true</EnableKeyMgtValidationInfoCache>
Enable JWT caching(repository/conf/api-manager.xml).
<EnableJWTCache>true</EnableJWTCache>
Enable oauth key caching(repository/conf/identity.xml).
<EnableOAuthCache>true</EnableOAuthCache>
For store UI improvement we can enable recently added API cache and tag cache.
Memory allocated for the server increase by modifying /bin/wso2server.sh
Default setting : -Xms256m -Xmx512m -XX:MaxPermSize=256m.
New setting : -Xms2048m -Xmx2048m -XX:MaxPermSize=1024m

33. *
Tune up
NHTTP and PT HTTP transports(If nhttp /repository/conf/nhttp.properties).
http.socket.timeout=60000
snd_t_core=200
snd_t_max=250
snd_io_threads=16
lst_t_core=200
lst_t_max=250
lst_io_threads=16
Axis2Client.xml
<parameter name="defaultMaxConnPerHost">1000</parameter>
<parameter name="maxTotalConnections">30000</parameter>
Set open files limit to 200000 by editing /etc/sysctl.conf(in linux)
sudo sysctl -p

34. *
Tune up
masterdatasource.properties
<maxActive>250</maxActive>
<testOnBorrow>false</testOnBorrow>
<validationInterval>120000</validationInterval>
Mysql maximum connections
mysql> show variables like "max_connections";
max_connections was 151
set to global max_connections = 250;
repository/conf/tomcat/catalina-server.xml
maxThreads="750"
minSpareThreads="150"
disableUploadTimeout="false"
enableLookups="false"
connectionUploadTimeout="120000"
maxKeepAliveRequests="600"
acceptCount="600"

35. *
Call to action page
๏ Product documentation home page(http://docs.wso2.
org/display/AM170/WSO2+API+Manager+Documentation)
๏ Article on fine tuning API Manager(http://sanjeewamalalgoda.
blogspot.com/2014/01/wso2-carbon-420-based-products-general.
html)
๏ Product home page(http://wso2.com/products/api-manager/)

36. *
Business Model

37. *
Q & A

38. Contact us !