Deploying Tomcat Clusters in an Advanced Blackboard TM Environment Speaker: Steve Feldman Director, Software Performance Engineering and Architecture [email_address]
Agenda and Introductions
Goals, Objectives and Outcomes
Hands on Learning
Total Time: 50 Minutes
Note to Attendees
Tomcat clustering is not a standard configuration of the Blackboard Academic Suite.
Clusters have been set-up and tested by Blackboard Engineers, but have not been certified as a valid configuration.
The technology is proven and used throughout the world.
The same can be said of server virtualization
Many Blackboard clients use and deploy virtual servers.
The guidance provided in this presentation is to help clients take advantage of the power of clustering, but help clients understand that they must have knowledge on setting up and managing clusters.
The goals of this presentation are:
Introduce the concepts and techniques for planning and deploying Tomcat Application Clusters.
Help administrators understand the value of Tomcat clusters for performance and availability.
Show administrators how to set-up their very own Tomcat cluster.
Define what are clusters.
Define what are nodes.
Explain the differences between cluster nodes and load balanced nodes.
Review the configuration requirements for setting up a Tomcat cluster
Review the configuration requirements for setting up web server distribution to the cluster.
Explain issues that might come up with clusters and provide techniques for diagnosing and resolving the issues.
Convince every customer to go home and start setting up clusters in their development environments.
At the end of the session, administrators will be able to do the following:
Define the difference between a load-balanced configuration and a tomcat cluster.
Set-up a Tomcat cluster or a load-balanced nodes
Configure clustering for quality of service.
What is a Tomcat Cluster?
A cluster is a collection of two or more Tomcat instances running in parallel and working together to provide robust scalability, performance and reliability.
Clusters can be multi-homed and reside physically on the same hardware.
Clusters can be distributed and reside on distributed hardware.
Clusters are indistinguishable to end users presenting a seamless experience
Clusters distribute workload
Minimize unnecessary queuing
When configured for a small JVM it can reduce the amount of JVM pause times.
Clusters can help better utilize the hardware you run on
Clusters enable a scale-up model to go along with load-balanced installations (scale-out model)
Clusters can provide additional availability
Not recommend using Tomcat Clusters as the primary failover solution
Clusters are cool…and because they are cool you should consider them!
Just one person’s opinion!
Visual Look at a Tomcat Cluster Traffic is directed by the Web Server like a software load balancer Nodes know little about there partners Recommend 1 node Per Core Web Server Instance Node 1 Node 2 Node N
What is an Application Node?
An application node is a single instance of Tomcat.
Nodes can be part of a cluster.
Means they are aware of each other
Part of a group
Nodes can be part of a load-balanced configuration.
A single physical server can have multiple logical nodes not in a cluster, but technically load-balanced by JK
Clustered Nodes versus Load-Balanced Nodes
A load-balanced node is the following:
Physically distinct from a hardware perspective.
Has its own web server managing traffic.
Has a dedicated configuration and resource definition.
Has no understanding of failover definition.
Relies on third-party resources like load-balancers.
A clustered node is the following:
Can be physically or logically distinct from a hardware perspective.
Shares a web server with potential N+ nodes.
Shares configuration and resource definition with other nodes.
Has failover definition as part of the application configuration.
Load-Balanced and Clustered Nodes Hardware or Software Load Balancer Cluster Load-Balanced Nodes unaware of each other. Nodes aware of each other. Web Server Instance Node 1 Node 2 Node N Web Server Instance Node 1 Node 2 Node N
To enable session replication in Tomcat, three different paths can be followed to achieve the exact same thing:
Using session persistence, and saving the session to a shared file system (PersistenceManager)
Using session persistence, and saving the session to a shared database (JDBCManager)
Using in-memory-replication, using the SimpleTcpCluster that ships with Tomcat 5 (server/lib/catalina-cluster.jar)
Recommend Using SimpleTcpCluster
Guidelines for Setting Up a Cluster
The cluster configuration is completely described in the server.xml file under $BLACKBOARD_HOME/apps/tomcat/conf
You can set-up a true cluster or settle for load-balanced JVMs.
Either way you win.
Create multiple clusters in a multi-homed configuration initially
Each multihomed cluster is capable of failing over to each other rather then across physical systems
Eliminates the need for cross-over connections between servers.
Copy $BLACKBOARD_HOME/apps/tomcat to $BLACKBOARD_HOME/apps/tomcat_node2
Change all references of $BLACKBOARD_HOME/apps/tomcat to now use $BLACKBOARD_HOME/apps/tomcat_node2
Modify server.xml in new directory to reflect unique server port for AJP13 connector and shutdown.
Cluster ports need to be unique in order to create a socket unless IP Addresses are defined and uniquely defined.
Modify CATALINA_PID in blackboard-tomcat.sh or .bat file to point to the right node and log file directory.
Setting Up JK Load Distribution
Apache and IIS will route traffic based on JK definition.
Best to set-up a new workers.properties
Located under $BLACKBOARD_HOME/apps/tomcat/conf/jk/workers.properties
Change httpd.conf to point to new workers.properties (Unix)
Managed from within IIS
Blackboard Web Site Jakarta
Simply copy workers.properties and rename old file
Example Workers.Properties # One virtual worker worker.list=loadbalancer # Configure the virtual worker as a load balancer worker.loadbalancer.type=lb worker.loadbalancer.balanced_workers=tomcat1,tomcat2 worker.loadbalancer.method=Request # Attempt to worker.loadbalancer.sticky_session=1 # Actual workers, not available directly worker.tomcat1.port=8009 worker.tomcat1.host=localhost worker.tomcat1.type=ajp13 worker.tomcat1.lbfactor=50 worker.tomcat2.port=8209 worker.tomcat2.host=localhost worker.tomcat2.type=ajp13 worker.tomcat2.lbfactor=50
Web Server Configuration
Apache manages Tomcat worker information in httpd.conf
Establish JK Mount (Bottom of File)
Probably called new worker list “loadbalancer”
Windows manages Tomcat worker information un uriworkermap.properties
Must note new worker name for Cluster (aka: worker list)
Things to Consider
Apache has a limit of 1024 maxclients
More then enough for a 2, 3 or 4 node cluster
Must always keep maxclients = (Tomcat Max Server Processes) * (# of Tomcat Nodes per Cluster)
Recompiling Apache is one way to solve the problem
Advanced Feature: Setting Up QOS Models Change JKMount **JK Mount says /webapps/* but can be changed to any single webapp if you choose.