GlassFish v2.1
Clustering & Enterprise Manager
Alexis Moussine-Pouchkine
Didier Burkhalter
Sun Microsystems
1

Some vocabulary
• Cluster – a group a homogenous GlassFish instances
administered as a whole
• Load-Balancing – a strategy and implementation to
balance HTTP, IIOP, and other incoming requests.
Enough to provide service continuity
• High-Availability (HA) – replication of stateful data to
survive instance crash. All about session continuity

Dynamic Clustering with Project Shoal
• GMS with Project Shoal
http://shoal.dev.java.net
>
Dynamic clusters implemented with JXTA by default
>
Ease of use in cluster setup
>
Differentiates between a member that leaves the group
>
because of failure or for administrative reason
> Configurable
>Failure timeout detection
># of retries for failed member
>Heartbeat timeouts

In-Memory Replication
• What?
HTTP session state
>
Stateful EJB session state
>
Single Sign-On state
>
Container state (timers, …)
>
• How?
> Default is In-Memory replication with JXTA for application state
> Can also use HADB for 99.999% uptime

Clustering in GlassFish v2
JMX = Java Management Extensions

Memory Replication
Typical cluster topology
Instance 1 Instance 2
Instance 3 Instance 4
Machine 1 Machine 2

Memory Replication
Typical cluster topology Example: Maximize
Availability on 4 node
cluster on 2 machines
Instance 1 Instance 2
Instance 3 Instance 4
Machine 1 Machine 2

Memory Replication
Typical failover scenario
• Location transparency
> fail over request can go to any instance in the cluster
• 2 Cases: Failover request lands on
> Case 1: instance with replica data:
ownership taken, processing continues
> Case 2: instance without replica data
> instance sends broadcast request
> instance with replica data transfers data back to requester and deletes
its copy after an acknowledge
> JXTA makes this easy (propagation communication channels are
scoped within the “group” (i.e. the cluster members)

HTTP Session State Failover
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
W Load instance 1 instance 2
Balancer
S
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
instance 4 Cluster instance 3

HTTP Session State Failover
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
W Load instance 1 instance 2
Balancer
S
Case 1:
Instance has Servlet Servlet
replica data (HTTP Session) (HTTP Session)
replicas replicas
instance 4 Cluster instance 3

HTTP Session State Failover
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
W Load instance 1 instance 2
Balancer
S
Case 2:
Instance does not Servlet Servlet
have replica data (HTTP Session) (HTTP Session)
Broadcast…
replicas replicas
instance 4 Cluster instance 3

HTTP Session State Failover
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
W Load instance 1 instance 2
S Balancer
Case 2:
Instance does not Servlet Servlet
have replica data (HTTP Session) (HTTP Session)
and ownership
replicas replicas
transfer…
instance 4 Cluster instance 3

Cluster Dynamic Shape Change
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
W Load instance 1 instance 2
S Balancer
Shape Change
instance1 fails Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
instance 4 Cluster instance 3

Cluster Dynamic Shape Change
Servlet Servlet
(HTTP Session) (HTTP Session)
X
replicas replicas
W Load instance 1 instance 2
S Balancer
X
Shape Change
instance1 fails Servlet Servlet
(HTTP Session) (HTTP Session)
Instance2 and 4
see the failure
replicas replicas
instance 4 Cluster instance 3

Cluster Dynamic Shape Change
Servlet Servlet
(HTTP Session) (HTTP Session)
replicas replicas
W Load instance 1 instance 2
S Balancer
Shape Change
instance1 fails Servlet Servlet
(HTTP Session) (HTTP Session)
Instance2 and 4
see the failure
Instance4 selects replicas replicas
Instance2 as new
partner—new instance 4 Cluster instance 3
connections
established
the reverse happens when an instance joins or re-joins the cluster

Memory Replication Configuration
Our hope was to say…
• “This page left intentionally blank” ;-)
> Meaning “zero configuration required”
• We came close to that goal…

Memory Replication Configuration
Out of the box…
• Create a domain
> Use the ‘cluster’ profile – defaults for replication are handled
> Enables GMS—heartbeat enabled
> persistence-type = \"replicated\"
• Create a cluster and instances
• Deploy your application with availability-
enabled=true
• That’s it

Memory Replication Configuration
Making your app distributable
• <distributable/> element
> Required in web.xml
> Indicates you believe your application is ready to run in a cluster
• Serializable objects required
> HTTP Session state
> EJB technology Stateful Session Bean state

Configuring the availability service

Sun GlassFish Enterprise Server
Enterprise Manager
Customer
Advocate eLearning
Credit
Customer Focused
24x7 Support
Support Team
GlassFish
Open Source
Sun VIP
Patches &
Application Server
Interoperability
Upgrades
Support

Enterprise Manager
• SNMP Monitoring
• Performance Advisor
• Performance Monitor

Enterprise Manager
• Requires GlassFish Enterprise v2.1 subscription
• Download via SunSolve (www.sunsolve.com)
SNMP Monitoring: 140750-01
>
Performance Advisor: 140751-01
>
Performance Monitor: 140749-01
>
GlassFish v3: Available in Update Center soon
>
• Evaluation bundles in development

Enterprise Manager
SNMP Monitoring
• Leverage existing enterprise management tools
• SNMP v1, v2c adapter
> JSR 77/J2EE MIB
> Exposes JMX attributes via SNMP
• Deploys as a war file
• Can monitor instances through DAS or directly
• Configured via GlassFish properties
asadmin set standalone-instance1.property.snmp-adapter-port=10162 asadmin set
clustered-instance1.property.snmp-adapter-port=10163 asadmin set clustered-
instance2.property.snmp-adapter-port=1016

GlassFish SNMP Architecture

Enterprise Manager
Performance Advisor - Tuning
25

Enterprise Manager
Performance Advisor - Alerts
• Proactive notification of “events of interest”
Physical memory alert
>
Memory leak alert
>
CPU Usage trend alert
>
JVM Throughput alert
>
GC Pause Time alert
>
Log entry alert
>
• Alert notification methods
> Written to log file
> Optionally sent to email address(es)

Enterprise Manager
Performance Advisor - Alerts
27

Enterprise Manager
Performance Monitor
• Diagnose and monitor GlassFish and JVM performance
• Rich GUI application
> VisualVM distribution with integrated GlassFish plugin
• Chart performance over time
• Read-only tooling (vs. admin console)
• Local and (secure) remote monitoring

Enterprise Manager
Performance Monitor
• GlassFish observability
HTTP Service view
>
Web View
>
EJB View
>
JDBC View
>
Web Services View
>
Cluster View
>
Node Agent View
>
Pool monitoring (JDBC, EJB, MDB)
>
• Heap View
• GC View

Enterprise Manager
Performance Monitor
30

Questions ?
alexis.mp@sun.com
http://blogs.sun.com/alexismp
31