Your SlideShare is downloading. ×
Mobicents Summit 2012 - Vladimir Ralev - Mobicents Load Balancer and High Availability
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Mobicents Summit 2012 - Vladimir Ralev - Mobicents Load Balancer and High Availability

1,136
views

Published on

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,136
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. HA and SIP Load Balancing Design documentation Revised: Sep, 2012
  • 2. Important!● HA here doesnt imply replication.● HA quality units ○ one nine, two nines, three nines and so on ○ can claim them without any replication ■ its not even cheating● It is sufficient to partition your calls onto a set of machines ○ The SIP protocol doesnt mind address changes ○ Can go into production against any production-ready phone
  • 3. Deployment Scenarios Overview● Pure IP load-balancing ○ No SIP-based affinity, only IP-based affinity ○ Might violate some SIP rules ○ Not recommended● Standalone SIP-based load-balancing ○ Provides SIP-based affinity ○ Provides SIP protocol compliance● Distributed load balancing ○ IP load balancer in front ○ Multiple SIP load balancers at the back end ○ Eliminates the Single Point of Failure problem ○ Scales better when SIP LB capacity is exceeded● Cooperative load balancing with HTTP (integrated, mod_jk and mod_cluster)
  • 4. Deployment Scenarios Overview● Pure IP load-balancing ○ No SIP-based affinity, only IP-based affinity NOT ENTIRELY BAD ○ Might violate some SIP rules ○ Not recommended● Standalone SIP-based load-balancing ALMOST USELESS ○ Provides SIP-based affinity ○ Provides SIP protocol compliance● Distributed load balancing ○ IP load balancer in front ○ Multiple SIP load balancers at the back end ○ Eliminates the Single Point of Failure problem ○ Scales better when SIP LB capacity is exceeded● Cooperative load balancing with HTTP (integrated, mod_jk and mod_cluster)
  • 5. Deployment Scenarios Overview● New this year!● DNS load balancing ○ No SIP message affinity ○ Affinity is temporary per UA by TTL ○ No built-in heartbeats with the SIP servers - must do it on your own, with a module or by the UA ○ You better try to keep all your IPs up ■ Use IP takeover for fast recovery
  • 6. Two kinds of DNS load balancing ● Dynamic record DNS ○ Round-Robin DNS ● DNS SRV ○ Built-in load balancing with statistical weights ○ Requires support from the SIP phones (very common)_service._proto.name TTL class SRV priority weight port targetFor instance:_sip._tcp.example.com. 86400 IN SRV 0 5 5060 sipserver.example.com._sip._tcp.example.com. 86400 IN SRV 0 5 5060 sipserver.example.com.
  • 7. Deployment Scenarios Overview● New this year!● Dedicated IP load balancing by SIP headers ○ Equivalent to DNS SRV without the need of phone support
  • 8. Pure IP load balancing● Low priority● Not very useful - just to clarify theoretically ○ Routes based on IP/UDP/TCP (Layer 3/4) fields - IP address, source or destination port, etc ○ Can not make routing decisions based on SIP messages (BigIP F5, etc are exception) ○ SIP AS-initiated requests and responses hard to route ○ IP LB is not a SIP entity.
  • 9. Pure IP load balancing (conted)Retransmissions ● UDP ○ Retransmissions will get sprayed randomly ○ Forks and race conditions ● TCP ○ Works fine ■ Unless the TCP connection fails and the new connection ends up on new node
  • 10. Via headers in IP load balancing● Via headers should contain the original node address, not the address of the IP load balancer. Otherwise the SIP phones will follow the SIP spec and route responses through the IP load balancer, creating additional traffic.● Via headers are per-transaction. Mid-transaction fail-over is not supported although we are able to recover from it with retransmissions.● Before JBCP 1.2.6 and MSS 1.3 the Via headers were with IP balancer addresses.● Via headers must be IP balancer headers only if the load balancer is capable of Via branch affinity
  • 11. Standalone Mobicents SIP LoadBalancer● The SIP LB is a stateless SIP proxy● Responses and subsequent requests will follow the same path - UDP follows Vias and TCP/TLS follows the established connections as required per SIP spec.● SIP LB can make routing decisions based on SIP headers or content. It parses the SIP messages.● The Standalone SIP LB is a Single Point of Failure
  • 12. Standalone Load Balancer (proxy)Note: In case of SIP AS or SIP LB failure in in step 4 theresponse will be lost.
  • 13. Distributed Load Balancer● IP Load Balancer in front of the SIP LB● The SIP LB will advertise the IP LB address instead of its own● Support for multiple SIP LBs. The IP LB will distribute the load among the SIP LBs● Support for bidirectional load-balancing
  • 14. Distributed Load Balancer● The SIP LBs may maintain shared state when it is needed for certain load-balancing algorithms● Certain algorithms dont need shared state (like consistent hash)
  • 15. Bidirectional Distributed SIP LB ● The SIP Application Servers not only receive client requests, but can initiate SIP requests and transactions on their own
  • 16. Solving the case for both directions inthe Distributed LBWhen requests come from SIP phones(clients) it is clear weshould use an IP load balancer in front of the external ports ofthe SIP LBs. However, when the Application Servers areinitiating requests there are two options: 1. The Application Server is always aware which SIP LBs are alive, so if one dies the AS will pick another one on its own. No IP load balancer is needed. This method works in terms of heartbeat resolution. 2. Put an IP load balancer between the Application Servers and the internal ports of the SIP load balancers as shown in the next slide.
  • 17. Distributed Load Balancer (2 IP LBs)BIDIRECTIONAL
  • 18. Bidirectional Distributed SIP LBDeployment Scenarios ● The SIP LB can be configured with separate ports for inbound and outbound messages (simply specify the internalPort property in the SIP LB configuration file) ● Two IP LBs - use separate IP LBs for request that come from clients and requests that come from servers ● One IP LB - use the same IP LB for both types of requests. The problem with this one is that direction analysis must be done using Via header. If the SIP AS is a non-Record- Routing proxy application then non-initial requests initiated by the callee will bypass the SIP AS and there will be no SIP AS Via header to give a hint that the request comes from the callee and should no go to the SIP AS.
  • 19. Distributed Load Balancer (2 IP LBs)BIDIRECTIONAL
  • 20. Distributed Load Balancer (or 1 IP LB)BIDIRECTIONAL
  • 21. Converged andCooperative Load Balancing SIP, HTTP and other protocols
  • 22. Integrated HTTP forwarding● The Mobicents Load Balancer supports HTTP forwarding● SIP and HTTP can use common consistent hash affinity key to group and fail-over together SIP and HTTP sessions● Example ○ SIP URI sip:app1@host.com ○ HTTP URL http://host.com/page?appsession=app1 ○ app1 is sip user in the URI and appsession parameter ○ app1 will be hashed against the AS nodes for both and will cause the SIP and the HTTP request to always stick to the same node● When there is no key, the balancer behaves like mod_jk and analyses jvmRoute component to selet node● Alternatively, you can use mod_jk and mod_cluster
  • 23. Distributed Load Balancer (3 IP LBs)BIDIRECTIONAL
  • 24. HTTP and SIP consistent hashing
  • 25. Session affinity with mod_jk
  • 26. mod_jk supportmod_jk and mod_cluster can be manipulated by changing thejsessionid cookie to reroute requests to a node of choice!Additionally mod_cluster can be controlled by the MCCPprotcol
  • 27. mod_jk hints
  • 28. Rolling Upgrades from the LB● Each node is bootstrapped with a version system property ○ Each node is started with -Dversion=1● The version is advertised in the SIP LB heartbeat● The SIP LB has awareness of the groups with particular version and can detect conditions that jeopardize the opertaions ○ More than two versions ○ Node count dangerously low ○ Stalled upgrade with idle nodes
  • 29. Cluster groupsDivide the cluster into subgroups that failover only internallysubclusterMap=( 192.168.1.1, 192.168.1.2 ) (10.10.10.10, 20.20.20.20, 30.30.30.30)
  • 30. SIP LB Internal Architecture What is needed for the SIP LB to support the deployment scenarios?
  • 31. Quick SIP LB functional spec● Dumb SIP parsing - as dumb as possible with JSIP stateless.● Pluggable routing decision algorithms. Pass the message to the algorithm and it will return the node where to send the message.● Shared store - use JBoss Cache 3.2.1● No need to translate between TCP and UDP. The SIP AS will be able to handle both anyway.● Support separate SIP ports for inbound and outbound messages (add an internalPort property in the config file)● Support for single SIP port (delete the internalPort property)● RMI and JGroups heartbeats (right now RMI)● Protocol to allow AS or other entity to give instructions to the SIP LB.(like the mod_cluster protocol)
  • 32. Example AlgorithmsSample pluggable algorithms for the SIP load balancer
  • 33. #1 - Call-ID affinity workflow
  • 34. #2 - Consistent-hash on Call-ID
  • 35. #3- Persistent Consistent Hash
  • 36. Balancer Algorithm Interfacepublic interface BalancerAlgorithm { SIPNode processExternalRequest(Request request); SIPNode processHttpRequest(HttpRequest request); void processInternalRequest(Request request); void processExternalResponse(Response response); void processInternalResponse(Response response); void nodeRemoved(SIPNode node); void nodeAdded(SIPNode node); Properties getProperties(); void setProperties(Properties properties); BalancerContext getBalancerContext(); void jvmRouteSwitchover(String fromJvmRoute, String toJvmRoute); void init(); void stop(); void assignToNode(String id, SIPNode node);} ● Click here to see the full interface with documentatione/browse/trunk/tool ● Click here to see one example algorithm Call-ID affinity with association map
  • 37. The SIP LB configuration file# The binding address of the load balancerhost=127.0.0.1# The RMI port used for heartbeat signalsrmiRegistryPort=2000# The SIP port used where client should connectexternalPort=5060# The SIP port from where servers will receive messages# Delete if you want to use only one port for both inbound and outbound)internalPort=5065# The HTTP port for HTTP forwarding.# If you like to have integrated HTTP load balancer, this is the entry pointhttpPort=8080#Specify UDP or TCP (for now both must be the same)internalTransport=UDPexternalTransport=UDP# If you are using IP load balancer, put the IP address and port hereexternalIpLoadBalancerAddress=127.0.0.1externalIpLoadBalancerPort=111# Requests initited from the App Servers can route to this address (if you are using 2 IP load balancers for bidirectional SIP LB)internalIpLoadBalancerAddress=127.0.0.1internalIpLoadBalancerPort=111# Designate extra IP addresses as serer nodes#extraServerNodes=222.221.21.12:21,45.6.6.7:9003,33.5.6.7,33.9.9.2
  • 38. ...the SIP LB configuration file# Call-ID affinity algortihm settings. This algorithm is the default. No need to uncomment it.#algorithmClass=org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm# This property specifies how much time to keep an association before being evitcted.# It is needed to avoid memory leaks on dead calls. The time is in seconds.#callIdAffinityMaxTimeInCache=500# Uncomment to enable the consistent hash based on Call-ID algorithm.#algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm# This property is not required, it defaults to Call-ID if not set, cna be "from.user" or "to.user" when you want the SIP URIusername#sipHeaderAffinityKey=Call-ID#specify the GET HTTP parameter to be used as hash key#httpAffinityKey=appsession# Uncomment to enable the persistent consistent hash based on Call-ID algorithm.#algorithmClass=org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm# This property is not required, it defaults to Call-ID if not set#sipHeaderAffinityKey=Call-ID#specify the GET HTTP parameter to be used as hash key#httpAffinityKey=appsession#This is the JBoss Cache 3.1 configuration file (with jgroups), if not specified it will use default#persistentConsistentHashCacheConfiguration=/home/config.xml# Call-ID affinity algortihm settings. This algorithm is the default. No need to uncomment it.#algorithmClass=org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm# This property specifies how much time to keep an association before being evitcted.# It is needed to avoid memory leaks on dead calls. The time is in seconds.#callIdAffinityMaxTimeInCache=500
  • 39. ...the SIP LB configuration file# Uncomment to enable the consistent hash based on Call-ID algorithm.#algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm# This property is not required, it defaults to Call-ID if not set, cna be "from.user" or "to.user" when you want the SIP URIusername#sipHeaderAffinityKey=Call-ID# and specify the GET HTTP parameter to be used as hash key#httpAffinityKey=appsession# Uncomment to enable the persistent consistent hash based on Call-ID algorithm.#algorithmClass=org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm# This property is not required, it defaults to Call-ID if not set#sipHeaderAffinityKey=Call-ID# and specify the GET HTTP parameter to be used as hash key#httpAffinityKey=appsession#This is the JBoss Cache 3.1 configuration file (with jgroups), if not specified it will use default#persistentConsistentHashCacheConfiguration=/home/config.xml#NEW PROPERTIES IN MSS 1.2#If a node doesnt check in within that time, it is considered deadnodeTimeout=5100#The consistency of the above condition is checked every heartbeatInterval millisecondsheartbeatInterval=5000#JSIP stack configuration.....
  • 40. Example Configurations How to configure common scenarios? Note: All distributed configurations must use a consistent hash routing algorithm.Example with Call-ID header hashing:# Uncomment to enable the consistent hash based on Call-ID algorithm.algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm# This property is not required, it defaults to Call-ID if not setsipHeaderAffinityKey=Call-ID#specify the GET HTTP parameter to be used as hash keyhttpAffinityKey=appsession
  • 41. Two SIP LBs with client poolsRequest coming from phone.If a load balancer fails then a phone pool will experience outage.
  • 42. Two SIP LBs with Internal and External IP LBs in the same network with sample configurations. Request coming from phone.
  • 43. Two SIP LBs with Internal and External IP LBs in the same network with sample configurations. Request coming from Application Server.
  • 44. Two SIP LBs with Internal and External IP LBs in the same network with sample configurations. Full picture
  • 45. IP load balancer problems
  • 46. SIP LB Topics 2011
  • 47. Large scale tests● Simulate the IP load balancing and rewriting the addresses● Run many nodes, many load balancers● Execute the test as part of a larger scenario● Simulate application server ○ Heartbeat ○ Handling and initiating requests● TLS
  • 48. Cluster groupsDivide the cluster into subgroups that failover only internallysubclusterMap=( 192.168.1.1, 192.168.1.2 ) (10.10.10.10, 20.20.20.20, 30.30.30.30)
  • 49. Worst-case testingLoad balancing without affinity
  • 50. Rolling Upgrades from the LB● Each node is bootstrapped with a version system property● The version is advertised in the SIP LB heartbeat● The SIP LB has awareness of the groups with particular version and can detect conditions that jeopardize the opertaions ○ More than two versions ○ Node count dangerously low ○ Stalled upgrade with idle nodes
  • 51. Performance testing10K requests/s
  • 52. Moving the load balancing on theserver side● Eliminates the worst case● Some requests will go deeper in the pipeline costing more● Cant change the Route headers● Delay the load balancing decision as much as possible
  • 53. NIO in the SIP LBBIO is limited at 2500-10000 concurrent sockets on servers