This document discusses GTP load balancing techniques using Cubro products. It describes how outer IP load balancing alone is insufficient and can cause issues. It recommends using inner IP load balancing along with session correlation and control plan correlation to properly handle subscriber sessions across network changes. This is achieved using Packetmasters and Sessionmasters working together. Scaling to handle higher traffic is possible by upgrading devices and adding more powerful units like the EXA24160.

1. GTP Load Balancing
How to do GTP Load Balancing with Cubro
Products
27 Sep 2017
www.cubro.com

2. • GTP: GPRS Tunneling Protocol
• GTP is used to transport packet data from the eNodeB to the
internet via an IP tunnel.
GTP Load Balancing

3. GTP Load Balancing in L 4
GTP is an IP in IP Tunnel used on many mobile Interfaces like GN, S5, S8 and S1U.
GTP consists of two types of packets GTP-C and GTP-U
GTP-U = is the user plan where the user traffic is transported
GTP-C = is the control plane of the protocol

4. GTP Load Balancing in L4
• IP address range is different between outer and inner IP
• Outer IP address range is from a few hundreds to thousands
• Inner IP address range is millions as each subscriber has a unique IP
• Typically a Probe needs GTP-U and GTP-C to produce useful meta data (this cannot be
done with outer IP LB)

5. Outer IP Load Balancing
Outer IP address is in L4 which means any NPB can do load balancing based on a 5 tuple hash
If only Outer IP LB is done on GTP traffic, following issues will happen:
 Monitoring session for a user is interrupted when the customer moves to another location
 LB could be asymmetric due to the small amount which means the output ports can be
overloaded leading to packet drop and bad monitoring quality
 Corresponding GTP-C traffic is not on the same port
 On LTE not on the same interface (S11)

6. LB Configuration in EX (outer tunnel)
Hash configuration (defines the field which is used for LB)
Configure output group (defines the amount and which ports are used
for the LB)
Add a rule for the in ports to the LB
group
(Binds the ports where the traffic is
received to the load balancing group
and the defined outputs to load
balance the traffic based on the
configured hash)

7. Typical Application
• For most of the Probes on the market this will not work because the
Inner IP traffic could be disintegrated - the traffic of the subscriber
session can be distributed to different Probes.
• This application is usable for capture solution or in the first stage of
an inner IP load balancing solution (see next pages)

8. Inner IP LB with session correlation and
control plan correlation
• LP on Outer IP to reduce the traffic from100 Gbit to multiple 10 Gbit links
• Sessionmaster now decodes the GTP-C or and S11 and S6a user plan to get all meta data to do a
full subscriber based correlation needed because 2G - 3G - 4G PS Domain traffic has some
differentiation and therefore different methods must be used to handle the traffic properly
(especially in a fall back situations and when the customer is moving and the session jumps
between network elements, it is not easy to forward the traffic always to the same probe)
• Each Sessionmaster is connected to each Probe and also between each other to perform this
challenging application
• In the case of more then 3 – 5 Probes an additional Packetmaster is used to steer the traffic (see
next page)

9. GTP LB Application for 1-1,2 TB
Mixed traffic 2G/3G/4G (including S1-MME/S11/S6a/S1-U)
Input aggregation & first outer tunnel load balancing, session and endpoint out learning meta
data provider, l7 traffic tagging subscriber based, output load balancing based on L7 tagging
to multiple probes.
(The first and the last units can be one unit depending on the amount of ports used)

10. Simple DPI Integration – Inner IP LB
Mixed traffic (including S1-MME/S11/S6a/S1-U)
Functions:
EX48(1): Auto study for eNB/xGW IP address from S1MME, LB traffic to EX48D
EX48D: Find user IP from inner layer, by matching eNB/xGW IP combination, then LB the traffic by inner layer IP
EX48(2): Aggregate output traffic from EX40D by VLAN (same user has same VLAN tag)
Performance:
EX40D:80Gbps/U (40G per CPU, 2x)
(EXA40D can also be upgraded to EXA24160, to get better performance per unit)

11. Simple DPI Integration – IMSI Based LB
Mixed traffic (including S1-MME/S11/S6a/S1-U)
Functions:
EXA40(1): Auto Study for eNB/xGW IP address from S1MME, LB traffic to EXA40D
EXA40D(1): Processing control plane traffic and get correlation meta data, then send meta data to EXA40(2),
and output control plane traffic by IMSI
EXA40D(2): Correlation user plane and control plane meta data, then output user plane traffic by IMSI
EXA40(2): Aggregate output traffic from EXA40D(1)/(2) by IMSI based VLAN (same IMSI has same VLAN tag)
Performance:
EXA40D(1):80Gbps/U (40G per CPU, 2x)
EXA40(2): 5~6Gbps/U (Control plane decoding only)

12. Review of the two DPI Integration
• Auto study of information of eNB/xGW from traffic - no
manual configuration work on thousands of IP addresses
• Have enough access ports (up to 96 10GE), and the port
number can be extended by adding simple aggregation
switches
• Have the same performance on user plane processing,
i.e. 80G bps per 2xCPU unit
Since complex solution needs correlation, special processing
on control plane traffic by decoding is needed, which is a
performance consuming task. Each 2xCPU units can handle
5Gbps signaling, i.e. 5% of 100G all-mixtures-traffic. The user
needs extra 3~4 units to do correlation.

13. Scale-out possibility
Mixed traffic (including S1-MME/S11/S6a/S1-U)
When the future traffic to be load balanced will in the line of 100GE, we can upgrade:
 EXA48 (Position 1) to EX32100 (32x100G)
 EXA40D (Position 1 and 2) to EXA24160 (each unit can handle 120G+ user plane traffic and 10G control
plane traffic)
 By EX32100, we can connect up to 32 EXA24160 nodes for 2.4T traffic and have 20xEXA24160 (to
replace EXA40D (2)) to process user plane and 12xEXA24160 (to replace EXA40D (1)) to process control
plane traffic.
 The final output traffic can use EX32100/EX48600 combination to many 10G/40G end points

14. Cubro Product Portfolio for GTP LB
EX32 (G4 Series)
EX48600
EX32100
EXA40/40D
EXA24160
Upto L4 Packet Oriented Up to L7 Session Oriented
EX20400

15. Packetmaster EX32100
 MPLS tag/detag
 VLAN tag/detag / Q in Q
 Header modification Layer 4
 Load balancing Layer 4
 GRE de/encapsulation
 VXLAN de/encapsulation
 All ports activated
 All software activated
 Low power design
 Jumbo Frames 12000 Bytes
(Management ports on the back)
Packet load 3,2 Tbps
Ports 40 Gbit 32 QSFP
Ports 100 Gbit 32 QSFP28
GUI WEB/CLI/GUI
Packet buffer YES 24 MB
Delay < 700 ns
Dual Power YES
112 x 10 Gbit (with breakout cable) + 4 x 100 Gbit
128 x 10 Gbit (with breakout cable)
1.2 Bpps forwarding performance:
L2 FDB/IPV4 Host /IPV6 Host/MPLS Labels (unified) (IP accurate)
320K/160K/80K/160K
IPV4 Routes (LPM) / IPV6 LPM 64K/16K (IP with mask)
iPACL/iVACL/iRACL – 64b keys (5-tuple with wildcard)
eACL 6K per pipe (5-tuple with wildcard)

16. Sessionmaster EXA40 / EXA40D
 1 Octeon III CPU (28 cores each)
 1 x 64 GB DDR Memory
 1 x 80 Billion instructions per sec
 Up to 40 Gbps L7 filtering performance
(forwarding only)
 Up to 20 Gbps L7 correlation
performance
 Up to 10 Gbps L7 metadata
processing
 The D Model comes with two CPU
and double the performance

17. Sessionmaster EXA24160
 2 x Octeon III CN7890 CPU (48 cores each)
 2 x 64 GB DDR Memory
 2 x 240 Billion instructions per sec
 Up to 160 Gbps L7 filtering performance (forwarding only)
 Up to 120 Gbps L7 correlation performance
 Up to 60 Gbps L7 metadata processing

18. Packetmasters – L 7 Filtering
Packetmaster EXA48600
Packetmaster EXA32100
These advanced units are a mixture between Sessionmaster and Packetmaster
• L7 filtering in line rate up to multiple 100 Gbit
• But cannot handle session like the Sessionmaster

19. EXA32100
• EX32100 / EXA48600 can do LB on the inner IP, and
remove the GTP tunnel if needed
• EXA32100 can help to offload the Sessionmaster and the
amount of units.

20. Next generation GTP LB
Aggregation and filtering stage (EX32100)
GTP-U load balancing (user plan) (EXA32100)GTP-C load balancing control plan) (EXA24160)
This BOM represents a solution for 400 – 500 Gbps full
correlated LB for 2G/3G/4G, included signaling correlation

21. The complete picture

22. Thank you
For more information contact us at: support@cubro.com