HAProxy scale out using open source

Why HAProxy?
High availability
Powerful loadbalancer for websites due to its
proxy nature
Open Source
Enterprise ready
HAProxy - Scale out using open source | by Ingo Walz
2

3
Enterprise options
ALOHA HAProxy Loadbalancer Appliance
HAProxy Enterprise Edition - HAPEE
http://www.haproxy.com/

4
Who's using it?
http://www.haproxy.org/they-use-it.html

5
Featureset
Content switching / filtering 
Asymetric load balancing 
Priority activation 
SSL offloading 
HTTP compression 
TCP buffering 
Priority queue / rate shaping 
Direct server return (DSR) 
http://en.wikipedia.org/wiki/Load_balancing_(computing)#Load_balancer_features

6
Looks familiar?

7
Fix your Single-Server Environment
Congratulation, your whole
environment is one Single Point
Of Failure!

8
Fix your Single-Server Environment
Always try to follow the principle:
One function per component
Not anymore because it scales the best, more because it's
the cleanest way to manage them.
Configure Services, not Servers

9
Make your application cluster capable
You need to deploy to a various number of different
machines
Do not use something like NFS to workaround
A CI will help you
Session clustering
Avoid to work on the filesystem to save data / user
input
Use central technologies to save your data (e.g.
databases)

Installation
HAProxy - Scale out using open source | by Ingo Walz 10

11
Known procedure
$ wget http://www.haproxy.org/download/1.5/src/haproxy-1.5.6.tar.gz
$ tar xvzf haproxy-1.5.6.tar.gz
$ cd haproxy-1.5.6
$ ./configure USE_OPENSSL=1 USE_PCRE=1
$ make
$ sudo make install
http://www.haproxy.org/#down

Configuration

/usr/local/etc/haproxy/haproxy.cfg
13
Configuration sections
global
# process wide and eventually OS specific
# some have CLI equivalents
[ .. ]
defaults
# set default parameters for all following sections
[ .. ]
frontend
# describes a set of listening sockets accepting client connections
[ .. ]
backend
# describes a set of servers to which the proxy will connect
# to forward incoming connections
[ .. ]
listen
# defines a complete proxy with its frontend and backend parts combined in one section.
# It is generally useful for TCP-only traffic
[ .. ]

14
TCP vs HTTP loadbalancing
defaults
mode tcp # Can balance everything, the default
defaults
mode http # But you want that!
Layer 7 loadbalancing advantages
● Request inspection
● Content switching
● Header manipulation
● Cookie persistence
● Advanced health checks

15
Loadbalance your nginx

16
Loadbalance your nginx
global
maxconn 4096
daemon
defaults
mode http
timeout connect 5000
timeout client 50000
timeout server 50000
frontend www_fe
bind :80
# Close connection to server but keep open for client
option http-server-close
default_backend www
backend www_be
server nginx1 10.0.0.10:80 check

17
Still a single point of failure

18
HA with HAProxy & keepalived

19
HA with HAProxy & keepalived
vrrp_script chk_haproxy {
script "killall -0 haproxy"
interval 2
weight 2
}
vrrp_instance VIRTUAL {
interface eth0
virtual_router_id 10
state MASTER #state BACKUP
priority 100 #priority 101
advert_int 1
virtual_ipaddress {
10.0.0.30
}
track_script {
chk_haproxy
}
}
/etc/keepalived/keepalived.conf

20
HA with public IPs

21
HA with public IPs - failover

The final step

23
Loadbalance MySQL - TCP

24
frontend mysql_fe
bind :3306
mode tcp
default_backend mysql_be
backend mysql_be
mode tcp
option mysql-check user haproxy
server mysql1 10.0.0.40:3306 check
server mysql2 10.0.0.45:3306 check backup
mysql~> INSERT INTO mysql.user (Host,USER) VALUES
('10.0.0.20','haproxy'); FLUSH PRIVILEGES;
mysql~> INSERT INTO mysql.user (Host,USER) VALUES
('10.0.0.25','haproxy'); FLUSH PRIVILEGES;

25

26

All about SSL

28
Setup SSL Pass-Through
frontend www_fe
bind :80
bind :443
mode tcp
default_backend www_be
backend www_be
mode tcp
No HTTP mode possible – how to inspect encrypted headers?

29
SSL Termination – why you should offload
Single configuration point for all certificates
Certificates not widely spread across the
infrastructure
Offload the decryption load
Typically, your HAProxy will have a bit of CPU
left
You need to decrypt to inspect the request information

30
Setup SSL offloading
frontend www_fe
bind :80
bind :443 ssl crt /etc/haproxy/sample.pem
# Close connection to server but keep open for client
backend www_be
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout
ssl/sample.key -out ssl/sample.crt
$ cat ssl/sample.key ssl/sample.crt > ssl/sample.pem

31
SNI – How it works
● Multiple certificates per IP / frontend profile
● Client and server need to support it

32
Setup SNI
frontend www_fe
bind :80
# sample.pem as default, other pem's based on host header
bind :443 ssl crt /etc/haproxy/sample.pem crt /etc/haproxy/certs/
# Content switch based on certificate (and based on host)
use_backend sample1 if { ssl_fc_sni sample1 }
use_backend sample2 if { ssl_fc_sni sample2 }
backend sample1
backend sample2
backend www_be

Secure your entry point

34
Protect against syn flooding
# Consider this amount of clients as valid
$ sysctl -w net.ipv4.tcp_max_syn_backlog=”4096”
# Once net.ipv4.tcp_max_syn_backlog is reached, enable syn cookies
$ sysctl -w net.ipv4.tcp_syncookies=1
# Enable reverse path filtering, is the source routable through the incoming interface?
$ sysctl -w net.ipv4.conf.all.rp_filter=1

35
Basic iptables
$ cat iptables.sh
#!/bin/bash
iptables -F # Drop current table
# Drop incoming traffic (eth0 is the public available interface)
iptables -i eth0 -P INPUT DROP
iptables -i eth0 -P FORWARD DROP
# Allow outgoing traffic
iptables -P OUTPUT ACCEPT
# Allow ping
iptables -i eth0 -A INPUT -p ICMP -j ACCEPT
# Allow SSH (this should be avoided, SSH to haproxy via internal interface / through VPN)
iptables -i eth0 -A INPUT -j ACCEPT -p tcp --dport 22
# Allow HTTP
# Allow HTTPS
# Allow connections from localhost on every port
iptables -i eth0 -A INPUT -j ACCEPT -s 127.0.0.1
# Already opened connections are accepted on every port (required for some daemons)
iptables -i eth0 -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

36
Use HAProxy to secure your environment
frontend www_fe
bind :80
bind :443 ssl crt /usr/local/etc/haproxy/sample.pem
#detect and reject shellshock requests
reqdeny ^[^:]+:s*(s*)s+{
reqdeny ^[^:]+:s+.*?(<<[^<;]+){5,}
#This rule to display SSLv3 error message
acl sslv3 ssl_fc_protocol SSLv3
http-request allow if sslv3
use_backend backend_sslv3 if sslv3
backend backend_sslv3
mode http
errorfile 503 /usr/local/etc/haproxy/pages/poodle.http

37
Check SSLv3 error message
$ openssl s_client -connect 10.0.0.30:443 -ssl3
[ … ]
SSL-Session:
Protocol : SSLv3
---
GET /
[ … ]
<html>
<head>
<title>SSLv3 detected</title>
</head>
[ … ]
</html>

38
Persistent HTTP loadbalancing
backend www_be
cookie PHPSESSID prefix
server nginx1 10.0.0.10:80 cookie nginx1 minconn 10 maxconn 20
check
check
# Set-Cookie:PHPSESSID=nginx1~7cmjd41klupaderap0q7tve357; path=/
Persistence only if PHPSESSID cookie is set!
backend www_be
cookie server insert indirect nocache
check
check
# Set-Cookie:server=nginx1; path=/

39
ACL
Extract data from request / response stream
Perform content switching
Conditional request handling
Can help you to secure your environment
E.g. display an error message for SSLv3

40
Loadbalancing algorithms (most useful)
roundrobin
leastconn
Suggested if you have very long sessions
source
Only useful in TCP environments
Other methods available
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#4-balance

41
“Reverse proxy” usage
frontend www_fe
bind :80
bind :443
[ … ]
use_backend nginx1_backend if { path_beg /nginx1 }
use_backend nginx2_backend if { path_beg /nginx2 }
backend nginx1_backend
reqrep ^([^ :]*) /nginx1(/.*) 1 /2
server nginx1 10.0.0.10:80 cookie nginx1 check
backend nginx2_backend
reqrep ^([^ :]*) /nginx2(/.*) 1 /2
server nginx2 10.0.0.15:80 cookie nginx1 check
You need to cut nginx1/2 from the request

42
Statistics

Run the example

44
Reminder

45
Used technologies
Vagrant >= 1.5.2
ChefDK >= 0.2.0
Berkshelf

https://github.com/iwalz/zendcon-haproxy

47
Project structure

48
Vagrant
$ vagrant plugin install vagrant-omnibus
$ vagrant plugin install vagrant-berkshelf
Omnibus for chef solo support
Berkshelf to manage cookbook dependencies

49
Vagrant up
$ git clone https://github.com/iwalz/zendcon-haproxy
$ cd zendcon-haproxy
$ vagrant up haproxy1
$ vagrant up haproxy2
$ vagrant up nginx1
$ vagrant up nginx2
$ vagrant up mysql1
$ vagrant up mysql2
Don't simply use `vagrant up`, the Berkshelf
Dependencies will be messed up

50
How to continue
Architecture documentation (outdated, but still useful)
http://www.haproxy.org/download/1.3/doc/architecture.txt
Official documentation
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html
Haproxy.com Blog
http://blog.haproxy.com/

Questions?

HAProxy scale out using open source

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to HAProxy scale out using open source

Similar to HAProxy scale out using open source (20)

Recently uploaded

Recently uploaded (20)

HAProxy scale out using open source