SDN X CLOUD NATIVE
MEETUP #10
APPLICATION-BASED ROUTING
Hung-Wei Chiu(hwchiu)
hwchiu@thundertoken.tw
WHO AM I
Hung Wei Chiu (Hwchiu)
https://blog.hwchiu.com
DevOps Engineer @ ThunderToken
Interest in Networking/DevOps/Kubernetes/SDN/Programming
Software Defined Wide Area Network (SD-WAN)
SD-WAN
Create a virtual-overlay to abstract underlying private/public WAN connections
LTE
MPLS
Wifi
Fiber
Route WAN traffic along the best route
Latency
QoS
SD-WAN
Managed by a centralized controller
Remotely program edge devices and reduce provisioning times.
Minimizing the need to manually configure network devices
Security
IPSec
Firewall
SD-WAN
Traffic
Improve performance through a combination WAN connections
Fail-over
Simplifies the network
Deployments
Configurations
Operations
DSL
MPLS
LTE
DSL
MPLS
LTE
DSL
MPLS
LTE
Controller
Internet
Application Based Routing
APPLICATION-BASED ROUTING
Assume there’re multiple WAN Connections
Different Latency
Route the traffic based on application
Priority
Game/VoIP/…etc
Public Internet
10 sec
2 sec
3 sec500 ms
Public Internet
10 sec
2 sec
3 sec500 ms
File Transfer
Web Browse
How Can We Implement it In a Linux Host
CHALLENGES
Which Application
Policy
Route Traffic
DPI
Deep Packet Inspection
Well-known tuple (L3 + L4)
TCP/UDP
Port (53/67/80/443)
DPI
L7 application
Organized format
Guess by pattern
SSL Termination
User should import a trusted CA from DPI devices
NDPI
Open Source Software (OSS)
Based on C++ language
Support 185+ protocols
NDPI
Support capture packets by pcap
Decision Tree
Example (Skype)
nDPI Engine
Packet PacketPacket Packet Packet
Packet Packet Packet
Packet Packet Packet
Packet = skype
Connections Results
Packet = Quic
Packet = Facebook
APPROACHES
With Linux Kernel
Without Linux Kernel
OpenvSwitch/Openflow
SOURCE (WITH KERNEL)
Use the pcap to catch packets from a
specific interface
By iptables and our implemented daemon
PCAP APPROACH
Implement a daemon
Capture packets by libpcap
Feed packets to nDPI engine to get its
type
Application
nDPI Engine
User-space
Kernel-space
Physical
libpacp
enp0s1 enp0s2 wlan1
BPF BPF BPF
IPTABLES APPROACH
Implement a daemon
Capture packets by iptables module
(NFQueue)
Feed packets to nDPI engine to get its
type
Network Stack
Application
nDPI Engine
User-space
Kernel-space
Physical
libnetfilter_queue
enp0s1 enp0s2 wlan1
netfilter
nfqueue
IPTABLES APPROACH
Use netlink to pass packets from/to user-
kernel
Flexible than pcap approach
Iptables rules
IPTABLES APPROACH
Use netlink to pass packets from/to user-
kernel
Flexible than pcap approach
Iptables rules
Iptables -A INPUT -j NFQUEUE --queue-num 0
Iptables -A FORWARD -j NFQUEUE --queue-num 0
Iptables -A FORWARD -i br0 -j NFQUEUE --queue-
num 0
Next Challenge..
How route packet by nDPI result?
ROUTE
Tag the packets and route by tag
Open the RAW socket to transmit that
packet(?)
NFQUEUE
We can send packets back to linux kern
el
Via netlink
It’s based on sk_buff structure
NFQUEUE
We can use the mark to represent the
application ID.
And then we can use that mark in the
iptables to accept/drop that packet
http://lt.netfilter.org/projects/libnetfilter_queue/doxygen/
group__Queue.html
Network Stack
Application
nDPI Engine
User-space
Kernel-space
Physical
libnetfilter_queue
enp0s1 enp0s2 wlan1
netfilter
nfqueue
- Receive Packets
- Detect Packet
- Mark Packet and send back
to Kernel
Routing
- Accept/Drop by mark
- Route ??
Iptables -i FORWARD -m mark --mark 0x0003 -j DROP
Iptables -i FORWARD -m mark --mark 0x0003 -j ACCEPT
But how to use that mark to route via different interfaces?
LINUX ROUTINGTABLE
Route packets by destination IP address in default.
Policy Route
Source IP address
L3/L4 protocols
Tos/Mark
LINUX ROUTINGTABLE
Multiple Routing table
Number from 0 - 32767 (high to low)
Build-in
0(local)
32766(main)
32767(default)
Custom
LINUX ROUTINGTABLE
echo 201 hwchiu.test >> /etc/iproute2/rt_tables
Use ip rule to manipulate the lookup order of tables.
ip rule add fmmark 10 table 201
Ip rule add from 140.113.235.234 fwmark 25 table 202
ip rule show
LINUX ROUTINGTABLE
Use ip route add to add the routing rule into table.
ip route add default via 10.0.2.2 dev enp0s3 table 201
Network Stack
Application
nDPI Engine
User-space
Kernel-space
Physical
libnetfilter_queue
enp0s1 enp0s2 wlan1
netfilter
nfqueue
- Receive Packets
- Detect Packet
- Mark Packet and send back
to Kernel
PRDB
main local default
custom
nfqueue
User-space
Kernel-Space
Physical Interface
enp0s1
Driver
netfilter_system
kernel_system
Thread
nfqueue_verdict
netfilter_system
enp0s2
Driver
call nDPI
How About Performance ?
PERFORMANCE
All packets have same l3/l4 tuple belong to
same connection (mostly)
We don’t need to detect all packets to
know its application.
Just pass the unknown connection packets
to nDPI engine
PERFORMANCE
Use the connmark to set the mark to
connection tracking entry.
Save the mark based on its skb_buff
Iptables -t mangle -j CONNMARK --save-
mark
nfqueue
User-space
Kernel-Space
Physical Interface
enp0s1
Driver
netfilter_system
kernel_system
Thread
nfqueue_verdict
netfilter_system
enp0s2
Driver
call nDPI
PERFORMANCE
You can also add those connection
tuple(l3/l4) to hardware to get the high
performance
Remember, only few packets need to be
detected.
Thanks!

Application-Based Routing