This document provides an overview and instructions for configuring dynamic routing on pfSense using the FRR routing daemon. It discusses the key differences between interior routing protocols like OSPF and exterior protocols like BGP. It then provides step-by-step instructions for installing the FRR package, configuring the global settings, and setting up OSPF and BGP configurations including necessary preparations and neighbor/interface definitions. Tips are also provided for converting from OpenBGPD or Quagga configurations to FRR format.

1. Dynamic Routing with FRR
December 2017 Hangout
Jim Pingle

2. About this Hangout
● Project News
● Why use a routing protocol?
● Terminology
● BGP Intro
● OSPF Intro
● Compare/Contrast BGP and OSPF
● FRR Intro
● FRR package for pfSense
● Configuring FRR/Zebra
● BGP Setup Prep
● BGP Configuration
● OSPF Setup Prep
● OSPF Configuration
● Converting OpenBGPD to FRR
● Converting Quagga to FRR

3. Project News
●
2.4.2-RELEASE-p1 / 2.3.5-RELEASE-p1
– Security and bug fixes
– https://www.netgate.com/blog/pfsense-2-4-2-release-p1-and-2-3-5-release-p1-now-available.html
●
Article on Snort and openappid for application detection
– https://www.netgate.com/blog/application-detection-on-pfsense-software.html
●
Netgate was at the DPDK Summit, FD.io mini summit, and AWS re:Invent conferences
●
New Netgate products TNSR and SCLR announced
– Based on VPP from FD.io for a data plane, may also support netmap in the future
●
VPP is capable of performance far beyond FreeBSD or Linux kernel networing
●
Currently using Linux as a base OS to run VPP
– 40+Gbit/s IPsec
– TNSR is high-throughput enterprise focused, SCLR is Small/Medium Business
– TNSR is CLI-based, SCLR will have a web interface
– RESTCONF API
– First offering is TNSR for AWS Transit VPC to connect between AWS VPC regions at high speeds
●
Sneak peek at upcoming c3000 based platform, PLCC-B
– https://twitter.com/pfsense/status/943879263165517824
– C3xxx CPU, built-in switch, exact specs of production units are not yet set
●
Netgate office is moving to a new location in Austin next month
– There should be minimal, if any, disruption in system lead times and deliveries

4. Disclaimer
● In some areas this hangout will stay light on detail
– No time to get into the intricacies of how the protocols work, advanced setups,
etc
● Some terminology and examples will be simplified, so if you are very
familiar with BGP or OSPF you might find some omissions or
contradictions made here for the sake of simplicity
● Many more advanced configurations are possible than will be covered in
this introductory hangout!
– Not covering advanced settings or concepts such as multiple areas, route maps,
communities, access lists, prefix lists, etc.

5. Why use a routing protocol?
● Easier than maintaining static routes
●
Each router knows its own connected networks and will tell other routers about them, rather
than having to manually configure routes everywhere
●
Scales better for larger networks vs manual configuration
●
Can react dynamically to events (e.g. WAN or other path failures)
● Can calculate best path to reach another network when multiple paths are possible
● Generally falls into two classes:
– IGP - Interior Gateway Protocol, manages routing between internal segments of a single network
●
Example: OSPF between local segments or site-to-site links
– EGP - Exterior Gateway Protocol, routing between discrete networks
●
Example: BGP between a site and its ISP

6. Terminology
● Autonomous System (AS) - The network of an entire organization
● AS Number - A number assigned to a specific Autonomous System
– Depending on peer capabilities, can either be 2 byte (0-65534) or 4 byte (0-4294967294) value
● Area - A set of related routers
– An Area is typically represented as an IPv4 address in dotted quad notation, such as 0.0.0.0
● Router ID - Used to uniquely identify a router
– Usually an IPv4 address on a router, but can be any IPv4 address so long as it is unique to the router
● Neighbor - A peer in a directly connected network that is running the same routing protocol
● Adjacency - A relationship between two neighbors
● Next Hop - Host through which traffic for a route is delivered
● Metric / Preference / Weight - Used to signify which paths are best when there are multiple choices

7. BGP Introduction
● BGP stands for Border Gateway Protocol
● Routes between autonomous systems
● Each Neighbor must have a BGP connection defined for every other Neighbor to exchange routes
● Neighbors connect to each other using TCP port 179
● Uses path length and other configured policies to make decisions
● Most commonly used for exterior use cases such as reaching the Internet (ebgp)
● Can also be used inside an AS for interior routing (ibgp)
● Example use cases
– ISP full feeds (ebgp)
●
When full feeds are available from multiple ISPs, routing decisions can be made about what traffic to send along each path
● Lots o' RAM and CPU for that!
– Public IP allocation routing (ebgp)
– Internal routing (ibgp)

8. OSPF Introduction
●
OSPF stands for Open Shortest Path First
●
Routes internally within an autonomous system or single routing domain (not over the Internet, only local directly connected
paths)
●
Routers do not need to know about other routers beforehand, they find other routers automatically
●
Uses protocol 89 (OSPF), not TCP or UDP
●
Uses multicast to distribute routes inside a broadcast domain
– Connections between OSPF neighbors must be capable of using multicast!
●
Uses Area designations to divide portions of networks
– Every router inside an area must keep a database of networks inside that area, so a single area doesn’t scale well on large networks
– Routes between areas can be summarized
– Different types of areas accept different types of routes, but this is beyond the scope of what we’re talking about today (e.g. stub areas)
●
Example use cases
– Edge router to L3 switch or internal router
– VPN or dedicated site-to-site circuit
– Multiple paths for VPN or other networks

9. Choosing between BGP and OSPF
●
BGP can work externally and internally, but is better at the border or handling border-related
tasks
● OSPF is for internal use only
● The two can work together, BGP at the edge (or setup to let routers find a path to the edge),
OSPF internally so routers know how to reach each other and other local networks
● Both can work with IPv4 and IPv6, depending on the implementations
– OSPF/OSPFv2 for IPv4, OSPF6/OSPFv3 for IPv6
– FRR uses BGP version 4 which can handle IPv4 or IPv6
●
BGP has strictly defined peers, better security, autonomous systems
● BGP requires a full mesh where every peer defines every other peer, thus scales poorly on
large internal networks
– Route reflectors and confederation can help, but are beyond the scope of what we’re covering today

10. Choosing between BGP and OSPF
● OSPF works much more dynamically, less admin overhead for each node
●
OSPF converges faster in most common cases
● BGP uses TCP, OSPF requires multicast and uses its own protocol (89)
● BGP can use TCP-MD5 signatures for authentication/security
– Not on OpenVPN, but on regular interfaces
– OSPF can have password authentication, but it uses a different mechanism
● FRR is capable of redistributing BGP routes to OSPF and vice versa
– Route maps can be used to filter what routes go where, at least on the BGP side
– The OSPF GUI in the pfSense FRR package does not have proper support for route-maps but
the same effect can be obtained using the “do not redistribute” list
●
OSPF routers know the entire topology to every segment within the area, BGP may only
know the next hop depending on the setup

11. What is FRR?
● FRRouting, also known as Free Range Routing
● Fork of the Quagga project from large contributors looking to
increase the pace of development
● Handles dynamic routing protocols such as BGP, OSPF,
OSPFv3/OSPF6, IS-IS, LDP, PIM, and RIP
– pfSense FRR GUI only currently supports BGP, OSPF, and OSPF6
● Like Quagga, it has dedicated daemons for each protocol, plus
the zebra demon to coordinate everything

12. FRR Package for pfSense
●
Available on 2.4.2 and 2.3.5
● Designed as a replacement for both Quagga and OpenBGPD which both have problems
– The Quagga package only had GUI controls for OSPF, but could be manually configured in some cases for BGP
– The OpenBGPD package had several behavioral problems which limited its usefulness
– Due to both Quagga and OpenBGPD having “bgpd” binaries they could not be installed at the same time
– OpenBGPD had problems with IPsec, PFKEY errors, the problem does not occur with FRR (e.g. AWS VPC VPN)
● By unifying features in one package, BGP and OSPF can both be used without manual workarounds
● Extensive GUI for configuring OSPF and BGP
●
Improved status combined from Quagga and OpenBGPD and enhanced
– Status > FRR menu entry
– Filtering, data limiting to help deal with large routing tables
– Multiple status views, tab per protocol, plus general status with summary info
● May gain support for additional protocols in the future
● Why FRR and not something else like BIRD? Familiarity, stability on FreeBSD, and preference

13. FRR Package for pfSense
● Installing the package
– Installs via the package manager like any other package, System > Package Manager, Available Packages tab
– The package adds Services menu entries for each section of FRR (Global/Zebra, BGP, OSPF, OSPF6), plus a Status menu link
● Navigating the package
– Too many tabs in each section to show them all at once, so each section only shows its own options, plus links to other sections
– Tabs in [Brackets] go to other sections
●
Example: When in BGP, click [OSPF] to switch to the OSPF section with its own set of tabs
●
Always enable the main FRR tab checkbox in addition to the individual daemons (master on/off switch!)
– Example: To enable OSPF, visit OSPF Settings and check Enable OSPF Routing, then on the Global Settings tab, check Enable FRR
● Status section has an All page with some information from each enabled section, plus tabs for each section with more detail
●
Do not be intimidated by the complexity of the GUI!
– There are many, many options in FRR but odds are if you don’t know or recognize them, you probably don’t need them for basic use cases
– FRR is very powerful, but not everyone needs all of that power, and it isn’t practical to make numerous “Advanced” sections for everything
– If any terms or concepts are unfamiliar, read through the FRR user guide at https://frrouting.org/user-guide/
● Dynamic routes in routing table have a “1” flag when viewed at Diagnostics > Routes

14. Configuring FRR/Zebra
● No matter which protocol is used, first enable the main settings
of FRR
● Navigate to Services > FRR Global/Zebra
● Check Enable FRR
● Enter a Master Password
● Set logging as desired
● Enter a Router ID (e.g. LAN IP Address of this firewall)

15. BGP Setup Preparations
● Determine the setup type (e.g. site to site, ISP uplink, etc)
– This is example is a site-to-site example, but others are similar
● Determine the local AS number
– If this is local/private, use a private AS number in the range 64512-65534 (16 bit/2 byte AS) or 4200000000-
4294967294 (32 bit/4 byte AS)
– If this is a public allocation, ask your RIR and/or ISP
● Determine the remote neighbor address and AS number
– Consult the administrator of the far-side firewall
– If this is you, and it’s internal, pick another private AS number
● Decide which networks will be advertised through BGP
– If this is private, it’s up to you
– If this is for a vendor link, it must be agreed upon by both sides
– If it’s public, it will be the address block allocated to you
● Firewall rules must allow tcp/179 from the neighbor to this device, e.g. on WAN or VPN link

16. Configuring BGP
● Configure FRR/Zebra as mentioned previously
● Navigate to the BGP Settings via [BGP] tab or Services > FRR BGP
● Check Enable BGP Routing
● Set logging as desired
● Enter the AS number for this router
● Enter a Router ID, or leave it blank to use the ID defined in the global
settings
● Under Network Distribution, enter the local networks which will be
advertised to BGP neighbors (e.g. LAN, DMZ, or a summary of all)
● Save the settings

17. Configuring BGP
● Switch to the Neighbors tab, add an entry for each neighbor/address family (e.g. one
entry for IPv4, one for IPv6)
● Enter the Name/Address, typically the IP address of the neighbor in a directly-
connected network
● Enter a Description for the neighbor to make it easier to identify
● Enter the Remote AS, which is the AS number of the neighbor
● Other options may be set as needed, depending on the requirements of the peer, but
no other options are required
● Save, and visit the BGP tab and Save there again to ensure all settings are activated
● Navigate to the Status page and check if an adjacency has been formed to the
neighbor

18. OSPF Setup Preparations
● Determine which interfaces will be involved in OSPF, and how
– Active interfaces will advertise OSPF to other OSPF neighbors (e.g. VPN links, links to other
routers or L3 switches)
– Passive interfaces will have routes to their networks distributed to other OSPF neighbors
(e.g. LAN)
– For configurations with multiple links to the same remote, decide which interfaces are the
preferred path
● Determine the OSPF area(s) this router will be involved in (e.g. 0.0.0.0)
● Firewall rules must allow the OSPF protocol on active interfaces
● Active interfaces must support multicast in a way that OSPF can use
– OpenVPN must be static key site-to-site or tap with a subnet

19. Configuring OSPF
● Configure FRR/Zebra as mentioned previously
● Navigate to the OSPF Settings via [OSPF] tab or Services > FRR OSPF
● Switch to the OSPF Interfaces tab, since interfaces must be added before configuring the general OSPF settings
● Add entries on this tab for each interface involved with OSPF
●
Enter a Description for the interface to identify it
●
For interfaces that link to other routers, enter a Metric to help OSPF decide which link to use when there are multiple choices
(e.g. redundant VPN), lower metrics are preferred by the protocol
●
Enter an Area for the interface if it differs from the main area for this router
●
For local interfaces not directly involved in OSPF, such as a LAN that contains no other routers, check Interface is Passive
– This will configure the network in OSPF but advertisements are disabled
●
If there are multiple links to the same destination which could create a loop, check Accept Filter which tells FRR to ignore
routes received from peers for the network on this interface
– This can help with multiple links and OpenVPN so OpenVPN doesn’t fail due to its own interface route being in the table unexpectedly
● Save the settings, add the rest of the interfaces

20. Configuring OSPF
● Switch the the OSPF Settings tab
● Check Enable OSPF Routing
● Set logging as desired
● Enter a Router ID, or leave it blank to use the ID defined in the global settings
● Enter an Area ID, or use 0.0.0.0 if one is not known
● Choose any Redistribute options that you want
– Redistribute Connected will distribute routes for any directly connected subnets
– Redistribute Kernel will distribute routes present in the kernel, such as static routes from pfSense defined under System >
Routing
– Redistribute static will distribute routes defined on the main FRR/Global Settings tab
● Change any other options if needed to match the other routers in the area
● Save the settings
● Check the status to see if an adjacency has been formed to the neighbor(s)

21. Converting from OpenBGPD to FRR
● Groups are not required in FRR as they were in OpenBGPD
– If there was only one neighbor in a group in OpenBGPD, do not make a group in FRR, simply
define the remote AS directly on the neighbor
● The Neighbor Parameters in OpenBGPD are nearly all present in FRR but they may
not work 100% the same, or may require additional parameters
– Example: “Local Address X” on OpenBGPD is “Update Source” on FRR and requires also
choosing the correct address family, such as IPv4, and picking the source from an interface and
VIP list rather than entering it manually
– Example: TCP MD5 options on OpenBGP are under “Password” on FRR with the Type set to
“FRR and setkey Bidirectional”
● For converting a raw configuration, refer to the FRR docs to locate equivalent
statements

22. Converting from Quagga to FRR
● Since FRR is a fork of Quagga, the configuration is nearly identical
● The GUI has some slight differences but nothing major for OSPF
● The configuration cannot be automatically imported from Quagga,
but it is simple to note down the settings, take screenshots, or refer
to the config.xml and re-create an identical configuration, screen
for screen within the FRR OSPF section
● For a manual configuration it should import as-is, but if any
problems are encountered, refer to the FRR docs

23. Bonus Slide: FRR for AWS VPC VPN
● Exact details vary depending on the setup, but are not that different from a standard setup
● Global/Zebra
– Enable FRR
– Router ID: Virtual IP address for local end of tunnel
●
BGP Settings
– BGP: Enable
– AS Number: Local AS
– Hold Time: Hold Time set in VPC
– Networks: Add local networks to the list, local networks from your IPsec P2 (no route-map selected)
●
BGP Neighbor
– Name/Address: VPC remote address
– Remote AS: VPN remote AS number
– Update Source: Virtual IP address for local end of tunnel
● If you are using a Netgate device with the factory image, the VPC wizard can get you started, but …
– Remove OpenBGPD first, then install FRR
– Edit/save the VIPs created by the wizard before configuring FRR to work around a bug that’s being addressed in the next update of the
wizard

24. Conclusion
● Questions?
● Ideas for hangout topics? Post on forum, comment on the blog
posts, Reddit, etc