Nathalie Trenaman| 13 April 2021 |
Lebanese University, Faculty
of Sciences
Routing Security
Introduction to


BGP routing
3
Internet building blocks
ASN (Autonomous System Number)
4
ASN (Autonomous System Number)
Internet building blocks
ASN Addresses Interconnect
Autonomous System
5
Routing on the Internet
“BGP protocol”
Can I trust
AS2?
Routing table


194.x.x.x = AS2
Routing table


193.x.x.x = AS1
Is AS1
correct?
AS1


193.x.x.x
AS2


194.x.x.x
AS2: “I have 194.x.x.x”
AS1: “I have 193.x.x.x”
6
Route Propagation
AS15 AS756
R1
AS33
AS164
66.2.9.0/24
M
ED=700
MED=500
LP=100
LP=50
AS25 AS5
R2
LP=40
tra
ffi
c
route
7
Accidents Happen
• Fat Fingers


- 2 and 3 are really close on our keyboards….


• Policy Violations (leaks)


- Oops, we did not want this to go on the public Internet


- Infamous incident with Pakistan Telecom and YouTube
8
Incidents Are Common
• 2019 Routing Security Review


- 12,600 incidents


- 4,4% of all ASNs affected


- 3,000 ASNs are victims of at least one incident


- 1,300 ASNs caused at least one incident
Source: https://bgpstream.com
9
How Bad Is It?
10
Routing on the Internet
Can I
trust B?
Routing table


194.x.x.x = B
Routing table


193.x.x.x = A
Is A
correct?
A


193.x.x.x
B


194.x.x.x
B: “I have 194.x.x.x”
A: “I have 193.x.x.x”
RIPE
Database
“Internet Routing Registry”
11
Problem Statement
• Some IRR data can not be fully trusted


- Accuracy


- Incomplete data


- Lack of maintenance


• Not every RIR has an IRR


- Third party databases need to be used


- No verification of who holds IPs/ASNs
•
Problem Statement
13
Internet Routing Registry
• Many exist, most widely used


- RIPE Database


- RADB


• Verification of holdership over resources


- RIPE Database for RIPE Region resources only


- RADB allows paying customers to create any object


- Lots of the other IRRs do not formally verify holdership
Introduction to RPKI
15
Resource Public Key Infrastructure
• Ties IP addresses and ASNs to public keys


• Follows the hierarchy of the registries


• Authorised statements from resource holders


- “ASN X is authorised to announce my Prefix Y”


- Signed, holder of Y
16
RPKI Certificate Structure
Member Member Member
ROA ROA ROA
Certificate hierarchy follows allocation hierarchy
ARIN APNIC RIPE LACNIC AFRINIC
17
RPKI Chain of Trust
ROA
signature
LIR’s Resources
signature
public key
ALL Resources
signature
public key
18
RPKI Chain of Trust
RIPE NCC Root Certificate


Self-signed
ALL Resources
Root’s private key
signature
public key
19
RPKI Chain of Trust
LIR Certificate


Signed by the Root private key
LIR’s Resources
Root’s private key
signature
public key
20
RPKI Adoption
21
Two elements of RPKI
Signing
Create your ROAs
Validating
Verifying others
ROAs
23
ROA (Route Origin Authorisation)
• A ROA is…


• LIRs can create a ROA for each one of their
resources (IP address ranges)


• Multiple ROAs can be created for an IP range


• ROAs can overlap
24
What is in a ROA ?
Prefix The network for which you are
creating the ROA
The ASN that’s supposed to be
originating the BGP Announcement
Origin
ASN
Max
Length
The Maximum prefix length accepted
for this ROA
25
RPKI Chain of Trust
ALL Resources
LIR’s Resources
Root’s private key signature
signature
public key
public key
26
Route Origin Authorisation
Prefix


is authorised to be announced by


AS Number
LIR’s private key
ROA
signature
27
RPKI Chain of Trust
ROA
signature
LIR’s Resources
signature
public key
ALL Resources
signature
public key
28
Hosted or Delegated RPKI
RIPE
ROA ROA
ROA ROA
ROA
Member Member Member
ROA
Member-X CA Member-Y CA
RIPE NCC Hosted System
29
Hosted RPKI
• Automatic signing and key roll overs


- One click setup of resource certificate


- User has a valid and published certificate for as long as
they are the holder of the resources


- All the complexity is handled by the hosted system


• Lets you focus on creating and publishing
ROAs


- Match your intended BGP configuration
30
Delegated RPKI
• Run your own Certification Authority software


- Dragon research Labs, RPKI toolkit


- NLNetLabs, Krill


• Setup connection with RIPE NCC CA


• Generate a certificate and get it signed by the
parent CA


• Run your own repository
31
First login to the dashboard
32
Creating ROAs
33
Reviewing changes
34
Checking the effects
/23
35
193.0.24.0/21


AS2121


Max Length: /21
ROA
193.0.24.0/21
193.0.24.0/22 193.0.28.0/22
193.0.24.0/23


AS2121


Max Length: /24
ROA
193.0.30.0/23


AS2121


Max Length: /23
ROA
✖
✖
✔︎
/23 /23 /23 /23
/23
/24 /24 /24 /24 /24 /24 /24 /24
/24 /24
✖
✔︎✔︎✔︎✔︎
✖
✖ ✖
✖
✖
✖
✖
36
RPKI Adoption
37
ROA Adoption
38
ROA Accuracy
Validation Tools
40
Two elements of RPKI
Signing
Create your ROAs
Validating
Verifying others
41
Routing on the Internet
Is A
correct?
A


192.0.2.0/24
B


193.0.24.0/21
A: “I have 192.0.2.0/24”
1. Create route
authorisation record
(ROA)
2. Validate route
RPKI Repository
A is authorised
to announce
192.0.2.0/24
BGP
42
Trust Anchor Locator (TAL)
RIPE NCC ARIN APNIC AFRINIC
LACNIC
Validator
Repository Repository Repository Repository Repository
• Location of RIR repositories


• Root’s public key
TAL TAL TAL TAL
List of ROAs
Cerfificates
43
RPKI Validators
• Software that creates a local “validated cache”
with all the valid ROAs


- Downloads the RPKI repository from the RIRs


- Validates the chain of trust of all the ROAs and associated
CAs


- Talks to your routers using the RPKI-RTR Protocol
44
Relying Party
RIPE NCC ARIN APNIC AFRINIC
LACNIC
Validator
Repository Repository Repository Repository Repository
List of ROAs
Cerfificates
45
RPKI-RTR
ROAs
ROAs
VALIDATOR SOFTWARE
Verification
Validated
Cache
RPKI-RTR
ROUTERS
RIR REPOSITORIES
46
Relying Party
ROA
AS111 10.0.7.30/22
AS222 10.0.6.10/24
AS333 10.4.17.5/20
AS111 10.0.7.30/22
AS111 10.0.7.30/22
AS111 10.0.7.30/22
BGP Announcements
BETTER ROUTING DECISIONS
47
RIPE NCC Validator
• https://github.com/RIPE-NCC/rpki-validator


• Version 3.1


• Java-based, web interface, white-list
functionality


• Can speak RPKI-RTR
48
Alternatives
• All are open source:


- Routinator - https://github.com/NLnetLabs/
routinator/


- FORT - https://github.com/NICMx/FORT-validator/


- OctoRPKI - https://github.com/cloudflare/cfrpki


- RPKI-client - https://rpki-client.org/


- Prover - https://github.com/lolepezy/rpki-prover


- Rpstir2 - https://github.com/bgpsecurity/rpstir2
ROA Validation
50
Two elements of RPKI
Signing
Create your ROAs
Validating
Verifying others
51
ROA Validation
• Routers receive data from the validated cache
via RPKI-RTR


• Based on this and on BGP announcements,
you have to make decisions


- Accept or discard the BGP Announcement


- As temporary measure, you could influence other
attributes, such as Local Preference
52
ROAs
ROAs
ROA Validation
BGP Validation
VALID INVALID
VALID INVALID UNKNOWN
NOT FOUND
53
Invalid ROA
• Invalid ROA


- The ROA in the repository cannot be validated by the client
(ISP) so it is not included in the validated cache


• Invalid BGP announcement


- There is a ROA in validated cache for that prefix but for a
different AS.


- Or the max length doesn’t match.


• If no ROA in the cache then announcement is
“unknown”
54
Whitelisting
• If there is an invalid ROA for a network that’s
important for you or your customers, you can
whitelist it


• This is done on your local validator software


- It creates a “fake” ROA for the resources you want


• It allows you to contact the operator to fix their
ROA


- Think of e-mail, contact forms, etc…
55
Take the Poll!
Status of RPKI ROV
Name Type Details Status
Telia Transit Signed & Filtering Safe
Cogent Transit Signed & Filtering Safe
GTT Transit Signed & Filtering Safe
NTT Transit Signed & Filtering Safe
Hurricane Electric Transit Signed & Filtering Safe
Tata Transit Signed & Filtering Safe
PCCW Transit Signed & Filtering Safe
RETN Transit Partially Signed &
Filtering
Safe
Cloud
fl
are Cloud Signed & Filtering Safe
Amazon Cloud Signed & Filtering Safe
Net
fl
ix Cloud Signed & Filtering Safe
Wikimedia
Foundation
Cloud Signed & Filtering Safe
Scaleway Cloud Signed & Filtering Safe
• Source: isbgpsafeyet.com
57
Where do we go from here ?
• RPKI is only one of the steps towards full BGP
Validation


- Paths are not validated


• We need more building blocks


- BGPSec (RFC)


- ASPA (draft)


- AS-Cones (draft)
Questions
nathalie@ripe.net


rpki@ripe.net

Routing Security

  • 1.
    Nathalie Trenaman| 13April 2021 | Lebanese University, Faculty of Sciences Routing Security
  • 2.
  • 3.
    3 Internet building blocks ASN(Autonomous System Number)
  • 4.
    4 ASN (Autonomous SystemNumber) Internet building blocks ASN Addresses Interconnect Autonomous System
  • 5.
    5 Routing on theInternet “BGP protocol” Can I trust AS2? Routing table 
 194.x.x.x = AS2 Routing table 
 193.x.x.x = AS1 Is AS1 correct? AS1 
 193.x.x.x AS2 
 194.x.x.x AS2: “I have 194.x.x.x” AS1: “I have 193.x.x.x”
  • 6.
  • 7.
    7 Accidents Happen • FatFingers - 2 and 3 are really close on our keyboards…. • Policy Violations (leaks) - Oops, we did not want this to go on the public Internet - Infamous incident with Pakistan Telecom and YouTube
  • 8.
    8 Incidents Are Common •2019 Routing Security Review - 12,600 incidents - 4,4% of all ASNs affected - 3,000 ASNs are victims of at least one incident - 1,300 ASNs caused at least one incident Source: https://bgpstream.com
  • 9.
  • 10.
    10 Routing on theInternet Can I trust B? Routing table 
 194.x.x.x = B Routing table 
 193.x.x.x = A Is A correct? A 
 193.x.x.x B 
 194.x.x.x B: “I have 194.x.x.x” A: “I have 193.x.x.x” RIPE Database “Internet Routing Registry”
  • 11.
    11 Problem Statement • SomeIRR data can not be fully trusted - Accuracy - Incomplete data - Lack of maintenance • Not every RIR has an IRR - Third party databases need to be used - No verification of who holds IPs/ASNs
  • 12.
  • 13.
    13 Internet Routing Registry •Many exist, most widely used - RIPE Database - RADB • Verification of holdership over resources - RIPE Database for RIPE Region resources only - RADB allows paying customers to create any object - Lots of the other IRRs do not formally verify holdership
  • 14.
  • 15.
    15 Resource Public KeyInfrastructure • Ties IP addresses and ASNs to public keys • Follows the hierarchy of the registries • Authorised statements from resource holders - “ASN X is authorised to announce my Prefix Y” - Signed, holder of Y
  • 16.
    16 RPKI Certificate Structure MemberMember Member ROA ROA ROA Certificate hierarchy follows allocation hierarchy ARIN APNIC RIPE LACNIC AFRINIC
  • 17.
    17 RPKI Chain ofTrust ROA signature LIR’s Resources signature public key ALL Resources signature public key
  • 18.
    18 RPKI Chain ofTrust RIPE NCC Root Certificate Self-signed ALL Resources Root’s private key signature public key
  • 19.
    19 RPKI Chain ofTrust LIR Certificate Signed by the Root private key LIR’s Resources Root’s private key signature public key
  • 20.
  • 21.
    21 Two elements ofRPKI Signing Create your ROAs Validating Verifying others
  • 22.
  • 23.
    23 ROA (Route OriginAuthorisation) • A ROA is… • LIRs can create a ROA for each one of their resources (IP address ranges) • Multiple ROAs can be created for an IP range • ROAs can overlap
  • 24.
    24 What is ina ROA ? Prefix The network for which you are creating the ROA The ASN that’s supposed to be originating the BGP Announcement Origin ASN Max Length The Maximum prefix length accepted for this ROA
  • 25.
    25 RPKI Chain ofTrust ALL Resources LIR’s Resources Root’s private key signature signature public key public key
  • 26.
    26 Route Origin Authorisation Prefix isauthorised to be announced by AS Number LIR’s private key ROA signature
  • 27.
    27 RPKI Chain ofTrust ROA signature LIR’s Resources signature public key ALL Resources signature public key
  • 28.
    28 Hosted or DelegatedRPKI RIPE ROA ROA ROA ROA ROA Member Member Member ROA Member-X CA Member-Y CA RIPE NCC Hosted System
  • 29.
    29 Hosted RPKI • Automaticsigning and key roll overs - One click setup of resource certificate - User has a valid and published certificate for as long as they are the holder of the resources - All the complexity is handled by the hosted system • Lets you focus on creating and publishing ROAs - Match your intended BGP configuration
  • 30.
    30 Delegated RPKI • Runyour own Certification Authority software - Dragon research Labs, RPKI toolkit - NLNetLabs, Krill • Setup connection with RIPE NCC CA • Generate a certificate and get it signed by the parent CA • Run your own repository
  • 31.
    31 First login tothe dashboard
  • 32.
  • 33.
  • 34.
  • 35.
    /23 35 193.0.24.0/21 AS2121 Max Length: /21 ROA 193.0.24.0/21 193.0.24.0/22193.0.28.0/22 193.0.24.0/23 AS2121 Max Length: /24 ROA 193.0.30.0/23 AS2121 Max Length: /23 ROA ✖ ✖ ✔︎ /23 /23 /23 /23 /23 /24 /24 /24 /24 /24 /24 /24 /24 /24 /24 ✖ ✔︎✔︎✔︎✔︎ ✖ ✖ ✖ ✖ ✖ ✖ ✖
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
    40 Two elements ofRPKI Signing Create your ROAs Validating Verifying others
  • 41.
    41 Routing on theInternet Is A correct? A 
 192.0.2.0/24 B 
 193.0.24.0/21 A: “I have 192.0.2.0/24” 1. Create route authorisation record (ROA) 2. Validate route RPKI Repository A is authorised to announce 192.0.2.0/24 BGP
  • 42.
    42 Trust Anchor Locator(TAL) RIPE NCC ARIN APNIC AFRINIC LACNIC Validator Repository Repository Repository Repository Repository • Location of RIR repositories • Root’s public key TAL TAL TAL TAL List of ROAs Cerfificates
  • 43.
    43 RPKI Validators • Softwarethat creates a local “validated cache” with all the valid ROAs - Downloads the RPKI repository from the RIRs - Validates the chain of trust of all the ROAs and associated CAs - Talks to your routers using the RPKI-RTR Protocol
  • 44.
    44 Relying Party RIPE NCCARIN APNIC AFRINIC LACNIC Validator Repository Repository Repository Repository Repository List of ROAs Cerfificates
  • 45.
  • 46.
    46 Relying Party ROA AS111 10.0.7.30/22 AS22210.0.6.10/24 AS333 10.4.17.5/20 AS111 10.0.7.30/22 AS111 10.0.7.30/22 AS111 10.0.7.30/22 BGP Announcements BETTER ROUTING DECISIONS
  • 47.
    47 RIPE NCC Validator •https://github.com/RIPE-NCC/rpki-validator • Version 3.1 • Java-based, web interface, white-list functionality • Can speak RPKI-RTR
  • 48.
    48 Alternatives • All areopen source: - Routinator - https://github.com/NLnetLabs/ routinator/ - FORT - https://github.com/NICMx/FORT-validator/ - OctoRPKI - https://github.com/cloudflare/cfrpki - RPKI-client - https://rpki-client.org/ - Prover - https://github.com/lolepezy/rpki-prover - Rpstir2 - https://github.com/bgpsecurity/rpstir2
  • 49.
  • 50.
    50 Two elements ofRPKI Signing Create your ROAs Validating Verifying others
  • 51.
    51 ROA Validation • Routersreceive data from the validated cache via RPKI-RTR • Based on this and on BGP announcements, you have to make decisions - Accept or discard the BGP Announcement - As temporary measure, you could influence other attributes, such as Local Preference
  • 52.
    52 ROAs ROAs ROA Validation BGP Validation VALIDINVALID VALID INVALID UNKNOWN NOT FOUND
  • 53.
    53 Invalid ROA • InvalidROA - The ROA in the repository cannot be validated by the client (ISP) so it is not included in the validated cache • Invalid BGP announcement - There is a ROA in validated cache for that prefix but for a different AS. - Or the max length doesn’t match. • If no ROA in the cache then announcement is “unknown”
  • 54.
    54 Whitelisting • If thereis an invalid ROA for a network that’s important for you or your customers, you can whitelist it • This is done on your local validator software - It creates a “fake” ROA for the resources you want • It allows you to contact the operator to fix their ROA - Think of e-mail, contact forms, etc…
  • 55.
  • 56.
    Status of RPKIROV Name Type Details Status Telia Transit Signed & Filtering Safe Cogent Transit Signed & Filtering Safe GTT Transit Signed & Filtering Safe NTT Transit Signed & Filtering Safe Hurricane Electric Transit Signed & Filtering Safe Tata Transit Signed & Filtering Safe PCCW Transit Signed & Filtering Safe RETN Transit Partially Signed & Filtering Safe Cloud fl are Cloud Signed & Filtering Safe Amazon Cloud Signed & Filtering Safe Net fl ix Cloud Signed & Filtering Safe Wikimedia Foundation Cloud Signed & Filtering Safe Scaleway Cloud Signed & Filtering Safe • Source: isbgpsafeyet.com
  • 57.
    57 Where do wego from here ? • RPKI is only one of the steps towards full BGP Validation - Paths are not validated • We need more building blocks - BGPSec (RFC) - ASPA (draft) - AS-Cones (draft)
  • 58.