Linx851

© 2014 Renesys Corporation Confidential & Proprietary1
The End of Undetected BGP Route Hijacking
(Detecting malicious behavior using global data analytics)
Speaker
Earl Zmijewski, PhD
earl@renesys.com
+1 603-643-9300, x103
19 May 2014
LINX 85
Congress Centre, London

Agenda
•  A few examples
•  The Innocent, the Unusual and the Bizarre
•  What makes BGP vulnerable?
•  You already know this!
•  BGP hijacks in 2013
•  Major Sources
•  Techniques Employed
•  Overall Summary for the Year
•  Fundamental detection issues
•  Why is this so hard?
•  How Renesys benefits the community
•  We keep an eye on things so you don’t have to!

A Few Examples
(Looking for Anomalies in Traceroute and BGP Data)

Are either of these scenarios unusual & why?
4
Scenario 1
Traffic between two floors
of the same office building
in Singapore takes over
350ms round trip, traveling
via San Jose, California
Scenario 2
Traffic from Western
Europe to the US takes
around 70ms round trip,
traveling via Iceland’s
incumbent provider

Scenario 2 is a Man-in-the-Middle incident
5
Scenario 1
Innocent. NTT won’t peer
with Tinet in Singapore;
Tinet must drag traffic to
San Jose to hand it off to
NTT, who drags it home
again to Singapore.
Scenario 2
Unusual. Iceland’s Siminn
hijacked routes of major firms
for weeks and passed the
traffic along. In general,
Internet traffic never flows via
Iceland (cost, geography).

© 2014 Renesys Corporation Confidential & Proprietary6 6
Scenario 1
Latencies to Google’s
public DNS servers
increase dramatically from
South America
Scenario 2
Latencies to a Microsoft
network (hosting important
domains) decrease
momentarily from E. Europe
Are either of these scenarios unusual & why?

Scenario 2 involves route hijacking
7
Scenario 1
Unusual. Google departs
Brazil for unexplained reasons.
DNS queries answered from
California. No route hijacking
involved.
(See our October 30th blog.)
Scenario 2
Unusual. Microsoft network
(more specific of routed
prefix) is hijacked,
misdirection limited to
immediate vicinity. Not
Man-in-the-Middle! Traces
terminated at the hijacker.
Google left South America for months,
eventually came back and for the better.

Renesys collects BGP & traceroute data worldwide
– used to detect route hijacks, path and latency changes/impairments

What makes BGP
vulnerable?

Why & how infrastructure protocols fail?
– Internet runs on the “honor system” at all levels
•  BGP protocol is simple & the same on every router
§  BGP routers relay messages to neighbors about routes
§  Routes are constructed hop-by-hop, beyond the originator’s
control
•  BGP policy is complex, local, & without any global
coordination
§  Local policies for accepting, rejecting and propagating routes
§  This is good: Great flexibility to support business objectives
§  This is bad: Vulnerability to bogus route propagation

BGP Hijacks in 2013

Two Major Sources of MITM Hijacks in 2013
12
•  Beltelecom (AS 6697)
•  Belarus incumbent
•  Several downstream AS “origins”
•  Traces pass only through Beltelecom, not the claimed origin
•  Siminn (AS 6677)
•  Iceland incumbent
•  Several downstream AS “origins”
•  Traces pass only through Siminn, not the claimed origin
•  Siminn conceded redirection of traffic, but claimed router bug
•  A few other players, but more limited impacts.

NYC to Los Angeles during Belarus Hijacks
13

Germany to California during Iceland Hijacks
14

Denver to Denver during Iceland hijacks
15

BGP MITM requires both corrupt and clean paths
• At least three possible approaches:
•  Pilosov & Kapela, Defcon 2008
•  Announce more-specifics
•  AS path poisoning to maintain one clean path from attacker to victim
•  Never observed “in the wild”. See our Black Hat DC 2009 talk.
•  BGP Communities (first seen in 2013)
•  Used to limit propagation by upstream providers
•  Tricky to get right, especially since the upstreams might be interconnected
•  We’ve seen use of communities evolve over time
•  Announcements only to peers, not transit providers (first seen in 2013)
•  Peering relationships are between competitors
•  Peers will not propagate routes (otherwise they provide free transit!)
•  Requires provider with many peers in order to get appreciable amounts of
traffic

Exploiting BGP Communities
Hijacker announces prefix p to regional provider P with BGP community P:71990
What does this mean? Whois information tells us …
remarks To deny prefix propagation,
remarks use P:7DNNA, where
remarks D – Deny announcements:
remarks 1 – International peers
remarks 2 – In-country peers
…
remarks NN – Upstream:
remarks 01 – Provider 1
remarks 02 – Provider 2
…
remarks 99 – All Upstreams
…
remarks A – Action:
remarks 0 – Do not announce prefix
•  The hijacker tells P not to
announce p to any of its
international peers, implying
that it should announce p only
to its domestic peers and
customers.
•  In this way, the hijack of p is
constrained to a geographic
region.
•  Regional traffic for p is
misdirected to the attacker, who
can then forward it onward to its
rightful owner via a clean path
through another provider.

Overall Statistics: 1 Jan – 31 Dec 2013
•  Major hijacks occurred on 102 days in this time period
(29% of the days in 2013), most were MITM
•  Over almost 1,500 networks (prefixes) were targeted, geolocating
to over 150 cities worldwide. Targets included financial services,
various governments, NSPs, content providers.
•  Hijacked networks contain over 500,000 domains (FQDNs)
•  Traffic detouring lasted from minutes to months! One hijack
persisted from late October 2013 until late January 2014!
•  Techniques were refined over time.
•  Global impact varied depending on providers accepting the bogus
routes and techniques employed

Hijacks by City: 1 Jan – 31 Dec 2013

Global Impacts – 9 February 2013
– We identify the level of acceptance of bogus routes, by country
Geolocation of network subject to traffic detouring

Global Spread of Bogus routes – 27 Feb 2013
– Major US provider, one ccTLD, many others
Geolocation of network subject to traffic detouring

How can all BGP hijacks
be discovered?

•  Each of the 42,000+ ASes on the Internet could …
•  Use their favorite route monitoring service or
•  Publish their routing policies in an IRR and keep them updated,
allowing others to do the monitoring
•  Use global historical routing data to report on anomalies
•  Find suspicious origins, AS paths
•  Use global traceroute data to look for MITM or hosts in the
attacker’s network that answer (e.g., Turkey answering for
Google’s DNS servers).
•  Difficulty: The Internet is a very dynamic place. Easy to generate
an overwhelming number of false positives.
Two Approaches for Complete Global Coverage

•  Suppress alerts for new originations that meet certain
conditions:
•  Origination has been seen in the past (e.g., traffic engineering)
•  Origination seen by very few peers or a very short duration
•  New origin is part of the same organization as typical origin
(e.g., AT&T has over 100 ASNs)
•  New origin is a DDoS mitigation service (e.g., Prolexic)
•  Obvious typos (e.g., edit distance 1 from legitimate announcement)
•  … many more rules to filter innocent errors / typical usage
Eliminating False Positives

•  Score new originations by unusualness
•  Hijacking AS now operating in a new, distant geography?
•  Hijacked prefix hosting important domains?
•  Traces pass through hijacking AS onto legitimate AS (MITM)?
•  Traces terminate at hijacking AS (impersonation)?
•  … many more rules to score novelty and call out the most
suspicious originations for review
Look for Novelty in the Rest

•  We get a handful of reports per day, many are false
positives and easily dismissed. Occasionally, we get a “live
one”.
•  Here is one automated report entry from April 2014:
AS6697 (Beltelecom, BY) announced 1 pfx(s) that are
potentially hijacks of currently routed address space:
XXX.YYY.151.0/24 (ZZZZ, GB), seen by 310 peers for an
average of 1.16 hours. This is a more specific of
prefix: XXX.YYY.128.0/19 (ZZZZZ), Origin=NNNN (ZZZZZ,
GB)
Automated BGP Hijack Reporting

Fundamental Detection
Issues

Fundamental Detection Issues I
•  Internet infrastructure incidents impacting a specific
enterprise typically occur beyond the enterprise’s
perimeter and may be invisible from the enterprise’s
vantage points
•  Widely distributed global sensor network is necessary
to minimize false negatives (otherwise, regional or
provider specific events can be missed)
•  Multiple data sources and sophisticated analytics are
necessary to detect deviations from “normal” and
minimize false positives

Fundamental Detection Issues II
How do we distinguish malicious/suboptimal from legitimate?
Best case: An enterprise knows “Ground Truth” for what it cares about.
Monitoring system then alerts on any deviations.
Actual case: “Ground Truth” is rarely fully and correctly specified, and
never will be for most organizations.
Monitor long enough to determine the “approximately
correct”★ state, alert on deviations, automate response
processing
★ with thanks to Leslie Valiant, author of Probably Approximately Correct.

How Renesys Benefits the
Community

The End of Undetected BGP Route Hijacking
•  It's no longer acceptable for route hijacking to go
unobserved. It's too important to only tell the people who
pay.
•  Renesys watches the entire Internet, 24x7, and the vast
majority of the affected parties are not our customers.
•  We tell them anyway.
•  Your support makes that possible; without LINX and the
other IXes, we would not have the global data to
understand “normal routing” in enough detail to support
this activity.

We even alert on silly things …
•  For example, global Carrier-Grade NAT announcements:
•  Cogent and
TransTelecom
stopped after we
alerted them.
•  MegaFon stopped
briefly in April.
•  The routing table is a
little cleaner as a
result.

•  A new more-specific prefix announced from a seemingly
unrelated origin will always generate an alert.
•  The following example is from 8 May 2014.
AS393327 (THE GEORGE W. BUSH FOUNDATION, US) announced
1 pfx(s) that are potentially hijacks of currently
routed address space:
12.203.53.0/24 (GEORGE W. BUSH PRESIDENTIAL CENT,
US), seen by 394 peers for an average of 0.57 hours.
This is a more specific of 12.128.0.0/9 (AT&T Bell
Laboratories, US) Origin=7018 (AT&T, US)
•  First US president with his own Autonomous System.
But false positives are unavoidable …

Thank you

Linx851

Recommended

Recommended

More Related Content

Similar to Linx851

Similar to Linx851 (20)

Linx851