Measuring the Internet Economy: How Networks Create Value

1. Measuring the Internet Economy:
How Networks Create Value
Bill Woodcoc
k
Executive Directo
r
Packet Clearing House
Version 1.
0
bdNOG1
3
June 10-12, 2021

2. Why Measure the Internet?
Tactical reasons:
To understand where the value of the product you’re selling comes from
As input to calculating Average Per-Bit Delivery Costs (APBDC)
So that you can bill your customers in di erent and more useful ways
To optimize and plan network interconnection (peering and transit)
Strategic reasons:
So you’ll know your business when you talk with your regulator or customers
To justify infrastructural investment and attention from policy-makers

3. Why Measure the Internet?
Although economists argue about methodologies and speci cs, there’s general
agreement that the Internet is responsible for a quarter to a third of new productivity in
most OECD countries, and a double-digit percentage globally.
It’s the responsibility of governmental policy-makers, of regulators, and of people in the
involved industries, to understand and support such engines of growth.
The Internet allows people without work to
fi
nd jobs, children to continue their education
during a pandemic, businesses to receive orders and ful ll them. The economic activity
produces tax revenue which pays for schools, roads, law enforcement, and healthcare.
Regardless of whether individual customers appreciate the work that you do, society
and posterity depend upon your work and its success.

4. Theodore Vail used the Network E
ff
ect to justify an AT&T monopoly in 1908.
Value increases as a function of the number of users of a compatible
product or service.
In communications, value increases as a function of the number of other
people who can be reached through a network.
The Network Effect

5. The Network Effect
Users: 2
Value: 1
Value per user: 0.5
Marginal value
of next user: 2
Users: 4
Value: 6
Value per user: 1.5
Marginal value
of next user: 4
Users: 6
Value: 15
Value per user: 2.5
Marginal value
of next user: 6
Users: 5
Value: 10
Value per user: 2
Marginal value
of next user: 5
Users: 3
Value: 3
Value per user: 1
Marginal value
of next user: 3

6. The Network Effect
Source: CAIDA

7. The Network Effect
Network e
ff
ects apply not only to underlying communications infrastructure like ber
cables, but at all layers of the network. Some examples:
Layer 2: Ethernet adjacencies
Layer 3: Global reachability with the Internet Protocol
Layer 5: BGP route-servers like those which allowed the IXPs in Jakarta and Sao
Paulo to so quickly outpace those of Europe and the United States
Layer 8: Speakers of a common language, or people who share a common
understanding of a scienti
fi
c principle

8. Internet Service Provider
Lifecycle Phase 1:
Simple Aggregator
Single Transit Provider
Customers
ISP Network
IXPs

9. Internet Service Provider
Lifecycle Phase 1:
Simple Aggregator
Single Transit Provider
Customers
ISP Network
IXPs
Vertical connections are
transit: service travels
downward, money travels
upward.

10. Redundant Transit Providers
Customers
ISP Network
IXPs
Internet Service Provider
Lifecycle Phase 2:
Redundancy & Efficiency

11. Redundant Transit Providers
Customers
ISP Network Single IXP
IXPs
Internet Service Provider
Lifecycle Phase 3:
Local Peering

12. Redundant Transit Providers
Customers
ISP Network Single IXP
IXPs
Horizontal connections are peering:
bandwidth is exchanged, but money
is not.
Internet Service Provider
Lifecycle Phase 3:
Local Peering

13. Redundant Transit Providers
Customers
ISP Network
Multipl
e
IXPs
IXPs
Internet Service Provider
Lifecycle Phase 4:
Diverse Peering

15. IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

16. IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

17. IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

18. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

19. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

20. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
C
C

21. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2
1
1 1
2
C
C
1

22. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
3
2
2
2
1
1 1
3
2
4
C
C
1

23. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
IXP IXP IXP
IXP
4
4
3
2
2
2
1
1 1
3
2
IXP
4
C
C
1

24. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
IXP IXP IXP
IXP
4
3
2
2
2
1
1 1
3
2
4
IXP
4
C
C
1

25. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
IXP IXP IXP
IXP
4
3
2
2
2
1
1 1
2
3
4
IXP
4
C
C
1

26. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
IXP IXP IXP
IXP
4
3
2
2
2
1
1 1
3
4
IXP
4
C
C
1
2

27. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

28. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
Transit Cone
Customers: 3
Value: 6

29. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
Transit Cone
Customers: 5
Value: 15

30. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
Transit Cone
Customers: 9
Value: 45

31. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
Transit Cone
Customers: 16
Value: 136

32. C C
C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
Customers: 24
Value: 300
Transit Cone + Peering

33. Source: CAIDA

34. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1

35. C C
C C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
C
IXP
IXP IXP IXP
IXP
4
4
4
3 3
2
2
2 2
1
1
1
1
IXP
IXP IXP
C
C
C
C
C C

36. So What Do We Measure?
The vast majority of bandwidth is produced in Internet exchange points.
Bandwidth produced in IXPs comes from interconnections between participating
networks across the switch fabric, or across passive ber crossconnects.
Aggregate bandwidth produced on the switch fabric is typically a public number.
The amount of bandwidth produced on private pieces of ber is known only to
the two parties to each interconnection, but traceroutes will reveal whether the
connection is occurring across the switch fabric or a private connection.
A tiny remaining fraction of bandwidth is produced within transit cones, and
traceroute analysis can identify this, but not quantify it, as well.

37. So What Do We Measure?
The ratio of bandwidth produced through each of these three mechanisms is
not directly measurable, for many reasons.
The ratio of interconnections facilitated by each of the three mechanisms can
be determined by analyzing traceroutes, but each of the three types have
characteristically di
ff
erent tra
ffi
c capacities and average utilization.
Di
ff
erent business practices and regulatory environments strongly a ect the
ratios of tra
ffi
c generated using each mechanism:
• Regulatorily-protected monopoly incumbents in many developing economies
• Data-collection-driven preference for switching over crossconnects

38. So What Do We Measure?
More speci
fi
cally, at exchange points, we sum the count of outbound bits
across all ISP-facing switch ports.
Not inbound, because that wouldn’t catch multicast, but would catch
bogons and subsequently-mitigated DDoS.
Not all ports, because that would double-count inter-switch tra c.
If we want to get really fancy, we can deduplicate broadcast tra c, but
there really shouldn’t be broadcast tra
ffi
c on an IX subnet.

39. So What Do We Measure?
In an ISP network, measurement is used for both engineering and billing.
We typically use
fl
ow-based measurement, in addition to summing bits in
both directions at the peering and upstream transit edges.
This allows a “local tra
ffi
c is free, international tra c is metered” two-
category billing model.
If you capture customer-facing port bit-counts as well, that enables a “on-
net, peer, and transit” three-category model.
Old-style customer-port-bit-count-only doesn’t allow di erentiated
pricing, nor does it tell you which customers are costing you most.

40. What Questions Can We Answer?
If we measure Internet tra
ffi
c comprehensively, ideally using both bit-counts
and sparsely-sampled
fl
ows, we can answer many questions:
• Which of our customers make us pro
fi
table, and which cost us money?
• Is our country or region a net importer or net exporter of bandwidth?
• Where is the bandwidth our users are consuming produced?
• Where are the consumers of the bandwidth we’re producing?
• Which protocols and applications are gaining momentum, and which are
falling into disuse?

41. Thanks, and Questions?
Bill Woodcoc
k
Executive Directo
r
Packet Clearing Hous
e
woody@pch.ne
t
https://pch.net