The document discusses the operational management of a 2 Tbps global IP network operated by Fiberring using open source tools. Key topics include network monitoring, capacity planning, attack detection, and device backup strategies, highlighting the importance of scalability and ease of integration. The conclusion emphasizes the effectiveness of open source solutions supported by skilled personnel in running a global network efficiently.

1. b
Running a 2 Tbps global IP network
using Open Source tools
Bart van der Sloot (Managing Director)
Samer Abdel-Hafez (Network Design Engineer)

2. Agenda
1. Introduction to FiberRing and AS16265:
a 2Tbps Global IP Network
2. Network management and operations challenges
3. Common Requirements: why Open Source?
4. Network Monitoring
5. Capacity Planning
6. Attack Detection
7. Device back-ups
8. Considerations for the future
9. Conclusions

3. 1. Ocom: 1 group, 4 operating companies
Infrastructure
as a Service (IaaS)
Bare Metal Servers
CDN
Cloud
Colocation
Data Centers
Amsterdam
US East Coast
Data Center
Construction
Energy Efficient
Modular
Network Services
IP Transit
Internet Access
Longhaul MPLS
Remote IX
Amsterdam metro
services (dark fiber,
waves, MPLS)
Locations
Amsterdam, Frankfurt
US East Coast, US West Coast
Singapore
More to follow
Team
> 350 People
> 60 Software Developers
>20 Nationalities

4. 1. FiberRing Global Network

5. 1. FiberRing Global Network
Total traffic: 2.5Tbps
(>1Tbps growth in last 12 months, >4 Tbps total capacity)
• AS16265 – LeaseWeb Global
Backbone
• AS38930 – FiberRing
• Various customer networks
Traffic distribution
> 500 peers
> 30 private peers
> 30 Internet Exchanges
> 10 Transit providers
3 100GE ports (Ams-IX)
> 480 10GE ports peering/Transit
> 120 10G waves in backbone
….a challenge to manage!

6. 2. Challenges to address
• Network monitoring
• Capacity planning
• Attack detection
• Network devices configurations backup

7. 3. Common requirements
1. Easy to use and maintain
2. Scalability up to 1000’s “objects”
3. Trivial integration with internal systems
4. Long life span expectation
5. Easy access to updates
6. Extensive documentation provided

8. 3. Why open source tools
• Increased control over development
• Largely tested and documented
• Cost reduction
• Easy to extend to support own code

9. 4. Network monitoring
• OpsView (nagios)
• NMIS
• Custom RRDs
• Custom code for specific monitoring
(e.g. BGP state)

10. 5. Capacity planning
• PMACCT combined with UI developed
in-house
• Focus on traffic trends per destination AS
• Immediate results are measured by
monthly cost reduction
• Long term results are measured in
customer response
• Overall quality has greatly improved

11. 6. Attack detection
• Nfdump
• Small dumps covering 60 seconds spans
• Immediate view of the attack sources and
destinations
• NOC response is almost instantaneous
• Very little training needed
• Huge reduction in sourced attacks
• Customer experience improved
• Requires resources & expertise to maintain

12. 7. Network devices config backup
• Oxidized and GitLab
• Oxidized is a RANCID replacement
• Design team contributes to Oxidized
development
• Oxidized is easy to integrate with internal
systems
• Oxidized uses GitLab to store configuration
updates
• GitLab provides extensive overview over HTTP
and ability to store internal repository for free

13. 8. Future considerations
• Replace RRDs with time series database
(influxdb)
• Extend oxidized-script tools to our needs
• Logs visualization tools
• Hire developers to enhance the existing
tools
• Early discussions on the development of
a new NMS
• Automation

14. 8. Conclusions
• We can run a global IP Network with ~ 15
people in Network Design and Support
• Including network expansions, new PoPs, peering & transit
negotiations
• Including managing internal networks of our customers, e.g.
LeaseWeb
• There are good Open Source tools available to support
this, but they require staff that understand both
“networking”, and “programming”
If you want to hop onto our network and try our services: come to our booth!
- Special offer for Remote Ams-IX connectivity –
Interested in joining a global fast growing company?
We are looking for Design & Support Engineers in Amsterdam….

15. b
Questions?
Bart van der Sloot b.vandersloot@fiberring.com
Samer Abdel-Hafez s.abdel@fiberring.com
Colin Boekhout c.boekhout@fiberring.com