Using Docker and SDN for telco-grade
applications
@NicoJanssens – Bell Labs
@FlorianOtel – Nuage Networks
KEY TAKE-AWAYS
Evolution: Traditional telco -> agile methodologies and micro-service
oriented solutions
Use Case: Using Docker as enabling technology to facilitate this
transformation
Lesson learned: Advanced SDN support required to facilitate more
sophisticated deployment scenarios
3
COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
WHO ARE WE?
Nico Janssens
Bell Labs researcher @ ALU
Virtualization enthusiast
Hacker
Using Docker since v0.7 (on a raspberry
pi …)
Florian Otel
EMEA Director for Business Development
and Strategy @ Nuage Networks
First time using Docker: Oct 2013
Operational costs pressures push Telcos to virtualize environments while
preserving non-functional requirements
• 5 nines availability
• Reliability
• Performance and response times
MOVING AWAY FROM TRAD TELCO SERVICE DESIGN
Additional non-functional
requirements to take into account
• Scalability
• Elasticity
• Agility
• Operability and portability
Low overhead
Portability
Micro-service architectures
Active eco-system
Public image registries
Facilitates dev-ops methodology
MOVING AWAY FROM TRAD TELCO SERVICE DESIGN
BELL LABS VIEW
DOCKER AND TELCO SERVICES
BELL LABS PROJECTS
New Network Analytics Service
Next-gen communication
services
New Communication Service
Key Goal: Simplify interactions among people, machines,
and their environments
• From transaction-oriented Web model to persistent
conversations
• Uniform interaction model for people, machines, and
objects
• Rich context-based communications and collaboration
Mobile 5G Network
WIFI Network
Edge
Cloud
HubDevice
Distributed Cloud
Platform
For Heterogeneous
HW infrastructure
Home
High Performance & Predictability
Small footprints
Central
Cloud
New Home/IoT Service Platform
Sensor network
WiFi network
Mobile 5G network
USE CASE ARCHITECTURE
MICRO-SERVICE CHAT ARCHITECTURE
load balancer
WS server
message
broker
key-value
database
document-
oriented db
Stateless Workers
Chat-Thread Mgr
User Mgr
Group Mgr
User View Mgr
User Presence Mgr
Push Notification Adapter
Call Mgr
…
TURN/STUN Media Server
SDK
mobile client
web browser
machine buddies
web storage
distributed
object store
and FS
RT + batch
analytics
HTTP server
location and
geo-fencing
service
USE-CASE DEVELOPMENT – Why we chose Docker
8
Low overhead ✔
Portability ✔
Micro-service architectures ✔
Active eco-system ✔
Public image registries ✔
Facilitates dev-ops methodology ✔
…
DOCKER TOOLBOX
DOCKER COMPOSE
DOCKER REGISTRY
INITIAL PRODUCTION DESIGN
20 node cluster with RANCHER and DOCKER
Datacenter
Host Host Host
…
Rancher
Agent
Rancher
Agent
Private IP Net
Server
Firewall
NAT, DHCP,
LB,DNS
Docker
Daemon
Docker
Daemon
VM
Rancher
Agent
Docker
Daemon
WorkerContainer WorkerContainer
Container
Native Process
Rancher Cluster Manager
• Native Docker support
• Re-usage of Docker Compose
files
• Low entry hurdle
• Dashboard
• Redundancy
Public Cloud
Operator
MISSING: Mature SDN support for federation
Mgmt Server
Rancher
Mgmt
Docker
Registry
Mgmt Server
Rancher
Mgmt
Docker
Registry
Server
Firewall
NAT, DHCP,
LB,DNS
TELCO-GRADE SERVICES – Network fabric requirements
 Hybrid deployment scenarios
 Policies
 Stringent QoS requirements
 Security constraints
10
11/23/2015
Software Defined Networking
VPN
VPN
VPN
VPN
Internet
Network
Virtualization
Massive IP
Scale
Policy Based
Endpoint Attachment
Proven technologies of VPNs (network virtualization), IP (scaling) and
Mobile ( policy) applied to SDN
Nuage templates and role-based workflow
Compute
Management
Tenant / Application
Request
Compute Request
completed in sub-second
time
Service velocity is not hindered by network-centric operations
Auto-instantiation
00:01
IP
Address
WAN interconnect
Policy/Security
Zones
L2 /L3
Service AD
Network Change
Completed automatically
Service chaining
Template->Instances
Virtualized Services Directory
(VSD)
Virtualized Services Controller
(VSC)
Virtual Routing & Switching
(VRS)
Nuage Networks Virtualized Services Platform (VSP)
• Business/IT policy engine (definition of rules)
• Service templates and Analytics
• SDN Controller
• Instantaneous programming of the network
• Virtualized service edge
• Event-driven automation
• Integration of bare-metal assets
Provide unified IP Fabrics
Unified IP Fabric
Secure GW Secure GW
VRS
VRS
Virtualized
Services
Controller
Virtualized
Services
Directory
VRS
VRS
Virtualized
Services
Controller
Virtualized
Services
Directory
Public Cloud Private Cloud
MP-BGP
Exchange Network information
XMPP (Span of control)
Exchange Application Policies
- Home/Visiting Policy approach
Legacy VM
L2 and/or L3
(VLAN, VXLAN, GRE)
Virtual Router and Switch
(VRS)
VRS-H*
VRS-G
VRS-X
VRS-V
Citrix XEN
Hypervisors
VMware vSphere
Hypervisors
Microsoft Hyper-V
Hypervisors
SW Gateway for
Bare Metal Servers
&
Appliances
KVM
Hypervisors
VRS-K
Hardware gateway
for Bare
Metal/MaaS
VRS-T
L2-L4 VIRTUAL SWITCH
• Open vswitch-based
• Provides both VXLAN
and MPLSoGRE
tunnel encapsulation
options
• Programmed through
OpenFlow from VSC,
Encapsulates VM flow
into preferred protocol
(L2 or L3)
• Detects VM
instantiation and
teardown
OpenFlow
XMPP
Virtualized
Services
Directory
Virtualized
Services
Controller
Virtual
Routing &
Switching
Host
Virtual
Routing &
Switching
Hypervisor
Virtual Router and Switch (VRS)
• VIRTUAL MACHINE
BASED
• SDN CONTROLLER
• POWERED BY
SERVICE ROUTER
OPERATING SYSTEM
(SROS)
• PEERING &
FEDERATION
• AUTO-DISCOVERY
• TENANT SLICING
Virtualized Services
Controller
(VSC)
SROS
BASED
SMNP/CLI
BGP/IGP
SERVICE MGR
Forwarding dB
RIB/FIB
XMPP
OPENFLOW
Control path
to VRS
Message bus for:
Event Notifications
Policy Push
Security
Load Balancer
OpenFlow
XMPP
Virtualized
Services
Directory
Virtualized
Services
Controller
Virtual
Routing &
Switching
Virtualized Services Controller (VSC)
Host
OpenFlow
XMPP
Virtualized Services
Directory
(VSD)
• VIRTUAL MACHINE
BASED
• SERVICE DEFINITION
• POLICY
ESTABLISHMENT
• SERVICE TEMPLATING
• ANALYTICS ENGINE &
REPORTING
NETWORKS
SECURITY
QOS
STATISTICS
ZONE POLICIES:
WEB ACCESS
BACKEND LOGIC
ETC.
CRM APP :- VM
“80MBPS – REAL
TIME”
THRESHHOLD
ALARM
UI
REST API
Message
Bus
Domain
Zones
Subnets
Policies
VPN
Public
Internet
Virtualized
Services
Directory
Virtualized
Services
Controller
Virtual
Routing &
Switching
Virtualized Services Director (VSD)
UI
Host
Network virtualization with Nuage Networks
Federated Datacenter Services (multiple CMS)
Cloud Service
Management Plane
Datacenter
Control Plane
Datacenter
Data Plane
Brooklyn Datacenter - Zone
1
Virtualized
Services
Directory
Virtualized
Services
Controller
HOST
HOST
HOST
HOST
HOST
HOST
CloudManagertoHypervisorcommunications
HOST
HOST
HOST
Brooklyn Datacenter - Zone
2
HOST
HOST
HOST
Network Services
Manhattan Datacenter - Zone
2
Federation of
Controllers
Edge
Router
MPLS
(MP-
BGP)
Service
Provider
Control Plane
Service Provider
Data Plane
Business
VPN Service
Private
Datacenter
Domain
Subnets
VPNInternet
Zones
Policies
Live Demo
Please pray the Demo Gods be with us today
… else we will show you slides with screenshots 
Additional information
Nuage SDN resources -- repos:
* official / stable: github.com/nuagenetworks/
* unofficial / “radioactive”:
github.com/FlorianOtel/docker-vsp-plugin
github.com/FlorianOtel/gonuageshell
SDN performance metrics: Network connectivity and policy for 100’000
Docker containers in 7.5 minutes
(demonstrated at Network Tech Field Day 2014)
Thank you!
Nico Janssens & Florian Otel
@NicoJanssens
@FlorianOtel

DockerCon EU 2015: Using Docker and SDN for telco-grade applications

  • 1.
    Using Docker andSDN for telco-grade applications @NicoJanssens – Bell Labs @FlorianOtel – Nuage Networks
  • 2.
    KEY TAKE-AWAYS Evolution: Traditionaltelco -> agile methodologies and micro-service oriented solutions Use Case: Using Docker as enabling technology to facilitate this transformation Lesson learned: Advanced SDN support required to facilitate more sophisticated deployment scenarios
  • 3.
    3 COPYRIGHT © 2014ALCATEL-LUCENT. ALL RIGHTS RESERVED. WHO ARE WE? Nico Janssens Bell Labs researcher @ ALU Virtualization enthusiast Hacker Using Docker since v0.7 (on a raspberry pi …) Florian Otel EMEA Director for Business Development and Strategy @ Nuage Networks First time using Docker: Oct 2013
  • 4.
    Operational costs pressurespush Telcos to virtualize environments while preserving non-functional requirements • 5 nines availability • Reliability • Performance and response times MOVING AWAY FROM TRAD TELCO SERVICE DESIGN
  • 5.
    Additional non-functional requirements totake into account • Scalability • Elasticity • Agility • Operability and portability Low overhead Portability Micro-service architectures Active eco-system Public image registries Facilitates dev-ops methodology MOVING AWAY FROM TRAD TELCO SERVICE DESIGN BELL LABS VIEW
  • 6.
    DOCKER AND TELCOSERVICES BELL LABS PROJECTS New Network Analytics Service Next-gen communication services New Communication Service Key Goal: Simplify interactions among people, machines, and their environments • From transaction-oriented Web model to persistent conversations • Uniform interaction model for people, machines, and objects • Rich context-based communications and collaboration Mobile 5G Network WIFI Network Edge Cloud HubDevice Distributed Cloud Platform For Heterogeneous HW infrastructure Home High Performance & Predictability Small footprints Central Cloud New Home/IoT Service Platform Sensor network WiFi network Mobile 5G network
  • 7.
    USE CASE ARCHITECTURE MICRO-SERVICECHAT ARCHITECTURE load balancer WS server message broker key-value database document- oriented db Stateless Workers Chat-Thread Mgr User Mgr Group Mgr User View Mgr User Presence Mgr Push Notification Adapter Call Mgr … TURN/STUN Media Server SDK mobile client web browser machine buddies web storage distributed object store and FS RT + batch analytics HTTP server location and geo-fencing service
  • 8.
    USE-CASE DEVELOPMENT –Why we chose Docker 8 Low overhead ✔ Portability ✔ Micro-service architectures ✔ Active eco-system ✔ Public image registries ✔ Facilitates dev-ops methodology ✔ … DOCKER TOOLBOX DOCKER COMPOSE DOCKER REGISTRY
  • 9.
    INITIAL PRODUCTION DESIGN 20node cluster with RANCHER and DOCKER Datacenter Host Host Host … Rancher Agent Rancher Agent Private IP Net Server Firewall NAT, DHCP, LB,DNS Docker Daemon Docker Daemon VM Rancher Agent Docker Daemon WorkerContainer WorkerContainer Container Native Process Rancher Cluster Manager • Native Docker support • Re-usage of Docker Compose files • Low entry hurdle • Dashboard • Redundancy Public Cloud Operator MISSING: Mature SDN support for federation Mgmt Server Rancher Mgmt Docker Registry Mgmt Server Rancher Mgmt Docker Registry Server Firewall NAT, DHCP, LB,DNS
  • 10.
    TELCO-GRADE SERVICES –Network fabric requirements  Hybrid deployment scenarios  Policies  Stringent QoS requirements  Security constraints 10 11/23/2015
  • 11.
    Software Defined Networking VPN VPN VPN VPN Internet Network Virtualization MassiveIP Scale Policy Based Endpoint Attachment Proven technologies of VPNs (network virtualization), IP (scaling) and Mobile ( policy) applied to SDN
  • 12.
    Nuage templates androle-based workflow Compute Management Tenant / Application Request Compute Request completed in sub-second time Service velocity is not hindered by network-centric operations Auto-instantiation 00:01 IP Address WAN interconnect Policy/Security Zones L2 /L3 Service AD Network Change Completed automatically Service chaining Template->Instances
  • 13.
    Virtualized Services Directory (VSD) VirtualizedServices Controller (VSC) Virtual Routing & Switching (VRS) Nuage Networks Virtualized Services Platform (VSP) • Business/IT policy engine (definition of rules) • Service templates and Analytics • SDN Controller • Instantaneous programming of the network • Virtualized service edge • Event-driven automation • Integration of bare-metal assets
  • 14.
    Provide unified IPFabrics Unified IP Fabric Secure GW Secure GW VRS VRS Virtualized Services Controller Virtualized Services Directory VRS VRS Virtualized Services Controller Virtualized Services Directory Public Cloud Private Cloud MP-BGP Exchange Network information XMPP (Span of control) Exchange Application Policies - Home/Visiting Policy approach Legacy VM
  • 15.
    L2 and/or L3 (VLAN,VXLAN, GRE) Virtual Router and Switch (VRS) VRS-H* VRS-G VRS-X VRS-V Citrix XEN Hypervisors VMware vSphere Hypervisors Microsoft Hyper-V Hypervisors SW Gateway for Bare Metal Servers & Appliances KVM Hypervisors VRS-K Hardware gateway for Bare Metal/MaaS VRS-T L2-L4 VIRTUAL SWITCH • Open vswitch-based • Provides both VXLAN and MPLSoGRE tunnel encapsulation options • Programmed through OpenFlow from VSC, Encapsulates VM flow into preferred protocol (L2 or L3) • Detects VM instantiation and teardown OpenFlow XMPP Virtualized Services Directory Virtualized Services Controller Virtual Routing & Switching Host Virtual Routing & Switching Hypervisor Virtual Router and Switch (VRS)
  • 16.
    • VIRTUAL MACHINE BASED •SDN CONTROLLER • POWERED BY SERVICE ROUTER OPERATING SYSTEM (SROS) • PEERING & FEDERATION • AUTO-DISCOVERY • TENANT SLICING Virtualized Services Controller (VSC) SROS BASED SMNP/CLI BGP/IGP SERVICE MGR Forwarding dB RIB/FIB XMPP OPENFLOW Control path to VRS Message bus for: Event Notifications Policy Push Security Load Balancer OpenFlow XMPP Virtualized Services Directory Virtualized Services Controller Virtual Routing & Switching Virtualized Services Controller (VSC) Host
  • 17.
    OpenFlow XMPP Virtualized Services Directory (VSD) • VIRTUALMACHINE BASED • SERVICE DEFINITION • POLICY ESTABLISHMENT • SERVICE TEMPLATING • ANALYTICS ENGINE & REPORTING NETWORKS SECURITY QOS STATISTICS ZONE POLICIES: WEB ACCESS BACKEND LOGIC ETC. CRM APP :- VM “80MBPS – REAL TIME” THRESHHOLD ALARM UI REST API Message Bus Domain Zones Subnets Policies VPN Public Internet Virtualized Services Directory Virtualized Services Controller Virtual Routing & Switching Virtualized Services Director (VSD) UI Host
  • 18.
    Network virtualization withNuage Networks Federated Datacenter Services (multiple CMS) Cloud Service Management Plane Datacenter Control Plane Datacenter Data Plane Brooklyn Datacenter - Zone 1 Virtualized Services Directory Virtualized Services Controller HOST HOST HOST HOST HOST HOST CloudManagertoHypervisorcommunications HOST HOST HOST Brooklyn Datacenter - Zone 2 HOST HOST HOST Network Services Manhattan Datacenter - Zone 2 Federation of Controllers Edge Router MPLS (MP- BGP) Service Provider Control Plane Service Provider Data Plane Business VPN Service Private Datacenter Domain Subnets VPNInternet Zones Policies
  • 19.
    Live Demo Please praythe Demo Gods be with us today … else we will show you slides with screenshots 
  • 20.
    Additional information Nuage SDNresources -- repos: * official / stable: github.com/nuagenetworks/ * unofficial / “radioactive”: github.com/FlorianOtel/docker-vsp-plugin github.com/FlorianOtel/gonuageshell SDN performance metrics: Network connectivity and policy for 100’000 Docker containers in 7.5 minutes (demonstrated at Network Tech Field Day 2014)
  • 21.
    Thank you! Nico Janssens& Florian Otel @NicoJanssens @FlorianOtel

Editor's Notes

  • #5 Evolution: Dedicated HW + middleware + dedicated apps -> Software decoupled from HW
  • #7 Chat-based communication: people, services, devices) Context awareness: call topics, personalized experience, …) Multi-media services: live video distribution, augmented video experience, …) Innovation: Rich Interactive Messages – Move beyond text, photos, and videos. Bring your messages to life. Zero-Latency Chat – Enable chat to control devices and objects. Fast Buddy Building – Create new chat buddies as quickly as a Web page.
  • #8 Intensive data analytics support – RT as well as historical data processing QoS critical multi-media services Message brokering support facilitating (semi-) RT A2P message delivery Various plugins to integrate with external services
  • #9 Since the teams working on this research project are located in different labs, Docker Toolbox has been a great help to quickly setup a dev environment without spending/wasting lots of cycles for people to get bootstrapped. Additionally, Docker Compose has been an interesting orchestration solution + we’ve been extensively using Registry to distribute our Docker images. Docker has proven to be a great vehicle for us so far to rapidly develop and deploy new micro service systems like the comm solution that we’re presenting as a Use Case during the presentation. Docker enables us to focus on the core functionalities of our systems, without spending/wasting lot’s of cycles on operational issues. In addition, Docker has established a very active eco-system, making it hard for instance to _not_ find a docker image of a particular server that is needed to build, test and deploy new features.
  • #10 Active pre-production testing, we deployed the system for internal use by Alcatel-Lucent employees We’re currently also evaluating the latest GA version of Docker Swarm.
  • #14 13
  • #15 Public cloud is not as flexible to networking as people expect -> build a flexible fabric that is agnostic Standardize protocols like VXLAN (MPLS does not work 5-10% model B interconnects max + not possible to use MPLS in Clouds + on top Intercloud uses proprietary protocols where we use standardized solutions - Unified fabric between Public, Private and enterprise sites - Make environment that can operate independently - own span of control - seamless network policies
  • #19 Private VPN networks of major Service Providers utilize MPLS to deliver IP-VPN’s. These IP-VPNs communicate internally (between the SP’s PE routers) using MP-BGP to signal paths and route changes within the VPN instance. It makes sense for Nuage to use these existing protocols within the wide area to facilitate the seamless extension of services from the datacenter to the tenants IPVPN. Nuage has implemented the widely deployed Service Router Operating System (SROS) from Alcatel-Lucent to facilitate open communication and protocol interworking between the datacenter and the wide area network. As compute resources are consumed or moved within the datacenter or even across datacenters the Nuage VSP will instantaneously update the BGP routing information to the tenants VPN so that paths between the cloud hosted resources and the clients within the VPN are maintained and correct at all times; without requiring any network configuration or staffing overheads.