• Save
Developing production OpenFlow controller with Trema
Upcoming SlideShare
Loading in...5
×
 

Developing production OpenFlow controller with Trema

on

  • 2,579 views

 

Statistics

Views

Total Views
2,579
Views on SlideShare
2,513
Embed Views
66

Actions

Likes
4
Downloads
0
Comments
0

3 Embeds 66

https://twitter.com 61
http://jam.cpi.ad.jp 4
http://gvtkne.blogspot.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Developing production OpenFlow controller with Trema Developing production OpenFlow controller with Trema Presentation Transcript

  • Developing productionOpenFlow controller with TremaApr. 20, 2013Yasunobu ChibaPart-time developer, Trema ProjectCopyright (C) 2013 NEC Corporation
  • Agenda• Provide tips and typical mistakes whendeveloping OpenFlow controller with Trema Clibrary• Explain a production OpenFlow controllerdevelopment use caseCopyright (C) 2013 NEC Corporation
  • THINGS TO KNOW ABOUT OPENFLOWAND TREMA BEFORE DEVELOPINGOPENFLOW CONTROLLERCopyright (C) 2013 NEC Corporation
  • Trema is not an OpenFlow controller• Trema is a programming framework and just abuilding block for implementing OpenFlowcontrollers• You may use any other building blocks as wellas Trema– For high availability, scalability, network protocolsupport, and etc.Copyright (C) 2013 NEC Corporation
  • OpenFlow switch is not a database• OpenFlow switch is not a database and does nothave ACID properties• Error messages are returned from switchasynchronously• Send a Barrier Request to complete a previouslysent Flow Mod (or any other) operation• See and reuse Transaction Manager:– https://github.com/trema/apps/tree/master/transaction_managerCopyright (C) 2013 NEC Corporation
  • Asynchronous events happen beforefeatures request/reply exchange• You may receive asynchronous OpenFlowevents such as Packet-In from unknownswitches that features reply has not beenreceived• In Trema, Switch Ready event is triggeredwhen a features reply is receivedCopyright (C) 2013 NEC Corporation
  • send_openflow_message() does notsend message actually• send_openflow_message() just pushes a givenmessage to an appropriate send queue• The send queue is automatically flushed insidethe main loop• Message ordering is NOT guaranteed amongswitchesCopyright (C) 2013 NEC Corporation
  • libtrema is not fully thread-safe• The followings may be thread-safe but notguaranteed for all functions– buffer, doubly linked list, hash, linked list, log,match table, OpenFlow message, packet info,packet parser, stat, utility, wrapper• OpenFlow application interface is NOT thread-safe• Call send_openflow_message() from the mainthread that trema_run() is runningCopyright (C) 2013 NEC Corporation
  • Length of send queue is limited• The maximum length of send queue (forsending OpenFlow messages) is limited andfixed• send_openflow_message() may return false ifthe send queue is fullCopyright (C) 2013 NEC Corporation
  • Do not call flush_messenger()• flush_messenger() flushes all send queues andreceive queues immediately andsynchronously• Calling it in an OpenFlow event handler mayrecursively call the event handler!• Call only if you known how it works very wellCopyright (C) 2013 NEC Corporation
  • start_trema() may call fork(2)• start_trema() calls fork(2) if your controller isto be daemonized• Use caution if you call functions that may beaffected by fork(2)Copyright (C) 2013 NEC Corporation
  • CONTROLLER DEVELOPMENT USECASECopyright (C) 2013 NEC Corporation
  • Requirements• Functional Requirements– Manage association among virtual networks (basedon non-OpenFlow technology) and switch ports– Associate a switch port with MAC addresses locatedon the switch port– All operations above can be done via REST interface• Non-functional Requirements– 1K+ switches must be handled– 10K+ active virtual networks must be managed– 10K+ end hosts must be connected to virtual networksCopyright (C) 2013 NEC Corporation
  • Requirements – cont’dCopyright (C) 2013 NEC CorporationSwitch #1 Switch #2 Switch #3 Switch #XVirtual Network #NVirtual Network #2Virtual Network #110,000+ ActiveVirtual Networks1,000+ Switches10,000+ End HostsPhysical NetworkVirtual Networks
  • Design• Load balancer + three-tiered architecture + αCopyright (C) 2013 NEC CorporationDispatcher(Load Balancer)WorkerWorkerWorkerBackendDatabaseConfigurationInterfaceExternal ApplicationOpenFlow SwitchRESTOpenFlow+ RESTOpenFlow+ REST
  • ImplementationCopyright (C) 2013 NEC CorporationREST I/FController ClusterOpenFlow Load Balancer (LVS)Virtual Network Manager Virtual Network Manager Virtual Network ManagerConfiguration Frontend Configuration Frontend Configuration FrontendREST Interface Load Balancer (LVS)Backend DB (MySQL)WatchdogTrema Trema TremaOpenFlowVirtual NetworkAgentRESTVMLegacyNetworkOpenFlow SwitchREST OpenFlowVXLAN Tunnel End-PointVirtual NetworksLegacy L2/L3 NetworkVMLegacyNetworkVirtual NetworkAgentOpenFlow SwitchVXLAN Tunnel End-Point
  • Evaluation setupCopyright (C) 2013 NEC CorporationVnet Manager+ Config.Frontend#1Vnet Manager+ Config.Frontend#2Vnet Manager+ Config.Frontend#NBackend DBLoadBalancereth0 eth0 eth0 eth0eth1 eth2 eth2 eth2 eth2Database Network192.168.17.0/24192.168.16.1 192.168.16.2 192.168.16.N192.168.48.16192.168.17.64 192.168.17.1 192.168.17.2 192.168.17.N 192.168.16.254eth180/tcp6633/tcpREST Clienteth1 eth1 eth1InternalControlNetwork192.168.16.0/24192.168.64.17 192.168.64.18 192.168.64.19 192.168.64.16eth0ManagementNetwork192.168.64.0/20192.168.64.(17+N)ControlandData Network192.168.48.0/20eth0 - 192.168.65.1VXLAN TEPVnetAgentVM Host #1Open vSwitchhp001sp001eth1 - 192.168.49.1hp128sp128hp002sp002eth0 - 192.168.65.NVXLAN TEPVnetAgentVM Host #NOpen vSwitchhp001sp001eth1 - 192.168.49.Nhp128sp128hp002sp002192.168.63.253
  • Evaluation items and results• # of switches that can be managed– 410 - 412 switches per a single Virtual NetworkManager were connected and initialized properly• Switch daemons were not able to run due toinsufficient memory (system memory was 2 GB)– 1024 switches were connected and initialized withthree Virtual Network ManagersCopyright (C) 2013 NEC Corporation
  • Evaluation items and results• # of virtual networks that can be managed– 16384 virtual networks that have 8 ports (hosts)each were successfully created with 1024 switchesand three Virtual Network Managers• Virtual network setup time– Setup time did not increase even if we have anumber of virtual networks– Database access time was constant and a minorfactorCopyright (C) 2013 NEC Corporation
  • To be continued…Copyright (C) 2013 NEC Corporation