Rob Sherwood CTO Big Switch Networks Plenaries Session ONS2015: http://bit.ly/ons2015sd ONS Inspire! Webinars: http://bit.ly/oiw-sd Watch the talk (video) on ONS Content Archives: http://bit.ly/ons-archives-sd

3. Built with SDN, Bare Metal Switches, and
Merchant Silicon
DESIGN OF BARE METAL FABRICS
Rob Sherwood
Open Networking Summit 2014

4. Application Plane
Control Plane
Data Plane
Switch Light for LinuxSwitch Light for Broadcom
Big Virtual Switch
Big Network Controller

5. THIS YEAR: SDN FABRICS ON BARE METAL
• Big Switch’s Core Technologies
• SDN: control separated from data plane: centralized, automated
• Switch Light: own both the Controller and Switch software stacks
• Commodity Hardware: low-cost hardware and merchant silicon
• Open: APIs, source, protocols
• Fabric data plane: unified L2/L3 multipath, active/active design
• Live Demonstration:
• Resilient under systemic failure
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 3
New

6. WHAT’S INSIDE A SWITCH?
©2013 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 4
Application
Network OS
Hardware Driver
Box
Silicon

7. COMPONENT ECOSYSTEM AND BARE METAL
©2013 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 5
Application
Network OS
Driver
Box
Silicon
Single
Vendor
Closed
Product
Traditional
Networking
(past)
Single
Vendor
Closed
Product
Traditional
Networking
(today)
ODM Chip
ODM Box
SDN
Controller
Vendor
OpenFlow
Model
SDN
Hardware
Vendor
OpenFlow
Traditional
Network
Stack/OS
Vendor
ODM Chip
Bare Metal
Vision
ODM Box

8. SWITCH LIGHT IS SDN ON BARE METAL
• The OpenFlow Ecosystem Model didn’t work for us
• Hard: one OF agent that is all things to all people
• OF Wire protocol is (mostly) fine, but needs work for existing hardware
• Economic incentives are backwards for SDN hardware vendors
• Out-of-phase release cycles caused low feature velocity
• Switch Light Model: download OS image and OF agent from Controller
• Easy : Customize switch software to application and controller version
• Add extensions to wire protocol to validate before standardization
• Perfect economic incentives
• One engineering team  high feature velocity
©2013 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 6

9. BARE METAL + MERCHANT SI: LOWERS FABRIC COSTS
• Brand name network vendors are using the same components
• “Bare metal” is more than “white box”
• Name brand vs. no-name vendors
• Buy your network hardware just like servers
• Vendors already successful in the server model have advantage
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 7
Business
Agility is
Increased
Revenue
Financial
Benefit of
SDN
Reduced
OpEx Reduced
CapEx
CapEx
Savings is
Not Trivial

10. ARCHITECTURE IS OPEN FROM THE DATA PLANE UP
• Open data plane
• Re-use existing virtualization formats: no proprietary protocols/new hardware
• Compatible: deployed firewalls, IDS, WAN accelerators, Server offload NICs
• Open control plane
• OF1.3: All extensions are open source and will be taken to ONF
• Open REST APIs is primary interface: CLI is just a REST client
• Integration with OpenStack/Quantum and custom BSS/OSS systems
• Open source components: software engineering best practice
• Accelerates adoption, eases porting
• Open Network Linux: bare metal Linux distribution in OCP
• Indigo Open Flow agent: many 3rd parties have ported to their hardware
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 8

11. THIS YEAR: SDN FABRICS ON BARE METAL
• Big Switch’s Core Technologies
• SDN: control separated from data plane: centralized, automated
• Switch Light: own both the Controller and Switch software stacks
• Commodity Hardware: low-cost hardware and merchant silicon
• Open: APIs, source, protocols
• Fabric data plane: unified L2/L3 multipath, active/active design
• Live Demonstration:
• Resilient under systemic failure
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 9
New

12. 1
2
3
4
5
6
1 2 3 4 5 6
WHAT’S A FABRIC?
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 10
Chassis-based
Fabric
FabricBackplane
Supervisor 2
Line Card
Line Card
Line Card
Line Card
Line Card
Supervisor 1
Spine
Leaf
SpineSpine
LeafLeafLeaf
Leaf-Spine
Fabric
Controller 2
Controller 1

13. FABRIC USE-CASE (1/3): CONNECTIVITY, POLICY
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 11
Spine
Leaf
SpineSpine
LeafLeafLeaf
Server
Server
Server
Server
Server
Server
Router
Router
• Most common fabric use-case
• Policy: connectivity, ACLs, QoS
• Optional VM orchestration
• e.g., OpenStack, HyperV

14. Existing
Production
Network
FABRIC USE-CASE (2/3): MONITORING
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 12
Spine
Leaf
SpineSpine
LeafLeafLeaf
IDS
VoIP QoS Tool
• BigTap: Scale-out monitoring
• Monitoring “as a Service”
• Programmable via REST
• RBAC
• Overlapping policy support
• Physically isolated from prod
• In/out heavily over-subscribed
• Load balance across tools
• “My first SDN App”
Tap Traffic

15. FABRIC USE-CASE (3/3): PATCH PANEL
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 13
Spine
Leaf
SpineSpine
LeafLeafLeaf
Packet Generator
Packet Generator
• Programmatically create large
virtual L1 topologies
• Automate complex tests
without rewiring
• Blindly forward packets by port
• Similar to Pseudo-wire
• Even CRC failures, runts, etc.
• Non-product: in our Q/A labs
DUT
DUT
DUT
DUT
DUT
DUT
*DUT: Device Under Test

16. THIS YEAR: SDN FABRICS ON BARE METAL
• Big Switch’s Core Technologies
• SDN: control separated from data plane: centralized, automated
• Switch Light: own both the Controller and Switch software stacks
• Commodity Hardware: low-cost hardware and merchant silicon
• Open: APIs, source, protocols
• Fabric data plane: unified L2/L3 multipath, active/active design
• Live Demonstration:
• Resilient under systemic failure
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 14
New

17. DEMO: RESILIENCY UNDER SYSTEMIC FAILURES
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 15
Leaf 1 Leaf 2
Spine 1 Spine 2
Traffic
Generator
Primary
Controller
Backup
Controller
1
2
3 4
5

18. CONCLUSION
• Big Switch has a common architecture for its products
• Based on SDN, Switch Light, Open, and Merchant Silicon
• Fabric technology common across applications
• Stepping back a bit: Industry is maturing
• Storage+Network+Compute convergence means new players
• Bare metal means less vertical integration, more competition
• Reality: hardware has already commoditized
• Every major vendor is shipping products on Broadcom Trident+/Trident2
• Price war has already started: seeing less than $100/10G port
• Claim: Value will move up the stack in to software
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 16

19. THANKS!
©2014 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 17

20. CLAIM: SDN SCALES BETTER THAN DISTRIBUTED NETWORKS
History teaches us that hierarchies solve scaling problems
• LAN: OSPF  OSPF Areas
• WAN: BGP  BGP Route Reflectors
• Edge: Enterprise Wireless  Enterprise Wireless controllers
So of course adding SDN controllers makes data center fabrics scale better
• Faster bring up time, no route convergence
• Global knowledge  global optimization
Almost all commercial fabrics have “controllers”
• Different choices for who is “master of state”
©2013 BIG SWITCH NETWORKS, INC. WWW.BIGSWITCH.COM 18