Open Network Install Environment
NETWAYS OSDC 2014, Berlin
Nat Morris
9th April 2014
cumulusnetworks.com 2
• Overview
• What Is ONIE?
• Lessons Learned
• ONIE Development
• Demo
Agenda
IP Fabric Networking Landscape
cumulusnetworks.com 3
Network
Hardware
NetworkOS
Open Closed
The Expanding Landscape
hardware
operating system
appapp
hardware
operating system
app app
Single Vendor Blob Multi-Vendor...
Understanding Characteristics of a Leaf Switch
5cumulusnetworks.com
10/40 Gigabit
spine uplink ports
Serial
console port
E...
Understanding Characteristics of a Spine Switch
6cumulusnetworks.com
Serial
console port
Ethernet Out-of-Band
Management P...
Bare Metal Switch Provisioning
§ Similar approach to installing OS on server
§  BIOS + PXE = U-Boot + ONIE (Open Network...
Choice
cumulusnetworks.com 8
Choice
cumulusnetworks.com 9
What Is ONIE?
§ Network OS Install Environment
§  Provides an environment for network OS installer discovery
and executi...
Hardware Vendors
cumulusnetworks.com 11
Operating System Vendors
cumulusnetworks.com 12
ONIE: Anatomy of a Network Switch
cumulusnetworks.com 13
(	
  Management	
  Interfaces	
  )	
   (	
  Data	
  Plane	
  )	
 ...
ONIE: Uses Management Interfaces
cumulusnetworks.com 14
(	
  Management	
  Interfaces	
  )	
   (	
  Data	
  Plane	
  )	
  ...
ONIE: Bare Metal Install – First Time Boot Up
cumulusnetworks.com 15
Boot Loader
(HW Vendor Supplied)
ONIE
(HW Vendor Supp...
ONIE: Subsequent Reboots – Vendor’s OS is Already Installed
cumulusnetworks.com 16
Boot Loader
(HW Vendor Supplied)
ONIE
(...
ONIE: Network OS Installer Discovery and Install Behavior
cumulusnetworks.com 17
Configure Network
Interface
Locate Instal...
Why	
  Not	
  Use	
  PXE?	
  
cumulusnetworks.com 18
•  IPv6	
  and	
  HTTP	
  out	
  of	
  the	
  box	
  
•  Use	
  exisS...
ONIE: Other Behaviors
§ Provides a mechanism for a network OS to invoke
the above behaviors.
cumulusnetworks.com 19
Reins...
Lessons Learned
§ Hardware Platforms
§  Diverse mix of CPU complex designs, even within a
single CPU family.
§  HW Vend...
ONIE Ongoing Development
§ Support Multiple CPU Architectures
§  PowerPC – Today
§  x86 – VM Prototype available today
...
ONIE Ongoing Development
§ New Features
§  DHCPv6
§  DNS Service Discovery / Multicast DNS discovery
§  HW Vendor diag...
ONIE x86 Strategy
§  Use existing BIOS from hardware vendors
§  During manufacturing install GRUB2 and ONIE-x86 on
the m...
Block Device Partitioning
§  Partition using GUID Partition Table (GPT) format
§  GPT supported by Linux, GRUB, UEFI
§ ...
GRUB Menu
cumulusnetworks.com 25
NOS Installer Duties
§  Create new GPT partition(s)
§  Create file systems on partition(s)
§  Install NOS files into pa...
Resources
§ ONIE Websites
§  Main Page: http://www.onie.org/
§  Source Code: https://github.com/onie/onie/
§  Document...
© 2013 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trade...
Back Up Slides – Cumulus Background
cumulusnetworks.com 29
Pushing Changes Down
cumulusnetworks.com 30
CPU, RAM, Flash, etc. Switch Silicon
Front Panel Ports
lldpd
Routing Tables AR...
Cumulus Linux Hardware Compatibility List
Cumulus Networks and Partner Confidential
40G Portfolio Part number Description ...
Upcoming SlideShare
Loading in...5
×

OSDC 2014: Nat Morris - Open Network Install Environment

213

Published on

ONIE defines an open source “install environment” that runs on this management subsystem utilizing facilities in a Linux/BusyBox environment. This environment allows end-users and channel partners to install the target network OS as part of data center provisioning, in the fashion that servers are provisioned.
ONIE enables switch hardware suppliers, distributors and resellers to manage their operations based on a small number of hardware SKUs. This in turn creates economies of scale in manufacturing, distribution, stocking, and RMA enabling a thriving ecosystem of both network hardware and operating system alternatives.

Published in: Software, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
213
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

OSDC 2014: Nat Morris - Open Network Install Environment

  1. 1. Open Network Install Environment NETWAYS OSDC 2014, Berlin Nat Morris 9th April 2014
  2. 2. cumulusnetworks.com 2 • Overview • What Is ONIE? • Lessons Learned • ONIE Development • Demo Agenda
  3. 3. IP Fabric Networking Landscape cumulusnetworks.com 3 Network Hardware NetworkOS Open Closed
  4. 4. The Expanding Landscape hardware operating system appapp hardware operating system app app Single Vendor Blob Multi-Vendor Ecosystem app app cumulusnetworks.com 4
  5. 5. Understanding Characteristics of a Leaf Switch 5cumulusnetworks.com 10/40 Gigabit spine uplink ports Serial console port Ethernet Out-of-Band Management Port 1/10 Gigabit Ethernetnetwork access ports * SFP+ ports can be grouped together into a single QSFP 40G port via reverse connecting breakout cable options * QSFP ports can be broken out into four SFP+ ports via copper or optical transceiver options
  6. 6. Understanding Characteristics of a Spine Switch 6cumulusnetworks.com Serial console port Ethernet Out-of-Band Management Port * QSFP ports can be broken out into four SFP+ ports via copper or optical breakout cable options 40G / 10G Fabric ports
  7. 7. Bare Metal Switch Provisioning § Similar approach to installing OS on server §  BIOS + PXE = U-Boot + ONIE (Open Network Install Environment) §  Supported hardware (HCL) preloaded with ONIE §  ONIE available on GitHub •  http://onie.github.io/onie/ cumulusnetworks.com 7 bare metal server operating system app app app BIOS and PXE bare metal switch operating system app app app U-Boot and ONIE
  8. 8. Choice cumulusnetworks.com 8
  9. 9. Choice cumulusnetworks.com 9
  10. 10. What Is ONIE? § Network OS Install Environment §  Provides an environment for network OS installer discovery and execution §  Like a pre-installed kickstarter §  Defined by its behaviors §  Implemented using a modern Linux kernel and BusyBox § An Open Source Project within OCP §  http://www.onie.org/ § Evolving … cumulusnetworks.com 10
  11. 11. Hardware Vendors cumulusnetworks.com 11
  12. 12. Operating System Vendors cumulusnetworks.com 12
  13. 13. ONIE: Anatomy of a Network Switch cumulusnetworks.com 13 (  Management  Interfaces  )   (  Data  Plane  )   CPU SoC DRAM Boot Flash Mass Storage Switching ASIC Serial   Console   Ethernet   Mgmt  Port   10Gb   Port   40Gb   Port  …   10Gb   Port   40Gb   Port   …   PCIe  
  14. 14. ONIE: Uses Management Interfaces cumulusnetworks.com 14 (  Management  Interfaces  )   (  Data  Plane  )   CPU SoC DRAM Boot Flash Mass Storage Switching ASIC Serial   Console   Ethernet   Mgmt  Port   10Gb   Port   40Gb   Port  …   10Gb   Port   40Gb   Port   …   PCIe  
  15. 15. ONIE: Bare Metal Install – First Time Boot Up cumulusnetworks.com 15 Boot Loader (HW Vendor Supplied) ONIE (HW Vendor Supplied) Installer (OS Vendor) Boot  Loader   •  Low  Level  boot  loader,  configures  CPU  complex   •  Loads  and  boots  ONIE   ONIE   •  Linux  Kernel  with  Busybox   •  Configures  management  Ethernet  interface   •  Locates  and  executes  an  OS  installer   •  Provides  tools  and  environment  for  installer   OS  Installer   •  Available  from  network  or  USB   •  Linux  executable   •  Installs  vendor  OS  into  mass  storage   Network OS (OS Vendor Supplied) Fetches   Installs  
  16. 16. ONIE: Subsequent Reboots – Vendor’s OS is Already Installed cumulusnetworks.com 16 Boot Loader (HW Vendor Supplied) ONIE (HW Vendor Supplied) Boot  Loader   •  Low  Level  boot  loader,  configures  CPU  complex   •  Loads  and  boots  OS  vendor’s  installed  OS   Network  OS   •  Configures  Switching  ASIC   •  Runs  Network  Protocols   •  Provides  CLI   Network OS (OS Vendor Supplied) ONIE   •  SSll  exists,  but  is  not  used   •  Available  for  uninstall  /  re-­‐install  operaSons  
  17. 17. ONIE: Network OS Installer Discovery and Install Behavior cumulusnetworks.com 17 Configure Network Interface Locate Installer Run Installer •  Uses  DHCPv4,  DHCPv6   •  Configures  Ethernet  interface  for  IPv4  /  IPv6   •  Configures  DNS  and  hostname   •  Determines  the  locaSon  of  an  installer  executable   •  Examines  local  file  systems,  e.g.  USB  flash  drives   •  Uses  DHCP  opSons,  DNS  Service  Discovery,  MulScast   DNS  and  IPv6  Neighbors   •  Downloads  installer  via  URL   •  Passes  various  environment  variables  to  installer   •  Launches  installer  
  18. 18. Why  Not  Use  PXE?   cumulusnetworks.com 18 •  IPv6  and  HTTP  out  of  the  box   •  Use  exisSng  Linux  device  drivers.    No  need  to  write  new  ones   •  Integrated  automaSon  
  19. 19. ONIE: Other Behaviors § Provides a mechanism for a network OS to invoke the above behaviors. cumulusnetworks.com 19 Reinstall Remove currently installed OS and return to the “out of box” provisioning state Uninstall Completely wipe out everything, except ONIE Rescue Reboot box into ONIE for repair, debug and forensics Update Install a new ONIE version Diag Run HW Vendor’s diag (optional)
  20. 20. Lessons Learned § Hardware Platforms §  Diverse mix of CPU complex designs, even within a single CPU family. §  HW Vendors need freedom to customize ONIE §  Need mechanism to run HW vendor diag § Recurring Themes §  TLV based EEPROM format widely adopted §  Common HW designs reduce development time cumulusnetworks.com 20
  21. 21. ONIE Ongoing Development § Support Multiple CPU Architectures §  PowerPC – Today §  x86 – VM Prototype available today §  Thinking about ARM, MIPS §  Maintain ONIE behaviors across architectures § Testing and Compliance §  Enhance and extend regression test suites §  Develop compliance test suites cumulusnetworks.com 21
  22. 22. ONIE Ongoing Development § New Features §  DHCPv6 §  DNS Service Discovery / Multicast DNS discovery §  HW Vendor diagnostic § Releases §  Quarterly release cadence §  Ongoing maintenance §  Enhancements cumulusnetworks.com 22
  23. 23. ONIE x86 Strategy §  Use existing BIOS from hardware vendors §  During manufacturing install GRUB2 and ONIE-x86 on the mass storage block device §  NOS Installer adds partitions, installs software and updates GRUB2 configuration. cumulusnetworks.com 23
  24. 24. Block Device Partitioning §  Partition using GUID Partition Table (GPT) format §  GPT supported by Linux, GRUB, UEFI §  Supports dual-booting multiple operating systems cumulusnetworks.com 24 Partition # Name R/W Notes 1 GRUB Boot r/o Used by GRUB and GPT 2 ONIE BOOT r/o ONIE Kernel 3 ONIE CONFIG r/w Configuration 4+ For NOS use
  25. 25. GRUB Menu cumulusnetworks.com 25
  26. 26. NOS Installer Duties §  Create new GPT partition(s) §  Create file systems on partition(s) §  Install NOS files into partition(s) §  Update ONIE-CONFIG using ONIE provided CLIs §  onie-boot-entry-add! §  onie-boot-entry-remove! §  onie-boot-entry-show! §  onie-boot-default! §  onie-boot-update! cumulusnetworks.com 26
  27. 27. Resources § ONIE Websites §  Main Page: http://www.onie.org/ §  Source Code: https://github.com/onie/onie/ §  Documentation: http://onie.github.io/onie/ cumulusnetworks.com 27
  28. 28. © 2013 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis. § Thank You! cumulusnetworks.com 28
  29. 29. Back Up Slides – Cumulus Background cumulusnetworks.com 29
  30. 30. Pushing Changes Down cumulusnetworks.com 30 CPU, RAM, Flash, etc. Switch Silicon Front Panel Ports lldpd Routing Tables ARP Table Devices Bridge FDB Filter Tables Bonds VLANs LinuxKernel Virtual Kernel Ports Bridging mstpd ACLRouting Suite Quagga snmpd vconfig iptable ebtable ip6tableiproute2 VXLAN Bridges Switch HAL brctl Switch Driver UserSpace Quagga daemon, Quagga.conf, and vtysh CLI and /etc/network/ interfaces switchd
  31. 31. Cumulus Linux Hardware Compatibility List Cumulus Networks and Partner Confidential 40G Portfolio Part number Description Merchant Silicon Cumulus Linux Release Dell S6000 32 x 40G-QSFP+ Trident II 2.1 or later Edge-Core AS6700-32X 32 x 40G-QSFP+ Trident II 2.0 or later Penguin Computing Arctica 3200XL 32 x 40G-QSFP+ Trident II 2.0 or later Quanta QCT T5032-LY6* 32 x 40G-QSFP+ Trident II 2.0 or later 10G Portfolio Part number Description Merchant Silicon Cumulus Linux Release Agema AG-7448CU 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident 1.5.0 or later Dell S4810 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident 2.0.x or later** Edge-Core AS5600-52X 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident+ 1.5.0 or later Edge-Core AS5710-54X 48 x 10G-SFP+ and 6 x 40G-QSFP+ Trident II 2.1 or later** Edge-Core AS5710-96X 96 x 10G-SFP+ and 8 x 40G-QSFP+ Trident II 2.1 or later** Penguin Computing Arctica 4804X 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident+ 1.5.1 or later Quanta QCT T-3048-LY2* 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident+ 1.5.0 or later Quanta QCT T5048-LY8* 48 x 10G-SFP+ and 6 x 40G-QSFP+ Trident II 2.1 or later** 1G Portfolio Part number Description Merchant Silicon Cumulus Linux Release Edge-Core AS4600-54T 48 x 1G-T and 4 x 10G-SFP+ Apollo2 2.0 or later Penguin Computing Artica 4804i 48 x 1G-T and 4 x 10G-SFP+ Triumph2  1.5.1 or later Quanta QCT T1048-LB9* 48 x 1G-T and 4 x 10G-SFP+ FireBolt3 1.5.0 or later 31
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×