This document summarizes a proof of concept for a software defined data center using OpenStack and Midokura MidoNet software defined networking. The POC used 4 controllers, 8 Ceph storage nodes, and 16 compute nodes with Midokura providing logical layer 2-4 networking services. Key lessons learned included planning the underlay network configuration, optimizing Zookeeper connections, and improving OpenStack deployment processes which can be complex. Performance testing showed Ceph throughput was higher for reads than writes and SSD journaling improved IOPS. The streamlined workflow provided by the software defined infrastructure could help reduce costs and management complexity for organizations.

1. Walk Through a Software
Defined Everything PoC
Cynthia Thomas
Systems Engineer - Midokura
Chris Janiszewski
OpenStack Architect - Red Hat

2.  To enable data center services through an abstraction of compute,
network, and storage functionality into a pool of resources simply
consumed as a service.
 Enable users to simply and expeditiously provision or decommission
an application
 Automatically balance applications across data center resources to
optimize efficiency
 Elastically scale out data center resources to meet application demands
 Provide complete isolation to prevent unintended resource
sharing/intrusion
 Be available all the time, tolerant to failure, and maintain data consistent
 Elegantly metered, managed, monitored, and configured
 Leverage universal on and off premises computing standards
Objectives

3. Use Case
 OpenStack Neutron Production-Grade Plugin
 Independent Control and Data Plane
 Database/Topology Management
 Dynamic Routing Protocol on GWs
 Advanced VRF Features
 Flow Tracing Troubleshooting
3

4.  SDN distributed architecture that is built to
scale with enhanced security.
 Leading SDS OpenStack solutions
 Integration of Software Defined
Technologies with Enterprise OpenStack
 Hardware Management and Monitoring
Proof of concept

5. Midonet Platform
v
Any Application
Midokura Enterprise MidoNet
Logical L2
Any Network Hardware
OpenStack, vSphere, Custom Platforms
Logical
Firewall
Logical Layer 4
Load Balancer
KVM, ESXi, LXC, Docker
Logical L3
Logical Switching – Layer 2 over Layer 3, decoupled from
the physical network
Logical Routing – Routing between virtual networks
without exiting the software container
Logical Firewall – Distributed Firewall, Kernel Integrated,
High Performance
Logical Layer 4 Load Balancer – Application Load
Balancing in software
MidoNet API – RESTful API for integration into any Cloud
Management Platform
Distributed Networking Services

6. Architecture Overview

7. Midonet Topology

8. Ceph Architecture

9. Extreme Cloud Administration Toolkit
 Open Source
 Lights-out management
 Automation and Deploy – bare metal and service VMs
 Hardware monitoring and management
 Firmware update
 Parallel shell commands
 RPM Repo
 DNS, HTTP, DHCP, TFTP, NFS
 SOL Console

10. Design

11. Infrastructure

12. 4x controller nodes:
– CPU: 2 socket Intel Haswell (12 core)
– Memory: 64 GB
– Storage: 4x 4 TB HDD (RAID-10 Virtualization)
8x Ceph OSD nodes:
– CPU: 1 socket Intel Haswell (6 core)
– Memory: 64 GB
– Storage: 8x 6 TB HDD (OSD) ; 2x 200 GB SSD (Journal)
16x compute nodes:
– CPU: 2 socket Intel Haswell (20 core)
– Memory: 384 GB
All nodes:
2x 32 GB Class 10 SD Cards (Operating System)
2x 10 Gb and 1Gb NIC
Hardware

13. OpenStack Services

14. MidoNet Configuration
 Most default configurations used
 Dedicated links for BGP Gateways
 BGP timers made more aggressive
 LRO off on Gateways
 Port-groups for Gateways
 Increased Client connections to Zookeeper
14

15.  Replication 3x
 64 OSDs spread equally between 8 nodes
 SSD to HDD ratio 1:4
 4096 Placement Groups based on formula:
Total PGs = (OSDs * 100) / # of replicas (increase to
next power of 2)
 Aggregated 10Gb NICs with VLAN isolation for Public
and Cluster network
Ceph Configuration

16. MidoNet Lessons Learned
 Plan out the Underlay (MTU, VLANs, IP) as later changes
affect OpenStack services like RabbitMQ
 Connection count for Zookeeper matters else all midolmen
cannot join the party
 Gateway failover was impressive!
 Horizon doesn’t link enough Network objects
 Flow Tracing makes debugging an SDN so much easier!
16

17.  “Split Brain” for odd number of database nodes
 Restarting RabbitMQ is not a simple task
 Power Outages happens
 VXLAN offload only works on single UDP port
 OpenStack manual deployment is still a
complex process
OpenStack - Lessons Learned

18.  Throughput - 5x Read vs Write
 IOPS read (seq) BS 4K - more then 80K (no SSD)
 SSD for journaling – double performance
 No SSD for journaling – partition outer edge of the disk
 20Gbps network bottleneck
 OVS balance-tcp > Linux bond
 Isolate Cluster and Public networks
 SSD to HDD ratio and SSD size
Ceph – Lessons Learned

19. Streamlined Workflow

20.  Rapid deployments and scale up of new applications
 Reduced management cost and reduced complexity
 Management tools that can support the management of thousands of
physical servers
 Ability to scale to thousands of VMs per cloud administrator
 Reduced cost per VM
 Advanced and agile networking that uses Network Virtualization
Overlays
 Tenant isolation over shared infrastructure
 Simplified underlying Network infrastructure that uses open standards
L3 routing protocols
 Improved IT productivity with reduced time to deploy resources
Business Benefits

21. Whitepaper - https://lenovopress.com/redp5233
Learn More
THANK YOU