Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Containers for the Enterprise: Delivering OpenShift on OpenStack for Performance and Scale

682 views

Published on

Imagine being able to stand up thousands of tenants with thousands of apps, running thousands of Docker-formatted container images and routes, all on a self-healing cluster. Now, take that one step further with all of those images being updatable through a single upload to the registry, and with zero downtime. In this session, Steve Gordon of the Red Hat OpenStack Platform team will show you just that. Steve will walk through a recent benchmarking deployment using the Cloud Native Computing Foundation’s (CNCF) new 1,000 node cluster with OpenStack and Red Hat’s OpenShift Container Platform, the enterprise-ready Kubernetes for developers.

Published in: Software
  • Be the first to comment

Containers for the Enterprise: Delivering OpenShift on OpenStack for Performance and Scale

  1. 1. DELIVERING OPENSHIFT ON OPENSTACK FOR PERFORMANCE AND SCALE STEVE GORDON (@xsgordon)
  2. 2. GOAL ● 1000 OpenShift Container Platform nodes; hosted on ● 300 physical Red Hat OpenStack Platform nodes 2
  3. 3. 3 APPLICATION CONTAINERS 1. Code: mysqld 2. Configuration: /etc/my.cnf 3. Data: /var/lib/mysql 4. Other stuff :-) CODE, CONFIGURATION, DATA
  4. 4. 4 EXPOSITION OF RESOURCES
  5. 5. 5 CONSUMPTION OF RESOURCES
  6. 6. OPENSHIFT ON OPENSTACK 6 Architectural tenets: ● Technical independence ● Contextual awareness ● Avoiding redundancy ● Simplified management Reference architecture: ● red.ht/2ibNmvX
  7. 7. PERFORMANCE AND SCALE TESTING 7
  8. 8. WHERE TO TEST? 8 The CNCF cluster is made up of 1000 nodes deployed at Switch, Las Vegas by Intel for the use of the CNCF community: Compute Node Spec ● 2x Intel E5-2680v3 12-core ● 256GB RAM ● 2x Intel S3610 400GB SSD ● 1x Intel P3700 800GB NVMe PCIe SSD ● 1x QP Intel X710 Storage Node Spec ● 2x Intel E5-2680v3 12-core ● 128GB RAM ● 2x Intel S3610 400GB SSD ● 10x Intel 2TB NLSAS HDD ● 1x QP Intel X710" Got a nefarious plan for taking over the world using CNCF related open source projects? Head to https://github.com/cncf/cluster.
  9. 9. WHAT TO TEST? Goal: 1000 OpenShift Container Platform nodes, on 300 Red Hat OpenStack Platform nodes ● Push deployment to its limit, identify: ○ Bottlenecks, ○ Config changes, and ○ Best practices ● Document, file, and fix issues as appropriate. 9
  10. 10. ARCHITECTURE 10 ● OpenStack UnderCloud
  11. 11. ARCHITECTURE 11 ● OpenStack UnderCloud ● OpenStack Overcloud control plane
  12. 12. ARCHITECTURE 12 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute
  13. 13. ARCHITECTURE 13 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions ● Load Generators
  14. 14. ARCHITECTURE 14 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions
  15. 15. ARCHITECTURE 15 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions ● Load generators
  16. 16. SYSTEM VERIFICATION TEST SUITE ● Red Hat OpenShift Performance and Scalability team’s upstream test suites ● Main tests are ○ cluster-loader ○ Networking/synthetic ○ Workload Generator ○ Reliability/Longevity ● https://github.com/openshift/svt 16
  17. 17. 17 CLUSTER-LOADER ARCHITECTURE Start Parse args & config Config Obj End False Create Namespace True X Exists ? Items < N False Create X Iterate Item Count True True False X can be: ● Quota ● Template ● Service ● User ● Pod ● RC
  18. 18. CLUSTER LOADER TEST 18 OpenShift (Kubernetes) Object Quantity Nodes 1,000 Namespaces (projects) 13,000 Pods 52,000 Build Configs 39,000 Templates 78,000 Image Streams 13,000 Deployment Configs and Services 39,000 (incl. 13,000 Replication Controllers) Secrets 260,000 Routes 39,000 Component # of bugs filed/encountered Kubernetes 8 Installer 8 Images 5 Docker 5 Containers 3 Storage 2 Ansible 2 Management 1 iptables 1 Deployments 1 Command 1 Builds 1 Auth 1
  19. 19. NOTABLE SCALE ISSUES ENCOUNTERED 19 etcd Disk Utilization API Server CPU
  20. 20. NOTABLE SCALE ISSUES ENCOUNTERED 20 Ansible Installer https://github.com/ansible/ansible/issues/16749
  21. 21. FUTURE OPPORTUNITIES ● Reference architecture available at red.ht/2ibNmvX ● Containers and OpenStack: A platform for distributed applications, paper available at http://red.ht/2hSfIPs ● Learn more about Red Hat Summit at redhat.com/summit 21
  22. 22. THANK YOU 22

×