Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
New features of
Azure Cloud Provider
in OpenShift 3.10
Takayoshi Tanaka, Software Maintenance Engineer
tatanaka@redhat.com
Takeaways
Let you know Azure Cloud Provider is getting more and
more useful.
Let you know some upstream project/feature ca...
Agenda
What’s Azure Cloud Provider in OpenShift
Big Changes for Azure Cloud Provider
Experimental Features
Azure Cloud Provider
Concepts Underlying the Cloud Controller Manager
Azure Cloud Provider
kubernetes:
◦integrating Cloud features
◦Azure is one of the implementations
OpenShift
◦includes kube...
OpenShift 3.10 on Azure
includes kubernetes 1.10 – actually 1.10.1 + α
kubernetes now supports, but OpenShift doesn’t:
◦ c...
Red Hat OpenShift on Azure
https://www.redhat.com/en/about/press-releases/red-hat-and-microsoft-co-
develop-first-red-hat-...
Looking back the announcment
10000 feet overview
◦ Launch OpenShift cluster with one command
◦ First-class citizen in Azur...
Before starting…
Requirements for Azure Cloud Provider
Virtual Machines (VMs) must belong to a single resource group & VNE...
Node Name = VM name:
DO NOT include capital letters
Be sure these values are same
◦ Azure VM name: az vm create –n <name>
...
Big Changes
Big Changes
Installer Improvement
Upgrade Issue (3.9 to 3.10)
Azure Disk/File Mount options
Azure File default permission ...
Installer Improvement:
Internal Hostname in Azure NIC
Be sure to set internal hostname in the Azure NIC.
- Create a new NI...
Installer Improvement:
Support creating azure.conf
Advanced installer creates azure.conf.
Note: If you need more parameter...
How to create a service principal?
Execute the following three commands.
If you’re using Azure Cloud Shell, try the follow...
Installer Improvement:
Support creating storage class
Advanced installer creates storage class for Azure Disk/File.
opensh...
Upgrade Issue (3.9 to 3.10):
relocate azure.conf
OpenShift 3.10 doesn’t allow to locate azure.conf under /etc/azure.
Use /...
Azure Disk/File Mount options
You can specify mount options for Azure Disk and Azure File as a Volume.
Storage Class and P...
Azure File default permission
changed to 0755
The permission changed from 0777 to 0755. (at OCP 3.9 or newer)
The owner of...
Improved Azure Disk
mount/unmount performance
It takes huge time to unmount and mount an Azure Disk when a new version of ...
Experimental
Features
RED HAT DOESN’T TESTED THESE FEATURES.
Experimental Features
Advanced Options for ServiceType: LoadBalancer
Azure REST API authentication
Monitoring
Service Type: LoadBalancer
Node Virtual Machines
apiVersion: v1
kind: Service
metadata:
name: alb-svc
spec:
loadBalancerIP...
Advanced Options
Internal Load Balancer
Internal Load Balancer with subnet
DNS Label name
https://github.com/kubernetes/cl...
Service Type: LoadBalancer
for huge cluster
One Availability Set has maximum 200 VMs.
If you deploy over 200 OpenShift nod...
Azure REST API authentication:
Managed Service Identity
Managed Service Identity:
◦ You don’t have to provide password to ...
Monitoring - Prometheus
Prometheus monitors Azure API Request Duration Seconds & Error.
◦ Prometheus is Technology Preview...
New features of Azure Cloud Provider in OpenShift Container Platform 3.10
Upcoming SlideShare
Loading in …5
×

0

Share

Download to read offline

New features of Azure Cloud Provider in OpenShift Container Platform 3.10

Download to read offline

These slides are presented at London OpenShift meetup in August 2018.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

New features of Azure Cloud Provider in OpenShift Container Platform 3.10

  1. 1. New features of Azure Cloud Provider in OpenShift 3.10 Takayoshi Tanaka, Software Maintenance Engineer tatanaka@redhat.com
  2. 2. Takeaways Let you know Azure Cloud Provider is getting more and more useful. Let you know some upstream project/feature can work on OpenShift (however, without support)
  3. 3. Agenda What’s Azure Cloud Provider in OpenShift Big Changes for Azure Cloud Provider Experimental Features
  4. 4. Azure Cloud Provider Concepts Underlying the Cloud Controller Manager
  5. 5. Azure Cloud Provider kubernetes: ◦integrating Cloud features ◦Azure is one of the implementations OpenShift ◦includes kubernetes = almost all Cloud Provider features could work ◦Our tested features is describes in product documents
  6. 6. OpenShift 3.10 on Azure includes kubernetes 1.10 – actually 1.10.1 + α kubernetes now supports, but OpenShift doesn’t: ◦ cluster auto scale ◦ https://github.com/feiskyer/autoscaler/blob/master/cluster- autoscaler/cloudprovider/azure/README.md ◦ Azure VM Scale Set Private Preview in OpenShift in the future ◦ Windows Container Support ◦ If you have an interest, contact Red Hat sales in charge of you, or me.
  7. 7. Red Hat OpenShift on Azure https://www.redhat.com/en/about/press-releases/red-hat-and-microsoft-co- develop-first-red-hat-openshift-jointly-managed-service-public-cloud https://azure.microsoft.com/en-gb/blog/openshift-on-azure-the-easiest-fully-managed- openshift-in-the-cloud/ https://www.youtube.com/watch?v=WDr9TSyK9DA
  8. 8. Looking back the announcment 10000 feet overview ◦ Launch OpenShift cluster with one command ◦ First-class citizen in Azure ◦ Including Open Service Broker for Azure If you have an interest, please contact me later. I'll pass your info to our team. $ az openshift create -n oscluster -g osrg ` –node-vm-size Standard_DS4_v3` --l eastus
  9. 9. Before starting… Requirements for Azure Cloud Provider Virtual Machines (VMs) must belong to a single resource group & VNET ◦ Some improvements are ongoing VM name requirement ➔ Next page Internal Load Balancer is unavailable for master public end point OCP 3.7 or later is required for Managed Disk Service Type: LoadBalancer supports only Basic Azure Load Balancer ◦ Node VMs must belong to Availability Set (* I’ll explain later) Sovereign Cloud (Azure Germany, China etc) are available. ◦ Some extra configurations are required VM scale set is not tested in OCP
  10. 10. Node Name = VM name: DO NOT include capital letters Be sure these values are same ◦ Azure VM name: az vm create –n <name> ◦ Internal DNS Name: az nic create --internal-dns-name <name> ◦ Hostname (DO not edit manually) ◦ OpenShift node name (inventory file) This name should not include capital letters ◦ OK: ocp-node1 ◦ NG: OCP-Node1
  11. 11. Big Changes
  12. 12. Big Changes Installer Improvement Upgrade Issue (3.9 to 3.10) Azure Disk/File Mount options Azure File default permission changed to 0755 Improved Azure Disk mount/unmount performance
  13. 13. Installer Improvement: Internal Hostname in Azure NIC Be sure to set internal hostname in the Azure NIC. - Create a new NIC - Apply to the existing NIC $ az network nic create -g MyResourceGroup --vnet- name MyVnet --subnet MySubnet -n MyNic --internal- dns-name node1 $ az network nic update -g MyResourceGroup - n MyNic --internal-dns-name node1
  14. 14. Installer Improvement: Support creating azure.conf Advanced installer creates azure.conf. Note: If you need more parameters such as VNET or security group, you have to add them to azure.conf and restart services manually. openshift_cloudprovider_kind=azure openshift_cloudprovider_azure_client_id=<AAD_app_id> openshift_cloudprovider_azure_client_secret=<AAD_password> openshift_cloudprovider_azure_tenant_id=<AAD_tenant_id> openshift_cloudprovider_azure_subscription_id=<subscription_id> openshift_cloudprovider_azure_resource_group=<resource_group_name> openshift_cloudprovider_azure_location=<location>
  15. 15. How to create a service principal? Execute the following three commands. If you’re using Azure Cloud Shell, try the following script. https://gist.github.com/tanaka- takayoshi/802fcfbd8d544495977c2e8660604bc3 $ az account show -o json $ az group show -n <resource_group_name> -o json $ az ad sp create-for-rbac --name <app_name> --role contributor --scopes <resource_group_id> -o json
  16. 16. Installer Improvement: Support creating storage class Advanced installer creates storage class for Azure Disk/File. openshift_storageclass_name=slow-disk openshift_storageclass_provisioner=azure-disk openshift_storageclass_parameters={'storageaccounttype' : 'Standard_LRS', 'kind': 'Managed'}
  17. 17. Upgrade Issue (3.9 to 3.10): relocate azure.conf OpenShift 3.10 doesn’t allow to locate azure.conf under /etc/azure. Use /etc/origin/cloudprovider instead of /etc/azure. Before upgrading the cluster, please copy the file and edit the master and node configuration files.
  18. 18. Azure Disk/File Mount options You can specify mount options for Azure Disk and Azure File as a Volume. Storage Class and Persistent Volume has a mountOptions parameters. apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: slow mountOptions: - dir_mode=0777 - file_mode=0777 parameters: kind: Managed storageaccounttype: Standard_LRS provisioner: kubernetes.io/azure-disk reclaimPolicy: Delete apiVersion: v1 kind: PersistentVolume metadata: name: static-pv-file spec: accessModes: - ReadWriteOnce azureFile: secretName: azure-secret secretNamespace: openshift shareName: share1 capacity: storage: 1Gi mountOptions: - dir_mode=0777 - file_mode=0777 persistentVolumeReclaimPolicy: Delete
  19. 19. Azure File default permission changed to 0755 The permission changed from 0777 to 0755. (at OCP 3.9 or newer) The owner of the directory is different from the UID of container by default. ◦ https://docs.openshift.com/container- platform/3.10/install_config/persistent_storage/persistent_storage_azure_file.html#azu re-file-before-you-begin Configure mountOptions to ensure the permission. Use SecurityContext to run a container by specific GID. (OCP 3.10) Use SecurityContext (RunAsUser) & mountOptions (uid) to run a container by specific UID.
  20. 20. Improved Azure Disk mount/unmount performance It takes huge time to unmount and mount an Azure Disk when a new version of pod is located another node. It could happen when a new version of pod is located to a different node than the node where a current version of pod is running. pod pod
  21. 21. Experimental Features RED HAT DOESN’T TESTED THESE FEATURES.
  22. 22. Experimental Features Advanced Options for ServiceType: LoadBalancer Azure REST API authentication Monitoring
  23. 23. Service Type: LoadBalancer Node Virtual Machines apiVersion: v1 kind: Service metadata: name: alb-svc spec: loadBalancerIP: 40.121.183.52 type: LoadBalancer ports: - port: 80 selector: app: my-app IP should be created in advance. Or you don’t have to specify the ip.
  24. 24. Advanced Options Internal Load Balancer Internal Load Balancer with subnet DNS Label name https://github.com/kubernetes/cloud-provider-azure/blob/master/docs/development- loadbalancer.md annotations: service.beta.kubernetes.io/azure-load-balancer-internal: "true" annotations: service.beta.kubernetes.io/azure-load-balancer-internal: "true" service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "apps-subnet" annotations: service.beta.kubernetes.io/azure-dns-label-name: ""
  25. 25. Service Type: LoadBalancer for huge cluster One Availability Set has maximum 200 VMs. If you deploy over 200 OpenShift nodes, you have to deploy two or more AvailabilitySet. One ALB can have only one AvailabilitySet. So, you have to deploy one ALB per AvailabilitySet. Node VMs – group A Node VMs – group B annotations: service.beta.kubernetes.io/azure-load-balancer-mode: “group-a,group-b"
  26. 26. Azure REST API authentication: Managed Service Identity Managed Service Identity: ◦ You don’t have to provide password to configuration file ◦ https://docs.microsoft.com/en-us/azure/active-directory/managed-service- identity/overview Steps ◦ Enable MSI on all VMs for OpenShift nodes ◦ Grant your VM access to a Resource Group ◦ Update azure.conf useManagedIdentityExtension: true aadClientId: aadClientSecret:  Remove two lines  Add this line
  27. 27. Monitoring - Prometheus Prometheus monitors Azure API Request Duration Seconds & Error. ◦ Prometheus is Technology Preview at 3.10

These slides are presented at London OpenShift meetup in August 2018.

Views

Total views

786

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

12

Shares

0

Comments

0

Likes

0

×