Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Dive OpenShitt on Azure & .NET Core on OpenShift


Published on

London OpenSift meetup Aug 14

Published in: Technology
  • Be the first to comment

Deep Dive OpenShitt on Azure & .NET Core on OpenShift

  1. 1. Deep Dive OpenShift on Azure & .NET Core on OpenShift Takayoshi Tanaka @TanakaTakayoshi Red Hat K.K. (Japan)
  2. 2. Notes: This slide is available online. As I have tested at OCP 3.5 and .NET Core 2.0 preview2, something will be changed at the latest OCP 3.6 and .NET Core 2.0 RTM. If you have any question or comments, feel free to contact me: email: Twitter: @TanakaTakayoshi
  3. 3. Background Red Hat K.K. (Japan) ◦ Software Maintenance Engineer ◦ OpenShift ◦ Red Hat solutions on Azure ◦ .NET Core on RHEL Personal ◦ Microsoft MVP for VSDT ◦ C# Lang, .NET Core on Linux ◦ Blogs: ◦ Red Hat Developers ◦ Personal Blog “Silver light and Blue sky” VSDT: Visual Studio & Development Technologies
  4. 4. Goal ◦ Learn about OpenShift on Azure Reference Architecture ◦ How to integrate Azure Features with OpenShift ◦ .NET Core 2.0/integrating OpenShift features with ASP.NET Core
  5. 5. Reference Architecture Document is now available ◦ Deploying Red Hat OpenShift Container Platform 3 on Microsoft Azure
  6. 6. OpenShift Ansible - Azure ARM Template
  7. 7. Installation Summary 1 • ARM Template for Azure Resources (VM, LB, NW…) 2 • Custom Script Extension with ARM • generate config. files & execute ansible 3 • Ansible Installer for OpenShift
  8. 8. Known Issue Available only in the Azure Marketplace VM • duplicated billing. Custom image (.vhd) is on the roadmap. No official Red Hat is available (self-support only) • You should troubleshoot by yourself. The OpenShift VM configuration is fixed • 3 masters with etcd (same hosts), 3 infra nodes, 3+ nodes, 1 bastion
  9. 9. 構成図
  10. 10. 構成図
  11. 11. 構成図
  12. 12. 構成図
  13. 13. 構成図
  14. 14. VMs 1 bastion 3 masters with etcd 3 infra nodes (router/docker registry) 3+ nodes Support request required for increasing cpu core limit. This limitation is due to design of ARM template. You can install all-in-one OpenShift on 1 host (not supported)
  15. 15. Examples: Integrating Azure Features Availability Set Azure Load Balancer ◦ master endpoint ◦ backend is a group of masters ◦ routing endpoint ◦ backend is a group of infra nodes (routers) Azure VHD for Persistent Volume (PV) ◦ Virtual Hard Disk for Azure VM (VHD) ◦ Dynamic provisioning Available at OCP 3.5+
  16. 16. How does Azure VHD for PV work? node service /etc/azure.conf 1. node service receives Volume Mount request 2. Load azure.conf (API auth etc) 3. (if dynamic provisioning) Create an empty VHD 4. Mount VHD to Azure VM 5. Create filesystem if needed 6. Mount filesystem to container Depending on kubernetess Azure Volume Plugin VHD attach
  17. 17. How to configure azure.conf See the document for more detail. Easy 3 steps with Azure CLI 2.0 $ az account list -o json //Retrieve tenantID & id $ az group show --name <ResourceGroupName> -o json //Retrieve id & location $ az ad sp create-for-rbac --name <ResourceGroupName> --role contributor --scopes "<Resource Id>“ -o json //Retrieve appId, password
  18. 18. Azure VHD for PV Notes Managed Disk is unavailable ◦ kubernetes Azure Disk plugin is not supported Managed Disk Be sure to confirm VM name to hostname ◦ Also specification of kubernetes plugin Configure DNS yourself ◦ VMs can be communicate with their VM name. ◦ If not using Azure internal DNS ◦ If using VNET peering or other
  19. 19. More Azure Features Azure Active Directory Open ID Connect ◦ authentication for master ◦ LDAP integration with AAD+AAD DS or AD is also available. Azure Blob Storage for OpenShift internal docker registry ◦ object storage is suitable for docker registry storage Azure File Storage ◦ File storage is also available for PV ◦ Linux kernel CIFS module with SMB 3 is still experimental Operation Management Suite integration ◦ Log Analysis also available for containers
  20. 20. How to set up OpenID connect? Create Azure AD App using the Microsoft Azure portal
  21. 21. How to set up LDAP auth with AD? Option A) AAD + AAD DS + (VNET peering or VNET-to-VNET VPN) * AAD DS only supports Classic VNET and requires private network from ARM VNET. AAD AAD DS classic VNET ARM VNET OCP master ldap://xx.xx.xx.xx/
  22. 22. AAD DS configuration example - name: "aad_ds_provider" challenge: true login: true mappingMethod: claim provider: apiVersion: v1 kind: LDAPPasswordIdentityProvider attributes: id: - dn email: - userPrincipalName name: - cn preferredUsername: - cn bindDN: "cn=adadmin,ou=AADDC Users,DC=example,DC=onmicrosoft,DC=com" bindPassword: "<password>" insecure: true url: "ldap://XXX.XX.XX.XX/OU=AADDC Users,DC=example,DC=onmicrosoft,DC=com? userPrincipalName?sub?(memberof=CN=ocpgroup,OU=AADDC Users,DC=example,DC=onmicrosoft,DC=com)" master-config.yaml ou: AADDC Users AAD default OU userPrincipalName will be email
  23. 23. How to set up LDAP auth with AD? Option B) on premise AD + VPN Connect on premise Network and ARM Network with VPN. AD on premise NW ARM NW OCP master ldap://xx.xx.xx.xx/
  24. 24. Storage Technology Comparison Type References Notes Azure Blob Storage Object storage Extended Registry Configuration Microsoft Azure storage driver Deploying Your Own Private Docker Registry on Azure Azure Blob Storage Only Available for docker registry storage Azure VHD Filesystem on external Disk Persistent Storage Using Azure Disk Configuring for Azure About disks and VHDs for Azure Linux VMs Depends on k8s plugin Azure File Storage NFS Persistent Storage Using Azure File Configuring for Azure How to use Azure File Storage with Linux Depends on k8s plugin Experimental. External NFS service NFS N/A Should maintain yourself or buy 3rd party service.
  25. 25. Operation Management Suite (OMS) Log Analysis & other features for on-premised to cloud Containers (Preview) solution in Log Analytics now support OpenShift
  26. 26. Installing OMS Agent Adding OMS agent directly on Linux Host Or, install agent as a OpenShift daemonset
  27. 27. Container solution
  28. 28. Future: Windows Container? No roadmap: Windows Container kubernets has roadmap for working with Windows Container “Capability” is existing.
  29. 29. .NET Application Model 35
  30. 30. .NET Core Inside 36 .NET Core App (C#/VB) IL Assembly (exe, dll) Roslyn CoreFX (.NET Core Class Library) ManagedUnmanaged OS Native ABI etc (F#) compile r CoreCLR (.NET Core Runtime) .NET Core SDK Tools
  31. 31. .NET Core on OpenShift s2i build ◦ “Source code in the Git repo” To “docker Image” ◦ can run out of OpenShift s2i image ◦ parameters for simple customize ◦ more customization are available with s2i scripts Template project ◦ Start .NET Core on OpenShift with few clicks at the portal ◦ All in one: deploymentconfig, service, route etc…
  32. 32. s2i build & deploy flow builder pod SCM(git) internal registry deployer pod pod deploymentConfigbuildConfig $ dotnet build $ dotnet publish $ dotnet <dll>
  33. 33. .NET Core 2.0 launch start today! rh-dotnet supports csproj at .NET Core 2.0 ◦ rpm version will be available ◦ s2i for .NET Core 2.0 & ASP.NET Core 2.0 ◦ Runtime image & s2i image (s2i image only at 1.x) More new features coming ◦ Announcing .NET Standard 2.0 ◦ Announcing .NET Core 2.0 ◦ Introducing ASP.NET Core 2.0 ◦ Announcing Entity Framework Core 2.0
  34. 34. Use Case Examples ◦Schedule Jobs with .NET Core ◦Switching Configuration for Dev & Prod Environment ◦Razor Page & C# 7.1 ◦Redis for HTTP Session storage with multi pods Notes: All examples are built on .NET Core 2.0 preview. We’re actively working on it now.
  35. 35. Schedule Job with .NET Core Run .NET Core Console App as a cron job: Cron Jobs Example Repository Web portal does not support cron jobs, so use the CLI. $ oc create imagestream cronjobexample $ oc create -f cronjob-buildconfig.yaml $ oc create -f cronjob.yaml
  36. 36. Schedule Jobs with .NET Core spec: containers: - command: - /opt/rh/rh-dotnet20/root/bin/dotnet - bin/Release/netcoreapp2.0/CronJobExample.dll image: imagePullPolicy: Always name: lipsum-cron resources: {} restartPolicy: Never securityContext: {} terminationGracePeriodSeconds: 30 schedule: '*/1 * * * *' command to execute: should be the full path command to execute: *scl should be enabled. To be fixed in my example image should be specified with full URL OCP 3.6 will support imagestreamtag. Replace with your internal registry’s IP and port
  37. 37. Switching Configuration for Dev & Prod Environment How to treat different environments with one code ◦ Connect to different database ◦ Use Redis as a cache - only in a production environment ◦ Integrate with a different OpenID account Use Environment feature in ASP.NET Core ◦ Specified by environment variables. Configuration can be injected specific to each environment.
  38. 38. Switching with Environment Startup class • constructor • Configure method • ConfigurreService method Check IHostingEnvironment Can’t inject IHostingEnvironment into ConfigerService method
  39. 39. Switching ConfigureXXXServices ConfigureXXXServices
  40. 40. Switching ConfigureXXX ConfigureXXX
  41. 41. Inject configuration from Environment Variable
  42. 42. Loading configuration from Secret use OpenShift secret feature.
  43. 43. Razor Page + C# 7.1 RazorPage: ◦ Simpler application than original MVC: “Page-focused scenarios” ◦ WebMatrix like easy development ◦ Razor Page is enabled with MVC C# 7.1: ◦ available at .NET Core 2.0 & ASP.NET Core 2.0 ◦ C# 7.1 in Razor page is not working at Preview 2 by bug (see issue) ◦ It should be fixed at 2.0 RTM.
  44. 44. HTTP session for multi pods By default: ◦ Sticky session: request goes to the same pod in same user session ◦ HTTP session is stored in the memory of each pod ◦ HTTP session is encrypted by pod specific key It means: When a pod has died, a user session will be lost.  How to keep HTTP session
  45. 45. IDistributedCache & IDataProtection IDistributedCache ◦ Provide distribution cache ◦ Available for storing session ◦ ASP.NET Core team provides SQLServer and Redis IDataProtection ◦ Provide key management for encryption ◦ Encrypt http session ◦ By default, generate machine (=pod) specific key and store in local file ◦ ASP.NET Core team provides NFS, Redis and AzureStorage (Preview)
  46. 46. machine A pod B pod C ASP.NET Core ASP.NET Core ASP.NET Core Session Data A’ Each pod has a different key. Can’t decrypt session data When loading another pod from a different session load with same id. ~/.aspnet default implementation of IDataProtection
  47. 47. machine A machine B machine C ASP.NET Core ASP.NET Core ASP.NET Core Session Data Use DataProtection.Redis
  48. 48. Configuration for Redis public void ConfigureServices(IServiceCollection services) { // You can retrieve this connection string from Azure Portal. var conn = Configuration["REDIS_CONNECTION_STRING"]; var redis = ConnectionMultiplexer.Connect(conn); services.AddDataProtection() .PersistKeysToRedis(redis, "DataProtection-Keys"); services.AddDistributedRedisCache(option => { option.Configuration = conn; option.InstanceName = "master"; }); services.AddSession(); },password=<password>,ssl=True,abortConnect=False
  49. 49. High Level Debugging .NET Core 58 GDB/LLDB MICore MIDE/Engine.Impl MIDE(*)/AD7.Impl VSCode Debugger AD7 Interface *MIDE: MIDebugEngine: GitHub repository *vsdbg can be used only in VS products and might not be distributed. MIText VS Debugger VS Debugger Engine vsdbg (closed license*) windbg See more: Architecture of MIEngine
  50. 50. Remote Debugging .NET Core vsdbg provided by Microsoft ◦ only trusted communication is required ◦ SSH is generally available ◦ VS remote debugger tools is also available on Windows ◦ Due to the license limitation, VS products (VS, VS Code, VS for mac) are only available for debugging. * Low level debugger is provided by Red Hat ◦ sos ◦ Not providing graphical debugger interface 59
  51. 51. Remote debugging to a container on OpenShift “oc rsh” is available instead of ssh vsdbg should be manually installed ◦ install script is unavailable as s2i image doesn’t have unzip ◦ download vsdbg on local and rsync see more detail in my wiki
  52. 52. Remote debug from Visual Studio Code .vscode/launch.json 61 { "name": ".NET Core Docker Remote Attach", "type": "coreclr", "request": "attach", “processId”: “1”, "pipeTransport": { "pipeProgram": “oc", “pipeArgs”: [ “rsh”, “-T”, “”], “quoteArgs”:false, “debuggerPath”: “/opt/app-root/src/vsdbg/vsdbg”, "pipeCwd": "${workspaceRoot}" }, "sourceFileMap": { "/opt/app-root/src": "${workspaceRoot}“ } }
  53. 53. Summary OpenShift on Azure ◦ Reference Architecture is a good place to start. ◦ More Azure features available-- Authenticating with OpenID and others .NET Core 2.0/ASP.NET Core 2.0 on OpenShift ◦ csproj support ◦ cronjob for .NET Core console app ◦ OpenShift secret & configuration. ASP.NET Core environment ◦ Remote debugging