Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Edge Computing Architecture using GPUs and Kubernetes

587 views

Published on

Speaker: Nobuyuki Tamaoki (VirtualTech Japan)
Title: Edge Computing Architecture using GPUs and Kubernetes
Agenda:
- What’s “Edge” Computing?
- Introduce to our Edge Computing POC project

Published in: Technology
  • Be the first to comment

Edge Computing Architecture using GPUs and Kubernetes

  1. 1. Edge Computing Architecture using GPUs and Kubernetes 日本仮想化技術株式会社 VitrualTech.jp 2018/8/14 1
  2. 2. VirtualTech Japan Inc. 日本仮想化技術株式会社 • Company name: VirtualTech Japan Inc. (called VTJ) • Address: 1-8-1 Shibuya Shibuya-ku Tokyo • Founded: Dec 2006 • President and CEO: Toru Miyahara • Number of employees: 8 (Engineer: 7, Business Development: 1) • Our service: • Consulting NFV/OpenStack for Japan telco company • NTT Docomo’s large-scale OpenStack services • NTT West’s one of management systems of fixed network service using OpenStack • Plan to consulting Edge + GPU Computing • Corporate Web Site: http://virtualtech.jp 2 Corporate profile
  3. 3. Our expertise at OpenStack We are experts in Open Infrastructure, OpenStack and NFV. 3 2014/11 OpenStack Summit Paris We spoke the knowledge and tips when building and operating OpenStack Cloud on 100 Physical Servers. (Neutron HA, VXLAN performance,,,) 2012/10 OpenStack Summit San Diego We announced baremetal provisioning framework which handles barematel machine like a virtual machine. This is merged upstream in Grizzly. 2015/10 OpenStack Summit Tokyo We (NTT West, Canonical and VTJ) spoke ”Requirements for Providing Telecom Services on OpenStack-based Infrastructure”. Corporate profile
  4. 4. 1. OpenStack 2. Kubernetes 3. Kubernetes on OpenStack 4. OpenStack on Kubernetes 5. Edge Cloud 6. NFV Cloud Network Function Virtualization Definition of words 4 Kubernetes OpenStack OpenStack Kubernetes 3. Kubernetes on OpenStack 4. OpenStack on Kubernetes Kubernetes Hardware Hardware Under Cloud Over Cloud 5. Edge Cloud 6. NFV Cloud Relation of OpenStack and Kubernetes Using Cloud/Container technology at Telco company Device Kubernetes Hardware OpenStack OpenStack Hardware Access Point Inter net
  5. 5. Today‘s agenda • What’s “Edge” Computing? • Introduce to our Edge Computing POC project 5
  6. 6. Questions about “Edge” Computing We have any questions about “Edge” Computing. • Can you tell me about your “Edge” ? • What’s “Edge” Computing ? • What’s key points of “Edge” Computing ? 6 What’s “Edge” Computing?
  7. 7. Can you tell me about your “Edge” ? • I know that the definition of “Edge” is different for each person. 1. Edge of Network nodes 2. Edge of Cloud / Computing 3. Server side of IoT Application etc • We want ”Edge” Computing that can be used in various use cases. 7 What’s “Edge” Computing?
  8. 8. What’s “Edge” Computing ? • We want ”Edge” Computing that can be used in various use cases. • I joined OpenStack Summit Vancouver. I watched some Telco Edge Computing projects. (AT&T, China Mobile and Verizon) • China Mobile’s use cases of “Edge” Computing From presentation “ Edge TIC – Future edge cloud for China mobile” • Enterprise Private Network (as like SD-WAN) • CDN Deployment • Live Sporting Event • Real time data backhaul of Unmanned Aerial Vehicle • V2X Service (V is Vehicle) 8 What’s “Edge” Computing?
  9. 9. What’s “Edge” Computing ? • We want ”Edge” Computing that can be used in various use cases. • AT&T and China Mobile are combining NFV, “Edge” and MANO, and are beginning to create the Next-gen Network Service Infra. 9 What’s “Edge” Computing? NFV Edge MANO MA NO MANO: NFV Management and Orchestration NF V NF V Ed ge Ed ge Ed ge Regional (4+) Province (100+) City (600+) County (3000+) AP (100K+) Software • MANO: ONAP • NFV: OPNFV (Based OpenStack) • Edge: Akraino (Based Kubernetes on OpenStack) The number above is the assumed value of China Mobile.
  10. 10. What’s key points of “Edge” Computing ? • It’s important to think about both ”Technical side” and ”Business side” for success “Edge” Computing. • Technical side • We have to solve the technical problem related to “Edge” Computing. • Container , Kubernetes and Kubernetes ecosystem (include Kubernetes on OpenStack) • Running Kubernetes production, logging and problem solving • Business side • We have to think the business model using “Edge” Computing. • We are ready to help thinking your services and solutions leveraging “Edge” Computing + GPUs. • Cost reduction, Operation optimization • Create new business and new revenue (ex. Selling edge nodes for advertise items) 10 What’s “Edge” Computing?
  11. 11. “Edge” Computing + GPUs is Big impact! • Operation side Ex. Auto healing for Cloud Infra. • Service side Ex. Live Sporting Event 11 What’s “Edge” Computing? NFV Edge MANO Big data & Log Streaming GPU 1. Error occur at Network services 2. Detect error from logs Policy Engine 3. Action (change routing) OpenStack Auto healing SIG is alpha. Edge Edge GPU Football Stadium Camera Camera Edge 1. Streaming from Cameras 2. Processing Streaming Data GPU Users3. Streaming GPU power-ed live videos 0. Booked Edge nodes
  12. 12. Summary • I know that the definition of “Edge” is different for each person. • We want ”Edge” Computing that can be used in various use cases. • AT&T and China Mobile are combining NFV, “Edge” and MANO, and are beginning to create the Next-gen Network Service Infra. • It’s important to think about both ”Technical side” and ”Business side” for success “Edge” Computing. • “Edge” Computing + GPUs is Big Impact! • We are ready to help thinking your services and solutions leveraging “Edge” Computing + GPUs. 12 What’s “Edge” Computing?
  13. 13. Today‘s agenda • What’s “Edge” Computing? • Introduce to our Edge Computing POC project 13
  14. 14. Summary of Our Edge Computing POC • This’s a use case of Japanese telecom company. • Starting 5G/Edge Computing POC project using Kubernetes and NVIDIA GPUs. • This Edge Computing runs CPUs/GPUs at edge nodes. • Kubernetes manage Edge Computing Infra + GPUs. • We’re using Canonical Juju/MAAS (Provisioning tools) for Zero- touch provisioning. 14 Edge Computing POC
  15. 15. Understanding Multi-Access Edge Computing 15
  16. 16. About Multi-Access Edge Computing (MEC) External Factor 5G Network is ready Edge Throughput: 100Mbps Latency: 1ms Peak Data Rate: 20Gbps All Telco company need to promote 5G Network Internal Factor Cost reduction and productivity improvement Next-generation Network Virtualization (NFV / SD-WAN) production is planning Edge Computing POC
  17. 17. Understanding MEC 5G/MEC use case MBB: Mobile Broadband mMTC: massive Machine Type Communications Dense Inf Society Connected vehicles VR office/factory/tactile Throughput Latency Reliability Availability Energy Efficiency User/Device density Implications of 5G RAN and IoT on OpenStack based edge computing. より引用 [ OpenStack Summit にて AT&T, Ericsson 発表 ] https://www.openstack.org/videos/sydney-2017/implications-of-5g-ran-and-iot-on-openstack-based-edge-computing From AT&T‘s MEC POC
  18. 18. Disaggregated CoreDisaggregated RAN Understanding MEC (cond.) AT&T’s MEC Architecture 5G Application Ecosystem IoT Connected Car MBB RU DU UPF UPF Macro Radio & Small cell Antennas 5G Base Stations Edge Cloud Centralized Cloud CCF Internet CU-CP CU-UP NFV MANO (Management & Orchestration) CU: Centralized Unit CP: Control Plane UP: User Plane UPF: User Plane Function CCF: Core Control Function RU: Radio Unit DU: Digital Unit Implications of 5G RAN and IoT on OpenStack based edge computing. より引用
  19. 19. Understanding MEC (cond.) • Building Docker / Kubernetes controller • Zero-touch Provisioning is key • Planning thousands of locations • Support emerging technology at edge node (GPU, SmartNIC, FPGA, etc) • Planning collaboration with SDN/NFV and Orchestration Feedback from AT&T’s MEC project Edge Computing POC
  20. 20. Edge Computing+GPUs 20
  21. 21. Proof of Concept(POC) #1 The scope of POC#1 is the following. • Building edge controller and container nodes using Kubernetes • Zero-touch Provisioning • Support GPUs at container nodes The scope of POC#2 is planning. Edge Computing POC
  22. 22. Container / Compute Nodes Edge Computing + GPUs Architecture NFV MANO Edge Controllers Physical Provisioning Application Provisioning SDN / SDS Monitoring / Alerting Orchestrator GPU Hi speed networking General purpose Low energy Hi speed storage GPU Server GPU Server Storage Server Storage Server Object Storage Servers w/t SmartNIC Servers Scope of Edge Cloud ServerServer Server Edge Computing POC
  23. 23. Container nodes Scope of Edge Computing + GPUs POC#1 NFV MANO Edge Controllers Physical Provisioning Application Provisioning SDN / SDS Monitoring / Alerting Orchestrator GPU Hi speed networking General purpose Low energy Hi speed storage GPU Server GPU Server Storage Server Storage Server Object Storage Servers w/t SmartNIC Servers Scope of Edge Cloud ServerServer Server Edge Computing POC
  24. 24. Components for Edge Computing Components • Edge Cloud • Edge Controllers • Physical Provisioning: Ubuntu MAAS • Application Provisioning: Ubuntu Juju • Orchestrator: Kubernetes • SDN(Software Defined Network): Flannel (I believe Juniper Contrail needs it) • Monitoring/Alerting: Prometheus, Grafana • Container nodes • GPU Server • General Purpose Server: Intel and ARM Server Edge Computing POC
  25. 25. Questions: VM vs Container • Existing Apps running on VMs will remain VMs. (You can migrate VMs to Containers, but cost does not match.) • New Apps such as IoT, Edge Computing and AI will be advanced with Containers. • NFV (their service infrastructure such as 5G and Fixed service) is currently VMs, Next generations will be Containers. (AT&T planed) • Large size (Servers > 100), prepare "Kubernetes on OpenStack" and let the user choose VMs or Containers. • Middle size (20 < Servers < 100), the user choice "Kubernetes" or "OpenStack". • Small size (Servers < 20), the user choice "Kubernetes". 25 Edge Computing POC
  26. 26. Kubernetes Kubernetes vs ”Kubernetes on OpenStack” • Kubernetes • Kubernetes on OpenStack 26 Kubernetes Container ContainerContainer Container ContainerContainer Kubernetes’s Good: • common to use Kubernetes to manage containers • Light weight controller • Auto healing is very good Kubernetes’s Bad: • No Multi-Tennant • No Network Policy related SDN • No Persistent Storage Kubernetes OpenStack “Kubernetes on OpenStack“ add missing features of Kubernetes. However, OpenStack’s controller isn’t Light weight. We have to think to apply it. Edge Computing POC
  27. 27. POC environment 27
  28. 28. Container nodes POC#1 environment (H/W) Edge Controllers • Physical Provisioning • Monitoring /Alerting • Application Provisioning • Orchestrator • SDN POC#1のControllersの多重化は行わない想定 • GPU • General purpose Edge Computing POC
  29. 29. Container nodes POC#1 environment (S/W) Edge Controllers • Physical Provisioning • Monitoring /Alerting • Application Provisioning • Orchestrator • SDN POC#1のControllersの多重化は行わない想定 • GPU • General purpose MAAS/Juju Prometheus / Grafana Kubernetes Flannel nvidia docker docker Edge Computing POC
  30. 30. ・Normal x86_64 Server ・Juju/MAAS ・Prometheus ・Grafana ・apt local repository ・Normal x86_64 Server ・Kubernetes Master node ・docker image pool ・Normal x86_64 Server ・Kubernetes Master node (・docker image pool) ・Normal x86_64 Server w/GPU ・Kubernetes Worker node ・Normal x86_64 Server ・Load Balancer ・ARM64 Server ・Kubernetes Worker node ・1GbE以上の Switch ・10GbE Switch IPMI IPMI IPMI IPMI IPMI Port VLANでも構いませ ん IPMI ・作業用端末 MAAS, メンテナンス用 for Podデプロイ, 一般通信用 ・確認用端末 ※Podデプロイの通信を独立させる構想は、以後のPoCでの検証対象とします 必要に応じて移動 • Ubuntu Server • Juju/MAAS • Kubernetes • GPU Server • ARM Server • Flannel • Prometheus • Grafana POC#1 environment (Our Testbed) Edge Computing POC
  31. 31. Next Step • Try OSS about Edge Computing + AI/DL • From AT&T OSS • Airship: Infrastructure project for OpenStack and Kubernetes • Akraino: Edge Computing Framework • Acumos AI: develop ML models for cloud optimization use-cases • From Kubernetes issues • Container Network (Calico, Tungsten Fabric, Cilium, etc) • Container Security (Istio, etc) • Persistent Storage (Ceph, Rook, etc) • Application deployment (Spinnaker, etc) 31
  32. 32. Summary of Our Edge Computing POC • This’s a use case of Japanese telecom company. • Starting 5G/Edge Computing POC project using Kubernetes and NVIDIA GPUs. • This Edge Computing runs CPUs/GPUs at edge nodes. • Kubernetes manage Edge Computing Infra + GPUs. • We’re using Canonical Juju/MAAS (Provisioning tools) for Zero- touch provisioning. 32 Edge Computing POC
  33. 33. Apendix • What’s Kubernetes • OpenStack Summit Feedback 33
  34. 34. Kubernetes 参考) Kubernetesとは • Kubernetes(以降、K8sと記す)はコンテナのオーケストレー ションツールです。 • コンテナの配置だけでなく、ロードバランシングや、コンテナ のオートスケールやオートヒーリングを実現します。 • コンテナ基盤のセルフヒーリングやローリングアップデートの 機能を備えます。 34 C C C Server C C C Server C C C Server Orchestrator Dockerなど のコンテナ ソフトウェア
  35. 35. 参考) Kubernetes と OpenStack • レイヤの異なるものなので比較するものではありません • コンテナの管理に特化したのが Kubernetes • 仮想マシンやコンテナやベアメタルサーバを管理するのが OpenStack • OpenStack上でKubernetesを動かすK8s on OpenStackもあり ます • アプリケーションがコンテナ/仮想マシンを選択することにな ります。 • 仮想マシン上のアプリケーションをコンテナに移植することは可能で すが、全てのアプリケーションを移行することは現実的ではありませ ん。 35
  36. 36. OpenStack Summit Feedback (1 1) AT&T's "Network AI" Network AI: AT&T’s Framework for Its Open Source Efforts That Will Drive our Software-Defined Network in 2018 and Beyond http://about.att.com/innovationblog/att_framework 36
  37. 37. 37 “The OpenStack and Kubernetes Smørrebrød (Open Sandwich)”より引用
  38. 38. OpenStack on Kubernetes 38 Under Cloud Over Cloud Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Server Control Plane Control Plane Control Plane Control Plane Control Plane Control Plane Control Plane Control Plane Control Plane Contain er node Contain er node Control Plane Contain er node Control node Control node Control node Control node Control node Control node Control node Compute node Compute node Control node Compute node 1. Single Node Bootstrap 2. Expand Control Plane 3. Deploy Additional Masters 4. Deploy Compute Hosts Kubeadm Self-hosted Deployment •Keystone •Nova •Glance •Heat •Ironic •Ceph Discover baremetal servers using Ironic Over Cloud で OpenStack Under Cloud で Kubernetes
  39. 39. OpenStack Summit Feedback 2) 2) Acumos AI Project A federated platform for managing AI and ML applications and sharing AI models. AT&T and Tech Mahindra contributed the initial Acumos code, now freely available for download. The Linux Foundation Launches Open Source Acumos AI Project https://www.acumos.org/news/2018/03/26/the-linux- foundation-launches-open-source-acumos-ai-project/ 39
  40. 40. OpenStack Summit Feedback 3) 3) Telus, Canadian telco comapny, AI Challenge Telus's AI Challenge is excellent. You can watch the following video. I will share interesting slide by email. Artificial Intelligence driven Orchestration, Challenges and Opportunities https://www.openstack.org/videos/vancouver-2018/artificial- intelligence-driven-orchestration-challenges-and- opportunities 40
  41. 41. In 2020, Begin to utilize Containers and AI 41
  42. 42. “Software Defined Service Operation” 42
  43. 43. Challenge to utilize AI/ML at NFV and Edge 43
  44. 44. AI/ML Orchestration Eco system 44
  45. 45. 参考)仮想化通信 コンテナ関連記事1 Spinnakerのパイプラインによる自動デプロイ https://tech.virtualtech.jp/entry/2018/06/25/180834 KubernetesクラスタへSpinnakerを構築 https://tech.virtualtech.jp/entry/2018/05/23/134107 Jujuで構築するKubernetesでGPUを使うには https://tech.virtualtech.jp/entry/2018/03/28/113217 Jujuでデプロイするサービスを追加する方法 https://tech.virtualtech.jp/entry/2018/07/04/120958 KubernetesでPodを作る方法 https://tech.virtualtech.jp/entry/2018/04/06/100843 Kubernetesと永続ストレージの使い方 https://tech.virtualtech.jp/entry/2018/05/28/190542 45
  46. 46. 参考)仮想化通信 コンテナ関連記事2 Kubernetesで永続ストレージを構築するツールのRook(の最新版)を使う https://tech.virtualtech.jp/entry/2018/07/17/121740 JujuでCalicoを使ったKubernetes環境を構築する(前編) https://tech.virtualtech.jp/entry/2018/06/15/115735 JujuでCalicoを使ったKubernetes環境を構築する(後編) https://tech.virtualtech.jp/entry/2018/06/15/162143 Juju + MAAS を使用したOpenstack構築 + 監視設定 https://tech.virtualtech.jp/entry/2018/04/24/131339 Juju + MAAS を使用したKubernetes構築 + 監視設定 https://tech.virtualtech.jp/entry/2018/03/29/104216 Airshipを試してみた https://tech.virtualtech.jp/entry/2018/07/19/175614 46
  47. 47. 47

×