Kubernetes上で動作する機械学習モジュールの配信＆管理基盤Rekcurd について
•
2 likes•10,353 views
Rekcurd is a tool for deploying and managing machine learning models on Kubernetes. It provides a dashboard for uploading models, switching between models without stopping services, and versioning models. Rekcurd integrates with Kubernetes for high availability, auto-scaling, load balancing and auto-healing of machine learning services. The presentation demonstrates how Rekcurd addresses common tasks in serving machine learning models on Kubernetes.
More Related Content
What's hot
What's hot (20)
Similar to Kubernetes上で動作する機械学習モジュールの配信＆管理基盤Rekcurd について
Similar to Kubernetes上で動作する機械学習モジュールの配信＆管理基盤Rekcurd について (20)
More from LINE Corporation
More from LINE Corporation (20)
Recently uploaded
Recently uploaded (20)
Kubernetes上で動作する機械学習モジュールの配信＆管理基盤Rekcurd について
- 1. Kubernetes Drucker Rekcurd Keigo Hattori, Search & Clova Center
- 2. • • → Rekcurd • Kubernetes Deployment Pod • Kubernetes → Rekcurd
- 3. Who I am @keigohtr Keigo Hattori keigohtr Keigo Hattori Software Engineer 2009 2009~2017.10 2017.11~ LINE x Clova Apitore
- 4. Join us! • Rekcurd GitHub OSS https://github.com/rekcurd Issue/Pull Request • • We are hiring!
- 7. TITLE
- 16. Clova
- 18. • • → Rekcurd • Kubernetes Deployment Pod • Kubernetes → Rekcurd
- 19. BackgroundSVM Deep LearningMachine Learning Keras Chainer Caffe scikit-learn gensim logistic regression Random Forest Neural Network Perceptron libsvm liblinear Theano
- 21. 1. Data i. Collection ii. Cleaning/Cleansing 2. Feature i. Preprocessing ii. Dictionary 3. Training i. Algorithm ii. Parameter tuning iii. Evaluation 4. Others i. Server setup ii. Versioning (data, parameter, model, result) Tasks in building ML model
- 23. • ü ü ü
- 24. • ü ü ü
- 25. 1. High Availability 2. Management i. Upload the latest ML model ii. Switch a model without stopping services iii. Versioning models 3. Monitor i. Load balancing ii. Auto healing iii. Auto scaling iv. Performance/Results check 4. Others i. Server setup (development/staging/production) ii. Integration to the existing services iii. AB testing iv. Managing many ML services v. Logging Tasks in serving ML service Rekcurd x Kubernetes
- 26. Rekcurd • • •
- 27. Rekcurd • Rekcurd • Rekcurd dashboard (x Kubernetes) • Rekcurd client
- 28. Rekcurd
- 29. • pip install Flask ü FW • gRPC ü gRPC spec gRPC Rekcurd
- 30. Rekcurd • Input format • Output format Field Type Description input string/bytes/list Tensor option string (json format) Rekcurd option fields key-value Field Type Description label string/bytes/list Tensor score float/list multi-class option string (json format) Rekcurd option fields key-value
- 32. Rekcurd dashboard • Rekcurd ü Rekcurd ü • WebUI ü Kubernetes
- 36. Add Application (2/3) • Base Image • Git • Git Branch • • CPU/Memory
- 37. Add Application (3/3) • • Autoscale Policy • Deploy Policy •
- 38. Add Application
- 39. Application /
- 40. Add Model ML
- 41. Add Model
- 42. Switch Model ML
- 44. Service URL
- 45. Model
- 49. Rekcurd client
- 50. • pip install SDK ü URL ü gRPC spec gRPC • Netflix Feign ü DNS + App + Kubernetes Rekcurd Rekcurd client
- 52. Kubernetes gRPC
- 54. Kubernetes
- 55. Rancher ü LINE Private Cloud Kubernetes Rancher 2.x ü Clova Rancher1.6 Kubernetes Rekcurd Rekcurd GCP/AWS/kubeadm
- 56. HA /
- 57. Kubernetes • Auto healing ü Deployment Pod • Auto scaling ü HorizontalAutoScaler Pod • Rolling update ü Deployment Pod ü Pod Rekcurd dashboard
- 58. Dev Prod Node HA Node Dev Prod
- 59. Kubernetes Node SL: dev Node SL: stg Node SL: prod Node SL: prod namespace x node selector x affinity Pod App: hoge SL: dev Pod App: hoge SL: stg Pod App: hoge SL: prod Pod App: hoge SL: prod Pod App: hoge SL: prod Rekcurd dashboard
- 60. • namespace ü service level namespace Pod • node selector ü node service level service level node Pod • affinity ü Pod application name HA node Pod namespace x node selector x affinity Rekcurd dashboard
- 61. gRPC Load Balancing Rancher gRPC Load balance
- 62. Ingress • nghttpx ingress controller ü http2 Load balancer (nginx ingress controller ) Rekcurd Service mesh (Istio) • Ingress Load balancer annotation ü DNS Host name routing http://<app-name>-<service-level>.<domain> Rekcurd dashboard
- 63. fluentd logger
- 64. ü fluentd Kubernetes Daemonset ü Pod stdout/stderr Forwarding ü Clova kibana logger fluentd-kubernetes
- 65. Node/Pod Docker image image Node Disk full
- 66. Online Storage Kubernetes Node SL: dev Node SL: stg Node SL: prod Node SL: prod MySQL & Online storage Pod App: hoge SL: dev Pod App: hoge SL: stg Pod App: hoge SL: prod Pod App: hoge SL: prod Rekcurd dashboard
- 67. • Node Online storage (e.g. WebDAV) ü LINE AWS S3 Private online storage ü goofys Node • Pod Node volume ü Node Online storage • Application ML MySQL ü Pod DB MySQL & Online storage Rekcurd dashboard
- 68. Rekcurd on Kubernetes Architecture
- 69. Rekcurd 1. Rekcurd dashboard Kubernetes 2. Pod git pull Rekcurd 3. Rekcurd
- 70. git pull Rekcurd
- 71. Rekcurd 1. Rekcurd dashboard Kubernetes 2. Pod git pull Rekcurd 3. Rekcurd Docker Image Docker • Docker • Image
- 72. Docker
- 73. • Rekcurd Docker Hub ü “docker pull rekcurd/rekcurd:python-latest” ü Rekcurd OK ü Dockerfile private docker registry image No and Yes
- 74. AB Canary
- 77. LDAP ü LDAP ML
- 78. Kubeflow
- 79. Kubeflow • TensorFlow • • WebUI • etc... Kubeflow
- 80. Rekcurd
- 81. Rekcurd Apache2.0 https://github.com/rekcurd Welcome Feedback and Pull Request!!! Rekcurd
- 83. We are hiring!!!
- 84. 1. High Availability 2. Management i. Upload the latest ML model ii. Switch a model without stopping services iii. Versioning models 3. Monitor i. Load balancing ii. Auto healing iii. Auto scaling iv. Performance/Results check 4. Others i. Server setup (development/staging/production) ii. Integration to the existing services iii. AB testing iv. Managing many ML services v. Logging Tasks in serving ML service Rekcurd x Kubernetes
- 85. THANK YOU
- 86. @keigohtr Keigo Hattori keigohtr Keigo Hattori @line_clova #Clova LINE Clova http://clova-blog.line.me/ja/