Deploying and Managing Artificial Intelligence Services using the Open Data Hub Project on Openshift Container Platform

1. 1
Deploying and Managing Artificial
Intelligence Services using the Open
Data Hub Project on Openshift
Container Platform
Orgad Kimchi
Associate Manager, Consulting
Sep 18 2019

2. ● Introduction
● Why Red Hat for AI/ML?
● AI as Service High Level Architecture
● What is Open Data Hub?
● Representative Open AI/ML Architectures
● Demo
2
Agenda

3. ● Artificial Intelligence/Machine Learning (AI/ML) is a critical part of the
Digital Transformation journey for many customers. autonomous vehicles,
manufacturing, are just some of the key markets being transformed by
AI/ML.
● Customers trying to adopt AI/ML face significant hurdles today.
Proprietary systems are unable to keep up with the rapid evolution of the
AI/ML ecosystem.
● The cost, complexity and vendor lock-in of existing solutions pose
significant challenges for AI/ML implementations at scale.
3
Introduction

4. ● Proprietary AI/ML solutions are unable to keep up with the rapidly evolving AI/ML
ecosystem. Open source and open standards architectures from Red Hat provide
the necessary agility, flexibility and transparency needed to evolve the customer’s
AI/ML environment over time.
● Vendor lock-in also can be expensive and limit choice. Red Hat has a large
ecosystem of partners and systems integrators to help the customers with all the
facets of an end-to-end AI/ML solution. This approach will utilize the customer’s
existing infrastructure and partner integrations. One example is the OpenShift
certification with NVidia GPUs and workload affinity integrations.
● Rapid automation, massive scalability and efficient lifecycle operations with
containers and Kubernetes are the foundations of the Red Hat AI/ML solution.
4
Why Red Hat for AI/ML?

5. 5
AI as Service High Level Architecture

6. 6
End-to-End Reference AI Architecture on Openshift

7. ● The Open Data Hub is a machine-learning-as-a-service platform built on Red
Hat's Kubernetes-based OpenShift Container Platform, Ceph Object Storage,
and Kafka/Strimzi
● integrating a collection of open source projects. It inherits from upstream
efforts such as Kubeflow and is the base of Red Hat's internal data-science
and ML service.
● Data scientists can create models using Jupyter notebooks, and select from
popular tools such as TensorFlow™, scikit-learn, Apache Spark™ and more
for developing models.
● Teams can spend more time solving critical business needs and less on
installing and maintaining infrastructure with the Open Data Hub.
Source : https://opendatahub.io/
7
What is Open Data Hub?

8. ● Ceph is an open source object store that is massively scalable. It can run natively in
OpenShift or as a standalone cluster for optimized performance. Ceph provides a
scalable Ceph Storage Cluster native to Openshift, allowing the distributed storage
of massive data sets as typical of AI/ML workflows.
● Ceph is ideal for storing unstructured data from multiple sources which is also ideal
for large AI/ML dataset ingestions. Ceph provides S3 RESTful API that is widely
supported and is simple to use, making AI/ML data that is stored and transformed
easily accessible.
● Ceph is deployed on OpenShift via Rook (https://rook.io), a storage operator that
provides a user friendly way for deployment and integration of Ceph into the
OpenShift ecosystem.
8
Included Components

9. ● Apache Spark™ operator is an open source operator implementation of Apache
Spark™.
● It is developed as part of the Radanalytics community (https://radanalytics.io/) to
provide distributed Spark cluster workloads on Openshift.
● This implementation creates a Spark cluster with master and worker/executor
processes.
● Distributed parallel execution as provided by Spark clusters are typical and essential
for the success of AI/ML workloads.
9
Included Components

10. JupyterHub (https://jupyter.org/hub) is an open source multi-user notebook platform that ODH provides with
multiple notebook image streams that incorporate embedded features such as Spark libraries and connectors.
JupyterHub provides many features such as multi-user experience for data scientists allowing them to run
notebooks in their own workspaces. Authentication can also be customized as a pluggable component to support
authentication protocols such as OAuth. Data scientists can use familiar tools such as Jupyter notebooks for
developing complex algorithms and models. Frameworks such as numpy, scikit-learn, Tensorflow and more are
available for use.
Prometheus (https://prometheus.io/) is an open source monitoring and alerting tool that is widely adopted across
many enterprises. Prometheus can be configured to monitor targets by scraping or pulling metrics from the
target’s HTTP endpoint and storing the metric name and a set of key-value pairs in a time series database. For
graphing or querying this data, Prometheus provides a web portal with rudimentary options to list and graph the
data. It also provides an endpoint for more powerful visualization tools such as Grafana to query the data and
create graphs. An Alert Manager is also available to create alert rules to produce alerts on specific metric
conditions.
Grafana (https://grafana.com/) is an open source tool for data visualization and monitoring. Data sources such as
Prometheus can be added to Grafana for metrics collection. Users create Dashboards that include
comprehensive graphs or plots of specific metrics. It includes powerful visualization capabilities for graphs, tables,
and heatmaps. Ready-made dashboards for different data types and sources are also available giving Grafana
users a head start. It also has support for a wide variety of plugins so that users can incorporate
community-powered visualisation tools for things such as scatter plots or pie charts.
10
Included Components

11. 11
AI WorkFlow

12. Reproducibility
A fundamental concern for many AI/ML use cases is reproducibility. This
implies both that results are reproducible and that the environments used to
produce these results are reproducible.
Reproducible application environments and reproducible application
deployments are a core feature of OpenShift, and the same functionality that
provides this capability to general distributed applications can also enable
reproducible machine learning models, pipelines, systems, and applications.
12
Why OpenShift?

13. Security, access control, and isolation
An equally important set of concerns for machine learning systems is related to security. Machine learning
systems often deal with sensitive or valuable data (including confidential data,
OpenShift: the namespace mechanism provides lightweight isolation between distinct applications; internal
service routing means that model services deployed within the same namespace as an application are not
accessible to the outside world by default; secret management makes it possible for components to securely
store credentials for sensitive data sources; and namespace isolation, quotas, and scheduling policies combine
to keep misbehaving components from impacting others.
For deployments that require exposing model services beyond the scope of an application in OpenShift, the
3Scale API gateway and RH-SSO from Red Hat’s application development portfolio provide powerful tools to
authenticate, authorize, meter, and gate access to these services.
13
Why OpenShift?

14. Elasticity, scale-out, and federation
An essential part of systems designed for the cloud, and for the contemporary
hybrid cloud in particular, is that their components scale elastically to exploit
available resources and meet shifting demand, potentially even scaling or
migrating across multiple clouds (e.g., some combination of internal clouds and
distinct public cloud providers).
These capabilities are provided by fundamental functionality in OpenShift, which is
an abstraction layer for the hybrid cloud: scaling application components (whether
on-demand or in response to application metrics), migrating stateless and stateful
services, and scheduling applications against resources federated from multiple
clouds.
14
Why OpenShift?

15. Flexible scheduling of heterogeneous resources
Machine learning systems don’t just require an advanced scheduler for basic
applications that run on identical commodity hardware: they may also benefit from the
ability to schedule particular tasks where they can take advantage of hardware
accelerators like GPUs.
OpenShift is flexible enough both to schedule conventional application components as
well as specialized compute workloads, including those that require close coupling
between parallel tasks, guaranteed memory or network bandwidth, or access to
accelerator hardware.
15
Why OpenShift?

16. 16
Project Road Map for 2019

17. Fraud Detection Using OpenDataHub on OpenShift
https://www.youtube.com/watch?v=IcQ2bhsw_kQ
17
Use Cases

18. The Open Data Hub operator deploys and manages various components using the
Operator SDK in the Operator Framework.
There are two options to deploy ODH operator: Manual and using Operator Lifecycle
Manager (OLM). Both require Openshift 3.11 or 4.0 and an installation of Ceph using
Rook operator.
The latest version of the Open Data Hub operator project is located here:
https://gitlab.com/opendatahub/opendatahub-operator
The latest version of the Open Data Hub operator image is located here:
https://quay.io/opendatahub/opendatahub-operator
18
Installation

19. 19
DEMO

20. Cloud Solution Blueprint for AI/ML
Reference Architecture
Samson Wick
William Benton
Neeraj Kuppam
20
Credits

21. linkedincom/company/red-hat
youtubecom/user/RedHatVideos
facebookcom/redhatinc
twittercom/RedHat
21
Red Hat is the world’s leading provider of enterprise
open source software solutions Award-winning support,
training, and consulting services make Red Hat a trusted
adviser to the Fortune 500
Thank you
OPTIONALSECTIONMARKERORTITLE