Azure ML Deployment using ACI

1. Azure ML Deployment using ACI

2. Components – Directory Structure
1. echo_score.py has py
extension while train.ipynb is
a notebook file
2. Data is loaded manually into
data folder. However, the
right way is to first create a
data store and then data
asset. Post that, we can use
that data asset in the code.
3. Train.ipynb has to run on
Python 3.10 SDK V2 since
there were issues installing
sklearn lib in earlier SDK
versions.

3. Train File – Define Workspace
# Define workspace variables
from azureml.core import Workspace, Dataset
import numpy as np
import pandas as pd
secret_subscription_id_value = ‘XXXXXXXXXXX'
resource_group = ‘XXXXXXXXXXX'
workspace_name = ‘XXXXXXXXXXX'
workspace = Workspace(secret_subscription_id_value, resource_group, workspace_name)
1. Id values can be stored in key vault. However, that code did not work.

4. Train File – Save Model Artifacts
# Save the model and its artifacts
import joblib
joblib.dump(kmeans, "../model_artifacts/kmeans.joblib")
joblib.dump(enc, "../model_artifacts/enc.bin", compress=True)
joblib.dump(model_rf, "../model_artifacts/taxi_demand_prediction.joblib")
1. At the end of training, you store the model artifacts into Azure ML compute.
This is referred in the scoring file.

5. Train File – Register Model
# Register model
from azureml.core.model import Model
import urllib.request
model = Model.register(workspace,
model_name="taxi_demand_prediction",
model_path="../model_artifacts/")
1. When you register the model,
all the artifacts stored in Azure
ML compute instance are
registered as a folder in the
model registry (view on the
right)

6. Train File – Environment Setup
# Environment setup
from azureml.core import Environment
from azureml.core.model import InferenceConfig
env = Environment(name="taxi_demand_prediction")
python_packages = ['azure-ml-api-sdk','numpy', 'pandas', 'seaborn', 'matplotlib',
'scipy', 'scikit-learn', 'joblib','requests']
for package in python_packages:
env.python.conda_dependencies.add_pip_package(package)
1. While setting up environment for inference, the first lib “'azure-ml-api-sdk”
required to be installed as well.

7. Train File – Inference Config
# Inference configuration setup
inference_config = InferenceConfig(
environment=env,
source_directory="../source_dir",
entry_script="echo_score.py",
)

8. Train File – Local Deployment
# Local Deployment
from azureml.core.webservice import LocalWebservice
from azureml.core.webservice import AciWebservice
deployment_config = LocalWebservice.deploy_configuration(port=6789)
service = Model.deploy(
workspace,
"taxidemandprediction",
[model],
inference_config,
deployment_config,
overwrite=True,
)
service.wait_for_deployment(show_output=True)
print(service.get_logs())
1. There should be no underscore in
the deployment name. Hence, we
the name is “taxidemandprediction”

9. Train File – Local Testing
# Local Testing
import requests
import json
uri = service.scoring_uri
requests.get("http://localhost:6789")
headers = {"Content-Type": "application/json"}
data = {"pickup_latitude":"-
73.980492","pickup_longitude":"40.777981","tpep_pickup_datetime":"2020-02-13
23:40:00"}
data = json.dumps(data)
response = requests.post(uri, data=data, headers=headers)
print(response.json())
service.get_logs() # Get logs

10. Train File – Remote ACI Deployment
# Remote Deployment
deployment_config = AciWebservice.deploy_configuration(
cpu_cores=0.5, memory_gb=1, auth_enabled=True
)
service = Model.deploy(
workspace,
"taxidemandprediction",
[model],
inference_config,
deployment_config,
overwrite=True,
)
service.wait_for_deployment(show_output=True)
print(service.get_logs())
1. ACI is a serverless setup made by Azure
for low cost real time deployments.
Alternatives are Kubernetes.
2. For batch inferences, we can use azure
compute itself.
3. We can also do “bring your own
container” and only use Azure for
deployment.

11. Train File – ACI Remote Testing
import requests
import json
from azureml.core import Webservice
service = Webservice(workspace=workspace, name="taxidemandprediction")
scoring_uri = service.scoring_uri
# If the service is authenticated, set the key or token
key, _ = service.get_keys()
# Set the appropriate headers
headers = {"Content-Type": "application/json"}
headers["Authorization"] = f"Bearer {key}"
# Make the request and display the response and logs
data = {"pickup_latitude":"-
73.980492","pickup_longitude":"40.777981","tpep_pickup_datetime":"2020-02-13 23:40:00"}
data = json.dumps(data)
resp = requests.post(scoring_uri, data=data, headers=headers)
print(resp.text)

12. Other issues faced
1. The model path given in score file refers to the path in model
registry. The path added was
“taxi_demand_prediction/model_artifacts/kmeans.joblib” before
and we changed it to “model_artifacts/kmeans.joblib”. Similar
changes were done to other paths in score file.
2. Always make sure to do local testing on compute and then deploy on
ACI. The logs are detailed during local testing and it is easier to
debug.

13. Key views - Models

14. Key views – Endpoints
1. End point also
provides a Rest API
for the deployment
that can be queried
from outside Azure
setup.

15. Key views – Compute used for Training