Before IT leaders and their teams begin to dig into the nitty-gritty technical aspects of containerizing AI/ML workloads, some principles are worth thinking about up front. Here are six fact to consider. today’s big IT trends, AI/ML and containers, have become part of the same conversation at many organisations. They’re increasingly paired together, as teams look for better ways to manage their AI and ML workloads – enabled by a growing menu of commercial and open source technologies for doing so. The best news for IT leaders is that tooling and processes for running machine learning at scale in containers has improved significantly over the past few years,” says Blair Hanley Frank, enterprise technology analyst at ISG. “There is no shortage of available open source tooling, commercial products, and tutorials to help data scientists and IT teams get these systems up and running. https://wecode-inc.com/service/next-generation-technology.html

1. 2022: AI/ML Workloads in Containers: 6 Key Facts

2. Introduction:
Before IT leaders and their teams begin to dig into the nitty-gritty
technical aspects of containerizing AI/ML workloads, some principles are
worth thinking about up front. Here are six essentials to consider.

3. Table of Content
● AI/ML workloads represent workflows
● The benefits are similar to other containerized workloads
● Teams need to be aligned
● The "pay attention" points don’t really change
● Containers won’t fix all underlying issues
● Be smart about build vs. buy

4. AI/ML Workloads Represent
workflows
1.

5. AI/ML Workloads Represent Workflows
“Data gets gathered, cleaned, and processed,” Haff says. Then, the work continues: “Now it’s time to train
a model, tuning parameters based on a set of training data. After model training, the next step of the
workflow is [deploying to] production. Finally, data scientists need to monitor the performance of models in
production, tracking prediction, and performance metrics.”
“Traditionally, this workflow might have involved two or three handoffs to different individuals using
different environments,” Haff says. “However, a container platform-based workflow enables the sort of
self-service that increasingly allows data scientists to take responsibility for both developing models and
integrating into applications.”

6. 2.
The Benefits are Similar to
Other Containerized
Workloads

7. The benefits are similar to other containerized
workloads
Nauman Mustafa, head of AI & ML at Autify, sees three overarching benefits of containerization in the
context of AI/ML workflows:
● Modularity: It makes important components of the workflow – such as model training and deployment –
more modular. This is similar to how containerization can enable more modular architectures, namely
microservices, in the broader world of software development.
● Speed: Containerization “accelerates the development/deployment and release cycle,” Mustafa says.
(We’ll get back to speed in a moment.)
● People management: Containerization also makes it “[easier] to manage teams by reducing cross-
team dependencies,” Mustafa says. As in other IT arenas, containerization can help cut down on the
“hand off and forget” mindset as work moves from one functional group to another.

8. Teams Need to be Aligned
3.

9. Teams Need to be Aligned
“Make sure everyone involved in building and operating machine learning workloads in a containerized
environment is on the same page,” says Frank from ISG. “Operations engineers may be familiar with
running Kubernetes, but may not understand the specific needs of data science workloads. At the same
time, data scientists are familiar with the process of building and deploying machine learning models, but
may require additional help when moving them to containers or operating them going forward.”
“In a world where repeatability of results is critical, organizations can use containers to democratize access to
AI/ML technology and allow data scientists to share and replicate experiments with ease, all while being
compliant with the latest IT and InfoSec standards,” says Sherard Griffin, director of global software
engineering at Red Hat.

10. The "Pay Attention"
Points Don’t Really
Change
4.

11. The "Pay Attention" Points Don’t Really Change
Here are three examples of operational requirements that you’ll need to pay attention to, just like with
other containerized applications:
● Resource allocation: Mustafa notes that proper resource allocation remains critical to optimizing
cost and performance over time. Provision too much and you’re wasting resources (and money)
over time; too little and you’re setting yourself up for performance problems.
● Observability: Just because you can’t see a problem does not render it out of existence. “Ensure
that you have the necessary observability software in place to understand how your multi-container
applications behave,” Frank says.
● Security: “From a security point of view, launching AI/ML solutions is no different from launching
other solutions in containers,” Alexandra Murzina, ML engineer at Positive Technologies. That
means tactics such as applying the principle of least privilege (both to people and the containers
themselves), using only trusted, verified container images, runtime vulnerability scanning, and other
security layers should remain top of mind.

12. Containers Won’t Fix all
Underlying Issues
5.

13. Containers Won’t Fix all Underlying Issues
Just as automation won’t improve a flawed process (it just helps that flawed process run faster and more
frequently), containerization is not going to address fundamental problems with your AI/ML workloads.
If you’re baking bias into your ML models, for example, running them in containers will do nothing to
address that potentially serious issue.
“Containers are very beneficial for running AI/ML workloads,” says Raghu Kishore Vempati, director of
technology at Capgemini Engineering. “[But] containerizing AI/ML workloads alone doesn’t make the model
more efficient. It only provides a way to accelerate the productivity associated with training the models and
inferring on them.”

14. Be Smart About Build vs.
Buy
6.

15. Be Smart About Build vs. Buy
As with most technical choices, there’s a “should we or shouldn’t we?” decision in terms of containerizing
AI/ML workloads. Also like most important technical choices, nothing comes free.
“There is a cost associated with containerizing machine learning workflows, which may not be justified for tiny
teams, but for large teams, benefits outweigh the cost,” Mustafa from Autifly says.
IT leaders and their teams should do it with clear goals or reasons in mind – “just because we can” shouldn't
be the only reason on your list.
“Don’t overcomplicate an already complex situation,” Frank says. “Make sure that containerizing ML
workloads will provide business value beyond the intellectual exercise.”
Source: enterprisersproject

16. Next-Gen Tech Services
Mobile Application Development
Cloud Computing Services
Quality Assurance
Digital Marketing
Visit: www.wecode-inc.com
Email: sales@wecode-inc.com