ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj

Agile Data Science &
Machine Learning
Operations

Introduction
Key takeaways
Q & A
Agenda
© 2024 Wavicle Data Solutions 2

Agile transformations – my view!
4
© 2024 Wavicle Data
Solutions
• Agile is awesome!
• Agile processes provide better visibility to customer stakeholders
• There’s a framework for all needs
• Better suitable for distributed teams
• Removes a lot of boilerplate

Classic Agile Framework
5
© 2024 Wavicle Data Solutions
Product Backlog Sprint Backlog
Daily Scrum
Meeting
24 hours
2 to 4 weeks
Sprint
Retrospective
Potentially Shippable
Product Increment

Classic Agile Framework
6
Daily Scrum
Meeting
24 hours
2 to 4
weeks
Sprint
Retrospective
Product Increment
Scrum Of
Scrum
(SoS)
Daily Scrum
Meeting
24 hours
2 to 4
weeks
Sprint
Retrospective
Product Increment
Daily Scrum
Meeting
24 hours
2 to 4
weeks
Sprint
Retrospective
Product Increment
Daily Scrum
Meeting
24 hours
2 to 4
weeks
Sprint
Retrospective
Product Increment
Scrum
Team1
Scrum
Team2
Scrum
Team4
Scrum
Team3

Benefits
Relevant Output Better Time-to-Market Real Feedback Cut Losses Early
7
By defining requirements
just before development
(as opposed to all
upfront in a project), the
features are more likely
to meet the most current
needs. The stakeholders’
initial requests often do
not map to their real
needs. Agile practices
help you discover the
true needs earlier in the
process.
By delivering incremental
product features such as
exploratory data reports,
dashboards, or Minimal
Viable Models, users gain
value before the project’s
end.
By soliciting feedback on
the functional product,
the data scientists can
more accurately assess
whether their deliverables
work “in the real-world”.
Meanwhile, the product
manager can assess
whether the deliverables
provide the intended
business value.
No matter what you do,
some data science
projects will not succeed.
The sooner you get
feedback that you are
headed to a dead end, the
sooner you can pivot to
related objectives or a
completely new project.

5
2
1 4
3
The verdict
Challenges
Why Data Science? Agile DS Solution
The preparation

1. Why Data Science?
1
0
Solutions
• Data Science is a study of data to gain insights.
• Data types – raw, structured, unstructured etc
• Use of scientific methods and algorithms
• Data is useless without Data science

11
Solutions
1
1

2. Is Agile for Data Science?
• Yes! (In most cases)
1
2
Solutions
• Data Science is naturally non-linear and fits well into the
Agile Framework. It does tend to lack clear upfront
understanding; therefore, the flexibility is required to
pivot based on findings.
• There are some variations in expectations and process
including:
- More ambiguity in planning due to exploratory nature of
data
- Not every iteration will deliver an expected result
- Fully staffed team must include Scrum Master, Data
Analyst, Business Analyst, Data Scientist, and ML
Engineer to be successful

3. Preparing for Agile
Data Science

Required Roles
14
Role Definition Allocation
Business Analyst
The business analyst (BA) is responsible for documenting all the business interviews and technology interactions
as they relate to use case development and business expectations. The BA is responsible for documenting the
use cases.
50%
Scrum Master
The scrum Master is responsible for supporting the Agile environment by holding all Agile ceremonies,
developing the feature and sprint backlogs, and helping the team remove blockers.
50%
Data Analyst
The data analyst (DA) is responsible for the exploration and understanding of the data sets. They produce
artifacts like data dictionaries and data visualizations to help determine importance or bias of variables.
100%
Product Owner
The product owner (PO) is responsible for the deliverables required and has knowledge of the business need.
They will act as the main point for escalation as needed. They are responsible for reviewing project status,
helping clear roadblocks identified, and providing feedback on outputs.
50%
Business
Stakeholders and
SMEs
The business stakeholders and SMEs are members of the business affected by/and have knowledge of the need.
They are responsible for working with the development team to create and prioritize use cases, as well as
provide definition around the data and the value derived from outputs.
25%
Data Scientist
The data scientist (DSc) is responsible for performing exploratory data analysis, determining the list of features,
and building/testing their models.
100%
ML Ops Engineer
The ML Ops engineer is responsible for all ML Operations such as deploying machine learning models to
production with appropriate governance, monitoring, and CI/CD.
100%

Data Value Pyramid
Action!
Gain Insight
Explore & Visualize
Validate, Define, Organize
Identify & Access Raw Data
The World of Data!
Developing policy and making decisions based on generated insights
Action
1
2
3
4
Teams need to determine what is the data relevant to the industry/study and
how to ingest into internal systems for analysis
Identify & Access Raw Data
5
Data Science, Machine Learning, Generative AI processes to gain business
insights
Gain Insight
Determine how the data is trending and what picture is it making
Explore & Visualize
This includes defining fields, organizing into schema, and applying Feature
Stores in the case for Data Science
Validate, Define, Organize

ML Workflow Decomposition
1
6
Solutions

MLOps
1
7
Solutions

Gotchas in Agile Data Science
• Watch out for the following and try to avoid:
1
8
1. Misunderstanding of the process and lack of buy-in buy stakeholders and
team members
2. Applying software specific framework expectations. This can inhibit the
exploratory nature of data science
3. Expecting hard deadlines. Due to the R&D nature of data science, hard
deadlines and inflexible management may derail the value of the effort.
4. Limiting exploratory activity. Data science research often requires longer
time horizons that are difficult to know during the planning phase.
5. Improper staffing of pods. In most cases, each pod requires a business
analyst, data analyst, data scientist, and machine learning engineer in
order to successfully complete all tasks in the process.

Cross Industry Standard
Process for the
development of ML
Applications with
Quality Assurance
4. The Solution - CRISP-ML(Q)
• MLOps Frameworks

• Define business
objectives
• Translate business
objectives into ML
objectives
• Collect and verify
data
• Assess the project
feasibility
• Create POC
CRISP-ML(Q) Phase
•
• Business & Data
• Feature Selection
• Data Selection
• Class Balancing
• Cleaning Data
• Feature Engineering
• Data Augmentation
• Data Standardization
•
• Data Engineering
• Define quality
measure of the
model
• ML Algorithm
selection
• Domain Knowledge
• Model Training
• Transfer Learning
• Model Compression
• Ensemble Learning
• Model
Documentation
•
• ML Model Engineering
• Validate model’s
performance
• Determine
robustness
• Increase model’s
explain ability
• Model deployment
• Model evaluation
documentation
•
• ML Model Evaluation

CRISP-ML(Q) Phase
Model Deployment Model Monitoring and Maintenance
2
Evaluate model under production condition
User acceptance and usability
Model governance
Deploy according to the selected strategy (A/B
Testing, Multi-armed bandits)
Monitor the efficiency and efficacy of the model
prediction serving
Compare to the previously specified success criteria
(thresholds)
Retrain model if required
Collect new data
Perform labelling of the new data points
Repeat tasks from the Model Engineering
and Model Evaluation phases
Continuous, integration, training and
deployment of the model

• Venkatesa Prasannaa Selvaraj
• Director – Data Analytics
• Venkatesa.prasannaa@wavicledata.com
Thank you!

ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj

Recommended

Recommended

More Related Content

Similar to ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj

Similar to ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj (20)

More from AgileNetwork

More from AgileNetwork (20)

Recently uploaded

Recently uploaded (20)

ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj

Editor's Notes