Human-in-the-loop (HITL) machine learning integrates human and machine intelligence to enhance algorithm training and testing, allowing for continuous improvement through human feedback. It involves various data labeling techniques to develop AI models for applications like natural language processing and computer vision. The HITL approach is evolving towards automation while ensuring human oversight, significantly impacting business workflows in the future.

1. Human-in-the-loop in
Machine Learning

2. Uncertain
Confident
Active Learning
Output
AI Classifier
Human Annotation
Human-in-the-loop Machine Learning
2

3. What is Human-in-the-loop?
▪ Human in the loop machine learning is a process that leverages both human and
machine intelligence to create machine learning-based AImodels.
▪ When a system is unable to provide a solution or solve a problem, human
intervention is needed in the training and testing phases of an algorithm
development.
▪ This creates a recurring feedback loop where every iteration of algorithm provides
better solutions.Humans can also modify the algorithm to improve accuracy.
3

4. What is HITL Machine Learning?
Combination of human & machine
intelligence.
Incorporate human feedback into
learning loop of machines.
Humans are involved in training,
tuning and testing of algorithms.
80% of the time, the algorithm is left
alone with the human involvement
limited to 19% and remaining 1% left
to randomness.
4

5. Human-in-the-loop AI
Increase efficiency of every human expert that work
with the AIsystem
Increase the level of quality by collecting and sharing
data and data-driven insights
Capture silent knowledge and expertise of human
experts in a single system
Human expert confirms,
rejects or labels output
AI model
generates output
Model learns from
human and accuracy
constantly improves
5

6. Combining Machine Learning with
Human Contributions
Crowdsourcing of subjective
parameters.
▪ Contextualize data.
▪ Use user expertise in identifying
and recognizing patterns.
▪ Process data that is difficult
to process automatically.
▪ Ask users to improvealgorithms.
▪ Human input to improve machine inference.
▪ Use machine inference to monitor and
improve user performance.
6

7. Types of Data Labelling
The data labelling can be categorized into different types such as
▪ Semantic segmentation to train visual perception annotation-based model
▪ Landmark annotation to train facial recognition
▪ NLPannotation, text annotation and sentiment analysis to train language
or voice recognition model.
▪ The data are annotated and labelled to develop AI devices that communicates
with humans.
▪ HITLcan create different types of training data sets for different types of machine
learning models built for different fields.
7

8. Object & activity
detection
Person
tracking
Face
recognition
Real - time live
stream
Content
moderation
Celebrity
recognition
Video
Analysis
8

9. Image Annotation for Computer Vision
9

10. Typical Workflow
for a Machine
Learning Model Deploy
the model
Generate
example
data
Train a model
Clean
Fetch
Prepare
Train
modelEvaluate
model
Deploy to
production
Monitor/
Collect data/
Evaluate
10

11. Lifecycle of HITL Machine Learning
▪ Inception - A manual workflowgets created.
▪ Iteration - The workflow is improvediteratively.
▪ Transition - The workflow transitions from a complete manual to semi-automated.
▪ Monitoring - Complete automation shifts HITLinto a validation mechanism.
11

12. Inception
▪ Here, we follow a "Divide and Conquer" approach where we categorize the process
into smaller components and train the machine learning models accordingly.
▪ The goal of the Machine learning model is defined and conceptualized. We fetch the
raw unstructured data, clean, label the same and prepare for the next stage.
12

13. Iteration
▪ We increase the efficiency, time cost, quality and accuracy of the machine learning
model to achieve better outcomes and obtain higher quality data.
▪ We train and evaluate the model to determine its efficiency in an iterative way.
▪ The iterative process on the workflow components benefits us to structure our
machine learning modelefficiently.
13

14. Transition
▪ We deploy the machine learning model to production once desired human efficiency
and quality levels are achieved.
▪ Combining these automated mechanisms with the human component creates the
actual Human-in-the-Loop machine learning system.
▪ The transition adopts into the following.
▪ Fully manual - The human receives the automated system’s decision along with the
input data, which can be used to evaluate the model’s performance before it can
independently act.
14

15. Types of Transition
▪ Augmented manual - The human is augmented by the automation, acts as a
gate-keeper for automated decisions, approving or declining — and correcting —
those automated decisions.
▪ Semi-automated - A fraction of the automation’s decisions stop being monitored by
a human. The rest remains under the Augmented manual model.
▪ Automated - The model reaches a desired accuracy level and humans stop being
involved in the loop.
15

16. Monitoring
▪ HITL is never fully trust what is automated. Even at the Automated phase, there
occurs the touch of humans to verify that the automated processes acts as
intended.
▪ Once the monitoring process is operationalized, we must ensure that it is precise and
accurate to avoid the potential risks ahead.
16

17. Who uses Human-in-the-loop
Machine Learning?
▪ HITL can be used for manifold AI projects including NLP, computer vision,
sentiment analysis, transcription, and a vast amount of other use cases.
▪ Any deep learning AI can benefit from human intelligence involved into the
loop at some point.
17

18. Logistics
▪ Determine and predict demand.
▪ Patterns of the route.
▪ Characterize congestion areas.
Retail management
▪ Why do users buy what they buy?
▪ Onset of fashions
▪ Products development
▪ Targeted advertisement and influence
Application Areas
18

19. 19

20. Out ofstock
Out ofstock
Out ofstock
No visible label present
cannot box as an out - of -
stock item.
20

21. Business Address Order Number
21

22. Conclusion
▪ HITL will significantly change the way business workflows are carried out in future
by creating a pipeline that includes data collection, model training, testing,
deployment andmaintenance.
▪ This is a very excitingtime to be involved in this field as industry and institutions
are pushing the limits furtherevery day.
22

23. hello@mitosistech.com
www.mitosistech.com
IND: +91-78240 35173
US: +1-(415) 251-2064