The document discusses semi-supervised learning, which combines labeled and unlabeled data to enhance learning efficiency while minimizing expensive labeling costs. It outlines the key assumptions of semi-supervised learning, including continuity, cluster, and manifold assumptions, and compares the method to human learning processes. The document aims to provide insights into leveraging the strengths of both supervised and unsupervised learning for better model training.

1. Semi Supervised Learning
Devansh

2. Context
Supervised learning is amazing. It has been used to solve a whole bunch of
problems.
It is also very intuitive when it comes to the idea. With lots of data and a directed
learning mechanism, it makes sense that our agents should be able to learn
specific things.
Labeling/annotating data can get extremely expensive. For eg. medical images
are typically very expensive to annotate (check out the video on SinGAN-Seg for
more details).
There is a lot of unlabeled data that is lying around, which can be used for cheap.

3. Enter Semi-Supervised Learning (Prequel)
Don’t confuse this with Self-Supervised Learning (I make that mistake a lot.)
How can we use the simpler concept and higher performance of Supervised
Learning with the cheap, large scale data availability of using Unsupervised
Learning?
That’s a good question. How would you try to use both processes to leverage the
strengths of both types (pause the video and think for a second)

4. An Image to help you think

5. SSL: Overview
We take a (relatively) small amount of labelled
data. Use that to learn.
Then take the unlabeled data use your model
to figure out the (pseudo) labels. Use that to
train our models further.
(Optional)
Rinse and Repeat. Do some other things like
add noise if you have the resources/want to
get creative

6. Transduction vs Induction
Often you will see learning presented as
either transductive or inductive learning.
Transductive learning: I give you specific
examples and ask you to predict specific
examples.
Inductive learning: I give you examples, and
you figure out a pattern. Use that pattern to
predict samples.
Sound similar? Think back to the video on
discriminative vs generative classification.

7. Assumptions used in Semi Supervised Learning
Take a second to pause. Think about the kinds of assumptions you make when
implementing SSL. This is a good habit when dealing with ML/Data.
Understanding your implicit assumptions gives you a way to figure out how to
improve.
SSL algorithms have 3 assumptions. To use SSL we need to make atleast one:
● Continuity assumption
● Cluster assumption
● Manifold assumption

8. Assumptions- Continuity
Wikipedia: Points that are close to each other are more likely to share a label. This
is also generally assumed in supervised learning and yields a preference for
geometrically simple decision boundaries. In the case of semi-supervised learning,
the smoothness assumption additionally yields a preference for decision
boundaries in low-density regions, so few points are close to each other but in
different classes.

9. Assumptions: Cluster
Wikipedia: The data tend to form discrete clusters, and points in the same cluster
are more likely to share a label (although data that shares a label may spread
across multiple clusters).
Gives rise to feature learning with clustering algorithms.

10. Assumptions: Manifold
The data lie approximately on a manifold of much lower dimension than the input
space. Learning the manifold using both the labeled and unlabeled data can avoid
the curse of dimensionality. Then learning can proceed using distances and
densities defined on the manifold.

11. SSL-> Most human way to learn?
Semi Supervised Learning is often compared to how we humans learn.
Think back to your favorite skills. How did you learn to do them? The SSL formula:
1) Have the learner train through specific guided examples with lots of detail.
2) As the learner becomes more proficient, we start using mixing in samples that
aren’t fully worked out. Learner must work through them on their own.
3) If we really want to check proficiency of the learner, we throw in
incorrect/broken examples, and have the student correct the mistakes.
Let’s see how this squares with our learning process. Let’s take 3 skills I learnt:
Math, French, and Brazilian JiuJitsu.

12. Reach out to me
Check out my other articles on Medium. : https://machine-learning-made-
simple.medium.com/
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn: https://www.linkedin.com/in/devansh-devansh-516004168/
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
My Substack: https://devanshacc.substack.com/
Live conversations at twitch here: https://rb.gy/zlhk9y
Get a free stock on Robinhood: https://join.robinhood.com/fnud75