The document discusses the concept of 'machine unlearning', outlining its importance in privacy and usability while challenging the traditional method of retraining machine learning models. It addresses the technical requirements for effective unlearning, the types of data removal requests, and verification methods to ensure the accuracy of unlearned models. Furthermore, it highlights the difficulties posed by stochasticity, incrementality, and performance degradation in developing unlearning algorithms.

1. How can a learnt model unlearn something
Framework for "Machine Unlearning"
Saradindu Sengupta
Senior ML Engineer @Nunam
Where I work on building learning systems to forecast health and failure of Li-ion batteries.
PyData Global 2023

2. Overview
1. A brief overview of what is unlearning and why not just retrain.
a. Why we came her - Applications for machine unlearning
b. A brief overview of previous research work
2. Challenges to be encountered for an unlearning algorithm
a. Stochasticity
b. Incrementality
c. Degradation
3. Unlearning framework
a. Technical Design requirements to be met
4. Type of removal requests to be handled
a. Feature-wise
b. Item-wise
c. Class-wise
5. Verification
a. How to define evaluation metric for the unlearning algorithm
b. How to define if the influence of forgetting the dataset is truly removed

3. Why we came here
Privacy
Security Usability
● Facebook Privacy Policy Change
● iCloud photo hacking
● “Right to be forgotten” regulations
stipulates “individuals have the
right to be forgotten”.
● Polluted training data would pollute
model outcome
● Polygraph, a worm detection
program conclusively demonstrated
that. [Perdisci, Dagon, and et.al,
“Misleading worm signature
generators using deliberate noise
injection”]
● Recommendation engine
● Netflix account sharing would
pollute the content
recommendation
Why not just retrain ?

4. Why not retrain ?
Training cost for ResNet-50 decreased by 38% overall [Google Cloud Cost] due to hardware optimization and parallelism
but total cost have increased significantly.
[1] COUNTING THE COST OF TRAINING LARGE LANGUAGE MODELS
[2] (Sharir et al., 2020)
Model
Params
(Billions) Token
Days to
train
Price to
Train
Cost per 1M
params
GPT-3XL 1.3 26 0.4 2,500 1.92
GPT-J 6 120 8 45000 7.5
GPT-3 6.7B 6.7 134 11 40000 5.97
T-5 11B 11 34 9 60000 5.45
GPT-3 13B 13 260 39 150000 11.54
GPT-3 NeoX 20 400 47 525000 26.25
GPT 70B 70 1400 85 2500000 35.71
GPT 175B 175 3500 110.5 8750000 50

5. Research Space - How we came here
[Y. Cao and J. Yang, 2015]
● Introduced the term ‘machine unlearning’
● Provided deterministic algorithm for
unlearning
[A. Ginart and et al. , 2019]
Introduced probabilistic unlearning
inspired from differential privacy
[(Guo et al., 2020] [Izzo et al., 2021] [Neel et al., 2021] [Ullah et al., 2021]
Provided theoretical error boundness to probabilistic unlearning
[Cauwenberghs and Poggio, 2001] [Tveit et al., 2003]
Introduced decremental learning
[Du et al., 2019] [Golatkar et al.,2020b,a]
[Nguyen et al., 2020]
Introduced unlearning for deep
learning

6. Challenges for an efficient unlearning algorithm
1. Stochasticity
a. The stochasticity of the training process makes identifying a single data point that influences
weight very difficult
2. Incrementality
a. The nature of incrementality in training , where a single instance of data influences the
subsequent instances and it is itself influenced by previous samples, makes the process of
removing influence tricky
3. Catastrophic Unlearning
a. While removing influences of subset of data, the nature of degradation of its performance can be
exponential which makes the process hard to quantify.

7. Unlearning Framework
Training Dataset
Pre-trained
Model Forget Dataset
Unlearning Model
Unlearned Model
Training Dataset
Evaluation Metric
Retraining without forget dataset
Check how good the unlearning algorithm
is compared to retrained model

8. Unlearning Framework: Design Requirements
1. Completeness
a. An unlearned model should be making same prediction as a retrained model on incoming
samples
b. Metrics can be derived from adversarial attacks
2. Timeliness
a. A retrained and an unlearnt model should work same but the first process taking more time
than later
b. This metric is a compromise with completeness as a model retrained will have better
accuracy, although might be negligible but time to retrain would be costly
3. Verifiability
a. An unlearnt model should provide mechanism to check the effects the unlearning request
b. To that end, backdoor attacks can be useful

9. Type of removal requests
1. Item-wise
a. Remove certain items/samples from training data
2. Feature-wise
a. Unlearning at the feature level
b. When misclassified samples leaks error for specific features
3. Class-wise
a. Unlearning at class level
b. It can be implicit in many scenarios

10. Verification and Evaluation Metrics
Unlearning verification and evaluation metrics or tests overlap in some areas but inherently while the
first is used for model optimization the later is used for model evaluation. Some of the used unlearning
model verification tests are mentioned below.
Verification
1. Feature Injection test
2. Forgetting Measuring
3. Information Leakage
4. Membership Inference Attack
5. Backdoor Attack
6. Slowdown Attack
7. Interclass Confusion Test
8. Federated Verification
9. Cryptographic Protocol
Evaluation Metric
1. Accuracy
2. Completeness
3. Unlearn Time
4. Relearn Time
5. Layer-wise Distance
6. Activation Distance
7. JS Divergence
8. Membership Inference Attack
9. ZRF Score
10. Model Inversion Attack

11. Verification
2 of the most commonly references
Feature Injection test
Membership Inference Attack
Kong, Z., & Alfeld, S. (2022). Approximate Data
Deletion in Generative Models. ArXiv.
/abs/2206.14439
Shokri, R., Stronati, M., Song, C., & Shmatikov, V.
(2016). Membership Inference Attacks against
Machine Learning Models. ArXiv. /abs/1610.05820

12. Evaluation Metric
Layer-wise Distance
Unlearn Time
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli,
M. (2021). Fast Yet Effective Machine Unlearning. ArXiv.
https://doi.org/10.1109/TNNLS.2023.3266233
Y Cao, J Yang(2015). Towards Making Systems Forget with
Machine Unlearning

13. Thank You
/in/saradindusengupta
@iamsaradindu /saradindusengupta