I will start by describing privacy risk in deep learning, in particular memorization of inputs during network training and attacks that expose these memorized inputs. I will then talk about methods to mitigate memorization, and increase privacy, introducing the concept of differential privacy as a measure for exposure risk, and the idea of using generative networks to create synthetic data to protect the original training data. Finally, I will remark on benefits in generalization that come as a side effect of privacy enhancing deep learning methods.

1. PRIVACY AND DEEP LEARNING -
FRIENDS OR FOES?
KLAUDIUS KALCHER

2. Privacy and AI - Topics

3. Private Data in Deep Learning Models
Private Data Private Data?Private Data?

4. Generative AI
cat: 82%
dog: 18%
AI Inference
Generative AI

5. Synthetic Shakespeare
Generative AI for Synthetic Text

6. Application for Text Generation
Gmail Smart Compose

7. Generative AI for Images

8. Originality of GAN-generated Images

9. Originality of GAN-generated Faces
Generated Faces
5 Nearest Neighbors
From Training Set

10. However...
Progressive Growing of GANs for Improved Quality, Stability, and Variation, Karras et al. (2017/2018)

11. A case of Overfitting?
Training loss decreases
Validation loss starts to increase
Best model in terms of
generalization

12. Attack Scenario: Attribute Inference
Q: given some attributes of individual A in the training dataset -
what are the others?
Age Date of Birth Sex ZIP Code Blood Pressure Medication Heart Attack
49 03.12. F 86923 140/90 N Y
62 28.02. M 86923 135/85 Y N
55 14.09. M 83129 160/80 N Y
67 13.03. F 83235 140/80 N N
... ... ... ... ... ... ...
??

13. Attack Scenario: Model Inversion
Reconstructed Original
Access only to API:
1. Reverse engineer model
2. Invert model
Model Inversion Attacks that Exploit
Confidence Information and Basic
Countermeasures,
Fredrikson et al. (2015)

14. Attack Scenario: Membership Inference
Q: given individual A - was A in the training dataset?
Harmful examples:
- Medical study databases
- Customer databases
- ...

15. The issue of Model Capacity
# Parameters > # Training examples
e.g. ImageNet Competition: 1.2 m examples
# Parameters of top models:
Alexnet - 61 m
VGG - 138 m
Inception V1 - 7 m
Resnet-50 - 25.5 m

16. Avoiding Overfitting using Regularization?
Regularization methods
● Early stopping
● Dropout
● Weight decay
● Weight quantization
It’s not just overfitting!

17. Measuring Memorization
Overfitting vs Secret Exposure
Maximum exposure reached
before overfitting begins!
Training is memorization
The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets, Carlini et al. (2018)

18. Differential Privacy
≈ 1 + 𝜖 for small 𝜖

19. Differentially Private Stochastic Gradient Descent
Deep Learning with Differential Privacy, M. Abadi et al. (2016)
Implementation in
tensorflow-privacy

20. Private Aggregation of Teacher Ensembles (PATE)
All answers must be the
consensus of all teachers
Privacy and machine learning: two unexpected allies?, Nicolas Papernot and Ian Goodfellow (2018)

21. Teacher Ensemble Predictions in PATE

22. Training a Differentially Private Student Model
Private Open

23. Training on Synthetic Data
Real Data
NAME AGE GENDER ITEM EUR DATE TIME
Mary 25y female Book 12€ 4/2/19 8:12
John 72y male Pizza 34€ 4/2/19 18:12
...
Bill 18y male Swim 6€ 4/4/19 10:02
Bill 18y male Shoes 123€ 4/4/19 12:32
Synthetic Data
NAME AGE GENDER ITEM EUR DATE TIME
Kim 29y female Amazon 236€ 4/4/19 12:32
Kim 29y female Zalando 36€ 4/4/19 18:58
...
Brian 82y male Beer 6€ 4/2/19 21:32
Sue 24y female Sushi 12€ 4/2/19 21:32
ModelPrivate OpenGenerative Model
(must have randomness)

24. Privacy vs. Accuracy - a Tradeoff?
Theory
Reduced influence
of individual
Better generalization
Practice
Difficulty estimating
privacy loss
Conservative upper
bounds
Reduced accuracy

25. We believe in the power of data.
We believe in the right for privacy.
We are here to make it possible!
We’re hiring!
klaudius.kalcher@mostly.ai