This document discusses private machine learning using techniques like homomorphic encryption and secure multi-party computation. It explains how these techniques allow a machine learning model to be trained on encrypted data without decrypting it, and make predictions on encrypted data without seeing the raw inputs. Specifically, it covers homomorphic encryption schemes like Paillier encryption that allow private addition and public multiplication on ciphertexts. It also discusses secret sharing techniques used in secure multi-party computation frameworks like SPDZ that replace computation with communication between parties who hold secret shares of the data. The goal is to enable machine learning on sensitive data while preserving privacy.

1. Private Machine Learning
Morten Dahl
(Cyber)Security for Software Engineers meetup, June 2018
Cryptography behind

2. Why?

3. Machine Learning Process
data set prediction modeltraining

4. clinical photos CNNtransfer learning
machine learning on sensitive information could make a big impact

5. Potential Bottlenecks
access to data
monetise on data
risk management
incentivise use
encrypted trainingencrypted data set encrypted prediction

6. Prediction

7. Unencrypted Prediction
w0
w1
w2
x0 x1 x2 x0*w0 + x1*w1 + x2*w2=dot( , )
xw x
dot(x, w)

8. Prediction on Encrypted Data
xw
Enc(x0)*w0 + Enc(x1)*w1 + Enc(x2)*w2=
w0
w1
w2
Enc(x0) Enc(x1) Enc(x2)dot( , )
private addition
public multiplication
Enc(x)
Enc(dot(x, w))
Enc(x0*w0 + x1*w1 + x2*w2)=
Enc(x0*w0) + Enc(x1*w1) + Enc(x2*w2)=
homomorphic
encryption scheme

9. Paillier Encryption
c = Enc(x, r)
x
r1
r2
r3
c1
c2
c3
Plaintexts
Ciphertexts

10. Paillier Encryption
c = Enc(x, r) = g^x * r^n mod n^2
public encryption key
g = 36
n = 35
n^2 = 1225 Enc(5, 4) = 36^5 * 4^35 mod 1225 = 674
Enc(5, 2) = 36^5 * 2^35 mod 1225 = 718
g^x r^n* mod n^2

11. Private Addition
Enc(x, r) * Enc(y, s)
Enc(5, 2) * Enc(5, 4)
= 718 * 674
= 57
= 36^10 * 8^35
= Enc(10, 8)
= (g^x * r^n mod n^2) * (g^y * s^n mod n^2)
= Enc(x+y, r*s)
= g^(x + y) * (r * s)^n mod n^2

12. Public Multiplication
Enc(x, r) ^ w
= (g^x * r^n mod n^2) ^ w
= Enc(x*w, r^w)
= g^(x*w) * (r^w)^n mod n^2
Enc(5, 2) ^ 2
= 718 * 718
= 1024
= 36^10 * 4^35
= Enc(10, 4)

13. What’s Next?
computationally expensive available operations
4096 bit modulus
data expansion
private multiplication
…
Secret Sharing
replace computation with communication

14. Prediction on Secret Shared Data
dot(x, w)
x
w Share1(x)
Share1(dot(x, w))
w Share2(x)
Share2(dot(x, w))

15. Secret Sharing in SPDZ
x1 = Share1(x, r) = r mod m
public parameter
x2 = Share2(x, r) = x - r mod m
m = 10
Share2(5, 7) = 5 - 7 mod 10 = 8
Share1(5, 7) = 7 mod 10 = 7
x1 + x2 = x mod m
x1 = r
x2 = x - r

16. Private Addition
x1
x2
y1
y2
x1 + x2 = x y1 + y2 = y
z1 + z2
= (x1 + y1) + (x2 + y2)
= (x1 + x2) + (y1 + y2)
= x + y
z1 = x1 + y1
z2 = x2 + y2

17. Public Multiplication
x1
x2
w
w
x1 + x2 = x
z1 + z2
= (x1 * w) + (x2 * w)
= (x1 + x2) * w
= x * w
z1 = x1 * w
z2 = x2 * w

18. Private Multiplication
x1
x2
y1
y2
a1
a2
a1 + a2 = a
b1
b2
b1 + b2 = b
c1
c2
x - a
x - a
y - b
y - b
z1 + z2
= …
= x * y
z1 = (x - a)*(y - b)
+ (x - a)*b1
+ (y - b)*a1
+ c1
z2 = (x - a)*b2
+ (y - b)*a2
+ c2
x1 + x2 = x y1 + y2 = y
c1 + c2 = a * b
(
( )
), ,
, ,
random triple

19. Performance
logistic regression
Sigmoid evaluation, 100 features, servers on Google cloud (2 vCPU, 10 GB)

20. Getting Involved

21. ◆ Federated Learning
◆ Homomorphic Encryption
◆ Multi-Party Computation
◆ Gradient Validation Markets
Tools for Safe AI
Slides originally by Andrew Trask

22. Model
Jane’s
Data
Jack’s
Data Joe’s
Data
AI Inc.
for Safe AIFederated Learning
Multi-Party Computation
+ Federated Learning

23. mortendahl.github.io
Thank you
@mortendahlcs
openmined.org
@openminedorg