This document summarizes the paper "Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo" by Y.-X. Wang, S. Fienberg, and A. Smola. It proposes a differentially private matrix factorization algorithm called Stochastic Gradient Monte Carlo (SGMC) that achieves state-of-the-art accuracy while providing strong privacy guarantees. SGMC adds calibrated noise to gradients during training to publish model parameters with (ε, δ)-differential privacy. It outperforms prior differentially private collaborative filtering methods by leveraging posterior sampling to balance privacy and accuracy.

1. Privacy for Free:
Posterior Samplingand Stochastic Gradient Monte Carlo
Y.-X. Wang, S. Fienberg, and A. Smola
[Fast Differentially Private Matrix Factorization @ RecSys2015]
1

2. 2

3. (ε, δ)
A
X X’
3
Pr[A(X) À S] f exp(✏) Pr[A(X®
) À S] +

4. A ε
4
ε
A(X) X
X
X X’
A S
A(X’) A(X)
Pr[A(X) À S] f exp(✏) Pr[A(X®
) À S], ≈S À Range(A)
S
Pr[A(X) = S]
Pr[A(X®
) = S]

5. 5
X R R
Pr[R]
q(X, R) X R
X R
Δq
(2εΔq)
q(X, R) − | f(X) − R |
∫ exp(ε q(X, R)) dPr(R)
exp(✏q(X, R)) Pr[R]

6. 6
π(θ) θ
l(x | θ) θ x
X={x} θ
B l(x | θ)
4B
l(x | θ) 2B
Ç✓ Ì Pr[✓X] ◊ exp
≥
x l(✓x) ⇡(✓)

7. 7
f(U, V)=1
2
P
(x,y)2D (rxy u>
x vy)
2
+ 2 (kUk2
F +kVk2
F )

8. 8
y Y x X
(x, y) D
R
UV
rxy
uxvy
l(Rij | θ) ∝ −(Rij − ui
⊤
vj)2

9. 9
PrX
PrX Pr’X L1
δ
A
(ε, (1+eε
) δ)
✓t+1 } ✓t * ⌘t
⇠
(⇡(✓) + N
⌧
≥⌧
i (l(xi✓)
⇡
+ Normal(0, ⌘t)

10. Bibliography I
C. C. Aggarwal and P. S. Yu, editors.
Privacy-Preserving Data Mining: Models and Algorithms.
Springer, 2008.
J. A. Calandrino, A. Kilzer, A. Narayanan, E. W. Felten, and V. Shmatikov.
You might also like: Privacy risks of collaborative filtering.
In IEEE Sympo. on Security and Privacy, pages 231–246, 2011.
C. Dwork, F. McSherry, K. Nissim, and A. Smith.
Calibrating noise to sensitivity in private data analysis.
In Proc. of the 3rd Theory of Cryptography Conference, pages 265–284, 2006.
[LNCS 3876].
Y. Koren.
Collaborative filtering with temporal dynamics.
In Proc. of the 15th ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data
Mining, pages 447–455, 2009.
Z. Liu, Y.-X. Wang, and A. J. Smola.
Fast differentially private matrix factorization.
In Proc. of the 9th ACM Conf. on Recommender Systems, 2015.
10

11. Bibliography II
F. McSherry and I. Mironov.
Differentially private recommender systems: Building privacy into the netflix prize
contenders.
In Proc. of the 15th ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data
Mining, pages 627–635, 2009.
F. McSherry and K. Talwar.
Mechanism design via differential privacy.
In Proc. of the 48th IEEE Sympo. on Foundations of Computer Science, pages 94–103,
2007.
R. Salakhutdinov and A. Mnih.
Probabilistic matrix factorization.
In Advances in Neural Information Processing Systems 20, pages 1257–1264, 2008.
Y.-X. Wang, S. E. Fienberg, and A. Smola.
Privacy for free: Posterior sampling and stochastic gradient monte carlo.
In Proc. of the 32nd Int’l Conf. on Machine Learning, 2015.
11