Literature Review – Talk, By Kato Mivule, COSC891 Fall 2013, Computer Science Department, Bowie State University "Signal Processing and Machine Learning with Differential Privacy Algorithms and challenges for continuous data" Sarwate and Chaudhuri (2013)

1. Signal Processing and Data Privacy
Literature Review – Talk
By Kato Mivule
COSC891 Fall 2013
Signal Processing and Machine Learning with Differential Privacy
Algorithms and challenges for continuous data
Sarwate, A.D.; Chaudhuri, K., "Signal Processing and Machine Learning with Differential Privacy: Algorithms and Challenges for
Continuous Data," Signal Processing Magazine, IEEE , vol.30, no.5, pp.86,94, Sept. 2013
doi: 10.1109/MSP.2013.2259911
Bowie State University Department of Computer Science

2. Signal Processing and Data Privacy
Agenda
•
•
•
•
•
•
Introduction.
Privacy Definition Challenge.
Differential Privacy Definition.
Differential Privacy Challenges.
Differential Privacy Applications in Signal Processing.
Conclusion.
Bowie State University Department of Computer Science

3. Signal Processing and Data Privacy
Introduction: Privacy Preserving Data Mining
• Maintaining the privacy of individuals is imperative.
• Individuals expect their data to be kept private despite willingness to share
such info.
• The Challenge is how extract knowledge from large scale data while
maintaining privacy.
Bowie State University Department of Computer Science

4. Signal Processing and Data Privacy
Introduction: Privacy Definition
• Privacy definition is problematic.
• There are different meaning for these words across different communities:
• Privacy
• Confidentiality
• Security
• “There is no real separation between individuals’ identity and their data—the
pattern of data associated with an individual is itself uniquely identifying.”
Sarwate and Chaudhuri (2013)
Bowie State University Department of Computer Science

5. Signal Processing and Data Privacy
Differential Privacy Definition
• Cryptographically Motivated.
• Proposed by Cynthia Dwork (2006).
• Imposes confidentiality by returning perturbed query responses from
databases:
•
𝒇 𝒙 + 𝑳𝒂𝒑𝒍𝒂𝒄𝒆(𝟎, 𝒃)
•
𝒃=
•
∆𝒇
𝜺
∆𝒇 = 𝑴𝒂𝒙 𝒇 𝑫 𝟏 − 𝒇 𝑫 𝟐
• The end user of the database cannot know if a data item has been altered.
• An attacker cannot gain information about any data item in the database.
Bowie State University Department of Computer Science

6. Signal Processing and Data Privacy
Differential Privacy Definition:
ε-differential privacy is satisfied if the results to a query run on database D1 and D2 should
probabilistically be similar, and meet the following condition:

𝑷[𝒒 𝒏 (𝑫 𝟏 )∈𝑹]
𝑷[𝒒 𝒏 𝑫 𝟐 ∈𝑹]
≤ 𝒆𝜺

Where D1 and D2 are the two databases


P is the probability of the perturbed query results D1 and D2.
qn() is the privacy granting procedure (perturbation).


qn(D1) is the privacy granting procedure on query results from database D1.
qn(D2) is the privacy granting procedure on query results from database D2.


R is the perturbed query results from the databases D1 and D2 respectively.
𝒆 𝜺 is the exponential e epsilon value.
Bowie State University Department of Computer Science

7. Signal Processing and Data Privacy
Differential Privacy Definition – Types
• Query-based Differential Privacy.
• Input perturbation based differential privacy – add Laplace noise to the data.
• Output perturbation – add Laplace noise to the query results.
Bowie State University Department of Computer Science

8. Signal Processing and Data Privacy
Differential Privacy – Input and Output Perturbation
Image Source: Sarwate, A.D.; Chaudhuri, K., "Signal Processing and Machine Learning with Differential Privacy: Algorithms and Challenges for
Continuous Data," Signal Processing Magazine, IEEE , vol.30, no.5, pp.86,94, Sept. 2013 doi: 10.1109/MSP.2013.2259911
Bowie State University Department of Computer Science

9. Signal Processing and Data Privacy
Differential Privacy Challenges
• There is a tension between data privacy and data utility.
• Evaluating the impact of privacy restrictions on utility.
• Data dimensionality.
• Trade-off points between data privacy and utility.
Bowie State University Department of Computer Science

10. Signal Processing and Data Privacy
Differential Privacy Challenges – Utility Quantification
•
Mean Squared Error – for statistical estimation.
•
•
Computed as follows: Original and privatized datasets 𝑋 − 𝑋 ′
𝑑
𝑗=1
𝑛
′ 2
𝑖=1(𝑥 𝑖𝑗 −𝑥 𝑖𝑗 )
𝑛𝑑
•
Expected loss – for classification.
•
Comparative analysis – quantify various differential privacy algorithms.
•
Where n is the number of records and d the number of variables in the table.
Bowie State University Department of Computer Science

11. Signal Processing and Data Privacy
Differential Privacy Challenges – Limitations
• Time Series and Filtering problems.
• Understanding the fundamental limits for continuous data may shed some
light on which signal processing tasks are possible under differential privacy.
• The optimal differential privacy parameter adjustment for acceptable levels of
data utility – adjustment of the 𝜺 value.
• A single data set could be used in multiple computations – challenge is how
to keep the same differential privacy across the board.
Bowie State University Department of Computer Science

12. Signal Processing and Data Privacy
Differential Privacy – Signal Processing Applications
•
Apply differential piracy in signal processing problems.
•
Apply signal processing to the differential privacy problem of data utility.
•
Time Series and Filtering problems.
•
Differential privacy of a query sequence in Fourier domain and use homomorphic
encryption for distributed noise addition.
•
Kalman filtering applied to a differentially private time series.
•
Kalman filtering used on aggregated signals after input and output perturbation.
Bowie State University Department of Computer Science

13. Signal Processing and Data Privacy
Differential Privacy – Applications
• Integrate differential privacy in signal processing.
• Use signal processing to enhance data utility and privacy.
• Use signal processing to filter out unneeded noise.
• Research areas – applying differential privacy in signal processing for:
•
•
•
•
Image processing
Network information systems
Cryptographic approaches
Social networks
Bowie State University Department of Computer Science

14. Signal Processing and Data Privacy
Conclusion
•
A general overview of the differential privacy is given.
•
The paper focused on differential privacy and the applications in signal processing.
•
The paper suggests research areas for applying differential privacy in signal
processing.
•
Implementation is left to the readers.
Bowie State University Department of Computer Science

15. Signal Processing and Data Privacy
References
•
Sarwate, A.D.; Chaudhuri, K., "Signal Processing and Machine Learning with Differential Privacy: Algorithms
and Challenges for Continuous Data," Signal Processing Magazine, IEEE , vol.30, no.5, pp.86,94, Sept. 2013
doi: 10.1109/MSP.2013.2259911
•
Mivule, K; Turner, C; and Y. Ji, S., “Towards A Differential Privacy and Utility Preserving Machine Learning
Classifier,” in Procedia Computer Science, 2012, vol. 12, pp. 176–181.
•
Mivule, Kato, "Utilizing Noise Addition for Data Privacy, an Overview", Proceedings of the International
Conference on Information and Knowledge Engineering (IKE 2012), Pages 65-71, Las Vegas, NV, USA.
Bowie State University Department of Computer Science