This document provides an overview of data privacy challenges in multi-agent learning systems. It discusses literature on privacy issues in multi-agent systems, including problems related to autonomy, trust, and defining privacy. Key privacy issues for multi-agent systems are identified as information collection, processing, and dissemination by agents. Several proposed abstract architectures aim to address privacy in multi-agent design, but challenges remain regarding trade-offs between privacy and data utility.

1. An Overview of Data Privacy in Multi-Agent Learning Systems
Kato Mivule, Darsana Josyula, and Claude Turner
Computer Science Department
Bowie State University
Bowie, Maryland, USA
COGNITIVE 2013
1

2. OUTLINE
• Introduction
• Literature Review
• Privacy Issues in Multi-agents
• Proposed Abstract Privacy Architectures
• Work in Progress
• Conclusion
2
COGNITIVE 2013

3. 3
COGNITIVE 2013
This an
overview of
data privacy in
multi-agent
systems,
challenges and
future areas of
work.
Need to engineer
multi-agent
systems that
comply with data
privacy laws.
Entities have to
comply to data
privacy laws.
There are data
privacy challenges
that multi-agents
must encounter.
INTRODUCTION

4. 4
COGNITIVE 2013
MOTIVATION
• Design and develop multi-agent learning systems that take into account
data privacy considerations.

5. LITERATURE REVIEW
5
COGNITIVE 2013
Research
Interest
• Privacy in multi-agent systems has been of research interest for some time.
Forner
(1996):
• Privacy in multi-agents was still problematic due to privacy design challenges.
Wong et al.
(2000)
• Security and trust in multi-agent systems was problematic; proposed a security
and trust architecture, agents self-authenticate.
Yu et al.
(2003)
• Privacy may have various meanings and importance for different agents; there
should be room for a diversity of perceptions on privacy.

6. Ramchurn et al.
(2004):
Three problematic
areas of trust in multi-
agent systems.
(i) Multi-agent
interactions.
(ii) Multi-agent
interrelation.
(iii) Multi-agent
cooperation.
6
COGNITIVE 2013
LITERATURE REVIEW

7. LITERATURE REVIEW – MULTI-AGENTS
7
COGNITIVE 2013
Jung et al.
(2012):
• Critical to autonomous computing; Security and Trust issues
need to be addressed.
Martins et al.
(2012):
• Need to conform to the three canons of privacy and security,
namely, Confidentiality, Accessibility, and Integrity.
Nagaraj
(2012):
• Privacy and security Requirements for multi-agents, is often
neglected during the Requirements design phase.

8. THE PROBLEM: DEFINING PRIVACY
Friedewald et al.
(2010):
Privacy is an evolving
and shifting complex
multi-layered concept,
described differently by
different people.
Katos et al. (2011):
Privacy is a human and
socially driven
distinctive made up of
human mannerisms,
perceptions, and
opinions.
Spiekermann (2012):
Privacy is a fuzzy
concept often confused
with security, and, as
such, difficult to
implement.
A concise definition
of privacy is
problematic.
8
COGNITIVE 2013
Data utility:
How useful a privatized
dataset is to the user of
that particular
privatized dataset.
Faces same definition
challenges as data
privacy.

9. BACKGROUND – DEFINING PRIVACY IN MULTI-AGENTS
Agents: Wooldridge
(2003):
Computer systems
that are located in a
particular
environment with the
capability of
independent and
autonomous action
so as to achieve
intended specific
goals.
Multi-Agents:
Wooldridge (2003):
A group of
autonomous agents
combined into one
system,
independently
solving simpler
problems while
communicating with
each other to
accomplish bigger
and complex
objectives.
Multi-agent systems
(MAS): Da Silva
(2005):
Multi-agent systems
are formed to deal
with complex
applications in a
distributed systems
environment.
Privacy and Security
Issues: Da Silva
(2005):
Observed that
examining data in
distributed
environments is a
difficult problem
since agents face
several restrictions
that include privacy
issues with sensitive
data.
9
COGNITIVE 2013

10. Data privacy and security:
Pfleeger et al. (2006):
Privacy:
A controlled disclosure in
which an entity decides when
and to whom to disclose its
data.
Security:
Has to do with access
control, as in who is allowed
legitimate access to data and
systems.
10
COGNITIVE 2013
BACKGROUND – UNDERLYING PRIVACY NOTIONS

11. 11
COGNITIVE 2013
BACKGROUND - UNDERLYING PRIVACY NOTIONS
The three aspects of
information security
Pfleeger et al. (2006):
Confidentiality:
Ensuring the
concealment and privacy
of data and systems,
Availability:
Ensuring the availability
of data and systems at
all times, and lastly,
Integrity:
Ensuring that data and
systems are altered by
only the authorized.

12. Such et al. (2012):
Multi-agents are
vulnerable to three
information-related
activities:
Information
collection:
agents collect and
store data about an
individual.
Information
processing:
whereby agents
modify data that has
been collected.
Information
dissemination:
whereby agents
publish data.
12
COGNITIVE 2013
PRIVACY ISSUES IN MULTI-AGENT SYSTEMS

13. Klusch et al. (2003):
• Autonomy and
privacy of agents in a
distributed
environment.
Albashiri (2010):
• Multi heterogeneous
agent systems, have
to specify suitable
communication and
interfacing
protocols.
13
COGNITIVE 2013
PRIVACY ISSUES IN MULTI-AGENT SYSTEMS

14. Rashvand et al., (2010):
Multi-agent security
requirements :
Service-agent
protection:
agents are protected
from external threats.
System vulnerability:
agents are protected
from insecure internal
processes.
Protective security
services:
main goal of an agent is
to provide security.
14
COGNITIVE 2013
PRIVACY ISSUES IN MULTI-AGENT SYSTEMS

15. Nagaraj (2012):
• Agent
misbehavior
(e.g., denial of
service attacks).
Martins et al.,
(2012):
• Key security
concerns of
Authentication,
Confidentiality,
and Integrity
must be taken
into considered.
Krupa et al. (2012):
• ‘Privacy
Enforcing Norms’
in which agents
learn a set of
acceptable
privacy social
behavior(rules).
• Penalties given to
agents that
misbehave.
15
COGNITIVE 2013
PRIVACY ISSUES IN MULTI-AGENT SYSTEMS

16. Krupa (2012):
Implementing
privacy for agents is
still problematic:
Agents have to
learn how to sense
privacy violations.
Managing multi-
agents without
centralization to
deter privacy
abuses.
Flexible solutions
to the
inapplicability of
most existing
privacy algorithms.
16
COGNITIVE 2013

17. AN ABSTRACT ARCHITECTURE FOR PRIVACY PRESERVING MULTI-AGENT SYSTEM
17
COGNITIVE 2013

18. AN ABSTRACT ARCHITECTURE FOR PRIVACY PRESERVING MULTI-AGENT SYSTEM
18
COGNITIVE 2013

19. AN ABSTRACT ARCHITECTURE FOR PRIVACY PRESERVING MULTI-AGENT SYSTEM
19
COGNITIVE 2013

20. AN ABSTRACT ARCHITECTURE FOR PRIVACY PRESERVING MULTI-AGENT SYSTEM
20
COGNITIVE 2013

21. THE UNDERLYING CHALLENGES OF DATA PRIVACY AND UTILITY STILL IMPACT PRIVACY DESIGN IN MULTI-AGENTS
21
COGNITIVE 2013
• Trade-offs between privacy and utility are sought.
• The problem of data privacy in multi-agents still remains a challenge.

22. REFERENCES
M. Wooldridge, “An Introduction to Multi-Agent Systems.” Chichester, England: John Wiley and Sons, 2003, ISBN-10: 0470519460.
J.C. Da Silva, C. Giannella, R. Bhargava, H. Kargupta, and M. Klusch, “Distributed data mining and agents”, Eng Appl Artif Intell, pages 791–807, 2005.
K.A. Albashiri, “An investigation into the issues of Multi-Agent Data Mining”, Dissertation, University of Liverpool, 2010
A.L. Samuel, “Some studies in machine learning using the game of checkers”. IBM Journal. Res. Dev. 3, 3, pages 210-229, 1959. DOI=10.1147/rd.33.0210
T. Mitchell, “ Machine Learning”, McGraw Hill. ISBN 0-07-042807-7, page 2, 1997.
IBM, “Big Data”, Online, [Retrieved: March, 2013] http://www-01.ibm.com/software/data/bigdata/
W. Davies, “Agent-Based Data-Mining”, First Year Report, University, 15 August 1994, Online, [Retrieved: March 2013]
http://www.agent.ai/doc/upload/200403/davi94_1.pdf
C.P. Pfleeger and S.L. Pfleeger, “Security in Computing” (4th Edition). Prentice Hall PTR, Upper Saddle River, NJ, USA, pages 10, 606, 2006.
US Department of Homeland Security, “Handbook for Safeguarding Sensitive Personally Identifiable Information at The Department of Homeland Security”, October
2008. Online, [Retrieved February, 2013] http://www.dhs.gov/xlibrary/assets/privacy/privacy_guide_spii_handbook.pdf
E. Mccallister, and K. Scarfone, “Guide to Protecting the Confidentiality of Personally Identifiable Information ( PII ) Recommendations of the National Institute of
Standards and Technology”, NIST Special Publication 800-122, 2010.
V. Rastogi, S. Hong, and D. Suciu, “The boundary between privacy and utility in data publishing”, VLDB, September pp. 531-542, 2007.
M. Sramka, R. Safavi-Naini, J. Denzinger, and M. Askari, “A Practice-oriented Framework for Measuring Privacy and Utility in Data Sanitization Systems”, ACM, (EDBT’ 10)
Article 27, 10 pages, 2010. DOI=10.1145/1754239.1754270
S.R. Sankar, “Utility and Privacy of Data Sources: Can Shannon Help Conceal and Reveal Information”, Information Theory and Applications Workshop (ITA), pages 1-7,
2010 .
R.C. Wong, et al, “Minimality attack in privacy preserving data publishing.” VLDB, pages 543-554, 2007.
W. Davies, and P. Edwards, “Distributed learning: An agent-based approach to data-mining”. In working notes of ICML'95, Workshop on Agents that Learn from Other
Agents, 1995.
D. Caragea, A. Silvescu, and V. Honavar, “Agents that learn from distributed and dynamic data sources.” In Proceedings of the Workshop on Learning Agents, pages 53-
61, 2000.
S. Ontañón, and E. Plaza, “Recycling data for multi-agent learning”. In Proceedings of the 22nd international conference on Machine learning (ICML '05). ACM, pages
633-640, 2005.
R. Cissée, “An architecture for agent-based privacy-preserving information filtering." In Proceedings of 6th International Workshop on Trust, Privacy, Deception and
Fraud in Agent Systems, 2003.
L. Crepin, Y. Demazeau, O. Boissier, and F. Jacquenet, "Sensitive data transaction in Hippocratic multi-agent systems." Engineering Societies in the Agents World IX, pages
85-101, 2009.
22
COGNITIVE 2013

23. REFERENCES
T. Léauté, and B. Faltings, “Privacy-Preserving Multi-agent Constraint Satisfaction”, International Conference on Computational Science and Engineering, Vol. 3, pages
17-25, 2009.
JM. Such, A. Espinosa, A. GarcíA-Fornes, and C. Sierra, "Self-disclosure decision making based on intimacy and privacy." Journal of Information Sciences Vol 211, 2012,
pages 93-111.
JM. Such, A. Espinosa, A. GarcíA-Fornes, and C. Sierra, “A Survey of Privacy in Multi-agent Systems”, Knowledge Engineering Review, in press, 2012.
M. Klusch, S. Lodi, and G. Moro, “Issues of agent-based distributed data mining.”, In Proceedings of the second international joint conference on Autonomous agents and
multiagent systems, ACM, pages 1034-1035, 2003, DOI=10.1145/860575.860782
H.F. Rashvand, K. Salah, J.M.A Calero, L. Harn: “Distributed security for multi-agent systems - review and applications”. IET Inf. Secur. 4(4), pages 188–201, 2012.
S. V. Nagaraj, "Securing Multi-agent Systems: A Survey." Advances in Computing and Information Technology, pages 23-30, 2012.
R.A. Martins, M.E. Correia, and A.B. Augusto, "A literature review of security mechanisms employed by mobile agents," Information Systems and Technologies (CISTI), 7th
Iberian Conference, pages 1-4, 2012.
Y. Krupa and L. Vercouter "Handling privacy as contextual integrity in decentralized virtual communities: The PrivaCIAS framework." Web Intelligence and Agent Systems,
pages 105-116, 2012.
Y. Krupa, "PrivaCIAS: Privacy as Contextual Integrity in Decentralized Multi-Agent Systems." PhD dissertation, Université de Caen, 2012.
S. Chakraborty, Z. Charbiwala, H. Choi, KR. Raghavan, and MB. Srivastava, "Balancing behavioral privacy and information utility in sensory data flows." Pervasive and
Mobile Computing, Volume 8, Issue 3, Pages 331–345 2012.
C. Dwork, “Differential Privacy”, Automata, Languages and Programming, Lecture Notes in Computer Science, Springer, Vol. 4052, pages 1-12, 2006.
V. Ciriani, S.D. Di Vimercati, S. Foresti, and P. Samarati, “Theory of privacy and anonymity”. In Algorithms and theory of computation handbook (2 ed.), pages 18-18,
Chapman and Hall/CRC, 2010, ISBN:978-1-58488-820-8.
P. Samarati and L. Sweeney, “Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression”, IEEE Symp on
Research in Security and Privacy, pp. 384–393, 1998.
US Federal Election Comission, Campaign Finance Disclosure Portal, Online, [Retrieved: March 2013] http://www.fec.gov/pindex.shtml
L.N. Foner, "A security architecture for multi-agent matchmaking." In Proceeding of Second International Conference on Multi-Agent System, Mario Tokoro. Pages 80-86,
1996.
C.H. Wong, and K. Sycara. "Adding security and trust to multiagent systems." Applied Artificial Intelligence Vol. 14, no. 9, pages 927-941, 2002.
E. Yu, and L. Cysneiros. "Designing for Privacy in a Multi-agent World." Trust, Reputation, and Security: Theories and Practice, pages: 259-269, 2003.
S.D. Ramchurn, D. Huynh, and N.R. Jennings. "Trust in multi-agent systems." The Knowledge Engineering Review Vol. 19, no. 1, pages 1-25, 2004.
Y. Jung, M. Kim, A. Masoumzadeh, and J.BD. Joshi. "A survey of security issue in multi-agent systems." Artificial Intelligence Review, Vol. 37, no. 3, pages 239-260, 2012.
R.A. Martins, M.E. Correia, and A.B. Augusto, "A literature review of security mechanisms employed by mobile agents," 7th Iberian Conference on Information Systems
and Technologies (CISTI), pages1-4, 2012.
V. Katos, F. Stowell, and P. Bednar, “Surveillance, Privacy and the Law of Requisite Variety”, Data Privacy Management and Autonomous Spontaneous Security, Lecture
Notes in Computer Science Vol. 6514, pages 123–139, 2011.
S. Spiekermann, “The challenges of privacy by design,” Communications of the ACM, vol. 55, no. 7, page 38, 2012.
M. Friedewald, D. Wright, S. Gutwirth, and E. Mordini, “Privacy, data protection and emerging sciences and technologies: towards a common framework,” Innovation:
The European Journal of Social Science Research, vol. 23, no. 1, pp. 61–67, Mar. 2010.
23
COGNITIVE 2013

24. THANK YOU.
Contact: kmivule at gmail dot com
24
COGNITIVE 2013