• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
T11. Normative multi-agent systems
 

T11. Normative multi-agent systems

on

  • 488 views

14th European Agent Systems Summer School. Tutorial slides

14th European Agent Systems Summer School. Tutorial slides

Statistics

Views

Total Views
488
Views on SlideShare
488
Embed Views
0

Actions

Likes
0
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    T11. Normative multi-agent systems T11. Normative multi-agent systems Presentation Transcript

    • Tutorial Normative Reasoning and Decision MakingGuido Boella & Leon van der Torre
    • Agreement Technologies• WG1: “Semantics”• WG2: “Norms”• WG3: “Organizations”• WG4: “Argumentation and Negotiation”• WG5: “Trust” 2
    • Some Norms• You are obliged to return the book in 2 weeks• Role X is permitted to access resource Y• Actions of agent X count as actions of agent Y• Advocates should act in the interest of their clients• Violations should be sanctioned• Don’t do to them what you don’t want them to do to you• People should know the law• Every adult has the right to vote• Make your homework!• We start at 09:00 3
    • Merriam-Webster Online Dictionary• ‘normative’: ‘conforming to or based on norms’, – as in normative behavior, normative judgments – not: ‘of, relating to, or determining norms or standards’, as in normative tests, or ‘prescribing norms’, as in normative rules of ethics or normative grammar.• ‘norm’: ‘a principle of right action binding upon the members of a group and serving to guide, control, or regulate proper and acceptable behavior’. – Not: ‘an authoritative standard or model’, ‘an average like a standard, typical pattern, widespread practice or rule in a group’, and various definitions used in mathematics. 4
    • Normative Systems (1971)“When a deductive correlation is such that thefirst sentence of the ordered pair is a case andthe second is a solution, it will be callednormative. If among the deductive correlations ofthe set there is at least one normativecorrelation, we shall say that the set hasnormative consequences. A system of sentenceswhich has some normative consequences will becalled a normative system.” C. Alchourron and E. Bulygin. Normative Systems. Springer, 1971. 5
    • Normative Systems (1993)• Normative systems are “systems in the behavior of which norms play a role and which need normative concepts in order to be described or specified” J.-J. Meyer and R. Wieringa. Deontic Logic in Computer Science: Normative System Specification. John Wiley & Sons, 1993.• Many distinct notions of “normative systems” – Social expectation, legal law, linguistic imperative…• Role of norms in computer science is changing – Solutions based on multiagent systems increasing 6
    • Layout• Challenges due to uncertainty and imprecision in normative reasoning and decision making1. Normative reasoning in computer science2. Imprecision & uncertainty normative systems3. Decision making in normative systems1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 7
    • AgentLink RoadMap (2003) 8
    • 2005: Normative MultiAgent Systems• are multiagent systems with normative systems• in which agents can decide whether to follow the explicitly represented norms, and• the normative systems specify how and in which extent the agents can modify the norms.’’ G. Boella, L. van der Torre, H. Verhagen, Introduction to normative multiagent systems. Computational and Mathematical Organization Theory, double special issue on normative multiagent systems, 2006. “The normchange definition”1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 9
    • 2007: Normative MultiAgent System• is a multiagent system organized by means of• mechanisms to represent, communicate, distribute, detect, create, modify, and enforce norms, and• mechanisms to deliberate about norms and detect norm violation and fulfillment.” G. Boella, L. van der Torre, and H. Verhagen, Introduction to the special issue on normative multiagent systems. Autonomous Agents and MultiAgent Systems, Aug. 2008. “The mechanism design definition”1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 10
    • Layout• Challenges due to uncertainty and imprecision in normative reasoning and decision making1. Normative reasoning in computer science2. Imprecision & uncertainty normative systems3. Decision making in normative systems1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 11
    • Norms: Certain and Precise?• Norms are rules to distinguish right from wrong.• Dilemmas: when may we shoot down a hijacked plane?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 12
    • Violations: Certain and Precise?• Norms describe what counts as a violation.• Budget deficit > 3 % of GDP is a violation?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 13
    • Design: Uncertain and Imprecise• Norms are rules to guide, control or regulate behavior.• What are the effects of new norms on behavior?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 14
    • Guide, Control, Regulate Behavior What’s new with respect to constraints?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 15
    • Guide, Control, Regulate BehaviorWays to deal with violations, representation of permissive norms,the evolution of norms over time (in deontic logic), the relationbetween the cognitive abilities of agents and the global propertiesof norms, how agents can acquire norms, how agents can violatenorms, how an agent can be autonomous (in normative agentarchitectures and decision making), how norms are created bylegislator, emerge spontaneously or are negotiated among agents,how norms are enforced, how constitutive or counts-as norms areused to describe institutions, how norms are related to social andlegal concepts, how norms structure organizations, how normscoordinate groups and societies, how contracts are related tocontract frames & contract law, how legal courts are related, hownormative systems interact, …1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 16
    • Normative Reasoning Norms are rules to guide, control or regulate behavior. Infrastructure for open communities. Reasoning in open environments. Trust and reputation. More than distinguishing right from wrong. More than describing what counts as a violation. More than constraints. Uncertain and imprecise.1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 17
    • Security and Reliability• Norms make systems more secure & reliable.• How to prevent over-regulation?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 18
    • Trust• Norms build up trust in international trade. ESCROW• Which mechanisms for electronic commerce?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 19
    • Example: ESCROW• Norms describing coordination – Agent communication: sales agreement – Agent transaction: payment for shipment• Norms describing fees and compensation – ESCROW gets percentage (from seller or buyer) – Return merchandize within 5 days• Norms ensure fulfillment of some obligations – Seller is paid for goods, or goods will be returned – (under assumption of trustworthiness of ESCROW)• Some obligations can still be violated: – Seller does not ship goods, shipper unreliable, …2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems4. Norm change 20
    • Virtual Communities• Social norms are emerging in Second Life.• How to prevent exclusion from communities?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 21
    • Autonomous Systems• aut- + nomos: making ones own norms.• How to define global policies about local ones?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 22
    • Applications Normative Reasoning Norms are rules used to control or regulate behavior. Risk management for computer security. Designing trust mechanisms for electronic commerce. Recognition of emerging social norms in Second Life. Local and global policies for autonomous systems.1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 23
    • Layout• Challenges due to uncertainty and imprecision in normative reasoning and decision making1. Normative reasoning in computer science2. Imprecision & uncertainty normative systems3. Decision making in normative systems1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 24
    • Examples: How to decide …• … whether to fulfill or violate a norm?• … whether behavior counts as a violation?• … whether an excuse for a violation is acceptable?• … whether a violation should be sanctioned?• … whether to accept a norm / contract / … ?• ... whether to accept sanctions of a norm?• … which contract to propose / norm to create?• … whether an agent conforms to the norm?• … whether a procedure is compliant to the norm?• … on a constitution for Europe?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 25
    • Basis: Decision Making With Norms• Introducing obligations in agent architectures• tl O tl tl al cl hl act obs B I P D obligations as normative goals• Introducing obligations in agent programming1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 26
    • Exercise: When Violate a Norm?• When you are not able to fulfill it• When the sanction is too low• When the probability to be caught is too low• …1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 27
    • Exercise: When Violate a Norm?• When you are not able to fulfill it• When the sanction is too low• When the probability to be caught is too low• When you don’t know the norm• When there is another more important desire• When there is another more important obligation• When you have a good excuse• When you like to take risks• When other agents violate the norm• When the norm does not make sense1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 28
    • When to Violate/Fulfill: Challenge• Multiagent structure of a normative system – Many agents involved in agent interaction – Interaction is highly structured• Normative system regulates their roles – Powers of roles described by counts-as norms, and – Their behavior is regulated by procedural norms.1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 29
    • Norm Compliance• Attacker = agent trying to profit from violation – Violation undetected, no sanction, or less than profit normative system Attacker / adversary $$• Anticipate rather than regiment (mechanism design)1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 30
    • Profile NS of Attacker A (for Op)1. NS desires p (“your wish is my command”).2. Absence of p is considered as violation of A. ! Anderson’s reduction of deontic logic to modal logic.3. NS desires that there are no violations.4. If violation, then NS is motivated to sanction.5. NS does not like to sanction.6. A does not like being sanctioned.7. NS has the power to count absence of p as violation.8. NS has the power to enforce sanction.Example: You are obliged to return the book in 2 weeksQuestion: Why do we need all of them? Is it complete?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 31
    • When to Accept a Contract? normative system trustee $$ trustor• Decision making of trustor taking profiles of trustee and normative system into account1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 32
    • Challenge: When to Accept a Norm •Immanuel Kant, Metaphysics of Morals, 1785• “Act only according to that maxim whereby you can at the same time will that it should become a universal law.”1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 33
    • Emergence of Norms• Don’t do to them, what you don’t want them to do to you• Social delegation cycle – Agent desires – Merging: Social goal – Planning: Norm – Acceptance: Agent desires• I accept if you fulfill the norm on the assumption that all others fulfill it1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 34
    • Autonomous Systems• Legal institutions in context of legal institutions – Norm dynamics described by counts-as norms• Strong permissions in norm creation (Bulygin)• Global policies about local policies – Von Wright: transmission of will by nested norms1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 35
    • Hierarchical Normative Systems Global authority Norm. 2 $$ Norm. 1• Decision making of norm. system 1 taking profiles of norm. system 2 and global authority into account1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems 36
    • Decision Making in NMASNorms are rules used to control or regulate behavior. Norm conformance and compliance. Multiagent structure of normative systems. Norm acceptance for emergence of norms. Hierarchical normative systems for autonomy. 37
    • Concluding Remarks• Challenges due to uncertainty and imprecision in normative reasoning and decision making1. Normative reasoning in computer science2. Imprecision & uncertainty normative systems3. Decision making in normative systems 38
    • http:deonticlogic.org 39
    • Further Reading• Normative multiagent systems proceedings: – Computational and Mathematical Organization Theory, double special issue on normative multiagent systems, 2006. – Normative Multiagent Systems, 18.03. - 23.03.2007, Schloss Dagstuhl, Germany, 2007. – Autonomous Agents and Multiagent Systems Journal, special issue on normative multiagent systems, August 2008. – 4rd International Workshop on Normative Multiagent Systems (NORMAS2009), Dagstuhl, March 2009• Introductions to the area and its challenges – Ten philosophical problems in deontic logic – Ten challenges for normative multiagent systems – Ten guidelines for norms in computer science 40
    • Ten Philosophical Problems1. How can deontic logic be reconstructed in accord with the philosophical position that norms are neither true nor false?2. When is a set of norms to be termed `coherent?3. How can deontic logic accommodate possible conflicts of norms? How can the resolution of apparent conflicts be semantically modeled?4. How do we reason with contrary-to-duty obligations which are in force only in case of norm violations?5. How to define dyadic deontic operators with regard to given sets of norms and facts?6. How to distinguish various kinds of permissions and relate them to obligations?7. How can meaning postulates and intermediate terms be modeled in semantics for deontic logic reasoning?8. How to define counts-as conditionals and relate them to obligations and permissions?9. How to revise a set of regulations or obligations? Does belief revision offer a satisfactory framework for norm revision?10. Can the belief merging framework deal with the problem of merging sets of norms? J. Hansen, G. Pigozzi and L. van der Torre, Ten philosophical problems in deontic logic, Dagstuhl DROPS proceedings 07122, 2007. 41
    • Ten Challenges for NorMAS1. Tools for agents supporting communities in their task of recognizing, creating, and communicating norms to agents.2. Tools for agents to simplify normative systems, recognize when norms have become redundant, and to remove norms.3. Tools for agents to enforce norms. In a distributed approach, roles should be defined for agents in charge of monitoring and sanctioning.4. Tools for agents to preserve their autonomy.5. Tools for agents to construct organizations.6. Tools for agents to create intermediate concepts and normative ontology, for example to decide about normative gaps.7. Tools for agents to decide about norm conflicts.8. Tools for agents to voluntarily give up some norm autonomy by allowing automated norm processing in agent acting and decision making9. Tools for conviviality.10. Tools for legal responsibility of the agents and their principals. G. Boella, L. van der Torre, H. Verhagen, Introduction to the special issue on normative multiagent systems. Autonomous Agents and MultiAgent Systems, Aug 2008. 42
    • Ten Guidelines 43
    • Guideline 11. Motivate which definition of normative multiagent system is used.• Norms explicitly represented in system – (the ‘strong’ interpretation: too strict?)• Norms explicitly represented in specification – (the ‘weak’ interpretation: too general?) – Norm compliance, norm implementation, ...• Something else 44
    • Guideline 22. Make explicit why norms are a kind of (soft) constraints deserving special analysis.• Ways to deal with violations, representation of permissive norms, the evolution of norms over time (in deontic logic), the relation between the cognitive abilities of agents and the global properties of norms, how agents can acquire norms, how agents can violate norms, how an agent can be autonomous (in normative agent architectures and decision making), how norms are created by legislator, emerge spontaneously or are negotiated among agents, how norms are enforced, how constitutive or counts-as norms are used to describe institutions, how norms are related to social and legal concepts, how norms structure organizations, how norms coordinate groups and societies, how contracts are related to contract frames & contract law, how legal courts are related, how normative systems interact? 45
    • Guideline 33. Explain why and how norms can be changed at runtime.• E.g., legislators and voting on acceptance, observe behavior and violations to modify… 46
    • Guideline 44. Discuss the use and role of norms as a mechanism in a game-theoretic setting.• D. Lewis “master and slave” game• E. Bulygin “rex, minister and subject” game• G. Boella c.s.: violation games, institutionalized games, negotiation games, norm creation games, control games Norms are rules specifying violation games. 47
    • Guideline 55. Clarify role of norms in the multiagent system.• Norms guide (…) desired system behavior• Norms are incentives to motivate agents – Gneezy and Rustichini’s daycare example• Norms organize systems – Modularity, abstractions 48
    • Guideline 66. Relate the notion of “norm” to the legal, social, or moral literature.Five phases in normas design:1.off-line norm design2.norm representation3.norm manipulation4.social reality5.moral reality 49
    • Guideline 77. Use norms not only to distinguish right from wrong, but also to resolve dilemmas, and use norms not only describe violations, but in general to coordinate, organize, guide, regulate or control interaction among agents. 50
    • Guideline 88. Distinguish norms from obligations, prohibitions and permissions.• Deontic logic: logical relations obligations, etc – normative system is typically left implicit• Two distinct philosophical traditions – Von Wright: norms and normative propositions – Alchourron: prescriptive and descriptive obligations J. Hansen. Imperatives and Deontic Logic. PhD thesis, University of Leipzig, 2008. 51
    • Guideline 99. Use the deontic paradoxes only to illustrate the normative multiagent system.1. A certain man should go to the assistance of his neighbors,2. If he goes, he should tell them he is coming3. If he does not go, he should not tell them that he is coming4. He does not go.{Oa,O(a ! t),¬a ! O(¬t),¬a} {Oa, a ! O(t),¬a ! O(¬t),¬a} 52
    • Guideline 1010. Consider regulative norms in relation to other kinds of norms and concepts.• in relation to permissive norms, constitutive norms, procedural norms, agents, roles, groups, societies, rights, duties, obligations, time, beliefs, desires, intentions, goals, roles, and other kinds of norms and other social-cognitive computer science concepts. 53
    • The Question• Could (or should) “norms” play a similar role in computer science like “service”, “contract” or “trust”? – Since the use of norms is a key element of human social intelligence, norms may be essential too for artificial agents that collaborate with humans, or that are to display behavior comparable to human intelligent behavior. – Norms are thought to ensure efficiency at the level of the multiagent system whilst respecting individual autonomy.• We have to build more flexible normative multiagent systems, and test them in practice, before we know where they can be used best. 54
    • Roles in conceptual modelling
    • Objects, relationships, and rolesEach modelling language has its own reference ontology.The duality of objects and relationships is present in all mod-elling languages. E.g., variables and constants vs predicatesand relations.But there is another basic ontological distinction: roles.
    • Roles, first appearanceBachman and Daya 1977’s role DB data model:“Records and n-tuples are role oriented. These deal withemployees, customers, patients, or students: role types. Thisis in contrast with DB theory: each record should representall aspects of some one entity in the real world. The reasonfor confusion is that neither the roles of the real world northe entities of the real world are a subset of the other”
    • “If members of a set are records of various types, large piecesof redundant code had to be written in order to addressthe semantically equivalent, but syntactically distinct fields(items) of the members when iterating through a set.By letting all the entity types involved play the same role andby defining the fields as fields of the role, not of the types, itbecame possible to process all members with the same pieceof code.”
    • Natural types vs rolesSowa 1984’s conceptual structures distinguish between natu-ral types “that relate to the essence of the entities” and roletypes “that depend on an accidental relationship to someother entity”: child depends on parent, pet on owner, etc.Sowa: role types are subtypes of natural types. Child, Pet,Quotient and Food are subtypes of the natural types Person,Animal, Number and Physical-Substance.
    • Properties of roles in Object Orientation from Steimann 2000 1. A role comes with its own properties and behavior. Hence, it is a type. E.g., a director of a department commands other members and makes buy-orders. 2. Roles depend on relationships: e.g., an employee is re- lated to an employer, a student to a school, etc. 3. Objects of unrelated types can play the same role. E.g., both a person and an organization are customers.
    • 4. An object may play different roles. E.g., a person can be a student and an employee.5. An object may play the same role several times. E.g., a person can hold several employments.6. An object may acquire and abandon roles dynamically. E.g., a person can acquire the role of student or of em- ployee.7. The sequence in which roles may be acquired and relin- quished can be subject to restrictions. E.g., a person becomes a teaching assistant only if it is a student.
    • 8. Roles can play roles. E.g., an employee can be a project leader, a role of the employee. 9. A role can be transferred from one object to another. E.g., the salary of an open position may be specified independently of the person that will be employed.10. The state of an object can vary depending on the role in which it is being addressed: this should be viewed as a separate instance of the object. E.g., an employee has an address per job and also a private one.11. If an object plays several of roles simultaneously, it re- sponds according to the role in which it is being ad- dressed. E.g., an person gives the address of the em- ployee it is playing.
    • 12. Roles restrict access. This corresponds to an object hav- ing different perspectives, facets, or aspects. E.g., the private phone number of an employee can be invisible when the person is playing the employee role.13. Different roles may share structure and behavior: role definitions inherit from each other. E.g., the role first year student can inherit the behavior of giving exams from the role student.14. An object and its roles share identity. Since roles do not exist by themselves they cannot have an identity.15. An object and its roles have different identities. E.g., the number of passengers of an airline can be greater than the number of person who travelled with it.
    • Possible views of roles1. Roles as named places in relationships.2. Roles as specifications or generalizations of natural types.3. Roles as adjunct instances.
    • Role as a named place in a relationshipIf two places of a relationship are of the same type, thenthe type would not suffice to distinguish them. Role namesshould serve to identify the places.Every place of a relationship has a role name in the ER model(and UML).For example, the is-parent-of (or is-child-of ) relationship issufficiently characterized by the role names parent and child.However, it fails to account for the fact that roles come withtheir own properties and behavior.
    • Roles as specializationsIf roles are types, how are the roles and the natural typesrelated?A role type is more specific than the natural type of its roleplayers: if Customer and Supplier are subtypes of Person.But, not only persons, but also organizations like companies,etc., can be customers and suppliers. However, declaringCustomer and Supplier as subtypes of both Person and Or-ganization is a problem.
    • Roles as generalizationsIf roles are not subtypes, could they be supertypes?No, if roles are more specific than the types of their players.But this is false, anyway: a person has many properties notrequired of a customer or supplier. Rather, being a customeror supplier imposes its required properties on persons andorganizations, making the former supertypes of the latter.From the extensional point of view, roles are supertypes stat-ically, while dynamically they are subtypes.
    • Roles as adjunct instancesRoles as independent types the instances of which are carriersof role specific state and behavior, but not identity.An object and its roles are related by a special played-by rela-tion, and an object and its roles form an aggregate (subject).Picking up a role corresponds to the creation of a new in-stance of the corresponding role type and its integration intothe compound.
    • Ontological properties of rolesFollowing Sowa, Guarino and Welty 2002 presented an onto-logical distinction between role and natural types.A role is a concept that is: • founded: the individuals of a role stand in relation to other individuals, • lacks semantic rigidity: they can enter and leave the ex- tent of the concept without losing their identity.
    • A natural type, on the other hand, is characterized by: • semantic rigidity: an individual of a natural type cannot drop this type without losing its identity, • lack of foundation: for an individual to belong to that type it is not required that it stands in relationship to others.E.g., Person is a natural type, Student is a role.Adolescent and Adult are states or phases, lacking both se-mantic rigidity and foundation.
    • Dependence of rolesFine 1995’s notion of dependence: “to say that an object xdepends upon an F is to say that an F will be ineliminablyinvolved in any definition of x”.Masolo et al. 2004’s definitional dependence: e.g., the def-inition of the concept of student makes reference not to aspecific school but to the concept of school, etc.
    • Handbook Norms Change Permission Institution Agency Deontic Logic as Normative Reasoning Leon van der Torre Individual and Collective Reasoning Group Computer Science and Communication University of Luxembourg EASSS 2012 (Valencia) May 2012 Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution Agency 1 Handbook 2 Problem 1 - In what sense are obligations different from norms? 3 Problem 2 - How do norms change? 4 Problem 3 - How to relate various kinds of permissions? 5 Problem 4 - What is the role of constitutive norms? 6 Problem 5 - Where does deontic logic meet game theory? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyVolume 1: Deontic logic and imperatives A. BACKGROUND 1. R. Hilpinen and P. McNamara, Deontic logic: history, overview. 2. J. Hansen, Imperative logic: history, overview. B. DEONTIC PROBLEMS 3. S. Hansson, Varieties of permission. 4. M. Sergot, Contrary-to-duty norms. 5. L. Goble, Prima facie norms, normative conflicts and dilemmas. 6. M. Sergot, The theory of normative positions. 7. A.J.I. Jones and D. Grossi, Constitutive norms and counts as conditionals. 8. P. McNamara, Logics for supererogation and allied normative concepts. C. CANDIDATES FOR A NEW STANDARD 9. S. Hansson, Alternative semantics for deontic logic. 10. X. Parent and L. van der Torre, Input/output logic. 11. J. Hansen, Imperativist approach to deontic logic. 12. L. Lindahl and J. Odelstad, TJS. Normative systems with intermediaries. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyVolume 2: Norms in systems, and interactions D. DYNAMICS 1 ?, Time and deontic logic. 2. J. Broersen, J. Horty and J.-J. Ch. Meyer, Actions and deontic logic. 3. G. Boella, G. Pigozzi and L. van der Torre, Normative system change. 4. J. van Benthem and F. Liu, Deontic logic and changing preferences. 5. D. Gabbay, Reactive approaches. E. INTERACTIONS 6 R. Thomason, The formalization of practical reasoning: problems and prospects. 7 B. Kooi and A. Tamminga, Deontic logic and game theory. 8 K. van Fintel and T. Gillies, Deontic logic and natural language. 9 G. Sartor, G. Governatori, and A. Rotolo. Deontic logic and legal reasoning. F. INTEGRATION INTO SYSTEMS 10. D. Garg, Modalities for access control. 11. J. Lobo, E. Lupu and A. Russo, Distributed systems management. 12. F. Cuppens and N. Cuppens-Boulahia, Obligations for usage control. 13. G. Aucher and G. Boella, Privacy and epistemic obligations. 14. R. Parikh, L. Oldeloohuisl, and C. Baskent, Epistemic norms. 15. G. Governatori, Rule, Norms and compliance. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyIn what sense are obligations different from norms? BQ Can norms have truth values? (Jorgensen, 1938) Imperatives, promises, legal statutes, and moral standards are usually not viewed as being true or false. E.g.: “John, leave the room!” BQ Is a logic of norms possible? Without truth, norms cannot: be premise or conclusion in an inference, be termed consistent or contradictory, or be compounded by truth-functional operators. BQ Can deontic logic be built on descriptive obligations only? (Makinson, 1998) turns the Bad Questions into a Good Question: Problem 1. How can deontic logic be reconstructed in accord with the philosophical position that norms are neither true nor false? Explained in following slides in three steps. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyStep 1: Traditional Deontic Logic, axiomatic (1951) In what sense are obligations different from norms? SDL = normal modal logic of type KD extends the propositional tautologies with the axioms K : O(x → y ) → (Ox → Oy ) and D : ¬(Ox ∧ O¬x), closed under modus ponens x, x → y /y and Necessitation x/Ox Prohibition & permission defined Fx = O¬x and Px = ¬O¬x. BQ Which representation for conditional norms O(x|a)? O(x|a) = O(a → x), or O(x|a) = a → Ox, or something else BQ Factual or deontic detachment? Factual detachment: (O(x|a) ∧ a) → Ox Deontic detachment: (O(x|a) ∧ Oa) → Ox Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyStep 2: Traditional Deontic Logic, possible worlds (1969) In what sense are obligations different from norms? DSDL3 = dyadic modal logic Kripke models ￿W , ≥￿ where ≥ is total pre-order O(x|a) iff x holds in all the most preferred a worlds Axiomatized by Lewis (counterfactuals, 1973) and Spohn BQ Which representation for conditional norms O(x|a)? BQ Which logical relations hold among conditional norms? BQ Factual or deontic detachment? Factual detachment: (O(x|a) ∧ a) → Ox Deontic detachment: (O(x|a) ∧ Oa) → Ox BQ How are logical relations and detachment related? For example, transitivity and deontic detachment Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyStep 3: Modern Deontic Logic, iterative (1998) In what sense are obligations different from norms? Makinson’s iterative detachment approach Normative code N is pairs of propositional formulas (a, x) Gross output: out(N, A) = apply N as rules on A E.g., E0 = ∅,En+1 = En ∪ {x|(a, x) ∈ N, En ∪ A implies a} Net output: consistency constraint for output E.g., En+1 = En ∪ {x|(a, x) ∈ N, En ∪ A implies a} if cons. En otherwise BQ Which representation for conditional norms O(x|a)? GQ How to detach obligations from conditional norms? Factual detachment: ((a, x) ∧ a) → Ox Deontic detachment: ((a, x) ∧ Oa) → Ox BQ How are logical relations and detachment related? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyCandidates for a New Standard Makinson’s iterative detachment approach GQ Which approaches do not fit into this general framework? GQ What are the alternatives to this iterative framework? Algebraic Programming Non-Monotonic a:b/Oc Reactive Iterative a in out(C,b) Diagnostic q / ¬V(n) ! p Labeled Imperativistic Op!p,Oq!q ! O(p/q)!p,!q !p,!q ! O(p/q) Dynamic Input/Output K O Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 1 - Some Good Questions Makinson’s iterative detachment approach GQ When is a normative system coherent? GQ When is a norm redundant? GQ When are two normative systems equivalent? GQ How are the three questions above related? GQ Which meta-logical properties can or should hold? Law2case principle Strong case2law principle GQ How to prevent the pragmatic oddity without creating the drowning problem? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 2 - How do norms change? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 2 - How do norms change? Problem 1 - In what sense are obligations different from norms? GQ How is obligation change different from norm change? How to formalize the relation between them? Obligations can change while normative system remains same Detachment of obligation, delegation, etc A relatively clear and well studied subject (in 70s and 80s) Norm change: the new challenge (workshops 2007, 2010) Norm revision and contraction, e.g. change of legal code Norm evolution, e.g. change of social norms Merging normative systems, e.g. merger of companies Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 2 - How do norms change? Problem 2.1. How to revise a set of norms? Alch´urron & Makinson [1981] study changes of legal code: o addition amendment: revision derogation: contraction Legal code is a non-empty and finite set of propositions. Starting point of belief revision (with G¨rdenfors), a.k.a. AGM a AGM 1985: minimality represented by recovery postulate More advanced variants, e.g. for iterated belief revision Also some issues with prioritized norms we do not consider Defeasible deontic logic derived from non-monotonic logic Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution Agency AGM 85 C. E. Alchourrón, P. Gärdenfors, and D. Makinson. On the logic of theory change: Partial meet contraction and revision Leon van der Torre Log., 50(2):510–530, 1985. Reasoning functions. J. Symb. Deontic Logic as Normative
    • Handbook Norms Change Permission Institution Agency AGM 85 Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 2.1 - How to revise a set of norms? BQ Do the AGM postulates hold for norm revision? E.g., success postulate criticized in belief revision GQ Does AGM offer a satisfactory framework for norm revision? Role of minimal change in norm change Revision of conditional norms Example If we have {(￿, a), (a, b)} and we have that c is an exception to the obligation to do b, then we need to have b ￿∈ out(N, c). Two solutions seem to be {(¬c, a), (a, b)} or {(￿, a), (a ∧ ¬c, b)}. GQ What triggers the change of a norm? AGM says a new or removed norm GQ Do general patterns in the revision of norms exist? If so, how to formalize them? GQ Derogation is contraction? How about annulment? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 2 - How do norms change? GQ How to formalize the evolution of (social) norms? E.g., norm about phone calls during meetings No persons who change the norm like in legal code Change of social norm triggered by norm violations Social delegation cycle: relates agent desires and social goals GQ How to merge sets of norms? Problem only recently addressed in the literature. GQ What is norm merging? 1 Merging norms that belong to the same normative system 2 Merging norms that belong to different systems Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyMerging norms: same normative system Example (Cholvy & Cuppens 1999) An organization that works with secret documents has two rules. R1 = It is obligatory that any document containing some secret information is kept in a safe, when nobody is using this document. R2 = If nobody has used a given document for five years, then it is obligatory to destroy this document by burning it. To derive a contradiction we need to add an IC that keeping a document and destroying it are contradictory actions ⇒ The notion of coherence between norms can involve information other than norms. Is this a genuine problem of merging norms? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyMerging norms: different systems Example (Gr´goire, 2004) e The Belgian-French bilateral agreement preventing double taxation: Resident in B ∧¬ France State Worker ⇒ Taxable in B Resident in B ∧¬ Work in France ⇒ Taxable in B Belief merging would simply give a set containing the two initial rules. But if someone is not a State worker in France (but works in France), we would get contradictory requirements from the Belgium and the France parties. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 2.3 - How to merge sets of norms? Belief merging (a.k.a. belief fusion) is a ‘generalization’ of belief revision. Belief merging dedicated to the combination of information coming from different (conflicting) sources. GQ Can belief merging offer a satisfactory framework for norms? GQ Do distinctions in belief merging correspond to something? Belief merging distinguishes majoritarian and egalitarian operators. They try to capture the intuitions that often guide the aggregation of individual preferences into a social one. These intuitions seem to have nothing to say when we try to model the merging of sets of norms. GQ What are alternative frameworks for norm merging? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 3 - Permissive norms BQ How to solve the following problem with permissions: Example (Ross’ paradox) Px implies P(x ∨ y ), e.g., if a person is permitted to smoke, he is also permitted to smoke or kill. BQ How to formalize free choice permissions: Example (Free choice permission) P(x ∨ y ) implies Px ∧ Py , e.g., “You may either sleep on the sofa-bed or sleep on the guest room bed” implies “You may sleep on the sofa-bed and you may sleep on the guest room bed”. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 3 - Permissive norms BQ Is permission unilateral (Op → Pp) or bilateral (Pp → ¬Op)? BQ What is difference between weak (negative, tacit) and strong (positive, implicit and explicit) permission? von Wright (1951): Permission is dual of obligation Px =def ¬O¬x (weak permission) Negative permission defined without permissive norms Px =def ¬x ￿∈ out(N, A) (weak permission) BQ Are permissive norms necessary for normative systems? Example (Bulygin 1986) Permissive norms are needed when there multiple authorities. If a higher authority permits you, a lower authority cannot prohibit it. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 3 - Permissive norms GQ What is the difference between permission and authorization? Example (Makinson1986) A priest may be authorized to marry people, in the sense that his actions may have that effect, without being permitted to do so. GQ What is a right? Example If you have the right to enter, then a guard must let you in. Concept of permission is multi-faceted. Problem 3. How to relate various kinds of permissions and relate them to obligations? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 3 - Permissive norms GQ How to formalize relation permissions and permissive norms? Something is positively permitted iff the code explicitly presents it. However, permissions follow from explicit ones. GQ How do permissive norms influence obligations? Obligations imply permissions, but how about the other way? GQ What is role of permissive norms in norm change / merging? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 3 - Permissive norms Example Consider a set G consisting of the norm (work, tax), and a set P consisting of (18y, vote). Does it follow from our mini-code that voting is permitted on condition of being employed? In one sense yes: there is nothing in the code that forbids it (negative permission). In another sense no: a person may be employed at the age of 17 and not covered by the explicit permission (positive static permission). But in another sense, yes again. For if we were to forbid a person to vote on the condition of being employed we would be creating an incoherence in the code, in its application to people who are both employed and 18 or over (dynamic positive permission). Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 3 - Permissive norms Kinds of permissions used in different circumstances: Negative permissions seem to answer to the needs of the citizen, who needs to now that his action is not forbidden. Since it’s difficult to establish these kinds of permissions (also because of ambiguities in the code), positive static permissions seem to suit better the need to know what is explicitly permitted. Lastly, dynamic permissions are the answers to the need of the legislator, who need to anticipate the effect of changing an existing corpus of norms by adding a prohibition. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 4 - What is the role of constitutive norms? Normative system contains definitions, expressions of purposes, conceptual rules, etc.: we call them constitutive norms. Example “An act of theft is punished by a prison sentence not exceeding 5 years or a fine.” GQ What is the role of constitutive norms? (What can’t we do if we have only regulative and permissive norms) Meaning postulates and intermediate concepts Creation of social reality, counts-as conditionals Legal and institutional powers of agents Defining the way normative systems can change Interpretation issues ... Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyExample meaning postulates and intermediate concepts “An act of theft is punished by a prison sentence not exceeding 5 years or a fine.” “Someone commits an act of theft if that person has taken a movable object from the possession of another person into his own possession with the intention to own it, and if the act occurred without the consent of the other person or some other legal authorization.” A person in the sense of the law is a human being that has been born. A movable object is any physical object that is not a person or a piece of land. A movable object is in the possession of a person if that person is able to control its uses and location. The owner of an object is within the limits of the law entitled to do with it whatever he wants, namely keep it, use it, transfer possession or ownership of the object to another person, and destroy or abandon it. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyMeaning postulates and intermediate concepts Intermediate concepts link legal terms to terms that describe natural facts. GQ Why are there intermediate concepts? Wedberg & Ross (1950s): a legal term like ’ownership’ (x is the owner of y at time t) serves the purpose of economy of expression of a set of legal rules. To regulate normative change and legal interpretation by judge To resolve conflicts Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 4 - Meaning postulates and intermediate concepts Example (Intermediates to resolve conflicts) Suppose that (￿, ¬dog) forbids dogs on the premises There is also a higher order principle that no blind person may be required to give up her guide dog. The judge cannot change the statute. What he can do is to conclude that the statute does not apply to guide dogs. So the statute must be re-interpreted as reading (￿, ¬tdog) with the additional intermediate (dog ∧ ¬guidedog, tdog) ∈ T , and thus no conflict arises for the case of blind persons that want to keep their guide dog. GQ What is the exact process of creating and modifying theoretical terms in order to resolve conflicts? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 4 - Constitutive norms GQ How do regulative, permissive and constitutive norms interact? Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 4 - Constitutive norms There is no consensus on the logic of counts-as conditionals, probably due to the fact that the concept is not studied in depth yet. For example, the adoption of the transitivity rule T for Jones & Sergot’s logic is criticized by Artosi, Rotolo & Vida: They claim that non-monotonicity is a crucial feature of count-as conditionals: Example: Suppose that in an auction if the agent x raises one hand, this may count as making a bid. It is clear that this does not hold if x raises one hand and scratches his own head. Leon van der Torre Deontic Logic as Normative Reasoning
    • Handbook Norms Change Permission Institution AgencyProblem 5 - Where does deontic logic meet game theory? Leon van der Torre Deontic Logic as Normative Reasoning
    • Where Does Deontic Logic Meet Game Theory? Leon van der Torre EASSS 2012 1
    • Why do we need deontic detachment?•  Thomason 1980: –  Context of justification (e.g. a judge, looking back) –  Context of deliberation (normative decision making)•  Consider Chisholm’s paradox without DD –  Day after tomorrow you must help neighbors –  Tomorrow you must tell•  S.O. Hansson: If you don’t tell & you help, then you never received “moral cue for action” to tellR. Thomason, “Deontic logic as founded on tense logic." In New Studies in 2Deontic Logic, R. Hilpinen, ed., D. Reidel, Dordrecht, 1981, pp. 165-176.
    • Quick History Practical Reasoning•  Aristotle: theoretical & practical reasoning•  40s Practical Reas. splits into two branches –  Classical decision and game theory in economics –  Bounded rationality and means-end reasoning in AI•  90s unification in multi-agent systems –  Russell and Norvig’s “modern approach” to AI –  Shoham’s DTGT foundations of MAS•  Deontic logic: obligations treated as goals in AI –  Distinguished from theory of normative systems 3
    • NORMAS•  Deontic logic and normative reasoning are one•  They should be studied both from a bounded rationality or AI perspective, and from a DTGT or economic perspective: they should be studied from a multi-agent perspective –  The DL handbook: “a modern approach”•  Consequences: –  Action and time: DTGT model –  Normative decision theory: BOID 4
    • NORMAS•  State of art in action and time Deontic Logic•  Same situation as in preference-based DL: –  Semantics is relatively clear and intuitive –  Languages and proof systems are to be improved•  All you need to know action and time in DL is in Horty’s “Agency and Deontic Logic” (2001)J. Broersen, L. van der Torre: What an Agent Ought To Do. Artificial Intelligence 5and Law 11(1): 45-61 (2003)
    • DTGT vs Deontic Logic•  Models are the same, choices over branching time, with only one distinction: –  DTGT: each agent has its own utility –  DL: there is a single global utility•  Savage’s sure thing principle plays central role•  There exist more detailed DTGT models and we can use them as more detailed DL models 6
    • The Gambling Problem•  Obligation to be and obligation to do –  Not interdefinable•  Granularity of indeterministic actions 7
    • The Driving Example•  Assume a collision•  Dominance act utilitarianism –  There was no dominant action•  Orthodox perspective on the agents ought –  You should have changed lanes•  The issue of moral luck (Horty, 2001, p.121). –  Orthodox ought: agent looks back in time –  Dominant: agent may regret, but not be blamed•  What if utility of 5 and 3? (Whiff and Poof) 8
    • Procrastinates Choice•  Field = agent’s reasoning is limited to subtree –  “Intention” in BDI agent theory?•  Possibilist interpretation –  It is possible to write the review, so he accept it•  Strategic interpretation –  He will not write it, so he should not accept it 9
    • A Challenge•  The bystander effect: –  One person watches a child in water and saves it –  One hundred people watch child and it drowns•  How to reason about this in deontic logic? –  If deontic logic is going to play a role in ethics, it has to say something about examples like this 10
    • A Challenge•  The bystander effect: –  One person watches a child in water and saves it –  One hundred people watch child and it drowns•  How to reason about this in deontic logic? –  If deontic logic is going to play a role in ethics, it has to say something about examples like this•  Introduce normative systems in DTGT model –  Needs an explicit mechanism of norm creation 11
    • Summary•  How to combine deontic logic & game theory? –  Mechanism design, norm is a mechanism –  Introduce norms in Horty’s model•  Computational methods for deontic logic –  Algorithms, complexity, proof systems 12