3. Some Norms
• You are obliged to return the book in 2 weeks
• Role X is permitted to access resource Y
• Actions of agent X count as actions of agent Y
• Advocates should act in the interest of their clients
• Violations should be sanctioned
• Don’t do to them what you don’t want them to do to you
• People should know the law
• Every adult has the right to vote
• Make your homework!
• We start at 09:00
3
4. Merriam-Webster Online Dictionary
• ‘normative’: ‘conforming to or based on norms’,
– as in normative behavior, normative judgments
– not: ‘of, relating to, or determining norms or standards’, as in
normative tests, or ‘prescribing norms’, as in normative rules
of ethics or normative grammar.
• ‘norm’: ‘a principle of right action binding upon the
members of a group and serving to guide, control, or
regulate proper and acceptable behavior’.
– Not: ‘an authoritative standard or model’, ‘an average like a
standard, typical pattern, widespread practice or rule in a
group’, and various definitions used in mathematics.
4
5. Normative Systems (1971)
“When a deductive correlation is such that the
first sentence of the ordered pair is a case and
the second is a solution, it will be called
normative. If among the deductive correlations of
the set there is at least one normative
correlation, we shall say that the set has
normative consequences. A system of sentences
which has some normative consequences will be
called a normative system.”
C. Alchourron and E. Bulygin. Normative Systems. Springer, 1971.
5
6. Normative Systems (1993)
• Normative systems are “systems in the behavior
of which norms play a role and which need
normative concepts in order to be described or
specified”
J.-J. Meyer and R. Wieringa. Deontic Logic in Computer Science: Normative System Specification. John Wiley & Sons, 1993.
• Many distinct notions of “normative systems”
– Social expectation, legal law, linguistic imperative…
• Role of norms in computer science is changing
– Solutions based on multiagent systems increasing
6
7. Layout
• Challenges due to uncertainty and imprecision
in normative reasoning and decision making
1. Normative reasoning in computer science
2. Imprecision & uncertainty normative systems
3. Decision making in normative systems
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 7
9. 2005: Normative MultiAgent Systems
• are multiagent systems with normative systems
• in which agents can decide whether to follow the
explicitly represented norms, and
• the normative systems specify how and in which
extent the agents can modify the norms.’’
G. Boella, L. van der Torre, H. Verhagen, Introduction to normative multiagent
systems. Computational and Mathematical Organization Theory, double
special issue on normative multiagent systems, 2006.
“The normchange definition”
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 9
10. 2007: Normative MultiAgent System
• is a multiagent system organized by means of
• mechanisms to represent, communicate,
distribute, detect, create, modify, and enforce
norms, and
• mechanisms to deliberate about norms and
detect norm violation and fulfillment.”
G. Boella, L. van der Torre, and H. Verhagen, Introduction to the special issue
on normative multiagent systems. Autonomous Agents and MultiAgent
Systems, Aug. 2008.
“The mechanism design definition”
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 10
11. Layout
• Challenges due to uncertainty and imprecision
in normative reasoning and decision making
1. Normative reasoning in computer science
2. Imprecision & uncertainty normative systems
3. Decision making in normative systems
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 11
12. Norms: Certain and Precise?
• Norms are rules to distinguish right from wrong.
• Dilemmas: when may we shoot down a hijacked plane?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 12
13. Violations: Certain and Precise?
• Norms describe what counts as a violation.
• Budget deficit > 3 % of GDP is a violation?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 13
14. Design: Uncertain and Imprecise
• Norms are rules to guide, control or regulate behavior.
• What are the effects of new norms on behavior?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 14
15. Guide, Control, Regulate Behavior
What’s new with respect to constraints?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 15
16. Guide, Control, Regulate Behavior
Ways to deal with violations, representation of permissive norms,
the evolution of norms over time (in deontic logic), the relation
between the cognitive abilities of agents and the global properties
of norms, how agents can acquire norms, how agents can violate
norms, how an agent can be autonomous (in normative agent
architectures and decision making), how norms are created by
legislator, emerge spontaneously or are negotiated among agents,
how norms are enforced, how constitutive or counts-as norms are
used to describe institutions, how norms are related to social and
legal concepts, how norms structure organizations, how norms
coordinate groups and societies, how contracts are related to
contract frames & contract law, how legal courts are related, how
normative systems interact, …
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 16
17. Normative Reasoning
Norms are rules to guide, control or regulate behavior.
Infrastructure for open communities.
Reasoning in open environments.
Trust and reputation.
More than distinguishing right from wrong.
More than describing what counts as a violation.
More than constraints.
Uncertain and imprecise.
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 17
18. Security and Reliability
• Norms make systems more secure & reliable.
• How to prevent over-regulation?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 18
19. Trust
• Norms build up trust in international trade.
ESCROW
• Which mechanisms for electronic commerce?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 19
20. Example: ESCROW
• Norms describing coordination
– Agent communication: sales agreement
– Agent transaction: payment for shipment
• Norms describing fees and compensation
– ESCROW gets percentage (from seller or buyer)
– Return merchandize within 5 days
• Norms ensure fulfillment of some obligations
– Seller is paid for goods, or goods will be returned
– (under assumption of trustworthiness of ESCROW)
• Some obligations can still be violated:
– Seller does not ship goods, shipper unreliable, …
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems
4. Norm change 20
21. Virtual Communities
• Social norms are emerging in Second Life.
• How to prevent exclusion from communities?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 21
22. Autonomous Systems
• aut- + nomos: making ones own norms.
• How to define global policies about local ones?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 22
23. Applications Normative Reasoning
Norms are rules used to control or regulate behavior.
Risk management for computer security.
Designing trust mechanisms for electronic commerce.
Recognition of emerging social norms in Second Life.
Local and global policies for autonomous systems.
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 23
24. Layout
• Challenges due to uncertainty and imprecision
in normative reasoning and decision making
1. Normative reasoning in computer science
2. Imprecision & uncertainty normative systems
3. Decision making in normative systems
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 24
25. Examples: How to decide …
• … whether to fulfill or violate a norm?
• … whether behavior counts as a violation?
• … whether an excuse for a violation is acceptable?
• … whether a violation should be sanctioned?
• … whether to accept a norm / contract / … ?
• ... whether to accept sanctions of a norm?
• … which contract to propose / norm to create?
• … whether an agent conforms to the norm?
• … whether a procedure is compliant to the norm?
• … on a constitution for Europe?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 25
26. Basis: Decision Making With Norms
• Introducing obligations in agent architectures
•
tl O tl
tl al cl
hl act
obs B I P
D
obligations as normative goals
• Introducing obligations in agent programming
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 26
27. Exercise: When Violate a Norm?
• When you are not able to fulfill it
• When the sanction is too low
• When the probability to be caught is too low
• …
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 27
28. Exercise: When Violate a Norm?
• When you are not able to fulfill it
• When the sanction is too low
• When the probability to be caught is too low
• When you don’t know the norm
• When there is another more important desire
• When there is another more important obligation
• When you have a good excuse
• When you like to take risks
• When other agents violate the norm
• When the norm does not make sense
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 28
29. When to Violate/Fulfill: Challenge
• Multiagent structure of a normative system
– Many agents involved in agent interaction
– Interaction is highly structured
• Normative system regulates their roles
– Powers of roles described by counts-as norms, and
– Their behavior is regulated by procedural norms.
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 29
30. Norm Compliance
• Attacker = agent trying to profit from violation
– Violation undetected, no sanction, or less than profit
normative system Attacker / adversary
$$
• Anticipate rather than regiment (mechanism design)
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 30
31. Profile NS of Attacker A (for Op)
1. NS desires p (“your wish is my command”).
2. Absence of p is considered as violation of A.
! Anderson’s reduction of deontic logic to modal logic.
3. NS desires that there are no violations.
4. If violation, then NS is motivated to sanction.
5. NS does not like to sanction.
6. A does not like being sanctioned.
7. NS has the power to count absence of p as violation.
8. NS has the power to enforce sanction.
Example: You are obliged to return the book in 2 weeks
Question: Why do we need all of them? Is it complete?
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 31
32. When to Accept a Contract?
normative system trustee
$$
trustor
• Decision making of trustor taking profiles of
trustee and normative system into account
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 32
33. Challenge: When to Accept a Norm
•Immanuel
Kant,
Metaphysics
of Morals,
1785
• “Act only according to that
maxim whereby you can at
the same time will that it
should become a universal
law.”
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 33
34. Emergence of Norms
• Don’t do to them, what you don’t want them to
do to you
• Social delegation cycle
– Agent desires
– Merging: Social goal
– Planning: Norm
– Acceptance: Agent desires
• I accept if you fulfill the norm on the
assumption that all others fulfill it
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 34
35. Autonomous Systems
• Legal institutions in context of legal institutions
– Norm dynamics described by counts-as norms
• Strong permissions in norm creation (Bulygin)
• Global policies about local policies
– Von Wright: transmission of will by nested norms
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 35
36. Hierarchical Normative Systems
Global authority Norm. 2
$$
Norm. 1
• Decision making of norm. system 1 taking profiles of
norm. system 2 and global authority into account
1. Normative reasoning in computer science
2. Uncertainty and imprecision in normative reasoning
3. Decision making in normative systems 36
37. Decision Making in NMAS
Norms are rules used to control or regulate behavior.
Norm conformance and compliance.
Multiagent structure of normative systems.
Norm acceptance for emergence of norms.
Hierarchical normative systems for autonomy.
37
38. Concluding Remarks
• Challenges due to uncertainty and imprecision
in normative reasoning and decision making
1. Normative reasoning in computer science
2. Imprecision & uncertainty normative systems
3. Decision making in normative systems
38
40. Further Reading
• Normative multiagent systems proceedings:
– Computational and Mathematical Organization Theory, double special
issue on normative multiagent systems, 2006.
– Normative Multiagent Systems, 18.03. - 23.03.2007, Schloss Dagstuhl,
Germany, 2007.
– Autonomous Agents and Multiagent Systems Journal, special issue on
normative multiagent systems, August 2008.
– 4rd International Workshop on Normative Multiagent Systems
(NORMAS2009), Dagstuhl, March 2009
• Introductions to the area and its challenges
– Ten philosophical problems in deontic logic
– Ten challenges for normative multiagent systems
– Ten guidelines for norms in computer science
40
41. Ten Philosophical Problems
1. How can deontic logic be reconstructed in accord with the philosophical position that
norms are neither true nor false?
2. When is a set of norms to be termed `coherent'?
3. How can deontic logic accommodate possible conflicts of norms? How can the
resolution of apparent conflicts be semantically modeled?
4. How do we reason with contrary-to-duty obligations which are in force only in case of
norm violations?
5. How to define dyadic deontic operators with regard to given sets of norms and facts?
6. How to distinguish various kinds of permissions and relate them to obligations?
7. How can meaning postulates and intermediate terms be modeled in semantics for
deontic logic reasoning?
8. How to define counts-as conditionals and relate them to obligations and permissions?
9. How to revise a set of regulations or obligations? Does belief revision offer a
satisfactory framework for norm revision?
10. Can the belief merging framework deal with the problem of merging sets of norms?
J. Hansen, G. Pigozzi and L. van der Torre, Ten philosophical problems in deontic logic, Dagstuhl DROPS proceedings 07122, 2007.
41
42. Ten Challenges for NorMAS
1. Tools for agents supporting communities in their task of recognizing, creating, and
communicating norms to agents.
2. Tools for agents to simplify normative systems, recognize when norms have
become redundant, and to remove norms.
3. Tools for agents to enforce norms. In a distributed approach, roles should be
defined for agents in charge of monitoring and sanctioning.
4. Tools for agents to preserve their autonomy.
5. Tools for agents to construct organizations.
6. Tools for agents to create intermediate concepts and normative ontology, for
example to decide about normative gaps.
7. Tools for agents to decide about norm conflicts.
8. Tools for agents to voluntarily give up some norm autonomy by allowing automated
norm processing in agent acting and decision making
9. Tools for conviviality.
10. Tools for legal responsibility of the agents and their principals.
G. Boella, L. van der Torre, H. Verhagen, Introduction to the special issue on normative
multiagent systems. Autonomous Agents and MultiAgent Systems, Aug 2008.
42
44. Guideline 1
1. Motivate which definition of normative
multiagent system is used.
• Norms explicitly represented in system
– (the ‘strong’ interpretation: too strict?)
• Norms explicitly represented in specification
– (the ‘weak’ interpretation: too general?)
– Norm compliance, norm implementation, ...
• Something else
44
45. Guideline 2
2. Make explicit why norms are a kind of (soft)
constraints deserving special analysis.
• Ways to deal with violations, representation of permissive norms, the
evolution of norms over time (in deontic logic), the relation between the
cognitive abilities of agents and the global properties of norms, how agents
can acquire norms, how agents can violate norms, how an agent can be
autonomous (in normative agent architectures and decision making), how
norms are created by legislator, emerge spontaneously or are negotiated
among agents, how norms are enforced, how constitutive or counts-as norms
are used to describe institutions, how norms are related to social and legal
concepts, how norms structure organizations, how norms coordinate groups
and societies, how contracts are related to contract frames & contract law,
how legal courts are related, how normative systems interact?
45
46. Guideline 3
3. Explain why and how norms can be changed at
runtime.
• E.g., legislators and voting on acceptance,
observe behavior and violations to modify…
46
47. Guideline 4
4. Discuss the use and role of norms as a
mechanism in a game-theoretic setting.
• D. Lewis “master and slave” game
• E. Bulygin “rex, minister and subject” game
• G. Boella c.s.: violation games, institutionalized
games, negotiation games, norm creation
games, control games Norms are rules specifying violation games.
47
48. Guideline 5
5. Clarify role of norms in the multiagent system.
• Norms guide (…) desired system behavior
• Norms are incentives to motivate agents
– Gneezy and Rustichini’s daycare example
• Norms organize systems
– Modularity, abstractions
48
49. Guideline 6
6. Relate the notion of “norm” to the legal,
social, or moral literature.
Five phases in normas design:
1.off-line norm design
2.norm representation
3.norm manipulation
4.social reality
5.moral reality
49
50. Guideline 7
7. Use norms not only to distinguish right from
wrong, but also to resolve dilemmas, and use
norms not only describe violations, but in
general to coordinate, organize, guide, regulate
or control interaction among agents.
50
51. Guideline 8
8. Distinguish norms from obligations, prohibitions
and permissions.
• Deontic logic: logical relations obligations, etc
– normative system is typically left implicit
• Two distinct philosophical traditions
– Von Wright: norms and normative propositions
– Alchourron: prescriptive and descriptive obligations
J. Hansen. Imperatives and Deontic Logic. PhD thesis, University of Leipzig, 2008.
51
52. Guideline 9
9. Use the deontic paradoxes only to illustrate the
normative multiagent system.
1. A certain man should go to the assistance of his
neighbors,
2. If he goes, he should tell them he is coming
3. If he does not go, he should not tell them that he is
coming
4. He does not go.
{Oa,O(a ! t),¬a ! O(¬t),¬a} {Oa, a ! O(t),¬a ! O(¬t),¬a}
52
53. Guideline 10
10. Consider regulative norms in relation to other
kinds of norms and concepts.
• in relation to permissive norms, constitutive
norms, procedural norms, agents, roles, groups,
societies, rights, duties, obligations, time,
beliefs, desires, intentions, goals, roles, and
other kinds of norms and other social-cognitive
computer science concepts.
53
54. The Question
• Could (or should) “norms” play a similar role in
computer science like “service”, “contract” or “trust”?
– Since the use of norms is a key element of human social
intelligence, norms may be essential too for artificial agents
that collaborate with humans, or that are to display behavior
comparable to human intelligent behavior.
– Norms are thought to ensure efficiency at the level of the
multiagent system whilst respecting individual autonomy.
• We have to build more flexible normative multiagent
systems, and test them in practice, before we know
where they can be used best.
54
56. Objects, relationships, and roles
Each modelling language has its own reference ontology.
The duality of objects and relationships is present in all mod-
elling languages. E.g., variables and constants vs predicates
and relations.
But there is another basic ontological distinction: roles.
57. Roles, first appearance
Bachman and Daya 1977’s role DB data model:
“Records and n-tuples are role oriented. These deal with
employees, customers, patients, or students: role types. This
is in contrast with DB theory: each record should represent
all aspects of some one entity in the real world. The reason
for confusion is that neither the roles of the real world nor
the entities of the real world are a subset of the other”
58. “If members of a set are records of various types, large pieces
of redundant code had to be written in order to address
the semantically equivalent, but syntactically distinct fields
(items) of the members when iterating through a set.
By letting all the entity types involved play the same role and
by defining the fields as fields of the role, not of the types, it
became possible to process all members with the same piece
of code.”
59. Natural types vs roles
Sowa 1984’s conceptual structures distinguish between natu-
ral types “that relate to the essence of the entities” and role
types “that depend on an accidental relationship to some
other entity”: child depends on parent, pet on owner, etc.
Sowa: role types are subtypes of natural types. Child, Pet,
Quotient and Food are subtypes of the natural types Person,
Animal, Number and Physical-Substance.
60. Properties of roles in Object Orientation
from Steimann 2000
1. A role comes with its own properties and behavior. Hence,
it is a type. E.g., a director of a department commands
other members and makes buy-orders.
2. Roles depend on relationships: e.g., an employee is re-
lated to an employer, a student to a school, etc.
3. Objects of unrelated types can play the same role. E.g.,
both a person and an organization are customers.
61. 4. An object may play different roles. E.g., a person can be
a student and an employee.
5. An object may play the same role several times. E.g., a
person can hold several employments.
6. An object may acquire and abandon roles dynamically.
E.g., a person can acquire the role of student or of em-
ployee.
7. The sequence in which roles may be acquired and relin-
quished can be subject to restrictions. E.g., a person
becomes a teaching assistant only if it is a student.
62. 8. Roles can play roles. E.g., an employee can be a project
leader, a role of the employee.
9. A role can be transferred from one object to another.
E.g., the salary of an open position may be specified
independently of the person that will be employed.
10. The state of an object can vary depending on the role in
which it is being addressed: this should be viewed as a
separate instance of the object. E.g., an employee has
an address per job and also a private one.
11. If an object plays several of roles simultaneously, it re-
sponds according to the role in which it is being ad-
dressed. E.g., an person gives the address of the em-
ployee it is playing.
63. 12. Roles restrict access. This corresponds to an object hav-
ing different perspectives, facets, or aspects. E.g., the
private phone number of an employee can be invisible
when the person is playing the employee role.
13. Different roles may share structure and behavior: role
definitions inherit from each other. E.g., the role first
year student can inherit the behavior of giving exams from
the role student.
14. An object and its roles share identity. Since roles do not
exist by themselves they cannot have an identity.
15. An object and its roles have different identities. E.g., the
number of passengers of an airline can be greater than
the number of person who travelled with it.
64. Possible views of roles
1. Roles as named places in relationships.
2. Roles as specifications or generalizations of natural types.
3. Roles as adjunct instances.
65. Role as a named place in a relationship
If two places of a relationship are of the same type, then
the type would not suffice to distinguish them. Role names
should serve to identify the places.
Every place of a relationship has a role name in the ER model
(and UML).
For example, the is-parent-of (or is-child-of ) relationship is
sufficiently characterized by the role names parent and child.
However, it fails to account for the fact that roles come with
their own properties and behavior.
66. Roles as specializations
If roles are types, how are the roles and the natural types
related?
A role type is more specific than the natural type of its role
players: if Customer and Supplier are subtypes of Person.
But, not only persons, but also organizations like companies,
etc., can be customers and suppliers. However, declaring
Customer and Supplier as subtypes of both Person and Or-
ganization is a problem.
67. Roles as generalizations
If roles are not subtypes, could they be supertypes?
No, if roles are more specific than the types of their players.
But this is false, anyway: a person has many properties not
required of a customer or supplier. Rather, being a customer
or supplier imposes its required properties on persons and
organizations, making the former supertypes of the latter.
From the extensional point of view, roles are supertypes stat-
ically, while dynamically they are subtypes.
68.
69. Roles as adjunct instances
Roles as independent types the instances of which are carriers
of role specific state and behavior, but not identity.
An object and its roles are related by a special played-by rela-
tion, and an object and its roles form an aggregate (subject).
Picking up a role corresponds to the creation of a new in-
stance of the corresponding role type and its integration into
the compound.
70. Ontological properties of roles
Following Sowa, Guarino and Welty 2002 presented an onto-
logical distinction between role and natural types.
A role is a concept that is:
• founded: the individuals of a role stand in relation to
other individuals,
• lacks semantic rigidity: they can enter and leave the ex-
tent of the concept without losing their identity.
71. A natural type, on the other hand, is characterized by:
• semantic rigidity: an individual of a natural type cannot
drop this type without losing its identity,
• lack of foundation: for an individual to belong to that
type it is not required that it stands in relationship to
others.
E.g., Person is a natural type, Student is a role.
Adolescent and Adult are states or phases, lacking both se-
mantic rigidity and foundation.
72. Dependence of roles
Fine 1995’s notion of dependence: “to say that an object x
depends upon an F is to say that an F will be ineliminably
involved in any definition of x”.
Masolo et al. 2004’s definitional dependence: e.g., the def-
inition of the concept of student makes reference not to a
specific school but to the concept of school, etc.
73. Handbook Norms Change Permission Institution Agency
Deontic Logic as Normative Reasoning
Leon van der Torre
Individual and Collective Reasoning Group
Computer Science and Communication
University of Luxembourg
EASSS 2012 (Valencia)
May 2012
Leon van der Torre Deontic Logic as Normative Reasoning
74. Handbook Norms Change Permission Institution Agency
1 Handbook
2 Problem 1 - In what sense are obligations different from norms?
3 Problem 2 - How do norms change?
4 Problem 3 - How to relate various kinds of permissions?
5 Problem 4 - What is the role of constitutive norms?
6 Problem 5 - Where does deontic logic meet game theory?
Leon van der Torre Deontic Logic as Normative Reasoning
75. Handbook Norms Change Permission Institution Agency
Volume 1: Deontic logic and imperatives
A. BACKGROUND
1. R. Hilpinen and P. McNamara, Deontic logic: history, overview.
2. J. Hansen, Imperative logic: history, overview.
B. DEONTIC PROBLEMS
3. S. Hansson, Varieties of permission.
4. M. Sergot, Contrary-to-duty norms.
5. L. Goble, Prima facie norms, normative conflicts and dilemmas.
6. M. Sergot, The theory of normative positions.
7. A.J.I. Jones and D. Grossi, Constitutive norms and counts as conditionals.
8. P. McNamara, Logics for supererogation and allied normative concepts.
C. CANDIDATES FOR A NEW STANDARD
9. S. Hansson, Alternative semantics for deontic logic.
10. X. Parent and L. van der Torre, Input/output logic.
11. J. Hansen, Imperativist approach to deontic logic.
12. L. Lindahl and J. Odelstad, TJS. Normative systems with intermediaries.
Leon van der Torre Deontic Logic as Normative Reasoning
76. Handbook Norms Change Permission Institution Agency
Volume 2: Norms in systems, and interactions
D. DYNAMICS
1 ?, Time and deontic logic. 2. J. Broersen, J. Horty and J.-J. Ch. Meyer,
Actions and deontic logic. 3. G. Boella, G. Pigozzi and L. van der Torre,
Normative system change. 4. J. van Benthem and F. Liu, Deontic logic and
changing preferences. 5. D. Gabbay, Reactive approaches.
E. INTERACTIONS
6 R. Thomason, The formalization of practical reasoning: problems and
prospects. 7 B. Kooi and A. Tamminga, Deontic logic and game theory. 8 K.
van Fintel and T. Gillies, Deontic logic and natural language. 9 G. Sartor, G.
Governatori, and A. Rotolo. Deontic logic and legal reasoning.
F. INTEGRATION INTO SYSTEMS
10. D. Garg, Modalities for access control. 11. J. Lobo, E. Lupu and A. Russo,
Distributed systems management. 12. F. Cuppens and N. Cuppens-Boulahia,
Obligations for usage control. 13. G. Aucher and G. Boella, Privacy and
epistemic obligations. 14. R. Parikh, L. Oldeloohuisl, and C. Baskent,
Epistemic norms. 15. G. Governatori, Rule, Norms and compliance.
Leon van der Torre Deontic Logic as Normative Reasoning
77. Handbook Norms Change Permission Institution Agency
In what sense are obligations different from norms?
BQ Can norms have truth values? (Jorgensen, 1938) Imperatives,
promises, legal statutes, and moral standards are usually not
viewed as being true or false. E.g.: “John, leave the room!”
BQ Is a logic of norms possible? Without truth, norms cannot:
be premise or conclusion in an inference,
be termed consistent or contradictory, or
be compounded by truth-functional operators.
BQ Can deontic logic be built on descriptive obligations only?
(Makinson, 1998) turns the Bad Questions into a Good Question:
Problem 1. How can deontic logic be reconstructed in accord with
the philosophical position that norms are neither true nor false?
Explained in following slides in three steps.
Leon van der Torre Deontic Logic as Normative Reasoning
78. Handbook Norms Change Permission Institution Agency
Step 1: Traditional Deontic Logic, axiomatic (1951)
In what sense are obligations different from norms?
SDL = normal modal logic of type KD
extends the propositional tautologies with the axioms
K : O(x → y ) → (Ox → Oy ) and D : ¬(Ox ∧ O¬x), closed
under modus ponens x, x → y /y and Necessitation x/Ox
Prohibition & permission defined Fx = O¬x and Px = ¬O¬x.
BQ Which representation for conditional norms O(x|a)?
O(x|a) = O(a → x), or
O(x|a) = a → Ox, or something else
BQ Factual or deontic detachment?
Factual detachment: (O(x|a) ∧ a) → Ox
Deontic detachment: (O(x|a) ∧ Oa) → Ox
Leon van der Torre Deontic Logic as Normative Reasoning
79. Handbook Norms Change Permission Institution Agency
Step 2: Traditional Deontic Logic, possible worlds (1969)
In what sense are obligations different from norms?
DSDL3 = dyadic modal logic
Kripke models W , ≥ where ≥ is total pre-order
O(x|a) iff x holds in all the most preferred a worlds
Axiomatized by Lewis (counterfactuals, 1973) and Spohn
BQ Which representation for conditional norms O(x|a)?
BQ Which logical relations hold among conditional norms?
BQ Factual or deontic detachment?
Factual detachment: (O(x|a) ∧ a) → Ox
Deontic detachment: (O(x|a) ∧ Oa) → Ox
BQ How are logical relations and detachment related?
For example, transitivity and deontic detachment
Leon van der Torre Deontic Logic as Normative Reasoning
80. Handbook Norms Change Permission Institution Agency
Step 3: Modern Deontic Logic, iterative (1998)
In what sense are obligations different from norms?
Makinson’s iterative detachment approach
Normative code N is pairs of propositional formulas (a, x)
Gross output: out(N, A) = apply N as rules on A
E.g., E0 = ∅,En+1 = En ∪ {x|(a, x) ∈ N, En ∪ A implies a}
Net output: consistency constraint for output
E.g., En+1 = En ∪ {x|(a, x) ∈ N, En ∪ A implies a} if cons.
En otherwise
BQ Which representation for conditional norms O(x|a)?
GQ How to detach obligations from conditional norms?
Factual detachment: ((a, x) ∧ a) → Ox
Deontic detachment: ((a, x) ∧ Oa) → Ox
BQ How are logical relations and detachment related?
Leon van der Torre Deontic Logic as Normative Reasoning
81. Handbook Norms Change Permission Institution Agency
Candidates for a New Standard
Makinson’s iterative detachment approach
GQ Which approaches do not fit into this general framework?
GQ What are the alternatives to this iterative framework?
Algebraic Programming
Non-Monotonic
a:b/Oc
Reactive
Iterative
a in out(C,b)
Diagnostic
q / ¬V(n) ! p Labeled Imperativistic
Op!p,Oq!q ! O(p/q)!p,!q !p,!q ! O(p/q)
Dynamic
Input/Output
K O
Leon van der Torre Deontic Logic as Normative Reasoning
82. Handbook Norms Change Permission Institution Agency
Problem 1 - Some Good Questions
Makinson’s iterative detachment approach
GQ When is a normative system coherent?
GQ When is a norm redundant?
GQ When are two normative systems equivalent?
GQ How are the three questions above related?
GQ Which meta-logical properties can or should hold?
Law2case principle
Strong case2law principle
GQ How to prevent the pragmatic oddity
without creating the drowning problem?
Leon van der Torre Deontic Logic as Normative Reasoning
83. Handbook Norms Change Permission Institution Agency
Problem 2 - How do norms change?
Leon van der Torre Deontic Logic as Normative Reasoning
84. Handbook Norms Change Permission Institution Agency
Problem 2 - How do norms change?
Problem 1 - In what sense are obligations different from norms?
GQ How is obligation change different from norm change?
How to formalize the relation between them?
Obligations can change while normative system remains same
Detachment of obligation, delegation, etc
A relatively clear and well studied subject (in 70s and 80s)
Norm change: the new challenge (workshops 2007, 2010)
Norm revision and contraction, e.g. change of legal code
Norm evolution, e.g. change of social norms
Merging normative systems, e.g. merger of companies
Leon van der Torre Deontic Logic as Normative Reasoning
85. Handbook Norms Change Permission Institution Agency
Problem 2 - How do norms change?
Problem 2.1. How to revise a set of norms?
Alch´urron Makinson [1981] study changes of legal code:
o
addition
amendment: revision
derogation: contraction
Legal code is a non-empty and finite set of propositions.
Starting point of belief revision (with G¨rdenfors), a.k.a. AGM
a
AGM 1985: minimality represented by recovery postulate
More advanced variants, e.g. for iterated belief revision
Also some issues with prioritized norms we do not consider
Defeasible deontic logic derived from non-monotonic logic
Leon van der Torre Deontic Logic as Normative Reasoning
86. Handbook Norms Change Permission Institution Agency
AGM 85
C. E. Alchourrón, P. Gärdenfors, and D. Makinson. On the logic of theory change: Partial
meet contraction and revision Leon van der Torre Log., 50(2):510–530, 1985. Reasoning
functions. J. Symb. Deontic Logic as Normative
87. Handbook Norms Change Permission Institution Agency
AGM 85
Leon van der Torre Deontic Logic as Normative Reasoning
88. Handbook Norms Change Permission Institution Agency
Problem 2.1 - How to revise a set of norms?
BQ Do the AGM postulates hold for norm revision?
E.g., success postulate criticized in belief revision
GQ Does AGM offer a satisfactory framework for norm revision?
Role of minimal change in norm change
Revision of conditional norms
Example
If we have {(, a), (a, b)} and we have that c is an exception to
the obligation to do b, then we need to have b ∈ out(N, c). Two
solutions seem to be {(¬c, a), (a, b)} or {(, a), (a ∧ ¬c, b)}.
GQ What triggers the change of a norm?
AGM says a new or removed norm
GQ Do general patterns in the revision of norms exist?
If so, how to formalize them?
GQ Derogation is contraction? How about annulment?
Leon van der Torre Deontic Logic as Normative Reasoning
89. Handbook Norms Change Permission Institution Agency
Problem 2 - How do norms change?
GQ How to formalize the evolution of (social) norms?
E.g., norm about phone calls during meetings
No persons who change the norm like in legal code
Change of social norm triggered by norm violations
Social delegation cycle: relates agent desires and social goals
GQ How to merge sets of norms?
Problem only recently addressed in the literature.
GQ What is norm merging?
1 Merging norms that belong to the same normative system
2 Merging norms that belong to different systems
Leon van der Torre Deontic Logic as Normative Reasoning
90. Handbook Norms Change Permission Institution Agency
Merging norms: same normative system
Example (Cholvy Cuppens 1999)
An organization that works with secret documents has two rules.
R1 = It is obligatory that any document containing some secret
information is kept in a safe, when nobody is using this document.
R2 = If nobody has used a given document for five years, then it is
obligatory to destroy this document by burning it.
To derive a contradiction we need to add an IC that keeping a
document and destroying it are contradictory actions
⇒ The notion of coherence between norms can involve
information other than norms.
Is this a genuine problem of merging norms?
Leon van der Torre Deontic Logic as Normative Reasoning
91. Handbook Norms Change Permission Institution Agency
Merging norms: different systems
Example (Gr´goire, 2004)
e
The Belgian-French bilateral agreement preventing double taxation:
Resident in B ∧¬ France State Worker ⇒ Taxable in B
Resident in B ∧¬ Work in France ⇒ Taxable in B
Belief merging would simply give a set containing the two initial
rules. But if someone is not a State worker in France (but works in
France), we would get contradictory requirements from the
Belgium and the France parties.
Leon van der Torre Deontic Logic as Normative Reasoning
92. Handbook Norms Change Permission Institution Agency
Problem 2.3 - How to merge sets of norms?
Belief merging (a.k.a. belief fusion) is a ‘generalization’ of belief
revision. Belief merging dedicated to the combination of
information coming from different (conflicting) sources.
GQ Can belief merging offer a satisfactory framework for norms?
GQ Do distinctions in belief merging correspond to something?
Belief merging distinguishes majoritarian and egalitarian
operators.
They try to capture the intuitions that often guide the
aggregation of individual preferences into a social one.
These intuitions seem to have nothing to say when we try to
model the merging of sets of norms.
GQ What are alternative frameworks for norm merging?
Leon van der Torre Deontic Logic as Normative Reasoning
93. Handbook Norms Change Permission Institution Agency
Problem 3 - Permissive norms
BQ How to solve the following problem with permissions:
Example (Ross’ paradox)
Px implies P(x ∨ y ), e.g., if a person is permitted to smoke, he is
also permitted to smoke or kill.
BQ How to formalize free choice permissions:
Example (Free choice permission)
P(x ∨ y ) implies Px ∧ Py , e.g., “You may either sleep on the
sofa-bed or sleep on the guest room bed” implies “You may sleep
on the sofa-bed and you may sleep on the guest room bed”.
Leon van der Torre Deontic Logic as Normative Reasoning
94. Handbook Norms Change Permission Institution Agency
Problem 3 - Permissive norms
BQ Is permission unilateral (Op → Pp) or bilateral (Pp → ¬Op)?
BQ What is difference between weak (negative, tacit) and strong
(positive, implicit and explicit) permission?
von Wright (1951): Permission is dual of obligation
Px =def ¬O¬x (weak permission)
Negative permission defined without permissive norms
Px =def ¬x ∈ out(N, A) (weak permission)
BQ Are permissive norms necessary for normative systems?
Example (Bulygin 1986)
Permissive norms are needed when there multiple authorities. If a
higher authority permits you, a lower authority cannot prohibit it.
Leon van der Torre Deontic Logic as Normative Reasoning
95. Handbook Norms Change Permission Institution Agency
Problem 3 - Permissive norms
GQ What is the difference between permission and authorization?
Example (Makinson1986)
A priest may be authorized to marry people, in the sense that his
actions may have that effect, without being permitted to do so.
GQ What is a right?
Example
If you have the right to enter, then a guard must let you in.
Concept of permission is multi-faceted.
Problem 3. How to relate various kinds of permissions and relate
them to obligations?
Leon van der Torre Deontic Logic as Normative Reasoning
96. Handbook Norms Change Permission Institution Agency
Problem 3 - Permissive norms
GQ How to formalize relation permissions and permissive norms?
Something is positively permitted iff the code explicitly
presents it. However, permissions follow from explicit ones.
GQ How do permissive norms influence obligations?
Obligations imply permissions, but how about the other way?
GQ What is role of permissive norms in norm change / merging?
Leon van der Torre Deontic Logic as Normative Reasoning
97. Handbook Norms Change Permission Institution Agency
Problem 3 - Permissive norms
Example
Consider a set G consisting of the norm (work, tax), and a set P
consisting of (18y, vote). Does it follow from our mini-code that
voting is permitted on condition of being employed?
In one sense yes: there is nothing in the code that forbids it
(negative permission).
In another sense no: a person may be employed at the age of
17 and not covered by the explicit permission (positive static
permission).
But in another sense, yes again. For if we were to forbid a
person to vote on the condition of being employed we would
be creating an incoherence in the code, in its application to
people who are both employed and 18 or over (dynamic
positive permission).
Leon van der Torre Deontic Logic as Normative Reasoning
98. Handbook Norms Change Permission Institution Agency
Problem 3 - Permissive norms
Kinds of permissions used in different circumstances:
Negative permissions seem to answer to the needs of the
citizen, who needs to now that his action is not forbidden.
Since it’s difficult to establish these kinds of permissions (also
because of ambiguities in the code), positive static
permissions seem to suit better the need to know what is
explicitly permitted.
Lastly, dynamic permissions are the answers to the need of the
legislator, who need to anticipate the effect of changing an
existing corpus of norms by adding a prohibition.
Leon van der Torre Deontic Logic as Normative Reasoning
99. Handbook Norms Change Permission Institution Agency
Problem 4 - What is the role of constitutive norms?
Normative system contains definitions, expressions of purposes,
conceptual rules, etc.: we call them constitutive norms.
Example
“An act of theft is punished by a prison sentence not exceeding 5
years or a fine.”
GQ What is the role of constitutive norms? (What can’t we do if
we have only regulative and permissive norms)
Meaning postulates and intermediate concepts
Creation of social reality, counts-as conditionals
Legal and institutional powers of agents
Defining the way normative systems can change
Interpretation issues
...
Leon van der Torre Deontic Logic as Normative Reasoning
100. Handbook Norms Change Permission Institution Agency
Example meaning postulates and intermediate concepts
“An act of theft is punished by a prison sentence not exceeding 5
years or a fine.”
“Someone commits an act of theft if that person has taken a
movable object from the possession of another person into his
own possession with the intention to own it, and if the act
occurred without the consent of the other person or some
other legal authorization.”
A person in the sense of the law is a human being that has
been born.
A movable object is any physical object that is not a person or
a piece of land.
A movable object is in the possession of a person if that
person is able to control its uses and location.
The owner of an object is within the limits of the law entitled
to do with it whatever he wants, namely keep it, use it,
transfer possession or ownership of the object to another
person, and destroy or abandon it.
Leon van der Torre Deontic Logic as Normative Reasoning
101. Handbook Norms Change Permission Institution Agency
Meaning postulates and intermediate concepts
Intermediate concepts link legal terms to terms that describe
natural facts.
GQ Why are there intermediate concepts?
Wedberg Ross (1950s): a legal term like ’ownership’ (x is
the owner of y at time t) serves the purpose of economy of
expression of a set of legal rules.
To regulate normative change and legal interpretation by judge
To resolve conflicts
Leon van der Torre Deontic Logic as Normative Reasoning
102. Handbook Norms Change Permission Institution Agency
Problem 4 - Meaning postulates and intermediate concepts
Example (Intermediates to resolve conflicts)
Suppose that (, ¬dog) forbids dogs on the premises
There is also a higher order principle that no blind person may be
required to give up her guide dog. The judge cannot change the
statute. What he can do is to conclude that the statute does not
apply to guide dogs. So the statute must be re-interpreted as
reading (, ¬tdog) with the additional intermediate
(dog ∧ ¬guidedog, tdog) ∈ T , and thus no conflict arises for the
case of blind persons that want to keep their guide dog.
GQ What is the exact process of creating and modifying
theoretical terms in order to resolve conflicts?
Leon van der Torre Deontic Logic as Normative Reasoning
103. Handbook Norms Change Permission Institution Agency
Problem 4 - Constitutive norms
GQ How do regulative, permissive and constitutive norms interact?
Leon van der Torre Deontic Logic as Normative Reasoning
104. Handbook Norms Change Permission Institution Agency
Problem 4 - Constitutive norms
There is no consensus on the logic of counts-as conditionals,
probably due to the fact that the concept is not studied in
depth yet.
For example, the adoption of the transitivity rule T for Jones
Sergot’s logic is criticized by Artosi, Rotolo Vida: They
claim that non-monotonicity is a crucial feature of count-as
conditionals:
Example: Suppose that in an auction if the agent x raises one
hand, this may count as making a bid. It is clear that this
does not hold if x raises one hand and scratches his own head.
Leon van der Torre Deontic Logic as Normative Reasoning
105. Handbook Norms Change Permission Institution Agency
Problem 5 - Where does deontic logic meet game theory?
Leon van der Torre Deontic Logic as Normative Reasoning
106. Where Does Deontic Logic Meet
Game Theory?
Leon van der Torre
EASSS 2012
1
107. Why do we need deontic detachment?
• Thomason 1980:
– Context of justification (e.g. a judge, looking back)
– Context of deliberation (normative decision making)
• Consider Chisholm’s paradox without DD
– Day after tomorrow you must help neighbors
– Tomorrow you must tell
• S.O. Hansson: If you don’t tell you help, then
you never received “moral cue for action” to tell
R. Thomason, “Deontic logic as founded on tense logic. In New Studies in 2
Deontic Logic, R. Hilpinen, ed., D. Reidel, Dordrecht, 1981, pp. 165-176.
108. Quick History Practical Reasoning
• Aristotle: theoretical practical reasoning
• 40s Practical Reas. splits into two branches
– Classical decision and game theory in economics
– Bounded rationality and means-end reasoning in AI
• 90s unification in multi-agent systems
– Russell and Norvig’s “modern approach” to AI
– Shoham’s DTGT foundations of MAS
• Deontic logic: obligations treated as goals in AI
– Distinguished from theory of normative systems
3
109. NORMAS
• Deontic logic and normative reasoning are one
• They should be studied both from a bounded
rationality or AI perspective, and from a DTGT
or economic perspective: they should be
studied from a multi-agent perspective
– The DL handbook: “a modern approach”
• Consequences:
– Action and time: DTGT model
– Normative decision theory: BOID
4
110. NORMAS
• State of art in action and time Deontic Logic
• Same situation as in preference-based DL:
– Semantics is relatively clear and intuitive
– Languages and proof systems are to be improved
• All you need to know action and time in DL is in
Horty’s “Agency and Deontic Logic” (2001)
J. Broersen, L. van der Torre: What an Agent Ought To Do. Artificial Intelligence 5
and Law 11(1): 45-61 (2003)
111. DTGT vs Deontic Logic
• Models are the same, choices over branching
time, with only one distinction:
– DTGT: each agent has its own utility
– DL: there is a single global utility
• Savage’s sure thing principle plays central role
• There exist more detailed DTGT models and
we can use them as more detailed DL models
6
112. The Gambling Problem
• Obligation to be and obligation to do
– Not interdefinable
• Granularity of indeterministic actions
7
113. The Driving Example
• Assume a collision
• Dominance act utilitarianism
– There was no dominant action
• Orthodox perspective on the agent's ought
– You should have changed lanes
• The issue of moral luck (Horty, 2001, p.121).
– Orthodox ought: agent looks back in time
– Dominant: agent may regret, but not be blamed
• What if utility of 5 and 3? (Whiff and Poof)
8
114. Procrastinate's Choice
• Field = agent’s reasoning is limited to subtree
– “Intention” in BDI agent theory?
• Possibilist interpretation
– It is possible to write the review, so he accept it
• Strategic interpretation
– He will not write it, so he should not accept it
9
115. A Challenge
• The bystander effect:
– One person watches a child in water and saves it
– One hundred people watch child and it drowns
• How to reason about this in deontic logic?
– If deontic logic is going to play a role in ethics, it has
to say something about examples like this
10
116. A Challenge
• The bystander effect:
– One person watches a child in water and saves it
– One hundred people watch child and it drowns
• How to reason about this in deontic logic?
– If deontic logic is going to play a role in ethics, it has
to say something about examples like this
• Introduce normative systems in DTGT model
– Needs an explicit mechanism of norm creation
11
117. Summary
• How to combine deontic logic game theory?
– Mechanism design, norm is a mechanism
– Introduce norms in Horty’s model
• Computational methods for deontic logic
– Algorithms, complexity, proof systems
12