Presentation at the 15 International Conference on Artificial Intelligence and Law (ICAIL 2015), San Diego, June 2015 . In this paper we discuss some reasons why temporal logic might not be suitable to model real life norms. To show this, we present a novel deontic logic contrary-to-duty/derived permission paradox based on the interaction of obligations, permissions and contrary-to-duty obligations. The paradox is inspired by real life norms.

1. Thou Shalt is not You Will
Guido Governatori
ICAIL 2015
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 1/20

2. Deontic Logic History
1951 Georg Henrik von Wright. Deontic Logic.
1959 Soul Kripke. A Completeness Theorem in Modal Logic.
1962 Roderick Chisholm. Contrary-to-Duty Imperatives and Deontic Logic.
1965 William W. Hansson. Semantics for Deontic Logic.
1986 Marek Sergot et al. British Nationality Act as a Logic Program.
1991 Henning Herrestad. Norms and Formalization.
1992 Andrew J. Jones and Marek Sergot. Deontic logic in the representation of law:
Towards a methodology.
2015 Thou Shalt is not You Will
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 2/20

3. Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 3/20

4. Aim of the paper
Can we use (linear) temporal logic to verify the compliance of a system
with a set of norms?
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 4/20

5. Aim of the paper
Can we use (linear) temporal logic to verify the compliance of a system
with a set of norms?
Can we use (linear) temporal logic to model norms?
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 4/20

6. Background
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 5/20

7. Linear Temporal Logic 101 (Syntax)
• Xφ: at the next time φ holds;
• Fφ: eventually φ holds (sometimes in the future φ); and
• Gφ: globally φ holds (always in the future φ).
In addition we have three binary operators:
• φ U ψ (until): φ holds until ψ holds;
• φ W ψ (weak until): φ holds until ψ holds and ψ might not hold.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 6/20

8. Linear Temporal Logic 101 (Syntax)
• Xφ: at the next time φ holds;
• Fφ: eventually φ holds (sometimes in the future φ); and
• Gφ: globally φ holds (always in the future φ).
In addition we have three binary operators:
• φ U ψ (until): φ holds until ψ holds;
• φ W ψ (weak until): φ holds until ψ holds and ψ might not hold.
Interdefinability
• Fφ ≡ U φ,
• Gφ ≡ ¬F¬φ,
• φ W ψ ≡ (φ U ψ) ∨ Gφ
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 6/20

9. Linear Temporal Logic 102 (Semantics)
TS, σ |= a
s0
a
s1 s2 s3
TS, σ |= Xa
s0 s1
a
s2 s3
TS, σ |= a U b
s0
a ∧ ¬b
s1
a ∧ ¬b
s2
b
s3
TS, σ |= Fa
s0
¬a
s1
¬a
s2
a
s3
TS, σ |= Ga
s0
a
s1
a
s2
a
s3
a
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 7/20

10. Linear Temporal Logic 102 (Semantics)
TS, σ |= a
s0
a
s1 s2 s3
TS, σ |= Xa
s0 s1
a
s2 s3
TS, σ |= a U b
s0
a ∧ ¬b
s1
a ∧ ¬b
s2
b
s3
TS, σ |= Fa
s0
¬a
s1
¬a
s2
a
s3
TS, σ |= Ga
s0
a
s1
a
s2
a
s3
a
A formula φ is true in a fullpath σ iff it is true at the first element of the fullpath.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 7/20

11. Linear Temporal Logic 102 (Semantics)
TS, σ |= a
s0
a
s1 s2 s3
TS, σ |= Xa
s0 s1
a
s2 s3
TS, σ |= a U b
s0
a ∧ ¬b
s1
a ∧ ¬b
s2
b
s3
TS, σ |= Fa
s0
¬a
s1
¬a
s2
a
s3
TS, σ |= Ga
s0
a
s1
a
s2
a
s3
a
A formula φ is true in a fullpath σ iff it is true at the first element of the fullpath.
A formula is true in a state S
TS, s |= φ iff ∀σ: σ[0] = s, TS, σ |= φ.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 7/20

12. Obligation, Prohibition and Permission
Obligation A situation, an act, or a course of action to which a bearer is legally bound, and
if it is not achieved or performed results in a violation.
Prohibition A situation, an act, or a course of action which a bearer should avoid, and if it is
achieved results in a violation.
Permission Something is permitted if the obligation or the prohibition to the contrary does
not hold.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 8/20

13. Achievement vs Maintenance Obligations
• For an achievement obligation, a certain condition must occur at least once before the
deadline
‘Customers must pay before the delivery of the good, after receiving the invoice’
• For maintenance obligations, a certain condition must obtain during all instants before
the deadline:
‘After opening a bank account, customers must keep a positive balance until bank
charges are taken out’
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 9/20

14. Dura lex sed lex
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 10/20

15. A Privacy Act
Section 1: (Prohibition to collect personal medical information)
Offence: It is an offence to collect personal medical information.
Defence: It is a defence to the prohibition of collecting personal medical information,
if an entity immediately destroys the illegally collected personal medical
information before making any use of the personal medical information
Section 2: An entity is permitted to collect personal medical information if the entity acts
under a Court Order authorising the collection of personal medical information.
Section 3: (Prohibition to collect personal information) It is forbidden to collect personal
information unless an entity is permitted to collect personal medical
information.
Offence: an entity collected personal information
Defence: an entity being permitted to collect personal medical information.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 11/20

16. Making Sense of the Act
• Collection of medical information is forbidden.
• Destruction of the illegally collected medical information excuses the illegal collection.
• Collection of medical information is permitted if there is an authorising court order.
• Collection of personal information is forbidden.
• Collection of personal information is permitted if the collection of medical information is
permitted
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 12/20

17. Dilemma Structure
• b (“collection of medical information”) is forbidden
• c (“destruction of medical information”) compensates the illegal collection
• b is permitted if a (“acting under a court order”)
• d (“collection of personal information”) is forbidden
• d is permitted if b is permitted
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 13/20

18. Running Out of Time
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 14/20

19. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

20. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
• Prohibitions: G¬ (never)
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

21. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
• Prohibitions: G¬ (never)
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

22. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
• Prohibitions: G¬ (never)
Fp ≡ ¬G¬p
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

23. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
• Prohibitions: G¬ (never)
Fp ≡ ¬G¬p
In deontic logic the dual of obligation is permission.
Pp ≡ ¬O¬p
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

24. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
• Prohibitions: G¬ (never)
Fp ≡ ¬G¬p
In deontic logic the dual of obligation is permission.
Pp ≡ ¬O¬p
Obligation implies permission
Op → Pp
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

25. Running out of time (1)
How do we model obligations in LTL?
• Achievement obligations: F (sometimes in the future)
• Maintenance obligations: G (always in the future)
• Prohibitions: G¬ (never)
Fp ≡ ¬G¬p
In deontic logic the dual of obligation is permission.
Pp ≡ ¬O¬p
Obligation implies permission
Op → Pp
How do we model permissions in LTL?
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 15/20

26. Dilemma Structure
• b (“collection of medical information”) is forbidden
• c (“destruction of medical information”) compensates the illegal collection
• b is permitted if a (“acting under a court order”)
• d (“collection of personal information”) is forbidden
• d is permitted if b is permitted
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 16/20

27. Formalising the Dilemma: Take 1
1 G¬b, (G¬b ∧ b) → Gc;
2 a → Fb;
3 G¬d;
4 Fb → Fd.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 17/20

28. Formalising the Dilemma: Take 1
1 G¬b, (G¬b ∧ b) → Gc;
2 a → Fb;
3 G¬d;
4 Fb → Fd.
G¬b ∧ b ≡ ⊥
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 17/20

29. Formalising the Dilemma: Take 1
1 G¬b, (G¬b ∧ b) → Gc;
2 a → Fb;
3 G¬d;
4 Fb → Fd.
G¬b ∧ b ≡ ⊥
G¬b ∧ Fb ≡ ⊥ G¬d ∧ Fd ≡ ⊥
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 17/20

30. Formalising Compensation
Contrary-to-duty obligation
Oα ¬α → Oβ
Violation triggered obligation
Oα ∧ ¬α → Oβ
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 18/20

31. Formalising Compensation
Contrary-to-duty obligation
Oα ¬α → Oβ
Violation triggered obligation
Oα ∧ ¬α → Oβ
New “compensation operator” ⊗.
TS, σ |= φ ⊗ ψ iff ∀i ≥ 0, TS, σi |= φ; or
∃j, k : 0 ≤ j ≤ k, TS, σj |= ¬φ and TS, σk |= ψ.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 18/20

32. Formalising the Dilemma: Take 2
1 ¬a → (¬b ⊗ c);
2 a → Fb;
3 G¬b → G¬d;
4 Fb → Fd.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 19/20

33. Formalising the Dilemma: Take 2
1 ¬a → (¬b ⊗ c);
2 a → Fb;
3 G¬b → G¬d;
4 Fb → Fd.
t0
¬a
t1
¬a, b
t3
¬a, c, d
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 19/20

34. Formalising the Dilemma: Take 2
1 ¬a → (¬b ⊗ c);
2 a → Fb;
3 G¬b → G¬d;
4 Fb → Fd.
t0
¬a
t1
¬a, b
t3
¬a, c, d
the trace is (weakly) compliant in LTL, but the prohibition of ‘d’ is violated.
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 19/20

35. Really Running Out of Time
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20

36. Really Running Out of Time Conclusions
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20

37. Really Running Out of Time Conclusions
• Solution:
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20

38. Really Running Out of Time Conclusions
• Solution: do not use temporal logic to model norms
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20

39. Really Running Out of Time Conclusions
• Solution: do not use temporal logic to model norms
• CLAIM: the problem is not limited to temporal logic (most deontic logics have the same
issue)
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20

40. Really Running Out of Time Conclusions
• Solution: do not use temporal logic to model norms
• CLAIM: the problem is not limited to temporal logic (most deontic logics have the same
issue)
• Solution: Norm based semantics (Calardo, Governatori, Rotolo: A Preference-Based
Semantics for CTD Reasoning. DEON 2014: 49-64)
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20

41. Really Running Out of Time Conclusions
• Solution: do not use temporal logic to model norms
• CLAIM: the problem is not limited to temporal logic (most deontic logics have the same
issue)
• Solution: Norm based semantics (Calardo, Governatori, Rotolo: A Preference-Based
Semantics for CTD Reasoning. DEON 2014: 49-64)
• Solution: Defeasible Deontic Logic of Violation (FCL/PCL) (Governatori: Representing
business contracts in RuleML, International Journal of Cooperative Information Systems
(2005) 14: 181-216)
Thou Shalt is not You Will Copyright NICTA 2015 Guido Governatori 20/20