This document discusses ISO 24617-2, a standard for dialogue act annotation. It introduces the motivation for creating a standard, outlines the metamodel for dialogue annotation, and describes DiAML (Dialogue Act Markup Language). The metamodel defines the minimal units of annotation as functional segments and dialogue acts. It also specifies entity structures, dialogue act structures, and their semantics. Dependence relations and rhetorical relations between acts are represented. DiAML defines an XML representation of the annotation structures and their semantics.

1. ISO 24617-2: A Semantically-based Standard
for Dialogue Annotation
Bryan Hang Zhang
Universität des Saarlandes
2014 September

2. Outline:
• Motivation of building ISO Dialogue
Annotation (DA)
• Intro to the Meta Model of DA
• Intro to the DiAML
( Dialogue Makeup language )
• Task: Automatic Annotation Task
2Universität des Saarlandes

3. Motivation of ISO Dialogue
Annotation Scheme
3Universität des Saarlandes

4. Definition of dialogue act:
1. Dialogue act evolves from speech act.
However, proper definitions and concept of
dialogue act varies.
4Universität des Saarlandes

5. In reality there is a wide range of
taxonomies and inventories about
dialogue act.
5Universität des Saarlandes

6. ISO 24617-2 dialogue act annotation is created
That’s why we need a standard for Dialogue act
annotation which is inter-operable so annotated
corpora can be reused.
6Universität des Saarlandes

7. ISO 24617-2 Dialogue Act Annotation
• Based on DIT++ (Dynamic Interpretation
Theory) taxonomy.
• Information State Approach:
Dialogue
acts
Information state of
participants
UPDATE OPERATION
7Universität des Saarlandes

8. Medamodel for Dialogue Annotation
8Universität des Saarlandes

9. (DiAML) the Dialogue Act Markup Language
abstract syntax : specifies the possible annotation
structures in set-theoretical terms;
semantics : specifies the interpretation of the
structures defined by the abstract syntax;
concrete syntax : defines an XML representation
9Universität des Saarlandes

10. Representation
Structure (XML)
Annotation structure
Defined Semantics
10Universität des Saarlandes

11. Dialogue
acts
Information state of
participants
UPDATE OPERATION
11Universität des Saarlandes

12. Dialogue
acts
Information state of
participants
UPDATE OPERATION
12Universität des Saarlandes

13. Dialogue
act
Semantic content Communicative Function
Description of objects,
properties, actions.
Specification of how the
information states should be
updated with semantic content
13Universität des Saarlandes

14. Please consider these utterances:
Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Apology Answer
14Universität des Saarlandes

15. Dialogue
acts
Semantic content
Communicative Function
Communicative Function
Communicative Function
…..
15Universität des Saarlandes

16. Semantic content
Dialogue
acts
Communicative Function
Communicative Function ADimension 1
Dimension 2
Dimension 3
Dimension …
9 dimensions are
formulated
Communicative Function A
Communicative Function A
Communicative Function A
16Universität des Saarlandes

17. Semantic content
Dialogue
acts
Communicative Function
Communicative Function ADimension 1
Dimension 2
Dimension 3
Dimension …
10 dimensions are
formulated
Communicative Function A
Communicative Function A
Communicative Function A
17Universität des Saarlandes

18. Communicative Functions:
Dimension-specific functions
General purpose functions
Specific for a particular dimension
Turn Release function can be specific for Turn
Management dimension
Can be applied for all dimension
18Universität des Saarlandes

19. Medamodel for Dialogue Annotation
Minimal unit of dialogue
annotation (the raw data)
19Universität des Saarlandes

20. Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.Functional
Segment2
Functional
Segment3
Apology Answer
D: TASK
Set question
Functional
Segment 1
D: SOCAIL
OBLIGATION
D: TASK
DA 1
Dialogue act 1
DA 3 DA 2
Semantic Content
Semantic Content Semantic Content
20Universität des Saarlandes

21. Medamodel for Dialogue Annotation
Minimal unit of dialogue
annotation (the raw data)
21Universität des Saarlandes

22. Qualifiers
• Qualifiers are applicable to general purpose
functions(GPFs).
Qualifiers Type Communicative functions
Sentiment
qualifiers
Additive Q Accept happily
Conditional
qualifiers
Specified Q
e.g. Promise, Offer…
Certainty
qualifiers
e.g.Answer, Confirmation…
22Universität des Saarlandes

23. (un)Certainty Qualifier
Uncertain
certain
23Universität des Saarlandes

24. Medamodel for Dialogue Annotation
ENTITY STRUCTURE
DIALOGUE ACT STRUCTURE
24Universität des Saarlandes

25. Relations
There are various relations in coherent dialogue:
• Functional Dependence Relations
• Feedback Dependence Relations
• Rhetorical relations
25Universität des Saarlandes

26. Dependence Relations
1.Functional Dependences
• Functional dependence relations occur
with dialogue acts that are responsive
in nature.
• Examples: Answer-Question,
Apology- Accept Apology,
26Universität des Saarlandes

27. Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.Functional
Segment2
Functional
Segment3
Apology Answer
D: TASK
Set question
Functional
Segment 1
D: SOCAIL
OBLIGATION
D: TASK
DA 1
Dialogue act 1
DA 3 DA 2
27Universität des Saarlandes

28. fs1 (“Is there an earlier connection?”),
fs2 (“No there isn’t”) fs3 (“I’m sorry”)
• fs2 links to fs1 as the answer to the
question, semantic content (No, there
isn’t) is also needed for the link.
28Universität des Saarlandes

29. • Functional dependences is now part of
inherent attribute of the a functional
segment.
• This justice the intuition that the functional
dependence is an inherent part of a
responsive dialogue act.
• An Answer cannot exist without a Question
Accept-apology cannot exist without an
Apology.
29Universität des Saarlandes

30. Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.Functional
Segment2
Functional
Segment3
Functional
Segment 1
Dialogue act 1
Dialogue Act 3 Dialogue Act 2
30Universität des Saarlandes

31. In the past, a separate structure
31Universität des Saarlandes

32. Now it is a built-in ATTRIBUTE
32Universität des Saarlandes

33. Medamodel for Dialogue Annotation
33Universität des Saarlandes

34. Dependence Relations
2.Feedback Dependences
• Feedback dependence relation is
determined by semantic content of a
previous dialogue act.
34Universität des Saarlandes

35. • Speaker B here is checking the correctness
of his perception of what A said, which
sounded a bit strange
• (A may be hesitating between Tuesday and
Thursday or maybe A was non-native
speaker who had difficulty pronouncing ‘th’)
35Universität des Saarlandes

36. A feedback can be a non-local and distant feedback,
36Universität des Saarlandes

37. Functional
Segment2
Functional
Segment3
Functional
Segment 1
Dialogue act 1
Dialogue Act 3 Dialogue Act 2
37Universität des Saarlandes

38. 38Universität des Saarlandes

39. Medamodel for Dialogue Annotation
39Universität des Saarlandes

40. Rhetorical Relations
Rhetorical Relations are the parts of a
coherent text are connected explicitly or
implicitly
In other terms such as
‘coherence relations’, or ‘discourse relations’
40Universität des Saarlandes

41. • A: You keep losing them
• A: They easily slip behind the couch
• A: Where would you position your buttons?
• A: I think that has some impact on many
things.
A1 Question Act A2 Inform ActMotivation
Relation
Event A1 Event A2Semantic Causal
Relation
41Universität des Saarlandes

42. Functional
Segment2
Functional
Segment3
Functional
Segment 1
Dialogue act 1
Dialogue Act 3 Dialogue Act 2
42Universität des Saarlandes

43. Element->
Attribute->
43Universität des Saarlandes

44. Universität des Saarlandes 44
DiAML syntax and semantics

45. DiAML =Dialogue Act Markup Language
define:
• Abstract syntax:
set-theoretical annotation structure
• Concrete syntax:
XML representation of the structure.
45Universität des Saarlandes

46. Abstract Syntax
-Annotation Structure
Linguistic information added to the segments of
language data, independent of format
46Universität des Saarlandes

47. Medamodel for Dialogue Annotation
47Universität des Saarlandes

48. Functional Segment 1
Holds the semantic content
Entity
Dialogue Act Structue
Capture the function aspect
α
s
48Universität des Saarlandes

49. • Entity structures:
<s,α> or <s, α, E, δ>
• s: functional segment
• α: Dialogue act structure
• E: a set of entity structures (α) that have a
function/ feedback dependence relations
•
• δ: specifies function or feedback dependence
relation.
49Universität des Saarlandes

50. Medamodel for Dialogue Annotation
ENTITY STRUCTURE
DIALOGUE ACT STRUCTURE
50Universität des Saarlandes

51. Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Functional
Segment2
Functional
Segment3
Apology Answer
D: TASK
Set question
Functional
Segment 1
D: SOCAIL
OBLIGATION
D: TASK
DA 1
Dialogue act 1
DA 3 DA 2
51Universität des Saarlandes

52. Dialogue Act Structure
Senders
Addressee(S)
Dimension-specific communicative
function
General-
purpose
function
specify
dimension
Qualifier
52Universität des Saarlandes

53. Medamodel for Dialogue Annotation
53Universität des Saarlandes

54. ENTITY STRUCTURE e =
DIALOGUE ACT STRUCTURE
only captures the functional part of the Dialogue act.
Or
Semantics
Structure Interpretation
Complete Dialogue Act
Interpretation
54Universität des Saarlandes

55. • The Semantics of a Dialogue Act Structure
• :interpretation of the entity structure
• :semantic content of the segments
• So
55Universität des Saarlandes

56. Semantics for a certain class of communicative functions
Functions Update semantics units
F(Answer)=U1∪ U2 ∪ U9 ∪ U7
56Universität des Saarlandes

57. Semantic
Updates units
Information-transfer functions
57Universität des Saarlandes

58. Functional Part
Semantic Part
58Universität des Saarlandes

59. An entity
structure A non empty set of
entity structures
Rhetorical relations
Entities Link structures
Updates are
sequential or unified
59Universität des Saarlandes

60. Semantics of Dependence Relations and
Qualifiers
60Universität des Saarlandes

61. Concrete Syntax
-representation in XML
The format in which annotation is rendered.
Independent of content
61Universität des Saarlandes

62. Conceptual Inventories
Abstract Syntax
Set-theoretical
Annotation Structure
62Universität des Saarlandes

63. Set-theoretical
Annotation Structure
Representation Structure (XML)
63Universität des Saarlandes

64. Representation
Structure (XML)
Annotation structure
Defined Semantics
64Universität des Saarlandes

65. Representation
Structure (XML)
Annotation structure
Un-ambiguity Completeness
65Universität des Saarlandes

66. Automatic Incremental Annotation
Petukhova and Bunt (2011) report on
an incremental, token-based approach
to the segmentation and annotation of
spoken dialogue, with a focus on the
recognition of their communicative
functions.
66Universität des Saarlandes

67. Local Classifiers
Global Classifiers
Communicative
functions
AMI input data
features
Dimensions
Five previous and five
following tokens
67Universität des Saarlandes

68. ISO 24617-2 annotation scheme can
be effectively used for automatic annotation.
F-scores for incremental token-based recognition of
communicative functions.
68Universität des Saarlandes

69. Conclusion
• The requirement that semantic annotations have a
formal semantics was shown to have direct
consequences for the design of annotation structures.
• For dialogue act annotation, functional and feedback
dependence relations have to be expressed with
attributes in the XML elements representing dialogue act
structures. This simplifies the representations and make
them semantically fully transparent.
• For the ISO standard for dialogue act annotation, this
realization came just in time to prevent the proposal of
incorrect annotation representations
69Universität des Saarlandes

70. Reference
• Bunt, H. (2009a) The DIT++ taxonomy for functional dialogue markup. In
D. Heylen, C. Pelachaud, R. Catizone and D. Traum (eds.)
• Bunt, H. (2011). The semantics of dialogue acts.
• Traum, D. & S. Larsson (2003) The Information State Approach
to Dialogue Act Management.
• ISO 24617-2: A semantically-based standard for dialogue
annotation Harry Bunt*, Jan Alexandersson, Jae-Woong Choe,
Alex Chengyu Fang, Koiti Hasida,Volha Petukhova, Andrei
Popescu-Belis and David Traum
70Universität des Saarlandes

71. Thank You!
謝謝大家！
감사합니다!
71Universität des Saarlandes