Your SlideShare is downloading. ×
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Summarization and Visualization of Digital Conversations
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Summarization and Visualization of Digital Conversations

215

Published on

Paper presented at SPIM worksop at LREC2010, Malta.

Paper presented at SPIM worksop at LREC2010, Malta.

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
215
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Summarization and Visualization of Digital Conversations Vincenzo Pallotta Joint work withRodolfo Delmonte, University of Venice, Italy Marita Ailomaa, EPFL, Switzerland
  • 2. Digital Conversations•  The Web – Social Media – Forums – Blogs•  Meetings•  VoIP•  Call centers•  Help Desk SPIM 2010 - Malta 2
  • 3. Captured Meetings SPIM 2010 - Malta 3
  • 4. Virtual Collaboration SPIM 2010 - Malta 4
  • 5. SPIM 2010 - Malta 5
  • 6. SPIM 2010 - Malta 6
  • 7. SPIM 2010 - Malta 7
  • 8. 1st Hypothesis…V. Pallotta, Content-based retrieval of distributed multimedia conversational data. In E.Vargiu, A. Soro, G. Armano, G. Paddeu (eds.) Information Retrieval and Mining inDistributed Environments, Springer Verlag, series: Studies in Computational Intelligence(ISSN: 1860-949X) to Appear, 2010. SPIM 2010 - Malta 8
  • 9. Challenges for (spoken) conversation processing•  dealing with multiple speakers•  dealing with foreign language and associated accents•  incorporating non-speech audio dialogue acts –  (e.g., clapping, laughter, silence?)•  conversational segmentation and summarization•  discourse analysis, such as: –  analyzing speaking rates –  turn taking (frequency, durations) –  concurrence/disagreement •  which often provides insights into speaker emotional state, –  attitudes toward topics and other speakers –  roles/relationships. M. Maybury: Keynote at the SIGIR 2007 Workshop Searching Spontaneous Conversational Speech SPIM 2010 - Malta 9
  • 10. Capturing and Processing Conversations•  Informal Meetings •  Executive Summaries•  Focus Groups •  Topic highlights •  Issue tracking•  Classes •  Project management•  Interviews •  Mediation•  Debates •  Semantic Search•  Podcasts•  Comments•  Forums SPIM 2010 - Malta 10
  • 11. 2nd Hypothesis… SPIM 2010 - Malta 11
  • 12. What type of content is user looking for from conversations? 40 •  Users look for 35 30 IM2 set MS set argumentative 25 information 20 15 10 –  Decision Making 5 –  Conflict Resolution 0 Factual Thematic Process Outcome •  Information Retrieval is 80 70 IM2 set: argumentative not sufficient 60 50 MS set: argumentative –  Need for more context 40 30 –  Answers not found in 20 words spoken 10 0 IR sufficient IR irrelevant IR insufficientPallotta, Seretan, Ailomaa ACL 2007 SPIM 2010 - Malta 12
  • 13. 3rd Hypothesis… SPIM 2010 - Malta 13
  • 14. …in what form? SPIM 2010 - Malta 14
  • 15. …more demographic details SPIM 2010 - Malta 15
  • 16. …and still more SPIM 2010 - Malta 16
  • 17. 4th Hypothesis… SPIM 2010 - Malta 17
  • 18. Two reviews from ACL…•  "The idea of using argument structure annotation to aid dialogue summarization is very promising. For an abstractive summary of dialogues this seems almost like an inevitable step and I am always glad to see people take on the hard task of abstractive summarization.“•  "I think the general approach of detecting the argumentative structure is the correct one to take and the authors are laying groundwork for a solid abstractive system." SPIM 2010 - Malta 18
  • 19. Our Approach…•  Topic Segmentation•  Recognition of argumentative episodes: –  Based on the GETARUNS system•  Automatic recognition of argumentative structure: –  Novel discourse parsing algorithm•  Retrieval through: –  Question Answering –  Abstractive summaries –  Visualization of arguments SPIM 2010 - Malta 19
  • 20. Meeting Description SchemaDISCUSS(issue) <- PROPOSE(alternative)1702.95 David: so - so my question is should we go ahead and get na- -nine identical head mounted crown mikes ? {qy} 61a REJECT(alternative) 1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a PROVIDE(justification) 1714.09 B: because theres no point in doing that if its John: because theres no point in doing that if its going to to be better . {s} {s} 61b+ not not goingbe anyany better . 61b+ ACCEPT(justification) 1712.69 David: okay . {s^bk} 62b PROPOSE(alternative) 1716.85 John: so why dont we get one of these with the crown with a different headset ? {qw^cs} 63a PROVIDE(justification) 1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if its preferable and if it is then well get more . {s^cs^2} 64b 1725.47 Mark: comfort . {s} ACCEPT(alternative) 1721.56 David: yeah . {s^bk} 63b 1726.05 Lucy: yeah . {b} 1727.34 John: yeah . {b} Why was David’s proposal on microphones rejected? SPIM 2010 - Malta 20
  • 21. Abstractive SummaryDISCUSS(issue) <- PROPOSE(alternative)1702.95 David: so - so my question is should we go ahead and get na- - • David proposal was: “go ahead and get ninenine identical head mounted crown mikes ? {qy} 61a REJECT(alternative) 1708.89 John: not before having one come here and have identical head mounted some people try it out . {s^arp^co} 61b.62a crown mikes” PROVIDE(justification) 1714.09 B: because theres no point in doing that if its John: because theres no point in doing that if its going to to be better . {s} {s} 61b+ not not goingbe anyany better . 61b+ • David’s proposal was ACCEPT(justification) rejected. 1712.69 David: okay . {s^bk} 62b • John provided an PROPOSE(alternative) alternative: “get one of 1716.85 John: so why dont we get one of these with the crown with a different headset ? {qw^cs} 63a these with crown with a PROVIDE(justification) 1722.4 John: and - and see if that works . {s^cs} 63a+.64a different headset”. John’s proposal was accepted by 1723.53 Mark: and see if its preferable and if it is then well get more . {s^cs^2} 64b 1725.47 Mark: comfort . {s} ACCEPT(alternative) the majority of participants. 1721.56 David: yeah . {s^bk} 63b 1726.05 Lucy: yeah . {b} 1727.34 John: yeah . {b} SPIM 2010 - Malta 21
  • 22. Argumentative Labeling with GETARUNS•  Primitive Discourse Relations labels: –  statement, narration, adverse, result, cause, motivation, explanation, question, hypothesis, elaboration, permission, inception, circumstance, obligation, evaluation, agreement, contrast, evidence, hypoth, setting, prohibition.•  Mapped into Argumentative labels: –  ACCEPT, REJECT/DISAGREE, PROPOSE/ SUGGEST, EXPLAIN/JUSTIFY, REQUEST EXPLANATION/JUSTIFICATION.Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogueUnderstanding. Proceedings LREC 2010 (P31 Dialogue Corpora). SPIM 2010 - Malta 22
  • 23. Evaluation ICSI corpus of meetings (Janin et al., 2003) Precision: 81.26% Recall: 97.53% Total Correct Incorrect Precision Found Accept 662 16 678 98% Reject 64 18 82 78% Propose 321 74 395 81% Request 180 1 181 99% Explain 580 312 892 65% Disfluency 19 0 19 100% Total 1826 421 2247 81%Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogueUnderstanding. Proceedings LREC 2010 (P31 Dialogue Corpora). SPIM 2010 - Malta 23
  • 24. Applications for Visualizationand Summarization of DigitalConversations SPIM 2010 - Malta 24
  • 25. Conversational Graphs [7:00] # Yes, uh, Ive a question, uh, whats mean exactly advance chip on print? Whats the meaning of that? [7:10] 7 5 [7:02] Yeah [7:2] [7:10] I think its um uh a multiple uh chip design uh and its maybe printed on to the (curcuit) board. [7:20] 8 7 [7:21] Mm-hmm. [7:21] [7:21] Uh I could find out more about that uh before the next fi- next meeting. [7:26] 8.1 8 [7:24] Yeah, is it means its on the - x#x is it on the micro-processor based or uh - [7:30] 9 8 [7:32] I dont know, but Ill find out more on our next meeting. [7:35] 10 11 11:09 [7:34] [O]okay, uh, that would be great, so if you find out from the technology backgroud, okay, so that would be good[.] [7:39] 12 10 [7:39] Sounds good. [7:40] [7:41] Why was the plastic eliminated as a possible material? [7:44] 13 3 [7:43] Because um it gets brittle - [7:46] 14 13 3 [7:47] cracks - [7:48] 14 13 3 [7:48] uh-huh [7:49] [7:51] um [7:51] 14 13 3 [7:53] We want - we expect these um these remote controls to be around for several hundred years. [7:59] 14 13 3 [8:00] So $ we could $ (??) - good expression [8:6] [8:02] (I would gi-) [8:2] [8:02] Wow $ Good expression, (well) after us $ [8:12] [8:05] Which - [8:6] [8:12] Um, speak for yourself, I (??) $ - [8:16] [8:13] Alth- I think - [8:15] [8:14] $ [8:16] [8:16] I think with the wood though youd run into the same types of problems (??) I mean it chips, it- if 15:14 you drop it, ehm, its - Im not su- $ [8:27] 15 16 (15:3?) 16:15 SPIM 2010 - Malta 25
  • 26. Mapping to Bales IPA categories SPIM 2010 - Malta 26
  • 27. Improving Opinion Mining SPIM 2010 - Malta 27
  • 28. Attitude scores re-ranking NESTLÉ twittrratr Interanalytics Δ 21% Positive 13% 34% Neutral 85% 40% -45% Negative 3% 16% 13% Not Clear 0% 10% 10% Total 100% 100% Reliability 33% 80% Scores Powered by: SPIM 2010 - Malta 28
  • 29. Abstractive Summaries ofDigital Conversations SPIM 2010 - Malta 29
  • 30. Conversation Memos (1)GENERAL INFORMATION ON PARTICIPANTS•  The participants to the meeting are 7.•  Participants less actively involved are Ami and Don who only intervened respectively for 38 and 68 turns.LEVEL OF INTERACTIVITY IN THE DISCUSSION•  The speaker that has held the majority of turns is Adam with a total of 722 turns, followed by Fey with a total of 561.•  The speaker that has undergone the majority of overlaps is Adam followed by Jane.•  The speaker that has done the majority of overlaps is Jane followed by Fey.•  Jane is the participant that has been most competitive. SPIM 2010 - Malta 30
  • 31. Conversation Memos (2)DISCUSSION TOPICS•  The discussion was centered on the following topics: " "schemas, action, things and domain.•  The main topics have been introduced by the most important speaker of the meeting.•  The participant who introduced the main topics in the meeting is: Adam.•  The most frequent entities in the whole dialogue partly coincide with the best topics, and are the following: action, schema, things, source-path-goal, person, spg, roles, bakery, intention, specific, case, categories, information, idea. SPIM 2010 - Malta 31
  • 32. Conversation Memos (3)ARGUMENTATIVE CONTENT EPISODE ISSUE No. 7The following participants: In this episode we have the following argumentative exchanges between the "Andreas, Dave, Don, Jane, Morgan following speakers: Don, Morgan.expressed their dissent 52 times. However Morgan provides the following explanation: Dave, Andreas and Morgan expressed [oh, that-s_, good, .] then he , overlapped by Don, continues: dissent in a consistently smaller [because, we, have, a_lot, of, breath, noises, .] percentage. Don accepts the previous explanation:The following participants: [yep, .] "Adam, Andreas, Dave, Don, Jane, Morgan then he provides the following explanation: [test, .]asked questions 55 times. Morgan continues:The remaining 1210 turns expressed positive [in_fact, if, you, listen, to, just, the, channels, of, people, not, talking, it-s_, like, ..., .] content by proposing, explaining or then he , overlapped by Don, disagrees with the raising issues. However Adam, Dave and previous explanation Andreas suggested and raised new [it-s_, very, disgust, ..., .] issues in a consistently smaller Don, overlapped by Morgan, asks the following percentage. question:The following participants: Adam, Andreas, [did, you, see, hannibal, recently, or, something, ?] Dave, Don, Jane, Morgan expressed Morgan provides the following positive answer: [sorry, .] acceptance 213 times. then he provides the following explanation: [exactly, .] [it-s_, very, disconcerting, .] [okay, .] … SPIM 2010 - Malta 32
  • 33. Conclusion•  Conversational Search and Condensation is extremely challenging –  Classical approaches simply don’t work –  Sense-making is needed•  One possible “sense”: –  Argumentative structure•  Possible outputs: –  Question Answering –  Abstractive Summaries –  Conversation Graphs•  Future Work: –  Improving performance of the classifier –  Build the linking structure of arguments –  Approach generation SPIM 2010 - Malta 33
  • 34. Summarization andVisualization of Digital Conversations Vincenzo Pallotta Joint work withRodolfo Delmonte & Marita Ailomaa

×