Your SlideShare is downloading. ×
0
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
What is Computer-Aided Summarisation and does it really work?
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

What is Computer-Aided Summarisation and does it really work?

1,135

Published on

Invited talk at Department of linguistics, Tuebingen University, Germany in May 2007

Invited talk at Department of linguistics, Tuebingen University, Germany in May 2007

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,135
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • In the first two stages the summariser identifies the overall structure of the text and the main topics. In the third stage, copy/paste operations followed by post-editing are used to produce the summary. The two first stages can be replaced by automatic methods
  • Transcript

    • 1. What is computer-aided summarisation and does it really work? Constantin Orasan http://clg.wlv.ac.uk/projects/CAST/
    • 2. Structure <ul><li>Introduction </li></ul><ul><li>CAST </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions </li></ul>
    • 3. Computer-aided summarisation <ul><li>Combines automatic methods with human input </li></ul><ul><li>Relies on automatic methods to identify the important information </li></ul><ul><li>Humans can decide to include this information and/or additional one </li></ul><ul><li>Humans post-edit the information to produce a coherent summary </li></ul>
    • 4. <ul><li>Automatic summarisation (AS) </li></ul><ul><li>Produces summaries automatically with the help of computers </li></ul><ul><li>Does not require human input </li></ul><ul><li>The quality is low (especially when compared to human summaries) </li></ul><ul><li>Computer-aided summarisation (CAS) </li></ul><ul><li>Uses automatic methods to produce summaries, but </li></ul><ul><li>Allows the humans to postedit the result </li></ul><ul><li>High quality, but less effort </li></ul>
    • 5. Why CAS can work? <ul><li>Endres-Niggemeyer (1998) identifies three stages in human summarisation: document exploration , relevance assessment and summary production </li></ul><ul><li>We hypothesise the first two stages can be replaced by automatic methods </li></ul><ul><li>Craven (1996) and Narita (2000) tried to help humans summarisers using automatic means </li></ul>
    • 6. Computer-aided summarisation tool (CAST) <ul><li>Work funded by Arts and Humanities Research Council </li></ul><ul><li>Work done together with Laura Hasler </li></ul><ul><li>The most important outcome of the project is the tool </li></ul><ul><li>Allows the user to run automatic methods to identify important sentences </li></ul><ul><li>In order to produce an abstract, the user can take sentences and edit them </li></ul>
    • 7. CAST- the tool (II) <ul><li>At present CAST contains the following methods: </li></ul><ul><ul><li>Keyword method </li></ul></ul><ul><ul><li>Indicating phrases </li></ul></ul><ul><ul><li>Surface clues </li></ul></ul><ul><ul><li>Lexical cohesion </li></ul></ul><ul><li>These methods were chosen because they are highly customisable and domain independent </li></ul><ul><li>The user can select the setting which is the most appropriate for a particular text/genre </li></ul>
    • 8.  
    • 9.  
    • 10.  
    • 11. Feedback from the user <ul><li>We analysed the work of our human summariser </li></ul><ul><ul><li>Term-based summarisation was used first to produce 30% summaries </li></ul></ul><ul><ul><li>Whenever a useful sentence was found lexical chains were used to identify related sentences </li></ul></ul><ul><ul><li>Avoids to run too many automatic methods because it becomes confusing </li></ul></ul><ul><ul><li>Requested a way to know which sentences have been included in the summary </li></ul></ul>
    • 12. Evaluation <ul><li>Our assumption about CAS is that it is possible to produce summaries in less time without any loss in quality </li></ul><ul><li>2 experiments were carried out: </li></ul><ul><ul><li>We recorded the time for producing summaries with and without CAST </li></ul></ul><ul><ul><li>Showed pairs of summaries and asked humans to pick the better one </li></ul></ul>
    • 13. Experiment 1 <ul><li>Used one professional summariser </li></ul><ul><li>69 texts from CAST corpus were used </li></ul><ul><li>Summaries were produced with and without the tool at one year distance </li></ul><ul><li>Without CAST With CAST Reduction % </li></ul><ul><li>Newswire texts 498secs 382secs 23.29% </li></ul><ul><li>New Scientist texts 771secs 623secs 19.19% </li></ul>
    • 14. Experiment 1 <ul><li>We evaluated the term-based summariser used in the process </li></ul><ul><li>We found correlation between the success of the automatic summariser and the time reduction </li></ul>
    • 15. Experiment 2 <ul><li>Turing-like experiment where we asked humans to pick the better summary in a pair </li></ul><ul><li>Each pair contained one summary produced with CAST and one without CAST </li></ul><ul><li>17 judges were shown 4 randomly selected pairs </li></ul>
    • 16. Experiment 2 <ul><li>In 41 pairs the summary produced with CAST was preferred </li></ul><ul><li>In 27 pairs the summary produced without CAST was preferred </li></ul><ul><li>Our assumption was that there is no difference between them </li></ul><ul><li>Chi-square shows that there is no statistically significant difference with 0.05 confidence </li></ul>
    • 17. Conclusions <ul><li>Computer-aided summarisation really works for professional summarisers </li></ul><ul><li>and reduces the time necessary to produce summaries by about 20% </li></ul><ul><li>It would be interesting to try with non-professional summarisers </li></ul><ul><li>Try on other texts </li></ul><ul><li>Compare to other computer-aided methods </li></ul>

    ×