IMPACT Final Conference - Language Parallel Sessions -  Landsbergen
Upcoming SlideShare
Loading in...5
×
 

IMPACT Final Conference - Language Parallel Sessions - Landsbergen

on

  • 2,013 views

 

Statistics

Views

Total Views
2,013
Views on SlideShare
441
Embed Views
1,572

Actions

Likes
0
Downloads
9
Comments
0

5 Embeds 1,572

http://impactocr.wordpress.com 1098
http://www.digitisation.eu 447
http://impact.dlsi.ua.es 22
http://impact.sherrydesign.co.uk 4
http://webcache.googleusercontent.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

IMPACT Final Conference - Language Parallel Sessions -  Landsbergen IMPACT Final Conference - Language Parallel Sessions - Landsbergen Presentation Transcript

  • Named Entity Work in IMPACT Dr. Frank Landsbergen, Institute for Dutch Lexicology
  • Structure of this talk
    • Named Entities (NE‘s): definition
    • NE‘s in OCR and end-user retrieval
    • NE-lexicon building
  • Structure of this talk
    • Named Entities (NE‘s): definition
    • NE‘s in OCR and end-user retrieval
    • NE-lexicon building
  • What are named entities?
    • The Examiner (September 1845)
  • What are named entities?
    • The Examiner (September 1845)
    • Named entity: a word or string of words referring to a proper location, person, or organisation (or date, time, etc.)
    • Named Entities (NE‘s): definition
    • NE‘s in OCR and end-user retrieval
    • NE-lexicon building
  • From image to OCR’ed text On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa, and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party.
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa, and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Napoleon
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa, and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon , while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Napoleon
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa, and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Frederic?k? Barbarosa
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa , and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Frederic?k? Barbarosa
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa , and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Frederic?k? Barbarosa
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa , and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic , dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Leipzig
  • Retrieval On Thursday, the 4th inst., her Majesty continued her journey through Gelnhausen, where stand the ruins of the palace of the Emperor Frederick Barbarossa , and Hanau, a tolerably large place containing about 14,000 inhabitants, and the scene of more than one warlike exploit. It was gallantly defended for nine months during the thirty years' war, and in later days Napoleon, while in hot retreat from Leipsic, dashed through the allied troops of the Austrians Bavarians under its walls. The Queen reached I i'ankfort at three o'clock in the afternoon, and descended at the Hotel d'Angleterre, where a body of the City guard was drawn up to receive her Majesty. In the evening, after dinner, there was a reception or levee, at which the Larl and Countess of Westmoreland, the Burgomaster, and some others of the authorities of Frankfort were pre- sented. Prince Metternich and the King of Bavaria dined with the Royal party. User searches for: Frankfurt
  • Retrieval
    • Enable user to find NE’s regardless of historical spelling variation
    • Leipsic, Dares Salem, Frankfort
    • Improve quality of OCR for NE’s
    • I i'ankfort, Larl of Westmoreland
    • Named Entities (NE‘s): definition
    • NE‘s in OCR and end-user retrieval
    • NE-lexicon building
  • Building an NE-lexicon
    • Database of named entities
    • Related entries (wordforms) in the database are linked
    • Entries are enriched (e.g. PND)
    Amsterdam Amsterdams Amsteldam Aemsteldam Amstelredamme Amsterdam
  • Building an NE-lexicon
    • Database of named entities
    • Related entries (wordforms) in the database are linked
    • Entries are enriched (e.g. PND)
    Frederik Hendrik PND-id: 118590200 Frederik Frederick Frederic Fredric Fr édéric Frederiks Amsterdam Amsterdams Amsteldam Aemsteldam Amstelredamme Amsterdam
  • Building an NE-lexicon: a 4 step-process
    • data collection (NE-lists, text)
    • NE-tagging
    • enrichment (POS-tagging, lemmatizing, adding name structure, variants)
    • database creation
  • Building an NE-lexicon: NE-tagging
    • NE-extraction software
    • Stanford University, with added module
    • Statistical NE-recognition:
      • Software ‘trains itself’ with manually tagged data
      • automatic search for features + weights that ‘predict’ NE’s in tagged text
      • uses trained model to mark NE’s in untagged text
    • Manual tagging of training data: Attestation Tool
  • NE-recognition
    • Statistical NE-recognition software works very well with modern data. But how about historical material?
    • Benchmark tests, comparing results for modern & historical data
  • NE-recognition
    • NE-recognition on historical data not necessarily poorer than on modern data
    • More important is the resemblance between training set and extraction set (time period, genre)
    • Added spelling variation module improves performance by ‘smoothening’ variation
  • Building an NE-lexicon: enrichment
    • Lemmatization
    • POS-tagging
    • Person name structure
    • Variant matching
    • Link to autority files
    • NE-variant matching module that links:
    • Historical wordform to modern lemma
    • Aemsteldam > Amsterdam
    • Historical wordforms to each other
    • Frederik, Fredrik, Fréderick
  • The NE-lexicon
  • IMPACT-work on NE‘s so far
    • Toolkit for NE-lexicon building available (NERT, Attestation Tool)
    • NE-lexica of Dutch, English, German
  • IMPACT-work on NE‘s so far
    • Toolkit for NE-lexicon building available (NERT, Attestation Tool)
    • NE-lexica of Dutch, English, German
    Hendricus Hamer Hendricus Hamerus Hendrikkus Hindrikus Hendericus Henderikus Heudricus Henricus Hendrikus Henriçus henriecus Hamer Amerus Haamer Hammer
  • Contact
    • Frank Landsbergen
    • Institute for Dutch Lexicology
    • Leiden, Netherlands
    • f [email_address]