OCR challenges in historic documents and the contribution of IMPACT

IMPACT Centre of Competence
IMPACT Centre of CompetenceIMPACT Centre of Competence
OCR challenges in historic documents  and the contribution of IMPACT Clemens Neudecker, KB National Library of the Netherlands
Background ,[object Object],[object Object],[object Object],[object Object],[object Object]
IMPACT – Improving access to text ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Historic material:  different   problems ,[object Object],[object Object],[object Object],[object Object],[object Object]
Bad OCR results… la 112 B ik e my lat arrived the >Pylades,-. lliot; aod. Abe- 3ineva, CNeee 4orn Neath, ' titch ,cuim; ,'t;ohn_ IoMelwl fri ytiil SUn- .die8; ,FrietndiLp, St&ar, froniidon, 'Ui wine and grocerieu ;: ;aletn, Bker, from Liverpool,. witfi eoal.;' 4Stalled the AluidonG.: ceror' Lkndon, with sundries; : ;Two Rrothwsj'@ Whe~atn-;- Pylade', Eiot; Har'tinny,; ;: Fisbley; ::Iiiveiy Peggy:-(flth add tie JAne, Redman, for eathly Newpot;agd llford; -Tw Br.otherAs, lawces, fos Lysixowjvithbinehol V pirI-ihzure;vi etsey, Per- wIliti; iIudstry, ModA - ~tbi ,Al~t,,'enniugs, for .:IP1~iOntI, StIth Ltu .c*ar An'l? Hawkinss foir ouck , + iii ballasto I _______~ ~ ~ ~~~Ai
Bleed  t hrough &  s hine  t hrough Effects are high, since it is the same ink (though lighter) and the shaping of characters is directly disturbed. When the printing ink was not dry, the letters of the one page also appear on the other page.  Also, if a paper is relatively thin the ink of the other side of the page may shine through.  Effects on OCRing General description
IMPACT: Binarisation
Annotations in the text Effects are high, since both segmentation as well as the recognition process itself is disturbed.  All notes, lines, drawings created by users, but also stamps, tapes etc. used within libraries. Effects on OCRing General description
IMPACT: Improved binarisation ,[object Object],[object Object],[object Object]
Warping of paper Partly a relatively high effect, especially if it is connected with bad printing (e.g. characters not aligned on the baseline of a line). Due to humidity the single page of an old book is very rarely really flat, in contrast it is warped. Even with putting the paper against a glass plate the warping will not disappear.  Effects on OCRing General description
IMPACT: Border removal
IMPACT: Geometric correction I
IMPACT: Geometric correction II
Gothic  typeface Effects are high since such fonts and  characters are often not recognised correctly. Historic fonts, obsolete characters such as the long s Effects on OCRing General description
IMPACT: Improved recognition
Complex l ayout   Effects are high since text is not ordered in the right way  Effects on OCRing Due to difficult layouts, pages can be segmented incorrectly General description
IMPACT: Segmentation Blocks/Regions   Words   Glyphs
IMPACT: Functional extension parser ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Bad  p rinting: blurred, broken, faded characters   Effects are high since characters are  broken or bound together.  According to the printing technology used letters may be blurred, broken or dotted.  Effects on OCRing General description
IMPACT: Cooperative correction ,[object Object],[object Object],[object Object]
IMPACT: Word spotting ,[object Object],[object Object],[object Object]
Historical language Historical variants of the Dutch word ‘wereld’ (world): werelt weerelt wereld weerelds wereldt werelden weereld werrelts waerelds weerlyt wereldts vveerelts waereld weerelden waerelden weerlt werlt werelds sweerels zwerlys swarels swerelts werelts swerrels weirelts tsweerelds werret vverelt werlts werrelt worreld werlden wareld weirelt weireld waerelt werreld werld vvereld weerelts werlde tswerels werreldts weereldt wereldje waereldje weurlt wald we ë led
IMPACT: Historical dictionaries ,[object Object],[object Object],[object Object],[object Object],werreid FineReader with IMPACT dictionary of  historical Dutch werreld ,[object Object],[object Object]
IMPACT: Linguistic post-correction ,[object Object]
IMPACT: Interoperability framework Interaction, Modularisation, Evaluation
Thank you! ,[object Object],[object Object],[object Object],[object Object]
1 of 26

Recommended

Developers guide java by
Developers guide javaDevelopers guide java
Developers guide javaVaibhav Mankar
1.7K views23 slides
Bratislava WS - Schlarb - ONB - technical tools_pdf by
Bratislava WS - Schlarb - ONB - technical tools_pdfBratislava WS - Schlarb - ONB - technical tools_pdf
Bratislava WS - Schlarb - ONB - technical tools_pdfIMPACT Centre of Competence
965 views25 slides
IMPACT Final Conference - Katrien Depuydt by
IMPACT Final Conference - Katrien DepuydtIMPACT Final Conference - Katrien Depuydt
IMPACT Final Conference - Katrien DepuydtIMPACT Centre of Competence
3.5K views46 slides
Language tools bne-5-10-2011 by
Language tools bne-5-10-2011Language tools bne-5-10-2011
Language tools bne-5-10-2011IMPACT Centre of Competence
271 views65 slides
New Words New Toys by
New Words New ToysNew Words New Toys
New Words New Toysresec
1.7K views42 slides
Reborn Digital: text, transmission, and technology by
Reborn Digital: text, transmission, and technologyReborn Digital: text, transmission, and technology
Reborn Digital: text, transmission, and technologyPip Willcox
449 views40 slides

More Related Content

More from IMPACT Centre of Competence

Session6 01.helmut schmid by
Session6 01.helmut schmidSession6 01.helmut schmid
Session6 01.helmut schmidIMPACT Centre of Competence
239 views30 slides
Session1 03.hsian-an wang by
Session1 03.hsian-an wangSession1 03.hsian-an wang
Session1 03.hsian-an wangIMPACT Centre of Competence
98 views16 slides
Session7 03.katrien depuydt by
Session7 03.katrien depuydtSession7 03.katrien depuydt
Session7 03.katrien depuydtIMPACT Centre of Competence
109 views32 slides
Session7 02.peter kiraly by
Session7 02.peter kiralySession7 02.peter kiraly
Session7 02.peter kiralyIMPACT Centre of Competence
202 views26 slides
Session6 04.giuseppe celano by
Session6 04.giuseppe celanoSession6 04.giuseppe celano
Session6 04.giuseppe celanoIMPACT Centre of Competence
116 views19 slides
Session6 03.sandra young by
Session6 03.sandra youngSession6 03.sandra young
Session6 03.sandra youngIMPACT Centre of Competence
285 views20 slides

More from IMPACT Centre of Competence(20)

Recently uploaded

STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf by
STKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdfSTKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdf
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdfDr. Jimmy Schwarzkopf
20 views29 slides
"Running students' code in isolation. The hard way", Yurii Holiuk by
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk Fwdays
17 views34 slides
Special_edition_innovator_2023.pdf by
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdfWillDavies22
18 views6 slides
Voice Logger - Telephony Integration Solution at Aegis by
Voice Logger - Telephony Integration Solution at AegisVoice Logger - Telephony Integration Solution at Aegis
Voice Logger - Telephony Integration Solution at AegisNirmal Sharma
39 views1 slide
Case Study Copenhagen Energy and Business Central.pdf by
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdfAitana
16 views3 slides
SAP Automation Using Bar Code and FIORI.pdf by
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdfVirendra Rai, PMP
23 views38 slides

Recently uploaded(20)

STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf by Dr. Jimmy Schwarzkopf
STKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdfSTKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdf
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays17 views
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2218 views
Voice Logger - Telephony Integration Solution at Aegis by Nirmal Sharma
Voice Logger - Telephony Integration Solution at AegisVoice Logger - Telephony Integration Solution at Aegis
Voice Logger - Telephony Integration Solution at Aegis
Nirmal Sharma39 views
Case Study Copenhagen Energy and Business Central.pdf by Aitana
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdf
Aitana16 views
SAP Automation Using Bar Code and FIORI.pdf by Virendra Rai, PMP
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdf
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f... by TrustArc
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc11 views
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
STPI OctaNE CoE Brochure.pdf by madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb14 views
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn22 views
6g - REPORT.pdf by Liveplex
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdf
Liveplex10 views
Serverless computing with Google Cloud (2023-24) by wesley chun
Serverless computing with Google Cloud (2023-24)Serverless computing with Google Cloud (2023-24)
Serverless computing with Google Cloud (2023-24)
wesley chun11 views

OCR challenges in historic documents and the contribution of IMPACT

  • 1. OCR challenges in historic documents and the contribution of IMPACT Clemens Neudecker, KB National Library of the Netherlands
  • 2.
  • 3.
  • 4.
  • 5. Bad OCR results… la 112 B ik e my lat arrived the >Pylades,-. lliot; aod. Abe- 3ineva, CNeee 4orn Neath, ' titch ,cuim; ,'t;ohn_ IoMelwl fri ytiil SUn- .die8; ,FrietndiLp, St&ar, froniidon, 'Ui wine and grocerieu ;: ;aletn, Bker, from Liverpool,. witfi eoal.;' 4Stalled the AluidonG.: ceror' Lkndon, with sundries; : ;Two Rrothwsj'@ Whe~atn-;- Pylade', Eiot; Har'tinny,; ;: Fisbley; ::Iiiveiy Peggy:-(flth add tie JAne, Redman, for eathly Newpot;agd llford; -Tw Br.otherAs, lawces, fos Lysixowjvithbinehol V pirI-ihzure;vi etsey, Per- wIliti; iIudstry, ModA - ~tbi ,Al~t,,'enniugs, for .:IP1~iOntI, StIth Ltu .c*ar An'l? Hawkinss foir ouck , + iii ballasto I _______~ ~ ~ ~~~Ai
  • 6. Bleed t hrough & s hine t hrough Effects are high, since it is the same ink (though lighter) and the shaping of characters is directly disturbed. When the printing ink was not dry, the letters of the one page also appear on the other page. Also, if a paper is relatively thin the ink of the other side of the page may shine through. Effects on OCRing General description
  • 8. Annotations in the text Effects are high, since both segmentation as well as the recognition process itself is disturbed. All notes, lines, drawings created by users, but also stamps, tapes etc. used within libraries. Effects on OCRing General description
  • 9.
  • 10. Warping of paper Partly a relatively high effect, especially if it is connected with bad printing (e.g. characters not aligned on the baseline of a line). Due to humidity the single page of an old book is very rarely really flat, in contrast it is warped. Even with putting the paper against a glass plate the warping will not disappear. Effects on OCRing General description
  • 14. Gothic typeface Effects are high since such fonts and characters are often not recognised correctly. Historic fonts, obsolete characters such as the long s Effects on OCRing General description
  • 16. Complex l ayout Effects are high since text is not ordered in the right way Effects on OCRing Due to difficult layouts, pages can be segmented incorrectly General description
  • 18.
  • 19. Bad p rinting: blurred, broken, faded characters Effects are high since characters are broken or bound together. According to the printing technology used letters may be blurred, broken or dotted. Effects on OCRing General description
  • 20.
  • 21.
  • 22. Historical language Historical variants of the Dutch word ‘wereld’ (world): werelt weerelt wereld weerelds wereldt werelden weereld werrelts waerelds weerlyt wereldts vveerelts waereld weerelden waerelden weerlt werlt werelds sweerels zwerlys swarels swerelts werelts swerrels weirelts tsweerelds werret vverelt werlts werrelt worreld werlden wareld weirelt weireld waerelt werreld werld vvereld weerelts werlde tswerels werreldts weereldt wereldje waereldje weurlt wald we ë led
  • 23.
  • 24.
  • 25. IMPACT: Interoperability framework Interaction, Modularisation, Evaluation
  • 26.