Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Good Lynx, bad Lynx: Document enrichment for historical ecologists

445 views

Published on

Presentation given at CLIN 28: http://clin28.cls.ru.nl

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Good Lynx, bad Lynx: Document enrichment for historical ecologists

  1. 1. G O O D LY N X , B A D LY N X : D O C U M E N T E N R I C H M E N T F O R H I S T O R I C A L E C O L O G I S T S M A R I E K E VA N E R P, J E S S E D E D O E S , T H O M A S VA N G O E T H E M A N D K AT R I E N D E P U Y D T Image by Böhringer Friedrich (Own work) [CC BY-SA 2.5 2.5)], via Wikimedia Commons
  2. 2. T H E S E R P E N S P R O J E C T • SERPENS: Contextual Search and Analysis of Pest and Nuisance Species through Time in the KB Newspaper Collection • research pilot project • May 2017- May 2018 • Partners:
  3. 3. W H Y N E W S PA P E R S ? • they give an insight into daily life • (reasonably) uniformly formatted • contain a lot of information • easily accessible (thanks to the National Library’s infrastructure)
  4. 4. T H E P L A N
  5. 5. F I R S T H U R D L E : O C R • The older the source, the harder it is to read • OCR errors may result in relevant documents being missed and irrelevant documents retrieved • In SERPENS we do not try to ‘fix’ the OCR, but rank documents by OCR quality through lexicon overlap
  6. 6. A M B I G U I T Y • Wolf: animal • Wolf: last name • Wolf in sheep’s clothes • … • Context of the document needed to find the right meaning Image source: https://www.powned.tv/media/73380/johndewolfff_gr.jpg? anchor=center&mode=crop&quality=80&width=1024&height=576&rnd=131081516640000000
  7. 7. T H E P L A N
  8. 8. BACK TO THE DRAWING BOARD image source: https://d2tyltutevw8th.cloudfront.net/media/image/drawing-board-640x250-1416349729.jpg
  9. 9. S E R P E N S C AT E G O R I E S • Natural history • Nuisance, material damage • Nuisance, immaterial damage • Pest control • Hunt for economic reasons • Prevention • Accidents • Figurative • Other image source: http://mmr.adlibhosting.com/madigopacx/wwwopac.ashx?
  10. 10. T R A I N I N G A N E W T O P I C C L A S S I F I E R • Manually classified 7,902 documents • Replace occurrences of “bunzing” & “Lynx” with “—ANIMAL—“ • 10-fold cross-validation • Mallet & fastText Image source: https://wallpaperclicker.com/storage/wallpaper/ machine-learning-smart-brain-wallpaper-26779524.jpg
  11. 11. R E S U LT S FA S T T E X T P R E C I S I O N R E C A L L F 1 N AT U R A L H I S T O RY ( 8 7 7 ) 5 8 . 1 7 7 2 . 7 2 6 4 . 6 4 N U I S A N C E - M AT E R I A L ( 3 1 2 ) 3 2 . 0 9 4 9 . 6 8 3 8 . 9 9 N U I S A N C E - I M M AT E R I A L ( 8 6 ) 0 0 0 P E S T C O N T R O L ( 2 9 1 ) 3 4 . 2 6 1 2 . 8 9 1 8 . 7 3 H U N T- E C O N O M I C ( 9 5 1 ) 7 8 . 5 0 7 0 . 3 9 7 4 . 2 2 A C C I D E N T S ( 2 2 0 ) 6 7 . 7 4 2 8 . 6 4 4 0 . 2 6 F I G U R AT I V E ( 3 1 4 ) 5 0 . 3 1 2 6 . 2 0 3 4 . 4 5 O T H E R ( 4 7 7 1 ) 8 8 . 0 9 9 2 . 8 1 9 0 . 3 9 T O TA L ( 7 8 3 2 ) 7 7 . 6 2 7 7 . 6 2 7 7 . 6 2
  12. 12. R E S U LT S M A L L E T ( N A I V E B AY E S ) P R E C I S I O N R E C A L L F 1 N AT U R A L H I S T O RY ( 8 7 7 ) 8 0 . 0 1 7 9 . 2 8 7 9 . 5 1 N U I S A N C E - M AT E R I A L ( 3 1 2 ) 5 9 . 1 8 5 8 . 4 1 5 8 . 1 5 N U I S A N C E - I M M AT E R I A L ( 8 6 ) 4 5 . 5 0 1 6 . 1 5 2 2 . 0 8 P E S T C O N T R O L ( 2 9 1 ) 5 0 . 6 5 5 2 . 2 8 5 1 . 1 8 H U N T- E C O N O M I C ( 9 5 1 ) 8 6 . 1 6 8 3 . 8 2 8 4 . 9 1 A C C I D E N T S ( 2 2 0 ) 6 5 . 6 5 5 6 . 8 5 5 9 . 8 9 F I G U R AT I V E ( 3 1 4 ) 6 8 . 0 6 6 0 . 0 7 6 3 . 4 2 O T H E R ( 4 7 7 1 ) 9 3 . 7 6 9 6 . 4 2 9 5 . 0 8 T O TA L ( 7 8 3 2 ) ( M A C R O ) 7 2 . 1 1 6 5 . 9 2 6 7 . 1 4
  13. 13. R E S U LT S M A L L E T ( N A I V E B AY E S ) P R E C I S I O N R E C A L L F 1 N AT U R A L H I S T O RY ( 8 7 7 ) 8 0 . 0 1 7 9 . 2 8 7 9 . 5 1 N U I S A N C E - M AT E R I A L ( 3 1 2 ) 5 9 . 1 8 5 8 . 4 1 5 8 . 1 5 N U I S A N C E - I M M AT E R I A L ( 8 6 ) 4 5 . 5 0 1 6 . 1 5 2 2 . 0 8 P E S T C O N T R O L ( 2 9 1 ) 5 0 . 6 5 5 2 . 2 8 5 1 . 1 8 H U N T- E C O N O M I C ( 9 5 1 ) 8 6 . 1 6 8 3 . 8 2 8 4 . 9 1 A C C I D E N T S ( 2 2 0 ) 6 5 . 6 5 5 6 . 8 5 5 9 . 8 9 F I G U R AT I V E ( 3 1 4 ) 6 8 . 0 6 6 0 . 0 7 6 3 . 4 2 O T H E R ( 4 7 7 1 ) 9 3 . 7 6 9 6 . 4 2 9 5 . 0 8 T O TA L ( 7 8 3 2 ) ( M A C R O ) 7 2 . 1 1 6 5 . 9 2 6 7 . 1 4
  14. 14. C O M M O N LY M I X E D U P C O R R E C T S Y S T E M P R E D I C T I O N F R E Q U E N C Y N AT U R A L H I S T O RY O T H E R 1 4 0 F I G U R AT I V E O T H E R 1 3 9 H U N T- E C O N O M I C O T H E R 1 3 8 O T H E R N AT U R A L H I S T O RY 1 3 1 P E S T C O N T R O L N U I S A N C E - M AT E R I A L 1 0 8
  15. 15. T R E N D S I N A N I M A L P E R C E P T I O N 0.00 0.20 0.40 0.60 0.80 1.00 80 90 0 10 20 30 Relativenumberofarticles Decade Bunzing 1880 - 1940 accidents figurative nuisance material nuisance immaterial economic natural history pest control other
  16. 16. C O M M U N I T Y W O R K S H O P 2 5 J A N U A RY 2 0 1 8
  17. 17. S N E A K P E E K : D E M O Try it on 9 March: CLARIAH “Toogdag” National Library of the Netherlands https://www.clariah.nl/evenementen/agenda/53
  18. 18. Questions? image source: https://www.thesun.co.uk/wp-content/uploads/ 2017/07/nintchdbpict0001286085811.jpg

×