This study links ship records from muster rolls to newspaper articles by applying domain heuristics and text classification techniques. The researchers generated candidate links and then applied two types of filters: domain-specific filtering using heuristics like matching ship captain names and dates, and text classification to determine if articles were about ships. Combining the captain name heuristic with date restrictions produced very high precision links, though approximate recall was lower. The historian preferred these high precision links. The study aims to evaluate different techniques for linking historical datasets rather than introduce new methods.

1. Linking historical ship records to
a newspaper archive
Andrea Bravo Balado
Victor de Boer, Guus Schreiber
VU University Amsterdam

2. Context: dutchshipsandsailors.nl/
2

3. Dutch Ships and Sailors (DSS) datasets
3

4. Results published as Linked Data
4

5. Data visualizations
5

6. This study
• Increasing number of historical databases are
being digitized
• Finding matching occurrences of the same
object in different datasets is both relevant
(for historical research) and non-trivial
– “Instance mapping”
• This paper: case study of linking ship instances
in two maritime datasets
6

7. Focus on methodology
• This study is not about developing new
techniques
• This study is about methodology:
– What combination of existing techniques gets the
“best” result?
– What the “best” result is depends on context (i.e.,
goal of the historical research)
• This is a case study, so be wary of
generalization
7

8. Data
• Muster rolls (Northern Dutch Maritime
Museum)
– Period: 1803-1937
– 77,043 records of 34,552 sea men
– 17,098 mentions of 4,935 ships
• Newspaper archive (Dutch National Library)
– Period: 1618-1995
– 7K newspapers, 9M pages (coverage: 10%)
– Text generated via OCR
8

9. Timeline newspapers in the archive
9

10. Example muster roll record (in Dutch)
10

11. Example newspaper article (in Dutch)
11

12. Approach
• Generate candidate set of links
• Apply two types of filters to the candidate set
– Domain-specific filtering
• Using domain heuristics about ship identification
– Text classification of newspaper articles
• Determine whether the article is about a ship
• Combine filters
12

13. Baseline generation
• Find all ship instances in the muster rolls
• Query newspaper archive for first 100 hits
with this name
– API: http://www.delpher.nl/
• Result set is expected to have high recall but
low precision
13

14. Evaluation
• No gold standard
• Manual assessment of all links is infeasible
• Sampling method for evaluating candidates
– 50 candidates per technique
– 3 assessors (domain expert plus two authors)
– Inter-observer agreement: Cohen’s kappa = 0.65
• Recall: approximation, based on the estimated
number of correct links (using the baseline)
14

15. Domain-specific filtering
• Heuristic 1: co-occurrence of name of ship
captain
– Common practice in historical maritime
documentation
• Heuristic 2: date of newspaper article is within
ship lifetime (as indicated by muster roll)
– Average life span of ship is 30 years
15

16. Text classification
• Task: decide whether a newspaper article is
about a ship
• Two techniques used
– Naive Bayes and Support Vector Machine (SVM)
with Sequential Minimal Optimisation (SMO)
– WEKA implementation
– Training set: 200 samples (121 positive, 79
negative)
16

17. Configuration
• Filter 1a: captain name
• Filter 1b: time restriction
• Filter 2: combine filters 1a + 1b
• Filter 2 + text classification
17

18. Results
18

19. Analysis
• Captain’s name turns out to be a strong
heuristic
• Time restriction much less useful
• When combined, precision becomes very
high, at the cost of (approximate) recall
• Text classification has high precision (no false
positives)
• Text classification combined with heuristic
filtering has negative effect
19

20. Discussion
• Interestingly, the historian preferred very high
precision at the cost of recall
• Consequently, 16K links published as Linked
Data (precision 0.96; approximate recall 0.13)
• Links are to departure/arrival listing, but also
to shipwrecks and sales
• In case of good heuristics the contribution of
generic techniques is at best minimal
• Absence of gold standard is realistic
20

21. Limitations
• Evaluation
– 50 samples
– Choice of assessors
– Approximation of recall
• Data
– OCR quality of newspaper articles
– Digitized newspaper archive covers only 10%
21

22. Acknowledgements
• Jurjen Leinenga, domain expert
• CLARIN-NL
http://www.clarin.nl
• BiographyNet, Netherlands eScience Center
http://esciencecenter.nl
• Online appendix with details of results at
http://dx.doi.org/10.6084/m9.figshare.1189228
22

23. QUESTION TIME
23