First, an interdisciplinary research approach of the Digital Humanities will be presented with a research project at the Bibliotheca Hertziana (MPI for Art History), Rome serving as an example. In the Institute's research, questions about the historical understanding of social space and its change during the so-called Long Middle Ages play a central role. The investigation of the relationship between historical maps and texts is intended to explore the historical understanding of space and the knowledge associated with it by taking up approaches from cognitive linguistics. Cognitive maps depict culturally specific spatial knowledge and practices. Annotation and analysis of historical texts and maps pose special challenges for knowledge representation and processing. The research data will be published as Linked Open Data in the Semantic Web. Subsequently, problems and challenges for machine learning in this area will be discussed.

1. Guenther Goerz, Chiara Seidl, Martin Thiering
Bibliotheca Hertziana –Max Planck Institute for the History of Art, Rome
FAU Erlangen-Nuremberg, Department of Computer Science, Digital
Humanities
Technical University Berlin, Department of Linguistics
Linked Biondo
Modelling Geographical Features
in Renaissance Texts and Maps

2. G. Goerz, FAU, CS DH & BHR 2

3. Research Question and Methodology
• Cognition of geographical space in history:
How would you start ?
– Toponyms and definite place descriptions
• Text annotation: Named Entity Recognition,
geographic verification
– Spatial relations: topology, orientation, distance,…
• Cognitive-linguistic annotation
Constructions figure- spatial_indicator – ground
– Comparisons with contemporary maps
• Cognitive maps (Common Sense Geography)
– Spatial objects and relations
– Epistemological modelling
G. Goerz, FAU, CS DH & BHR 3

4. Problem statement
• Common sense conceptualizations of geographic
concepts and relations in ancient and early modern
texts and maps
– Analytic methods of cognitive computational linguistics
Corpus construction, annotation, and parsing
– Formal two-level representation
• (Cognitive) linguistic
• Conceptual – general semantics (onto-logical)
– Linguistic and historical interpretation and evaluation
– Long term goal: Synthetic
Reconstruction of cognitive maps/sketches
G. Goerz, FAU, CS DH & BHR 4

5. Sources: Preparation and Conditioning
• Flavio Biondo Italia Illustrata (1474)
– “Topographically ordered historical account” (Clavuot)
– Connecting antiquity and presence
• Text editions: Pontari (la), White, Castner (la, en)
• Preprocessing
– OCR correction, sentence separation, etc.
– Word lists, statistics, n-grams
• scripts, antconc, voyant-tools, ...
– Concordance (KWIC)
– Part-of-Speech Tagging (TreeTagger, Collatinus)
– wordij network: “semantic” clustering
• Map image processing
G. Goerz, FAU, CS DH & BHR 5

6. Toponyms and spatial relations
• Text annotation (semi-automatic)
– Recogito 2 annotation tool with geographic
verfication (gazetteers, e.g. Pleiades)
– Spatial Role Labeling (manually: brat)
• Machine Learning? … corpus problem
– Narrative order: virtual trips
• Map annotation
– Recogito 2 (incl. visualization)
– Cartometric investigations (Guckelsberger)
G. Goerz, FAU, CS DH & BHR 6

7. Investigations on Renaissance maps
• Biondo’s use of maps ? (“Petrarca”, Ptolemy)
• Which maps to choose ?
– Six large maps of Italy (15. c.; Milanesi
2007/08); Paulinus Minorita (14. c.); Tabula
Peutingeriana
– Ptolemaic maps (ca. 30 traditional and
“novae” – mostly after 1450!)
– Sea charts (Portolans) before 1450 (max. 10)
G. Goerz, FAU, CS DH & BHR 7

8. G. Goerz, FAU, CS DH & BHR 8

9. G. Goerz, FAU, CS DH & BHR 9

10. L23nova
G. Goerz, FAU, CS DH & BHR 10

11. DARE map view of Ptolemaeus L23n
G. Goerz, FAU, CS DH & BHR 11

12. Generated research data
• Recogito data export
– CSV (tables), GeoJSON, RDF/OA, JSON-LD,…
– Case study: Ptolemaic tabulae novae & text
• Spatial Role Labeling
G. Goerz, FAU, CS DH & BHR 12
according to
predefined
cognitive-linguistic
taxonomy

13. Table for Ptolemy L23nova
G. Goerz, FAU, CS DH & BHR 13

14. Cognitive linguistics and Gestalt theory
• Mental models are based on universal cognitive
mechanisms and Gestalt-theoretical principles (!)
– cf. “Primary theory” (Smith/Mark 2001)
• Particularly relevant: spatial division of figure and
background in identifying and locating objects
• Spatial relations are represented through grammatical
markers and semantic fields
• From representational viewpoint: Mental models store
information on events and objects of the external
world, especially
– for orientation in space and references to places,
for topological and geometrical knowledge
G. Goerz, FAU, C.S. - AG DH

15. Spatial Construals and its Spatial
Parameters
• Gestalt principles of figure–ground (TRAJECTOR-
LANDMARK) asymmetries;
trajectory/path of TR and LM
• OBJECT CLASSIFICATIONS, mental rotations, 2,5/3-D
sketch, geometrical dimensions
• FRAMES OF REFERENCE (relative; intrinsic; absolute)
• TOPONYMS
(place / city names, buildings, bridges, churches,
fountains, walls, streets, squares, rivers, hills, gates,
memorials, temples, sites, regions, etc.)
G. Goerz, FAU, C.S. - AG DH

16. Spatial Construals and its Spatial
Parameters
• LANDMARKS
• DISTANCES (scale, scope, size),
encoded in adjectives, adverbs, verbs but mostly in
adpositions and case systems
• METRICAL SYSTEMS
(verbal systems such as posture verbs, classificatory
verbs and case systems)
• PERSPECTIVE
(bird’s eye, hodological, vectorial perspective)
• ELEMENTS OF COMMON SENSE KNOWLEDGE
(traveller reports, myths etc.)
G. Goerz, FAU, C.S. - AG DH

17. Spatial Construals and its Spatial
Parameters
• MOTION EVENT: SOURCE = Point of departure of TR
• PATH/TRAJECTORY = Movement of TR from SOURCE[TR(X)]
to GOAL[LM(Y)]
• GOAL = GOAL of TR'S movement to LM(Y);
often a container such as a room, city, town, church
etc.
• DISTANCE = proximate1[PROX], medial2[MED], distal3[DIST]
between TR and LM
• PROFILE = TRAJECTOR'S specification of LANDMARK
• Conceptualization of spatial structure: Static concepts
include a REGION, LOCATION, and dynamic concepts
include PATH and PLACEMENT of TR
G. Goerz, FAU, C.S. - AG DH

18. Spatial Role Labeling
in a Cognitive Linguistic Framework
• Given: SpatialML annotation scheme
• Definition of a brat “configuration” (taxonomy)
• Annotation sentence by sentence with brat, manually
• Parallel text: transfer to Latin
• XML/RDF export
– to be combined with dependency relations
– information integration with NER results
• Evaluation and Interpretation
– Evaluation of the use of prototypical functions with
lemmata (Latin/English) in order of frequency
– Specially: Landmarks, toponyms, frames of reference,
perspectives
G. Goerz, FAU, C.S. - AG DH

19. Part-of-
Speech
Analysis
(excerpt)

21. G. Goerz, FAU, CS DH & BHR 21
T1 region 6 14 Etruriae
T2 definite_description 0 14 Finis Etruriae
T3 river 18 25 Tiberim
T4 settlement 40 45 Romam
T6 spatial_indicator 15 17 ad
E1 spatial_indicator:T6 figure:T1 ground:T3
T7 action 30 39 perducens
T1 / T2 T6 T3 T7 T4
© Kordjamshidi 2013

22. Machine Learning for Spatial Role
Labeling?
• Kordjamshidi et al. (ca. 2010 ff.), KU Leuven
– Hybrid approach (with klog rules; cf. “explainable AI”)
– CoNLL competition, training corpus (campus)
– new hybrid framework since 2015 (Univ. Illinois)
• Problem of training data
– only for English, different text sort
– Latin from manual labeling, very small
• Experiment with Sarah Schulz (U. Stuttgart, 2018)
– Software (above) incomplete: missing modules
– Therefore using open software: mod:nlpnet (base:
NLTK, numpy)

23. Machine Learning for Spatial Role
Labeling?
• Experiment with Sarah Schulz : mod:nlpnet
– Multilayer perceptron for POS tagging
– Convolutional NN for SRL tagging
• adapted to SpRL, actually figure–sp_ind-ground
– Results for English with problematic training
corpus: anything but representative
• find identifier: accuracy 0.96
• find figure and ground: precision 0.54, recall 0.25
• recall figure: 0.19, ground: 0.30

24. wordij network: “semantic” clustering
G. Goerz, FAU, CS DH & BHR 24

25. G. Goerz, FAU, CS DH & BHR 25

26. Semantic enhancement of data
• What is the meaning of annotations ?
• Semantics of annotation components defined in terms of a
formal ontology
• Formal ontology: knowledge modelling
– two methodological levels
• Reference ontology CIDOC CRM (ISO 21127) with extension
CRMgeo
– CRM event-based; linguistic-pragmatic approach
– Refining of domain object descriptions with technical
terms (“types”) from thesauri (“Pleiades vocabulary”)
– Use of authority files
G. Goerz, FAU, CS DH & BHR 26

27. participate in
Actors Conceptual Objects
Physical Entities
Temporal Entities
affect
Types
refine
Appellations
identify/name
location
occur atwithin
Time-Spans
Places
CIDOC CRM
Top Level Classes
© T. Gill G. Goerz, FAU, CS DH & BHR 27

28. CRM + CRMgeo in OWL-DL (Protégé)
G. Goerz, FAU, CS DH & BHR 28

29. CRM + CRMgeo in OWL-DL (Protégé)
G. Goerz, FAU, CS DH & BHR 29

30. hmap + CRM + CRMgeo
• hmap: domain ontology for historical maps
– Map Metadata
ID, Cartographer, Creator, Title, Place, Time,
Size, Material, Technique, Projection, Scale,
Orientation, Reference System, …
– Image (Reproduction) Metadata: Map, URL,
Dimension, Rights,...
– Content: Annotated places and connections
UUID, Transcription, Anchor, Type, URI
(Gazetteer), Label, Lat, Lng, Place Type,
Verif_status, …
G. Goerz, FAU, CS DH & BHR 30

31. In search of a semantic platform
• How can we perform the semantic
enhancement of annotation data ?
• How can we publish them as Linked
Open Data ?
• Transformation with VRE WissKI,
usage as Linked Open Data platform
• SPARQL query interface, RDF export,…
G. Goerz, FAU, CS DH & BHR 31

32. What is WissKI ?
(“Scholarly Communication Infrastructure“)
• Extension of CMS Drupal, customizable
• Web-based multi-user system
• Open source & open standards (Semantic
Web)
• Object-based documentation, multiple media
types
• Ontology-based representation (ECRM/OWL),
extensible by application ontologies &
controlled vocabularies
G. Goerz, FAU, CS DH & BHR 32

33. WissKI Software Architecture
G. Goerz, FAU, CS DH & BHR 33

34. Layered
Ontologies
34 G. Goerz, FAU, CS DH & BHR

35. WissKI
• Create, Navigate, Find : main modes
• Create: Data input
– Form based or text based
– (automatic) linking
– data enrichment with external sources
– various import and export formats
• Pathbuilder: define semantics of fields
G. Goerz, FAU, CS DH & BHR 35

36. Input Form: Semantic Modelling
36
Albrecht Dürer
Nürnberg
E84 Information Carrier
→ P108i was produced by →
E12 Production
→ P14 carried out by →
E21 Person
→ P131 is identified by →
E82 Actor Appellation
→ P3 has note →
„Albrecht Dürer“
E84 Information Carrier
→ P108i was produced by →
E12 Production
→ P7 took place at →
E53 Place
→ P87 is identified by →
E48 Place Name
→ P3 has note →
„Nürnberg “
© HohmannG. Goerz, FAU, CS DH & BHR

37. Linked Open Data (... SPARQL endpoint)
DNB GND
Geonames
BritishMuseum
Linking Open Data cloud diagram 2017, by Andrejs Abele, John P. McCrae, Paul Buitelaar, Anja
Jentzsch and Richard Cyganiak. http://lod-cloud.net/
G. Goerz, FAU, CS DH 37

38. Next steps
• “In the last analysis all maps are cognitive maps”
(Blakemore & Harley)
• Abstract conceptual knowledge represented by
object schemata – propositional and “depictional”
• Translation of descriptions of spatial objects and their
spatial relations
– Triples [figure, spatial_indicator, ground]
– Constructing a spatial property graph
– Generating sketches of cognitive maps to represent and
process reifications of cognitive objects on an
epistemological level, i.e. frame of reference, topology,
direction, trajectory, distance, and shape
• Spatial reasoning ?
G. Goerz, FAU, C.S. - AG DH

39. Grazie!
G. Goerz, FAU, CS DH & BHR 39

40. G. Goerz, FAU, CS DH & BHR 40
CRM + CRMgeo
in OWL-DL
(Protégé)