The document provides an overview of a tutorial presented at the DCMI Conference in Seoul regarding the usage and querying of Wikidata as a linking hub in the linked data cloud. It covers key concepts such as external identifiers, mapping relations, and the process of linking external data to Wikidata through tools like Mix-n-Match, while also addressing quality control and community involvement in Wikidata. Additionally, it outlines challenges in performance for federated SPARQL queries and emphasizes the importance of community consensus for decision-making in Wikidata.

1. Wikidata as a hub for the linked data cloudWikidata as a hub for the linked data cloud
TUTORIAL AT DCMI CONFERENCE, SEOUL, 2019-09-25TUTORIAL AT DCMI CONFERENCE, SEOUL, 2019-09-25
Tom Baker, Joachim Neubert, Andra Waagmeester
Slides (partitially) at https://jneubert.github.io/wd-dcmi2019/#/

2. OverviewOverview
Part 1: Using and querying Wikidata
Part 2: Wikidata as a linking hub
Part 3: Applications based on Wikidata
Part 4: Wikidata usage scenarios
Scenarios
Intro and details
Hands-on: Mix-n-match
Quality control tools and procedures
Wikidata community

3. Wikidata as linking hubWikidata as linking hub

4. The idea of linking hubsThe idea of linking hubs
Connect concepts via identifiers/URLs
Existing hubs: , , ...
Image by Jakob Voss
VIAF sameAs.org

5. Different linking propertiesDifferent linking properties
1. (datatype URL)
generic link to URL in the meaning of skos:exactMatch
2. : more than 4000 specialized properties (datatype external identifier)
exact match
Pxxxx

6. Examples for external identifiersExamples for external identifiers
GND / VIAF identifiers
geogaphical entities
proteins
Swedish cultural heritage objects
African plants
baseball players
TED conference speakers

8. Property definitionsProperty definitions
subject item for the property
examples
constraints on values, cardinality, etc.
: creates a clickable link for the ID
start at the property page, e.g., for the ISSN:
formatter URL
https://www.wikidata.org/wiki/Property:P236

9. Property DocumentationProperty Documentation

11. Beyond sameness - mapping relationsBeyond sameness - mapping relations
Wikidata external ids imply "sameness" of linked concepts
even with geographic names, other mapping relations are required in some
cases.
examples:
close matches, e.g., "Yugoslavia" (1918-1992) (Wikidata) ≅ "Yugoslavia (until
1990)" (STW)
related matches, e.g. a company and its founder

12. Mapping relation type (P4390)Mapping relation type (P4390)
introduced after a community discussion in October 2017
to be used as qualifier for external id entries
fixed value set - SKOS mapping relations (exact, close, broad, narrow, related
match)

13. EXAMPLE AT ITEMEXAMPLE AT ITEM ASSESSMENT CENTERASSESSMENT CENTER

14. How does that relate to the Linked Data model?How does that relate to the Linked Data model?
Internal data model and storage (Wikibase) is transformed to RDF for:
RDF dumps
Query Service

15. RDF linking from WikidataRDF linking from Wikidata
: linked data URI
e.g., , (vs. formatter URL
)
linked external RDF resources
plus ~950.000 relations to individual URIs
formatter URI for RDF resource
http://sws.geonames.org/$1/
https://www.geonames.org/$1
List of 130+ relationships to external RDF datasets
26+ million
exact match

16. Links in the RDF dumpsLinks in the RDF dumps
Output has full URLs to external resources, however with Wikidata-specific
properties:
This creates a hurdle for generic Linked Data browsers and tools - not even
is translated to skos:exactMatch
wd:Q123 wdt:P234 "External-ID" ;
wdtn:P234 <http://example.com/reference/External-ID>
exact
match

17. Federated SPARQL queriesFederated SPARQL queries
Example use case: GND authority has information about the
professions/occupations of people which is not known in Wikidata.
So get that information dynamically from a GND SPARQL endpoint.
Here, we are interested in economists, in particular.

18. From Wikidata to a remote endpointFrom Wikidata to a remote endpoint
From a remote endpoint to WikidataFrom a remote endpoint to Wikidata
<== not working currently
query to WDQS
query to GND endpoint

19. Several points for attentionSeveral points for attention
Direction and sequence of statements often matters for performance
To reach out from Wikidata, endpoints have to be ( )
In the other direction, access is normally not restricted
Some federated queries get extremely slow, when large sets of bindings exist before the remote
service is invoked
be sure to exclude variables bound to blank nodes ('unknown value' in Wikidata)
approved full list

20. Further reading on Wikidata/RDFFurther reading on Wikidata/RDF
( )
Critical comments/suggestions:
RDF dump format (documentation)
Waagmeester: Integrating Wikidata and other linked data sources -
Federated SPARQL queries more examples
Malyshev et al.: Semantic Technology Usage in Wikipedia’s
Knowledge Graph
Freire/Isaac: Technical usability of
Wikidatas linked data

21. Application process for a new propertyApplication process for a new property
Double-check, that the property does not already exist
Prepare a property proposal in the according section, e.g., Wikidata:Property
proposal/Authority control

23. Hints for getting it approved smoothlyHints for getting it approved smoothly
Clearly lay out the motivation and planned use for the property
Provide working examples (with the formatter URI you are suggesting)
Be responsive to comments

24. Wikidata as a universal linking hubWikidata as a universal linking hub
easy extensibility with new properties for external identifiers
immense fund of existing items, with the full set of SKOS mapping relations for
more or less exact mappings to these
immediate extensibility with new items

25. Linking content via Mix-n-MatchLinking content via Mix-n-Match

26. Mix-n-match is a widely used tool (by Magnus Manske) to link external databases,
catalogs, etc. to existing Wikidata items (or to create new ones).

27. Example list:Example list:
Newspapers and journalsNewspapers and journals
from the 20th Century Press Archivefrom the 20th Century Press Archive

32. Please navigate to our example catalog
Mix-n-match manual

33. TasksTasks
Login through Widar
In "Automatically matched" list:
Connect matching items
Remove non-matching entries
In "Unmatched" list:
Search for existing items
Create missing items ( )suggested properties

34. Supplementary materialSupplementary material

35. Item creation "on the go"Item creation "on the go"
With Mix-n-match "New item": rudimentary, no references
Custom list of prepared QuickStatements insert blocks ( from STW
Thesaurus for Economics - please don't mess with it, this is work in progress)
Workflow-wise, use same sequence for M-n-m input and prepared insert blocks
example

36. Recommendations for item creationRecommendations for item creation
Pay attention to (much more relaxed then
Wikipedias)
Explain your plan and ask for feedback in the
to make mass edits ( )
Source every statement ( )
Create input in
Check with a few statements, verify result
Run as batch, document input and batch URL
Wikidata's notability criteria
Wikidata project chat
Apply for a bot account example
hints
QuickStatements text format

37. Matching from WD to the external database entriesMatching from WD to the external database entries

39. Normally requires an endpoint for the external source, where you can search for
the labels, aliases or other data of Wikidata items
Insert statement for external id into Wikidata can be prepared for cut&paste or
even semi-automatic execution in QuickStatements
Some hints and linked code here

40. Import catalog data to Mix-n-MatchImport catalog data to Mix-n-Match

41. Prepare dataPrepare data
... as tab-separated table (one line per record) with three columns
1. identitfier
2. name
3. description
Input file for the example used earlier

42. Pay attention toPay attention to
description column: include everything useful for intelectual identification
order: the sequence may help structuring your workflow (e.g., most used entries
first)

43. Load data via web interfaceLoad data via web interface
... at https://tools.wmflabs.org/mix-n-match/import.php

48. Sync existing ids from WikidataSync existing ids from Wikidata

52. Quality control tools and proceduresQuality control tools and procedures
Perception: Anybody can edit anything - so Wikidata is no reliable source of
knowledge
Seen as a threat for information systems based on Wikidata
particularly by some large Wikipedias (e.g., the English one)
Basic policy to address this: Statements should be referenced

53. QA support for editorsQA support for editors
Contraint definition for properties
raise warnings during data input, when, e.g.
a format definition (ISBN, DOI etc.) is violated
a supposedly unique identifier is added to more than one item
generated lists of constraint violations (e.g. )
Constraints can be very helpful, but do not cover complex cases
ZDB ID format

54. More QA support for editorsMore QA support for editors
Additional reports can be created via SPARQL queries
Shape Expressions (ShEx) allow to define complex constraints and conformance
checks
ShEx Primer
How to get started with Shape Expressions in Wikidata?

55. Revision control and patrolingRevision control and patroling
Versioned edits and version control
Manual and tool supported vandalism prevention
Watchlists
Automated flagging of suspect edits (e.g., "new editor deleteing statements")
Technically very easy to revert edits
Semi-protection or protection of oftenly-vandalized items
Patroling

57. Automated tools for vandalism detectionAutomated tools for vandalism detection
Fighting to keep up with rate of human edits in Wikidata (multiple per second)
... requires reducing the manual workload, e.g. via
Objective Revision Evaluation Service ( )
and other rule-based and machine-learning tools
Wikidata Abuse Filter
ORES

58. Ongoing researchOngoing research
Heindorf et al.: Vandalism Detection in Wikidata
Sarabadani et al.: Building automated vandalism detection tools for
Wikidata

59. The Wikidata communityThe Wikidata community
Everybody can participate
No central "committee" or decision structure
Desisions are made via discussion and community consensus

60. Main entry point for all kind of discussions
Resolved discussions archived after 7 days -
Beginner's questions welcome (but please try to find the answer online before,
particularly in the , which has a search link to the help pages)
Compared to Wikipedia, the overall atmosphere is constructive (though
exceptions exist sometimes in some sub-communities)
English is the lingua franca, but a few questions show up in other languages,
and receive comments, too
Project chatProject chat
searchable
FAQ

61. User page and user talk pageUser page and user talk page
Introduce yourself - especially if you work in a professional context with
Wikidata ( )
Activate notifications to your email address
Be responsive to comments on your talk page
You can address other users on their talk page, too
example

62. Talk pages of propertiesTalk pages of properties
Questions on the use of a certain property
Suggestions for changes or enhancements of the property definition ( )
Consider adding properties you are interested in to your watchlist
example

63. WikiProjectsWikiProjects

65. Often a great source to find documentation about the community consensus in
certain fields
Many WikiProjects pages contain data structuring recommendations - see, e.g.,
for
Current WikiProjects on Wikidata
periodicals

66. Thank you - questions welcome!Thank you - questions welcome!
Joachim Neubert
j.neubert@zbw.eu
Jneubert on WD