The document discusses the Library of Congress Web Archives which contains over 18 million online resources including books, images, newspapers, and over 11,000 archived websites. It describes how the Library of Congress exposes metadata about these resources via an API to enable services to be built on top. The Web Observatory uses schema.org vocabulary to describe different resources in repositories and data catalogs to help make them discoverable across observatories. The goal is for web observatories around the world to work together using common standards to map and understand the digital universe at a global scale through observation and experimentation.

1. Web Obervatories, Data
Analytics and Archives
Professor Dame Wendy Hall
University of Southampton, UK and Kluge Chair, Library
of Congress
16th June 2016

2. 2

3. 

4. Library of Congress Web Archives
Over 18 million resources
available via the LOC website
Books, images, drawings,
newspapers, and websites
Over 11 thousand archived
websites supported by the
Internet Archive and counting

5. Library of Congress API
LOC exposes their resource data via
API to enable services to be built on top
>

6. Building Web Observatory Services on
the LOC website
The Web Observatory uses schema.org vocab
to describe different resources
• simple, lightweight metadata (URL, name,
description, author)
The Web Observatory lens can be added to
existing repositories and data catalogues
• helping make them discoverable across Web
Observatories

7. Enriching LOC Resources with Schema.org
LOC API (JSON)
>
WO Schema.org (JSON-LD)

11. Archives Unleashed 2.0 – used datasets
found through the Observatory

12. Archives Unleashed 2.0 – the results can be
made accessible through the Observatory

13. Studying the Web across continents
• Astronomers obtain a very high
resolution picture of the sky from small
telescopes a long distance apart.
• Many web research labs, contributing
across the globe, help build an accurate
picture of human activity at planetary
scale.
– transcending parochial social, political,
economic, legal interpretations

14. understanding web evolution:
- observation
- experimentation

15. Web Observatory Infrastructure today
• An initiative supported by the Web Science Trust
– http://www.webscience.org
• A growing number of sites using common metadata for
hosted datasets and apps (schema.org)
– http://index.webobservatory.org
• Some WO sites use purpose-built software that:
– Allows their community members to list and share
public or private datasets and apps
– Provides for discovery and access to listed datasets and
apps across WO sites
– Provides APIs for app development using listed datasets:
– http://webobservatory.soton.ac.uk
Thanassis Tiropanis – University of Southampton – t.tiropanis@southampton.ac.uk

16. index.webobservatory.org
Follow us: @wo_team
Contact us: hello@webobservatory.org

17. Web Observatory Infrastructure tomorrow
• A distributed catalogue across WO sites
• WO sites use common technical standards for
– Describing locally or remotely hosted datasets and apps
– Accessing datasets and apps across sites
– APIs for developing apps and visualisations
– Meaningful terms and conditions
– Implementing ethical practice
Thanassis Tiropanis – University of Southampton – t.tiropanis@southampton.ac.uk

18. The Web Observatory: A Middle Layer for Broad Data. (2014). Tiropanis, T, Hall, W, Hendler, J A, De Larinaga, C. Big Data, 2(3).

20. Datasets
20

21. Apps

22. Who is editing what in Wikipedia?
22

23. The Web of Observatories
Web
Observatory
Web
Observatory
Web
Observatory
Web
Observatory
Web
Observatory

24. 24© Web Science Trust 2013
Observing the Web
How do we catalogue Observatories and content?
•https://www.w3.org/wiki/WebSchemas/SchemaDot
OrgProposals
•https://www.w3.org/wiki/WebSchemas/WebObsSch
ema
•We’re building a crawler and a search engine

25. 25© Web Science Trust 2013
Observing the Web
The ambition is to map the digital
universe!

26. webscience.org/web-observatory
Follow us: @wo_team
Contact us: hello@webobservatory.org

27. Digital Vellum
The Self Archiving Web
Who is archiving what?