SlideShare a Scribd company logo
Interrogating the archived
UK web
Peter Webster (British Library)
@pj_webster / @UKWebArchive
webarchive.org.uk
www.bl.uk 2
Big UK Domain Data for the Arts &
Humanities
• Led by Dr Jane Winters (IHR) (@jfwinters)
• In partnership with the British Library and the Oxford Internet Institute
• With the help of Niels Brügger (Aarhus University, Denmark) @NielsBr
• Co-investigators: Ralph Schroeder, Eric Meyer (@etmeyer), Helen
Hockx-Yu (@hhockx)
• The team includes: Jonathan Blaney, @JoshCowls, @anjacks0n
• Funded by the AHRC, Jan 2014 – March 2015
• http://buddah.projects.history.ac.uk/
www.bl.uk 3
Project aims
• To highlight the value of web archives as a source for A&H, & to
transform the way in which researchers interact with the data
• To establish a theoretical and methodological framework for the analysis
of web archives, focusing on the JISC UK Web Domain Dataset
• To explore the ethical implications of big data research, and particularly
as they relate to the web
• To inform collection development and access arrangements for the UK
web archive at the British Library
www.bl.uk 4
Project outputs
• a suite of tools to support analysis of web archives by A&H researchers
• an enhanced interface through which researchers access the archived
material held by the British Library
• a history of the development of UK web space from 1996 to 2013,
analysing technical, social, organisational and cultural developments
and trends in the dataset
• a series of case studies across a range of A&H disciplines
• two project workshops, bringing together researchers, archivists,
technologists, and digital preservation professionals
• a free online training module illustrating the use of web archives and the
application of big data techniques and methods.
www.bl.uk 5
Forthcoming event
Web archives as big data
Wednesday, 3 December 2014 from 09:45 to 17:30
IHR, Senate House, United Kingdom
Booking at: http://tinyurl.com/webarchives
www.bl.uk 6
A new class of primary source ?
Deswarte and Webster,
“Web Archives: A New Class of Primary Source for
Historians?”
IHR Digital History seminar, 2013, reporting on predecessor
project (AADDA)
http://tinyurl.com/qca3yy5
www.bl.uk 7
webarchivehistorians.org (@HistWebArchives)
www.bl.uk 8
The UK Web Archive: three archives in one
Open UK Web Archive (2004-)
• c.14,000 sites
• Curated, selective, permission-based
• webarchive.org.uk
Legal Deposit UK Web Archive (2013-)
• legal framework
• c.4-5 million hosts per year
• onsite only
JISC UK Web Domain Dataset
www.bl.uk 9
JISC UK Web Domain Dataset 1996-2013
• Funded by JISC to create a research collection of UK
websites
• Collaboration between the Internet Archive, JISC and the
British Library
• Copy of subset of the Internet Archive’s web collection that
relates to the UK
• c.300 million resources, 60TB in total
• No local access – possible through the Internet Archive
• Can be used to generate secondary datasets
www.bl.uk 10
Use cases (generalised)
• Full-text/facet search -> individual resource
• Full-text/facet search -> analysis/visualisation
• Search -> corpus creation -> annotation/curation
• Corpus creation -> full-text search -> individual resource
• Corpus -> search -> analysis/visualisation
• Derived datasets -> take-away
• Direct access to WARC -> take-away
www.bl.uk 11
What do we know about each resource ?
From the crawl data
• crawl date
• URL (/page.html, host.domain.co.uk, domain.co.uk, .co.uk)
• file format
• file size
www.bl.uk 12
What do we know about each resource ?
From the full-text index
• page title
• link destinations (host.domain.co.uk, domain.co.uk, .co.uk)
• author (sometimes)
• language (sometimes)
www.bl.uk 13
What *don’t* we know ?
• subject
• geographic scope
• publisher
• date of publication
• date of last amendment
www.bl.uk 14
Thank you !
peterwebster6@gmail.com
@pj_webster / @UKWebArchive
britishlibrary.typepad.co.uk/webarchive
webarchive.org.uk
peterwebster.me

More Related Content

What's hot

Dariah de and e research alliance, germany
Dariah de and e research alliance, germanyDariah de and e research alliance, germany
Dariah de and e research alliance, germany
SistemaBibliotecarioSapienza
 
British Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination WorkshopBritish Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination Workshop
labsbl
 
World Archaeology Congress paper
World Archaeology Congress paperWorld Archaeology Congress paper
World Archaeology Congress paper
dejp3
 
British Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination WorkshopBritish Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination Workshop
labsbl
 
Hughes british library june 9th
Hughes british library june 9thHughes british library june 9th
Hughes british library june 9th
lorna_hughes
 
Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...
Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...
Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...
Bernhard Haslhofer
 
Future Directions for The European Library
Future Directions for The European Library Future Directions for The European Library
Future Directions for The European Library
The European Library
 
Europeana in a Research Context
Europeana in a Research ContextEuropeana in a Research Context
Europeana in a Research Context
TU Delft, Netherlands
 
Kuali Invades the UK: OLE's success in partnering with UK academic libraries.
Kuali Invades the UK: OLE's success in partnering with UK academic libraries.Kuali Invades the UK: OLE's success in partnering with UK academic libraries.
Kuali Invades the UK: OLE's success in partnering with UK academic libraries.
Andrew Preater
 
Tanya Szrajber, The British Museum Collection Database
Tanya Szrajber, The British Museum Collection DatabaseTanya Szrajber, The British Museum Collection Database
Tanya Szrajber, The British Museum Collection Database
Andrew Prescott
 
Historical newspapers in the context of Digital Library of Slovenia
Historical newspapers in the context of Digital Library of SloveniaHistorical newspapers in the context of Digital Library of Slovenia
Historical newspapers in the context of Digital Library of Slovenia
Europeana Newspapers
 
Scotland's National Collections and the Digital Humanities - Malcolm Chapman
Scotland's National Collections and the Digital Humanities - Malcolm ChapmanScotland's National Collections and the Digital Humanities - Malcolm Chapman
Scotland's National Collections and the Digital Humanities - Malcolm Chapman
tarastar
 
Glenn Cumiskey - UKAD 2016 forum
Glenn Cumiskey - UKAD 2016 forumGlenn Cumiskey - UKAD 2016 forum
Glenn Cumiskey - UKAD 2016 forum
The-National-Archives
 
Mass observation behind bars | DCDC14
Mass observation behind bars | DCDC14Mass observation behind bars | DCDC14
Mass observation behind bars | DCDC14
ResearchLibrariesUK
 
From Research Library to Research Services
From Research Library to Research ServicesFrom Research Library to Research Services
From Research Library to Research Services
Saskia Scheltjens
 
NCompass Live: Database Roadshow Teaser
NCompass Live: Database Roadshow TeaserNCompass Live: Database Roadshow Teaser
NCompass Live: Database Roadshow Teaser
Nebraska Library Commission
 
BL Labs Presentation to Social Sciences Team at the British Library
BL Labs Presentation to Social Sciences Team at the British LibraryBL Labs Presentation to Social Sciences Team at the British Library
BL Labs Presentation to Social Sciences Team at the British Library
labsbl
 
Exhibition recommendation using British Museum data and Event Registry - ESWC...
Exhibition recommendation using British Museum data and Event Registry - ESWC...Exhibition recommendation using British Museum data and Event Registry - ESWC...
Exhibition recommendation using British Museum data and Event Registry - ESWC...
eswcsummerschool
 
Missing links closing talk - with notes
Missing links closing talk - with notesMissing links closing talk - with notes
Missing links closing talk - with notes
Kevin Ashley
 
Dealing with research data
Dealing with research dataDealing with research data
Dealing with research data
Elena Simukovic
 

What's hot (20)

Dariah de and e research alliance, germany
Dariah de and e research alliance, germanyDariah de and e research alliance, germany
Dariah de and e research alliance, germany
 
British Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination WorkshopBritish Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination Workshop
 
World Archaeology Congress paper
World Archaeology Congress paperWorld Archaeology Congress paper
World Archaeology Congress paper
 
British Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination WorkshopBritish Library Labs Presentation at the Accelerating Human Imagination Workshop
British Library Labs Presentation at the Accelerating Human Imagination Workshop
 
Hughes british library june 9th
Hughes british library june 9thHughes british library june 9th
Hughes british library june 9th
 
Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...
Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...
Maphub und Pelagios: Anwendung von Linked Data in den Digitalen Geisteswissen...
 
Future Directions for The European Library
Future Directions for The European Library Future Directions for The European Library
Future Directions for The European Library
 
Europeana in a Research Context
Europeana in a Research ContextEuropeana in a Research Context
Europeana in a Research Context
 
Kuali Invades the UK: OLE's success in partnering with UK academic libraries.
Kuali Invades the UK: OLE's success in partnering with UK academic libraries.Kuali Invades the UK: OLE's success in partnering with UK academic libraries.
Kuali Invades the UK: OLE's success in partnering with UK academic libraries.
 
Tanya Szrajber, The British Museum Collection Database
Tanya Szrajber, The British Museum Collection DatabaseTanya Szrajber, The British Museum Collection Database
Tanya Szrajber, The British Museum Collection Database
 
Historical newspapers in the context of Digital Library of Slovenia
Historical newspapers in the context of Digital Library of SloveniaHistorical newspapers in the context of Digital Library of Slovenia
Historical newspapers in the context of Digital Library of Slovenia
 
Scotland's National Collections and the Digital Humanities - Malcolm Chapman
Scotland's National Collections and the Digital Humanities - Malcolm ChapmanScotland's National Collections and the Digital Humanities - Malcolm Chapman
Scotland's National Collections and the Digital Humanities - Malcolm Chapman
 
Glenn Cumiskey - UKAD 2016 forum
Glenn Cumiskey - UKAD 2016 forumGlenn Cumiskey - UKAD 2016 forum
Glenn Cumiskey - UKAD 2016 forum
 
Mass observation behind bars | DCDC14
Mass observation behind bars | DCDC14Mass observation behind bars | DCDC14
Mass observation behind bars | DCDC14
 
From Research Library to Research Services
From Research Library to Research ServicesFrom Research Library to Research Services
From Research Library to Research Services
 
NCompass Live: Database Roadshow Teaser
NCompass Live: Database Roadshow TeaserNCompass Live: Database Roadshow Teaser
NCompass Live: Database Roadshow Teaser
 
BL Labs Presentation to Social Sciences Team at the British Library
BL Labs Presentation to Social Sciences Team at the British LibraryBL Labs Presentation to Social Sciences Team at the British Library
BL Labs Presentation to Social Sciences Team at the British Library
 
Exhibition recommendation using British Museum data and Event Registry - ESWC...
Exhibition recommendation using British Museum data and Event Registry - ESWC...Exhibition recommendation using British Museum data and Event Registry - ESWC...
Exhibition recommendation using British Museum data and Event Registry - ESWC...
 
Missing links closing talk - with notes
Missing links closing talk - with notesMissing links closing talk - with notes
Missing links closing talk - with notes
 
Dealing with research data
Dealing with research dataDealing with research data
Dealing with research data
 

Similar to Peter webster interrogating the archived uk web

Supporting research with open services at the British Library, Sara Gould, Op...
Supporting research with open services at the British Library, Sara Gould, Op...Supporting research with open services at the British Library, Sara Gould, Op...
Supporting research with open services at the British Library, Sara Gould, Op...
Crossref
 
Digital Research Support by Stella Wisdom, 20th & 21st Century Collections
Digital Research Support by Stella Wisdom, 20th & 21st Century CollectionsDigital Research Support by Stella Wisdom, 20th & 21st Century Collections
Digital Research Support by Stella Wisdom, 20th & 21st Century Collections
Digital Research and Curator Team @ British Library
 
Prospects and pitfalls in using web archives for research
Prospects and pitfalls in using web archives for researchProspects and pitfalls in using web archives for research
Prospects and pitfalls in using web archives for research
Peter Webster
 
Supporting the Digital Scholar: Experiences from the British Library Labs
Supporting the Digital Scholar:Experiences from the British Library LabsSupporting the Digital Scholar:Experiences from the British Library Labs
Supporting the Digital Scholar: Experiences from the British Library Labs
labsbl
 
Digging into the Web Archive at the British Library 2014-11-27
Digging into the Web Archive at the British Library 2014-11-27Digging into the Web Archive at the British Library 2014-11-27
Digging into the Web Archive at the British Library 2014-11-27
Andy Jackson
 
British Library Labs Leeds Roadshow 2018
British Library Labs Leeds Roadshow 2018British Library Labs Leeds Roadshow 2018
British Library Labs Leeds Roadshow 2018
labsbl
 
Quantifying the impacts of investment in humanities archives
Quantifying the impacts of investment in humanities archivesQuantifying the impacts of investment in humanities archives
Quantifying the impacts of investment in humanities archives
Eric Meyer
 
British Library Labs - Bodleian - University of Oxford
British Library Labs - Bodleian - University of OxfordBritish Library Labs - Bodleian - University of Oxford
British Library Labs - Bodleian - University of Oxford
labsbl
 
BL Labs presentation given to the Digital Scholarship Team
BL Labs presentation given to the Digital Scholarship TeamBL Labs presentation given to the Digital Scholarship Team
BL Labs presentation given to the Digital Scholarship Team
labsbl
 
International Digital Library Initiatives
International Digital Library InitiativesInternational Digital Library Initiatives
International Digital Library Initiatives
Dept of Library and Information Science Tumkur University
 
British Library Labs - Open University Presentation - 3 April 2014, 1100-1200
British Library Labs - Open University Presentation - 3 April 2014, 1100-1200British Library Labs - Open University Presentation - 3 April 2014, 1100-1200
British Library Labs - Open University Presentation - 3 April 2014, 1100-1200
labsbl
 
Presentation to the National Science Library of the Chinese Academy of Sciences
Presentation to the National Science Library of the Chinese Academy of SciencesPresentation to the National Science Library of the Chinese Academy of Sciences
Presentation to the National Science Library of the Chinese Academy of Sciences
labsbl
 
Digital archaeology and museums
Digital archaeology and museumsDigital archaeology and museums
Digital archaeology and museums
dejp3
 
Building a Collection of the Historical UK Web for scholarly use
Building a Collection of the Historical UK Web for scholarly useBuilding a Collection of the Historical UK Web for scholarly use
Building a Collection of the Historical UK Web for scholarly use
ALISS
 
Working with the archived web, 1996-2013
Working with the archived web, 1996-2013 Working with the archived web, 1996-2013
Working with the archived web, 1996-2013
labsbl
 
More than just books - British Library Labs Presentation given at MSc Compute...
More than just books - British Library Labs Presentation given at MSc Compute...More than just books - British Library Labs Presentation given at MSc Compute...
More than just books - British Library Labs Presentation given at MSc Compute...
labsbl
 
The Biodiversity Heritage Library
The Biodiversity Heritage LibraryThe Biodiversity Heritage Library
The Biodiversity Heritage Library
Martin Kalfatovic
 
eResearch-Oz-Bellamy
eResearch-Oz-BellamyeResearch-Oz-Bellamy
eResearch-Oz-Bellamy
Craig Bellamy
 
Open, Digital, Inclusive: Unleashing Knowledge
Open, Digital, Inclusive: Unleashing KnowledgeOpen, Digital, Inclusive: Unleashing Knowledge
Open, Digital, Inclusive: Unleashing Knowledge
labsbl
 
British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...
British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...
British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...
labsbl
 

Similar to Peter webster interrogating the archived uk web (20)

Supporting research with open services at the British Library, Sara Gould, Op...
Supporting research with open services at the British Library, Sara Gould, Op...Supporting research with open services at the British Library, Sara Gould, Op...
Supporting research with open services at the British Library, Sara Gould, Op...
 
Digital Research Support by Stella Wisdom, 20th & 21st Century Collections
Digital Research Support by Stella Wisdom, 20th & 21st Century CollectionsDigital Research Support by Stella Wisdom, 20th & 21st Century Collections
Digital Research Support by Stella Wisdom, 20th & 21st Century Collections
 
Prospects and pitfalls in using web archives for research
Prospects and pitfalls in using web archives for researchProspects and pitfalls in using web archives for research
Prospects and pitfalls in using web archives for research
 
Supporting the Digital Scholar: Experiences from the British Library Labs
Supporting the Digital Scholar:Experiences from the British Library LabsSupporting the Digital Scholar:Experiences from the British Library Labs
Supporting the Digital Scholar: Experiences from the British Library Labs
 
Digging into the Web Archive at the British Library 2014-11-27
Digging into the Web Archive at the British Library 2014-11-27Digging into the Web Archive at the British Library 2014-11-27
Digging into the Web Archive at the British Library 2014-11-27
 
British Library Labs Leeds Roadshow 2018
British Library Labs Leeds Roadshow 2018British Library Labs Leeds Roadshow 2018
British Library Labs Leeds Roadshow 2018
 
Quantifying the impacts of investment in humanities archives
Quantifying the impacts of investment in humanities archivesQuantifying the impacts of investment in humanities archives
Quantifying the impacts of investment in humanities archives
 
British Library Labs - Bodleian - University of Oxford
British Library Labs - Bodleian - University of OxfordBritish Library Labs - Bodleian - University of Oxford
British Library Labs - Bodleian - University of Oxford
 
BL Labs presentation given to the Digital Scholarship Team
BL Labs presentation given to the Digital Scholarship TeamBL Labs presentation given to the Digital Scholarship Team
BL Labs presentation given to the Digital Scholarship Team
 
International Digital Library Initiatives
International Digital Library InitiativesInternational Digital Library Initiatives
International Digital Library Initiatives
 
British Library Labs - Open University Presentation - 3 April 2014, 1100-1200
British Library Labs - Open University Presentation - 3 April 2014, 1100-1200British Library Labs - Open University Presentation - 3 April 2014, 1100-1200
British Library Labs - Open University Presentation - 3 April 2014, 1100-1200
 
Presentation to the National Science Library of the Chinese Academy of Sciences
Presentation to the National Science Library of the Chinese Academy of SciencesPresentation to the National Science Library of the Chinese Academy of Sciences
Presentation to the National Science Library of the Chinese Academy of Sciences
 
Digital archaeology and museums
Digital archaeology and museumsDigital archaeology and museums
Digital archaeology and museums
 
Building a Collection of the Historical UK Web for scholarly use
Building a Collection of the Historical UK Web for scholarly useBuilding a Collection of the Historical UK Web for scholarly use
Building a Collection of the Historical UK Web for scholarly use
 
Working with the archived web, 1996-2013
Working with the archived web, 1996-2013 Working with the archived web, 1996-2013
Working with the archived web, 1996-2013
 
More than just books - British Library Labs Presentation given at MSc Compute...
More than just books - British Library Labs Presentation given at MSc Compute...More than just books - British Library Labs Presentation given at MSc Compute...
More than just books - British Library Labs Presentation given at MSc Compute...
 
The Biodiversity Heritage Library
The Biodiversity Heritage LibraryThe Biodiversity Heritage Library
The Biodiversity Heritage Library
 
eResearch-Oz-Bellamy
eResearch-Oz-BellamyeResearch-Oz-Bellamy
eResearch-Oz-Bellamy
 
Open, Digital, Inclusive: Unleashing Knowledge
Open, Digital, Inclusive: Unleashing KnowledgeOpen, Digital, Inclusive: Unleashing Knowledge
Open, Digital, Inclusive: Unleashing Knowledge
 
British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...
British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...
British Library Labs Presentation at Ed Tech Hackathon 2013 - hackathoncentra...
 

More from Digital History

Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020
Digital History
 
Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020
Digital History
 
Commemorating the Great War on Twitter
Commemorating the Great War on TwitterCommemorating the Great War on Twitter
Commemorating the Great War on Twitter
Digital History
 
Community Archives and Ethics
Community Archives and EthicsCommunity Archives and Ethics
Community Archives and Ethics
Digital History
 
Contemporary web archives ihr
Contemporary web archives ihrContemporary web archives ihr
Contemporary web archives ihr
Digital History
 
The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...
The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...
The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...
Digital History
 
The Language of Migration in the Victorian Press: A Corpus Linguistic Approach
The Language of Migration in the Victorian Press: A Corpus Linguistic ApproachThe Language of Migration in the Victorian Press: A Corpus Linguistic Approach
The Language of Migration in the Victorian Press: A Corpus Linguistic Approach
Digital History
 
Identifying responses to revolution
Identifying responses to revolutionIdentifying responses to revolution
Identifying responses to revolution
Digital History
 
Chance encounters with the past
Chance encounters with the pastChance encounters with the past
Chance encounters with the past
Digital History
 
The lives and criminal careers of juvenile offenders
The lives and criminal careers of juvenile offendersThe lives and criminal careers of juvenile offenders
The lives and criminal careers of juvenile offenders
Digital History
 
History of teaching ihr
History of teaching ihrHistory of teaching ihr
History of teaching ihr
Digital History
 
Tudor Intelligence Networks - Ruth Ahnert
Tudor Intelligence Networks - Ruth AhnertTudor Intelligence Networks - Ruth Ahnert
Tudor Intelligence Networks - Ruth Ahnert
Digital History
 
The Pictorial publisher - Agents technologies and the illustrrated book in Br...
The Pictorial publisher - Agents technologies and the illustrrated book in Br...The Pictorial publisher - Agents technologies and the illustrrated book in Br...
The Pictorial publisher - Agents technologies and the illustrrated book in Br...
Digital History
 
Cordell scientific american
Cordell scientific americanCordell scientific american
Cordell scientific american
Digital History
 
Mapping paris
Mapping parisMapping paris
Mapping paris
Digital History
 
Political Meetings Mapper with British Library Labs: mapping the origins of B...
Political Meetings Mapper with British Library Labs: mapping the origins of B...Political Meetings Mapper with British Library Labs: mapping the origins of B...
Political Meetings Mapper with British Library Labs: mapping the origins of B...
Digital History
 
European or Imperial Metropolis? Depictions of London in British Newspapers, ...
European or Imperial Metropolis? Depictions of London in British Newspapers, ...European or Imperial Metropolis? Depictions of London in British Newspapers, ...
European or Imperial Metropolis? Depictions of London in British Newspapers, ...
Digital History
 
The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...
The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...
The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...
Digital History
 
Emma Bayne: ‘Traces Through Time overview and next steps’
Emma Bayne: ‘Traces Through Time overview and next steps’ Emma Bayne: ‘Traces Through Time overview and next steps’
Emma Bayne: ‘Traces Through Time overview and next steps’
Digital History
 
Ihr june15-evans
Ihr june15-evansIhr june15-evans
Ihr june15-evans
Digital History
 

More from Digital History (20)

Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020
 
Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020Ihr dig hist_teachingpanel_feb2020
Ihr dig hist_teachingpanel_feb2020
 
Commemorating the Great War on Twitter
Commemorating the Great War on TwitterCommemorating the Great War on Twitter
Commemorating the Great War on Twitter
 
Community Archives and Ethics
Community Archives and EthicsCommunity Archives and Ethics
Community Archives and Ethics
 
Contemporary web archives ihr
Contemporary web archives ihrContemporary web archives ihr
Contemporary web archives ihr
 
The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...
The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...
The ‘Digital Thematic Deconstruction’ of early modern urban maps and bird’s-e...
 
The Language of Migration in the Victorian Press: A Corpus Linguistic Approach
The Language of Migration in the Victorian Press: A Corpus Linguistic ApproachThe Language of Migration in the Victorian Press: A Corpus Linguistic Approach
The Language of Migration in the Victorian Press: A Corpus Linguistic Approach
 
Identifying responses to revolution
Identifying responses to revolutionIdentifying responses to revolution
Identifying responses to revolution
 
Chance encounters with the past
Chance encounters with the pastChance encounters with the past
Chance encounters with the past
 
The lives and criminal careers of juvenile offenders
The lives and criminal careers of juvenile offendersThe lives and criminal careers of juvenile offenders
The lives and criminal careers of juvenile offenders
 
History of teaching ihr
History of teaching ihrHistory of teaching ihr
History of teaching ihr
 
Tudor Intelligence Networks - Ruth Ahnert
Tudor Intelligence Networks - Ruth AhnertTudor Intelligence Networks - Ruth Ahnert
Tudor Intelligence Networks - Ruth Ahnert
 
The Pictorial publisher - Agents technologies and the illustrrated book in Br...
The Pictorial publisher - Agents technologies and the illustrrated book in Br...The Pictorial publisher - Agents technologies and the illustrrated book in Br...
The Pictorial publisher - Agents technologies and the illustrrated book in Br...
 
Cordell scientific american
Cordell scientific americanCordell scientific american
Cordell scientific american
 
Mapping paris
Mapping parisMapping paris
Mapping paris
 
Political Meetings Mapper with British Library Labs: mapping the origins of B...
Political Meetings Mapper with British Library Labs: mapping the origins of B...Political Meetings Mapper with British Library Labs: mapping the origins of B...
Political Meetings Mapper with British Library Labs: mapping the origins of B...
 
European or Imperial Metropolis? Depictions of London in British Newspapers, ...
European or Imperial Metropolis? Depictions of London in British Newspapers, ...European or Imperial Metropolis? Depictions of London in British Newspapers, ...
European or Imperial Metropolis? Depictions of London in British Newspapers, ...
 
The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...
The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...
The Challenge of Digital Sources in the Web Age: Common Tensions Across Three...
 
Emma Bayne: ‘Traces Through Time overview and next steps’
Emma Bayne: ‘Traces Through Time overview and next steps’ Emma Bayne: ‘Traces Through Time overview and next steps’
Emma Bayne: ‘Traces Through Time overview and next steps’
 
Ihr june15-evans
Ihr june15-evansIhr june15-evans
Ihr june15-evans
 

Recently uploaded

RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
zuzanka
 
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
National Information Standards Organization (NISO)
 
Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10
nitinpv4ai
 
How to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 InventoryHow to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 Inventory
Celine George
 
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skillsspot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
haiqairshad
 
Bonku-Babus-Friend by Sathyajith Ray (9)
Bonku-Babus-Friend by Sathyajith Ray  (9)Bonku-Babus-Friend by Sathyajith Ray  (9)
Bonku-Babus-Friend by Sathyajith Ray (9)
nitinpv4ai
 
How Barcodes Can Be Leveraged Within Odoo 17
How Barcodes Can Be Leveraged Within Odoo 17How Barcodes Can Be Leveraged Within Odoo 17
How Barcodes Can Be Leveraged Within Odoo 17
Celine George
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
PsychoTech Services
 
Stack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 MicroprocessorStack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 Microprocessor
JomonJoseph58
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
ssuser13ffe4
 
Juneteenth Freedom Day 2024 David Douglas School District
Juneteenth Freedom Day 2024 David Douglas School DistrictJuneteenth Freedom Day 2024 David Douglas School District
Juneteenth Freedom Day 2024 David Douglas School District
David Douglas School District
 
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptxPengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Fajar Baskoro
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
TechSoup
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
Himanshu Rai
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
Nguyen Thanh Tu Collection
 
skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)
Mohammad Al-Dhahabi
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
iammrhaywood
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
Krassimira Luka
 
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
EduSkills OECD
 

Recently uploaded (20)

RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
 
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
Jemison, MacLaughlin, and Majumder "Broadening Pathways for Editors and Authors"
 
Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10Haunted Houses by H W Longfellow for class 10
Haunted Houses by H W Longfellow for class 10
 
How to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 InventoryHow to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 Inventory
 
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skillsspot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
 
Bonku-Babus-Friend by Sathyajith Ray (9)
Bonku-Babus-Friend by Sathyajith Ray  (9)Bonku-Babus-Friend by Sathyajith Ray  (9)
Bonku-Babus-Friend by Sathyajith Ray (9)
 
How Barcodes Can Be Leveraged Within Odoo 17
How Barcodes Can Be Leveraged Within Odoo 17How Barcodes Can Be Leveraged Within Odoo 17
How Barcodes Can Be Leveraged Within Odoo 17
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
 
Stack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 MicroprocessorStack Memory Organization of 8086 Microprocessor
Stack Memory Organization of 8086 Microprocessor
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
 
Juneteenth Freedom Day 2024 David Douglas School District
Juneteenth Freedom Day 2024 David Douglas School DistrictJuneteenth Freedom Day 2024 David Douglas School District
Juneteenth Freedom Day 2024 David Douglas School District
 
Pengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptxPengantar Penggunaan Flutter - Dart programming language1.pptx
Pengantar Penggunaan Flutter - Dart programming language1.pptx
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
 
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem studentsRHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
 
skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)skeleton System.pdf (skeleton system wow)
skeleton System.pdf (skeleton system wow)
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
 
Temple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation resultsTemple of Asclepius in Thrace. Excavation results
Temple of Asclepius in Thrace. Excavation results
 
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...
 

Peter webster interrogating the archived uk web

  • 1. Interrogating the archived UK web Peter Webster (British Library) @pj_webster / @UKWebArchive webarchive.org.uk
  • 2. www.bl.uk 2 Big UK Domain Data for the Arts & Humanities • Led by Dr Jane Winters (IHR) (@jfwinters) • In partnership with the British Library and the Oxford Internet Institute • With the help of Niels Brügger (Aarhus University, Denmark) @NielsBr • Co-investigators: Ralph Schroeder, Eric Meyer (@etmeyer), Helen Hockx-Yu (@hhockx) • The team includes: Jonathan Blaney, @JoshCowls, @anjacks0n • Funded by the AHRC, Jan 2014 – March 2015 • http://buddah.projects.history.ac.uk/
  • 3. www.bl.uk 3 Project aims • To highlight the value of web archives as a source for A&H, & to transform the way in which researchers interact with the data • To establish a theoretical and methodological framework for the analysis of web archives, focusing on the JISC UK Web Domain Dataset • To explore the ethical implications of big data research, and particularly as they relate to the web • To inform collection development and access arrangements for the UK web archive at the British Library
  • 4. www.bl.uk 4 Project outputs • a suite of tools to support analysis of web archives by A&H researchers • an enhanced interface through which researchers access the archived material held by the British Library • a history of the development of UK web space from 1996 to 2013, analysing technical, social, organisational and cultural developments and trends in the dataset • a series of case studies across a range of A&H disciplines • two project workshops, bringing together researchers, archivists, technologists, and digital preservation professionals • a free online training module illustrating the use of web archives and the application of big data techniques and methods.
  • 5. www.bl.uk 5 Forthcoming event Web archives as big data Wednesday, 3 December 2014 from 09:45 to 17:30 IHR, Senate House, United Kingdom Booking at: http://tinyurl.com/webarchives
  • 6. www.bl.uk 6 A new class of primary source ? Deswarte and Webster, “Web Archives: A New Class of Primary Source for Historians?” IHR Digital History seminar, 2013, reporting on predecessor project (AADDA) http://tinyurl.com/qca3yy5
  • 8. www.bl.uk 8 The UK Web Archive: three archives in one Open UK Web Archive (2004-) • c.14,000 sites • Curated, selective, permission-based • webarchive.org.uk Legal Deposit UK Web Archive (2013-) • legal framework • c.4-5 million hosts per year • onsite only JISC UK Web Domain Dataset
  • 9. www.bl.uk 9 JISC UK Web Domain Dataset 1996-2013 • Funded by JISC to create a research collection of UK websites • Collaboration between the Internet Archive, JISC and the British Library • Copy of subset of the Internet Archive’s web collection that relates to the UK • c.300 million resources, 60TB in total • No local access – possible through the Internet Archive • Can be used to generate secondary datasets
  • 10. www.bl.uk 10 Use cases (generalised) • Full-text/facet search -> individual resource • Full-text/facet search -> analysis/visualisation • Search -> corpus creation -> annotation/curation • Corpus creation -> full-text search -> individual resource • Corpus -> search -> analysis/visualisation • Derived datasets -> take-away • Direct access to WARC -> take-away
  • 11. www.bl.uk 11 What do we know about each resource ? From the crawl data • crawl date • URL (/page.html, host.domain.co.uk, domain.co.uk, .co.uk) • file format • file size
  • 12. www.bl.uk 12 What do we know about each resource ? From the full-text index • page title • link destinations (host.domain.co.uk, domain.co.uk, .co.uk) • author (sometimes) • language (sometimes)
  • 13. www.bl.uk 13 What *don’t* we know ? • subject • geographic scope • publisher • date of publication • date of last amendment
  • 14. www.bl.uk 14 Thank you ! peterwebster6@gmail.com @pj_webster / @UKWebArchive britishlibrary.typepad.co.uk/webarchive webarchive.org.uk peterwebster.me