Beyond Open Access: Open Data, Web services, and Semantics Eric Kansa UC Berkeley School of Information Unless otherwise indicated, this work is licensed under a Creative Commons Attribution 3.0 License <http://creativecommons.org/licenses/by/3.0/> Sarah Whitcher Kansa Alexandria Archive Institute

Web 2.0 and Open Find research faster, easier Build on the work of others Facilitate creative / innovative reuse (“remixes”, “mashups”)

Find research faster, easier

Build on the work of others

Facilitate creative / innovative reuse (“remixes”, “mashups”)

Web 2.0 Challenges Smaller, highly specialized communities Structured data is hard for Web 2.0 collaboration Work-flow, incentive, user experience

Smaller, highly specialized communities

Structured data is hard for Web 2.0 collaboration

Work-flow, incentive, user experience

Web 2.0 Challenges Smaller, highly specialized communities Structured data is hard for Web 2.0 collaboration Work-flow, incentive, user experience

Smaller, highly specialized communities

Structured data is hard for Web 2.0 collaboration

Work-flow, incentive, user experience

Open Context makes primary research easy to share , use and cite … … and is freely accessible across the globe

OCHRE, Open Context OCHRE: Fully supports ArchaeoML global schema using a native XML database and free java client Open Context: Uses a subset of the ArchaeoML schema (via MySQL/PHP) for web-browser access. Not a repository (partnerships needed) Common services, including complex querying and analysis for pooled content

OCHRE: Fully supports ArchaeoML global schema using a native XML database and free java client

Open Context: Uses a subset of the ArchaeoML schema (via MySQL/PHP) for web-browser access. Not a repository (partnerships needed)

Common services, including complex querying and analysis for pooled content

Global Schemas: ArchaeoML Simple, general schema makes it easier to pool diverse content Not overly determined, support multiple research agendas Hard to implement . But we’re gaining experience UML Diagram of a subset of ArchaeoML

Simple, general schema makes it easier to pool diverse content

Not overly determined, support multiple research agendas

Hard to implement . But we’re gaining experience

Open Context Components Two Parts : Open access interfaces for browse, “tagging”, analysis, export of public data Penelope data import and schema mapping tool for publishing

Two Parts :

Open access interfaces for browse, “tagging”, analysis, export of public data

Penelope data import and schema mapping tool for publishing

Faceted Browse Data from multiple projects browsed, queried (even with Boolean algebra), and results pooled together Individual records dynamically exposed

Data from multiple projects browsed, queried (even with Boolean algebra), and results pooled together

Individual records dynamically exposed

Summary Statistics

Records in Open Context Contextual relationships: (Spatial containment)

Media Record

Discussion Caveats Caveats : Up to now, we’ve been validating a data model. Need user experience study & optimization Too early to talk of use of “classic” Web 2.0 tools. (First major complete dataset went live in December)

Caveats :

Up to now, we’ve been validating a data model. Need user experience study & optimization

Too early to talk of use of “classic” Web 2.0 tools. (First major complete dataset went live in December)

Open Context Tagging

Open Context Tagging Pingback: Register of a link made to a set tagged as “weaving tools” from a weblog To Do: Link Open Context tagging with Del.icio.us / Connotea

Open Context & Search PHP/MySQL: Drives many dynamic content websites, relatively simple standard technology. “Bleeding edge” difficult for our target community. Easy integration of Open Source Tools (RSS-feeds, ping-back, etc.) Open to Search Engines: Increasingly important research tools (Harley 2006) 2 Diane Harley, Sarah Earl-Novell, Jennifer Arter, Shannon Lawrence, and C. Judson King, Jr. “The Influence of Academic Values on Scholarly Publication and Communication Practices”, Research & Occasional Paper Series: CSHE.13.06 <http://cshe.berkeley.edu/publications/docs/ROP.Harley.AcademicValues.13.06.pdf>.

PHP/MySQL: Drives many dynamic content websites, relatively simple standard technology. “Bleeding edge” difficult for our target community.

Easy integration of Open Source Tools (RSS-feeds, ping-back, etc.)

Open to Search Engines: Increasingly important research tools (Harley 2006)

Open Context Referrals from Search Engine Queries (based on 3,611 searches, Jan-Mar 2008) Note: 2585 Search requests for “context” excluded from this diagram. Person Place Object “ Long Tail”- 53% of searches

Ownership in Open Context Citation information with stable URL direct to the item being cited.

Ownership in Open Context Unique for every single item in the database!

Ownership in Open Context Zotero (www.zotero.org) uses COinS (a micro-format) metadata to make bibliographic references

Ownership in Open Context Copyright ownership and Creative Commons license information, including RDF metadata Internet-wide standard metadata, links ownership & permissions

Penelope Pictures Suggests need for more information about people! Links with Facebook? LinkedIn? Field notes should document people more?

Suggests need for more information about people!

Links with Facebook? LinkedIn?

Field notes should document people more?

Web 2.0 and Open Find research faster, easier Build on the work of others Facilitate creative / innovative reuse (“remixes”, “mashups”)

Find research faster, easier

Build on the work of others

Facilitate creative / innovative reuse (“remixes”, “mashups”)

Records in Open Context XML data output, enables: (1) Sharing between web resources (2) Custom presentation (Zooarcheology, or Brown University-specific style templates, etc.)

Faceted Browse Petra Great Temple: 128,187 locations / objects 1.1 million descriptions 1626 media objects ( more to come ) 298,500 relationships

Petra Great Temple:

128,187 locations / objects

1.1 million descriptions

1626 media objects ( more to come )

298,500 relationships

Faceted Browse RESTful Web Service under the facet browse feature Very flexible way to query Open Context to get XML data

RESTful Web Service under the facet browse feature

Very flexible way to query Open Context to get XML data

Back to the Long Tail: Open Search Recently implemented the “Open Search Standard” Send Open Context “Google-like” search string, return results in Atom (XML) format No need to understand ArchaeoML Easy way for distributed search across multiple projects! ( More use of all initiatives )

Recently implemented the “Open Search Standard”

Send Open Context “Google-like” search string, return results in Atom (XML) format

No need to understand ArchaeoML

Easy way for distributed search across multiple projects! ( More use of all initiatives )

XML/XSLT enables us to have multiple interfaces optimized for different users / types of content Media heavy? Data analysis/visualization (Swivel tools?) Requires in-depth study of researchers as they work User Experience Image by Jeff Kubina via Flickr (CC-by license) <http://www.flickr.com/photos/kubina/296367267/>

XML/XSLT enables us to have multiple interfaces optimized for different users / types of content

Media heavy?

Data analysis/visualization (Swivel tools?)

Requires in-depth study of researchers as they work

Open Context Components Two Parts : Open access interfaces for browse, “tagging”, analysis, export of public data Penelope data import and schema mapping tool for publishing

Two Parts :

Open access interfaces for browse, “tagging”, analysis, export of public data

Penelope data import and schema mapping tool for publishing

Schema Mapping into ArchaeoML Importer an important part, most people work with Excel, Filemaker, Access… Goal: Individuals can upload their own data, map them into ArchaeoML and submit for review and publishing

Importer an important part, most people work with Excel, Filemaker, Access…

Goal: Individuals can upload their own data, map them into ArchaeoML and submit for review and publishing

Penelope 2

Penelope 2 Petra Great Temple: 12 individual databases (some very large) ~200 text documents with embedded tables 1600+ related media files “ Stress-test” for Penelope!

Petra Great Temple:

12 individual databases (some very large)

~200 text documents with embedded tables

1600+ related media files

“ Stress-test” for Penelope!

Web 2.0 Structured Data Web resources using highly generalized data structures see growing popularity Swivel Many Eyes (IBM) Freebase (Metaweb) Common store for structured data Freebase and Open Context share similar data models Still haven’t made tools for easy upload… Freebase (www.freebase.com)

Web resources using highly generalized data structures see growing popularity

Swivel

Many Eyes (IBM)

Freebase (Metaweb)

Common store for structured data

Freebase and Open Context share similar data models

Still haven’t made tools for easy upload…

Schema mapping is major challenge Leave to Open Context editors Leave to schema geeks who may draw data of interest from a repository Semi-automate (make suggested mappings) Requires in-depth study of researchers as they work User Experience Image by Jeff Kubina via Flickr (CC-by license) <http://www.flickr.com/photos/kubina/296367267/>

Schema mapping is major challenge

Leave to Open Context editors

Leave to schema geeks who may draw data of interest from a repository

Semi-automate (make suggested mappings)

Requires in-depth study of researchers as they work

Requires in-depth study of researchers as they work Key to cracking the nut of making “Web 2.0” work for archaeology Summary: Web 2.0 & User Experience Image by Jeff Kubina via Flickr (CC-by license) <http://www.flickr.com/photos/kubina/296367267/>

Requires in-depth study of researchers as they work

Key to cracking the nut of making “Web 2.0” work for archaeology

Open Context Developers Eric Kansa (Lead developer, tagging system, interface design) Ahrash Bissell (Penelope design, usability) Nathan Hirth (XML, XSLT, schema mapping) David Schloen (ArchaeoML schema) Sarah W. Kansa (Usability, interface design, documentation) Jeanne Lopiparo (Interface and graphic design, usability) Michael Ashley (Filemaker item-view mockup) Chris Hoffman (Usability, optimization )

Eric Kansa (Lead developer, tagging system, interface design)

Ahrash Bissell (Penelope design, usability)

Nathan Hirth (XML, XSLT, schema mapping)

David Schloen (ArchaeoML schema)

Sarah W. Kansa (Usability, interface design, documentation)

Jeanne Lopiparo (Interface and graphic design, usability)

Michael Ashley (Filemaker item-view mockup)

Chris Hoffman (Usability, optimization )

Special Thanks University of Chicago: OCHRE Project The Electronic Frontier Foundation Doris and Donald Fisher SAA 2008 Conference Presidio Archaeology: NPS, Golden Gate National Rec. Area Science Commons Internet Archive (media repository services)