Distributed Databases and Applications Presentation
Upcoming SlideShare
Loading in...5
×
 

Distributed Databases and Applications Presentation

on

  • 423 views

 

Statistics

Views

Total Views
423
Views on SlideShare
423
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Distributed Databases and Applications Presentation Distributed Databases and Applications Presentation Presentation Transcript

  • Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC Berkeley
  • Distributed Databases – Discipline-specific
    • The Species Analyst ( TSA )
    • The Integrated Taxonomic Information System ( ITIS )
    • FishNet
    • The Mammal Networked Information System ( MaNIS )
    • HerpNET
    • The Ornithological Information System (ORNIS)
  • Distributed Databases – International
    • European Natural History Science Information Network ( ENHSIN )
    • Biological Collection Access for Europe ( BioCASE )
    • Australia Virtual Herbarium ( AVH )
    • Red Mundial de Información Sobre Biodiversidad, Comisión Nacional para el Conocimiento y Uso de la Biodiversidad ( REMIB , CONABIO)
  • Distributed Databases – Regional
    • Mountain and Plains Spatio-Temporal Database-Informatics ( MaPSTeDI )
    • Ocean Biogeographic Information System ( OBIS )
    • Pacific Basin Information Node, National Biological Information Infrastructure ( PBIN , NBII)
    • Species Link, Centro de Referência em Informação Ambiental ( Species Link , CRIA)
    • A Virtual Herbarium of the Chicago Region ( vPlants )
    • Spatial Analysis of Local Vegetation Inventories Across Scales ( SALVIAS )
  • Distributed Databases – Intra-institutional
    • Berkeley Natural History Museums ( BNHM )
    • Association of Biological Collections, UC Davis
  • Distributed Databases – “Nodes”
    • LifeMapper
    • National Biodiversity Information Infrastructure ( NBII )
    • Global Biodiversity Information Facility ( GBIF )
  • GBIF Work Programmes
    • NODES
    • ECAT – Electronic Catalogue of Names of Known Organisms
    • DIGIT – Digitisation of Natural History Collections
    • OCB – Outreach and Capacity Building
    • DADI – Data Access and Database Interoperability
  • Taxonomic Database Working Group
    • Standards development and maintenance
      • Access to Biological Collections Data ( ABCD )
      • Darwin Core Version 2 ( DwC2 )
      • Structure of Descriptive Data (SDD)
      • DiGIR
      • Others…
  • DiGIR Di stributed G eneric I nformation R etrieval John Wieczorek, Stan Blum, Dave Vieglais, P.J. Schwartz
  • Information Retrieval
    • Distributed - a protocol for retrieving structured data from multiple, heterogeneous databases across the Internet.
    • Generic - a protocol independent of the data retrieved and of the software to retrieve it.
  • Project Rationale
    • Avoid multiple incongruous development efforts
    • Pool resources and create a support community of experts
    • Solve scalability problems
  • Design Goals
    • Use open protocols and standards, such as HTTP and XML
    • Decouple the protocol, software and semantics
    • Make new data provider installations as easy as possible
    • Develop open source software with GNU General Public Licensing (It’s free).
  • DiGIR Component Summary
  • DiGIR Architecture
    • Provider
  • Provider
    • Receives requests
    • Retrieves data from database
    • Sends results to requestor
    • Supplies metadata to describe content, contacts, and capabilities
    • Logs requests
  • DiGIR Architecture
    • Portal Engine
  • Portal Engine
    • The entry point for an application
    • Can query a registry to discover potential providers
    • Can determine, based on provider metadata, whether a provider should be queried
    • Can send requests to multiple providers
  • Portal Engine, continued
    • Assembles responses from providers
    • Returns packaged results to the requesting application
    • Communicates via protocol compliant messaging only
    • Logs activity
  • Registry
    • Provides a “yellow pages” to advertise the existence and capabilities of a provider
    • Provides a means to discover potential providers of interest
    • May be public or private
    • Need not be a part of the architecture
  • DiGIR Architecture
    • Provider
    • Registry (register)
  • DiGIR Architecture
    • Portal Engine
    • Registry (discover)
  • DiGIR Protocol
    • Defines request and response message formats for communication between provider, portal engine, and applications
      • Metadata requests
      • Search requests
      • Inventory requests
    • Remains unfettered by the structure of the data it transfers
  • DiGIR Architecture
    • Application
  • DiGIR Architecture
    • Application
    • Protocol (request)
    • Portal Engine
  • DiGIR Architecture
    • Application
    • Protocol (request)
    • Portal Engine
    • Protocol (request)
    • Provider
  • DiGIR Architecture
    • Application
    • Protocol (request)
    • Portal Engine
    • Protocol (response)
    • Provider
  • DiGIR Architecture
    • Application
    • Protocol (response)
    • Portal Engine
  • Applications
    • Must be able to assemble and send a request document to a portal
    • Must be able to receive and interpret a response document from the portal
    • Must do something incredibly useful and interesting with the data
    • This is where the real fun is!
  • Hot topics – Interesting problems
    • Persistent unique identifiers
    • Web services (converters, translators, calculators, transformation services)
    • Data validation (taxonomic and geographic thesauri, ecological niche modeling, expedition analysis, outlier detection,…)
    • Spatial query interfaces
    • Data quality feedback mechanisms
    • Automated georeferencing, event gazetteers
    • Concept libraries, schema extensions, schema libraries, federated ontologies
  • MaNIS – The Mammal Networked Information System It’s more than just a pangolin…
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database Online MS Access Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database Online MS Access Database Online MS Access Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database UAM Oracle Database Online MS Access Database Online MS Access Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database Online MS Access Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MVZ-MaNIS Presentation Layer LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • MaNIS Network Configuration MaNIS DiGIR Portal MaNIS DiGIR Portal MaNIS DiGIR Portal MVZ- MaNIS Presentation Layer UMNH- MaNIS Presentation Layer UWBM- MaNIS Presentation Layer … … … … … … … … LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • “ CalNet” Network Configuration CalNet DiGIR Portal LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • Event Gazetteer Network Configuration BioGeomancer Web Service CalNet DiGIR Portal LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • NBII Network Configuration NBII DiGIR Portal LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • GBIF Network Configuration GBIF Presentation Layers NBII DiGIR Portal LACM MS Access Database Online MS Access Database MVZ Sybase Database LSUMZ 4D-Mac Database Online MS Access Database UAM Oracle Database Online MS Access Database DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider DiGIR Provider CAS SQL Server Database
  • Intra-Network Configuration (BNMH) BNHM DiGIR Portal BNHM Presentation Layer PHMA Working Database Online Database UCBG Working Database DiGIR Provider UCJEPS Working Database Online Database UCMP Working Databases (4) Online Database Essig Working Database Online Database Online Database
  • Other Network Configurations Working Database Online Database DiGIR Provider DiGIR Provider DiGIR Portal Working Database Online Database DiGIR Provider Working Database Online Database DiGIR Portal Working Database Online Database DiGIR Provider DiGIR Portal Working Database
  • Other Network Configurations Working Database Online Database DiGIR Provider DiGIR Provider DiGIR Portal Working Database Online Database DiGIR Provider Working Database Online Database DiGIR Portal Working Database Online Database DiGIR Provider DiGIR Portal Working Database
  • Other Network Configurations Working Database Online Database DiGIR Provider DiGIR Provider DiGIR Portal Working Database Online Database DiGIR Provider Working Database Online Database DiGIR Portal Working Database Online Database DiGIR Provider DiGIR Portal Working Database
  • Other Network Configurations Working Database Online Database DiGIR Provider DiGIR Provider DiGIR Portal Working Database Online Database DiGIR Provider Working Database Online Database DiGIR Portal Working Database Online Database DiGIR Provider DiGIR Portal Working Database
  • Other Network Configurations Working Database Online Database DiGIR Provider DiGIR Provider DiGIR Portal Working Database Online Database DiGIR Provider Working Database Online Database DiGIR Portal Working Database Online Database DiGIR Provider DiGIR Portal Working Database
  • Other Network Configurations Working Database Online Database DiGIR Provider DiGIR Portal Working Database Working Database
  • Project Information
    • DiGIR is a collaborative open source development project on SourceForge ( https:// sourceforge.net/projects/digir ).
    • Software and documentation are available on the DiGIR web site ( http://digir.net ).
    • MaNIS is an international network collaboration among mammal specimen collections ( http:// elib.cs.berkeley.edu/manis ).
  • Hot topics – Interesting problems
    • Persistent unique identifiers
    • Web services (converters, translators, calculators, transformation services)
    • Data validation (taxonomic and geographic thesauri, ecological niche modeling, expedition analysis, outlier detection,…)
    • Spatial query interfaces
    • Data quality feedback mechanisms
    • Automated georeferencing, event gazetteers
    • Concept libraries, schema extensions, schema libraries, federated ontologies