TAPIR PyWrapper3, at GBIF GB14 nodes meeting (2007)


Published on

TAPIR PyWrapper3 training workshop for the GBIF GB14 NODES meeting in Amsterdam 15th October 2007.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Some keywords; the main topics of the talk
  • http://commons.wikimedia.org/wiki/Image:Tapir_hooves.jpg
  • http://commons.wikimedia.org/wiki/Image:Central_American_Tapir-Belize20.jpg
  • http://commons.wikimedia.org/wiki/Image:Milton_Tapir_sniffing.JPG
  • http://commons.wikimedia.org/wiki/Image:Python_molure_13.JPG
  • http://commons.wikimedia.org/wiki/Image:Dusit_zoo_tapir.jpg
  • Photo: Field been from Boreal, accession NGB11518, 2005-03-05, Dag Endresen [http://r142b.ngb.se/ngb/2005-03--the-making-of-seeds-pictures/index.php?offset=19&size=medium&stp=1]
  • Photo: Field been from Boreal, accession NGB11518, 2005-03-05, Dag Endresen [http://r142b.ngb.se/ngb/2005-03--the-making-of-seeds-pictures/index.php?offset=19&size=medium&stp=1]
  • Photo: Field been from Boreal, accession NGB11518, 2005-03-05, Dag Endresen [http://r142b.ngb.se/ngb/2005-03--the-making-of-seeds-pictures/index.php?offset=19&size=medium&stp=1]
  • http://tapir.grinfo.net/queryforms/qf_tapir.cgi?url=http://tapir.grinfo.net//pywrapper.cgi?dsa=SINGER <?xml version='1.0' encoding='UTF-8'?> <request> <header /> <inventory count='true' start='0' limit='40' xmlns:singer='http://singer.grinfo.net/schema/SINGER-1.0' > <concepts> <concept path='singer:/sourcename'/> <concept path='singer:/taxonomy/genus' /> <concept path='singer:/taxonomy/species' /> <concept path='singer:/taxonomy/subspecies' /> <concept path='singer:/holding/ID' /> <concept path='singer:/holding/name' /> <concept path='singer:/origin/collecting/countrysource' /> <concept path='singer:/origin/collecting/countrysourceID' /> <concept path='singer:/status/biologicalstatus' /> <concept path='singer:/status/biologicalstatusID' /> </concepts> <filter> <like> <concept path='singer:/taxonomy/genus' /> <literal value='cice*' /> </like> </filter> </inventory> </request>
  • http://tapir.grinfo.net/queryforms/qf_tapir.cgi?url=http://tapir.grinfo.net//pywrapper.cgi?dsa=SINGER http://apps3.fao.org/wiews/wiews.jsp
  • * Illustration: Corn earworm pupae that will be used to produce control parasites for release in the field. Photo by Scott Bauer. [http://www.ars.usda.gov/is/graphics/photos/k5554-2.htm]
  • OMG-LSR, Object Management Group – Life Science Research [http://www.omg.org/lsr/]
  • http://www.biocase.org/whats_biocase/unit_net.shtml
  • Arnica montana SPIMED Medicinal plants , Photographer Katarina Wdelsbäck (NGB Picture Archive, image 003907)
  • Image: Field of wheat at Alnarp. Photographer Dag Terje Endresen (NGB Picture Archive, image 002981) Image: Spider in a spiderweb Image: Dag Terje Filip Endresen in Benin Image: Michael Mackay
  • TAPIR PyWrapper3, at GBIF GB14 nodes meeting (2007)

    1. 1. Cover slide TAPIR P yWrapper The third generation PyWrapper October 15, 2007 NODES9, GB14, Amsterdam Dag Terje Filip Endresen Nordic Gene Bank (Sweden) Bioversity International (Italy)
    2. 2. TOPICS <ul><li>TAPIR </li></ul><ul><li>PyWrapper </li></ul>
    3. 3. <ul><li>TAPIR </li></ul>
    4. 4. TDWG Access Protocol for Information Retrieval - TAPIR <ul><li>[ http://www.tdwg.org/activities/tapir/ ] </li></ul><ul><li>TAPIR specifies a standardised, stateless, HTTP transmittable, XML-based request and response protocol for accessing structured data . </li></ul><ul><li>TAPIR combines and extends features of the BioCASE and DiGIR protocols . </li></ul><ul><li>The work on the TAPIR protocol was initiated at the 2004 TDWG conference in New Zealand. </li></ul><ul><li>The mandate of TAPIR is to unify the protocol of the DiGIR and BioCASE wrappers, which is most common in current implementations for sharing of biodiversity datasets. </li></ul><ul><li>Three alternative TAPIR reference implementations are developed: </li></ul><ul><ul><li>PyWrapper [ http://www.pywrapper.org ] </li></ul></ul><ul><ul><li>TapirLink [ http://sourceforge.net/projects/digir/ ] </li></ul></ul><ul><ul><li>TapirDotNET [ http:// sourceforge .net/projects/ tapirdotnet / ] </li></ul></ul>
    5. 5. <ul><li>Biodiversity data exchange tools </li></ul>
    6. 6. PyWrapper3 <ul><li>PyWrapper3 is based on the previous PyWrapper version 1 and 2 from the BioCASE project. </li></ul><ul><li>PyWrapper3 comes with a small python application server (CherryPy) that can be run as a standalone web server. </li></ul><ul><li>The recommended configuration is to run PyWrapper behind an Apache web server. </li></ul><ul><li>PyWrapper3 includes a LSID resolver sevice. </li></ul><ul><li>The PyWrapper3 middleware can be used to implement distributed information systems by using the TAPIR data exchange protocol. </li></ul><ul><li>All datasources wrapped by PyWrapper can be accessed in a uniform way, no matter what data structure they have. </li></ul>
    7. 7. PyWrapper Provider Software <ul><li>Required configuration: </li></ul><ul><li>Web server : Any CGI compliant web server: Apache, IIS etc. (The built in CherryPy web server can also be used). </li></ul><ul><li>Database : major databases are supported, including MySQL, Oracle, SQLServer, Sybase, Access, PostgreSQL. Theoretically any database with a Python library should work. </li></ul><ul><li>Python PyWrapper is developed with the Python programming language. (Install version 2.3 or later). </li></ul><ul><li>[http://trac.pywrapper.org/pywrapper/wiki/Downloads] </li></ul><ul><li>[With SubVersion svn://svn.pywrapper.org/pywrapper ] </li></ul>
    8. 8. Installation <ul><li>PyWrapper should run on any operating system that supports Python [ http://www.python.org/ ] . </li></ul><ul><li>Python runs on Windows, Linux/Unix, Mac OS X, OS/2, Amiga, Palm Handhelds, and Nokia mobile phones. Python has also been ported to the Java and .NET virtual machines. </li></ul><ul><li>Python is distributed under an OSI-approved open source license that makes it free to use, even for commercial products. </li></ul><ul><li>You will also need to install these Python libraries: </li></ul><ul><ul><li>PyXML </li></ul></ul><ul><ul><li>mxDateTime </li></ul></ul><ul><ul><li>lxml for libxml2 </li></ul></ul><ul><ul><li>pyPgSQL for PostgreSQL or MySQLdb for MySQL. </li></ul></ul>
    9. 9. Installation <ul><li>Step 1: Download the latest PyWrapper3 installer. </li></ul><ul><li>Step 2: Uncompress to a folder of your choice. </li></ul><ul><ul><li>Example: “/usr/local/pywrapper/” </li></ul></ul><ul><ul><li>Example: “C: pywrapper ” </li></ul></ul><ul><li>Step 3. Installation … </li></ul><ul><li>Local installation : If you have a Subversion client installed, you may use the automatic installer. </li></ul><ul><li>(Local Python and libraries is installed to your pywrapper folder.) </li></ul><ul><ul><li>promt$ svn export svn://svn.pywrapper.org:80/pywrapper/trunk pywrapper </li></ul></ul><ul><ul><li>promt$ cd pywrapper/tools </li></ul></ul><ul><ul><li>promt$ /bin/sh install.sh </li></ul></ul><ul><ul><li>This will require that you have a bash shell, and probably that you have a Unix line system like e.g. FreeBSD, Linux or Mac OsX… </li></ul></ul>
    10. 10. <ul><li>System installation: </li></ul><ul><li>Make sure you have Python 2.4 or later installed. </li></ul><ul><li>Install the Python libraries: </li></ul><ul><ul><li>PyXML </li></ul></ul><ul><ul><li>mxDateTime </li></ul></ul><ul><ul><li>lxml for libxml2 </li></ul></ul><ul><ul><li>pyPgSQL if you will use PostgreSQL </li></ul></ul><ul><ul><li>MySQLdb if you will use MySQL </li></ul></ul><ul><li>$promt setup.py install (Unix-like systems). </li></ul><ul><li>For Windows systems, execute the installer. </li></ul><ul><li>Execute: pywrapper/tools/ adapt_installation.py </li></ul><ul><li>(file permissions and shebang line in the python scripts) </li></ul>Installation
    11. 11. PyWrapper configuration <ul><li>After successful installation you will need to configure your data provider. Follow the instructions from the PyWrapper documentation web page to configure </li></ul><ul><li>Data sources . If you provide more datasets or several databases they will be configured as individual data sources. </li></ul><ul><li>Database connection . So the software can access your database. </li></ul><ul><li>Database structure . Define the relevant tables, the primary keys and foreign keys. </li></ul><ul><li>Data model . Map your database model to the standard represented by the XML Schemas you choose. </li></ul>[http://trac.pywrapper.org/pywrapper/wiki/Documentation] [http://trac.pywrapper.org/pywrapper/wiki/DraftInstallationGuide]
    12. 12. <ul><li>Start standalone server: </li></ul><ul><li>run ./start_server.py (default port is 8080) </li></ul><ul><li>promt $ cd webapp/ </li></ul><ul><li>promt $ ./start_server.py 8088 </li></ul><ul><li>Configure PyWrapper v3.1.0dev CherryPy server using /somewhere/pywrapper/config/webapp.cfg </li></ul><ul><li>mounting PyWrapper webapps to /pywrapper </li></ul><ul><li>run server on port 8088 </li></ul><ul><li>http://localhost:8088/pywrapper </li></ul><ul><li>start engine... </li></ul><ul><li>http://localhost:8088/pywrapper </li></ul><ul><li>/pywrapper/config/config.ini </li></ul>Start PyWrapper
    13. 13. <ul><li>PyWrapper comes with a graphical web based configuration tool </li></ul>Configuration
    14. 14. BioCASE Install third party software [ http://localhost/pywrapper/utilities/testlibs.cgi ] Follow the links from the Library test page. The column for installed version will display the installed version after successful installation. <ul><li>To update the PyWrapper software: </li></ul><ul><li>Download the new release. </li></ul><ul><li>Unzip to a temporary folder. </li></ul><ul><li>Execute the setup.py and follow the instructions. </li></ul>
    15. 15. Example of a service request <ul><li>All exchanged data is formatted with XML tags. </li></ul>
    16. 16. Example of a service response
    17. 17. Example 2, service request
    18. 18. Example 2, service response singer:/sourcename singer:/taxonomy/genus singer:/taxonomy/species singer:/taxonomy/subspecies singer:/holding/ID singer:/holding/name singer:/origin/collecting/countrysource singer:/origin/collecting/countrysourceID singer:/status/biologicalstatus singer:/status/biologicalstatusID ...
    19. 19. Taxonomic Database Working Group <ul><li>Darwin Core 2 - Element definitions designed to support the sharing and integration of primary biodiversity data&quot;. [http://wiki.tdwg.org/twiki/bin/view/DarwinCore/WebHome], [http://darwincore.calacademy.org/] </li></ul><ul><li>Access to Biological Collection Data (ABCD) 2.06 - An evolving comprehensive standard for the access to and exchange of data about specimens and observations (a.k.a. primary biodiversity data)“. </li></ul><ul><li>[http://www.tdwg.org/activities/abcd/], </li></ul><ul><li>[http://www.bgbm.org/TDWG/CODATA/Schema/] </li></ul>
    20. 20. PyWrapper/BioCASE [http://www.pywrapper.org/] <ul><li>PyWrapper runs on MS Windows, Mac OS X, Linux, BSD, Solaris... </li></ul><ul><li>PyWrapper works with many different databases , PostgreSQL, MySQL, Oracle, MS Access, MS SQL Server.... </li></ul><ul><li>PyWrapper works with UNICODE </li></ul><ul><li> ضاإطقكغب שּׁשׁﭻﭗﭼﱠ אָבּדּוּ </li></ul><ul><li>PyWrapper is OpenSource </li></ul><ul><li>PyWrapper is developed in the Python programming language </li></ul>CVS
    21. 21. <ul><li>Life Science IDentifiers </li></ul><ul><li>LSID is a digital name tag. </li></ul><ul><li>LSIDs are GUIDs, Global Unique Identifiers. </li></ul><ul><li>[http://lsid.sourceforge.net/] </li></ul><ul><li>Structure urn:lsid: authority : namespace : object : revision </li></ul><ul><li>Example (fictive) urn:lsid:eurisco.org:accession:H451269 </li></ul><ul><li>The LSID concept introduces a straightforward approach to naming and identifying data resources stored in multiple, distributed data stores . </li></ul><ul><li>LSID define s a simple, common way to identify and access biologically significant data ; whether that data is stored in files, relational databases, in applications, or in internal or public data sources, LSID provides a naming standard to support interoperability. </li></ul><ul><li>Developed by OMG-LSR and W3C, implemented by IBM. </li></ul>W3C :: LSID
    22. 22. Distributed BioCASE/PyWrapper network
    23. 23. Special thanks to <ul><li>Bioversity International [http://www.bioversityinternational.org] </li></ul><ul><li>GBIF, Global Biodiversity Information Facility [http://www.gbif.org] </li></ul><ul><li>BioCASE , The Biological Collection Access Service for Europe. [http://www.biocase.org] </li></ul><ul><li>TDWG , Taxonomic Database Working Group [http://www.tdwg.org] </li></ul>
    24. 24. Germplasm data indexing tools <ul><li>We have recently built data indexing methodologies for access to germplasm data with BioCASE/PyWrapper. </li></ul><ul><li>This is planned to build a Global Accession Information Network (GAIN). </li></ul><ul><li>Development in close cooperation with GBIF, which themselves index basic biodiversity data from a similar approach. </li></ul><ul><li>[http://chm.grinfo.net/index.php] </li></ul>
    25. 25. Thank you for listening!