Scratchpad training


Published on

2011 Scratchpad training course presentation.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Before I do this let me say a few words about the case for the Scratchpads: Why do we need them. - Taxonomy / biodiversity studies have lots of big questions which require large aggregations of data to address. Issue like climate change, changing patterns of land use, protection of food supplies, ecosystem services etc.
  • So Drupal was selected to be the foundation for the Scratchpad project. It supports the central management of multiple sites, providing us with a mechanism to supply customised community based websites to anyone that requests a site. THis is the home page of the Scratchpad project. Users can access a variety of resources from here. In parctiuclar they can also apply for a new site.
  • Fill in form…
  • Get a template empty site that they can populate and add new users. The Scratchpads have been built to support a wide range of activities required by taxonomists. These functions are easy to navigate, helping people to construct their sites quickly and easily. Crucially users and theme and focus their site on one or more of a thange of activities. This is crucial to building a sense of community amongst the userbase.
  • The project has been running since March 2007 and in that time we have built up…These form an ecosystem of different communities working toward their own goals in a self managed environment. A high proportion of the sites fail - Probably about 80% have relativly little content. The system is construicted to accomdate failure.
  • Some use cases for the Scratchpads are illustrated on this rather elaborate slide here….
  • From a technical perspective Scratchpad sit on a single Virtual Machine in the NHM server room. Currently they occypy about 100 GB of space. The server architecture is based on a LAMP configuration. Its all backed up using Tivoli storage manager. Eventually these go to tape as part of the NHM’s storage management and are picked up by a courrier and stored off site. The marginal costs of creating a new site are exceptionally low - probably about £3 per site, taking into account storage and staff time.
  • Developers site with instructions on how to set up your own Scratchpad server. You can also access individual modules from here. Functionality is partitions into modules of code which with with the Drupal core. Moduels that are not
  • Scratchpads are build on three main parts, taxonomy, content types, and views. You can add data of various types to a Scratchpad. For example you can add images, bibliographic citations, or simple pages. Each image, bibliographic citation or page represents one so called node on a scratchpad. And each node has a unique URL. The taxonomy is the backbone of the Scratchpad, which gathers the data together. You can have several taxonomy vocabularies on your Scratchpad. On taxonomic websites, the main vocabulary is usually a biological classification, but it can also be a list of collectors or geographic regions for example. Each node is tagged, that is linked to one or more terms in the classification. Now we can view the data in various ways. One way is to look at just one node, this is the node view. Another way it to look at all the data on the Scratchpad linked to one vocabulary term and this is the taxon page view. You can also create your own views to look at only those data you want to see.
  • As an example of a Scratchpad focussed on a taxonomic group, I will give you an overview of my Milichiidae website. I have some general information on the family, mostly as simple pages, like the front page or as tables. Then I have some special information on the genus Milichiella, which I recently revised.
  • This includes species pages, which I will show later, an interactive Lucid key, a page where one can compare images of different species side by side, and a phylogenetic tree. As you can see at the bottom, other users, even anonymous users, which are not logged in, can comment on a page if this function has been enabled. However, comments are usually not published immediately, but have to be activated by the administrator of the site, to avoid spam.
  • Each Scratchpad has a ‘Content’ menu, which has links to the various content types present on the respective Scratchpad, the default ones as well as those custom-made for this Scratchpad. The default ones are for example biblio, countries map, … Clicking on ‘Image’, for example, will bring up the image gallery page.
  • This is a list of the image galleries I have on my Scratchpad. Clicking on the ‘Behaviour’ gallery will bring up a list of thumbnails for the images in this gallery and clicking on one of the images will open a kind of slide show.
  • This is a myrmecophilous milichiid stalking Crematogaster ants. I can just click on the arrow to see the next image.
  • or click on another link to see the image node with all the details.
  • Image details with copyright information and the tags associated with this node, like the gallery it belongs to, keywords and the classification. Clicking on a classification tag will always bring up the taxon page for this taxon, in this case Milichia patrizii.
  • A species page is build from all the data on a Scratchpad that are tagged with the respective classification term. Each content type is represented by one or more what we call panels or widgets. In the present case I have a panel for ‘Nomenclature & Description’ which is a content type I created myself. There is the default image panel, a specimen map and a list of the specimens the map is based on. There is a panel for ‘Page’ nodes and one for the bibliography. There are also several panels showing data from other web sources, like BHL, which you see here, but also Yahoo and Flickr images, GBIF, Gen- and Morphbank, Wikipedia and Google Scholar. This is a rare species, so except for BHL none of these sources has data on Milichia patrizii. The ‘Links’ panel is one I created myself to provide links to other useful information on the web that is not covered by the other web panels. At the bottom you have a special panel which is called Authors & Sources and shows a list of the contributors to this page, in this case me and BHL. Clicking on the citation button will generate a permanent copy of the species page as it is now, together with the citation and a link to the page. You can use this citation when you want to refer to this page in a paper.
  • [Skip?] I only want to show two more of the options you have on a Scratchpad. One is the glossary function. You can add a glossary which is basically the same as a classification with terms and definitions of the term. You can also define extra fields, like my ‘Explanation’ and ‘Character evolution’. When one of the glossary terms occurs on a page it is highlighted and hovering with the mouse over the term will show the definition.
  • [Skip?] The other content type, which we have recently improved a lot, is the character project. This is a character matrix, where you can enter controlled, text or numeric characters. The matrix can be exported to SDD format, which can be imported into Lucid. Lucid will generate the key, which you can then show on your Scratchpad. Same for phylogenies, you can export the matrix to Nexus format, generate a phylogenetic tree with the respective software, and then show this key on your Scratchpad. [don’t skip] We now also have the option to publish Scratchpad data in a journal, but this will be the topic of Vlad’s talk, later.
  • I will demonstrate the main content types later. Here I just want to point out some special functions.
  • Scratchpad training

    1. 1. Vince Smith & Ed Baker Natural History Museum, London, UK Training session
    2. 2. Overview of the day <ul><li>Expectations, introductions & goals </li></ul><ul><li>Overview presentation (30-40 minutes) </li></ul><ul><li>Overview of (10 minutes) </li></ul><ul><li>Training course options </li></ul><ul><ul><li>Basic training (2-5 hours) </li></ul></ul><ul><ul><li>Advanced training (3 hours) </li></ul></ul><ul><ul><li>Publication tool (1 hour) </li></ul></ul><ul><li>Final Q&A </li></ul><ul><li>Feedback form </li></ul>
    3. 3. Why Scratchpads?
    4. 4. 21st Century taxonomy <ul><li>Inventory the Earth’s species </li></ul><ul><li>Document their relationships </li></ul><ul><li>“ Publish” & apply these data </li></ul>Goal… The challenge for modern taxonomists <ul><li>1.8 M described spp. (10M names) </li></ul><ul><li>300M pages (over last 250 years) </li></ul><ul><li>1.5-3B specimens </li></ul>Data set… People… <ul><li>4-6,000 scientists </li></ul><ul><li>30-40,000 “pro-amateurs” </li></ul><ul><li>Many more citizen scientists? </li></ul>
    5. 5. Towards a solution The European Distributed Institute of Taxonomy <ul><li>A Network of Excellence (NoE) </li></ul><ul><li>29 leading European, North American, </li></ul><ul><li>& Russian natural history collections-based </li></ul><ul><li>institutions </li></ul><ul><li>Circa 12M €, funded under EU FP6 </li></ul><ul><li>March 2006 - February 2011 </li></ul>Products… <ul><li>Funding </li></ul><ul><li>Training & outreach </li></ul><ul><li>Websites </li></ul><ul><li>Integrated scientific activities </li></ul><ul><li>Inventories </li></ul><ul><li>Computer tools </li></ul>
    6. 6. Fast Intuitive Fit for use What is a Scratchpad? A website for you & your community Your data 1 “ Published” & reviewed on your site 3 Uploaded & tagged 2
    7. 7. Scratchpads Overview <ul><li>Hosted websites for taxonomists </li></ul><ul><li>Research & publication platform </li></ul><ul><li>Modular (Drupal) & flexible </li></ul><ul><li>Supports the taxonomic workflow </li></ul><ul><li>Bottom-up design, agile dev. </li></ul><ul><li>Ecosystem of communities (200) </li></ul><ul><li>2,500+ users (unpaid) from 2007 </li></ul>
    8. 8. Scratchpads A multisite implementation of Drupal
    9. 9. Scratchpad Application Form Free to any individual or community
    10. 10. Flexible Template Site With easy navigation
    11. 11. Sites: 200+ Users: 2500+ Pages: 300k+ Circa 2 new sites/wk. Since Mar. ‘07 10% access per mnth An ecosystem of communities 40% accessed per mnth
    12. 13. Sustaining Scratchpads ViBRANT: Building on EDIT & the Scratchpads “ To set up the means, tools and infrastructure to produce a more rational and a more effective framework for European Biodiversity research” <ul><li>A Virtual Research Environment ( Scratchpads ) where users can safely store, share and manage their research information. </li></ul><ul><li>Analytical services for users to build identification keys and phylogenetic trees. </li></ul><ul><li>A publication platform for users to automatically compile taxonomic manuscripts from their research database. </li></ul><ul><li>A portal for users to centrally access publicly accessible biodiversity research information and literature. </li></ul><ul><li>Training, support & sociological study , helping research communities to use these tools and services. </li></ul><ul><li>A standards compliant technical architecture that can be sustained by the biodiversity research community. </li></ul>
    13. 14. ViBRANT logistics <ul><li>Starts December 2010 (36 months) </li></ul><ul><li>Project website ( http: //vbrant . eu ) </li></ul><ul><li>Virtual Research Communities, CP-CSA, EU FP7 </li></ul><ul><li>€ 6.2M Euros (EU Contribution €4.75M) </li></ul><ul><li>17 Partners in 9 countries, 603 person months </li></ul> of 14 Collaboration… <ul><li>ESFRI Projects: LifeWatch, ELIXIR & EMBRC </li></ul><ul><li>GBIF - controlled vocabularies, nodes & observational data recording </li></ul><ul><li>PESI, 4D4Life & related EU projects </li></ul><ul><li>Encyclopedia of Life, Barcode of Life & Biodiversity Heritage Library </li></ul><ul><li>South African National Biodiversity Institute & Atlas of living Australia </li></ul>
    14. 15. ViBRANT Project plan The “chromosome” SERVICE WP5. Data WP6. Publishing WP4. Standards WP8. Mobilisation RESEARCH WP2. Architecture WP7. Literature NETWORKING WP3. Training
    15. 16. Scratchpad Technical Overview
    16. 17. Scratchpad Object Model
    17. 18. Scratchpad management Scalable & sustainable technology One virtual machine, open-source, self-archiving, backed-up, multi-site configuration ( easy to move & upgrade, secure & reliable, screencasts, low admin., low marginal costs ) Hardware, software & user support
    18. 19. Scratchpad Developers Site
    19. 20. Scratchpad SVN
    20. 21. Scratchpad Functionality Summary
    21. 22. Taxonomy Taxonomy import, management and navigation
    22. 23. Bibliographic data Reference manager / Endnote support for bibliographies
    23. 24. Images Image galleries, image upload & annotation
    24. 25. Phylogeny Nexus / Newick import for visualizing phylogenies
    25. 26. Character Matricies Molecular & morphological character matricies (discrete, morphometric and text characters)
    26. 27. Distribution Maps Presence absence country maps
    27. 28. Specimens & locations Specimen & location records (DwC)
    28. 29. Pages, Forums, Blogs, Newsletters Web fora with e-mail integration User blogs Static web pages Newsletters with e-mail integration
    29. 30. Mass Import Import from CSV text file to any content type
    30. 31. Create & switch between content in any language Multilingual Support
    31. 32. Administration Simple & advanced admin. AutoTag content (automatically adds structure) Private Groups Customized user profiles
    32. 33. Putting it all together - an example site
    33. 34. Scratchpad structure Taxonomy biblio Content types image page taxon page view term 1 term 2 term 3 node view list view custom view
    34. 35. Knowledgebase on a family of flies General Information
    35. 36. User Comments Special Information
    36. 37. Taxonomy Taxonomy import, management and navigation Classification
    37. 38. ‘ Content’ menu
    38. 39. Image galleries
    39. 41. Link to image details
    40. 42. Image details Classification and other tags
    41. 43. Species page Nomenclature and taxonomy Images Bibliography Specimen Specimen map Links Pages Authors & Sources BHL Species description Countries map
    42. 44. Glossary Definition of term Explanation Character evolution
    43. 45. character matrix export to SDD and Nexus Milichia online revision
    44. 46. Phylogeny Nexus / Newick import for visualizing phylogenies
    45. 47. Pages, Forums, Blogs, Newsletters Web forums with e-mail integration User blogs Rich text editor Newsletters with e-mail integration
    46. 48. Special functions Multilingual support Private groups – an area on your site that is visible only to selected users Autotagging – automatically add structure to your content
    47. 49. Paper publication
    48. 50. User Support
    49. 51. Getting help
    50. 52. The Sandbox
    51. 53. of Built in help, the FAQ & issues queue
    52. 54. Screencast s, but…
    53. 55. Questions?
    54. 56. Scratchpad Training http://training[X] Username: username Password: password Sites will be available for 2 weeks Scratchpad Log-in Scratchpad Training Files Example Completed Site - Carnidae