ckan 2.0: a deeper look

649 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
649
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
14
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • gather_stage - Aggregates all the remote identifiers for a particular source (ie identifiers for a CSW server, files for a WAF). fetch_stage - Fetches all the remote documents and stores them on the database. import_stage - Performs all the processing for transforming the remote content into a CKAN dataset: validates the document, parses it, converts it to a CKAN dataset dict and saves it in the database.
  • ckan 2.0: a deeper look

    1. 1. ckan 2.0: a deeper look Internship @ Academia Sinica Report #2 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Taiwan License. Presenter: Cheng-Jen Lee (Sol) Email: u10313335 AT citi.sinica.edu.tw
    2. 2. Aug 5, 2013 2 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migration ● OpenID & UNIX user auth
    3. 3. Aug 5, 2013 3 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migration ● OpenID & UNIX user auth
    4. 4. Aug 5, 2013 4 Jetty8/9+solr4 ● For ckanext-spatial ● IKAnalyzer: 2012FF_fh1 ● Different schema
    5. 5. Aug 5, 2013 5 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migrate to pomelo ● OpenID & UNIX user auth
    6. 6. Aug 5, 2013 6 ckanext-spatial ● Geo related plugins for CKAN ● spatial_metadata ● Spatial Search Widget ● Dataset Extent Map ● WMS Preview ● GeoJSON Preview ● CSW Server ● Spatial Harvesters
    7. 7. Aug 5, 2013 7 Spatial Search Widget ● Map widget integrated on the search form ● Live Demo
    8. 8. Aug 5, 2013 8 Dataset Extent Map
    9. 9. Aug 5, 2013 9 WMS Preview
    10. 10. Aug 5, 2013 10 GeoJSON Prevew ● a GeoJSON previewer ● Live Demo
    11. 11. Aug 5, 2013 11 CSW Server Spatial Harvesters ● still in investigation ● CSW Server – to server metadata from the CKAN instance ● Spatial Harvesters – for importing spatial metadata into CKAN – 3 formats: CSW, WAF, doc – 3 stages: gather, fetch, import
    12. 12. Aug 5, 2013 12 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migration ● OpenID & UNIX user auth
    13. 13. Aug 5, 2013 13 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migration ● OpenID & UNIX user auth
    14. 14. Aug 5, 2013 14 Import/Export ● Importing Datasets with the CKAN RESTful API – request = urllib2.Request('http://www.my_ckan_site. com/api/action/package_create') ● Export: json, csv, and db dump – paster db ... – Restore and migration?
    15. 15. Aug 5, 2013 15 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migration ● OpenID & UNIX user auth
    16. 16. Aug 5, 2013 16 Migration ● paster db simple-dump-json/csv – datasets and users cannot be imported simultaneously ● paster db dump – SQLAlchemy error (maybe the version of PostgreSQL...) ● Raw db dump ● Target machine: – Apache + mod_wsgi OK→ – Sub-url redirection sucks – PostgreSQL 8.4 (instead of 9.1) ● postgresql-8.4-postgis is no longer supported
    17. 17. Aug 5, 2013 17 Agenda ● Jetty8/9+solr4 ● ckanext-spatial ● ckanext-harvest ● Import/Export ● Migration ● OpenID & UNIX user auth
    18. 18. Aug 5, 2013 18 OpenID ● No longer supported!
    19. 19. Aug 5, 2013 19 UNIX user auth ● Data.gov.uk: – User log-on and credentials are handled by Drupal – Drupal provides an internal API for CKAN to confirm the details
    20. 20. Aug 5, 2013 20 Thanks for your attention! Any Q?

    ×