Your SlideShare is downloading. ×
  • Like
Ckan tutorial odw2013 131109
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Ckan tutorial odw2013 131109

  • 593 views
Published

 

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
593
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
18
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • 由 Open Knowledge Foundation 支持
  • 由 Open Knowledge Foundation 支持
  • 公布網址
  • Data.gov.uk 即是 drupal+CKAN
  • Store the raw data and metadata. Visualise structured data with interactive tables, graphs and maps.
  • TGOS: 國土資訊圖資服務平台

Transcript

  • 1. CKAN Tutorial @ odw2013 Presenter: 李承錱 Cheng-Jen Lee (Sol) Email: cjlee AT iis.sinica.edu.tw This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Taiwan License.
  • 2. 2013/11/9 Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Installation ● Harvesters ● Issues
  • 3. 2013/11/9 Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Installation ● Harvesters ● Issues
  • 4. 2013/11/9 About CKAN ● The Comprehensive Knowledge Archive Network ● A powerful data management system ● Publishing ● Sharing ● Finding ● Using Data
  • 5. 2013/11/9 About CKAN 61 instances
  • 6. 2013/11/9 CKAN 2
  • 7. 2013/11/9 Feature Tour
  • 8. 2013/11/9 Demo Site demo.ckan.org
  • 9. 2013/11/9 Feature Tour (1) Publish & Find Datasets Add Dataset Basic Information
  • 10. 2013/11/9 Feature Tour (1) Publish & Find Datasets Add Data Under the Dataset
  • 11. 2013/11/9 Feature Tour (1) Publish & Find Datasets Add Metadata About the Dataset
  • 12. 2013/11/9 Feature Tour (1) Publish & Find Datasets Filter By Keywords
  • 13. 2013/11/9 Feature Tour (1) Publish & Find Datasets Filter By Geographical Features
  • 14. 2013/11/9 Feature Tour (2) Store & Manage Data Data Explorer: recline_preview (csv, xls) json_preview pdf_preview ckanext-spatial
  • 15. 2013/11/9 Feature Tour (2) Store & Manage Data Graphing data
  • 16. 2013/11/9 Feature Tour (3) Engage with Users & Others Share
  • 17. 2013/11/9 Feature Tour (3) Engage with Users & Others Organization
  • 18. 2013/11/9 Feature Tour (3) Engage with Users & Others Manage Users of an Organization
  • 19. 2013/11/9 Feature Tour (3) Engage with Users & Others Manage Role of Members Admin: edit datasets & members Editor: edit datasets Viewer: view (private) datasets Note: Public datasets are visible to everyone
  • 20. 2013/11/9 Feature Tour (3) Engage with Users & Others Harvest and Federation
  • 21. 2013/11/9 Feature Tour (3) Engage with Users & Others History
  • 22. 2013/11/9 Feature Tour (4) Customise & Extend ● RESTful JSON APIs – The Action API – The DataStore API – The FileStore API... ● Extensions (over 60) – ckanext-harvest – ckanext-spatial Open source is good! ● Themable ● Integrates with other CMS (ex. Drupal)
  • 23. CKAN and 5 ★ Open Data 2013/11/9
  • 24. CKAN and 5 ★ Open Data ● ★ Make your stuff available on the Web (whatever format) under an open license 2013/11/9
  • 25. CKAN and 5 ★ Open Data ● ★★ Make it available as structured data (e.g., Excel instead of image scan of a table) ★★★ Use non-proprietary formats (e.g., CSV instead of Excel) – Accept any data format – Beautiful data demonstration 2013/11/9
  • 26. CKAN and 5 ★ Open Data ● ★★★★ Use URIs to denote things, so that people can point at your stuff – Permanent link for each dataset – Get Dataset URI through API ● ★★★★★ Link your data to other data to provide context – Linked data and RDF for metadata 2013/11/9
  • 27. CKAN and 5 ★ Open Data ● RDF for metadata – DCAT and Dublin Core – curl -L -H "Accept:application/rdf+xml" http://thedatahub.org/dataset/gold-prices 2013/11/9
  • 28. 2013/11/9 Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Installation ● Harvesters ● Issues
  • 29. 2013/11/9 United Kingdom DATA.GOV.UK
  • 30. 2013/11/9 United States DATA.GOV
  • 31. 2013/11/9 Brazil DADOS.GOV.BR
  • 32. 2013/11/9 European Union PUBLICDATA.EU
  • 33. 2013/11/9 Geospatial Data Explorer: Lat/Long field
  • 34. 2013/11/9 Geospatial Data Explorer: GeoJSON
  • 35. 2013/11/9 Geospatial Data Explorer: WMS
  • 36. 2013/11/9 Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Installation ● Harvesters ● Issues
  • 37. 2013/11/9 System Architecture
  • 38. What You Should Know ● Python, Pylons ● ckan plugins toolkit ● SQLAlchemy, SQL ● HTML, JavaScript ● Babel ● Web Server (Nginx, uWSGI...) 2013/11/9
  • 39. 2013/11/9 Install from Source ● Virtual environment ● Checkout the source (via GIT) – https://github.com/okfn/ckan ● Create a CKAN config file ● Setup Jetty & Solr ● Initialize Database (user, db) ● Link to who.ini ● Create a Sysadmin User ● Deployment (nginx + uWSGI) ● Install other extensions...
  • 40. 2013/11/9 Installation Notes ● https://ckan-docs-tw.readthedocs.org/
  • 41. 2013/11/9 Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Installation ● Harvesters ● Issues
  • 42. 2013/11/9 Harvesters ● ckanext-harvest – Remote harvesting extension – https://github.com/okfn/ckanext-harvest ● Source Type – CKAN (built-in) – CSW – WAF – Custom (csv/xls/website… etc)
  • 43. 2013/11/9 Harvested from TGOS CSW service
  • 44. 2013/11/9 Harvesters http://Mydomain.com/harvest
  • 45. 2013/11/9 Harvesters Add a new harvest source
  • 46. 2013/11/9 Harvesters Create a harvest job
  • 47. 2013/11/9 Harvesters Overview of harvested datasets
  • 48. 2013/11/9 Harvesters Background Process ● Manually – (pyenv) $ paster --plugin=ckanext-harvest harvester gather_consumer -c /etc/ckan/default/production.ini – (pyenv) $ paster --plugin=ckanext-harvest harvester fetch_consumer -c /etc/ckan/default/production.ini – (pyenv) $ paster --plugin=ckanext-harvest harvester run -c /etc/ckan/default/production.ini
  • 49. 2013/11/9 Harvesters Background Process ● Automatically – Supervisor (for gather & fetch consumer) – Cron (for run)
  • 50. 2013/11/9 Harvesters Custom harvester ● Implement the harvester interface to perform harvesting operations ● Three stages ● gather: get the identification ● fetch: fetch the contents ● import: create ckan package(dataset)
  • 51. 2013/11/9 Harvesters The harvesting interface from base import HarvesterBase class SRDAHarvester(HarvesterBase): def _set_config(self,config_str): def info(self): ... def gather_stage(self, harvest_job): def fetch_stage(self, harvest_object): def import_stage(self, harvest_object): See the extension site for details An example (SRDA): http://goo.gl/ZMnND7
  • 52. 2013/11/9 Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Installation ● Harvesters ● Issues
  • 53. 2013/11/9 Issues ● CJK Support – CJK Search – Some broken translations – Frequent updates ● Extensions compatibilities ● Tons of tweaks needed ● Performance Issue ● Complicated Architecture
  • 54. ● Official Documents: – http://docs.ckan.org/en/latest/ ● Installation Notes (in Chinese): – https://ckan-docs-tw.readthedocs.org/ ● CKAN Development Discussions: – http://lists.okfn.org/mailman/listinfo/ckan-dev ● CKAN Taiwan Interest Group: – https://groups.google.com/forum/#!forum/ckan-taiwan-interest-group 2013/11/9 Resources
  • 55. Thanks for your attention! 2013/11/9 Any Q? Email: u10313335 AT citi.sinica.edu.tw http://about.me/sollee