ckan 2.0 Introduction (20140618 updated)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

ckan 2.0 Introduction (20140618 updated)

on

  • 200 views

 

Statistics

Views

Total Views
200
Views on SlideShare
198
Embed Views
2

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 2

http://www.slideee.com 2

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 地理資訊圖資雲服務平台 <br />
  • 由 Open Knowledge Foundation 支持 <br />
  • 由 Open Knowledge Foundation 支持 <br />
  • 公布網址 <br />
  • Data.gov.uk 即是 drupal+CKAN <br />
  • Store the raw data and metadata. Visualise structured data with interactive tables, graphs and maps. <br />
  • TGOS: 國土資訊圖資服務平台 <br />

ckan 2.0 Introduction (20140618 updated) Presentation Transcript

  • 1. CKAN 2 Introduction Presenter: 李承錱 Cheng-Jen Lee (Sol) Email: cjlee AT iis.sinica.edu.tw This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Taiwan License.
  • 2. Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Issues 2014/6/18 2
  • 3. Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Issues 2014/6/18 3
  • 4. About CKAN ● The Comprehensive Knowledge Archive Network ● A powerful data management system ● Publishing ● Sharing ● Finding ● Using Data 2014/6/18 4
  • 5. About CKAN 83 instances around the world in May 2014 2014/6/18 5
  • 6. CKAN 2 2014/6/18 6
  • 7. Feature Tour 2014/6/18 7
  • 8. Demo Site demo.ckan.org 2014/6/18 8
  • 9. Feature Tour (1) Publish & Find Datasets Add Dataset Basic Information 2014/6/18 9
  • 10. Feature Tour (1) Publish & Find Datasets Add Data Under the Dataset 2014/6/18 10
  • 11. Feature Tour (1) Publish & Find Datasets Add Metadata About the Dataset 2014/6/18 11
  • 12. Feature Tour (1) Publish & Find Datasets Filter By Keywords 2014/6/18 12
  • 13. Feature Tour (1) Publish & Find Datasets Filter By Geographical Features 2014/6/18 13
  • 14. Feature Tour (2) Store & Manage Data Data Explorer: recline_preview (csv, xls) json_preview pdf_preview ckanext-spatial 2014/6/18 14
  • 15. Feature Tour (2) Store & Manage Data Graphing data 2014/6/18 15
  • 16. Feature Tour (3) Engage with Users & Others Share 2014/6/18 16
  • 17. Feature Tour (3) Engage with Users & Others Organization 2014/6/18 17
  • 18. Feature Tour (3) Engage with Users & Others Manage Users of an Organization 2014/6/18 18
  • 19. Feature Tour (3) Engage with Users & Others Manage Role of Members Admin (管理者): edit datasets & members Editor (編輯): edit datasets Viewer (成員): view (private) datasets Note: Public datasets are visible to everyone 2014/6/18 19
  • 20. Feature Tour (3) Engage with Users & Others Harvest and Federation 2014/6/18 20
  • 21. Feature Tour (3) Engage with Users & Others History 2014/6/18 21
  • 22. Feature Tour (4) Customise & Extend ● RESTful JSON APIs – The Action API – The DataStore API – The FileStore API... ● Extensions (over 60) – ckanext-harvest – ckanext-spatial Open source is good! ● Themable ● Integrates with other CMS (ex. Drupal) 2014/6/18 22
  • 23. CKAN and 5 ★ Open Data 2014/6/18 23
  • 24. CKAN and 5 ★ Open Data ● ★ Make your stuff available on the Web (whatever format) under an open license 2014/6/18 24
  • 25. CKAN and 5 ★ Open Data ● ★★ Make it available as structured data (e.g., Excel instead of image scan of a table) ★★★ Use non-proprietary formats (e.g., CSV instead of Excel) – Accept any data format – Beautiful data demonstration – Datastore : Indexing for structured data 2014/6/18 25
  • 26. CKAN and 5 ★ Open Data ● ★★★★ Use URIs to denote things, so that people can point at your stuff – Permanent link for each dataset – Get Dataset URI through API ● ★★★★★ Link your data to other data to provide context – Linked data and RDF for metadata 2014/6/18 26
  • 27. CKAN and 5 ★ Open Data ● RDF for metadata – DCAT and Dublin Core – curl -L -H "Accept:application/rdf+xml" http://thedatahub.org/dataset/gold-prices 2014/6/18 27
  • 28. Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Issues 2014/6/18 28
  • 29. United Kingdom DATA.GOV.UK 2014/6/18 29
  • 30. United States DATA.GOV 2014/6/18 30
  • 31. Brazil DADOS.GOV.BR 2014/6/18 31
  • 32. European Union PUBLICDATA.EU 2014/6/18 32
  • 33. Japan DATA.GO.JP 2014/6/18 33
  • 34. Tainan DATA.TAINAN.GOV.TW (NEW!) 2014/6/18 34
  • 35. Geospatial Data Explorer: Lat/Long field 2014/6/18 35
  • 36. Geospatial Data Explorer: GeoJSON 2014/6/18 36
  • 37. Geospatial Data Explorer: WMS 2014/6/18 37
  • 38. Agenda ● About CKAN ● Feature Tour – Publish & Find Datasets – Store & Manage Data – Engage with Users & Others – Customise & Extend ● CKAN and 5 ★ Open Data ● Showcase ● Issues 2014/6/18 38
  • 39. Issues ● CJK Support – CJK Search – Some broken translations – File name ● Extensions compatibilities ● Tons of tweaks needed ● Performance Issue ● Complicated Architecture 2014/6/18 39
  • 40. System Architecture 2014/6/18 40
  • 41. Issues (Cont'd) ● What You Should Know – Python & Pylons – ckan plugins toolkit – SQLAlchemy & SQL – HTML, JavaScript – Babel (Translation) – Web Server (UNIX, Apache, Nginx ...) 2014/6/18 41
  • 42. Resources ● Official Documents: – http://docs.ckan.org/en/latest/ ● Installation Notes (in Chinese): – https://ckan-docs-tw.readthedocs.org/ ● CKAN Development Discussions: – http://lists.okfn.org/mailman/listinfo/ckan-dev ● CKAN Taiwan Interest Group: – https://groups.google.com/forum/#!forum/ckan-taiwan-interest-group 2014/6/18 42
  • 43. Thanks for your attention! Any Q? Email: u10313335 AT citi.sinica.edu.tw http://about.me/sollee 2014/6/18 43
  • 44. CKAN 2: Additional Topics Presenter: 李承錱 Cheng-Jen Lee (Sol) Email: u10313335 AT citi.sinica.edu.tw This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Taiwan License.
  • 45. Agenda ● Extended Topic 1: Installation ● Extended Topic 2: Harvesters 2014/6/18 45
  • 46. Install from Source ● Virtual environment ● Checkout the source (via GIT) – https://github.com/okfn/ckan ● Create a CKAN config file ● Setup Jetty & Solr ● Initialize Database (user, db) ● Link to who.ini ● Create a Sysadmin User ● Deployment (Apache + Nginx) ● Install other extensions... 2014/6/18 46
  • 47. Installation Notes ● https://ckan-docs-tw.readthedocs.org/ 2014/6/18 47
  • 48. Agenda ● Extended Topic 1: Installation ● Extended Topic 2: Harvesters 2014/6/18 48
  • 49. Harvesters ● ckanext-harvest – Remote harvesting extension – https://github.com/okfn/ckanext-harvest ● Source Type – CKAN (built-in) – CSW – WAF – Custom (csv/xls/website… etc) 2014/6/18 49
  • 50. Harvested from TGOS CSW service 2014/6/18 50
  • 51. Harvesters http://Mydomain.com/harvest 2014/6/18 51
  • 52. Harvesters Add a new harvest source 2014/6/18 52
  • 53. Harvesters Create a harvest job 2014/6/18 53
  • 54. Harvesters Overview of harvested datasets 2014/6/18 54
  • 55. Harvesters Background Process ● Manually – (pyenv) $ paster --plugin=ckanext-harvest harvester gather_consumer -c /etc/ckan/default/production.ini – (pyenv) $ paster --plugin=ckanext-harvest harvester fetch_consumer -c /etc/ckan/default/production.ini – (pyenv) $ paster --plugin=ckanext-harvest harvester run -c /etc/ckan/default/production.ini 2014/6/18 55
  • 56. Harvesters Background Process ● Automatically – Supervisor (for gather & fetch consumer) – Cron (for run) 2014/6/18 56
  • 57. Harvesters Custom harvester ● Implement the harvester interface to perform harvesting operations ● Three stages ● gather: get the identification ● fetch: fetch the contents ● import: create ckan package (dataset) 2014/6/18 57
  • 58. Harvesters The harvesting interface from base import HarvesterBase class SRDAHarvester(HarvesterBase): def _set_config(self,config_str): def info(self): ... def gather_stage(self, harvest_job): def fetch_stage(self, harvest_object): def import_stage(self, harvest_object): See the extension site for details An example (SRDA): http://goo.gl/ZMnND7 2014/6/18 58