Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
STEVEN DE COSTA
Steering Group Member
@STARL3N
PUBLIC DATA AND CKAN
Introduction 3
Link Digital
Executive Director
CKAN Association
Steering Group Member
Open Knowledge Australia
Co-Secretar...
STEVEN DE COSTA
Steering Group Member
@STARL3N
STEVEN DE COSTA
Steering Group Member
@STARL3N
6Link Digital’s Network Map
Data portal software:
1. Open Source
2. Large and expanding installation base
within Government worldwide
3. Expanding use...
LEARN MORE AT CKAN.org
DOWNLOAD FROM github.com/ckan/ckan
1. >> Organisations (optionally with sub-organisations)
2. >> >> Datasets
3. >> >> >> Resources
4. >> Platform Custodian
5...
1. >> Constitution
2. >> >> Parliamentary Legislation and Acts (Jurisdiction = Platform)
3. >> >> >> Ministries (Organisat...
1. User registration
2. User management
3. Custodian workflows (manage datasets and data resources)
4. Directory Browse by...
1. Create an organistation (usually done by platform owner)
2. Login as member of organisation
3. Click ‘add dataset’
4. S...
1. Join the dev mailing list (monitored by tech team):
https://lists.okfn.org/mailman/listinfo/ckan-dev
2. Search Stack ov...
CKAN API DOCUMENTATION
1. Get JSON-formatted lists of a site’s datasets, groups or other CKAN objects
2. Get a full JSON representation of a data...
1. CKAN as an Information Asset Register
2. FileStore – For hosting of data and resources
3. DataStore - provides a databa...
Delivering the world’s best open
data management system
The purpose of the CKAN Association
is to support sustainable grow...
Community interests, or needs, can be generalised as:
1. Users need an enterprise level open data management system tailor...
Who owns or directly manages the CKAN project?
The project, its releases and future direction are cooperatively managed by...
What is the current structure of
the CKAN Association?
See http://ckan.org
The steering group carries on a
number of busin...
FORMAL MEMBERSHIPS
It's easy to find people willing to fund or work on the flashier parts of CKAN, and there is some funding for
core work su...
Foundations of Open Government 25
My Administration is committed to creating an
unprecedented level of openness in Governm...
The role of Government in Australia 26
It should support civil society and its multiplicity of voices and activities.
It s...
Business Cases: Theory of the firm & Transaction costs 27
Where is there work to do with Open Data? 28
Published Data Handbook 29
Published Data Handbook 30
Recipe Name What you’ll need Ingredients Method Perfect for…
31Establishing Data Publishing Policy
32Establishing Data Management Systems
33Educating Data Publishers
34Facilitation within tech communities
Establishing enterprise solution architecture 35
36Extract, Transform and Load Connectors
37Cleaning of Messy Data
38Geospatial Integration
39Resource View Integration
40Enriched Data and Analytics
41The perfect storm
42Drupal interface
43CKAN interface
44What the DFMP does
45Launched Globally as CKAN Galleries
46Launched Globally as CKAN Galleries
47Government as an API - Digital Transformation Office
48Data.vic.gov.au on CKAN, Drupal and AWS
Classifying data 49
The three tiers are:
internal
whole of Government
open
50Data.vic.gov.au data classification
51The ‘go. no go’ gates for going open
OEH Data Management System 52
Take Note: What is NOT good 53
Take Note: What is best… 54
CKAN and Australian open data updates for Wikimedia - 7 October 2015
CKAN and Australian open data updates for Wikimedia - 7 October 2015
CKAN and Australian open data updates for Wikimedia - 7 October 2015
CKAN and Australian open data updates for Wikimedia - 7 October 2015
Upcoming SlideShare
Loading in …5
×

CKAN and Australian open data updates for Wikimedia - 7 October 2015

774 views

Published on

Another joint presentation covering CKAN and updates from what is going on in Australia surrounding open data initiatives.
This was my first chance to also showcase the work of WA Parks and Wildlife and Florian Mayer's approach as a data pipeline plumber.

Published in: Technology
  • Be the first to comment

CKAN and Australian open data updates for Wikimedia - 7 October 2015

  1. 1. STEVEN DE COSTA Steering Group Member @STARL3N PUBLIC DATA AND CKAN
  2. 2. Introduction 3 Link Digital Executive Director CKAN Association Steering Group Member Open Knowledge Australia Co-Secretary & Treasurer Data Shades Founder GovHack National Organiser AWS User Group CBR Organiser
  3. 3. STEVEN DE COSTA Steering Group Member @STARL3N
  4. 4. STEVEN DE COSTA Steering Group Member @STARL3N
  5. 5. 6Link Digital’s Network Map
  6. 6. Data portal software: 1. Open Source 2. Large and expanding installation base within Government worldwide 3. Expanding use cases in the wider data ecosystem 4. Python web app, PostgreSQL DB 5. Built for machines, custodians and end users WHAT IS CKAN?
  7. 7. LEARN MORE AT CKAN.org
  8. 8. DOWNLOAD FROM github.com/ckan/ckan
  9. 9. 1. >> Organisations (optionally with sub-organisations) 2. >> >> Datasets 3. >> >> >> Resources 4. >> Platform Custodian 5. >> >> Organisation Custodian, Editor or Member 6. >> Published or Private datasets CKAN STRUCTURE
  10. 10. 1. >> Constitution 2. >> >> Parliamentary Legislation and Acts (Jurisdiction = Platform) 3. >> >> >> Ministries (Organisation) 4. >> >> >> >> Programs (Sub-Organisations) 5. >> >> >> >> >> Projects (Datasets) 6. >> >> >> >> >> >> Outcomes (Resources) CKAN USE CASE PARADIGM
  11. 11. 1. User registration 2. User management 3. Custodian workflows (manage datasets and data resources) 4. Directory Browse by organisation or group 5. Faceted search for multiple fields (supporting end user discovery) 6. Resource views to preview data (a recently improved feature) 7. Metadata view CKAN UI
  12. 12. 1. Create an organistation (usually done by platform owner) 2. Login as member of organisation 3. Click ‘add dataset’ 4. Step 1: Add a title, description and other metadata 5. Step 2: Add resources (links to data or upload data files for hosting) 6. Step 3: Add any additional info CKAN CUSTODIAN WORKFLOW
  13. 13. 1. Join the dev mailing list (monitored by tech team): https://lists.okfn.org/mailman/listinfo/ckan-dev 2. Search Stack overflow under CKAN: http://stackoverflow.com/search?q=CKAN 3. Check the roadmap on waffle.io: https://waffle.io/ckan/ideas-and-roadmap 4. Join a tech team meeting: http://ckan.org/about/technical-team/ WHERE TO GET HELP
  14. 14. CKAN API DOCUMENTATION
  15. 15. 1. Get JSON-formatted lists of a site’s datasets, groups or other CKAN objects 2. Get a full JSON representation of a dataset, resource or other object 3. Search for packages or resources matching a query 4. Create, update and delete datasets, resources and other objects 5. Get an activity stream of recently changed datasets on a site CKAN API
  16. 16. 1. CKAN as an Information Asset Register 2. FileStore – For hosting of data and resources 3. DataStore - provides a database for structured storage of data together with a powerful Web- accessible Data API 4. License Selection (machine ready?) 5. Harvesting A FEW MORE POINTS
  17. 17. Delivering the world’s best open data management system The purpose of the CKAN Association is to support sustainable growth and development of CKAN while also protecting the interests of the CKAN community. The Association values a healthy and thriving community which continues to deliver the best open data management system in the world. CKAN ASSOCIATION: STATEMENT OF PURPOSE
  18. 18. Community interests, or needs, can be generalised as: 1. Users need an enterprise level open data management system tailored to meet their needs now and into the future. 2. Individual contributors need a project that is rewarding to work for, inclusive and active. COMMUNITY INTERESTS
  19. 19. Who owns or directly manages the CKAN project? The project, its releases and future direction are cooperatively managed by its community of users and contributors. Association delegates, staff and office holders may be active within the project but will exercise no more or less influence than any other contributor or user. More information about CKAN and how to contribute can be found at CKAN.org. DELEGATION TO SERVE AND PROTECT
  20. 20. What is the current structure of the CKAN Association? See http://ckan.org The steering group carries on a number of business activities. This includes raising revenue, managing resources and directing projects or programs of activity relevant to the CKAN Association’s statement of purpose. COMPLETENESS OF PURPOSE
  21. 21. FORMAL MEMBERSHIPS
  22. 22. It's easy to find people willing to fund or work on the flashier parts of CKAN, and there is some funding for core work such as meetings and preparing releases. But there are a few important things that aren't getting addressed. Fix Security Issues There are a small number of known issues that high profile sites work around by implementing workflows and tight controls over users and data published. Everyone else would benefit from a more-secure-by-default CKAN. Clear Technical Debt CKAN, has some parts that should be replaced or removed. This work doesn't add any new features and worse, threatens to break upgrades for existing sites and extensions. It is necessary work though. Unless performance and technical limitations are addressed they will grow and start turning users away from CKAN. Ref: https://github.com/ckan/ideas-and-roadmap/issues/152 - Raised by Ian Ward, Tech Team Lead HOW MEMBERSHIPS BENEFIT CKAN
  23. 23. Foundations of Open Government 25 My Administration is committed to creating an unprecedented level of openness in Government. We will work together to ensure the public trust and establish a system of transparency, public participation, and collaboration. Openness will strengthen our democracy and promote efficiency and effectiveness in Government. Sources: http://www.whitehouse.gov/the_press_office/TransparencyandOpenGovernment Hope Poster by Shepard Fairey: http://en.wikipedia.org/wiki/File:Barack_Obama_Hope_poster.jpg
  24. 24. The role of Government in Australia 26 It should support civil society and its multiplicity of voices and activities. It should provide the economic framework and the essential infrastructure for public and private enterprise. Source: http://www.australiancollaboration.com.au/pdf/FactSheets/Role-government-FactSheet.pdf
  25. 25. Business Cases: Theory of the firm & Transaction costs 27
  26. 26. Where is there work to do with Open Data? 28
  27. 27. Published Data Handbook 29
  28. 28. Published Data Handbook 30 Recipe Name What you’ll need Ingredients Method Perfect for…
  29. 29. 31Establishing Data Publishing Policy
  30. 30. 32Establishing Data Management Systems
  31. 31. 33Educating Data Publishers
  32. 32. 34Facilitation within tech communities
  33. 33. Establishing enterprise solution architecture 35
  34. 34. 36Extract, Transform and Load Connectors
  35. 35. 37Cleaning of Messy Data
  36. 36. 38Geospatial Integration
  37. 37. 39Resource View Integration
  38. 38. 40Enriched Data and Analytics
  39. 39. 41The perfect storm
  40. 40. 42Drupal interface
  41. 41. 43CKAN interface
  42. 42. 44What the DFMP does
  43. 43. 45Launched Globally as CKAN Galleries
  44. 44. 46Launched Globally as CKAN Galleries
  45. 45. 47Government as an API - Digital Transformation Office
  46. 46. 48Data.vic.gov.au on CKAN, Drupal and AWS
  47. 47. Classifying data 49 The three tiers are: internal whole of Government open
  48. 48. 50Data.vic.gov.au data classification
  49. 49. 51The ‘go. no go’ gates for going open
  50. 50. OEH Data Management System 52
  51. 51. Take Note: What is NOT good 53
  52. 52. Take Note: What is best… 54

×