Opening up Government Data Jenny Telford Director of Census Products and Services Australian Bureau of Statistics

Opening up ABS Data How far we have come Case Study of the 2006 Census Output The challenges: Policy Organisational Technical

How far we have come

Case Study of the 2006 Census Output

The challenges:

Policy

Organisational

Technical

How far we have come The past Data available in paper publications The ‘paper on the web’ era The present Content designed for the web Interactive applications The future More interactivity Data as a service

The past

Data available in paper publications

The ‘paper on the web’ era

The present

Content designed for the web

Interactive applications

The future

More interactivity

Data as a service

Case Study: 2006 Census Output Aim to make data: Relevant Visible Usable Timely Accurate Accessible

Aim to make data:

Relevant

Visible

Usable

Timely

Accurate

Accessible

2006 Census Output Objectives Provide a range of products aimed at meeting individual user needs Tourists Harvesters Miners Better access to more data Changes to delivery not content Improve the quantity and quality of our metadata.

Provide a range of products aimed at meeting individual user needs

Tourists

Harvesters

Miners

Better access to more data

Changes to delivery not content

Improve the quantity and quality of our metadata.

The Product Range

TableBuilder High end application for experienced users Create custom tables direct from unit record data Confidentiality built into the application Range of formats csv, xls, mid/mif, esri, GML, etc… Create tables, maps and charts

High end application for experienced users

Create custom tables direct from unit record data

Confidentiality built into the application

Range of formats

csv, xls, mid/mif, esri, GML, etc…

Create tables, maps and charts

Where to next? Opening up data even further Data delivered as services Increased emphasis on geospatial display Aggregating Census data with other sources Mashups Maplets Widgets

Opening up data even further

Data delivered as services

Increased emphasis on geospatial display

Aggregating Census data with other sources

Mashups

Maplets

Widgets

Opening up data – the challenges Protecting our reputation/brand Authenticity of open data Protecting respondent confidentiality Equity of access to all Making data comparable Technical challenges

Protecting our reputation/brand

Authenticity of open data

Protecting respondent confidentiality

Equity of access to all

Making data comparable

Technical challenges

Protecting our Reputation ABS has a reputation for quality accurate data Direct link between reputation and response rates Risk of mashing up ABS data with data from a less reliable source Loss of direct control

ABS has a reputation for quality accurate data

Direct link between reputation and response rates

Risk of mashing up ABS data with data from a less reliable source

Loss of direct control

Data Authenticity Can you trust what you see? ABS brand = Quality and Trust The digital watermark “ Official Source” stamp

Can you trust what you see?

ABS brand = Quality and Trust

The digital watermark

“ Official Source” stamp

Confidentiality is key! Protect the privacy of individuals and businesses CORE value – cannot be compromised Range of technical and procedural protections in place Adds technical complexity

Protect the privacy of individuals and businesses

CORE value – cannot be compromised

Range of technical and procedural protections in place

Adds technical complexity

Equity of Access Key ABS principle Aim to make our data as accessible and usable as possible – for all We can’t possibly publish data in every format the quest for open standards Google gadgets, Yahoo widgets etc… The potential of RSS

Key ABS principle

Aim to make our data as accessible and usable as possible – for all

We can’t possibly publish data in every format

the quest for open standards

Google gadgets, Yahoo widgets etc…

The potential of RSS

Data Cohesion – making things comparable Importance of standards Changing classifications Time series Comparing geographic areas – not as simple as it sounds

Importance of standards

Changing classifications

Time series

Comparing geographic areas – not as simple as it sounds

We don’t just publish numbers We publish metadata (and a lot of it) Fitness for purpose you decide Quality Statements need to carry a link with the data.

We publish metadata (and a lot of it)

Fitness for purpose

you decide

Quality Statements

need to carry a link with the data.

Copyright and IP issues Policies aim to promote (but not abuse) use of ABS data More aligned to the past Copyright - the 500 cell limit Does this still work? Creative Commons

Policies aim to promote (but not abuse) use of ABS data

More aligned to the past

Copyright - the 500 cell limit

Does this still work?

Creative Commons

The risk of being too interesting 2006 Census results 1.7 million hits in the 3 hours after release CPI is every quarter 300 hits per second at 11.30 am What would these numbers look like if we had automated data feeds as well as static data!

2006 Census results

1.7 million hits in the 3 hours after release

CPI is every quarter

300 hits per second at 11.30 am

The future Some things will continue Continued push to open data More data, more formats, more uses! Data as a service Opportunities and challenges.

Some things will continue

Continued push to open data

More data, more formats, more uses!

Data as a service

Opportunities and challenges.