1. Reproductions of this material, or any parts of it, should refer to the IMF Statistics Department as the source.
IMF Statistics Department
Andreas Hake
Division Chief, IMF Statistics Department
2016 SDMX Expert Group Meeting
Aguascalientes, Mexico, October 17-20, 2016
1
Making data discoverable in SDMX
Managing metadata with an SDMX registry
2. IMF Statistics Department
Evolution of data dissemination diagram
2
Human
Readable
Human
Readable
and
Indexed
Human,
Machine
Readable
and
Indexed
Human,
Machine
Readable,
Rich
Metadata-
driven and
Indexed
10/18/2016
6. IMF Statistics Department
Human & Machine Readable, Indexed and
Rich Metadata-driven Datasets
Metadata-driven datasets for
• an SDMX registry enabled
• machine to machine
• automated data exchange
SDMX structures help making data more discoverable
• Data Structures Definition (DSD),
• Provision Agreements, etc..
SDMX notifications services reduce data exchange delay
between data provider and consumer
Data is more discoverable
610/18/2016
For hundreds of years, tables were the main way to disseminate statistical data
In the early years of the internet, data came online, but still mainly in tables and other two-dimensional presentations (e.g. charts and maps)
These are so popular because they are easy for humans to understand
But finding the right table or chart can be tricky…
Data dissemination has evolved to a point where rich, metadata-driven discovery can help users (human or machine) find and access data quickly
As the internet developed, disseminated datasets became bigger and multidimensional
These datasets were indexed in two ways:
Internally, so human users could search for the variables of interest
Externally, so human users could search for the right datasets
The SDDS National Summary Data Page (NSDP) is an example of such an index
Implementation of SDMX provides a standard for providing machine readable data
Early SDMX versions focused on providing a structure for internal definition of datasets
Dissemination mostly by bilateral exchange or posting or hosting datasets
More than just a SDMX registry
fully integrated, metadata-driven service using the SDMX standard as the foundation
enhance data dissemination process building on the SDMX information model
used to describe datasets by defining the
dimensionality of a dataset
allowable content for each of the dimensions
Provision agreements are used to define which data complier or agency supplies which data, aiding discovery
Registrations can drive notifications, supporting automated machine retrieval of data
Human users can browse all datasets registered with SDMX Central, no matter where the SDMX data resides
Load Data to Validate, Transform, Publish, Register and Receive SDMX notification