Workshop of the Expert Group on SDMX. OECD, Paris, 22-25 October 2018.

1. in the Sea of Standards
Workshop of the Expert Group on SDMX
OECD, Paris, 23 October 2018
Xavier Badosa
Statistical Institute of Catalonia
(Idescat)
JSON-stat

2. SDMX
DDI JSON-stat
XBRL
GSBPM GSIM
Dublin Core MARC
PREMIS
METS
OFX IFX
FpML
OAGIS
BPMN
finXML ebXML ...

3. SDMX
DDI JSON-stat
XBRL
GSBPM GSIM
Dublin Core MARC
PREMIS
METS
OFX IFX
FpML
OAGIS
BPMN
finXML ebXML ...

4. SDMX
DDI JSON-stat
XBRL
Standards

5. SDMX
DDI JSON-stat
XBRL
Formats

6. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-stat
XBRL
SDMX-ML
Formats

7. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Formats
OData

8. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Census JSON BLS JSON BEA JSON
ONS JSON StatCan-JSON
…
WB JSON
Formats
OData

9. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Census JSON BLS JSON BEA JSON
ONS JSONWB JSON StatCan-JSON
…
Formats
OData
*-JSON

10. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
…
*-JSON
OData
Apples and oranges

11. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Exchange
*-JSON
OData

12. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Download
*-JSON
OData

13. SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Web APIs
*-JSON
OData

14. Exchange Download Web APIs
1 to 1 (known) 1 to many (unknown)

15. agreement
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
no agreement

16. agreement
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
facilitate the reuse
of our data

17. agreement
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
user-friendly
approach

18. Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
format-agnosticagreement
features
tools

19. SDMX-ML
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
as many popular
formats as possible

20. SDMX-ML
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
CSV
PX
Excel
R
SAS
SPSS
…

21. SDMX-JSON
JSON-stat
PX-JSON
Web APIs
*-JSON
OData

22. 7,000Human Languages
Ole Xavier
Danish Catalan
Hello 你好 ‫مرحبا‬
bridge standard(s)

23. 7,000Human Languages
Danish Catalan
English
bridge standard(s)
native
shared adoption

24. Towards a topography of statistical web API formats

25. *
*
* * * * * * * * * * *
* * * * * * * * * * *

26. OData
JSON-statPX-JSON
SDMX-JSON
*
*
* * * * * * * * * * *
* * * * * * * * * * *

27. SDMX Roadmap 2020
2. Making data usage easier
2.1 Encourage the use of SDMX for data dissemination and for
more interactive data visualisation
2.4 Provide easier access to SDMX for national organisations
through more SDMX-compatible exchange formats, based on the
SDMX Information Model, and better IT tools.

28. SDMX Roadmap 2020
2. Making data usage easier
‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX
implementations, and better IT infrastructure and tools available.

29. JSON-stat SDMX-JSON
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible format

30. JSON-stat SDMX-JSON
tools
features
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible format

31. "observations": {
"0:0:0:0:0:0": [36031.7608618513, 0, null],
"0:0:1:0:0:0": [19533.898066707, 0, null],
"0:0:2:0:0:0": [16497.8627951443, 0, null]
}
SDMX-JSON
"value": [
36031.7608618513,
19533.898066707,
16497.8627951443
]
JSON-stat?

32. [
{
"value": 36031.7608618513,
"SEX": "All persons",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
},
{
"value": 19533.898066707,
"SEX": "Men",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
},
{
"value": 16497.8627951443,
"SEX": "Women",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
}
]
!

33. [
{
"value": 36031.7608618513,
"SEX": "All persons",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
},
{
"value": 19533.898066707,
"SEX": "Men",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
},
{
"value": 16497.8627951443,
"SEX": "Women",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
}
]

34. SDMX-JSON

35. SDMX-JSON
mapDataSetsToJsonStat()
Experimental
2014
JavaScript

36. SDMX-JSON
Python
JavaScript

37. JSON-stat

38. JavaScript
JSON-stat
fromSDMX()

39. Python
JavaScript
JSON-stat

40. Java
Python
JavaScript
JSON-stat

41. Julia
Java
Python
JavaScript
JSON-stat

42. R
Julia
Java
Python
JavaScript
JSON-stat

43. CLI
R
Julia
Java
Python
JavaScript
sdmx2jsonstat
csv2jsonstat
jsonstat2csv
jsonstat2arrobj
JSON-stat

44. "observations": {
"0:0:0:0:0:0": [36031.7608618513, 0, null],
"0:0:1:0:0:0": [19533.898066707, 0, null],
"0:0:2:0:0:0": [16497.8627951443, 0, null]
}
sdmx2jsonstat
"value": [
36031.7608618513,
19533.898066707,
16497.8627951443
]
jsonstat2arrobj
[
{
"value": 36031.7608618513,
"SEX": "All persons",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIME_PERIOD": "2017",
"COUNTRY": "OECD countries"
},
…
R
Julia
Java
Python
JavaScript
JSON-stat
CLI

45. R
Julia
Java
Python
JavaScript
JSON-stat
CLI

46. JSON-stat SDMX-JSON
tools
features

47. light

48. SDMX-ML Generic Data 16.5 MB
PX-JSON 5.1 MB
JSON-stat 0.8 MB
Statistics Sweden
Dataset AM040
13 x 10 x 5 x 1 x 584 = 87,600 obs.
light

49. OECD
Dataset PDB_GR
53 x 32 x 6 x 48 = 488,448 obs.
SDMX-ML Generic Data 5.2 MB
SDMX-JSON flat flavor 2.9 MB
JSON-stat 1.4 MB
SDMX-ML Compact Data 3.8 MB
light

50. JSON-stat SDMX-JSON
tools
features

51. JSON-stat SDMX-JSON
simple
semantically
rich
semantically
dimensions
attributes
measuresdimensions

52. SDMX-JSON
SDMX IM
SDMX COGs
...
SDMX IM
SDMX COGs
...
SDMX-ML

53. SDMX-JSON
SDMX IM
SDMX COGs
...
SDMX IM
SDMX COGs
...
SDMX-ML
SDM
XLAND

54. SDMX-JSON
SDMX IM
SDMX COGs
...
SDMX IM
SDMX COGs
...
SDMX-ML
JSON-stat

55. JSON-stat SDMX-JSON
simple
semantically
rich
semantically
designed to
adapt & grow

56. JSON-stat
simple
semantically
designed to
adapt & grow
most properties
optional
extensible

57. SDMX-JSONJSON-stat

59. JSON-stat
SDMX
extension
as an SDMX-compatible format

60. JSON-stat SDMX-JSON
mappingstandardized
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible format
2. Making data usage easier
‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX
implementations, and better IT infrastructure and tools available.

61. JSON-stat SDMX-JSON
mappingstandardized
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible format
2. Making data usage easier
‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX
implementations, and better IT infrastructure and tools available.
Thank You

62. Boiling Ocean by tosa muu | CC BY-NC
Speech Icon by mcmurryjulie (pixabay.com) | CC0
Old Phone Icon by SVGSilh.com | CC0
Download Icon by SVGSilh.com | CC0
Loudspeaker Bullhorn Icon by SVGSilh.com | CC0
Road in the Mojave Desert by Xavier Badosa | CC BY
The Bai Chay Bridge under Construction by Iman Imanudin | Used with permission
Soma (blocks image) by Dru! | CC BY-NC
Moon by NASA | CC0
Credits