Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
in the Sea of Standards
Workshop of the Expert Group on SDMX
OECD, Paris, 23 October 2018
Xavier Badosa
Statistical Instit...
SDMX
DDI JSON-stat
XBRL
GSBPM GSIM
Dublin Core MARC
PREMIS
METS
OFX IFX
FpML
OAGIS
BPMN
finXML ebXML ...
SDMX
DDI JSON-stat
XBRL
GSBPM GSIM
Dublin Core MARC
PREMIS
METS
OFX IFX
FpML
OAGIS
BPMN
finXML ebXML ...
SDMX
DDI JSON-stat
XBRL
Standards
SDMX
DDI JSON-stat
XBRL
Formats
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-stat
XBRL
SDMX-ML
Formats
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Formats
OData
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Census JSON BLS JSON BEA JSON
ONS JSON StatCa...
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Census JSON BLS JSON BEA JSON
ONS JSONWB JSON...
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
…
*-JSON
OData
Apples and oranges
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Exchange
*-JSON
OData
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Download
*-JSON
OData
SDMX-EDI SDMX-JSONSDMX-CSV
DDI JSON-statCSV-stat
XBRL CSV
SDMX-ML PX PX-JSON
Web APIs
*-JSON
OData
Exchange Download Web APIs
1 to 1 (known) 1 to many (unknown)
agreement
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
no agreement
agreement
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
facilitate the reuse
of our data
agreement
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
user-friendly
approach
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
format-agnosticagreement
features
tools
SDMX-ML
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
as many popular
formats as possible
SDMX-ML
Exchange Download Web APIs
1 to 1 1 to many(known) (unknown)
CSV
PX
Excel
R
SAS
SPSS
…
SDMX-JSON
JSON-stat
PX-JSON
Web APIs
*-JSON
OData
7,000Human Languages
Ole Xavier
Danish Catalan
Hello 你好 ‫مرحبا‬
bridge standard(s)
7,000Human Languages
Danish Catalan
English
bridge standard(s)
native
shared adoption
Towards a topography of statistical web API formats
*
*
* * * * * * * * * * *
* * * * * * * * * * *
OData
JSON-statPX-JSON
SDMX-JSON
*
*
* * * * * * * * * * *
* * * * * * * * * * *
SDMX Roadmap 2020
2. Making data usage easier
2.1 Encourage the use of SDMX for data dissemination and for
more interactiv...
SDMX Roadmap 2020
2. Making data usage easier
‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX
implementations,...
JSON-stat SDMX-JSON
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible format
JSON-stat SDMX-JSON
tools
features
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible format
"observations": {
"0:0:0:0:0:0": [36031.7608618513, 0, null],
"0:0:1:0:0:0": [19533.898066707, 0, null],
"0:0:2:0:0:0": [1...
[
{
"value": 36031.7608618513,
"SEX": "All persons",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIM...
[
{
"value": 36031.7608618513,
"SEX": "All persons",
"FREQUENCY": "Annual",
"SERIES": "Unemployment",
"AGE": "Total",
"TIM...
SDMX-JSON
SDMX-JSON
mapDataSetsToJsonStat()
Experimental
2014
JavaScript
SDMX-JSON
Python
JavaScript
JSON-stat
JavaScript
JSON-stat
fromSDMX()
Python
JavaScript
JSON-stat
Java
Python
JavaScript
JSON-stat
Julia
Java
Python
JavaScript
JSON-stat
R
Julia
Java
Python
JavaScript
JSON-stat
CLI
R
Julia
Java
Python
JavaScript
sdmx2jsonstat
csv2jsonstat
jsonstat2csv
jsonstat2arrobj
JSON-stat
"observations": {
"0:0:0:0:0:0": [36031.7608618513, 0, null],
"0:0:1:0:0:0": [19533.898066707, 0, null],
"0:0:2:0:0:0": [1...
R
Julia
Java
Python
JavaScript
JSON-stat
CLI
JSON-stat SDMX-JSON
tools
features
light
SDMX-ML Generic Data 16.5 MB
PX-JSON 5.1 MB
JSON-stat 0.8 MB
Statistics Sweden
Dataset AM040
13 x 10 x 5 x 1 x 584 = 87,60...
OECD
Dataset PDB_GR
53 x 32 x 6 x 48 = 488,448 obs.
SDMX-ML Generic Data 5.2 MB
SDMX-JSON flat flavor 2.9 MB
JSON-stat 1.4...
JSON-stat SDMX-JSON
tools
features
JSON-stat SDMX-JSON
simple
semantically
rich
semantically
dimensions
attributes
measuresdimensions
SDMX-JSON
SDMX IM
SDMX COGs
...
SDMX IM
SDMX COGs
...
SDMX-ML
SDMX-JSON
SDMX IM
SDMX COGs
...
SDMX IM
SDMX COGs
...
SDMX-ML
SDM
XLAND
SDMX-JSON
SDMX IM
SDMX COGs
...
SDMX IM
SDMX COGs
...
SDMX-ML
JSON-stat
JSON-stat SDMX-JSON
simple
semantically
rich
semantically
designed to
adapt & grow
JSON-stat
simple
semantically
designed to
adapt & grow
most properties
optional
extensible
SDMX-JSONJSON-stat
JSON-stat
SDMX
extension
as an SDMX-compatible format
JSON-stat SDMX-JSON
mappingstandardized
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible ...
JSON-stat SDMX-JSON
mappingstandardized
Easier JSON-stat support
in SDMX infrastructures
JSON-stat as an SDMX-
compatible ...
Boiling Ocean by tosa muu | CC BY-NC
Speech Icon by mcmurryjulie (pixabay.com) | CC0
Old Phone Icon by SVGSilh.com | CC0
D...
JSON-stat in the Sea of Standards
Upcoming SlideShare
Loading in …5
×

JSON-stat in the Sea of Standards

Workshop of the Expert Group on SDMX. OECD, Paris, 22-25 October 2018.

  • Be the first to comment

JSON-stat in the Sea of Standards

  1. 1. in the Sea of Standards Workshop of the Expert Group on SDMX OECD, Paris, 23 October 2018 Xavier Badosa Statistical Institute of Catalonia (Idescat) JSON-stat
  2. 2. SDMX DDI JSON-stat XBRL GSBPM GSIM Dublin Core MARC PREMIS METS OFX IFX FpML OAGIS BPMN finXML ebXML ...
  3. 3. SDMX DDI JSON-stat XBRL GSBPM GSIM Dublin Core MARC PREMIS METS OFX IFX FpML OAGIS BPMN finXML ebXML ...
  4. 4. SDMX DDI JSON-stat XBRL Standards
  5. 5. SDMX DDI JSON-stat XBRL Formats
  6. 6. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-stat XBRL SDMX-ML Formats
  7. 7. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Formats OData
  8. 8. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Census JSON BLS JSON BEA JSON ONS JSON StatCan-JSON … WB JSON Formats OData
  9. 9. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Census JSON BLS JSON BEA JSON ONS JSONWB JSON StatCan-JSON … Formats OData *-JSON
  10. 10. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON … *-JSON OData Apples and oranges
  11. 11. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Exchange *-JSON OData
  12. 12. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Download *-JSON OData
  13. 13. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Web APIs *-JSON OData
  14. 14. Exchange Download Web APIs 1 to 1 (known) 1 to many (unknown)
  15. 15. agreement Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) no agreement
  16. 16. agreement Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) facilitate the reuse of our data
  17. 17. agreement Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) user-friendly approach
  18. 18. Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) format-agnosticagreement features tools
  19. 19. SDMX-ML Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) as many popular formats as possible
  20. 20. SDMX-ML Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) CSV PX Excel R SAS SPSS …
  21. 21. SDMX-JSON JSON-stat PX-JSON Web APIs *-JSON OData
  22. 22. 7,000Human Languages Ole Xavier Danish Catalan Hello 你好 ‫مرحبا‬ bridge standard(s)
  23. 23. 7,000Human Languages Danish Catalan English bridge standard(s) native shared adoption
  24. 24. Towards a topography of statistical web API formats
  25. 25. * * * * * * * * * * * * * * * * * * * * * * * *
  26. 26. OData JSON-statPX-JSON SDMX-JSON * * * * * * * * * * * * * * * * * * * * * * * *
  27. 27. SDMX Roadmap 2020 2. Making data usage easier 2.1 Encourage the use of SDMX for data dissemination and for more interactive data visualisation 2.4 Provide easier access to SDMX for national organisations through more SDMX-compatible exchange formats, based on the SDMX Information Model, and better IT tools.
  28. 28. SDMX Roadmap 2020 2. Making data usage easier ‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX implementations, and better IT infrastructure and tools available.
  29. 29. JSON-stat SDMX-JSON Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format
  30. 30. JSON-stat SDMX-JSON tools features Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format
  31. 31. "observations": { "0:0:0:0:0:0": [36031.7608618513, 0, null], "0:0:1:0:0:0": [19533.898066707, 0, null], "0:0:2:0:0:0": [16497.8627951443, 0, null] } SDMX-JSON "value": [ 36031.7608618513, 19533.898066707, 16497.8627951443 ] JSON-stat?
  32. 32. [ { "value": 36031.7608618513, "SEX": "All persons", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 19533.898066707, "SEX": "Men", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 16497.8627951443, "SEX": "Women", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" } ] !
  33. 33. [ { "value": 36031.7608618513, "SEX": "All persons", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 19533.898066707, "SEX": "Men", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 16497.8627951443, "SEX": "Women", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" } ]
  34. 34. SDMX-JSON
  35. 35. SDMX-JSON mapDataSetsToJsonStat() Experimental 2014 JavaScript
  36. 36. SDMX-JSON Python JavaScript
  37. 37. JSON-stat
  38. 38. JavaScript JSON-stat fromSDMX()
  39. 39. Python JavaScript JSON-stat
  40. 40. Java Python JavaScript JSON-stat
  41. 41. Julia Java Python JavaScript JSON-stat
  42. 42. R Julia Java Python JavaScript JSON-stat
  43. 43. CLI R Julia Java Python JavaScript sdmx2jsonstat csv2jsonstat jsonstat2csv jsonstat2arrobj JSON-stat
  44. 44. "observations": { "0:0:0:0:0:0": [36031.7608618513, 0, null], "0:0:1:0:0:0": [19533.898066707, 0, null], "0:0:2:0:0:0": [16497.8627951443, 0, null] } sdmx2jsonstat "value": [ 36031.7608618513, 19533.898066707, 16497.8627951443 ] jsonstat2arrobj [ { "value": 36031.7608618513, "SEX": "All persons", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, … R Julia Java Python JavaScript JSON-stat CLI
  45. 45. R Julia Java Python JavaScript JSON-stat CLI
  46. 46. JSON-stat SDMX-JSON tools features
  47. 47. light
  48. 48. SDMX-ML Generic Data 16.5 MB PX-JSON 5.1 MB JSON-stat 0.8 MB Statistics Sweden Dataset AM040 13 x 10 x 5 x 1 x 584 = 87,600 obs. light
  49. 49. OECD Dataset PDB_GR 53 x 32 x 6 x 48 = 488,448 obs. SDMX-ML Generic Data 5.2 MB SDMX-JSON flat flavor 2.9 MB JSON-stat 1.4 MB SDMX-ML Compact Data 3.8 MB light
  50. 50. JSON-stat SDMX-JSON tools features
  51. 51. JSON-stat SDMX-JSON simple semantically rich semantically dimensions attributes measuresdimensions
  52. 52. SDMX-JSON SDMX IM SDMX COGs ... SDMX IM SDMX COGs ... SDMX-ML
  53. 53. SDMX-JSON SDMX IM SDMX COGs ... SDMX IM SDMX COGs ... SDMX-ML SDM XLAND
  54. 54. SDMX-JSON SDMX IM SDMX COGs ... SDMX IM SDMX COGs ... SDMX-ML JSON-stat
  55. 55. JSON-stat SDMX-JSON simple semantically rich semantically designed to adapt & grow
  56. 56. JSON-stat simple semantically designed to adapt & grow most properties optional extensible
  57. 57. SDMX-JSONJSON-stat
  58. 58. JSON-stat SDMX extension as an SDMX-compatible format
  59. 59. JSON-stat SDMX-JSON mappingstandardized Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format 2. Making data usage easier ‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX implementations, and better IT infrastructure and tools available.
  60. 60. JSON-stat SDMX-JSON mappingstandardized Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format 2. Making data usage easier ‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX implementations, and better IT infrastructure and tools available. Thank You
  61. 61. Boiling Ocean by tosa muu | CC BY-NC Speech Icon by mcmurryjulie (pixabay.com) | CC0 Old Phone Icon by SVGSilh.com | CC0 Download Icon by SVGSilh.com | CC0 Loudspeaker Bullhorn Icon by SVGSilh.com | CC0 Road in the Mojave Desert by Xavier Badosa | CC BY The Bai Chay Bridge under Construction by Iman Imanudin | Used with permission Soma (blocks image) by Dru! | CC BY-NC Moon by NASA | CC0 Credits

×