Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JSON-stat in the Sea of Standards

334 views

Published on

Workshop of the Expert Group on SDMX. OECD, Paris, 22-25 October 2018.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

JSON-stat in the Sea of Standards

  1. 1. in the Sea of Standards Workshop of the Expert Group on SDMX OECD, Paris, 23 October 2018 Xavier Badosa Statistical Institute of Catalonia (Idescat) JSON-stat
  2. 2. SDMX DDI JSON-stat XBRL GSBPM GSIM Dublin Core MARC PREMIS METS OFX IFX FpML OAGIS BPMN finXML ebXML ...
  3. 3. SDMX DDI JSON-stat XBRL GSBPM GSIM Dublin Core MARC PREMIS METS OFX IFX FpML OAGIS BPMN finXML ebXML ...
  4. 4. SDMX DDI JSON-stat XBRL Standards
  5. 5. SDMX DDI JSON-stat XBRL Formats
  6. 6. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-stat XBRL SDMX-ML Formats
  7. 7. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Formats OData
  8. 8. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Census JSON BLS JSON BEA JSON ONS JSON StatCan-JSON … WB JSON Formats OData
  9. 9. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Census JSON BLS JSON BEA JSON ONS JSONWB JSON StatCan-JSON … Formats OData *-JSON
  10. 10. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON … *-JSON OData Apples and oranges
  11. 11. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Exchange *-JSON OData
  12. 12. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Download *-JSON OData
  13. 13. SDMX-EDI SDMX-JSONSDMX-CSV DDI JSON-statCSV-stat XBRL CSV SDMX-ML PX PX-JSON Web APIs *-JSON OData
  14. 14. Exchange Download Web APIs 1 to 1 (known) 1 to many (unknown)
  15. 15. agreement Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) no agreement
  16. 16. agreement Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) facilitate the reuse of our data
  17. 17. agreement Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) user-friendly approach
  18. 18. Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) format-agnosticagreement features tools
  19. 19. SDMX-ML Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) as many popular formats as possible
  20. 20. SDMX-ML Exchange Download Web APIs 1 to 1 1 to many(known) (unknown) CSV PX Excel R SAS SPSS …
  21. 21. SDMX-JSON JSON-stat PX-JSON Web APIs *-JSON OData
  22. 22. 7,000Human Languages Ole Xavier Danish Catalan Hello 你好 ‫مرحبا‬ bridge standard(s)
  23. 23. 7,000Human Languages Danish Catalan English bridge standard(s) native shared adoption
  24. 24. Towards a topography of statistical web API formats
  25. 25. * * * * * * * * * * * * * * * * * * * * * * * *
  26. 26. OData JSON-statPX-JSON SDMX-JSON * * * * * * * * * * * * * * * * * * * * * * * *
  27. 27. SDMX Roadmap 2020 2. Making data usage easier 2.1 Encourage the use of SDMX for data dissemination and for more interactive data visualisation 2.4 Provide easier access to SDMX for national organisations through more SDMX-compatible exchange formats, based on the SDMX Information Model, and better IT tools.
  28. 28. SDMX Roadmap 2020 2. Making data usage easier ‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX implementations, and better IT infrastructure and tools available.
  29. 29. JSON-stat SDMX-JSON Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format
  30. 30. JSON-stat SDMX-JSON tools features Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format
  31. 31. "observations": { "0:0:0:0:0:0": [36031.7608618513, 0, null], "0:0:1:0:0:0": [19533.898066707, 0, null], "0:0:2:0:0:0": [16497.8627951443, 0, null] } SDMX-JSON "value": [ 36031.7608618513, 19533.898066707, 16497.8627951443 ] JSON-stat?
  32. 32. [ { "value": 36031.7608618513, "SEX": "All persons", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 19533.898066707, "SEX": "Men", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 16497.8627951443, "SEX": "Women", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" } ] !
  33. 33. [ { "value": 36031.7608618513, "SEX": "All persons", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 19533.898066707, "SEX": "Men", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, { "value": 16497.8627951443, "SEX": "Women", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" } ]
  34. 34. SDMX-JSON
  35. 35. SDMX-JSON mapDataSetsToJsonStat() Experimental 2014 JavaScript
  36. 36. SDMX-JSON Python JavaScript
  37. 37. JSON-stat
  38. 38. JavaScript JSON-stat fromSDMX()
  39. 39. Python JavaScript JSON-stat
  40. 40. Java Python JavaScript JSON-stat
  41. 41. Julia Java Python JavaScript JSON-stat
  42. 42. R Julia Java Python JavaScript JSON-stat
  43. 43. CLI R Julia Java Python JavaScript sdmx2jsonstat csv2jsonstat jsonstat2csv jsonstat2arrobj JSON-stat
  44. 44. "observations": { "0:0:0:0:0:0": [36031.7608618513, 0, null], "0:0:1:0:0:0": [19533.898066707, 0, null], "0:0:2:0:0:0": [16497.8627951443, 0, null] } sdmx2jsonstat "value": [ 36031.7608618513, 19533.898066707, 16497.8627951443 ] jsonstat2arrobj [ { "value": 36031.7608618513, "SEX": "All persons", "FREQUENCY": "Annual", "SERIES": "Unemployment", "AGE": "Total", "TIME_PERIOD": "2017", "COUNTRY": "OECD countries" }, … R Julia Java Python JavaScript JSON-stat CLI
  45. 45. R Julia Java Python JavaScript JSON-stat CLI
  46. 46. JSON-stat SDMX-JSON tools features
  47. 47. light
  48. 48. SDMX-ML Generic Data 16.5 MB PX-JSON 5.1 MB JSON-stat 0.8 MB Statistics Sweden Dataset AM040 13 x 10 x 5 x 1 x 584 = 87,600 obs. light
  49. 49. OECD Dataset PDB_GR 53 x 32 x 6 x 48 = 488,448 obs. SDMX-ML Generic Data 5.2 MB SDMX-JSON flat flavor 2.9 MB JSON-stat 1.4 MB SDMX-ML Compact Data 3.8 MB light
  50. 50. JSON-stat SDMX-JSON tools features
  51. 51. JSON-stat SDMX-JSON simple semantically rich semantically dimensions attributes measuresdimensions
  52. 52. SDMX-JSON SDMX IM SDMX COGs ... SDMX IM SDMX COGs ... SDMX-ML
  53. 53. SDMX-JSON SDMX IM SDMX COGs ... SDMX IM SDMX COGs ... SDMX-ML SDM XLAND
  54. 54. SDMX-JSON SDMX IM SDMX COGs ... SDMX IM SDMX COGs ... SDMX-ML JSON-stat
  55. 55. JSON-stat SDMX-JSON simple semantically rich semantically designed to adapt & grow
  56. 56. JSON-stat simple semantically designed to adapt & grow most properties optional extensible
  57. 57. SDMX-JSONJSON-stat
  58. 58. JSON-stat SDMX extension as an SDMX-compatible format
  59. 59. JSON-stat SDMX-JSON mappingstandardized Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format 2. Making data usage easier ‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX implementations, and better IT infrastructure and tools available.
  60. 60. JSON-stat SDMX-JSON mappingstandardized Easier JSON-stat support in SDMX infrastructures JSON-stat as an SDMX- compatible format 2. Making data usage easier ‘Easy-to-use’ SDMX-compatible formats, more harmonised SDMX implementations, and better IT infrastructure and tools available. Thank You
  61. 61. Boiling Ocean by tosa muu | CC BY-NC Speech Icon by mcmurryjulie (pixabay.com) | CC0 Old Phone Icon by SVGSilh.com | CC0 Download Icon by SVGSilh.com | CC0 Loudspeaker Bullhorn Icon by SVGSilh.com | CC0 Road in the Mojave Desert by Xavier Badosa | CC BY The Bai Chay Bridge under Construction by Iman Imanudin | Used with permission Soma (blocks image) by Dru! | CC BY-NC Moon by NASA | CC0 Credits

×