Standards for statistical data dissemination: a wish list

Standards for statistical data dissemination a wish list Xavier Badosa (@badosa) Statistical Institute of Catalonia OECD Statistical Information System Collaboration Community (SIS-CC) Workshop 2012. Paris, 12-14 March

For humans

For humans (naked)

For humans (naked)(accessed and read by humans)

For humans (naked)(accessed and read humans)

For humans with machines(accessed by humans,but read by machines) For humans (naked) (accessed and read humans)

For machines

For machines built by humans

For machines built by humans(accessed and read by machines)

API D A T A

General-purpose“Apps”Third-party“Apps”that solve specific needsStatistics as platform “O.S.”

How I stopped worrying about mywebsite and learned to love the wholeInternet Matt McAlister Director of Digital Strategy Guardian Media Group

We don’t operate in theeyeball market maximum data exposure & reachWe operate in the accuracy preservationreferencemarket brand recognition

E A BCHA Big orgsNG Stats-savvyE B2B

E A BCHA Big orgsNG Stats-savvyE B2B Shared environment

D I SS EM I N A T IO N

like the ? WWW

For machines built by humans

don’t make me think KISS principlerespect the conventions develop also for the weakest link

is a platform An example

is widely used An example

2006 is widely used2009 50,000 applications

is widely used# mashups

is widely usedhad more than1 provider

is widely used= is a standard

“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.” Anil Dash Expert Labshttp://dashes.com/anil/2009/12/the-twitter-api-is-finished.html

KISS principle“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.” Anil Dash Expert Labshttp://dashes.com/anil/2009/12/the-twitter-api-is-finished.html

develop for the weakest link“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.” Anil Dash Expert Labshttp://dashes.com/anil/2009/12/the-twitter-api-is-finished.html

provide a useful service“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.” Anil Dash Expert Labshttp://dashes.com/anil/2009/12/the-twitter-api-is-finished.html

get close to the end users“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.” Anil Dash Expert Labshttp://dashes.com/anil/2009/12/the-twitter-api-is-finished.html

langs / APIs ~ browsers / websites“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.” Anil Dash Expert Labshttp://dashes.com/anil/2009/12/the-twitter-api-is-finished.html

“these APIs are simple enough for weekendhackers to build interesting projects on, and(…) easy to implement even on mobile devicesand in almost any programming language.”

2 APIsREST API 100 x 1.5 = 150 pp.Streaming API 6 x 3 = 18 pp. < 200 pp.

208 pp. < 200 pp.books.google.com

The Fifteen Minute Rule A person of reasonable ability should be able toget from zero to ‘Hello World’ in fifteen minutes. Michael E. Driscoll Dataspore, Metamarkets

http://lcsd05.cs.tamu.edu/slides/keynote.pdf

18 pp.1 PDF18 pp.

18 pp. 1 PDF143 pp.

161 pp. 1 PDF143 pp.

161 pp.ZIP 8 MSW

1,109 pp.ZIP 8 MSW 46 pp. 341 pp. 269 pp. 68 pp. 176 pp. 34 pp. 4 pp. 10 pp.

1,109 pp.Samples

1,278 pp.ZIP 1 PDF 169 pp.

1,327 pp. PDF 49 pp.

1,377 pp. PDF 50 pp.

1,408 pp. PDF 31 pp.

1,408 pp. 1,344 pp.books.google.com

high barrier to entry APIdevelopers

ECHAN SDMXGE Shared environment

ECHAN SDMXG amazingly flexibleE Shared environment

ECHAN SDMXG amazingly complexE Shared environment

ECHAN SDMXG amazingly complexESDMX-ML is a meta-language

D I S TypePad WordPressS EM I N A T I existing Twitter clientsO N

The Metcalfe-Bray Law D The value of a I markup language is proportional S TypePad WordPress approximatelyS to the square E of theM number of I different N software implementations A that can process T it. I Tim Bray existing Twitter clientsO Google N

The Metcalfe-Bray Law D The value of a I markup language is proportional S Stats provider Stats provider approximatelyS to the square E of theM number of I different N software implementations A that can process T it. I Tim Bray existing clientsO Google N

D I Are we S Stats provider Stats provider so specialS that E weM can’t I benefit N from A existing T libraries I and clients? existing clientsO N

Are we so special that we Special things can’t benefit from existing libraries and clients?Common things

Population Search API

Population Search API Atom

Population Search API Atom OpenSearch

Population Search APIAtom OpenSearch SDMX

Population Search API

Rectifications API

Rectifications API Atom OpenSearch

AtomOpenSearchhttp://www.opensearch.orgOpen Data Protocolhttp://www.odata.orgGoogle Data Protocolhttp://code.google.com/apis/gdata/Open Publication Distribution Systemhttp://opds-spec.org

Are we so special that we Special things can’t benefit from existing libraries and clients?Common things

Are we so special SDMX? that we Special things can’t benefit from existing libraries and clients?Common things

Are we so special SDMX? that we Special things can’t benefit from Simplified SDMX? existing libraries and clients?Common things

A Complex System That Works Is InvariablyFound To Have Evolved From A Simple System That Worked. “Gall’s Law” Systemantics How Systems Work and Especially How They Fail John Gall

The parallel proposition also appears to be true: A Complex System Designed From Scratch Never Works And Cannot Be Made To Work. You Have To Start Over, Beginning With A Working Simple System. “Gall’s Law” Systemantics How Systems Work and Especially How They Fail John Gall

What’s common practice in the API space?

Open APIs and the Semantic Web (John Musser, ProgrammableWeb)http://www.slideshare.net/jmusser/j-musser-semtechjun2011

“REST” RESTish Pragmatic REST REST-inspiredOpen APIs and the Semantic Web (John Musser, ProgrammableWeb)http://www.slideshare.net/jmusser/j-musser-semtechjun2011

<KeyFamilyRef>ALFS_SUMTAB</KeyFamilyRef> <Series> <SeriesKey> <Value concept="LOCATION" value="AUS" /> <Value concept="SUBJECT" value="YGTT01L1_ST" /> <Value concept="FREQUENCY" value="A" /> </SeriesKey> <Attributes> <Value concept="TIME_FORMAT" value="P1Y" /> </Attributes> <Obs> <Time>2000</Time> <ObsValue value="19153" /> </Obs> <Obs> <Time>2001</Time> <ObsValue value="19413" /> </Obs> <Obs> <Time>2002</Time>SDMX-ML <ObsValue value="19651" /> </Obs> <Obs> <Time>2003</Time> <ObsValue value="19895" /> </Obs> <Obs> <Time>2004</Time> <ObsValue value="20127" />fragment </Obs> <Obs> <Time>2005</Time> <ObsValue value="20395" /> </Obs> <Obs> <Time>2006</Time> <ObsValue value="20698" /> </Obs> <Obs> <Time>2007</Time> <ObsValue value="21072" /> </Obs> <Obs> <Time>2008</Time> <ObsValue value="21499" /> </Obs> <Obs> <Time>2009</Time> <ObsValue value="21955" /> </Obs> <Obs> <Time>2010</Time> <ObsValue value="22342" /> </Obs> </Series>

<KeyFamilyRef>ALFS_SUMTAB</KeyFamilyRef><KeyFamilyRef>ALFS_SUMTAB</KeyFamilyRef> <Series> <SeriesKey><Series> <Value concept="LOCATION" value="AUS" /> <SeriesKey> <Value concept="SUBJECT" value="YGTT01L1_ST" /> <Value concept="FREQUENCY" value="A" /> <Value concept="LOCATION" value="AUS" /> </SeriesKey> <Attributes> <Value concept="SUBJECT" value="YGTT01L1_ST" /> <Value concept="TIME_FORMAT" value="P1Y" /> </Attributes> <Value concept="FREQUENCY" value="A" /> <Obs> </SeriesKey> <Time>2000</Time> <ObsValue value="19153" /> <Attributes> </Obs> <Obs> <Value concept="TIME_FORMAT" value="P1Y" /> <Time>2001</Time> <ObsValue value="19413" /> </Attributes> </Obs> <Obs> <Obs> <Time>2002</Time> <Time>2000</Time> <ObsValue value="19651" /> </Obs> <ObsValue value="19153" /> <Obs> <Time>2003</Time> </Obs> <ObsValue value="19895" /> <Obs> </Obs> <Obs> <Time>2001</Time> <Time>2004</Time> <ObsValue value="20127" /> <ObsValue value="19413" /> </Obs> <Obs> </Obs> <Time>2005</Time> <Obs> <ObsValue value="20395" /> </Obs> <Time>2002</Time> <Obs> <Time>2006</Time> <ObsValue value="19651" /> <ObsValue value="20698" /> </Obs> </Obs> <Obs> <Obs> <Time>2007</Time> <ObsValue value="21072" /> <Time>2003</Time> </Obs> <Obs> <ObsValue value="19895" /> <Time>2008</Time> <ObsValue value="21499" /> </Obs> </Obs> <Obs> <Obs> <Time>2009</Time> <Time>2004</Time> <ObsValue value="21955" /> </Obs> <ObsValue value="20127" /> <Obs> <Time>2010</Time> </Obs> <ObsValue value="22342" /> <Obs> </Obs> </Series> <Time>2005</Time>

<KeyFamilyRef>ALFS_SUMTAB</KeyFamilyRef> "KeyFamilyRef" : "ALFS_SUMTAB" ,<Series> "Series" : { <SeriesKey> "SeriesKey" : { <Value concept="LOCATION" value="AUS" /> "location" : "AUS" , <Value concept="SUBJECT" value="YGTT01L1_ST" /> "subject" : "YGTT01L1_ST" , <Value concept="FREQUENCY" value="A" /> "frequency" : "A" </SeriesKey> }, <Attributes> "Attributes" : { <Value concept="TIME_FORMAT" value="P1Y" /> "time_format" : "P1Y" </Attributes> }, <Obs> "Obs" : [ <Time>2000</Time> {"2000" : 19153}, <ObsValue value="19153" /> {"2001" : 19413}, </Obs> {"2002" : 19651}, <Obs> {"2003" : 19895}, <Time>2001</Time> {"2004" : 20127}, <ObsValue value="19413" /> {"2005" : 20395}, </Obs> {"2006" : 20698}, <Obs> {"2007" : 21072}, <Time>2002</Time> {"2008" : 21499}, <ObsValue value="19651" /> {"2009" : 21955}, </Obs> {"2010" : 22342} <Obs> ] <Time>2003</Time> <ObsValue value="19895" /> </Obs> <Obs> <Time>2004</Time> <ObsValue value="20127" /> </Obs> <Obs> <Time>2005</Time>

"KeyFamilyRef" : "ALFS_SUMTAB" ,"Series" : { "SeriesKey" : { "location" : "AUS" , "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : { "time_format" : "P1Y" }, "Obs" : [ {"2000" : 19153}, {"2001" : 19413}, {"2002" : 19651}, {"2003" : 19895}, {"2004" : 20127}, {"2005" : 20395}, {"2006" : 20698}, {"2007" : 21072}, {"2008" : 21499}, {"2009" : 21955}, {"2010" : 22342} ]

"KeyFamilyRef" : "ALFS_SUMTAB" ,"Series" : { "SeriesKey" : { "location" : "AUS" , "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : { "time_format" : "P1Y" }, "Obs" : { "2000" : 19153, "2001" : 19413, "2002" : 19651, "2003" : 19895, "2004" : 20127, "2005" : 20395, "2006" : 20698, "2007" : 21072, "2008" : 21499, "2009" : 21955, "2010" : 22342 }

"KeyFamilyRef" : "ALFS_SUMTAB" ,"Series" : { "SeriesKey" : { "location" : "AUS" , "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : { "time_format" : "P1Y" }, "Obs" : { "time": [ "2000","2001","2002","2003", "2004","2005","2006","2007", "2008","2009","2010" ], "value": [ 19153,19413,19651,19895, 20127,20395,20698,21072, 21499,21955,22342 ] }

"KeyFamilyRef" : "ALFS_SUMTAB" , "Series" : { "SeriesKey" : { "location" : "AUS" ,Natural to programmers "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : { “time_format" : "P1Y" }, "Obs" : { "time": [ "2000","2001","2002","2003", "2004","2005","2006","2007", "2008","2009","2010" ], "value": [ 19153,19413,19651,19895, 20127,20395,20698,21072, 21499,21955,22342 ] }

"KeyFamilyRef" : "ALFS_SUMTAB" ,a “data” format "Series" : { "SeriesKey" : { "location" : "AUS" ,Natural to programmers "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : {string, number, bool, null “time_format" : "P1Y" },array (vector, list, sequence) "Obs" : { "time": [ "2000","2001","2002","2003",object (struct, hash table, "2004","2005","2006","2007",associative array, map, "2008","2009","2010" ],record, keyed list, dictionary) "value": [ 19153,19413,19651,19895, 20127,20395,20698,21072, 21499,21955,22342 ] }

"KeyFamilyRef" : "ALFS_SUMTAB" ,a “data” format "Series" : { "SeriesKey" : { "location" : "AUS" ,Natural to programmers "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : { “time_format" : "P1Y"“JSON shines as a }, "Obs" : {programming language- "time": [independent representation "2000","2001","2002","2003", "2004","2005","2006","2007",of typical programming "2008","2009","2010"language data structures.” ], "value": [ James Clark 19153,19413,19651,19895, Technical lead for the W3C XML 20127,20395,20698,21072, activity which developed 21499,21955,22342 XML 1.0 Recommendation ] }

"KeyFamilyRef" : "ALFS_SUMTAB" , "Series" : { "SeriesKey" : { "location" : "AUS" ,Natural to programmers "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : {value[0] = 19153 “time_format" : "P1Y"value[1] = 19413 },value[2] = 19651 "Obs" : { "time": [value[3] = 19895 "2000","2001","2002","2003",value[4] = 20127 "2004","2005","2006","2007",value[5] = 20395 "2008","2009","2010" ],value[6] = 20698 "value": [value[7] = 21072 19153,19413,19651,19895,value[8] = 21499 20127,20395,20698,21072, 21499,21955,22342value[9] = 21955 ]value[10] = 22342 }

"KeyFamilyRef" : "ALFS_SUMTAB" , "Series" : { "SeriesKey" : { "location" : "AUS" ,Natural to programmers "subject" : "YGTT01L1_ST" , "frequency" : "A" }, "Attributes" : { "time_format" : "P1Y" }, "Obs" : { "time": [ "2000","2001","2002","2003", "2004","2005","2006","2007", "2008","2009","2010" ], "value": [ 19153,19413,19651,19895, 20127,20395,20698,21072, 21499,21955,22342 ] }

From document viewerNatural to programmers to application platform

Fragmented device market

Storage Offline GeolocationAudio and Video Canvas Drag and Drop … JS APIs

SGML / DTD XML / XSD

JSONP callbackJSON with padding

http://caniuse.com/#feat=cors

Date: Wed, 22 Feb 2012 17:12:52 GMTServer: ApacheExpires: Wed, 22 Feb 2012 18:12:52 GMTAccess-Control-Allow-Origin: *Content-Length: 697Connection: closeContent-Type: application/json;charset=utf-8200 OKhttp://api.idescat.cat/pob/v1/sug.json?q=bar

XML SDMX-MLJSON SDMX-JS?

XMLJSON

XML Xml2Json() JSONOn-the-fly conversion

XML Xml2Json() JSONOn-the-fly conversion<text>Hello, World!</text>

XML Xml2Json() JSONOn-the-fly conversion<text>Hello, World!</text>"text": "Hello, World!"

XML Xml2Json() JSONOn-the-fly conversion<text>Hello, World!</text><text>Goodbye, Life!</text>"text": "Hello, World!"

XML Xml2Json() JSONOn-the-fly conversion<text>Hello, World!</text><text>Goodbye, Life!</text>"text": ["Hello, World!", "Goodbye, Life!"]

XML Xml2Json() JSONOn-the-fly conversion<text>Hello, World!</text>"text": ["Hello, World!"]

XML JSONPredefined map

XML JSONPredefined map namespaces? elements/attributes? all strings?

“XMLish” JSONPredefined map

{"@concept" : "OBS_STATUS", "@value" : "P"}}},{"generic:Time" : "2011-04", "generic:ObsValue" : “XMLish”{"@value" : "106.56246"}, "generic:Attributes" :{"generic:Value" : {"@concept" : "OBS_STATUS","@value" : "P"}}}, {"generic:Time" : "2011-05", JSON"generic:ObsValue" : {"@value" : "113.26596"},"generic:Attributes" : {"generic:Value" :{"@concept" : "OBS_STATUS", "@value" : "P"}}},{"generic:Time" : "2011-06", "generic:ObsValue" :{"@value" : "114.22037"}, "generic:Attributes" :{"generic:Value" : {"@concept" : "OBS_STATUS","@value" : "P"}}}, {"generic:Time" : "2011-07","generic:ObsValue" : {"@value" : "108.77534"},"generic:Attributes" : {"generic:Value" :{"@concept" : "OBS_STATUS", "@value" : "P"}}},{"generic:Time" : "2011-08", "generic:ObsValue" :{"@value" : "116.37424"}, "generic:Attributes" :

natural-bornJSON

natural-born JSON“JSON-G” genuine

natural-born JSON“JSON-G”“JSON-C”compact/customized

Price of an apple?Save as many bytesas possible

Price of an apple?Nutrition valueAllergiesRecipesPriceForbidden fruit inthe Garden of Eden

FlexibilityNutrition valueAllergiesRecipesPriceForbidden fruit inthe Garden of Eden

detail=fulldetail=dataonlydetail=serieskeysonlydetail=nodata

ResponseGroup=Images,Reviews

fields=id,name,picture

fields=title,teaserbyline,none

:(id,first-name,last-name,positions:(title))

fields=id,scope/type

fields=entry(id,gd:when)

fields=entry(@gd:*,id,gd:when)

fields=entry(title,link[@rel=self or @rel=edit],media:group(media:thumbnail))

fields=entry[media:group/yt:duration/@seconds<= 120 and yt:statistics/@viewCount > 20]

simple flexible RESTish JSONlightweightdeveloper-centric mobile-friendly

simple flexible RESTish JSONlightweight Semantic Web-ish?developer-centric mobile-friendly

I have a dream for the Web [in which computers] becomecapable of analyzing all the data on the Web – the content,links, and transactions between people and computers. A‘Semantic Web’, which should make this possible, has yet toemerge, but when it does, the day-to-day mechanisms oftrade, bureaucracy and our daily lives will be handled bymachines talking to machines. The ‘intelligent agents’ peoplehave touted for ages will finally materialize. Tim Berners-Lee Director of the W3C, 1999

Statistical “Cube” Data. The group will produce avocabulary, compatible with SDMX, for expressingsome kinds of statistical data. This need not be asexpressive as all of SDMX, but may provide a subsetas in the RDF Data Cube vocabulary. It may also include ways toannotate data to indicate its assumptions and comparability.

The MainstreamRelevance “Law”The mainstreamrelevance of acommunicationenvironment isproportional tothe quantity ofrubbish in thatenvironment.

The MainstreamRelevance “Law” TV e-mail Twitter ...

simple? flexible RESTish JSON?lightweight? Semantic Web-ish?developer-centric? mobile-friendly?

For some web developers the need to understandthe RDF data model and associated serializations andquery language (SPARQL) has proved a barrier toadoption of linked data. This project seeks to developAPIs, data formats and supporting tools to overcomethis barrier. Including, but not limited to, accessinglinked data via a developer-friendly JSON format.

Simplify + JSONify SDMX?Reuse/adapt a generic data format? semantic

Simplify + JSONify SDMX?Reuse/adapt a generic data format? semanticCreate a new simple standard for stats?

http://json-stat.org

http://json-stat.orglightweight

http://json-stat.orgsimple

http://json-stat.orgdeveloper-centric

http://json-stat.orgflexible

http://code.google.com/p/json-stat/

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1,1,1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "U" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "value": [7539618] "label" : "When" , "desc" : "Longer text", "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index" : { "B" : 0 } , "label" : { "B" : "Catalonia" } } } }}

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } }"id"} , : ["metric", "time", "geo"], "time" : {"size" : [1, 1, 1],// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index" : { "B" : 0 } , "label" : { "B" : "Catalonia" } } } }}

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "index" : { "pop" : 0 } , "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} "label" : { "pop" : "Population" } } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index" : { "B" : 0 } , "label" : { "B" : "Catalonia" } } } }}

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index" : { "B" : 0 } , "label" : { "B" : "Catalonia" } } } }}

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text","index" : {"2011-01-01" : 0} , "category" : { "index" : { "B" : 0 } , "label" : { "B" : "Catalonia" }"label" : {"2011-01-01" : "January 1st, 2011"} } } }}

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": "Latest population figure of Catalonia", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } ,"index" : { "ISO-3166-2:ES-CT" : 0 } ,// "time" : { "label" : "When" , "desc" : "Longer text","label" : { "ISO-3166-2:ES-CT" : "Catalonia" } "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": "Population of Catalonia, January 1st, 2011", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "type" : { "pop" : "count" } , "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "base" : { "pop" : "Person" } , "geo" : {// "symbol" : { "label" : "Where" , "desc" : "Longer text", "pop" : null } , "mult" "category" : { : { "pop" : 0 } "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "long" : 2.178389, "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "lat" : 41.383531 "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"2011-01-01":0} , "label": {"2011-01-01":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {"index" : {"20110101" : 0} ,// "label" : "Where" , "desc" : "Longer text", "category" : {"label" : {"20110101" : "January 1st, 2011"} "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "index" "dimension" : {: { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "from" : "20000101" "metric": { ,// "to" : "20110101" "label" : "What" , "desc" : "Longer text", "category" : { } "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {"index" : {"20110101" : 0} ,// "label" : "Where" , "desc" : "Longer text", "category" : {"label" : {"20110101" : "January 1st, 2011"} "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

{ "label": “Population of Catalonia, January 1st, 2011", "value": [7539618] , "dimension" : { "id" : ["metric", "time", "geo"], "size" : [1, 1, 1], "metric": {// "label" : "What" , "desc" : "Longer text", "category" : { "index" : { "pop" : 0 } , "label" : { "pop" : "Population" } , "unit" : {"type" : {"pop" : "count"} , "base" : {"pop" : "Person"} ,"symbol" : { "pop" : null} ,"mult" : {"pop" : 0} } } } , "time" : {// "label" : "When" , "desc" : "Longer text", "category" : { "index": {"20110101":0} , "label": {"20110101":"January 1st, 2011"} } } , "geo" : {// "label" : "Where" , "desc" : "Longer text", "category" : { "index":{"ISO-3166-2:ES-CT":0} , "label":{"ISO-3166-2:ES-CT":"Catalonia"} } } }}

"value": [3732196, 3807422 , 7539618]

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex" : { "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } }}

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex" : { "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } } Names not predetermined!}

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex" : { "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } } value[dimension["sex"].category.index["F"]]}

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex" : { "category" : { "id" : ["M", "F", "T"] , "label" : ["Men", "Women", "Total"] }} value[dimension["sex"].category.index["F"]]

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex" : { "category" : { "id" : ["M", "F", "T"] , "label" : ["Men", "Women", "Total"] }} for( ; ; ){ ... } value[dimension["sex"].category.index["F"]]

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex" : { "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } }}

"value": [3732196, 3807422 , 7539618]"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 1, 1, 3]"sex3" : { "attribute" : "sex" , "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } }}

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618]

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618] ↓ cryptic ↑ single model

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618] ↓ cryptic ↑ single model"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 12, 1, 3]

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618] 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 12, 1, 3]

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618] 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Male Female Total"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 12, 1, 3]

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618] 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Male Female “CSV revisited” Total"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 12, 1, 3]

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618]getDataById(ds, {"sex" : "M" , "time" : "20090101"})"id" : ["metric", "time", "geo", "sex"] ,"size" : [1, 12, 1, 3]

"value": [ 3058628, 3115336, 3201029, 3309850, 3366329, 3468235, 3543706, 3578176, 3661028, 3713765, 3724515, 3732196, 3203371, 3246029, 3305411, 3394296, 3446990, 3526971, 3590991, 3632332, 3703050, 3761655, 3787866, 3807422, 6261999, 6361365, 6506440, 6704146, 6813319, 6995206, 7134697, 7210508, 7364078, 7475420, 7512381, 7539618]"status": [ "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "p", "e", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "p", "e", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "p", "e"]

"value": [3732196, 3807422 , 7539618]"sex" : { "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } , }}

"value": [3732196, 3807422 , 7539618]"sex" : { "category" : { "index" : {"M" : 0 , "F" : 1 , "T" : 2} , "label" : { "M" : "Men" , "F" : "Women" , "T" : "Total" } , "uri" : "http://jsonstat.dataprovider.org/dimension/sex.json", }}

"value": [3732196, 3807422 , 7539618]"sex" : { "category" : { "uri" : "http://jsonstat.dataprovider.org/dimension/sex.json", }}

The JSON-stat CommunityWelcomes You!

The JSON-stat CommunityWelcomes You! Needs You!

Standards for statistical data dissemination a wish list Xavier Badosa (@badosa) Statistical Institute of Catalonia Thank you OECD Statistical Information System Collaboration Community (SIS-CC) Workshop 2012. Paris, 12-14 March

Dan Taylor borman818 / Daniel Borman Christian Cable Ian Muttoo Donald Macleod Lushbunny shaggy359Wikimedia Commons http://en.wikipedia.org/wiki/File:IBM_PC_5150.jpg http://en.wikipedia.org/wiki/File:Asimo_look_new_design.jpg Richard Cyganiak

Photos of products and webpages have been taken from the respectivecompanies’ websites. 1993 Mosaic webpage, © Board of Trustees of theUniversity of Illinois, taken from the Computer History Museum website.Superman logo (free licence) from vectortemplates.com. Authors’pictures taken from respective websites or Twitter accounts, except TimBerners-Lee taken from a video session. HTML5 logo from W3C. Image fromStanley Kubrick’s Dr. Strangelove or: How I Learned to Stop Worrying andLove the Bomb, © Columbia Pictures. Homo erectus’ drawing taken fromrecursos.cnice.mec.es (Spanish Ministry of Education & Science). Homosapiens neanderthalis’ drawing taken from vanedf11.blogspot.com.

http://xavierbadosa.com	1335
http://json-stat.org	448
http://stataccess.blogspot.com	141
http://stataccess.blogspot.se	103
http://stataccess.blogspot.ch	81
http://translate.googleusercontent.com	50
http://www.linkedin.com	40
http://ultimate	35
https://twitter.com	18
http://stataccess.blogspot.ca	15
http://stataccess.blogspot.co.uk	15
http://stataccess.blogspot.com.es	15
https://twimg0-a.akamaihd.net	14
http://stataccess.blogspot.pt	10
http://stataccess.blogspot.co.nz	9
http://stataccess.blogspot.de	9
http://stataccess.blogspot.com.au	9
http://stataccess.blogspot.fr	8
http://abtasty.com	7
http://stataccess.blogspot.com.br	7
http://stataccess.blogspot.it	7
http://stataccess.blogspot.in	6
http://stataccess.blogspot.jp	3
http://www.statsblogs.com	2
http://tweetedtimes.com	2
https://si0.twimg.com	2
http://stataccess.blogspot.mx	2
http://stataccess.blogspot.cz	1
http://stataccess.blogspot.com.ar	1
http://bottlenose.com	1
https://mercy.blackboard.com	1
http://twitter.com	1
http://a0.twimg.com	1

Standards for statistical data dissemination: a wish list

by Xavier Badosa

Accessibility

Categories

Upload Details

Usage Rights

33 Embeds 2,399

Statistics

Standards for statistical data dissemination: a wish list Presentation Transcript