Proposing a discovery process to infer the implicit schema of a set of JSON documents

1. Discovering Implicit
Schemas in JSON Data
Javier Luis Cánovas Izquierdo, Jordi Cabot
{javier.canovas,jordi.cabot}@inria.fr
ICWE Conference
July 2013

2. APIs everywhere

3. APIs everywhere

4. flickr/FortMeade
Format War

5. flickr/FortMeade
Format War
XML vs. JSON

6. 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2009 2010 2011 2012 2013
JSON-based API XML-based API
Source: ProgrammableWeb.com
XML vs. JSON

7. flickr/Eleaf

8. Also called JavaScript Object Notation, is a text-
based open standard designed for human-
readable data interchange. It is derived from the
JavaScript scripting language for representing
simple data structures and associative arrays,
called objects. Despite its relationship to
JavaScript, it is language-independent, with
parsers available for many languages.
JSON (/ˈdʒeɪsɒn/ jay-sawn, /ˈdʒeɪsən/ jay-sun)
Source: Wikipedia

9. Also called JavaScript Object Notation, is a text-
based open standard designed for human-
readable data interchange. It is derived from the
JavaScript scripting language for representing
simple data structures and associative arrays,
called objects. Despite its relationship to
JavaScript, it is language-independent, with
parsers available for many languages.
JSON (/ˈdʒeɪsɒn/ jay-sawn, /ˈdʒeɪsən/ jay-sun)
… and it is schemaless

10. flickr/jean-louis zimmermann
Example

17. flickr/Mads Boedker
Our proposal

18. Discovery of schemas in JSON

21. Creation Refinement

22. Creation Refinement
C1 C2 C3 C4 C5

23. R1 R2 R3
Creation Refinement

24. Creation Refinement

25. M1 M2 M3 M4

26. Discovering dependencies

27. Implementation

28. Not enough…
• Allowing developers to enrich the partial schemas
• Ideas from database normalization theory and XML-based approaches
Improve discovery process
• Some metrics: effectiveness, coverage, etc.
Evaluate the discovery process
• Automatize the generation of documentation
• Identifying use patterns in the JSON-based API
Promote JSON-based APIs
• Generation of mash-ups
• …others?
Additional uses

29. Check it out!
https://code.google.com/a/eclipselabs.org/p/json-discoverer/
Except where otherwise noted, content on this presentation is licensed under a Creative Commons Attribution 3.0 License.
Do you wanna try?