The document discusses LM Datasets, which provide a generic representation for hierarchical data using a structure (S) and record (R) to define the relationships and properties. This avoids the need for custom code between different services, instead allowing common operations like checking if an ID exists or getting the parent node. The system also supports templates, views, and normalization to reduce duplicate data storage. Key advantages highlighted are keeping the data representation open and generic.

1. LM Datasets
Promote data and code sharing
on the web
David Semeria Ruby Social Club Milano
david@lmframework.com 16th December 2010
@hymanroth

2. Objects
Properties (data)
Methods (code)
Interface
LM Datasets 2

3. Objects
Properties (data)
Methods (code) Functional abstraction (GOOD)
Interface
LM Datasets 3

4. Objects
Properties (data) Data abstraction (BAD)
Methods (code) Functional abstraction (GOOD)
Interface
LM Datasets 4

5. Objects
Properties (data) Data abstraction (BAD)
Methods (code) Functional abstraction (GOOD)
Interface
Context: web services
Interoperability is key
LM Datasets 5

6. Interoperability
Browser
Twitter Facebook
Flickr Bit.ly
LM Datasets 6

7. Interoperability
Browser
Twitter Facebook
Flickr Bit.ly
LM Datasets 7

8. How Much Glue Code?
Twitter Facebook Facebook Twitter
Twitter Flickr Facebook Flickr
Twitter Bit.ly Facebook Bit.ly
Flickr Twitter Bit.ly Twitter
Flickr Facebook Bit.ly Facebook
Flickr Bit.ly Bitl.ly Flickr
12 sets of code
N2 - N
LM Datasets 8

9. The General Case
Browser
Service A Service B
Choose from N options Choose from N options
LM Datasets 9

10. The General Case
Browser
Service A Service B
Choose from N options Choose from N options
For N = 100 N2 – N = 99,900
LM Datasets 10

11. The Problem
APIs are better than nothing, but they
remain a major impediment to a fully
writable Web.
(The same applies to corporate intranets)
LM Datasets 11

12. Datasets
A generic Global data definitions
representation for
hierarchical data
Permissions
LIBRARY
( Front and back end )
Key word: GENERIC
LM Datasets 12

13. Hierarchical Structures
root
node
node node
leaf leaf
node node
leaf leaf leaf leaf
LM Datasets 13

14. A 'people' tree
root
people
sport music
Id: bowie Id: clapton
name: “David Bowie” name: “Eric Clapton”
soccer formula1
Id: maldini Id: gerrard Id: alonso Id: hamilton
name: “Paolo Maldini” name: “Steven Gerrard” name: “Fernando Alonso” name: “Lewis Hamilton”
LM Datasets 14

15. Generic Representation
S root node 1
node 2
node 1 leaf 1
leaf 2
node 2
R node 1 record
node 2 record
leaf 1 record
leaf 2 record
LM Datasets 15

16. JSON Example
ds: { s: { root: { people: 1 },
people: { music: 1, sport: 1 },
sport: { soccer: 1, forumla1: 1 },
music: { bowie: 1, clapton: 1 },
soccer: { maldini: 1, gerrard: 1 },
formula1: { alonso: 1, hamilton: 1 }
},
r: { people: { name: “People”, color: “green” },
music: { name: “Music” color: “black” },
sport: { name: “Sport” color: “white” },
soccer: { name: “Soccer”, color “red” },
formula1: { name: “Formula One”, color: “yellow” },
bowie: { name: “David Bowie”, color: “black” },
clapton: { name: “Eric Clapton”, color: “black” },
Maldini: { name: “Paolo Maldini”, color: “red” },
Gerrard: { name: “Steven Gerrard”, color: “red” },
Alonso: { name: “Fernando Alonso”, color: “red” },
Hamilton: { name: “Lewis Hamilton”, color: “silver” }
}
};
LM Datasets 16

17. Some Code Examples
➔ Leverage structure
➔ No need for recursive tree walking
➔ Leverage native operations
➔ Object property look-up much faster than array iteration.
LM Datasets 17

18. ID Exists ?
function IdExists (id){
return ds.r[id] != null;
}
LM Datasets 18

19. Node or Leaf ?
function nodeOrLeaf (id){
return (ds.s[id]) ?'node' :'leaf';
}
// assumes id exists
LM Datasets 19

20. Node contains id ?
function contains (nodeId, id){
if (ds.s[nodeId][id]){
return true;
}
return false
}
// assumes nodeId exists
LM Datasets 20

21. Parent Node
function parentNode (id){
for ( var k in ds.s ){
if (ds.s[k][id]){
return k;
}
}
//error
}
LM Datasets 21

22. Move Item
function move ( toNodeId, id ){
delete( ds.s[parenNode(id)][id] );
ds.s[toNodeId][id] = 1;
}
// assumes all ids exist
LM Datasets 22

23. Templates
DATASET
FLOW
+ HTML
TEMPLATES
LM Datasets 23

24. NODE TEMPLATE:
Flowing Templates
<DIV style = “border: 2px solid {color}; padding: 10px”></DIV>
LEAF TEMPLATE:
<P><SPAN style = “color:{color}”>{name}</SPAN></P>
LM Datasets 24

25. Flowing Templates
NODE TEMPLATE:
<DIV style = “border: 2px solid {color}; padding: 10px”></DIV>
LEAF TEMPLATE:
<P><SPAN style = “color:{color}”>{name}</SPAN></P>
OUTPUT:
David Bowie
Eric Clapton
Paolo Maldini
Steven Gerrard
Fernando Alonso
Lewis Hamilton
LM Datasets 25

26. Demo 1
LM Datasets 26

27. Data Definitions
EXAMPLE DEFINITION
Name Age
type string type integer
minLen 1 minVal 0
maxLen 50 maxVal 150
canBeNumeric false
regex (w| )*
function checkName
LM Datasets 27

28. Inheritance
PEOPLE PLACES THINGS ......
BASIC INFO
DETAILED INFO EMAIL INFO
DETAILED & EMAIL INFO
LM Datasets 28

29. Inheritance Across Root Types
PEOPLE SERVICE
BASIC INFO TWITTER
DETAILED INFO TWITTER INFO
TWITTER USER is a sub-type of both:
SERVICE / TWITTER / TWITTER INFO
TWITTER USER
PEOPLE / BASIC INFO
LM Datasets 29

30. Inheritance
Demo 2
LM Datasets 30

31. Normalization
Just like in the relational model, Dataset
normalization means we don't store the
same information twice....
LM Datasets 31

32. Viewsets and Recordsets
VIEWSET A VIEWSET B
refs
RECORD SET 1 sparse RECORD SET 2
SERVER
LM Datasets 32

33. Demo 3
windows
LIVERPOOL MILAN #1 MILAN #2 DREAM TEAM
view sets
VS - LIVERPOOL VS - MILAN VS – DREAM TEAM
RECORD SET FOOTBALLERS
SERVER
LM Datasets 33

34. Demo 3
windows
LIVERPOOL MILAN #1 MILAN #2 DREAM TEAM
view sets
VS - LIVERPOOL VS - MILAN VS – DREAM TEAM
RECORD SET FOOTBALLERS
SERVER
LM Datasets 34

35. Demo 3
windows
LIVERPOOL MILAN #1 MILAN #2 DREAM TEAM
view sets
VS - LIVERPOOL VS - MILAN VS – DREAM TEAM
RECORD SET FOOTBALLERS
SERVER
LM Datasets 35

36. Demo 3
windows
LIVERPOOL MILAN #1 MILAN #2 DREAM TEAM
view sets
VS - LIVERPOOL VS - MILAN VS – DREAM TEAM
RECORD SET FOOTBALLERS
SERVER
LM Datasets 36

37. Demo 3
windows
LIVERPOOL MILAN #1 MILAN #2 DREAM TEAM
view sets
VS - LIVERPOOL VS - MILAN VS – DREAM TEAM
RECORD SET FOOTBALLERS
SERVER
LM Datasets 37

38. Demo 3
windows
LIVERPOOL MILAN #1 MILAN #2 DREAM TEAM
view sets
VS - LIVERPOOL VS - MILAN VS – DREAM TEAM
RECORD SET FOOTBALLERS
SERVER
LM Datasets 38

39. Summary
➔ Don't hide your data in objects
LM Datasets 39

40. Summary
➔ Don't hide your data in objects
➔ APIs can be an obstacle (representation)
LM Datasets 40

41. Summary
➔ Don't hide your data in objects
➔ APIs can be an obstacle (representation)
➔ Above all, KEEP IT GENERIC !!
LM Datasets 41

42. Summary
➔ Don't hide your data in objects
➔ APIs can be an obstacle (representation)
➔ Above all, KEEP IT GENERIC !!
Questions are welcome:
david@lmframework.com
@hymanroth
LM Datasets 42