Normalization of nested data, a powerful concept which keeps you from pulling your hair out when multiple sources of truth and nested object hierarchies creep into your application

1. Data normalization &
memoized denormalization
Branislav Holý
1

2. Data normalization
● A process that transforms denormalized data to normalized data
2
const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
}
};

3. Data normalization
● A process that transforms denormalized data to normalized data
3
const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
}
};
const entities = {
users: {
'user-1': {
id: 'user-1',
name: 'John'
}
},
articles: {
...
}
};

4. Data normalization
● A process that transforms denormalized data to normalized data
4
const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
}
};
const entities = {
users: {
'user-1': {
id: 'user-1',
name: 'John'
}
},
articles: {
'article-1': {
id: 'article-1',
title: 'My Awesome Article',
author: 'user-1'
}
}
};

5. Data normalization
● A process that transforms denormalized data to normalized data
5
const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
},
viewers: [{
id: 'user-2',
name: 'Alice'
}, {
id: 'user-1',
name: 'John'
}]
};

6. Data normalization
● A process that transforms denormalized data to normalized data
6
const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
},
viewers: [{
id: 'user-2',
name: 'Alice'
}, {
id: 'user-1',
name: 'John'
}]
};
const entities = {
users: {
'user-1': { ... },
'user-2': { ... }
},
articles: {
'article-1': {
id: 'article-1',
title: 'My Awesome Article',
author: 'user-1',
viewers: ['user-2', 'user-1']
}
};

7. Should I use it?
● Yes… if your data are entities that have relations between each other
7

8. Advantages
● Single source of truth
8

9. const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
},
viewers: [{
id: 'user-2',
name: 'Alice'
}, {
id: 'user-1',
name: 'John'
}]
};
Advantages
● Single source of truth
○ Denormalized data
■ name in multiple places
9

10. const entities = {
users: {
'user-1': {
id: 'user-1',
name: 'John'
},
'user-2': { ... }
},
articles: {
'article-1': {
id: 'article-1',
title: 'My Awesome Article',
author: 'user-1',
viewers: ['user-2', 'user-1']
}
};
Advantages
● Single source of truth
○ Denormalized data
■ name in multiple places
○ Normalized data
■ name in a single place
1

11. Advantages
● Single source of truth
● Easy to write
○ We have to modify a single place only
11

12. Disadvantage
● Difficult to read
○ We cannot use:
normalizedArticle.user.name
12

13. Disadvantage
● Difficult to read
○ We cannot use:
normalizedArticle.user.name
○ The user field is a string!
13

14. Disadvantage
● Difficult to read
○ We cannot use:
normalizedArticle.user.name
○ The user field is a string!
○ But we can denormalize the article object!
… you’ll see later.
14

15. Normalizr
● It is a library that converts denormalized data to normalized data
● It uses schema to describe relations between entities
● https://github.com/paularmstrong/normalizr
15

16. How to normalize data?
● Create schemas for your entities
● Call normalize()
16

17. Schema
import { schema } from 'normalizr';
const user = new schema.Entity('users');
const article = new schema.Entity('articles', {
author: user,
viewers: [user]
});
17

18. Schema
import { schema } from 'normalizr';
const user = new schema.Entity('users');
const comment = new schema.Entity('comments', {
commenter: user
});
const article = new schema.Entity('articles', {
author: user,
comments: [comment]
});
18

19. normalize() function
import { normalize } from 'normalizr';
const normalizedData = normalize(denormalizedData, schema);
○ denormalizedData - nested data (API response)
○ schema - normalizr schema
19

20. 20
const denormalizedArticles = [
{
id: 'article-1',
title: 'My Awesome Article',
author: { id: 'user-1', name: 'John' },
comments: [{
id: 'comment-1',
text: 'Such a great article!',
commenter: { id: 'user-2', name: 'Alice' }
}, {
id: 'comment-2',
text: 'Thanks!',
commenter: { id: 'user-1', name: 'John' }
}]
},
{
id: 'article-2',
title: 'My Terrible Article',
author: { id: 'user-1', name: 'John' },
comments: []
}
]

21. 21
const denormalizedArticles = [
{
id: 'article-1',
title: 'My Awesome Article',
author: { id: 'user-1', name: 'John' },
comments: [{
id: 'comment-1',
text: 'Such a great article!',
commenter: { id: 'user-2', name: 'Alice' }
}, {
id: 'comment-2',
text: 'Thanks!',
commenter: { id: 'user-1', name: 'John' }
}]
},
{
id: 'article-2',
title: 'My Terrible Article',
author: { id: 'user-1', name: 'John' },
comments: []
}
]
normalize(denormalizedArticles, [article]);

22. 22
const denormalizedArticles = [
{
id: 'article-1',
title: 'My Awesome Article',
author: { id: 'user-1', name: 'John' },
comments: [{
id: 'comment-1',
text: 'Such a great article!',
commenter: { id: 'user-2', name: 'Alice' }
}, {
id: 'comment-2',
text: 'Thanks!',
commenter: { id: 'user-1', name: 'John' }
}]
},
{
id: 'article-2',
title: 'My Terrible Article',
author: { id: 'user-1', name: 'John' },
comments: []
}
]
normalize(denormalizedArticles, [article]);
{
entities: {
users: {
'user-1': { id: 'user-1', name: 'John' },
'user-2': { id: 'user-2', name: 'Alice' }
},
comments: { ... },
articles: { ... }
},
result: ['article-1', 'article-2']
}

23. 23
const denormalizedArticles = [
{
id: 'article-1',
title: 'My Awesome Article',
author: { id: 'user-1', name: 'John' },
comments: [{
id: 'comment-1',
text: 'Such a great article!',
commenter: { id: 'user-2', name: 'Alice' }
}, {
id: 'comment-2',
text: 'Thanks!',
commenter: { id: 'user-1', name: 'John' }
}]
},
{
id: 'article-2',
title: 'My Terrible Article',
author: { id: 'user-1', name: 'John' },
comments: []
}
]
normalize(denormalizedArticles, [article]);
{
entities: {
users: { ... },
comments: {
'comment-1': {
id: 'comment-1',
text: 'Such a great article!',
commenter: 'user-2'
},
'comment-2': {
id: 'comment-2',
text: 'Thanks!',
commenter: 'user-1'
}
},
articles: { ... }
},
result: ['article-1', 'article-2']
}

24. 24
const denormalizedArticles = [
{
id: 'article-1',
title: 'My Awesome Article',
author: { id: 'user-1', name: 'John' },
comments: [{
id: 'comment-1',
text: 'Such a great article!',
commenter: { id: 'user-2', name: 'Alice' }
}, {
id: 'comment-2',
text: 'Thanks!',
commenter: { id: 'user-1', name: 'John' }
}]
},
{
id: 'article-2',
title: 'My Terrible Article',
author: { id: 'user-1', name: 'John' },
comments: []
}
]
normalize(denormalizedArticles, [article]);
{
entities: {
users: { ... },
comments: { ... },
articles: {
'article-1': {
id: 'article-1',
title: 'My Awesome Article',
author: 'user-1',
comments: ['comment-1', 'comment-2']
},
'article-2': {
id: 'article-2',
title: 'My Terrible Article',
author: 'user-1',
comments: []
}
}
},
result: ['article-1', 'article-2']
}

25. How to denormalize data?
● Use the same schemas
● Call denormalize()
○ denormalize(['article-1', 'article-2'], [article], entities);
25

26. How to denormalize data?
● Use the same schemas
● Call denormalize()
○ denormalize(['article-1', 'article-2'], [article], entities);
○ It denormalizes the whole data structure
○ It does not memoize previously normalized data
26

27. How to denormalize data?
● Use the same schemas
● Call denormalize()
○ denormalize(['article-1', 'article-2'], [article], entities);
○ It denormalizes the whole data structure
○ It does not memoize previously normalized data
● Use selectors!
27

28. Reselect
● A selector is a function that gets data from a state
○ const selector = state => state.name; // a simple selector without reselect
● Selectors can compute derived data from a state
○ They keep a single source of truth
● Selectors (with Reselect) are efficient (memoized)
○ They are not recomputed unless one of their arguments is changed
● Selectors (with Reselect) are composable
○ They can be used as input to other selectors
28

29. Reselect
● The selectors created with
createSelector are memoized
● The first arguments of
createSelector are other
selectors
● The last argument is a function which
has results of the previous selectors
in its input arguments
29
import { normalize } from 'normalizr';
const appState = {
counter: 0,
user: { name: 'John' }
};
const getCounter = state => state.counter;
const getUser = state => state.user;
const getName = createSelector(
getUser,
user => user.name
);
const getUpperCaseName = createSelector(
getUserName,
userName => userName.toUpperCase()
);
const getNameWithCounter = createSelector(
getUpperCaseName,
getCounter,
(userName, num) =,> `${userName} ${num}`
);

30. Selectors for Users
30
normalize(denormalizedArticles, [article]);
{
entities: {
users: {
'user-1': { id: 'user-1', name: 'John' },
'user-2': { id: 'user-2', name: 'Alice' }
},
comments: { ... },
articles: { ... }
},
result: ['article-1', 'article-2']
}
const getUsers = entities => entities.users;

31. Selectors for Comments
31
const getUsers = entities => entities.users;
const getNormalizedComments = entities =>
entities.comments;
normalize(denormalizedArticles, [article]);
{
entities: {
users: { ... },
comments: {
'comment-1': {
id: 'comment-1',
text: 'Such a great article!',
commenter: 'user-2'
},
'comment-2': {
id: 'comment-2',
text: 'Thanks!',
commenter: 'user-1'
}
},
articles: { ... }
},
result: ['article-1', 'article-2']
}

32. Selectors for Comments
const getComments = createSelector(
getNormalizedComments,
getUsers,
(normalizedComments, users) => Object.keys(normalizedComments).reduce(
(comments, commentId) => {
const normalizedComment = normalizedComments[commentId];
const userId = normalizedComment.commenter;
return {
...comments,
[commentId]: {
...normalizedComment,
commenter: users[userId]
}
};
},
{}
)
);
32

33. Selectors for Articles
33
const getUsers = entities => entities.users;
const getNormalizedComments = entities =>
entities.comments;
const getComments = createSelector(...);
const getNormalizedArticles = entities =>
entities.articles;
normalize(denormalizedArticles, [article]);
{
entities: {
users: { ... },
comments: { ... },
articles: {
'article-1': {
id: 'article-1',
title: 'My Awesome Article',
author: 'user-1',
comments: ['comment-1', 'comment-2']
},
'article-2': {
id: 'article-2',
title: 'My Terrible Article',
author: 'user-1',
comments: []
}
}
},
result: ['article-1', 'article-2']
}

34. Selectors for Articles
const getArticles = createSelector(
getNormalizedArticles,
getComments,
getUsers,
(normalizedArticles, comments, users) => Object.keys(normalizedArticles).reduce(
(articles, articleId) => {
const normalizedArticle = normalizedArticles[articleId];
const userId = normalizedComment.commenter;
return {
...articles,
[articleId]: {
...normalizedArticle,
author: users[userId],
comments: normalizedArticle.comments.map(commentId => comments[commentId])
}
};
},
{}
));
34

35. Selectors
35
const getUsers = entities => entities.users;
const getNormalizedComments = entities =>
entities.comments;
const getComments = createSelector(...);
const getNormalizedArticles = entities =>
entities.articles;
const getArticles = createSelector(...);
const normalizedData = {
entities: {
users: { ... },
comments: { ... },
articles: { ... }
},
result: ['article-1', 'article-2']
};
const denormalizedArticles = getArticles(
normalizedData.entities
);
const denormalizedData = normalizedData.result
.map(articleId => denormalizedArticles[articleId]);

36. 36
Thank you!
Don’t forget to use…

37. 37
Thank you!
Don’t forget to use…

38. 38
Thank you!
Any questions?

39. 39

40. 40

41. Advanced normalizr features
41
● Custom ID attribute
● Process strategy
● Merge strategy

42. Custom ID attribute
42
● User object: { userId: 'user-1', name: 'John' }

43. Custom ID attribute
43
● User object: { userId: 'user-1', name: 'John' }
● Schema:
const user = new schema.Entity('users', {}, {
idAttribute: 'userId'
});

44. Custom ID attribute for M:N
44
[
{
id: 'user-1',
name: 'John',
skills: [
{ level: 10, skill: { id: 'skill-1', name: 'JavaScript' } },
{ level: 1, skill: { id: 'skill-2', name: 'C++' } }
]
},
{
id: 'user-2',
name: 'Alice',
skills: [
{ level: 8, skill: { id: 'skill-1', name: 'JavaScript' } }
]
}
]

45. Custom ID attribute for M:N
45
const skill = new schema.Entity('skills');
const userSkill = new schema.Entity(
'userSkills',
{ skill },
{
idAttribute: (value, parent) => `${value.skill.id}-${parent.id}`
}
);
const user = new schema.Entity('users', {
skills: [userSkill]
});

46. Custom ID attribute for M:N
46
{
skills: {
'skill-1': {
id: 'skill-1',
name: 'JavaScript'
},
'skill-2': {
id: 'skill-2',
name: 'C++'
}
},
userSkills: { ... },
users: { ... }
},
result: ['user-1', 'user-2']
}

47. Custom ID attribute for M:N
47
{
skills: { ... },
userSkills: {
'skill-1-user-1': { level: 10, skill: 'skill-1' },
'skill-2-user-1': { level: 1, skill: 'skill-2' },
'skill-1-user-2': { level: 8, skill: 'skill-1' }
},
users: { ... }
},
result: ['user-1', 'user-2']
}

48. Custom ID attribute for M:N
48
{
skills: { ... },
userSkills: { ... },
users: {
'user-1': {
id: 'user-1',
name: 'John',
skills: ['skill-1-user-1', 'skill-2-user-1']
},
'user-2': {
id: 'user-2',
name: 'Alice',
skills: ['skill-1-user-2']
}
}},
result: ['user-1', 'user-2']
}

49. Process strategy
49
● Pre-process entities before the normalization
● Modify the shape of entities
● Add additional data

50. Process strategy
50
[
{
id: 'user-1',
name: 'John',
skillLevels: [10, 1],
skills: [
{ id: 'skill-1', name: 'JavaScript' },
{ id: 'skill-2', name: 'C++' }
]
},
{
id: 'user-2',
name: 'Alice',
skillLevels: [8],
skills: [{ id: 'skill-1', name: 'JavaScript' }]
}
]

51. Process strategy
51
const skill = new schema.Entity('skills');
const userSkill = new schema.Entity('userSkills', { skill }, { idAttribute });
const user = new schema.Entity(
'users',
{ skills: [userSkill] },
{
processStrategy: ({ id, name, skillLevels, skills }) => ({
id,
name,
skills: skills.map((skill, index) => ({
level: skillLevels[index],
skill
}))
})
}
);

52. Process strategy
52
{
skills: { ... },
userSkills: {
'skill-1-user-1': { level: 10, skill: 'skill-1' },
'skill-2-user-1': { level: 1, skill: 'skill-2' },
'skill-1-user-2': { level: 8, skill: 'skill-1' }
},
users: { ... }
},
result: ['user-1', 'user-2']
}

53. Merge strategy
53
● Merge entities with the same IDs

54. Merge strategy
54
{
skills: {
'skill-1': {
id: 'skill-1',
name: 'JavaScript'
},
'skill-2': {
id: 'skill-2',
name: 'C++'
}
},
userSkills: { ... },
users: { ... }
},
result: ['user-1', 'user-2']
}
We want user IDs here.
We want user IDs here.

55. Merge strategy
55
const user = new schema.Entity(
'users',
{ skills: [userSkill] },
{
processStrategy: ({ id, name, skillLevels, skills }) => ({
id,
name,
skills: skills.map((skill, index) => ({
level: skillLevels[index],
skill: {
...skill,
users: [id]
}
}))
})
}
);

56. Merge strategy
56
const skill = new schema.Entity(
'skills',
{},
{
mergeStrategy: (entityA, entityB) => ({
...entityA,
...entityB,
users: [...entityA.users, ...entityB.users]
})
}
);

57. Merge strategy
57
{
entities: {
skills: {
'skill-1': {
id: 'skill-1',
name: 'JavaScript',
users: ['user-1', 'user-2']
},
'skill-2': {
id: 'skill-2',
name: 'C++',
users: ['user-1']
}
},
userSkills: { ... },
users: { ... }
},
result: ['user-1', 'user-2']
}

58. 58
Thank you!
Don’t forget to use…

59. 59
Thank you!
Don’t forget to use…

60. 60
Thank you!
Any questions?