The world is inundated with data, and we rely on our visual cortex to interpret and extract useful information. The other four senses, especially taste, smell, and touch, are under-utilised and often denied access to the wealth of the information age. Founder Jack Zhao of Australia’s leading data visualisation studio Small Multiples, and Jen Lo, Creative Director of artisan chocolatier Bakedown Cakery, will explore the use of multi-sensory mediums as a means of communicating complex data. They will be providing examples from ‘Not a Single Origin’, a design project that challenged the status quo, by transforming population census data to delicious chocolates. ‘Not a Single Origin’ aimed to promote public awareness around the demographic diversity in Sydney. Combining code and confectionery, Small Multiples and Bakedown Cakery used experimental 3D printing techniques to embed population data and maps of each suburb on their respective chocolate, literally bringing the streets to your taste buds. Jack Zhao and Jen Lo will also discuss what’s next in their sensory exploration quest, with interactive and delicious surprises as the team begins to experiment with human speech.

1. MAKING DATA DELICIOUS
Jack Zhao and Jen Lo

2. ●
●
●
The world is inundated
with data
Network, Teamwork (2002), Uta Eisenreich. Which 3 kids would you invite to your birthday party? http://www.hier-eisenreich.org/

3. We primarily rely on our vision to make sense
of data.
Distellamap by Ben Fry, https://benfry.com/distellamap/

4. HYPER-REALITY https://vimeo.com/166807261
Our vision is overloaded.

5. We want to make data delicious

6. data visualiser chocolatier

7. What we’re going to share

8. We want to communicate data
through all human senses.

9. Why chocolate?

10. https://www.bakedowncakery.com/shop/til-death-do-us-part-heart
Because chocolate is delicious

11. https://www.bakedowncakery.com/
Chocolate is beautiful

12. https://www.bakedowncakery.com
Chocolate is expressive.

13. https://thenewdaily.com.au/entertainment/tv/2016/08/25/the-bachelor-chocolate-bath/
Chocolate is the feels.

14. Jen Lo Jack Zhao
Harry Morris
Dasha Yurovskaya
Robert Yuan Anna Nordon

15. So we made some chocolate

16. then we made some more...

17. Motivations

18. Design innovations taking place across data
visualisation, physical interactions with data,
and artisan chocolate making.

19. Each chocolate is directly named after
Japanese expressions used to describe texture.
Chocolate has become
an edible canvas for
artistic expression
Motivation
http://www.nendo.jp/en/works/chocolatexture-2/

20. Digital fabrication
Motivation
https://ultimaker.com/

21. The Mayku FormBox, powered by any
vacuum cleaner, it works with a world of
materials and helps the ideas real.
Whether crafting by hand or partnering
with a 3D printer, the FormBox functions
as a desktop production line.
Democratising
manufacturing
Motivation
https://www.mayku.me

22. Data visualisation is
becoming more
accessible
Motivation
http://deck.gl/showcases/wind/
Powerful tools are becoming available on web browsers
for experts and laymen alike

23. Data Physicalisation
Motivation
https://www.media.mit.edu/projects/making-data-matter/overview
Data physicalization of the human brain, visualizing bundles
of axons, which connect different regions of the brain

24. Tactile interactions
with data
Motivation
http://www.jan-barth.de/fwp_portfolio/databot
Feeling the size of the files you’re trying to move

25. Physical interactions
with data
Motivation
https://developer.apple.com/design/human-interface-guidelines/ios/user-interaction/feedback
Haptic feedback in everyday devices

26. Design goals

27. 1. To challenge the tyranny of the pixel*
Moere, Andrew Vande. "Beyond the tyranny of the pixel: Exploring the physicality of information visualization." Information Visualisation, 2008. IV'08.
12th International Conference. IEEE, 2008.

28. "we see with our
brains not our eyes.”
To challenge the tyranny of the pixel
https://www.nature.com/articles/221963a0
Paul Bach-y-Rita hypothesized in the 1960s

29. See with your tongue
http://vision.wicab.com/index.php
http://hci.ucsd.edu/234/234ExtraReading/BachYRitaKercel2003SensorySubstitution.pdf
To challenge the tyranny of pixel
Wicab BrainPort translates information from a wearable
camera into electrical simulation patterns on the tongue,
helping vision impaired people to ‘see’.

30. Ammassalik
wooden maps
These maps can be used inside mittens, float if
they fall in the water
https://www.atlasobscura.com/articles/greenland-wooden-maps-ammassalik

31. 2. To use food as a carrier of information

32. Food carries our identities
Food as a carrier of information
http://menzelphoto.com/
What the World Eats by Peter Menzel

33. Food carries
memories
https://tangible.media.mit.edu/project/transformative-appetite/
Food as a carrier of information
Transformative Appetite - MIT Media Lab

34. Food tells stories
https://www.facebook.com/PollutedWaterPopsicles
Food as a carrier of information

35. Gastronomification of
data
https://thomaslevine.com/data-music/beyond-visualization
Food as a carrier of information
Thomas Levine

36. Design Process: Inspirations

37. Edible data
Egguality - Wiebke Jansen
Inspiration
http://data-cuisine.net/data-dishes/egguality/

38. Drinkable data
Multivariate Beer - Nathan Yau
Inspiration
https://flowingdata.com/2015/05/20/brewing-multivariate-beer/

39. https://www.meshu.io/
Wearable data
Encode Ring
Inspiration
https://encodering.com
Meshu

40. Data materialisation
Barack Obama: Next Industrial Revolution
Inspiration

41. Not a Single Origin:
the design process

42. https://notasingleorigin.com/
Not a Single Origin
Immigrants come from all over the world to settle
in Sydney. We celebrate this cultural diversity and
people’s connections with their new land.
Materialising cultural
identity

43. The Data
Top 3 regions: Europe, Asia, and Middle East
12 ancestries
Not a Single Origin

44. The Data
12 suburbs
12 chocolate pieces
Not a Single Origin

45. Matching ancestries with flavours

46. Matching ancestries with flavours

47. Not a Single Origin
The UX Design of
Chocolate

48. Ancestry as flavour
12 suburbs
12 chocolate pieces
Not a Single Origin

49. Location as form
Not a Single Origin

50. Prototyping: transfer sheets
Not a Single Origin

51. Prototyping: 3D printing
Not a Single Origin

52. Custom mould
Not a Single Origin

53. Test run
Not a Single Origin

54. The new world of small batch
manufacturing
Not a Single Origin

55. Custom mould complete!
Not a Single Origin

56. Prototyping: packaging
Not a Single Origin

57. Packaging
Not a Single Origin

58. Packaging
Not a Single Origin

59. Packaging
Not a Single Origin

60. The final product
Not a Single Origin

61. Shape of My Words

62. Materialising speech
Shape of My Words
What if we can eat each other’s words.
Sending information from mouth to mouth (M2M).

63. Speech:
voice
Shape of My Words
Mel-frequency cepstrum (MFCC)
analysis to confirm we can reliably
visualise voice

64. Speech:
emotion
Shape of My Words
clmtrackr tracks a face and
outputs the coordinate positions
of the face model.

65. Voice as shapes
Shape of My Words

66. Prototyping: voice to shape
Shape of My Words

67. Wave Interference Patterns
Shape of My Words
Melania Trump
“want”
Michelle Obama
“want”

68. Shape of My Words
Making:
voice to shape

69. Congratulations
Memories
You

70. Heteronyms | Same spelling & different
pronunciation
BowLead
Homographs | Same spelling & sound, different meaning
AddressCurrent
Homophones | Sound alike, different meaning &
spelling
Advice,AdviseRight,Write

71. Birthday Condolences You
AngerHappinessDisgust
DisgustFearSadness
AngerSadSurprise

72. Shape of My Words
Making:
emotion to flavour
Anger
Disgust
Fear
Sadness
Surprise
Happiness
Bitterness
Saltiness
Spiciness
Fruitiness

73. Shape of My Words
Vacuum moulds

74. Shape of My Words
Progress so far

75. What we’ve learnt

76. Edible media
make design more
accessible
● More people love to eat than looking at charts
● More sensory input modes = more bandwidth for information to
flow
● Food predates and will outlast pixels

77. Role of the
designer is
changing
The ‘product’ in the future might become
just a kind of 3D template or even more
something along the lines of a generator to
start a parametric design process to
incorporate the differs requirements of
consumers - Benedikt Groß
https://benedikt-gross.de/

78. Q&A
Small Multiples / Jack Zhao
https://small.mu
smallmultiples
Bakedown Cakery / Jen Lo
https://bakedowncakery.com
bakedowncakery
Not a Single Origin
https://notasingleorigin.com