1. Autograff
Optimality Principles
in the Procedural Generation of Graffiti Style
Graffiti synthesis, a motion centric approach
Daniel Berio
http://doc.gold.ac.uk/autograf
@colormotor

2. Graffiti art

3. Graffiti art

4. Graffiti – canvas in motion

5. Graffiti art - tags
• Elementary “Atom” of graffiti art
• Highly stylized signature denoting an artist’s pseudonym
• Generated by rapidly executed and well learned movements
– Different style are referred to as “hand styles”
– In graffiti jargon, a well made tag has ”flow”

6. Embodied perception
• Well known relations between kinematics and geometry of human movements
(e.g. power laws) (Lacquaniti et al. 1983)
• Observation of a human made trace triggers the mental recovery of the
movement underlying its production
(Freedberg and Gallese 2007, Longcamp et al. 2006, Pignocchi 2010)
• Such recovery influences aesthetic appreciation (Leder et al. 2012)

7. Embodied perception
• Well known relations between kinematics and geometry of human movements
(e.g. power laws) (Lacquaniti et al. 1983)
• Observation of a human made trace triggers the mental recovery of the
movement underlying its production
(Freedberg and Gallese 2007, Longcamp et al. 2006, Pignocchi 2010)
• Such recovery influences aesthetic appreciation (Leder et al. 2012)
• It follows that an appropriate simulation of movement may trigger a similar
effect in the viewer for computer generated traces.

8. Graffiti Taxonomy
Evan Roth, Graffiti Taxonomy https://www.moma.org/collection/works/147174

9. Common letter structure

10. Bi-level representation of “style”

11. Movement centric curve generation
Trajectory formation model Stylized trajectoriesControl polygon / motor plan
Parameters ?

12. Data driven approach
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO
Trajectory formation model Stylized trajectoriesStructure (motor plan)

13. Graphonomics
The scientific field ”concerned with the systematic
relationships involved in the generation and analysis of the
handwriting and drawing movements, and the resulting
traces of writing and drawing instruments”
(Kao, Hoosain, & Van Galen, 1986)
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

14. Graphonomics - principles
• Aiming movements assume a “bell shaped” speed profile (Morasso, 1981)
• Handwriting movements can be decomposed into a discrete number of
aiming movement primitives (strokes)
(Teulings and Schomaker 1993, Mussa Ivaldi and Solla 2004, Sosnik et al. 2004, Plamondon et al. 2014)
– Also characterized by the same bell shaped speed profile.
– Each stroke is aimed at a virtual (imaginary) target
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

15. Kinematic Theory - Sigma Lognormal Model (Plamondon et al. 2014)
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

16. Kinematic Theory - Sigma Lognormal Model (Plamondon et al. 2014)
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

17. Kinematic Theory - Sigma Lognormal Model (Plamondon et al. 2014)
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

18. Sigma Lognormal Model
STRUCTURE
“Virtual Targets”
KINEMATICS/HANDSTYLE
“Dynamic parameters”
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

19. User input
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

20. Reconstruction
(a) (b)
(c)
virtual targets
Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

21.  Randomly perturb parameters of sigma lognormal model (e.g. +- 10%)
Data Augmentation

22. Video 1

23. Optimisation approach
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Generating Calligraphic Trajectories with Model Predictive Control, 2017, Graphics Interface
Trajectory formation model Stylized trajectoriesStructure (motor plan)
Optimization / performance criterion

24. Computational Motor Control
• Complex hand and arm motions tend to be smooth
– Minimization of a cost or performance criterion.
– Minimum Square Derivative models (Flash & Hogan 1985, Flash 1983, Dingwell et al. 2004)
• Minimization of the squared magnitude of high derivatives of position
such as jerk (3rd, change in acceleration), snap (4th, change in jerk) etc…
• Optimal feedback control - Minimal intervention principle
(Todorov & Jordan 2002)
– Deviations from an average trajectory are only corrected if they interfere with
the required task precision.
– Higher variability → reduced effort → smoother trajectory
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Generating Calligraphic Trajectories with Model Predictive Control, 2017, Graphics Interface

25. Cost function
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Generating Calligraphic Trajectories with Model Predictive Control, 2017, Graphics Interface

26. Gaussian targets
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Generating Calligraphic Trajectories with Model Predictive Control, 2017, Graphics Interface

27. Video 2
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Generating Calligraphic Trajectories with Model Predictive Control, 2017, Graphics Interface

28. Procedural generation – random covariance
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Dynamic Graffiti Stylisation with Stochastic Optimal Control, 2017, MOCO

29. Procedural generation – tied/semi-tied covariance
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Dynamic Graffiti Stylisation with Stochastic Optimal Control, 2017, MOCO

30. Video 3
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Dynamic Graffiti Stylisation with Stochastic Optimal Control, 2017, MOCO

31. Generative Glyphs
Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie
Dynamic Graffiti Stylisation with Stochastic Optimal Control, 2017, MOCO

33. The end
Daniel Berio
d.berio@gold.ac.uk - http://www.enist.org
More info:
http://doc.gold.ac.uk/autograff
Relevant references:
Berio D., Akten M., Fol Leymarie F., Grierson M., Plamondon R.
Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks
Proc. of 4th Int’l Conf. on Movement Computing (MOCO). London, UK, 2017.
Berio D., Calinon S., Fol Leymarie F.
Dynamic Graffiti Stylisation with Stochastic Optimal Control
ACM Proceedings of the 4th International Conference on Movement and Computing. London, UK, June 2017.
Berio D., Calinon S., Fol Leymarie F.
Generating Calligraphic Trajectories with Model Predictive Control
Proceedings of Graphics Interface. Edmonton, Canada: Canadian Human-Computer Communications Society, May 2017.