Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming

35 views

Published on

Presentation for GECCO 2014: https://dl.acm.org/citation.cfm?id=2598303

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming

  1. 1. E N G A G I N G T H E W O R L D Philippe Pasquier pasquier@sfu.ca 1 Matthieu Macret mmacret@sfu.ca Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming Wednesday, July 16, 2014
  2. 2. The Synthesis Calibration Problem 2 Target sound Sound Synthesizer i1 i2 ... in = Synthesized sound Input parameters Wednesday, July 16, 2014
  3. 3. Automatic calibration of Sound synthesizers 3 Synthesis  technique Op/miza/on  technique Reference Addi)ve HMM Heise  (AES  Conven)on  2009) Substra)ve Par)cle  Swarm Yoshimura  (Speech   Communica)on  and  Tech.  1999) Substra)ve Neural  Network Roth  (AES  Conven)on  2011) FM Cellular  Automata Serquera  (Applica)ons  of   Evolu)onary  Computa)on  2010) FM GA Horner  (Computer  Music  Journal   1993) Addi)ve GA Horner  (Computer  Music  Journal   1996) Granular GA Fujinaga  (ICMC  1994) Wednesday, July 16, 2014
  4. 4. ModFM synthesizer (Macret SMC 2012) 4 Wednesday, July 16, 2014
  5. 5. ModFM synthesizer (Macret SMC 2012) 4 Target Wednesday, July 16, 2014
  6. 6. ModFM synthesizer (Macret SMC 2012) 4 Target Wednesday, July 16, 2014
  7. 7. ModFM synthesizer (Macret SMC 2012) 4 Wednesday, July 16, 2014
  8. 8. ModFM synthesizer (Macret SMC 2012) 4 Synthesized Wednesday, July 16, 2014
  9. 9. ModFM synthesizer (Macret SMC 2012) 4 Synthesized Wednesday, July 16, 2014
  10. 10. More complex synthesizer • VST Instruments with GA (Yee-King ICMC 2011) 5 Wednesday, July 16, 2014
  11. 11. OP-1 Synthesizer (Macret SMC 2013) 6 Mul)-­‐objec)ve  Gene)c  Algorithm Wednesday, July 16, 2014
  12. 12. OP-1 Synthesizer (Macret SMC 2013) 6 Mul)-­‐objec)ve  Gene)c  Algorithm Target Wednesday, July 16, 2014
  13. 13. OP-1 Synthesizer (Macret SMC 2013) 6 Mul)-­‐objec)ve  Gene)c  Algorithm Target Wednesday, July 16, 2014
  14. 14. OP-1 Synthesizer (Macret SMC 2013) 6 Mul)-­‐objec)ve  Gene)c  Algorithm Wednesday, July 16, 2014
  15. 15. OP-1 Synthesizer (Macret SMC 2013) 6 Mul)-­‐objec)ve  Gene)c  Algorithm Synthesized Wednesday, July 16, 2014
  16. 16. OP-1 Synthesizer (Macret SMC 2013) 6 Mul)-­‐objec)ve  Gene)c  Algorithm Synthesized Wednesday, July 16, 2014
  17. 17. General Sound Synthesis Problem 7 Target sound Sound Synthesizer i1 i2 ... in = Synthesized sound Input parameters Wednesday, July 16, 2014
  18. 18. Genetic Programming - Garcia (DAFx 2001) 8 Produc)on  rules in  had  hoc  synthesis   system Sound Synthesizer i1 i2 ... in = Synthesized sound Target sound GP GA Wednesday, July 16, 2014
  19. 19. Coevolutionary Mixed-typed Cartesian GP • Coevolution of: –Mixed-typed Cartesian Genetic Programming •Evolve synthesizer architecture –Genetic Algorithm •Optimize Input Parameters 9 Wednesday, July 16, 2014
  20. 20. Pure Data • Graphical programming environment for audio, video, and image processing • http://puredata.info/ • Popular among musician, artist and sound designers 10 Wednesday, July 16, 2014
  21. 21. 11 Wednesday, July 16, 2014
  22. 22. 11 Wednesday, July 16, 2014
  23. 23. 11 Wednesday, July 16, 2014
  24. 24. Mixed-typed Cartesian GP 12 • Harding (GECCO 2012) • Characteristics: –Directed Acyclic Graph –Mixed-typed Wednesday, July 16, 2014
  25. 25. 13 0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1 CGP  graph Pd  Patch Wednesday, July 16, 2014
  26. 26. 14 Gene Input  1 Input  2 Extra Param. Descrip/o n Output   Type 3 control control -­‐ Division control 4 signal control -­‐ Oscillator signal Wednesday, July 16, 2014
  27. 27. 15 0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1 CGP  graph Pd  Patch Wednesday, July 16, 2014
  28. 28. Fitness function • MFCC (Mel-Frequency Cepstral Coefficients) in windows of 23 ms 16 Euclidean  distance   between  synthesized   sound  and  target   sound Wednesday, July 16, 2014
  29. 29. Fitness function • MFCC (Mel-Frequency Cepstral Coefficients) in windows of 23 ms 16 Euclidean  distance   between  synthesized   sound  and  target   sound Wednesday, July 16, 2014
  30. 30. Evaluation 17 i1 i2 ... in i1 i2 ... in i1 i2 ... in i1 i2 ... in GA  popula)on  (150  individuals) CGP  Popula)on  (5  individuals) Pd PatchPd PatchPd Patch Pd PatchPd PatchPd Patch Pd PatchPd PatchPd Patch Pd PatchPd PatchPd Patch Pd PatchPd PatchPd Patch Wednesday, July 16, 2014
  31. 31. Genetic Algorithm 18 • Binary encoding of chromosome • Binary tournament selection • 2-points crossover • Bitflip mutation Wednesday, July 16, 2014
  32. 32. Mixed-typed Cartesian GP • Integer encoding of chromosome • Evolutionary Strategy 1+4 • Mutation-only approach 19 Wednesday, July 16, 2014
  33. 33. 20 Initialize CGP /GA population Evaluate CGP / GA population Variate GA population Select best CGP individual Variate CGP population Termination criteria ?End No Yes Wednesday, July 16, 2014
  34. 34. Clarinet sound 21 Wednesday, July 16, 2014
  35. 35. Clarinet sound 21 Wednesday, July 16, 2014
  36. 36. Clarinet sound 21 Wednesday, July 16, 2014
  37. 37. Clarinet sound 21 Target Wednesday, July 16, 2014
  38. 38. Clarinet sound 21 Target Wednesday, July 16, 2014
  39. 39. Clarinet sound 21 Wednesday, July 16, 2014
  40. 40. Clarinet sound 21 Synthesized Wednesday, July 16, 2014
  41. 41. Clarinet sound 21 Synthesized Wednesday, July 16, 2014
  42. 42. Clarinet sound 22 Wednesday, July 16, 2014
  43. 43. 23 Pd  patch  used  to  generate  the  target  sound Evolved  Pd  Patch Contrived sound Wednesday, July 16, 2014
  44. 44. 23 Pd  patch  used  to  generate  the  target  sound Evolved  Pd  Patch Contrived sound Wednesday, July 16, 2014
  45. 45. 23 Pd  patch  used  to  generate  the  target  sound Evolved  Pd  Patch Contrived sound Wednesday, July 16, 2014
  46. 46. 23 Pd  patch  used  to  generate  the  target  sound Evolved  Pd  Patch Contrived sound Wednesday, July 16, 2014
  47. 47. 23 Pd  patch  used  to  generate  the  target  sound Evolved  Pd  Patch Contrived sound Wednesday, July 16, 2014
  48. 48. Stats 24 Wednesday, July 16, 2014
  49. 49. Conclusions • New approach to automate the design of sound synthesizers using Coevolutionary Mixed-typed Cartesian Programming • Synthesizers represented as reusable Pd patches 25 Wednesday, July 16, 2014
  50. 50. Future works • Our methods involves many parameters – Parameter sensitivity analysis • Extend our system to reproduce not only one target sound but a set of target sounds • Improve our fitness function – Move to a multi-objective fitness function • Quantitative/Qualitative comparison with previous similar systems 26 Wednesday, July 16, 2014

×