This document summarizes a talk on computational creativity given by Dr. Cole D. Ingraham. It discusses top-down and bottom-up approaches to computational creativity. Top-down approaches involve defining the structure of a program, while bottom-up approaches use machine learning and large datasets. The talk compares these approaches and provides examples like Google's Wavenet model. It also summarizes Dr. Ingraham's work on the Amper Music platform, which uses a hybrid approach combining top-down structure with bottom-up machine learning to generate personalized music.

1. NYAI #7 (SPEAKER SERIES):
Data Science to Operationalize
Machine Learning (Matthew Russell)
& Computational Creativity
(Dr. Cole D. Ingraham DMA)

2. Top-down vs. Bottom-up
Computational Creativity
Dr. Cole D. Ingraham
NYAI Nov. 22, 2016

3. About Me
• originally from Santa Clara, CA
• DMA Music Composition
• currently working as lead developer at
Amper Music (www.ampermusic.com)
• www.coleingraham.com

4. Abstract
• what does computational creativity mean, and what are some approaches?
• what are some examples of these approaches?
• how to know the best way a particular problem could be solved?
• primarily concerned with choosing an appropriate approach, less about
implementation

5. Defining Creativity
• Creativity: the use of the imagination or original ideas, especially in the production of an
artistic work.
• Artistic: having or revealing natural creative skill; aesthetically pleasing
• Aesthetic: a set of principles underlying and guiding the work of a particular artist or
artistic movement
• very circular, difficult to concretely define

6. Defining Creativity
• my personal requirements:
• originality/novelty
• having some unified set of guiding principals

7. Top-Down Approach
• finding an answer that:
• is novel and useful (either for the individual or for society)
• demands that we reject ideas we had previously accepted
• results from intense motivation and persistence
• comes from clarifying a problem that was originally vague
https://en.wikipedia.org/wiki/Computational_creativity#Defining_creativity_in_computational_terms

8. Bottom-Up Approach
• artificial neural networks
• machine learning
• data, data, and more data
https://en.wikipedia.org/wiki/Computational_creativity#Defining_creativity_in_computational_terms

9. Comparing Approaches
• Top-Down
• primarily code driven
• dependent on defining structure
• requires considerable development time to be effective
• very “hands on” (things only improve as the code base improves)
• Bottom-Up
• primarily data driven
• heavy use of statistical analysis
• requires considerable amounts of data to be effective

10. Comparing Approaches
• Top-Down
• you define the structure of your program
• Bottom-Up
• you learn the structure of your program from analyzing data

11. GenerationVs.Analysis
• Generation
• create something that does not exist
• novel output
• example use: music composition
• Analysis
• extract information from something that exists
• generalization of the input

12. Generation & Analysis
• the creation of something new can/should be informed by analysis of previous work
• in order to analyze previous work, the work must already exist
• ... chicken and egg

13. Generation by Analysis
• analysis produces generalizations/averages of all input
• this can lead to largely homogenous output

14. Generation,Then Analysis
• with some target in mind, generate output, then analyze it and see how close you got
• heuristic “guess and check” with no generation-time feedback

15. Generation, No Analysis
• create output without analyzing anything before or after
• you must know what you are going to get
• ... or you must not care what you are going to get

16. How to know what approach use?

17. It depends!

18. Generation / Analysis
• examples:
• compile-time code optimization: generate, analyze, repeat n times
• absolute realism: analyze, then generate (conceptually)
• generative abstract art: generate, do not analyze (sometimes)

19. Novelty
• introducing variance into the programs output
• important considerations:
• how much variance?
• how to determine what can be varied?
• how much/little control over the variance you want/need?

20. How MuchVariance?
• largely subjective and project dependent
• beware of probability-driven decisions:
• probabilities say how frequently something happens
• probabilities do not inherently say why something should happen

21. What Can BeVaried?
• largely subjective and project dependent
• example - text generation:
• synonyms can vary a sentence without fundamentally changing its meaning
• sentence structure and grammar are far less flexible

22. How to ControlVariance?
• largely subjective and project dependent (sensing a theme?)
• some options:
• explicitly defined seeding and propagation of pseudo-randomness
• working with finite quantities of input parameters

23. A Note on Randomness
• nondeterministic code is a nightmare to QA
• interaction between many deterministic operations can create seemingly random
outcome
• (my opinion) it’s a better use of time to work on deterministically defined variance
than it is to weed out undesired results from nondeterministic code

24. In the News

25. Google
• neural network based approach (TensorFlow)
• uses a large training set
• limited by availability of data (not a problem for Google)

26. Wavenet
(Google DeepMind)
• input: raw audio
• output: raw audio
• aimed at improving speech synthesis (very good at this)
• also used for music generation (less good at this)

27. Wavenet
“Since WaveNets can be used to model any audio signal, we thought it
would also be fun to try to generate music. Unlike the TTS experiments, we
didn’t condition the networks on an input sequence telling it what to play
(such as a musical score); instead, we simply let it generate whatever it
wanted to. When we trained it on a dataset of classical piano music, it
produced fascinating samples [...]”
https://deepmind.com/blog/wavenet-generative-model-raw-audio/

28. Wavenet
• “trained it on a dataset of classical piano music”
• single instrument, single genre
• “we didn’t condition the networks on an input sequence telling it what to play
(such as a musical score)”
• no large scale structural awareness
• “let it generate whatever it wanted to”
• no external input

29. ...
what about top-down news?

30. Top-Down News?
• none that I’m aware of
• not a buzzword like neural networks
• (my opinion): everyone uses a top-down approach to some degree, they just don’t
feel the need to talk about it

31. Why am I talking about it then?
• to emphasize that:
• AI is more than just data science
• defining the structure of your problem is integral to finding its solution
• there are many more valid approaches to creative AI than what gets all the
attention
• one should use the right tool for the right job
• some balance between top-down and bottom-up is often the right choice

33. Project Example:Amper Music
• Goal: create an AI music composition platform that lets users create personalized,
professional quality music instantly with no experience required.
• Requirements:
• speed: it must be fast!
• quality: it must be believable!
• control: it must be collaborative!

34. Requirement: Speed
• neural networks are s l o w
• hand tuned algorithms can be fast
• lookup operations are very fast

35. Requirement: Quality
• neural networks can be very accurate, given a training set of sufficient size and quality
• most musical training (theory, performance practice) can be defined in code
• we:
• don’t (currently) have access to an appropriate training set
• do have a team of developers who are also highly trained professional musicians

36. Requirement: Control
• neural networks are “black boxes that just do what they do”
• defining music as a hierarchical structure offers handles to various aspects directly
• multiple levels of control:
• intuitive enough for the non-musician to use effectively
• powerful enough for the professional musician to use without feeling limited

37. Our Solution
• define various levels of musical structure in a declarative manner
• use machine learning / neural networks to generate data for all defined structural
levels offline, as data becomes available, that can be used to augment runtime
decisions
• structure the program in a way that can be easily scaled with machine learning, but
works without it
• keep the “black box” parts of the program as minimal and confined as possible

38. Tooling
• runtime: Haskell
• offline: Haskell, Python, anything else (it’s offline!)

39. Tooling• Why Haskell?
• compiled (fast)
• functional language (great for music and AI, like LISP)
• statically typed (safe, easy to maintain and refactor)
• Why Python?
• availability of libraries (TensorFlow as one example)
• scripting layer on top of C(++) (easier to use, still fast)

40. Tooling• Why Haskell?
• compiled to machine code (fast)
• functional language
• statically typed (safe, easy to maintain and refactor)
• Why not Python (for runtime)?
• interpreted / compiled to bytecode
• not a functional language (I wanted to use a functional language)

41. Thank you

42. Social Event:
THE STOREHOUSE
69 West 23rd
, New York,
NY 10010 (2nd
Floor)