Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The Art, Joy, & Power of
Creating Musical Programs
David Koelle @dmkoelle
#JFugue
play(“C”);
Musically-Inclined Programmers!
Programming-Inclined Musicians!
Anyone who wants to infuse fun
into programming!
Who is JF...
import org.jfugue.player.Player;
public class HelloWorld {
public static void main(String[] args) {
Player player = new Pl...
import org.jfugue.player.Player;
public class HelloWorld {
public static void main(String[] args) {
Player player = new Pl...
Sequencer sequencer = MidiSystem.getSequencer();
Sequence sequence = sequencer.getSequence();
Track track = sequence.creat...
public class TwelveBarBlues {
public static void main(String[] args) {
Pattern pattern = new ChordProgression(“I IV V”)
.d...
Real-World Examples
Log Files Classic Novels
Log Files Classic Novels
Brian Tarbox
“I want to know
the status of
my video
servers without
staring at a
screen.”
time
video server log files
time
video server log files
o server log files
video stream is created
video stream has failed
video stream is tricked
video stream is destroyed
video content is added
S...
video stream is created
video stream has failed
video stream is tricked
video stream is destroyed
video content is added
S...
video stream is created
video stream is destroyed
video stream has failed
[BASS_DRUM]s
[ACOUSTIC_SNARE]s
[CRASH_CYMBAL_1]s...
Listening to Video on Demand Servers
Learn more at
Log4JFugue.org
Winner,
2010 Duke’s
Choice Award
Integrates with
Spring Framework
Scala version
now available
Log Files Classic Novels
Hannah Davis
“I want to
create a new
interactive
experience to
perceive classic
novels.”
TransProse creates music
from literature
Songs are based on the emotional
content of a novel
Introducing TransProse
unique word prevalence
average sentence length
book length
punctuation
presence of main characters
descriptive words
chapt...
unique word prevalence
average sentence length
book length
punctuation
presence of main characters
descriptive words
chapt...
Emolex – 14K tagged words
The Art of Making the Music
Stable Tones
Emotion Counts
Musical Considerations
Melodic Shape
minor/major key
tempo
note duration
harmony
pitch
changing between keys
different instrument voices
rhythm
time signature
...
minor/major key
tempo
note duration
harmony
pitch
changing between keys
different instrument voices
rhythm
time signature
...
amount of emotion
low emotion counts
high emotion counts
spikes in emotion
active:passive emotions
positive:negative emoti...
Learn more at
MusicFromText.com
TIME
Discovery
Wired
io9
NBC
Engadget
Fast Company
Popular Science
CNET
Der Spiegel
Le Mon...
More Real-World Examples
Interactive Experiences & Games
Algorithmic/Generative Music
Music Based on Data
Education
Art & Design
Software Support f...
SoundOfShopping.com
Turning barcodes into music
“I'm analyzing microblogs like YikYak,
Tumblr and Twitter to get trending
topics. I'm trying to translate text to
quantita...
Making Music from a Chess Match
Soundtrack for Astrophotography Show
Based on 6000 naked-eye stars:
declination (latitude)  pitch
right ascension (longit...
“Since JFugue can turn MIDI into text,
algorithmically created music can be
made using the same technology your
phone uses...
A Tour of the Magic of JFugue
Easy to Use
Based on Music Theory
Fun Things are Baked In
Consistent “System”
Event-Based Architecture
Five Bits of JFugue...
Play music with two lines of code:
Player player = new Player();
player.play(“C”);
Ease of Use
Music is easy to specify
and intentionally human-readable
C Middle-C
C#6h C# note, 6th octave, half duration
EbMAJ^w E-fla...
Music is easy to specify
and intentionally human-readable
I[Flute] Change instrument to Flute
T[Allegro] Change tempo to A...
Pattern melodyVerseOne = new Pattern("T115 V0
I[Synth_Choir]")
.add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi")
....
public class TwelveBarBlues {
public static void main(String[] args) {
Pattern pattern = new ChordProgression(“I IV V”)
.d...
Define a Scale in terms of Intervals
Define a Chord in terms of Intervals
Create a “I IV V” Chord Progression
Set the Root...
Rhythm rhythm = new Rhythm()
.addLayer("O..oO...O..oOO..")
.addLayer("..S...S...S...S.")
.addLayer("````````````````")
.ad...
Pattern prog = ChordProgression
.fromChords("C#4min E4maj B3maj A3maj")
.eachChordAs("$_i $_i Ri $_i")
.getPattern()
.setI...
MidiParser parser = new MidiParser();
LilyPondListener listener =
new LilyPondListener();
parser.addListener(listener);
pa...
StaccatoParser parser = new StaccatoParser();
LilyPondListener listener =
new LilyPondListener();
parser.addListener(liste...
StaccatoParser parser = new StaccatoParser();
MusicXmlListener listener =
new MusicXmlListener();
parser.addListener(liste...
MidiParser parser = new MidiParser();
MyParserListener listener = new MyParserListener();
parser.addParserListener(listene...
Sending to / Receiving from MIDI Devices
Creating New Patterns in Real-Time
Sending Musical Events “Before” They Occur
Ext...
Making Music from DNA
Making Music from DNA
Many people have done this many ways…
http://www.mimg.ucla.edu/faculty/miller_jh/gene2music/
previou...
Download JFugue
Make beautiful and creative things!
http://www.jfugue.org
Optionally:
Contribute to JFugue!
Live coding, T...
Thank You!
David Koelle @dmkoelle
#JFugue
The Art, Joy, and Power of Creating Musical Programs (JFugue at SXSW Interactive 2015)
The Art, Joy, and Power of Creating Musical Programs (JFugue at SXSW Interactive 2015)
Upcoming SlideShare
Loading in …5
×

The Art, Joy, and Power of Creating Musical Programs (JFugue at SXSW Interactive 2015)

1,804 views

Published on

Describes how the ability to easily create musical programs can change the world.

Published in: Software
  • Be the first to comment

The Art, Joy, and Power of Creating Musical Programs (JFugue at SXSW Interactive 2015)

  1. 1. The Art, Joy, & Power of Creating Musical Programs David Koelle @dmkoelle #JFugue
  2. 2. play(“C”);
  3. 3. Musically-Inclined Programmers! Programming-Inclined Musicians! Anyone who wants to infuse fun into programming! Who is JFugue for?
  4. 4. import org.jfugue.player.Player; public class HelloWorld { public static void main(String[] args) { Player player = new Player(); player.play(“C”); } } “Hello, World!” in JFugue
  5. 5. import org.jfugue.player.Player; public class HelloWorld { public static void main(String[] args) { Player player = new Player(); player.play(“C”); } } “Hello, World!” in JFugue
  6. 6. Sequencer sequencer = MidiSystem.getSequencer(); Sequence sequence = sequencer.getSequence(); Track track = sequence.createTrack(); ShortMessage onMessage = new ShortMessage(); onMessage.setMessage(ShortMessage.NOTE_ON, 0, 60, 128); MidiEvent noteOnEvent = new MidiEvent(onMessage, 0); track.add(noteOnEvent); ShortMessage offMessage = new ShortMessage(); offMessage.setMessage(ShortMessage.NOTE_OFF, 0, 60, 128); MidiEvent noteOffEvent = new MidiEvent(offMessage, 200); track.add(noteOffEvent); sequencer.start(); try { Thread.sleep(track.ticks()); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } “Hello, World!” without JFugue
  7. 7. public class TwelveBarBlues { public static void main(String[] args) { Pattern pattern = new ChordProgression(“I IV V”) .distribute(“7%6”) .allChordsAs(“$0 $0 $0 $0 $1 $1 $0 $0 $2 $1 $0 $0”) .eachChordAs(“$0i $1i $2i $3i $4i $3i $2i $1i”) .getPattern() .setInstrument(“Acoustic Bass”) .setTempo(120); new Player().play(pattern); } } Advanced JFugue… still fits on one slide
  8. 8. Real-World Examples
  9. 9. Log Files Classic Novels
  10. 10. Log Files Classic Novels
  11. 11. Brian Tarbox “I want to know the status of my video servers without staring at a screen.”
  12. 12. time video server log files
  13. 13. time video server log files
  14. 14. o server log files
  15. 15. video stream is created video stream has failed video stream is tricked video stream is destroyed video content is added SNMP trap emitted Interesting Events in Video On Demand (a video stream is content that is being played)
  16. 16. video stream is created video stream has failed video stream is tricked video stream is destroyed video content is added SNMP trap emitted Interesting Events in Video On Demand
  17. 17. video stream is created video stream is destroyed video stream has failed [BASS_DRUM]s [ACOUSTIC_SNARE]s [CRASH_CYMBAL_1]s Mapping Events to Music
  18. 18. Listening to Video on Demand Servers
  19. 19. Learn more at Log4JFugue.org Winner, 2010 Duke’s Choice Award Integrates with Spring Framework Scala version now available
  20. 20. Log Files Classic Novels
  21. 21. Hannah Davis “I want to create a new interactive experience to perceive classic novels.”
  22. 22. TransProse creates music from literature Songs are based on the emotional content of a novel Introducing TransProse
  23. 23. unique word prevalence average sentence length book length punctuation presence of main characters descriptive words chapters/sections changes between tones/moods narrator perspective pacing, plot action/movement locations/environment overall mood/tone/emotion real world or mental landscape smaller individual scenes character ages, voices time period average syllable count amount of dialog Textual Things to Analyze
  24. 24. unique word prevalence average sentence length book length punctuation presence of main characters descriptive words chapters/sections changes between tones/moods narrator perspective pacing, plot action/movement locations/environment overall mood/tone/emotion real world or mental landscape smaller individual scenes character ages, voices time period average syllable count amount of dialog Textual Things to Analyze
  25. 25. Emolex – 14K tagged words
  26. 26. The Art of Making the Music
  27. 27. Stable Tones Emotion Counts Musical Considerations Melodic Shape
  28. 28. minor/major key tempo note duration harmony pitch changing between keys different instrument voices rhythm time signature melody lyrics/vocals # of notes in key used (complexity) Musical Elements to Use
  29. 29. minor/major key tempo note duration harmony pitch changing between keys different instrument voices rhythm time signature melody lyrics/vocals # of notes in key used (complexity) Musical Elements to Use
  30. 30. amount of emotion low emotion counts high emotion counts spikes in emotion active:passive emotions positive:negative emotion amt of emotion per section total emotions in chapter prevalent emotions in novel stability of note from root note more consonant notes more dissonant notes interesting melodic movement tempo major/minor key and octave note duration main melody additional melodies Mappings From Analysis to Music
  31. 31. Learn more at MusicFromText.com TIME Discovery Wired io9 NBC Engadget Fast Company Popular Science CNET Der Spiegel Le Monde … Lots of press More complex music Motifs for main characters Additional instruments and genres Future Work
  32. 32. More Real-World Examples
  33. 33. Interactive Experiences & Games Algorithmic/Generative Music Music Based on Data Education Art & Design Software Support for Cultural Music General Categories
  34. 34. SoundOfShopping.com Turning barcodes into music
  35. 35. “I'm analyzing microblogs like YikYak, Tumblr and Twitter to get trending topics. I'm trying to translate text to quantitative data (semantics, topic, gender, time of day, etc.) to then interpolate and put through a genetic algorithm to generate music.” -Tyler Music from Live-Streaming Data
  36. 36. Making Music from a Chess Match
  37. 37. Soundtrack for Astrophotography Show Based on 6000 naked-eye stars: declination (latitude)  pitch right ascension (longitude)  time magnitude  volume and duration
  38. 38. “Since JFugue can turn MIDI into text, algorithmically created music can be made using the same technology your phone uses for predicting the next word” www.frizzythegame.com brendan@frizzythegame.com JFugue and Markov chaining
  39. 39. A Tour of the Magic of JFugue
  40. 40. Easy to Use Based on Music Theory Fun Things are Baked In Consistent “System” Event-Based Architecture Five Bits of JFugue Magic
  41. 41. Play music with two lines of code: Player player = new Player(); player.play(“C”); Ease of Use
  42. 42. Music is easy to specify and intentionally human-readable C Middle-C C#6h C# note, 6th octave, half duration EbMAJ^w E-flat major chord, first inversion, whole duration Ease of Use
  43. 43. Music is easy to specify and intentionally human-readable I[Flute] Change instrument to Flute T[Allegro] Change tempo to Allegro (120bpm) m560.8q Play a 560.8 Hz tone at ¼-duration Rq Rest (takes duration just like a note) Ease of Use
  44. 44. Pattern melodyVerseOne = new Pattern("T115 V0 I[Synth_Choir]") .add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi") .add("Aqit Ehtq Rhs Gi Aqi+C#6qi Bi+D6i Aih+C#6ih Rhqi B5i") .add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi") .add("Aqit Ehtq Rhs Gi Aqi+C#6qi Bi+D6i Aih+C#6ih Rh B D6"); Ease of Use
  45. 45. public class TwelveBarBlues { public static void main(String[] args) { Pattern pattern = new ChordProgression(“I IV V”) .distribute(“7%6”) .allChordsAs(“$0 $0 $0 $0 $1 $1 $0 $0 $2 $1 $0 $0”) .eachChordAs(“$0i $1i $2i $3i $4i $3i $2i $1i”) .getPattern() .setInstrument(“Acoustic Bass”) .setTempo(120); new Player().play(pattern); } } Ease of “Expression”
  46. 46. Define a Scale in terms of Intervals Define a Chord in terms of Intervals Create a “I IV V” Chord Progression Set the Root of a Chord Progression Get the Chords from a Chord Progression Get the Notes from a Chord … Based on Music Theory
  47. 47. Rhythm rhythm = new Rhythm() .addLayer("O..oO...O..oOO..") .addLayer("..S...S...S...S.") .addLayer("````````````````") .addLayer("...............+"); new Player() .play(rhythm.getPattern().repeat(2)); Fun Things are Baked In
  48. 48. Pattern prog = ChordProgression .fromChords("C#4min E4maj B3maj A3maj") .eachChordAs("$_i $_i Ri $_i") .getPattern() .setInstrument("GUITAR") .setVoice(0); Rhythm rhythm = new Rhythm() .addLayer("..X...X...X...XO"); new Player() .play(new Pattern(prog, rhythm).repeat(2)); Consistent “System”
  49. 49. MidiParser parser = new MidiParser(); LilyPondListener listener = new LilyPondListener(); parser.addListener(listener); parser.parse(midi sequence); listener.displayEngravedSheetMusic(); Event-Based Architecture
  50. 50. StaccatoParser parser = new StaccatoParser(); LilyPondListener listener = new LilyPondListener(); parser.addListener(listener); parser.parse(“C D E F G A B”); listener.displayEngravedSheetMusic(); Event-Based Architecture
  51. 51. StaccatoParser parser = new StaccatoParser(); MusicXmlListener listener = new MusicXmlListener(); parser.addListener(listener); parser.parse(“C D E F G A B”); listener.save(new File(“my_song.xml”)); Event-Based Architecture
  52. 52. MidiParser parser = new MidiParser(); MyParserListener listener = new MyParserListener(); parser.addParserListener(listener); parser.parse(MidiSystem.getSequence(new File(“some midi file"))); System.out.println("There are "+listener.counter+" 'C' notes in this music."); class MyParserListener extends ParserListenerAdapter { public int counter; @Override public void onNoteParsed(Note note) { if (note.getPositionInOctave() == 0) { counter++; } } } Event-Based Architecture
  53. 53. Sending to / Receiving from MIDI Devices Creating New Patterns in Real-Time Sending Musical Events “Before” They Occur Extending the Staccato Parser Using Replacement Maps to Play Solfege Writing Functions for Musical Effects (e.g., Trill) Scratching the Surface…
  54. 54. Making Music from DNA
  55. 55. Making Music from DNA Many people have done this many ways… http://www.mimg.ucla.edu/faculty/miller_jh/gene2music/ previouswork.html
  56. 56. Download JFugue Make beautiful and creative things! http://www.jfugue.org Optionally: Contribute to JFugue! Live coding, Tools for Melodic Structure, Making Awesome Music… What Next?
  57. 57. Thank You! David Koelle @dmkoelle #JFugue

×