What Shazam doesn't want you to know

What Shazam doesn't want you to know

• 1.
  What Shazam doesn'twant you to know Roy van Rijn Software Craftsman
• 2.
  Imagine this...
• 3.
  Friday afternoon
• 4.
  Getting inspiration
• 5.
  Singing along...
• 6.
  Meet Stewie!
• 7.
  Music matching
• 8.
  Shazam is magic...alien technology! Start the application.
• 9.
  Let it listenfor +/- 20 seconds.
• 10.
  It tells you:Music matching
• 11.
  Saturday morning
• 12.
  Specifying the audioformat: private AudioFormat getFormat() { float sampleRate = 44100; int sampleSizeInBits = 8; int channels = 1; boolean signed = true ; boolean bigEndian = true ; return new AudioFormat( sampleRate, sampleSizeInBits, channels, signed, bigEndian); } The microphone
• 13.
  Accessing the microphonefinal AudioFormat format = getFormat(); DataLine.Info info = new DataLine.Info(TargetDataLine. class , format); final TargetDataLine line = (TargetDataLine) AudioSystem. getLine (info); line.open(format); line.start(); The microphone
• 14.
  Reading the sound:out = new ByteArrayOutputStream(); running = true ; try { while ( running ) { int count = line.read( buffer , 0, buffer . length ); if (count > 0) { out .write( buffer , 0, count); } } out .close(); } catch (IOException e) { throw new RuntimeException(e); } The microphone
• 15.
  0 0 01 2 5 0 -1 -3 -4 -5 -2 0 1 2 0 2 (etc) Inside the byte array
• 16.
  Plotting a graph
• 17.
  Membrane, cochlear, brainThe human ear
• 18.
  Hertz
• 19.
  The amount ofcycles per second
• 20.
  i.e. one sinewave. Frequencies...?
• 21.
  To match musicwe need frequencies, not waves. Time vs Frequency
• 22.
  Fourier Transformation
• 23.
  Fourier Transformation
• 24.
  Fourier Transformation
• 25.
  Fourier Transformation
• 26.
  Fourier Transformation
• 27.
  Excellent explanation byStuart Riffle http://altdevblogaday.com Fourier Transformation
• 28.
  We've lost trackof time! Frequency domain
• 29.
  Solution: Apply transformationon pieces byte audio[] = out .toByteArray(); final int amountSlices = audio. length / SLICE_SIZE ; Complex[][] results = new Complex[amountChucks][]; for ( int slice = 0;slice < amountSlices; slice++) { Complex[] complex = new Complex[ SLICE_SIZE ]; for ( int i = 0;i< SLICE_SIZE ;i++) { complex[i] = new Complex(audio[(slice* SLICE_SIZE )+i], 0); } results[slice] = FFT. fft (complex); } Windowing
• 30.
  From wikipedia: SpectumAnalyzer A spectrum analyzer or spectral analyzer is a device used to examine the spectral composition of some electrical, acoustic , or optical waveform. Spectrum Analyzer
• 31.
  Demo: AphexTwin
• 32.
  Sunday
• 33.
  Determine the keypoints (in ranges):
• 34.
  34 41 92129 186
• 35.
  39 41 117130 218
• 36.
  40 42 106129 191
• 37.
  40 47 117121 217
• 38.
  40 53 81129 208
• 39.
  40 48 109132 260
• 40.
  39 45 89135 247
• 41.
  40 42 84125 251
• 42.
  40 41 81121 232
• 43.
  38 42 113131 245 (etc...) Matching the song
• 44.
  Playing/decoding MP3 files:JLayer (real time MP3 decoder) jl1.0.1.jar MP3SPI (Java plugin, based on JLayer) mp3spi1.9.4.jar Tritonus (implementation of Java Sound API) tritonus_share.jar Something to match against
• 45.
  Harvesting my musiccollection: public void harvest(File rootDirectory) { String[] itemsInDirectory = rootDirectory.list(); for (String itemInDirectory:itemsInDirectory) { if (itemInDirectory.endsWith( &quot;.mp3&quot; )) { //Assume mp3 file File mp3File = new File(mp3Directory, itemInDirectory); captureAudio(mp3File); } else if ( new File(mp3Directory, itemInDirectory).isDirectory()) { //Directory? Recurse! harvest( new File(mp3Directory, itemInDirectory)); } } } Something to match against
• 46.
  We have: Setof +/- 3000 files of reference data (songs)