Classification of blue whale D calls and fin whale 40-Hz calls using deep learning
Blue whales (Balaenoptera musculus) and fin whales (Balaenoptera physalus) are closely related species that have both experienced declines in population size. The vocalizations of these species provide data on populations sizes and migratory patterns, but the similarity of a subset of their vocalizations presents issues for automated processing. The seasonal variation, proximity to feeding areas, and correlation with feeding behaviors provide evidence that both the blue whale D calls and fin whale 40-Hz calls are feeding-specific calls. The overlapping frequency ranges and frequency modulated nature of the calls, the common genetic background of the two species, and the similar function of the calls are possible contributors to the difficulties in the detection and accurate classification of these calls. Detectors which target downswept call types within this frequency range will experience high recall of both blue whale D calls and fin whale 40-Hz calls, which would require further human annotation to classify each. Building on the success of using deep neural networks for the detection of low-frequency whale calls and for the classification of high-frequency whale vocalizations, we adapt Caffe's freely available AlexNet implementation to tackle the task of distinguishing between blue whale D calls and fin whale 40Hz calls. We obtained over 1387 hours of audio taken from passive acoustic monitoring recorded between 2009-2013 off the coast of Southern California. Ground truth annotation by human experts provided 4796 D calls and 415 40-Hz calls. Using spectrograms of these vocalizations, we extracted high level features from our network and trained an SVM classifier to predict the call type. Our system achieved an overall 97.6% classification accuracy with a 98.6% precision and 98.8% recall for blue whale D calls. Augmenting current blue whale D call detection methods with an additional classification step will allow researchers to eliminate fin whale 40-Hz calls and move through their data faster. Additionally, this method is very flexible and allows researchers the ability to add additional call types to pursue more fine-grained classifiers.
More Related Content
Viewers also liked
Viewers also liked (14)
Similar to Classification of blue whale D calls and fin whale 40-Hz calls using deep learning
Similar to Classification of blue whale D calls and fin whale 40-Hz calls using deep learning (20)
Recently uploaded
Recently uploaded (20)
Classification of blue whale D calls and fin whale 40-Hz calls using deep learning
- 1. Jeremy Karnowski1 & Yair Movshovitz-Attias2 1 University of California, San Diego, USA 2 Carnegie Mellon University, Pittsburgh, USA jkarnows@cogsci.ucsd.edu @mwakanosya Classification of blue whale D calls and fin whale 40-Hz calls using deep learning
- 2. Passive Acoustic Monitoring ● Blue whale and fin whale population sizes are declining. ● Vocalizations found from passive acoustic monitoring can provide massive amounts of data on population sizes and migratory patterns. 2
- 3. “Since the fin whale detectors can be triggered by blue whale calls, a separate detection algorithm for blue whales is being developed to allow for differentiation between the two.“ Weirathmueller , Wilcock , Soule (DCLDE 2011) “Finally, ambiguity could arise in distinguishing blue whale D calls from fin whale 40-Hz calls in an LTSA even though D calls have a distinctly broader bandwidth (Oleson et al. 2007)” Širović, Williams, Kerosky, Wiggins, and Hildebrand (2013) A Problem 3
- 4. Fin Whales and Blue Whales ● Closely related species, so call production may be similar ● Evidence suggests that the fin whale 40-Hz call may be feeding call, similar to the blue whale D call (Watkins (1981); Sirovic, Williams, Kerosky, Wiggins, and Hildebrand (2013)) D call 40-Hz call 4
- 5. Whistle Classification Dynamic Time Warping (Deecke & Janik, 2006) Feature Selection (Gannier et al., 2000) Spectrogram Correlation (Mellinger & Clark, 2000) Generalized Power-Law (Helble, Ierley, D’Spain, Roch, Hildebrand, 2012) 5
- 6. GPU and Deep Learning Packages Academic Hardware Grant Request Form ● Shallow Network: ○ Recognizing transient low-frequency whale sounds by spectrogram correlation (Mellinger, Clark 2000) ● Deep Network: ○ Practical deep neural nets for detecting marine mammals (Nouri, DCLDE 2013) (Setup) 6
- 7. Data Collection Over 1387 hours of audio recorded between 2009-2013 off the coast of Southern California 7
- 8. Dataset Creation Annotations Audio files Creating annotation files for each audio file for visual inspection Creating audio dataset and spectrogram dataset All vocalizations Spectrograms (<100 Hz) Scaled (0-255) as image 4796 D calls 415 40-Hz calls 8
- 9. AlexNet + SVM Input 10 samples 1 sample 96 55x55 256 27x27 256 13x13 384 13x13 384 13x13256x256 4096 4096 SVM ? Class conv max norm conv max norm conv conv conv max full full fc7fc6conv5conv4conv3conv2conv1 Extract high level features Classify each sample 9 4096-dim feature vectors
- 10. fc7 Feature Vectors in 2D (PCA) blue whale D calls fin whale 40-Hz calls 10
- 11. AlexNet + SVM Input 10 samples 1 sample 96 55x55 256 27x27 256 13x13 384 13x13 384 13x13256x256 4096 4096 SVM ? Class conv max norm conv max norm conv conv conv max full full fc7fc6conv5conv4conv3conv2conv1 Extract high level features Classify each sample ● Linear SVM (C=0.0625) ● 10-fold cross-validation ● For each image, classify each sample as 0 (fin) or 1 (blue) ○ Take average of 10 samples and label call blue if > 0.5 11
- 12. Results Confusion Matrix True Predicted Blue whale Fin whale Blue whale 4738 58 Fin whale 66 349 97.62% Accuracy For blue whales: 98.63% Precision 98.79% Recall 12
- 14. Results 14 Correctly classified blue whale D calls Misclassified blue whale D calls Lower frequency range? Different slope? Harmonics? Different call?
- 15. Future Directions ● Compare this method with other classification methods ● Clean up noise in spectrogram before classification ● Obtain more data from noisier environments to make the detectors more robust ● Add in detection: Use GPL detector (Helble et al. 2012) and then classification on the found calls. ○ Determine the time savings for users 15
- 16. Contributions ● Created more targeted classification datasets ● Used deep learning methods for a novel whistle classification task ● Very good performance in accuracy, precision, and recall ● Easy to modify - researchers can add in additional categories: 50-Hz calls and other false tonal detections 16
- 17. Thanks! (for all the fish) Funding Support Elizabeth Vu Tyler Helble John Hildebrand Edwin Hutchins Christine Johnson 17