SlideShare a Scribd company logo
1 of 17
Download to read offline
Jeremy Karnowski1
& Yair Movshovitz-Attias2
1
University of California, San Diego, USA
2
Carnegie Mellon University, Pittsburgh, USA
jkarnows@cogsci.ucsd.edu @mwakanosya
Classification of blue whale D calls and fin
whale 40-Hz calls using deep learning
Passive Acoustic Monitoring
● Blue whale and fin whale population sizes are declining.
● Vocalizations found from passive acoustic monitoring can
provide massive amounts of data on population sizes and
migratory patterns.
2
“Since the fin whale detectors can be triggered by blue
whale calls, a separate detection algorithm for blue whales
is being developed to allow for differentiation between the
two.“
Weirathmueller , Wilcock , Soule (DCLDE 2011)
“Finally, ambiguity could arise in distinguishing blue whale D
calls from fin whale 40-Hz calls in an LTSA even though D
calls have a distinctly broader bandwidth (Oleson et al.
2007)”
Širović, Williams, Kerosky, Wiggins, and Hildebrand (2013)
A Problem
3
Fin Whales and Blue Whales
● Closely related species, so call production may be similar
● Evidence suggests that the fin whale 40-Hz call may be
feeding call, similar to the blue whale D call
(Watkins (1981); Sirovic, Williams, Kerosky, Wiggins, and Hildebrand (2013))
D call 40-Hz
call
4
Whistle Classification
Dynamic Time Warping
(Deecke & Janik, 2006)
Feature Selection
(Gannier et al., 2000)
Spectrogram Correlation
(Mellinger & Clark, 2000)
Generalized Power-Law
(Helble, Ierley, D’Spain, Roch, Hildebrand, 2012)
5
GPU and Deep Learning Packages
Academic Hardware Grant Request Form
● Shallow Network:
○ Recognizing transient low-frequency whale sounds by
spectrogram correlation (Mellinger, Clark 2000)
● Deep Network:
○ Practical deep neural nets for detecting marine
mammals (Nouri, DCLDE 2013)
(Setup) 6
Data Collection
Over 1387 hours of audio recorded between 2009-2013 off the coast of Southern California 7
Dataset Creation
Annotations
Audio files
Creating annotation files for each audio file for visual inspection
Creating audio dataset and spectrogram dataset
All vocalizations Spectrograms (<100 Hz) Scaled (0-255) as image
4796 D calls
415 40-Hz calls
8
AlexNet + SVM
Input
10 samples 1 sample
96
55x55
256
27x27
256
13x13
384
13x13
384
13x13256x256 4096 4096
SVM ?
Class
conv
max
norm
conv
max
norm
conv conv conv
max
full full
fc7fc6conv5conv4conv3conv2conv1
Extract high level features Classify
each sample
9
4096-dim feature vectors
fc7 Feature Vectors in 2D (PCA)
blue whale
D calls
fin whale
40-Hz calls
10
AlexNet + SVM
Input
10 samples 1 sample
96
55x55
256
27x27
256
13x13
384
13x13
384
13x13256x256 4096 4096
SVM ?
Class
conv
max
norm
conv
max
norm
conv conv conv
max
full full
fc7fc6conv5conv4conv3conv2conv1
Extract high level features Classify
each sample
● Linear SVM (C=0.0625)
● 10-fold cross-validation
● For each image, classify each sample as 0 (fin) or 1 (blue)
○ Take average of 10 samples and label call blue if > 0.5
11
Results
Confusion Matrix
True
Predicted
Blue whale Fin whale
Blue whale 4738 58
Fin whale 66 349
97.62% Accuracy
For blue whales: 98.63% Precision
98.79% Recall
12
Results
Precision Recall Curve
13
Results
14
Correctly classified blue whale D calls
Misclassified blue whale D calls
Lower frequency range? Different slope? Harmonics?
Different call?
Future Directions
● Compare this method with other classification methods
● Clean up noise in spectrogram before classification
● Obtain more data from noisier environments to make
the detectors more robust
● Add in detection: Use GPL detector (Helble et al. 2012)
and then classification on the found calls.
○ Determine the time savings for users
15
Contributions
● Created more targeted classification datasets
● Used deep learning methods for a novel whistle
classification task
● Very good performance in accuracy, precision, and
recall
● Easy to modify - researchers can add in additional
categories: 50-Hz calls and other false tonal detections
16
Thanks!
(for all the fish)
Funding
Support
Elizabeth Vu
Tyler Helble
John Hildebrand
Edwin Hutchins
Christine Johnson
17

More Related Content

Viewers also liked

Presentation1
Presentation1Presentation1
Presentation1jnmtimbo
 
The dolphin Dikran and Pilar
The dolphin  Dikran and PilarThe dolphin  Dikran and Pilar
The dolphin Dikran and PilarSandra Sanchez
 
Emmanuel blue whale1
Emmanuel blue whale1Emmanuel blue whale1
Emmanuel blue whale1Kswartz3
 
The Blue Whale Eric
The Blue Whale  EricThe Blue Whale  Eric
The Blue Whale EricMs. Valcik
 
Fennelly blue whale
Fennelly blue whaleFennelly blue whale
Fennelly blue whaleksolender
 
Jared beck power point
Jared beck power pointJared beck power point
Jared beck power pointJohn Hoopman
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...GeeksLab Odessa
 
Australian Aboriginal Culture in Art
Australian Aboriginal Culture in ArtAustralian Aboriginal Culture in Art
Australian Aboriginal Culture in Artmcrawf
 
Whales By Ajay And Trupti
Whales By Ajay And TruptiWhales By Ajay And Trupti
Whales By Ajay And Truptisubzero64
 
Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016
Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016
Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016Universitat Politècnica de Catalunya
 
Folklore of rajasthan
Folklore of rajasthanFolklore of rajasthan
Folklore of rajasthanAyushi Rajput
 
Africa Art and Culture
Africa Art and CultureAfrica Art and Culture
Africa Art and CultureSara Allen
 

Viewers also liked (14)

Presentation1
Presentation1Presentation1
Presentation1
 
Anna and alexandra
Anna and alexandraAnna and alexandra
Anna and alexandra
 
The dolphin Dikran and Pilar
The dolphin  Dikran and PilarThe dolphin  Dikran and Pilar
The dolphin Dikran and Pilar
 
Emmanuel blue whale1
Emmanuel blue whale1Emmanuel blue whale1
Emmanuel blue whale1
 
The Blue Whale Eric
The Blue Whale  EricThe Blue Whale  Eric
The Blue Whale Eric
 
Fennelly blue whale
Fennelly blue whaleFennelly blue whale
Fennelly blue whale
 
Jared beck power point
Jared beck power pointJared beck power point
Jared beck power point
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
 
Australian Aboriginal Culture in Art
Australian Aboriginal Culture in ArtAustralian Aboriginal Culture in Art
Australian Aboriginal Culture in Art
 
Whales By Ajay And Trupti
Whales By Ajay And TruptiWhales By Ajay And Trupti
Whales By Ajay And Trupti
 
Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016
Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016
Deep Learning for Computer Vision (1/4): Image Analytics @ laSalle 2016
 
Folklore of rajasthan
Folklore of rajasthanFolklore of rajasthan
Folklore of rajasthan
 
Africa Art and Culture
Africa Art and CultureAfrica Art and Culture
Africa Art and Culture
 
2 ay b
2 ay b2 ay b
2 ay b
 

Similar to Classification of blue whale D calls and fin whale 40-Hz calls using deep learning

Universal plane wave compounding for high quality us imaging using deep learning
Universal plane wave compounding for high quality us imaging using deep learningUniversal plane wave compounding for high quality us imaging using deep learning
Universal plane wave compounding for high quality us imaging using deep learningShujaat Khan
 
Deep Learning-Based Universal Beamformer for Ultrasound Imaging
Deep Learning-Based Universal Beamformer for Ultrasound ImagingDeep Learning-Based Universal Beamformer for Ultrasound Imaging
Deep Learning-Based Universal Beamformer for Ultrasound ImagingShujaat Khan
 
Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...
Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...
Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...Shamman Noor Shoudha
 
Channel Models for Massive MIMO
Channel Models for Massive MIMOChannel Models for Massive MIMO
Channel Models for Massive MIMOCPqD
 
Sonar Principles Asw Analysis
Sonar Principles Asw AnalysisSonar Principles Asw Analysis
Sonar Principles Asw AnalysisJim Jenkins
 
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)NAVER Engineering
 
Wavemeter for VisibleNIRDiGangi2006.pdf
Wavemeter for VisibleNIRDiGangi2006.pdfWavemeter for VisibleNIRDiGangi2006.pdf
Wavemeter for VisibleNIRDiGangi2006.pdfennio12
 
Basics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.ppt
Basics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.pptBasics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.ppt
Basics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.pptpebojey210
 
Introduction to deep learning based voice activity detection
Introduction to deep learning based voice activity detectionIntroduction to deep learning based voice activity detection
Introduction to deep learning based voice activity detectionNAVER Engineering
 
Telomere-to-telomere assembly of a complete human chromosomes
Telomere-to-telomere assembly of a complete human chromosomesTelomere-to-telomere assembly of a complete human chromosomes
Telomere-to-telomere assembly of a complete human chromosomesGenome Reference Consortium
 
KHMiga-AGBT.020923.upload.pdf
KHMiga-AGBT.020923.upload.pdfKHMiga-AGBT.020923.upload.pdf
KHMiga-AGBT.020923.upload.pdfKarenMiga
 
Scaling Secure Computation
Scaling Secure ComputationScaling Secure Computation
Scaling Secure ComputationDavid Evans
 
Fisheries Acoustics 102 - Applications
Fisheries Acoustics 102 - ApplicationsFisheries Acoustics 102 - Applications
Fisheries Acoustics 102 - ApplicationsBob McClure
 
Toward a Global Coral Reef Observatory
Toward a Global Coral Reef ObservatoryToward a Global Coral Reef Observatory
Toward a Global Coral Reef ObservatoryLarry Smarr
 
ECHOES & DELPH Seismic - Advances in geophysical sensor data acquisition
ECHOES & DELPH Seismic - Advances in geophysical sensor data acquisitionECHOES & DELPH Seismic - Advances in geophysical sensor data acquisition
ECHOES & DELPH Seismic - Advances in geophysical sensor data acquisitionIXSEA-DELPH
 
Advances in geophysical sensor data acquisition
Advances in geophysical sensor data acquisitionAdvances in geophysical sensor data acquisition
Advances in geophysical sensor data acquisitionIXSEA-DELPH
 
Phylogenomic Supertrees. ORP Bininda-Emond
Phylogenomic Supertrees. ORP Bininda-EmondPhylogenomic Supertrees. ORP Bininda-Emond
Phylogenomic Supertrees. ORP Bininda-EmondRoderic Page
 

Similar to Classification of blue whale D calls and fin whale 40-Hz calls using deep learning (20)

Universal plane wave compounding for high quality us imaging using deep learning
Universal plane wave compounding for high quality us imaging using deep learningUniversal plane wave compounding for high quality us imaging using deep learning
Universal plane wave compounding for high quality us imaging using deep learning
 
Deep Learning-Based Universal Beamformer for Ultrasound Imaging
Deep Learning-Based Universal Beamformer for Ultrasound ImagingDeep Learning-Based Universal Beamformer for Ultrasound Imaging
Deep Learning-Based Universal Beamformer for Ultrasound Imaging
 
KinPhy
KinPhyKinPhy
KinPhy
 
Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...
Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...
Audio Visual Emotion Recognition Using Cross Correlation and Wavelet Packet D...
 
Channel Models for Massive MIMO
Channel Models for Massive MIMOChannel Models for Massive MIMO
Channel Models for Massive MIMO
 
Sonar Principles Asw Analysis
Sonar Principles Asw AnalysisSonar Principles Asw Analysis
Sonar Principles Asw Analysis
 
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
캡슐 네트워크를 이용한 엔드투엔드 음성 단어 인식, 배재성(KAIST 석사과정)
 
Wavemeter for VisibleNIRDiGangi2006.pdf
Wavemeter for VisibleNIRDiGangi2006.pdfWavemeter for VisibleNIRDiGangi2006.pdf
Wavemeter for VisibleNIRDiGangi2006.pdf
 
Basics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.ppt
Basics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.pptBasics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.ppt
Basics-Od-Digital-Audio-Multimedia-Technologies-Unit-III.ppt
 
Introduction to deep learning based voice activity detection
Introduction to deep learning based voice activity detectionIntroduction to deep learning based voice activity detection
Introduction to deep learning based voice activity detection
 
Telomere-to-telomere assembly of a complete human chromosomes
Telomere-to-telomere assembly of a complete human chromosomesTelomere-to-telomere assembly of a complete human chromosomes
Telomere-to-telomere assembly of a complete human chromosomes
 
KHMiga-AGBT.020923.upload.pdf
KHMiga-AGBT.020923.upload.pdfKHMiga-AGBT.020923.upload.pdf
KHMiga-AGBT.020923.upload.pdf
 
Scaling Secure Computation
Scaling Secure ComputationScaling Secure Computation
Scaling Secure Computation
 
Fisheries Acoustics 102 - Applications
Fisheries Acoustics 102 - ApplicationsFisheries Acoustics 102 - Applications
Fisheries Acoustics 102 - Applications
 
Sept2016 sv nist_intro
Sept2016 sv nist_introSept2016 sv nist_intro
Sept2016 sv nist_intro
 
Toward a Global Coral Reef Observatory
Toward a Global Coral Reef ObservatoryToward a Global Coral Reef Observatory
Toward a Global Coral Reef Observatory
 
BWE2.ppt
BWE2.pptBWE2.ppt
BWE2.ppt
 
ECHOES & DELPH Seismic - Advances in geophysical sensor data acquisition
ECHOES & DELPH Seismic - Advances in geophysical sensor data acquisitionECHOES & DELPH Seismic - Advances in geophysical sensor data acquisition
ECHOES & DELPH Seismic - Advances in geophysical sensor data acquisition
 
Advances in geophysical sensor data acquisition
Advances in geophysical sensor data acquisitionAdvances in geophysical sensor data acquisition
Advances in geophysical sensor data acquisition
 
Phylogenomic Supertrees. ORP Bininda-Emond
Phylogenomic Supertrees. ORP Bininda-EmondPhylogenomic Supertrees. ORP Bininda-Emond
Phylogenomic Supertrees. ORP Bininda-Emond
 

Recently uploaded

AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 

Recently uploaded (17)

AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 

Classification of blue whale D calls and fin whale 40-Hz calls using deep learning

  • 1. Jeremy Karnowski1 & Yair Movshovitz-Attias2 1 University of California, San Diego, USA 2 Carnegie Mellon University, Pittsburgh, USA jkarnows@cogsci.ucsd.edu @mwakanosya Classification of blue whale D calls and fin whale 40-Hz calls using deep learning
  • 2. Passive Acoustic Monitoring ● Blue whale and fin whale population sizes are declining. ● Vocalizations found from passive acoustic monitoring can provide massive amounts of data on population sizes and migratory patterns. 2
  • 3. “Since the fin whale detectors can be triggered by blue whale calls, a separate detection algorithm for blue whales is being developed to allow for differentiation between the two.“ Weirathmueller , Wilcock , Soule (DCLDE 2011) “Finally, ambiguity could arise in distinguishing blue whale D calls from fin whale 40-Hz calls in an LTSA even though D calls have a distinctly broader bandwidth (Oleson et al. 2007)” Širović, Williams, Kerosky, Wiggins, and Hildebrand (2013) A Problem 3
  • 4. Fin Whales and Blue Whales ● Closely related species, so call production may be similar ● Evidence suggests that the fin whale 40-Hz call may be feeding call, similar to the blue whale D call (Watkins (1981); Sirovic, Williams, Kerosky, Wiggins, and Hildebrand (2013)) D call 40-Hz call 4
  • 5. Whistle Classification Dynamic Time Warping (Deecke & Janik, 2006) Feature Selection (Gannier et al., 2000) Spectrogram Correlation (Mellinger & Clark, 2000) Generalized Power-Law (Helble, Ierley, D’Spain, Roch, Hildebrand, 2012) 5
  • 6. GPU and Deep Learning Packages Academic Hardware Grant Request Form ● Shallow Network: ○ Recognizing transient low-frequency whale sounds by spectrogram correlation (Mellinger, Clark 2000) ● Deep Network: ○ Practical deep neural nets for detecting marine mammals (Nouri, DCLDE 2013) (Setup) 6
  • 7. Data Collection Over 1387 hours of audio recorded between 2009-2013 off the coast of Southern California 7
  • 8. Dataset Creation Annotations Audio files Creating annotation files for each audio file for visual inspection Creating audio dataset and spectrogram dataset All vocalizations Spectrograms (<100 Hz) Scaled (0-255) as image 4796 D calls 415 40-Hz calls 8
  • 9. AlexNet + SVM Input 10 samples 1 sample 96 55x55 256 27x27 256 13x13 384 13x13 384 13x13256x256 4096 4096 SVM ? Class conv max norm conv max norm conv conv conv max full full fc7fc6conv5conv4conv3conv2conv1 Extract high level features Classify each sample 9 4096-dim feature vectors
  • 10. fc7 Feature Vectors in 2D (PCA) blue whale D calls fin whale 40-Hz calls 10
  • 11. AlexNet + SVM Input 10 samples 1 sample 96 55x55 256 27x27 256 13x13 384 13x13 384 13x13256x256 4096 4096 SVM ? Class conv max norm conv max norm conv conv conv max full full fc7fc6conv5conv4conv3conv2conv1 Extract high level features Classify each sample ● Linear SVM (C=0.0625) ● 10-fold cross-validation ● For each image, classify each sample as 0 (fin) or 1 (blue) ○ Take average of 10 samples and label call blue if > 0.5 11
  • 12. Results Confusion Matrix True Predicted Blue whale Fin whale Blue whale 4738 58 Fin whale 66 349 97.62% Accuracy For blue whales: 98.63% Precision 98.79% Recall 12
  • 14. Results 14 Correctly classified blue whale D calls Misclassified blue whale D calls Lower frequency range? Different slope? Harmonics? Different call?
  • 15. Future Directions ● Compare this method with other classification methods ● Clean up noise in spectrogram before classification ● Obtain more data from noisier environments to make the detectors more robust ● Add in detection: Use GPL detector (Helble et al. 2012) and then classification on the found calls. ○ Determine the time savings for users 15
  • 16. Contributions ● Created more targeted classification datasets ● Used deep learning methods for a novel whistle classification task ● Very good performance in accuracy, precision, and recall ● Easy to modify - researchers can add in additional categories: 50-Hz calls and other false tonal detections 16
  • 17. Thanks! (for all the fish) Funding Support Elizabeth Vu Tyler Helble John Hildebrand Edwin Hutchins Christine Johnson 17