SlideShare a Scribd company logo
Music Emotion Recognition
A State of the Art Review




Dr Scott Beveridge
Interdisciplinary Perspectives on Music, Emotion and Technology
Glasgow Caledonian University, 25th June 2012


© Fraunhofer IDMT
Outline

 Why emotion?


 Definition of Music Emotion Recognition


 History and motivation for interdisciplinary approach


 Current experiments




© Fraunhofer IDMT
Why emotion?
Emotion and Meaning in Music




© Fraunhofer IDMT
Emotion and Meaning in Music

 Humans treat music with great importance


 Music has a very powerful effect
            One of the primary reasons behind
             humans enjoyment of music


 Henry is an example of the listener
  perspective
            Music is useful for those who
             compose and perform also
                 Method of communication and
                  expression

© Fraunhofer IDMT
Emotion and Meaning in Music

 Emotion is a great way of creating information which facilitates browsing and
  organization of music
            Vast number of audio tracks online


 Why are we doing this?
 MER Applications
            Music organization and browsing – personal and commercial
            Academic – Music Digital Libraries (MDL), archiving
            Health – music therapy, pain management (Knox 2011)




© Fraunhofer IDMT
Emotion and Meaning in Music

 Pin down some aspects of the affective process to make the problem
  computationally tractable
            Expression versus Induction


 Most work in MER focusses on expression
            „What the music is trying to say to you‟
            This is easier to decide on with some types of music than others
                     Film music




© Fraunhofer IDMT
A Definition of Music Emotion
Recognition (MER)
 MER has two steps


           1. Identification, recognition and extraction of musical characteristics
              which express emotions in music


           2. Modelling these characteristics in order to make prediction on
              emotions expressed by „new‟ music




© Fraunhofer IDMT
Emotion Recognition       Happy = Fast Tempo,
Supervised Learning               Major Mode

                      Sad = Slow Tempo,
     Happy                  Minor Mode




         Sad




                      ?
© Fraunhofer IDMT
What can machines learn?
Conceptualization of Emotion

 Assign emotion labels (Classification)
            Previous example
            Exuberant, Anxious, Depressed, Content


 Define a point in 2D space (Numeric Prediction)


 Predict emotions which vary over time (Time-continuous
  prediction)




© Fraunhofer IDMT
A Brief History of MER
Background

 Music Psychology           Engineering
 Cate Hevner        1935

                    1988    Kayatose (Symbolic)

 Patrik Juslin       2001
                    1935
                    2003    Feng (Signal-based)




                    NOW



© Fraunhofer IDMT
A Brief History of MER
Popular Music

 Popular music is becoming…..popular!




© Fraunhofer IDMT
Popular Music
Challenges

 By definition popular music is
            Made commercially
                 Limits the scope of expressed emotions
            Made using ever-changing technologies
                 Over production (dynamic compression)


 This generally leads to homogeneity in the popular music genre


 To overcome these problems psychologists, musicologists, philosophers, and
  engineers must work together



© Fraunhofer IDMT
A Brief History of MER
Why Interdisciplinary?

 Olighara 2003
            “One 39 year old male Chinese” annotator for a corpus of Western
             contemporary popular music
 Wu 2006
            10 second music clips and no mention of music


 Schellenberg12
            Manual tempo calculation
 Yang07
            Expression/Induction distinction



© Fraunhofer IDMT
Current Experiments

 Based on 2 steps in MER
            Features
                 Tone Objects
                 Statistical properties of melody


            Modelling
                 Predict tension gradients for use in syncronisation
                 Includes creation of new features
                     Feature fusion




© Fraunhofer IDMT
Current Experiments
Tone Objects

 Objective: Find novel ways of describing popular music by creating new
             musical features


 Existing features
            Tempo, Mode, Key, Instrument Timbre


 New features
            Examine existing features based on tone objects
                 Musical notes of the main melody




© Fraunhofer IDMT
Current Experiments
Tone Objects

 Extracting tone objects involves
  many signal processing
  techniques
            Source separation
            Computational Auditory
             Scene Analysis (CASA)
                 Identify the main
                  melody


 Results shows that tone objects
  help identify particular types of
  emotion


© Fraunhofer IDMT
Current Projects
Main Melody Statistics

 In linguistics, Zipf’s Law shows that: [CAREFUL! These figures might be
  incorrect]
  Given some corpus of natural language the frequency of any word is inversely
                 proportional to its rank in the frequency table
  Word              # of occurrences                Word                 # of occurrences
  the               69,971                          Unison               69,971
  of                36,411                          Major 3rd            36,411
  and               28,852                          Perfect 5th          28,852




 Studies1 have shown that Zipf law statistics have a relationship with aesthetic
  aspects of music – pleasant, beautiful
 Can Zipf‟s law statistics be applied in emotion classification?
                                          1   http://sger.cs.cofc.edu/
© Fraunhofer IDMT
Current Projects
Tension Prediction

 Objective: Track time-continuous tension gradients in film music
 Applications in syncronization task
            Helps creators of films and adverts find music with specific
             characteristics
 Approach:
            Step 1: Extract time-continuous features from a collection of film
             music
            Step 2: Conduct a study which asks people to rate time-continuous
             tension
            Step 3: Build models with the data which predicts tension gradients
             in new music
 An example of supervised learning!

© Fraunhofer IDMT
Current Projects
Tension Prediction – Feature Extraction

 Step1: Extract time-continuous features




© Fraunhofer IDMT
Current Projects
Tension Prediction – Participant Testing

 Step 2: Asked participants to rate music
  based on perceived tension


 General agreement




© Fraunhofer IDMT
Current Projects
Tension Prediction – Participant Testing

 Features most correlated with tension:


            Timbral Complexity: The rate of change of timbre (How many
             „different sounding‟ instruments are present


            Spectral Dissonance: Perceived roughness


            Pure Tonalness: A measure of how „tone-like‟ a sound is




© Fraunhofer IDMT
Current Projects
Tension Prediction – Demonstration




© Fraunhofer IDMT
The Future of MER

 Automatic MER systems are only the beginning


 For MER systems to be truly effective it is necessary to adopt a user-centred
  approach
            Emotions elicited in music are created through social factors and
             environment
                 Listening with friends
                 Listening on the way to work


 Profile users to create bespoke emotion recommendation systems based on
            Geo-location, time of day, skipping behaviour


© Fraunhofer IDMT
Music Emotion Recognition
A State of the Art Review




                         Thank you !!

                      bevest@idmt.fraunhofer.de
                    http://tinyurl.com/bevestLinkedIn


© Fraunhofer IDMT

More Related Content

Similar to Interdisciplinary Perspectives on Emotion, Music and Technology

Computational models of symphonic music
Computational models of symphonic musicComputational models of symphonic music
Computational models of symphonic music
Emilia Gómez
 
Musicand interactivity 1
Musicand interactivity 1Musicand interactivity 1
Musicand interactivity 1
david mcandrew
 
Ai lab workshop(211206)
Ai lab workshop(211206)Ai lab workshop(211206)
Ai lab workshop(211206)
António Oliveira
 
Wilkie
WilkieWilkie
Wilkie
anesah
 
An Introduction To Speech Sciences (Acoustic Analysis Of Speech)
An Introduction To Speech Sciences (Acoustic Analysis Of Speech)An Introduction To Speech Sciences (Acoustic Analysis Of Speech)
An Introduction To Speech Sciences (Acoustic Analysis Of Speech)
Jeff Nelson
 
Sonification and Speaker Object
Sonification and Speaker ObjectSonification and Speaker Object
Sonification and Speaker Object
柏豪 紀
 
Marc-André Rappaz - Metaphors, gestures, and emotions in music
Marc-André Rappaz - Metaphors, gestures, and emotions in musicMarc-André Rappaz - Metaphors, gestures, and emotions in music
Marc-André Rappaz - Metaphors, gestures, and emotions in music
swissnex San Francisco
 
The effect of music listening on work performance.
The effect of music listening on work performance.The effect of music listening on work performance.
The effect of music listening on work performance.
PeacefulNature
 
Human Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A ReviewHuman Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A Review
Editor IJCATR
 
Denktank 2010
Denktank 2010Denktank 2010
Denktank 2010
ocor203
 
Extraction and Conversion of Vocals
Extraction and Conversion of VocalsExtraction and Conversion of Vocals
Extraction and Conversion of Vocals
IRJET Journal
 
IRJET- Music Genre Recognition using Convolution Neural Network
IRJET- Music Genre Recognition using Convolution Neural NetworkIRJET- Music Genre Recognition using Convolution Neural Network
IRJET- Music Genre Recognition using Convolution Neural Network
IRJET Journal
 
Automatic Music Transcription
Automatic Music TranscriptionAutomatic Music Transcription
Automatic Music Transcription
Khyati Ganatra
 
Creating an Entertaining and Informative Music Visualization
Creating an Entertaining and Informative Music VisualizationCreating an Entertaining and Informative Music Visualization
Creating an Entertaining and Informative Music Visualization
icchp2012
 
CONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUES
CONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUESCONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUES
CONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUES
AM Publications
 
Animal Voice Morphing System
Animal Voice Morphing SystemAnimal Voice Morphing System
Animal Voice Morphing System
editor1knowledgecuddle
 
Theories of speech perception.pptx
Theories of speech perception.pptxTheories of speech perception.pptx
Theories of speech perception.pptx
sherin444916
 
Mood Detection
Mood DetectionMood Detection
Mood Detection
Puneet Singh
 
Towards a Computational Model of Melody Identification in Polyphonic Music
Towards a Computational Model of Melody Identification in Polyphonic MusicTowards a Computational Model of Melody Identification in Polyphonic Music
Towards a Computational Model of Melody Identification in Polyphonic Music
Ronildo Oliveira
 
MLConf2013: Teaching Computer to Listen to Music
MLConf2013: Teaching Computer to Listen to MusicMLConf2013: Teaching Computer to Listen to Music
MLConf2013: Teaching Computer to Listen to Music
Eric Battenberg
 

Similar to Interdisciplinary Perspectives on Emotion, Music and Technology (20)

Computational models of symphonic music
Computational models of symphonic musicComputational models of symphonic music
Computational models of symphonic music
 
Musicand interactivity 1
Musicand interactivity 1Musicand interactivity 1
Musicand interactivity 1
 
Ai lab workshop(211206)
Ai lab workshop(211206)Ai lab workshop(211206)
Ai lab workshop(211206)
 
Wilkie
WilkieWilkie
Wilkie
 
An Introduction To Speech Sciences (Acoustic Analysis Of Speech)
An Introduction To Speech Sciences (Acoustic Analysis Of Speech)An Introduction To Speech Sciences (Acoustic Analysis Of Speech)
An Introduction To Speech Sciences (Acoustic Analysis Of Speech)
 
Sonification and Speaker Object
Sonification and Speaker ObjectSonification and Speaker Object
Sonification and Speaker Object
 
Marc-André Rappaz - Metaphors, gestures, and emotions in music
Marc-André Rappaz - Metaphors, gestures, and emotions in musicMarc-André Rappaz - Metaphors, gestures, and emotions in music
Marc-André Rappaz - Metaphors, gestures, and emotions in music
 
The effect of music listening on work performance.
The effect of music listening on work performance.The effect of music listening on work performance.
The effect of music listening on work performance.
 
Human Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A ReviewHuman Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A Review
 
Denktank 2010
Denktank 2010Denktank 2010
Denktank 2010
 
Extraction and Conversion of Vocals
Extraction and Conversion of VocalsExtraction and Conversion of Vocals
Extraction and Conversion of Vocals
 
IRJET- Music Genre Recognition using Convolution Neural Network
IRJET- Music Genre Recognition using Convolution Neural NetworkIRJET- Music Genre Recognition using Convolution Neural Network
IRJET- Music Genre Recognition using Convolution Neural Network
 
Automatic Music Transcription
Automatic Music TranscriptionAutomatic Music Transcription
Automatic Music Transcription
 
Creating an Entertaining and Informative Music Visualization
Creating an Entertaining and Informative Music VisualizationCreating an Entertaining and Informative Music Visualization
Creating an Entertaining and Informative Music Visualization
 
CONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUES
CONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUESCONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUES
CONTENT BASED AUDIO CLASSIFIER & FEATURE EXTRACTION USING ANN TECNIQUES
 
Animal Voice Morphing System
Animal Voice Morphing SystemAnimal Voice Morphing System
Animal Voice Morphing System
 
Theories of speech perception.pptx
Theories of speech perception.pptxTheories of speech perception.pptx
Theories of speech perception.pptx
 
Mood Detection
Mood DetectionMood Detection
Mood Detection
 
Towards a Computational Model of Melody Identification in Polyphonic Music
Towards a Computational Model of Melody Identification in Polyphonic MusicTowards a Computational Model of Melody Identification in Polyphonic Music
Towards a Computational Model of Melody Identification in Polyphonic Music
 
MLConf2013: Teaching Computer to Listen to Music
MLConf2013: Teaching Computer to Listen to MusicMLConf2013: Teaching Computer to Listen to Music
MLConf2013: Teaching Computer to Listen to Music
 

Recently uploaded

Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 

Recently uploaded (20)

Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 

Interdisciplinary Perspectives on Emotion, Music and Technology

  • 1. Music Emotion Recognition A State of the Art Review Dr Scott Beveridge Interdisciplinary Perspectives on Music, Emotion and Technology Glasgow Caledonian University, 25th June 2012 © Fraunhofer IDMT
  • 2. Outline  Why emotion?  Definition of Music Emotion Recognition  History and motivation for interdisciplinary approach  Current experiments © Fraunhofer IDMT
  • 3. Why emotion? Emotion and Meaning in Music © Fraunhofer IDMT
  • 4. Emotion and Meaning in Music  Humans treat music with great importance  Music has a very powerful effect  One of the primary reasons behind humans enjoyment of music  Henry is an example of the listener perspective  Music is useful for those who compose and perform also  Method of communication and expression © Fraunhofer IDMT
  • 5. Emotion and Meaning in Music  Emotion is a great way of creating information which facilitates browsing and organization of music  Vast number of audio tracks online  Why are we doing this?  MER Applications  Music organization and browsing – personal and commercial  Academic – Music Digital Libraries (MDL), archiving  Health – music therapy, pain management (Knox 2011) © Fraunhofer IDMT
  • 6. Emotion and Meaning in Music  Pin down some aspects of the affective process to make the problem computationally tractable  Expression versus Induction  Most work in MER focusses on expression  „What the music is trying to say to you‟  This is easier to decide on with some types of music than others  Film music © Fraunhofer IDMT
  • 7. A Definition of Music Emotion Recognition (MER)  MER has two steps 1. Identification, recognition and extraction of musical characteristics which express emotions in music 2. Modelling these characteristics in order to make prediction on emotions expressed by „new‟ music © Fraunhofer IDMT
  • 8. Emotion Recognition Happy = Fast Tempo, Supervised Learning Major Mode Sad = Slow Tempo, Happy Minor Mode Sad ? © Fraunhofer IDMT
  • 9. What can machines learn? Conceptualization of Emotion  Assign emotion labels (Classification)  Previous example  Exuberant, Anxious, Depressed, Content  Define a point in 2D space (Numeric Prediction)  Predict emotions which vary over time (Time-continuous prediction) © Fraunhofer IDMT
  • 10. A Brief History of MER Background Music Psychology Engineering Cate Hevner 1935 1988 Kayatose (Symbolic) Patrik Juslin 2001 1935 2003 Feng (Signal-based) NOW © Fraunhofer IDMT
  • 11. A Brief History of MER Popular Music  Popular music is becoming…..popular! © Fraunhofer IDMT
  • 12. Popular Music Challenges  By definition popular music is  Made commercially  Limits the scope of expressed emotions  Made using ever-changing technologies  Over production (dynamic compression)  This generally leads to homogeneity in the popular music genre  To overcome these problems psychologists, musicologists, philosophers, and engineers must work together © Fraunhofer IDMT
  • 13. A Brief History of MER Why Interdisciplinary?  Olighara 2003  “One 39 year old male Chinese” annotator for a corpus of Western contemporary popular music  Wu 2006  10 second music clips and no mention of music  Schellenberg12  Manual tempo calculation  Yang07  Expression/Induction distinction © Fraunhofer IDMT
  • 14. Current Experiments  Based on 2 steps in MER  Features  Tone Objects  Statistical properties of melody  Modelling  Predict tension gradients for use in syncronisation  Includes creation of new features  Feature fusion © Fraunhofer IDMT
  • 15. Current Experiments Tone Objects  Objective: Find novel ways of describing popular music by creating new musical features  Existing features  Tempo, Mode, Key, Instrument Timbre  New features  Examine existing features based on tone objects  Musical notes of the main melody © Fraunhofer IDMT
  • 16. Current Experiments Tone Objects  Extracting tone objects involves many signal processing techniques  Source separation  Computational Auditory Scene Analysis (CASA)  Identify the main melody  Results shows that tone objects help identify particular types of emotion © Fraunhofer IDMT
  • 17. Current Projects Main Melody Statistics  In linguistics, Zipf’s Law shows that: [CAREFUL! These figures might be incorrect] Given some corpus of natural language the frequency of any word is inversely proportional to its rank in the frequency table Word # of occurrences Word # of occurrences the 69,971 Unison 69,971 of 36,411 Major 3rd 36,411 and 28,852 Perfect 5th 28,852  Studies1 have shown that Zipf law statistics have a relationship with aesthetic aspects of music – pleasant, beautiful  Can Zipf‟s law statistics be applied in emotion classification? 1 http://sger.cs.cofc.edu/ © Fraunhofer IDMT
  • 18. Current Projects Tension Prediction  Objective: Track time-continuous tension gradients in film music  Applications in syncronization task  Helps creators of films and adverts find music with specific characteristics  Approach:  Step 1: Extract time-continuous features from a collection of film music  Step 2: Conduct a study which asks people to rate time-continuous tension  Step 3: Build models with the data which predicts tension gradients in new music  An example of supervised learning! © Fraunhofer IDMT
  • 19. Current Projects Tension Prediction – Feature Extraction  Step1: Extract time-continuous features © Fraunhofer IDMT
  • 20. Current Projects Tension Prediction – Participant Testing  Step 2: Asked participants to rate music based on perceived tension  General agreement © Fraunhofer IDMT
  • 21. Current Projects Tension Prediction – Participant Testing  Features most correlated with tension:  Timbral Complexity: The rate of change of timbre (How many „different sounding‟ instruments are present  Spectral Dissonance: Perceived roughness  Pure Tonalness: A measure of how „tone-like‟ a sound is © Fraunhofer IDMT
  • 22. Current Projects Tension Prediction – Demonstration © Fraunhofer IDMT
  • 23. The Future of MER  Automatic MER systems are only the beginning  For MER systems to be truly effective it is necessary to adopt a user-centred approach  Emotions elicited in music are created through social factors and environment  Listening with friends  Listening on the way to work  Profile users to create bespoke emotion recommendation systems based on  Geo-location, time of day, skipping behaviour © Fraunhofer IDMT
  • 24. Music Emotion Recognition A State of the Art Review Thank you !! bevest@idmt.fraunhofer.de http://tinyurl.com/bevestLinkedIn © Fraunhofer IDMT

Editor's Notes

  1. V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. expressive
  2. V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. expressive
  3. V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. Expressive<<David Cameron’s Fav album>>Who cares? What does this mean? and Why is this in the news?Because Music is important to people
  4. V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. expressive
  5. MER and the tasks mentioned is a very young field. Psychology longer. Going to give a history in a few slides
  6. Recently updated literature review. Reason Mention Survey of 62 conference articles and journal papersMany different disciplines Computer Science and engineering
  7. Recently updated literature review. Reason Mention Survey of 62 conference articles and journal papersMany different disciplines Computer Science and engineering