Music Emotion RecognitionA State of the Art ReviewDr Scott BeveridgeInterdisciplinary Perspectives on Music, Emotion and T...
Outline Why emotion? Definition of Music Emotion Recognition History and motivation for interdisciplinary approach Cur...
Why emotion?Emotion and Meaning in Music© Fraunhofer IDMT
Emotion and Meaning in Music Humans treat music with great importance Music has a very powerful effect            One o...
Emotion and Meaning in Music Emotion is a great way of creating information which facilitates browsing and  organization ...
Emotion and Meaning in Music Pin down some aspects of the affective process to make the problem  computationally tractabl...
A Definition of Music EmotionRecognition (MER) MER has two steps           1. Identification, recognition and extraction ...
Emotion Recognition       Happy = Fast Tempo,Supervised Learning               Major Mode                      Sad = Slow ...
What can machines learn?Conceptualization of Emotion Assign emotion labels (Classification)            Previous example ...
A Brief History of MERBackground Music Psychology           Engineering Cate Hevner        1935                    1988   ...
A Brief History of MERPopular Music Popular music is becoming…..popular!© Fraunhofer IDMT
Popular MusicChallenges By definition popular music is            Made commercially                 Limits the scope of...
A Brief History of MERWhy Interdisciplinary? Olighara 2003            “One 39 year old male Chinese” annotator for a cor...
Current Experiments Based on 2 steps in MER            Features                 Tone Objects                 Statistic...
Current ExperimentsTone Objects Objective: Find novel ways of describing popular music by creating new             musica...
Current ExperimentsTone Objects Extracting tone objects involves  many signal processing  techniques            Source s...
Current ProjectsMain Melody Statistics In linguistics, Zipf’s Law shows that: [CAREFUL! These figures might be  incorrect...
Current ProjectsTension Prediction Objective: Track time-continuous tension gradients in film music Applications in sync...
Current ProjectsTension Prediction – Feature Extraction Step1: Extract time-continuous features© Fraunhofer IDMT
Current ProjectsTension Prediction – Participant Testing Step 2: Asked participants to rate music  based on perceived ten...
Current ProjectsTension Prediction – Participant Testing Features most correlated with tension:            Timbral Compl...
Current ProjectsTension Prediction – Demonstration© Fraunhofer IDMT
The Future of MER Automatic MER systems are only the beginning For MER systems to be truly effective it is necessary to ...
Music Emotion RecognitionA State of the Art Review                         Thank you !!                      bevest@idmt.f...
Upcoming SlideShare
Loading in …5
×

Interdisciplinary Perspectives on Emotion, Music and Technology

406 views
311 views

Published on

Recent EPSRC funded research by audio technology staff at GCU has resulted in the development of links with researchers at the cutting edge of music emotion and technology research. The seminar will consist of three talks on this exciting cross-disciplinary area.


"Is Music Ever Sad?" Barry Maguire MA, Department of Philosophy, Princeton University

"Music Emotion Recognition, A State of the Art Review", Dr Scott Beveridge, Semantic Music Technologies, Fraunhofer Institute of Digital Media Technology

"Practical Applications of Semantic Music Technologies", Jakob Abesser Dipl.-Ing. Semantic Music Technologies, Fraunhofer Institute of Digital Media Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
406
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. expressive
  • V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. expressive
  • V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. Expressive<<David Cameron’s Fav album>>Who cares? What does this mean? and Why is this in the news?Because Music is important to people
  • V. Moving video. Most interesting part is the music has some ‘residual effect’ – Henry is animated, lucid …. expressive
  • MER and the tasks mentioned is a very young field. Psychology longer. Going to give a history in a few slides
  • Recently updated literature review. Reason Mention Survey of 62 conference articles and journal papersMany different disciplines Computer Science and engineering
  • Recently updated literature review. Reason Mention Survey of 62 conference articles and journal papersMany different disciplines Computer Science and engineering
  • Interdisciplinary Perspectives on Emotion, Music and Technology

    1. 1. Music Emotion RecognitionA State of the Art ReviewDr Scott BeveridgeInterdisciplinary Perspectives on Music, Emotion and TechnologyGlasgow Caledonian University, 25th June 2012© Fraunhofer IDMT
    2. 2. Outline Why emotion? Definition of Music Emotion Recognition History and motivation for interdisciplinary approach Current experiments© Fraunhofer IDMT
    3. 3. Why emotion?Emotion and Meaning in Music© Fraunhofer IDMT
    4. 4. Emotion and Meaning in Music Humans treat music with great importance Music has a very powerful effect  One of the primary reasons behind humans enjoyment of music Henry is an example of the listener perspective  Music is useful for those who compose and perform also  Method of communication and expression© Fraunhofer IDMT
    5. 5. Emotion and Meaning in Music Emotion is a great way of creating information which facilitates browsing and organization of music  Vast number of audio tracks online Why are we doing this? MER Applications  Music organization and browsing – personal and commercial  Academic – Music Digital Libraries (MDL), archiving  Health – music therapy, pain management (Knox 2011)© Fraunhofer IDMT
    6. 6. Emotion and Meaning in Music Pin down some aspects of the affective process to make the problem computationally tractable  Expression versus Induction Most work in MER focusses on expression  „What the music is trying to say to you‟  This is easier to decide on with some types of music than others  Film music© Fraunhofer IDMT
    7. 7. A Definition of Music EmotionRecognition (MER) MER has two steps 1. Identification, recognition and extraction of musical characteristics which express emotions in music 2. Modelling these characteristics in order to make prediction on emotions expressed by „new‟ music© Fraunhofer IDMT
    8. 8. Emotion Recognition Happy = Fast Tempo,Supervised Learning Major Mode Sad = Slow Tempo, Happy Minor Mode Sad ?© Fraunhofer IDMT
    9. 9. What can machines learn?Conceptualization of Emotion Assign emotion labels (Classification)  Previous example  Exuberant, Anxious, Depressed, Content Define a point in 2D space (Numeric Prediction) Predict emotions which vary over time (Time-continuous prediction)© Fraunhofer IDMT
    10. 10. A Brief History of MERBackground Music Psychology Engineering Cate Hevner 1935 1988 Kayatose (Symbolic) Patrik Juslin 2001 1935 2003 Feng (Signal-based) NOW© Fraunhofer IDMT
    11. 11. A Brief History of MERPopular Music Popular music is becoming…..popular!© Fraunhofer IDMT
    12. 12. Popular MusicChallenges By definition popular music is  Made commercially  Limits the scope of expressed emotions  Made using ever-changing technologies  Over production (dynamic compression) This generally leads to homogeneity in the popular music genre To overcome these problems psychologists, musicologists, philosophers, and engineers must work together© Fraunhofer IDMT
    13. 13. A Brief History of MERWhy Interdisciplinary? Olighara 2003  “One 39 year old male Chinese” annotator for a corpus of Western contemporary popular music Wu 2006  10 second music clips and no mention of music Schellenberg12  Manual tempo calculation Yang07  Expression/Induction distinction© Fraunhofer IDMT
    14. 14. Current Experiments Based on 2 steps in MER  Features  Tone Objects  Statistical properties of melody  Modelling  Predict tension gradients for use in syncronisation  Includes creation of new features  Feature fusion© Fraunhofer IDMT
    15. 15. Current ExperimentsTone Objects Objective: Find novel ways of describing popular music by creating new musical features Existing features  Tempo, Mode, Key, Instrument Timbre New features  Examine existing features based on tone objects  Musical notes of the main melody© Fraunhofer IDMT
    16. 16. Current ExperimentsTone Objects Extracting tone objects involves many signal processing techniques  Source separation  Computational Auditory Scene Analysis (CASA)  Identify the main melody Results shows that tone objects help identify particular types of emotion© Fraunhofer IDMT
    17. 17. Current ProjectsMain Melody Statistics In linguistics, Zipf’s Law shows that: [CAREFUL! These figures might be incorrect] Given some corpus of natural language the frequency of any word is inversely proportional to its rank in the frequency table Word # of occurrences Word # of occurrences the 69,971 Unison 69,971 of 36,411 Major 3rd 36,411 and 28,852 Perfect 5th 28,852 Studies1 have shown that Zipf law statistics have a relationship with aesthetic aspects of music – pleasant, beautiful Can Zipf‟s law statistics be applied in emotion classification? 1 http://sger.cs.cofc.edu/© Fraunhofer IDMT
    18. 18. Current ProjectsTension Prediction Objective: Track time-continuous tension gradients in film music Applications in syncronization task  Helps creators of films and adverts find music with specific characteristics Approach:  Step 1: Extract time-continuous features from a collection of film music  Step 2: Conduct a study which asks people to rate time-continuous tension  Step 3: Build models with the data which predicts tension gradients in new music An example of supervised learning!© Fraunhofer IDMT
    19. 19. Current ProjectsTension Prediction – Feature Extraction Step1: Extract time-continuous features© Fraunhofer IDMT
    20. 20. Current ProjectsTension Prediction – Participant Testing Step 2: Asked participants to rate music based on perceived tension General agreement© Fraunhofer IDMT
    21. 21. Current ProjectsTension Prediction – Participant Testing Features most correlated with tension:  Timbral Complexity: The rate of change of timbre (How many „different sounding‟ instruments are present  Spectral Dissonance: Perceived roughness  Pure Tonalness: A measure of how „tone-like‟ a sound is© Fraunhofer IDMT
    22. 22. Current ProjectsTension Prediction – Demonstration© Fraunhofer IDMT
    23. 23. The Future of MER Automatic MER systems are only the beginning For MER systems to be truly effective it is necessary to adopt a user-centred approach  Emotions elicited in music are created through social factors and environment  Listening with friends  Listening on the way to work Profile users to create bespoke emotion recommendation systems based on  Geo-location, time of day, skipping behaviour© Fraunhofer IDMT
    24. 24. Music Emotion RecognitionA State of the Art Review Thank you !! bevest@idmt.fraunhofer.de http://tinyurl.com/bevestLinkedIn© Fraunhofer IDMT

    ×