The document describes a musical system called the Emotion-Driven Music Engine (EDME) that is able to produce instrumental music with a specified emotional content. The system uses a hybrid approach that combines music selection/classification with music transformation. It was tested through experiments that evaluated its ability to accurately classify and transform music features related to emotional dimensions like valence and arousal. The results showed the system could reliably discriminate between emotions and produce emotionally expressive music, with contributions including its flexibility, reliability through experimental calibration, and adoption of both discrete and dimensional emotion representation.

1. A Musical System For Emotional Expression
António Pedro Oliveira
University of Coimbra, Portugal

2. 2
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

3. 3
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

4. Motivation
 Emotions are widely accepted as being an
important factor in the society
 Music is almost everywhere and it is a powerful
stimulus capable of influencing our emotions
 Computational systems with the capability of
producing music with an appropriate emotional
content have an enormous application potential
4
Introduction Background EDME Conclusion

5. Aim
 Conceive a computational system for the
control of the emotional content of
produced music, so that it expresses a given
emotional specification
 Music: solely instrumental
5
Introduction Background EDME Conclusion

6. 6
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

7. Emotional Expression with
Music
 Four approaches:
 Music Transformation
 Music Composition
 Music Selection/Classification
 Hybrid Approaches
 Our approach:
 Hybrid that consists in combining
selection/classification with transformation
7
Introduction Background EDME Conclusion

8. 8
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

9. Architecture –
Offline Stage
9
Introduction Background EDME Conclusion

10. Architecture –
Online Stage
10
Introduction Background EDME Conclusion

11. Experiments
 Initial phase
11
Introduction Background EDME Conclusion

12. Initial phase
 Manually Built Knowledge Base (short
version)
 Happy music: high loudness, major scale
 Sad music: violin, slow tempo
 Activating music: high loudness, fast tempo
 Relaxing music: low loudness, slow tempo
12
Introduction Background EDME Conclusion

13. Experiments
 Stages of the experiments
13
Introduction Background EDME Conclusion

14. Experiments
 First Experiment - Preliminary Evaluation of
the Classification Module
 Hypothesis: There is a small amount of features
that may predict arousal/valence
 Valence: 2 features, CC – 0.76
 Arousal: 4 features, CC – 0.77
14
Introduction Background EDME Conclusion

15. Experiments
 Second experiment (two parts)
 First part - Extended Evaluation of the Classification
Module
 Hypothesis: There is a small amount of features that
may predict arousal/valence
 Valence: 4 features, CC – 0.70
 Arousal: 3 features, CC – 0.77
15
Introduction Background EDME Conclusion

16. Experiments
 Second part - Analysis of Audio Features
 Hypothesis: There are audio features emotionally-
relevant
 Valence: Spectral Sharpness and Loudness are important
 Arousal: Spectral Similarity, Spectral Dissonance and
Spectral Sharpness are important
16
Introduction Background EDME Conclusion

17. Experiments
 Third experiment (three parts)
 First part - Improvement of the Classification Module
 Hypothesis: There is a small amount of features that
may predict arousal/valence
 Valence: 5 features, CC – 0.69
 Arousal: 3 features, CC – 0.71
17
Introduction Background EDME Conclusion

18. Experiments
 Second part – Evaluation of the Transformation
Algorithms
 Hypothesis: It is possible to change musical features to
transform emotional content
 High positive coefficients for tempo were confirmed
 The increase of register correlates positively with
valence and negatively with arousal
 Some of the features can be helpful in finding scales more
appropriate to some emotions
 Instruments are essentially relevant to the arousal
 Change from normal to staccato articulation has a
correlation with the increase of valence
18
Introduction Background EDME Conclusion

19. Experiments
 Third part – Melodic Analysis
 Hypothesis: The analysis of the melodic line alone turns
the emotionally-relevant features more visible
19
Introduction Background EDME Conclusion
Emotional Dimension CC Features
Valence – data of first experiment 0.79 5 features
Valence – data of second
experiment
0.62 5 features
Valence – data of third experiment 0.41 4 features
Arousal – data of first experiment 0.85 4 features
Arousal – data of second
experiment
0.72 4 features
Arousal – data of third experiment 0.54 3 features

20. Calibration and Validation
 Hypothesis: 13 features are enough to
discriminate emotional expression
 Valence: 7 features, CC – 0.85
 Arousal: 6 features, CC – 0.83
 Use SAM to obtain emotional answers
 Controlled environment
 Statistical analysis:
 System’s classification and subject’s classification
are probably measuring the same concept
20
Introduction Background EDME Conclusion

21. 21
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

22. Contributions
 The system proposed has the advantage of
being able to produce outputs of acceptable
quality quite independently from the music
base
 It is also quite flexible: the music base can be
completely redefined to adapt to the specific
needs of a given scenario
22
Introduction Background EDME Conclusion

23. Contributions
 The system is also reliable, thanks to the
experimental calibration using different
subjects
 We adopted both the discrete and
dimensional representation of emotions
 We used techniques of human emotional
recognition for validation and calibration of
the system
23
Introduction Background EDME Conclusion

24. Future Work
 Emotion-Driven Music Composition
 Artificial Intelligence approaches
 Test in applications contexts
 Healthcare
 Entertainment
24
Introduction Background EDME Conclusion

25. The End
25