Ami lunch apo v0.6

1. A Musical System For Emotional Expression
António Pedro Oliveira
University of Coimbra, Portugal

2. 2
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

3. 3
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

4. Motivation
 Emotions are widely accepted as being an
important factor in the society
 Music is almost everywhere and it is a powerful
stimulus capable of influencing our emotions
 Computational systems with the capability of
producing music with an appropriate emotional
content have an enormous application potential
4
Introduction Background EDME Conclusion

5. Aim
 Conceive a computational system for the
control of the emotional content of
produced music, so that it expresses a given
emotional specification
 Music: solely instrumental
5
Introduction Background EDME Conclusion

6. 6
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

7. Emotional Expression with
Music
 Four approaches:
 Music Transformation
 Music Composition
 Music Selection/Classification
 Hybrid Approaches
 Our approach:
 Hybrid that consists in combining
selection/classification with transformation
7
Introduction Background EDME Conclusion

8. 8
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

9. Architecture –
Offline Stage
9
Introduction Background EDME Conclusion

10. Architecture –
Online Stage
10
Introduction Background EDME Conclusion

11. Experiments
 Initial phase
11
Introduction Background EDME Conclusion

12. Initial phase
 Manually Built Knowledge Base (short
version)
 Happy music: high loudness, major scale
 Sad music: violin, slow tempo
 Activating music: high loudness, fast tempo
 Relaxing music: low loudness, slow tempo
12
Introduction Background EDME Conclusion

13. Experiments
 Stages of the experiments
13
Introduction Background EDME Conclusion

14. Experiments
 First Experiment - Preliminary Evaluation of
the Classification Module
 Hypothesis: There is a small amount of features
that may predict arousal/valence
 Valence: 2 features, CC – 0.76
 Arousal: 4 features, CC – 0.77
14
Introduction Background EDME Conclusion

15. Experiments
 Second experiment (two parts)
 First part - Extended Evaluation of the Classification
Module
 Hypothesis: There is a small amount of features that
may predict arousal/valence
 Valence: 4 features, CC – 0.70
 Arousal: 3 features, CC – 0.77
15
Introduction Background EDME Conclusion

16. Experiments
 Second part - Analysis of Audio Features
 Hypothesis: There are audio features emotionally-
relevant
 Valence: Spectral Sharpness and Loudness are important
 Arousal: Spectral Similarity, Spectral Dissonance and
Spectral Sharpness are important
16
Introduction Background EDME Conclusion

17. Experiments
 Third experiment (three parts)
 First part - Improvement of the Classification Module
 Hypothesis: There is a small amount of features that
may predict arousal/valence
 Valence: 5 features, CC – 0.69
 Arousal: 3 features, CC – 0.71
17
Introduction Background EDME Conclusion

18. Experiments
 Second part – Evaluation of the Transformation
Algorithms
 Hypothesis: It is possible to change musical features to
transform emotional content
 High positive coefficients for tempo were confirmed
 The increase of register correlates positively with
valence and negatively with arousal
 Some of the features can be helpful in finding scales more
appropriate to some emotions
 Instruments are essentially relevant to the arousal
 Change from normal to staccato articulation has a
correlation with the increase of valence
18
Introduction Background EDME Conclusion

19. Experiments
 Third part – Melodic Analysis
 Hypothesis: The analysis of the melodic line alone turns
the emotionally-relevant features more visible
19
Introduction Background EDME Conclusion
Emotional Dimension CC Features
Valence – data of first experiment 0.79 5 features
Valence – data of second
experiment
0.62 5 features
Valence – data of third experiment 0.41 4 features
Arousal – data of first experiment 0.85 4 features
Arousal – data of second
experiment
0.72 4 features
Arousal – data of third experiment 0.54 3 features

20. Calibration and Validation
 Hypothesis: 13 features are enough to
discriminate emotional expression
 Valence: 7 features, CC – 0.85
 Arousal: 6 features, CC – 0.83
 Use SAM to obtain emotional answers
 Controlled environment
 Statistical analysis:
 System’s classification and subject’s classification
are probably measuring the same concept
20
Introduction Background EDME Conclusion

21. 21
Outline of the Presentation
 Introduction
 Background
 Emotion-Driven Music Engine (EDME)
 Conclusion

22. Contributions
 The system proposed has the advantage of
being able to produce outputs of acceptable
quality quite independently from the music
base
 It is also quite flexible: the music base can be
completely redefined to adapt to the specific
needs of a given scenario
22
Introduction Background EDME Conclusion

23. Contributions
 The system is also reliable, thanks to the
experimental calibration using different
subjects
 We adopted both the discrete and
dimensional representation of emotions
 We used techniques of human emotional
recognition for validation and calibration of
the system
23
Introduction Background EDME Conclusion

24. Future Work
 Emotion-Driven Music Composition
 Artificial Intelligence approaches
 Test in applications contexts
 Healthcare
 Entertainment
24
Introduction Background EDME Conclusion

25. The End
25