The document proposes a computer system to control affective content in music production. The system would have four main stages: 1) represent emotions in a two-dimensional or lexical space, 2) systematize relationships between music features and emotions, 3) segment and extract features from music, and 4) select and manipulate pre-composed music to match intended emotional descriptions. The emotional output of the manipulated music would be validated through experiments collecting physiological and subjective feedback to refine the system's mappings between music features and emotions.

1. António Pedro OliveiraAntónio Pedro Oliveira
and Amílcar Cardosoand Amílcar Cardoso
06/12/200706/12/2007
A Computer System to Control
Affective Content in Music
Production

2. António Pedro Oliveira
and Amílcar Cardoso
2
OutlineOutline
 IntroductionIntroduction
 Proposed SystemProposed System
 Emotional output validationEmotional output validation
 Prototype (demo)Prototype (demo)
 ConclusionConclusion

3. António Pedro Oliveira
and Amílcar Cardoso
3
OutlineOutline
 IntroductionIntroduction
 Proposed SystemProposed System
 Emotional output validationEmotional output validation
 Prototype (demo)Prototype (demo)
 ConclusionConclusion

4. António Pedro Oliveira
and Amílcar Cardoso
4
IntroductionIntroduction
 Problem
 No automatic method to express and induce an
intended emotional description with music
 Objective
 Design, implement and assess a computer
system that can control the affective content of
pre-composed music represented at a symbolic
level

5. António Pedro Oliveira
and Amílcar Cardoso
5
Introduction – Topics of ResearchIntroduction – Topics of Research
 Music and emotions psychology
 Relations between emotions and musical
features
 Affective computingAffective computing
 Techniques to recognize emotions
 Music computingMusic computing
 Approaches to generate affective music

6. António Pedro Oliveira
and Amílcar Cardoso
6
OutlineOutline
 IntroductionIntroduction
 Proposed SystemProposed System
 Emotional output validationEmotional output validation
 Prototype (demo)Prototype (demo)
 ConclusionConclusion

7. António Pedro Oliveira
and Amílcar Cardoso
7
Proposed System - ArchitectureProposed System - Architecture
(first stage)(first stage)

8. António Pedro Oliveira
and Amílcar Cardoso
8
Emotions representationEmotions representation
 2 Dimensional and Lexical space2 Dimensional and Lexical space

9. António Pedro Oliveira
and Amílcar Cardoso
9
Proposed System - ArchitectureProposed System - Architecture
(second stage)(second stage)

10. António Pedro Oliveira
and Amílcar Cardoso
10
Computational systematization –Computational systematization –
emotions and musicemotions and music
 Systematize relations between musicalSystematize relations between musical
features in a computational contextfeatures in a computational context

11. António Pedro Oliveira
and Amílcar Cardoso
11
Proposed System - ArchitectureProposed System - Architecture
(third stage)(third stage)

12. António Pedro Oliveira
and Amílcar Cardoso
12
Music SegmentationMusic Segmentation
 Segment music to allow a betterSegment music to allow a better
control of affective contentcontrol of affective content

13. António Pedro Oliveira
and Amílcar Cardoso
13
Music Features ExtractionMusic Features Extraction
 Extract features to relate withExtract features to relate with
emotionsemotions

14. António Pedro Oliveira
and Amílcar Cardoso
14
Proposed System - ArchitectureProposed System - Architecture
(fourth stage)(fourth stage)

15. António Pedro Oliveira
and Amílcar Cardoso
15
Control of affective content –Control of affective content –
music selection and manipulationmusic selection and manipulation
 Selection and manipulation of pre-
composed symbolic music

16. António Pedro Oliveira
and Amílcar Cardoso
16
Music SelectionMusic Selection
 Select relations from the knowledge base by matching theSelect relations from the knowledge base by matching the
emotional descriptionemotional description
 Select music from the music base according to selectedSelect music from the music base according to selected
relationsrelations
 Similarity metrics among music features (melody, rhythm, etc.)

17. António Pedro Oliveira
and Amílcar Cardoso
17
Music ManipulationMusic Manipulation
 Focus of the workFocus of the work

18. António Pedro Oliveira
and Amílcar Cardoso
18
Music SequencingMusic Sequencing
 Music sequencing is done with theMusic sequencing is done with the
help of k-means clusteringhelp of k-means clustering
 e.g., by note density (3 clusters)

19. António Pedro Oliveira
and Amílcar Cardoso
19
Music transformationMusic transformation
 Produce a set of musical segmentsProduce a set of musical segments
that sound fluentthat sound fluent

20. António Pedro Oliveira
and Amílcar Cardoso
20
Inclusion of composition andInclusion of composition and
arrangementarrangement
 Possible use of third party software /Possible use of third party software /
algorithms of composition andalgorithms of composition and
arrangement to manipulate musicarrangement to manipulate music

21. António Pedro Oliveira
and Amílcar Cardoso
21
OutlineOutline
 IntroductionIntroduction
 Proposed SystemProposed System
 Emotional output validationEmotional output validation
 Prototype (demo)Prototype (demo)
 ConclusionConclusion

22. António Pedro Oliveira
and Amílcar Cardoso
22
Emotional output validationEmotional output validation
 Assessment methodologyAssessment methodology
 Produced music is played in experimental
contexts (e.g., clinical or entertainment)
 Data is collected about the perceived and
induced emotions
 Facial electromyography, skin conductance, heart rate
measures and questionnaires

23. António Pedro Oliveira
and Amílcar Cardoso
23
Emotional output validationEmotional output validation
 Recognized emotional state is compared with
the intended emotional state
 Comparisons will be used to refine the mappings
in the knowledge base
 New mappings can be added; old mappings can be
adapted or discarded with Case-Based or Rule-Based
techniques

24. António Pedro Oliveira
and Amílcar Cardoso
24
OutlineOutline
 IntroductionIntroduction
 Proposed SystemProposed System
 Emotional output validationEmotional output validation
 Prototype (demo)Prototype (demo)
 ConclusionConclusion

25. António Pedro Oliveira
and Amílcar Cardoso
25
Prototype - DemoPrototype - Demo

26. António Pedro Oliveira
and Amílcar Cardoso
26
ExamplesExamples
 Manipulated music samplesManipulated music samples
 Different affective content (e.g., timbre,
loudness, tempo and register)
 http://student.dei.uc.pt/~apsimoes/PhD/Music/epia200
 High arousal
 High valence
 Low arousal
 Low valence

27. António Pedro Oliveira
and Amílcar Cardoso
27
OutlineOutline
 IntroductionIntroduction
 Proposed SystemProposed System
 Emotional output validationEmotional output validation
 Prototype (demo)Prototype (demo)
 ConclusionConclusion

28. António Pedro Oliveira
and Amílcar Cardoso
28
ConclusionConclusion
 Design a computational model to control
music affective content grounded on
psychological research
 Validate the model with users to promote a
reliable expression and induction of emotions
by using music
 System can be applied in areas that intend to
produce music given an emotional input
 Music Therapy, soundtracks for arts, movies, dance,
theater, virtual environments, computer games, etc.