The document discusses how human listening and perception of sound is influenced by numerous cultural and contextual factors. It explores how culture can impact the processing of auditory stimuli and affect how people analyze complex sound scenes. The document also examines how cultural background shapes musical expectations and influences judgments of qualities like mood, similarity, and genre. It suggests computational models of audition should account for these cultural influences to better reflect human perception.

1. The impact of Cultural Context on the
Perception of Sound and Musical Language
(and its relevance for the development of Computational Models of Audition)
The Lisbon Consortium
Luís Gustavo Martins
lmartins@porto.ucp.pt
Lisbon, June 26th 2013

2. CATÓLICA'PORTO+
ARTES+
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

3. CATÓLICA'PORTO+
ARTES+
Outline
• Human listening comprises many layers of information analysis and processing.
• From the transduction of low-level auditory stimuli in the ears…
• … to their organization in the brain,
• higher-level factors take an important role in the auditory process:
• Memory
• Attention
• Experience
• Culture
• Context
• …
• As a result, listening in general is influenced by a large number of variables
interacting, cooperating and competing with each other in a rather complex network.
• Recent research in the field of computational sound and music analysis is now more
aware of the impact of the cultural background in the perception of sound and music.

4. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
Sensation
• neurological process by which we become aware of our environment
(Jandt, 2012).
• You are not directly aware of what is in the physical world…
• … but, rather, of your own internal sensations.
• Is variation in human sensation attributable to culture?
Source: h"p://www.sagepub.com/upm1data/45975_Chapter_3.pdf=

5. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?

6. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
“Kids are encouraged to have fun outdoors, playing in the nature.”

7. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
“Kids are encouraged to have fun outdoors, playing in the nature.”

8. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
“Unattended kids may become extremely violent when playing outdoors!”

9. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
“Unattended kids may become extremely violent when playing outdoors!”

10. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
Dogs in the Western Culture
Source: http://www.sagepub.com/upm-data/45975_Chapter_3.pdf
domesticated animal life—probably into animals used for
recreation, and pets. Look at the picture of the puppy and c
Most of us see this puppy in the category of pet, for whic
loving feelings. Puppies are cute, cuddly, warm, loving crea
of a man holding u
capture your feeling
this picture uncomf
people eat dogs? The
on where you catego
cultures and food in o
acceptable as watchd
not kept in the home
unclean and a low fo
is an insult among Ar
strong ideas about
human consumptio
some countries think
of eating corn on th
food is fit only for p
salo, raw pig fat wit
might cause nausea
horse meat from Cal
Belgium, France, and
Many consider dogs as pets. (The
author’s first dog, Smokey.)

11. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
Dogs in the Eastern Culture
Source: http://www.sagepub.com/upm-data/45975_Chapter_3.pdf
CHAPTER 3 Culture’s Influence on Perception 67
Your reaction of disgust to the picture is a culturally learned interpretation—and that inter-
pretation can be quite strong. In 1989, California made it a misdemeanor for any person to
Can you explain your feelings about this photograph? As China’s economy boomed and
affluence spread, attitudes toward dogs changed. Traditional Chinese may have eaten dog
meat because it was thought to improve blood circulation. Urban Chinese today are more
likely to have dogs as pampered companions.

12. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
What do you (can you!) “see” here?

13. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
Humans (independent of their cultural context) develop, from early ages, a
“model” for “the human face”... So maybe, you see a face...

14. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
Or, maybe you know what a saxophone looks like, and how it is (usually) played...

15. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
What if you never left your native tribe, deep in the African savana?
Can you still perceive something from the figure?

16. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
Or maybe you are a kid with strong “cultural” references from cartoons?
(and a large imagination)

17. CATÓLICA'PORTO+
ARTES+
Can Culture and Context affect our Perception?
• Nisbett (2003) has demonstrated that:
• humans sense and perceive the world in ways unique to their
environments
• by contrasting Eastern and Western cultures.
• Perception and thought are not independent of the cultural environment
• therefore, our brains are both shaped by the external world and shape
our perception of the external world.
• Sensation is the neurological process of becoming aware of our
environment and is affected by our cultures (Jandt, 2012).
Source: h"p://www.sagepub.com/upm1data/45975_Chapter_3.pdf=

18. CATÓLICA'PORTO+
ARTES+
Sensing and Perceiving the World around us
A World of Sounds

19. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Sound production and propagation

20. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Sound propagation – waves of energy

21. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Sound sensors: the Human Ear THE HUMAN AUDITORY SYSTEM
Figure 1: A sketch of the peripheral region of the human auditory system (adapted
from [Beranek, 1990]).
a pre-processing of the acoustical signal, the most notable aspect being frequency analysis.

22. CATÓLICA'PORTO+
ARTES+
A world of Sounds
Sound sensors: the Human Ear

23. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Beyond sensing: higher order Auditory processes and their interactions
MUSIC LISTENING
ABSTRACT
KNOWLEDGE
STRUCTURES
EVENT
STRUCTURE
PROCESSING
EXTRACTION
OF
ATTRIBUTES
AUDITORY
GROUPING
PROCESSES
MENTAL
REPRESENTATION OF
SOUND
ENVIRONMENT
TRANSDUCTION
TRANSDUCTION
ATTENTIONAL
PROCESSES
Figure 2: The main types of auditory processing and their interactions (adapted
from [McAdams and Bigand, 1993]).
possible to extract perceptual attributes which provide a representation of each element in

24. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Auditory Scene Analysis (ASA) - The lake analogy…
Source: http://www.cns.nyu.edu/~david/courses/perception/lecturenotes/localization/localization.html

25. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Auditory Scene Analysis – The Cocktail Party Effect
Source: http://online.wsj.com/article/SB10001424052702303459004577361850069498164.html

26. CATÓLICA'PORTO+
ARTES+
A World of Sounds
Auditory Scene Analysis - Polyphonic, multitimbral, complex musical compositions
• Listeners can still identify:
• Melodic lines
• Instruments (or groups of instruments playing)
• Harmony
• Beat
• Rhythm
• …

27. CATÓLICA'PORTO+
ARTES+
Perception Principles
The Gestalt Theory of Perception

28. CATÓLICA'PORTO+
ARTES+
Perception Principles
The Gestalt Theory of Perception

29. CATÓLICA'PORTO+
ARTES+
Perception Principles
Auditory Scene Analysis (ASA)
- the Perceptual Organization of Sound
http://webpages.mcgill.ca/staff/Group2/abregm1/web/

30. CATÓLICA'PORTO+
ARTES+
Perception Principles
Segregation of a melody from interfering tones
• Bottom-up VS Top-Down approaches
Source: http://webpages.mcgill.ca/staff/Group2/abregm1/web/downloadstoc.htm
“At the intermediate separations, familiarity with the melody can sometimes
allow listeners to hear it out, but there is a still a tendency to hear runs of tones
that group melodic and distractor notes.”

31. CATÓLICA'PORTO+
ARTES+
Music Perception
How is Similarity defined, for Music?
(And what impact has cultural context and background on its definition?)
Query:
Similar
Music?

32. CATÓLICA'PORTO+
ARTES+
Music Perception
How is Similarity defined, for Music?
(And what impact has cultural context and background on its definition?)
“- This all sounds like Fado to my hears…”
“- Not to me… I’m a Fado singer…I can tell a difference!”
=

33. CATÓLICA'PORTO+
ARTES+
Music Perception
How is Similarity defined, for Music?
(And what impact has cultural context and background on its definition?)
Saudades de Coimbra
(Fado) - Verdes Anos
Amália Rodrigues -
Fado Português
João Braga - Fado de
Lisboa
“- This all sounds like Fado to my hears…”
“- Not to me… I’m a Fado singer…I can tell a difference!”
=

34. CATÓLICA'PORTO+
ARTES+
Music Perception
How is Similarity defined, for Music?
(And what impact has cultural context and background on its definition?)
“- This all sounds like Techno to my hears…”
“ – No to me… I’m a DJ… I can tell a difference!”

35. CATÓLICA'PORTO+
ARTES+
Music Perception
How is Similarity defined, for Music?
(And what impact has cultural context and background on its definition?)
“- This all sounds like Techno to my hears…”
“ – No to me… I’m a DJ… I can tell a difference!”
The Big Fake
Traffic Signs
(ELECTRO)
Je T'aime (Digital Dog mix)
Armand Van Helden
(DEEP HOUSE)
Exorcise (A1)
Umek
(TRANCE)
Memories
David Guetta
(COMMERCIAL
HOUSE)

36. CATÓLICA'PORTO+
ARTES+
Music Perception
“Music is different things and does different things in different cultures; the
bundles of elements and functions which are music for any given culture may
overlap minimally with those of another culture, even for those cultures where
"music" constitutes a discrete and identifiable category of human activity in its
own right.” (Cross, 2001)
http://techno.org/electronic-music-guide/
- Expert listeners in a genre, or in a
given culture can perceive subtle
variances in a music genre, and
identify sub-genres, artist styles,
etc.
- They make part of a “culture”
around that musical genre / style
- “Music", like speech, is a
product of both our biologies
and our social interactions
(Cross, 2001)

37. CATÓLICA'PORTO+
ARTES+
Music Perception
How is Similarity defined, for Music?
(And what impact has cultural context and background on its definition?)
• It is an ill–posed question…
• It is multidimensional, may use different and simultaneous criteria, such as:
• Pitch
• Harmony
• Timbre
• Rhythm
• Genre
• Texture
• …
• It may depend on the context of the query
• Difficult to infer…
• It will certainly be culturally biased
• Difficult to match expectations

38. CATÓLICA'PORTO+
ARTES+
Effects of Culture on Music Perception
• Perception of Pitch (Wong et al., 2012)
• Studies on Chinese and Canadian population with Amusia
• Amusia: a neurogenetic disorder affecting the music (pitch and rhythm) processing
• Contonese is a tone language, Canadian French and English are not tone languages
• Cantonese speakers as a group tend to show enhanced pitch perception ability compared to
speakers of Canadian French and English.
• This enhanced ability occurs in the absence of differences in rhythmic perception and persists
even after relevant factors such as musical background and age were controlled.
• These findings not only provide critical evidence for a double association of music and
speech, but also argue for the reconceptualization of communicative disorders within a
cultural framework.
• Along with recent studies documenting cultural differences in visual perception, our
auditory evidence challenges the common assumption of universality of basic mental
processes and speaks to the domain generality of culture-to-perception influences.
Source: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0033424

39. CATÓLICA'PORTO+
ARTES+
Effects of Culture on Music Perception
• Memory and Musical Expectation for Tones (Curtis and Bharucha, 2009)
• Explored how musical culture shapes one’s listening experience.
• Western participants heard a series of tones drawn from either:
• the Western major mode (culturally familiar)
• or the Indian thaat Bhairav (culturally unfamiliar)
• and then heard a test tone.
• They made a speeded judgment about whether the test tone was present in the prior series
of tones.
• False alarm rates were higher for test tones congruent with the major mode than for
test tones congruent with Bhairav.
• In contrast, false alarm rates were lower for test tones incongruent with the major mode
than for test tones incongruent with Bhairav.
• These findings suggest that one’s internalized cultural knowledge may drive musical
expectancies when listening to music of an unfamiliar modal system.
Source: http://openscholar.purchase.edu/meagan_curtis/files/curtisbharucha2009musicperception.pdf

40. CATÓLICA'PORTO+
ARTES+
Effects of Culture on Music Perception
• A cross cultural study on Music Mood Perception (Hu and Lee, 2012)
• People's cultural background is often assumed to be an important factor in music mood
perception.
• This paper reports on a study comparing mood judgments on a set of 30 songs by American
and Chinese people.
• Results show that mood judgments do indeed differ between American and Chinese
respondents.
• Furthermore, respondents’ mood judgments tended to agree more with other
respondents from the same culture than those from the other group.
Source: http://ismir2012.ismir.net/event/papers/535-ismir-2012.pdf

41. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Typical Pipeline
Hearing
Representation
Analysis
Understanding
Acting
Interaction
Signal Processing
Feature Extraction
Machine Learning
Analysis and Retrieval
Human
Computer
Interaction

42. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Different perspectives involved
Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
TECHNOLOGICAL PERSPECTIVE
2.1
Musically
relevant data
2.2
Music
representations
Models
2.3
Data processing
methdologies
2.4
Knowledge-driven
methdologies
2.5
Estimation of
elements related to
musical concepts
Estimated
musical
concepts
Other
estimated
concepts
2.6
Evaluation
methodologies
SOCIO-CULTURAL PERSPECTIVE
4.1
Social aspects
4.2
Multiculturality
USER PERSPECTIVE
3.1
User behaviour
3.2
User interaction
EXPLOITATION PERSPECTIVE
5.1
Music distribution
applications
5.2
Creative tools
5.3
Other exploitation areas

43. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Technological Perspective:

44. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
TECHNOLOGICAL PERSPECTIVE
2.1
Musically
relevant data
2.2
Music
representations
Models
2.3
Data processing
methdologies
2.4
Knowledge-driven
methdologies
2.5
Estimation of
elements related to
musical concepts
Estimated
musical
concepts
Other
estimated
concepts
2.6
Evaluation
methodologies
SOCIO-CULTURAL PERSPECTIVE
4.1
Social aspects
4.2
Multiculturality
USER PERSPECTIVE
3.1
User behaviour
3.2
User interaction
EXPLOITATION PERSPECTIVE
5.1
Music distribution
applications
5.2
Creative tools
5.3
Other exploitation areas

45. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
• Music Information Research (MIR) involves the understanding and modeling of
music-related data in its full contextual complexity.
• Music is a communication phenomenon that involves people and
communities immersed in specific social and cultural context.
• MIR aims at processing musical data that captures the social and cultural
context and at developing data processing methodologies with which to
model the whole musical phenomenon.
Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=

46. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
• Music Computing research (IT in general) does not respond to the world's
multi-cultural reality.
• Data models, cognition models, user models, interaction models,
ontologies, ... are culturally biased.
• Music information is more than CDs and metadata.
h"p://compmusic.upf.edu=;=h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
==

47. CATÓLICA'PORTO+
ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
• Promote a multicultural approach to Music Computing research.
• Advance in the formalization and description of music, making it more accessible to
computational approaches.
• Reduce the gap between audio signal descriptions and semantically/culturally
meaningful music concepts.
• Develop information modelling techniques applicable to different music repertories.
• Develop computational models to represent culture specific music contexts.
h"p://compmusic.upf.edu=;=h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
==

48. CATÓLICA'PORTO+
ARTES+
compmusic
Music information modeling
Sound and Music Computing
Music Information Processing
ComputationalMusicology
Ontologies
Cognition
models
Interaction
models
Data
models
User
models
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
h"p://compmusic.upf.edu==

49. CATÓLICA'PORTO+
ARTES+
References
• Richard Nisbett, “The Geography of Thought: How Asians and Westerners Think Differently ... and Why”, New York: Free
Press, 2003, ISBN 978-0-7432-1646-3
• Fred E. Jandt, “An Introduction to Intercultural Communication”, Chapter 3 - “Culture’s Influence on Perception”, SAGE
Publications, 2012, http://www.sagepub.com/upm-data/45975_Chapter_3.pdf
• Ian Cross, “Music, cognition, culture and evolution”, Annals of the New York Academy of Sciences Vol 930, 2001, pp 28-42.
• Wong PCM, Ciocca V, Chan AHD, Ha LYY, Tan L-H, et al. (2012) “Effects of Culture on Musical Pitch Perception.”, PLoS ONE
7(4): e33424. doi:10.1371/journal.pone.0033424
• Meagan E. Curtis, Jamshed J. Bharucha, “Memory and Musical Expectation for Tones in Cultural Context”, Music Perception
VOLUME 26, ISSUE 4, PP. 365–375, ISSN 0730-7829, ELECTRONIC ISSN 1533-8312
• Xiao Hu, Jin Ha Lee, “A CROSS-CULTURAL STUDY OF MUSIC MOOD PERCEPTION BETWEEN AMERICAN AND CHINESE
LISTENERS”, 13th International Conference on Music Information Retrieval (ISMIR), 2012, Porto, Portugal
• Bregman, A. (1990). Auditory Scene Analysis -- The Perceptual Organization of Sound.
• Bregman, A., & Ahad, P. (1996). Demonstrations of Auditory Scene Analysis: The Perceptual Organization of Sound.
• Wang, W. (Ed.). (2010). Machine Audition - Principles, Algorithms and Systems. IGI Global.

50. CATÓLICA'PORTO+
ARTES+
References
• Xavier Serra, Michela Magas, Emmanouil Benetos, Magdalena Chudy, Simon Dixon, Arthur Flexer, Emilia Gómez, Fabien
Gouyon, Perfecto Herrera, Sergi Jorda, Oscar Paytuvi, Geoffroy Peeters, Jan Schlüter, Hugues Vinet, Gerhard Widmer,
“Roadmap for Music Information ReSearch”, Geoffroy Peeters (editor), 2013, Creative Commons BY-NC-ND 3.0 license, ISBN:
978-2-9540351-1-6 - http://mires.eecs.qmul.ac.uk
• Wang, D., & Brown, G. (2006). Computational Auditory Scene Analysis: Principles, Algorithms and Applications.

51. CATÓLICA'PORTO+
ARTES+
THANK YOU!
Luís Gustavo Martins
lmartins@porto.ucp.pt
http://lgustavomartins.wordpress.com/about/
http://artes.ucp.pt
http://artes.ucp.pt/citar
http://artes.ucp.pt/ccd