An overview is provided of the Stravinsqi-Jun2014 algorithm and its performance on the MediaEval 2014 C@merata Task. Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input. The algorithm parses a symbolic representation of a piece of music as well as a query string consisting of a natural language expression, and identifies where event(s) specified by the query occur in the music. The output for any given query is a list of time windows corresponding to the locations of relevant events. To evaluate the algorithm, its output time windows are compared with those specified by music experts for the same query-piece combinations. In an evaluation consisting
of twenty pieces and 200 questions, Stravinsqi-Jun2014 had recall .91 and precision .46 at the measure level, and recall .87 and precision .44 at the beat level. Important potential applications of this work in music-educational software and musicological research are discussed.
http://ceur-ws.org/Vol-1263/mediaeval2014_submission_50.pdf
4. • Stravinsqi
stands
for
STaff
Representation
Analysed
VIa
Natural
language
String
Query
Input.
1. Get
question
string
and
division
value,
analyse
it
and
split
into
compound
questions
if
necessary.
2. Load
staff
names,
time
signatures,
and
point-‐set
representations
of
the
piece.
3. Produce
time-‐interval
answers
to
elemental
question(s).
4. Cross-‐check
timings
of
answers
to
compound
questions.
5. Convert
time-‐interval
answers
to
required
format.
5. • Stravinsqi
stands
for
STaff
Representation
Analysed
VIa
Natural
language
String
Query
Input.
1. Get
question
string
and
division
value,
analyse
it
and
split
into
compound
questions
if
necessary.
2. Load
staff
names,
time
signatures,
and
point-‐set
representations
of
the
piece.
3. Produce
time-‐interval
answers
to
elemental
question(s).
4. Cross-‐check
timings
of
answers
to
compound
questions.
5. Convert
time-‐interval
answers
to
required
format.
6. • Stravinsqi
stands
for
STaff
Representation
Analysed
VIa
Natural
language
String
Query
Input.
1. Get
question
string
and
division
value,
analyse
it
and
split
into
compound
questions
if
necessary.
2. Load
staff
names,
time
signatures,
and
point-‐set
representations
of
the
piece.
3. Produce
time-‐interval
answers
to
elemental
question(s).
4. Cross-‐check
timings
of
answers
to
compound
questions.
5. Convert
time-‐interval
answers
to
required
format.
7. • Stravinsqi
stands
for
STaff
Representation
Analysed
VIa
Natural
language
String
Query
Input.
1. Get
question
string
and
division
value,
analyse
it
and
split
into
compound
questions
if
necessary.
2. Load
staff
names,
time
signatures,
and
point-‐set
representations
of
the
piece.
3. Produce
time-‐interval
answers
to
elemental
question(s).
4. Cross-‐check
timings
of
answers
to
compound
questions.
5. Convert
time-‐interval
answers
to
required
format.
8. • Stravinsqi
stands
for
STaff
Representation
Analysed
VIa
Natural
language
String
Query
Input.
1. Get
question
string
and
division
value,
analyse
it
and
split
into
compound
questions
if
necessary.
2. Load
staff
names,
time
signatures,
and
point-‐set
representations
of
the
piece.
3. Produce
time-‐interval
answers
to
elemental
question(s).
4. Cross-‐check
timings
of
answers
to
compound
questions.
5. Convert
time-‐interval
answers
to
required
format.
9. • Stravinsqi
stands
for
STaff
Representation
Analysed
VIa
Natural
language
String
Query
Input.
1. Get
question
string
and
division
value,
analyse
it
and
split
into
compound
questions
if
necessary.
2. Load
staff
names,
time
signatures,
and
point-‐set
representations
of
the
piece.
3. Produce
time-‐interval
answers
to
elemental
question(s).
4. Cross-‐check
timings
of
answers
to
compound
questions.
5. Convert
time-‐interval
answers
to
required
format.
12. #|
Let's
begin
by
looking
at
Morley's
'April
is
in
my
mistress'
face'.
|#
(setq
notation-‐name
"f1")
(setq
notation-‐path&name
(merge-‐pathnames
(make-‐pathname
:name
notation-‐name
:type
"krn")
notation-‐path))
(setq
question-‐number
"001")
13. #|
*********************************
****
1.
ANALYSE
QUESTION
STRING
****
*********************************
|#
;
Load
question
string
and
division
value.
(setq
question&division
(c@merata2014-‐question-‐file2question-‐string
question-‐number
question-‐path&name
notation-‐name))
;
-‐-‐>
("F#
followed
two
crotchets
later
by
a
G"
1)
(setq
division
(second
question&division))
;
-‐-‐>
1
14. ;
Split
up
compound
questions.
(setq
questions
(followed-‐by-‐splitter
(first
question&division)))
;
-‐-‐>
(("F#"
0)
("G"
2))
;
Try
US
version
of
the
question
string.
(setq
questions
(followed-‐by-‐splitter
"F#
followed
two
quarter
notes
later
by
a
G"))
;
-‐-‐>
(("F#"
0)
("G"
2))
15. ;
Try
junk
version
of
the
question
string.
(setq
questions
(followed-‐by-‐splitter
"F#
blah
followed
two
crotchets
later
by
a
blah
G"))
;
-‐-‐>
(("F#
blah"
0)
("blah
G"
2))
;
Try
reference
to
bars
rather
than
a
specific
duration.
(setq
questions
(followed-‐by-‐splitter
"F#
followed
two
bars
later
by
a
G"))
;
-‐-‐>
(("F#"
0)
("G"
2
"bars"))
16. ;
Try
'consecutive'
question.
(setq
questions
(followed-‐by-‐splitter
"F#
followed
by
two
consecutive
crotchets"))
;
-‐-‐>
(("F#"
0)
("crotchet"
0)
("crotchet"
0))
17. ;
Try
linguistic
variant
of
compound
query.
(setq
questions
(followed-‐by-‐splitter
(concatenate
'string
"F#
then
two
bars
later
a
G
"
"then
consecutive
fifths
in
the
left
hand")))
#|
-‐-‐>
(("F#"
0)
("G"
2
"bars")
("fifths
in
the
left
hand"
0)
("fifths
in
the
left
hand"
0))
|#
18. ;
Back
to
original
question.
(setq
questions
(followed-‐by-‐splitter
(first
question&division)))
;
-‐-‐>
(("F#"
0)
("G"
2))
19. #|
***************************************
****
2.
LOAD
REPRESENTATIONS
OF
PIECE
****
***************************************
|#
#|
Load
staff
names
and
names
of
the
opening
clefs
on
each
staff.
|#
(setq
staff&clef-‐names
(staves-‐info2staff&clef-‐names
notation-‐path&name))
#|
-‐-‐>
(("Bass"
"bass
clef")
("Tenor"
"tenor
clef")
("Alto"
"treble
clef")
("Soprano"
"treble
clef"))
|#
20. #|
Load
time-‐signature
information
and
any
changes
across
the
piece.
|#
(setq
ontimes-‐signatures
(append-‐ontimes-‐to-‐time-‐signatures
(kern-‐file2ontimes-‐signatures
notation-‐path&name)))
;
-‐-‐>
((1
4
4
0))
;
If
there
was
a
change,
it
might
look
like:
;
-‐-‐>
((1
4
4
0)
(9
3
2
32))
25. ;
This
is
the
function
of
interest
on
this
occasion...
(setq
ans-‐pitch
(mapcar
#'(lambda
(x)
(pitch-‐class-‐time-‐intervals
(first
x)
point-‐set
staff&clef-‐names))
questions))
#|
-‐-‐>
(((32
36)
(57
115/2)
(58
60))
((12
13)
(12
13)
(24
32)
(26
27)
(28
32)
(46
48)
(48
50)
(49
50)
(50
52)
(50
52)
(54
57)
(60
62)
(60
61)
(61
62)
(62
63)
(63
64)))
|#
26. #|
****************************************************
****
4.
Cross-‐check
timings
for
compound
questions
****
****************************************************
|#
;
Put
the
answers
next
to
one
another
in
one
list.
(setq
time-‐intervals
(list
ans-‐harmonic-‐interval
ans-‐melodic-‐interval
ans-‐dur-‐pitch
ans-‐pitch
ans-‐dur
ans-‐nadir-‐apex
ans-‐triad
ans-‐triad-‐inversion
ans-‐texture
ans-‐cadence
ans-‐word&event
ans-‐word
ans-‐artic
ans-‐dur-‐rest
ans-‐tied&event))
33. • The
Stravinsqi
algorithm
has
effectively
solved
seven
of
the
twelve
C@merata
task
categories
(pitch,
duration,
pitch
and
duration,
articulation,
voice
specific,
lyrics,
and
melodic
interval)
• As
for
the
remaining
five
categories,
precision
for
one
category
(compound
queries)
can
be
improved
by
fixing
a
selection-‐criteria
bug
• More
data
are
required
for
the
triad,
cadence,
and
texture
query
categories
• A
big
“thank
you”
to
Richard
Sutcliffe
and
team
34. • The
Stravinsqi
algorithm
has
effectively
solved
seven
of
the
twelve
C@merata
task
categories
(pitch,
duration,
pitch
and
duration,
articulation,
voice
specific,
lyrics,
and
melodic
interval)
• As
for
the
remaining
five
categories,
precision
for
one
category
(compound
queries)
can
be
improved
by
fixing
a
selection-‐criteria
bug
• More
data
are
required
for
the
triad,
cadence,
and
texture
query
categories
• A
big
“thank
you”
to
Richard
Sutcliffe
and
team
35. • The
Stravinsqi
algorithm
has
effectively
solved
seven
of
the
twelve
C@merata
task
categories
(pitch,
duration,
pitch
and
duration,
articulation,
voice
specific,
lyrics,
and
melodic
interval)
• As
for
the
remaining
five
categories,
precision
for
one
category
(compound
queries)
can
be
improved
by
fixing
a
selection-‐criteria
bug
• More
data
are
required
for
the
triad,
cadence,
and
texture
query
categories
• A
big
“thank
you”
to
Richard
Sutcliffe
and
team
36. • The
Stravinsqi
algorithm
has
effectively
solved
seven
of
the
twelve
C@merata
task
categories
(pitch,
duration,
pitch
and
duration,
articulation,
voice
specific,
lyrics,
and
melodic
interval)
• As
for
the
remaining
five
categories,
precision
for
one
category
(compound
queries)
can
be
improved
by
fixing
a
selection-‐criteria
bug
• More
data
are
required
for
the
triad,
cadence,
and
texture
query
categories
• A
big
“thank
you”
to
Richard
Sutcliffe
and
team
37. Researchers
at
CCARH
for
creating
and
hosting
kern
scores
Collaborators
at:
Johannes
Kepler
University:
Gerhard
Widmer,
Andreas
Arzt,
Sebastian
Böck,
and
Sebastian
Flossmann
Center
for
Mind
and
Brain,
UC
Davis:
Petr
Janata,
Fred
Barrett,
and
Joy
Geng
Aalborg
University:
David
Meredith
University
of
Lyon:
Barbara
Tillmann
The
Open
University:
Robin
Laney,
Alistair
Willis,
and
Paul
Garthwaite
Funding
bodies:
Austrian
Science
Fund
(FWF)
project
number
Z159
(Wittgenstein
Grant),
NSF
grant
#1025310,
ESPRC
38. Code
for
Stravinsqi
(as
well
as
other
projects):
http://www.tomcollinsresearch.net
40. Allegro non tanto
#$$$$$ % &% &%
'$$$$$
!"
!"
% &%
6
#$$$$$
'$$$$$ &%
11
#$$$$$
'$$$$$
!
P
22
!
P
33
!
P
32
!
P
21
!
P
34
!
P
23
!
P
31
!
P
11
!
P
12
Beginning
of
op.56
no.1
by
Chopin
