Teaching courses on coding game development & VR in the school of computer science Thank Izzy and the Organising committee for inviting me Teach students how to make Tunepal as part of my OOP class Author & inventor of Tunepal
Store the score in a tunebook Thesession.org run by Jeremy Keith Bill Black’s transcriptions of Ceol Rince na Heireann O’Neills American Marching band Ceol Rince na Heireann O Farrells Bulmer & Sharpley Heinrik Norebecks tune collections
Psion version late 1990’s Windows version early 2000’s working for Bear Stearns.
Palmtop User magazine 2002
2004 Poster for a mobile learning conference
MATT2 and Tansey Heinrik Norebecks collection I was working on a different problem Genuine Eurica moment the day I made it work for the first time Sent Tansey a copy of my thesis
Even by the standards of the time! It was pretty shit looking One cant be good at everything
11 Feb 2010 iPhone app 1 July 2010 with Tunepal for Android 15 December 2010 with Tunepal HD for iPad Zeitgeist Phones could do the transcription Also phones were small & unobtrusive enough to bring to a session No music score display
Fuzzy Histogram clusters notes together based on similar durations!! Ornamentation Filtering Harmonicity changes All detailed in my PhD thesis and various publications ABCJS by Paul Rosen and Gregory DykeABC2PS by Michael MethfesselABC2PS iPhone port by Jeremy HuxtableABC4J by Lionel Gueganton.ABC2MIDI by Seymour Shlien
Marco Cascorino Morgan Conlon Jamie Osler Lisa Shields Jonathan Lynch & Scott Baker Siobhán Nic Gaoithín, Oifig na Gaeilge
New website Geotagged maps (no longer work!) Translations New tunebooks iPad version
Sunday Times top 50 cultural apps Mention Avery Wang from Shazam
Top 50 apps of all time
Met Avery Wang of Shazam who offered me a job
Wrote some articles for this Kept the apps updated
dynamically track and publish records of folk music playing It captures, stores, and posts the names of tunes played in Irish traditional music sessions on a public website Disambiguate between noise and music based on spectral flux Tunes were annotated based on majority voting of several 12 second samples 20-50% accuracy , longitudinal statistics) of tune playing practices, the changing repertoire of a session. to examine whether instrumentation of traditional and folk arts is and should be possible Semi structured interviews and less formal during session with anchor musicians At the same time, TuneTracker seeks to respect the privacy of musicians by not recording or streaming any actual audio. Deployed for five months in Dublin, Ireland at The Cobblestone pub Amateurs can intelligently practice certain (e.g., most popular) tunes in a session. Professionals will be able to reflect on the repertoire of their sessions. TuneTracker is also an academic exercise in intervention
Some of the anchor musicians of a particular night requested that we turn off TuneTracker during their session time semi-structured  interviews (N=7) and less formal interviews during session observations (N=4) with anchor musicians. In other words, TuneTracker is imprinting on amateurs a deterministic image of what a session is—that it follows a formula One anchor told us he received a text from a friend saying “I see you’re in The Cobblestone tonight from the tunes that are being played.” Through the tunes, the musicians were being tracked.
When seeing the low percent of polkas (3%) in her session, an anchor told us, “Now, we’ll have to do something about that!
I don’t think technology should be pushing the music. I think the musicians should be pushing the music...It’s a bit [like] a genie out of the bottle though.
In the second quote, the musicians see TuneTracker as a tool, not a substitute for the tradition. For them, any artifact that can encourage more people to play and reflect on traditional music, thereby preserving the tradition, is beneficial. TuneTracker will encourage visiting musicians to join in sessions.
TuneTracker has agency —it deliberately introduces a power asymmetry  by wresting control of sessions away from the musicians and into a technological artifact. TuneTracker will be prescriptive: spelling out what tunes people should demand in sessions. Because TuneTracker is authoritatively clothed in digital garb, it asserts itself as the final truth—whether it be regarding tune names, tune notes, the order of tunes in a set, etc. Thus, TuneTracker threatens to destroy tradition by marginalizing the human element in the traditional process Just like any resource (e.g., sheet music), it can be misused
Bryan Duggan Lise DenBrock Breandán Knowlton Chris Xue 50 million cultural heritage items (from books and paintings to 3D objects and audiovisual material) that celebrate over 3,500 cultural institutions across Europe. Comhaltas 30 second extracts Replace the Java applet with record and transcription functionality implemented in HTML5 Return recordings of music, not just music scores. Make all the functionality of the Tunepal, including query-by-playing work similarly across all devices including smartphones. Open-source Tunepal and make an API server available to other projects.
A Search Through Time: Connecting Live Playing to Archive Recordings of Traditional Music 6th. International Workshop on Folk Music Analysis, 15-17 June, 2016. Hosted in TU Dublin Europeanna Sounds Conference in Paris
user trials for the week of Feadh Ceoil na hEireann in various locations in Sligo in August 2015. In total 40 users tested the new version of Tunepal
Limitations of existing Tunepal
Support for different instruments such as banjo, concertina Not key invariant searches. Many of the searches were failing because the players were playing ionj
2 of the challenges that were taken on my Pierre Beauguitte in 2015 Approaches these problems with great academic rigour Coming the completion of his PhD
2015 Pierre Beauguitte co-supervised with Professor John Kelleher Immersed himself in traditional music & culture Attended classes in the conservatory O’Donohues pub Traditional music ensemble, Peter Browne, Ciaran Hanrahan, Thomas Dorely Gig with Mark Redmond, piper,
Similar work done by IZZY Co supervising 5 years of outstandiung scholarship to bear on these problems 30 tunes Foinn Seisiun´ books published by the Comhaltas Ceoltoir ´ ´ı Eireann ´ organisation, available with Creative Commons licence. These offer good quality, homogeneous examples of the heterophony inherent to an Irish traditional dance music session. • Grey Larsen’s MP3s for 300 Gems of Irish Music for All Instruments, commercially available. These consist of studio quality recordings of tunes played on Irish flute, tin whistle and anglo concertina. • personal recordings of the second author, a renowned musician, on the Irish flute. These are available together with the annotations.
Tony software . After importing an audio file, Tony offers estimates using the pYIN algorithm (notelevel) presented in the next section. These were then manually corrected, by adding new notes, merging repeated notes and adjusting frequencies. The annotations were finally post-processed to convert these frequencies to the closest MIDI note references Exported to CSV files
a note is considered correctly transcribed if its onset is distant from the reference note by less than 50ms and its pitch by less than a quarter of a tone
Precision, recall and F-measures are computed with the mir eval framework It is interesting to observe that MATT achieves the highest F-measures on the solo recordings. However, only the difference with Silvet is statistically signiﬁcant On the session recordings, Silvet achieves signiﬁcantly higher pitch precisions Why?
musical key consists of a tonic note, represented by a pitch class (C, C#, D...), and a mode, or rather mode family, which can be minor or major 24 candidate keys, for the 12 semitones of the octave and the two considered modes The standard approach to identifying keys in a musical piece is to use key-proﬁles They can be seen as vectors assigning weights to the twelve semitones The paper compares different approaches to generating keyprofiles from the literature with two new approaches introduced by the author based on CADENCES and modes in traditional music Pierre annotated an audio dataset 636 items from the fionn session tunebook, Grey Larsen’s 300 gems, also a symbolic dataset drawn from thesession.org Ran an experiment which extracted the pitch class histogram from the tune in audio or symbolic format Audio files used deep chromas Key was evaluated by comparing against the 24 key profiles one for each semitone. Cross product Highest score
Hence it appears that inferring keys from heterophonic or polyphonic audio is easier than on monophonic recordings. An explanation for this is that the harmonic content is richer in heterophonic and polyphonic signals. Cadences outperform the existing key profiles Propose an improvement by adding weights 10 fold cross validation is used to estimate the size of a grid search to evaluate the optimum weights Using the evaluated weights Pierre is able to improve the mirex score again Parametric profiles
Two new types of error neighbour key errors and relative key errors. The high frequencies of these two types of errors can be explained by the speciﬁc characteristics of Irish traditional music minor keys are harder to detect than major keys The key annotations on the datasets were made by the ﬁrst author, and it is possible that other annotators could annotate some of the tunes differently Ambiguous
Simple and compound refer to the beat subdivision, duple and triple refer to the grouping of beats Show examples Down the broom
Slide Slide Brosna Slip Jig – Drops of Brandy
obtain an onset detection function by a method of spectral difference The autocorrelation functions is then computed on a 5s window of the SD function. The quaver duration q is determined by the fuzzy histogram algorithm, a clustering algorithm A quantised lag vector is calculated what exactly is? Fed into a logistic regression classifier 10 fold cross validation
Used a dataset the collection of recordings accompanying the Foinn Seisiun books published by the Comhaltas Ceoltoirı Eireann organisation, which he labelled with Labelled with tune type and beat subdivision 1st experiment easy, Tune type similar to meter detection, reel, hoprnpipe and barndance 326 unique recordings The models of both experiments predict a label for a 5s window
2 experiments 1. simple, compound
15787 tunes from thesession,org Audio files from the previous experiment tune id in the search space Key deviation used in the ground truth manually annotated to provide ground truth 4 random offsets are chosen on each tune, spaced by a minimum of 5 seconds such that overlap is allowed. Our dataset thus consists of 2000 12 second excerpts https://dl.acm.org/citation.cfm?id=2660188 pitch class is be represented by integers Ornamentation notes are filtered out Transcribes to chromas, pYin, Melodia, Silvet Quantised using the fuzzy histogram algorithm
Pitch class histograms are generated for the symbolic audio 120 valued vector. A Gaussian kernel of size 150 cents . The vector is normalised to sum to 1. Makes the representation close to the audio one Audio PCH STFT, summed, 120 valued vector Aligned using Bhattacharyya coefficient. Select the shift that has the highest score that provides the best alignment these symbolic PCH can be pre-computed. Once this optimal shift k is found, the symbolic quaver sequence is transposed by round(k/10) semitones, as the unit for k is 10 cents
Wu and Manber (1992) present a bit-parallel algorithm for finding occurrences
QbSH tMIREX Top-10 hit rate (1 point is scored for a hit in the top 10 and 0 is scored otherwise) Best (Top 1) Hit rate a new way of ranking is now introduced worst possible rank (WPR) worst possible rank (WPR), is a way of ranking results that appropriately deals with draws Relative difference? Original with a tip – glass ceiling All – Go through all possible semitone transpositions and pick the best – brute force
PCH to the best a novel approach. Existing literature has often focused on audio-to-audio alignment in the context of cover song identiﬁcation (EllisandPoliner, 2007;Serraetal.,2008), or has relied on keyannota tion when dealing with symbolic music The method introduced here performs audio-to-symbolic alignment without the need for key annotation, thus alleviating the difﬁculty of deﬁning key in ITM Without reducing accuracy and by adding 50% to the tune identification task compared to 12 times more for the brute force approach
Pierres work shows us the road ahead for tunepal
Other examples move from Webworld to Azure hosting Need to move to ssl Browser updates Europeanna search API changed Caused the archive searches to fail for the past 6 months or so, despite my best efforts to track it down
Katie Kilroy is a Senior Analyst with Arlo Smart Home Technologies who works with all things data, process, and systems on a daily basis button accordion at Irish trad sessions around Cork City
Average of 65,505 per month 1.4 searches per minute
A generation of musicians it was how they learned the names of tunes Used my teachers, professionals, amatures alike Find a tune in any key on any instrument Tunepal is too important for one person! 4 million interactions. 2 million music searches! Thanks to Pierre 70% geotagged
10 Years of Tunepal: Reflections & Future Directions
10 Years of Tunepal:
Reflections & Future
Dr Bryan Duggan
School of Computer Science
Technological University of Dublin
• A search-by-playing app &
website for Android and IOS
• 20K App users
• 23K music scores
• Around 1K music searches
• 2,322,682 tap-to-record
• 2,042,877 title searches
• 3,619,592 tunes downloaded
• Links to Comhaltas archive
• MIDI Playback & music score
• Links to youtube, thesession
• Grant from the Dept of Culture Sport and
• Had to complete all the work in 10 weeks
– App available in Irish and English
– New professionally designed UX
– Music score display
– Add additional tunebooks
– iPad version Tunepal HD
– Tune editing & composition
In the media
• Irish Times
• Irish Daily mail
• Sunday Times - Top twenty
• John Creedon’s Fleadh
• Top 25 grossing apps on
iTunes (2 days)
• Duggan, B., Gainza, M. & Cunningham, P.
Machine annotation of sets of traditional Irish
dance tunes. Paper presented at the Ninth
International Conference on Music Information
Retrieval (ISMIR), Drexel University, Philadelphia,
• Bryan Duggan, Brendan O'Shea, (2011) "Tunepal:
searching a digital library of traditional music
scores", OCLC Systems & Services, Vol. 27 Iss: 4,
pp.284 - 297
• Dynamically track and
publish records of folk music
• Deployed for five months in
Dublin, Ireland at The
• Proceedings of the 2014
conference on Designing
• Norman Makato Su, Indiana
2015 Europeanna Collaboration
• Replace the Java applet with record and
transcription functionality implemented in
• Return recordings of music, not just music scores.
• Make all the functionality of the Tunepal,
including query-by-playing work similarly across
all devices including smartphones.
• Open-source Tunepal and make an API server
available to other projects.
• Magical to explore!
• Jersey for Java API server
• HTML5 Audio
• Node, npm
• Materialize with Angular.JS
• Europeana APIs
• Works on Chrome, Firefox, Edge
• Show another demo!!
Transcription Accuracy Study
• Tony software 30 pieces of audio
• Variety of instruments, solo and ensemble
• Midi format
• Used as ground truth to evaluate 4 different algorithms
– pYIN, Melodia, MATT, Silvet
• Frame-Level Evaluation: Melody Extraction Task
• Note-Level Evaluation: Note Tracking Task
• Beauguitte, P., Duggan, B., Kelleher, J. (2016). A Corpus
of Annotated Irish Traditional Dance Music Recordings:
Design and Benchmark Evaluations. 17th ISMIR 2016.
Beauguitte, P., Duggan, B. and Kelleher, J. (2017) Key inference from
Irish traditional music scores and recordings.14th Sound and Music
Computing Conference, July 5-8, 2017, Espoo, Finland.
• Categorised into
– simple duple: 4 4 (reel, hornpipe, ﬂing, barndance) and 2 4
– simple triple: 3 4 (waltz, mazurka)
– compound duple: 6 8 (jigs) and 12 8 (slides)
– compound triple: 9 8 (slip jigs)
• Infer rhythmic information from audio recordings of Irish
• Beauguitte, P., Duggan, B. & Kelleher, J. D. (2018). Rhythm
inference from audio recordings of Irish traditional music.
Proceedings of the 8th International Workshop on Folk
Music Analysis, 26-29 June 2018, Thessaloniki (Greece)
• Relies on the repetitive nature of this musical
Keeping the apps up to date
• Example from this year
• Android 9
– Switch from Eclipse to Android Studio
– Switch to Gradle from ad hock build system
– Remove ActionBarSherlock library
– A lot of time spent fixing UX bugs and crashes on certain phones in
– Gmail sending bug
– Still random crashes
• Europeanna searches failing
– Lots of tracking down and dead ends trying to find what the problem
– Trying to build a 3 year old web app with a complex toolchain, java
scriot, ecs6, node, npm, bower, git
Listening to ClareFM.ie this morning.
Heard a great tune…..interrupted by the Galway Races Used Tunepal……..it’s called The
Parting………got it from The Session.org. So much fun……….. listening and learning Irish Trad.
Thank you so much for the good you have done and are doing!
It is ironic, but one of my mates has proudly proclaimed she has never had a cell phone,
computer, nor opened an email account. She is utterly fascinated with Tunepal
“I’m sure you’re getting emails every day saying how much folks love your tunepal site and
iphone app. I have the app and LOVVVVVVVVVE it! I use it every day. Awesome awesome
Getting a lot of use out of this app now. Everybody I have told so far is very impressed.”
It’s pure magic. Thanks a million for having made this diamond and share it with everyone. If
there was a God, I’m quite sure that he would have you blessed !
I just wanted to thank you for the great idea and the great job you’ve done with Tunepal. It’s an
excellent and very useful tool. Just what I whished for quite often.
I am really impressed with the app. It was 100% correct in identifying tunes when I played them
on flute or whistle. Having worked in a previous life in ultrasonic testing of materials, I can
appreciate some of the problems involved.
Thank you, Tunepal!!! You’ve saved lives!!!!!!
Fair play to Bryan it’s a wonderful App I have learned lots of tunes from
it and Names of Tunes as well
Amazing! Congrats! I must be at least 30k of the tune searches 🤔☺ I use
it so much. I love it
• Technology has transformed the playing and learning of
• Tunepal is beloved by its users
– “It changed my life”
• 10 years of data on usage
• The future
– Tune synchronisation
– Better transcription accuracy
– Key invariant matching
– Modern Tunepal app
– Redevelop the server
– Get involved!
• Get the code and contribute:
• Contact me
Tunepal is a query-by-playing search engine for
traditional music that handles around 60K music searches each month.