Large-Scale Capture of Producer-Defined Musical
Semantics
Ryan Stables
School of Digital Media Technology
Birmingham City U...
Problem...
Problem Definition
Producer:
Audio effects parameters
usually refer to low-level
attributes.
Professionally produced audio
o...
Problem Definition
Descriptors need to represent
the views of music producers.
These may change with genre,
musical instrum...
Project Aims
1. Gather large amounts of semantics data during the music
creation/production process.
Develop a series of D...
Model...
Project overview
Server
Descriptor name...
Save...Load...
Save...
Semantic Descriptor
Parameter Space
Feature Set
Pre/Post...
(1) Plug-in interface
Parameters can be set
experimentally.
Semantic descriptors to be
stored in text field.
Descriptors ca...
(2) Feature Extraction
Features are extracted from the
selected region.
The parameter space is stored.
Semantic descriptor...
(3) Mapping
NLP Algorithms to identify
semantic correlation.
Dimensionality reduction to
find correlation in
features/param...
Design Constraints...
Architecture
Requirements:
Maximisation of user-base.
Transparency: Access to the processing chain.
Design decisions:
Stan...
Analysis framework
LibXtract.
Hard-coded, C library.
Around 400 combined
audio features*.
[Bullock, 2007]
Vamp.
Plug-in wi...
Mini-Project...
Mini-Project: Aims
Analyse the production requirements of musicians.
Birmingham Conservatoire
The Music Producers Guild
Th...
Mini Project: Schematic
Plug-in
development
Interface design
Algorithm
Development
Server, network,
data distribution
User...
Positions and Timescale
2 x PhD Students: 1 x C4DM (QMUL) & 1 x DMT (BCU).
3 x Advisory roles.
Timescale: 6-months from Se...
References
Bullock, J. (2007).
Libxtract: A lightweight library for audio feature extraction.
In Proceedings of the Intern...
Upcoming SlideShare
Loading in …5
×

Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)

897 views

Published on

This talk was given by Ryan Stables (Birmingham City University) at the "Semantic Media @ The British Library" event on 23 September 2013.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
897
On SlideShare
0
From Embeds
0
Number of Embeds
156
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)

  1. 1. Large-Scale Capture of Producer-Defined Musical Semantics Ryan Stables School of Digital Media Technology Birmingham City University
  2. 2. Problem...
  3. 3. Problem Definition Producer: Audio effects parameters usually refer to low-level attributes. Professionally produced audio often requires extensive training. Researcher: Lack of semantically annotated music production datasets. How can we map low-level descriptors to perceived muscial timbre?
  4. 4. Problem Definition Descriptors need to represent the views of music producers. These may change with genre, musical instruments, etc... Various terms may be used to define similar things (colour, texture etc...)
  5. 5. Project Aims 1. Gather large amounts of semantics data during the music creation/production process. Develop a series of DAW plug-ins. Extract information and anonymously upload it to a server. 2. Identify correlation and patterns in the semantics data. 3. Use the data to improve/aid music production tasks.
  6. 6. Model...
  7. 7. Project overview Server Descriptor name... Save...Load... Save... Semantic Descriptor Parameter Space Feature Set Pre/Post Gain Analysis... Natural Language Processing Dimensionality Reduction Etc... (1) (2)(3) Figure : Schematic Overview of the Semantic Audio Feature Extraction Project.
  8. 8. (1) Plug-in interface Parameters can be set experimentally. Semantic descriptors to be stored in text field. Descriptors can be loaded through same interface. Parameters are stored and/or set. Figure : Semantic Audio plug-in: Multi-band distortion
  9. 9. (2) Feature Extraction Features are extracted from the selected region. The parameter space is stored. Semantic descriptors are sent as targets. Additional metadata is sent, if available. Server Descriptor name... Save...Load... Save... Semantic Descriptor Parameter Space Feature Set Pre/Post Gain Analysis... Natural Language Processing Dimensionality Reduction Etc... Figure : Stored attributes.
  10. 10. (3) Mapping NLP Algorithms to identify semantic correlation. Dimensionality reduction to find correlation in features/parameters. Additional data partitions based on metadata (Genre, instrument, etc...) Results sent back to user plug-in. Server Descriptor name... Save...Load... Sav Semantic Paramete Feature Se Pre/ Analysis... Natural Language Processing Dimensionality Reduction Etc... Figure : Results processing
  11. 11. Design Constraints...
  12. 12. Architecture Requirements: Maximisation of user-base. Transparency: Access to the processing chain. Design decisions: Stand-alone plug-ins. MultiFX. Plug-ins within a plug-in. Analysis-only. Other: Free field vs. fixed word. Before and after. Metadata pane.
  13. 13. Analysis framework LibXtract. Hard-coded, C library. Around 400 combined audio features*. [Bullock, 2007] Vamp. Plug-in within a plug-in. Hosts LibXtract features, amongst others. [Cannam et al., 2006]
  14. 14. Mini-Project...
  15. 15. Mini-Project: Aims Analyse the production requirements of musicians. Birmingham Conservatoire The Music Producers Guild The Birmingham Music Network Build a series of prototype systems for the collection of musical semantics data. Use these systems to collect data from a small group of musicians during the production process. Evaluate the results in order to identify a suitable system for future research. Demonstrate the feasibility of a wider research project in this area.
  16. 16. Mini Project: Schematic Plug-in development Interface design Algorithm Development Server, network, data distribution User Testing Data Aquisition Results Analysis Figure : Schematic Overview of the Mini-Project.
  17. 17. Positions and Timescale 2 x PhD Students: 1 x C4DM (QMUL) & 1 x DMT (BCU). 3 x Advisory roles. Timescale: 6-months from September 2013. Future: collaborative grant application. Thanks! ryan.stables@bcu.ac.uk
  18. 18. References Bullock, J. (2007). Libxtract: A lightweight library for audio feature extraction. In Proceedings of the International Computer Music Conference, volume 43. Cannam, C., Landone, C., Sandler, M. B., and Bello, J. P. (2006). The sonic visualiser: A visualisation platform for semantic descriptors from musical signals. In ISMIR, pages 324–327.

×