Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
From SALAMI to Social Machines:Music Information Retrieval as anExemplar of Digital Research, or                    Resear...
Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
...the imminent flood of scientific data expected from the next generation of experiments, simulations, s ensors and satel...
BioEssays,, 26(1):99–105, January 2004                                         http://research.microsoft.com/en-us/collabo...
www.einfrastructuresouth.ac.uk
A Big Picture                e-infrastructure                                          The FourthMore machines            ...
E. Science laboris                     • Data Analysis Pipelines                     • Workflows are the new              ...
“Blink”“Automating the Audio Production Process”
Reuse, Recycling, Repurposing• Paul writes workflows for identifying biological  pathways implicated in resistance to  Try...
“A biologist would rathershare their toothbrush than     their gene name” “Data mining: my  data’s mine and your data’s mi...
http://www.myexperiment.org/
Paul’sPaul’s Pack                                     Workflow 16                                  QTL Research           ...
Research Objects                             Reproducibility, Integrated Publishing• Workflow                    Distribut...
Linked Data support      rdf.myexperiment.org1. Use URIs as names for things2. Use HTTP URIs so that people   can look up ...
Sean Bechhofer                                  SELECT?wf ?uriSELECT?pack ?contrib              WHERE {WHERE {            ...
Jeremy Frey“Publication @ Source”
To Do                      Ingredient List                                 Dissolve 4-      Add K2CO3                    H...
Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
The Problem                           Ichiro Fujinaga                        INT.   VERSE   VERSE BRIDG VERSE BRIDG VERSE...
SALAMI• Structural Analysis of Large amounts of Music  Information• Musical analysis has traditionally been conducted by  ...
Structural Analysis of Large Amounts of Music Information    23,000 hours of   Digital Music    recorded music            ...
Ground TruthAshley Burgoyne
LeadFunctionLarge scaleSmall scale              Annotation               Example
Segment Ontology                                                 class structureOntology models properties from musicologi...
http://www.music-ir.org/mirex/
Meandre              Stephen Downie          http://seasr.org/meandre/
ABCDBC
It’s web-like!                           “Ground Truth”                                            Community              ...
http://musicnet.mspace.fm/blog/music-linked-data-workshop/
Digital Music               Digital Music                Digital Music                 Digital Music               Collect...
Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
http://force11.org/
method data
The R dimensionsReusable. The key tenet of Research Replayable. Studies might involveObjects is to support the sharing and...
Research repeat     Record          repeatMachine      paper          Machine                          REPRODUCE          ...
The Executable Thesis                        new data                        executable                          thesis  P...
Computational Research ObjectsResearch Objects that are1. The research record for repeatable, reproducible, ... etc2. Desc...
Notifications and automatic re-runs          Autonomic        Self-repair           Curation                      New rese...
Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
http://www.bodleian.ox.ac.uk/bodley/library/special/projects/whats-the-score
SOCIAMThe Theory and Practice of     Social Machines
The Order of Social Machines Real life is and must be full of all kinds of social constraint – the very processes from whi...
Some other machines?
Dimensions• Number of people              • Empowering of• Number of machines              individuals, groups, crowds• Sc...
Building a Social MachineVirtual World(Network of social interactions)                                  Dave Robertson    ...
The users of a website, the website, andthe interactions between them, togetherform our fundamental notion of a “machine”
That Big Picture                e-infrastructure                                          The FourthMore machines         ...
An Agenda1. Science has much to learn from an industry /    R&D that is already digital end-to-end   • Insights into ICT c...
david.deroure@oerc.ox.ac.ukwww.oerc.ox.ac.uk/people/dderwww.scilogs.com/eresearch@dderSlide credits: Christine Borgman, Ca...
Links• Semantic Media  http://semanticmedia.org.uk/• myExperiment project wiki  http://wiki.myexperiment.org/• Workflow Fo...
•   D. De Roure, C. Goble and R. Stevens. The Design and Realisation of the myExperiment    Virtual Research Environment f...
Fourth Quadrant Semantic Media
Fourth Quadrant Semantic Media
Fourth Quadrant Semantic Media
Fourth Quadrant Semantic Media
Fourth Quadrant Semantic Media
Fourth Quadrant Semantic Media
Upcoming SlideShare
Loading in …5
×

Fourth Quadrant Semantic Media

1,120 views

Published on

"From SALAMI to Social Machines: Music Information Retrieval as an Exemplar of Digital Research, or Fourth Quadrant Research". Keynote by David De Roure at Semantic Media Launch, Barbican, 3 October 2012

  • Great quote about 'social machines' on slide 47; few people reference this though they're implementing such social machines. Will be interesting to see the results of SOCIAM (slide 48 & http://social.org & EPSRC info: http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/J017728/1 )
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Fourth Quadrant Semantic Media

  1. 1. From SALAMI to Social Machines:Music Information Retrieval as anExemplar of Digital Research, or ResearchFourth Quadrant Semantic Media David De Roure
  2. 2. Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
  3. 3. ...the imminent flood of scientific data expected from the next generation of experiments, simulations, s ensors and satellites Source: CERN, CERN-EX-0712023, http://cdsweb.cern.ch/record/1203203
  4. 4. BioEssays,, 26(1):99–105, January 2004 http://research.microsoft.com/en-us/collaboration/fourthparadigm/
  5. 5. www.einfrastructuresouth.ac.uk
  6. 6. A Big Picture e-infrastructure The FourthMore machines Big Data The Future! Big Compute Quadrant Conventional Social online Computation Networking R&D More people
  7. 7. E. Science laboris • Data Analysis Pipelines • Workflows are the new rock and roll • Machinery for coordinating the execution of services and linking together resources • Repetitive and mundane boring stuff made easier Carole Goble
  8. 8. “Blink”“Automating the Audio Production Process”
  9. 9. Reuse, Recycling, Repurposing• Paul writes workflows for identifying biological pathways implicated in resistance to Trypanosomiasis in cattle• Paul meets Jo. Jo is investigating Whipworm in mouse.• Jo reuses one of Paul’s workflow without change.• Jo identifies the biological pathways involved in sex dependence in the mouse model, believed to be involved in the ability of mice to expel the parasite.• Previously a manual two year study by Jo had failed to do this. Carole Goble
  10. 10. “A biologist would rathershare their toothbrush than their gene name” “Data mining: my data’s mine and your data’s mine”
  11. 11. http://www.myexperiment.org/
  12. 12. Paul’sPaul’s Pack Workflow 16 QTL Research Results Object produces Included in Published in Included in Feeds intoLogs produces Included in Included inMetadata Slides Paper produces Published in Common pathways Workflow 13 Results
  13. 13. Research Objects Reproducibility, Integrated Publishing• Workflow Distributed Third Party Alien – Provenance Tenancy Store – Conservation & Preservation – Executable Publication Carole Goble• Human – Credit Tracking – Unit of Scholarship – Crowd management• Semantics – Acquisition & Publishing – Encoding, Encapsulation & Annotation: OAI-ORE, AO… Technical Objects Social Objects
  14. 14. Linked Data support rdf.myexperiment.org1. Use URIs as names for things2. Use HTTP URIs so that people can look up those names3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL)4. Include links to other URIs so that they can discover more things
  15. 15. Sean Bechhofer SELECT?wf ?uriSELECT?pack ?contrib WHERE {WHERE { ?wf mebase:has-current-version ?v. ?pack rdf:type mepack:Pack. ?v mecomp:executes-dataflow ?d. ?pack ore:aggregates ?contrib. ?d mecomp:has-component ?c.} ?c rdf:type mecomp:WSDLProcessor. ?c mecomp:processor-uri ?uri. }
  16. 16. Jeremy Frey“Publication @ Source”
  17. 17. To Do Ingredient List Dissolve 4- Add K2CO3 Heat at reflux Cool and add Heat at Cool and add Extract with Combine organics, Remove Fuse compound to silica & List flourinated powder for 1.5 hours Br11OCB reflux until water (30ml) DCM dry over MgSO4 & solvent in column in ether/petrol Fluorinated biphenyl 0.9 g Br11OCB 1.59 g biphenyl in completion (3x40ml) filter vacuo Potassium Carbonate 2.07 g butanone A digital lab book Butanone 40 ml Plan replacement that Add Cool Add Reflux Liquid- Remove Column Add Reflux Cool Add Dry Filter Fuse liquid Solvent Chromatography extraction by Rotary Evaporation Butanone dried via silica column and Sample of 4- 0.9031 grammes Weigh Inorganics dissolve 2 layers. Added brine ~20ml. text image 3 of 40 Measure ml excess Measure g Silica Ether/ Petrol Ratio chemists wereProcess flourinated able to use, and measured into 100ml RB flask.Record Used 1ml extra solvent to wash out biphenyl Annotate container. DCM MgSO4 Annotate 1 1 2 2 1 3 1 4 3 5 2 6 2 7 4 8 9 10 11 12 13 14 Add Cool Add Reflux Add Remove Column Add Reflux Cool Liquid- Dry Filter Fuse text liquid (Buchner) Solvent Chromatography Sample of Butanone Annotate extraction by Rotary Sample of Br11OCB Water Annotate Annotate Evaporation K2CO3 Measure Powder liked. Weigh Weigh Measure text Started reflux at 13.30. (Had to change heater stirrer) Only reflux 40 text Washed MgSO4 with text ml for 45min, next step 14:15. Organics are yellow solution DCM ~ 50ml 2.0719 g g 30 ml 1.5918 Key Observation Types Future Questions Process weight - grammes Whether to have many subclasses of processes or fewer with annotations measure - ml, drops Combechem Input How to depict destructive processes annotate - text 30 January 2004 Jeremy Frey Literal How to depict taking lots of samples temperature - K, C ° gvh, hrm, gms Observation What is the observation/process boundary? e.g. MRI scan
  18. 18. Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
  19. 19. The Problem Ichiro Fujinaga INT. VERSE VERSE BRIDG VERSE BRIDG VERSE O . E E UT
  20. 20. SALAMI• Structural Analysis of Large amounts of Music Information• Musical analysis has traditionally been conducted by individuals and on a small scale• Computational approach, combined with the huge volume of data now available, will 1. Deliver substantive corpus of musical analyses in common framework for music scholars and students 2. Establish a methodology and tooling so that community can sustain and enhance this resource www.diggingintodata.org
  21. 21. Structural Analysis of Large Amounts of Music Information 23,000 hours of Digital Music recorded music Collections Music Information Retrieval Community Student-sourced Community ground truth Software Supercomputer Linked Data Repositories
  22. 22. Ground TruthAshley Burgoyne
  23. 23. LeadFunctionLarge scaleSmall scale Annotation Example
  24. 24. Segment Ontology class structureOntology models properties from musicological domain• Independent of Music Information Retrieval research and signal processing foundations• Maintains an accurate and complete description of relationships that link them Kevin Page and Ben Fields
  25. 25. http://www.music-ir.org/mirex/
  26. 26. Meandre Stephen Downie http://seasr.org/meandre/
  27. 27. ABCDBC
  28. 28. It’s web-like! “Ground Truth” Community Digital Audio “Signal” Structural Analysis
  29. 29. http://musicnet.mspace.fm/blog/music-linked-data-workshop/
  30. 30. Digital Music Digital Music Digital Music Digital Music Collections Collections Collections Collectionsground truth ground truth ground truth Community Community Community Software Software Expertise Expertise Software Expertise Expertise Results Results papers Results Results papers papers Evaluation Evaluation Papers Infrastructure Infrastructure (sociotechnical) Evaluations Evaluations (sociotechnical) Evaluations
  31. 31. Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
  32. 32. http://force11.org/
  33. 33. method data
  34. 34. The R dimensionsReusable. The key tenet of Research Replayable. Studies might involveObjects is to support the sharing and single investigations that happen inreuse of data, methods and milliseconds or protracted processesprocesses. that take years.Repurposeable. Reuse may also Referenceable. If research objectsinvolve the reuse of constituent are to augment or replace traditionalparts of the Research Object. publication methods, then they mustRepeatable. There should be be referenceable or citeable.sufficient information in a Research Revealable. Third parties must beObject to be able to repeat the able to audit the steps performed instudy, perhaps years later. the research in order to be convincedReproducible. A third party can of the validity of results.start with the same inputs and Respectful. Explicit representationsmethods and see if a prior result can of the provenance, lineage and flowbe confirmed. of intellectual property. Replacing the Paper: The Twelve Rs of the e-Research Record” on http://blogs.nature.com/eresearch/
  35. 35. Research repeat Record repeatMachine paper Machine REPRODUCE papersoftware softwareMachine Machine Software REPRODUCE OR REPEAT? paperworkflow workflow wf softwaresoftwareMachine Software Machine blogs.nature.com/eresearch/
  36. 36. The Executable Thesis new data executable thesis PhD Student new results
  37. 37. Computational Research ObjectsResearch Objects that are1. The research record for repeatable, reproducible, ... etc2. Describe process (method) for enactment/execution3. Usable by machines as well as humans – Social Objects – Semantically described – Programmatically accessible – Designed for assistance and automation – Designed for scale and heterogeneity4. Composable with a distributed computational model?
  38. 38. Notifications and automatic re-runs Autonomic Self-repair Curation New research?Machines are users too
  39. 39. Overview• Digital Research• SALAMI as an exemplar• Reconstruct / Reuse / Repurpose• Social Machines
  40. 40. http://www.bodleian.ox.ac.uk/bodley/library/special/projects/whats-the-score
  41. 41. SOCIAMThe Theory and Practice of Social Machines
  42. 42. The Order of Social Machines Real life is and must be full of all kinds of social constraint – the very processes from which society arises. Computers can help if we use them to create abstract social machines on the Web: processes in which the people do the creative work and the machine does the administration… The stage is set for an evolutionary growth of new social engines. Berners-Lee, Weaving the Web, 1999
  43. 43. Some other machines?
  44. 44. Dimensions• Number of people • Empowering of• Number of machines individuals, groups, crowds• Scale of data • Time criticality• Varieties of data • Extent of wide area• Type of machine communication problem solving • Need for urgent• Type of human mobilization problem solving • Specification of goal state SOCIAM – The Theory and Practice of Social Machines – commences October 2012, led by Nigel Shadbolt at University of Southampton.
  45. 45. Building a Social MachineVirtual World(Network of social interactions) Dave Robertson Model of social interactionDesign and Participation andComposition Data supply Physical World (people and devices)
  46. 46. The users of a website, the website, andthe interactions between them, togetherform our fundamental notion of a “machine”
  47. 47. That Big Picture e-infrastructure The FourthMore machines Big Data The Future! Big Compute Quadrant Conventional Social online Computation Networking R&D More people
  48. 48. An Agenda1. Science has much to learn from an industry / R&D that is already digital end-to-end • Insights into ICT challenges2. What can we learn from (e-)Science? • Metadata capture at source, end-to-end semantics • Social objects, semantic objects, audio objects? • Reproducible/reconstructable/machine-assisted analysis/production • Interactivity, intersection of digital and physical3. Designing the Social Machines of music* • Human generated metadata / music? * And also the music of Social Machines!
  49. 49. david.deroure@oerc.ox.ac.ukwww.oerc.ox.ac.uk/people/dderwww.scilogs.com/eresearch@dderSlide credits: Christine Borgman, Carole Goble, Faith Lawrence & MikeJewell, Sean Bechhofer, Jeremy Frey, Ichiro Fujinaga, Stephen Downie, AshleyBurgoyne, Kevin Page, Ben Fields, Nigel Shadbolt, Dave Robertsonwww.myexperiment.org/packs/337http://www.slideshare.net/davidderoure/fourth-quadrant-semantic-media
  50. 50. Links• Semantic Media http://semanticmedia.org.uk/• myExperiment project wiki http://wiki.myexperiment.org/• Workflow Forever project (Wf4Ever) http://www.wf4ever-project.org/• Future of Research Communication (FORCE11) http://force11.org/• Theory and Practice of Social Machines (SOCIAM) http://sociam.org/
  51. 51. • D. De Roure, C. Goble and R. Stevens. The Design and Realisation of the myExperiment Virtual Research Environment for Social Sharing of Workflows Future Generation Computer Systems 25, pp. 561-567.• S. Bechhofer, I. Buchan, D De Roure et al. Why linked data is not enough for scientists, Future Generation Computer Systems• D. De Roure, David and C. Goble, Anchors in Shifting Sand: the Primacy of Method in the Web of Data. WebSci10, April 26-27th, 2010, Raleigh, NC, US.• D. De Roure, S. Bechhofer, C. Goble and D. Newman, Scientific Social Objects, 1st International Workshop on Social Object Networks (SocialObjects 2011).• D. De Roure, K. Belhajjame, P. Missier, P. et al Towards the preservation of scientific workflows. 8th International Conference on Preservation of Digital Objects (iPRES 2011).• Carole A. Goble, David De Roure and Sean Bechhofer Accelerating scientists’ knowledge turns. Will be available at www.springerlink.com• Khalid Belhajjame, Oscar Corcho, Daniel Garijo et al Workflow-Centric Research Objects: First Class Citizens in Scholarly Discourse, SePublica2012 at ESWC2012, Greece, May 2012• Kevin R. Page, Ben Fields, David De Roure et al Reuse, Remix, Repeat: The Workflows of MIR, 13th International Society for Music Information Retrieval Conference (ISMIR 2012) Porto, Portugal, October 8th-12th, 2012• Jun Zhao, Jose Manuel Gomez-Perezy, Khalid Belhajjame et al, Why Workflows Break - Understanding and Combating Decay in Taverna Workflows, eScience 2012, Chicago, October 2012

×