David De Roure
 @dder
Intersection, Scale, and Social Machines:
The Humanities in the Digital World
DIRECTOR, UNIVERSITY OF OXFORD E-RESEARCH CENTRE
Data-intensive research
Human-intensive research
Music
Scholarly Communication
The Big Picture(s)

Challenging Assumptions
ChristineBorgman
13,785,659	
  total	
  volumes	
  
6,871,154	
  book	
  6tles	
  
364,473	
  serial	
  6tles	
  
4,824,980,650	
  pages	
  
618	
  terabytes	
  
163	
  miles	
  
11,201	
  tons	
  
5,372,477	
  public	
  domain	
  volumes	
  
10,000,000,000,000,000 bytes archived!
New Forms of Data
▶ Internet data, derived from social
media and other online interactions
(including data gathered by
connected people and devices, eg
mobile devices, wearable
technology, Internet of Things)
▶ Tracking data, monitoring the
movement of people and objects
(including GPS/geolocation data,
traffic and other transport sensor
data, CCTV images etc)
▶ Satellite and aerial imagery (eg
Google Earth, Landsat, infrared,
radar mapping etc) http://www.oecd.org/sti/sci-tech/new-data-for-
understanding-the-human-condition.htm
The	
  Big	
  Picture	
  
More people
Moremachines
Big Data
Big Compute
Conventional
Computation
“Big Social”
Social Networks
e-infrastructure
Online R&D
(Science 2.0)
Digital
Scholarship
@dder
theODI.org
Data Detect Store AnalyticsFilter Analysts
@dder
There is no such thing as the Internet of Things
There is no such thing as a closed system
Humans are creative and subversive
The Rise of the Bots A Swarm of Drones
Accidents happen (in the lab, bin)
Holding machines to account Software vulnerability
Where are the throttle points?
@dder
F i r s t
Edwards, P. N., et al. (2013) Knowledge Infrastructures: Intellectual Frameworks and
Research Challenges. Ann Arbor: Deep Blue. http://hdl.handle.net/2027.42/97552
Social Machines

Empowered Citizens
Social	
  Machines	
  Defini6on	
  TBL	
  
Pip Willcox
https://twitter.com/CR_UK/status/446223117841494016/
Some people's smartphones
had autocorrected the word
"BEAT" to instead read
"BEAR".
"Thank you for choosing an
adorable polar bear," the
reply from the WWF said.
"We will call you today to set
up your adoption."
http://www.bbc.com/news/technology-26723457
SOCIAM: The Theory and Practice of Social Machines is funded by the UK Engineering and Physical Sciences Research Council
(EPSRC) under grant number EPJ017728/1 and comprises the Universities of Southampton, Oxford and Edinburgh. See sociam.org
“Yet	
  Wikipedia	
  and	
  its	
  stated	
  ambi6on	
  to	
  “compile	
  the	
  sum	
  of	
  all	
  
human	
  knowledge”	
  are	
  in	
  trouble.	
  The	
  volunteer	
  workforce	
  that	
  
built	
  the	
  project’s	
  flagship,	
  the	
  English-­‐language	
  Wikipedia—and	
  
must	
  defend	
  it	
  against	
  vandalism,	
  hoaxes,	
  and	
  manipula6on—
has	
  shrunk	
  by	
  more	
  than	
  a	
  third	
  since	
  2007	
  and	
  is	
  s6ll	
  shrinking…	
  	
  
The	
  main	
  source	
  of	
  those	
  problems	
  is	
  not	
  mysterious.	
  The	
  loose	
  
collec6ve	
  running	
  the	
  site	
  today,	
  es6mated	
  to	
  be	
  90	
  percent	
  
male,	
  operates	
  a	
  crushing	
  bureaucracy	
  with	
  an	
  oYen	
  abrasive	
  
atmosphere	
  that	
  deters	
  newcomers	
  who	
  might	
  increase	
  
par6cipa6on	
  in	
  Wikipedia	
  and	
  broaden	
  its	
  coverage…”	
  
	
  http://www.technologyreview.com/featuredstory/520446/the-decline-of-wikipedia/
“Panoptes has been designed so
that it’s easier for us to update
and maintain, and to allow
more powerful tools for project
builders. It’s also open source
from the start, and if you find
bugs or have suggestions about
the new site you can note them
on Github (or, if you’re so
inclined, contribute to the
codebase yourself).”
"	
  
http://blog.zooniverse.org/2015/06/29/a-whole-new-zooniverse/
http://monsterspedia.wikia.com/wiki/File:Argus-Panoptes.jpg
Panoptes
Musical Social Machines

Social Machines of Scholarship
INT. VERSE VERSE VERSE VERSEBRIDGEBRIDGE OUT.
ê
The	
  Problem	
  
signal
understanding
Ichiro Fujinaga
salami.music.mcgill.ca
Jordan B. L. Smith, J. Ashley Burgoyne, Ichiro Fujinaga, David De Roure, and J.
Stephen Downie. 2011. Design and creation of a large-scale database of structural
annotations. In Proceedings of the International Society for Music Information
Retrieval Conference, Miami, FL, 555–60
Sequence alignment
http://en.wikipedia.org/wiki/Sequence_alignment#/media/File:Histone_Alignment.png
Dan Edelstein, Robert Morrissey, and Glenn Roe, To Quote or not to Quote: Citation Strategies in the Encyclopédie.
Journal of the History of Ideas , Volume 74, Number 2, April 2013 . pp. 213-236. 10.1353/jhi.2013.0012
Glenn Roe
Digital	
  Music	
  
Collec6ons	
  
Grad-­‐sourced	
  
ground	
  truth	
  
Community	
  
SoYware	
  
Linked	
  Data	
  
Repositories	
  
Supercomputer	
  
23,000 hours of
recorded music
Music Information
Retrieval Community
SALAMI
Ashley Burgoyne
www.music-ir.org/mirex
Music Information Retrieval Evaluation eXchange
Audio Onset Detection
Audio Beat Tracking
Audio Key Detection
Audio Downbeat Detection
Real-time Audio to Score Alignment(a.k.a
Score Following)
Audio Cover Song Identification
Discovery of Repeated Themes & Sections
Audio Melody Extraction
Query by Singing/Humming
Audio Chord Estimation
Singing Voice Separation
Audio Fingerprinting
Music/Speech Classification/Detection
Audio Offset Detection
Downie, J. Stephen, Andreas F. Ehmann, Mert Bay and M. Cameron Jones. (2010).
The Music Information Retrieval Evaluation eXchange: Some Observations and
Insights. Advances in Music Information Retrieval Vol. 274, pp. 93-115
Stephen	
  Downie	
  
http://chordify.net/
Digital	
  Material	
  
Pip Willcox
Kevin Page
David Weigl
Interfaces, for
computer and
human
!
Sonifying	
  the	
  Variants	
  
•  From	
  Play	
  to	
  Sonifica6on	
  
•  Using	
  First	
  Folio	
  and	
  Quartos	
  data	
  
•  Parsing	
  the	
  TEI	
  XML,	
  conver6ng	
  it	
  with	
  rule	
  set	
  into	
  numbers,	
  
sonifying	
  the	
  data	
  to	
  produce	
  sounds	
  
34
Sonification	
  
Iain Emsley
Studying Social Machines

Scholarship of Social Machines
Ecosystem
Perspective
•  We see a community of
living, hybrid organisms,
rather than a set of
machines which happen to
have humans amongst
their components
•  Their successes and
failures inform the design
and construction of their
offspring and successors
time
Social Machine instances
 @dder
Observer of
one social
machine
Observers using third
party observatory
Observer of
multiple social
machines
Human
participants in
Social
Machine
Human participants in
multiple Social Machines
Observer of Social
Machine infrastructure
1	
  
4	
  
2	
  
3	
  
5	
  
6	
  
SM
SM
SM
Social Machine
Observing Social
Machines
7	
  
@dder
De Roure, D.,
Hooper, C., Page,
K., Tarte, S., and
Willcox, P. 2015.
Observing Social
Machines Part 2:
How to Observe?
ACM Web Science
The Web
Observatory
Tiropanis, T., Hall, W., Shadbolt, N., De Roure, D.,
Contractor, N. and Hendler, J. 2013. The Web Science
Observatory, IEEE Intelligent Systems 28(2) pp 100–104.
ThanassisTiropanis
Simpson, R., Page, K.R. and
De Roure, D. 2014.
Zooniverse: observing the
world's largest citizen science
platform. In Proceedings of
the companion publication of
the 23rd international
conference on World Wide
Web, 1049-1054.
Kevin Page
STORYTELLING AS A STETHOSCOPE
FOR SOCIAL MACHINES
1.  Sociality through storytelling potential
and realization
2.  Sustainability through reactivity and
interactivity
3.  Emergence through collaborative
authorship and mixed authority
Zooniverse	
  is	
  a	
  highly	
  
storified	
  Social	
  Machine	
  
Facebook	
  doesn’t	
  allow	
  
for	
  improvisa6on	
  
Wikipedia	
  assigns	
  
authority	
  rights	
  rigidly	
  
http://ora.ox.ac.uk/objects/ora:8033
Tarte, S.M., De Roure, D. and Willcox, P. 2014. Working out the Plot: the Role of
Stories in Social Machines. SOCM2014: The Theory and Practice of Social
Machines, Seoul, Korea, International World Wide Web Conferences pp. 909–914
Pip Willcox
Tarte, S. Willcox, P., Glaser, H. and De Roure, D. 2015. Archetypal Narratives in Social
Machines: Approaching Sociality through Prosopography. ACM Web Science 2015.
SégolèneTarte
Scholarly Communication

Preface
Elizabeth Williamson
Richard O’Bierne
A computationally-enabled
sense-making network of
expertise, data, software,
models and narratives
Big Data, in a
Big Data Centre
Pip Willcox and Kevin Page
 	
  
consume	

	
  	
  
produce	

	
  	
  
compose	
  
perform	
  
capture	

	
  	
  
distribute	

	
  	
  
	
  	
  
	
  	
  
	
  	
  
Mark	
  Sandler	
  
Curate	
  	
  	
  	
  	
  	
  Preserve	
  
!
Notifications and automatic re-runs
Machines are users too
Autonomic
Curation
Self-repair
New research?
The	
  R	
  Dimensions	
  
Research	
  Objects	
  facilitate	
  research	
  that	
  is	
  
reproducible,	
  repeatable,	
  replicable,	
  reusable,	
  
referenceable,	
  retrievable,	
  reviewable,	
  
replayable,	
  re-­‐interpretable,	
  reprocessable,	
  
recomposable,	
  reconstructable,	
  repurposable,	
  
reliable,	
  respecful,	
  reputable,	
  revealable,	
  
recoverable,	
  restorable,	
  reparable,	
  refreshable?”	
  
@dder 14 April 2014
sci	
  method	
  
access	
  
understand	
  
new	
  use	
  
social	
  
cura6on	
  
Research	
  
Object	
  
Principles	
  
De Roure, D. 2014. The future
of scholarly communications.
Insights: the UKSG journal,
27, (3), 233-238.
DOI 10.1629/2048-7754.171
https://www.gartner.com/technology/research/digital-marketing/transit-map.jsp
Intersection, Scale, and
Social Machines:
The Humanities in the
Digital World
First	
  Folio	
  Social	
  Machines	
  
Metadata
Story of the
First Folio
Social
Machines Annotation
David De Roure and Pip Willcox
‘“Coniunction, with the participation of Society”: Citizens, Scale, and
Scholarly Social Machines’
Beyond the PDF: Born-Digital Humanities, Boston, 27–28 April 2015
Pip Willcox
PipWillcox
david.deroure@oerc.ox.ac.uk @dder
Thanks to Tim Crawford, Mark d’Inverno, Stephen Downie,
Iain Emsley, Ichiro Fujinaga, Chris Lintott, Grant Miller,
Terhi Nurmikko-Fuller, Kevin Page, Carolin Rindfleisch,
Glenn Roe, Mark Sandler, Ségolène Tarte, David Weigl, and
Pip Willcox.
http://www.slideshare.net/davidderoure/humanities-in-the-digital-world
Supported by SOCIAM: The Theory and Practice of Social Machines, funded by the UK Engineering and Physical
Sciences Research Council (EPSRC) under grant number EP/J017728/1; Fusing Semantic and Audio Technologies for
Intelligent Music Production and Consumption (FAST) funded by EPSRC under grant number EP/L019981/1; and
Transforming Musicology, funded by the UK Arts and Humanities Research Council under the Digital Transformations
programme. Thanks also to the Andrew W. Mellon Foundation.
www.oerc.ox.ac.uk	

david.deroure@oerc.ox.ac.uk	

@dder

Humanities in the Digital World

  • 1.
    David De Roure @dder Intersection, Scale, and Social Machines: The Humanities in the Digital World DIRECTOR, UNIVERSITY OF OXFORD E-RESEARCH CENTRE
  • 2.
  • 3.
  • 4.
  • 5.
    13,785,659  total  volumes   6,871,154  book  6tles   364,473  serial  6tles   4,824,980,650  pages   618  terabytes   163  miles   11,201  tons   5,372,477  public  domain  volumes   10,000,000,000,000,000 bytes archived!
  • 6.
    New Forms ofData ▶ Internet data, derived from social media and other online interactions (including data gathered by connected people and devices, eg mobile devices, wearable technology, Internet of Things) ▶ Tracking data, monitoring the movement of people and objects (including GPS/geolocation data, traffic and other transport sensor data, CCTV images etc) ▶ Satellite and aerial imagery (eg Google Earth, Landsat, infrared, radar mapping etc) http://www.oecd.org/sti/sci-tech/new-data-for- understanding-the-human-condition.htm
  • 7.
    The  Big  Picture   More people Moremachines Big Data Big Compute Conventional Computation “Big Social” Social Networks e-infrastructure Online R&D (Science 2.0) Digital Scholarship @dder
  • 8.
  • 9.
    Data Detect StoreAnalyticsFilter Analysts @dder
  • 10.
    There is nosuch thing as the Internet of Things There is no such thing as a closed system Humans are creative and subversive The Rise of the Bots A Swarm of Drones Accidents happen (in the lab, bin) Holding machines to account Software vulnerability Where are the throttle points? @dder
  • 11.
    F i rs t
  • 12.
    Edwards, P. N.,et al. (2013) Knowledge Infrastructures: Intellectual Frameworks and Research Challenges. Ann Arbor: Deep Blue. http://hdl.handle.net/2027.42/97552
  • 13.
  • 14.
    Social  Machines  Defini6on  TBL   Pip Willcox
  • 15.
    https://twitter.com/CR_UK/status/446223117841494016/ Some people's smartphones hadautocorrected the word "BEAT" to instead read "BEAR". "Thank you for choosing an adorable polar bear," the reply from the WWF said. "We will call you today to set up your adoption." http://www.bbc.com/news/technology-26723457
  • 17.
    SOCIAM: The Theoryand Practice of Social Machines is funded by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant number EPJ017728/1 and comprises the Universities of Southampton, Oxford and Edinburgh. See sociam.org
  • 18.
    “Yet  Wikipedia  and  its  stated  ambi6on  to  “compile  the  sum  of  all   human  knowledge”  are  in  trouble.  The  volunteer  workforce  that   built  the  project’s  flagship,  the  English-­‐language  Wikipedia—and   must  defend  it  against  vandalism,  hoaxes,  and  manipula6on— has  shrunk  by  more  than  a  third  since  2007  and  is  s6ll  shrinking…     The  main  source  of  those  problems  is  not  mysterious.  The  loose   collec6ve  running  the  site  today,  es6mated  to  be  90  percent   male,  operates  a  crushing  bureaucracy  with  an  oYen  abrasive   atmosphere  that  deters  newcomers  who  might  increase   par6cipa6on  in  Wikipedia  and  broaden  its  coverage…”    http://www.technologyreview.com/featuredstory/520446/the-decline-of-wikipedia/
  • 20.
    “Panoptes has beendesigned so that it’s easier for us to update and maintain, and to allow more powerful tools for project builders. It’s also open source from the start, and if you find bugs or have suggestions about the new site you can note them on Github (or, if you’re so inclined, contribute to the codebase yourself).” "   http://blog.zooniverse.org/2015/06/29/a-whole-new-zooniverse/ http://monsterspedia.wikia.com/wiki/File:Argus-Panoptes.jpg Panoptes
  • 21.
    Musical Social Machines SocialMachines of Scholarship
  • 22.
    INT. VERSE VERSEVERSE VERSEBRIDGEBRIDGE OUT. ê The  Problem   signal understanding Ichiro Fujinaga
  • 23.
    salami.music.mcgill.ca Jordan B. L.Smith, J. Ashley Burgoyne, Ichiro Fujinaga, David De Roure, and J. Stephen Downie. 2011. Design and creation of a large-scale database of structural annotations. In Proceedings of the International Society for Music Information Retrieval Conference, Miami, FL, 555–60
  • 24.
  • 25.
    Dan Edelstein, RobertMorrissey, and Glenn Roe, To Quote or not to Quote: Citation Strategies in the Encyclopédie. Journal of the History of Ideas , Volume 74, Number 2, April 2013 . pp. 213-236. 10.1353/jhi.2013.0012 Glenn Roe
  • 26.
    Digital  Music   Collec6ons   Grad-­‐sourced   ground  truth   Community   SoYware   Linked  Data   Repositories   Supercomputer   23,000 hours of recorded music Music Information Retrieval Community SALAMI
  • 27.
  • 28.
    www.music-ir.org/mirex Music Information RetrievalEvaluation eXchange Audio Onset Detection Audio Beat Tracking Audio Key Detection Audio Downbeat Detection Real-time Audio to Score Alignment(a.k.a Score Following) Audio Cover Song Identification Discovery of Repeated Themes & Sections Audio Melody Extraction Query by Singing/Humming Audio Chord Estimation Singing Voice Separation Audio Fingerprinting Music/Speech Classification/Detection Audio Offset Detection Downie, J. Stephen, Andreas F. Ehmann, Mert Bay and M. Cameron Jones. (2010). The Music Information Retrieval Evaluation eXchange: Some Observations and Insights. Advances in Music Information Retrieval Vol. 274, pp. 93-115
  • 29.
  • 30.
  • 31.
  • 32.
    Kevin Page David Weigl Interfaces,for computer and human !
  • 34.
    Sonifying  the  Variants   •  From  Play  to  Sonifica6on   •  Using  First  Folio  and  Quartos  data   •  Parsing  the  TEI  XML,  conver6ng  it  with  rule  set  into  numbers,   sonifying  the  data  to  produce  sounds   34 Sonification   Iain Emsley
  • 35.
  • 36.
    Ecosystem Perspective •  We seea community of living, hybrid organisms, rather than a set of machines which happen to have humans amongst their components •  Their successes and failures inform the design and construction of their offspring and successors
  • 37.
  • 38.
    Observer of one social machine Observersusing third party observatory Observer of multiple social machines Human participants in Social Machine Human participants in multiple Social Machines Observer of Social Machine infrastructure 1   4   2   3   5   6   SM SM SM Social Machine Observing Social Machines 7   @dder De Roure, D., Hooper, C., Page, K., Tarte, S., and Willcox, P. 2015. Observing Social Machines Part 2: How to Observe? ACM Web Science
  • 39.
    The Web Observatory Tiropanis, T.,Hall, W., Shadbolt, N., De Roure, D., Contractor, N. and Hendler, J. 2013. The Web Science Observatory, IEEE Intelligent Systems 28(2) pp 100–104. ThanassisTiropanis
  • 40.
    Simpson, R., Page,K.R. and De Roure, D. 2014. Zooniverse: observing the world's largest citizen science platform. In Proceedings of the companion publication of the 23rd international conference on World Wide Web, 1049-1054. Kevin Page
  • 41.
    STORYTELLING AS ASTETHOSCOPE FOR SOCIAL MACHINES 1.  Sociality through storytelling potential and realization 2.  Sustainability through reactivity and interactivity 3.  Emergence through collaborative authorship and mixed authority Zooniverse  is  a  highly   storified  Social  Machine   Facebook  doesn’t  allow   for  improvisa6on   Wikipedia  assigns   authority  rights  rigidly   http://ora.ox.ac.uk/objects/ora:8033 Tarte, S.M., De Roure, D. and Willcox, P. 2014. Working out the Plot: the Role of Stories in Social Machines. SOCM2014: The Theory and Practice of Social Machines, Seoul, Korea, International World Wide Web Conferences pp. 909–914
  • 42.
  • 43.
    Tarte, S. Willcox,P., Glaser, H. and De Roure, D. 2015. Archetypal Narratives in Social Machines: Approaching Sociality through Prosopography. ACM Web Science 2015. SégolèneTarte
  • 44.
  • 46.
  • 47.
  • 48.
    A computationally-enabled sense-making networkof expertise, data, software, models and narratives Big Data, in a Big Data Centre
  • 49.
    Pip Willcox andKevin Page
  • 50.
        consume     produce     compose   perform   capture     distribute                 Mark  Sandler   Curate            Preserve   !
  • 51.
    Notifications and automaticre-runs Machines are users too Autonomic Curation Self-repair New research?
  • 52.
    The  R  Dimensions   Research  Objects  facilitate  research  that  is   reproducible,  repeatable,  replicable,  reusable,   referenceable,  retrievable,  reviewable,   replayable,  re-­‐interpretable,  reprocessable,   recomposable,  reconstructable,  repurposable,   reliable,  respecful,  reputable,  revealable,   recoverable,  restorable,  reparable,  refreshable?”   @dder 14 April 2014 sci  method   access   understand   new  use   social   cura6on   Research   Object   Principles   De Roure, D. 2014. The future of scholarly communications. Insights: the UKSG journal, 27, (3), 233-238. DOI 10.1629/2048-7754.171
  • 53.
  • 54.
    Intersection, Scale, and SocialMachines: The Humanities in the Digital World
  • 55.
    First  Folio  Social  Machines   Metadata Story of the First Folio Social Machines Annotation David De Roure and Pip Willcox ‘“Coniunction, with the participation of Society”: Citizens, Scale, and Scholarly Social Machines’ Beyond the PDF: Born-Digital Humanities, Boston, 27–28 April 2015 Pip Willcox
  • 56.
  • 57.
    david.deroure@oerc.ox.ac.uk @dder Thanks toTim Crawford, Mark d’Inverno, Stephen Downie, Iain Emsley, Ichiro Fujinaga, Chris Lintott, Grant Miller, Terhi Nurmikko-Fuller, Kevin Page, Carolin Rindfleisch, Glenn Roe, Mark Sandler, Ségolène Tarte, David Weigl, and Pip Willcox. http://www.slideshare.net/davidderoure/humanities-in-the-digital-world Supported by SOCIAM: The Theory and Practice of Social Machines, funded by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/J017728/1; Fusing Semantic and Audio Technologies for Intelligent Music Production and Consumption (FAST) funded by EPSRC under grant number EP/L019981/1; and Transforming Musicology, funded by the UK Arts and Humanities Research Council under the Digital Transformations programme. Thanks also to the Andrew W. Mellon Foundation.
  • 58.

Editor's Notes

  • #35  ----- Meeting Notes (25/10/15 20:38) ----- Pipeline to transform the XML into numbers according to a simple set of rules. These numbers are then transformed into sound in the black box. Mention the Hinman collator here and stereoscopy. Used the First Folio Hamlet and the Quartos variants as the test data. One stream Two steams to create an audio version of a steroscopic illusion.