1. The Cost of Collaboration for Code and Art
Evidence from a Remixing Community
Benjamin Mako Hill1,2
mako@mit.edu
Andrés Monroy-Hernández2,3
amh@microsoft.com
1
Massachusetts Institute of Technology
2
Berkman Center, Harvard University
3
Microsoft Research
February 26, 2013
revision: b047d49 (2013/02/26)

2. Peer Production and Remixing
Benkler (2006); Raymond (1999); Lessig (2008); von Hippel (2005); Giles (2005);
Wilkinson and Huberman (2007); Kittur and Kraut (2008), etc.
2 / 17

3. Remixing
The reworking and recombination of existing creative artifacts.
Most commonly in reference to music, video, and interactive media.
Widespread, and an important new communication modality (e.g.,
Manovich 2005; Lessig 2009)
Especially among use youth (Jenkins 2006; Palfrey and Gasser 2008)
3 / 17

4. Criticism of Peer Production
Others have suggested that peer production
and remixing are amateurish and poor quality
(e.g., Lanier, 2010; Keen, 2007).
4 / 17

5. “Could Hamlet have been written by a committee,
or the Mona Lisa painted by a club? Could the
New Testament have been composed as a
conference report? Creative ideas do not spring
from groups. They spring from individuals. The
divine spark leaps from the finger of God to the
finger of Adam.”
– A. Whitney Griswold (1957)
5 / 17

6. Research Questions
RQ1: Are remixes, on average, higher quality
than single-authored works?
6 / 17

7. Research Questions
RQ1: Are remixes, on average, higher quality
than single-authored works?
RQ2: Are code-intensive remixes, on average,
higher quality than media-intensive
remixes?
6 / 17

8. Research Site
A computer programming
environment
7 / 17

9. Research Site
A computer programming
environment
An online community for
sharing projects
7 / 17

10. Research Site
A computer programming
environment
An online community for
sharing projects
Designed and built around
collaboration through remixing
7 / 17

17. Data and Measures
1,271,085 projects shared before April 1, 2011.
10 / 17

18. Data and Measures
1,271,085 projects shared before April 1, 2011.
Outcome:
Loveits
10 / 17

19. Data and Measures
1,271,085 projects shared before April 1, 2011.
Outcome:
Loveits
Independent Variables:
Collaborativeness (Is project a remix?).
Control blocks (similar to lines of code)
Media elements including image and sound files.
10 / 17

20. Data and Measures
1,271,085 projects shared before April 1, 2011.
Outcome:
Loveits
Independent Variables:
Collaborativeness (Is project a remix?).
Control blocks (similar to lines of code)
Media elements including image and sound files.
Additional controls:
Number of views
Creator tenure, gender and age
10 / 17

21. Methods
Negative binomial regression model on a count of loveits.
loveits = β + β log blocks + β log media + βisremix +
β log blocks × isremix + β log media × isremix +
βage + βjoined + βfemale + β log prevloveits + β log views
11 / 17

22. Results: Plots of Prototypical Projects
Blocks Media Elements
0.35
0.30
0.25
Loveits
De Novo
0.20 Remix
0.15
0.10
50 100 150 200 10 20 30 40
12 / 17

23. Results: Plots of Prototypical Projects
Blocks Media Elements
0.35
0.30
0.25
Loveits
De Novo
0.20 Remix
0.15
0.10
50 100 150 200 10 20 30 40
RQ1: Remixes are rated lower.
12 / 17

24. Results: Plots of Prototypical Projects
Blocks Media Elements
0.35
0.30
0.25
Loveits
De Novo
0.20 Remix
0.15
0.10
50 100 150 200 10 20 30 40
RQ1: Remixes are rated lower.
RQ2: Code-intense remixes are rated higher.
Media-intense remixes are rated lower.
12 / 17

25. Example: Code Intense Project
Original Remix
13 / 17

26. Example: Code Intense Project
Original Remix
Remix received about the same loveits than the
antecedent.
13 / 17

27. Example: Media Intense Project
Original Remix
14 / 17

28. Example: Media Intense Project
Original Remix
Remix received less loveits than antecedent.
14 / 17

29. Limitations
Ratings are a single and imperfect measure of
quality
15 / 17

30. Limitations
Ratings are a single and imperfect measure of
quality
Code & Media ≈ Functional & Artistic
15 / 17

31. Limitations
Ratings are a single and imperfect measure of
quality
Code & Media ≈ Functional & Artistic
Remixes and antecedents projects may
compete for loveits
15 / 17

32. Limitations
Ratings are a single and imperfect measure of
quality
Code & Media ≈ Functional & Artistic
Remixes and antecedents projects may
compete for loveits
Generalizability beyond Scratch, beyond kids,
beyond remixing.
15 / 17

33. Takeaways and Final Thoughts
Remixes are rated lower than de novo projects by
Scratch users.
16 / 17

34. Takeaways and Final Thoughts
Remixes are rated lower than de novo projects by
Scratch users.
More code = smaller gap
More media = wider gap
16 / 17

35. References
Benkler, Y. (2006, May). The Wealth of Networks: How Social Production Transforms Markets and
Freedom. Yale University Press.
Giles, J. (2005, December). Internet encyclopaedias go head to head. Nature 438(7070), 900–901.
Griswold, A. W. (1957, June). Baccalaureate address.
Keen, A. (2007, June). The Cult of the Amateur: How Today’s Internet is Killing Our Culture (3rd
Printing ed.). Crown Business.
Kittur, A. and R. E. Kraut (2008). Harnessing the wisdom of crowds in wikipedia: quality through
coordination. San Diego, CA, USA, pp. 37–46. ACM.
Lanier, J. (2010, January). You Are Not a Gadget: A Manifesto (1 ed.). Knopf.
Lessig, L. (2008, October). Remix: Making Art and Commerce Thrive in the Hybrid Economy.
Penguin Press HC.
Raymond, E. S. (1999). The cathedral and the bazaar: Musings on Linux and open source by an
accidental revolutionary. Sebastopol, CA: O’Reilly and Associates.
von Hippel, E. (2005). Democratizing innovation. Cambridge, Massachusetts: The MIT Press.
Wilkinson, D. M. and B. A. Huberman (2007). Cooperation and quality in wikipedia. In Proceedings
of the 2007 international symposium on Wikis, Montreal, Quebec, Canada, pp. 157–164. ACM.
17 / 17

36. Example and
Data Appendix

37. Empirical Model
loveits = β + β log blocks + β log media + βisremix +
β log blocks × isremix + β log media × isremix +
βage + βjoined + βfemale + β log prevloveits + β log views
19 / 17

38. Empirical Model
loveits = β + β log blocks + β log media + βisremix +
β log blocks × isremix + β log media × isremix +
βage + βjoined + βfemale + β log prevloveits + β log views
[RQ1] Are remixes, on average, rated as higher quality
than de novo projects?
19 / 17

39. Empirical Model
loveits = β + β log blocks + β log media + βisremix +
β log blocks × isremix + β log media × isremix +
βage + βjoined + βfemale + β log prevloveits + β log views
[RQ1] Are remixes, on average, rated as higher quality
than de novo projects?
[RQ1] Are code-intensive remixes, on average, rated as
higher quality than media-intensive remixes?
19 / 17

40. Regression Results
(Intercept) −3.359∗ −4.828∗
log blocks 0.166∗ 0.076∗
log media 0.384∗ 0.086∗
log isremix 0.279∗ −0.091∗
female 0.000 0.284∗
age −0.006∗ −0.002∗
joined −0.020∗ −0.008∗
prevloveits 0.568∗ 0.076∗
log blocks×
isremix 0.059∗ 0.021∗
log media×
isremix −0.305∗ −0.103∗
views 1.226∗
N 1239470 1239470
log L -901476 -707714
∗
∗ indicates significance at p < .01
20 / 17