Kurt Luther¹
Kelly Caine²
Kevin Ziegler¹
Amy Bruckman¹
¹Georgia Tech
²Indiana University
Why It Works (When It Works):
Suc...
2Wikipedia http://www.wikipedia.org/
3Linux http://www.linux.org/
4Mozilla Firefox http://www.mozilla.com/en-US/firefox/firefox.html
5Mass Animation (Live Music) http://www.massanimation.com/
6Scratch http://scratch.mit.edu/
7Star Wars Uncut http://www.starwarsuncut.com/
Kurt Luther¹
Kelly Caine²
Kevin Ziegler¹
Amy Bruckman¹
¹Georgia Tech
²Indiana University
Why It Works (When It Works):
Suc...
Preview
Background
Study 1 (Qualitative)
Methods
Results & Discussion
Study 2 (Quantitative)
Methods
Results & Discussion
...
Background
Online Creative Collaboration
People meet and communicate primarily over the Internet
with the shared goals of ...
Background
Newgrounds
Oldest and largest host of Flash-animated movies and
games (founded 1995)
Top 1,000 most-visited web...
12Newgrounds http://www.newgrounds.com/
Background
Collabs on Newgrounds
Hundreds organized each year on Newgrounds discussion
forums
Each collab has 1 leader and...
Traditional definitions of success unsuitable because OSS
projects are ongoing and iterative (no concept of
“completion”)
...
Success defined as completing a collab; incomplete
collabs are “failed” or “dead”
Ratings, popularity, learning often view...
Study 1 (Qualitative)
Methods
Semi-structured interviews with 17 animators
14 interviewed via phone, 3 via email
Most part...
Planning
& Structure
Study 1 (Qualitative)
Results & Discussion
17
Reputation
& Experience
Communication
& Dedication
Coll...
Successful leaders have a clear
goal and a plan for achieving it
Collaborations fail because people get an
idea in their h...
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
Technical specifications (“specs”)
deemed especial...
Study 1 (Qualitative)
Results & Discussion
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
P1: Col...
Leaders with standing in the
community are more likely to
succeed
It definitely helps when the leader is
someone who alrea...
P2: Animators who are well-
known in the community are more
likely to lead successful collabs.Planning
& Structure
Reputat...
Leaders should be skilled and
experienced animators, first and
foremost
I think it also is important that people have
a su...
P3: Animators who have
experience with Flash and past
collabs are more likely to lead
successful collabs.
Experience with ...
Successful leaders are
communicative, dedicated, and
personally invested
A collaboration can’t succeed without
[communicat...
Artists look for collabs that
generate high activity and
discussion in the community
The thing about collabs, it’s all abo...
P4: Collabs whose members
frequently communicate are more
likely to be successful.
P5: Collabs whose leaders
frequently co...
Study 2 (Quantitative)
Methods
Scraped 137,328 forum threads on Newgrounds
Randomly sampled 892 threads into 3 bins:
high ...
Study 2 (Quantitative)
Methods
1) Content analysis of collab thread first posts
(themes, specs, gatekeeping policies, auth...
Study 2 (Quantitative)
Results & Discussion
87.4% of collabs failed
Only 112 of 892 collabs succeeded
Success is difficult...
Study 2 (Quantitative)
Results & Discussion
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
Leader...
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
P1: Collabs with initial planning
and structure, e...
Communication
& Dedication
Leaders of successful collabs have
more completed solo projects and
collabs, more awards, more ...
P2: Animators who are well-
known in the community are more
likely to lead successful collabs.Planning
& Structure
Reputat...
Leaders of successful collabs post
333% more replies
All members of successful collabs
post 228% more replies
Posts per da...
P4: Collabs whose members
frequently communicate are more
likely to be successful.
P5: Collabs whose leaders
frequently co...
Comparisons & Contrasts
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
There are only two ways we...
Comparisons & Contrasts
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
Successful collabs are run...
Comparisons & Contrasts
Planning
& Structure
Reputation
& Experience
Communication
& Dedication
Successful collabs and OSS...
Implications
Online creative collaboration expanding to new domains
(e.g. video, music, animation)
Which lessons learned f...
Implications
Possibility 1: Skilled, experienced leaders
In both collabs and OSS projects, leaders with skill and experien...
42Pipeline coming soon
Thank You
Co-authors:
Amy Bruckman, Kelly Caine, Kevin Ziegler
Study participants
Georgia Tech ELC Lab members
Newgrounds ...
Upcoming SlideShare
Loading in...5
×

Success Factors in Online Creative Collaboration

483

Published on

Presented by Kurt Luther (GVU Center, Georgia Tech) in the session "Create, Donate, Collaborate" at the 2010 ACM Conference on Supporting Group Work (GROUP 2010), in Sanibel Island, GA.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
483
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Success Factors in Online Creative Collaboration

  1. 1. Kurt Luther¹ Kelly Caine² Kevin Ziegler¹ Amy Bruckman¹ ¹Georgia Tech ²Indiana University Why It Works (When It Works): Success Factors in Online Creative Collaboration 1
  2. 2. 2Wikipedia http://www.wikipedia.org/
  3. 3. 3Linux http://www.linux.org/
  4. 4. 4Mozilla Firefox http://www.mozilla.com/en-US/firefox/firefox.html
  5. 5. 5Mass Animation (Live Music) http://www.massanimation.com/
  6. 6. 6Scratch http://scratch.mit.edu/
  7. 7. 7Star Wars Uncut http://www.starwarsuncut.com/
  8. 8. Kurt Luther¹ Kelly Caine² Kevin Ziegler¹ Amy Bruckman¹ ¹Georgia Tech ²Indiana University Why It Works (When It Works): Success Factors in Online Creative Collaboration 8
  9. 9. Preview Background Study 1 (Qualitative) Methods Results & Discussion Study 2 (Quantitative) Methods Results & Discussion Comparisons & Contrasts Implications 9
  10. 10. Background Online Creative Collaboration People meet and communicate primarily over the Internet with the shared goals of working together and creating new artifacts. a.k.a Yochai Benkler’s peer production Canonical examples are Wikipedia, open-source software (OSS) projects 10
  11. 11. Background Newgrounds Oldest and largest host of Flash-animated movies and games (founded 1995) Top 1,000 most-visited websites (Alexa.com) 2,200,000 registered members 180,000 member-uploaded movies and games Solo projects: single-author, individually made Collabs: multi-author, collaboratively made Accepted movies and games can attract 800,000- 1,000,000 views or more 11
  12. 12. 12Newgrounds http://www.newgrounds.com/
  13. 13. Background Collabs on Newgrounds Hundreds organized each year on Newgrounds discussion forums Each collab has 1 leader and 2 to 50+ artists Leader starts a collab thread on the forum, describes the idea in the first post, tries to recruit artists Production is modularized, not specialized Multi-author system allows leader to “co-author” up to 10 collaborators 13
  14. 14. Traditional definitions of success unsuitable because OSS projects are ongoing and iterative (no concept of “completion”) Success definitions tend to be complicated, often multi- dimensional English & Schweik’s 6-part classification Crowston et al.’s 18 measures of success Many attempts to identify success factors Self-selection for tasks, charismatic leader, meritocratic or rational culture, modular division of labor, use of collaborative technologies… Success Definitions & Factors Open-Source Software 14
  15. 15. Success defined as completing a collab; incomplete collabs are “failed” or “dead” Ratings, popularity, learning often viewed as secondary to completion Successful collabs… get exposure (via hosting on Newgrounds), increase the creators’ Batting Average, and may generate sponsorship income What factors influence success in collabs? Success Definitions & Factors Collabs 15
  16. 16. Study 1 (Qualitative) Methods Semi-structured interviews with 17 animators 14 interviewed via phone, 3 via email Most participants were leaders, some were artists, but all had collab experience Ranged from novice to expert, amateur to pro Represented six countries 16
  17. 17. Planning & Structure Study 1 (Qualitative) Results & Discussion 17 Reputation & Experience Communication & Dedication Collab just started Collab in progress
  18. 18. Successful leaders have a clear goal and a plan for achieving it Collaborations fail because people get an idea in their head and they can’t accurately convey their vision to the people they want to participate. And the collaboration doesn’t appear very appealing because the person doesn’t seem organized. –J.R. Planning & Structure Reputation & Experience Communication & Dedication Study 1 (Qualitative) Results & Discussion 18
  19. 19. Planning & Structure Reputation & Experience Communication & Dedication Technical specifications (“specs”) deemed especially important You have to give people really concrete boundaries in terms of how to put their movie together—not the creative part, but the technical aspects of it—in order for it to succeed to begin with. –L.C. Study 1 (Qualitative) Results & Discussion 19
  20. 20. Study 1 (Qualitative) Results & Discussion Planning & Structure Reputation & Experience Communication & Dedication P1: Collabs with initial planning and structure, especially technical specifications, are more likely to be successful. Initial planning and structure: Number of planning/structural elements provided by leader in first post of collab thread 20
  21. 21. Leaders with standing in the community are more likely to succeed It definitely helps when the leader is someone who already has the respect of the other users… When a popular artist asks people to get involved, they already assume it’s going to have a certain level of quality to it. If it’s some unknown person who wants you to be involved in a collab, you don’t know what the quality is going to be like. –T.F. Reputation & Experience Planning & Structure Communication & Dedication Study 1 (Qualitative) Results & Discussion 21
  22. 22. P2: Animators who are well- known in the community are more likely to lead successful collabs.Planning & Structure Reputation & Experience Communication & Dedication Well-known: Number of forum posts, number of awards won, and Batting Average Study 1 (Qualitative) Results & Discussion 22
  23. 23. Leaders should be skilled and experienced animators, first and foremost I think it also is important that people have a substantial amount of experience on their own with Flash and animation before they take the leadership role in a project of that nature. Just because it leads to a lot of frustration when you know more than the person who’s essentially going to be giving you orders. –L.C. Reputation & Experience Planning & Structure Communication & Dedication Study 1 (Qualitative) Results & Discussion 23
  24. 24. P3: Animators who have experience with Flash and past collabs are more likely to lead successful collabs. Experience with Flash: Number of solo projects hosted on Newgrounds Experience with past collabs: Number of collabs hosted on Newgrounds Planning & Structure Reputation & Experience Communication & Dedication Study 1 (Qualitative) Results & Discussion 24
  25. 25. Successful leaders are communicative, dedicated, and personally invested A collaboration can’t succeed without [communication]. At least, not a complex one. A movie director doesn’t walk onto the set and say, “I want you to do this, this, and this. I’ll be back in a few hours to check on you.” A movie director is constantly there giving his guidance. –J.R. Planning & Structure Reputation & Experience Communication & Dedication Study 1 (Qualitative) Results & Discussion 25
  26. 26. Artists look for collabs that generate high activity and discussion in the community The thing about collabs, it’s all about community. If only one person’s interested, they’ll just regularly bump up the thread, but after awhile they’ll just give up or somebody just comes along and tells them, “No, this is rubbish.” But if it’s a really good idea, loads of people will take part and the thread will just keep being bumped and bumped up to the top page. –R.W. Planning & Structure Reputation & Experience Communication & Dedication Study 1 (Qualitative) Results & Discussion 26
  27. 27. P4: Collabs whose members frequently communicate are more likely to be successful. P5: Collabs whose leaders frequently communicate are more likely to be successful. Communication frequency: Number of replies by each leader/member in collab thread Planning & Structure Reputation & Experience Communication & Dedication Study 1 (Qualitative) Results & Discussion 27
  28. 28. Study 2 (Quantitative) Methods Scraped 137,328 forum threads on Newgrounds Randomly sampled 892 threads into 3 bins: high activity (>50 replies), medium (10-50), low (<10) Collected ID, title, reply count for each thread; date, author & message for each reply in thread Collected completed solo projects & completed collabs, awards won, Batting Average for each thread creator 28
  29. 29. Study 2 (Quantitative) Methods 1) Content analysis of collab thread first posts (themes, specs, gatekeeping policies, authorship policies, restrictions, leader contact info) 2) Categorization of collab thread outcomes (either success or failure) 3) T-tests comparing first posts (successful vs. failed collabs) 4) T-tests comparing leaders (successful vs. failed collabs) 5) T-tests comparing activity dynamics (successful vs. failed collabs) 29
  30. 30. Study 2 (Quantitative) Results & Discussion 87.4% of collabs failed Only 112 of 892 collabs succeeded Success is difficult—what makes it more likely? 30
  31. 31. Study 2 (Quantitative) Results & Discussion Planning & Structure Reputation & Experience Communication & Dedication Leaders of successful collabs put more planning/structural elements in the first post Themes** 0.94 vs. 0.73 Specs** 3.74 vs. 2.03 Gatekeeping policies* 0.39 vs. 0.25 Authorship policies** 0.29 vs. 0.11 Restrictions* 0.63 vs. 0.40 Leader contact info** 0.83 vs. 0.52 *p < 0.05, **p < 0.01 31
  32. 32. Planning & Structure Reputation & Experience Communication & Dedication P1: Collabs with initial planning and structure, especially technical specifications, are more likely to be successful. Study 2 (Quantitative) Results & Discussion Supported 32
  33. 33. Communication & Dedication Leaders of successful collabs have more completed solo projects and collabs, more awards, more forum posts, and higher Batting Averages Past solo projects** 2.80 vs. 1.70 Past collabs** 5.42 vs. 3.35 Awards won** 1.60 vs. 0.60 Forum posts* 986 vs. 586 Batting Average** 3.17 vs. 2.92 *p < 0.05, **p < 0.01 Planning & Structure Reputation & Experience Study 2 (Quantitative) Results & Discussion 33
  34. 34. P2: Animators who are well- known in the community are more likely to lead successful collabs.Planning & Structure Reputation & Experience Communication & Dedication Study 2 (Quantitative) Results & Discussion Supported P3: Animators who have experience with Flash and past collabs are more likely to lead successful collabs. Supported 34
  35. 35. Leaders of successful collabs post 333% more replies All members of successful collabs post 228% more replies Posts per day** (leaders only) 4.24 vs. 1.86 Posts per day** (all members) 24.22 vs. 7.28 *p < 0.05, **p < 0.01 Planning & Structure Reputation & Experience Communication & Dedication Study 2 (Quantitative) Results & Discussion 35
  36. 36. P4: Collabs whose members frequently communicate are more likely to be successful. P5: Collabs whose leaders frequently communicate are more likely to be successful. Planning & Structure Reputation & Experience Communication & Dedication Study 2 (Quantitative) Results & Discussion Supported Supported 36
  37. 37. Comparisons & Contrasts Planning & Structure Reputation & Experience Communication & Dedication There are only two ways we know of to make extremely complicated things. One is by engineering, and the other is by evolution. – Danny Hillis Collabs are engineered The leader sets expectations early on and reduces conflicts later on by devising a plan and sticking to it. OSS projects evolve The leader’s problem definition is open to distributed iteration; members organize themselves. 37
  38. 38. Comparisons & Contrasts Planning & Structure Reputation & Experience Communication & Dedication Successful collabs are run by leaders with technical skills and a record of success. Successful OSS projects have similar attributes, but members also emphasize “soft skills” (e.g. charisma). A certain base level of design and coding skill is required… but it’s far from the whole story. –Eric Raymond 38
  39. 39. Comparisons & Contrasts Planning & Structure Reputation & Experience Communication & Dedication Successful collabs and OSS projects are both characterized by frequent communication among leaders and all members. Collab leaders go down with the ship, while OSS leaders are vulnerable to “forking.” A dissatisfied [OSS] community… can always leave and start again under new leadership. – Joseph Reagle 39
  40. 40. Implications Online creative collaboration expanding to new domains (e.g. video, music, animation) Which lessons learned from Wikipedia and OSS transfer, and which don’t? Why? Clear similarities and differences, just within two domains (collabs vs. OSS) Are broader principles of online creative collaboration possible? 40
  41. 41. Implications Possibility 1: Skilled, experienced leaders In both collabs and OSS projects, leaders with skill and experience in their domain are more successful. Possibility 2: Frequent communication Both successful collabs and successful OSS projects had leaders and members who communicated frequently. Caveat: Must test these principles in other domains, to avoid assumptions of generalization 41
  42. 42. 42Pipeline coming soon
  43. 43. Thank You Co-authors: Amy Bruckman, Kelly Caine, Kevin Ziegler Study participants Georgia Tech ELC Lab members Newgrounds staff NSF CreativeIT award #0855952 Flickr photos by ian_munroe, macinate, and kmndr used with permission 43

×