How to design a social computing system that people want to use


Published on

Guest lecture by Kurt Luther in Prof. Leysia Palen's "Social Computing" course, Department of Computer Science, University of Colorado, Boulder, January 2014.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Has anyone seen this movie? (Field of Dreams)Awesome 80s movie starring Kevin CostnerFarmer fallen on hard times, starts hearing whispers “If you build it, they will come”Decides to build a baseball field in his cornfield, causing ghosts of baseball players from the 1920s to appearPeople come from all around the world to watch them playIn the early days of social computing, this was a pretty safe betYou could build something and often they *would* comee.g. MOOSE Crossing, 1997
  • Unfortunately it’s less true nowThere’s way more stuff out thereJust having a website or piece of software isn’t enough to get people to use itI’ve learned this the hard way and wanted to share some lessons from the Social Computing field, along with my personal experiencesI’ve built a few social computing systems that people actively use
  • One is Proveit, a tool for managing references in Wikipedia articlesBuilt with colleagues at Georgia Tech and Wikimedia FoundationMore than 7,000 active users (Jimmy Wales)Integrated into English Wikipedia
  • Who Gives a Tweet?A site for rating people’s tweetsBuilt with Michael Bernstein (Stanford), Paul Andre (Facebook)Covered by press (CNN, TIME, etc.)More than 40,000 tweets rated
  • Pipeline: tool for crowdsourcing creative projects (dissertation work)Colleagues at Georgia TechUsed for animations, films, games, artworks – more than 100 projectsFeel free to check it out (
  • I’m mainly going to talk about Pipeline, and here are 8 recommendations I can distill from that experienceI hope they will be useful to youCaveats: just my experience, results may varyBuilding a creativity support toolUltimate goal was research
  • First recommendation is pick a community to focus onTempting to build a system for “everyone” but is very difficult, often ends up less than ideal for anyonePick a community that will hold your interest because it’ll be a while
  • I focused on Newgrounds, community of Flash animation enthusiasts2 million members100k projectsBeen around since 1995Example of extreme coordination challenges: distributed, volunteers, amateurs, complex multimedia artifact, expressive goalsI’m a fan of animation and online creativity so this was a good fit
  • Once you’ve picked a target community, you need to deeply understand how it works and what’s not workingParticipation builds credibilityUser-centered design: designing “for” users, users in a reactive roleParticipatory design: designing “with” users (co-designers)Meta-design: designing for designersPurposefully underdesign to allow for unanticipated usesEspecially relevant for creativity support tools
  • Interviews: Focus on leadership, challenges, overall processContent analysis: How many succeeded/failedElements of successful collabs
  • Poor technological supportNewgrounds – most collabs organized on forumsDesigned for conversation, not complex collaboration1 project – 317 replies over 11 pages!
  • As you know, we built Pipeline to streamline this process, to support redistributed leadership, and encourage more ambitious, successful projects.But the trick with social software, especially for research purposes, is you need a critical mass of people using it to get any idea of how well it works.So from the beginning of our design process, we were seeking out ways to involve users, not just for testing, but to generate interest in the community.
  • That leads to my next recommendation.
  • Here’s an example email I sent to my informants. Notice the highlighted part: “I’d like to invite you to be one of the first people to try Pipeline. One reason is because your interview helped make Pipeline what it is – you’re an expert, we value your opinion, and we really will change Pipeline based on your feedback. Another reason is to show my gratitude for your help. Without you, Pipeline could not exist.”
  • Consider an open-source model
  • Released as free OSS on GitHubCustomizable to augment (not replace) existing communities500+ users, 100+ projects
  • Keep community updated via blogLet them know exactly what’s changed, been added
  • I interned thereNice, but created some problems- They have different priorities (business)Wrote code that wasn’t deployed for two yearsMeanwhile what do you do?
  • Organized “Game Jam 6” in collaboration with Newgrounds community leaders and staffPrize money contributed by staffTeams have 72 hours to create a new, working video game for a given theme (hallucinations)Teams randomly assigned just before contests. 4 per team: artists, programmer, musician
  • About a dozen games were completed, many used Pipeline. Not everyone was successful, but many were.
  • One of my favorite games from the contest was “Trippin Kitchen”, where you play a grandmother who accidentally drank expired milk, causing her to trip out. She uses her frying pan to defend her kitchen against rogue hamburgers and other culinary enemies.
  • We wanted to see if Pipeline really works.
  • Holiday Flood: collaborative artwork made by 30 artists from around the world over 6 weeks.The goal was to produce 24 artworks, 2 for each of the 12 days of Christmas.Not only that, they had to submit the artworks on a given day. So “12 Drummers Drumming” was submitted December 14, “11 Pipers Piping” on Dec 15, etc.But there’s more. Notice each artwork has a thumbnail in the corner with the artist’s signature. When all these thumbnails are combined, they fit together like a puzzle, producing…
  • … this hidden Season’s Greeting message. All of this was part of an interactive gallery submitted to Newgrounds, complete with a custom arrangement of the “12 Days of Christmas” carol.The complexity of Holiday Flood shows how Pipeline enabled a more complex, ambitious, successful project. But we also triangulated this evidence with the experiences of actual users. As this interview quote shows, Holiday Flood’s leaders agreed Pipeline played a big role.
  • Just as people appropriate content and ideas, they will appropriate processesEspecially with creativity support tools, you can’t anticipate what users will do, who will want itRarely only one community like yours
  • Unexpected use example:Pipeline software tracks when members complete tasksYet leader manually checked off tasks, because it’s motivating for members
  • American Anthem, about a speed skater with autism who wants to compete in OlympicsIndependent film, greenlit to be made into a feature filmPreproduction work on Pipeline
  • Newgrounds-themed texture pack for Minecraft
  • GISHWHES 2012, 2013
  • Here are the recommendations again. I hope they’ll be useful to you in this class and beyond.
  • And maybe we can revise the Field of Dreams quote from the beginning. “If you build it WITH THEM, they will come!”
  • How to design a social computing system that people want to use

    1. 1. How to Design a Social Computing System that People Want to Use Kurt Luther Carnegie Mellon University
    2. 2. “If you build it, they will come!” 1
    3. 3. “Nobody came!”  2
    4. 4. 3
    5. 5. 4
    6. 6. 5
    7. 7. 7 Recommendations 1. 2. 3. 4. 5. 6. 7. Focus on a community and learn what it needs Use informants as beta testers Release early and often Form a relationship with the staff Organize events around your software Evaluate and iterate Embrace unexpected uses 6
    8. 8. 1. Focus on a Community • Systems designed for “everyone” often aren’t great for anyone • Find a community that interests you and focus on them 7
    9. 9. 1. Focus on a Community 8
    10. 10. 1. Learn What the Community Needs • Triangulate with mixed methods • Participate, don’t just observe • Involve users throughout design process – User-centered design (Norman & Draper 1986) – Participatory design (Schuler & Namioka 1993) – Meta-design (Fischer & Scharff 2000) 9
    11. 11. 1. Learn What the Community Needs • Research methods – Interviews with 17 Newgrounds members with collab experience – 5 years of participantobservation on forums – Content analysis of ~900 collab threads Joseph Blanchette, 24, USA Eric Carlson, 19, USA Luis Castanon, 27, USA Michael Frank, 19, USA Tom Fulp, 29, USA James Hole, 16, Australia Tyler Koch, 19, USA Massimo Maitan, 21, Australia Anders-Martin Meister, 16, Estonia Ross O’Donovan, 19, Australia Kraig Phillips, 27, USA Joseph Rooks, 21, USA Kester Smith, 21, UK “Sven”, 18, Netherlands Hans Van Harken, 17, Spain Robert Westgate, 21, UK “William”, 19, USA 10
    12. 12. 1. Learn What the Community Needs • Study results – More than 80% of collabs never completed – Leadership is crucial, but leaders often overwhelmed – Leaders try to increase success by simplifying projects, minimizing interaction – Poor technological support (Luther & Bruckman 2008; Luther et al. 2010) 11
    13. 13. 1. Learn What the Community Needs 12
    14. 14. 1. Learn What the Community Needs 13
    15. 15. 2. Use Informants as Beta Testers • They already know you • They’re knowledgeable about the community • They’re excited to see their comments acted upon 14
    16. 16. 2. Use Informants as Beta Testers 15
    17. 17. 3. Release Early and Often • Don’t just show up with a “complete” project • Early announcements provide more time for community buy-in • Frequent releases mean frequent opportunities for exposure – Keep the community informed • Linus’s Law – “Given enough eyeballs, all bugs are shallow” (Raymond 1999) 16
    18. 18. 3. Release Early and Often 17
    19. 19. 3. Release Early and Often 18
    20. 20. 4. Form a Relationship with the Staff • Staff can help in many ways – – – – – – Advertising Providing resources or prizes Endorsing your project Giving access to user data Offering insights into the community Generally making life easier • Much trickier if you’re a competitor 19
    21. 21. main office, near Philadelphia, PA 4. Form a Relationship with the Staff 20
    22. 22. 5. Organize Events around Your Software • Attract a critical mass of users in a short time • Online contests can be motivating, even if the prize is attention instead of money (Nickerson & Monroy-Hernandez 2011) • Special events are a hallmark of a vibrant online community (Kim 2000) 21
    23. 23. 5. Organize Events around your Software 22
    24. 24. 5. Organize Events around your Software 23
    25. 25. 5. Organize Events around your Software 24
    26. 26. 6. Evaluate and Iterate • Does your system do what you designed it to do? • How are you measuring success? – For creativity support tools, consider process as much as output • Iterate based on what you find 25
    27. 27. 6. Evaluate and Iterate • Evaluating Pipeline – Research questions • (How) does Pipeline foster more ambitious, successful projects? • (How) does Pipeline change leadership? – Methods • Case study of Holiday Flood – Interactive artwork organized on Pipeline • 1100+ Pipeline log events • 140+ Newgrounds forum posts • Interviews with 5 most active members 26
    28. 28. 12 Drummers Drumming (Dec. 14) 6 Geese Laying (Dec. 20) 11 Pipers Piping (Dec. 15) 5 Golden Rings (Dec. 21) 10 Lords Leaping (Dec. 16) 9 Ladies Dancing (Dec. 17) 8 Maids Milking (Dec. 18) 6. Evaluate and Iterate 7 Swans Swimming (Dec. 19) 4 Calling Birds (Dec. 22) 3 French Hens (Dec. 23) 2 Turtle Doves (Dec. 24) A Partridge in a Pear Tree (Dec. 25) 27
    29. 29. “Holiday Flood was all planned and plotted. It’s the reason we needed Pipeline. I doubt it would have worked out any other way.” –Renae (co-leader) 6. Evaluate and Iterate 28
    30. 30. 7. Embrace Unexpected Uses • Appropriation (Salovaara 2008) – Within the community – Outside the community • Unanticipated use is still use! – Learn from it and design for it 29
    31. 31. 7. Embrace Unexpected Uses 30
    32. 32. 7. Embrace Unexpected Uses 31
    33. 33. 7. Embrace Unexpected Uses 32
    34. 34. 7. Embrace Unexpected Uses 33
    35. 35. 7 Recommendations 1. 2. 3. 4. 5. 6. 7. Focus on a community and learn what it needs Use informants as beta testers Release early and often Form a relationship with the staff Organize events around your software Evaluate and iterate Embrace unexpected uses 34
    36. 36. “If you build it with them, they will come!” 35
    37. 37. Questions? • Thanks for listening! • Contact me –, • Try my software – Pipeline: – ProveIt: • Fix my software – Pipeline: – ProveIt: 36