Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (9)
Similar to Designing Successful Heritage Crowdsourcing Projects
Similar to Designing Successful Heritage Crowdsourcing Projects (20)
More from Mia
More from Mia (20)
Recently uploaded
Recently uploaded (20)
Designing Successful Heritage Crowdsourcing Projects
- 1. Designing Successful Heritage Crowdsourcing Projects Mia Ridge, @mia_out Digital Curator, British Library Friedrich-Meinecke-Institute, Freie Universität Berlin, December 2015
- 2. https://www.flickr.com/photos/statensarkiver/8975684669 Overview Definitions Key examples Designing successful heritage crowdsourcing projects
- 3. About me
- 5. 'Collecting experiences of WWI'
- 7. Definitions
- 8. What is crowdsourcing? Crowdsourcing (Jeff Howe and Mark Robinson, Wired, 2006): 'taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call'
- 9. What is crowdsourcing? Crowdsourcing (Jeff Howe and Mark Robinson, Wired, 2006): 'taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call' Or, as Clay Shirky's cognitive surplus, 'the spare processing power of millions of human brains'
- 10. Heritage crowdsourcing as volunteering https://www.flickr.com/photos/statelibraryofnsw/4659373140
- 11. Crowdsourcing in cultural heritage Asking the public to help with tasks that contribute to a shared, significant goal or research interest related to cultural heritage collections or knowledge.
- 12. Crowdsourcing in cultural heritage Asking the public to help with tasks that contribute to a shared, significant goal or research interest related to cultural heritage collections or knowledge. The activities and/or goals should be inherently rewarding.
- 13. Basically... Transforming input content into output content ...via a powerful purpose and / or enjoyable tasks that people want to help you with
- 14. Why crowdsourcing in GLAMs?
- 15. Digitisation backlog: collections are big, resources are small https://www.flickr.com/photos/statensarkiver/8975684427
- 16. Fix the 'semantic gap', enhance discoverability
- 17. Access external specialist expertise https://www.flickr.com/photos/usnationalarchives/6990188224/
- 19. Defining success
- 23. Key examples
- 24. reCAPTCHA
- 25. NLA Trove: OCR correction http://trove.nla.gov.au/
- 26. 179 million lines of text corrected
- 27. NYPL 'What's on the Menu?'
- 29. Task complexity vs audience size https://www.flickr.com/photos/library_of_congress/2162650585
- 34. The power of niche projects
- 39. Designing successful crowdsourcing projects • Project design • User experience (interface, interaction) design – Onboarding, first task – Sustained participation despite changes in motivation over time
- 41. Who participates in crowdsourcing? • People who are passionate about your subject / people who like doing the task you're offering • People who can't volunteer in regular hours or at your venues • Super-volunteers and passers-by • Amateurs, professionals, 'pro-ams'
- 42. Motivations for participation • Altruistic – helping to provide an accurate record of local history • Intrinsic – reading 18thC handwriting is an enjoyable puzzle or they're interested in the subject • Extrinsic – an academic collecting a quote from a primary source
- 43. Intrinsic motivations for participation • fun • the pleasure in doing hobbies • the enjoyment in learning • mastering new skills, practicing existing skills • recognition • community • passion for the subject State Library of Queensland, Australia https://secure.flickr.com/photos/statelibraryqueensland/3198305152/
- 44. Motivations as design guide People crave: • satisfying work to do • the experience of being good at something • time spent with people we like • the chance to be a part of something bigger (Jane McGonigal, 2009)
- 46. Basically... Transforming input content into output content ...via a powerful purpose and / or enjoyable tasks that people want to help you with
- 48. Link text to motivations 'With a few keystrokes, you could bring a family together' 'We know the names of these children; can you help us tell their stories?' 'Kill Time. Make History.' 'Historians need your help!'
- 50. Design for casual and super-contributors http://blog.oldweather.org
- 51. Simple tasks as stepping stones http://www.fossilfinder.org/
- 52. Scaffold tasks, minimise distractions NYPL 'What's on the Menu?'
- 53. Onboarding and casual games
- 54. Onboarding and casual games Tutorials built into the interface Feedback on progress towards goals Skills matched to challenge Low risk of failure
- 55. Provide social proof; anticipate anxiety It’s easy! No registration required! So far: 1,331,935 dishes transcribed from 17,545 menus
- 56. Design for on-going participation
- 57. Nurture curiosity 'Discuss this personnel record' http://www.measuringtheanzacs.org
- 59. Up the challenge: mobiles and tablets http://familysearch.org/
- 60. Project design
- 61. Project design • Plan to store and process results from crowdsourcing • Plan to measure success • Plan to contribute to your engagement strategy and digitisation goals • Understand the appetite for risk • Reality check your plans
- 62. Don't forget to invite people
- 63. #party host https://www.flickr.com/photos/nlireland/5786204856 Crowdsourcing as hosting a party
- 64. ‘no plan survives contact with the crowd’ With apologies to Helmuth von Moltke the Elder
- 65. Going off-piste '...None of the above is ready to send to you as I have more research to do' https://www.flickr.com/photos/swedish_heritage_board/10207262464
- 66. Balancing acts
- 67. Planning a graceful exit https://www.flickr.com/photos/fylkesarkiv/4545543824
- 68. Thank you! Questions? Mia Ridge @mia_out Digital Curator, British Library Friedrich-Meinecke-Institute, Freie Universität Berlin, December 2015
Editor's Notes
- Thank you for the invitation to speak. Abstract: Based on a review of the most successful international crowdsourcing projects, this talk will look at the attributes of successful crowdsourcing projects in cultural heritage, including interface and interaction design, participation in community discussion, and understanding participant motivations.
- Start with definitions; then show some key examples and discuss what can be learnt from them to apply when designing other projects. We’ll also look at the role of the participant community and the importance of using participant motivations as design guidance.
- Before we get going...
- Why am I here talking about this? Worked in museums for a long time, got interested in opportunity between public engagement and need to enhance collections. Made games c2010, then did PhD in digital history, including crowdsourcing as a stepping stone to engagement with the practices and skills of history. Edited a book.
- Also some experience working with participatory WWI projects. In many ways did the opposite of the advice I will give you today, but in some ways that's how I know the advice is good. My advice - don't have a series of complicated, specialist tasks on a site with syntax that baffles even the reasonably tech-proficient.
- Still, it matched what a tiny number of people wanted to do, so it did achieve some success in gathering structured data about allied WWI military units and collecting personal accounts of the war held in public collections.
- Where it came from... Open call to unknown people sometimes scares people in cultural heritage. Problematic coining - not usually a crowd or outsourcing - but a bit late now. I've given two other ways of looking at it, both closer to hobbies, leisure activities.
- Where it came from... Open call to unknown people sometimes scares people in cultural heritage. Problematic coining - not usually a crowd or outsourcing - but a bit late now. I've given two other ways of looking at it, both closer to hobbies, leisure activities.
- Where it came from... Open call to unknown people sometimes scares people in cultural heritage. Problematic coining - not usually a crowd or outsourcing - but a bit late now. I've given two other ways of looking at it, both closer to hobbies, leisure activities.
- My definition – partly proscriptive as well as descriptive. Tasks like tagging, collecting, transcribing, describing cultural heritage collections, undertaken by distributed, possibly anonymous participants. Participation possibly as altrustic or intrinsically-motivated 'volunteers', or as side-effect of playing games or own work on cultural / historical materials.
- No financial rewards so has to be rewarding.
- Those terms 'powerful purpose' and 'enjoyable tasks' are key and why I'll talk about design and participant motivations today.
- GLAMs are galleries, libraries, archives, museums.
- Manually enhancing collections records is expensive and time-consuming. Very few orgs have the resources for straight digitisation.
- Even when metadata or information records are created by professional cataloguers, the content is often designed for internal or specialist users and doesn't use the everyday language our audiences might use to find material.
- There's a lot of specialist expertise outside the museum. There's an online community for almost every topic or type of item under the sun.
- Well-designed projects can help people discover new interests, or just encourage them to have a brief moment of deeper engagement with cultural heritage
- If I'm talking about 'successful projects', what does that mean?
- The number of tasks completed. To achieve this, must also have succeeded in letting people know about your project and providing an interface that lets enough of them get started on their first tasks.
- You can also look at the number or type of people engaged in the tasks. This is often important for organisations whose mission is to reach the public, whether that's to give them an experience of contemporary science or access to their history through specific collections. The Zooniverse projects have reached well over a million people worldwide. Some of scientific projects might also look at the impact of publications on social media and in journals that result from their projects. Museums might look at the number of researchers who find their digitised collections.
- Finally, you can look at the number of people who are deeply engaged - people whose feelings or knowledge about the material or the underlying disciplines change to the extent that they change some aspect of their behaviour. http://herbariaunited.org/wiki/Harry_Corbyn_Levinge or http://herbariaunited.org/wiki/Augustin_Ley
- Lots of types of tasks and output data, source materials, validation methods, organisational structures, goals...
- You've all helped correct text (or transcribe audio)
- Designed to let people get on with correcting errors they'd come across when doing their work but satisfying enough task that people do it for fun. Minimised barriers to participation.
- Every time I talk about this site I have to go check their stats because they go up all the time. The task design is satisfying enough that some people go to the site just to transcribe. A few people spent a lot of time doing the task - a pattern we'll see in nearly every project.
- Really focussed design. Altruistic and subject specialist motivation; clear sense of what to do next… Also topical content - if there are new menus, menus relevant to events (Superbowl, in this case); tantalising snippets of content... Low friction design - friction is anything that makes it harder or delays participation. While minimising friction, also look for points that might cause anxiety. Empathy for your participants is a huge design asset.
- Complex task - marking up transcriptions in XML - on difficult source material. Has a small number of very productive super-taggers… manual validation creates backlog and delay in approving content reduces feelings of reward. Post to the blog about progress help make up for it. Media coverage helped - each round drew in a few contributors.
- Old Weather is a 'Zooniverse' project that aims to extract weather information from historic ships logs for use by climate scientists. The 'Zooniverse' is the organisation behind many citizen science projects, some of which have developed into partial citizen history projects. Powerful purpose!
- Zooniverse originally created forums to help answer questions when busy after launch; but turned out to provide place for wonderful discussion. Here volunteers have compiled detailed guides to understanding handwriting on particular ships
- The power of finding the right audience, particularly through something as fast as social media. There are a lot of map fans in the world and they were eager to interact with the BL's collection.
- Able to learn from lots of previous projects; put own twist on
- Matched with computational techniques. Fed data to software, asked it to learn the visual characteristics of things depicted. Able to collect images with 'sideburns'. In this interface you can correct or mark for review, helping the software learn. (Makes you question the definition of a sideburn!)
- There is a long tradition of avocational historians, sometimes working on collaborative projects - local historians in the UK sometimes undertake long-term, complex projects. FreeBMD began in 1998 to 'provide free Internet access to the Civil Registration index information from England and Wales'. It has over 250 million records. Polish vs passion projects
- Two key stages in UX - convincing people to start and convincing people to stay
- http://discovery.ucl.ac.uk/171071/ 'many 'amateur' content creators are 'extremely self motivated, enthusiastic, and dedicated' and test the boundaries between 'between definitions of amateur and professional, work and hobby, independent and institutional' and quotes Leadbeater and Miller's 'The Pro-Am Revolution' on people who pursue an activity 'as an amateur, mainly for the love of it, but sets a professional standard'.'
- Why will people join in? Lots of reasons - same task (such as transcribing sections of a historic document) could be undertaken for altruistic, intrinsic or extrinsic reasons… Usually more than one motivation per person. Motivations change over time – different when deciding to start participating to when continuing. Often similar to motivations for volunteering in a GLAM. Much research on motivations for participation in non-commercial crowdsourcing projects comes from citizen science, or other 'community-based peer-production projects' like open source software. (Intrinsic motivation - behavior, such as a hobby, that is initiated without obvious external incentives. External motivation is activated by external incentives, such as direct or indirect monetary compensation, or recognition by others.)
- Some things are just intrinsically fun, whether it's playing with a hose or collecting football cards. Some of the sense of progression or mastery comes from the community, not only from the interface. Given the availability of digital platforms, the largest investment should be community interaction. Think about the hobbies of people you know (or your hobbies). There's a passionate community for every topic, if you look hard enough on the internet.
- Another way of looking at it.. Source: http://www.aam-us.org/resources/publications/museum-magazine/museums-as-happiness-engineers and http://www.youtube.com/watch?v=zJ9j7kIZuoQ&feature=plcp
- 'once you pop you can't stop' is your design goal: make a task small and satisfying enough that people want more.
- Those terms 'powerful purpose' and 'enjoyable tasks' are key and why I'll talk about design and participant motivations today.
- Design is important because lots of projects around; people will compare yours to others
- Match 'microcopy' messages to motivations. Demonstrate a close match between the crowdsourcing project and the mission of the organisation running it
- Your contribution makes a difference immediately… Effective design that makes correcting text a satisfying interaction. The user experience is further enhanced by the immediate appearance of the corrected text on the page (alongside the editing history). This shows participants the value of their contribution by making their corrections immediately available for the benefit of other users.
- '16,400 little boxes – one for each person who's contributed to oldWeather. The area of each box is proportional to the number of pages transcribed, between us all we've done 1,090,745 pages.' Some people do a lot of the work, and a lot of people do some of the work. This represents all 16,400 people who have transcribed at least one page for Old Weather (back in 2012) Source: http://blog.oldweather.org/2012/09/05/theres-a-green-one-and-a-pink-one-and-a-blue-one-and-a-yellow-one/
- Gives people a chance to get familiar with the material before trying other tasks
- Minimised everything not essential to the task on this interface. This tightly scaffolded user experience means low cognitive overhead, faster and more satisfying tasks. Excellent example of a 'microtask'.
- Easy-to-learn game-play; Simple controls; 'Forgiving' game-play with low risk of failure; Carefully managed complexity levels with a shallow learning curve, guidance through early levels, and inclusive, accessible themes; Sense of rapid progress and achievement Build any tests for skill or experience requirements into the interface Build tutorials for new skills into application at the point where its needed; provide good feedback on actions
- Easy-to-learn game-play; Simple controls; 'Forgiving' game-play with low risk of failure; Carefully managed complexity levels with a shallow learning curve, guidance through early levels, and inclusive, accessible themes; Sense of rapid progress and achievement = flow! Build any tests for skill or experience requirements into the interface; Build tutorials for new skills into application at the point where its needed; provide good feedback on actions
- Further down the page, requests for help with specialist tasks - 'need a second pair of eyes. Help fix misspellings, fill in missing data... ' If you can't bear to see a misspelled word go by, you might not be able to resist that. Ecosystems of tasks are a good way to attract a wider range of contributors to help with different processes within a project. They also include proof of the value of your contributions - 'x menus digitized and counting...' and a way to check out the content before committing to action.
- Motivations for participation change over time... One way to keep people motivated is to provide more responsibilities, or more complex tasks as their skills and knowledge grow. In some cases, new research questions or projects emerge from the community – this helps keep people engaged and is a great way to demonstrate impact.
- Allow participants to return to records, add info not strictly required for your task.
- Tiny tasks that collectively contribute to getting all the needed data
- Tiny screens and lack of physical keyboards a challenge but big opportunity re leisure time
- https://www.flickr.com/photos/internetarchivebookimages/14592517038
- Lesson – you can’t just issue invitations, you have to stick around. Allow time for community interaction and marketing
- Release early and often (if you can). Projects change once a community finds them. Allow time to update after launch as things will need to be tweaked and participants often have good ideas
- Sometimes participants get distracted by the task
- Find balance between participants' interests and organisational goals, even in projects that seem purely contributory
- With any luck, one day you will run out of content to crowdsource - what then? Where does your community go?