Have you ever pondered on how you might be able to better engage your crowd on projects or jobs? You might call it a “campaign description,” “brief,” or project “scope,” but whatever your name is, this presentation is for you. Thought leader, Matt Johnston, CMO of uTest, discuses the ways in which we reach out to our crowd. A few questions explored: - What formats do you use and find most effective? - What methods are used for distribution? - How do you guide remote workers, and how much direction is needed? - How might we engage clients to understand the value of campaign descriptions?

1. Crowdsourcing Keys
The Never-Ending Journey To Define Scope & Briefs
|
Matt Johnston | @matjohnston | CMO @ uTest | April 2012

2. The Challenge
What We’re
Talking About
|
2

3. Boiled Down
• Two critical components to c-sourcing success:
1. What’s the nature & intent of the project?
2. Who’s the best fit to do a project?
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 3

4. The Challenge
Who’s The
Best Fit
|
4

5. Who’s The Best Fit?
• This 2nd question gets the attention, so we’ll focus elsewhere
• For uTest, matching algorithms based upon
– Based upon profile
– Based upon merit-based ratings
– Based upon other criteria (eg: load balancing)
• Structured data is vital to good matches
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 5

6. The Challenge
Defining Nature &
Intent Of Projects
|
6

7. First Things First
• Fundamental nature of your projects
– Creative brief ≠ technical spec ≠ advertising guidance
– Design vs. animation vs. dev vs. testing vs. advertising vs. content
• What customer is really trying to do and cares about
– Effectiveness vs. efficiency
- Signal-to-noise ratio?
- Precision?
- Brute force?
• Those who can’t articulate the
problem won’t find the solution.
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 7

8. Defining Work To Be Done
• Our journey:
– 2008 – We launched… and we sucked
– 2009 – More sucking
– 2010 – Started wrapping head around nature of the problem
– 2011 – More sophisticated
– 2012 – Solved most of the problem… and created several new ones
- Launched 4 new testing types, each w/ their own project definition challenges
• Key takeaways for us:
– Garbage in, garbage out
– Real customer of such info is our community
– Nature of project matters:
- If goal is predictability (technical work), unstructured data is the enemy
- If goal is creativity, unstructured data is still the enemy (just a necessary evil)
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 8

9. The Challenge
Examples
Past & Future
|
9

10. Examples: Project Setup
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 10

11. Examples: Project Scope
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 11

12. Input / Output Matters
• Even with right questions, we had room for improvement:
– Ux for customers to create input
- Information architecture and GUI design
- Integrations with legacy systems
– Ux for community to consume output
- Information architecture and GUI design
- Change management
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 12

13. Examples: Project Setup
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 13

14. Examples: Project Scope
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 14

15. Bottomline
• Two critical components to c-sourcing success:
1. What’s the nature & intent of the project?
2. Who’s the best fit to do a project?
• Should spend equal calories, hours and brain cells on each
|
In-The-Wild Testing for Functional + Security + Load + Usability + Localization 15

16. The Challenge
Questions?
Answers.
Matt Johnston | CMO @ uTest
mattj@utest.com | @matjohnston
|
16