TopCoder
Developing software through crowdsourcing
How TopCoder came
about?Jack Hughes
Tallan Inc. (Business
Data Services)
Sold for $920M
Lessons learnt from
Tallan?
Not happy with the operations of Tallan
Expensive and time consuming recruitment
process
Evolving technologies results in high employee
turnover rate
More can be done to save clients time and money
Crowdsourcing was the solution!
TopCoder v1.0
Consulting Business Model
Host competitions with monetary rewards from
sponsors
Build community strength
Gain popularity
Winner gets a job with the sponsor
HR firm
• Looking to
hire
Client
• Held
Competitions
TopCoder • Winner got
hired
Crowd
TopCoder v2.0
Competition Based Model
Crowd sourcing Software Development
Tasks of Software Development Process
Task
Conceptual
ization
Specificatio
n
Architec
ture
Component
Production
Application
Assembly
Certific
ation
Deploy
ment
v.2.0 PM PM PM COM PM PM PM
• Looking for
software
application
Client
• Held
Competitions
TopCoder • Winner got
rewarded
Crowd
TopCoder v2.1
Competition Based Model
Dissatisfied with $20 mln in revenue
Platform Managers costs too high
Tasks of Software Development Process
• Looking for
software
application
Client
• Held
Competitions
TopCoder • Winner got
rewarded
Crowd
Task
Conceptual
ization
Specificatio
n
Architec
ture
Component
Production
Application
Assembly
Certific
ation
Deploy
ment
v.2.0 PM PM PM COM PM PM PM
v2.1 PM & COM PM & COM COM COM COM PM
PM &
COM
Did it work?
2007 2008 2009
Total Revenue
(million)
$18.31 $18.75 $8.44
Cost of Platform
Managers
(million)
$4.89 $4.59 $1.88
Cost of PM vs
TR (%)
26.7% 24.5% 22.3%
Decrease in contest from 2008 to 2009
Switched from hourly based consultation to monthly
fee
Types of Competition
Algorithm Competition
Attract new members
Retain existing members
Software Development Competition
Provide clients with specific solutions
Expand TopCoder’s catalog
Algorithm Competitions
Timed competitions
Available in Java, C#, C++ & VB.Net
Single Round Match consists:
Coding Phase
Challenging Phase
System Testing Phase
Algorithm Competitions
Coding Phase
Lasts 75 mins
Contestants presented with the same 3 questions
Points awarded for successfully compiled solutions
Total time elapsed
Challenging Phase
Lasts 15 mins
Challenge the functionality of other competitors’ code
Successful challenge:
Defendant – loss of original submission points
Challenger – 50 point reward
Unsuccessful challenge:
Challenger – reduction of 25 points
Algorithm Competitions
System Testing Phase
Applied to all submitted code (not successfully
challenged)
Flawed code submission will lose all points
Automated tester applies a set of inputs
if submission’s output does not match expected
output, submission is considered flawed
All successful challenges will be added to the set of
inputs for the System Testing Phase
Example of an Algorithm
Competition
Example of Algorithm
Competition
011100011
Encrypt by adding to each digit to the sum of the
adjacent digits
[0+1], [0+1+1], [1+1+1] etc.
123210122
How does it attract & retain the
TopCoder Community
Think about your lab practices/tests
Challenging
Prove your worth
Self improvement
Community
TopCoder Community Activeness & Earning
Distribution
Latent
Pool
Normal
Active
Pool
82.5% 17.5% 95%
5%
Motivating the Community
Money is the most attractive thing
TopCoder ratings
How to motivate the latent
pool?
Monetary Award
$50 for successful compilation of code by latent
pool (for 5 rounds)
1.5 times prize money if they are the winner
Prospective Jobs

Top coder

  • 1.
  • 2.
    How TopCoder came about?JackHughes Tallan Inc. (Business Data Services) Sold for $920M
  • 3.
    Lessons learnt from Tallan? Nothappy with the operations of Tallan Expensive and time consuming recruitment process Evolving technologies results in high employee turnover rate More can be done to save clients time and money Crowdsourcing was the solution!
  • 4.
    TopCoder v1.0 Consulting BusinessModel Host competitions with monetary rewards from sponsors Build community strength Gain popularity Winner gets a job with the sponsor HR firm • Looking to hire Client • Held Competitions TopCoder • Winner got hired Crowd
  • 5.
    TopCoder v2.0 Competition BasedModel Crowd sourcing Software Development Tasks of Software Development Process Task Conceptual ization Specificatio n Architec ture Component Production Application Assembly Certific ation Deploy ment v.2.0 PM PM PM COM PM PM PM • Looking for software application Client • Held Competitions TopCoder • Winner got rewarded Crowd
  • 6.
    TopCoder v2.1 Competition BasedModel Dissatisfied with $20 mln in revenue Platform Managers costs too high Tasks of Software Development Process • Looking for software application Client • Held Competitions TopCoder • Winner got rewarded Crowd Task Conceptual ization Specificatio n Architec ture Component Production Application Assembly Certific ation Deploy ment v.2.0 PM PM PM COM PM PM PM v2.1 PM & COM PM & COM COM COM COM PM PM & COM
  • 7.
    Did it work? 20072008 2009 Total Revenue (million) $18.31 $18.75 $8.44 Cost of Platform Managers (million) $4.89 $4.59 $1.88 Cost of PM vs TR (%) 26.7% 24.5% 22.3% Decrease in contest from 2008 to 2009 Switched from hourly based consultation to monthly fee
  • 8.
    Types of Competition AlgorithmCompetition Attract new members Retain existing members Software Development Competition Provide clients with specific solutions Expand TopCoder’s catalog
  • 10.
    Algorithm Competitions Timed competitions Availablein Java, C#, C++ & VB.Net Single Round Match consists: Coding Phase Challenging Phase System Testing Phase
  • 11.
    Algorithm Competitions Coding Phase Lasts75 mins Contestants presented with the same 3 questions Points awarded for successfully compiled solutions Total time elapsed Challenging Phase Lasts 15 mins Challenge the functionality of other competitors’ code Successful challenge: Defendant – loss of original submission points Challenger – 50 point reward Unsuccessful challenge: Challenger – reduction of 25 points
  • 12.
    Algorithm Competitions System TestingPhase Applied to all submitted code (not successfully challenged) Flawed code submission will lose all points Automated tester applies a set of inputs if submission’s output does not match expected output, submission is considered flawed All successful challenges will be added to the set of inputs for the System Testing Phase
  • 14.
    Example of anAlgorithm Competition
  • 15.
    Example of Algorithm Competition 011100011 Encryptby adding to each digit to the sum of the adjacent digits [0+1], [0+1+1], [1+1+1] etc. 123210122
  • 16.
    How does itattract & retain the TopCoder Community Think about your lab practices/tests Challenging Prove your worth Self improvement
  • 17.
    Community TopCoder Community Activeness& Earning Distribution Latent Pool Normal Active Pool 82.5% 17.5% 95% 5%
  • 18.
    Motivating the Community Moneyis the most attractive thing TopCoder ratings
  • 20.
    How to motivatethe latent pool? Monetary Award $50 for successful compilation of code by latent pool (for 5 rounds) 1.5 times prize money if they are the winner Prospective Jobs

Editor's Notes

  • #3 http://news.cnet.com/CMGI-to-buy-Tallan-for-920-million/2100-12_3-264732.html