The document discusses a study conducted by Aspiring Minds to predict which candidates for an IT services company would be able to learn programming through a 3-month training program. 1371 candidates were tested on English, quantitative ability, logical reasoning, and computer programming skills. The company shared data on candidates classified as high, average, or low performers. Aspiring Minds designed a genetic algorithm to build a model to predict performance. The model found that quantitative ability was not a useful predictor and that exposure to programming reduced the importance of logical ability. It also identified thresholds in English, computer programming, and logical skills for shortlisting candidates.

1. Aspiring Minds
www.aspiringminds.com
Outcome predictions using
test scores
Who can learn programming through a short 3-
month training?
Varun Aggarwal
Presented at Education Workshop at ICML, 2015

2. Case of a large IT Services company
• Companies wants to hire candidates who when trained for 3-months will learn programming to be deployed as
software engineers.
• 1371 candidates tested on AMCAT at the time of joining a large IT company. Modules tested:
• English Comprehension
• Quantitative Ability
• Logical Reasoning
• Computer Programming
• Company shares who succeeded and who did not. Candidates classified as High Performers, Average Performers and
Mid-Performers.

3. The Model Type
• Should be theoretically plausible
• Models need to be interpretable
• Models need to be simple
• Trade-off models between type 1 and type 2 error required
Class 1
Class 0
Not allowed
L > 400
E + 0.4*L > 870
E + 0.7*Q > 980
English
Logical
Designed a genetic
algorithm based system to
search the model-space

4. HP
76.66
AP
15.36
PP
7.98
Test Shortlist rate
40%
AP Rejected
26.88%
PP Rejected
32.88%
HP
82.49
AP
11.24
PP
6.27
Test Shortlist rate
29.31 %
AP Rejected
39.17%
PP Rejected
47.26%
HP
86.09
AP
9.1
PP
4.8
Test Shortlist rate
21.72%
AP Rejected
50.75%
PP Rejected
59.62%
Poor Performers
11.89%
Average Performer
18.38%
High Performers
69.73%
CURRENT ORGANIZATION
We can help the organization…

5. Knowledge Discovery:
Quant Ability not a useful parameter
Type 1
Type2

6. 0
10
20
30
40
50
60
15 20 25 30 35 40 45 50
%lowperformersremoved
Shortlist Rate
EL
ELC
COUNTER-INTUITIVE TO BUSINESS PEOPLE!!!
CP model gets you lower type 1 and higher SR

7. Knowledge Discovery:
Exposure to programming reduces need of logical ability
1
• Computer > 360
2
• 0.47*English + 0.98*Computer + Logical > 1196
• The first filter eliminates bottom 28%
candidates in Programming.
•Above a minimum benchmark, lack of
Programming knowledge can be offset by high
Logical Ability and English skills.
1
• 0.42* English + Logical > 722
2
• Logical > 464
OR
• Logical ability needs to above the mean.
• Above a minimum benchmark, lack of logical
Ability can be offset by high English
comprehension skills

8. Who gets a job? Is there a bias?
Who can succeed in a short-duration programming training?
What behaviors lead to consistent job failures?
Learnings