The document discusses using situational judgement tests (SJTs) to predict job performance. It presents data showing that candidates who can correctly identify the worst answer choice on SJTs consistently underperform on the job compared to those who can identify the best or worst options. Specifically, an insurance salesperson study found those who identified the wrong answer scored 11% lower on sales targets, while a manager study found a 14% difference in performance ratings between those who did or did not identify the worst choice. The document concludes SJTs can improve predictions of on-the-job success by assessing a candidate's ability to discern ineffective responses.

1. Aspiring Minds
www.aspiringminds.com
Learning from data
What behaviors makes people succeed?
Varun Aggarwal

2. What is the right answer?
a) Clearly define daily tasks for the team members, evaluate what they have done at the end of the day and establish
deliverables for the next day.
b) Design and clearly communicate a process for implementing fines and other monetary disincentives for not meeting the
agreed timelines.
c) Motivate the team to work harder and meet their deadlines by assuring them a monetary incentive at the end of the
peak season.
d) Rotate the work responsibilities amongst the team members, and assign them tasks other than what they have been
doing so far. The freshness of work would motivate them.
You are working as an Operations Manager in an air conditioners manufacturing firm. Since it is a seasonal
business, the work pressure is high in the months December to June and negligible from July to November. Now
the peak season is starting again and you notice that your team members are not very motivated, working at a very
slow pace and not meeting deadlines. What will you do?

3. Ask: What is the wrong answer?!
a) Clearly define daily tasks for the team members, evaluate what they have done at the end of the day and establish
deliverables for the next day.
b) Design and clearly communicate a process for implementing fines and other monetary disincentives for not meeting the
agreed timelines.
c) Motivate the team to work harder and meet their deadlines by assuring them a monetary incentive at the end of the
peak season.
d) Rotate the work responsibilities amongst the team members, and assign them tasks other than what they have been
doing so far. The freshness of work would motivate them.
You are working as an Operations Manager in an air conditioners manufacturing firm. Since it is a seasonal
business, the work pressure is high in the months December to June and negligible from July to November. Now
the peak season is starting again and you notice that your team members are not very motivated, working at a very
slow pace and not meeting deadlines. What will you do?

4. Scoring different answers
• Ask the candidate for the best strategy and worst strategy
• Tag one option as the wrong strategy: say wa
• If candidate endorses ws = wa, provide +1 (wa score)
• If candidate endorses bs = wa, provide -1 (ba score)
• Do not tag/care for a best answer (ba)

5. People who can’t identify wrong answer
consistently fail…
Job Profile Tests Performance criteria Result
Direct Sales Sales SJT
Percentage of each individual’s
sales target achieved
rwa = 0.35 (0.08)
rba= 0.25 (0.15)
Insurance Sales
Logical Ability,
Personality, Insurance
readiness
Sales SJT
Performance ratings provided to
individuals by their supervisors
rwa = 0.26 (0.02)
rba= 0.14 (0.00)
Goes to 11% from 5% with SJT
scores.
Manager
English,
Logical Ability,
Quant Ability,
Personality
Managerial SJT
Performance ratings provided to
individuals by their supervisors
rwa = 0.39 (0.10)
rba= 0.36 (0.10)
Goes to 34% from 20% with ‘worst
score’
Regression: 0.47
wa: 1.00
ba: 1.80
* Work done with Steve Stemler, Wesleyan University

6. Learnings
• Answer choices selected by candidates reflect their thinking process.
• Responses predict real-time situational judgement and on-job reactionary decisions.