Strategic management

1. Machine learning and
human capital
complementarities:
Experimental
evidence on bias
mitigation
By: Prithwiraj Choudhury, Evan Starr, Rajshree Agarwal
Presented by: Yasmeen YAISH

2. • How can firms mitigate input
incompleteness bias to unlock the
potential of ML?
• How may human capital complement
ML to do so?
Research questions

3. What does most of ML literature focuses on?
Identify the set of possible ways the machine could
be misled, and then feed the ML technology as a way
to train the algorithm.
What did they miss?
If and how human capital may serve as a
complement to ML as a potential solution to bias
arising from input incompleteness.
Gap

4. Hypothesis
ML will not address biases due to input
incompleteness without complementary
domain specific expertise, and user-interface
complexities of ML require that humans who
provide such expertise also have
complementary vintage specific human capital.

5. Mechanism
Observational tests = Assumptions:
1. Patent language dynamically changes over time.
2. Complementarities between ML and domain-expertise.
Experimental tests:
3. Advantage of ML: Does Machine learning make my work
better and faster?
4. Disadvantage of ML: Bias of ML to to find the most similar
but not the most relevant.
How the human capital can compliment the bias in ML?
5. Domain expertise.
6. CS&E.
ØSample: 221 MBA students, random samples / compared ML
vs. Domain expertise vs. CS&E.

6. Experiment Variables
Dependent variables:
• The silver bullet patent.
• Productivity.
Independent variables:
• ML – Sigma / not using.
• Expert Advice – Receive / didn’t.
• CS&E – Degree/ No degree.
Controls:
• Section.
• Gender.
• Whether the individual has a partner.
• U.S. citizen.

7. Results
Where: Managers to consider 2 attributes of human capital: Domain expertise & vintage specific skills.
Why: Domain expertise complements ML by correcting for the (strategic) incompleteness of the input
to the ML tool, while vintage-specific skills (CS&E) ensure the ability to properly operate the technology.
• Narrow relevant prior art = ML
patent promising outcomes.
• No domain specific expertise = ML
bias predictions.
• Vintage-specific skills help navigate
the ML user interface.

8. Contributions
• Growing literature on bias in ML: Incompleteness as a source of bias.
• Role of domain-specific expertise as a complement to ML.
• Role of vintage specific skills as a complement to ML.
• Productivity differentials arising from vintage-specific skills (CS&E) contribute to the strategic
management of innovation literature on pace of technology substitution.

9. What makes the paper published in the top journal?
This study is the first to provide evidence
on both its relevance for prediction, and
offer a potential solution that does not
require opacity of ML algorithms, by
highlighting the complementary role of two
critical human capital attributes—domain-
specific expertise and vintage specific skills.

10. Limitations
• Unable to examine performance improvements over longer
periods of time.
• It is unclear if constant exposure and learning-by-doing by
workers would cause the relative differences between the
groups to grow or shrink over time.
• Generalizability: MBA students vs. highly skilled individuals.
• Finding the silver bullet was relatively uncommon.
• Research context and technology vintages are very specific—
applicable to USPTO's development of the ML tool Sigma
relative to Boolean search.
• Sigma is a relatively early stage ML tool.
• Results should be interpreted with caution.

11. Future directions
• Examine performance improvements over longer periods of time.
• Clarify if constant exposure and learning-by-doing by workers would cause the relative differences
between the groups to grow or shrink over time.
• Replicate the experiment using only highly skilled individuals.
• Investigate the the silver bullet in an even larger sample.
• Replicate the experiment with another ML tool other than Sigma to support the results.
• Add to the budding empirical research examining evolution in the productivity of all ML technologies,
and their contingencies.
• Study how long ML technologies will privilege those with computer science and engineering
backgrounds.

12. THANK YOU !