Strategic management

1. Machine
learning and
human capital
complementariti
es: Experimental
evidence on
bias mitigation
Prithwiraj Choudhury | Evan Starr |
Rajshree Agarwal (2020)
Professor: Francesco Castellaneta
M2 RMI Student: Hai Hoa LU
PAGE 1

2. Contents
Synopsis of the study
Paper’s main ideas and theocratical
contribution
Assessment on Strength/Weaknesses
Ideas for future researches
PAGE 2

3. Synopsis of the study
Phenomenon
▪ The bias in ML and the complementary of human capitals that mitigate the bias
Research questions
▪ How can firms mitigate such bias to unlock the potential of ML?
▪ And, how may human capital complement ML to do so?
Empirical context
▪ Machine Learning (ML) technology used for patent Examination
▪ Characteristics for patent applications: novelty and nonobviousness
Proposed from the authors that individuals possess
▪ Domain expertise
▪ Vintage-specific skills
PAGE 3

4. Literature reviews and Gaps
PAGE 4
• AI's substitution for humans in
cognitive tasks is overstated (Agrawal,
Gans, & Goldfarb, 2018; Autor, 2014).
• Agrawal et al. (2018) state ML
technologies can substitute humans
for prediction tasks (in routine,
codifiable tasks), but not for judgment
tasks (tacit knowledge, flexibility,
judgment, and creativity)
• Budding research acknowledges cognitive biases in
both humans and ML technologies (Bolukbasi et
al., 2016; Kleinberg et al., 2017; Polonski, 2018)
• ML technologies may be less susceptible than
humans to cognitive biases
• Two distinct biases in ML technologies: biases in
the model/ML algorithm, and biases/sample
selection in the training dataset
• Adversarial ML literature focus on identifying set
of possible misled, however feeding these misled
to train algorithm is hard to imply
Comparative advantage
of humans and machines
in cognitive tasks
Relative cognitive biases
of humans and machines
• 3rd bias not mention: Input
incompleteness—when all
relevant information
required for search and
prediction is not provided -
“adversarial machine
learning”
• Human capital serve as the
complement to ML as a
potential solution to bias
arising from input
incompleteness
GAPS

5. Empirical
settings
RS context: The U.S. Patent and Trademark Office (USPTO) examination process
Characteristics for patent applications: novelty and nonobviousness
Two key challenges to review the patents:
▪ No strategy altering
▪ The application lack relevant info
Tool to search for prediction use of prior art: Boolean and ML technology named
sigma
Individuals possess:
▪ Domain expertise: the skills and knowledge accumulated through prior
learning within a domain (Simon, 1991) – the dynamic update of knowledge
cause input incompleteness-> ML to make a reliable prediction about future
that is unfamiliar to its training dataset -> domain expert bring tacit info and
adjustment based on prior knowledge
▪ Vintage-specific skills: the skills and knowledge accumulated through the prior
familiarity of tasks with the technology (Chari & Hopenhayn, 1991) – posit on
users of ML technologies
Hypothesis: ML will not address biases due to input incompleteness without
complementary domain specific expertise, and user-interface complexities of ML
require that humans who provide such expertise also have complimentary
vintage specific human capital.
Testing methods:
▪ Observational: testing the dynamic updating of langue describing knowledge
base; the relevance of domain expertise
▪ Experimental: 221 graduated MBA students considered as novice examiners
examine patents with 5 claims over 5 days – test the CS&E knowledge base
PAGE 5

6. Theoretical contributions
▪ Social sciences and management
literature on bias in ML prediction by
building upon and extending the
adversarial ML literature in computer
science.
▪ Substitution effects of technologies on
human capital, particularly when related
to prediction
▪ Our results related to vintage-specific
human capital also contribute to the
literature on strategic renewal and the
literature on career specialization
PAGE 6

7. Limitations
PAGE 7
▪ Boundary conditions: focus on the early stages of the
evolution of ML technologies in a one-shot experiment. It is
unclear whether the provision of expert advice and
vintage-specific human capital will cause the relative to grow
or to shrink
▪ MBA students consider as novice users of process
technology that need to be replicated in a similar context for
confirmation
▪ The results of experimental research may be driven by
outliers despite the larger sample
▪ Research context and technology vintage are very specific
USPTO's development of the ML tool Sigma relative to
Boolean search -> suggest that budding more empirical
research examining the evolution in productivity of all ML
technologies to see their contingencies.

8. PAGE 8
▪ STRENGTHS
▪ Shed light on salience bias arising from
input-incompleteness
▪ The choice of empirical settings fixed with the paper’s
research questions
▪ Weaknesses
▪ The specify of patent context requires new language
and descriptors that cause the bias from input
incompleteness and uncertainty on future’s prediction
is the key for human complementary which condition is
hard to amplify
Strengths &
Weaknesses

9. Ideas for future researches
▪ The research put the setting on the window of times to observe the improvement after
complementing human capitals with the ML technology process
▪ Applying different methodology instead of experiments with MBA students
▪ Enlarge the samples of studies to avoid the outliers results
▪ The similar application to the domain of different research contexts such as self-driving
cars/automation transportation, within the hospital and medical context for the role of nursing,
etc.
▪ Exploring the agency theory within input incompleteness
PAGE 9

10. PAGE 10
Thank you!