Critical Thinking Skills of Doctors Analyzed

M.D. CRITICAL THINKING AND DECISION MAKING 1
CRITICAL THINKING AND DECISION-MAKING IN MODERN DAY PHYSICIANS
by:
James Patrick Dunlea
A thesis submitted to the faculty of
Cornell University
In partial requirement of the requirements for the
2015-2016 UNDERGRADUATE HONORS PROGRAM
Human Development
Cornell University
May, 2016
Advisors:
Professor Stephen J. Ceci, Department of Human Development
Professor Wendy M. Williams, Department of Human Development

ABSTRACT
The present study investigated how individuals from different occupations/educational levels
critically think and make decisions. Specifically, undergraduates, graduate students, and post-
graduates in both medical and legal fields were sampled to explore differences in critical
thinking ability and decision-making preferences within and between groups. Members of the
general population were also surveyed. Subjects completed a battery of six cognitive tests—
namely, the Critical Reflection Test (CRT), Abridged Numeracy Scale, Linda Problem, Wason
Cards Task, and two versions of the Beads Task—to gauge their critical thinking and decision
making preferences. Results indicated that individuals from different occupations differed in
terms of critical thinking abilities. Specifically, physicians were found to underperform on
certain critical-thinking measures when compared to other highly educated samples, such as
lawyers. Results from the present study warrant the re-examination of both the medical education
system and the cognitive strategies employed by physicians.

BIOGRAPHICAL SKETCH
James Patrick Dunlea was born and raised in Wayne, Illinois. He will be graduating with Honors
from Cornell University with a Bachelor of Science in Human Development in May 2016.
Starting in the fall of 2016, James will be matriculating as a full-time graduate student at
Northwestern University’s Pritzker School of Law to pursue a Masters of Science in Law (MSL).
He will be concentrating in Regulatory Analysis and Strategy. Upon completion of his MSL
degree, James hopes to establish a psycho-legal research career and matriculate as a J.D./Ph.D.
student studying the underpinnings of punishment, reward, and the psychological roots of the
bias that is feeding the current trend favoring mass-incarceration/retribution over
healing/education.

ACKNOWLEDGMENTS
I would like to thank Professor Stephen J. Ceci and Professor Wendy M. Williams for
their support, guidance, and hospitality throughout the duration of my time in the Child Witness
and Cognition Lab. Additionally, this project would not have been possible without doctoral
student Kayla A. Burd—words cannot express how thankful I am for your unconditional support,
time invested in me, and friendship throughout my undergraduate years. I would also like to
thank Jonathan G. Lash for his generous assistance with my project. Thank you as well to the
other graduate Child Witness and Cognition Lab members Rebecca K. Helm and Michael D.
Creim for their invaluable insights with this project.
Finally, I would also like to thank my mother, Helen Dunlea, and my father, Sean
Dunlea, and my siblings, Kayla and Patrick, for giving me the world and so much more.

TABLE OF CONTENTS
Page Number
Biographical Sketch………………………………………………………………….....................4
Acknowledgments…………………………………………………………………………………5
List of Tables……………………………………………………………………………………...8
List of Figures……………………………………………………………………………………10
Introduction………………………………………………………………………………………11
Comparing Medical Decision-Making to Non-Medical Groups………………………...15
The Present Study………………………………………………………………………..18
Methodology……………………………………………………………………………………..19
Participants……………………………………………………………………………….19
Design……………………………………………………………………………………20
Materials…………………………………………………………………………………20
Procedures………………………………………………………………………………..23
Results……………………………………………………………………………………………24
Cognitive Reflection Test (CRT) …………………………………………………...…...24
Abridged Numeracy Scale………………………………………………………...……..26
Linda Problem……………………………………………………………...…………….28
Wason Cards Task……………………………………………………………………….32
Beads Tasks……………………………………………………………………………...35
Discussion………………………………………………………………………………………..40
Implications……………………………………………………………………………....44
Limitations and Future Directions……………………………………………………….49

References………………………………………………………………………………………..52
Appendices……………………………………………………………………………………….66
Appendix A……………………………………………………………..………………..66
Appendix B……………………………………………………………..………………..67
Appendix C…………………………………………………………………..…………..68
Appendix D………………………………………………………………..……………..69
Appendix E……………………………………………………………………………....70
Appendix F……………………………………………………………………………….71

LIST OF TABLES
Table Page Number
1. Means and Standard Deviations for CRT Score—Between Groups Comparison………25
2. Means and Standard Deviations for CRT Scores—Within Legal Education
Comparison……………………………………………………………………………...26
3. Means and Standard Deviations for CRT Scores—Within Medical Education
Comparison……………………………………………………………………………...26
4. Means and Standard Deviations for Abridged Numeracy Scale Scores—Between Groups
Comparison……………………………………………………………………………...27
5. Means and Standard Deviations for Abridged Numeracy Scale Scores— Within Legal
Education Comparison…………………………………………………………………..28
6. Means and Standard Deviations for Abridged Numeracy Scale Scores— Within Medical
Education Comparison………………………………………………………………......29
7. Linda Problem Between Groups with J.D.s as Reference Point…………………………30
8. Linda Problem Between Groups with M.D.s as Reference Point………………………..31
9. Linda Problem Within Groups with J.D.s as Reference Point…………………………...32
10. Linda Problem Within Groups with M.D.s as Reference Point………………………….32
11. Wason Cards Task Between Groups with J.D.s as Reference Point…………………….33
12. Wason Cards Task Between Groups with M.D.s as Reference Point……………….….34
13. Wason Cards Task Within Groups with J.D.s as Reference Point……………………....34
14. Wason Cards Task Within Groups with M.D.s as Reference Point……………………..35
15. Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task…....36
16. Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within

Legal Education…………………………………………………………………………37
17. Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within
Medical Education………………………………………………………………………37
18. Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task—
Between Between Groups………………………………………..………………………38
19. Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task
Within Legal Education…………………………………………………………………39
20. Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task
Within Medical Education………………………………………………………………39

LIST OF FIGURES
Figure Page Number
1. Mean CRT Composite Scores by Educational Track…………………….……….......…60
2. Percent Correct on the Linda Problem by Educational Track…………………………...61
3. Percent Correct on the Wason Cards Task by Educational Track……………….………62
4. Beads Drawn on Easy Beads Task by Education Track………………………..………..63
5. Beads Drawn on Difficult Beads Task by Education Track……………………………..64
6. Abridged Numeracy Scale Mean Composite Scores by Educational Track……………..65

Critical Thinking and Decision-Making in Modern Day Physicians
In our increasingly growing age of technology, information is both literally and
figuratively at our fingertips. In response to the new, technology-savvy generation, the way
information is disseminated has evolved in tandem with the trend favoring convenience
(Motiwalla, 2007). Information has become routinely published and widely accessible on the
world-wide-web. Unlike in more primitive times, the curious, information-seeking individual is
no longer prompted to leave his or her own comfort to find the knowledge he or she seeks
(Bennett, Maton, & Kervin, 2008). Additionally, Bennett and colleagues (2008) propose that
citizens of the modern era have access to more information than one can quantitatively perceive;
simply put, we are living in a robustly information-saturated environment.
This increase in the amount of information available may be viewed as the result of
humans’ obsession for minimizing both risk and uncertainty (Tversky, Kahneman, & Thaler
1991). Furthermore, Tversky and colleagues (1991) argue that the drive to create this so called
“information-obsessed society” may be reduced to human innate desire to control the
surroundings in which we live. In short, it is the consensus among researchers that it is
normative human inclination to strive for empirical truth. There is undoubtedly no feasible way
to completely erase risk and uncertainty throughout decision-making processes. Understanding
this fact, researchers believe that the best way to minimize uncertainty is to ascertain factual
quanta of knowledge from outside sources that are reputable, clear, and accessible (Henrich &
Gil-White, 2000). The current information-saturated society attempts to do just this. Being
cognizant and proactive about self-education with the relevant and available information, a
decision-making individual is capable of mentally employing cognitive cost-benefit analyses to

decide which course of action will yield the most fruitful and least harmful experience to
themselves (Jaeger, Webler, Rosa, & Renn, 2013).
However, this cost-benefit analysis is not always correctly executed; human decision-
making is inherently imperfect due, in part, to constraints on our information processing (e.g.,
working memory limits). For example, when experiencing stress or cognitive dissonance while
making a decision, heuristics are often implemented to make quick plans of action (Neth &
Gigerenzer, 2015). Although heuristics offer quick ways to make judgments about one’s course
of action, this algorithm-type cognitive tool may draw upon extraneous information that may
lead to flawed evaluations. When using heuristics to make a decision or form an opinion,
tangential pieces of stored knowledge that have been encoded in memory may be introduced into
decision-making processes in the form of biases (Gigerenzer & Gaissmaier, 2011). Bringing in
bias introduces complications in the decision-making process by increasing the probability that a
prediction error—an incorrect assessment of the situation and subsequent action—will occur
(Neth & Gigerenzer, 2015). By increasing the prediction error, one is proportionally amplifying
the chance that he/she misjudges the situation being confronted, and acts in a potentially harmful
manner to oneself or others.
Interestingly, flawed decision-making may begin before one is actually confronted with a
stimulus that requires a decision. Researchers have found that memory—sometimes described as
a “metaphorical ball of malleable putty”—is continually undergoing shaping and re-coding
processes by the experiences one has throughout his or her lifetime (Ceci & Bruck, 1995). This
memory reconstruction effectively plays a significant role in cognitive processes and the way
one makes decisions (Reyna & Lloyd, 1997). Memory reconstruction is a process that is capable
of altering the architecture of previously formed memories. This process is therefore able to

deceive the brain into “thinking” that all of its stored memories are true. A commonly held lay,
albeit incorrect, perception of memory is that the brain works analogously to a video camera
(Ceci & Bruck, 1995): When one experiences a series of events, it seems as if the events are
seamlessly recorded by the brain. However, contemporary research supports the claim that there
is no such veridicality in information encoding and storage processes; it is safe to say that false
memories—whether they are purposefully or passively accumulated—appear real, and at play in
daily life (Ceci & Bruck, 1995).
To exemplify the relevance and subtleness of how memory manipulation may occur,
conceptualize the following scenario: if one was given a list of the words to memorize such as
‘winter’, ‘frosty’, ‘cold’, ‘ice’, ‘sledding’, ‘igloo’, ‘Eskimo’, ‘holiday’, ‘white’, ‘mittens’, and
‘flake’ and after an elapsed period of time, asked if he or she heard the word ‘snow’, it is quite
likely that this word will be falsely remembered being on the list of words (Roediger &
McDermott, 1995). In their study, Roediger and McDermott found that unsuspecting college
participants falsely remembered the non-represented word (in this case, ‘snow’) 40% of the time.
This is because the word ‘snow’ has been stored in the same mental scheme as the other words
presented. These words have been categorized in a similar manner within the individual’s neural
network. Therefore, it is difficult to discriminate between the actual words and semantic
associations that were unconsciously activated by our brain processes. Since people may draw
from their stored repertoire of knowledge to make decisions, it is plausible that their cognitive
sets may contain pockets of falsely remembered memories, potentially derailing decision-making
(Burke & Miller, 1999).
In addition to automatic memory alteration processes that may eventually taint decision-
making, individual cognitive predispositions may also lead to skewed information digestion. As

suggested by many researchers, the increase in information does not necessarily linearly
correlate with the general public’s access, consumption, and, most importantly, correct
internalization of this knowledge (Perlis, Purang, & Anderson, 1998; Blum-Kulka & Weizman,
1988). For example, Kahan, Jenkins-Smith, and Braman (2011) argue that the scientific
community has largely been unsuccessful at convincing the public about certain empirical
findings. Numerous examples of this can be found. Debates regarding the appropriateness of
prescribing and using medical marijuana (Adler & Colbert, 2013), the true effects of hydraulic
fracturing—commonly termed “fracking”—(Sovacool, 2014), and the importance (or lack there
of) of eating breakfast in the morning (Levitsky & Pacanowski, 2013) have drawn serious
attention to the notion that “facts”—seemingly deductive products of empirical research, testing,
and data collection—are able to be framed in different manners.
The skewed consumption and internalization of facts largely stems from the ideology that
the general population heeds because it is culturally predisposed to believe, while other ideas are
discredited for the same reason (Tomasello, Carpenter, Call, Behne & Moll, 2005). This
phenomenon is termed “cultural cognition” and can further be illustrated in Hmong culture,
where tonic-clonic seizures, commonly known as gran mal seizures, are viewed as being divinely
inspired (Fadiman, 1997). The Hmong people believe that when an epileptic individual seizes, a
wicked spirit, or a dab, infiltrates that person’s body; it is this physical manifestation that causes
the associated jerking and shaking movements. Although the Hmong do view this as being a
serious condition, they believe only a select few are chosen to be inflicted with this condition;
since affected individuals are rare, they are often elected to become shamans or sages. Therefore,
it is unsurprising that Hmong people living in the United States have been documented as
refusing medications to treat epileptic children. To many Westernized individuals, though, there

is confusion as to why a debate about administering medicine to an epileptic child exists in the
first place. The answer to the question as to why the debate exists can be grounded in cultural
cognition theory (Kahan et al., 2011). To the Hmong people, it is held true that epilepsy is a
revered celestial affliction, whereas to most Western individuals, it is true epilepsy is a serious,
threatening condition that must be treated with medication. These are two “facts”, yet they are
certainly contradictory to some degree. Therefore, if individuals from each respective culture
were asked which side of the debate was “correct”, different conclusions would be drawn. This
example is analogous to how people are biased towards drawing certain conclusions despite
being given the same information. The area of “framing effects” in judgment and decision-
making research is replete with similar examples.
Comparing Medical Decision-Making to Non-Medical Groups
In sum, there are empirical and theoretical bases for the conclusion that the common
person is susceptible to making a flawed decision—a nervous suitor may be mistaken for dull on
his first date, a female corporate executive’s desire to come off as authoritative may be perceived
by her co-workers as boorish, etc. The contexts in which a mistake can be made are endless.
Despite the large body of literature on the dangers of flawed cognition, few researchers have
examined what factors may influence certain groups more prone to make irrational decisions.
The current study will examine how formal education shapes the way one thinks and
subsequently makes decisions. Specifically, physicians’ cognitive styles and decision-making
processes will be examined and compared to both members of the general population and other
highly educated professionals, namely, lawyers.
Physicians, throughout history, have been held in a high regard (Maulitz, 1988). In
modern times, they surpass other highly educated, prestigious occupations such as physicists,

chemists, mathematicians, architects, politicians, military officers, engineers, lawyers, and
economists, an empirically-based ranking of occupational prestige (Ganzeboom, De Graaf, &
Treiman, 1992; Ganzeboom & Treiman, 1996; Nakao & Treas, 1992). As cited in Malmsheimer
(1988), the general public often views physicians as “essentially flawless individuals, dedicated
to healing both the physical and emotional ills of patients and often providing advice about the
solutions to nonmedical problems” (pp. 173). One of the characteristics most used to describe
physicians is altruism—the trait that entails an individual serving the needs of others with no
tangible benefit for himself/herself (McClelland, 2014).
Seeing the modern-day physician through a rose-colored lens perpetuates the “doctor
knows best” mentality; although this perception is beneficial in terms of establishing trust
between physician and patient, it also sows the seed that depicts physicians as immune to the
fallacious reasoning that afflicts the rest of us. Moreover, researchers have found that physicians
themselves may hold similar perceptions about themselves—Bornstein and Emler (2001) found
that a widespread consensus among physicians is that they were more or less immune to making
errors. Furthermore, Russo and Schoemaker (2002) purport that individuals who often engage in
fallacious decision-making tend to believe that they are exceptional decision-makers.
In order to examine whether the “infallible doctor” conception is accurate, the types of
fallacies in which a physician might make needs to be investigated: in the next chapter ethical,
perceptual, and reasoning fallacies will be discussed and put in the context of medicine. The
previous research raises a concern for medical professionals specifically, but also for those who
are in any decision-making role.
In 1998, British gastroenterologist Dr. Andrew Jeremy Wakefield published an
interesting article in the British Journal of Medicine. In this article, Dr. Wakefield cited data that

pointed to the conclusion that the vaccine against measles, mumps, and rubella (MMR) may lead
children to acquire developmental disorders, such as autism, and digestive tract inflammation,
called non-specific colitis. The article went viral. Thirteen years after Wakefield’s initial
publication, the British Journal of Medicine retracted the publication, ridiculing the physician for
fraudulent data collection, and stripped Wakefield of his license to practice (Godlee, 2011).
Although the article has been retracted by the journal, scientists must continue the fight to
convince the public that there is no causal connection between vaccinations and
developmental/gastrointestinal disorders (see news story by Haberman, 2015). This is an
example of an ethical fallacy in which a physician failed to abide by a code of conduct honoring
truthful data collection and reporting. Although ethical lapses are a very important flaw in the
medical field, my study will explore cognitive fallacies that lead to flawed decision-making.
Another type of commonly made mistake in decision-making is a perceptual fallacy. This
type of fallacy can more simply be described as a sort of “inattentional blindness” where an
observer fails to notice a seemingly blatant perceptual element in the larger environment (Drew,
Võ, Wolfe, 2013). In 2013, Drew and colleagues conducted an experiment that put the perceptual
keenness of seasoned radiologists to the test: 24 radiologists who routinely scanned for lung
cancer tumors were allowed 3 minutes to scroll through a chest computerized tomography (CAT)
scan to identify any abnormal nodules. To the unsuspecting physicians, a picture of a gorilla—
one that is 48 times the size of a typical nodule—was superimposed on one of the CAT scans
they saw. Drew and colleagues (2013) reported that 83% of seasoned radiologists failed to see
the gorilla’s picture in the CAT scans and reported that there was nothing wrong. This startling
statistic demonstrates that even highly trained professionals, such as physicians, are sometimes

flawed in their decision making abilities—in this case, in misperceiving their surroundings and
using information to draw incorrect conclusions.
To demonstrate reasoning fallacies in the medical profession, Hoffrage and Gigerenzer
(1998) tested 48 German physicians of varying subspecialties on their ability to calculate
positive predictive values on certain medically related diagnostic tests. In order to calculate the
positive predictive value for a patient, Hoffrage and Gigerenzer (1998) provided the physicians
with the base rate in the general population for a disease, the percentage of cases in which the
diagnostic test correctly predicts a positive test result, and the percentage in which the diagnostic
test incorrectly predicts a positive test results. The researchers reported that physicians were able
to correctly calculate the positive predictive value only 10% of the time (Hoffrage & Gigerenzer,
1998). In one variation of the study where the physicians were asked to predict the probability
that a patient had colorectal cancer based on the statistics provided, answers ranged between 1
percent and 99 percent! Therefore, Hoffrage and Gigerenzer (1998) were able to reveal the
deficits in critical thinking skills and reasoning that characterize physicians.
The Present Study
The current study focused on the reasoning type cognitive fallacy when making
decisions. In the study, the frequency of reasoning fallacies from a physician sample representing
the general population and a sample of lawyers was compared. In order to ascertain how the
curriculum of one’s formal education shapes his/her cognition, the frequency of reasoning
fallacies made by undergraduate students who have not yet matriculated into medical/law school
(“pre-medical/law students”) and current medical/law students was also compared. This
comparison was achieved by administering a battery of critical thinking measures to current

undergraduate pre-med/pre-law students, matriculated medical/law students, degree-bearing
physicians/lawyers and to members of the general population via an online (Qualtrics) survey.
As cited by Loftus, Levidow, and Duensing (1992), group differences in memory exist
among individuals from different occupations. In their study, Loftus and colleagues found that
artists and architects exhibited a superior memory when compared to law enforcement agents,
and homemakers, for example. As there is evidence for differences in cognitive mechanisms in
memory (Loftus et al., 1992), it was hypothesized that there would also be differences in the way
people of different occupations yield to reasoning fallacies that impact critical-thinking and
decision-making. More specifically, it was predicted that both physicians and lawyers, on
average, will score higher on critical thinking tests than members of the general population.
However, it was posited that physicians’ critical thinking abilities will fall short when pitted
against individuals holding a law degree due to the latter group’s training that emphasizes logic
and deductive reasoning (e.g., in contract law class). Additionally, it was hypothesized that
lawyers and physicians would have different decision-making preferences—specifically, lawyers
would be more motivated than physicians to make decisions that are clearly backed by
substantial and unambiguous evidence.
Methodology
Participants
A total of 738 participants completed all tasks included in the present study. 239
participants were recruited via Cornell University’s Psychology Experiment Sign-Up System
(SONA), while 203 participants were recruited from Amazon’s Mechanical Turk. Additionally,
294 individuals were recruited via the snowball sampling method.

Participants were offered different incentives based on their recruitment platform:
participants who were recruited via Cornell University’s Psychology Experiment Sign-Up
System (SONA) were awarded 1 SONA credit for their participation, while those who were
recruited from Amazon’s Mechanical Turk were given $0.50 for their participation. Individuals
who were recruited by the snowball sampling method (e.g., via email and social media outlets,
such as Facebook) were not compensated, but instead thanked for their participation. Of the
entire sample, 38.3% participants identified as male, 61.3% as female, and .4% did not identify
with either gender. Participant ages ranged from 18 to 78 (M = 29.76, SD = 13.61).
Design
A cross-sectional, correlational design was utilized that compared individuals’
performances on a battery of critical thinking and decision-making tasks. Specifically,
individuals were grouped according to self-report of his/her current professional field.
Regardless of professional field, all participants were administered an identical battery of
cognitive tasks. Participant scores on all cognitive measures were then used in subsequent
regression analyses.
Measures
Professional field. All participants completed an initial questionnaire regarding intended
professional track (e.g., “pre-med”, “pre-law”) or current professional field. If the participant
indicated that he/she was enrolled as a student in either medical school or law school, he/she was
asked if he/she intended to specialize in a specific sub-field of law or medicine, respectively.
Participants indicating that they were graduates of medical school or law school were asked to
indicate specialty (if any) and number of years since receiving his/her professional degree (See
Appendix A).

Cognitive Reflection Test (CRT). The Cognitive Reflection Test (Frederick, 2005) is a
short questionnaire that measures cognitive reflection—the measurement used to determine how
capable one is to appreciate or recognize the effects of delayed gratification. For example, an
individual scoring high on cognitive reflective abilities would be able to appreciate the benefits
of interest rates offered by the market over time. Additionally, CRT scores can be used to further
predict the degree to which one relies on mental heuristics in decision-making processes and
his/her time preferences in cognitive processes. Liberali, Reyna, Furlan, Stein, and Pardo (2012)
report a moderate to high reliability (α = 0.64 – 0.74) (See Appendix B).
Abridged numeracy scale (5-item). The abridged numeracy scale used is part of a
longer, 11-item measure developed by Lipkus, Samsa, and Rimer (2001). In this task,
participants were prompted to respond to questions such as “Imagine that we rolled a fair, six-
sided die 1,000 times. Out of 1,000 rolls, how many times do you think the die would come up
even (2, 4, or 6)?”. These questions were used to gauge participants’ mathematical and statistical
competencies (See Appendix C).
The Linda problem. Participants’ adeptness to make a rational probability judgment was
tested with the Linda Problem (Tversky & Kahneman, 1983). In the Linda problem, a fictional
character, Linda, is portrayed as a socially liberal woman invested in “issues of discrimination
and social justice”. Following this description, participants are prompted to chose whether Linda
would be more likely to be solely a “bank teller” or “a bank teller and active in the feminist
movement”. When answering the Linda problem, many participants may find themselves relying
upon irrelevant information (the statements portraying Linda through a liberal lens) and make a
conjunction fallacy—a misjudgment in logical cognition that occurs when two simultaneous
conditions are deemed more likely to occur than either one independently. Previous work

suggests that an individual’s performance on the Linda problem may be related to reliance on
possibly fallacious heuristic based decision making strategies (Tversky & Kahneman, 1983) (See
Appendix D).
Wason selection task. The Wason Selection Task (Wason, 1968) measures participants’
capabilities to deductively reason and subsequently make a decision after being given a set of
rules. Participants were asked to answer an “if–then” logic puzzle using four white-faced cards
labeled with either black-type letters or numbers (“A”, “K”, “4”, and “7”, respectively).
Specifically, participants were given a rule-based statement (“If a card has an even number on
one side, then it has a vowel on the other”) and subsequently asked to select a card dyad that
fulfilled the conditions set in the aforementioned rule (See Appendix E).
Beads task. The tendency for an individual to jump to conclusions was tested by a
common probabilistic reasoning measure, the Beads Task (Averbeck, Evans, Chouhan, Bristow,
& Shergill, 2011; Garety et al., 1991; Garety & Freeman, 1999; Huq, Garety, & Hemsley, 1988).
Jumping to conclusions, as defined by Garety and colleagues (1991), provides information
regarding the tendency of an individual to “request [a small amount of] information to reach a
decision” (p. 570). Although previous research has used the Beads Task to assess “reasoning
bias [encountered] across the continuum of delusional ideation” (p. 1255) in delusional and
delusion-prone individuals (Warman, Lysaker, Martin, Davis, & Haudenschield, 2006), new
research has used the Beads Task with healthy, neurotypical individuals (Evans et al., 2012).
In the present study, two versions (“easy” vs. “difficult”) of the Beads Task were
administered to participants. In the “easy” version of the Beads Task, participants were presented
with two jars (labeled “A” and “B”, respectively). In the “easy” version, Jar “A” was comprised
of 15% blue beads and 85% red beads; conversely, Jar “B” was filled with beads that were 15%

red and 85% blue. The participants were instructed that “The computer will now select one of the
jars and draw random beads from it”. Given this information, participants were instructed to
guess which jar the given bead was drawn from. The same set of instructions was given during
the “difficult” Bead Task. The sole difference between the “easy” and “difficult” Bead Task
versions was that the “difficult” Bead Task displayed jars with 60:40 bead color ratios to
participants as opposed to 85:15 bead color ratios. In both conditions, participants were not
limited to the number of beads they were able to draw (See Appendix F).
Demographic questionnaire. Lastly, participants responded to a number of demographic
questions, including questions regarding age; gender; race; highest level of education completed;
number of academic courses in science, technology, engineering, and math (STEM) fields; and
household income. If the participant was financially dependent on a caretaker/guardian, he/she
was asked to report the income of his/her caretaker/guardian.
Procedures
All participants completed the present study online using the Qualtrics platform.
Participants accessed the survey link on Cornell’s SONA system, Amazon’s Mechanical Turk, or
by clicking on the survey link received via email or a social networking website.
After clicking on the survey link, participants gave informed consent for their
participation. Next, participants indicated his/her intended or current professional track/field.
Then, participants completed a battery of twelve cognitive and decision-making tasks based on
random assignment. More specifically, participants answered three CRT questions, five
Abridged Numeracy Scale questions, the Linda problem, the Wason Selection Task problem, and
completed two trials of the Beads Task. Lastly, participants completed several demographic
questions.

Results
What follows are a series of analyses that examined both inter-professional differences in
cognitive performance (e.g., comparing physicians with lawyers), and intra-professional
differences in cognitive performance that focus on the effects of training within a profession
(e.g., comparing law school students with full-fledged law school graduates).
CRT
Between-group comparisons. In order to test the hypothesis that there would be
differences in critical-thinking abilities between participants from different professions, mean
composite scores on the CRT were compared across education/occupation groups. Results from
a one-way ANOVA revealed a significant effect of one’s education level/occupation on mean
CRT scores, F (7, 718) = 6.25, p < .001, ηp
2
= .057. Pair-wise comparisons using the Bonferroni
correction revealed several group differences: “pre-medical” undergraduate students had
significantly lower average CRT scores than medical school students (p = .024), law school
students (p = .003), lawyers (p = .001), and those with careers as engineers (p = .005). On
average, participants whose careers were classified as “Other” had significantly lower CRT
scores compared to law school students (p = .01), lawyers, (p = .003), and engineers, (p = .017).
No other comparisons revealed significant differences (See Table 1).
Within-group comparisons.
Legal education. In order to explore whether differences in critical-thinking abilities
would emerge among participants with legal education, mean composite scores on the CRT were
compared among “pre-law” students, current law school students, and law school graduates to
examine whether attending law school influenced CRT performance or if those who became
lawyers were self-selecting into the profession. Results from a one-way ANOVA revealed a

significant effect of education on mean CRT score, F (2, 158) = 4.00, p = .02, ηp
2
= .048. Pair-
wise comparisons using the Bonferroni correction revealed that “pre-law” undergraduate
students had significantly lower CRT scores compared to law school graduates, (p = .031). No
other comparisons revealed significant differences (see Table 2).
Table 1
Means and Standard Deviations for CRT Score—Between Groups Comparison
CRT score
Educational Track n M SD
Pre-Med 116 1.51 a*,b**, c**, d**
1.18
Medical School 43 2.16 a*
.97
M.D. 53 1.79 .91
Pre-Law 54 1.74 1.09
Law School 60 2.18b**, e*
.98
J.D. 45 2.30c**
.90
Other (non-engineers) 282 1.63e*, f*
1.16
Other (engineers) 73 2.12d**, f*
1.07
Total 726 1.80 1.12
Notes. * p <.05, ** p <.01
Medical education. In order to explore whether differences in critical-thinking abilities
exist among participants with varying degrees of medical education, mean composite scores on
the CRT were compared among “pre-med” students, current medical school students, and
medical school graduates. Results from a one-way ANOVA revealed a significant effect of
education on mean CRT score, F (2, 211) = 6.34, p = .002, ηp
2
= .057. Pair-wise comparisons

using the Bonferroni correction revealed that “pre-med” undergraduate students had significantly
lower CRT scores compared to current medical school students, (p = .002). No other
comparisons revealed significant differences (see Table 3).
Table 2
Means and Standard Deviations for CRT Scores—Within Legal Education Comparison
CRT score
Pre-Law 54 1.74 1.09
Law School 60 2.18a*
.98
J.D. 45 2.30a*
.90
Notes. * p <.05
Table 3
Means and Standard Deviations for CRT Scores—Within Medical Education Comparison
CRT score
Pre-Med 116 1.51 a**
1.18
Medical School 43 2.16 a**
.97
M.D. 53 1.79 .91
Notes. ** p <.01
Abridged Numeracy Scale
Between-group comparisons. In order to explore whether numeracy skills—“the
understanding of basic probability and mathematical concepts” (Lipkus, Samsa, & Rimer, 2001,

pp. 37)—differed among participants, a one-way ANOVA was used to compare mean composite
scores on the Abridged Numeracy Scale. This analysis revealed a significant effect of one’s
education on mean numeracy scores, F (7, 712) = 3.12, p = .003, ηp
2
= .03. Pair-wise
comparisons using the Bonferroni correction revealed that medical school graduates had
significantly higher numeracy scores than “pre-med” undergraduate students, (p = .021). No
other comparisons revealed significant differences amongst all other groups, (see Table 4).
Table 4
Means and Standard Deviations for Abridged Numeracy Scale Scores—Between Groups
Comparison
Abridged Numeracy Scale score
Pre-Med 115 3.72a*
1.51
Medical School 41 4.51a*
.93
M.D. 50 4.34 .96
Pre-Law 52 3.75 1.61
Law School 62 4.27 1.16
J.D. 50 4.16 1.48
Other (non-engineers) 278 3.93 1.21
Other (engineers) 71 4.06 1.30
Total 720 4.00 1.29
Notes. * p <.05
Legal education. In order to discover whether differences in numeracy among “pre-law”
students would emerge, current law school students, and law school graduates, mean Abridged

Numeracy Scale scores were compared within groups. Results from a one-way ANOVA
revealed no significant differences amongst these groups, with means ranging only between 3.76
and 4.27, F (2, 162) = 1.99, p = .139, ηp
2
= .024, (see Table 5).
Table 5
Means and Standard Deviations for Abridged Numeracy Scale Scores —Within Legal
Education Comparison
Pre-Law 52 3.75 1.61
Law School 62 4.27 1.16
J.D. 50 4.16 1.48
Notes. Pairwise comparisons revealed no significant differences among groups.
Medical education. Investigating whether there would be differences in numeracy among
“pre-med” students, current medical school students, and medical school graduates, mean
abridged numeracy scale scores were compared within these groups. Results from a one-way
ANOVA revealed a significant effect of education on mean numeracy scores, F (2, 204) = 7.23,
p = .001, ηp
2
= .066. Subsequent pair-wise comparisons using the Bonferroni correction revealed
that “pre-med” undergraduate students had significantly lower numeracy scores compared to
current medical school students, (p = .004). Additionally, analyses indicated “pre-med”
undergraduate students also had significantly lower numeracy scores compared to medical
school graduates, (p = .019). No other comparisons revealed significant differences, (see Table
6).
The Linda Problem
Between-group comparisons.

Legal education. A binary logistic regression was utilized to explore the effects of one’s
formal education/career path on performance on the Linda problem. First, law school graduates
were used as the comparison group. The omnibus test was significant, (p < .001). Analyses
revealed that lawyers were significantly more accurate when answering the Linda Problem
compared to any other group. Specifically, lawyers boasted an accuracy rate 2.21 times that of
“pre-med” undergraduate students (p < .001), 1.56 times that of medical school students (p =
.008), 1.63 times that of physicians (p = .004), 2.34 times that of “pre-law” undergraduate
students (p < .001), 1.53 times that of law students (p = .007), 1.74 times that of individuals
whose careers were categorized as “Other” (p = .001), and 2.68 times that of engineers, (p <
.001). No other comparisons revealed significant differences, ps > .05 (see table 7).
Table 6
Means and Standard Deviations for Abridged Numeracy Scale Scores —Within Medical
Education Comparison
Pre-Med 115 3.72a**, b*
1.51
Medical School 41 4.51a**
.93
M.D. 50 4.34 b*
.96
Notes.* p <.05, ** p <.01
Medical education. A binary logistic regression was utilized to explore the effects of
one’s formal education/career path on performance on the Linda problem. Here, medical school
graduates were used as the comparison group. The omnibus test of the model was significant, (p
< .001). Analyses revealed that lawyers were significantly more accurate when answering the
Linda Problem than physicians were: specifically, lawyers were 1.56 times more likely to

correctly answer the Linda Problem correctly than physicians, (p = .004). Conversely, physicians
were 1.64 times more likely to answer the Linda Problem correctly than individuals whose
occupations were categorized as “Other”, (p = .007). No other comparisons revealed significant
differences, ps > .05 (see table 8).
Table 7
Linda Problem Between Groups with J.D.s as Reference Group
Educational Track b SE Wald Accuracy Rate
Pre-Med -1.72 0.37 21.13 0.34***
Medical School -1.16 0.44 7.06 0.48**
M.D. -1.22 0.42 8.39 0.46**
Pre-Law -1.82 0.44 17.51 0.32***
Law School -1.10 0.41 7.31 0.49**
J.D. --- --- 45.74 0.75a
Other (non-engineers) -1.35 0.4 11.42 0.43**
Other (engineers) -2.04 0.35 34.46 0.28***
Note. * p <.05, ** p <.01, *** p <.00
Within-group comparisons. In order to examine the effects of formal education/career
path on cognitive performance, a series of logistic regressions were run to contrast performance
by those still engaged in formal training (law and med students) with those who already
completed such training (full-fledged physicians and attorneys). Such contrasts inform the
question of primacy or causality—does training improve problem-solving, or are there
preexisting differences that are not further amplified by training?
Legal education. A binary logistic regression was utilized to explore the effects of legal
education on performance on the Linda problem. Here, law school graduates were used as the
comparison group. The omnibus test of the model was significant, (p < .001). Specifically,
analyses revealed that lawyers were 2.34 times more accurate when answering the Linda
Problem than “pre-law” undergraduate students, (p < .001). Additionally, law school graduates

were 1.53 times more accurate than law school students, (p = .01). Both of these last two
findings suggest that training within a field does impart cognitive advantages over and above any
self-selection mechanism. No other comparisons revealed significant differences, ps > .05 (see
table 9).
Table 8
Linda Problem Between Groups with M.D.s as Reference Group
Pre-Med -0.50 0.33 2.22 0.34
Medical School 0.06 0.40 0.02 0.48
M.D. --- --- 45.76 0.46a
Pre-Law -0.60 0.40 2.25 0.32
Law School 0.12 0.37 0.10 0.49
J.D. 1.22 0.42 8.39 0.75**
Other (non-engineers) -0.82 0.30 7.27 0.43**
Other (engineers) -0.12 0.36 0.12 0.28
Note. * p <.05, ** p <.01, *** p <.001
Medical education. A binary logistic regression was utilized to determine the effects of
medical education on performance on the Linda Problem. Here, medical school graduates were
used as the comparison group. The omnibus test of the model was not significant, (p = .185),
suggesting that the experience of medical training did not influence reasoning accuracy on this
task. Current medical school students were no more accurate when answering the Linda problem
than full-fledged physicians, (p = .959), and although physicians were 1.35 times more accurate
than “pre-medical” students, this advantage was not statistically reliable (p = .136). No other
comparisons revealed significant differences, ps > .05 (see table 10).

Table 9
Linda Problem Within Groups with J.D.s as Reference Group
Educational
Track
b SE Wald Accuracy Rate
Pre-Law -1.803 0.428 17.773 0.32***
Law School -1.03 0.402 6.583 0.49*
J.D. --- --- 17.814 0.75a
Note. * p <.05, ** p <.01, *** p <.001
Table 10
Linda Problem Within Groups with M.D.s as Reference Group
Educational
Track
Pre-Med -0.50 0.33 2.22 0.34***
Medical School 0.02 0.40 0.003 0.48
M.D. --- --- 3.36 0.46a
Note. * p <.05, ** p <.01, *** p <.001
The Wason Cards Task
Between-group comparisons.
Legal education. A binary logistic regression was utilized to determine the effects of
one’s formal education/career path on performance on the Wason Cards Task. First, law school
graduates were used as the comparison group. The omnibus test of the model was significant, (p
= .002). Analyses revealed that lawyers were approximately 4 times more accurate than
physicians, (p = .021). No other comparisons revealed significant differences, ps > .05 (see Table
11).
one’s formal education/career path on performance on the Wason Cards Task. Here, physicians

were used as the comparison group. The omnibus test of the model was significant, (p = .002).
Analyses revealed that medical school students were 3.8 times more accurate than physicians, (p
= .031). Additionally, pre-law” undergraduate students were 3.4 times more accurate than
physicians, (p = .045), law students were 5.8 times more accurate than physicians, (p = .001),
and lawyers were 4 times more accurate than physicians (p = .021). Engineers were also 4 times
more accurate than physicians, (p = .016). No other comparisons revealed significant differences,
ps > .05 (see Table 12).
Legal education. A binary logistic regression was utilized to determine the effects of
legal education level on performance on the Wason Cards problem. Here, law school graduates
were used as the comparison group. The ombinus test of the model was not significant, (p =
.224). There was no reliable trend for current law school students to outperform law school
graduates on the Wason Cards task, (p = .959). Similarly, there was no trend for “pre-law”
undergraduate students to perform worse than law school graduates, (p = .851). No other
comparisons revealed significant differences, ps > .05 (see Table 13).
Table 11
Wason Cards Task Between Groups with J.D.s as Reference Group
Pre-Med -0.60 0.42 2.06 0.12
Medical School -0.08 0.48 0.03 0.19
M.D. -1.56 0.67 5.32 0.05*
Pre-Law -0.18 0.47 2.06 0.17
Law School 0.53 0.41 0.03 0.29
J.D. --- --- 5.32 0.20a
Other (non-engineers) 0.02 0.43 2.06 0.20
Other (engineers) -0.62 0.37 0.03 0.12
Note. * p <.05, ** p <.01, *** p <.001

Table 12
Wason Cards Task Between Groups with M.D.s as Reference Group
Pre-Med 0.96 0.65 2.17 0.12
Medical School 1.48 0.69 4.63 0.19*
M.D. --- --- 21.99 0.05a
Pre-Law 1.37 0.69 4.00 0.17*
Law School 2.08 0.64 10.45 0.29*
J.D. 1.56 0.67 5.32 0.20*
Other (non-engineers) 1.58 0.66 5.79 0.20*
Other (engineers) 0.94 0.62 2.30 0.12
Note. * p <.05, ** p <.01, *** p <.001; a = reference point
Table 13
Wason Cards Task Within Groups with J.D.s as Reference Group
Educational
Track
Pre-Law -0.09 0.46 0.04 0.17
Law School 0.55 0.41 1.79 0.29
J.D. --- --- 2.99 0.20a
Note. * p <.05, ** p <.01, *** p <.001
medical education on performance on the Wason Cards problem. Here, physicians were used as
the comparison group. The omnibus test of the model was not significant, (p = .069). Trends
revealed that “pre-med” undergraduate students had accuracy rates that were 2.4 times higher
than the accuracy rate of medical school graduates, although this superiority failed to reach
significance (p = .141). In contrast, medical school students were 3.8 times more accurate than
physicians when answering the Wason Cards Task, (p = .034), suggesting that training itself did
not impart an advantage, assuming that the current cohort of medical school students are not

being exposed to dramatically different types of cognitive training than the cohort of physicians.
No other comparisons revealed significant differences, ps > .05 (see Table 14).
Table 14
Wason Cards Task Within Groups with M.D.s as Reference Group
Educational
Track
Pre-Med 0.96 0.65 2.17 0.12
Medical School 1.46 0.69 4.50 0.19*
M.D. --- --- 4.59 0.05a
Note. * p <.05, ** p <.01, *** p <.001

Beads Task
Easy Version
differences in the amount of information individuals of different educational/occupational
backgrounds prefer when making decisions, the mean number of drawn beads during the easy
version of the beads task was compared utilizing a one-way ANOVA. Analyses revealed a
significant effect of one’s education on the number of beads drawn, F (7, 629) = 2.11, p = .04,
ηp
2
= .023. Despite the model’s overall significance, no reliable differences in the number of
beads pulled were found between any of the compared groups (see Table 15).
Legal education. Exploring whether there would be differences in the amount of
information individuals of varying legal educational backgrounds prefer when making decisions,
the mean number of drawn beads during the easy version of the beads task was compared
utilizing a one-way ANOVA. Analyses revealed no significant effect of one’s education on the
number of beads drawn although it appeared to be trending toward significance, F (2, 122) =

2.57, p = .08, ηp
2
= .04, (see Table 16).
Medical education. In order to discover if there would be differences in the amount of
information individuals of varying medical educational backgrounds prefer when making
decisions, the mean number of drawn beads during the easy version of the beads task was
compared utilizing a one-way ANOVA. Analyses revealed no significant effect of one’s
education on the number of beads drawn, F (2, 122) = .052, p = .95, ηp
2
= .001(see Table 17).
Table 15
Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task
Number beads drawn
Pre-Med 115 3.91 3.33
Medical School 28 4.04 3.68
M.D. 48 3.83 2.56
Pre-Law 49 3.57 3.23
Law School 31 5.03 5.87
J.D. 42 6.00 6.15
Other (non-engineers) 262 4.43 3.90
Total 637 4.41 4.03
Note. Pairwise comparisons revealed no significant differences among groups.

Table 16
Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within Legal
Education
Number beads drawn
Pre-Law 49 3.57 3.23
Law School 31 5.03 5.87
J.D. 42 6.00 6.15
Table 17
Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within
Medical Education
Number beads drawn
Pre-Med 115 3.91 3.33
M.D. 48 3.83 2.56
Difficult Version
differences in the amount of information individuals of different educational/occupational
backgrounds prefer when making decisions, the mean number of drawn beads during the difficult
version of the beads task was compared utilizing a one-way ANOVA. Analyses revealed a
significant effect of one’s education on the number of beads drawn, F (7, 696) = 2.89, p = .005,
ηp
2
= .028. Specifically, pair-wise comparisons using the Bonferroni correction revealed that law

school graduates drew significantly more beads than both “premed” undergraduate students, (p =
.011), and those whose occupations were classified as “Other”, (p = .046). No other comparisons
revealed significant differences, (see Table 18).
Legal education. In order to explore if there are differences in the amount of information
individuals with varying legal educational backgrounds prefer when making decisions, the mean
number of drawn beads during the difficult version of the beads task was compared utilizing a
one-way ANOVA. Analyses revealed no significant effect of one’s education on the number of
beads drawn, F (2, 154) = 2. 15, p = .12, ηp
2
= .027 (see Table 19).
Table 18
Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task—Between
Between Groups
Number beads drawn
Pre-Med 113 6.67a*
5.57
M.D. 50 8.48 7.45
Pre-Law 50 7.22 6.06
Law School 59 8.37 a*, b*
6.95
J.D. 47 10.43 8.58
Other (non-engineers) 275 7.39b*
5.48
Total 704 7.87 6.14
Note. * p <.05

Medical education. In order to explore if there are differences in the amount of
information individuals with varying medical educational backgrounds prefer when making
decisions, the mean number of drawn beads during the difficult version of the beads task was
compared utilizing a one-way ANOVA. Once again, analyses revealed no significant effect of
one’s education on the number of beads drawn, F (2, 202) = 2.00, p = .138, ηp
2
= .019 (see Table
20).
Table 19
Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task Within
Legal Education
Number beads drawn
Pre-Law 50 7.22 6.06
Law School 59 8.37 6.95
J.D. 47 10.43 8.58
Table 20
Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task Within
Medical Education
Number beads drawn
Pre-Med 113 6.67 5.57
M.D. 50 8.48 7.45

Discussion
This study examined the effect of one’s formal education on his/her critical thinking
abilities and decision-making preferences. More specifically, the present study explored both
inter-professional and intra-professional differences in cognitive performance. Rather
surprisingly, the results of the present study found that highly educated individuals who
completed either medical or law school did not have higher accuracy rates than individuals from
the general population on critical thinking measures. However, it should be noted that
individuals who have obtained a law degree had, on average, better accuracy on critical thinking
measures than medical school graduates. Finally, results indicated no differences in decision-
making preferences among medical and law school graduates. Exploratory analyses not only
suggested that individuals drawn from the general population may have similar critical thinking
profiles than medical school graduates, but also that they appear to have similar decision-making
preferences to both law school graduates and medical school graduates.
In conclusion, the hypothesis that medical and law school graduates would have better
critical thinking abilities than individuals from the general population was only partially
supported. The results offered more substantial support for the hypothesis that law school
graduates would have more adept critical thinking abilities than their medical school graduate
counterparts. Contrary to previous assumptions, the hypothesis that there would be differences in
decision-making preferences between law school graduates and medical school graduates was
not supported by the results.
Inter-Professional Critical Thinking Differences
Analyses revealed that medical school graduates outperformed members from the general
population on the Linda Problem. Additionally, law school graduates outperformed individuals

whose occupations were classified as “Other” on the CRT and the Linda Problem. Relatedly,
Fredrick (2005) reported that highly educated individuals studying at extremely rigorous
undergraduate universities had higher average CRT scores than individuals from the general
population. Additionally, individuals whose occupations were classified as “Other” had accuracy
rates that were indistinguishable from those of medical school graduates on the CRT. Lastly, it
was found that both medical and law school graduates performed tantamount to members of the
general population on the Wason Cards Task. In sum, the hypothesis that both medical and law
school graduates would perform better on measures of critical thinking than members of the
general population was only partially supported.
Additionally, analyses revealed trends indicating that law school graduates had higher
average scores than medical school graduates on all critical thinking tasks (see Figures 1 – 3).
However, the difference in physicians’ and lawyers’ mean composite CRT scores failed to reach
statistical significance. Taken together, the above results may call attention to core differences in
cognitive profiles among members of the legal and medical professions. The results may also
indicate that the curriculum implemented in medical education may not necessarily accentuate
the importance of critical thinking to the extent that legal education does. In conclusion, the
hypothesis that law school graduates would achieve higher accuracy rates on critical thinking
tasks compared to medical school graduates was offered partial support.
Intra-Professional Critical Thinking Differences
Exploratory analyses revealed notable trends concerning critical thinking trajectories
throughout medical and legal disciplines. Not surprisingly, results revealed that “pre-medical”
undergraduate students had significantly lower accuracy rates on the CRT than medical school
students did; however, somewhat shockingly, medical school graduates had statistically
indistinguishable accuracy rates with “pre-medical” undergraduates on the CRT. In short, critical

thinking gains made by medical school curriculum were found to be diminished in medical
school graduates. Furthermore, exploratory analyses also revealed that medical school graduates
had similar rates of accuracy to both “pre-medical” undergraduate students and current medical
school students when answering the Linda Problem. Possibly most surprisingly, it was found that
“pre-medical” undergraduate students performed slightly better than physicians on the Wason
Cards Task; similarly, current medical school students significantly outperformed the medical
school graduates on this task. Therefore, these results expose a noticeable degeneration in
physicians’ critical thinking skills after they complete their extensive medical education.
Alternatively, these results may indicate that the sample of full-fledged physicians was not as
cognitively elite as the students. Future research will be needed to determine which of these
explanations are true (e.g., controlling MCAT scores in analyses).
Analyses revealed that, unlike their medically-oriented peers, lawyers continue to reap
critical thinking gains after completing their formal legal education: Law school graduates
boasted higher accuracy rates than “pre-law” undergraduates on the CRT. Further, despite failing
to reach significance, law school graduates also had slightly higher accuracy rates than current
law school students on the CRT. Additionally, exploratory analyses revealed that lawyers
outperformed both “pre-law” undergraduates and current law students on the Linda Problem.
This trend offers support to the suggestion that law school graduates may continue to accrue
certain critical thinking skills throughout their professional career. The only instance where law
school graduates’ accuracy rates waned in compared to current law school students was on the
Wason Cards Task—it was found that law school students outperformed both “pre-law”
undergraduates and law school graduates on this task. It is possible that this particular cohort of

law students was advantaged by current changes to the curriculum that influenced reasoning on
the CRT, but this is sheer surmise and will require follow-up research to determine.
Inter-Professional Decision-Making Differences
It was hypothesized that law school graduates would prefer to make decisions based on
more evidence and certainty than medical school graduates. However, this hypothesis was not
supported: no significant differences in decision-making preferences were found between that
law or medical school graduates. However, analyses revealed trends indicating that, on average,
lawyers drew more beads than physicians on both the “Easy” and “Difficult” Beads Tasks (see
Figure 4). Although failing to reach significance, this trend suggests there might be differences in
nuanced decision-making preferences between medical and legal professionals. More
specifically, this trend suggests that lawyers may have a higher intrinsic need to make more well-
informed decisions compared to medical school graduates.
Intra-Professional Decision-Making Differences
Exploratory analyses revealed trends suggesting that one’s formal education may produce
slightly different decision-making preferences among between members of different fields. For
example, results from the “Easy” Beads Task reveal that both “pre-medical” and “pre-law”
undergraduate students rely on similar amounts of information before feeling comfortable to
make a well-informed decision. However, trends reveal that, unlike individuals pursuing a
medical career, individuals interested in law demonstrate an increased preference for additional
information before making a decision. In fact, analyses revealed that medical school students and
medical professionals failed to show any preference for increasing the amount of information
needed to make a decision from the time they were “pre-medical” undergraduate students. A
similar pattern emerged for the “Difficult” Beads Task: Analyses revealed a trend suggesting

legal education may place emphasis on gathering substantive amounts of concrete evidence
before making a decision (see Figure 5). However, analyses revealed a less influential effect of
one’s medical education on such decision-making preferences.
Inter-Professional Numeracy Differences
Results did not support the hypothesis that numeracy differences exist between medical
school and law school graduates. Despite failing to reach significance, trends revealed that both
graduate medical and legal curricula might lead to a slight increases in numeracy skills.
Similarly, there was a non-significant trend indicating a very slight decline in numeracy for both
individuals in medical and legal fields after completion of graduate schooling (see Figure 6).
Intra-Professional Numeracy Differences
Exploratory analyses revealed that one’s formal education did not impact numeracy for
individuals interested in pursuing medical or legal careers. Specifically, it was found that
medical school students and graduates both had significantly higher numeracy scores than “pre-
medical” undergraduate students. Based on these results, it is presumed that graduate medical
education may have an effect on numeracy ability. Therefore, results suggest that the medical
curriculum may emphasize numeracy more so than traditional legal education does. This makes
sense in that law school does not entail quantitative reasoning coursework where some medical
training does entail further exposure to physical science.
Implications
Broadly speaking, the results of present study suggest that the medical education fails to
instill and maintain critical thinking abilities, careful decision-making preferences, and numeracy
abilities. Traditionally, the Medical College Admission Test (MCAT) was composed of four
sections—verbal reasoning, writing, biological sciences, and physical sciences. However,

medical school curriculum developers, practicing physicians, and other healthcare practitioners
noted that the MCAT was testing progressively outdated concepts—the test needed to be
revamped as the categories tested were seen as less representative of the multifaceted and
complex topics confronting modern day physicians (Kaplan, Satterfield, & Kington, 2012).
Therefore, in the spring of 2015, the Association of American Medical Colleges (AAMC)
decided to implement two new sections—“Psychological, Social, and Biological foundations of
Behavior” and “Critical Analyses and Reading”—to the MCAT exam. According to Kaplan,
Satterfield, and Kington (2012), the new MCAT exam would “build a better physician” (pp.
1265). This substantive shift in the MCAT offers support to the notion that up until the current
year, the medical education system has not equipped contemporary physicians with the
educational tools needed to fully address the issues in modern progressive society. Recognizing
this need for reform in medical school curricula, the current study sheds light on the flaws of the
current medical education training.
Critical Thinking. As previously mentioned, physicians’ lackluster performance on all
critical thinking tasks prompts the following question—why do physicians fail to critically think
at the level of other highly-educated professionals, such as lawyers? Despite the fact that future
research must be conducted to address this question in greater depth, there are serious and
unnerving implications associated with this trend. Inadequate critical thinking skills may allow
physicians to fall prey to irrational and flawed cognition (Croskerry, 2003; Elstein & Schwarz,
2002). For example, physicians who fail to critically think may misinterpret patient symptoms,
laboratory results, or other patient-related information to make faulty diagnoses or prescriptions.
Such flawed cognition and misinterpretation could lead to negative health outcomes for
individuals coming in contact with modern day physicians (Croskerry, 2003; Elstein & Schwarz,

2002). Additionally, trends indicating a decline in critical thinking skills from medical school
students to post-graduate physicians should be of concern to medical professionals and educators
alike.
In order to address this decline in critical thinking skills, educators, physicians, and other
curriculum developers should consider requiring medical professionals to complete regular
“cognitive upkeep” training after they have received their graduate medical degrees (Crosskery,
2013; Weinberger, Pereira, Iobst, Mechaber, & Bronze, 2010). By doing so, the critical thinking
abilities could be equated with the critical thinking cognitive trajectories of law school graduates.
Additionally, the quality of medical education as a whole should be examined. Weinberger and
colleagues (2010) propose that modern-day medical education fails to train medical school
students and physicians to integrate and apply information outside of the explicit knowledge that
is learned from the textbook. Instead of critically thinking and adapting to the evolving needs and
problems presented by the current healthcare system, medical school students and physicians are
hindered by their reliance on specific quanta of facts (Weinberger et al., 2010). Additionally, the
rigor of the medical education system should be examined to help explain the trend suggesting a
decline in physicians’ critical thinking skills post-medical school—provided that this is not an
artifact of cohort differences rather than reflecting genuine post-medical school functioning. It
has been widely documented that the medical education system requires long working hours,
countless assessments, and intrinsic drive (Chang, Eddins-Folensbee, & Coverdale, 2012;
Mandal et al., 2012; Rosen, Gimotty, Shea, & Bellini, 2006). Some assert that such rigorous
standards and non-optimal working environments for medical professionals may be linked to a
“burnout”—a condition leading to a decrease in both cognitive performance and patient care
among those pursuing a medical profession (Montgomery, Todorova, Baban, & Panagopoulou,

2013; Shanafelt, Bradley, Wipf, & Back, 2002). In short, many factors associated with the
medical education system may be to blame for the decline in critical thinking skills in medical
professionals after graduating from medical school, assuming such decline is real and not a
cohort effect.
Decision-Making Preferences. As previously discussed, distinctive trends in decision-
making preferences were apparent among medical school and law school graduates. Specifically,
the results suggested that medical school graduates had a weaker preference to rely on large
amounts of evidence before making decisions compared to law school graduates. This trend may
have relevance to professionals in the medical field. As documented by many researchers,
individuals who rely upon little information when making decisions may be efficient in terms of
wasting little time pondering future actions. Although seemingly helpful, this speed benefit
comes at a potentially serious cost—these individuals may also be at risk for subconsciously
including extraneous information in his/her cognitive evaluation that may ultimately lead to
flawed decision-making (Gigerenzer & Gaissmaier, 2011; Jaeger, Webler, Rosa, & Renn, 2013;
Neth & Gigerenzer, 2015). Specifically, this trend has implications for medical professionals
because physicians are often required to make rapid decisions regarding a patient’s health. For
example, in her article “Missing a Cancer Diagnosis” (2014), columnist Susan Gubar recounted
her own physician misinterpreting multiple retrospectively blatant signs of ovarian cancer: Gubar
reports that physicians often attribute “[b]loating, satiety, fatigue and constipation” to
“menopause, aging, indigestion, irritable bowel syndrome, depression, laziness or whining.”
However, these symptoms are also associated with more severe illnesses such as ovarian cancer.
A report by the National Coalition on Health Care and Best Doctors, Inc. (2013) supports
Gubar’s anecdote: They estimate that upwards of 28% of oncologists reach faulty diagnoses due

to misinterpretation of patient symptoms. Together, Gubar’s experience and the report by the
National Coalition on Health Care and Best Doctors, Inc. reveal how modern day physicians may
rely on limited amounts of information before coming to faulty conclusions.
Furthermore, the ontogeny of these decision-making preferences should also be
examined. Block and colleagues (2013) report that modern day physicians are being forced to
spend an alarmingly small amount of time with patients compared to physicians of the past: The
average physician spends approximately eight minutes in total with each patient before making a
diagnosis or treatment plan—“[r]educed work hours in the setting of increasing complexity of
medical inpatients, growing volume of patient data, and increased supervision may limit the
amount of time interns spend with patients” (pp. 1042). Based upon the presented information,
future policy-based interventions should take aim at allowing physicians to spend more time with
each patient. By increasing patient-physician interaction times, physicians will be allowed to
holistically evaluate each patient and make a beneficial, carefully thought-out, and individualized
treatment plan (Dugdale, Epstein, & Pantilat, 1999). Dugdale and colleagues (1999) propose that
increased patient-physician interaction time may lead to “effective information gathering by
patients, more information provided by physicians, more conversation by patients relative to the
physician, and more expression of affect” (pp. 35). Therefore, increased patient-physician
interaction time may allow physicians’ to gather more information per patient. Ultimately, this
may lead to a shift in physician cognition that emphasizes the importance of drawing upon
adequate amounts of information when making important conclusions.
Numeracy. The results from the Abridged Numeracy Task also raise questions pertaining
to the efficacy of a rigorous STEM-heavy education for individuals interested in pursuing a
medical career. Intuitively, one may expect that an individual exposed to a STEM-intensive

curriculum would have higher numeracy skills compared to another individual without such
exposure; however, the results from the present study do not show significant differences in
numeracy among individuals pursuing less STEM-intensive career paths. To address this issue,
medical education pedagogy and curriculum development should be examined to identify
educational domains that can be tweaked so that numeracy ability may be increased. For
example, educators and curriculum-developers should aim to dismantle the current trend that
favors students relying heavily on problem-solving tools, such as calculators, to answer
numeracy-based problems (see Mead, 2014).
Limitations and Future Directions
The current study investigated the relationship between one’s educational track and
his/her critical thinking and decision making skills. In order for the present results to generalize
to a broader, more representative population, future research should aim to explore critical
thinking and decision-making differences among specific medical specialties (e.g.,
otolaryngologist, radiologists, emergency medicine physicians). By doing so, researchers can
make more accurate claims regarding the effect of more specific types of medical education on
cognitive patterns. In addition to exploring the effect of medical education by specialty on
medical school graduates’ cognitive profiles, future studies may aim to explore specific intervals
that may serve as “critical periods” for learning certain skills throughout one’s medical
education. For example, one may find that first year medical school students experience a
significant gain in critical-thinking skills. If this is true, educators may be able to determine what
salient feature of first year medical school curriculum is responsible for this increase in critical
thinking abilities; therefore, this salient component of medical school curriculum can be
implemented in subsequent years of schooling as well.

Additionally, future studies should aim to sample a more diverse array of individuals. In
the present study, snowball sampling was used to gather medical school students, law school
students, physicians, and lawyers. Therefore, the present study’s sample had relatively small
variance in academic, socioeconomic, and ethnic diversity. A homogenous participant cohort
may also help explain non-significant differences in some analyses. Increased diversity may
yield more distinctive differences in critical thinking and decision-making profiles may become
more evident. Scientific sampling is needed to distinguish between age/training effects and
cohort differences; as noted earlier, it could be that for some reason the full-fledged physicians in
the current study were trained differently or possessed lower cognitive ability than the med
school students, and it is one of these factors rather than medical training per se that explains
why they did poorer than students on some of the tasks. Furthermore, using different cognitive
tests may also help elicit clearer, more generalizable results. Many of the measures used in the
present study, such as the CRT and Abridged Numeracy Scale, were known to be internally
consistent but nevertheless comprised of only a few questions. That these measures were not
very sensitive in measuring the constructs they were meant to tap into may be due to their
uneven sampling of the cognitive domain. Additionally, the Abridged Numeracy Scale had
extremely high accuracy rates for all participant groups tested, thus restricting the range and
setting limits on the regression analyses. Future studies may aim to use more sensitive measures
to tap into numeracy ability among participant groups.
Finally, future studies may aim to utilize cognitive tasks that produce more generalizable
results for the decision-making tasks. Although the present study’s Beads Task may give
rudimentary insight into decision-making preferences among participants of different
occupations, the task can be modified so that professionals can make decisions with information

they are more familiar with dealing on a daily basis. For example, in future studies using
physicians in the participant sample, red and blue beads may be replaced with pieces of
medically relevant information needed to make a diagnosis. Similarly, for lawyers, the original
red and blue beads may be replaced with pieces of evidence needed to reach grounds for criminal
prosecution or acceptance of a plea offer.
Although physicians are perceived as among the most highly knowledgeable and
prestigious professionals, the present findings suggest that physicians, too, may at times fall prey
to cognitive reasoning errors. In addition to vast amounts of rudimentary medical knowledge,
modern-day and future physicians are now being held responsible for keeping up with the rapidly
changing medical system: They are asked to make and assess complex diagnoses, and make
quick, yet evidence-based treatment decisions in the patients’ best-interest. It is unsurprising that
physicians may have an occasional lapse in critical thinking or decision-making processes.
If physicians are to successfully deal with such complex cases, increased working
demands, and greater culpability, they must become increasingly skilled in more rigorous critical
thinking and decision-making skills. In order to adequately provide present and future physicians
with these fundamental cognitive skills, medical school curricula should consider placing a
greater emphasis on teaching the importance of critical thinking skills and appropriate decision-
making strategies.

REFERENCES
Adler, J.N., & Colbert, J.A. (2013). Medical use of marijuana – polling results. NEJM: The New
England Journal of Medicine, 368, 866 – 868. doi: 10.1056/NEJMclde1305159
Averbeck, B.B., Evans, S., Chouhan, V., Bristow, E., & Shergill, S.S. (2011). Probabilistic
learning and inference in schizophrenia. Schizophrenia Research, 127, 115 – 122. doi:
10.1016/j.schres.2010.08.009
Bennett, S., Maton, K. A., & Kervin, L. (2008). The ‘digital natives’ debate: a critical review of
the evidence. British Journal of Educational Technology, 39, 775 – 786. doi:
10.1111/j.1467-8535.2007.00793.x
Block, L., Habicht, R., Wu, A.W., Desai, S.V., Wang, K., Silva, K.N., Niessen, T., Oliver, N., &
Feldman, L. (2013). In the wake of the 2003 and 2011 duty hours regulations, how do
internal medicine interns spend their time? Journal of Internal Medicine, 28, 1042 –
1047. doi: 10.1007/s11606-013-2376-6
Blum-Kulka, S., & Weizman, E. (1988). The inevitability of misunderstandings: Discourse
ambiguities. Text, 8, 649 – 679. doi: 10.1515/text.1.1988.8.3.219
Bornstein, B.H., & Elmer, A.C. (2001). Rationality in medical decision making: a review of the
literature on doctors’ decision-making biases. Journal of Evaluation in Clinical Practice,
7, 97 – 107. doi: 10.1046/j.1365-2753.2001.00284.x
Burke, L.A., & Miller, M.K. (1999). Taking the mystery out of intuitive decision making. The
Academy of Management Executive, 13, 91 – 99. doi: 10.5465/AME.1999.2570557
Ceci, S.J., & Bruck, M. (1995). Jeopardy in the courtroom: A scientific analysis of children’s
testimony. Washington, D.C: American Psychological Association.

Chang, E., Eddins-Folensbee, F., & Coverdale, J. (2012). Survey of prevalence of burnout,
stress, depression, and the use of supports by medical students at one school. Academic
Psychiatry, 36, 177 – 182. doi: 10.1176/appi.ap.11040079
Crosskery, P. (2013). From mindless to mindful practice—cognitive bias and clinical decision
making. New England Journal of Medicine, 368, 2445 – 2448. doi:
10.1056/NEJMp1303712
Crosskery, P. (2002). The importance of cognitive errors in diagnosis and strategies to minimize
them. Academic Medicine, 78, 775 – 780. doi: 10.1097/00001888-200308000-00003
Drew, T., Võ, M.L.H., & Wolfe, J.M. (2013). The invisible gorilla strikes again: Sustained
inattentional blindness in expert observers. Psychological Science, 24, 1848 – 1853. doi:
0.1177/0956797613479386
Dugdale, D.C., Epstein, R., & Pantilat, S.Z. (1999). Time and the patient-physician relationship.
Journal of General Internal Medicine, 14, 34 – 40. doi: 10.1046/j.1525-
1497.1999.00263.x
Elstein, A. S., & Schwarz, A. (2002). Clinical problem solving and diagnostic decision making:
Selective review of the cognitive literature. British Medical Journal, 324, 729 – 732.
Evans, S., Almahdi, B., Sultan, P., Sohanpal, I., Brandner, B., Collier, T., Shergill, S.S., Cregg,
R., & Averbeck, B.B. (2012). Performance on a probabilistic inference task in healthy
subjects receiving ketamine compared with patients with schizophrenia. Journal of
Psychopharmacology, 26, 1211 – 1217. doi: 10.1177/0269881111435252
Fadiman, A. (1997). The spirit catches you and you fall down: A Hmong child, her American
doctors, and the collision of two cultures. New York, NY: Farrar, Straus & Giroux.

Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic
Perspectives, 19, 25 – 42. doi: 10.1257/089533005775196732
Ganzeboom, H.B.G., & De Graaf, P.M., & Treiman, D.J. (1992). A standard international socio-
economic index of occupational status. Social Science Research, 21, 1 – 56. doi:
10.1016/0049-089X(92)90017-B
Ganzeboom, H.B.G., & Treiman, D.J. (1996). Internationally comparable measures of
occupational status for the 1998 international standard classification of occupations.
Social Science Research, 25, 201 – 239. doi: 10.1006/ssre.1996.0010
Garety, P. A., & Freeman, D. (1999). Cognitive approaches to delusions: A critical review of
theories and evidence. British Journal of Clinical Psychology, 38, 113 – 154. doi:
10.1348/014466599162700
Garety, P.A., Hemsley, D.R., & Wessely, S.M.R.C. (1991). Reasoning in deluded schizophrenic
and paranoid patients: Biases in performance on a probabilistic inference task. The
Journal of Nervous and Mental Disease, 179, 194 – 201. doi:
http://dx.doi.org/10.1097/00005053-199104000- 00003
Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of
Psychology, 62, 451 – 482. doi: 10.1146/annurev-psych-120709-145346
Godlee, F., Smith, J., Marcovitch, H. (2011). Wakefield’s article linking MMR vaccine and
autism was fraudulent: Clear evidence of falsification of data should now close the door
on this damaging vaccine scare. BMJ: British Medical Journal, 342, 64 – 66.
Gubar, S. (2014, January 2). Missing a cancer diagnosis. The New York Times. Retrieved from
http://well.blogs.nytimes.com/2014/01/02/missing-a-cancer-diagnosis/

Haberman, C. (2015, February 1). A discredited vaccine study’s continuing impact on public
health. The New York Times. Retrieved from http://www.nytimes.com/2015/02/02/us/a-
discredited-vaccine-studys-continuing-impact-on-public-health.html?_r=0
Henrich, J., & Gil-White, F.J. (2001). The evolution of prestige: Freely conferred deference as a
mechanism for enhancing the benefits of cultural transmission. Evolution and Human
Behavior, 22, 165 – 196. doi: 10.1016/S1090-5138(00)00071-4
Hoffrage, U., & Gigerenzer, G. (1998). Using natural frequencies to improve diagnostic
inferences. Academic Medicine, 73, 538 – 540. doi:
http://dx.doi.org/10.3389/fpsyg.2015.00642
Huq, S.F., Garety, P.A., & Hemsley, D.R. (1988). Probabilistic judgments in deluded and non-
deluded subjects. The Quarterly Journal of Experimental Psychology, 40, 801 – 812. doi:
10.1080/14640748808402300
Jaeger, C.C., Webler, T., Rosa, E.A., & Renn, O. (2013) Risk, uncertainty, and rational action.
New York, NY: Earthscan.
Kahan, D.M., Jenkins-Smith, H., & Braman, D. (2011). Cultural cognition of scientific
consensus. Journal of Risk Research, 14, 147 – 174. doi: 10.1080/13669877.2010.511246
Kaplan, R.M., Satterfield, J.M., & Kington, R.S. (2012). Building a better physician—the case
for the new MCAT. New England Journal of Medicine, 366, 1265 – 1268. doi:
10.1056/NEJMp1113274
Mead, K. (2014). Exploring everyday math without using technology (Unpublished doctoral
dissertation). State University of New York (SUNY), Fredonia, New York.
Levitsky, D.A., & Pacanowski, C.R. (2013). Effect of skipping breakfast on subsequent energy
intake, Physiology & Behavior, 119, 9 – 16. doi: 10.1016/j.physbeh.2013.05.006

Liberali, J.M., Reyna, V.F., Furlan, S., Stein, L.M., & Pardo, S.T. (2012). Individual differences
in numeracy and cognitive reflection, with implications for biases and fallacies in
probability judgment. Journal of Behavioral Decision Making, 25, 361 – 381. doi:
10.1002/bdm.752.
Lipkus, I.M., Samsa, G., & Rimer, B.K. (2001). General performance on a numeracy sale among
highly educated samples. Medical Decision Making, 21, 37 – 44. doi:
10.1177/0272989X0102100105
Loftus, E.F., Levidow, B., & Duensing, S. (1992). Who remembers best? Individual differences
in memory for events that occurred in a science museum. Applied Cognitive Psychology,
6, 93 – 107. doi: 10.1002/acp.2350060202
Malmsheimer, R. (1988). Doctors only: the evolving image of the American physician.
Contributions in medical studies. New York, NY: Greenwood Press.
Mandal, A., Ghosh, A., Sengupta, G., Bera, T., Das, N., & Mukherjee, S. Factors affecting the
performance of undergraduate medical students: A perspective. Indian Journal of
Community Medicine: Official Publication of Indian Association of Preventive & Social
Medicine, 37, 126-129. doi: 10.4103/0970-0218.96104
Maulitz, R.C. (1988). The physician and authority: a historical appraisal. In King, N.M.P.,
Churchill, L.R., & Cross, A.W. (Eds.), The physician as captain of the ship (pp. 3 – 21).
Netherlands: Springer. doi: 10.1007/978-0-585-27589-5_1
Montgomery, A., Todorova, I., Baban, A., & Panagopoulou, E. (2013). Improving quality and
safety in the hospital: The link between organization culture, burnout, and quality of care.
British Journal of Health Psychology, 18, 656 – 662. doi: 10.1111/bjhp

Motiwalla, L.F. (2007). Mobile-learning: A framework and evaluation. Computers and
Education, 49, 581 – 596. doi: 10.1016/j.compedu. 2005.10.011
National Coalition on Health Care & Best Doctors, Inc. (2013). Exploring diagnostic accuracy in
cancer: A nationwide survey of 400 leading cancer specialists. Boston, MA.
Nakao, K., & Treas, J. (1992). The 1989 socioeconomic index of occupations: Construction from
the 1989 occupational prestige scores. Chicago: National Opinions Research Center.
Neth, H., & Gigerenzer, G. (2015). Heuristics: Tools for an uncertain world. In R. Scott & S.
Kosslyn (Eds.), Emerging trends in the social and behavioral sciences: An
interdisciplinary, searchable, and linkable resource (pp. 1 – 18). New York, NY: Wiley
Online Library. doi: 10.1002/9781118900772.etrds0394
Perlis, D., Purang, K., & Anderson, C. (1998). Conversational adequacy: Mistakes are the
essence. International Journal of Human-Computer Studies, 48, 553 – 575. doi:
http://dx.doi.org/10.1006/ijhc.1997.0181
Reyna, V.F., & Lloyd, F. (1997). Theories of false memory in children and adults. Learning and
Individual Differences, 9, 95 – 123. doi: 10.1016/S1041-6080(97)90002-9
Roediger, H.L., & McDermott, K.B. (1995). Creating false memories: Remembering words not
presented in lists. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 21, 803 – 814. doi: http://dx.doi.org/10.1037/0278-7393.21.4.803
Rosen, I.M., Gimotty, P.A., Shea, J.A., & Bellini, L.M. (2006). Evolution of sleep quantity, sleep
deprivation, mood disturbances, empathy, and burnout among interns. Academic
Medicine, 81, 82 – 85. doi: 10.1097/00001888-200601000-00020
Russo, J.E., & Schoemaker, P.J. (2002). Winning decisions: Getting it right the first time. M.
Hittleman (Ed.). New York, NY: Random House, Inc.

Shanafelt, T.D., Bradley, K.A., Wipf, J.E., & Back, A.L. (2002). Burnout and self-reported
patient care in an internal medicine residency program. Annals of Internal Medicine, 136,
358 – 367. doi: 10.1001/archinternmed.2012.3199
Sovacool, B.K. (2014). Diversity: Energy studies need social science. Nature, 511, 529 – 30.
Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. Understandings and sharing
intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28, 675 –
691. doi: http://dx.doi.org/10.1017/S0140525X05000129
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction
fallacy and probability judgment. Psychological Review, 90, 293 – 315. doi:
http://dx.doi.org/10.1037/0033-295X. 90.4.293
Tversky, A., Kahneman, D., Thaler, R.H. (1991). The endowment effect, loss aversion, and
status quo bias. The Journal of Economic Perspectives, 5, 193 – 201. doi:
10.1257/jep.5.1.193
Wakefield, A.J., Murch, S.H., Anthony, A., Linnell, J., Casson, D.M., Malik, M., Berelowitz,
M., Dhillon, A.P., Thompon, M.A., Harvey, P., Valentine, A., Davies, S.E., Walker-
Smith, J.A. (1998). Illeal-lymphoid-nodular hyperplasia, non-specific colitis, and
pervasive developmental disorder in children.
Warman, D.M., Lysaker, P.H., Martin, J.M., Davis, L., & Haudenschield, S.L. (2007). Jumping
to conclusions and the continuum of delusional beliefs. Behaviour Research and
Therapy, 45, 1255– 1269. doi: 10.1016/j. brat.2006.09.002
Wason, P.C. (1968). Reasoning about a rule. The Quarterly Journal of Experimental Psychology,
20, 273 – 281. doi: 10.1080/14640746808400161

Weinberger, S.E., Pereira, A.G., Iobst, W.F., Mechaber, A.J., Bronze, M.S. (2010). Competency-
based education and training in internal medicine. Annals of Internal Medicine, 153, 751-
756. doi: 10.1059/0003-4819-153-11-201012070-00009

Figure 1. Mean CRT composite scores by educational track.
0
0.5
1
1.5
2
2.5
3
Undergraduate Graduate Post-Graduate
MeanCompositeCRTScore
Education Level
Medical Track Law Track

Figure 2. Percent correct on the Linda problem by educational track.
0
10
20
30
40
50
60
70
80
90
100
PercentCorrect
Educational Track

Figure 3. Percent correct on the Wason cards task by educational track.
0
5
10
15
20
25
30
35
PercentCorrect
Educational Track

Figure 4. Beads drawn on easy beads task by education track.
0
1
2
3
4
5
6
7
8
MeanBeadsDrawn
Education Level

Figure 5. Beads drawn on difficult beads task by education track.
0
2
4
6
8
10
12
NumnerBeadsDrawn
Education Level

Figure 6. Abridged numeracy scale mean composite scores by educational track.

Appendix A

Appendix B

Appendix C

Appendix D

Appendix E

Appendix F
Easy Version

Difficult Version

Critical Thinking Skills of Doctors Analyzed

Recommended

Recommended

More Related Content

Similar to Critical Thinking Skills of Doctors Analyzed

Similar to Critical Thinking Skills of Doctors Analyzed (20)

Critical Thinking Skills of Doctors Analyzed