Limitations of self report

Running head: Limitations of Self-Report data and Peer Report Data 1
Limitations of Self-Report data and Peer-Report Data
Andrew Pearce
Psych 6113
Yorkville University

Psychoanalytic Theory Compared with the Neo-Analytical Theory 2
The Psychoanalytic Theory Compared with the Neo-Analytical Theory
Every person is a dynamic, complex and unique individual featuring equally dynamic and
complex personality characteristic’s all influenced by a matrix of psychological forces (Muris,
2006; Johnson, 2010). As a result there are several personality theories at the psychologist’s
disposal to ascertain the specifics and reasons for a person’s personality development and
temperament. According to Friedman and Schustack (2012) there are eight personality
perspectives (I.e., Psychoanalytic, Neo-Analytic/Ego, Biological, Behaviorist, Cognitive, Trait,
Humanistic/Existential, Interactionist). Each of these personality theories offers a perspective on
the various psychological forces that influence a person’s personality, as well as speak to “how
and why people act and behave as they do?”
Amongst the eight personality perspectives there are sources of assessment information, such
as self-report data and peer-report data that can be employed to gather reliable and relevant
information pertaining to person’s personality, including core traits, motivations, expressive
styles, and emotional expressiveness (Melchert, 2011; Friedman and Schustack, 2012).
The International Personality Item Pool – Neuroticism, Extraversion, and Openness (IPIP-
NEO) is a prime example of a personality test that relies on self-reporting. The IPIP-NEO is a
personality questionnaire that assesses people on the Big Five Model. The long-form IPIP-NEO
contains 300 items, whereas the short form IPIP-NEO contains 120 items “designed to measure
the same traits as the original IPIP-NEO, but more efficiently with fewer items” (Personality
Test Center, 2014). Also, the IPIP-NEO-120 is a self-report test capable of assessing “behaviour
in online role playing games” (Eladhari & Mateas, 2008). In addition the IPIP-NEO has a mini
version. Research shows that the Mini-IPIP, a 20-item short form of the 50-item International

Personality Item Pool (Goldberg, 1999), is a psychometrically acceptable and practically useful
short measure of the Big Five factors of personality (Donnellan, Oswald, Baird, Lucas, 2006).
In short, the accuracy and reliability of the IPIP-NEO is due to its standardization, in that
individuals answer the same questions and their results are compared with other results
(Goldberg, Johnson, Eber, Hogan, Ashton, Cloninger, 2006). In addition, the IPIP-NEO has
been translated to work in various languages and cultures, which further supports its validity and
reliability (Church, Katigbak, 2002).
In all both the long-form and short-form of the IPIP-NEO personality test can assist in
determining core personality tendencies as well as facilitating companies find the person that is
most suitable for specific jobs, based on their tendency scores. Also, the IPIP-NEO helps
predict future behaviour, and it is consistent in assessing what it is responsible for testing
(García, Muñoz, & Ortiz, 2005). However, its accuracy can be affected by bias, including social
desirability responses.
The Minnestoa Multiphasic Personality Inventory (MMPI-2-RF) is another example of a
wide-ranging self-report and is commonly employed by the police force because of its reliability
to ascertain psychopathology, including thought dysfunction and behavioural dysfunction
(Cochrane, Tett, & Vandecreek, 2003). Participants who received a high scale in thought
dysfunction correlated with “a number of externalizing behaviour dysfunction including the use
of excessive force, inappropriate language, rude behaviour, and deceptiveness” (Sellbom & Ben-
Porath, 2005).
Unlike other personality assessments the MMPI-2-RF, which is comprised of 51 scale
measures of personality and psychopathology includes “nine scales to assess acquiescent
responding and content based invalid responding including over reporting and under reporting”

(Ben-Porath, & Rellegan, 2011). This is significant because there is always risk for bias in
personality inventories and assessments.
The IPIP-NEO and the MMPI-2-RF assessments reveal that self-reporting is accurate in
gathering reliable answers regarding a person’s motivations, thoughts, and needs, but is not free
from subjective bias. Similarly peer-report data is another effective means of providing an
external observation of a person’s behaviour, core tendencies, expressive styles, and emotional
expressiveness, but by no means is peer reporting insulated from subjective bias, in that peer-
report data is not privy to the inner psyche and motivations of a person being observed
(McDonald, 2008).
In all, self-report data and peer-report data are two commonly used sources of assessment
information that are not insulated from potential bias, which can affect the validity and reliability
of obtained information (Donaldson & Grant-Vallone, 2002). Therefore, the purpose of this
paper is to summarize limitations of self-reporting and peer-reporting, such as acquiescence
response set, social desirability response set, and bias knowledge base, which can all affect and
limit the reliability and viability of gathered assessment information (Melchert, 2011).
Limitations of Self-Report Data
Self-report data is the primary and reliable source of assessment information in ascertaining
the inner motivations, thoughts and needs of a person that influence a person’s behaviour (Wang,
Jome, Haase, Bruch, 2006).
Acquiescence response
Acquiescence response set is a limitation of self-reporting and occurs when a person has a
tendency to give affirmative answers regardless of the content of the question (Messick, 1967).
This happens when a “yes” or “no” question offers two reverses, in that to answer “no” would

present a negative correlation, therefore the person feels obliged to answer in the affirmative.
Also, an unbalance of positively keyed and negatively keyed items in an assessment can
contribute to “…the effects of hypothetical response biases toward ‘‘yea-saying’’ (often called
acquiescence response bias) and ‘‘nay-saying’’ (Furr, 2011). Whereas other research found that
“employing negatively keyed items to reduce acquiescence can actually impair response
accuracy” (Schriesheim & Hill, 1981).
Another plausible reason for acquiescence response could potentially be a person’s inability
to understand or the judge the question accordingly and as a result consistently answers in the
affirmative regardless of the content (Krosnick, 1991).
In all, acquiescence response as the result of an unbalance proportion of negative and positive
questions can infringe upon the reliability and viability of a personality test.
Social desirability
Social desirability response set is another factor that can limit the reliability and validity of an
assessment, including the IPIP-NEO. This is because respondents may not be comfortable to
present themself in a negative light or are focused on managing other people’s impressions of
themself. For whatever reason the respondent over-reports or under-reports in order that his or
her answers are socially desirable or considered moderate (Huang, Liao, and Chang, S, 1998).
Research shows that social desirability presents a threat to the validity and reliability of
assessments, in that “the test is not measuring what it is meant to assess due to the participants
bias to present him or herself in a favourable view, rather than expressing true behaviour and risk
being seen as a socially undesirable (Fisher, 1993). For example, if a person was applying for a
job that requires the completion of a MMPI the respondent may not answer truthfully to
questions such as “I tend to take things personally” or “what others think of me is no concern of

mine”, because the respondent’s answers could effect whether he or she gets the job, or not
(Paulhus, Lysy, Yik, 1994).
Respondent Knowledge
Another limitation of self-report data pertains to a respondent’s knowledge of themself.
Although a self-report is reliable and efficient in ascertaining and assessing a persons emotional
expressiveness (temperament) including “distress, mood, and other subjective states” (Melchert,
2011, p. 125), if a person is unaware of his or her core tendencies and behaviour then the
answers that are given can be misleading and inaccurate. For example, self-report data can
“misleading” if the person providing the information is not honest with his or her answers.
Research shows that people can prefer their facade or fantasy about themselves rather than being
truthful (Van de Mortel, 2008). This is not unheard of because people are not only capable of
not being forthcoming in answering questions honestly, but they can also be unwilling or unable
to see or acknowledge what one’s peers can clearly see. For example, a peer may be able to
observe a friends prejudice and reflect this prejudice in his evaluation, whereas the self-reporter
may justify his prejudice and under-report (McFarland, 2010).
In all, beyond the limitations of self-report data, peer-reporting is a means of “filling in the
gaps”, to establish reliability and validity of self-report data (Friedman and Schustack, 2012).
Limitations of Peer-Report Data
Peer-report data provided by friends and acquaintances is an essential and viable means of
gathering reliable and relevant information for assessment purposes (Hofstee, 1994). This is
because self-reporting can be limited in providing the full spectrum and scope of one’s social
tendencies, behavioural tendencies (Friedman and Schustack, 2012). Therefore, a peer-report
completed by a friend or acquaintance will not only provide a valuable evaluation of a person’s

expressive style and emotional expressiveness at home and work, but will also provide
information that will supplement or confirm the self-report data (Melchert, 2011). In fact,
research verifies that peer-reporting can provide more accurate predictions about future behavior
than self-reporting. For example, “in a sample of military recruits, higher peer-reports of
antisocial [personality disorder] predicted involuntary discharges due to discipline problems
better than the analogous self-reports (Fiedler, Oltmanns, & Turkheimer, 2004).
Also, due to the fact that “people grow and change and even their basic patterns of responding
may change” (Friedman and Schustack, 2012, p. 29), peer-report data can account for these
fluctuations and changes as well. However, peer-report data cannot escape potential bias; for
this reason, peer-reporting is just as susceptible to acquiescence response, social desirability
response and knowledge issues.
Acquiescence response
Acquiescence response is also a potential limitation of peer-reporting. That is because similar
to the issues described in self-report section, a person completing a peer report may also have
have a tendency to give affirmative answers regardless of the content due to negative correlations
as well as difficulty in understanding the question at hand (Krosnick, 1991).
Social desirability
Furthermore, peer-report data is not exempt from social desirability response set bias because
the respondent may not feel comfortable depicting his friend in a negative light (Ben-Porath &
Waller, 1992). For example a peer-reporter, who is about to complete a IPIP-NEO test on his
friend, may know very well that his friend is not “open to new experiences”, however the peer
reporter may not want to cast his friend as a rigid traditionalist, so he projects his friend more
favorably.

Another reason social desirability can occur is when a peer’s report could affect a person’s
job or affect their reputation in the eyes of others. For example the peer may be less honest in
answering questions regarding the persons political position, whereas an employee who is
nothing more than an acquaintance may be more forthcoming with information because the
evaluation does not personally affect them. Nonetheless a recent survey found that although
respondents will complete an IPIP NEO type of questionnaire “at times [they] do not provide
accurate responses to questions, especially in reference to topics of a sensitive nature such as
politics” (Gittelman, Lange, Cook, & Frede, 2015, p. 254).
Also, a peer may under-report or over-report based on the circumstance. For example, if
Child Services is strongly considering taking a parent’s children away from him because he
affirms and employs aversive punishment, the peer-reporter may be inclined to answer the
questions pertaining to punishment in such a way to present his friend in a socially acceptable
light (Huang, Liao, and Chang, S, 1998). Researchers have suggested that close acquaintances
and especially friends may also be motivated to evaluate the target person positively (Cohrs,
Kämpfe-Hargrave; Konstabel, Aavik, & Allik, 2006).
In all, it is possible that a person for whatever reason may be predisposed with a positive
outlook of himself, which could also extend to others during reporting (McDonald, 2008).
Peer Knowledge
Another limitation of peer-report data pertains to a peer’s knowledge of the person they are
evaluating. A peer may very well be able to observe patterns of behaviour and consistency (i.e
extraversion) that a person may not reveal on a self-report, because extraversion is an easily
observable trait, however, a peer’s assessment is still subjective in nature. As a result a peer may
be wrong in their observations. They might “see things that are not really there...[and] [miss]

things that are really there” (Friedman and Schustack, 2012, p. 262). For example, a peer may
mistake a person’s conscientious behaviour for lack of “openness to experience”, when in fact
the person is not closed minded, but also likes to live according to routines and schedules. Yet,
research suggests, “[peers] can usually judge others’ personal attributes quite accurately. They
may also be less prone to a social desirability bias” (Cohrs, Kämpfe-Hargrave, & Riemann,
2012; McCrae & Weiss, 2007).
Social rift and prejudice
In addition, a social rift between the peer-reporter and the self-reporter may influence a peer’s
evaluation, in that he or she states the self-reporter is neurotic and uncooperative, even hen this
might not be the case. As a result although peer-reports can provide additional information to
either support or reject self-report data, the peer information gathered may not necessarily be
more accurate than self-report data (Clifton, 2004).
Furthermore, prejudice is an aspect that can affect validity of a personality test as well. A
study conducted by McConnell and Leibold (2001) found that “participants with stronger
implicit prejudice toward Blacks were judged to behave more negatively in social interactions
with a Black experimenter” (Cohrs, Kämpfe-Hargrave, & Riemann, 2012).
In all, research appears to show that the validity and reliability of peer-report data is
correlated with the “agreement among multiple judges and with self-reports” (Funder, 1995)
Conclusion
Self-report data and peer-report data provide the counsellor and third parties with a means of
useful information to establish a formidable assessment of a person’s personality, including core
trait tendencies, inner motivations, expressive styles, and emotional expressiveness. Research
shows that “self-peer agreement for the Big Five usually ranges between .30 and .60 and is often

Psychoanalytic Theory Compared with the Neo-Analytical Theory
10
higher than .50, depending on the number of peer-reporters” (Cohrs, Kämpfe-Hargrave, &
Riemann, 2012; McCrae & Costa, 2003; McCrae & Weiss, 2007). However, assessment and
data stream provided by self-reporting and peer-reporting are not shielded from potential
acquiescence response bias, social desirability bias and knowledge response bias, as well as
ideological attitudes, prejudice, cultural and language issues (Cohrs, Kämpfe-Hargrave, &
Riemann, 2012). Therefore, due to the apparent and potential limitations of self-reporting and
peer-reporting data it is wise for a counsellor to employ a combination of self-report and peer-
report data in order to gather reliable and relevant information. If a counselor does not employ
both types of assessments the counselor runs the risk of obtaining
unreliable and incomplete [information], [which] can lead to interventions that are unhelpful
or even deleterious. Therefore…[it] [is] [important] [to] [collect] data from a variety of
sources to obtain the most relevant and reliable assessment information possible” (Friedman
and Schustack, 2012).
For example, one’s level of “agreeableness” is better assessed and verified by employing a
combination of self-report data and peer-report data because a person may view themself as
cooperative and easy to get along with, whereas the person’s co-workers may disagree or agree.
In all, the employment of a peer-report will either deny or verify one’s claims (Melchert, 2011).
In conclusion, self-report data and peer-report data help provide a reliable and relevant means
of gathering useful information pertaining to the dynamic, complex and unique individual who
features equally dynamic and complex personality characteristic’s (Muris, 2006; Johnson, 2010).
Although both self-report and peer-report clearly have potential limitations, the limitations can
be reduced by the counsellor being aware that these assessments can be affected by bias response
sets and one’s knowledge base of the subject being evaluated.

11
References
Ben-Porath Y. S., Waller, N. G. (1992). Five big issues in clinical personality assessment: A
rejoinder to Costa and McCrae. Psychological Assessment, 4 (1), 23–25. doi:10.1037/1040-
3590.4.1.23.
Ben-Porath, Y. S., & Tellegen, A. (2008). Empirical correlates of the MMPI-2 Restructured
Clinical (RC) Scales in mental health, forensic, and nonclinical settings: An introduction.
Journal of Personality Assessment, 90, 119–121. doi:10.1080/00223890701845120
Carver, C.S., & Scheier, M. F. (1996). Perspectives on personality (3rd
ed.) Boston: Allyn &
Bacon
Cochrane, R. E., Tett, R. P., & Vandecreek, L. (2003). Psychological testing and the selection of
police officers: A national survey. Criminal Justice and Behavior, 30, 511–537.
doi:10.1177/0093854803257241
Cohrs, J. C., Kämpfe-Hargrave, N., & Riemann, R. (2012). Individual differences in ideological
attitudes and prejudice: Evidence from peer-report data. Journal Of Personality And Social
Psychology, 103 (2), 343-361. doi:10.1037/a0028706
Clifton, A., Turkheimer, E., & Oltmanns, T. F. (2004). Contrasting perspectives on personality
problems: Descriptions from the self and others. Personality and Individual Differences.
Donaldson, I. S., Grant-Vallone, J. E. (2002). Understanding self-report bias in organizational
behavior research. Journal of Business and Psychology, 17 (2), 245-260.
10.1023/A:1019637632584
Donnellan, M. B., Oswald, F. L., Baird, B. M., Lucas, R. E. (2006). The Mini-IPIP Scales: Tiny-
yet-effective measures of the Big Five Factors of Personality. Psychological Assessment, 18
(2), 192-203. doi: 10.1037/1040-3590.18.2.192
Eladhari, M. P., & Mateas, M. (2008). Semi-autonomous avatars in world of minds: Proceedings
of the 2008 international conference on advances in computer entertainment technology.
Association for Computing Machinery, doi:10.1145/1501750.1501798.
Fisher, R. J. (1993), Social desirability bias and the validity of indirect questioning. Journal
of Consumer Research, 20 (09), 303-315.
Friedman, H. S., & Schustack, M. W. (2011). Personality: Classic theories and modern
research. (5th ed.). Boston: Allyn and Bacon.
Funder, D. C. (1995). On the accuracy of personality judgment: A realistic
approach. Psychological Review, 102, 652–670. doi:10.1037/0033-295X.102.4.652
Furr, R. M. (2011). Scale construction and psychometrics for social and personality psychology.

12
Thousand Oaks, CA: Sage.
García, J. F., Muñoz, M. E., & Ortiz, A. M. (2005). Personality and contextual variables in
teacher burnout: Personality and Individual Differences, 38 (4): 929–940.
doi:10.1016/j.paid.2004.06.018.
Gittelman, S., Lange, V., Cook, A. C., & Frede, M. F. (2015). Accounting for social-desirability
bias in survey sampling: A model for predicting and calibrating the direction and
magnitude of social-desirability bias, 55 (3), 242-254. doi: 10.2501/JAR-2015-006.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R. (2006).
The International Personality Item Pool and the future of public domain personality measures.
Journal of Research in Personality, 40, 84–96. doi: /10.1016/j.jrp.2005.08.007
Huang, C., Liao, H. and Chang, S. (1998). Social desirability and the clinical self-report
inventory: Methodological reconsideration. Journal of Clinical Psychology, 54 (4): 517-528.
Johnson, E. E. (2010). Psychology and Christianity: Five Views. Downers Grove, IL:
InterVarsity Press
Jones, I., & Wheadon, C. (2015). Peer assessment using comparative and absolute judgement.
Studies In Educational Evaluation, 47, 93-101. doi:10.1016/j.stueduc.2015.09.004
Konstabel, K., Aavik, T., & Allik, J. (2006). Social desirability and consensual validity of
personality traits. European Journal of Personality, 20, 549–566. doi:10.1002/per.593
Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude
measures in surveys. Applied Cognitive Psychology, 5, 213–236.
McDonald, J. D. (2008). Measuring personality constructs: The advantages and disadvantages of
self-reports, informant reports and behavioural assessments. Enquire, 1 (1), 1-18.
McFarland, S. (2010). Authoritarianism, social dominance, and other roots of generalized
prejudice. Political Psychology, 31, 453–477. doi:10.1111/j.1467-9221.2010.00765.x
Melchert, P. T. (2011). Foundations of Professional Psychology: The End of Theoretical
Orientations and the Emergence of the Biopsychosocial Approach. London: Elsevier
Messick, S. J. (1967). The psychology of acquiescence: An interpretation of research evidence.
Response set in personality assessment. Chicago: Aldine.
Neeley, M. S., & Cronley, L. M. (2004). When research participants do not tell it like it is:
Pinpointing the effects of social desirability bias using self vs. indirect-questioning.
Association for Consumer Research, 31, 432-433.
Paulhus, D. L., Lysy, D., & Yik, M. (1994). Self-presentation on a real-world job application

13
Unpublished manuscript, University of British Columbia.
Personality Test Center (2014). International Personality Item Pool Representation of the NEO
PI-R. Retrieved from http://personalitytest.net/ipip/index.html
Schriesheim, C. A., & Hill, K. D. (1981). Controlling acquiescence response bias by item
reversals: The effect on questionnaire validity. Educational and Psychological Measurement,
41, 1101–1114. Doi:/10.1177/001316448104100420.
Sellbom, M., & Ben-Porath, Y. S. (2005). Mapping the MMPI-2 Restructured Clinical scales
onto normal personality traits: Evidence of construct validity. Journal of Personality
Assessment, 85, 179–187. Doi:10.1207/s15327752jpa8502_10
Sjöberg, L. (2015). Correction for faking in self-report personality tests. Scandinavian Journal
of Psychology, 56(5), 582-591.
Van de Mortel, T. F. (2008). Faking it: social desirability response bias in self-report
research. Australian Journal of Advanced Nursing, 25, (4), 40-48.
Wang, N., Jome, L. M., Haase, R. F., Bruch, M. A. (2006). The role of personality and career
decision making self-efficacy in the career choice commitment of college students. Journal of
Career Assessment, 14 (3): 312–332. doi:10.1177/1069072706286474.

14

Limitations of self report

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Limitations of self report

Similar to Limitations of self report (14)

Limitations of self report