Thread 1 DB Forum 2: Scoring Applicant’s Responses Determining Scores to Responses             When interviewing candidates, evaluating their letters of recommendation, and reviewing their previous work experience, it is essential that each component contain elements of standardization to ensure that the applicant hired is 1) the best candidate for the position based on these measures, and 2) the candidate was objectively evaluated. The position I’ve chosen for the Job Analysis Project is a Police Officer. Interview Questions             Interview questions should be highly standardized, with specific time limits allotted to each applicant and uniform questions to be asked by the interviewers. An effective method of assessing the interview results would be to rate the job applicants on a scale of 1-5. In addition to this standardization, it is imperative that the interrater agreement is achieved. “Critical uses, such as hiring decisions, demand very high levels of reliability, well in excess of 75% agreement” (Heneman, Judge, & Kammeyer-Mueller, 2015, p. 332). One implication of scoring interview questions is that interviewees might lie during an interview to improve their scores. “The size of the improvement of these interview scores correlates with the interviewee’s cognitive ability and their ability to identify the targeted interview dimensions” (Buehl, Kuhnel, & Melchers, 2018, para. 2). Letters of Recommendation             Recommendation letters are tricky to objectively, accurately, and fairly score relative to other applicants. This is due to the various factors that recommendation letters address, some addressing areas that others do not. Another implication that further challenges scoring is comparing scores of standardized letters of recommendations and traditional narrative letters of recommendation (Young, 2017). In order to create more standardization in scoring letters of recommendation, I would consider implementing the method shown in Exhibit 6 (p. 392). The method of providing a scale for mental ability, consideration-cooperation-dependability-reliability, urbanity, and vigor will allow for measurement of the individual applicant and a statistical comparison between applicants as well (Heneman, Judge, & Kammeyer-Mueller, 2015). Previous Work Experience             Scoring previous work experience is a tedious and challenging task, as applicant’s work experience, knowledge, and skills learned vary greatly. The scoring of previous work experience for police applicants, specifically, is unique compared to the way a typical office position would be scored. The position requires a strength and knowledge-based training academy, so it would be difficult to score work experience based on those terms; however, morals, judgement, integrity, and objectivity could be measured through previous position requirements, and specific examples of presenting these values through job responsibilities. Considerations             Th ...

1. Thread 1
DB Forum 2: Scoring Applicant’s Responses
Determining Scores to Responses
When interviewing candidates, evaluating their letters
of recommendation, and reviewing their previous work
experience, it is essential that each component contain elements
of standardization to ensure that the applicant hired is 1) the
best candidate for the position based on these measures, and 2)
the candidate was objectively evaluated. The position I’ve
chosen for the Job Analysis Project is a Police Officer.
Interview Questions
Interview questions should be highly standardized, with
specific time limits allotted to each applicant and uniform
questions to be asked by the interviewers. An effective method
of assessing the interview results would be to rate the job
applicants on a scale of 1-5. In addition to this standardization,
it is imperative that the interrater agreement is achieved.
“Critical uses, such as hiring decisions, demand very high levels
of reliability, well in excess of 75% agreement” (Heneman,
Judge, & Kammeyer-Mueller, 2015, p. 332). One implication of
scoring interview questions is that interviewees might lie during
an interview to improve their scores. “The size of the
improvement of these interview scores correlates with the
interviewee’s cognitive ability and their ability to identify the
targeted interview dimensions” (Buehl, Kuhnel, & Melchers,
2018, para. 2).

2. Letters of Recommendation
Recommendation letters are tricky to objectively,
accurately, and fairly score relative to other applicants. This is
due to the various factors that recommendation letters address,
some addressing areas that others do not. Another implication
that further challenges scoring is comparing scores of
standardized letters of recommendations and traditional
narrative letters of recommendation (Young, 2017). In order to
create more standardization in scoring letters of
recommendation, I would consider implementing the method
shown in Exhibit 6 (p. 392). The method of providing a scale
for mental ability, consideration-cooperation-dependability-
reliability, urbanity, and vigor will allow for measurement of
the individual applicant and a statistical comparison between
applicants as well (Heneman, Judge, & Kammeyer-Mueller,
2015).
Previous Work Experience
Scoring previous work experience is a tedious and
challenging task, as applicant’s work experience, knowledge,
and skills learned vary greatly. The scoring of previous work
experience for police applicants, specifically, is unique
compared to the way a typical office position would be scored.
The position requires a strength and knowledge-based training
academy, so it would be difficult to score work experience
based on those terms; however, morals, judgement, integrity,
and objectivity could be measured through previous position
requirements, and specific examples of presenting these values
through job responsibilities.
Considerations
Through the hiring process, the hiring manager should
ensure that the process is free of contamination. Contamination

3. can occur through a variety of sources. These sources include
content domain, standardization, raters, and the rating situation.
In order to achieve the most control in the process, the material
listed on tests should be clearly defined, the time limit and
questions uniform for each applicant, and raters properly trained
(Heneman, Judge, & Kammeyer-Mueller, 2015, p. 331). The
scoring process should always maintain integrity and
objectivity. Titus 2:7-8 (ESV) says, “Show yourself in all
respects to be a model of good works, and in your teaching
show integrity, dignity, and sound speech that cannot be
condemned, so that an opponent may be put to shame, having
nothing evil to say about us.”
References
Buehl, A. K., Kuhnel, J., Macan, T., & Melchers, K. G. (2018).
Tell me sweet lies: How does faking in interviews affect
interview scores and interview validity?
Journal of Business and Psychology, 34
(1), 107-124, https://doi-
org.ezproxy.liberty.edu/10.1007/s10869-018-9531-3
Heneman, H. G. III, Judge, T. A., & Kammeyer-Mueller, J. D.
(2015).
Staffing organizations
(8th ed.). Mishawaka, IN: Pangloss Industries, McGraw-Hill
Huber, A. (2018). Exploring hiring practitioner preferences for
and assessment practices of prospective candidates.
Journal of Interior Design, 43
(4), 21-44.
https://doi-org.ezproxy.liberty.edu/10.1111/joid.12131
Young, V. N. (2017). Letters of recommendation: Association
with interviewer’s perceptions and preferences.

4. Journal of Otolaryngology-Head and Neck Surgery, 156
(6), 1108-1113. https://doi.org/
10.1177/0194599817706063
thread 2
Question
Assume you gave a general ability test, measuring both verbal
and computational skills, to a group of applicants for a specific
job. Also, assume because of severe hiring pressures, you hired
all of the applicants, regardless of their test scores. How would
you investigate the criterion-related validity of the test?
Answer
Introduction
This post will look at the ways to test the criterion validity of a
test particularly as it relates to predictive validity. In
Staffing Organizations
it is said that “Scores on the predictor and criterion are
obtained for a sample of individuals; the relationship between
the scores is then examined to make a judgment about the
predictor’s validity” (Heneman, Judge, & Kammeyer-Mueller,
2019. p. 340) According to Criteria, a provider of pre-
employment testing, “In the context of pre-employment testing,
predictive validity refers to how likely it is for test scores to
predict future job performance.” (What is Predictive Validity?,
2019). Checking the validity is useful in case adjustments must
be made to the tests for future applicants. If the test is a poor

5. predictor of performance, then it will have to be changed. To
test this, each new employee’s performance evaluations can be
compared to their scores on the general ability test. If
evaluations do not cover the necessary areas, then an additional
evaluation could be added during a probation period, or as an
internal study after a period of time for new hires. This would
allow a relatively unobtrusive method of gathering the
information. Also, criterion measures do not need to be
restricted to performance measures. Things such as attendance,
retention, safety, and customer service can also be used
(Heneman, Judge, & Kammeyer-Mueller, 2019. p. 340). The
next sections will explore measuring the accuracy of the verbal
and computational skills in turn.
Verbal Skills
Verbal skills could be measured using the evaluations
of peers, supervisors, subordinates, and customers. The type of
position and working conditions will affect how the
performance evaluations should be interpreted and compared to
the original test, but with the information gathered, one should
be able to identify any inconsistencies (or non-valid
predictions) between the general ability test and actual
performance. In
Staffing Organizations,
it is recommended that if a job requires a high level of verbal
skills, oral responses should be solicited (Heneman, Judge, &
Kammeyer-Mueller, 2019. p. 372). In this way, better, more
accurate comparisons can be made between predictions and
actual performance. If there are significant differences, the
general ability test should be adjusted, or the method of
conducting the general ability test should be adjusted
accordingly.
Computational Skills

6. The word “Computational” is an adjective that is
defined as “Of, relating to, or used for computation; relating to
or making use of computers.” (OED Online, 2019).
Computational skill in some form is necessary for most modern
jobs. According to
Staffing Organizations, “
If the job requires constant interaction with the computer,
applicants should use a computer to enter their responses.”
(Heneman, Judge, & Kammeyer-Mueller, 2019. p. 372). This is
perhaps a more easily measured dimension than verbal skills,
because computer assisted productivity can be more objective
while verbal skills is more subjective. A few things that could
be used as standards are the employee’s ability to receive and
reply to emails, their ability to create and share documents, and
the frequency that they require assistance from IT, to name a
few. Depending on the results of comparing test scores and
performance, the test might need to include company specific
measures to test ability to perform common computational tasks
that are necessary for all positions, or the test might need to be
made easier if performance is better than the test predicted.
Conclusion
The important thing for companies is to pay close attention to
measurements because they are essential for a systematic
staffing process. (Heneman, Judge, & Kammeyer-Mueller, 2019.
p. 314). Checking the validity of tests is an important part of
continually improving staffing systems. This post offered a few
suggestions to test the validity of a general ability test related
to verbal and computational skills, and hopefully offered some
insight into ensuring the test measures what it is intended to
measure.
References:

7. What is Predictive Validity? (2019). Retrieved from:
https://www.criteriacorp.com/resources/glossary_predictive_val
idity.php
Heneman, H. G., Judge, T., & Kammeyer-Mueller, J. (2019).
Staffing organizations
. Columbus, OH: Pangloss Industries.
"computational, adj." OED Online, Oxford University Press,
June 2019, www.oed.com/view/Entry/37969. Accessed 5 June
2019.