ggu2016summer grit assessment prelim report

8/8/2016
1
(C) 2016: Students of PSYC344 Summer Course – GGU
GRIT
Design & Preliminary Validation of a
Personality Assessment
GOLDEN GATE UNIVERSITY
PSYC344 – TESTS & ASSESSMENTS (SUMMER, 2016)
This report contains preliminary results from a beta-testing of a newly developed assessment
designed by Master’s students of psychology (I/O, Counseling, MFT) attending the
Golden Gate University’s “Tests & Assessments” course.
(C) 2016: Students of PSYC344 Summer Course – GGU 2
Content
Project Summary
Thank you & Designers
1. About the GRIT Assessment
 Definition
 Major factors and subfactors
 Demographic Breakdown (of participants)
2. Reliabilities
3. Factor analyses
4. GRIT scores by demographic groups
5. GRIT validation
6. Variance analyses
7. Results summary
Project Summary
Present project was carried out by students enrolled
in PSYC344 – Tests and Assessments (Summer, 2016)
course – offered at the Golden Gate University.
Major aim of the course is to introduce Master’s
level students in psychology (I/O, Counseling, MFT)
to basic concepts in assessment design and
validation.
Results of this report are based on our initial data
collection (beta); hence, is preliminary and should
be interpreted with caution. A standard assessment
design and validation process involves multiple data
collection phases.

8/8/2016
2
Thank you!
The completion of this project would not
have been possible without the help of those
who took the time to complete the
assessment. Hence, we would like to
sincerely thank all those who participated in
the data collection phase of this assessment.
 Data collection period: July 6 – July 15, 2016
 Total # of participants who started the assessment: 197
 Total # of participants who completed the assessment in
its entirety: 171
Designers
Students:
Matthan Churchill
Scott Harris
Toni Martin
Suzanne Mekenkamp
Rehab Nada
Katherine Orozco
Shirin Pradhan
Wayne Seeley
Wenting (Gloria) Zhang
Patrick McAndrew
Instructor: Stephen B. Jeong, Ph.D.
1. About the GRIT Assessment

8/8/2016
3
Definition
Gritological Rating Intensity Test
GRIT is viewed as a personality construct and is
defined as the degree to which a person “sees
things through” (i.e., persists) regardless of the
challenges / obstacles to its resolution.
Factors & Sub-factors
Resilience
(Sub-factor1)
Courage
(Sub-factor2)
Emotional
Toughness
(Sub-factor3)
GRIT
GRIT is conceptualized as consisting of three related sub-
factors: Resilience, Courage, and Emotional Toughness. A
preliminary set of 27 items were developed and analyzed.
Gender
Female respondents
outnumber male respondents
by more than double

8/8/2016
4
Age
50% of the participants who took
the assessment ranged in age
from 31 to 45
Age Same information in graphical
format
Ethnicity
64% of the participants who took
the assessment were either
Caucasian or Hispanic/Latino

8/8/2016
5
Ethnicity Same information in graphical
format
Education
Nearly half of the participants who
took the assessment had either a
Bachelor’s or Master’s degree
Education (percentages)
0.5
2
21.3
10.2
24.9
22.8
0.5
1.5
3
Education
High school (grades 9-12, no degree) High school graduate (or equivalent)
Some college (1-4 years, no degree) Associate's degree (including occupational or academic degrees)
Bachelor's degree (BA, BS, AB, etc) Master's degree (MA, MS, MENG, MSW, etc)
Professional school degree (MD, DDC, JD, etc) Doctorate degree (PhD, EdD, etc)
Prefer not to answer
Same information in graphical
format

8/8/2016
6
SES (Income Level)
Frequency % Valid % Cumulative %
<$50,000 36 18.3 18.3 32
$51,000 to $100,000 63 32 32 89.8
$101,000 to $150,000 30 15.2 15.2 51.8
$151,000 to $200,000 12 6.1 6.1 57.9
>$200,000 9 4.6 4.6 36.5
Prefer not to answer 20 10.2 10.2 100
Total 197 100 100
This information should be self-
explanatory
Faith/Spirituality
A little over half of the participants
who took the assessment reported
being Christian
Faith Same information in graphical
format

8/8/2016
7
Number of Siblings
Frequency Percent Valid % Cumulative %
None (only child) 15 7.6 7.6 98.5
1 35 17.8 17.8 47.2
2 43 21.8 21.8 69
3 31 15.7 15.7 84.8
4 12 6.1 6.1 90.9
>5 30 15.2 15.2 29.4
Prefer not to answer 3 1.5 1.5 100
Did not answer 28 14.2 14.2 14.2
Total 197 100 100
Thinking that “maybe” the number of
siblings one has might correlate with GRIT,
we asked participants about this...
We’ll see if they do in later slides...
Number of Siblings
0
5
10
15
20
25
None
(only
child)
1 2 3 4 >5 Prefer not
to answer
Did not
answer
Number of Siblings (%)
Same information in graphical
format
Self-Rating of Resilience
This question asked participants
to rate themselves on RESILIENCE
using a 10-point scale (10 being
highly resilient).
No one reported a RESILIENCE
rating less than 4

8/8/2016
8
Self-Rating of Resilience Same information in graphical
format
2. How “reliable” is the new GRIT
assessment?
NOTE: Reliability(as measuredby Cronbach’s Alphahere)
indicatesthe degreeto which the itemsin each subscale(e.g.,
courage,resilience,and emotionaltoughness)are similarly
measuringthe abstract concept that we are callingGRIT. We
arelookingforanAlphavalueof0.7or above.
Factor 1: RESILIENCE
Reliabilities (Coefficient Alpha)
197 people started the survey
but 172 completed all items
in the survey
Alpha is slightly short of the
ideal (0.70) but not bad for
beta phase

8/8/2016
9
Factor 2: COURAGE
Few of the items from this
sub-factor will need to be
revised but still not bad
in the survey
Factor 3: EMOTIONAL TOUGHNESS
Excellent Alpha – especially
for first-round data collection
in the survey
3. Principle Components Analysis
Results
NOTE: Principle ComponentAnalysisis a data reduction
technique similarto factor analysiswherebyan optimal
number of dimensionsis extracted fromthe datasetconsisting
of 27 items(in our GRITassessment)

8/8/2016
10
Principle Component
Analysis Results
>1
Although the Scree Plot here
shows a total of 8-factors
above Eigenvalue of 1, it
appears that our GRIT
assessment can be reduced to
two factors (as shown by
leveling after the steep drop)
Principle Component
Analysis Results
Table to the right shows
what are referred to as
factor loadings.
Typically, factor loadings of
>0.4 are considered to be
sufficiently high enough to
add value.
If you count the total
number of factor loadings in
each column, you can see
that the # of items with a
loading of 0.4 or higher are
primarily in the first two
columns; confirming that
our GRIT assessment
contains two main factors
4. GRIT Score Analysis by
Demographic Variables
NOTE: Here, we are examiningoverallGRIT scores for different
segmentsof our participantsample– e.g., age,gender,
ethnicity,etc. Look for the MEAN score for each group. Higher
the MEAN, the more GRIT an individual(or group in this case) is
said to possess.
Important Caveat:
Results are based on pilot data with items that are still in
development-- we must be very careful to draw firm conclusions !!

8/8/2016
11
GRIT Scores by Gender
Do GRIT scores differ by
GENDER?
Do GRIT scores
differ by AGE?GRIT Scores by AGE
Do GRIT scores
differ by
ETHNICITY?
GRIT Scores by Ethnicity

8/8/2016
12
Do GRIT scores
differ by Level of
EDUCATION?
GRIT Scores by Education
Do GRIT scores
differ by income?
GRIT Scores by Income
N Minimum Maximum Mean Std. Deviation
mn_GRIT
21 2.42 4.11 3.3948 .43286
<$50,000 mn_GRIT
36 2.15 4.35 3.5288 .43401
$51,000 to $100,000 mn_GRIT
63 2.69 4.54 3.6032 .36598
$101,000 to $150,000 mn_GRIT
30 2.88 4.46 3.5782 .36829
$151,000 to $200,000 mn_GRIT
12 3.23 4.08 3.6058 .29617
>$200,000 mn_GRIT
9 3.42 4.08 3.7222 .23535
Prefer not to answer mn_GRIT
20 2.58 4.27 3.5231 .42214
Descriptive Statistics
SES
GRIT Scores by Spirituality
Do GRIT scores
differ by
Spirituality?

8/8/2016
13
Do GRIT scores
differ by Number
of SIBLINGS?
GRIT Scores by # of Siblings
N Minimum Maximum Mean Std. Deviation
mn_GRIT 22 2.42 4.11 3.3594 .45394
None (only child) mn_GRIT
15 2.96 4.31 3.5462 .38577
1 mn_GRIT 35 2.69 4.35 3.6407 .38799
2 mn_GRIT 43 2.15 4.54 3.6190 .38588
3 mn_GRIT 31 2.58 4.46 3.6166 .36454
4 mn_GRIT 12 2.73 3.88 3.4744 .33783
>5 mn_GRIT 30 2.88 4.35 3.5269 .34551
Prefer not to answer mn_GRIT
3 3.08 3.85 3.3846 .40704
Descriptive Statistics
siblings
Do respondents’ GRIT scores
correspond to their self-
reported resilience
perceptions?
GRIT Scores correlated with Self-Ratings of
Resilience
Answer: Based on
the strength of
the correlation
(~.46), the answer
appears to be YES!
GRIT
Resilience
Courage
Emotional
Toughness
.87**
.82**
.89**
** = statistically significant at p<.01
* = statistically significant at p<.05
Correlations among GRIT & Subfactors

8/8/2016
14
5. GRIT Score Compared to
Validation Measures
Validation Measures
In order to verify that we are actually measuring the
concept that we are calling GRIT, the following
validation measures (shown on the next slide) were
used. Put differently, we used these external
measures to see if our GRIT assessment correlates
with each in the anticipated direction (i.e., either
positive or negative correlation)
Validation Measures (variable names marked in RED)
 BigFive – Neuroticism [B5-Neuro] – characterized by sadness, moodiness,
and emotional instability.
 BigFive – Openness [B5-Open] – characteristics such as imagination and
insight
 Ambition Scale [Ambition] – characterized by a strong desire to seek
achievement in various settings in both professional and personal settings
 GRIT Scale [v_GRIT] (This trait is characterized by courage, resolve, and
strength). A subset of Duckworth’s 2007 scale was used.
 Learning Orientation [Learning] – desire to constantly learn and improve
one’s knowledge/skills/ability
 Self-Control/Regulation [Self Control] – degree to which one is able to
control his/her emotions/behavior
 Generalized Self-Efficacy [Gen_SE] – perceived level of general confidence

8/8/2016
15
Correlations among GRIT & Validation Scales
Ambition B5-Neuro B5-Open V_GRIT Gen_SE Learning Self Control
GRIT
Resilience
Courage
Emotional
Toughness
.87**
.82**
.89**
** = statistically significant at p<.01
* = statistically significant at p<.05
Ambition
B5-Neurotic
B5-Open
vGRIT
G Self-Efficacy
Learning
Self-Control
Self-Report GRIT
.32**
-.68**
.36**
.28**
.68**
.51**
.32**
.46**
Correlations among GRIT & Validation Scales
Variance Analysis Results
GRIT (conceptual space)
General Self-Efficacy
50%
B5-Neurotic
(10%)
Ambition (5%)
Learning (2%)
NOTE: Above illustration shows that about 67% of the GRIT concept is captured by
General Efficacy, Neuroticism, Ambition, and Learning
GRIT

8/8/2016
16
This project was carried out by students enrolled in PSYC344 –
Tests and Assessments (Summer, 2016) course – offered at the
Golden Gate University. As a result of posting the link to the
assessment on Facebook and various other social media sites, we
obtained a total of 171 responses.
Overall analysis shows that our GRIT assessment is close to being
sufficiently reliable (near .70 Alpha). Our principle component
analysis shows that the 27 items comprise two, rather than three
factors. Finally, correlations among GRIT and the various validation
items appear to be consistent with our expectations; indicating
that we are indeed measuring a concept that represents the
“degree to which a person “sees things through” (i.e., persists)
regardless of the challenges / obstacles to its resolution.”
GRIT Results Summary
That’s about it...

ggu2016summer grit assessment prelim report

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to ggu2016summer grit assessment prelim report

Similar to ggu2016summer grit assessment prelim report (20)

Recently uploaded

Recently uploaded (20)

ggu2016summer grit assessment prelim report