Numeracy is often referred to as the habit of mind to see real-world problems through a quantitative lens. Typical QR programs in higher education aim to build this habit in students by the time they graduate — but to what extent do students already have this habit when they arrive? I will briefly review some of what is known about incoming freshmen’s quantitative inclinations, and describe an ongoing project to measure their quantitative habit of mind in a somewhat subversive fashion.
How “Quantitatively Inclined” are Incoming College Freshmen? by Dr. Matthew Salomone
1. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
How “Quantitatively Inclined” are Incoming Freshmen?
Northeast Consortium for Quantitative Literacy XVIII
Yale University — May 9, 2015
Matt Salomone
Assistant Professor, Mathematics Department
Director, Math Services
Coordinator, Quantity Across the Curriculum
Bridgewater State University
Bridgewater, MA 023251
1This work is supported by a Summer Grant from the Center for the Advancement of Research and
Scholarship at BSU.
4. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Do Freshmen “Want to Do the Dishes?”
How “Quantitatively Inclined” are Incoming Freshmen?
To what extent does a quantitative habit of mind exist among incoming
freshman students?
Use their extemporaneous persuasive writing to correlate:
1 Engagement & success with quant evidence vs. Student characteristics
2 vs. Writing readiness score
3 vs. Retention into second year
5. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Do Freshmen “Want to Do the Dishes?”
How “Quantitatively Inclined” are Incoming Freshmen?
To what extent does a quantitative habit of mind exist among incoming
freshman students?
Use their extemporaneous persuasive writing to correlate:
1 Engagement & success with quant evidence vs. Student characteristics
2 vs. Writing readiness score
3 vs. Retention into second year
Frame & Features of Assessment
1 Holistic scoring for extent & quality of QR (Modified QuIRK)
2 “State of nature” measurement (Not a “QR task”)
3 Potential new proxies for early math/QR course success?
6. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Verbal Skill = Key Enabler of Quantitative Reasoning
Reading Comprehension
High cognitive reading load impairs QR success (Gillmor et al. 2015)
Unusually high-quality attention required to avoid bias (Kahneman 2011)
Basic logical/statistical vocabulary often missing (Bennett, NNN list)
7. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Verbal Skill = Key Enabler of Quantitative Reasoning
Reading Comprehension
High cognitive reading load impairs QR success (Gillmor et al. 2015)
Unusually high-quality attention required to avoid bias (Kahneman 2011)
Basic logical/statistical vocabulary often missing (Bennett, NNN list)
Writing Fluency
Writing at core of many QR assessment rubrics (LEAP &c.)
Difficulty expressing simple Q ideas with precision (Schield 2008)
Students frequently miss opportunities to use QR (Miller 2004,
Rutz & Grawe 2009)
8. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Verbal Skill = Key Enabler of Quantitative Reasoning
Reading Comprehension
High cognitive reading load impairs QR success (Gillmor et al. 2015)
Unusually high-quality attention required to avoid bias (Kahneman 2011)
Basic logical/statistical vocabulary often missing (Bennett, NNN list)
Writing Fluency
Writing at core of many QR assessment rubrics (LEAP &c.)
Difficulty expressing simple Q ideas with precision (Schield 2008)
Students frequently miss opportunities to use QR (Miller 2004,
Rutz & Grawe 2009)
Key idea: It is never enough to assess only computational skill — QR always
involves a trade of information both out of a context (reading) and back into a
context (writing).
9. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Facts on the Ground at Bridgewater State University
BSU is a comprehensive, Masters-large university focused on undergraduate
education and with a heritage as a teacher-preparatory normal school.
By the numbers
≈ 8500 undergraduates
60% female
60% are one or more:
1 Of color (15%)
2 1st generation (50%)
3 Pell eligible (25%)
Hum/SS
47%
Educa.on
27%
Business
15%
Sci/Math
11%
Freshman Class of 2013 N = 1436
Average SAT-Math: 501
Average Elem. Alg.: 78.5
Average SAT-Verb: 492
Average Read. Comp.: 85.5
Developmental math: 35.2%
Developmental writing: 14.4%
10. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Writing Placement – Process & Prompt
Locally developed exam feeds: (60 minutes)
ENGL 101 + ENGL 144 ENGL 101 ENGL 102
Writing Rhetorically (3 cr.) Writing Rhetorically (3 cr.) Writing Rhet. w Sources (3 cr.)
Academic Strategies (1 cr.)
Score 3 or SAT-V 500 Score 4 or SAT-V 600
195 students (14.4%) 1,082 students (75.3%) 150 students (10.4%)
11. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Writing Placement – Process & Prompt
Locally developed exam feeds: (60 minutes)
ENGL 101 + ENGL 144 ENGL 101 ENGL 102
Writing Rhetorically (3 cr.) Writing Rhetorically (3 cr.) Writing Rhet. w Sources (3 cr.)
Academic Strategies (1 cr.)
Score 3 or SAT-V 500 Score 4 or SAT-V 600
195 students (14.4%) 1,082 students (75.3%) 150 students (10.4%)
Essay is rhetorical in nature — students must construct an argument based on
≈ 1 page of evidence. Beginning 2012, some evidence is quantitative.
12. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Writing Placement – Process & Prompt
Locally developed exam feeds: (60 minutes)
ENGL 101 + ENGL 144 ENGL 101 ENGL 102
Writing Rhetorically (3 cr.) Writing Rhetorically (3 cr.) Writing Rhet. w Sources (3 cr.)
Academic Strategies (1 cr.)
Score 3 or SAT-V 500 Score 4 or SAT-V 600
195 students (14.4%) 1,082 students (75.3%) 150 students (10.4%)
Essay is rhetorical in nature — students must construct an argument based on
≈ 1 page of evidence. Beginning 2012, some evidence is quantitative.
Significant differences in math skill vs. writing placement
0
20
40
60
80
100
120
140
TENGL
ENGL101
ENGL102
Accuplacer
Elementary
Algebra
Score
200
300
400
500
600
700
800
TENGL
ENGL101
ENGL102
SAT-‐Mathema3cs
Score
13. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
The Road Ahead: Assessing Responses for Extent & Quality
Need to re-assess writing samples (n = 120) for QR extent and quality.
Modified QuIRK rubric Two scorers per sample Sum of scores
Relevance Extent of Use Quality of Use
None No potential use of
numbers, or misc. use only
1 No explicit numbers/QR used 1 Misses opportunity for QR
completely, even for back-
ground/context
Peripheral Potential
uses of numbers to
provide useful de-
tail/background/reference
2 1–2 instances of explicit numeri-
cal evidence / QR
2 Uses numerical evidence incon-
sistently or non-contextually
Central Potential uses of
numbers to address central
theme
3 Explicit QR used throughout ar-
gument
3 Uses numerical evidence consis-
tently and in context, but some op-
portunities are missed
4 Uses numerical evidence
throughout, effectively, insightfully,
and thoroughly
14. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Ways of “Closing the Loop” with this Study
If QR engagement (extent) is ⊕ correlated with writing placement
QR/writing workshops with high school partners could reduce developmental
writing placements & forward-place more students into ENGL 102
15. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Ways of “Closing the Loop” with this Study
If QR engagement (extent) is ⊕ correlated with writing placement
QR/writing workshops with high school partners could reduce developmental
writing placements & forward-place more students into ENGL 102
If extent & quality varies significantly across student characteristics
Correlate longitudinally with extent & quality at capstone to look for “value
added”; offer focused faculty development for instructors of student groups
who are least engaged/successful
16. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Ways of “Closing the Loop” with this Study
If QR engagement (extent) is ⊕ correlated with writing placement
QR/writing workshops with high school partners could reduce developmental
writing placements & forward-place more students into ENGL 102
If extent & quality varies significantly across student characteristics
Correlate longitudinally with extent & quality at capstone to look for “value
added”; offer focused faculty development for instructors of student groups
who are least engaged/successful
If extent & quality is ⊕ correlated with 1st-2nd year retention
(Controlling for other known predictors of retention) Targeted interventions,
esp. in first-year writing courses, to increase QR engagement could have
positive retention benefits
17. How
“Quantitatively
Inclined” are
Incoming
Freshmen?
Matt Salomone
The Motivation
QR & Verbal Skills
Context @ BSU
Writing Placement
Assessment Ahead
Ways of “Closing the Loop” with this Study
If QR engagement (extent) is ⊕ correlated with writing placement
QR/writing workshops with high school partners could reduce developmental
writing placements & forward-place more students into ENGL 102
If extent & quality varies significantly across student characteristics
Correlate longitudinally with extent & quality at capstone to look for “value
added”; offer focused faculty development for instructors of student groups
who are least engaged/successful
If extent & quality is ⊕ correlated with 1st-2nd year retention
(Controlling for other known predictors of retention) Targeted interventions,
esp. in first-year writing courses, to increase QR engagement could have
positive retention benefits
We hope to have initial results by the end of summer & will be happy to share!