Quantitative reasoning courses have the potential to offer students skills, viewpoints, and critical thinking tools that retain their value throughout one’s entire life. The specific examples used in QR courses are supposed to lead the student to acquire skills and attitudes that are useful across many subjects, topics, and situations. If we could guarantee that students remember exactly five things about quantitative reasoning after they leave college, what should they be? And if we knew they would remember at most these five things, how would that change how we teach?

1. Five Things Worth Remembering
D. Wallace
NECQL 2015
Yale University

2. An early observation
• “The existence of schools is based on the belief that
students remember something of what they learn.”
• Tests across 4 and 11 month retention periods
• 75% and 70% retention, with worst retention on multiple
choice items.
• The authors point out this is WAY better than other studies
indicate.
• “Verbatim recognition, comprehension, and mental skills
tasks were retained better . . . than recall tasks.”
• Semb, George B., John A. Ellis, and John Araujo. "Long-term memory for
knowledge learned in school." Journal of Educational Psychology 85.2 (1993):
305.

3. How do we remember anything?
• Cognitive scientists Dan Willingham and
Robert Bjork, describing how memory works
• Taken from helpful sources:
https://pragmaticreform.wordpress.com/2013/11/16
/memory/, http://www.aft.org/periodical/american-
educator/summer-2003/ask-cognitive-scientist

4. Willingham:
• “your memory system lays its bets: if you think
about something carefully (and repeatedly),
you’ll probably have to think about it again, so
it should be stored. If you don’t think about
something very much, then you probably
won’t want to think about it again, so it need
not be stored.”

5. So we forget stuff because of insufficient
• Focus
• Time spent on it
• Attention spent
• Practice
• Usage
• Revisiting
• Consolidation
• Application

6. How to learn a fiddle tune:
anecdotal confirmation of Willingham’s observations
• 16 bars, maybe 200 notes, maybe 2 minutes long
• 2 hours in a day and it’s forgotten in a week: You
might learn to play it really well but you then
forget it!
• 5 minutes a day for a week and it’s there for a
month
• 5 minutes a week and it’s there for a few months
• 5 minutes a month and it’s there for a few years
• Once or twice a year and it’s there for life

7. Learning a fiddle tune permanently requires:
• Focus --- at the start especially
• Time spent on it --- more at first
• Attention spent --- can’t do anything else at the
same time!
• Practice --- until you have it
• Usage --- consistently thereafter for a while
• Revisiting --- at regular intervals
• Consolidation --- it improves your other tunes
also
• Application --- in this example there is no
distinction between usage and application

8. Bjork, illustrated by Kirby:

9. Crammed
= low storage
strength,
but high retrieval
strength
Forgotten
= low storage
strength, low
retrieval strength
Buried
= high storage
strength, low
retrieval strength
Mastered
= high storage
strength, high
retrieval strength
Through
additional
use
Through lack
of use
Through
lack
of use
Relearning
quickly

10. Willingham on pedagogy:
• Distributing practice: “It is virtually impossible to become
proficient at any mental task without extended,
dedicated practice distributed over time.”
• Overlearning: Keep pupils learning after they “know” the
material to prevent forgetting. “A good rule of thumb is
to put in another 20% of the time it to to master the
material.”
• Testing frequently: Testing exercises the recall function,
improving retrieval strength.
. . . worth remembering . . .

11. Bjork on pedagogy:
• Spacing (rather than massing) practice: Information that is
presented repeatedly over spaced intervals is learned
much better than information learned in a block.
• Interleaving: People actually learn content better when it is
interleaved with other content (although they don’t think
so).
• Testing: When information is successfully retrieved from
memory, its representation in memory is changed such that
it becomes more recallable in the future (Bjork, 1975); and
this improvement is often greater than the benefit resulting
from additional study (Roediger & Karpicke, 2006).
When you are studying, you aren’t recalling!

12. A useful metaphor
Wallace, Dorothy (2011) "Parts of the Whole : Cognition,Schemas, and Quantitative Reasoning,"
Numeracy: Vol. 4: Iss. 1, Article 9.

13. stuff to consider:
• An idea or concept can be understood at different
levels, or schemas. What level do we care about for
a given concept?
• We can trade a more sophisticated understanding for
time to reinforce a simpler approach.
• Concepts are built out of examples (Skemp, The
Psychology of Learning Mathematics)
• Wallace’s rule of thumb: it takes at least 5 examples
to solidify a new concept.

14. And what about the examples chosen?
• Things students will have the need and
opportunity to use throughout life are more likely
to be remembered because they must be
recalled regularly.
• For example (from the table of contents of a
popular QR text):
– Sets and Venn diagrams
– The power of compounding
– The law of large numbers
– Correlation and causality
– Fractal geometry

15. Things worth remembering:
If important things worth remembering have to be taught
through
repeated practice,
overlearning, and
spacing out
over a semester, that means that there cannot be too many
important things worth remembering in one single course
offering.
Five is just a number. But it does divide neatly into 10 or 15 (week
quarters or semesters). What if we planned to spend 3 entire weeks
on something important, spaced out over a semester?

16. What five things do we want students
to remember?
• Each of us could make up a list
• Could we agree on a list?
• Could a department or institution agree on a
list?

17. My five things
• The value of a rough estimate, and how to
make one
• Estimating errors and how they propagate
• Data can be presented graphically to make a
point– or lie
• Weasel words and why they weasel, or “units
matter”
• Growth and decline versus relative growth and
decline, and its effect on our lives

18. The value of a rough estimate,
and how to make one
• Five examples:
– Estimating an amount needed
– Personal budget
– Business budget
– Time estimate
– Scientific claim estimate
But you wouldn’t do them all as a unit. You would want
them distributed over time, presented repeatedly over
spaced intervals, and interleaved with other content.

19. Estimating an amount needed: geometry
Potential sources of error:
measurement error, windows and doors, coverage claimed by manufacturer
Error estimate for project? Initiates second worthy thing
Data 1: graphical display of error estimate versus estimated amount needed
Data 2: graphical display of percent error versus estimated amount needed
Leads to third worthy thing

20. Deconstructing that example
• QR/QL goals
– Estimation
– Error estimation/calculation
– Graphical displays of data
• Pedagogical goals
– Interleaving of topics

21. Personal budget:
Notoriously difficult to estimate! But probably one of the most important things we have to
estimate regularly in life.
Possible to measure by keeping records for several months
Leading to an error estimate and possibly lifestyle changes
AND a discussion of exceeding a budget, and credit,
which brings up another worthy thing.

22. Growth and decline versus relative growth and
decline, and its effect on our lives
• Seven examples:
– How money grows in a piggy bank
– How money grows in a bank account
– How money grows in a mutual fund
– How debt accrues on a credit card balance
– Population growth or decline
– Decline of value of money over time
– Half life
But you wouldn’t do them all as a unit. You would want
them distributed over time, presented repeatedly over
spaced intervals, and interleaved with other content.

23. . . . a discussion of exceeding a budget, and credit,
which brings up the way money grows or declines with
time. $5 per year versus 3% per year.

24. ---- distributed over time, presented repeatedly over
spaced intervals, and interleaved with other content.
• Estimate how much you would have in 30 years by either
method. (It looks like a calculation but it’s really an estimation
because stuff happens.)
• What do you think your budget will do during that period?
How do you estimate it will change?
• What if the amount added, or percent growth rate, varied?
Could you get error bounds on your first estimate?
• What if you contributed 5 each year AND got 3% interest?
What do you estimate then?
• What happens to the error margin then?
• Graphical displays of various kinds of growth
All of these can be approached with calculator or
spreadsheet, before a formal discussion of exponential
growth, etc.

25. Deconstructing that example
• QR/QL goals
– Estimation
– Error estimation/calculation
– Graphical displays of data
– Growth and decline over time
• Pedagogical goals
– Interleaving of topics
– Revisiting and reinforcing earlier topics
– Spacing out discussions of estimation

26. Next example: the business budget

27. Do we need all five worthy things to
approach this problem?
• A rough estimate,
– the actual budget
• Estimating errors,
– online “shopping”
• Data,
– Likely usage rates
• Weasel words,
– online “shopping”
• Growth and decline of money,
– business loan
You can probably think of a dozen more ways to do this.

28. Sample budget,
numbers could be removed of course.

29. Error estimation. Again? YES.
• “shopping” for an indoor slide, thanks to
Amazon
• $89.95, $43.95, $130.88, $274.95, $99.99,
$109.99, $79.99, $98,99, $314.99, etc.
• $4819.99, $4737.99, $4218.99 ??? Shipping?
Commercial indoor slide.

30. Growth and decline:
the business loan
• Note the high price tag of the budget.
• How loans work, interest and compounding.
• Possibly exponential growth.
• Loan schedules and how they work. Maybe a
spreadsheet.
• Depreciation of equipment: the decline of
value over time.

31. Data:
• To justify a business loan, you must have some
evidence that your business will bring in
enough money to cover the monthly payment
in addition to your expenses.
• What kind of survey could be constructed to
estimate this?
• What other data could be collected (from, say,
similar businesses) to justify a loan?
• How would you present that data to the bank,
and why?

32. Weasel words abound when shopping
for anything, so there is an opening for
that topic also.

33. Deconstructing that example
• QR/QL goals
– Estimation
– Error estimation/calculation
– Graphical displays of data
– Growth and decline over time
– Weasel words
• Pedagogical goals
– Interleaving of topics
– Revisiting and reinforcing earlier topics
– Spacing out discussions of estimation, error estimates,
and growth
– Accumulation of various displays of data

34. A hard example.

35. Should we believe it?
• How could you estimate each of those
quantities? (the perfect jigsaw)
• What kind of errors could you expect?
• How does population change over time, and
which parts of the picture does this affect?
• Could we estimate change in these quantities
over time as a result?
• How many ways can we think of displaying the
data?
• How might one manipulate the opinion of the
reader, based on how the data is presented?
• Weasel words: more, less, some, and how they
mislead.

36. Weasel words in this case.
• “Of course, more people would result in more
fossil fuels being burned, but that is only a
fraction of the carbon entering the
atmosphere.”
• “More people would greatly increase the
contribution of fossil fuels!”
How much more? How big a fraction? How big
an increase? Technical details look boring.
Weasel words are rarely boring.
A new genre:
Weasel punctuation.

37. Five things we might want students to
remember 10 years after the course is over:
• The value of a rough estimate, and how to
make one
• Estimating errors and how they propagate
• Data can be presented graphically to make a
point– or lie
• Weasel words and why they weasel, or “units
matter”
• Growth and decline versus relative growth and
decline, and its effect on our lives

38. A hundred things we don’t really care about:
• How much paint you need to paint a wall.
• Exactly how much carbon enters the
atmosphere of Europe per year.
• How much an indoor playground might cost.
• Etc.

39. Five things we might want to remember
ourselves while teaching the course:
• Distributing, or spacing practice over spaced
intervals improves memory storage and recall
• Overlearning until mastery improves memory
storage
• Testing frequently improves recall ability
• People actually learn content better when it is
interleaved with other content
• It takes at least 5 examples to solidify a
concept.

40. Notice that I have sneakily talked about 10
things worth remembering.
Some people like to remember in pictures.
Here are two pictures worth remembering.

41. Bjork, illustrated by Kirby:

42. A useful metaphor
Wallace, Dorothy (2011) "Parts of the Whole : Cognition,Schemas, and Quantitative Reasoning,"
Numeracy: Vol. 4: Iss. 1, Article 9.

43. Ok, 12 things.
Thank you very much for inviting me!