Learning Trajectory-Aligned Diagnostic Assessments for Early Algebra, Grades 6-8

Scaling Up Digital Design Studies | NC State University | College of Education
Seminar on formative assessment tasks in mathematics
January 5-7, 2016
Weizmann Institute of Science
The Challenge of Building and Implementing
Learning Trajectory-Aligned Diagnostic
Assessments for Early Algebra, Grades 6-8
Jere Confrey
Joseph D. Moore Distinguished University Professor
North Carolina State University
Research funded by:

The Team
Jere Confrey, Project Director
Ryan Seth Jones, Learning Scientist
Garron Gianopulos, Psychometrician
Basia Coulter, Visual Designer
Yungjae Kim, Lead Software Engineer
Pedro Larios, Software Engineer
Doug Ivers, Software Engineer
Meetal Shah, Graduate Student

Scaling Up Digital Design Studies:
• How we work
– Research-based
– Rapid Prototyping
– Agile Methods
– Partnerships with
Teachers and Students
– Design Research

Success of the Future
• A Stanford-hosted panel of highly successful, scientific technology
entrepreneurs--Astro Teller, inventor (Google), Christina Smolke,
bioengineer (Stanford), and Steve Jurvetson, venture capitalist (DFJ
General Partner), reflected on what learning is essential for success in
tomorrow’s technology world. They agreed on the importance of problem
selection, successful work in interdisciplinary teams, outstanding
communication skills, and the ability to learn quickly. They also called for
learners who are able to reconstruct knowledge from first principles and to
pivot as needed to adjust to new possibilities and options.
• In a nutshell, Teller advised students about the nature of their future
pursuits saying, “…if you plan for static stability, you are going to be really
frustrated. But if you build the skills of dynamic stability, it is going to be
awesome…”
• Jurvetson summarized their views of learning up saying “Iterative learning,
that is, focusing on the process learning that correlates with success, will be
the increasing locus of learning.”
(http://ecorner.stanford.edu/authorMaterialInfo.html?mid=3554)

How to leverage
student thinking to
strengthen
instruction?
The
Challenge

The SUDDS approach
• Create a digital system that:
– Represents the big ideas in a learning map
– Specifies the landmarks and obstacles in learning big ideas over
time and across grades based on research on learning
trajectories
– Periodically diagnoses student progress accurately in real time
– Allows teachers to address key conceptions based on student
thinking patterns
– Allows teachers to flexibly create fluid groups, based on student
learning profiles

• A learning trajectory connects what students bring to
instruction, to a target concept, and delineates a set of
landmarks and obstacles that students are likely to
encounter as they move from naïve to sophisticated
understandings.
A Key Concept: A Learning Trajectory

• Insert corridor picture
A Learning Trajectory Depicted

Read the following problem, and predict how students will solve it.

Score of “0” on rubric—
• Circles “Bigger” or “Smaller”,
• Indicates a belief that to be equal, parts must be congruent, or
• No discernible or intelligible distinctions
A score of “1” indicates that they make their prediction based on “qualitative
compensation”--- one piece is wider and the other is taller. (no example)

transitivity argument: shares from
both splits are still ½ of the same
size brownie)…
Score of “2” on rubric:
Student demonstrates that results of the two strategies are equivalent:
…or shows equivalence by
compensation or decomposition

PEEQ: Property of Equality of
Equipartitioning
• If two congruent shapes are each split for the same
number of persons, then the size of each share from
one of the shapes is equal to the size of each share
from the other shape, regardless of the shape of the
shares.

Equipartitioning a Rectangle for 4 people
• Discuss the following solution to the task “Share a rectangle
(i.e. rectangular cake) for 4 people.” Did each person receive
a fair share?
• How do you know?—Justify your answer.

Learning Trajectories
• Are not simply logical deconstructions of a math concept
• Surface as a result of use of appropriate tasks, tools,
and forms of discourse in classrooms
• Not just erroneous, but have roots of productive thinking
• Can be transformative, leading to new possible insights
• Sometimes have to be unearthed, sensed, and pursued

Epistemological Objects of LTs
• What is observed or heard has a meaning to the child as
a means to make sense of experience, and is a form of
knowledge claim; therefore from the child’s perspective,
it is an epistemological object. It permits the observer to
imagine how the child is interpreting the problem and
what follows from that.

• creating a need for the idea
• connections to prior or related ideas
• misconceptions and alternative conceptions
• student built representations and coordinating
representations
• mental models
• strategies
• sets of cases
• generalizations and formalizations
Types of Epistemological Objects of
LTs

What is the difference between this…?

…and this?

Leveraging LTs
• What structure is needed for digital curricula?
• Can we use learning trajectories to create such a
structure?

What is the SUDDS 6-8 Learning Map?
● A Learning Map is a navigational system that helps
students to explore the content to be learned organized
in a learner-centered way.

● Anticipate and participate in what they are going to learn
● Experience personalization, not isolation
● Find and use digital resources coherently
● Receive and use diagnostic feedback and guidance
● Form flexible, just-in-time groups
With a learning map,
students can:

Map Structure
Fields (4)
Regions (9)
Related
Standards
Related Learning Clusters (24)
Constructs (65)
Underlying Learning Trajectories and Indicator Levels

The Related Learning Clusters within
the big ideas

Regions and Clusters in Algebra
Representing Expressions,
Equations, and Inequalities
Evaluating Equations with
2 Variables and Exponents
Exploring Relations
and Functions
Representing and Using
Linear Functions

Representing Expressions, Equations,
and Inequalities

A Related Learning Cluster (RLC)
Representing Expressions, Equations, and
Inequalities
The associated
U.S. Standards
The proficiency levels of the
learning trajectory, listed
from bottom to top
Four constructs

• Access to the map: www.sudds.co
• Make an account

What is a Digital Learning System?
Internet Links
Confrey, November, 2015

Digital Learning Systems
• Not just components
• Relationship among components
• Relationship among users (students, teachers, parents,
administrators, researchers)
• Prediction (planning forward: anticipation)
• Feedback (results to different users for different
purposes )
• Analytics and experimentation

Purpose of Diagnostics
• To identify in students and classes the patterns of
performance that are known from research to block
progress
• To place results in a taxonomy to ease interpretation
• To make actionable information available in real time
• To guide subsequent instructional decisions
• To support formative practices

Taking a Diagnostic Assessment
Students take a 20-
minute diagnostic
test to explore the
ideas from the
cluster and
demonstrate their
understanding.

Results for Algebra Field Testing
• Describe the field-testing setting
• Describe the kinds of items used for algebra
• Show results for each of the four constructs in the select
RLC
• Draw some conclusions and implications

Field-Testing 2015-16
• Site: High-performing district in New Jersey
• Two middle schools, each with ~700 students
• These data are from 7th graders (13-14 year olds)
• Data are collected near the end of instruction on units
• Results are used formatively only
• Current forms of items limited to multiple choice, numeric
and select all.
• (this work is in its infancy→we welcome suggestions)

Epistemological objects in Early Algebra
Epistemological objects
of Learning Trajectories
Examples from the first Algebra RLC
misconceptions 2(x + 3)= 2x + 3; (x + 3)/(x + 2) = 3/2
Representations and their
coordination
Number lines, set notation, intervals, expressions, equations, drawn
images; coordinating equations and number line and figural images
models Solving equations as “pan balances”; variables as unknowns.
strategies Guess and check, adjust; substitution
cases
x + a = c,
bx = c
ax + b = c
a(x + b) − c
ax + b = cx
a(x + b) + c = dx + e, etc.
outliers
-x = a; 1/x = b; -x < -b; a − (x−b) = c;
b – x = c
generalizations Initial amount + (rate times number of stages) = total
formalizations Term, coefficient, range, all the properties

Results from “Describing Patterns and Relations…”
Describing Patterns and
Relations Using Algebraic
Expressions (C53)
Results
L1: Describes simple patterns
(relation or function) verbally
or with pictures or diagrams
(g.6)
67% matched a verbal description using x with the correct pattern
relating number of tables and people.
L2: Describes simple patterns
(relation of function) using x as
a variable (g.6)
A simple growing L was shown.
94% choose the correct algebraic expression.
L3: Associates coefficients,
factors, and terms in
interpreting verbal or pattern-
based descriptions of
expressions with conditions
(g.6)
86% mistakenly said 5 + 3x + 2 + x/3 had six terms.
46% correctly identified 3 as the coefficient of the second term
while 67% also agreed that it was 3x, revealing some confusion.
For coefficient of x/3, 46% incorrectly called it 3, while 29%
correctly called it 1/3.
L4: Describes how coefficients,
factors, and terms vary with
the conditions in the situation
(g.6)
For a given algebraic description of a growing pattern
(arrangements of tables and chairs), 65% selected new expression
that added chairs in certain positions. (They were also provided a
figure of the new patterns, so could have figured it out from
those.)

• 67% of students correctly found the verbal description associated
with the pattern (L1). Cognitive challenge: extend pattern or explain
rate of change. One more table adds two more seats.

Results from “Finding Equivalent Expressions, etc.”
Finding Equivalent
Expressions and
Substituting Values (C54)
Data Summary
L1: Interprets situations
algebraically with simple
expressions without
parentheses.
Overall the students could translate simple expressions into
algebra, but performance dropped to ~80% correct for
subtraction (3 less than x) and division (the quotient of x and 4).
L2: Realizes that different
forms of simple expressions,
including those with
parentheses, may produce
equivalent results.
80% correct: recognize equivalence to x only for simple
expressions (x × 1) + 0;
Performance drops to 60-70% correct for slightly more
complicated expressions ( x + 7) − 7 and 4x/4.
One third incorrectly chose 2(2x) − 4x as equal to x.
L3: Substitutes for x and
evaluates, applying order of
operations on simple
expressions, including those
with parentheses.
Perimeter problem with two stated constraints, one using
expressions:
77% of the students were able to select the one correct solution.
55% of students also identified a compelling distractor meeting
only the first condition.
For two other distractors meeting only the second constraint,
each was selected by about 15% of the students.

Finding Equivalent
Expressions and
Data Summary
algebraically or substitutes
values systematically to test
whether complex expressions
are equivalent, including:
combining like terms and
distributing positive factors into
parentheses.
For the expression 2(x + 1.50):
73% choose the correct match;
13% chose the wrong match indicating a failure to understand the
meaning of the coefficient 2.
algebraically, or simplifies and
evaluates complex expressions,
including: combining like terms
and distributing positive factors.
In a matching exercise at this level, students were asked to match a
complicated algebraic expression with its written description.
14% got three correct;
individually on those three, 47%, 60% and 34% got each individual
one correct.
In a similar problem, 72% identified the correct expression if two
parts were expressed separately;
Only 40% saw the equivalence of other expressions.
In two items linked to perimeter, performance feel to around 30%.

Finding Equivalent
Expressions and
Data Summary
algebraically, or simplifies and
evaluates complex expressions,
including: combining like terms
and distributing positive and
negative factors.
Task: select all equivalent expressions from a list for -2(-3x − 5):
92% select correctly one answer of -6x − 10;
Only 67% also recognized the alternative -6x + -10.
19% chose 6x − 10.
These demonstrate some weakness in multiplying with negative
numbers within expressions.
Task: solve 8 − 2(3x − 1):
37% were correct;
41% choosing none of the above.
21% did not distribute or added the x’s incorrectly.

Finding Equivalent Expressions and Substituting Values: Sample item--
L3 Substitutes for x and evaluates, applying order of operations on
simple expressions, including those with a parentheses,
• A triangle has a perimeter of 7. Two of the sides have
integer lengths equal to x and x+1. Check all that could
be lengths of three sides of the triangle.
(55% of students selected this
incorrect choice-adds to 7)
(77% of students selected this correct
answer)
(14%of students selected this incorrect
choice – has consecutive values)
(15%of students selected this incorrect
choice- has consecutive values)
1) 1,1,5
1) 2,2,3
1) 1,3,4
1) 6,7,8

The Role of Context
• An interesting question for the LTs and items is when to
include context in the problems.
• Sometimes context facilitates; sometimes it adds
difficulty
• Our approach to this has varied. Sometimes we see it
as inherent to the LT (in patterns, in translating
expressions); sometimes we make it, its own level at a
complex stage to check for integration of ideas.
• Needs future experimentation to inform its proper role in
diagnostics.

Results from “…Equations in One Variable…”
Representing and Solving
Equations in One Variable
with Justification (C55)
Results
L1: Understands x stands for a
number that makes the equation
true.
Given the equation, 2(x + 4) = 3x, the, students were asked to choose
one of four answers that solved the equation.
80% answered the problem correctly;
11% believed there was no correct answer provided;
7% answered 4, which might imply a failure to distribute.
L2: Solves simple equations by
inspection and guess, check, and
adjust.
Students were asked to predict what number to try next.
98% responded by suggestion a number between the two given values.
For similar but more difficult equation, 3x-5 = 2 (x-5), students are told
the results for x = 12, x = 5 and x = 0, and each time the differences
between the two sides of the equality are smaller.
38% recommended correctly that the next number should be less than
zero;
19% recommended trying number between 0 and 5;
38% recommended trying number between 5 and 12.

Results
L3: Maintains balance of
equation in order to solve
equations of the form x + p = q
(p, q, and x are rational
numbers).
In the given problem, students are told the equation in the form of
x+1.5 = total time. They are asked how to find x, the number of
hours worked.
73% of them answered correctly to subtract 1.5;
19% suggested adding it;
5% wanted to divide by 1.5.
L4: Maintains balance of
equation in order to solve
equations of the form px = q (p,
q, and x are rational numbers).
Given the formula d × v = m, with values for volume (v) and mass
(m), students were asked to find density.
83% correctly knew to divide by volume, but
8% produced their answer by incorrectly multiplying the two
numbers.
Students were asked to convert the length of a marathon into
kilometers and round.
50% were correct;
33% more were within one kilometer suggesting they rounded
incorrectly;
7% incorrectly multiplied, getting a smaller number of kilometers.

Results
L5: Justifies solutions to simple
equations using inverses and
identity properties to maintain
balance.
For the problem 124/x = 4,
just under 50% could identify both steps of the solution.
Nearly 20% suggested dividing both sides by 124. (This approach could
work, producing 1/x = 1/31.)
At this level, 65% of students thought adding the same amount to both
sides was justified as an additive inverse rather than an additive axiom
of equality
L6: Uses strategies to isolate x on
one side of equation in two or
more operations and maintains
balance to solve.
Students were asked to solve for an unknown using a representation of
a pan balance. When given unknowns on one side and a
representation of the equation (3x+ 6 = 12),
94% got them correct.
When given the figures only and a problem representing 3 x+ 7= 5x+ 1,
the percentage correct dropped to 84%.
L7: Describes situations using
linear equations and solves.
Students were given an isosceles triangle with a base of n and told the
other two sides were congruent and one was labeled 3n+4 equal to 28:
58% solved it correctly;
32% said there was not enough information, perhaps indicating a
failure to use the information about congruent sides.

Results
L8: Justifies method for solving
equations with x on one side only
using properties (inverse,
identity, distributivity,
commutativity, and axioms of
equality).
The students were asked to order the steps to solve
5 − 5x +2(7 − x) + 9 = 0:
87% got all 4 correct.
L9: Uses strategies to eliminate
x's on one side of equation,
maintains balance to solve, and
justifies.
Students were given 6w = -2w + 24:
89% choose adding 2w to both sides correctly and 91% identified the
correct solution.
Students were also given the equation x+36+ x3=x+74 ,
and asked to choose productive first steps from a set of options.
36% multiplied each term by a name for one (2/2, 4/4 and 3/3) to
produce a common denominator.
43% multiplied both sides by 12.
12% recognized either as a correct option.
10% chose only multiplying by 12, which is defensible as a sole answer.
35%% of the students incorrectly selected subtracting 7 from both
sides.
19% wanted to divide by x.
21% wanted to subtract 3.

Results
L10: Interprets meaning of one
solution, multiple solutions, and
no solution for equations with x's
on both sides.
Students were given the problem 12x −15 = 3(4x − 5) and are told it
has an infinite number of solutions. They are given other equations
which differ in one value. They are asked which of the others will have
no solution. The two correct answers change either the value 15 or 5.
73% recognized the effect of changing the 5;
55% saw the change of the 15;
Only 7% got them both.
On the second item at this level, students were shown a solution that
results in a true statement and therefore infinitely many solutions.
66% picked infinitely many.
24 said there was no solution.
19% thought that the expression 6 = 6 implied that 6 was the
solution.
14% said “rework the problem” because there must have been a
mistake.

Representing and Solving Equations in One Variable:
L10: Interprets meaning of one solution, multiple solutions, and no
solution for equations with x's on both sides
L7 Describes situations using linear equations and solves
Student responses:
• 66% correctly chose
option 4, but only 44%
only chose option 4.
24% of the students said there were no solutions
19% thought that the answer was 6,
14% incorrectly said a mistake was made. and the
student should start over.

Results from “…Inequalities in One Variable…”
Inequalities in One Variable
Results
L1: Understands that x< a means
that x can be equal to a whole
range of numbers.
Students were asked to match a set of descriptions of inequalities to
their symbolic display.
Only 67% of students got all of these correct.
92% correctly matched “50 less than a number is no more than 7”;
Only 70% correctly matched “ 7 more than a number is no more
than 50” to its symbolic representation.
“50 is less than or equal to a number plus 7” was a strong distractor
for 14% of the students on this item.

Results
L2: Identifies contexts in which
inequalities apply.
Students were asked to identify which of a set of solutions that
completely represent x ≤ 8. The solutions included two correct
answers:
32% chose one that used the infinity symbol;
68% chose a graph of a number line.
The incorrect solutions included a variety discrete set of values ≤ 8,
chosen by 60%, 31%, and 25% of the students.
8% misread the inequality sign choosing values greater than 8.
In the second item, students were asked to identify which solution
sets contain values that make an inequality true:
79% chose the complete answer as one option.
Shown four correct discrete-valued options,
65% identified all correct integer values,
17% chose the graph with an open circle,
30% used the infinity sign,
60% chose 8 (which was on the boundary),
46% chose the single value of 3.5.
In a third example, the inequality 5 > x:
86% choose values correctly,
14% misread the direction of the relationship.

Results
L3: Solves simple inequalities by
inspection, guess, check, and
adjust to find a range of solutions.
Students were given |x| > 5, and were asked to select all numbers
that made the inequality true.
85% chose 6.
62% chose -6.
(23% more students chose 6 than -6)
58% choose 23/4.
Overall, only 42% got full credit.
Given 3x − 4 ≥ 8,
Only 42% chose the correct solution.
39% of incorrect responders chose x≤ 8.
L4: Maintains balance of equation
in order to solve Inequalities of
the form x + p = q (p, q, and x are
rational numbers).
No items at this level

Results
L5: Maintains balance of equation
in order to solve Inequalities of the
form px <> q (p, q, and x are
rational, including switching
inequality for negative multipliers).
Students were highly successful for b/3 ≥ 12:
96% correctly chose b>36, 91% selected b = 36. In inequalities, it may
be the case that this form is easier than the x+p.
For the item -y < 16, the problem requires students to multiply or
divide by -1 and know to reverse the sign, unless they figure out the
solution by trial and error.
Only 16% answered this item correctly;
63% answered incorrectly y < 16.
10% chose y > 16 (a partial solution but students have to understand
the item demands a complete solution.
11% choose y < -16.
L6: Justifies solutions to simple
inequalities using inverses and
identity properties to maintain
balance.
L7: Uses strategies to isolate x for
inequalities with x on one side of
Inequality and maintains balance
to solve.

Results
L8: Describes situations using
linear inequalities and solves.
In the tee-shirt canon item, the majority of students found the
correct values to solve the given equation (10x + 15 ≥ 45).
93% selected the boundary condition correctly, but 85-88%
answered completely correct, suggesting difficulty with selecting
the interval and not just the boundary value.
When requested to identify the correct inequality in a context of
attending a fair, student performance dropped.
30% got full credit for identifying two possible inequalities—
77% chose one correct option, 54% chose the other.
For an item about bungee jumping, (2x + 6 + 20 < 300), 56%
correctly answered the problem.
31% incorrectly gave the solution that the bungee was < 300 (giving
as the solution the right side of the original equation).

Results
L9: Justifies method for solving
inequalities with x on one side
only using properties (inverse,
identity, distributive property,
commutativity, and axioms of
inequality.
When students were given a set of steps solving an inequality using
properties:
89% correctly chose the distributive property,
only 28% correctly chose the additive property of inequality for the
second step.
58% chose the additive inverse incorrectly.
Only 32% correctly chose the additive identity;
63% correctly chose the division axiom of equality.
When students were asked to debug a student solution for finding x
for -4(p+3) + 6 ≥ 22:
55% correctly found the error in the first step on distributing a
negative number;
22% mistakenly said the last step, where the inequality was
reversed, represented the error.

Results
L10: Uses strategies to eliminate x
on one side of inequality,
maintaining balance to solve, and
justifies.
For 4x + 40 < -8x – 8:
61% solved correctly;
21% failed to correctly reverse the inequality sign.
For 3x + 1 > 5x – 3:
79% correctly solved,
but 18% chose either x > 4 or x < 4.

Student Difficulties with Ranges and Intervals in
Inequalities
Two examples
EX. 1: which solution set contain values that make the
inequality true
EX 2: which of the following represents a complete solution

Student responses:
• 79% chose the complete answer as one option.
• 65% identified all correct integer values.
• 17% chose the graph with an open circle.
• 30% recognized the use of the infinity sign.
• 60% selected the value of 8 on the boundary.
• 46% chose the single value of 3.5

Student responses:
• 32%: the choice using the infinity symbol.
• 68%: the number-line graph with a closed circle.
• 60%: (incorrectly) selected {…, -2, -1, 0, 1, 2, 3, 4,
5, 6. 7, 8}, (integer domain)
• 31%: the single value 8.
• 25%: the single value 3.5.
• Only 2 of 62 students selected both correct
answers with no incorrect answers.

Overall Results for RLC
Summary Table for Results for Four Constructs
Representing and Solving Equations in
One Variable with Justification
Can substitute to solve showing some weakness on
distributivity.
Understands the pan balance model.
Need work on atypical equations with x in
denominator of a direct variation or with
fractional terms.
Showed weakness applying to geometric context
Need work on multiple or no solutions.
Representing and Solving Inequalities in
One Variable with Justification
Can solve simple inequalities.
Have difficulty with flipping the sign.
Need work on distributivity and axioms of
inequality.
Need practice on writing inequalities in applied
contexts.
Describing Patterns and Relations Using
Algebraic Expressions
Can solve only simple patterns.
Lack knowledge of formal terminology associated
with the structure of the expressions that
support later parameterization of expressions.
Demonstrate only rudimentary understanding of
how to adjust parameters in patterns and explain
structural changes.
Finding Equivalent Expressions and
Substituting Values
Translates easy expressions showing weakness in
subtraction.
Substitutes values successfully.
Recognizes equivalent expressions only in simple
cases and misses distributivity and multiplication
by negatives.
Has some difficulty associating expressions with
complex diagrams and contexts.

Implications of the Diagnostic Cycle
• It functions in support of a set of formative assessment
practices:
– Process oriented and criterion-oriented
– Occurs in real time
– It supports discourse
– Shares responsibility with students
• Adds to formative assessment:
– Personalizes results to students
– Shows representativeness for whole class
– Supports the creation of flexible groups
– Documents progress over time

The Great Challenge
• What to do with this information?
• We will be studying this in partnership with our districts
and schools next.
• Invite me back in 2-3 years???

Conclusions
• Important to systematically make the results of research on
students’ learning trajectory accessible to students and
teachers;
• Learning trajectories are useful ways to encapsulate empirical
results on learning;
• Learning Maps can help students and teachers visualize
content to be learned around big ideas and anticipate
reasoning patterns;
• Diagnostic assessments tied to proficiency levels in LT can
provide just in time information to guide instruction; and
• Diagnostic information can help know what to reteach, how to
target information for individual needs, how to promote
formative discussions, and how to form flexible groups.
• Much work remains, to translate into instructional actions.

• Thank you for your time and attention.

Learning Trajectory-Aligned Diagnostic Assessments for Early Algebra, Grades 6-8

More Related Content

What's hot

Viewers also liked

Similar to Learning Trajectory-Aligned Diagnostic Assessments for Early Algebra, Grades 6-8

Recently uploaded

Learning Trajectory-Aligned Diagnostic Assessments for Early Algebra, Grades 6-8