Tools To Assess The Quality Of The Curriculum

Tools to Assess the Quality of the Curriculum, I, A, IS
The Parable of the Low Hanging Fruit

In the beginning, all that was necessary to have effective curriculum was to hand a teacher the right text.

And because only national standardized tests (which had little relationship to the taught curriculum no matter the text), who really knew what effective curriculum, teaching, or learning looked like? It looked like this:

But one day, fearing that the US would lose its status as a world power because of its education system (A Nation at Risk), ED Reform began the first movement in American Education that has ever lasted more than 7 years.
100
95.1
100
90.2
90
92.2
85.4
80.5
84.3
80
ELA
75.6
Composite Performance Index (CPI)
76.5
70.7
70
68.7
60
60.8
Math
ELA
Math
53.0
50
Ed. Reform
2001 & 02
2003 & 04
2005 & 06
2007 & 08
2009 & 10
2011 & 12
2013 & 14
Ten years ago, only 24 % of the state’s 10th graders scored proficient or higher on the math MCAS exam.

And Since Ed. Reform began, we have developed many ways beyond buying a new textbook to fine-tune C, I, A, and IS. Many more are sure to follow. We truly are building the plane while flying it!

How do we know if our curriculum is working?
Processes
LASW
Calibration of standards (NEC Mentor)
Rubric calibration
Observation/Evaluation of teaching
Vertical Teaming
Power Standards
UBD curriculum development/rubrics for evaluation
Professional Learning Community
Critical Friends
Data analysis:
MCAS, AP, SAT analyses (Root Cause)
Local Assessments
Surveys
Interviews
Parent Comments
Student Work Analysis

The Change in Assessment from Education Reform
Old model
New Model
Go on

Tools/Use
c
I
A
IS

What Works? (Marzano)(They are in rank order)
In classrooms
In schools and districts
Guaranteed (taught curriculum) and viable curriculum

Opportunity to learn
Time
Guaranteed—and assessed throughout the year
Viable—challenging rigorous (not onerous)

Challenging Goals and Effective Feedback

Monitoring (timely feedback, formative, not summative, assessment)
Pressure to achieve

Parental involvement

Good communication is critical component)

Safe and Orderly Environment

School Climate
Positive reinforcement
Productive climate and culture conducive to learning

Collegiality

Authentic professional interactions, and professionalism , content knowledge, and high correlation with pedagogy
Leadership
Learning organization
Cooperation

Effective Instructional Strategies:

Flexible grouping, planning, setting goals
Interactive learning, ongoing feedback, personalization
Identifying similarities and differences, summarizing and note taking, reinforcing effort and providing recognition, homework and practice, graphic organizers, cooperative learning, setting objectives and providing feedback, generating and testing hypotheses, questions, cues, and advance organizers
Madeline Hunter: Anticipatory set, objective and purpose, input, modeling, checking for understanding, guided practice, independent practice

Classroom management (discipline, student socialization, teacher behavior, organization, interactions, equity)
Classroom curriculum design (curriculum assessment)

It’s the teacher.

Most classrooms have mediocre teachers—Elkind, Goodlad, Sizer, Wagner
WRITING AND MATH: only 32% of college-bound students are adequately prepared for college, and 58% are in remedial courses—College Knowledge

READING: 34% of college graduates can read a complex book and extrapolate from it. NCED Statistic
WRITING: 24% of students write at proficient level; 4% at Advanced-NAEP

The Research

A meta-analysis of effectiveness based on 35 years of educational research
Effective schools 72.4 % of students pass test
Ineffective schools 27.6% of students pass test

Teachers : Decisions made on a teacher level have a far greater impact than decisions made at the school level.
The least effective teacher showed gains of 14% in student achievement in one year.
Ineffective strategies: use lower order questions based on recall, teachers talk (lecture, teacher-centered class) instead of providing information in a variety of formats, imprecise feedback on tests (grades)—No clear idea of the essential concepts and the scaffolding necessary to get ALL students there.
The most effective had gains of 53% in one year.
The gain from an average teacher is 34%

.

The cumulative effect over 3 years: for the most effective teacher is a gain of 83% point gain; for the least effective teacher the gain is 29%.

The Research: What Works in Schools: Translating Research into Action, Robert J. Marzano.

Research:What Works at the SCHOOL level?

What Works for the TEACHER:
discipline, student socialization, teacher behavior, organization, interactions, equity: routines, classroom climate
Standards-based curriculum: backwards plan
Goal setting, measuring progress

Rigor—Higher order thinking skills in questioning, tests, quizzes
Student engagement (not teacher-centered)
Writing
High expectations for all students
“Front loaded” units—students know what the final product looks like (exemplars) and how they will be graded from the first day of the unit (rubric).
Gradual release of responsibility
Student self-assessment
Good Feedback

What to look for in evaluations and walk-throughs

Benchmarking the MCAS and Standardized TestsCALIBRATION
Grade 3 MAT at 77th Percentile = Proficient Grade 4 ELA MCAS
Grade 6 MAT at 49th Percentile = Proficient at Grade 6 ELA MCAS
Grade 7 MAT at 56th Percentile = Proficient Grade 10 ELA MCAS

Grade 3 MAT at 84th Percentile = Proficient Grade 4 math MCAS
Grade 7 MAT at 72nd Percentile = Proficient Grade 8 math MCAS

Drilling Down with Data
The PIM Process

Pim: http://www.doe.mass.edu/sdi/pim/
DATA

PIM
GUIDING QUESTION: WHY HAVEN’T STUDENTS IN THE TARGETED GROUP LEARNED THE SKILLS AND KNOWLEDGE DESCRIBED IN THE STUDENT LEARNING OBJECTIVES?

TestWiz

Looking at Student WorkCalibrationRubricsLow, medium, and high protocolTeaching to the rubric

Can you predict how your students will do on the MCAS based on their class work, your tests, your textbook assessments?

Why LASW?
Common expectations for writing (and reading)
Calibrate to MCAS (at least)
Common language
Consistent experience for students
Collaborative lesson planning
Action plans for three levels of learners

DATA: What do you ask of it?

Score Analysis: Score Point 2 (AVERAGE SCORE)This response demonstrates a fair understanding of the mathematical concepts involving integers that underlie the task by completing 3 of the 6 elements.
An incorrect number line is provided which shows the negative integers placed to the right of zero, and the positive integers placed to the left of zero.
The explanation is unacceptable because it does not demonstrate an understanding that negative integers are placed to the left of zero on the number line: because everything on the right side of the 0 is - what ever number and it just goes like you would count from 0 - when it stops say to -20°.
The response correctly indicates that +3 is the greater number and provides an acceptable explanation: cause 10 is below zero and +3 is above zero.
The response correctly indicates that -3 is the greater number. However, the explanation is circular and does not demonstrate an understanding of negative integers because it is based on the incorrect number line provided in part (a): because it is on - and if you look at the line above then the -3 is higher then the -10.

START HERE
You are here

Grade 4 ORQ
Based on the article, describe the challenges Annie Smith Peck faced throughout her life. Support your answer with important details from the article.

What is good feedback?

Focuses on goal
Is clear and positive
Identifies specific strengths
Points to areas needing improvement
Suggests a route of action student can take
Limits amount of feedback to what the learner can accomplish
Models how students can self assess
Gives models, rubrics
Is timely
For example: Hamburger model/6-trait rubric

What this rubric says to a student is: This is what you are
doing now, and this is what you can do to improve.

Recognize complexity and look at a variety of assessments and the effectiveness of possible interventions while limiting focus to three students
Commonalities among three tiers
Addresses high achievers’ needs. They are often ignored
The Case Study Method

Case Study Interventions

MCAS Long Essay

Setting Clear (and Common) Expectations

Needs Improvement: 3 (of 6 for content/organization) essay for grade 7 ELA. This is what the average score for 7th grade looks like.
NEXT STEPS? Root Cause?

Consider three levels of response

Math Rubric
Scoring Guide : Students' Heights Rubric Score point 4: The response shows a comprehensive understanding of stem-and-leaf plots and how to interpret and draw conclusions from them. Score point 3: The response shows a general understanding of stem-and-leaf plots and how to interpret and draw conclusions from them. Score point 2: The response shows a basic understanding of stem-and-leaf plots. Score point 1: The response shows a minimal understanding of stem-and-leaf plots. Score point 0: The response is incorrect or contains some correct work that is irrelevant to the skill or concept being measured.

NCS Mentorhttp://www.ncsmentor.com/default.htm
Score Analysis: Score Point 2The response demonstrates a basic understanding of a stem-and-leaf plot and how to interpret and draw conclusions from them by completing 3 of the 6 elements.
The response does not correctly identify 147 cm as the mode of the students' heights. Instead, a flawed strategy which averages the heights results in an incorrect mode: you just add all the students' heights up and divide that number with the number of heights recorded.
An incorrect stem-and-leaf plot is given which includes all of the students' heights but not in the correct format and, therefore, receives no credit.
The response correctly identifies 142 as the median height of the students. The explanation demonstrates a correct strategy for finding the median: to find the median you put the numbers in order and then the number that is in the middle is the median.
The response provides the correct conclusion about the heights of the two additional students: One of the new student's height had to be less than 142 centimeters and the other new student's height had to be more than 142. However, there is no attempt to explain how this conclusion was drawn or to provide a specific example.
Successfully completing 3 of the 6 elements earns this response 2 points.

1. Begin with the end in mind2. Set measurable goals

Understanding by Design
Standards-based teaching
Clear goals
Assessments matched to goals
Activities are the LAST part of the work.

Content priorities
Worth being
familiar with
Discussions
Quizzes,
formative assessments
homework
Important to know
and do
Big Ideas
Understandings
Major performance assessment or
Final unit exam

Stage 1 – Desired results
Stage 1 – Desired Results
Content Standard (s):
Provide a framework for curriculum design; generalizations that define parameters about what students are expected to know and be able to do
Essential Question (s):
Inquiry used to explore the generalization to enable students to earn the understanding
Understanding (s):
Students will understand that…
Insight into the generalization; what students will walk away with
Knowledge:
Student will know … Skills:Students will be able to …
Specific priorities about what students are expected to know and be able to do
Design Standard for ENDURING UNDERSTANDINGS
45

Three stages of backward design
1. Identify desired results
2. Determine acceptable evidence
Then, and only then
3. Plan learning experiences &
instruction
46

Stage 2 – Assessment EvidencePerformance Task (s)Other Evidence
Varied types, over time:

authentic tasks and projects
academic exam questions, prompts, and problems
quizzes and test items
informal checks for understanding
student self-assessments

Establishing Curricular Priorities
Assessment Types
Traditional
Quizzes & tests

Paper/pencil
Selected response
Constructed-response

Performance Tasks and Projects

Open-ended
Complex
authentic

worth being familiar with
important to know & do
‘big ideas’ worth understanding
48

Reliability:Snapshot vs. photo album
We need patterns that overcome inherent measurement error
Sound assessment (particularly of State Standards) requires multiple evidence over time – a photo album vs. a single snapshot
Should a teenager get their drivers license with just a written or just a performance assessment?

FACETS OF UNDERSTANDING:
Which of the following 6 facets to you expect students to do in this unit to demonstrate their understanding?
Explanation
Interpretation
Application
Perspective
Empathy
Self-knowledge

The complexity of curriculum improvement is growing yearly.
What Works?

Professional Learning Communities

“Every educator engages
in effective professional learning every day so
every student achieves”

Skills: Measuring Progress, Focus on Students First and on Results
Rigor: Higher Order Thinking Skills
Collaboration: Purposeful Co-labor-ing
Positive school culture
A resolution for continuous improvement
The Bottom Line: Students’ achievement in the district and their readiness for their future

SEVEN STRATEGIES FOR IMPROVING CURRICULUM, INSTRUCTION, and ASSESSMENT
An urgency and understanding of the problem presented through data
A shared vision of good teaching which includes rigor, relevance, and respect
Adult meetings that focus on instruction and model good teaching
Clear standards, assessments, and consistent understanding of quality student work
Supervision that is frequent, rigorous, and focused on instruction
PD that is primarily on-site, intensive, collaborative, and job-embedded
Diagnostic data that is used frequently by teams to assess learning and teaching

The BLACK BOX:What do we need to do to unpack the needs and potential of the classroom?

The DISCONNECT BETWEEN STANDARDS and THE CLASSROOM

Firm evidence shows that formative assessment is an essential component of classroom work and that its development can raise standards of achievement.
The main problem is that pupils can assess themselves only when they have a sufficiently clear picture of the targets that their learning is meant to attain. Surprisingly, and sadly, many pupils do not have such a picture, and they appear to have become accustomed to receiving classroom teaching as an arbitrary sequence of exercises with no overarching rationale.
A particular feature of the talk between teacher and pupils is the asking of questions by the teacher. This natural and direct way of checking on learning is often unproductive.
Excerpted from “Inside the Black Box”

Changes needed “Inside the Black Box”
Traditional classroom
practices
High expectations
Student self-efficacy is enhanced with good feedback:
from
LUCK
TASK DIFFICULTY ABILITY
to
HARD WORK
Formative assessment works significantly and with low achievers.
The bell curve
predicted and
expected failure
Attitudes
Grades versus standards
Learning must be interactive.
(The social construction of knowledge)
Grades rank, but don’t inform
The quality of feedback needs to be enhanced.
Student self assessment: rubrics and examples before unit
Questioning and convergent thinking versus HOTS
Calibration to standards

CALIBRATION: Baseline data, developing correlation to MCAS.
CONNECTING: Daily classroom plans to benchmarks to MCAS
Collaborative assessment of student work, common lessons to improve, continued assessment—The next step for high achievers, average students, and students who have greatest needs.
Open Response practice across the disciplines. One per unit exam, commonly chosen and commonly assessed
Commonly assessed ORQs by grade levels and departments
Common assessment of long essay in grades 4, 7 and 9 and 10.
Increased time
Targeted teaching
Common exams—finals, mid-terms, unit—over time
Common syllabi for teachers with the same course
Exams match benchmarks, student expectations

Possible Action Responses

Action Planning

SMART Goal is Specific, Measurable, Attainable, Reasonable, Time-Specific)
GOAL: To increase fifth grade low income and SPED math scores by 10 CPI points in the 2009 MCAS. To increase average scores by 10% each quarter.
Objectives:

To provide increased time and targeted instruction for students who received warning scores in math in grade 3, 4, and 5 in 2007.
To provide 2 or three additional periods of math weekly to these students
To develop a specific curriculum for general weaknesses (ORQ, SA, fractions) and targeted individualized contracts for specific student needs.
To assess student progress every two weeks on general weaknesses and the specific weaknesses of each student
To purchase Study Island software for all students to allow for supplementary practice at home and at school.

Rationale: (SWOT analysis: Strengths, Weaknesses, Opportunities, and Threats) What will help or hinder the plan? Consider responses to change: resistance, CBAM responses, and the counter-intuitiveness of change: we go slowly to go fast, instead of anger, embrace opposition, etc.
Strength: The district has been declared in need of improvement based on the underperformance of two subgroups. Strengths: The district has an aligned curriculum with clear benchmarks. The district uses TestWiz to analyze subgroup needs to provide targeted instruction. The district develops an ISSP for each student who has received a Warning in MCAS. There is some homeroom time and x-block time to provide targeted instruction to specific students.
Weakness: The district does not have time, much money or staff to add more instructional time.
Opportunities: Grant money is available, but only $3000. Software can provide support and differentiation.
Threats: Students do not like to attend before and after school sessions.

Sample Action Plan