2. 2
Real Time Embedded Assessments
Goals:
1. To understand the different types of assessments.
2. To understand the impact formative assessments have on
student achievement.
3. To understand the how teacher evaluations, both
formative and summative impact student achievement.
3. Real Time Embedded Assessments
What are assessments?
What is the purpose of these assessments?
Do these assessments provide the data we need?
Do grades motivate students?
Should teachers take responsibility for student learning?
3
4. Real Time Embedded Assessments
Kellough (1999) lists the purposes of assessments
• To assist student learning
• To identify students’ strengths and weaknesses
• To assess the effectiveness of a particular instructional
strategy
• To assess and improve the effectiveness of curriculum
programs
• To assess and improve teaching effectiveness
• To provide data that assist in decision making
• To communicate with and involve parents
4
5. Real Time Embedded Assessments
Kellough also suggests that students need the answers to
the following questions:
• Where am I going?
• Where am I now?
• How do I get where I am going?
• How will I know when I get there?
• Am I on the right track for getting there?
5
6. Types of Assessments
Marzano (2010) suggests that there are three types of
assessments
1. Obtrusive – interrupts the normal flow of activity in the
classroom
2. Unobtrusive – do not interrupt the flow of instruction
3. Student generated – students generate ideas about
the manner in which they will demonstrate their current
status on a given topic
6
7. Types of Assessments (cont.)
• Normed Referenced (look actual data)%tiles vs
standard
– a designated standard of average performance of
people of a given age, background, etc
• Criterion Referenced
– one that provides for translating test scores into a
statement about the behavior to be expected of a
person with that score or their relationship to a
specified subject matter
7
8. Types of Assessments (cont.)
• Authentic
- A form of assessment in which students are asked to
perform real-world tasks that demonstrate meaningful
application of essential knowledge and skills
• Portfolios
- A portfolio is a purposeful collection of student work
that exhibits the student's efforts, progress, and
achievements in one or more areas of the curriculum
• Observation
- teachers observe whether students achieve the
learning goals
8
9. Types of Assessments (cont.)
• Formative
- Information gathered and reported for use in the
development of knowledge and skills
• Summative
- Information gathered and reported for use in judging
the outcome of student knowledge and skills
9
10. Summative Assessments
• State tests (MEAP, MME, etc)
• District benchmark
• Chapter tests
• Unit tests
• Final exams
• All are graded in some form
• Typical grades often provide false information
• How do we turn these scores into usable grades
10
11. Formative Assessments - definitions
Bell and Cowie (2001) “the process used by teachers and
students to recognize and respond to learning in order to
enhance that learning, during the learning.”
Popham (2008) as a planned process in which
assessment-elicited evidence of students’ status is used
by teachers to adjust their ongoing instructional
procedures or by students to adjust their current learning
tactics.
11
12. Formative Assessments - definitions
Garrison and Ehringhaus (2011) view formative
assessments as providing the information necessary to
adjust teaching and learning while they are happening.
12
13. Formative Assessments
Research has shown the formative assessments
implemented properly provide dramatic gains in learning.
The work of Black and William (1998) found that the gains
in learning by using formative assessments were
“amongst the largest ever reported for educational
interventions.”
Formative assessment works and there is no particular
formula to follow and it appears to work very well for
slow learners (Popham, 2008)
13
14. Formative Assessment
Marzano (2010) explains the elements of formative
assessment
• Formative assessment is a process, not any particular
test
• It is used not just by teachers, but by both teachers and
students
• Formative assessment takes place during instruction
14
15. Marzano’s Elements (cont.)
• It provides assessment-based feedback to teachers and
students
• The function of this feedback is to help teachers and
students make adjustments that will improve students’
achievement of intended curricular aims
15
16. Formative Assessment
Please take a few minutes and discuss the
following term and be ready to provide the results.
• Polynomials
16
17. Formative Assessment
• Polynomials
Literal meaning is many numbers
An expression consisting of the sum of two or more terms
each of which is the product of a constant and a variable
raised to an integral power: ax 2 + bx + c
similar expression in more than one variable, as 4 x 2 y 3 −
3 xy + 5 x + 7
17
18. Formative Assessments
Please take a few minutes and discuss the
following term and be ready to provide the results.
• Quadratic Equations
18
19. Formative Assessments
• Quadratic Equations
An equation containing a single variable of degree 2. Its
general form is ax 2 + bx + c = 0, where x is the variable
and a, b, and c are constants ( a ≠ 0).
19
20. Feedback
Effective feedback is critical in the formative assessment
process.
Students need to know what skills and knowledge they are
to gain, how close are they to achieving those skills, and
what do they need to do next in order to be a successful
learner.
Provides motivation for students.
20
21. Feedback
Hattie and Timperley (2007) model for feedback
• Feedback about the task – whether answers are right or
wrong or directions to get more information.
• Feedback about the processing of the task – strategies
used or strategies that could be used.
• Feedback about self-regulation – feedback about student
self evaluation or self confidence.
• Feedback about the student as a person.
21
22. Feedback
Marzano’s (2003) best ways to use feedback
• Feedback should be “corrective” in nature – provide
students with an explanation of what they did right and
wrong.
• Feedback should be timely – immediately following an
assessment
22
23. Marzano Feedback (cont.)
• Feedback should be specific to a criterion – it should
reference a specific level or skill or knowledge.
• Students can effectively provide some of their own
feedback – students keeping track of their performance
as learning occurs.
23
24. Feedback
Marzano indicates that feedback must be based on
criterion or goals.
• Instructional goals narrow what students focus on.
• Students should be encouraged to personalize the
teacher’s goals.
24
25. Goals
Brookhart (2008) states that teachers must be sure to do
the following with each assignment:
• Require student work to demonstrate the content
knowledge or skills specified in the learning target.
• Require students to demonstrate the cognitive process
specified in the learning target.
25
26. Brookhart (cont.)
• Provide students with complete and clear directions.
• Specify the criteria for good work (which will be the
criteria for both feedback and final evaluation).
26
28. Feedback Strateties
Timing
• Provide immediate feedback for knowledge of facts
• Delay feedback slightly for more comprehensive reviews
of student thinking and processing
• Never delay feedback beyond when it would make a
difference to students
• Provide feedback as often as is practical, for all major
assignments
28
29. Feedback Strategies (cont.)
Amount
• Prioritize – pick the most important points
• Choose points that relate to major learning goals
• Consider the student’s developmental level
29
30. Feedback Strategies (cont.)
Mode
• Select the best mode for the message. Would a
comment in passing the student’s desk suffice? Is a
conference needed?
• Interactive feedback is best
• Give written feedback on written work
• Use demonstration if how to do something is an issue
30
31. Feedback Strategies (cont.)
Audience
• Individual feedback makes the student feel the teacher
values their learning
• Group/class feedback works if most of the class missed
the concept – re-teaching opportunity
31
32. Feedback Strategies (cont.)
Focus
• When possible, describe both the work and the process
• Comment on the student’s self-regulation if the comment
will foster self-efficacy
• Avoid personal comments
32
33. Feedback Strategies (cont.)
Function
• Describe – don’t judge
Valence
• Use positive comments that describe what was done
well
• Accompany negative descriptions of the work with
positive suggestions for improvement
33
34. Feedback Strategies (cont.)
Clarity
• Use vocabulary and concepts the student will
understand
• Tailor the amount and content of feedback to the
student’s developmental level
34
35. Feedback Strategies (cont.)
Specificity
• Make the degree of specificity to the student and the
task
• Make feedback specific enough that they know what to
do, but not so specific that it is done for them
• Identify errors or types of errors, but do not correct
everyone – leave some for the student to correct
35
36. Feedback Strategies (cont.)
Tone
• Choose words that communicate respect for the student
and the work
• Choose words that position the student as the agent
• Choose words that cause students to think or wonder
36
37. Types of Learners
Successful Students
• Successful students are typically interested in school
and learning and want to do well on assignments.
• They also greatly benefit from constructive feedback on
their skills and knowledge.
• These students do self-assessments spontaneously
whether or not the teacher provides opportunities for this
activity.
37
38. Types of Learners
Successful Students (cont.)
• Teachers may often neglect to provide feedback to these
successful students to spend more time with slower
learners.
• Successful students will achieve even more with proper
feedback.
• No student should be neglected by the teacher even
though they are perceived as successful.
38
39. Types of Learners
Struggling Students
• Struggling students are those who have fallen behind in
school or haven’t had positive learning experiences.
• These students struggle when they view the large gap in
their knowledge based on criterion.
• Therefore, criterion-referenced feedback is not the best
choice for these students.
39
40. Types of Learners
Struggling Students (cont.)
• Self-referenced is much more applicable – comparing
their current work to previous work.
• It is important to make suggestions for improvement in
small steps for struggling students.
• Gradual and small improvements are better for the
students than being overwhelmed and not improving at
all.
40
41. Types of Learners
Reluctant Students
• These are students who perceive themselves as failures
are accustomed to viewing any kind of feedback as
confirmation that they are stupid.
• All they hear is what they did wrong.
• The natural tendency for teachers is to do just that, tell
the students what they did wrong.
41
42. Types of Learners
Reluctant Students (cont.)
• Reluctant students will benefit from self-referenced
assessments.
• This may take more time, but when students see
success and progress, they become more willing to put
forth effort into the learning process.
42
43. How Technology Can Help
• Provide feed back
to students in
ways that enable
the students to
learn better.
• Eliminated the
drudgery of
assessment.
43
44. How Technology Can Help
• Assessing students
more accurately,
efficiently, and
quickly.
• Make evaluating
student skills
unobtrusive and easy.
44
49. How Technology Can Help
• Individualized assessment
• Immediate nature of the assessment
49
50. Real Time Assessment
• Create virtual real-
time picture of which
students need help,
where they need it,
and how the
teachers can help
them best.
50
51. How Technology Can Help
• Enhancing formative
assessment with
technology enables
teachers to embed
assessment into
instruction and provide
immediate feedback.
• It has become cheaper
(sometimes free) and
easier to use.
51
52. Technology Quiz
• The largest factor in determining use of technology in a
school?
1. Technology budget
2. Amount of professional development
3. Teachers interest
4. Principal interest
52
53. Quiz Answer
The largest factor in determining use of technology in a
school is the Principal’s interest that it be used
53
54. What Type of Technology?
• Differential Instruction
• Rubrics
• White Boards and Clickers
• Problem Based Learning
• Infographics
• ePortfolios
• Digital Storytelling
• Students as Teachers
• Commercial Tools and Games
• Free Internet Tools
54
55. Free Internet Tools
• Socrative
• Quizlet
• Infographics
• ASSISTments
• Star Fall
• ePals
• Twitter in Education
55
57. • A free public service to teach children to read with
phonics.
• http://www.starfall.com/
57
58. Grand Prize for a Grand Idea
• One school will receive a one year
Adobe Creative Cloud membership
to have complete access to all Adobe
products on the Creative Cloud.
58
60. What’s Next?
• Learning Analytics - enable teachers and
schools to tailor educational opportunities to each
student's level of need and ability.
60
61. Personal Learning Environments
• PLEs- allow
students and
teachers to direct
their own learning
by themselves or in
groups. They
generally involve a
number of tools that
learners choose to
use as they learn.
61
63. Assessing Teacher Performance
Real Example from a Texas school
• Pat Davenport’s reform efforts in Brazosport, Texas –
where she served as Assistant Superintendent of
Curriculum and Instruction—were chronicled in the best-
selling book: Closing the Achievement Gap: No Excuses
(2002)
63
64. Assessing Teacher Performance
1. Pre-test (data disaggregation)
2. Create time lines (instructional calendars)
3. Direct instruction (instructional focus)
4. Regular assessment 1 to 3 weeks or less
5. Re-teach (tutorials)
6. Enrichment
7. Maintenance
8. Monitor
64
65. Assessing Teacher Performance
The 8 Step Process
By
Patricia Davenport and Peggy Hinkley
• peggyhinckley.com/.../Updated_8_step_marketi
ng_piece.169194314.pd...
65
67. Real Time Embedded Assessment
And Determining and Supporting
Teacher/Administrator Growth
• Real-time, embedded assessments provide
information about performance and growth as
learning or performance improvement is taking
place (versus after the fact or ex post facto)
• They also provides real-time feedback to
learners and performers so they can use that
feedback to guide future improvement or
learning efforts
67
71. Assessing Teacher/Administrator
Performance
• What is in the report and
recommendations?
• How will the recommendations affect
teachers and principals?
• How do the recommendations link with
Real Time, Embedded Assessment?
71
72. The MCEE Recommendations
• Four Options for Teacher Evaluation
• Two Options for Administrator Evaluation
• Funding Recommendations
• Growth and Value Added Components
• Timelines
• Training and Support
• Other Considerations
72
73. Teacher Evaluation Options
1. Danielson’s Framework for Teaching
2. Marzano’s Teacher Evaluation Model
3. Silver and Stronge’s Thoughtful Classroom
4. Five Dimensions of Teaching and Learning
– One will become the “State Tool based on a
competitive RFP process”
– State will provide base funding
– May use one of the other three, but pay any cost
differential
– Must apply for a waiver to use any other tool/system
73
74. Administrator Evaluation Options
1. D. Reeves’ Leadership Performance Rubric
2. P. Reeves’ and P. McNeill’s School ADvance
Administrator Evaluation System (through
MASA/MI-ASCD)
– One will become the “State tool based on a
competitive RFP process”
– State will provide base funding
– May use either, but pay any cost differential
– Must apply for a waiver to use any other tool/system
74
75. Teacher Effectiveness Factors
75
Teacher Evaluation Factors
Observ/Eval Tool Data
Other Practice
Measures
Building-Level VAMs
Teacher Level VAMs
Other Student Growth
Data
40%
10%
5%
25%
76. Administrator Evaluation Factors
76
Administrator Evaluation Factors
Practice Measures
(based on evaluation
tool and parent, student,
and teacher feedback)
Student Growth
Measures (including
attendance, and other
SIP measures)
77. Practice Observations
• All persons contributing observations to an educator’s practice rating
must be trained by the tool vendor on observations using the
evaluation tool and undergo supervised practice
• Principal must conduct, at least, one of the required multiple
observations (no less than three) which are factored into the practice
rating. At least one observation must be unscheduled
• Observers should provide useful developmental (specific and
focused on development) feedback on the practice of the person
being observed
• Practice evidence can include student and parent feedback (plus
teacher for administrators) and portfolio documentation but not count
for more than 20
% of the practice rating.
77
78. Growth and Value Added
Components
• State will continue to develop, select, support
assessments aligned to state=adopted content
standards in all core curriculum areas (English,
Language Arts, Mathematics, Science, and Social
Studies)
• Also for assessments in high-volume, non-core areas
where state-adopted standards exist (Arts, Health,
Physical Education, Career and Technical Education and
many high school electives)
78
79. Growth and Value Added
Components Continued
• State will provide guidelines for evaluating 3rd party and
locally developed assessments and training on rigor in
the development and measurement of student learning
objectives
• State will provide (where possible and professionally
responsible) Value-Added Modeling (VAM) scores for
educators on state assessments in core areas.
• For non-core areas LEAs can use either State VAM
scores or local Growth Scores
79
80. Growth and Value Added
Components Continued
• VAM scores can be used for a teacher’s evaluation even
if that teacher does not directly teach that subject if there
is a reasonable connection to what the teacher does
teach or the support that teacher provides
• Collective school-wide VAM scores can be used as a
10% factor in teachers’ evaluations as long as there is a
reasonable connection between the score area and the
teacher
• Where no State VAM score exists, teachers can be
evaluated based on alternative Growth measures
80
81. Growth and Value Added
Components Continued
• The Growth portion of Administrators’ evaluations will be
based on progress made on school or district level
improvement plans and will include:
– School or District Level State VAM Scores with Reading
and Match scores rated equally
– Other Local Growth Measures (e.g. student learning
objectives, graduation rates, local common assessments,
State provided assessments in non-VAM score areas,
vendor provided assessments in any content areas,
pass/fail rates, % of students on track to graduate on-time,
etc.
– Student Attendance rates
81
82. Performance Rating Categories
• Professional (includes effective and highly
effective or their equivalents from evaluation
instruments
• Provisional (equates to minimally effective or its
equivalent from evaluation instruments)
• Ineffective (equates to ineffective or its
equivalent from evaluation instruments)
82
83. Implications of Educator
Effectiveness Ratings
• All educators (including those rated Professional) must have a
professional growth plan)
• After 3 consecutive years of Professional Ratings, can be placed on
alternating year evaluation and growth plan cycle
• After 3 consecutive years of Provisional Ratings, the educator
should be “counseled out of current role”
• After 2 consecutive years of Ineffective Ratings, the educator should
be terminated from employment in current LEA
• 3 consecutive years of Professional Ratings required to move from
Provisional to Professional Certificate
• Provisional renewal after 3 years based on progress toward
Professional Ratings and district recommendations
• Lay-off , recall, and compensation all tied to Effectiveness Ratings
83
84. Implications for the Use of REAL-
TIME, EMBEDDED Assessments
• Summative Assessment Results now extremely high stakes
• Educators will not want to wait for Summative Results to know
where they stand
• Real-Time Embedded Assessments provide means to monitor
progress and make adjustments before the high stakes
assessments occur
• Educators can include student work samples and other
formative and benchmark assessments in their evidence
portfolios and may need the additional evidence
• Educators can use classroom assessments to check
curriculum alignment with high stakes summative
assessments
84
85. How Will these Recommendations
Affect Educators in Your School?
• How will you communicate what you know?
• How will you and your staff engage in district
decision making?
• How are you using Real-Time Embedded
Assessments Now; and
• How might that need to change?
85
86. Some Broader Considerations
• How will you work with your staff to make sense
of the connections between the following :
– The mission of serving all students well;
– The importance of building of a culture of
collaboration and professionalism in the school;
and
– The desire of each educator for personal
success?
• How can Real-Time, Embedded Assessments
serve all three goals?
86
Poll Title: What is the largest factor determining the use of technology in a school?
http://www.polleverywhere.com/multiple_choice_polls/QAkybw9G7GHUU3l
