SlideShare a Scribd company logo
Class Notes 11-12-12
DQs:
DQs:

  What are the important elements of Stage 2?
DQs:

  What are the important elements of Stage 2?

  How does one “think like an assessor”?
DQs:

  What are the important elements of Stage 2?

  How does one “think like an assessor”?

     What is evidence of desired results? (alignment with Stage 1)
DQs:

  What are the important elements of Stage 2?

  How does one “think like an assessor”?

     What is evidence of desired results? (alignment with Stage 1)
     What is evidence of UNDERSTANDING?
DQs:

  What are the important elements of Stage 2?

  How does one “think like an assessor”?

     What is evidence of desired results? (alignment with Stage 1)
     What is evidence of UNDERSTANDING?


  How can my students and I "change the world" through performance
  assessments?
DQs:

  What are the important elements of Stage 2?

  How does one “think like an assessor”?

     What is evidence of desired results? (alignment with Stage 1)
     What is evidence of UNDERSTANDING?


  How can my students and I "change the world" through performance
  assessments?

  What does assessment look like school and district wide?
DQs:

  What are the important elements of Stage 2?

  How does one “think like an assessor”?

     What is evidence of desired results? (alignment with Stage 1)
     What is evidence of UNDERSTANDING?


  How can my students and I "change the world" through performance
  assessments?

  What does assessment look like school and district wide?

  How can I organize what work is left?
Summary of the rest...

November 12-Finish up on Stage 2

November 19-Stage 3

November 26-Work session-wrap up week

December 3 and 10: Presentations
STAGE TWO

 Determining acceptable evidence

 Thinking Like an Assessor
Stages of Backward Design
Stages of Backward Design
 Stage One: Identifying desired results (such as enduring
 understandings, essential questions and knowledge objectives).
Stages of Backward Design
 Stage One: Identifying desired results (such as enduring
 understandings, essential questions and knowledge objectives).

 Stage Two: Determining acceptable evidence to assess and to
 evaluate student achievement of desired results.
Stages of Backward Design
 Stage One: Identifying desired results (such as enduring
 understandings, essential questions and knowledge objectives).

 Stage Two: Determining acceptable evidence to assess and to
 evaluate student achievement of desired results.

 Stage Three: Designing learning activities to promote student
 mastery and success on assessment tasks.
3 Basic Questions in order to think
like an assessor:
3 Basic Questions in order to think
like an assessor:
 What evidence?
3 Basic Questions in order to think
like an assessor:
 What evidence?

 What tasks?
3 Basic Questions in order to think
like an assessor:
 What evidence?

 What tasks?

 Does Stage 2 align with Stage 1?
Not a natural process              -as we are
focused on what are the results?
Not a natural process              -as we are
focused on what are the results?


  See figure 7.3 on p. 151.
Not a natural process                  -as we are
focused on what are the results?


  See figure 7.3 on p. 151.

  The UbD template “short circuits” the
  tendency to think like “an activity designer.”
From snapshot to scrapbook
From snapshot to scrapbook
 Figure 7.4 and 7.5, pages 152-3.
From snapshot to scrapbook
 Figure 7.4 and 7.5, pages 152-3.

 What types of assessments are there?
Some basic rules regarding
assessment
Some basic rules regarding
assessment
 You’re not generating grades.
Some basic rules regarding
assessment
 You’re not generating grades.
 Align assessment with Stage 1
Some basic rules regarding
assessment
 You’re not generating grades.
 Align assessment with Stage 1
 Ask yourself: What kinds of evidence do we need?
Some basic rules regarding
assessment
 You’re not generating grades.
 Align assessment with Stage 1
 Ask yourself: What kinds of evidence do we need?
 Think like an assessor, not an activity designer.
Some basic rules regarding
assessment
 You’re not generating grades.
 Align assessment with Stage 1
 Ask yourself: What kinds of evidence do we need?
 Think like an assessor, not an activity designer.
 Snapshot to scrapbook
Some basic rules regarding
assessment
 You’re not generating grades.
 Align assessment with Stage 1
 Ask yourself: What kinds of evidence do we need?
 Think like an assessor, not an activity designer.
 Snapshot to scrapbook
 Authentic whenever possible
Some basic rules regarding
assessment
 You’re not generating grades.
 Align assessment with Stage 1
 Ask yourself: What kinds of evidence do we need?
 Think like an assessor, not an activity designer.
 Snapshot to scrapbook
 Authentic whenever possible
 Use 6 facets when designing assessments and be
 sure the assessment align with EQs and EUs.
The Performance Task
The Performance Task
  Designing around problems:

    John Dewey: “Does the question naturally suggest itself
    within some situation or personal experience? Or is it an
    aloof thing…? Is it the sort of thing that would arouse
    conversation and engage experimentation out side of
    school? Or is it made a problem for the pupil only because
    he cannot get the required mark or be promoted or win the
    teacher’s approval unless he deals with it? (1916)
GRASPS
GRASPS
G=Goals from the real world.
GRASPS
G=Goals from the real world.
R=Roles that are authentic and based in reality.
GRASPS
G=Goals from the real world.
R=Roles that are authentic and based in reality.
A=Audiences to whom students will present final products
and performances
GRASPS
G=Goals from the real world.
R=Roles that are authentic and based in reality.
A=Audiences to whom students will present final products
and performances
S=Situations involving a real-world conflict to be resolved,
decision to be made, investigation to be completed
GRASPS
G=Goals from the real world.
R=Roles that are authentic and based in reality.
A=Audiences to whom students will present final products
and performances
S=Situations involving a real-world conflict to be resolved,
decision to be made, investigation to be completed
P=Products and Performances culminating from the study
GRASPS
G=Goals from the real world.
R=Roles that are authentic and based in reality.
A=Audiences to whom students will present final products
and performances
S=Situations involving a real-world conflict to be resolved,
decision to be made, investigation to be completed
P=Products and Performances culminating from the study
S=Standards for evaluating project-based products and
performances
Work for the week
Work for the week-both classwork and homework:

  Listen to the Podcast and follow along with slides. Be sure to
  watch the NYC Urban Academy video.

  After watching “Teaching 2030”, describe in your own words the
  five major points the group makes. Make a Google Doc and share
  with Bill. (wcarozza@nec.edu)

  Read Chapter 7 in the text-Thinking Like an Assessor

  Develop a GRASPS-share a Google Doc with Bill similar to the
  blank one on p. 172 in workbook.
GRASPS-Performance Task



Let’s share.
An exemplary GRASPS
     with EQs and EUs
Looking at Assessment
from a School or District
      Perspective
General guidelines for
    Assessment
General guidelines for
               Assessment tests to assess
We need to look at more than just objective
knowledge and skill.
General guidelines for
               Assessment tests to assess
We need to look at more than just objective
knowledge and skill.

How do we design performances that enable us to make
precise judgment about the different parts of the
performance?
General guidelines for
               Assessment tests to assess
We need to look at more than just objective
knowledge and skill.

How do we design performances that enable us to make
precise judgment about the different parts of the
performance?

Try to have parallel versions of the same content across
different assessment formats.
General guidelines for
               Assessment tests to assess
We need to look at more than just objective
knowledge and skill.

How do we design performances that enable us to make
precise judgment about the different parts of the
performance?

Try to have parallel versions of the same content across
different assessment formats.

Given that a single application or product may or may not
link to larger goals, ask students to “show their work”, give
reasons for answers and show connections to larger ideas in
the answers.
Three Broad Types of
    Assessment
Three Broad Types of
           Assessment
Secured Assessments
Three Broad Types of
           Assessment
Secured Assessments

   No help is given, e.g. NECAP, NWEA
Three Broad Types of
           Assessment
Secured Assessments

   No help is given, e.g. NECAP, NWEA


   Norm Referenced vs. Criterion Referenced tests
Three Broad Types of
           Assessment
Secured Assessments

   No help is given, e.g. NECAP, NWEA


   Norm Referenced vs. Criterion Referenced tests


       NRTs: classify students from high to low in large sample-compared to other peers.
Three Broad Types of
           Assessment
Secured Assessments

   No help is given, e.g. NECAP, NWEA


   Norm Referenced vs. Criterion Referenced tests


       NRTs: classify students from high to low in large sample-compared to other peers.



       CRTs: student performance based against a criteria.
Three Broad Types of
           Assessment
Secured Assessments

   No help is given, e.g. NECAP, NWEA


   Norm Referenced vs. Criterion Referenced tests


       NRTs: classify students from high to low in large sample-compared to other peers.



       CRTs: student performance based against a criteria.



Classroom embedded Assessments
Three Broad Types of
           Assessment
Secured Assessments

   No help is given, e.g. NECAP, NWEA


   Norm Referenced vs. Criterion Referenced tests


       NRTs: classify students from high to low in large sample-compared to other peers.



       CRTs: student performance based against a criteria.



Classroom embedded Assessments

   Coming out of classroom activities (Stage 2)
Three Broad Types of
            Assessment
Secured Assessments

    No help is given, e.g. NECAP, NWEA


    Norm Referenced vs. Criterion Referenced tests


        NRTs: classify students from high to low in large sample-compared to other peers.



        CRTs: student performance based against a criteria.



Classroom embedded Assessments

    Coming out of classroom activities (Stage 2)


Composite Records
Three Broad Types of
            Assessment
Secured Assessments

    No help is given, e.g. NECAP, NWEA


    Norm Referenced vs. Criterion Referenced tests


        NRTs: classify students from high to low in large sample-compared to other peers.



        CRTs: student performance based against a criteria.



Classroom embedded Assessments

    Coming out of classroom activities (Stage 2)


Composite Records

    Best Works Portfolios (managed by students)
Three Broad Types of
            Assessment
Secured Assessments

    No help is given, e.g. NECAP, NWEA


    Norm Referenced vs. Criterion Referenced tests


        NRTs: classify students from high to low in large sample-compared to other peers.



        CRTs: student performance based against a criteria.



Classroom embedded Assessments

    Coming out of classroom activities (Stage 2)


Composite Records

    Best Works Portfolios (managed by students)


    Progress Portfolios (managed by teachers)
Three Broad Types of
            Assessment
Secured Assessments

    No help is given, e.g. NECAP, NWEA


    Norm Referenced vs. Criterion Referenced tests


         NRTs: classify students from high to low in large sample-compared to other peers.



         CRTs: student performance based against a criteria.



Classroom embedded Assessments

    Coming out of classroom activities (Stage 2)


Composite Records

    Best Works Portfolios (managed by students)


    Progress Portfolios (managed by teachers)


    Accountability Portfolios (e.g. KY and VT)
Assessment within a school or
 district-making data driven
      decisions-examples
Assessment within a school or
 district-making data driven
      decisions-examples
Program and Curriculum Development
Assessment within a school or
 district-making data driven
      decisions-examples
Program and Curriculum Development

 Making decisions on programs
Assessment within a school or
 district-making data driven
      decisions-examples
Program and Curriculum Development

 Making decisions on programs

Child Specific
Assessment within a school or
 district-making data driven
      decisions-examples
Program and Curriculum Development

 Making decisions on programs

Child Specific

 Making decisions about children
Assessment within a school or
 district-making data driven
      decisions-examples
Program and Curriculum Development

 Making decisions on programs

Child Specific

 Making decisions about children

School Climate
Assessment within a school or
 district-making data driven
      decisions-examples
Program and Curriculum Development

 Making decisions on programs

Child Specific

 Making decisions about children

School Climate

 Making decisions about school climate
An Assessment
  Philosophy
An Assessment
        Philosophy
Assessment values what is taught and what students learn.
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.

Assessment addresses the process and the product.
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.

Assessment addresses the process and the product.

Assessment is both formal (e.g. standardized or published tests) and on
going (e.g. observation and anecdotal records
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.

Assessment addresses the process and the product.

Assessment is both formal (e.g. standardized or published tests) and on
going (e.g. observation and anecdotal records

Assessment must be developmentally appropriate.
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.

Assessment addresses the process and the product.

Assessment is both formal (e.g. standardized or published tests) and on
going (e.g. observation and anecdotal records

Assessment must be developmentally appropriate.

Assessment exists in multiple forms.
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.

Assessment addresses the process and the product.

Assessment is both formal (e.g. standardized or published tests) and on
going (e.g. observation and anecdotal records

Assessment must be developmentally appropriate.

Assessment exists in multiple forms.

Assessment is used to evaluate programs and individual student
progress.
An Assessment
        Philosophy
Assessment values what is taught and what students learn.

Assessment drives instruction for every student.

Assessment addresses the process and the product.

Assessment is both formal (e.g. standardized or published tests) and on
going (e.g. observation and anecdotal records

Assessment must be developmentally appropriate.

Assessment exists in multiple forms.

Assessment is used to evaluate programs and individual student
progress.

Assessment honors a “value-added” approach.
ASSESSMENT MATRIX * HOPKINTON SCHOOL DISTRICT
                                         2005-0 6

GRADE     ASSESSMENT                                          DESCRIPTION
  K     Observation Survey An early literacy assessment that assesses the following:

                                  • Text Reading Level
                                  • Letter Identification
                                  • Concepts About Print
                                         o D e m onstrate understanding of concepts about book handling and
                                             conventions of printed language while teacher reads a book
                                  • Word Test
                                         o Read a list of 20 words
                                  • Writing Vocabulary
                                         o S t u dent has 10 minutes to write all the words he/she knows
                                  • Hearing and Recording Sounds in Words
                                         o S t u dent writes a dictated sentence which is scored by the number of
                                             sounds hea r d
        Write Traits Writing   Assesses six important “traits” of writing: * Idea Development
        Benchmark                * Organization
                                  * Voice
                                 * Word Choice
                                 * Sentence Fluency
                                 * Conventions


  1     DRA                       • The Developmental Reading Assessment provides teachers with a method
                                     for assessing and documenting students' development as readers over time
                                     within a literature-based instructional reading program.
                                  • The DRA is designed to be used in elementary classrooms with rich literate
                                     environments.
                                  • The assessments are conducted during one-on-one reading conferences as
                                     children read specially selected assessment texts. A set of 20 stories, which
2   DRA                    See above
    Write Traits Writing   See above
    Benchmark
    MAP testing in         The NWEA (Northwest Evaluation Association) MAP test provides teachers and
    Reading and Math       parents with a standardized assessment of a student’s ability. You can receive
                           significant results the same day (and much more not long after) and use the
                           information to drive your instruction. The test is taken by students on the computer.
                           This assessment is given both in the fall and spring.

3   DRA                    See above
    Write Traits Writing   See above
    Benchmark
    MAP testing in         See above
    Reading and Math
    NECAP                  State assessment in Math and Reading/Language Arts-tested in the fall

4   DRA                    See above
    Write Traits Writing   See above
    Benchmark
    MAP testing in         See above
    Reading and Math
    NECAP                  State assessment in Math and Reading/Language Arts-tested in the fall
    NAEP                   National Assessment of Educational Progress-known as the “Nation’s Report Card”.

5   DRA                    See above
    Write Traits Writing   See above
    Benchmark
    MAP testing in         See above
    Reading and Math
    NECAP                  State Assessment in Math, Reading/Language Arts, and in Writing-tested in the fall
10   Selected MAP         Primarily for Math IIs.
     testing
     NHEIAP               State Assessment in Math and Reading/Language Arts-tested in the spring
     PSAT/NMSQT           PSAT/NMSQT stands for Preliminary SAT/National Merit Scholarship Qualifying
                          Test. It's a standardized test that provides firsthand practice for the SAT.
     World Language
     Assessment

11   SAT                  The Scholastic Aptitude Test (SAT) is a standard submission test that is used by
                          colleges around the country to help identify students who will succeed at a given
                          college.
     Subject Tests (SAT   Subject Tests (formerly SAT II: Subject Tests) are designed to measure student
     II)                  knowledge and skills in particular subject areas, as well as the ability to apply that
                          knowledge.
     Advanced             Taken by students in A.P. courses in order to receive college credit.
     Placement Tests
     PSAT/NMSQT           PSAT/NMSQT stands for Preliminary SAT/National Merit Scholarship Qualifying
                          Test. It's a standardized test that provides firsthand practice for the SAT. It also
                          provides an opportunity for National Merit scholarship programs.
     ACT                  The ACT assesses high school students' general educational development and their
                          ability to complete college-level work.
     World Language
     Assessment
MAP testing
Examples from the Internet -Quantitative
              Assessment
Lexiles
Lexiles


A system for measuring the difficulty of text.
Lexiles


A system for measuring the difficulty of text.

Reading reports give Lexile scores
Lexiles


A system for measuring the difficulty of text.

Reading reports give Lexile scores

Check out Lexile.com
Uses of MAP data
Uses of MAP data
Benchmarking student performance in reading and math
Uses of MAP data
Benchmarking student performance in reading and math
Entry points of DI in reading and math
Uses of MAP data
Benchmarking student performance in reading and math
Entry points of DI in reading and math

Identification of PD for school/teachers
Uses of MAP data
Benchmarking student performance in reading and math
Entry points of DI in reading and math

Identification of PD for school/teachers
Construction of IEP goals and objectives
Uses of MAP data
Benchmarking student performance in reading and math
Entry points of DI in reading and math

Identification of PD for school/teachers
Construction of IEP goals and objectives

Cross comparison data with reading benchmarks
Uses of MAP data
Benchmarking student performance in reading and math
Entry points of DI in reading and math

Identification of PD for school/teachers
Construction of IEP goals and objectives

Cross comparison data with reading benchmarks

Selection of readability levels
Uses of MAP data
Benchmarking student performance in reading and math
Entry points of DI in reading and math

Identification of PD for school/teachers
Construction of IEP goals and objectives

Cross comparison data with reading benchmarks

Selection of readability levels
Conversation starters with parents, especially at parent
conferences
Part of assessment criteria to support pre-
algebra decision
Part of assessment criteria to support pre-
algebra decision
Used in RTI decisions, pre-referral meetings,
Child Study
Part of assessment criteria to support pre-
algebra decision
Used in RTI decisions, pre-referral meetings,
Child Study
Part of criteria to support admission to
enrichment reading
Part of assessment criteria to support pre-
algebra decision
Used in RTI decisions, pre-referral meetings,
Child Study
Part of criteria to support admission to
enrichment reading
Conversation at PD planning meetings with
Principal and teacher.
Part of assessment criteria to support pre-
algebra decision
Used in RTI decisions, pre-referral meetings,
Child Study
Part of criteria to support admission to
enrichment reading
Conversation at PD planning meetings with
Principal and teacher.
Part of the placement puzzle
Digital Portfolios
  Qualitative Assessment
Digital Portfolios
Digital Portfolios

 Based on research from the Coalition
  of Essential Schools
Digital Portfolios

 Based on research from the Coalition
  of Essential Schools
  Started in 1993 with districts in New York, New Hampshire,
   and Kentucky
Digital Portfolios

 Based on research from the Coalition
  of Essential Schools
  Started in 1993 with districts in New York, New Hampshire,
   and Kentucky

 The Richer Picture software
  currently in use for elementary,
  secondary, and professional
Digital Portfolios
Digital Portfolios

 We can use technology to create
  a richer picture of a student’s
  abilities
Digital Portfolios

 We can use technology to create
  a richer picture of a student’s
  abilities
 Portfolio can contain a set of
  work that demonstrates:
Digital Portfolios

 We can use technology to create
  a richer picture of a student’s
  abilities
 Portfolio can contain a set of
  work that demonstrates:
  a student’s accomplishments AND
What goes in a
  portfolio?
What goes in a
         portfolio?

 Samples of student work
What goes in a
         portfolio?

 Samples of student work

 Information to put the work in context
What goes in a
         portfolio?

 Samples of student work

 Information to put the work in context

 Reflection on the work
What goes in a
         portfolio?

 Samples of student work

 Information to put the work in context

 Reflection on the work

 Portfolios represent a subset of student work
What goes in a portfolio-
      specifically?
What goes in a portfolio-
        specifically?

Collect
What goes in a portfolio-
         specifically?

Collect
  2 – 4 entries from each course
What goes in a portfolio-
         specifically?

Collect
  2 – 4 entries from each course

  Each entry has student work, a summary and a reflection
What goes in a portfolio-
         specifically?

Collect
  2 – 4 entries from each course

  Each entry has student work, a summary and a reflection

Select
What goes in a portfolio-
         specifically?

Collect
  2 – 4 entries from each course

  Each entry has student work, a summary and a reflection

Select
  Student determines which entries provide the best
  evidence
What goes in a portfolio-
         specifically?

Collect
  2 – 4 entries from each course

  Each entry has student work, a summary and a reflection

Select
  Student determines which entries provide the best
  evidence

Reflect
Essential Questions
Essential Questions
Expectations
Essential Questions
Expectations

 What should students know and do?
Essential Questions
Expectations

 What should students know and do?

 What purpose does the portfolio serve?
Essential Questions
Expectations

 What should students know and do?

 What purpose does the portfolio serve?

 Who is the audience?
Essential Questions
Expectations

 What should students know and do?

 What purpose does the portfolio serve?

 Who is the audience?

Entries
Essential Questions
Expectations

 What should students know and do?

 What purpose does the portfolio serve?

 Who is the audience?

Entries

 How can students exhibit the expectations?
Essential Questions
Expectations

 What should students know and do?

 What purpose does the portfolio serve?

 Who is the audience?

Entries

 How can students exhibit the expectations?

 How do we ensure tasks are “portfolio worthy” –
Essential Questions
Essential Questions

Assessment
Essential Questions

Assessment

 How do we decide what’s good?
Essential Questions

Assessment

 How do we decide what’s good?

 How do we give consistent messages?
Essential Questions

Assessment

  How do we decide what’s good?

  How do we give consistent messages?

School Structures
Essential Questions

Assessment

  How do we decide what’s good?

  How do we give consistent messages?

School Structures

  Who does what? When does it happen?
Essential Questions

Assessment

  How do we decide what’s good?

  How do we give consistent messages?

School Structures

  Who does what? When does it happen?
   Expectations for teachers, students, administrators
Essential Questions

Assessment

  How do we decide what’s good?

  How do we give consistent messages?

School Structures

  Who does what? When does it happen?
   Expectations for teachers, students, administrators

   How do we use technology?
Why Digital Portfolios for your
    School or Classroom?
Why Digital Portfolios for your
    School or Classroom?

It helps to balance the quantitative side of
assessment with qualitative work.
Why Digital Portfolios for your
    School or Classroom?

It helps to balance the quantitative side of
assessment with qualitative work.

It honors student work product not just a score
on a test.
Why Digital Portfolios for your
    School or Classroom?

It helps to balance the quantitative side of
assessment with qualitative work.

It honors student work product not just a score
on a test.

Most importantly…it is a portfolio first; the
technology is the vehicle.
How Could DP’s Be Useful for
 your school or classroom?
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !

A tool for capturing, storing, and examining student work.!
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !

A tool for capturing, storing, and examining student work.!

Allows students to help explain their understanding-application of learning.!
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !

A tool for capturing, storing, and examining student work.!

Allows students to help explain their understanding-application of learning.!

Tool for us to drill down into specific learning outcomes and issues, e.g. examination
of reading video to highlight a student's difficulties reading.!
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !

A tool for capturing, storing, and examining student work.!

Allows students to help explain their understanding-application of learning.!

Tool for us to drill down into specific learning outcomes and issues, e.g. examination
of reading video to highlight a student's difficulties reading.!

Capturing artwork and other 3 dimensional work.
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !

A tool for capturing, storing, and examining student work.!

Allows students to help explain their understanding-application of learning.!

Tool for us to drill down into specific learning outcomes and issues, e.g. examination
of reading video to highlight a student's difficulties reading.!

Capturing artwork and other 3 dimensional work.

Capturing music and p.e. work as well-the arts in general.!
How Could DP’s Be Useful for
  your school or classroom?
Helps to document what students need to know and be able to do.!

To use in parent conferences-conversation starter-helps parents understand
educational jargon.!
                   !

A tool for capturing, storing, and examining student work.!

Allows students to help explain their understanding-application of learning.!

Tool for us to drill down into specific learning outcomes and issues, e.g. examination
of reading video to highlight a student's difficulties reading.!

Capturing artwork and other 3 dimensional work.

Capturing music and p.e. work as well-the arts in general.!

Highlights students’ academic and emotional behavior in areas outside of the
classroom.! !
How Could DP’s Be Useful at Your
School or Classroom? (continued)
How Could DP’s Be Useful at Your
School or Classroom? (continued)
How Could DP’s Be Useful at Your
School or Classroom? (continued)


 Getting students to "own" their learning-self
 reflection.
How Could DP’s Be Useful at Your
School or Classroom? (continued)


 Getting students to "own" their learning-self
 reflection.
 Teachers better understand how their new
 students learn thus aiding transition-keeps
 kids from starting all over again and makes
 their elementary years much more fluid
How Could DP’s Be Useful at Your
School or Classroom? (continued)


 Getting students to "own" their learning-self
 reflection.
 Teachers better understand how their new
 students learn thus aiding transition-keeps
 kids from starting all over again and makes
 their elementary years much more fluid
 More manageable than using paper over
What can be put in the
        DP?
What can be put in the
        DP?
Scanned work (e.g. student writing)
What can be put in the
        DP?
Scanned work (e.g. student writing)

Photos
What can be put in the
        DP?
Scanned work (e.g. student writing)

Photos

Online rubrics
What can be put in the
        DP?
Scanned work (e.g. student writing)

Photos

Online rubrics

Video
What can be put in the
        DP?
Scanned work (e.g. student writing)

Photos

Online rubrics

Video

Anything that can be put on a web site
Who can view it?


Teachers and students

Parents at conferences

Parents at home
Homework for next
        week
Complete reflections on Stage 1 and 2 which includes and integrates
School Wide Assessment. At least a page for both. Share a Google
Doc with me for each. (two docs)

Continue working on both Stage 1 and 2.

Begin brainstorming lesson plan ideas.

Read: (links on Diigo under tag “UbD”.)

  UbD Introduction

  UbD in a Nutshell
3X5


What was the big point you learned in class
today?

What is the main unanswered question you
leave class with today?

More Related Content

What's hot

Assessments
AssessmentsAssessments
Assessments
annabethMB
 
Assessment in the Curriculum Design Process
Assessment in the Curriculum Design Process Assessment in the Curriculum Design Process
Assessment in the Curriculum Design Process
Peter Gow
 
Standards Based Grading - Benefits & Implementation
Standards Based Grading - Benefits & ImplementationStandards Based Grading - Benefits & Implementation
Standards Based Grading - Benefits & Implementation
Martin William
 
ECIS Standards-Based Assessment Pre-Conference
ECIS Standards-Based Assessment Pre-ConferenceECIS Standards-Based Assessment Pre-Conference
ECIS Standards-Based Assessment Pre-ConferenceSue Williams
 
Authentic Assessment (多元評量) Methods Workshop (12/6/2013)
Authentic Assessment (多元評量) Methods Workshop (12/6/2013)Authentic Assessment (多元評量) Methods Workshop (12/6/2013)
Authentic Assessment (多元評量) Methods Workshop (12/6/2013)
Anita Chiou
 
Psychometrics 101: Know What Your Assessment Data is Telling You
Psychometrics 101: Know What Your Assessment Data is Telling YouPsychometrics 101: Know What Your Assessment Data is Telling You
Psychometrics 101: Know What Your Assessment Data is Telling You
ExamSoft
 
Providing the Spark for CCSS
Providing the Spark for CCSSProviding the Spark for CCSS
Providing the Spark for CCSS
Kristen Wheat
 
Assessment to the Core Webinar by Jay McTighe
Assessment to the Core Webinar by Jay McTigheAssessment to the Core Webinar by Jay McTighe
Assessment to the Core Webinar by Jay McTighe
School Improvement Network
 
Creating First Exposure Assignments
Creating First Exposure AssignmentsCreating First Exposure Assignments
Creating First Exposure Assignments
St. John Fisher College
 
Tlcp learning goals-sc-educ 9 f61
Tlcp learning goals-sc-educ 9 f61Tlcp learning goals-sc-educ 9 f61
Tlcp learning goals-sc-educ 9 f61chdavies
 
10 29 09 Day Of Differentiation
10 29 09 Day Of Differentiation10 29 09 Day Of Differentiation
10 29 09 Day Of Differentiation
PSU Summer Institute
 
Standards Based Grading
Standards Based GradingStandards Based Grading
Standards Based GradingEric Langhorst
 
Assessment
AssessmentAssessment
Building your assessments to meet course objectives -detc
Building your assessments to meet course objectives -detcBuilding your assessments to meet course objectives -detc
Building your assessments to meet course objectives -detcostromae
 
Grading Practices MPHS Presentation
Grading Practices MPHS PresentationGrading Practices MPHS Presentation
Grading Practices MPHS Presentationaschoenborn
 
Riroe assessment workshop part #1
Riroe assessment workshop part #1Riroe assessment workshop part #1
Riroe assessment workshop part #1
AnneWeerda
 
Introduction to e-Assessment
Introduction to e-AssessmentIntroduction to e-Assessment
Introduction to e-AssessmentJisc Scotland
 
Simple k12 performance task assessment
Simple k12 performance task assessmentSimple k12 performance task assessment
Simple k12 performance task assessmentJonathan Martin
 

What's hot (20)

Assessments
AssessmentsAssessments
Assessments
 
Assessment in the Curriculum Design Process
Assessment in the Curriculum Design Process Assessment in the Curriculum Design Process
Assessment in the Curriculum Design Process
 
Standards Based Grading - Benefits & Implementation
Standards Based Grading - Benefits & ImplementationStandards Based Grading - Benefits & Implementation
Standards Based Grading - Benefits & Implementation
 
ECIS Standards-Based Assessment Pre-Conference
ECIS Standards-Based Assessment Pre-ConferenceECIS Standards-Based Assessment Pre-Conference
ECIS Standards-Based Assessment Pre-Conference
 
Authentic Assessment (多元評量) Methods Workshop (12/6/2013)
Authentic Assessment (多元評量) Methods Workshop (12/6/2013)Authentic Assessment (多元評量) Methods Workshop (12/6/2013)
Authentic Assessment (多元評量) Methods Workshop (12/6/2013)
 
Psychometrics 101: Know What Your Assessment Data is Telling You
Psychometrics 101: Know What Your Assessment Data is Telling YouPsychometrics 101: Know What Your Assessment Data is Telling You
Psychometrics 101: Know What Your Assessment Data is Telling You
 
Providing the Spark for CCSS
Providing the Spark for CCSSProviding the Spark for CCSS
Providing the Spark for CCSS
 
Assessment to the Core Webinar by Jay McTighe
Assessment to the Core Webinar by Jay McTigheAssessment to the Core Webinar by Jay McTighe
Assessment to the Core Webinar by Jay McTighe
 
Creating First Exposure Assignments
Creating First Exposure AssignmentsCreating First Exposure Assignments
Creating First Exposure Assignments
 
Tlcp learning goals-sc-educ 9 f61
Tlcp learning goals-sc-educ 9 f61Tlcp learning goals-sc-educ 9 f61
Tlcp learning goals-sc-educ 9 f61
 
10 29 09 Day Of Differentiation
10 29 09 Day Of Differentiation10 29 09 Day Of Differentiation
10 29 09 Day Of Differentiation
 
Standards Based Grading
Standards Based GradingStandards Based Grading
Standards Based Grading
 
Assessment
AssessmentAssessment
Assessment
 
Gps mod 8 v2.11
Gps mod 8 v2.11Gps mod 8 v2.11
Gps mod 8 v2.11
 
Building your assessments to meet course objectives -detc
Building your assessments to meet course objectives -detcBuilding your assessments to meet course objectives -detc
Building your assessments to meet course objectives -detc
 
BackwardsDesignPedagogy_OUSD_Camargo
BackwardsDesignPedagogy_OUSD_CamargoBackwardsDesignPedagogy_OUSD_Camargo
BackwardsDesignPedagogy_OUSD_Camargo
 
Grading Practices MPHS Presentation
Grading Practices MPHS PresentationGrading Practices MPHS Presentation
Grading Practices MPHS Presentation
 
Riroe assessment workshop part #1
Riroe assessment workshop part #1Riroe assessment workshop part #1
Riroe assessment workshop part #1
 
Introduction to e-Assessment
Introduction to e-AssessmentIntroduction to e-Assessment
Introduction to e-Assessment
 
Simple k12 performance task assessment
Simple k12 performance task assessmentSimple k12 performance task assessment
Simple k12 performance task assessment
 

Viewers also liked

Opening HMS Staff Meeting 2009
Opening HMS Staff Meeting 2009Opening HMS Staff Meeting 2009
Opening HMS Staff Meeting 2009
Bill Carozza, Principal, Harold Martin School
 
iPod Touch Workshop
iPod Touch WorkshopiPod Touch Workshop
Podcasting-Bill Carozza
Podcasting-Bill CarozzaPodcasting-Bill Carozza
Tsunamis by Hunter
Tsunamis by HunterTsunamis by Hunter
Tsunamis by HunterSusan Cannon
 
Using the Cloud to be More Productive- LESCN April 2013
Using the Cloud to be More Productive- LESCN April 2013Using the Cloud to be More Productive- LESCN April 2013
Using the Cloud to be More Productive- LESCN April 2013
Bill Carozza, Principal, Harold Martin School
 
Mike Pecen Portfolio
Mike Pecen PortfolioMike Pecen Portfolio
Mike Pecen Portfoliompecen
 

Viewers also liked (6)

Opening HMS Staff Meeting 2009
Opening HMS Staff Meeting 2009Opening HMS Staff Meeting 2009
Opening HMS Staff Meeting 2009
 
iPod Touch Workshop
iPod Touch WorkshopiPod Touch Workshop
iPod Touch Workshop
 
Podcasting-Bill Carozza
Podcasting-Bill CarozzaPodcasting-Bill Carozza
Podcasting-Bill Carozza
 
Tsunamis by Hunter
Tsunamis by HunterTsunamis by Hunter
Tsunamis by Hunter
 
Using the Cloud to be More Productive- LESCN April 2013
Using the Cloud to be More Productive- LESCN April 2013Using the Cloud to be More Productive- LESCN April 2013
Using the Cloud to be More Productive- LESCN April 2013
 
Mike Pecen Portfolio
Mike Pecen PortfolioMike Pecen Portfolio
Mike Pecen Portfolio
 

Similar to Cn 11 12-12

Performance-based Assessment
Performance-based AssessmentPerformance-based Assessment
Performance-based Assessment
Beberly Fabayos
 
CRITERIA AND VALIDITY PRESENTATIONS.pptx
CRITERIA AND VALIDITY PRESENTATIONS.pptxCRITERIA AND VALIDITY PRESENTATIONS.pptx
CRITERIA AND VALIDITY PRESENTATIONS.pptx
MammeNkwelle
 
Intro to Curriculum Design
Intro to Curriculum DesignIntro to Curriculum Design
Intro to Curriculum Design
Peter Gow
 
Building a Peer Evaluation Program
Building a Peer Evaluation ProgramBuilding a Peer Evaluation Program
Building a Peer Evaluation Program
Qualtrics
 
An Overview of Assessment Design
An Overview of Assessment DesignAn Overview of Assessment Design
An Overview of Assessment Design
Peter Gow
 
Assessment (11)
Assessment (11)Assessment (11)
Assessment (11)
JenniferPankowskiBlu
 
Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...
Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...
Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...
Mark Eutsler
 
Assessment: Achieving improved efficiency, effectiveness, educational integri...
Assessment:Achieving improved efficiency, effectiveness, educational integri...Assessment:Achieving improved efficiency, effectiveness, educational integri...
Assessment: Achieving improved efficiency, effectiveness, educational integri...
Diana Quinn
 
Assessment 072
Assessment 072Assessment 072
Assessment 072fogleman
 
LOTW
LOTWLOTW
Making learning visible_mon_25_aug_1pm
Making learning visible_mon_25_aug_1pmMaking learning visible_mon_25_aug_1pm
Making learning visible_mon_25_aug_1pm
caulkins
 
Assessing learning in Instructional Design
Assessing learning in Instructional DesignAssessing learning in Instructional Design
Assessing learning in Instructional Designleesha roberts
 
Assessment
Assessment Assessment
Assessment
Maureen Cohen
 
Authentic Assessment
Authentic AssessmentAuthentic Assessment
Authentic Assessmentxanderjoy
 
Creating Rubrics
Creating RubricsCreating Rubrics
Creating Rubrics
Roberta Niche
 
Performance-Based Assessment
Performance-Based AssessmentPerformance-Based Assessment
Performance-Based Assessment
deped
 
Making learning visible_mon_25_aug_2.30pm
Making learning visible_mon_25_aug_2.30pmMaking learning visible_mon_25_aug_2.30pm
Making learning visible_mon_25_aug_2.30pm
caulkins
 
How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...
How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...
How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...
National Resource Center for Paraprofessionals
 

Similar to Cn 11 12-12 (20)

Performance-based Assessment
Performance-based AssessmentPerformance-based Assessment
Performance-based Assessment
 
CRITERIA AND VALIDITY PRESENTATIONS.pptx
CRITERIA AND VALIDITY PRESENTATIONS.pptxCRITERIA AND VALIDITY PRESENTATIONS.pptx
CRITERIA AND VALIDITY PRESENTATIONS.pptx
 
Intro to Curriculum Design
Intro to Curriculum DesignIntro to Curriculum Design
Intro to Curriculum Design
 
Building a Peer Evaluation Program
Building a Peer Evaluation ProgramBuilding a Peer Evaluation Program
Building a Peer Evaluation Program
 
An Overview of Assessment Design
An Overview of Assessment DesignAn Overview of Assessment Design
An Overview of Assessment Design
 
Assessment (11)
Assessment (11)Assessment (11)
Assessment (11)
 
Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...
Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...
Rubric\'s Cube--Complimenting, Critiquing, and Challenging Student Work (NELB...
 
Assessment: Achieving improved efficiency, effectiveness, educational integri...
Assessment:Achieving improved efficiency, effectiveness, educational integri...Assessment:Achieving improved efficiency, effectiveness, educational integri...
Assessment: Achieving improved efficiency, effectiveness, educational integri...
 
Assessment 072
Assessment 072Assessment 072
Assessment 072
 
LOTW
LOTWLOTW
LOTW
 
Making learning visible_mon_25_aug_1pm
Making learning visible_mon_25_aug_1pmMaking learning visible_mon_25_aug_1pm
Making learning visible_mon_25_aug_1pm
 
Assessing learning in Instructional Design
Assessing learning in Instructional DesignAssessing learning in Instructional Design
Assessing learning in Instructional Design
 
Assessment
Assessment Assessment
Assessment
 
Authentic Assessment
Authentic AssessmentAuthentic Assessment
Authentic Assessment
 
Creating Rubrics
Creating RubricsCreating Rubrics
Creating Rubrics
 
Assessments
AssessmentsAssessments
Assessments
 
Performance-Based Assessment
Performance-Based AssessmentPerformance-Based Assessment
Performance-Based Assessment
 
Making learning visible_mon_25_aug_2.30pm
Making learning visible_mon_25_aug_2.30pmMaking learning visible_mon_25_aug_2.30pm
Making learning visible_mon_25_aug_2.30pm
 
How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...
How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...
How Can We Use Assessment Techniques to Enhance the Learning of Pareducators?...
 
Rubrics
RubricsRubrics
Rubrics
 

More from Bill Carozza, Principal, Harold Martin School

Evernote for Educators
Evernote for EducatorsEvernote for Educators
Web 2.0 Tools to Support Classroom Innovation and Professional Learning
Web 2.0 Tools to Support Classroom Innovation and Professional LearningWeb 2.0 Tools to Support Classroom Innovation and Professional Learning
Web 2.0 Tools to Support Classroom Innovation and Professional Learning
Bill Carozza, Principal, Harold Martin School
 
The Power of Blogging for Educators LESCN April 1, 2013
The Power of Blogging for Educators LESCN April 1, 2013The Power of Blogging for Educators LESCN April 1, 2013
The Power of Blogging for Educators LESCN April 1, 2013
Bill Carozza, Principal, Harold Martin School
 
The Power of Social Media for School Leaders
The Power of Social Media for School LeadersThe Power of Social Media for School Leaders
The Power of Social Media for School Leaders
Bill Carozza, Principal, Harold Martin School
 
Productivity for School Leaders
Productivity for School LeadersProductivity for School Leaders
Productivity for School Leaders
Bill Carozza, Principal, Harold Martin School
 
The Power of Social Media for Teachers
The Power of Social Media for TeachersThe Power of Social Media for Teachers
The Power of Social Media for Teachers
Bill Carozza, Principal, Harold Martin School
 
The Power of Social Media for School Leaders
The Power of Social Media for School LeadersThe Power of Social Media for School Leaders
The Power of Social Media for School Leaders
Bill Carozza, Principal, Harold Martin School
 
Social Media and School Administrators
Social Media and School AdministratorsSocial Media and School Administrators
Social Media and School Administrators
Bill Carozza, Principal, Harold Martin School
 
Technology in Education, 4 10-12
Technology in Education, 4 10-12Technology in Education, 4 10-12
Technology in Education, 4 10-12
Bill Carozza, Principal, Harold Martin School
 
Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...
Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...
Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...
Bill Carozza, Principal, Harold Martin School
 
PSU Common Core Presentation
PSU Common Core PresentationPSU Common Core Presentation
PSU Common Core Presentation
Bill Carozza, Principal, Harold Martin School
 
Podcasting Sakai Conference
Podcasting Sakai ConferencePodcasting Sakai Conference
Podcasting
PodcastingPodcasting
Web 2.0 Intro for Administrators
Web 2.0 Intro for AdministratorsWeb 2.0 Intro for Administrators
Web 2.0 Intro for Administrators
Bill Carozza, Principal, Harold Martin School
 

More from Bill Carozza, Principal, Harold Martin School (20)

Evernote for Educators
Evernote for EducatorsEvernote for Educators
Evernote for Educators
 
Web 2.0 Tools to Support Classroom Innovation and Professional Learning
Web 2.0 Tools to Support Classroom Innovation and Professional LearningWeb 2.0 Tools to Support Classroom Innovation and Professional Learning
Web 2.0 Tools to Support Classroom Innovation and Professional Learning
 
The Power of Blogging for Educators LESCN April 1, 2013
The Power of Blogging for Educators LESCN April 1, 2013The Power of Blogging for Educators LESCN April 1, 2013
The Power of Blogging for Educators LESCN April 1, 2013
 
Prod for school leaders
Prod for school leadersProd for school leaders
Prod for school leaders
 
The Power of Social Media for School Leaders
The Power of Social Media for School LeadersThe Power of Social Media for School Leaders
The Power of Social Media for School Leaders
 
Productivity for School Leaders
Productivity for School LeadersProductivity for School Leaders
Productivity for School Leaders
 
The Power of Social Media for Teachers
The Power of Social Media for TeachersThe Power of Social Media for Teachers
The Power of Social Media for Teachers
 
The Power of Social Media for School Leaders
The Power of Social Media for School LeadersThe Power of Social Media for School Leaders
The Power of Social Media for School Leaders
 
Social Media and School Administrators
Social Media and School AdministratorsSocial Media and School Administrators
Social Media and School Administrators
 
Web 2.0 for admins cmtc 2011
Web 2.0 for admins cmtc 2011Web 2.0 for admins cmtc 2011
Web 2.0 for admins cmtc 2011
 
Blogspot
BlogspotBlogspot
Blogspot
 
25 Tricks and Tips for Your Mac
25 Tricks and Tips for Your Mac25 Tricks and Tips for Your Mac
25 Tricks and Tips for Your Mac
 
Class Notes NEC Summer Inst.-Day 3
Class Notes NEC Summer Inst.-Day 3Class Notes NEC Summer Inst.-Day 3
Class Notes NEC Summer Inst.-Day 3
 
K night 6 12-12
K night 6 12-12K night 6 12-12
K night 6 12-12
 
Technology in Education, 4 10-12
Technology in Education, 4 10-12Technology in Education, 4 10-12
Technology in Education, 4 10-12
 
Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...
Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...
Legal Tweets: Balancing the Promise of Social Media and Its Implications in S...
 
PSU Common Core Presentation
PSU Common Core PresentationPSU Common Core Presentation
PSU Common Core Presentation
 
Podcasting Sakai Conference
Podcasting Sakai ConferencePodcasting Sakai Conference
Podcasting Sakai Conference
 
Podcasting
PodcastingPodcasting
Podcasting
 
Web 2.0 Intro for Administrators
Web 2.0 Intro for AdministratorsWeb 2.0 Intro for Administrators
Web 2.0 Intro for Administrators
 

Cn 11 12-12

  • 3. DQs: What are the important elements of Stage 2?
  • 4. DQs: What are the important elements of Stage 2? How does one “think like an assessor”?
  • 5. DQs: What are the important elements of Stage 2? How does one “think like an assessor”? What is evidence of desired results? (alignment with Stage 1)
  • 6. DQs: What are the important elements of Stage 2? How does one “think like an assessor”? What is evidence of desired results? (alignment with Stage 1) What is evidence of UNDERSTANDING?
  • 7. DQs: What are the important elements of Stage 2? How does one “think like an assessor”? What is evidence of desired results? (alignment with Stage 1) What is evidence of UNDERSTANDING? How can my students and I "change the world" through performance assessments?
  • 8. DQs: What are the important elements of Stage 2? How does one “think like an assessor”? What is evidence of desired results? (alignment with Stage 1) What is evidence of UNDERSTANDING? How can my students and I "change the world" through performance assessments? What does assessment look like school and district wide?
  • 9. DQs: What are the important elements of Stage 2? How does one “think like an assessor”? What is evidence of desired results? (alignment with Stage 1) What is evidence of UNDERSTANDING? How can my students and I "change the world" through performance assessments? What does assessment look like school and district wide? How can I organize what work is left?
  • 10. Summary of the rest... November 12-Finish up on Stage 2 November 19-Stage 3 November 26-Work session-wrap up week December 3 and 10: Presentations
  • 11. STAGE TWO Determining acceptable evidence Thinking Like an Assessor
  • 13. Stages of Backward Design Stage One: Identifying desired results (such as enduring understandings, essential questions and knowledge objectives).
  • 14. Stages of Backward Design Stage One: Identifying desired results (such as enduring understandings, essential questions and knowledge objectives). Stage Two: Determining acceptable evidence to assess and to evaluate student achievement of desired results.
  • 15. Stages of Backward Design Stage One: Identifying desired results (such as enduring understandings, essential questions and knowledge objectives). Stage Two: Determining acceptable evidence to assess and to evaluate student achievement of desired results. Stage Three: Designing learning activities to promote student mastery and success on assessment tasks.
  • 16. 3 Basic Questions in order to think like an assessor:
  • 17. 3 Basic Questions in order to think like an assessor: What evidence?
  • 18. 3 Basic Questions in order to think like an assessor: What evidence? What tasks?
  • 19. 3 Basic Questions in order to think like an assessor: What evidence? What tasks? Does Stage 2 align with Stage 1?
  • 20.
  • 21. Not a natural process -as we are focused on what are the results?
  • 22. Not a natural process -as we are focused on what are the results? See figure 7.3 on p. 151.
  • 23. Not a natural process -as we are focused on what are the results? See figure 7.3 on p. 151. The UbD template “short circuits” the tendency to think like “an activity designer.”
  • 24. From snapshot to scrapbook
  • 25. From snapshot to scrapbook Figure 7.4 and 7.5, pages 152-3.
  • 26. From snapshot to scrapbook Figure 7.4 and 7.5, pages 152-3. What types of assessments are there?
  • 27.
  • 28. Some basic rules regarding assessment
  • 29. Some basic rules regarding assessment You’re not generating grades.
  • 30. Some basic rules regarding assessment You’re not generating grades. Align assessment with Stage 1
  • 31. Some basic rules regarding assessment You’re not generating grades. Align assessment with Stage 1 Ask yourself: What kinds of evidence do we need?
  • 32. Some basic rules regarding assessment You’re not generating grades. Align assessment with Stage 1 Ask yourself: What kinds of evidence do we need? Think like an assessor, not an activity designer.
  • 33. Some basic rules regarding assessment You’re not generating grades. Align assessment with Stage 1 Ask yourself: What kinds of evidence do we need? Think like an assessor, not an activity designer. Snapshot to scrapbook
  • 34. Some basic rules regarding assessment You’re not generating grades. Align assessment with Stage 1 Ask yourself: What kinds of evidence do we need? Think like an assessor, not an activity designer. Snapshot to scrapbook Authentic whenever possible
  • 35. Some basic rules regarding assessment You’re not generating grades. Align assessment with Stage 1 Ask yourself: What kinds of evidence do we need? Think like an assessor, not an activity designer. Snapshot to scrapbook Authentic whenever possible Use 6 facets when designing assessments and be sure the assessment align with EQs and EUs.
  • 37. The Performance Task Designing around problems: John Dewey: “Does the question naturally suggest itself within some situation or personal experience? Or is it an aloof thing…? Is it the sort of thing that would arouse conversation and engage experimentation out side of school? Or is it made a problem for the pupil only because he cannot get the required mark or be promoted or win the teacher’s approval unless he deals with it? (1916)
  • 39. GRASPS G=Goals from the real world.
  • 40. GRASPS G=Goals from the real world. R=Roles that are authentic and based in reality.
  • 41. GRASPS G=Goals from the real world. R=Roles that are authentic and based in reality. A=Audiences to whom students will present final products and performances
  • 42. GRASPS G=Goals from the real world. R=Roles that are authentic and based in reality. A=Audiences to whom students will present final products and performances S=Situations involving a real-world conflict to be resolved, decision to be made, investigation to be completed
  • 43. GRASPS G=Goals from the real world. R=Roles that are authentic and based in reality. A=Audiences to whom students will present final products and performances S=Situations involving a real-world conflict to be resolved, decision to be made, investigation to be completed P=Products and Performances culminating from the study
  • 44. GRASPS G=Goals from the real world. R=Roles that are authentic and based in reality. A=Audiences to whom students will present final products and performances S=Situations involving a real-world conflict to be resolved, decision to be made, investigation to be completed P=Products and Performances culminating from the study S=Standards for evaluating project-based products and performances
  • 45. Work for the week Work for the week-both classwork and homework: Listen to the Podcast and follow along with slides. Be sure to watch the NYC Urban Academy video. After watching “Teaching 2030”, describe in your own words the five major points the group makes. Make a Google Doc and share with Bill. (wcarozza@nec.edu) Read Chapter 7 in the text-Thinking Like an Assessor Develop a GRASPS-share a Google Doc with Bill similar to the blank one on p. 172 in workbook.
  • 47. An exemplary GRASPS with EQs and EUs
  • 48.
  • 49.
  • 50. Looking at Assessment from a School or District Perspective
  • 51.
  • 52.
  • 53.
  • 55. General guidelines for Assessment tests to assess We need to look at more than just objective knowledge and skill.
  • 56. General guidelines for Assessment tests to assess We need to look at more than just objective knowledge and skill. How do we design performances that enable us to make precise judgment about the different parts of the performance?
  • 57. General guidelines for Assessment tests to assess We need to look at more than just objective knowledge and skill. How do we design performances that enable us to make precise judgment about the different parts of the performance? Try to have parallel versions of the same content across different assessment formats.
  • 58. General guidelines for Assessment tests to assess We need to look at more than just objective knowledge and skill. How do we design performances that enable us to make precise judgment about the different parts of the performance? Try to have parallel versions of the same content across different assessment formats. Given that a single application or product may or may not link to larger goals, ask students to “show their work”, give reasons for answers and show connections to larger ideas in the answers.
  • 59. Three Broad Types of Assessment
  • 60. Three Broad Types of Assessment Secured Assessments
  • 61. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA
  • 62. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests
  • 63. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers.
  • 64. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria.
  • 65. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria. Classroom embedded Assessments
  • 66. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria. Classroom embedded Assessments Coming out of classroom activities (Stage 2)
  • 67. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria. Classroom embedded Assessments Coming out of classroom activities (Stage 2) Composite Records
  • 68. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria. Classroom embedded Assessments Coming out of classroom activities (Stage 2) Composite Records Best Works Portfolios (managed by students)
  • 69. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria. Classroom embedded Assessments Coming out of classroom activities (Stage 2) Composite Records Best Works Portfolios (managed by students) Progress Portfolios (managed by teachers)
  • 70. Three Broad Types of Assessment Secured Assessments No help is given, e.g. NECAP, NWEA Norm Referenced vs. Criterion Referenced tests NRTs: classify students from high to low in large sample-compared to other peers. CRTs: student performance based against a criteria. Classroom embedded Assessments Coming out of classroom activities (Stage 2) Composite Records Best Works Portfolios (managed by students) Progress Portfolios (managed by teachers) Accountability Portfolios (e.g. KY and VT)
  • 71. Assessment within a school or district-making data driven decisions-examples
  • 72. Assessment within a school or district-making data driven decisions-examples Program and Curriculum Development
  • 73. Assessment within a school or district-making data driven decisions-examples Program and Curriculum Development Making decisions on programs
  • 74. Assessment within a school or district-making data driven decisions-examples Program and Curriculum Development Making decisions on programs Child Specific
  • 75. Assessment within a school or district-making data driven decisions-examples Program and Curriculum Development Making decisions on programs Child Specific Making decisions about children
  • 76. Assessment within a school or district-making data driven decisions-examples Program and Curriculum Development Making decisions on programs Child Specific Making decisions about children School Climate
  • 77. Assessment within a school or district-making data driven decisions-examples Program and Curriculum Development Making decisions on programs Child Specific Making decisions about children School Climate Making decisions about school climate
  • 78. An Assessment Philosophy
  • 79. An Assessment Philosophy Assessment values what is taught and what students learn.
  • 80. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student.
  • 81. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student. Assessment addresses the process and the product.
  • 82. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student. Assessment addresses the process and the product. Assessment is both formal (e.g. standardized or published tests) and on going (e.g. observation and anecdotal records
  • 83. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student. Assessment addresses the process and the product. Assessment is both formal (e.g. standardized or published tests) and on going (e.g. observation and anecdotal records Assessment must be developmentally appropriate.
  • 84. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student. Assessment addresses the process and the product. Assessment is both formal (e.g. standardized or published tests) and on going (e.g. observation and anecdotal records Assessment must be developmentally appropriate. Assessment exists in multiple forms.
  • 85. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student. Assessment addresses the process and the product. Assessment is both formal (e.g. standardized or published tests) and on going (e.g. observation and anecdotal records Assessment must be developmentally appropriate. Assessment exists in multiple forms. Assessment is used to evaluate programs and individual student progress.
  • 86. An Assessment Philosophy Assessment values what is taught and what students learn. Assessment drives instruction for every student. Assessment addresses the process and the product. Assessment is both formal (e.g. standardized or published tests) and on going (e.g. observation and anecdotal records Assessment must be developmentally appropriate. Assessment exists in multiple forms. Assessment is used to evaluate programs and individual student progress. Assessment honors a “value-added” approach.
  • 87. ASSESSMENT MATRIX * HOPKINTON SCHOOL DISTRICT 2005-0 6 GRADE ASSESSMENT DESCRIPTION K Observation Survey An early literacy assessment that assesses the following: • Text Reading Level • Letter Identification • Concepts About Print o D e m onstrate understanding of concepts about book handling and conventions of printed language while teacher reads a book • Word Test o Read a list of 20 words • Writing Vocabulary o S t u dent has 10 minutes to write all the words he/she knows • Hearing and Recording Sounds in Words o S t u dent writes a dictated sentence which is scored by the number of sounds hea r d Write Traits Writing Assesses six important “traits” of writing: * Idea Development Benchmark * Organization * Voice * Word Choice * Sentence Fluency * Conventions 1 DRA • The Developmental Reading Assessment provides teachers with a method for assessing and documenting students' development as readers over time within a literature-based instructional reading program. • The DRA is designed to be used in elementary classrooms with rich literate environments. • The assessments are conducted during one-on-one reading conferences as children read specially selected assessment texts. A set of 20 stories, which
  • 88. 2 DRA See above Write Traits Writing See above Benchmark MAP testing in The NWEA (Northwest Evaluation Association) MAP test provides teachers and Reading and Math parents with a standardized assessment of a student’s ability. You can receive significant results the same day (and much more not long after) and use the information to drive your instruction. The test is taken by students on the computer. This assessment is given both in the fall and spring. 3 DRA See above Write Traits Writing See above Benchmark MAP testing in See above Reading and Math NECAP State assessment in Math and Reading/Language Arts-tested in the fall 4 DRA See above Write Traits Writing See above Benchmark MAP testing in See above Reading and Math NECAP State assessment in Math and Reading/Language Arts-tested in the fall NAEP National Assessment of Educational Progress-known as the “Nation’s Report Card”. 5 DRA See above Write Traits Writing See above Benchmark MAP testing in See above Reading and Math NECAP State Assessment in Math, Reading/Language Arts, and in Writing-tested in the fall
  • 89. 10 Selected MAP Primarily for Math IIs. testing NHEIAP State Assessment in Math and Reading/Language Arts-tested in the spring PSAT/NMSQT PSAT/NMSQT stands for Preliminary SAT/National Merit Scholarship Qualifying Test. It's a standardized test that provides firsthand practice for the SAT. World Language Assessment 11 SAT The Scholastic Aptitude Test (SAT) is a standard submission test that is used by colleges around the country to help identify students who will succeed at a given college. Subject Tests (SAT Subject Tests (formerly SAT II: Subject Tests) are designed to measure student II) knowledge and skills in particular subject areas, as well as the ability to apply that knowledge. Advanced Taken by students in A.P. courses in order to receive college credit. Placement Tests PSAT/NMSQT PSAT/NMSQT stands for Preliminary SAT/National Merit Scholarship Qualifying Test. It's a standardized test that provides firsthand practice for the SAT. It also provides an opportunity for National Merit scholarship programs. ACT The ACT assesses high school students' general educational development and their ability to complete college-level work. World Language Assessment
  • 90. MAP testing Examples from the Internet -Quantitative Assessment
  • 92. Lexiles A system for measuring the difficulty of text.
  • 93. Lexiles A system for measuring the difficulty of text. Reading reports give Lexile scores
  • 94. Lexiles A system for measuring the difficulty of text. Reading reports give Lexile scores Check out Lexile.com
  • 95.
  • 96.
  • 97.
  • 98. Uses of MAP data
  • 99. Uses of MAP data Benchmarking student performance in reading and math
  • 100. Uses of MAP data Benchmarking student performance in reading and math Entry points of DI in reading and math
  • 101. Uses of MAP data Benchmarking student performance in reading and math Entry points of DI in reading and math Identification of PD for school/teachers
  • 102. Uses of MAP data Benchmarking student performance in reading and math Entry points of DI in reading and math Identification of PD for school/teachers Construction of IEP goals and objectives
  • 103. Uses of MAP data Benchmarking student performance in reading and math Entry points of DI in reading and math Identification of PD for school/teachers Construction of IEP goals and objectives Cross comparison data with reading benchmarks
  • 104. Uses of MAP data Benchmarking student performance in reading and math Entry points of DI in reading and math Identification of PD for school/teachers Construction of IEP goals and objectives Cross comparison data with reading benchmarks Selection of readability levels
  • 105. Uses of MAP data Benchmarking student performance in reading and math Entry points of DI in reading and math Identification of PD for school/teachers Construction of IEP goals and objectives Cross comparison data with reading benchmarks Selection of readability levels Conversation starters with parents, especially at parent conferences
  • 106.
  • 107. Part of assessment criteria to support pre- algebra decision
  • 108. Part of assessment criteria to support pre- algebra decision Used in RTI decisions, pre-referral meetings, Child Study
  • 109. Part of assessment criteria to support pre- algebra decision Used in RTI decisions, pre-referral meetings, Child Study Part of criteria to support admission to enrichment reading
  • 110. Part of assessment criteria to support pre- algebra decision Used in RTI decisions, pre-referral meetings, Child Study Part of criteria to support admission to enrichment reading Conversation at PD planning meetings with Principal and teacher.
  • 111. Part of assessment criteria to support pre- algebra decision Used in RTI decisions, pre-referral meetings, Child Study Part of criteria to support admission to enrichment reading Conversation at PD planning meetings with Principal and teacher. Part of the placement puzzle
  • 112. Digital Portfolios Qualitative Assessment
  • 114. Digital Portfolios  Based on research from the Coalition of Essential Schools
  • 115. Digital Portfolios  Based on research from the Coalition of Essential Schools  Started in 1993 with districts in New York, New Hampshire, and Kentucky
  • 116. Digital Portfolios  Based on research from the Coalition of Essential Schools  Started in 1993 with districts in New York, New Hampshire, and Kentucky  The Richer Picture software currently in use for elementary, secondary, and professional
  • 118. Digital Portfolios  We can use technology to create a richer picture of a student’s abilities
  • 119. Digital Portfolios  We can use technology to create a richer picture of a student’s abilities  Portfolio can contain a set of work that demonstrates:
  • 120. Digital Portfolios  We can use technology to create a richer picture of a student’s abilities  Portfolio can contain a set of work that demonstrates:  a student’s accomplishments AND
  • 121. What goes in a portfolio?
  • 122. What goes in a portfolio?  Samples of student work
  • 123. What goes in a portfolio?  Samples of student work  Information to put the work in context
  • 124. What goes in a portfolio?  Samples of student work  Information to put the work in context  Reflection on the work
  • 125. What goes in a portfolio?  Samples of student work  Information to put the work in context  Reflection on the work  Portfolios represent a subset of student work
  • 126. What goes in a portfolio- specifically?
  • 127. What goes in a portfolio- specifically? Collect
  • 128. What goes in a portfolio- specifically? Collect 2 – 4 entries from each course
  • 129. What goes in a portfolio- specifically? Collect 2 – 4 entries from each course Each entry has student work, a summary and a reflection
  • 130. What goes in a portfolio- specifically? Collect 2 – 4 entries from each course Each entry has student work, a summary and a reflection Select
  • 131. What goes in a portfolio- specifically? Collect 2 – 4 entries from each course Each entry has student work, a summary and a reflection Select Student determines which entries provide the best evidence
  • 132. What goes in a portfolio- specifically? Collect 2 – 4 entries from each course Each entry has student work, a summary and a reflection Select Student determines which entries provide the best evidence Reflect
  • 135. Essential Questions Expectations What should students know and do?
  • 136. Essential Questions Expectations What should students know and do? What purpose does the portfolio serve?
  • 137. Essential Questions Expectations What should students know and do? What purpose does the portfolio serve? Who is the audience?
  • 138. Essential Questions Expectations What should students know and do? What purpose does the portfolio serve? Who is the audience? Entries
  • 139. Essential Questions Expectations What should students know and do? What purpose does the portfolio serve? Who is the audience? Entries How can students exhibit the expectations?
  • 140. Essential Questions Expectations What should students know and do? What purpose does the portfolio serve? Who is the audience? Entries How can students exhibit the expectations? How do we ensure tasks are “portfolio worthy” –
  • 143. Essential Questions Assessment How do we decide what’s good?
  • 144. Essential Questions Assessment How do we decide what’s good? How do we give consistent messages?
  • 145. Essential Questions Assessment How do we decide what’s good? How do we give consistent messages? School Structures
  • 146. Essential Questions Assessment How do we decide what’s good? How do we give consistent messages? School Structures Who does what? When does it happen?
  • 147. Essential Questions Assessment How do we decide what’s good? How do we give consistent messages? School Structures Who does what? When does it happen? Expectations for teachers, students, administrators
  • 148. Essential Questions Assessment How do we decide what’s good? How do we give consistent messages? School Structures Who does what? When does it happen? Expectations for teachers, students, administrators How do we use technology?
  • 149. Why Digital Portfolios for your School or Classroom?
  • 150. Why Digital Portfolios for your School or Classroom? It helps to balance the quantitative side of assessment with qualitative work.
  • 151. Why Digital Portfolios for your School or Classroom? It helps to balance the quantitative side of assessment with qualitative work. It honors student work product not just a score on a test.
  • 152. Why Digital Portfolios for your School or Classroom? It helps to balance the quantitative side of assessment with qualitative work. It honors student work product not just a score on a test. Most importantly…it is a portfolio first; the technology is the vehicle.
  • 153. How Could DP’s Be Useful for your school or classroom?
  • 154. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.!
  • 155. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! !
  • 156. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! ! A tool for capturing, storing, and examining student work.!
  • 157. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! ! A tool for capturing, storing, and examining student work.! Allows students to help explain their understanding-application of learning.!
  • 158. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! ! A tool for capturing, storing, and examining student work.! Allows students to help explain their understanding-application of learning.! Tool for us to drill down into specific learning outcomes and issues, e.g. examination of reading video to highlight a student's difficulties reading.!
  • 159. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! ! A tool for capturing, storing, and examining student work.! Allows students to help explain their understanding-application of learning.! Tool for us to drill down into specific learning outcomes and issues, e.g. examination of reading video to highlight a student's difficulties reading.! Capturing artwork and other 3 dimensional work.
  • 160. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! ! A tool for capturing, storing, and examining student work.! Allows students to help explain their understanding-application of learning.! Tool for us to drill down into specific learning outcomes and issues, e.g. examination of reading video to highlight a student's difficulties reading.! Capturing artwork and other 3 dimensional work. Capturing music and p.e. work as well-the arts in general.!
  • 161. How Could DP’s Be Useful for your school or classroom? Helps to document what students need to know and be able to do.! To use in parent conferences-conversation starter-helps parents understand educational jargon.! ! A tool for capturing, storing, and examining student work.! Allows students to help explain their understanding-application of learning.! Tool for us to drill down into specific learning outcomes and issues, e.g. examination of reading video to highlight a student's difficulties reading.! Capturing artwork and other 3 dimensional work. Capturing music and p.e. work as well-the arts in general.! Highlights students’ academic and emotional behavior in areas outside of the classroom.! !
  • 162. How Could DP’s Be Useful at Your School or Classroom? (continued)
  • 163. How Could DP’s Be Useful at Your School or Classroom? (continued)
  • 164. How Could DP’s Be Useful at Your School or Classroom? (continued) Getting students to "own" their learning-self reflection.
  • 165. How Could DP’s Be Useful at Your School or Classroom? (continued) Getting students to "own" their learning-self reflection. Teachers better understand how their new students learn thus aiding transition-keeps kids from starting all over again and makes their elementary years much more fluid
  • 166. How Could DP’s Be Useful at Your School or Classroom? (continued) Getting students to "own" their learning-self reflection. Teachers better understand how their new students learn thus aiding transition-keeps kids from starting all over again and makes their elementary years much more fluid More manageable than using paper over
  • 167. What can be put in the DP?
  • 168. What can be put in the DP? Scanned work (e.g. student writing)
  • 169. What can be put in the DP? Scanned work (e.g. student writing) Photos
  • 170. What can be put in the DP? Scanned work (e.g. student writing) Photos Online rubrics
  • 171. What can be put in the DP? Scanned work (e.g. student writing) Photos Online rubrics Video
  • 172. What can be put in the DP? Scanned work (e.g. student writing) Photos Online rubrics Video Anything that can be put on a web site
  • 173. Who can view it? Teachers and students Parents at conferences Parents at home
  • 174.
  • 175.
  • 176.
  • 177.
  • 178.
  • 179.
  • 180.
  • 181.
  • 182. Homework for next week Complete reflections on Stage 1 and 2 which includes and integrates School Wide Assessment. At least a page for both. Share a Google Doc with me for each. (two docs) Continue working on both Stage 1 and 2. Begin brainstorming lesson plan ideas. Read: (links on Diigo under tag “UbD”.) UbD Introduction UbD in a Nutshell
  • 183. 3X5 What was the big point you learned in class today? What is the main unanswered question you leave class with today?

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. \n
  63. \n
  64. \n
  65. \n
  66. \n
  67. \n
  68. \n
  69. \n
  70. \n
  71. \n
  72. \n
  73. \n
  74. \n
  75. \n
  76. \n
  77. \n
  78. \n
  79. \n
  80. \n
  81. \n
  82. \n
  83. \n
  84. \n
  85. \n
  86. \n
  87. \n
  88. \n
  89. \n
  90. \n
  91. \n
  92. \n
  93. \n
  94. \n
  95. \n
  96. \n
  97. \n
  98. \n
  99. \n
  100. \n
  101. \n
  102. \n
  103. \n
  104. \n
  105. \n
  106. \n
  107. \n
  108. \n
  109. \n
  110. \n
  111. \n
  112. \n
  113. \n
  114. \n
  115. \n
  116. \n
  117. \n
  118. \n
  119. \n
  120. \n
  121. \n
  122. \n
  123. \n
  124. \n
  125. \n
  126. \n
  127. \n
  128. \n
  129. \n
  130. \n
  131. \n
  132. \n
  133. \n
  134. \n
  135. \n
  136. \n
  137. \n
  138. \n
  139. \n
  140. \n
  141. \n
  142. \n
  143. \n
  144. \n
  145. \n
  146. \n
  147. \n
  148. \n
  149. \n
  150. \n
  151. \n
  152. \n
  153. \n
  154. \n
  155. \n
  156. \n
  157. \n
  158. \n