While you wait, please…
1. Make/Set up a Name Tent
2. Think about: How do we know that they
(students) know?
TETR 9031 Designing Learning
TENTATIVE Course Outline
Assignment
1 Jan 13 Introductions and course overview
2 Jan 20 ID Models, Lesson Planning
3 Jan 27 Lesson Planning continued
4 Feb 3 Learning Outcomes and Objectives LP Cont’d
5 Feb 10 Strategies, methods and channels Assignment #1 Due for 15%
6 Feb 17 Strategies, methods and channels
7 Feb 24 Strategies, methods and channels Assignment #2 Due for 15%
8 Mar 3 Assessment & Evaluation
9 Mar 10 Practical design problems
Pairs work on Lesson Plan Assignment #3
10 Mar 17 Practical design problems Assignment #3 presentations due for 30%
11 Mar 24 Practical design problems
Group work on Course Proposal Assignment #4
12 Mar 31 Practical design problems
13 Apr 7 Wrap up Assignment #4 presentations due for
30%
How does this apply to ID?
Assessment
5
Evaluation and Assessment
Assessment: A process to judge
improvement in quality, quantity or
speed as a result of learning.
Evaluation: A process to determine
the worth, value or significance of a
learning intervention
The Kirkpatrick Model
Level 1: Reactions
Level 2: Learning
Level 3: Behaviours
Level 4: Results
Level 5?: ROI
What’s your preferred assessment method?
General Approaches
• Summative (At end to grade) &
Formative (During to learn)
• Instructor v. Peers v. Self v.3rd
party
• Norm Referenced Testing (NRT)
v. Criterion Referenced Testing
(CRT)
Techniques
• Knowledge tests
• MCQ
• T/F
• Matching
• SA
• FITB
• Essay/Oral
• SKILL/BEHAVIOUR tests
• Demos
• Checklists and Rating Scales
• Anecdotal
• Self-Reports
• Portfolios
• Work documents
Formative and Summative
Formative evaluation is done by
the chef while cooking
Summative evaluation is done by
the guests while eating
How is this like instructional design?
4/17/2017 8
NRT and CRT assessments
• Norm-Referenced Tests
(NRT)
– Ranks test-takers, each
compared to all the
others
• Criterion-Referenced Tests
(CRT)
– Test items matched to
“objective” standards or
criteria
4/17/2017 9
How do you make a test to assess learning?
• What’s the performance called for by the LO? K, S or A?
– Select a test type from the options in the table below
– Are there practical issues that prevent using the ideal test type?
• Draft criterion-referenced test items.
– 2-3 per LO
– Should the LO add any conditions? If so, be sure your test items include them.
– Is there a standard to meet? If so, let the students know.
Test Types Knowledge Skills Attitudes
Role play or performance test YES YES
Interview or survey (of students and others) YES YES YES
On-the-job observations with checklists and
rating scales
YES
Essays & oral presentation YES YES
Objective tests: MCQ, T/F, SA, FITB, Matching YES
Work Products YES YES
Others?
Checklists and observation forms for testing
performance: What can you test for?
• Quantity
– How many? How often?
• Quality
– How well? How accurately?
• Speed
– How fast?
For example, here’s an observation form to be used by a restaurant manager training a
new employee who has learned how to prepare bagels.
Bagel
sliced?
2 equal halves? Smooth
surfaces?
Injuries?
Yes/No Yes Yes with 1
exception
Yes Yes but not
trainee’s fault
Rating (1 to 5) 4 overall
# observed 15 correctly prepared over 3 different observations, 1 incorrectly
done
Average time per bagel 30 seconds
Questions???????
1. In general, how would you test the participants in
each of these? On K, S, A or a combination?
A. A course designed to teach the use of a PC
B. A course to improve supervisory feedback to employees
1. Why should organizations test learners?
2. Why shouldn’t they?
1. What are some of the practical and organizational
barriers to testing? How might an ID do to overcome
them?
Small Groups
Design Problems carried over from
Week 7
Anna Sabramowicz – How to Change a Person’s Attitude thru
Lesson Design
• Watch this 3:35 minute video
• ANSWER THESE QUESTIONS
• What are the steps that your design has
to have to change a person’s attitude?
• What instructional methods would you
use in a F2F class? How about a Virtual
Class (VC)? Thru e-learning alone?
Pairs
Which instructional techniques would you use to teach this
lesson?
• Read the Tolerance handout
• Assume this handout was used by a
teacher as a rough lesson plan outline
• Assume the audience is young, diverse
and just entering the workforce
• Answer these questions:
• What do you think the LO should be?
• What would you put in the:
• Intro?
• Content Presentation?
• Conclusion?
• What would you change for a more
mature or less diverse or more
experienced audience?
Source: The Virtues Project Educator’s Guide, 2000, Linda Kavelin Popov
Individuals
Which instructional techniques would you use
to teach this lesson?
http://youtu.be/ruu2U3YoZ2Y
• What is the domain, K, S, or A?
• What do you think the LO should be?
• What would you put in the:
• Content Presentation?
• What techniques would you use to teach it?
• Finished early? How would you close the class? How would you intro it?
Individuals
Which instructional techniques would you use
to teach this lesson?
http://youtu.be/AEIn3T6nDAo
• What is the domain, K, S, or A?
• What do you think the LO should be?
• What would you put in the:
• Content Presentation?
• What techniques would you use to teach it?
• Finished early? How would you close the class? How would you intro it?
Individuals
Which instructional techniques would you use
to teach this lesson?
• What is the domain, K, S, or A?
• What do you think the LO should be?
• What would you put in the:
• Content Presentation?
• What techniques would you use to teach it?
• Finished early? How would you close the class? How would you intro it?
Individuals
• Summary
– Best Summaries Activity
• Dive Deeper
– Watch for links to sites and documents on
Blackboard
• What’s happening next week?
• Q & Eh?
Before we go…

Gbc id session 8 Assessment & Evaluation

  • 1.
    While you wait,please… 1. Make/Set up a Name Tent 2. Think about: How do we know that they (students) know?
  • 2.
    TETR 9031 DesigningLearning TENTATIVE Course Outline Assignment 1 Jan 13 Introductions and course overview 2 Jan 20 ID Models, Lesson Planning 3 Jan 27 Lesson Planning continued 4 Feb 3 Learning Outcomes and Objectives LP Cont’d 5 Feb 10 Strategies, methods and channels Assignment #1 Due for 15% 6 Feb 17 Strategies, methods and channels 7 Feb 24 Strategies, methods and channels Assignment #2 Due for 15% 8 Mar 3 Assessment & Evaluation 9 Mar 10 Practical design problems Pairs work on Lesson Plan Assignment #3 10 Mar 17 Practical design problems Assignment #3 presentations due for 30% 11 Mar 24 Practical design problems Group work on Course Proposal Assignment #4 12 Mar 31 Practical design problems 13 Apr 7 Wrap up Assignment #4 presentations due for 30%
  • 3.
    How does thisapply to ID?
  • 4.
  • 5.
    5 Evaluation and Assessment Assessment:A process to judge improvement in quality, quantity or speed as a result of learning. Evaluation: A process to determine the worth, value or significance of a learning intervention The Kirkpatrick Model Level 1: Reactions Level 2: Learning Level 3: Behaviours Level 4: Results Level 5?: ROI
  • 6.
    What’s your preferredassessment method? General Approaches • Summative (At end to grade) & Formative (During to learn) • Instructor v. Peers v. Self v.3rd party • Norm Referenced Testing (NRT) v. Criterion Referenced Testing (CRT) Techniques • Knowledge tests • MCQ • T/F • Matching • SA • FITB • Essay/Oral • SKILL/BEHAVIOUR tests • Demos • Checklists and Rating Scales • Anecdotal • Self-Reports • Portfolios • Work documents
  • 7.
    Formative and Summative Formativeevaluation is done by the chef while cooking Summative evaluation is done by the guests while eating How is this like instructional design?
  • 8.
    4/17/2017 8 NRT andCRT assessments • Norm-Referenced Tests (NRT) – Ranks test-takers, each compared to all the others • Criterion-Referenced Tests (CRT) – Test items matched to “objective” standards or criteria
  • 9.
    4/17/2017 9 How doyou make a test to assess learning? • What’s the performance called for by the LO? K, S or A? – Select a test type from the options in the table below – Are there practical issues that prevent using the ideal test type? • Draft criterion-referenced test items. – 2-3 per LO – Should the LO add any conditions? If so, be sure your test items include them. – Is there a standard to meet? If so, let the students know. Test Types Knowledge Skills Attitudes Role play or performance test YES YES Interview or survey (of students and others) YES YES YES On-the-job observations with checklists and rating scales YES Essays & oral presentation YES YES Objective tests: MCQ, T/F, SA, FITB, Matching YES Work Products YES YES Others?
  • 10.
    Checklists and observationforms for testing performance: What can you test for? • Quantity – How many? How often? • Quality – How well? How accurately? • Speed – How fast? For example, here’s an observation form to be used by a restaurant manager training a new employee who has learned how to prepare bagels. Bagel sliced? 2 equal halves? Smooth surfaces? Injuries? Yes/No Yes Yes with 1 exception Yes Yes but not trainee’s fault Rating (1 to 5) 4 overall # observed 15 correctly prepared over 3 different observations, 1 incorrectly done Average time per bagel 30 seconds
  • 11.
    Questions??????? 1. In general,how would you test the participants in each of these? On K, S, A or a combination? A. A course designed to teach the use of a PC B. A course to improve supervisory feedback to employees 1. Why should organizations test learners? 2. Why shouldn’t they? 1. What are some of the practical and organizational barriers to testing? How might an ID do to overcome them? Small Groups
  • 12.
    Design Problems carriedover from Week 7
  • 13.
    Anna Sabramowicz –How to Change a Person’s Attitude thru Lesson Design • Watch this 3:35 minute video • ANSWER THESE QUESTIONS • What are the steps that your design has to have to change a person’s attitude? • What instructional methods would you use in a F2F class? How about a Virtual Class (VC)? Thru e-learning alone? Pairs
  • 14.
    Which instructional techniqueswould you use to teach this lesson? • Read the Tolerance handout • Assume this handout was used by a teacher as a rough lesson plan outline • Assume the audience is young, diverse and just entering the workforce • Answer these questions: • What do you think the LO should be? • What would you put in the: • Intro? • Content Presentation? • Conclusion? • What would you change for a more mature or less diverse or more experienced audience? Source: The Virtues Project Educator’s Guide, 2000, Linda Kavelin Popov Individuals
  • 15.
    Which instructional techniqueswould you use to teach this lesson? http://youtu.be/ruu2U3YoZ2Y • What is the domain, K, S, or A? • What do you think the LO should be? • What would you put in the: • Content Presentation? • What techniques would you use to teach it? • Finished early? How would you close the class? How would you intro it? Individuals
  • 16.
    Which instructional techniqueswould you use to teach this lesson? http://youtu.be/AEIn3T6nDAo • What is the domain, K, S, or A? • What do you think the LO should be? • What would you put in the: • Content Presentation? • What techniques would you use to teach it? • Finished early? How would you close the class? How would you intro it? Individuals
  • 17.
    Which instructional techniqueswould you use to teach this lesson? • What is the domain, K, S, or A? • What do you think the LO should be? • What would you put in the: • Content Presentation? • What techniques would you use to teach it? • Finished early? How would you close the class? How would you intro it? Individuals
  • 18.
    • Summary – BestSummaries Activity • Dive Deeper – Watch for links to sites and documents on Blackboard • What’s happening next week? • Q & Eh? Before we go…