SlideShare a Scribd company logo
1 of 33
Download to read offline
Session 3: Principles of Assessment:
Validity and Reliability
Professor Jim Tognolini
Introduction to Modern Assessment
Theory: A basis for all assessments
During this session we will
•define reliability.
•define measurement error.
•examine the sources of measurement error.
•define validity and identify threats to validity.
•build assessment frameworks
•operationalise frameworks with Tables of Specification.
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
The reliability of results gives the extent to which the results are
consistent or error-free. The concept of reliability is closely associated with
the idea of consistency.
Reliability is not an all or nothing concept; there can exist degrees of
reliability.
How similar are results if students are assessed at different times? How
similar are results if students are assessed with a different sample of
equivalent tasks? How similar are results if essays have been marked by
different markers.
Reliability
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Measurement Error
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Sources of Measurement Error
The following are some of the sources of measurement error
1. Test taking skills
2. Comprehension of instructions
3. Sampling variance of items
4. Temporary factors such as health; fatigue; motivation; testing
conditions.
5. Memory fluctuations
6. Marking bias (especially in essays)
7. Guessing
8. Item types
The aim for test developers is to identify sources of measurement
error and minimise their impact.
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Validity
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Validity
The validity of the results if a test can best be defined as the extent to
which the results of a test measure what they purport to measure.
It is the interpretation (including inferences and decisions) that is
validated, not the test or the test score.
Messick also argued (1989) that validation can include the evaluation of
the consequences of the test; are the specific benefits likely to be
realised?
In 1999 the Standards (AERA, APA and NCME) suggested that validation
can be viewed as developing scientifically sound validity arguments to
support the intended interpretation of test scores and their relevance to
the proposed use.
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Threats to Validity
1. Factors in the test itself
I. Unclear direction (e.g. how to respond to guessing; recording
answers).
II. Reading vocab and sentence structure is too difficult
III. Inappropriate level of difficulty of test items (e.g. guessing)
IV. Poorly constructed test items
V. Ambiguity
VI. Test items (tasks) inappropriate for content being assessed
VII.Test too short
VIII.Improper arrangement of items
IX. Identifiable pattern of answers
2. Factors in test administration and scoring
I. Insufficient time
II. Cheating
III. Unreliable scoring
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Reliability is a necessary but insufficient condition for validity.
Relationship between Validity and Reliability
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Some basic assessment theory
• Validity and reliability are not deterministic – maximise
validity and reliability
• Validity is paramount
• Ways to minimise threats to validity and reliability
 Breadth of material sampled – increase validity
 Guessing
 Quality of items
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Assessment frameworks
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Preliminary Questions
• Why are we assessing?
• What are we assessing?
• What is the most appropriate way to assess these outcomes?
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Assessment framework
Definition of construct
Domains/strands
Sub-domains/sub-
strands
Outcomes/content
standards
…
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Example 1 - Mathematics
Mathematics
Domains/strands
Sub-domains/sub-
strands
Outcomes/content
standards
…
Number
Addition &
subtraction
Multiplication and
Division
Fractions and
Decimals
Measurement
Space
Chance
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Example 1 - Mathematics
Mathematics
Domains/str
ands
Sub-domains/sub-
strands
Outcomes/content
standards
Progress levels
Number
Addition &
subtraction
Students develop facility
with number facts and
computation with larger
numbers in addition and
subtraction and an
appreciation of the
relationship between those
facts
Early Stage 1
Combines,
separates and
compares
collections of
objects, describes
using everyday
language and
records using
informal methods
Stage 1
Uses concrete
materials and
mental strategies
for addition and
subtraction
involving one- and
two-digit numbers
Multiplication and
Division
Multiplication and Division
Students develop facility with
number facts and computation
with larger numbers in
multiplication and division and
an appreciation of the
relationship between those
facts
Early Stage 1
Groups and shares
collections of objects,
describes using
everyday language and
records using informal
methods
Stage 1
Models and uses
strategies for
multiplication and
division
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Example 1 - Mathematics
Mathematics
Domains/
strands
Sub-domains/sub-
strands
Outcomes/content
standards
Progress levels
Number
Addition &
subtraction
Students develop facility
with number facts and
computation with larger
numbers in addition and
subtraction and an
appreciation of the
relationship between
those facts
Stage 2
Uses mental and
written
strategies for
addition and
subtraction
involving two-,
three- and four-
digit numbers
Stage 3
Selects and
applies
appropriate
strategies for
addition and
subtraction with
numbers of any
size
Multiplication and
Division
Multiplication and Division
Students develop facility
with number facts and
computation with larger
numbers in multiplication
and division and an
appreciation of the
relationship between
those facts
Stage 2
Uses mental and
written strategies
for multiplication
and division
Stage 3
Selects and
applies
appropriate
strategies for
multiplication and
division
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Example developmental continuum for mathematics
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Assessment framework
Definition of construct
Domains/strands
Sub-domains/sub-
strands
Outcomes/content
standards
…
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Example 2 – Scientific literacy
Scientific literacy
Domains/strands Outcomes/content standards …
Formulating
Formulating or
identifying investigable
questions and
hypotheses, planning
investigations and
collecting evidence
Interpreting
Interpreting evidence
and drawing conclusions,
critiquing the
trustworthiness of
evidence and claims
made by others, and
communicating findings
Using
Using understandings for
describing and explaining
natural phenomena,
making sense of reports,
and for decision-making
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Example 2 – Scientific literacy
Scientific literacy
Domains/strands Outcomes/content standards …
Formulating
Formulating or identifying
investigable questions and
hypotheses, planning
investigations and collecting
evidence
Level 1 - Year 2
Responds to the teacher’s
questions, observes and
describes
Level 2 - Year 4
Given a question in a familiar
context, identifies a variable to
be considered, observes and
describes or makes non-
standard measurements and
limited records of data
Interpreting
Interpreting evidence and drawing
conclusions, critiquing the
trustworthiness of evidence and
claims made by others, and
communicating findings
Level 1 - Year 2
Describes what happened
Level 2 - Year 4
Makes comparisons between
objects or events observed
Using
Using understandings for
describing and explaining natural
phenomena, making sense of
reports, and for decision-making
Level 1 - Year 2
Describes an aspect or
property of an individual
object or event that has been
experienced or reported
Level 2 - Year 4
Describes changes to,
differences between or
properties of objects or events
that have been experienced or
reported
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Domain A
Domain B
Domain C
Level 1 - Year 2 Level 2 - Year 4 Level 3 – Year 6
Formulating - Domain A
Interpreting - Domain B
Using - Domain C
T1
T2
T7
T8T6
T11
T3 T10
T4
T5 T12
Example developmental continuum for scientific literacy
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
• Preparing a list of learning outcomes – these describe the types of
performances the students are expected to demonstrate (e.g. Knows
basic terms – “Writes a definition of each term”; “Identifies the term
that represents each weather element”; etc.)
• Outlining the course content – the content describes the area in which
each type of performance is to be demonstrated (e.g. “air pressure”;
“wind”; “temperature”; etc.)
• Preparing a chart that relates the relative emphasis of the learning
objectives to the content through the number, type and percentage of
items.
Building a table of specifications
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Table of specifications
Learning Outcomes
Content Area Basic Skills Application Problem
Solving
Total
Percentage
Fractions 5 5 5 15
Mixed
numbers
5 5 10 20
Decimals 5 15 10 30
Decimal to
Fraction
conversions
5 15 15 35
Total
Percentage
Points
20 40 40 100
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Table of specifications - English
CONTENT
COGNITIVE
LEVELS
WEIGHTAGE MARKS
Identifies Interprets Infers Weightage Marks
PLOT
1 SA
(2marks)
1 Essay
(3marks)
1SA
(2marks)
28% 7marks
CHARACTER
1 SA
(2marks)
1 Essay
(4marks)
24% 6marks
CRISIS
1
Performance
task
(8 marks)
32% 8marks
LANGUAGE
1 SA
(1mark)
1 SA
(1mark)
1 SA
(2marks)
16% 4marks
Weightage 20% 32% 48% 100
Marks 5marks 8marks 12marks 25marks
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Table of specifications - Geography
CONTENT COGNITIVE LEVELS WEIGHTAGE MARKS
Basic map skills &
Understanding
Application
Extended
Understanding
Weightage Marks
Physical
Landforms
1 SA
(2 marks)
1 Essay
(6 marks)
2 SAs
(4 marks)
24% 12 marks
Location
4 SA
(8 marks)
16% 8 marks
Climate
1 SA
(2 marks)
1 Perform. task
(16 marks)
1 SA
(2 marks)
40% 20 marks
Vegetation
2 SAs
(4 marks)
1 Essay
(6 marks)
20% 10 marks
Weightage 32% 44% 24% 100
Marks 16 marks 22 marks 12 marks 50 marks
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Constructing a test that operationally
defines the scale.
Test constructors are challenged by the need to
1. define items that enable students at different stages along the scale
to demonstrate that they have enough of the subject (construct) to
correctly answer the item;
2. ensure that the items are assessing the outcomes for the particular
location on the scale;
3. ensure that as the items are being written, the ones that are
intended to be located further towards the top of the scale on the line
are, in fact, are more demanding then those that are located towards
the bottom of the scale on the line; and
4. ensure that the reason that the items are more demanding is a
function of the property/variable that is being measured and not a
function of some other extraneous feature (validity).
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Assessment Literacy: Question 1
What is the most important thing to consider when selecting a
method for assessing performance against learning
objectives?
•how easy the assessment is to score
•how easy the assessment is to prepare
•how useful the assessment is at assessing the learning
objective
•how well the assessment is accepted by the school
administration
Standards, Standard Setting and
Maintenance, March 2015
27
Assessment Literacy: Question 2
Standards, Standard Setting and
Maintenance, March 2015
28
What does it mean when you are told that the test is
“reliable”?
A.student scores from the assessment can be used for a large
number of decisions
B.students who take the same test are likely to get similar
scores next time
C.the test score accurately assesses the content
D.the test score is more valid than teacher-based
assessments
Standards, Standard Setting and
Maintenance, March 2015
29
Assessment Literacy: Question 3
Standards, Standard Setting and
Maintenance, March 2015
30
Class teachers in a school want to assess their students’
understanding of the method for solving problems that they
have been teaching. Which one of the following would be the
most appropriate method for seeing whether the teaching had
been effective? Justify your answer.
•select a problem solving book with a problem solving test
already in it
•develop an assessment method consistent with what has
actually been taught in class
•select a problem solving test (like the PSA) that will give a
problem solving mark
•select an assessment that measures students’ attitudes to
problem solving strategies
Assessment Literacy: Question 3
The following Table of Specifications for a Mathematics
assessment was prepared by the classroom teacher. Use this
Table to answer items 4 and 5. Note: The numbers in the
cells refer to the number of items.
Standards, Standard Setting and
Maintenance, March 2015
31
Content Area
Bloom’s Taxonomy
Knowledge Comprehension Application Synthesis Analysis
Total
Place values
and number
sense
1 2 2 1 1 7
Space 2 3 3 2 0 10
Addition and
subtraction
2 4 4 5 1 16
Multiplication
& Division 1 3 2 2 2 10
Measurement
2 2 3 3 3 13
Total 8 14 14 13 7 56
Assessment Literacy: Question 4
How many items did the teacher aim to use to assess higher
order thinking skills, where higher order thinking skills are
those that assess items at or above Application in Bloom’s
Taxonomy?
A.14
B.34
C.7
D.None of the above
Standards, Standard Setting and
Maintenance, March 2015
32
Assessment Literacy: Question 5
Which one of the following statements BEST DEFINES a Table
of Specifications?
A.It ensures that the total number of marks for the
assessment will equal 100.
B.It classifies educational goals, learning objectives and
standards.
C.It relates the content to the cognitive level of the learning
objectives for the purpose of improving the validity of the
instrument.
D.It is a table that is used by teachers to reliably assess
students.
Standards, Standard Setting and
Maintenance, March 2015

More Related Content

What's hot

Quantitative Analysis (Language and Literature Assessment)
Quantitative Analysis (Language and Literature Assessment)Quantitative Analysis (Language and Literature Assessment)
Quantitative Analysis (Language and Literature Assessment)Joy Labrador
 
Achieving beneficial blackwash
Achieving beneficial blackwashAchieving beneficial blackwash
Achieving beneficial blackwashMaury Martinez
 
Reliability and validity of Research Data
Reliability and validity of Research DataReliability and validity of Research Data
Reliability and validity of Research DataAida Arifin
 
CHARACTERISTICS OF A GOOD INSTRUMENT
CHARACTERISTICS OF A GOOD INSTRUMENTCHARACTERISTICS OF A GOOD INSTRUMENT
CHARACTERISTICS OF A GOOD INSTRUMENTMusfera Nara Vadia
 
Evaluation.2011intro
Evaluation.2011introEvaluation.2011intro
Evaluation.2011introKAthy Cea
 
DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY
DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRYDEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY
DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRYPawan Sharma
 
How to make tests more reliable
How to make tests more reliableHow to make tests more reliable
How to make tests more reliableNawaphat Deelert
 
Test characteristics
Test characteristicsTest characteristics
Test characteristicsSamcruz5
 
Testing and test constructions
Testing and test constructionsTesting and test constructions
Testing and test constructionsSamcruz5
 
Characteristics of a good test
Characteristics  of a good testCharacteristics  of a good test
Characteristics of a good testAli Heydari
 
Testing in language programs (chapter 8)
Testing in language programs (chapter 8)Testing in language programs (chapter 8)
Testing in language programs (chapter 8)Tahere Bakhshi
 
Qualities of a Good Test
Qualities of a Good TestQualities of a Good Test
Qualities of a Good TestDrSindhuAlmas
 
Lesson 4 analysis of test results
Lesson 4 analysis of test resultsLesson 4 analysis of test results
Lesson 4 analysis of test resultsCarlo Magno
 

What's hot (20)

Validity
ValidityValidity
Validity
 
LANGUAJE TESTING
LANGUAJE TESTINGLANGUAJE TESTING
LANGUAJE TESTING
 
Quantitative Analysis (Language and Literature Assessment)
Quantitative Analysis (Language and Literature Assessment)Quantitative Analysis (Language and Literature Assessment)
Quantitative Analysis (Language and Literature Assessment)
 
Criteria of a good language test
Criteria of a good language testCriteria of a good language test
Criteria of a good language test
 
Test validity
Test validityTest validity
Test validity
 
Reliablity
ReliablityReliablity
Reliablity
 
Achieving beneficial blackwash
Achieving beneficial blackwashAchieving beneficial blackwash
Achieving beneficial blackwash
 
Reliability and validity of Research Data
Reliability and validity of Research DataReliability and validity of Research Data
Reliability and validity of Research Data
 
CHARACTERISTICS OF A GOOD INSTRUMENT
CHARACTERISTICS OF A GOOD INSTRUMENTCHARACTERISTICS OF A GOOD INSTRUMENT
CHARACTERISTICS OF A GOOD INSTRUMENT
 
Evaluation.2011intro
Evaluation.2011introEvaluation.2011intro
Evaluation.2011intro
 
Test appraisal
Test appraisalTest appraisal
Test appraisal
 
DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY
DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRYDEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY
DEVELOPMENT AND EVALUATION OF SCALES/INSTRUMENTS IN PSYCHIATRY
 
How to make tests more reliable
How to make tests more reliableHow to make tests more reliable
How to make tests more reliable
 
Test characteristics
Test characteristicsTest characteristics
Test characteristics
 
Validity
ValidityValidity
Validity
 
Testing and test constructions
Testing and test constructionsTesting and test constructions
Testing and test constructions
 
Characteristics of a good test
Characteristics  of a good testCharacteristics  of a good test
Characteristics of a good test
 
Testing in language programs (chapter 8)
Testing in language programs (chapter 8)Testing in language programs (chapter 8)
Testing in language programs (chapter 8)
 
Qualities of a Good Test
Qualities of a Good TestQualities of a Good Test
Qualities of a Good Test
 
Lesson 4 analysis of test results
Lesson 4 analysis of test resultsLesson 4 analysis of test results
Lesson 4 analysis of test results
 

Viewers also liked

Ahmad measuremenr reliability and validity
Ahmad measuremenr reliability and validityAhmad measuremenr reliability and validity
Ahmad measuremenr reliability and validityAhd Ryn
 
Reliability, validity, generalizability and the use of multi-item scales
Reliability, validity, generalizability and the use of multi-item scalesReliability, validity, generalizability and the use of multi-item scales
Reliability, validity, generalizability and the use of multi-item scalesdakter Cmc
 
Presentation Validity & Reliability
Presentation Validity & ReliabilityPresentation Validity & Reliability
Presentation Validity & Reliabilitysongoten77
 
Validity and reliability in assessment.
Validity and reliability in assessment. Validity and reliability in assessment.
Validity and reliability in assessment. Tarek Tawfik Amin
 
EAW Pivot Mount Rear Feet with Ring | Optics Trade
EAW Pivot Mount Rear Feet with Ring | Optics TradeEAW Pivot Mount Rear Feet with Ring | Optics Trade
EAW Pivot Mount Rear Feet with Ring | Optics TradeOptics-Trade
 
Communication Skills - Northwest Medical Partners
Communication Skills - Northwest Medical PartnersCommunication Skills - Northwest Medical Partners
Communication Skills - Northwest Medical Partnerspedenton
 
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...MedCouncilCan
 
Validity, reliability and feasibility
Validity, reliability and feasibilityValidity, reliability and feasibility
Validity, reliability and feasibilitysilpa $H!lu
 
Medical communication skill
Medical communication skillMedical communication skill
Medical communication skillHadi Hmoud
 
Reliability and validity1
Reliability and validity1Reliability and validity1
Reliability and validity1MMIHS
 
Reliability & validity
Reliability & validityReliability & validity
Reliability & validityshefali84
 
Basic Principles of Assessment
Basic Principles of AssessmentBasic Principles of Assessment
Basic Principles of AssessmentYee Bee Choo
 
Reliability
ReliabilityReliability
ReliabilityRoi Xcel
 
Classification of psychiatric disorders
Classification of psychiatric disordersClassification of psychiatric disorders
Classification of psychiatric disordersDr. Amit Chougule
 
Teaching Soft Skills in Healthcare Professions, by Lori Tyler
Teaching Soft Skills in Healthcare Professions, by Lori TylerTeaching Soft Skills in Healthcare Professions, by Lori Tyler
Teaching Soft Skills in Healthcare Professions, by Lori Tylerpearsonedhealthprof
 
Introduction to clinical communication skills.pptx 2011
Introduction to clinical communication skills.pptx 2011Introduction to clinical communication skills.pptx 2011
Introduction to clinical communication skills.pptx 2011Reina Ramesh
 

Viewers also liked (20)

Ahmad measuremenr reliability and validity
Ahmad measuremenr reliability and validityAhmad measuremenr reliability and validity
Ahmad measuremenr reliability and validity
 
Reliability, validity, generalizability and the use of multi-item scales
Reliability, validity, generalizability and the use of multi-item scalesReliability, validity, generalizability and the use of multi-item scales
Reliability, validity, generalizability and the use of multi-item scales
 
Presentation Validity & Reliability
Presentation Validity & ReliabilityPresentation Validity & Reliability
Presentation Validity & Reliability
 
Validity and reliability in assessment.
Validity and reliability in assessment. Validity and reliability in assessment.
Validity and reliability in assessment.
 
EAW Pivot Mount Rear Feet with Ring | Optics Trade
EAW Pivot Mount Rear Feet with Ring | Optics TradeEAW Pivot Mount Rear Feet with Ring | Optics Trade
EAW Pivot Mount Rear Feet with Ring | Optics Trade
 
Communication Skills - Northwest Medical Partners
Communication Skills - Northwest Medical PartnersCommunication Skills - Northwest Medical Partners
Communication Skills - Northwest Medical Partners
 
Confiabilidad kuder
Confiabilidad kuderConfiabilidad kuder
Confiabilidad kuder
 
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
Blueprinting and Choosing Appropriate Tools for Assessment of Student Perform...
 
Validity, reliability and feasibility
Validity, reliability and feasibilityValidity, reliability and feasibility
Validity, reliability and feasibility
 
Medical communication skill
Medical communication skillMedical communication skill
Medical communication skill
 
Reability & Validity
Reability & ValidityReability & Validity
Reability & Validity
 
Reliability and validity1
Reliability and validity1Reliability and validity1
Reliability and validity1
 
Principles of assessment
Principles of assessmentPrinciples of assessment
Principles of assessment
 
Reliability & validity
Reliability & validityReliability & validity
Reliability & validity
 
Basic Principles of Assessment
Basic Principles of AssessmentBasic Principles of Assessment
Basic Principles of Assessment
 
Reliability
ReliabilityReliability
Reliability
 
Classification of psychiatric disorders
Classification of psychiatric disordersClassification of psychiatric disorders
Classification of psychiatric disorders
 
Teaching Soft Skills in Healthcare Professions, by Lori Tyler
Teaching Soft Skills in Healthcare Professions, by Lori TylerTeaching Soft Skills in Healthcare Professions, by Lori Tyler
Teaching Soft Skills in Healthcare Professions, by Lori Tyler
 
Reliability
ReliabilityReliability
Reliability
 
Introduction to clinical communication skills.pptx 2011
Introduction to clinical communication skills.pptx 2011Introduction to clinical communication skills.pptx 2011
Introduction to clinical communication skills.pptx 2011
 

Similar to Laos Session 3: Principles of Reliability and Validity (EN)

Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)
Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)
Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)NEQMAP
 
Laos Session 2: Introduction to Modern Assessment Theory (continued) (EN)
Laos Session 2:  Introduction to Modern Assessment Theory (continued) (EN)Laos Session 2:  Introduction to Modern Assessment Theory (continued) (EN)
Laos Session 2: Introduction to Modern Assessment Theory (continued) (EN)NEQMAP
 
Alternatives in assessment
Alternatives in assessmentAlternatives in assessment
Alternatives in assessmentRochiLuu
 
PR2 Q2 3rd week.docx
PR2 Q2 3rd week.docxPR2 Q2 3rd week.docx
PR2 Q2 3rd week.docxmaryann165675
 
Assessment 072
Assessment 072Assessment 072
Assessment 072fogleman
 
Kimberly joyce m. mendeja
Kimberly joyce m. mendejaKimberly joyce m. mendeja
Kimberly joyce m. mendejakhimendeja
 
Rubrics for Strategies 4 & 7
Rubrics for Strategies 4 & 7Rubrics for Strategies 4 & 7
Rubrics for Strategies 4 & 7Jeremy
 
Laos Session 6: Developing Quality Assessment Items Extended Response Items
Laos Session 6: Developing Quality Assessment Items Extended Response ItemsLaos Session 6: Developing Quality Assessment Items Extended Response Items
Laos Session 6: Developing Quality Assessment Items Extended Response ItemsNEQMAP
 
Assessment 07
Assessment 07Assessment 07
Assessment 07fogleman
 
Laos Session 7: Developing Quality Assessment Items - Rubrics
Laos Session 7: Developing Quality Assessment Items - RubricsLaos Session 7: Developing Quality Assessment Items - Rubrics
Laos Session 7: Developing Quality Assessment Items - RubricsNEQMAP
 
Factors in constructing evaluative instruments
Factors in constructing evaluative instrumentsFactors in constructing evaluative instruments
Factors in constructing evaluative instrumentsCatherine Matias
 
Using GradeMark to improve feedback and engage students in the marking proces...
Using GradeMark to improve feedback and engage students in the marking proces...Using GradeMark to improve feedback and engage students in the marking proces...
Using GradeMark to improve feedback and engage students in the marking proces...Sara Marsham
 
Formative Evaluation - Instructional Design and Development
Formative Evaluation - Instructional Design and DevelopmentFormative Evaluation - Instructional Design and Development
Formative Evaluation - Instructional Design and DevelopmentMr. Mariadi
 
Decoding Assessment Innoteach
Decoding Assessment InnoteachDecoding Assessment Innoteach
Decoding Assessment InnoteachMelanio Florino
 
Creating Rubrics
Creating RubricsCreating Rubrics
Creating Rubricschedisky
 
Special topic (Authentic Assessment)
Special topic (Authentic Assessment)Special topic (Authentic Assessment)
Special topic (Authentic Assessment)yunisay13
 
Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)
Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)
Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)yunisay13
 
Assessment in Flexible Learning Mode
Assessment in Flexible Learning ModeAssessment in Flexible Learning Mode
Assessment in Flexible Learning ModeMartin Nobis
 

Similar to Laos Session 3: Principles of Reliability and Validity (EN) (20)

Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)
Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)
Laos Session 5: Analysing Test Items using Classical Test Theory (CTT)
 
Laos Session 2: Introduction to Modern Assessment Theory (continued) (EN)
Laos Session 2:  Introduction to Modern Assessment Theory (continued) (EN)Laos Session 2:  Introduction to Modern Assessment Theory (continued) (EN)
Laos Session 2: Introduction to Modern Assessment Theory (continued) (EN)
 
Alternatives in assessment
Alternatives in assessmentAlternatives in assessment
Alternatives in assessment
 
PR2 Q2 3rd week.docx
PR2 Q2 3rd week.docxPR2 Q2 3rd week.docx
PR2 Q2 3rd week.docx
 
My Seminar 3
My Seminar 3My Seminar 3
My Seminar 3
 
Assessment 072
Assessment 072Assessment 072
Assessment 072
 
Kimberly joyce m. mendeja
Kimberly joyce m. mendejaKimberly joyce m. mendeja
Kimberly joyce m. mendeja
 
Rubrics for Strategies 4 & 7
Rubrics for Strategies 4 & 7Rubrics for Strategies 4 & 7
Rubrics for Strategies 4 & 7
 
Laos Session 6: Developing Quality Assessment Items Extended Response Items
Laos Session 6: Developing Quality Assessment Items Extended Response ItemsLaos Session 6: Developing Quality Assessment Items Extended Response Items
Laos Session 6: Developing Quality Assessment Items Extended Response Items
 
Assessment 07
Assessment 07Assessment 07
Assessment 07
 
Laos Session 7: Developing Quality Assessment Items - Rubrics
Laos Session 7: Developing Quality Assessment Items - RubricsLaos Session 7: Developing Quality Assessment Items - Rubrics
Laos Session 7: Developing Quality Assessment Items - Rubrics
 
Factors in constructing evaluative instruments
Factors in constructing evaluative instrumentsFactors in constructing evaluative instruments
Factors in constructing evaluative instruments
 
Assessing student learning outcomes
Assessing student learning outcomesAssessing student learning outcomes
Assessing student learning outcomes
 
Using GradeMark to improve feedback and engage students in the marking proces...
Using GradeMark to improve feedback and engage students in the marking proces...Using GradeMark to improve feedback and engage students in the marking proces...
Using GradeMark to improve feedback and engage students in the marking proces...
 
Formative Evaluation - Instructional Design and Development
Formative Evaluation - Instructional Design and DevelopmentFormative Evaluation - Instructional Design and Development
Formative Evaluation - Instructional Design and Development
 
Decoding Assessment Innoteach
Decoding Assessment InnoteachDecoding Assessment Innoteach
Decoding Assessment Innoteach
 
Creating Rubrics
Creating RubricsCreating Rubrics
Creating Rubrics
 
Special topic (Authentic Assessment)
Special topic (Authentic Assessment)Special topic (Authentic Assessment)
Special topic (Authentic Assessment)
 
Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)
Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)
Special topic by Rosalie P. Mabana and Gemma Caranza(Authentic Assessment)
 
Assessment in Flexible Learning Mode
Assessment in Flexible Learning ModeAssessment in Flexible Learning Mode
Assessment in Flexible Learning Mode
 

More from NEQMAP

ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)
ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)
ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)NEQMAP
 
ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...
ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...
ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...NEQMAP
 
ພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວ
ພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວ
ພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວNEQMAP
 
ພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບ
ພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບ
ພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບNEQMAP
 
Laos Session 4: Developing Quality Assessment Items (EN)
Laos Session 4:  Developing Quality Assessment Items (EN)Laos Session 4:  Developing Quality Assessment Items (EN)
Laos Session 4: Developing Quality Assessment Items (EN)NEQMAP
 
ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...
ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...
ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...NEQMAP
 
ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...
ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...
ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...NEQMAP
 
ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...
ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...
ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...NEQMAP
 
Laos Session 1: Introduction to Modern Assessment Theory (EN)
Laos Session 1:  Introduction to Modern Assessment Theory (EN)Laos Session 1:  Introduction to Modern Assessment Theory (EN)
Laos Session 1: Introduction to Modern Assessment Theory (EN)NEQMAP
 

More from NEQMAP (9)

ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)
ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)
ພາກທີ່ 7: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ - ເກນ (Rubrics)
 
ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...
ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...
ພາກທີ່ 5: ການວິເຄາະ ການທົດສອບ ແລະ ການທົດສອບ ລາຍການ ດ້ວຍການນໍາໃຊ້ ທິດສະດີ ການທ...
 
ພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວ
ພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວ
ພາກທີ່ 6: ການສ້າງ ລາຍການ ປະເມີນຜົນ ດ້ານຄຸນນະພາບ – ລາຍການ ທີ່ໄດ້ສ້າງ ຄຳຕອບແລ້ວ
 
ພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບ
ພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບ
ພາກທີ່ 4: ການສ້າງ ລາຍການຕ່າງໆ ຂອງການປະເມີນຜົນ ດ້ານຄຸນນະພາບ
 
Laos Session 4: Developing Quality Assessment Items (EN)
Laos Session 4:  Developing Quality Assessment Items (EN)Laos Session 4:  Developing Quality Assessment Items (EN)
Laos Session 4: Developing Quality Assessment Items (EN)
 
ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...
ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...
ພາກທີ່ 3: ບັນດາຫຼັກການພື້ນຖານ ຂອງການປະເມີນຜົນ: ຄວາມຖືກຕ້ອງ ແລະ ຄວາມຫນ້າເຊື່ອຖ...
 
ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...
ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...
ພາກທີ່ 2: ສະເຫນີແນະນຳ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ (ສືບຕໍ່) ໂດຍ: ສາດສະດ...
 
ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...
ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...
ພາກທີ່ 1: ສະເຫນີແນະ ທິດສະດີ ການປະເມີນຜົນ ແບບ ທັນສະ ໄຫມ ໂດຍ: ສາດສະດາຈານ Jim To...
 
Laos Session 1: Introduction to Modern Assessment Theory (EN)
Laos Session 1:  Introduction to Modern Assessment Theory (EN)Laos Session 1:  Introduction to Modern Assessment Theory (EN)
Laos Session 1: Introduction to Modern Assessment Theory (EN)
 

Recently uploaded

3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptxmary850239
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17Celine George
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsEugene Lysak
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptxSlides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptxCapitolTechU
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxheathfieldcps1
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
A gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceA gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceApostolos Syropoulos
 

Recently uploaded (20)

3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George Wells
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptxSlides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
Finals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quizFinals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quiz
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptx
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
A gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceA gentle introduction to Artificial Intelligence
A gentle introduction to Artificial Intelligence
 

Laos Session 3: Principles of Reliability and Validity (EN)

  • 1. Session 3: Principles of Assessment: Validity and Reliability Professor Jim Tognolini
  • 2. Introduction to Modern Assessment Theory: A basis for all assessments During this session we will •define reliability. •define measurement error. •examine the sources of measurement error. •define validity and identify threats to validity. •build assessment frameworks •operationalise frameworks with Tables of Specification. Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 3. The reliability of results gives the extent to which the results are consistent or error-free. The concept of reliability is closely associated with the idea of consistency. Reliability is not an all or nothing concept; there can exist degrees of reliability. How similar are results if students are assessed at different times? How similar are results if students are assessed with a different sample of equivalent tasks? How similar are results if essays have been marked by different markers. Reliability Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 4. Measurement Error Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 5. Sources of Measurement Error The following are some of the sources of measurement error 1. Test taking skills 2. Comprehension of instructions 3. Sampling variance of items 4. Temporary factors such as health; fatigue; motivation; testing conditions. 5. Memory fluctuations 6. Marking bias (especially in essays) 7. Guessing 8. Item types The aim for test developers is to identify sources of measurement error and minimise their impact. Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 6. Validity Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 7. Validity The validity of the results if a test can best be defined as the extent to which the results of a test measure what they purport to measure. It is the interpretation (including inferences and decisions) that is validated, not the test or the test score. Messick also argued (1989) that validation can include the evaluation of the consequences of the test; are the specific benefits likely to be realised? In 1999 the Standards (AERA, APA and NCME) suggested that validation can be viewed as developing scientifically sound validity arguments to support the intended interpretation of test scores and their relevance to the proposed use. Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 8. Threats to Validity 1. Factors in the test itself I. Unclear direction (e.g. how to respond to guessing; recording answers). II. Reading vocab and sentence structure is too difficult III. Inappropriate level of difficulty of test items (e.g. guessing) IV. Poorly constructed test items V. Ambiguity VI. Test items (tasks) inappropriate for content being assessed VII.Test too short VIII.Improper arrangement of items IX. Identifiable pattern of answers 2. Factors in test administration and scoring I. Insufficient time II. Cheating III. Unreliable scoring Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 9. Reliability is a necessary but insufficient condition for validity. Relationship between Validity and Reliability Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 10. Some basic assessment theory • Validity and reliability are not deterministic – maximise validity and reliability • Validity is paramount • Ways to minimise threats to validity and reliability  Breadth of material sampled – increase validity  Guessing  Quality of items Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 11. Assessment frameworks Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 12. Preliminary Questions • Why are we assessing? • What are we assessing? • What is the most appropriate way to assess these outcomes? Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 13. Assessment framework Definition of construct Domains/strands Sub-domains/sub- strands Outcomes/content standards … Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 14. Example 1 - Mathematics Mathematics Domains/strands Sub-domains/sub- strands Outcomes/content standards … Number Addition & subtraction Multiplication and Division Fractions and Decimals Measurement Space Chance Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 15. Example 1 - Mathematics Mathematics Domains/str ands Sub-domains/sub- strands Outcomes/content standards Progress levels Number Addition & subtraction Students develop facility with number facts and computation with larger numbers in addition and subtraction and an appreciation of the relationship between those facts Early Stage 1 Combines, separates and compares collections of objects, describes using everyday language and records using informal methods Stage 1 Uses concrete materials and mental strategies for addition and subtraction involving one- and two-digit numbers Multiplication and Division Multiplication and Division Students develop facility with number facts and computation with larger numbers in multiplication and division and an appreciation of the relationship between those facts Early Stage 1 Groups and shares collections of objects, describes using everyday language and records using informal methods Stage 1 Models and uses strategies for multiplication and division Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 16. Example 1 - Mathematics Mathematics Domains/ strands Sub-domains/sub- strands Outcomes/content standards Progress levels Number Addition & subtraction Students develop facility with number facts and computation with larger numbers in addition and subtraction and an appreciation of the relationship between those facts Stage 2 Uses mental and written strategies for addition and subtraction involving two-, three- and four- digit numbers Stage 3 Selects and applies appropriate strategies for addition and subtraction with numbers of any size Multiplication and Division Multiplication and Division Students develop facility with number facts and computation with larger numbers in multiplication and division and an appreciation of the relationship between those facts Stage 2 Uses mental and written strategies for multiplication and division Stage 3 Selects and applies appropriate strategies for multiplication and division Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 17. Example developmental continuum for mathematics Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 18. Assessment framework Definition of construct Domains/strands Sub-domains/sub- strands Outcomes/content standards … Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 19. Example 2 – Scientific literacy Scientific literacy Domains/strands Outcomes/content standards … Formulating Formulating or identifying investigable questions and hypotheses, planning investigations and collecting evidence Interpreting Interpreting evidence and drawing conclusions, critiquing the trustworthiness of evidence and claims made by others, and communicating findings Using Using understandings for describing and explaining natural phenomena, making sense of reports, and for decision-making Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 20. Example 2 – Scientific literacy Scientific literacy Domains/strands Outcomes/content standards … Formulating Formulating or identifying investigable questions and hypotheses, planning investigations and collecting evidence Level 1 - Year 2 Responds to the teacher’s questions, observes and describes Level 2 - Year 4 Given a question in a familiar context, identifies a variable to be considered, observes and describes or makes non- standard measurements and limited records of data Interpreting Interpreting evidence and drawing conclusions, critiquing the trustworthiness of evidence and claims made by others, and communicating findings Level 1 - Year 2 Describes what happened Level 2 - Year 4 Makes comparisons between objects or events observed Using Using understandings for describing and explaining natural phenomena, making sense of reports, and for decision-making Level 1 - Year 2 Describes an aspect or property of an individual object or event that has been experienced or reported Level 2 - Year 4 Describes changes to, differences between or properties of objects or events that have been experienced or reported Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 21. Domain A Domain B Domain C Level 1 - Year 2 Level 2 - Year 4 Level 3 – Year 6 Formulating - Domain A Interpreting - Domain B Using - Domain C T1 T2 T7 T8T6 T11 T3 T10 T4 T5 T12 Example developmental continuum for scientific literacy Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 22. • Preparing a list of learning outcomes – these describe the types of performances the students are expected to demonstrate (e.g. Knows basic terms – “Writes a definition of each term”; “Identifies the term that represents each weather element”; etc.) • Outlining the course content – the content describes the area in which each type of performance is to be demonstrated (e.g. “air pressure”; “wind”; “temperature”; etc.) • Preparing a chart that relates the relative emphasis of the learning objectives to the content through the number, type and percentage of items. Building a table of specifications Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 23. Table of specifications Learning Outcomes Content Area Basic Skills Application Problem Solving Total Percentage Fractions 5 5 5 15 Mixed numbers 5 5 10 20 Decimals 5 15 10 30 Decimal to Fraction conversions 5 15 15 35 Total Percentage Points 20 40 40 100 Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 24. Table of specifications - English CONTENT COGNITIVE LEVELS WEIGHTAGE MARKS Identifies Interprets Infers Weightage Marks PLOT 1 SA (2marks) 1 Essay (3marks) 1SA (2marks) 28% 7marks CHARACTER 1 SA (2marks) 1 Essay (4marks) 24% 6marks CRISIS 1 Performance task (8 marks) 32% 8marks LANGUAGE 1 SA (1mark) 1 SA (1mark) 1 SA (2marks) 16% 4marks Weightage 20% 32% 48% 100 Marks 5marks 8marks 12marks 25marks Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 25. Table of specifications - Geography CONTENT COGNITIVE LEVELS WEIGHTAGE MARKS Basic map skills & Understanding Application Extended Understanding Weightage Marks Physical Landforms 1 SA (2 marks) 1 Essay (6 marks) 2 SAs (4 marks) 24% 12 marks Location 4 SA (8 marks) 16% 8 marks Climate 1 SA (2 marks) 1 Perform. task (16 marks) 1 SA (2 marks) 40% 20 marks Vegetation 2 SAs (4 marks) 1 Essay (6 marks) 20% 10 marks Weightage 32% 44% 24% 100 Marks 16 marks 22 marks 12 marks 50 marks Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 26. Constructing a test that operationally defines the scale. Test constructors are challenged by the need to 1. define items that enable students at different stages along the scale to demonstrate that they have enough of the subject (construct) to correctly answer the item; 2. ensure that the items are assessing the outcomes for the particular location on the scale; 3. ensure that as the items are being written, the ones that are intended to be located further towards the top of the scale on the line are, in fact, are more demanding then those that are located towards the bottom of the scale on the line; and 4. ensure that the reason that the items are more demanding is a function of the property/variable that is being measured and not a function of some other extraneous feature (validity). Capacity Development Workshop: Test and Item Development and Design, Laos, September 2016
  • 27. Assessment Literacy: Question 1 What is the most important thing to consider when selecting a method for assessing performance against learning objectives? •how easy the assessment is to score •how easy the assessment is to prepare •how useful the assessment is at assessing the learning objective •how well the assessment is accepted by the school administration Standards, Standard Setting and Maintenance, March 2015 27
  • 28. Assessment Literacy: Question 2 Standards, Standard Setting and Maintenance, March 2015 28 What does it mean when you are told that the test is “reliable”? A.student scores from the assessment can be used for a large number of decisions B.students who take the same test are likely to get similar scores next time C.the test score accurately assesses the content D.the test score is more valid than teacher-based assessments
  • 29. Standards, Standard Setting and Maintenance, March 2015 29 Assessment Literacy: Question 3
  • 30. Standards, Standard Setting and Maintenance, March 2015 30 Class teachers in a school want to assess their students’ understanding of the method for solving problems that they have been teaching. Which one of the following would be the most appropriate method for seeing whether the teaching had been effective? Justify your answer. •select a problem solving book with a problem solving test already in it •develop an assessment method consistent with what has actually been taught in class •select a problem solving test (like the PSA) that will give a problem solving mark •select an assessment that measures students’ attitudes to problem solving strategies Assessment Literacy: Question 3
  • 31. The following Table of Specifications for a Mathematics assessment was prepared by the classroom teacher. Use this Table to answer items 4 and 5. Note: The numbers in the cells refer to the number of items. Standards, Standard Setting and Maintenance, March 2015 31 Content Area Bloom’s Taxonomy Knowledge Comprehension Application Synthesis Analysis Total Place values and number sense 1 2 2 1 1 7 Space 2 3 3 2 0 10 Addition and subtraction 2 4 4 5 1 16 Multiplication & Division 1 3 2 2 2 10 Measurement 2 2 3 3 3 13 Total 8 14 14 13 7 56
  • 32. Assessment Literacy: Question 4 How many items did the teacher aim to use to assess higher order thinking skills, where higher order thinking skills are those that assess items at or above Application in Bloom’s Taxonomy? A.14 B.34 C.7 D.None of the above Standards, Standard Setting and Maintenance, March 2015 32
  • 33. Assessment Literacy: Question 5 Which one of the following statements BEST DEFINES a Table of Specifications? A.It ensures that the total number of marks for the assessment will equal 100. B.It classifies educational goals, learning objectives and standards. C.It relates the content to the cognitive level of the learning objectives for the purpose of improving the validity of the instrument. D.It is a table that is used by teachers to reliably assess students. Standards, Standard Setting and Maintenance, March 2015