This document provides information about an upcoming training workshop on designing assessment tools to measure learning outcomes in maritime education. The workshop will be held on July 27-28, 2022 in Iloilo City, Philippines and aims to help participants gain knowledge on how to develop assessment tools. Several key principles of assessment are discussed, including that assessment should drive learning, people learn by doing and reflecting, and repetition leads to mastery. The document outlines the goal and agenda for the workshop, which will include discussions, constructing new tools, peer review, revising existing tools, and validating tools for content and alignment. Examples of assessment tools such as rubrics, checklists, concept maps, and performance tests are also provided. The workshop seeks to

1. TRAINING-WORKSHOP ON
DESIGNING ASSESSMENT TOOLS
TO MEASURE LEARNING
OUTCOMES IN MARITIME
EDUCATION
2022 JULY 27-28
(Vis-Min Cluster, Iloilo City)

2. “IF AN ASSESSMENT
ASKS STUDENTS TO
EVALUATE AND CREATE
BUT OUR INSTRUCTION
ASKS ONLY THAT THEY
REMEMBER AND
COMPREHEND,
THEN WE’VE TAKEN A
WRONG DIRECTION.”
“PEOPLE LEARN BY DOING
AND REFLECTING, NOT BY
WATCHING AND
LISTENING”
“REPETITION THROUGHOUT
THE CURRICULUM LEADS TO
MASTERY”
“ASSESSMENT DRIVES
LEARNING”

3. GOAL
Gain knowledge on how to
develop assessment tools
How do we get there?
1. Discussion
2. Constructing New
Tools
3. Peer Review
4. Revising Existing
Tools
5. Validation for Content
and Alignment
3

4. WHAT ARE ASSESSMENT TOOLS?
Techniques used to
measure a student's
academic abilities,
fluency/mastery and
skills in a specific subject
or to measure one’s
progress toward an
academic proficiency in a
specific subject area
4

5. WHAT ARE SOME EXAMPLES OF
ASSESSMENT TOOLS?
1. Rubric
2. Checklist
3. Concept Map
4. Concept Test such as MCQ,
essay test, etc.
5. Performance Test
6. Curriculum Mapping
7. Focus Group Discussion
8. Portfolio
9. Structured Interview
10. Survey
5

6. • Assessment is a structured process of gathering evidences and making judgment
(evaluation) about an individual’s performance in terms of knowledge, skills or
competencies, and behavior or attitudes in relation to a well-defined accepted standards
• Evidence of Knowledge and Understanding may be provided by examination while
evidence of Proficiency is provided by practical demonstration, job performance, or work
products
• Assessment should be valid, authentic, current, sufficient, reliable, and objective
• “Assessment is feedback from the student to the faculty about the student’s learning while
Evaluation is feedback from the Faculty to the student about the student’s learning”
• Reveals whether the student has attained or not the learning outcomes, and therefore the
student’s level of competency
What it is, what for is it, and what does it reveal at the end?
6
ASSESSMENT

7. COMPETENCY
Competency is attained when a student is able to apply/demonstrate the knowledge
and skills up to the standards of required performance
Competency indicates fitness or ability to perform a certain task
Examples:
1. The crew is competent to use a fire extinguisher.
2. The ability to use a fire extinguisher is a competence all seafarers must have.
3. A candidate is deemed competent if he/she has demonstrated sufficient
knowledge, understanding, and proficiency to perform the competences given in
column 1 of the relevant tables of the STCW Code.
What does it mean?
7

8. COMPETENCY-BASED VS OUTCOMES-BASED ASSESSMENT
❖Competency-based assessment is the process of collecting evidences and
establishing conclusions whether the competence has been achieved and thus
confirms that the student can perform up to the standard
❖Outcomes-based assessment is the deliberate collection of evidence of student
learning based on outcomes that are criterion referenced
❖What is their similarity?
• Clear criteria for measuring fluency/proficiency
• Adaptive to student needs
• Learner support is provided when and where it's needed most
What is the difference? What do we need?
8

9. COMPETENCY-BASED VS OUTCOMES-BASED ASSESSMENT
❖What is their difference?
Outcomes-based assessment focuses on what a student can or should be
able to do either upon completion of course instruction or upon graduation
from a program while competency-based assessment is more commonly
used in relation to professional fields such as seafaring/maritime.
❖What do we need?
Write the competences using the vocabulary of learning outcomes, i.e. express the
required competence in terms of the student’s achievement of specific course
outcomes or module’s intended learning outcomes.
What is the difference? What do we need?
9

10. FORMATIVE AND SUMMATIVE ASSESSMENT
❖Formative assessment takes place during the process of learning; provides
an indication of how learning is progressing, thereby identifying certain
needs of the learner that should be addressed during the process
❖Summative assessment occurs at the end of the learning process; gathered
evidences are assessed as to whether a learner has met the requirements for
competence through the attainment or demonstration of course outcomes
❖What do we need?
Both are needed as each of the two types serves a different and legitimate
purpose
All learning outcomes must be assessed
What is the difference? What do we need?
10

11. DIAGNOSTIC, IPSATIVE, NORM-REFERENCED, AND CRITERION-REFERENCED ASSESSMENT
Diagnostic assessment identifies what students know and can do in different domains.
Since the results show what students understand, the faculty would then be able to
build on the students' strengths and address their specific needs by creating a realistic
roadmap that addresses any gaps of knowledge
Ipsative assessment is assessment and evaluation of student output in comparison
with his own previous performance. This motivates students to set goals and improve
their skills
Norm-referenced is assessment and evaluation of student output on the basis of his
performance in relation to other’s performance
Criterion-referenced is assessment and evaluation of student output on the basis of
his performance in relation to clearly stated criteria
These are types or methods of assessment together with Formative and Summative Assessments
What are these for? When do we use each of these ?
11

12. ASSESSMENT METHODS, TOOLS/ACTIVITIES, AND EVIDENCES
Assessment Method refers to the philosophical or pedagogical approach to assessing
such as written or practical assessment, formative or summative assessment
Assessment Tools are used for different assessment methods and are more specific.
ATs determine how to gather the evidence/s that will measure the
LO/CO/competence (e.g. Observation, Interview, Poster, Essay, Exams, etc.) and are
also used to mark assessments such as the Rubrics
Assessment Activities or assessment tasks are what we find in the AT. These are the
ways that will be used to generate the evidence/s (e.g. Role Play, FGD, Oral Reporting,
Performance Test, etc.)
Evidences are proof/s that should be gathered and kept as such will support the
results of assessment (e.g. Observation Sheets, Checklist, Interview Questions,
Journal, Rubrics, etc.)
How important are these in assessment?
12

13. MORE EXAMPLES OF ASSESSMENT TOOLS
1. Case Studies 12. Presentations
2. Problem Solving 13. Projects
3. Computer-Based Assessment 14. Thesis/Dissertation
4. Essays 15. Questionnaires
5. Diaries/Journals 16. Reports
6. Practical/Performance Test 17. Reflection Paper
7. Modified Essay Test 18. Questions for Self-Assessment
8. Multiple Choice Questions 19. Work-Based Assessment
9. Oral Test 20. Term Paper
10. Portfolios 21. Assignment/Homework
11. Poster-Making Session 22. Infographics
The use of these depends on the purpose/outcome the assessment is being made for:
13
The underlying principle is that the assessment tasks should comprise an authentic representation of the learning outcomes.

14. LEARNING OUTCOMES AND SAMPLES OF ASSESSMENT TOOLS/INSTRUMENTS/ACTIVITIES
An Assessment Tool is the same as the Assessment Instrument. It refers to the documented activities developed to
support the assessment method, and is used to collect the evidence of student’s competence (see Bloom’s Wheel
for more examples).
14
Categories of Learning Outcomes to be assessed Could be assessed by:
1. Thinking critically and making judgments such as developing arguments and reflecting Essays, Journals, Debate
2. Solving problems and developing plans such as identifying root cause and analyzing
data
Scenarios, Group Work,
Case Study
3. Performing procedures and demonstrating techniques such as computation and using
equipment
Role Play, Posters,
Simulation, Demonstration
4. Managing and developing oneself such as working cooperatively and working
independently
Portfolios, Contracts or
Agreements
5. Accessing and managing information such as researching and organizing data Annotated Bibliography,
Projects
6. Demonstrating knowledge and understanding such as recalling and describing Exams, Essays
7. Designing, creating, performing such as Visualizing and producing Presentation, Projects
8. Communicating such as verbal or written communication, individually or within a group Presentation, Observation

15. ASSESSMENT STANDARD AND CRITERIA
Assessment Standard
• Specifies the criteria for determining proficiency
• Comes from the Table of Competences of the amended STCW Code
• Refers to written procedural guidelines and rules regarding the planning and
implementation of national or institutional evaluation processes, established by
consensus by recognized stakeholders and approved by the maritime
community
Assessment Criteria
Qualities, characteristics, and aspects of assessment task that will be used to
measure the attainment of the learning/course outcome
Why do we have to use each of these in assessment?
15

16. ASSESSING EVIDENCES
• Review portfolios of documentation against the relevant STCW standards for
the competence being assessed
• Verify all submitted evidences for authenticity
• Verify testimonial evidences of others from the sources
• Interview candidates to establish authenticity
• Observe candidates at actual work or simulated work, if possible
How do we assess the evidence/s?
16

17. So, now that we know the nitty-gritty
of what assessment is about, and what it is for……..
How do we start designing assessment tools that we can
use to measure the learning outcomes
attained by our students?
17

18. CRITICAL REVIEW OF DETAILED TEACHING SYLLABUS
What does the DTS tell?
18
• A syllabus tells the student what the course is about, why is it taught, where it is
going, and what will be required to be successful in it
• DTS is the source of all important information for effective course delivery and
assessment
• Logically, the sequence of completing the DTS is: Course Outcome – Intended
Learning Outcomes – Assessment – KUPs – TLAs – Equipment, Materials &
References – Indicative Hours
• Specifically, for assessment purposes, the critical part of the DTS are the outcomes
as these are translations and articulations of the competences required in the STCW

19. CRITICAL REVIEW OF DETAILED TEACHING SYLLABUS
How should the learning outcomes be written?
19
Learning Outcomes must:
• Be expressed in terms of measurable or observable behaviors
• State the major knowledge, skills, attitude or ability the student will acquire
• Have the following required components:
1. The Action Verb which should be SMART (Specific, Measurable, Attainable,
Realistic, Time-bound)
Example: Describe the principles used in designing
2. The Condition - the context under which the learning behavior is to occur
Example: Orally describe the principles used in designing
3. The Standard – criteria of acceptable level of performance
Example: Orally describe the five principles used in designing

20. QUICK EXERCISE
For general connotation, a learning outcome (LO) may either be a Course Outcome (CO)
or an Intended Learning Outcome (ILO)
• Original •
Students should be able to design research.
• Better •
Students should be able to independently design and carry out experimental and
correlational research.
• Best •
Students should be able to independently design and carry out experimental and
correlational research that yields valid results
Let’s try analyzing the improvement made in the following learning outcome:
20

21. HOW TO DEVELOP THE ASSESSMENT PLAN/GUIDE
❖An assessment plan is an outline that includes all relevant portions of an assessment, tying
together each step of an assessment in consideration of the context in which the assessment will
occur.
❖Steps in developing an Assessment Plan
1. Write the Course Outcome followed by its Intended Learning Outcomes
2. Analyze the Outcome and identify the domain of learning where it belongs
3. Write the Assessment Standard to be used in accepting the outcome of assessment
(reference is column 4 of the Table of Competences of the STCW Code)
4. Analyze the standard and identify the assessment criteria (qualities, characteristics, and
aspects of assessment task that will be used to measure the attainment of the learning/
course outcome)
5. Analyze the statement of outcome and determine the assessment method to be used
6. Analyze the statement of outcome and determine the assessment tool/s to be used and the
evidences that will be generated from using the assessment tools.
7. Determine your feedback strategy
• Focus on feedback that encourages students to think through their learning and on
what is important for the learner - outcomes, process, and self-regulation
❖ Prepared by the Assessor of the Course
How will I know if my students have learned what I want them to?
21

22. HOW TO DEVELOP THE ASSESSMENT PLAN/GUIDE
How will I know if my students have learned what I want them to?
22

23. TABLE OF SPECIFICATIONS (TOS)
❖Table of Specifications is a tool used to design an assessment or test or exam and ensures the
coverage of ALL learning outcomes (LOs)
❖Purposes
• Categorizes the LOs to be assessed by the learning domain that has been developed
✓ Cognitive Learning for developing knowledge and intellectual skills
✓ Psychomotor Learning for developing skills
✓ Affective Learning for developing behavior and appropriate attitude
• Ensures that a fair and representative sampling of questions/exercises/activities/scenarios are
covered in the assessment tool according to the LOs that will be measured
• Provides the assessor and faculty with evidence that the AT has content validity and that it covers
what should be covered
What is TOS? What purpose does it serve?
23

24. TABLE OF SPECIFICATIONS (TOS) FOR COGNITIVE LEARNING
• May cover both formative and summative assessment of knowledge which is translated in terms
of intellectual behavior
• Includes recall or recognition of specific facts, procedural patterns, and concepts that serve in the
development of intellectual abilities and skills
• Categories of assessing cognitive learning start from the simplest to the most complex
• Application of OBE principles can be shown from the third category up to the highest level
• To have an adequate number of valid test items per learning/course outcome, it is recommended
to construct at least 3 parallel valid test items which may be used for the assessment periods of
prelim, midterm, and final, or for other purposes such as developing a test bank after passing
item analysis
• Parallel test items are different test items in terms of test construct but covers the same learning
outcome and content
• Valid test item means that the test item’s content (what it is about) and how the item is constructed
will indeed measure the learning outcome
How does it help the Faculty and Assessor in the assessment process? (Ref: Bloom, 1956; Bloom, et.al. 1990)
24

25. HOW TO DEVELOP THE TABLE OF SPECIFICATIONS FOR COGNITIVE LEARNING
❖Steps in developing the Table of Specifications for Cognitive Learning
1. From the Assessment Plan, copy the Course Outcomes followed by the Intended Learning
Outcomes of Cognitive Learning
2. From the DTS, copy the topic that correspond to each of the ILO
3. Analyze the action verb, the condition setting, and standard written in the CO to identify the
category/level of learning behavior in the TOS that will be used in developing the Assessment
Tool
4. Analyze the action verb, the condition setting, and standard written in the ILO to identify the
category/level of learning behavior in the TOS that will be used in developing the Assessment
Tool
5. Assign at least 3 assessment items and write their number accordingly under the chosen learning
behavior
6. After completing the requirements for all COs and ILOs, complete the required statistics of the last
two rows and last two columns of the TOS
❖ Prepared by the Faculty and Assessor of the Course
Cognitive learning is a change in knowledge attributable to the learner’s experience
25

26. 26
SAMPLE OF THE TABLE OF SPECIFICATIONS FOR COGNITIVE LEARNING
TABLE OF SPECIFICATIONS FOR COGNITIVE LEARNING OF THERMODYNAMICS
STCW Competence to be assessed: Plan and schedule operations
CO 1
Apply principles of
thermodynamics in solving
problems relating to heat
balance of the various cycles
ILO 1.1
Explain the mechanisms of heat
transfer and identify examples
and applications
ILO 1.2
Calculate parameters involving
the three mechanisms of heat
transfer
ILO 2.1
Explain the principles of heat
transfer, quantity of heat, and
heat capacity
ILO 2.2 Explain the process in
measuring heat capacity
Statement of Outcome
Levels of Intellectual Learning
Total No. of
Items
Topic
Total No. of Items
% of Items
Remembering Understanding Applying Analyzing Evaluating Creating
% of Items

27. HOW TO DEVELOP THE ASSESSMENT TOOLS FOR COGNITIVE LEARNING
❖Steps in developing the Assessment Tools for Cognitive Learning (Ref. Slide #19 & #20, Bloom’s Wheel)
1. From the TOS, identify the level of learning that will be used in assessing the student and refer to the
outermost circle of the Bloom’s Wheel for the kinds of Assessment Tool that can be used
2. Determine the preferred AT by identifying which one can show the standard expressed in the LO
3. After selecting the appropriate AT, write the assessment item by considering the action verb of the LO
and the condition required so as to generate the expected performance/response from the student being
assessed
4. Write at least 3 assessment items per LO. Be guided by the proper way of writing assessment/test items;
good test items should be valid, reliable, objective, not too easy neither too difficult
5. Construct the answer key for all assessment items; if AT will result in having a product, or concrete
output such as essays, case study, or any similar outputs, rubric must prepared in assessing the
outcomes of learning
6. After completing the requirements for all COs and ILOs, complete the required statistics of the last two
rows and last two columns of the TOS
❖ Prepared by the Faculty and Assessor of the Course
Cognitive learning is a change in knowledge attributable to the learner’s experience
27

28. THE BLOOM’S WHEEL
28

29. THE RUBRICS FOR PERFORMANCE OUTPUT/OUTCOME
ANALYTICAL RUBRIC
When to use:
• Several faculty are collectively assessing student work output. Descriptions promote consistent scoring
• Outside people will be examining rubric scores. Substantial feedback to students or faculty is desired
• Profiles of specific strengths/weaknesses are desired
HOLISTIC RUBRIC
When to use:
• There is no single correct answer/response to a task (e.g. a creative work)
• The focus is on overall quality, proficiency, or understanding of a specific content or skills
• The assessment is summative
• You are assessing so many students (e.g. 100 or more work outputs)
What is the rubric for? How do you develop it?
29

30. ANALYTIC RUBRIC FOR RESEARCH PAPER

31. HOLISTIC RUBRIC ON PRESENTATION

32. 32
HOW TO SCORE THE
RESULTS OF ASSESSMENT
BY USING THE RUBRIC?
HOW DO YOU ASSESS
THE LEARNING
OUTCOMES ATTAINED BY
THE STUDENT AFTER
USING THE RUBRIC?

33. TABLE OF SPECIFICATIONS (TOS) FOR PSYCHOMOTOR LEARNING
• May cover both formative and summative assessment of physical movement, coordination, and
use of the motor skill areas of learning
• Development of these skills requires practice
• Skill developed may be measured in terms of speed, precision, distance, procedure or techniques
in execution (depend on the learning/course outcome)
• Categories of learning are listed from the simplest behavior to the most complex
• To have an adequate number of valid test scenarios per learning/course outcome, it is
recommended to construct at least 3 parallel valid test scenarios which may be used for the
assessment periods of prelim, midterm, and final, or for other purposes such as rework or resit
• Parallel test scenarios are different test situations but covers the same learning outcome and
required condition
• Valid test scenario means that the parameters, conditions, and calculated/anticipated results of
using the AT will indeed measure the targeted learning outcome
How does it help the Faculty and Assessor in the assessment process? (Ref: Simpson, 1972)
33

34. HOW TO DEVELOP THE TOS FOR PSYCHOMOTOR LEARNING
Psychomotor skills need to be performed, practiced, and observed to determine mastery of the skill.
34
❖Steps in developing the Table of Specifications for Psychomotor Learning
1. From the Assessment Plan, copy the Course Outcomes followed by the Intended
Learning Outcomes of Psychomotor Learning
2. From the DTS, copy the topic that correspond to each of the ILO
3. Analyze the action verb, the condition setting, and standard written in the CO to identify
which category/level of skill learned/developed in the TOS will be used in developing
the Assessment Tool
4. Analyze the action verb, the condition setting, and standard written in the ILO to identify
which category/level of skill learned/developed in the TOS will be used in developing
the Assessment Tool
5. Assign at least 3 assessment scenarios and write their number/code accordingly under
the chosen skill learning behavior
6. After completing the requirements for all COs and ILOs, complete the required statistics
of the last two rows and last two columns of the TOS
❖ Prepared by the Faculty and Assessor of the Course

35. 35
SAMPLE OF THE TABLE OF SPECIFICATIONS FOR PSYCHOMOTOR LEARNING
TABLE OF SPECIFICATIONS FOR PSYCHOMOTOR LEARNING OF SEAM 1
STCW Competence to be assessed:
A-II/1. F1.C7 Use the IMO Standard Marine Communication Phrases and the English language in written and oral form
A-II/1. F1.C8 Transmit and receive information by visual signalling
A-II/5. F3.C1 Contribute to the safe operation of deck equipment and machinery
A-II/5. F3.C2 Apply occupational health and safety precautions
A-II/5. F4.C1 Contribute to shipboard maintenance and repair
Total No. of
Activities
% of Activities
Perception Set
Guided
Response
Mechanism
Expert
Response
Adaptation Origination
Statement of Outcome Topic
Levels of Skill Learning
Total No. of Items
% of Items

36. HOW TO DEVELOP THE ASSESSMENT TOOL FOR PSYCHOMOTOR LEARNING
36
❖Steps in developing the Assessment Tool for Psychomotor Learning
1. Be guided by the statement of the CO/ILO
2. Complete the following requirements in the Performance Test:
• CO/ILO Statement that is being assessed
• Category of Skill Learning being covered
• Title of the Activity
• Safety Briefing
• Assessment Scenario
• Initial Condition of the Activity
• Allocated Time/Duration
• Resources Required
• Briefing on the Activity
• Procedure of the Activity
• Conditions for Stopping the Activity
3. Write your Assessment Guide
4. Construct/Attach the rubric
❖ Conduct debriefing of student/s after the assessment, include lesson learned and feedback on the outcome
❖ Prepared by the Faculty and Assessor of the Course
Psychomotor skills need to be performed and observed to determine mastery of the skill.

37. SAMPLE OF PERFORMANCE ASSESSMENT
37
Student Name:_____________________
Document Name PERFORMANCE ASSESSMENT/PRACTICAL TEST
Course Code
Course Descriptive Title
CO/ILO
Category of Skill Learning Covered
Title of the Activity
Safety Briefing
Assessment Scenario
Initial Condition of the Activity
Allocated Time/Duration
Resources Required
Briefing on the Activity
Procedure of the Activity
Condition/s for Stopping the Activity
Assessment Guide
Rubric
Debriefing
Year/Section: _________ Date: ______

38. TABLE OF SPECIFICATIONS (TOS) FOR AFFECTIVE LEARNING
• May cover assessment of the student’s manner in which he deals with things, people, and his
environment emotionally such as feelings, values, appreciation, enthusiasm, motivation, and
attitudes
• Assessment of this learning requires keen observation and consistency
• Categories of learning are listed from the simplest behavior to the most complex
• To have an adequate number of valid assessment scenarios per learning outcome, it is
recommended to construct at least 3 parallel valid scenarios to establish the consistency of
learning outcome
• Valid test scenario means that the parameters, conditions, and calculated/anticipated results of
using the AT will indeed measure the targeted learning outcome
How does it help the Faculty and Assessor in the assessment process? (Ref: Krathwohl, Bloom, Masia, 1973)
38

39. HOW TO DEVELOP THE TABLE OF SPECIFICATIONS FOR AFFECTIVE LEARNING
Affective learning is concerned with how learners feel while they are learning
39
❖Steps in developing the Table of Specifications for Affective Learning
1. From the Assessment Plan, copy the Course Outcomes followed by the Intended
Learning Outcomes of Affective Learning
2. From the DTS, copy the topic that correspond to each of the ILO
3. Analyze the action verb, the condition setting, and standard written in the CO to identify
which category/level of affective learning in the TOS will be used in designing the
Assessment Tool
4. Analyze the action verb, the condition setting, and standard written in the ILO to identify
which category/level of affective learning in the TOS will be used in designing the
Assessment Tool
5. Assign at least 3 assessment scenarios/situations and write their number/code
accordingly under the chosen level of affective learning
6. After completing the requirements for all COs and ILOs, complete the required statistics
of the last two rows and last two columns of the TOS
❖ Prepared by the Faculty and Assessor of the Course

40. ASSESSMENT TOOLS FOR AFFECTIVE LEARNING
How learning experiences are internalized so they guide the learner’s attitudes, opinions, and
behavior in the future
40
❖Methods of assessing affective learning outcomes
1. Teacher Observation – can be unstructured by note-taking as in diary keeping, describing
student behavior like as anecdotal record; or structured observation by using checklist, rating
scale, rubrics
2. Student Self-Report – using interview or questionnaire
3. Peer Rating – by asking students to assess their classmate’s behavior using checklist, rating
scale, rubrics
❖ Prepared by the Faculty of the Course

41. Thank you!