It is often the case in the maritime industry that we give a great deal more thought to training than we do to assessment. This is unfortunate because training cannot be successful (or cannot be shown to be successful) without an objective and comprehensive assessment process. Your training may be excellent at this moment, but without quality assessments you have no way of knowing this for sure. This presentation gives a quick overview of 6 practical tips that can help you develop a more reliable and valid assessment process.

1. 6 TIPS FOR BETTER
ASSESSMENTS IN THE
MARITIME INDUSTRY

2. Goal:
Achieve
Comprehensive,
Valid and Reliable
Assessment
Comprehensive, Valid and Reliable Assessment

3. VALID
if it correctly
measures what it is
supposed to measure.
An assessment technique is:
RELIABLE
if it yields the same
results consistently
given the same input.
Comprehensive, Valid and Reliable Assessment

4. HOW DO WE
ACHIEVE OUR GOAL?
Comprehensive, Valid and Reliable Assessment

5. TIP 1
USE A COMBINATION
OF ASSESSMENT
TECHNIQUES

6. No single assessment
technique is perfect.
Achieve a better overall assessment
by combining techniques.
TIP 1: USE COMBINATION OF TECHNIQUES

7. Example:
Combine a
demonstrative
exam with a
multiple
choice exam
to improve
objectivity and
coverage.
TIP 1: USE COMBINATION OF TECHNIQUES

8. Highly
subjective
Each assessors can
grade differently.
Demonstrative exams (“Please
don a firefighting suit”) are
good at testing skills, but are:
Not
comprehensive
Doesn’t fully test
underlying
knowledge.
TIP 1: USE COMBINATION OF TECHNIQUES

9. Multiple choice exams are
weaker in testing skills but:
● Are highly objective
● Give additional insight
into depth of knowledge
TIP 1: USE COMBINATION OF TECHNIQUES

10. Using multiple
techniques
creates a larger,
basket of data
for final
assessment.
TIP 1: USE COMBINATION OF TECHNIQUES

11. TIP 2
SEPARATE THE
TRAINER FROM THE
ASSESSOR

12. TRAINER’S JOB:
● Support, instruct
and provide
resources
● Ensure candidate
has required
skills
TIP 2: SEPARATE TRAINER FROM ASSESSOR
ASSESSOR’S JOB:
● Ensure no
candidate is
assigned to a job
they are not
prepared for

13. If the trainer
and the
assessor is the
same person:
TIP 2: SEPARATE TRAINER FROM ASSESSOR

14. Objectivity when
assessing is affected:
• Candidate result
reflects training ability
• There is an existing
trainee-trainer
relationship
TIP 2: SEPARATE TRAINER FROM ASSESSOR

15. TIP 3
USE RUBRICS IN SKILL
ASSESSMENTS

16. RUBRIC:
a written guide for
assigning grades on
an assessment.
TIP 3: USE RUBRICS

17. Example Rubric for
Proficiency in donning firefighting gear:
Smoothly, without hesitation 5 points
Proficiently, hesitates occasionally 4 points
Makes occasional mistakes which they
correct themselves
3 points
Makes occasional mistakes with which
they require assistance
2 points
Makes multiple mistakes with which they
require assistance
1 point
Unable to without significant assistance 0 points
TIP 3: USE RUBRICS

18. The candidate demonstrates the ability to
don the firefighting suit and related gear.
Yes/No
Compare the previous rubric to:
TIP 3: USE RUBRICS

19. Rubrics add objectivity
and reliability,
providing better data
for a final decision.
TIP 3: USE RUBRICS

20. TIP 4
TRAIN AND SUPPORT
ASSESSORS

21. TIP 4: TRAIN AND SUPPORT ASSESSORS
Skilled
assessors are:
● Partly made by
experience
● Partly made by
training

22. 1. Train your assessors
○ Choose expert mariners with
interest in training and assessment
○ Conduct a 1 or 2 day training session
on best practices and guidelines
TIP 4: TRAIN AND SUPPORT ASSESSORS

23. 2.Support your assessors
○ Create documents on
assessment practices,
philosophy, etc.
○ Provide rubrics
○ Create a place for assessors
to exchange information
anywhere, anytime
TIP 4: TRAIN AND SUPPORT ASSESSORS

24. TIP 5
VARY YOUR
ASSESSMENTS
🎲

25. Using the same
questions on every test
leads to the contents
becoming common
knowledge.
🎲TIP 5: VARY YOUR ASSESSMENTS

26. Questions should be
varied for each
assessment.
This prevents candidates
from “studying to the test”.
🎲TIP 5: VARY YOUR ASSESSMENTS

27. One way to vary assessments:
List categories
of skills and
knowledge you
would like to
cover
Create
questions with
equal difficulty
and importance
for each
category
Randomly
select
questions
from each
category
for the test
🎲TIP 5: VARY YOUR ASSESSMENTS

28. This technique
works for almost
any type of
assessment:
written, oral,
demonstrative, etc.
There should be at least 10
questions available to choose
from for each question selected.
👷
🎲TIP 5: VARY YOUR ASSESSMENTS

29. TIP 6
IMPLEMENT
CONTINUOUS
IMPROVEMENT

30. CONTINUOUS
IMPROVEMENT:
A process by which we
periodically evaluate the
performance of a task with a
view toward improving it.
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT

31. Evaluate current
performance
quality with Key
Performance
Indicators (KPIs).
Apply
changes that
you hope will
improve the
outcomes.
After a
period of
time, re-
assess the
KPIs.
During each continuous
improvement cycle:
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT

32. If the effect is positive,
keep the changes.
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT
If not, undo the changes.

33. 1. Ask supervisors how
successful they believe
assessments are in ensuring
only qualified candidates
are put into service.
Some assessment-
related KPIs:
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT

34. 2.Ask candidates
whether the
assessment
accurately
reflected the
knowledge and
skills taught in
training.
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT

35. 3.Monitor exam outcomes for
trends. Are grades generally
increasing or decreasing?
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT

36. The bottom line for
continuous improvement:
If you are not actively engaged
in measuring and improving your
performance, then you are
passively engaged in
performance deterioration.
TIP 6: IMPLEMENT CONTINUOUS IMPROVEMENT

37. Assessment is about
giving us the data we need
in order to make an
informed, professional
decision about individuals.
Comprehensive, Valid and Reliable Assessment

38. If assessment techniques
are flawed, then the data
is incorrect or incomplete
- an informed decision
cannot be made.
Comprehensive, Valid and Reliable Assessment

39. It is not difficult to take
steps toward more reliable
and valid assessments -
steps which can improve
organizational performance,
safety and pride as a whole.
Comprehensive, Valid and Reliable Assessment

40. Download the
in-depth guide on
improving your
assessments here.

41. FOLLOW US
Twitter Linkedin BlogFacebook

42. CREDITS∎ Presentation template by SlidesCarnival
∎ Photographs by Unsplash, Pixabay, Pexels Liz Novak, US Navy Mass
Communication Specialist 3rd Class Anna Van Nuys, Maine Maritime
Academy, Peter Griffin, NATO Training Mission-Afghanistan, U.S.
Navy photo by Mass Communication Specialist 3rd Class Liz
Dunagan, U.S. Navy Photo by Mass Communication Specialist
Seaman Christopher Farrington, SuperJet International, Kevin
Collins, U.S. Navy photo by Photographer's Mate 3rd Class Gary B.
Granger, Mercator Media, U.S. Coast Guard,