This document analyzes the cognitive levels of questions on the Seafarer Expertise Test (UKP) administered by Polytechnic Bumi Akpelni Semarang in Indonesia, using Bloom's taxonomy. The analysis of 100 multiple choice questions found that 51% tested remembering, 43% tested understanding, and 6% tested applying cognitive skills, all considered lower-order thinking skills. No questions tested higher-order thinking skills such as analyzing, evaluating or creating. This suggests the UKP focuses more on lower-order thinking than higher-order thinking. The researchers conclude the test should incorporate more higher-order thinking questions appropriate for the university level students taking the exam.

1. Evaluating The Cognitive Level of Seafarers Expertise Test (UKP)
Sheet By Using Taxonomy’s Bloom
Haryani1*
, Dwi Sulistyorini1*
, Ali Akbar Rafsanjani3*
1*
haryani@akpelni.ac.id, 2*
rini@akpelni.ac.id, 3*
aliakbarrafsan@gmail.com
1Nautical Study Program of Polytechnic Bumi Akpelni Semarang
2
Technical Study Program of Polytechnic Bumi Akpelni Smarang
Jl. Pawiyatan Luhur II/17, Bendan Dhuwur Semarang
3
Semarang State University
Jl. Kelud Utara 3, Petompon Gajahmungkur Semarang
Abstract: Seafarers Expertise Test (UKP) has important roles in measuring student’s
competence in maritime skill and knowledge. Changing the implementation of UKP in
writing to be Computer Based - Assessment (CBA) is a demand for rapid technology
change. Since English is also used in delivering the test, so many students do not pass
the test, and should do remedial test more than three times. The difficulty of the test
becomes one of the reasons for its failure. The aim of this research was to investigate
the Cognitive Levels of UKP sheet which focused on “Plan a voyage and conduct
navigation part”. This research also used content analysis method and qualitative
descriptive design. The questions analyzed were consisted of 100 questions. The
source of the data was taken from document of Polytechnic Bumi Akpelni Semarang.
The writers analyzed the data using Taxonomy’s Bloom for categorizing the cognitive
level for each question. The result showed that there were 51% of Remembering, 43%
of Understanding and 6% of Applying which were categorized as Low Order Thinking
Skill (LOTS). On the other hand, the category of High Order Thinking Skill (HOTS) did
not exist in the test questions. It can be concluded that the questions of the test were
more focused on LOTS than HOTS.
Keywords : Bloom’s Taxonomy, CBA, Cognitive Levels, Test, and UKP

2. A test has an important role in teaching and learning activities. It is the instrument to
assess teaching and learning process. It can measure what students’ level has been
actuated and how far they achieved the purpose of the teaching. Test can help the
teachers know how far students understand about the material that they have learnt.
One of the tests in Maritime Education is Seafarer Expertise Test (Ujian keahlian
Pelaut/UKP). The purpose of the test is to assess the seafarer students’ ability about
his vocational skill before they worked as seafarer. This test focus on nautical
questions’ sheets which derived from Bank Soal. In this test, the test used English
which meant that students have to master about English such as the vocabulary.
To make sure the difficulties of the seafarer examination question or seafarer
expertise test (UKP), the tester has to create the standard of questions. It means that
it is not too easy and it is not too difficult. To clarify the difficulties of the question in
cognitive level, it should be a tool to measure the cognitive level of the question. One
of them is Bloom’s taxonomy. Based on Bloom et al (1956:22) defined that Bloom’s
Taxonomy is a tool to classify each item’s level of the test. In addition Lee,et al (2017)
explained that bloom taxonomy is one of the alternative tool to check the difficulty of
the cognitive questions. It starts from remember, understand, apply, analyze, evaluate
and create. Moreover, the are some previous studies about taxonomy bloom in
different item test.
Literature Review
Firstly, Puluhulawa (2015) reported that in English National Examination for Junior
High School in Indonesia contained most of Understanding level. It is still consisted
LOTS. Secondly, Febriana, Usman & Muslem (2019). they investigated text book
English for eleventh grade students by Kemendikbud. The result was the most item
contained 66% HOTS. Thirdly, Kamlashi & Sahan (2018). they investigated the final
examination item of Senior high school level using taxonomy bloom. The result was
most of them containing understanding level (44%). it meant that most of them were
still in LOTS level. Fourthly, Nashuddin (2017). He investigated National Examination
item test using taxonomy bloom and he also investigate in University level included
Daily Communication (RDC), Reading at the Workplace (RW), Reading for English
Language Proficiency Test (RELPT). the result showed that most students in Indonesia
Especially in Senior High School and University level still faced difficulty in HOTS level.
Lastly, Tangsakul et al (2017) stated that Taxonomy bloom could be used to analyze
multiple choice questions. For example, They had proved by analyzing reading
comprehension test which the result showed that most of the question still focused on
Lower Other Thinking Skill (LOTS) category. Based on the previous study and the
problem, the writers got gap to conduct the research. In this research, the writers
wanted to identify the cognitive level of the questions of seafarer expertise test from
Politechnic Bumi AKPELNI by using bloom taxonomy. The writers wanted to find the
level of Higher Order Thinking Skill (HOTS) and Lower Order Thinking Skill (LOTS)
test category focusing on Plan a voyage and conduct navigation part.

3. Bloom’s Taxonomy
Bloom taxonomy comes from cognitivist of the learning theory. Cognitivist theory
believes that learning is thinking. When the students learn, they will think. It is In line,
when they do the exam, it will create their thinking skill. The process of thinking
should have measurement goal. Bloom taxonomy is a tool to measure the critical
thinking of the learners. Based on Adam (2015) stated that Taxonomy bloom is more
focus with cognitive level rather than physicomotor or affective skill. It is in line with
Anwar & Sohail (2014), they stated that cognitive level contains knowledge and
thinking. The aspects forced someone to think rather than to feel. According to
Krathwohl (2002) stated that there are six levels of Bloom’s Taxonomy. They are
remembering, understanding, applying, analyzing, evaluating and creating. Those
levels of taxonomy’s bloom explained below:
a. Remembering is a memorizing previous material such as recalling fact, giving
definition and concept.
b. Understanding is a demonstration of understanding facts and ideas by interpreting,
comparing, describing
c. Applying is a solving problem by applying knowledge and facts.
d. Analyzing is an examination and breaking information by identifying the causes.
e. Evaluating is a present and defend the opinion by making judgments about the
information.
f. Creating is generate new products or ideas from the information.
Figure 1. Blooms’ taxonomy
According Assaly & Smadi, (2015) they reported that Krathwohl divided 6 categories
of lexical bloom became two parts. The cognitive level aspects divided into Higher-
Order Thinking Skill (HOTS) and Low-Order Thinking skills (LOTS). The Low-Order
Thinking skills (LOTS) are remembering, understanding and applying. On the other
hand, The Higher- Order Thinking Skill (HOTS) are analyzing, evaluating and
creating.

4. Method
In this research, the writers used content analysis as the method. Ary, Jacob and
Sorensen (2016: 457) explained that content analysis is a research method which
can be applied in written and virtual material as the source data of the research. The
source that can be analyzed using this method such as television program, news-
paper, magazine, textbook and many other documents. This paper used questions of
seafarer expertise test sheets as the analyzed material.
Collecting and Analyzing the Data
To collect the data, the writers highlighted each number of the instruction from the
questions for classifying the cognitive level of each question. There were 100
questions from the data which all the questions are multiple choice questions. There
writers got the data from the document of the data bank from the academy.
In analyzing the data, the writers used Taxonomy’s Bloom to categorize the
instruction of the questions. The writers put the instructions into 6 categories. They
were Remembering, Understanding, Applying, Analyzing, Evaluating and Creating.
Then the writers concluded the categories into HOTS and LOTS categories.
Findings
Based on the analysis, writers found the frequency of cognitive levels of seafarer
expertise test as follows:
Table 1. Cognitive level frequency of seafarer expertise test
No Category Frequency Percent
1 Remembering 51 51%
2 Understanding 43 43%
3 Applying 6 6%
4 Analyzing 0 0%
5 Evaluating 0 0%
6 Creating 0 0%
Total 100 100%
Based on the result, remembering level had 51 or 51% which the most dominant
category in this research. The second dominant was ‘understanding category’
consisted 43 or 43% from 100 questions. The third dominant category was applying
which consisted 6 or 6%. However, the category of evaluating and creating did not
exist in this research.

5. Remembering
The result of the table showed that the remembering was the most frequently
category from the questions of the test. It contained 51 or 51 % from 100 questions.
The questions referred to define, recall and name the material about navigation that
students had learnt in the University. It was in line with Murphy (2007:65) that in
knowledge level or remembering level students had to recall their short term
memory to answer the question. The example of questions contained remembering
such as:
31. A cumulative list of notices to mariners are published a year.
A. Three times
B. Once
C. Four times
D. Twice
From the question, the writers concluded that it was remembering level. The question recalled
the students’ memory from their knowledge during lecturing. The students had to
remembering how many times that notices to mariners are published. However, when the
students forget the knowledge, they could not answer the question.
48. The reference datum used in determining the heights of land features on most charts is
A. Mean sea level
B. Mean high water
C. Mean low water
D. Half-tide level
The question number 48 was one of remembering level. In this question, the
students had to name from the definition. The students had to recall their short term
memory from the material that had been learned. The students should remember the
term to answer the question.
100. The marking A is___
A. Way point
B. Whell Over Point
C. Paralell index
D. Altering course
According to the question, the level of the question is on remembering. In this
question, the students have to define the term from the question. The students had to
remember the definition from the term to answer the question.
Understanding
The understanding level consisted 43 number or 43%. The number was lower
than remembering. In this level, the students should understand the concept of the
material not only remember the material. The understanding referred to choose,
conclude and classify the question. It was in line with Murphy (2007:65) that in
understanding or comprehension level the students not only remember the concept
but also the students had to interpret the concept. The questions would be explained
bellow:

6. 18. What is TRUE concerning new editions of Light Lists?
A. Supplements to new editions are issued monthly by the U.S. Coast Guar
B. New editions are published by the National Ocean Service.
C. New editions are corrected through the date shown on the title page.
D. None of the above
In this question, it was on understanding level. The students should understand the
concept not only remember the concept to answer the question. The question
encouraged the students to choose the true statement from the concept which need
understanding.
65. The scale on a chart is given as 1:5,000,000. This means that .
A. 1 inch is equal to 5,000 inches on the Earth's surface
B. 1 nautical mile on the chart is equal to 5,000 inches on the Earth's surface
C. 1 inch is equal to 5,000,000 inches on the Earth's surface
D. 1 nautical mile on the chart is equal to 5,000,000 inches on the Earth's surface
Based on the question number 65, it located on understanding level. The students
had to
interpret the question to answer the question. The students had to explain the scale
from the question to answer the question based on their understanding about the
material that they had learned.
90. The depth of water on a chart is indicated as 23 meters. This is equal to____
A. 11.5 fathoms
B. 12.6 fathoms
C. 69.0 feet
D. 78.6 feet
The question was categorized as understanding question. It showed that the students
had to conclude or interpret the question to answer the question. The students not
only had to remember the concept but also understand the concept about the
material for answering the question above.
Applying
Applying category consisted 6 or 6 %. In this category the students had to apply the
concept of the material in different context. The statement was in line with Issacs
(1996:2) that in Applying category or application category the students has to use
and apply the concept based on the natural situation. It meant that the students had
to apply the concept based on the other situations naturally. The example of the
questions contained applying category explained bellow:
7. You will be loading in Boston Harbor to a maximum draft of 32'06". The charted
depth of an obstruction in the channel near Boston Light is 30 feet and you wish to
have 3 feet of keel clearance. The steaming time from the pier to the obstruction is
01h 05m. What is the latest time (ZD +4) you can sail on 17 May 1983 and meet
these requirements?
A. 1610
B. 1728

7. C. 1821
D. 2350
In this question, it was one of applying category. The students not only master the
material but also they should apply it based on the other situations. In this question
the students had to calculate the questions to get the answer of the question.
12. You are enroute to assist vessel Vessel A is underway at 4.5 knots on course
233°T, and bears 264°T at 68 miles from you. What is the course to steer at 13 knots
to intercept vessel A?
A. 249°
B. 256°
C. 262°
D. 268°
Discussion
Based on the question, it was categorized to applying category. The students had to
determine intercept Vessel A based on the clue of the question. The students also had
to calculate for answering the question. However, although the students remember
and understand the concept, sometimes the students cannot apply the concept in
different situation.
26. The distance covered by a ship on four successive days were 320, 300, 310 and
330 nautical miles respectively. Find the average daily run speed if each day is 24
hours long
A. 330 naut. Miles
B. 300 naut. Miles
C. 320 naut. Miles
D. 315 naut. Miles
In question 26, the students had to determine the average daily run speed based on
another condition. The students should be able to apply the concept with another
condition. The students had to master the concept and make calculation to answer
the question.
Based on the result of the research and the discussion, the writers concluded that
the questions of seafarer expertise test of UKP sheet still emphasized in Low Other
Thinking Skill (LOTS). It showed that only 3 categories which existed in this research
categorized as Low Other Thinking Skill (LOTS). Unfortunately, the writers did not
find High Other Thinking Skill (HOTS) in this question. The problem was not
language problem which the test used English. Siregar (2017) stated that language
does not give effect for the level of difficulty of test. The result was in line with
stetment of Nasrudin (2017) that most of Indonesian test was still in LOTS level. In
contrast, Sulistyo (2017) stated that students in higher education should be exposed
more to such activities which can improve their HOTS level starting from analyze,
evaluate and create.it meant that the test should more emphasis HOTS than LOTS
because it is in University level.

8. Conclusion
The aim of this research is to determine the cognitive level of the questions of
seafarer ability examination by using bloom taxonomy focusing on Plan a voyage and
conduct navigation part. The writers wanted to find the level of test which is Higher
Order Thinking Skill (HOTS) and Lower Order Thinking Skill (LOTS) category. The
result showed that the questions contained 51 % of Remembering, 43%
understanding and 6% of Applying category. Those categories were categorized as
Low Other Thinking Skill (LOTS).On the other hand, the category of High Order
Thinking Skill (HOTS) such as Analyzing, Evaluating and creating did not exist in this
research. From this finding, hopefully there will be a good policy from the
transportation ministry through PUKP 5 to validate the test in order to have Hots
level, since we are now facing 4.0 industry revolution era.
References
Adam, E. A (2015). Bloom’s taxonomy of cognitive learning objectives. Journal of
Medical Library,103 (3),152-153. Doi: 10.3163/1536-5050.103.3.010.
Anwar, N.H & Sohail, M.M (2014). Assessing the Learning Level of Students through
Bloom’s Taxonomy in Higher Education in Punjab. Journal of Educational and
Social Research MCSER Publishing, Rome-Italy,4(3),83-88. Doi:
10.5901/jesr.2014.v4n3p83.
Ary, Donald. Lucy Cheser Jacobs, and Christine K. Sorensen. 2016. Introduction to
Research in Education Eighth Edition. Wadsworth, Cengage Learning.
Assaly,R.I & Smadi,O.M (2015). Using Bloom’s Taxonomy to Evaluate the Cognitive
Levels of Master Class Textbook’s Questions. ELT Canadian Center of Science and
Education,5(8),101-110.
Bloom, B.J (1956). Taxonomy of Educational Objective. London. Longman.
Febriani.,Usman, B.,&Muslem (2019). Analysis of reading comprehension questions by
using revised bloom’s taxonomy on higher order thinking skill (hots). English
education journal,10(1),1-15.
Kamlatsi, I & Sahan, A (2018). Descriptive Analyses on English Test Items based on the
Application of Revised Bloom’s Taxonomy. Metathesis: Journal of English
language literature and teaching,2(2),203-210. DOI
10.31002/metathesis.v2i2.847.
Krathwohl, David R. (2002). Revision of Bloom‟s Taxonomy : An Overview, Theory
Into Practice,4(4),212-219.

9. Lee, Yew-Jin, Mijung Kim, Qingna Jin, Hye-Gyoung Yoon, and Kenji Matsubara. (2017).
East-Asian Primary Science: Curricula: An Overview Using Revised Bloom’s
Taxonomy. Singapore: Springer.
Murphy,J.M (2007). A Review of Bloom's Taxonomy and Kolb's Theory of Experiential
Learning: Practical Uses for Prior Learning Assessment. The Journal of
Continuing Higher Education,55(3),63-66.
Nasrudin (2017). Integrating research and inquiry and higher order thinking skills in
teaching English as a foreign language: Reflections on research.the first
conference on linguistics and english teaching (1st
iclet) 2017 proceedings,1(1),44-
58.
Puluhuawa, Y (2015). An Analysis of Senior High School National Examination Based
on Bloom’S Taxonomy. English Literature Culture and Science,6(1),100-119.
Siregar, F.N (2017) the influence of intelligence and creativity of students in English
speaking toward the comprehension of the English term of economics. The first
conference on Linguistics and English Teaching (1st ICLET) 2017 proceedings. 1.
(1).333-341.
Sulistiyo, H. P (2017). Analyzing Cognitive levels of Students‟ Handbook for business
English Subject’s Questions. The first conference on Linguistics and English
Teaching (1st
ICLET) 2017 proceedings. 1. (1).361-366.
Tangsakul,P.,Kijpoonphol,W.,Linh,DH.,& Kimura,L.N (2017). Using bloom’s revised
taxonomy to analyze reading comprehension questions in team up in English 1-3
and grade 9 English o-net tests. International Journal of Research
Grantaalayah,5(7),31-41.