The document discusses the benefits of implementing the IAMU-PAES-P (International Association of Maritime Universities - Peer Assisted Self-Evaluation Scheme - Philippines) project at MAAP (Maritime Academy of Asia and the Pacific). It summarizes the perceived benefits from the perspectives of the MAAP PAES-P coordinator, MAAP community members, and MAAP students. Key benefits included promoting communication and collaboration, improving effectiveness and efficiency, validating MAAP's compliance with international standards, and providing objective assessments and recommendations to enhance MAAP's operations. The highest rated benefit was that it provided a venue for MAAP to share best practices. The study found the PAES-P implementation provided valuable insights and opportunities for

1. The IAMU-PAES-P Project
Implementation Benefits
IAMU-PAES-P* is International Association of Maritime Universities -Peer Assisted
Self-Evaluation Scheme- Philippines
Prof. Angelica M Baylon, PhD
MAAP-ERO Director / PAES-P WG Team member and coordinator

2. 2
Outline
1. Background and Rationale
IAMU -PAESP Project meetings
IAMU-PAESP project objectives /purpose
IAMU-PAESP project 3 Distinctive phases
 Self-evaluation
Evaluation defined
Self-evaluation defined
PAES form ( its coverage)
 Verification
PAES Tools used before the visit
PAES Tools used during the actual visit
Principles of PAES
Conduct of PAES
 Validation
PAES written Report to MAAO
2. Related Literature/Studies
- None, if ever more on forms of
assessment and students learning etc
- Study is first of its kind and a challenge
3. Methodology
a. Observation
b. Interview
c. Survey Questionnaires pilot tested
d. Literature Search / content analysis
4. Results/Findings
Benefits from 3 perspectives
• From MAAP WG member/coordinator (experience,
observation and
• From MAAP officers, faculty and staff (observation,
interview and questionnaire)
Analyzed the benefits quantitatively and
summarized benefits as IAMU-PAES and problems
or challenges as P
• From MAAP Students (observation and interview)
5. Concluding Remarks /Recommendations
6. References

3. 3
The IAMU-PAESP project
in partnership with
The second working meeting was held in Varna
Bulgaria during the 2017 IAMU Annual General
Assembly in September.
The first kick-off meeting of the project was held at
the IAMU headquarters in Tokyo Japan, on May 26-27,
2017.

4. 4
Objectives of the PAESP project
1. to suggest improvement in the quality of academic curriculum
leading to Bachelor of Science (BSc) and Certificate of
Competencies (COCs);
2. to improve the learning experience of all concerned and to
analyze/improve the MET system’s organization.

5. 5
Three Distinctive Phases of PAES-P
With the programme of activities mutually agreed by the External Peers and by MAAP, the PAES-P
project consists of 3 distinctive phases:
1. Self- evaluation
2. Verification
3. Validation
• The term evaluation is derived from theory “valoir"
which means to be worth.
• Thus, evaluation is a process of judging the value of
institutional accomplishments.
• It is an act or process that involves the assignment of
a numerical index to whatever is being assessed.
• It is an act or process that allows one to make
judgment about the desirability or value of a measure.

6. 6
Self-evaluation Defined
according to De McFarland it is as judging the quality of one's work based on evidence and explicit
criteria to do better work in the future.
“Comparing before situation with a current situation.“( Riley guide)
In this PAESP project, self-evaluation is defined as rating the entire
MAAP institutional system based on personal opinion on the
prevailing or current situation at the time the various areas with
statements have been measured or evaluated.

7. 7
The PAES form
It covers nine (9) areas :
1. Organization and Management; Students
2. Program
3. Education and Process
4. Academic Staff and support Personnel
5. Professional Training and Internships
6. Facilities and Resources
7. Program objectives
8. Stakeholders involvement
9. Continuing Education
rated by three (3) MAAP evaluators namely :
1. The Quality Management Representative
(QMR);
2. the Assistant Vice-President for Academics
(AVP)
3. and his Academic Dean.
Z = (X- u)/6+ 5
standardization formula

8. 8
Verification
Basis or Tools Used by the PAESP team for verification before the visit
The basis of external peer evaluators were those supplied in advance by the host institution.
1. accomplished self-evaluation PAES forms by MAAP 3 staff
2. information/data voluntarily provided by MAAP
3. data collected from the institution's web pages , other web-
based sources of information, including data available from
other official sources, before their visit at MAAP

9. 9
Verification
Basis or Tools Used by the PAESP team for verification during the actual visit
(Oct 25-26,2017)
 meetings and interviews with the
management and staff responsible for
the different areas listed in the PAES
form ( 27 staff led by Vadm Santos)
 Supporting academic staff and
students from all years of study were
also interviewed.
 Various MAAP documents, resources,
& facilities were visited as per
scheduled agreed programme of
activites. Various documents shown
thru both formal & informal
procedures.

10. 10
Principle of Peer Evaluation
Verification
The PAES project followed the principle of peer evaluation wherein the
peers are of the same rank.
Those answered the PAESP forms from MAAP were the Assistant Vice-
President, the Dean, and the QMR.
Conduct of Peer Evaluation
The conduct of peer evaluation at MAAP was brief, but it was based on
objective methodologies with trained observers who provided constructive
feedback under an open communication and trust.

11. 11
Validation
After the external peers had verified and learned in depth about MAAP’s mode of operation and its
compliance with general educational principles and PAES statements, on February 11, 2018,
MAAP finally received the 14-page PAES-P report on MAAP site visit.
The report consists of commendable findings in the nine areas.
Importantly, the report has valuable suggestions to improve the
overall educational experience at MAAP with emphasis that their
recommendations can be carried out as part of the MAAP
procedures with no need for the external peers’ further
involvement.

12. 12
Related Literature or Studies
Hence, this study about the benefits of the PAES-P
implementation is a challenge and the first of its kind.
There is NO published related literature or studies
on how those involved in the implementation of the
PAES project view the methods and its benefits.
Most of the relevant studies or related works of literature about self or peer assessments deal
with different assessment forms and their benefits for students and teachers in general.

13. 13
Methodology
This study utilized the following methods of data collection :
observation; interview; questionnaire , literature review search and
documentary/content analysis
The accomplished 22 (out of 27) questionnaires

14. 14
Discussion/Findings
Perceived Benefits from three (3) perspectives :
1. the MAAP PAES-P coordinator/member of the working group;
2. the MAAP community who participated directly or indirectly
in the PAESP visit
3. and the MAAP students who were interviewed.

15. 15
From MAAP-PAESP- P working group member/
coordinator perspective
the initial kick-off meeting held in IAMU
headquarters Tokyo Japan
2017 May
2017 Sept the 2nd meeting with the PAES-P team
in Varna Bulgaria
2017 Sept-Oct the needed coordination at MAAP from September to October 2017 for the
PAESP site visit at MAAP and STCW 2017 Workshop on “Quality Education at
the MHEIs”
2018 Feb tasked to determine the benefits of the PAESP project implementation in the
Philippines and present them in the IAMU-MARINASTCW 2018 Conference
with theme “Philippine MET Towards Globally Competitive Officers” on
February 22-23, 2018 at Midas Hotel, Manila

16. 16
Perceived benefits
As coordinator, the PAES-P project is perceived as an educational and
training experience, with the opportunity to have traveled, learned from
foreign counterparts and gained new expert friends
The PAES-P project had provided copies of many learning
materials.
The PAES Manual, and most especially the different IAMU
works of literature on Key Performance Indicators (KPIs)
and papers on managing universities

17. 17
Perceived benefits
 one of the perceived benefits is that internal professional disagreements
have been resolved
As PAESP member, was excluded in accomplishing forms with ERO office
responsible in computing ratings , saw the rate of a colleague evaluating his
area as 2 out of 10 ( believed as a MAAP strength )
MAAP strictly followed the PAES-P guidelines
“ to never interfere with the rate provided by colleague's self-
assessment or evaluation as it is the role of the external PAES-P
WG to validate the score “
In the PAES-P final validation report
• items rated low by MAAP, were viewed as commendable
items rated very high by MAAP were provided suggestions for improvement.

18. 18
Perceived benefits
 strengthened ties with co-IAMU member Institutions from Europe: Croatia
( UR) ; Denmark ( SIMAC) and Sweden (CUT)
 gained confidence and scholarly experience to be grateful for in
lifetime
The PAES-P WG exhibited highest standard of
professionalism, honesty, & integrity in their
interaction with institutional personnel & students
which is indeed an excellent learning and
networking opportunity, as we focused on our
similarities rather than differences.

19. 19
From MAAP Community Perspective
The signed attendance sheet
during the PAES visit
became the basis for the
distributed 27 questionnaires
to MAAP officers/faculty/staff.

20. 20
From MAAP Community (27) Perspective
(30 comments reduced to 20 ; 20 further reduced to 8 Major Benefits). Out of 27 surveys,
only 22 accomplished forms returned on time, & the same were analyzed using SPSS
version 25. MAAP President was the first to submit the accomplished questionnaire
to ERO with his signature .

21. From MAAP
Community
Perspective
Rank Name Mean Final Rank
1 Provided venue for MAAP share its best practices and initiatives 4.82 1
2 Provided informationon areas of potential improvement, best practices, or alternatives
that will enhance MAAP operation.
4.73 2
2 Confirmed that the MAAP documents prepared for PACUCOA Level 3 Accreditation
indicates that MAAP system is at par with Internationalstandards and best practices
4.73
4 Provided an objective assessment of MAAP programs and operations 4.68 3
4 Supported MAAP’s external peer evaluation or benchmarking 4.68
6 Encouraged the institution to innovate, improve competitiveness, attractiveness and
self-perception
4.64 4
6 Encouraged teamwork and involvementof all concerned in MAAP processes when
preparing for audits
4.64
6 Lifted MAAP’s role and status from being passive to being active IAMU member 4.64
9 Promoted external objective assessment and critique in order to yield a quality product
or program.
4.59 5
9 Assured the faculty and administrators that MAAP programs are performingat peak
efficiencyto meet stakeholder needs
4.59
9 Encouraged MAAP management and personnel to strive for a more advanced and
deeper understanding of its processes
4.59
9 Encouraged a deeper approach to quality system 4.59
9 Brought combined expertise across all areas of administration, unlikely to occur with a
single consultant. The peer evaluators share a common approach and work collectively.
4.59
14 Increased responsibilityand autonomy 4.57 6
15 Initiated talks about possible future developments and strategic goals 4.55 7
15 Encouraged MAAP to consider certain education activities being done by other co-IAMU
members
4.55
17 Provided a new system of evaluation and standards common with other co-IAMU
member
4.50 8
18 Helped peers resolve internal professionaldisagreements 4.45 9
18 Provided an avenue to bring education and informationto MAAP faculty and
administrative participants
4.45
20 Ensured objective measurement of KPI level ( high or low) for future success of the
organization
4.41 10
Composite 4.60
Table 1 presents
the mean
perceptions of 22
out of 27 MAAP
officers, faculty and
staff who
participated/intervie
wed during the
PAES-P Site visit.
Table 1
( overall high regard to
the PAES-P
implementation visit
having a )
composite mean
of 4.60.

22. From MAAP
Community
Perspective
Rank Name Mean Final
Rank
1 Provided venue for MAAP share its best practices and initiatives 4.82 1
2 Provided information on areas of potential improvement, best
practices, or alternatives that will enhance MAAP operation.
4.73 2
2 Confirmed that the MAAP documents prepared for PACUCOA
Level 3 Accreditation indicates that MAAP system is at par with
International standards and best practices
4.73
4 Provided an objective assessment of MAAP programs and
operations
4.68 3
4 Supported MAAP’s external peer evaluation or benchmarking 4.68
6 Encouraged the institution to innovate, improve competitiveness,
attractiveness and self-perception
4.64 4
6 Encouraged teamwork and involvement of all concerned in MAAP
processes when preparing for audits
4.64
6 Lifted MAAP’s role and status from being passive to being active
IAMU member
4.64
9 Promoted external objective assessment and critique in order to
yield a quality product or program.
4.59 5
9 Assured the faculty and administrators that MAAP programs are
performing at peak efficiency to meet stakeholder needs
4.59
9 Encouraged MAAP management and personnel to strive for a
more advanced and deeper understanding of its processes
4.59
9 Encouraged a deeper approach to quality system 4.59
9 Brought combined expertise across all areas of administration,
unlikely to occur with a single consultant. The peer evaluators
share a common approach and work collectively.
4.59
14 Increased responsibility and autonomy 4.57 6
15 Initiated talks about possible future developments and strategic
goals
4.55 7
15 Encouraged MAAP to consider certain education activities being
done by other co-IAMU members
4.55
17 Provided a new system of evaluation and standards common with 4.50 8
The 2nd top benefits – provision of
information on areas of potential
improvement, best practices, or
alternatives that will enhance MAAP
operation ; confirmation that the
MAAP documents prepared for
accreditation indicate that MAAP
system is at par with International
standards and best practices and
having provided an objective
assessment of MAAP programs and
so on
The Highest Mean
rating = 4.82
(venue to share MAAP best
practices and initiatives)

23. From MAAP
Community
Perspective
Rank Name Mean Final
Rank
18 Helped peers resolve internal professional disagreements 4.45 9
18 Provided an avenue to bring education and information to
MAAP faculty and administrative participants
4.45
20 Ensured objective measurement of KPI level ( high or low) for
future success of the organization
4.41 10
Composite 4.60
On the other hand 3 /20 benefit statements obtained the lowest mean ratings (Table 1).
• resolve internal disagreements was ranked low - probably because that benefit was
experienced by only me - a member of the PAESP team who cited that as benefit
* other two statement benefits that were ranked lowest pertain to KPI and on info and educ.
- Indicating that majority of the respondents have not participated in any IAMU training workshop
on KPIs or have not read the various information materials or PAES forms . This findings is a
strong indicator, that training or seminar about PAES prior to its implementation would be
beneficial.
Lowest mean rating= 4.41( KPI and Info /education ) and
4.45 (resolve internal professional disagreement )

24. 24
From MAAP Community Perspective
( 20 statements were categorized into 8 Benefits and were ranked by 22/27 )
Major Benefits of PAESP Project Implementation Rank Final Rank
1. Promotes Communication/ Collaboration and Cooperation 1 1
2. Improves Effectiveness/ Efficiency 2 2
3.Provides Proofs/Validation/Confirmation 2
4. Objectivity/ Independence / Neutrality 2
5. Educates and trains 5 3
6. Demonstrates Responsibility/ Duty /Concern 6 4
7. Presents a Balanced View/ Sensible or Well-Adjusted 7 5
8. Ensures Accountability/ Answerability 8 6
Table 2
Table 2 reveals the ranking of the 8 categorized major benefits of PAESP visit
* No correct and wrong answers . It can be surmised that benefits depends on one’s position /experience
As evident in Table 2, promoting communication, collaboration, and cooperation or Unity
topped the significant benefits of PAESP project implementation as perceived by the respondents.
Least benefits perceived Accountability/ Scorecard balanced view and support

25. 25
Results both supports and confirms the past PAESP reports but this MAAP Study had given and more
scientific and detailed picture of the PAESP Implementation Benefits
General Dimensions of the 8 Benefits found is abbreviated as IAMU-PAES- P for easy recall
with the last -P as the perceived challenges.
I
A
M
U
P
A
E
S
P
 Information
(Educates or Trains)
 Accountability/ Autonomy
/Authenticity and Assurance
 Motivation
 Unity or Teamwork
 Proofs/Validation/
Objectivity
 Assistance
 Effectiveness/ Efficiency
 Sensible Support / Score
well-balanced
 Perceived problems/
challenges

26. 26
Information (Educates or Trains)I
External expertise is an avenue that had brought education and information to MAAP faculty and
administrative participants. The dialogue and discussions were opportunities to identify and at times
enlightened misunderstandings or misperceptions about the organization or administration.
Moreover, the PAES-P team had facilitated a one-
day seminar –workshop for MAAP and selected
HEIs in the country that was organized by
MARINA on Oct 27, 2017.

27. 27
Accountability/ Autonomy /Authenticity
and Assurance
A
Faculty and administrators gained assurance that MAAP sponsored programs are performing at peak
efficiency to meet stakeholder needs as it also provided all concerned, the opportunity to demonstrate
their respective duties, responsibilities or accountability during the visit.

28. 28
MotivationM
All concerned were motivated, inspired and duty-bound to strive for a more advanced and
more in-depth understanding of its processes; more profound approach to quality system
and possibly consider specific education activities being done by co-IAMU members.
The action motivated collegiality and open-
minded discussions among management,
staff members, students and peers for
possible improvements in the organization.

29. 29
U Unity or Teamwork
The PAES implementation at MAAP promoted
(“communication, collaboration, and cooperation”)
which is the MAAP slogan for 2018)

30. 30
P Proofs/Validation/Objectivity
The external experts, who are newly exposed
to MAAP as a new institution, had recognized
its strengths, weaknesses, and ways to
improve the institution, which may have been
overlooked by those working within the system.
And had provided OBJECTIVE assessment
MAAP documents prepared for the other
accreditation bodies have been confirmed,
proving that MAAP system is at par with
international standards and best practices.

31. 31
A Assistance
MAAP can count on them for a systematic approach to identification
of activities that may be improved, methods to decide on and actions
leading to identified goals as well as verification process.
All concerned were provided the
confidence that in case of questions
or need for advice or assistance,

32. 32
Effectiveness/ EfficiencyE
MAAP embraces peer evaluation both
institutional and external (CHED, PACUCOA,
MARINA, PSB, OPITO, DNV-GL, MILTT etc.).
Information on areas of potential improvement, best practices, or alternatives that will
enhance MAAP operation, were provided as the external peers brought a lot of their
international experiences. Planning skills were enhanced to ensure more effective task
management.

33. 33
S Sensible Support /
Scorecard well-balanced
The peer evaluators share a standard approach
and work collectively to provide the most reasonable
and sensible recommendations to MAAP.
Supported by detailed
standards ( KPIs) in a form
of scorecards rated
with no bias

34. 34
P Problems or Challenges Perceived
( with the benefits come the perceived problems )
Nevertheless, the benefits and joy
gained from this exercise had beaten
the challenges.
Benefits
Problems

35. 35
From MAAP Students Perspective

36. 36
From MAAP Students Perspective
As part of this study, the students (
1st class, 3rd C/ and 4th Cl ) were
individually interviewed by me , as
regards their PAES visit
experience.( Ref: photo taken with
the PAESP team)
And a lot more…

37. 37
From MAAP Students Perspective
About their experience or moments with the IAMU-PAES team, the following have been noted:
( Note: it was more of its effect on them and about their institution MAAP)
 Privileged as it is not always that they be called by foreign visitors
for an interview
 Proud to be part of MAAP
 Excited to be interviewed
 Mixed emotions including being nervous
 Confirmation that MAAP is excellent
 Enhanced confidence
 It was shore leave at MAAP and glad to be at MAAP during the
PAESP visit
 Confirmed that MAAP is the best maritime school to be visited by
PAESP
 Honored to be called and included to the group of students called
for the interview

38. 38
Concluding Remarks / Recommendations
Indeed, a key factor in the success of the external peer review was the MAAP leadership
and its community receptivity to the process.
MAAP community generally viewed
the external peer review team as… and the process as
than standard internal peer review.
That sense of partnership and collegiality was essential to
the quick and efficient implementation of changes in any at
MAAP, following the external peer review.
Success of PAESP

39. Concluding Remarks / Recommendations
The external peer review MAAP experience underscored t or highlighted he
importance of the on-going relationship with accrediting bodies such as PACUCOA,
DNV , MIILT PSB and OPITO , which we in MAAP intends to maintain.
Although external peer evaluator is not universally
required by the Philippines government at this
time,
this is a good best practice to possibly consider by
MARINA,
that would intensify focus on quality
educational and training services as we all
would like to maintain.

40. 40
Concluding Remarks / Recommendations
It is given in any literatures that in order to gain benefit from any
peer and self-evaluation schemes, those involved, needs training
in the specific scheme being used; as ideally they may play an
important role in devising the PAES scheme.
The involvements reported here, which involved (N =22)
group of MAAP community, included no training or at
least information for measures on PAES process (only 6/
22 from MAAP knows )
Nonetheless, the results show that MAAP community felt,
benefited from their involvement.
The results also present clear evidences that training or other measures (having been ranked
the least perceived benefits) might be beneficial for the success of succeeding PAESP
implementation

41. 41
Recommendations
 MOU between IAMU and MARINA may be intensified
 ( For the benefit of none IAMU members )
 Training the trainers may be initiated ( those who experienced PAES visit and
had attended IAMU –MARIAN training on KPIs)
 Enhancement of the PAES form may be considered
Over-all remarks and recommendations
From the MAAP experience, PAES project in the
Philippines is undeniably a valuable quality
improvement tool and a key to any institution
long-term success.

42. References
Francic, V. Askholn, J, Ljungklint, J. and Yutaka E (Feb 11, 2018), 14-page IAMU-PAESP Report
on the Site Visit to MAAP
Lam, Ricky (2010) A Peer Review Training Workshop: Coaching Students to give and Evaluate
Peer Feedback in TESL Canada Journal Vol 27 No 2 Spring 2010 pp. 114 -127 Retrieved at
https://files.eric.ed.gov/fulltext/EJ924064.pdf on Feb 14, 2018
McLeod, S. A. (2008) LIKERT Scale. Retrieved from www.simplypsychology.org/likert-scale.html
on February 11, 2018
Min, H.T. (2005) Training students to become successful peer reviewers. System, 33, pp 293-308.
Savita (2015) Self -evaluation, peer-evaluation , patient satisfaction and utilization review retrieved
from www.slideshare.net/MnSavita/self-evaluation-peer-evaluation on February 15, 2018
Zec, Damir (2015) IAMU Peer Assisted Self - Evaluation Scheme (PAES) Manual, University of
Rijeka