The document outlines the development and importance of accreditation in engineering education, highlighting various international accords such as the Washington Accord. It details the process of outcome-based education (OBE), emphasizing the need for alignment between program educational objectives and outcomes to meet industry standards. Additionally, it discusses the roles of stakeholders and evaluators in ensuring quality and continual improvement in engineering programs.

1. Accreditation & Outcome
Based Approach
Prof Megat Johari Megat Mohd Noor
UTM QRIM@KL & MJIIT
Lahore/Faisalabad/Jamshoro, Pakistan
September 2016

2. Introduction

3. 3
We are joining the Mutual Recognition Train
(MRT) !

4. 4
WASHINGTON
ACCORD
SYDNEY
ACCORD
DUBLIN
ACCORD
4 YEARS
3 YEARS
2 YEARS
IPEA
International Professional Engineers Agreement
(ENGINEERS MOBILITY FORUM)
APEC ENGINEER
FEANI / EUR-ACE / ENAEE
(EUROPE)
3 + 2
YEARS
NABEEA
(ASIA)
EDUCATION ACCORDS PRACTICE AGREEMENTS
IETA
International Engineering Technologist Agreement
(ENGINEERING TECHNOLOGISTS MOBILITY FORUM)
INTERNATIONAL
ENGINEERING
ALLIANCE (IEA)
(INTERNATIONAL
ENGINEERING MEETING, IEM)
AIET
Agreement of International Engineering Technician

5. 5
Development of International Engineering
Alliance
WA signed by 6
organisations
Development
of formal
peer review
processes
New Accords
and
Agreements
Development
of graduate
attribute
exemplars
28 Sep 1989
1990s
onwards
1997-2015
2001
onwards
IEA Established in 2007

6. 6
WASHINGTON ACCORD FULL SIGNATORY
1. Australia - Engineers Australia (1989)
2. New Zealand - Institution of Professional Engineers NZ (1989)
3. Canada - Engineers Canada (1989)
4. United States - Accreditation Board for Engineering and
Technology (1989)
5. United Kingdom - Engineering Council UK (1989)
6. Ireland - Engineers Ireland (1989)
7. Hong Kong China - The Hong Kong Institution of Engineers (1995)
8. South Africa - Engineering Council of South Africa (1999)
9. Japan - Japan Accreditation Board for Engineering Education
(2005)
10. Singapore - Institution of Engineers Singapore (2006)
11. Chinese Taipei - Institute of Engineering Education Taiwan (2007)
12. Korea - Accreditation Board for Engineering Education of Korea
(2007)
13. Malaysia - Board of Engineers Malaysia (2009)
14. Turkey - MUDEK (2011)
15. Russia - Association for Engineering Education of Russia (2012)
16. India - National Board of Accreditation (2014)
17. Sri Lanka - Institution of Engineers Sri Lanka (2014)
18. China - CAST (2016)
Provisional Status
19. Bangladesh
20. Pakistan
21. Phillippines
22. Peru
23. Costa Rica
24. Mexico
Potential Applicants
25. Thailand
26. Indonesia

7. 7
Accreditation

8. 8
Importance of Accreditation to Institutions of
Higher Learning
• Recognises institutional missions and goals
• Involves faculty/staff in evaluation and planning
• Assists institutions in determining the
acceptability of transfer credits
• Promotes “best practices” in education
• Increases visibility and reputation of the
institution
• Aids engineering schools to identify required
operational resources to institution
management

9. 9
Pakistan Washington Accord Route (2011 – 2016)
• Nominator (EME, GIKI)
• Mentor (Islamabad, Topi, Risalpur,
Faisalabad, Peshawar, Karachi,
Lahore)
• 1a Reviewer (Universities?)
• 1b Reviewer (ADM)
Prof Abang (MAL)
Prof Megat (MAL)
Prof Lock (SIN)
Kim (Korea)
Collins (UK)
Basil (NZ)
Nominator
Mentor
Prof Megat (MAL)
Ir Azlan (MAL)
Prof Lock (SIN)
Reviewer?

10. DECIDE
ENGINEERING
ACCREDITATION
BOARD (EAB)
Representatives of
Professional
Societies
Representatives of
Universities
Representatives of
Industries
Constituents in Accreditation
PAKISTAN
ENGINEERING
COUNCIL (PEC)
ACCREDITATION
DEPARTMENT
(AD)
EAB
EVALUATION
PANEL
ENDORSE
FACILITATE
RECOMMEND
Representatives of
Government
10

11. 11
EAB Manual
• From input based to outcome based
20.. 2014

12. 12
Programme
Objectives (…..)
& Outcomes
(…..)
Students
(…..)
Academic &
Support
Staff (…..)
Industry
Linkage (…..)
Facilities (…..)
Academic
Curriculum
(…..) Continual
Quality
I
mprovemen
t

13. 13
Programme Evaluators (PEVs)
 Chair (Criteria of appointment)
 Two members (Criteria of
appointment)
one member with extensive academic
experience and one member with
extensive industrial experience
- knowledgeable
- trained
- independent

14. Challenges
• Paradigm Shift – Outcome & Quality
• Maintain Fundamentals while Encourage
Inclusion of Latest Technology Advancement in
the Curriculum
• Allow Academic Innovation and Creativity
• Avoid Side-tracked
• Variety of Modes of Delivery

15. 15
Complex
Problems
Broadly Defined
Problems
Well defined
Problems
Solved using
limited theoretical
knowledge, but
normally requires
extensive practical
knowledge
Knowledge of
principles and
applied
procedures or
methodologies
In-depth knowledge
that allows a
fundamentals-
based first
principles analytical
approach
Depth of Knowledge Required

16. 16
Engineering & Technology Domain
Engineers
Technologists
Research & Design
Supervision &
Maintenance
Strong in
Mathematics,
Engineering
Sciences,
Professional
courses
(Theoretical)
Appropriate
Mathematics,
Engineering
Sciences,
Professional
courses
(Practical)
Education
Work
Engineering
Breadth & Depth
of Curricula
Technology
Breadth & Depth
of Curricula

17. 17
Programme EO / O Development/ Review
Internal Stakeholders
Teachers
Students
University
External Stakeholders
Potential Employers / Industry
Alumni
Regulatory Body
Course O / Content
Development / Review
1, 2, 3 ……
Course Implementation
1, 2, 3 ……
Course Assessment
1, 2, 3 ……
Teacher – Knowledge, Skills, Affective
Students – Teaching
Teacher – Descriptive Self Assessment
on Cohort’s Achievement
Programme Evaluation
Summative - direct
Exit Survey - indirect
Industry Survey - indirect
Alumni Survey - indirect
External – direct
Accreditation - direct
Educational Process & Stakeholders
Pull
factor
Internal Stakeholders
Teachers
Technicians
Students
Internal Stakeholders
Teachers
Students
External Stakeholders
Potential Employers / Industry
Alumni
Regulatory Body
External Assessor
Summative
Formative
/
Summative
Internal Stakeholders
Teachers
Specification

18. 18
(8) Leadership, governance
and administration
(6) Educational
resources
(1) Vision,mission and
learning outcomes
(5) Academic
Staff
(2a) Curriculum
Design
(2b) Curriculum
Delivery
INPUT (STUDENTS)
(4a) Selection of Students
(4b) Supporting Services
(7) Programme
Monitoring and
Review
(3) Student
Assessments
(9) Total Continual Quality
Improvements
OUTPUT
(GRADUATES)
STAKEHOLDER
NEEDS
AND
INSTITUTIONAL
MISSION
FEEDBACKS
FROM
STAKEHOLDERS
STAKEHOLDER’S
SATISFACTION
Academic IQA Practices in Perspective

19. 19
OBE
Programme Objective
(after 3-5 Years)
Programme Outcome
(at Exit)
Course/Unit/Learning Outcome
(Abilities & Intentional)
Directed & Coherent Curriculum
Graduate Relevant to Industry
Accountable

20. 20
Characteristics of OBE curricula
• Have programme objectives,
programme outcomes, course
outcomes and performance
indicators.
• Stated objectives and outcomes can
be assessed and evaluated.
• Centered around the needs of the
students and the stakeholders.

21. 21
Characteristics of OBE curricula
• Learning outcomes are intentional and
assessed using suitable performance
indicators.
• Programme objectives address the
graduates attainment in their career
within 3-5 years after their graduation.
• Programme outcomes (abilities attained
by students before they graduate) are
formulated based on the programme
objectives – TOP DOWN.

22. 22
Characteristics of OBE curricula
• Programme outcomes address Knowledge,
Skills and Attitudes to be attained by students.
• Course outcomes must satisfy the stated
programme outcomes. There is no need for
ANY (individual) course to address all
programme outcomes.
• Teaching/ Learning method may have to be
integrated to include different delivery
methods to complement the traditional
Lecturing method.

23. 23
OBE in a nut shell
 What do you want the students to have or
able to do?
 How can you best help students achieve
it?
 How will you know what they have
achieved it?
 How do you close the loop
 Knowledge, Skill, Affective
 PDCA
 Student Centred Delivery
 Assessment

24. 24
Plan, Do, Check & Act (PDCA)

25. 25
Strategy of OBE
• Top down curricula design
• Appropriate Teaching & Learning Methods
• Appropriate Assessment & Evaluation
Methods

26. 26
Different Levels of Outcomes
Programme Educational Objectives
Programme Outcomes
Course/subject Outcomes
Weekly/Topic Outcomes
Upon graduation
Upon subject completion
Upon weekly/topic completion
Few years after
Graduation – 3 to 5 years

27. Accreditation

28. 28
Preparation for Accreditation
• Comprehend the EAB Manual
• Prepare the SAR
• Address previous accreditation report
• Arrange the evidence
• Complete the Checklist
• Assign key persons according to accreditation
schedule

29. 29
SAR & Evidence

30. 30
Azlan
or
a man sitting
down in a
garden and a
lady passing
by?

31. Issues from
• Approval Report
• First Year Visit Report
• External Examiner’s Report
• Stakeholders’ Recommendations
• Recent Accreditation Visit
EVIDENCE BASED
Documents & Records
Interviews
Observations
CQI

32. Self Assessment Report
• Prepare a checklist of questions based on the following
criteria in preparing a SAR:
– Programme Objectives
– Programme Outcomes
– Curriculum and Learning Process
– Students
– Faculty
– Support staff
– Industry stakeholders
– Alumni
– Facilities and Infrastructure
– Institutional and Financial Support
– Continual Quality Improvement
– Industry Linkage
Exercise

33. Industrial Linkage
Using the Clause 9 of the PEC Manual, evaluate
the relevant part of the submitted SAR.
Exercise

34. 3.2.9 Criterion 9: Industrial Linkages
Students are expected to undertake assignments
from industry to provide solutions to complex
engineering problems. Students and faculty should be
encouraged to establish collaboration for R&D and
product development related projects, with due
regard to environmental and societal impact.
Feedback from the industry and employers is crucial
and an essential part of curriculum review process
used to evaluate attainment of the program
objectives.

35. 4.1.9 Industrial Linkages
4.1.9.1
Discuss the involvement of industry in
discussions and forums, professional
practice exposure, and collaborative
projects / research for the solutions
to engineering problems.

36. 36
List down HEI’s representatives (parties) that
will be involved in an accreditation visit
Exercise

37. 37
Item/Criteria List down the involved parties
Accreditation Planning
Opening Meeting
Programme Educational Objectives
(……)
Programme Outcomes (…….)
Curriculum (…….)
Students (……)
Academic & Support Staff (……)
Facilities (……)
Quality Management System (……)
Exit Meeting
Exercise

38. 38
• Sensible questioning
• Check records
• Observing processes
• Analyse inputs and outputs
• Organised using tables, matrices,
flowcharts and checklists
PROGRAMME EVALUATOR’S APPROACHES

39. 39
What are the six typical words that Programme
Evaluators (PEVs) would usually begin with,
when questioning?
Quiz
Programme Evaluators’ (PEVs’) Best friend –

40. 40
What are the methods/techniques employed by
Programme Evaluators (PEVs) when conducting
an accreditation exercise?
Quiz

41. 41
Cause for concerns at
Accreditation Decision Meeting
• Phases of OBE
– Planning
– Implementation
– Effectiveness
• CQI
• List of concerns
• Breadth & depth (taxonomy & complex problem)
• Staffing
• Industrial Training
• Commitment to change
• System failure
• Stagnant (no improvement)
• Repeat offender
• Safety

42. 42
Rubrics for New Programme, New Cycle &
Continuing

43. 43
What WA will be observing?
• Adherence to EAB document
• EAB PEV’s aplomb and decorum
• Probing questions (not interrogative)
• Discussion level
• Clarity of reports
• Graduate outcomes
• Health & safety at HEIs
• Equivalency of practice

44. 44
Opening Meeting
You are the Dean/HoD with three programmes
to be evaluated; Mechanical Engineering, Civil
Engineering and Electrical Engineering, for the
third cycle. Prepare a list of talk points to
address the Programme Evaluators (PEVs) at the
Opening Meeting.
Exercise

45. 45
• Welcome Evaluation Team
• Introduce team members
• Corrective & Preventive
Actions from previous
accreditation
• Short presentation on
Faculty/Dept/Prog strengths
• Fill up with the latest (within
a specified timeframe) if any
OPENING MEETING
10 minutes

46. 46
PEC Manual 2014
Programme Educational Objectives

47. Exercise
List down potential stakeholders
• Major
• Minor

48. 48
Programme Educational Objectives
• Broad statements: What graduates are
expected to achieve (BE) a few years after
graduation.
• Linked to programme outcomes
• Include feedback from employers, alumni,
academics and other stakeholders

49. PROGRAM EDUCATIONAL OBJECTIVE (PEO)
 Limit number of statements (manageable)
 No restatement of outcomes
 Forward looking and challenging
 Distinctive/unique features/having own
niche
 Specific, Measurable, Achievable, Result
oriented, and having a Time frame (SMART)
Programme Educational Objectives
49

50. 50
Write down your evaluation on the following
PEO statements
Exercise

51. 51
Write down your evaluation on the following
PEO statements
Exercise

52. 52
PEC Manual 2012
Programme Outcomes

53. 53
Programme Outcomes (PO)
What the graduates are expected to know and
able to perform or attain by the time of
graduation (skills, knowledge and
behaviour/attitude)
Need to distribute the outcomes throughout the
programme, and not one/two courses only addressing a
particular outcome
There must be a clear linkage between
Programme Objective and Outcomes

54. 54
Programme Outcomes
• Discuss on HEI’s possible approaches or
methods to demonstrate implementation of
the 12 programme outcomes
• Discuss on the possible models to show
attainment of the 12 programme outcomes
Exercise

55. 4 YEARS
WA 1
ENGINEERING
KNOWLEDGE
WA 2
PROBLEM
ANALYSIS
WA3
DESIGN
WA5
MODERN TOOLS
WA6 ENGR & SOC
WA7 ENV & SUST
WA8 ETHICS
WA4
INVESTIGATION
WA9
IND & TEAM
WA10
COMMUNICAT-
ION
WA11
PROJ MGMT &
FINANCE
WA12
LIFE LONG
PEO
WHAT YOU WANT YOUR GRADUATES TO BE IN 3 - 5 YEARS
EXTRA-CURRICULAR
UNIVERSITY
EXPERIENCE

57. 57
Problem Organised Project Work
or POPBL (Project Oriented Problem Based
Learning)
Problem Analysis Problem Solving Report
Literature Lectures Group Studies
Tutorials Field Work Experiment

58. 58
POPBL Requirements
• High degree of supervision
• Office space
• Lectures to be constantly changing or
renewed
• Flexibility in the distribution of resources

59. 59
Instructors/Supervisors
• Pedagogical skills
• Scientific skills
• Time management
• Project based on staff research

60. 60
Graduates’ Strength
AALBORG UNIV
• Strong in problem
solving
• Communication
• Cooperation
• General technical
knowledge
DENMARK
TECHNICAL UNIV
• Specialist
knowledge
• Technical
methodology

61. 61
Typical questions on PEO/PO
• What are the PEOs/POs?
• Who were involved in the development of the
PEOs/POs?
• How were they developed/improved?
• To what extent the stakeholders were
involved?
• How their attainment were determined?
• What were the improvements introduced?

62. 62
Curriculum
• Discuss on the possible relationship between
taxonomy levels and the different knowledge
profile with consideration to the 12 programme
outcomes
• What are typical probing questions in ascertaining
that student’s POs have been attained?
• What are typical probing questions in ascertaining
that student’s COs have been attained?
Exercise

63. 63
ASSESSMENT:
Processes that identify, collect, use and
prepare data for evaluation of achievement
of programme outcomes or educational
objectives.
EVALUATION:
Processes for interpretation of data and
evidence from assessment practices that
determine the programme outcomes are
achieved or result in actions to improve
programme.

64. Outcome-Based Assessment
Implementation
Strategy
Assessment
Strategy
Data
Sources/Assessment
instruments
Industrial project
Improve student
competence in
communication,
teamwork, and project
management
Exams, interview,
survey, observe,
assess skill level,
monitor
development of
skills
Reports, interview
schedule, survey,
observation records,
grades of exams and
projects, exit skill
checklist
Design course
Address industry
needs
Assessment by
industry, and
lecturers
List of assessment
criteria, observation,
reports, interview,
students evaluation,
exams, exit skill
checklist
64

65. 65
Assessment
– drives learning (necessary evil!)
– is formative or/and summative; to demonstrate
student’s competence in demonstrating a
specific outcome
– is the process that identify, collect, use and
prepare data that can be used to evaluate
attainment.

66. 66
Assessment
• Do not assess those that have not
been taught

67. 67
What Assessment?
• Assessing Student/Cohort (Course Outcome)
• Assessing Student/Cohort & Faculty
(Programme Outcome)

68. 68
Assessment Process
–Anecdotal vs. measured results
–Reliance on course grades only
–Over-reliance on indirect assessment
(survey)

69. 69
University Assessment & Evaluation
S
t
u
d
e
n
t
,
A
l
u
m
n
i
P
e
r
c
e
p
ti
o
n
E
m
p
l
o
y
e
r
,
I
n
d
u
s
t
r
y
P
e
r
c
e
p
ti
o
n
MEASURE & EVALUATE

70. 70
How will you know what they
have achieved it?
 Formative Assessment
 Summative Assessment
 Course Assessment
 Program Assessment
 Assessment Tools
 Direct and Indirect Assessment

71. 71
How do you close the loop ?
• Assessment Plan
• Who is doing what and when
• Stakeholder participation
• CQI in place

72. Programme Outcome Assessment Matrix
Outcome indicators
& core courses
PO 1 PO 2
Project Report A B
Course 1 B B
Course 2 C B
A: slightly, B: moderately, C:substantively - base on a
review of course materials (syllabus, learning objectives,
tests, other assessment…..)
Outcome 1: ability to …..
Outcome 2: ability to ….. 72

73. Course Assessment Matrix
Outcome-related
learning objectives
PO 1 PO 2
Explain A C
Perform calculation B B
Identify B B
Solve B C
A: slightly, B: moderately, C: substantively
Outcome 1: ability to …..
Outcome 2: ability to …..
73

74. 74
Rubric
4 – Exceeds
Criteria
3 – Meets Criteria 2 - Progressing
to Criteria
1 - Below
Expectations
Content Provides ample
supporting detail
to support solution/
argument
Provides adequate
supporting detail
to support solution/
argument.
Some details but
may include
extraneous
or loosely
related material.
Inconsistent or few
details that may
interfere with the
meaning of the text.
Organization Organizational
pattern is logical &
conveys
completeness
& wholeness.
Organizational
pattern is logical &
conveys completeness
& wholeness
with few lapses.
Little completeness
& wholeness,
though organization
attempted.
Little evidence of
organization or any
sense of wholeness
& completeness.
Style Uses effective
language; makes
engaging,
appropriate word
choices for
audience
& purpose.
Uses effective
language &
appropriate
word choices
for intended audience
& purpose.
Limited &
predictable
vocabulary, perhaps
not appropriate for
intended audience
& purpose.
Limited or
inappropriate
vocabulary for the
intended audience
& purpose.
Consistently
follows
the rules of
standard English.
Generally follows
the rules for standard
English.
Generally does not
follow the rules of
standard English.
Does not follow the
rules of standard
English.
Adopted from G.Rogers

75. 75
Performance Criteria/ Indicators -
Good Teamwork
Students are able to demonstrate
1. Positive contribution to the team project (minutes of
meeting)
2. Well prepared and participate in discussion
(observation)
3. Volunteer to take responsibility
4. Prompt and sufficient attendance
5. Aplomb and decorum

76. 76
Quiz
Courses PO1 PO2 PO9 PO10
C1 3 2 1 1
C2 2 1 2 2
C3 3 0 3 2
C4 2 1 3 1
Discuss on the potential problems, if any, where 3, 2, 1, and 0
refer to High, Moderate, Low, and No emphasis, respectively.
C1..4 refer to the courses, whereas POs 1,2,9 and 10 refer to
Programme Outcomes.
How would cohort POs attainment be obtained?

77. 77
Quiz
PO1 PO2 PO9 PO10
C1-CO1 + +
C1-CO2 + + +
C1-CO3 + + +
C1-CO4 + +
How would you design the assessment for
the above matrix?
CO: Course Outcomes +: There is assessment

78. 78
Quiz
Table 1 PO 1
Q1 CO1 +
Q2 CO2 +
Q3 CO3 +
Q4 CO4 +
Table 2 PO 1 PO9
Q1 CO1 + CO2 +
Q2 CO2 + CO3 +
Q3 CO3 + CO4 +
Q4 CO4 +
Discuss on the attainment
of COs and POs for both
Tables 1&2, where Qs are
questions set to address
the COs

79. 79
Quiz
Delivery Assessment
Lecture
Laboratory
PBL
Case Method
Project Based
Identify suitable assessment techniques for the
different delivery modes.

80. 80
Lessons learnt from accreditation activities
related to assessment
 Do not know the teaching plan
 Done without referring to the plan
 Do not know how to translate plan into assessment
 Assessing at low-medium level (not challenging)
 No feedback to students except at end of semester
 Do not know how to relate assessment to expected
outcomes
 Repetition
 Bulk marking
 Traditional assessments

81. 81
Big Picture
Programme or
Student
Improvement ?
Selective
Culminating
Hybrid
Taxonomy Level (Average, From, Up To)
Assessment – Constructive Alignment

82. 82
Curricula Models
Yr. 1
Yr. 4
Yr. 3
Yr. 2
K 70%
S&A
30%
K 70% K 70% K 70%
S&A
30%
S&A
30%
S&A
30%
Distribution of Knowledge, Skills & Attitude
elements throughout the 4 years
A B C D

83. 83
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
(Capstone Design Course)
MyOBE
Software

84. 84
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
(Capstone Design Course)
MyOBE Process
Module
Programme Outcomes
Attributes that are expected to be
attained by the students
Teaching and Assessment Plan
Planning of course outcome
Planning of course assessment
Course Assessment
Course Assessment Mark
Course Assessment Summary
Programme Evaluation
Programme Outcomes Summative
Trend Analysis

85. 85
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
(Capstone Design Course)
MyOBE
Snapshots
Lecturers’ Module:
Enter all course
assessment marks

86. 86
Evaluation of Outcomes at Programme Level
ECV3092 Civil Engineering Design
(Capstone Design Course)
MyOBE
Snapshots

87. 87
Final Year
Design Project
Final Year Courses
Third Year Courses
Second Year Courses
First Year Courses
Final Year Project
PO Attainment
Final Year Project
Final Year
Design Project
Final Year Courses
Third Year Courses
Second Year Courses
First Year Courses

88. 88
What constitutes strength?
 Exceeds the minimum standard set by the EAC Engineering
Accreditation Manual.
 Extensive benchmarking (not only via the external examiners
path) with more established programmes/institutions.
 The curriculum is built on strong fundamentals (engineering
sciences) and appropriate engineering knowledge according to
the discipline, which transcend national boundaries.
 Generic attributes (professional and/or interpersonal skills)
should also be evident to prepare graduates for the advanced
part of their career.

89. 89
What constitutes strength? Cont…
• A curriculum with clear (measurable) objective(s) and
outcomes (that satisfies the (12) EAC stipulated
outcomes)
• Involved stakeholders, both internal and external,
extensively
• An appropriate working load for students
determined through extensive consultation with the
academics (Usually a 15 – 16 credit per semester
loading)
• Blend of delivery methods

90. 90
What constitutes strength? Cont…
 Programme challenges students to achieve greater
heights than just satisfying the minimum standard
 Attain competency in the open-ended project
based and problem oriented courses
 Majority of the staff has PhD qualification and the
number available indicates a low staff-student
ratio (that enables greater contact with students)
 The academic staffs also conduct research that
permeates/contributes to teaching and learning.

91. 91
What constitutes strength? Cont…
• Over and above Industrial Training (extensive &
distributed professional exposure) that does not
compromise on the cognitive domain
• Ergonomics is taken seriously by the institution to reduce
occupational hazard
• Safety culture
• Show that they have the plan and the completion of the
quality cycles is widespread
• Monitoring of the QMS also indicates strength.

92. 92
What constitutes strength? Cont…
• Students’ ability to give opinion and articulate
with substance
• Students are clear of their goals upon graduation
and highly motivated during their course of study
(“constructive criticisms”)
• Widespread involvement of students in co-
curricular activities (not forced as part of
curriculum nor limited to small group of
students).

93. 93
What constitutes strength? Cont…
• Academic staff with Professional Engineer
status
• Academic staff are actively participating in
professional activities (not merely members)
• Design courses are taught by experienced
academics (with consultancy experience or
professional engineers).

94. 94
What constitutes strength? Cont…
• Up to date facilities are made available and
they exceed the recommended student-
equipment ratio appropriate to the relevant
discipline.
• Extensive electronics publications for life long
learning, project based courses and the final
year project

95. Taxonomy & Course Outcome

96. • Knowledge (list)
• Comprehension (explain)
• Application (calculate, solve, determine)
• Analysis (classify, predict, model,derived)
• Synthesis (design, improve)
• Evaluation (judge, select, critique)
Bloom’s Taxonomy
96

97. 97

98. 98
Higher order
lower order Intermediate

99. 99
Higher order
lower order Intermediate

100. Why are course outcomes important?
• Define the type and depth of learning students are
expected to achieve
• Provide an objective benchmark for formative,
summative, and prior learning assessment
• Clearly communicate expectations to learners
• Clearly communicate graduates’ skills to the
stakeholders
• Define coherent units of learning that can be further
subdivided or modularized for classroom or for other
delivery modes.
• Guide and organize the instructor and the learner.
100

101. Three components of a learning outcome
Reaching the “Standard” (criteria of acceptable level of
performance)
• describe the principles used in designing X.(Verb)
• orally describe the principles used in designing X. (Verb & Condition)
• orally describe the five principles used in designing X. (Verb &
Condition & Standard)
• design a beam. (Verb)
• design a beam using Microsoft Excel design template . (Verb &
Condition)
• design a beam using Microsoft Excel design template based on BS
5950:Part 1. (Verb & Condition & Standard)
101

102. Learning outcomes by adding a condition and
standard
Poor
• Students should be able to design research.
Better
• Students should be able to independently design and
carry out experimental and correlational research.
Best
• Students should be able to independently design and
carry out experimental and correlational research that
yields valid results.
Source: Bergen, R. 2000. A Program Guideline for Outcomes Assessment at Geneva College
102

103. WA Knowledge Profile (WK)
103

104. WA Knowledge Profile
(WK)
4 YEARS
WK1
natural sciences
WK3
engineering
fundamentals
WK2
mathematics,
numerical
analysis,
statistics,
computer and
information
science
WK4
engineering
specialist
knowledge
WK5
engineering
design
WK6
engineering
practice
WK8
research
literature
WK7
engineering in
society
104

105. Theory-based natural sciences WK1
Conceptually-based mathematics, numerical
analysis, statistics and formal aspects of
computer and information science to
support analysis and modelling
WK2
Theory-based engineering fundamentals WK3
Engineering specialist knowledge that
provides theoretical frameworks and bodies
of knowledge for the practice areas; much is
forefront
WK4
WA Knowledge Profile (Curriculum)
105

106. WA Knowledge Profile
Knowledge that supports Engineering design in
the practice areas
WK5
Knowledge of Engineering practice
(technology) in the practice areas
WK6
Comprehension of the role of Engineering in
society and identified issues in engineering
practice: ethics and professional responsibility
of an engineer to public safety; the impact of
engineering activity: economic, social,
cultural, environmental and sustainability
WK7
Engagement with selected knowledge in the
Research literature
WK8
106

107. WA Programme Outcome or
Graduate Attributes (WA)
107

108. 108
Washington Accord Graduate Attributes
PROGRAMME OUTCOMES
WA1 Engineering Knowledge Breadth & depth of knowledge
WA2 Problem Analysis Complexity of analysis
WA3 Design/Development of
Solutions
Breadth & uniqueness of engineering problems i.e. the extent to
which problems are original and to which solutions have
previously been identified and coded
WA4 Investigation Breadth & depth of investigation and experimentation
WA5 Modern Tool Usage Level of understanding of the appropriateness of the tool
WA6 The Engineer and Society Level of knowledge and responsibility
WA7 Environment and
Sustainability
Type of solutions
WA8 Ethics Understanding and level of practice
WA9 Individual and Team Work Role in and diversity of team
WA10 Communication Level of communication according to type of activities performed
WA11 Project Management and
Finance
Level of management required for differing types of activity
WA12 Life-long Learning Preparation for and depth of continuing learning

109. Engineering Knowledge
(WA1) Apply knowledge of mathematics, natural
science, engineering fundamentals and an
engineering specialisation to the solution of
complex engineering problems; (WK1 to WK4)
PROGRAMME OUTCOME
WK = Knowledge Profile = Curriculum
WA = Programme Outcome
109

110. Problem Analysis - Complexity of analysis
(WA2) Identify, formulate, research literature
and analyse complex engineering problems
reaching substantiated conclusions using first
principles of mathematics, natural sciences and
engineering sciences (WK1 – WK4)
PROGRAMME OUTCOME
110

111. Design/Development of Solutions – Breadth and
uniqueness of engineering problems i.e. the extent
to which problems are original and to which
solutions have previously been identified or codified
(WA3) Design solutions for complex engineering
problems and design systems, components or
processes that meet specified needs with appropriate
consideration for public health and safety, cultural,
societal, and environmental considerations (WK5)
PROGRAMME OUTCOME
111

112. Investigation - Breadth & Depth of Investigation
& Experimentation
(WA4) Conduct investigation of complex problems
using research based knowledge (WK8) and
research methods including design of
experiments, analysis and interpretation of data,
and synthesis of information to provide valid
conclusions
PROGRAMME OUTCOME
112

113. Modern Tool Usage - Level of understanding of the
appropriateness of the tool
(WA5) Create, select and apply appropriate
techniques, resources, and modern engineering
and IT tools, including prediction and modelling, to
complex engineering problems, with an
understanding of the limitations. (WK6)
PROGRAMME OUTCOME
113

114. The Engineer and Society - Level of knowledge
and responsibility
(WA6) Apply reasoning informed by contextual
knowledge to assess societal, health, safety, legal
and cultural issues and the consequent
responsibilities relevant to professional
engineering practice and solutions to complex
engineering problems. (WK7)
PROGRAMME OUTCOME
114

115. Environment and Sustainability - Type of solutions
(WA7) Understand and evaluate the sustainabilty
and impact of professional engineering work in the
solutions of complex engineering problems in
societal and environmental contexts (demonstrate
knowledge of and need for sustainable
development) (WK7)
PROGRAMME OUTCOME
115

116. PROGRAMME OUTCOME
Ethics - Understanding and level of practice
(WA8) Apply ethical principles and commit to
professional ethics and responsibilities and norms
of engineering practice. (WK7)
116

117. PROGRAMME OUTCOME
Individual and Team Work – Role in and diversity
of team
(WA9) Function effectively as an individual, and as
a member or leader in diverse teams and in multi-
disciplinary settings
117

118. Communication – Level of communication
according to type of activities performed
(WA10) Communicate effectively on complex
engineering activities with the engineering
community and with society at large, such as being
able to comprehend and write effective reports
and design documentation, make effective
presentations, and give and receive clear
instructions
PROGRAMME OUTCOME
118

119. PROGRAMME OUTCOME
Project Management and Finance – Level of
management required for differing types of
activity
(WA11) Demonstrate knowledge and
understanding of engineering and management
principles and economic decision-making and
apply these to one’s own work, as a member and
leader in a team, to manage projects and in
multidisciplinary environments
119

120. PROGRAMME OUTCOME
Life-long Learning – Preparation for and depth of
continuing learning
(WA12) Recognise the need for, and have the
preparation and ability to engage in independent
and life-long learning in the broadest context of
technological change
120

121. WK1
natural sciences
WK2
mathematics,
numerical
analysis,
statistics,
computer and
information
science
WK3
engineering
fundamentals
WK4
engineering
specialist
knowledge
WK5
engineering
design
WK6
engineering
practice
WK7
engineering in
society
WK8
research
literature
WA1
ENGINEERING
KNOWLEDGE
WA2
PROBLEM
ANALYSIS
WA3
DESIGN
WA5
MODERN TOOLS
WA6 ENGR & SOC
WA7 ENV & SUST
WA8 ETHICS
WA4
INVESTIGATION
WA9
IND & TEAM
WA10
COMMUNICAT-ION
WA11
PROJ MGMT & FINANCE
WA12
LIFE LONG
121
4 YEARS

122. WA Complex Problem (WP)
122

123. Complex Problem
Uncertain
Change
Difficult
Confusing
Intractable
Contentious
Decision
Strategy
Idea
Product
Need to think broadly and systematically
and see the big picture
123

124. Difficulty & Uncertainty
• Complexity – the problem contains a large
number of diverse, dynamic and
interdependent elements
• Measurement – it is difficult or practically
unfeasible to get good qualitative data
• Novelty – there is a new solution evolving
or an innovative design is needed
124

125. Scientific/Technical
Problems
can combine to
form
A
Complex Problem
125

126. Limited Explanation,
Prediction, Control
Results in an educated
guest
?
A limited number of
features are captured by
the Model
Operating with scare
resources
Difficult to measure
Complex causal Chains
Unbounded Systems, No
Experiment
Explanation, Prediction,
Control
Results in a Covering
Law
f(x,y,z)
All the Salient features
are captured by the
Model
Operating with adequate
resources
Measurable
Simple causal Chains
Isolatable Systems,
Controlled Experiment
Complex
Technical
126

127. Characteristics
Complex Problems
• No definitive problem boundary
• Relatively unique or unprecedented
• Unstable and/or unpredictable
problem parameters
• Multiple experiments are not
possible
• No bounded set of alternative
solutions
• Multiple stakeholders with different
views or interest
• No single optimal and/or objectively
testable solution
• No clear stopping point
Technical Problems
• Isolatable boundable problem
• Universally similar type
• Stable and/or predictable problem
parameters
• Multiple low-risk experiments are
possible
• Limited set of alternative solutions
• Involve few or homogeneous
stakeholders
• Single optimal and testable
solutions
• Single optimal solution can be
clearly recognised
127

128. WP1 Depth of Knowledge
required
Resolved with forefront in-depth engineering
knowledge (WK3, WK4, WK5, WK6 or WK8) which
allows a fundamentals-based, first principles analytical
approach
WP2 Range of conflicting
requirements
Involve wide-ranging or conflicting technical,
engineering and other issues.
WP3 Depth of analysis required Have no obvious solution and require abstract thinking,
originality in analysis to formulate suitable models.
WP4 Familiarity of issues Involve infrequently encountered issues
WP5 Extent of applicable codes Beyond codes of practice
WP6 Extent of stakeholder
involvement and level of
conflicting requirements
Involve diverse groups of stakeholders with widely
varying needs.
WP7 Interdependence Are high level problems including many component
parts or sub-problems.
EP1 Consequences Have significant consequences in a range of contexts.
EP2 Judgement Require judgement in decision making
Complex Engineering Problems have characteristic WP1 and some or all of WP2 to WP7, EP1 and EP2, that
can be resolved with in-depth forefront knowledge
Complex Problems (Need High Taxonomy Level)
128

129. Problem Oriented, Team-Based Project Work as a
Learning/Teaching Device
1. Problem-oriented project-organized education deals with
the solution of theoretical problems through the use of any
relevant knowledge, whatever discipline the knowledge
derives from. We are dealing with KNOW WHY (Research
Problems).
2. In design-oriented project work, the students deal with
KNOW HOW problems that can be solved by theories and
knowledge they have acquired in their previous lectures.
(Design Problems).
129

130. Preamble Complex activities means (engineering) activities or
projects that have some or all of the following
characteristics listed below
Range of
resources
Diverse resources (people, money, equipment,
materials, information and technologies).
Level of
interaction
Require resolution of significant problems arising
from interactions between wide ranging or
conflicting technical, engineering or other issues.
Innovation Involve creative use of engineering principles and
research-based knowledge in novel ways
Consequences to
society and
the environment
Have significant consequences in a range of
contexts, characterised by difficulty of prediction
and mitigation.
Familiarity Can extend beyond previous experiences by
applying principles-based approaches.
Complex Engineering Activities (Project based)
130

131. WA – WK – WP Relationships
WA1 – Engineering Knowledge
(Science, Mathematics & Engineering)
(WK1, WK2, WK3, WK4)
to solve
Complex Engineering Problems
WK2 - mathematics, numerical analysis,
statistics, computer and information science
(WA1)
WK1 - natural sciences (WA1)
WK3 - engineering fundamentals (WA1)
WK4 - engineering specialist knowledge
(WA1)
WP1 – Depth of Knowledge
required:
Resolved with forefront in-depth
engineering knowledge
(WK3, WK4, WK5, WK6 or WK8)
which allows a fundamentals-based,
first principles analytical approach
WK5 - engineering design (know how)
WA3 - Design
WK6 - engineering practice (know how)
WA5 - Modern Tools
WK8 - research literature (know why)
WA4 - Investigation
(know what)

132. to solve
Complex Engineering Problems
WK2 - mathematics, numerical analysis,
statistics, computer and information science (WA1)
WK1 - natural sciences (WA1)
WK3 - engineering fundamentals (WA1)
WK4 - engineering specialist knowledge
(WA1)
WP1 – Depth of Knowledge
required:
Resolved with forefront in-depth
engineering knowledge
(WK3, WK4, WK5, WK6 or WK8)
which allows a fundamentals-based,
first principles analytical approach
WK5 - engineering design
WA3 - Design
WK6 - engineering practice
WA5 - Modern Tools
WK8 - research literature
WA4 - Investigation
WP2 Range of conflicting requirements
WP3 Depth of analysis required
WP4 Familiarity of issues
WP5 Extent of applicable codes
WP6 Extent of stakeholder involvement and level
of conflicting requirements
WP7 Interdependence
EP1 Consequences
EP2 Judgement
Some or all
WP2 – WP7, EP1 & EP2

133. to solve
Complex Engineering Problems
WK2 - mathematics, numerical analysis,
statistics, computer and information science (WA1)
WK1 - natural sciences (WA1)
WK3 - engineering fundamentals (WA1)
WK4 - engineering specialist knowledge
(WA1)
WP1 – Depth of Knowledge
required:
Resolved with forefront in-depth
engineering knowledge
(WK3, WK4, WK5, WK6 or WK8)
which allows a fundamentals-based,
first principles analytical approach
WK5 - engineering design
WA3 - Design
WK6 - engineering practice
WA5 - Modern Tools
WK8 - research literature
WA4 - Investigation
WP2 Range of conflicting requirements
WP3 Depth of analysis required
WP4 Familiarity of issues
WP5 Extent of applicable codes
WP6 Extent of stakeholder involvement and level
of conflicting requirements
WP7 Interdependence
EP1 Consequences
EP2 Judgement
WK7 - engineering in society
WA6 - engineer & society
WA7 - environment & sustainability
WA8 - ethics
Breadth

134. Design Course WK2 - mathematics, numerical analysis,
statistics, computer and information science (WA1)
WK1 - natural sciences (WA1)
WK3 - engineering fundamentals (WA1)
WK4 - engineering specialist knowledge (WA1)
WP1 – Depth of Knowledge
required:
Resolved with forefront in-depth
engineering knowledge
(WK3, WK4, WK5, WK6 or WK8)
which allows a fundamentals-based,
first principles analytical approach
WK5 - engineering design
WA3 - Design
WK6 - engineering practice
WA5 - Modern Tools
WK8 - research literature
WA4 - Investigation
WP2 Range of conflicting requirements
WP3 Depth of analysis required (WA2)
WP4 Familiarity of issues
WP5 Extent of applicable codes
WP6 Extent of stakeholder involvement
and level of conflicting
requirements WK7 (WA6, WA7,
WA8)
WP7 Interdependence
EP1 Consequences
EP2 Judgement
WK7 - engineering in society
WA6 - engineer & society (WK7)
WA7 - environment & sustainability (WK7)
WA8 – ethics (WK7)
WA2 - Problem Analysis (WK 1-4)
WA9 - Individual and Team Work
WA10 - Communication
WA11 - Project Management and Finance
WA12 - Life-long Learning

135. Example 1: Complex Problem Solving
• Two villages in Timbuktu are separated from each other by
a valley, at its deepest section about 30 metres.
• The valley is dry all the year around, except for the four
months, from October to December each year, where
torrential rainfall can flood major parts of the valley to a
depth of over 12 metres in some site.
• The soil is generally lateritic with firm bedrock
underneath. A bridge connecting the two villages is in a
state of disrepair and has to be replaced.
• Write a project brief on how would you approach to
design for the replacement bridge.
• You are limited to the use of locally available building
materials.
135

136. Aspects
• Economics
• Social
• Environment
• Ethics
• Management
• Technology
• Analysis
• Evaluation
136

137. Thinking
• Site condition
• Weather
• Available technology
• Building materials
• Design
• Costing
• Scheduling
137

138. Solutions?
• Problem solving skills
• Formulate the problem
• Literature
• Experiment?
138

139. Assessment
• Report – style and content (flow)
• Display – attractive ?
• Viva / Articulation
• Teamwork
• Management – scheduling
139

140. Sandy soil
Fissured Rocks
Igneous rock
Clayey soil
Groundwater flow
Spring
River
Example 2: Complex Problem Solving
140

141. How does complexity relates to
curriculum?
• General Subjects
• Industrial Placement
• Core & Specialist (Engineering) Subjects –
Complex Problem Solving
• Elective Subjects – Complex Problem Solving
• Design Project – Complex Engineering Activities
• Final Year Project – Complex Problem Solving
141

142. Closing Remarks
142

143. Thank You

144. Appendix
144

145. Complex Problem Solving (CPS)
• Dynamic, because early actions determine the
environment in which subsequent decision must
be made, and features of the task environment
may change independently of the solver’s actions;
• Time- dependent, because decisions must be
made at the correct moment in relation to
environmental demands;
• Complex, in the sense that most variables are not
related to each other in a one-to-one manner
145

146. Microworld CPS Model
• The problem requires not one decision, but a
long series, in which early decisions condition
later ones.
• For a task that is changing continuously, the
same action can be definitive at moment t1
and useless at moment t2.
• Include novel solutions to an old dilemma in
general science (external validity vs.
experimental control)
146

147. Expert-novice CPS Model
• Expert-novice approach most of the time
produces conclusions that are crystal-clear.
• It almost guarantees statistically significant
results, because the groups compared (expert
and novices) are very different and tend to
perform very differently when confronted with
similar experimental situations (Sternberg
1995).
147

148. Naturalistic decision making (NDM)
• Naturalistic decision making (NDM) (e.g.,
Zsambok and Klein 1997, Salas and Klein
2001)
• ‘real-world’ task
• Example interviewing firefighters after
putting out a fire or a surgeon after she has
decided in real time what to do with a
patient.
148

149. Dynamic decision making DDM
• Dynamic decision making (DDM) (Brehmer
1992, Sterman 1994).
• Discrete dynamic decision tasks that change
only when the participant introduces a new
set of inputs.
• Variables like time pressure have been
successfully integrated in models like
Busemeyer and Townsend’s (1993) decision
field theory
149

150. Implicit learning in system control
• This tradition has used tasks like the sugar
factory (Berry and Broadbent 1984) or the
transportation task (Broadbent et al. 1986), that
are governed by comparatively simple
equations.
• The theorization and computational modeling in
this branch of CPS are extremely rich. Models
are based on exemplar learning, rule learning,
and both (e.g., Dienes and Fahey 1995, Gibson
et al. 1997, Lebiere et al. 1998).
150

151. European complex problem solving (CPS)
• Initiated by Dörner (Dörner and Scholkopf
1991, Dörner and Wearing 1995)
• A large number of tasks that have been
considered complex problem solving are
nowadays affordable for theory development
and computer modeling (e.g. Putz-Osterloh
1993, Vollmeyer et al. 1996, Burns and
Vollmeyer 2002, Schoppek 2002)
• Transport real-life complexity to the lab in a
way that can be partly controlled 151

152. Time related
• Time variant – time invariant (dynamic vs.
static systems)
• Continuous time – discrete time.
• Degree of time pressure – decision has to be
made quickly
152

153. Variable related
• Number and type (discrete/continuous) of
variables
• Number and pattern of relationships
between variables
• Non-Linear - Linear
153

154. System behaviour related
• Opaque - transparent.
• Stochastic - deterministic
• Delayed feedback - immediate feedback.
154

155. Delivery
• Knowledge-lean vs. knowledge-intensive
• Skill based vs planning based (reactive vs
predictive
• Learning vs. no learning during problem
solving
• Understanding-based vs. search-based
problems
• Ill-defined vs. well-defined
155

156. Conclusion
• Problem solving has been traditionally a
task-centered field. VanLehn (1989) think
that ‘task’ and ‘problem’ are virtually
synonymous.
156

157. 157

158. The author would like to thank the contributors of the clip arts
used in this presentation
158