quality in higher education the art and design of education quality

1. EQ2003 Pakistan Institute of Quality Control 1
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Quality in HigherQuality in Higher
Education:Education:
The Art and Design ofThe Art and Design of
Assessment StrategyAssessment Strategy
Dr Hasan Sohaib MuradDr Hasan Sohaib Murad
RectorRector, University of Management and Technology (UMAT), University of Management and Technology (UMAT)
ChairmanChairman, Association of Management Development Institutions in, Association of Management Development Institutions in
Pakistan (AMDIP)Pakistan (AMDIP)
Higher Education:Higher Education:
Emerging ScenarioEmerging Scenario
1.1. Disenchantment of stakeholdersDisenchantment of stakeholders
2.2. Increased demandIncreased demand
3.3. Shrinking public expenditureShrinking public expenditure
4.4. Increase in profit and nonprofit provisionIncrease in profit and nonprofit provision
5.5. Growth of transnational higher education,Growth of transnational higher education,
physical and virtualphysical and virtual
6.6. Control of quality usurped by the marketControl of quality usurped by the market
7.7. Problems of access, quality, and equityProblems of access, quality, and equity
8.8. Crises of regulatory frameworkCrises of regulatory framework
9.9. Conventions, Innovation, and AdaptationConventions, Innovation, and Adaptation
10.10. Global Good PracticesGlobal Good Practices

2. EQ2003 Pakistan Institute of Quality Control 2
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
<< No uniform quality assurance mechanism in placeNo uniform quality assurance mechanism in place
<< What graduates KNOW and can DO is unclear.What graduates KNOW and can DO is unclear.
<< “…effective public policy requires a systematic“…effective public policy requires a systematic
monitoring of the progress made by states and themonitoring of the progress made by states and the
nation in raising the education and skill levels ofnation in raising the education and skill levels of
American society.”American society.” National Center for Public Policy & HENational Center for Public Policy & HE
(2002)(2002)
<< “…“…most HE institutions do not have clearly defined,most HE institutions do not have clearly defined,
broadly understood definitions of quality, how tobroadly understood definitions of quality, how to
produce it, or what it costs to achieve.produce it, or what it costs to achieve.”” –– NationalNational
Center for Postsecondary Improvement (2002)Center for Postsecondary Improvement (2002)
<< Some external impetus is necessary to initiateSome external impetus is necessary to initiate
outcomes assessment in higher education.outcomes assessment in higher education.
Higher Education:Higher Education:
Emerging Scenario (Cont.)Emerging Scenario (Cont.)
Sources of ExternalSources of External
MonitoringMonitoring
nn AccreditationAccreditation
nn Q auditQ audit
nn Program assessment or reviewProgram assessment or review
nn External evaluation andExternal evaluation and
comparison of standardscomparison of standards
conducted byconducted by
government ministriesgovernment ministries
professional bodiesprofessional bodies
special agenciesspecial agencies

3. EQ2003 Pakistan Institute of Quality Control 3
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Hazards of External QHazards of External Q
MonitoringMonitoring
nn Produces bureaucracyProduces bureaucracy
nn Asks the wrong questionsAsks the wrong questions
nn Leads to game playingLeads to game playing
nn Produces only shortProduces only short--term responsesterm responses
nn Focuses on accountability, notFocuses on accountability, not
improvementimprovement
nn Has no lasting impact, esp. onHas no lasting impact, esp. on
student learningstudent learning
Accreditation AssessmentAccreditation Assessment
nn ExternalExternal
nn Fixed FrameworkFixed Framework
nn MeasurementMeasurement
nn Common StandardsCommon Standards
nn CertificationCertification
nn RecognitionRecognition
nn SelfSelf--DrivenDriven
nn Dynamic ProcessDynamic Process
nn DiagnosisDiagnosis
nn CustomizedCustomized
nn RegenerationRegeneration
nn SelfSelf--AssuranceAssurance

4. EQ2003 Pakistan Institute of Quality Control 4
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Assessment FirstAssessment First
Assessment is not a substitute ofAssessment is not a substitute of
Accreditation. It is the core activityAccreditation. It is the core activity
during accreditation. It can, however,during accreditation. It can, however,
go beyond accreditation to provide ago beyond accreditation to provide a
reliable source of timely and valuablereliable source of timely and valuable
information leading to intervention forinformation leading to intervention for
continuous improvement.continuous improvement.
Assessment: Core ProcessAssessment: Core Process
Quality as fitness for purposeQuality as fitness for purpose
1. Say what you do1. Say what you do
2. Do what you say2. Do what you say
3. Generate evidence3. Generate evidence
4. Evaluate outcomes4. Evaluate outcomes

5. EQ2003 Pakistan Institute of Quality Control 5
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Outcomes AssessmentOutcomes Assessment
The process of providing credibleThe process of providing credible
evidence of the outcomes ofevidence of the outcomes of
higher education undertaken forhigher education undertaken for
the purpose of improvingthe purpose of improving
programs and services within theprograms and services within the
institution.institution.
Why Assessment?Why Assessment?
nn Paradigm shift from education toParadigm shift from education to
learninglearning
nn Develop capacity to form selfDevelop capacity to form self--
concept and selfconcept and self--accreditationaccreditation
nn information for planned change andinformation for planned change and
setting prioritiessetting priorities
nn benchmarkingbenchmarking
nn preparation for external assessmentpreparation for external assessment

6. EQ2003 Pakistan Institute of Quality Control 6
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
The FutureThe Future
nn Increasingly required as a a tool for effectiveIncreasingly required as a a tool for effective
governancegovernance
nn Need for evidence of accountability willNeed for evidence of accountability will
increaseincrease
nn Awareness to the benefits of assessment willAwareness to the benefits of assessment will
increaseincrease
nn More electronic assessment methods will beMore electronic assessment methods will be
developeddeveloped
nn More sharing of assessment methods willMore sharing of assessment methods will
take placetake place
nn Faculty will learn more about learning andFaculty will learn more about learning and
student learning will improvestudent learning will improve
Outcome AssessmentOutcome Assessment
depends on:depends on:
nn What is the purpose?What is the purpose?
nn Who is doing it?Who is doing it?
nn availability of dataavailability of data
nn the way of reportingthe way of reporting
nn the criteria setthe criteria set
nn the follow upthe follow up
nn The difference realizedThe difference realized

7. EQ2003 Pakistan Institute of Quality Control 7
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Conditions for Self AnalysesConditions for Self Analyses
nn Internal motivationInternal motivation
nn Constructive valuesConstructive values
nn support of managementsupport of management
nn taking into account the specific identitytaking into account the specific identity
of departmentof department
nn clear fixed processclear fixed process
nn well organised projectwell organised project
Assessment InfrastructureAssessment Infrastructure
Mission
statement
goals
&
aims
expected
outcomes
process
learning
services
knowledge
services
community
services
output
graduates
research
services
input
management
policy
staff
students
resources
facilities
Mission
Realised
Goals
Achieved
Outcomes
Achieved
Stakeholders
Satisfied

8. EQ2003 Pakistan Institute of Quality Control 8
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Select or DesignSelect or Design
Assessment MethodsAssessment Methods
1. Match with goals1. Match with goals
2. Use multiple methods2. Use multiple methods
3. Combine direct and indirect measures3. Combine direct and indirect measures
4. Combine qualitative and quantitative4. Combine qualitative and quantitative
measuresmeasures
5. Consider pre5. Consider pre -- post design to assesspost design to assess
gainsgains
6. Use built6. Use built--in points of contact within points of contact with
studentsstudents
Success in AssessmentSuccess in Assessment
1.1. Develop commitment to changeDevelop commitment to change
2.2. Use assessment as a catalyst forUse assessment as a catalyst for
changechange
3.3. Ask departments to identify ownAsk departments to identify own
evaluation questionsevaluation questions
4.4. Hold departments accountable forHold departments accountable for
answering own questionsanswering own questions
5.5. Encourage information sharingEncourage information sharing
and dialogueand dialogue
6.6. Take seriously issues of dataTake seriously issues of data
quality and credibilityquality and credibility

9. EQ2003 Pakistan Institute of Quality Control 9
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Assessment EssentialsAssessment Essentials
1.1. Plan for assessmentPlan for assessment
2.2. Involve all stakeholdersInvolve all stakeholders
3.3. Develop goals and objectivesDevelop goals and objectives
4.4. Select or design assessmentSelect or design assessment
methodsmethods
5.5. Report assessment findingsReport assessment findings
6.6. Promote the use of assessmentPromote the use of assessment
findingsfindings
7.7. Evaluate the assessmentEvaluate the assessment
programprogram
Some Questions to Ask UpSome Questions to Ask Up
Front About AssessmentFront About Assessment
nn Why Should We Do This at All?Why Should We Do This at All?
nn What Have We Got Already?What Have We Got Already?
nn What Organizational/CulturalWhat Organizational/Cultural
Obstacles are We Likely toObstacles are We Likely to
Encounter?Encounter?
nn How Will We Use the ResultingHow Will We Use the Resulting
Information, and for What?Information, and for What?

10. EQ2003 Pakistan Institute of Quality Control 10
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Three Enduring CharacteristicsThree Enduring Characteristics
of Assessmentof Assessment
nn Assessment is AboutAssessment is About LearningLearning
nn Assessment is AboutAssessment is About PerformancePerformance
nn Assessment isAssessment is More Than GradingMore Than Grading
Some Cautions About Assessment:Some Cautions About Assessment:
Assessment Is…..Assessment Is…..
nn NotNot “Measuring Everything that Moves”“Measuring Everything that Moves”
nn NotNot Just “Checking up After the Fact”Just “Checking up After the Fact”
nn NotNot Searching for “Final” AnswersSearching for “Final” Answers
nn NotNot Always Being as “Precise” asAlways Being as “Precise” as
PossiblePossible
nn NotNot Ever Expecting to be DoneEver Expecting to be Done

11. EQ2003 Pakistan Institute of Quality Control 11
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Three Directives About GettingThree Directives About Getting
StartedStarted
nn Start With What You HaveStart With What You Have
nn Look for Opportunities Within theLook for Opportunities Within the
CurriculumCurriculum
nn Start Small, but StartStart Small, but Start
Starting With What You HaveStarting With What You Have
nn What Kinds of “Assessments?”What Kinds of “Assessments?”
nn Opportunities for AssessmentOpportunities for Assessment

12. EQ2003 Pakistan Institute of Quality Control 12
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Some CommonlySome Commonly--UsedUsed
Assessment ApproachesAssessment Approaches
nn Direct AssessmentDirect Assessment
ApproachesApproaches
nn Indirect AssessmentIndirect Assessment
ApproachesApproaches
FrequentlyFrequently--EncounteredEncountered
ObstaclesObstacles
nn Faculty ResistanceFaculty Resistance
nn Too Little TimeToo Little Time
nn Too Little “KnowToo Little “Know--HowHow

13. EQ2003 Pakistan Institute of Quality Control 13
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Some Prominent PitfallsSome Prominent Pitfalls
nn Linear ThinkingLinear Thinking
nn Waiting for PerfectionWaiting for Perfection
nn Fixed Targets and StandardsFixed Targets and Standards
nn Either Excessive or NonEither Excessive or Non--ExistentExistent
ConsequencesConsequences
Four Lessons from a National “BestFour Lessons from a National “Best
Practice” Benchmarking Study [APQC]Practice” Benchmarking Study [APQC]
nn Assessment Systems Must beAssessment Systems Must be
Consistent with Core Values andConsistent with Core Values and
Institutional MissionInstitutional Mission
nn Assessment Must be ConsequentialAssessment Must be Consequential
[but the best linkages are Indirect][but the best linkages are Indirect]
nn Good Assessment Systems Take TimeGood Assessment Systems Take Time
to Developto Develop
nn Assessment Changes the Role ofAssessment Changes the Role of
Leaders and How they LeadLeaders and How they Lead

14. EQ2003 Pakistan Institute of Quality Control 14
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Attitude and ActionAttitude and Action
nn For Attitude….the key word isFor Attitude….the key word is
“Improvement”“Improvement”
nn For Action….the key word isFor Action….the key word is
“Consistency”“Consistency”
It is more aboutIt is more about
Mindset than MethodMindset than Method
nn Most Questions About Learning areMost Questions About Learning are
not Just Matters of Opinionnot Just Matters of Opinion
nn Assessment is About ImprovingAssessment is About Improving
PracticePractice

15. EQ2003 Pakistan Institute of Quality Control 15
Quality in Higher Education
by Dr. Hasan Sohaib Murad
www.piqc.com.pk
Lessons Learned Along the WayLessons Learned Along the Way
nn Mission must driveMission must drive everythingeverything
nn Establish Strategic Plan early andEstablish Strategic Plan early and
revise appropriatelyrevise appropriately
nn Involve all stakeholdersInvolve all stakeholders
nn Maintain high level of openMaintain high level of open
communicationcommunication
nn Spend deliberate, highSpend deliberate, high--quality timequality time
on planningon planning
nn Never too early to evaluateNever too early to evaluate
existing processes and establishexisting processes and establish
new onesnew ones