Your SlideShare is downloading. ×
Assessing learning outcomes in Higher Education:the Brazilian experience - Renato Pedrosa
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Assessing learning outcomes in Higher Education: the Brazilian experience - Renato Pedrosa

1,591
views

Published on

Published in: Education

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,591
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. OECD – IMHE General ConferenceParis – September, 17-19, 2012Assessing learning outcomes in Higher Education:the Brazilian experienceRenato H. L. PedrosaDept. of Scientific and Technology Policy /Geosciences InstitituteGroup of Studies in Higher Education/Center for Advanced StudiesUniversity of Campinas - Unicamp CEAvrenato.pedrosa@reitoria.unicamp.br
  • 2. Assessing Learning Outcomes in HE CEAv 1. Scope: academic a. General education (?) b. Subject area (general subject area/specific subject area) 2. Scope: institutional, national, multinational (LA, EU, Asia), global 3. Purposes? a. Accreditation: regulation b. Evaluation: feedback to policy makers, stakeholders, students c. Comparative studies 4. Validity: more in a moment 5. Feasibility: practical issuesIMHE 2012 RHLPedrosa
  • 3. Why assess learning outcomes? CEAvIMHE 2012 RHLPedrosa
  • 4. Assessing Learning Outcomes in HE CEAv John Sexton this morning: “Are we just measuring the measurable?"IMHE 2012 RHLPedrosa
  • 5. The Brazilian Asessment System CEAv SINAES/ENADE • The Brazilian System of Higher Education Evaluation (SINAES) was established by national law in 2004. • 1996: there was already a national exam for most undergraduate programs. • SINAES includes as one of its components the National Student Performance Test (ENADE), which is mandatory for all students in their last year of studies. • SINAES also evaluates various institutional aspects, like faculty, infrastructure and others and each program is assigned a score, the Preliminary Program Score (CPC), on a 1-5 scale. • Program score norm-referenced, meaning that scores are attributed by splitting the programs in level of scores without reference to proficiency levels. Thus one actually cannot say at what level if students graduating from a program that scores 4 (next to highest) • But: most publicized and debated aspect of system is the Student Performance National Exam (ENADE), since the results for each program are widely publicized and used by media.IMHE 2012 RHLPedrosa
  • 6. The test - ENADE CEAv • It is composed by two components • A “general education” (GE) component: 10 items, 8 multiple choice, 2 open • A subject area (SA) component: 30 item, 27 multiple choice, 3 open • The final score is weighted 25% for the GE and 75% for the SA component • The score is used in two ways • As the score for the graduating class • To compute a “value added” score (with the High School Exit Test score) • Those two scores plus the institutional and program evaluation for the Preliminary Program Score (CPC) • 40% graduating ENADE Score, 30% “value added” score, 30% institutional scoreIMHE 2012 RHLPedrosa
  • 7. Validity of educational assessment CEAv systems Evidences presented to justify the intended interpretations and uses of results of exams/tests employed by an assessment system Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17– 64). Westport, CT: ACE/Praeger Publishers.IMHE 2012 RHLPedrosa
  • 8. Validity of educational assessment CEAv systems Evidences presented to justify the intended interpretations and uses of results of exams/tests employed by an assessment system Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17– 64). Westport, CT: ACE/Praeger Publishers.IMHE 2012 RHLPedrosa
  • 9. Validity aspects of ENADE and SINAES CEAv 1. “General education” component a. Contemporary social, cultural information asseessment b. Some analytical, writing, reasoning skills assessed (not much) c. How is that related to curriculum or any specific university/college activities? 2. Subject area component a. Coverage of all subareas? b. Number of items? 3. “Value added” score a. Some evidence it does not actually measure that, but is highly correlated to the graduating class score 4. Preliminary Program Score (CPC) a. Norm-referenced b. No indication regarding what the scores really mean 5. Big issue: student involvementIMHE 2012 RHLPedrosa
  • 10. Comments CEAv • ENADE/CPC are not valid, in their present form, as assessments of individual programs or intitutions regarding undergraduate education. Evidence is clear, well regarded programs by specialists and market get very low scores (student boycott being the main reason for that). • Certainly the “general education” component is very fragile as indicator of what a college education brings to the student in terms of general skills. That is evident from just analysing content of items and from performance of first and last year students.IMHE 2012 RHLPedrosa
  • 11. Comments CEAv • In spite of that, it has some validity for assessment of subgroups of institutions. For example, the public or the private systems. Or the university system, or the small colleges system. • But it depends heavily on the subject area. Some analysis done at Unicamp for the recent exams indicate that in engineering and basic sciences the exams seem appropriate as assessment tools (but much further study is needed). On the other hand, there is much concern about human and social sciences areas.IMHE 2012 RHLPedrosa
  • 12. Example: engineering CEAvIMHE 2012 RHLPedrosa
  • 13. OECD – IMHE General ConferenceParis – September, 17-19, 2012Assessing learning outcomes in Higher Education:the Brazilian experience Thank youRenato H. L. PedrosaDept. of Scientific and Technology Policy /Geosciences InstitituteGroup of Studies in Higher Education/Center for Advanced StudiesUniversity of Campinas - Unicamp CEAvrenato.pedrosa@reitoria.unicamp.br