1. Standards, Evaluation and
Accountability
State-of-Play and Challenges from
Latin America
Gilbert A. Valverde, Ph.D.
Comparative and International Education Policy Program
University at Albany – State University of New York
valverde@uamail.albany.edu
Working Group on Standards and Assessment
Program to Promote Educational Reform in Latin America

2. A World-Wide Shift of Policy Focus
• We are experiencing a global shift in education
policy priorities
▫ Not only is it important for children to have access
to education.
▫ The quality of what transpires within the
classroom also matters.
• Prioritizing quality, means focusing on the
content of schooling
▫ Global interest in standards, evaluation and
accountability is a signal of this policy shift.

3. Latin America
• Much discussion of the importance of the content of
schooling
• Important and persistent problems of quality
▫ National, regional, and international testing programs
have thoroughly documented the poverty of
educational outcomes in the region.
• The evidence has conclusively demonstrated that the
traditional repertoire of policy instruments is not
sufficient to address this problem.
• A growing consensus that standards, evaluation and
associated policy instruments merit serious
consideration.

4. Goals for this talk
• Discuss how initiatives in standards, assessment
and accountability are forwarding important
policy conversations on educational quality in
Latin America.
• Consider challenges and lessons from the Latin
American experience that may be of interest to
other parts of the world.

5. Basic premises
• Educational reform based on principals of
promotion of educational quality requires:
▫ A clear vision of what is to be accomplished
▫ A system to monitor how these goals are being
met.
▫ A commitment to act in accordance with the
results of monitoring efforts

7. Elements of Quality of Vision
• Research and policy experience indicate that
accomplishing high levels of quality in
educational outcomes requires the articulation
of a clear vision of pedagogical objectives that
operationalize goals that are
▫ rigorous,
▫ challenging,
▫ have well defined priorities and foci
▫ evidence-based

8. Challenges for Latin America
• A critical examination of the instruments of
curriculum policy in the region suggests an
urgent need to correct problems of
▫ Ambiguity
▫ Dispersion and lack of priorities
▫ Lack of focus
▫ Absence of academic rigor

9. An example of dispersion and lack of
focus
Mexico Japan
5
viejo
5 6
6 7
7
viejo / foco 8
8
9
9
10
nuevo
10
11
11
12
nuevo /
12
13
foco
13
14
14
15
15
16
16
17
17
18
18
0 5 10 15 20 25 30 35 40
0 5 10 15 20 25 30 35 40

11. 11
First National Assessments
1996
1996 1991
1998
1997
1993
2002
1998
1986
2005 1991
1996
1990
RAPID GROWTH
1996
•In 1980 – none
1996
•In 1989 – 2 (Chile, Costa Rica)
1996
•Today – universal
1982
1996
1993
Source: Sistemas de evaluación de aprendizajes en
América Latina. Balances y desafíos. PREAL, 2006.

12. Consequences of rapid growth
• Introduction was precipitous, without much preparatory work
with schools or other actors.
▫ Many claim that introduction was forced or imposed.
• In most cases the initial measurement model was not at all
aligned with the curriculum reforms that often were taking
place simultaneously.
• Results were quickly and extensively disseminated, most often
accompanied by condemnations of the quality of teaching
(which is never measured).
• Important questioning of the technical quality of testing efforts
▫ Much suspicion that testing systems are not committed to
technical quality.
• Tepid government commitment to testing.
▫ Often testing systems do not have budgets until a few months prior
to test administration.
▫ Low levels of investment, with unclear commitment to technical
quality.
▫ Reticence to act in accordance with test results

13. Continuing progress
• Growing participation in large-scale cross-national
assessments.
• Increasing investments in state-of-the-art technologies
• Awareness that disappointing outcomes measured by
tests imply need to explore new policy instruments
• Setting of more specific improvement targets
• Emerging “culture of evaluation”
• Capacity building, growing experience and exchange
• Openness to revision and improvement of approaches,
methods, analyses and reporting
• Teachers growingly interested in potential of properly
utilized assessments for professional development and
school improvement purposes
• More interest in returning results to schools and non-
traditional educational stakeholders

15. Nascent efforts in Accountability
• If the accountable unit is the school, requires a
census of schools, if the student, a census of
students
• No high-stakes testing in the Region
• “Low-Stakes” Tests in Costa Rica, El Salvador and
the Dominican Republic
▫ In the Dominican Republic, for example, the school-
leaving examination or “Prueba Nacional” contributes
only 30% of a student’s grade
▫ Students have 3 or 4 opportunities to pass the exam
▫ Few students fail to pass based on the test alone

16. Schools = primary units of
accountability
• Chile
▫ Yearly publication of league tables ranking all
schools
• El Salvador
▫ Publication of lists of highest performing schools
• Colombia (Bogotá)
▫ “Excellence” awards based on school test scores.

17. School Choice
• Extremely uncommon
• Chile is the notable exception
 Ministry distributes school-by-school test results
for each region, including
 average scores in each subject area,
 changes in scores since previous testing round,
and
 comparisons with schools serving similar
socioeconomic groups
 Special reports for parents since 2003
 Recent research has demonstrated conclusively
that despite the intentions, parents do not make
significant use of test results in choosing schools.

18. New efforts to promote accountability
• Teacher as the unit of accountability
▫ Chile:
 Awards for teachers in the best schools according to the
national test
 A new voluntary teacher test (including video, teacher
portfolio and content assessment) to compete for
“pedagogical excellence” designation
 Efforts to put together a required teacher certification
test for graduating teachers
▫ Mexico:
 Teacher bonuses based on complex formula that includes
student test results

19. However….
Still too early to say whether, in most Latin American
countries, institutionalized and permanent mechanisms
have been firmly established which enable continuous
setting and revising of learning goals and measuring their
achievement.
In only few countries, student testing and other kinds of
outcomes assessment and reporting are clearly framed or
aligned within a set of policies and norms about the kind of
utilization which will be given to the resulting information,
as part of a well integrated, clear and balanced set of
policies for the improvement of learning and for adequate
accountability.

21. New Policy Routes
• The most important challenge is to increase mean levels
of achievement in all schools.
• Increasing quality in outcomes requires
▫ Operationalization of a concrete vision of what can be
considered quality outcomes
▫ Assignment of responsibilities
▫ Monitoring
▫ Commitment to act according to monitoring results.
• A new policy instrument increasingly of interest:
standards
▫ There are educational standards or projects underway in:
Guatemala, Honduras, Panama, Colombia, Argentina and
Chile
▫ Following diverse models from the United States and
Australia

22. Main challenges
Clarification and social legitimation of assessment
•
purposes and processes
Clarification of learning goals
•
Selection of what will be measured and how, according to
•
purposes
Selection of appropriate institutional framework
•
Improvement of testing instruments and processes
•
More and better data analyses
•
More and better diversified reports
•
More and better dissemination of results
•
Appropriate use of results for decision making
•

23. Key question looking to the future
• Is it possible or desirable to compensate for the early
shortcomings in the introduction of assessment systems
by greater efforts in transparency and genuine
consultation?
• Can the region learn from failures to lay sufficient policy
groundwork in the introduction of assessment systems,
in order to avoid pushback against nascent efforts to
introduce standards that could lead to similar
questioning of their political legitimacy, pedagogical
value, and technical soundness?
• Can research and development efforts be made more
transparent so that a broad spectrum of actors will be
persuaded that governments are sufficiently concerned
with the technical quality of standards and the validity
and reliability of measures used in their assessment
systems?

24. • Will it be possible to introduce or maintain a
concern for benchmarking Latin American
standards against “world-class standards” when the
likelihood is that Latin American countries will
continue to be a the bottom of PISA, TIMSS and
other global league tables?
• Can the development of assessment systems in the
region be used to introduce a concern for evidenced-
based policy making in the region, to temper
enthusiasm for theoretical innovations with weak
evidentiary bases in the development of curriculum
policy?
• Can assessment and standards be brought into
alignment with a commitment to act in accordance
with evaluation findings in order to develop
accountability systems with greater potential for
stimulating educational quality?