Instituto Pedagógico de Caracas
EVALUACIÓN EN EL
  Departamento de Idiomas Modernos
         Cátedra de Lingüística
   PROCESO DE
 ENSEÑANZA DEL
  INGLÉS COMO
     LENGUA
  EXTRANJERA
           - Código IIU115 -

    Prof. Rosynella Cardozo R.
    Prof. Jonathan Magdalena
       Prof. Mirna Quintero
CONTENT

 1.- Review of main concepts

   a.- What is evaluation?
   b.- Generations in evaluation
   c.- Importance of evaluation
   d.- Evaluation types and purposes

 2.- Evaluation Vs. Assessment
   a.-Types of assessment

 3.- Principles of test design – test formats
CONTENT

 5.- Test analysis and design

 6.- Informal assessment

 7.- Self assessment

 8.- Evaluation
1- EVALUATION
Evaluation is a natural activity which consists of making value
judgments constantly. However, evaluation itself is not usually
carried out in a principled and systematic way.


The implications of evaluating in an educational context are more
powerful than those related to the social setting. As a result, it
becomes crucial that careful thought is given to make explicit
what is being evaluated and the criteria by which it is being
judged.


Therefore, Evaluation (in the pedagogical context) refers to the
act of making value judgments in a systematic way, using a
principled, well-defined criteria to determine the product
of education.
                     .
GENERATIONS IN EVALUATION

1st Generation: evaluators measure participants

2nd Generation: evaluators describe participants

3rd Generation: evaluators judge participants

4th Generation: evaluators negotiate with participants



                  Guba, E. and Lincoln, Y. (1989). Fourth Generation Evaluation.
                     Newbury Park: Sage Publications Inc.
IMPORTANCE OF EVALUATION
   To diagnose the needs of participants.
   To determine how effective a process is (so as to improve it).
   To orient or reorient a process.
   To obtain feedback about classroom practiced and progress.
   To confirm the validity of all features in the education
    context.
   To determine and monitor students’ weaknesses or
    strengths.
   To determine the program’s appropriateness.
   To check on the strategies and the students’ response to
    them.
   To take decisions.
                       .
TYPES OF
              EVALUATION



Congruent        Formative         Summative


              •Throughout      •At the end of
•Before the
              the whole        (a stage of) the
process                        process.
begins.       process.

              •To reinforce    •To quantify it
•To predict
              or improve it.   through the use
results.                       of grades.
PURPOSES OF EVALUATION
   Accountability: Summative. Determines whether there has been value for money;
    whether something has been effective or not. It informs to decide is to continue or
    to be drastically removed. How? Analysis of statistical data. Who? Policy makers
    and resource providers.

   Curriculum Development: Formative. Involves information to be used as the
    basis of future planning and action. Improvement and renewal of curriculum. How?
    Responses to questionnaires, interviews, diaries. Who? Teachers and curriculum
    developers.

   Teacher self-development: Formative. To raise consciousness on teachers and
    other practitioners about what actually happens in the classroom. How? Self-
    assessment, awareness-raising activities. Who? Teachers.

   Student’s outcome: Formative/Summative.To check on students’ behavior (non-
    linguistic factors) and performance (linguistic factors). How? Teacher, Self, and
    peer-assessment; formal and informal assessment. Who? Teachers and Students.
EVALUATION VS.
     ASSESSMENT

Assessment is the collection of data to describe or better
understand an issue. It is more often used in relation to
educational programs.
A definition to distinguish the two terms:
Assessment refers to the measurement of
performance to determine if the ends of
teaching have been achieved, whereas
evaluation refers to the judgments based
on that information.
EVALUATION AND ASSESSMENT
                                                   Ev
                                                      al
                 on                                      ua
               i                                            t
          u at                                               io
                                                                  n
     al         Analysis of documents
   Ev


                             Assessment
                                                   Teachers’




                                                                  Ev
                                            Appraisals
          n




                                                                    alu
        io




                      Administrators’
     at




                                                           at
   lu




                                            Counselors’




                                                              io n
             va




                              Community members’
            E
WHO SHOULD             USE    ASSESSMENT                        AND
EVALUATION?

                             Examples of why they need Assessment,
     Job Description
                             Research and Evaluation

     Policy Makers           set standards, focus on goals, monitor the
                             quality of education, formulate policies, direct
                             resources including personnel and money, and
                             determine effects of tests
     Administration          are school/departments meeting the goals of
                             the University, appropriateness of curriculums
                             and course, identify program strengths and
                             weaknesses, designate program priorities,
                             assess alternatives, plan and improve
                             programs
        Teachers             refine curriculum, perform individual
                             diagnosis and prescription, monitor student
                             progress, how much knowledge students are
                             retaining from current teaching methods,
                             provide feedback to students


      Researchers            is research meeting the goal of the proposal
                             (especially if funding is reliant on grant money
                             that requires progress reports), how to improve
                             the program, find unexpected outcomes
                .
TYPES OF
             ASSESSMENT



   Formal       Informal           Self - Peer
Assessment     Assessment         Assessment
 (TESTING)

             •Workshops       •Questionnaires
•Tests:
             •Projects        •Diaries
•Exams
             •Presentations   •Surveys
•Quizzes
             •Homework        •Desciptions

             •Etc.            •Etc.
2.- TEST CHARACTERISTICS
.- Validity
       A test is valid “if it measures accurately what it is
intended to measure” (Hughes, A., 1989).

.- Reliability
        A test is reliable if it measures consistently.
Results must be stable.

.- Practicality
       Aspects affecting time, money, effort, resources.
.- Washback
        Influence of tests on teaching and learning.
3.- PRINCIPLES OF TEST DESIGN –
TESTS FORMATS

- Guidelines for item design

- Formats

- Sample items
4.- COMMUNICATIVE TESTING


           Relevance

        Contextualization

         Meaningfulness

          Authenticity
5.- TEST ANALYSIS AND
DESIGN

     Analysis of authentic tests from different
 contexts prior to test/item design

6.- INFORMAL
ASSESSMENT

7.- SELF-
ASSESSMENT

Evaluacón ILE Introduction to the course - Preliminars

  • 1.
    Instituto Pedagógico deCaracas EVALUACIÓN EN EL Departamento de Idiomas Modernos Cátedra de Lingüística PROCESO DE ENSEÑANZA DEL INGLÉS COMO LENGUA EXTRANJERA - Código IIU115 - Prof. Rosynella Cardozo R. Prof. Jonathan Magdalena Prof. Mirna Quintero
  • 2.
    CONTENT 1.- Reviewof main concepts a.- What is evaluation? b.- Generations in evaluation c.- Importance of evaluation d.- Evaluation types and purposes 2.- Evaluation Vs. Assessment a.-Types of assessment 3.- Principles of test design – test formats
  • 3.
    CONTENT 5.- Testanalysis and design 6.- Informal assessment 7.- Self assessment 8.- Evaluation
  • 4.
    1- EVALUATION Evaluation isa natural activity which consists of making value judgments constantly. However, evaluation itself is not usually carried out in a principled and systematic way. The implications of evaluating in an educational context are more powerful than those related to the social setting. As a result, it becomes crucial that careful thought is given to make explicit what is being evaluated and the criteria by which it is being judged. Therefore, Evaluation (in the pedagogical context) refers to the act of making value judgments in a systematic way, using a principled, well-defined criteria to determine the product of education. .
  • 5.
    GENERATIONS IN EVALUATION 1stGeneration: evaluators measure participants 2nd Generation: evaluators describe participants 3rd Generation: evaluators judge participants 4th Generation: evaluators negotiate with participants Guba, E. and Lincoln, Y. (1989). Fourth Generation Evaluation. Newbury Park: Sage Publications Inc.
  • 6.
    IMPORTANCE OF EVALUATION  To diagnose the needs of participants.  To determine how effective a process is (so as to improve it).  To orient or reorient a process.  To obtain feedback about classroom practiced and progress.  To confirm the validity of all features in the education context.  To determine and monitor students’ weaknesses or strengths.  To determine the program’s appropriateness.  To check on the strategies and the students’ response to them.  To take decisions. .
  • 7.
    TYPES OF EVALUATION Congruent Formative Summative •Throughout •At the end of •Before the the whole (a stage of) the process process. begins. process. •To reinforce •To quantify it •To predict or improve it. through the use results. of grades.
  • 8.
    PURPOSES OF EVALUATION  Accountability: Summative. Determines whether there has been value for money; whether something has been effective or not. It informs to decide is to continue or to be drastically removed. How? Analysis of statistical data. Who? Policy makers and resource providers.  Curriculum Development: Formative. Involves information to be used as the basis of future planning and action. Improvement and renewal of curriculum. How? Responses to questionnaires, interviews, diaries. Who? Teachers and curriculum developers.  Teacher self-development: Formative. To raise consciousness on teachers and other practitioners about what actually happens in the classroom. How? Self- assessment, awareness-raising activities. Who? Teachers.  Student’s outcome: Formative/Summative.To check on students’ behavior (non- linguistic factors) and performance (linguistic factors). How? Teacher, Self, and peer-assessment; formal and informal assessment. Who? Teachers and Students.
  • 9.
    EVALUATION VS. ASSESSMENT Assessment is the collection of data to describe or better understand an issue. It is more often used in relation to educational programs. A definition to distinguish the two terms: Assessment refers to the measurement of performance to determine if the ends of teaching have been achieved, whereas evaluation refers to the judgments based on that information.
  • 10.
    EVALUATION AND ASSESSMENT Ev al on ua i t u at io n al Analysis of documents Ev Assessment Teachers’ Ev Appraisals n alu io Administrators’ at at lu Counselors’ io n va Community members’ E
  • 11.
    WHO SHOULD USE ASSESSMENT AND EVALUATION? Examples of why they need Assessment, Job Description Research and Evaluation Policy Makers set standards, focus on goals, monitor the quality of education, formulate policies, direct resources including personnel and money, and determine effects of tests Administration are school/departments meeting the goals of the University, appropriateness of curriculums and course, identify program strengths and weaknesses, designate program priorities, assess alternatives, plan and improve programs Teachers refine curriculum, perform individual diagnosis and prescription, monitor student progress, how much knowledge students are retaining from current teaching methods, provide feedback to students Researchers is research meeting the goal of the proposal (especially if funding is reliant on grant money that requires progress reports), how to improve the program, find unexpected outcomes .
  • 12.
    TYPES OF ASSESSMENT Formal Informal Self - Peer Assessment Assessment Assessment (TESTING) •Workshops •Questionnaires •Tests: •Projects •Diaries •Exams •Presentations •Surveys •Quizzes •Homework •Desciptions •Etc. •Etc.
  • 13.
    2.- TEST CHARACTERISTICS .-Validity A test is valid “if it measures accurately what it is intended to measure” (Hughes, A., 1989). .- Reliability A test is reliable if it measures consistently. Results must be stable. .- Practicality Aspects affecting time, money, effort, resources. .- Washback Influence of tests on teaching and learning.
  • 14.
    3.- PRINCIPLES OFTEST DESIGN – TESTS FORMATS - Guidelines for item design - Formats - Sample items
  • 15.
    4.- COMMUNICATIVE TESTING Relevance Contextualization Meaningfulness Authenticity
  • 16.
    5.- TEST ANALYSISAND DESIGN Analysis of authentic tests from different contexts prior to test/item design 6.- INFORMAL ASSESSMENT 7.- SELF- ASSESSMENT