Evaluation of Investment In
Workforce Education & Development
What we will do…
n
n

Define evaluation as a process.
Make decision-making a focus of evaluation:
Decision-making processe...
Decision types
Intended

Actual

PLANNING
Ends

RECYCLING

To determine
objectives

To judge and react to
attainments

STR...
What we will do…
n
n

Define evaluation as a process.
Make decision-making a focus of evaluation:
Decision-making processe...
Describe use of
criteria in evaluation
Criteria
n

n
n

A decision requires judging among two or more
alternatives.
Criteria are rules or tests for judging.
Crit...
Four kinds of values that impinge
on establishment of criteria
n

Instiutional values
E.g., “What does the Board of Truste...
What we will do…
n
n

Define evaluation as a process.
Make decision-making a focus of evaluation:
Decision-making processe...
Delineate criteria
for evaluation
Criteria for evaluation
n

Scientific
Internal and external validity, reliability, objectivity

n

Practical
Relevance, im...
Criteria for evaluation:
Scientific
n

Internal validity
No alternative explanations are possible for results obtained.

n...
Criteria for evaluation:
Practical
n

Relevance
Information serves decisions.

n

Importance
“Best” information culled to ...
Criteria for evaluation:
Efficiency
n

Conducted with the “best” use of resources.
What we will do…
n
n

Define evaluation as a process.
Make decision-making a focus of evaluation:
Decision-making processe...
Suggest a meta-methodology
for evaluation
Work breakdown of
evaluation design
Delineation of
information needs
Delineation of
information needs
Delineation of
information needs
Delineation of
information needs
Work breakdown of
evaluation design
Plan for
obtaining information
Plan for
obtaining information
Plan for
obtaining information
Work breakdown of
evaluation design
Plan for
providing information
Plan for
providing information
What we will do…
n
n

Define evaluation as a process.
Make decision-making a focus of evaluation:
Decision-making processe...
Specify four types
of evaluation
Evaluation &
decision-making
Evaluation types
Intended

CONTEXT
Ends

For planning
decisions

INPUT
Means

For structuring
decisions

Actual

PRODUCT
F...
Evaluation types:
Context
n
n
n
n
n

Defines relevant environment.
Identifies unmet needs and unused opportunities.
Diagno...
Evaluation types:
Input
n

Determines how to use resources to achieve aims.
Identifies relevant capabilities.
Specifies st...
Evaluation types:
Input (cont.)
n

n

Although context evaluation is systematic and
macro-analytic, input evaluation is ad...
Evaluation types:
Process
n

Aims:
Detects or predicts defects in procedural design or in
implementation in early stages.
...
Evaluation types:
Product
n

n

Interprets attainments at end of project cycle or as
often as necessary during cycle for a...
What we will do…
n
n

Define evaluation as a process.
Make decision-making a focus of evaluation:
Decision-making processe...
Present a general
evaluation model
Evaluation of Investment In
Workforce Education & Development
Upcoming SlideShare
Loading in …5
×

Definition of Evaluation

593 views
382 views

Published on

Passmore

Published in: Education, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
593
On SlideShare
0
From Embeds
0
Number of Embeds
127
Actions
Shares
0
Downloads
2
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Definition of Evaluation

  1. 1. Evaluation of Investment In Workforce Education & Development
  2. 2. What we will do… n n Define evaluation as a process. Make decision-making a focus of evaluation: Decision-making processes Decision settings Decision types n n n n n Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
  3. 3. Decision types Intended Actual PLANNING Ends RECYCLING To determine objectives To judge and react to attainments STRUCTURING Means To design procedures IMPLEMENTING To utilize, control, and refine procedures
  4. 4. What we will do… n n Define evaluation as a process. Make decision-making a focus of evaluation: Decision-making processes Decision settings Decision types n n n n n Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
  5. 5. Describe use of criteria in evaluation
  6. 6. Criteria n n n A decision requires judging among two or more alternatives. Criteria are rules or tests for judging. Criteria are yardsticks for values. Values = estimated or assessed worth. Values result from systems of beliefs, perceptions, interpretations of facts, judgments of needs. Values are shared and/or idiosyncratic. n Decision-makers apply criteria. They must “own” criteria. Often need assistance in identifying and finding criteria.
  7. 7. Four kinds of values that impinge on establishment of criteria n Instiutional values E.g., “What does the Board of Trustees expect our College to do?” n Values external to the institution E.g., “What does the State Legislature believe the College should produce?” n Subsystem maintenance values E.g., “What do my College colleagues believe we should do?” n Private values E.g, “What do I think our College should produce?”
  8. 8. What we will do… n n Define evaluation as a process. Make decision-making a focus of evaluation: Decision-making processes Decision settings Decision types n n n n n Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
  9. 9. Delineate criteria for evaluation
  10. 10. Criteria for evaluation n Scientific Internal and external validity, reliability, objectivity n Practical Relevance, importance, scope, credibility, timeliness, pervasiveness n Prudential Efficiency
  11. 11. Criteria for evaluation: Scientific n Internal validity No alternative explanations are possible for results obtained. n External validity Findings are generalizable across people, places, and time. n Reliability Accuracy of findings; reproducibility. n Objectivity Intersubjectivity; lack of bias.
  12. 12. Criteria for evaluation: Practical n Relevance Information serves decisions. n Importance “Best” information culled to serve decisions. n Scope All necessary information is provided. n Credibility Clients/users believe information. n Timeliness Information provided when needed. n Pervasiveness Information provided to all clients/users.
  13. 13. Criteria for evaluation: Efficiency n Conducted with the “best” use of resources.
  14. 14. What we will do… n n Define evaluation as a process. Make decision-making a focus of evaluation: Decision-making processes Decision settings Decision types n n n n n Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
  15. 15. Suggest a meta-methodology for evaluation
  16. 16. Work breakdown of evaluation design
  17. 17. Delineation of information needs
  18. 18. Delineation of information needs
  19. 19. Delineation of information needs
  20. 20. Delineation of information needs
  21. 21. Work breakdown of evaluation design
  22. 22. Plan for obtaining information
  23. 23. Plan for obtaining information
  24. 24. Plan for obtaining information
  25. 25. Work breakdown of evaluation design
  26. 26. Plan for providing information
  27. 27. Plan for providing information
  28. 28. What we will do… n n Define evaluation as a process. Make decision-making a focus of evaluation: Decision-making processes Decision settings Decision types n n n n n Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
  29. 29. Specify four types of evaluation
  30. 30. Evaluation & decision-making
  31. 31. Evaluation types Intended CONTEXT Ends For planning decisions INPUT Means For structuring decisions Actual PRODUCT For recycling decisions PROCESS For implementing decisions
  32. 32. Evaluation types: Context n n n n n Defines relevant environment. Identifies unmet needs and unused opportunities. Diagnoses problems that prevent needs from being met and opportunities from being used. Is macro-analytic and systematic. Involves contingency and congruence studies to set direction. Contingency - searches for opportunities and pressures outside system to promote improvement within. Congruence - compares actual and intended system performance to affirm objectives or consider new objectives.
  33. 33. Evaluation types: Input n Determines how to use resources to achieve aims. Identifies relevant capabilities. Specifies strategies for achieving goals. Fabricates specific designs to accomplish aims. n End product of input evaluation is one or more procedural designs, with a lists of costs and benefits. Assesses alternative designs in light of resource requirements/limits. Identifies procedural barriers, and risk/consequences of failure to overcome. Relevance of various designs to aims. n n Build or make? Project proposals embody the results of input evaluation.
  34. 34. Evaluation types: Input (cont.) n n Although context evaluation is systematic and macro-analytic, input evaluation is ad hoc and micro-analytic. Some questions asked: Is need documented? What strategies already exist? What are costs and benefits of competing strategies? What is track record for competing strategies? Are designs logical? Based on sound theory? Assumptions met? Legal? Moral? Side effects? Likely to be adopted?
  35. 35. Evaluation types: Process n Aims: Detects or predicts defects in procedural design or in implementation in early stages. Provides information for programmed decisions. Records implementation as it occurs. n Depends on context and input evaluation. More adequate context and input evaluation, the less critical the need for process evaluation. Less adequate context and input evaluation, the more critical process evaluation becomes. n More important than product evaluation in early development stages, but less important as more structure and knowledge is developed.
  36. 36. Evaluation types: Product n n Interprets attainments at end of project cycle or as often as necessary during cycle for accountability or improvement. Involves: Devising operational definitions of objectives. Obtaining criterion measures. Comparing measures with some absolute or relative standards. Making rational interpretations of outcomes using recorded context, input, and process information. n Criteria are instrumental or consequential: Instrumental -- related to intermediate accomplishments prerequisite to some end. Consequential -- pertain to fundamental, penultimate ends sought.
  37. 37. What we will do… n n Define evaluation as a process. Make decision-making a focus of evaluation: Decision-making processes Decision settings Decision types n n n n n Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
  38. 38. Present a general evaluation model
  39. 39. Evaluation of Investment In Workforce Education & Development

×