Standardized tests vs e-portfolios


Published on

This is a brief presentation on comparing standardized tests and scored e-portfolio artifacts.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • SurveyMonkey survey: 1. Please rate your overall impression of the validity of the Collegiate Learning Assessment(CLA)in providing feedback about students’ abilities in such areas as critical thinking, problem solving, and effective communication:
  • 3. Using a five-point scale in which a “5” means “serious and sustained effort” and a “1” means “almost no effort at all,” how would you rate the level of effort that you put into taking this test?
  • Standardized tests vs e-portfolios

    1. 1. Matching E-Portfolios to Standardized Tests for Value-Added Assessment Richard Robles EDLD 812: Data Improvement
    2. 2.  About the University Honors Program  UC’sinvolvement in the Voluntary System of Accountability  Comparing the Collegiate Learning Assessment to e-portfolio artifacts  What we have learned thus far
    3. 3. University Honors Program  Revised the program during AY 2007-08 to focus honors activities on experiential learning in the following thematic areas:  Community engagement  Global studies  Leadership  Research/Creative arts  Comprised of1,600 students from all of the colleges within the University (except Raymond Walters and Clermont Colleges)  Last 2 entering cohorts represent about 25% of the population  Average entering ACT is 31 (SAT 1346)
    4. 4. Voluntary System of Accountability  Voluntary initiative for 4-year public colleges and universities  Developed through a partnership between the AASCU and APLU to help institutions  Demonstrate accountability and stewardship to public  Measure educational outcomes to identify effective educational practices  Assemble information that is accessible, understandable, and comparable
    5. 5. Voluntary System of Accountability  Ohio is the only state to have all state institutions participating in the VSA  UC’s initial participation is the implementation of the Collegiate Learning Assessment
    6. 6. Collegiate Learning Assessment (CLA)  Developed by the Council for Aid to Education (CAE) in 2004 –  A standardized and nationally normed test that measures the institutional contributions (value added) to the learning gains made by students  Direct assessment of student learning
    7. 7. Collegiate Learning Assessment (CLA)  Areas of measurement  critical thinking  analytical reasoning / problem solving  written communication  Time required:  90 minutes for a performance task (complex problem to be solved)  75 minutes for two writing prompts (make an argument and break an argument)
    8. 8. The CLA’s Range  First-year students and then seniors  Longitudinal study is better  Cross-sectional study is possible (but more expensive)  The institution is the primary unit of analysis  Institutions can be compared for value-added gains
    9. 9. The “Dual Pilot” study at UC  To consider the applicability of learning portfolios (and especially e-portfolios) alongside the CLA among a group of first-year Honors students  Best strategies towards  The transition to become student scholars  The development of General Education baccalaureate competencies over the first year  The need for intervention mechanisms to facilitate further transitions
    10. 10. Methodology  Administer the CLA as a course requirement to 117 first- year, first-time baccalaureate seeking, Honors students  Average ACT is 31  Representing a range of UC colleges and programs  Half took the CLA tests to “make an argument” and to “critique an argument,” with the other half doing the problem-solving “performance task”
    11. 11. Additional information for gathering  Level of effort exercised in completing each instrument  Sense of engagement with this process  Sense of learned perceptions from completing the CLA test  Reactions to and self-assessment of their performance  Role of any “ceiling effect”
    12. 12. E-Portfolio Artifacts  Two essays posted in the e-portfolio with  Focus on personal academic development  Focus on designing a experiential learning project  Use of AAC&U metarubrics  Integrative Learning  Critical Thinking  Written Communication
    13. 13. PreliminaryResults:Overall Impression I believe the test is a waste of time and that it 6 6.6% would not be at all useful in testing students’ abilities I believe the test would be only mildly useful in 40 44% testing students’ abilities I believe this test would be useful in testing 36 39.6% students’ abilities I believe the test would be very useful in testing 9 9.9% students’ abilities I believe this test is one of the most useful tests I 0 0% have taken and that it will provide outstanding insight into students’ abilities
    14. 14. Preliminary Results: Testing Experience Taking this test was a very unpleasant experience 9 9.9% Taking this test was a somewhat unpleasant 25 27.5% experience Taking this test was neither a pleasant nor 43 47.3% unpleasant experience Taking this test was a somewhat fun, interesting or 12 13.2% pleasant experience Taking this test was a fun, interesting or pleasant 2 2.2% experience.
    15. 15. Preliminary Results: Effort Expensed Almost no effort at all 2 2.2% A little effort 5 5.5% Some effort 36 36.9% Very much effort 41 45.1% Serious and sustained effort 7 7.7%
    16. 16. Conclusions Thus Far  Questionable reliability of results  Lack of integration into the curriculum  Disconnect between the CLA and UC’s mission  Absence of ownership by the faculty  Questionable motivation by seniors taking the test  High costs to administer the test (over $100 per student)
    17. 17. Matching E-Portfolios to Standardized Tests for Value-Added Assessment Richard Robles EDLD 812: Data Improvement