AN ASSESSMENT-BASED      DEGREE    November 1, 2012  WCET Annual Conference
The Iron Triangle2                   Cost      Quality             Access
Scalability of Content3     Open Educational Resources: accessible      to millions     Independent Study     Requires ...
Types of OER              Lecture notes                  Physics lecture              Videos of classroom lectures      ...
What is OER’s value?   Learning   CertificatesBut… OER by itself does not typically  award formal educational credit   ...
Academic credit6       What is credit?         Assurance   that someone knows something         The something must be a...
Decoupling assessment from the7    classroom    “What you know is more important than where or                 how you lea...
Aspects of validity for credit by    assessment8          Identity   verification          Assessment      quality      ...
Identity verification9     Ryan Ruppe                                 Jeffery Turner                  Steve Winton
Threats to validity--courses10        Did the person actually go through the course?        Did the person do his/her ow...
Threats to validity--11     assessments        Is the person taking the assessment the same         person who is claimin...
Assessment quality12      A good assessment       Measures knowledge of the subject       Does not measure irrelevant ch...
Threats to assessment quality13        Assessments may not measure quite the same         content as the material studied...
Threats to assessment quality14        The assessment might not be scored         consistently enough          Different...
Appropriateness of content15        Match of assessment to OER          How  close are the assessment specifications to ...
Efficient assessment   Machine-scored competency/proficiency    exams can handle large numbers of    examinees and be use...
Scalability - Generalizability   Lessgeneralizable                   Course final exams and homework                   T...
Scalability – Large groupsLess scalable                   Individual assessments                     Portfolios         ...
A model for low-cost            educationOER University
What is Charter Oak State College?    Founded in 1973 in Connecticut    Enrollment about 2,000    Over 11,000 degrees a...
What is Charter Oak State College?   Aggregation model - with Cornerstone and    Capstone       Outcome assessment cours...
Student Profile                                   CONCENTRATION PLAN OF STUDY (CPS) FORM                                  ...
Program PlanningHi Karen,As per your request, Ive included a list below of tests/ independentstudies courses I am planning...
Class of 2011-12 & CBE2011-12 Charter Oak StateCollege Graduates:                                                         ...
What Is Excelsior College?   Independent institution serving adult students and other    non-traditional learners since 1...
What Is Excelsior College?   Aggregation model       EC online courses (available since 2004)       Two national exam s...
Excelsior College Exam-Based Degree Paths   New degree pathways emphasize Excelsior’s own CBE    products, along with a s...
Sample ASB Degree Template   (ETC ASB example)
Infrastructure Needs29        Common definitions of assessment-based         learning among the Regional Accreditors     ...
A New Narrative30        We need to add a chapter to the higher         education biography.        The story begins wit...
Upcoming SlideShare
Loading in …5
×

An assessment based_degree

325 views
258 views

Published on

WCET Annual Meeting Presentation

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
325
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Restricting access to the “best” students can increase quality.Increasing cost can increase quality.How can we have high quality while allowing open access and containing costs? Answer: scalability and decoupling
  • Assessment quality is not just about decoupled assessments. These aspects of quality apply in bricks-and-mortar and online courses too: anything that is used for academic credit needs to have good assessments. Note, though, that over the course of a semester, there are usually multiple assessments, so the individual assessments need not all be extremely rigorous: the aggregation of multiple assessments can lend reliability in cases in which a single assessment may not be highly reliable. However, in decoupled assessments, there is usually an “all your eggs in one basket” approach, so the basket needs to be very, very good.
  • The big advantage courses have over decoupled assessments is that the first issue is usually not a problem!Formative assessments (ones given throughout a course) need not worry about depth—aggregated, though, they should cover everything.
  • These two characteristics are the biggest pitfalls for the lower-stakes assessments usually seen as part of courses. Instructors often have no training in assessment.
  • Appropriate here is on two levels. It’s important for the assessment to match the content of what examinees are expecting to learn. But it’s also important for the objectives to be valuable to credit-granting bodies. If I build a beautiful assessment for an OER course on water management that relates nicely to the course objectives, that assessment will still not be appropriate for someone seeking credit specifically for a degree in literature. Validity in this area often balances with validity in scalability: it’s often hard to do a great job on appropriateness (good matching) while at the same time providing a truly decoupled assessment
  • Our typical student brings 5 transcripts to the table and may start a BA/BS program with about 60 credits completed. The Academic Advisor is the generic job at EC, and they specialize in helping students identify what they need.
  • We began developing our current models to serve 1) an international agency and 2) the challenge of providing a $10K degree. But we don’t develop “exclusives.”
  • An assessment based_degree

    1. 1. AN ASSESSMENT-BASED DEGREE November 1, 2012 WCET Annual Conference
    2. 2. The Iron Triangle2 Cost Quality Access
    3. 3. Scalability of Content3  Open Educational Resources: accessible to millions  Independent Study  Requires that learners be self-motivated
    4. 4. Types of OER  Lecture notes Physics lecture  Videos of classroom lectures  Lessons designed for OER  CoursesOpenStudy  Discussion groups Khan Academy Open University
    5. 5. What is OER’s value? Learning CertificatesBut… OER by itself does not typically award formal educational credit Why not?
    6. 6. Academic credit6  What is credit?  Assurance that someone knows something  The something must be appropriate for the particular academic program  To provide that assurance, both the someone and the something must be verified
    7. 7. Decoupling assessment from the7 classroom “What you know is more important than where or how you learned it.” Credit should be based on knowledge, not attendance Knowledge gained through independent study can be independently assessed
    8. 8. Aspects of validity for credit by assessment8  Identity verification  Assessment quality  Appropriateness of knowledge tested for a particular degree program  Scalability
    9. 9. Identity verification9 Ryan Ruppe Jeffery Turner Steve Winton
    10. 10. Threats to validity--courses10  Did the person actually go through the course?  Did the person do his/her own work?  Is the person who took the course the same person who is presenting the credential?
    11. 11. Threats to validity--11 assessments  Is the person taking the assessment the same person who is claiming the knowledge?  Is the person claiming the knowledge the same person who is presenting the credential?
    12. 12. Assessment quality12 A good assessment  Measures knowledge of the subject  Does not measure irrelevant characteristics  Gives a person the same score regardless of which form is taken  Gives people of the same ability the same score
    13. 13. Threats to assessment quality13  Assessments may not measure quite the same content as the material studied  Not all things in the world labeled “Sociology” cover the same topics!  Assessments may not cover material in enough depth A 10-question quiz is unlikely to cover the equivalent of a college semester’s worth of content
    14. 14. Threats to assessment quality14  The assessment might not be scored consistently enough  Different instructors have different standards  Assessments might measure irrelevant characteristics  Unnecessarily complicated questions  Questions with “giveaway” answers  Trick questions
    15. 15. Appropriateness of content15  Match of assessment to OER  How close are the assessment specifications to the learning objectives of the OER material?  Match of assessment to credit-granting body  How close are the assessment specifications/learning objectives to what is taught at the institution where credit is sought?
    16. 16. Efficient assessment Machine-scored competency/proficiency exams can handle large numbers of examinees and be used for multiple OER sources Machine-scored exams need not measure simply mindless regurgitation of facts For some types of OER, more specific assessments may be needed  Portfolios for highly specialized content Cost of a 3-credit multiple-choice assessment:
    17. 17. Scalability - Generalizability Lessgeneralizable  Course final exams and homework  Third-party assessments designed for a specific course  Competency/proficiency assessments Moregeneralizable
    18. 18. Scalability – Large groupsLess scalable  Individual assessments  Portfolios  Research papers  Oral examinations  Human-scored group assessments  Short answer questions  Essay questions  Machine-scored assessments  Multiple-choice exams  Machine-scored constructed responseMore scalable exams
    19. 19. A model for low-cost educationOER University
    20. 20. What is Charter Oak State College?  Founded in 1973 in Connecticut  Enrollment about 2,000  Over 11,000 degrees awarded, with alumni from all 50 states and a dozen countries  Partnerships with military branches  Associate and Baccalaureate degrees  New England Association of Schools and Colleges accreditation
    21. 21. What is Charter Oak State College? Aggregation model - with Cornerstone and Capstone  Outcome assessment courses  Writing, research, program planning Transfer credit CBE, portfolio assessment, ACE and National CCRS, CCAP, institutionally reviewed training and certifications
    22. 22. Student Profile CONCENTRATION PLAN OF STUDY (CPS) FORM Business Administration MINIMUM Number and Level of MINIMUM SUBJECT REQUIRED SOURCE: TEST OR INSTITUTION COURSE/TEST NO. & TITLE GRADE Credits Propose REQUIREMENTS CREDITS Lower UpperAccounting: 3 credits CLEP Financial Accounting Prop (3) Financial * Managerial * 3 credits StraighterLine Managerial Acct. B 3Finance: 3 credits DSST Prin. Finance Prop (3) Principles * or Managerial * or CorporateManagement: 3 credits Excelsior Coll. Organizational Behav Prop 3 Organizational BehaviorManagement: 3 credits Northwood U. MGT 230 Prin. Mgt. A 2.7 Intro/Principles Management*Marketing: 3 credits Northwood U. MKT 208 Prin. Mark. C+ 2.7 Principles of Marketing*Strategic Processes 3 credits COSC MGT 499 Strategic Prop 3Capstone Course Management
    23. 23. Program PlanningHi Karen,As per your request, Ive included a list below of tests/ independentstudies courses I am planning on taking for my degree.• Pending exams/courses: • Cornerstone course (IDS 101)- in progress • College Algebra - Straighterline, in progress • English Composition II - Straighterline, in progress • Mgmt Info Systems - DSST (area of concentration) • Environment & Humanity – DSST • *Business Ethics in Society - DSST (UL area of concentration) • Organizational Behavior - Excelsior Exam(UL area of concentration) • Principles of Public Speaking - DSST
    24. 24. Class of 2011-12 & CBE2011-12 Charter Oak StateCollege Graduates: Average of Test Credit Graduates Average of Test Count HoursNo Testing 310 64%Took 1 or more tests 176 5.23 17.64 36%Grand Total 486 5.23 17.64
    25. 25. What Is Excelsior College? Independent institution serving adult students and other non-traditional learners since 1971 Enrollment 34,000; alumni 141,000 Up to one-third of EC students are  Members of historically under-represented groups  Military or veterans Average age of graduates: late 30s Associate, Baccalaureate, and Masters degrees Middle States accreditation
    26. 26. What Is Excelsior College? Aggregation model  EC online courses (available since 2004)  Two national exam series developed in house: ECE and UExcel  Transfer credit  Consistently ranked #1 or #2 in acceptance of credit from other accredited institutions  Other CBE, portfolio assessment, ACE and National CCRS, institutionally reviewed training and certifications Largest pre-licensure nursing program in the country, entirely assessment based
    27. 27. Excelsior College Exam-Based Degree Paths New degree pathways emphasize Excelsior’s own CBE products, along with a single required Capstone course Degree planning templates show suggested sequence of exams; students with transfer credit have wider choices Supported Independent Study model is based on Open Courseware options reviewed and recommended for all exams Initial degree templates for BS and BA in Liberal Studies and AS and BS in General Business
    28. 28. Sample ASB Degree Template (ETC ASB example)
    29. 29. Infrastructure Needs29  Common definitions of assessment-based learning among the Regional Accreditors  Access to Federal Financial Aid  Proposal:a demonstration program that lets a set of low cost assessment approaches prove their merits.  National Database of existing credential and training assessments
    30. 30. A New Narrative30  We need to add a chapter to the higher education biography.  The story begins with Socratic learning, moves to libraries, adds faculty, morphs universities, carves land grants from the wilderness, upscales adult education into Community Colleges, and invests in for-profits.  Now the story must add competency-based credentialing

    ×