Your SlideShare is downloading. ×
  • Like
An assessment based_degree
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

An assessment based_degree

  • 165 views
Published

WCET Annual Meeting Presentation

WCET Annual Meeting Presentation

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
165
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Restricting access to the “best” students can increase quality.Increasing cost can increase quality.How can we have high quality while allowing open access and containing costs? Answer: scalability and decoupling
  • Assessment quality is not just about decoupled assessments. These aspects of quality apply in bricks-and-mortar and online courses too: anything that is used for academic credit needs to have good assessments. Note, though, that over the course of a semester, there are usually multiple assessments, so the individual assessments need not all be extremely rigorous: the aggregation of multiple assessments can lend reliability in cases in which a single assessment may not be highly reliable. However, in decoupled assessments, there is usually an “all your eggs in one basket” approach, so the basket needs to be very, very good.
  • The big advantage courses have over decoupled assessments is that the first issue is usually not a problem!Formative assessments (ones given throughout a course) need not worry about depth—aggregated, though, they should cover everything.
  • These two characteristics are the biggest pitfalls for the lower-stakes assessments usually seen as part of courses. Instructors often have no training in assessment.
  • Appropriate here is on two levels. It’s important for the assessment to match the content of what examinees are expecting to learn. But it’s also important for the objectives to be valuable to credit-granting bodies. If I build a beautiful assessment for an OER course on water management that relates nicely to the course objectives, that assessment will still not be appropriate for someone seeking credit specifically for a degree in literature. Validity in this area often balances with validity in scalability: it’s often hard to do a great job on appropriateness (good matching) while at the same time providing a truly decoupled assessment
  • Our typical student brings 5 transcripts to the table and may start a BA/BS program with about 60 credits completed. The Academic Advisor is the generic job at EC, and they specialize in helping students identify what they need.
  • We began developing our current models to serve 1) an international agency and 2) the challenge of providing a $10K degree. But we don’t develop “exclusives.”

Transcript

  • 1. AN ASSESSMENT-BASED DEGREE November 1, 2012 WCET Annual Conference
  • 2. The Iron Triangle2 Cost Quality Access
  • 3. Scalability of Content3  Open Educational Resources: accessible to millions  Independent Study  Requires that learners be self-motivated
  • 4. Types of OER  Lecture notes Physics lecture  Videos of classroom lectures  Lessons designed for OER  CoursesOpenStudy  Discussion groups Khan Academy Open University
  • 5. What is OER’s value? Learning CertificatesBut… OER by itself does not typically award formal educational credit Why not?
  • 6. Academic credit6  What is credit?  Assurance that someone knows something  The something must be appropriate for the particular academic program  To provide that assurance, both the someone and the something must be verified
  • 7. Decoupling assessment from the7 classroom “What you know is more important than where or how you learned it.” Credit should be based on knowledge, not attendance Knowledge gained through independent study can be independently assessed
  • 8. Aspects of validity for credit by assessment8  Identity verification  Assessment quality  Appropriateness of knowledge tested for a particular degree program  Scalability
  • 9. Identity verification9 Ryan Ruppe Jeffery Turner Steve Winton
  • 10. Threats to validity--courses10  Did the person actually go through the course?  Did the person do his/her own work?  Is the person who took the course the same person who is presenting the credential?
  • 11. Threats to validity--11 assessments  Is the person taking the assessment the same person who is claiming the knowledge?  Is the person claiming the knowledge the same person who is presenting the credential?
  • 12. Assessment quality12 A good assessment  Measures knowledge of the subject  Does not measure irrelevant characteristics  Gives a person the same score regardless of which form is taken  Gives people of the same ability the same score
  • 13. Threats to assessment quality13  Assessments may not measure quite the same content as the material studied  Not all things in the world labeled “Sociology” cover the same topics!  Assessments may not cover material in enough depth A 10-question quiz is unlikely to cover the equivalent of a college semester’s worth of content
  • 14. Threats to assessment quality14  The assessment might not be scored consistently enough  Different instructors have different standards  Assessments might measure irrelevant characteristics  Unnecessarily complicated questions  Questions with “giveaway” answers  Trick questions
  • 15. Appropriateness of content15  Match of assessment to OER  How close are the assessment specifications to the learning objectives of the OER material?  Match of assessment to credit-granting body  How close are the assessment specifications/learning objectives to what is taught at the institution where credit is sought?
  • 16. Efficient assessment Machine-scored competency/proficiency exams can handle large numbers of examinees and be used for multiple OER sources Machine-scored exams need not measure simply mindless regurgitation of facts For some types of OER, more specific assessments may be needed  Portfolios for highly specialized content Cost of a 3-credit multiple-choice assessment:
  • 17. Scalability - Generalizability Lessgeneralizable  Course final exams and homework  Third-party assessments designed for a specific course  Competency/proficiency assessments Moregeneralizable
  • 18. Scalability – Large groupsLess scalable  Individual assessments  Portfolios  Research papers  Oral examinations  Human-scored group assessments  Short answer questions  Essay questions  Machine-scored assessments  Multiple-choice exams  Machine-scored constructed responseMore scalable exams
  • 19. A model for low-cost educationOER University
  • 20. What is Charter Oak State College?  Founded in 1973 in Connecticut  Enrollment about 2,000  Over 11,000 degrees awarded, with alumni from all 50 states and a dozen countries  Partnerships with military branches  Associate and Baccalaureate degrees  New England Association of Schools and Colleges accreditation
  • 21. What is Charter Oak State College? Aggregation model - with Cornerstone and Capstone  Outcome assessment courses  Writing, research, program planning Transfer credit CBE, portfolio assessment, ACE and National CCRS, CCAP, institutionally reviewed training and certifications
  • 22. Student Profile CONCENTRATION PLAN OF STUDY (CPS) FORM Business Administration MINIMUM Number and Level of MINIMUM SUBJECT REQUIRED SOURCE: TEST OR INSTITUTION COURSE/TEST NO. & TITLE GRADE Credits Propose REQUIREMENTS CREDITS Lower UpperAccounting: 3 credits CLEP Financial Accounting Prop (3) Financial * Managerial * 3 credits StraighterLine Managerial Acct. B 3Finance: 3 credits DSST Prin. Finance Prop (3) Principles * or Managerial * or CorporateManagement: 3 credits Excelsior Coll. Organizational Behav Prop 3 Organizational BehaviorManagement: 3 credits Northwood U. MGT 230 Prin. Mgt. A 2.7 Intro/Principles Management*Marketing: 3 credits Northwood U. MKT 208 Prin. Mark. C+ 2.7 Principles of Marketing*Strategic Processes 3 credits COSC MGT 499 Strategic Prop 3Capstone Course Management
  • 23. Program PlanningHi Karen,As per your request, Ive included a list below of tests/ independentstudies courses I am planning on taking for my degree.• Pending exams/courses: • Cornerstone course (IDS 101)- in progress • College Algebra - Straighterline, in progress • English Composition II - Straighterline, in progress • Mgmt Info Systems - DSST (area of concentration) • Environment & Humanity – DSST • *Business Ethics in Society - DSST (UL area of concentration) • Organizational Behavior - Excelsior Exam(UL area of concentration) • Principles of Public Speaking - DSST
  • 24. Class of 2011-12 & CBE2011-12 Charter Oak StateCollege Graduates: Average of Test Credit Graduates Average of Test Count HoursNo Testing 310 64%Took 1 or more tests 176 5.23 17.64 36%Grand Total 486 5.23 17.64
  • 25. What Is Excelsior College? Independent institution serving adult students and other non-traditional learners since 1971 Enrollment 34,000; alumni 141,000 Up to one-third of EC students are  Members of historically under-represented groups  Military or veterans Average age of graduates: late 30s Associate, Baccalaureate, and Masters degrees Middle States accreditation
  • 26. What Is Excelsior College? Aggregation model  EC online courses (available since 2004)  Two national exam series developed in house: ECE and UExcel  Transfer credit  Consistently ranked #1 or #2 in acceptance of credit from other accredited institutions  Other CBE, portfolio assessment, ACE and National CCRS, institutionally reviewed training and certifications Largest pre-licensure nursing program in the country, entirely assessment based
  • 27. Excelsior College Exam-Based Degree Paths New degree pathways emphasize Excelsior’s own CBE products, along with a single required Capstone course Degree planning templates show suggested sequence of exams; students with transfer credit have wider choices Supported Independent Study model is based on Open Courseware options reviewed and recommended for all exams Initial degree templates for BS and BA in Liberal Studies and AS and BS in General Business
  • 28. Sample ASB Degree Template (ETC ASB example)
  • 29. Infrastructure Needs29  Common definitions of assessment-based learning among the Regional Accreditors  Access to Federal Financial Aid  Proposal:a demonstration program that lets a set of low cost assessment approaches prove their merits.  National Database of existing credential and training assessments
  • 30. A New Narrative30  We need to add a chapter to the higher education biography.  The story begins with Socratic learning, moves to libraries, adds faculty, morphs universities, carves land grants from the wilderness, upscales adult education into Community Colleges, and invests in for-profits.  Now the story must add competency-based credentialing