Assessment
Conundrum
Panel at the Georgia International Conference on Information Literacy
Olga Koz and Chris Sharpe, Kennesaw State University
September 30, 2016
Introduction
Main topics for discussion
Two mini-reports:
Assessment of Learning
Outcomes
Program Assessment
Problems & Concerns
1. There is no more ACRL's “Information Literacy Competency Standards
for Higher Education” prescribing general practices for all students
2. The disconnect between what is being taught and what is being tested.
3. What do we measure: inputs, outcomes, perceptions, outputs?
4. How do we measure or “Instrument dilemma”?
5. Library Assessment vs. Educational Assessment
6. Assessment of outcomes of “library orientations, ” one-shot training
sessions vs. assessment of a learning module or a course
Standards
All good assessment begins with
standards: statements of what we
want our students to know and be
able to do.
What do librarians value?
What our institutions value?
What professional or academic
communities value?
Assessment of
Learning
Outcomes
Dr. Olga Koz
1. What should I assess?
2. How should I assess?
3. Traditional assessment
4. Authentic assessment
What should I assess?
I have been asked I want to measure
How many workshops/in-class
presentations have I done?
Students’ acquired knowledge (only
formative assignments)
How many library orientations? Students’ performance (graded authentic
assignments)
How many students were in attendance? Students’ engagement rates (engage with
courseware and learning materials)
How many reference questions? Students’ metacognitive skills (hard to
assess)
How many consulting sessions? Students’ level of competency (combination
of knowledge & performance)
How many research guides have I created?
Inputs/Quantity Outcomes/Quality
How should I assess?
Traditional Library Assessment Traditional Teaching Assessment
Survey of users’ perceptions, self-report
data
Students’ perceptions (Teacher evaluation)
Short term intervention ( pre/post test),
Quizes or Exercises
Course or assignment related tests &
exams (designed by an instructor)
Standardized tests/exams (SAILS/TATIL;
iSkills™ )
Standardized tests/exams
Authentic Assessment
A form of assessment in which
students are asked to perform
real-world tasks that
demonstrate meaningful
application of essential
knowledge and skills -- Jon
Mueller
- Co-instructor for Capstone
Courses (two assignments)
and Literature Review
course
- Co-instructor for “Research
courses” with assignments
on literature/research
evaluation and synthesis
- Learning modules + an
assignment for several core
courses
Products/Assignments:
Creating “Analysis” & “Synthesis” Matrix
Literature Review (Dissertation, Capstone, Term paper or research
project)
Research
Citation analysis of dissertations
Content analysis of students works (annotated bibliogrpahies
, evalaution matrices and literature reviews)
Knowledge map (network analysis) of students’ literature
reviews
Program
Assessment
Chris Sharpe
C. Student/faculty satisfaction?
II. Comprehensive instruction assessment
plan
III. Methods
A. Authentic Assessment
1. Evaluate assignment based
on rubric
B. Pre/posttests
C. Surveys
D. National tests
IV. Challenges/issues
A. Lack of majors/minors/cohorts to
track

Assessment Conundrum

  • 1.
    Assessment Conundrum Panel at theGeorgia International Conference on Information Literacy Olga Koz and Chris Sharpe, Kennesaw State University September 30, 2016
  • 2.
    Introduction Main topics fordiscussion Two mini-reports: Assessment of Learning Outcomes Program Assessment
  • 3.
    Problems & Concerns 1.There is no more ACRL's “Information Literacy Competency Standards for Higher Education” prescribing general practices for all students 2. The disconnect between what is being taught and what is being tested. 3. What do we measure: inputs, outcomes, perceptions, outputs? 4. How do we measure or “Instrument dilemma”? 5. Library Assessment vs. Educational Assessment 6. Assessment of outcomes of “library orientations, ” one-shot training sessions vs. assessment of a learning module or a course
  • 4.
    Standards All good assessmentbegins with standards: statements of what we want our students to know and be able to do. What do librarians value? What our institutions value? What professional or academic communities value?
  • 5.
    Assessment of Learning Outcomes Dr. OlgaKoz 1. What should I assess? 2. How should I assess? 3. Traditional assessment 4. Authentic assessment
  • 6.
    What should Iassess? I have been asked I want to measure How many workshops/in-class presentations have I done? Students’ acquired knowledge (only formative assignments) How many library orientations? Students’ performance (graded authentic assignments) How many students were in attendance? Students’ engagement rates (engage with courseware and learning materials) How many reference questions? Students’ metacognitive skills (hard to assess) How many consulting sessions? Students’ level of competency (combination of knowledge & performance) How many research guides have I created? Inputs/Quantity Outcomes/Quality
  • 7.
    How should Iassess? Traditional Library Assessment Traditional Teaching Assessment Survey of users’ perceptions, self-report data Students’ perceptions (Teacher evaluation) Short term intervention ( pre/post test), Quizes or Exercises Course or assignment related tests & exams (designed by an instructor) Standardized tests/exams (SAILS/TATIL; iSkills™ ) Standardized tests/exams
  • 8.
    Authentic Assessment A formof assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills -- Jon Mueller - Co-instructor for Capstone Courses (two assignments) and Literature Review course - Co-instructor for “Research courses” with assignments on literature/research evaluation and synthesis - Learning modules + an assignment for several core courses Products/Assignments: Creating “Analysis” & “Synthesis” Matrix Literature Review (Dissertation, Capstone, Term paper or research project)
  • 9.
    Research Citation analysis ofdissertations Content analysis of students works (annotated bibliogrpahies , evalaution matrices and literature reviews) Knowledge map (network analysis) of students’ literature reviews
  • 10.
    Program Assessment Chris Sharpe C. Student/facultysatisfaction? II. Comprehensive instruction assessment plan III. Methods A. Authentic Assessment 1. Evaluate assignment based on rubric B. Pre/posttests C. Surveys D. National tests IV. Challenges/issues A. Lack of majors/minors/cohorts to track