3. Testing Instruments
Use of pre-created instruments to measure
particular traits or domains
Examples: Myers-Briggs Type Indicator, Moral
Development Inventory, Collegiate Learning
Assessment (CLA), Measurement of
Academic Proficiency and Progress (MAPP)
Participants are given score and compared
to normative data
Can be useful if measuring a specific area
3
4. Quasi-experiments
Seek to control variables by assessment
design
Can give confident results when
implemented
Can be resource intensive
Not used often in education
Pre-post test is most popular design
4
5. Ratings of Skills
Teacher/advisor/supervisor rates skills of student
Could be part of formal evaluation process
Helpful to have criteria for ratings
5
6. Capstone/Culminating Activity
Allows student opportunity to apply and
demonstrate knowledge and skills from a
variety of learning experiences
May be as part of a course
May not necessarily be planned event
6
7. Observation
Used to gather qualitative data in an
unobtrusive way
Need ongoing access to group you want
to collect data from
May take great deal of time to transcribe
notes and analyze data
7
8. Narrative/Journaling
Allows student to reflect on
experience
Can be analyzed using a
rubric or content analysis
Demonstrates writing skills,
critical thinking skills, and
can also provide insight into
other types of learning
Need to consider intent for
student and intent for
assessment
8
9. Portfolio
Way of documenting progression
towards goals
Can be electronic or paper
Used for a variety of learning outcomes
Provides opportunity for reflection by
student
Allows for feedback from
staff/faculty/other students
Can serve as an archive
9
11. Visual Collection
Captures images as the data to
analyze
Provides great detail
May be easy to alter images
Limited number of perspectives
Photo Journaling
http://www.usi.edu/depart/instires/air
99%20reflex/index.htm
11
13. Tracking
Simply tracking individuals served/affected
Important to gather data in detailed way
for future analysis
Helpful to have an ID number to connect
to other institutional databases
Need a systematic electronic
format
13
14. Checklists
Can be used by collector or collectee to
identify actions or activities that have
occurred or taken place
Often incorporated into surveys
Only indicate existence, not any judgment
14
15. Checklists
Mark group leadership characteristics
candidate demonstrated:
Listened to others
Summarized what was said
Tried to reconcile differing opinions
Resolved conflicts
Tried to build consensus
Helped group move forward
15
16. Surveys
Can be paper or electronic
Used to collect data from many people
quickly and easily
Limited resources needed
Unfortunately, this the default
16
17. Interviews
Used to obtain detailed information
and allow for direct follow-up
Can be in person or on the phone
Can gather rich data
Need to develop trust with interviewee
Can be expensive and time consuming
Takes a great deal of time to transcribe
notes and analyze data
17
18. Focus Groups
Can be done in person or online
Allow for direct follow-up
Need to develop trust between
moderator and participants
Provide depth of answers, but lack
breadth
Can be time consuming to collect and
analyze data
18
19. Concept Map
Used to demonstrate
relationships and
connections
Can demonstrate
critical thinking skills
Allows user to be
creative
19
21. Rubrics
Set of criteria to judge student
demonstration of learning
Completed by rater or learner
Can be holistic or component
Can be an effective assessment tool,
but underutilized
Stevens, D. D. & Levi, A. J. (2005). Introduction to rubrics: An assessment tool to save grading
time, convey effective feedback, and promote student learning. Sterling, VA: Stylus.
21
22. AAC&U VALUE
Critical Thinking Rubric
4 3 2 1
Problem/issue relevant to Problem/issue relevant to Problem/issue relevant to Problem/issue relevant to a
Explanation of issues
situation in context is clearly situation is stated and partially situation is stated different situation identified
stated described
An appropriate (for assignment) An adequate (for assignment)
Limited reputable sources are Questionable sources are
Evidence variety of reputable sources are variety of reputable sources are
selected and used. selected and used.
selected and used. selected and used.
Recognizes significant Responds to some implications
Shows emerging awareness of Presents position without
Influence of context and implications of context and of context and assumptions in
context and assumptions in consideration of assumptions or
assumptions assumptions in developing and developing and presenting a
presenting a position. context.
presenting a well qualified qualified position.
position.
Student's perspective is
Student's perspective is rational Student's perspective is clear
Own perspective, hypothesis, multifaceted and exhibits Only student's perspective is
and considered in light of other and an alternative is
or position complex and appropriate apparent.
perspectives. recognized.
consideration of other
perspectives.
A comprehensive conclusion Conclusions acknowledge
Conclusions are integrated from Conclusion is emerging with
Conclusions, implications synthesizes sources and has a sources with limited
sources with consideration of scant attention to implications
and consequences nuanced consideration of consideration of implications
implications and consequences. and consequences.
implications and consequences. and consequences.
22
23. Scoring Rubric
(based on AAC&U VALUE teamwork rubric)
Task Description:
Highest Level Comments
Contributes to team meetings Contributions to team meetings have
exceptional positive impact.
Facilitates the contributions of team Actively engages all (or nearly all) team
members members in ways that facilitate their
contributions.
Displays necessary work ethic Goes above and beyond the call. Completes
own assignments in a superior manner, while
also assisting team in completing other tasks
that contribute to team success.
Fosters constructive team climate Treats members respectfully and successfully
fosters constructive team climate by saying or
doing things which make others feel valued in
the group and able to contribute.
Response to conflict Addresses conflict directly and
constructively, helping to resolve it in a way
that strengthens overall team cohesiveness
and future effectiveness.
23
24. CAT Overview
Angelo, T. A., & Cross, K., P. (1993).
Classroom assessment techniques (2nd
Edition). San Francisco: Jossey-Bass.
24
25. Focused Listing
Assesses prior knowledge, recall, and
understanding.
This CAT focuses students’ attention on a single
important term, name, or concept from a particular
lesson or class section and directs them to list
several ideas that are closely related to that “focus
point.”
Helps educator determine the most effective
starting point for an activity.
Examples?
25
26. Empty Outlines
Assesses prior knowledge, recall, and
understanding.
The instructor provides students with an empty or
partially completed outline or an in-class
presentation or assignment and gives students
limited time to fill in the blank spaces.
Helps instructor know how well students grasped
main points.
Examples?
26
27. Minute Paper
Assesses prior knowledge, recall, and
understanding.
At the end of an activity students are asked to
respond briefly to some variation of “what was the
most important thing you learned in class.
Provides manageable amounts of timely and useful
feedback for a minimal investment of time and
energy.
Examples?
27
28. Muddiest Point
Assesses prior knowledge, recall, and
understanding.
The technique consists of asking students to jot
down a quick response to one questions: “What is
the muddiest point in ______?” The focus of the
Muddiest Point assessment might be any activity.
Provides information on what students find least
clear or most confusing about a an activity or
interaction.
Examples?
28
29. One Sentence Summary
Assesses synthesis and creative thinking.
This simple technique challenges students to answer
the questions “Who does what to whom, when,
where, how, and why?” about a given topic, and
then to synthesize those answers into a single
informative, grammatical, and long summary
sentence.
Enables educators to find out how concisely,
completely, and creatively, students can summarize
a large amount of information.
Examples?
29
30. Documented Problem Solutions
Assesses problem solving.
This technique prompts students to keep track of the
steps they take in solving a problem – to “show and tell”
how they worked it out. By analyzing these detailed
protocols – in which each solution step is briefly explained
in writing – teachers can gain valuable information on
their students’ problem-solving skills.
There are two main aims: 1) assess how students solve
problems and 2) assess how well students understand
and can describe their problem-solving methods.
Examples?
30
31. Directed Paraphrasing
Assesses application and performance.
Directed paraphrasing is an assessment technique
designed to assess and help develop the ability to
translate highly specialized information into language
that clients or customers understand. Students are
directed to paraphrase part of a policy or practice for a
specific audience and purpose, using their own words.
Provides feedback on students’ ability to summarize and
restate important information or concepts in their own
words.
Examples?
31
32. Application Cards
Assesses application and performance.
After students have heard or read about an
important principle, generalization, theory, or
procedure, they receive an index card and are
asked to write down at least one possible, real-world
application for what they have just learned.
You can know quickly how well students understand
the possible applications of what they have
learned.
Examples?
32
What testing instruments have you used in the past?What have been benefits of these instruments?
What are examples of quasi-experiments? Roommate matching Alcohol education programs Employee trainingWhat are the disadvantages of quasi-experiments?
What are examples of this technique? RA evaluation Feedback from student group advisorWhat do you think are some benefits?
This is typically conceived as “academic”. How might this be used in student affairs? Final project for a leadership course Actual leadership of a student organization after taking a leadership course Crisis RA has to deal with
Examples of the use of observation? Observe cigarette butts or noise level to determine effectiveness of policy Observe foot paths to assess appropriateness of outside lightingWhat are some benefits of observation?
Different formats: reflection paper, 1-minute paper, major take-awayHow can you envision using this technique? Student leaders journal about what they are learning RAs journal about challenges and how they are addressing them Sanction for judicialWhat would be the benefits of this technique?
How have you seen this being using in student affairs? Student leaders Extracurricular portfolioWhat would you think is the most useful benefit of a portfolio?
How have you seen this being using in student affairs? Student leaders Extracurricular portfolioWhat would you think is the most useful benefit of a portfolio?
How might you think this could be used?
Athletics, Faith Sharing groups
How many of you have used checklists? How have you used them?
RA hiring group exerciseTeamwork evaluation/student group evaluation
Satisfaction with student health planEffectiveness of residence life staffCharacteristics of incoming students
What are some examples? Student athletes tips for success Women in STEM majors experience
Examples? RAs/UGAs to find out best time to program Students of color to understand experience on campus Study abroad to find out what students learned
What else might you use a concept map for?
When might you use rubrics?-student evaluations-RA/student leader interviews
Needs assessmentExample: In an leadership development workshop, students list all of the leadership skills they can think of and identify ways they may have demonstrated those. The instructor can review these lists to understand what knowledge students have and what types of experiences are they coming in with.
Example: Program on finding an internship related to major. If there are blanks because students can’t answer, then staff can consider how to deliver that information differently.
Example: Use at the end of weekly staff meetings. The supervisor can determine if her outcomes for the meeting were achieved and if not can revisit those through other approaches.
Example:At the end of a residence hall floor meeting, RA passes out index cards asking students to respond. The questions can be answered then or in later meetings or via electronic communication. Benefits are that students questions can be answered and staff know what they need to articulate differently next time.
Example: new student leader meets with staff member to discuss process for putting on a campus event and is asked to summarize that process at the end of the meeting. The staff member can then correct any incorrect information regarding the process.
Examples: Many orientation staffs have “Behind Closed Doors” to allow students to role-model problems. While good experience, these are resource intensive since many staff need to be involved. Documented problem solutions could be another method of allowing OLs to practice how to solve a problem. Staff can review these and offer feedback. If the same issue (e.g. sexual assault) is given to multiple students, the solutions can be analyzed to see if there are common themes which can be addressed in a future staff meeting and revised in future training sessions.
Example: RAs need to explain the rationale of the college alcohol policy to their students. Hall directors can evaluate these paraphrasing for accuracy and provide feedback for improving the statements.
Example: Intramural officials are asked to articulate how they will apply the skills that they learned officiating intramural sports.
