This document discusses challenges and open questions around using rubrics and focus groups to assess behavior change after a training course. It considers whether to send questionnaires to students and supervisors, whether the rubric should be shared beforehand, whether focus groups should involve students and supervisors immediately and over time, how long it may take for behaviors to change, and how to address limitations of resources and funding. The document does not reach any conclusions and instead raises many questions around effectively evaluating training programs and their real-world impact.

1. The Rubric and
the Behavior
Change
Truth or Sci-Fi?

2. Will a Questionnaire Work
 Suggest sending to students who have
completed course plus their supervisor.
 But, does the supervisor know what he or
she is looking for? If not, does the whole
process lose credibility?
 Or should it just go to the supervisors?
 What if the supervisor is ‘crappy’?
 And if it does should the questionnaire
itself be in the form of a rubric?
 What if the student can’t immediately
implement what he or she learned?
 And is anyone at HQ interested?
 And who collects the data?

3. Should the Rubric be Shared with
Students Before the Course
Begins?
 Is then the rubric a guideline, a checkoff
sheet?
 Would this put more control in the learner’s
hands?
 But what about the intent of the rubric and
the product? This is where the rubric often
falls short.
 And does the student actually know how to
demonstrate the desired behavior?
 How can a rubric assess progression of skill
from mediocre to disciplined/

4. Does a Focus Group Do
What is Needed?
 Would a focus group be the best approach-
a focus group of the student-graduates?
Supervisors? Both?
 Could the focus group be done immediately
after the training, then several months later,
then again, several months after that?
 And what is the optimal time to wait?
 Is three months too long or too short?

5. How About…
 Coordinating with other training programs
then supervisors could be asked about
multiple training programs at one time.
 Or what about how long it may take for
someone to become proficient at the
behavioral change even though they have
the ‘book’ knowledge.
 Or realistically addressing funding,
resources, leveraging other partner
agencies?

6. And.
 Does the rubric address behavior or
performance?

7. Or stated differently: How
many..
 Licks does it take to get to the center of the
Tootsie Pop?