Hi everyone, I’m Ann Emery. For the past few years, I was an internal evaluator. I worked (here) at a non-profit youth center in downtown Washington, DC. Today I’m here to share a few of the reasons why I love internal evaluation.
When some people think about internal evaluation, the first things to come to mind are the negative aspects – words like pressures, stresses, and internal politics. So that’s why I’m here today – to share another perspective with you.
So let’s get down to business and start with the “most important” benefit of all… there’s no commuting! Everyone’s in the same building, so my “clients” work right down the hallway, not across the city. This is a big deal in a gridlocked city like DC.
Since everyone’s located in the same building, internal evaluators can simply walk down the hall to hold casual and informal meetings with staff over a cup of coffee. We can hold meetings anytime, anywhere, which is nice for us and even nicer for staff.
We can attend holiday parties (like this holiday party) or simply go out to lunch and grab some frozen yogurt (like this group lunch). Talking together creates a welcoming and friendly atmosphere, which is especially helpful for frontline staff, who get intimidated by formal meetings and board rooms.
Another practical advantage of internal evaluation is that there are NO TIMESHEETS and no billing codes. You’re a salaried employee, not a consultant who’s paid by the hour. This gives us lots of flexibility in our work schedules and in our evaluation plans.
So how do we take advantage of our flexible schedules? One way is that we can use our flexibility to conduct qualitative research. I used to attend graduation parties like this one, and I’d schedule interviews and focus groups right before or right after the events to make participation as convenient as possible.
Since we’re not tightly bound by contracts and timesheets, we also have the flexibility to be a little more innovative in our evaluation approaches. Here’s an example where I tried out different interactive focus group techniques with program participants.
We have opportunities to volunteer at all sorts of events hosted by our organizations [like these].I can chat informally with the program participants when I’m volunteering. These conversations help me understand their reading levels and comfort with English vs. Spanish, which helps me design surveys and other instruments that are a good fit.
Our flexibility makes it easy to form partnerships with nearby universities. For example, we can team up with graduate students and professors to dive deeper into certain projects. The external perspective also comes in handy when we get wrapped inside office politics.
We can make sure data and evaluation are always on the meeting agenda, which makes data-driven decisions happen naturally. Since I reporteddirectly to the Chief Operating Officer, we talked about evaluation at our weekly staff meetings, instead of having to wait until an annual report.
Whether it’s a weekly staff meeting,or a quarterly board of directors meeting [like this one], the evaluation process and results remain high priorities in the organization – which means results get put into action and used to make improvements.
How exactly does this happen? Well, different leaders prefer different evaluation approaches. We get to know the leaders personally, so we can tailor our formats, styles, etc. to each and every personality type – like dashboards for one manager and oral presentations for another.
We can also build long-term relationships with mid-level managers and frontline staff. I got to partner with our computer instructor (here) to teach Excel certification courses to our staff. We used real program data, and showed staff how to analyze their own data, which puts more meaning behind the numbers.
We can also lead brown bags about everything from logic models to research design. As a result of this capacity building, internal evaluators become more of a “coach,” guiding staff through evaluation rather than making decisions on their behalf.
We get to evaluate a variety of programs! Since I worked at a multi-service youth organization, I had opportunities to evaluate everything from our afterschool drop-in center to our tennis lessons for teeny tiny little 5-year-olds.
We can also use a variety of evaluation designs and methods. We can use our insider knowledge of the program’s stage and the staff capacity level to use anything from a pretest-posttest design, to a simple feedback form, to an RCT.
And finally, internal evaluators get the personal satisfaction of building an organizational culture of learning. Staff are constantly working together, thinking about data, participating in the evaluation process, and using data to improve programs.
Now, when you think about internal evaluation, I hope some of the positive aspects come to mind – words like innovative, teaching, learning, partnerships – words that bring to mind the collaborative, participatory, and utilization-focused nature of internal evaluation.
Thank you very much for listening! I hope this presentation has given you a different perspective. If you’d like to learn more, please contact me through email, Twitter, or my blog. Thanks!
http://www.flickr.com/photos/epsos/5591761716/sizes/l/in/photostream/http://www.flickr.com/photos/joeshlabotnik/2463318076/sizes/l/in/photostream/ no commuting!http://www.flickr.com/photos/oatsy40/6901864973/sizes/l/in/photostream/