Recurring Steps in the Process Scoring Guide Due Da.docx

Recurring Steps in the Process Scoring
Guide
Due Date: End of Unit 10.
Percentage of Course Grade: 35%.
Recurring Steps in the Process Scoring Guide Grading Rubric
Criteria
Non-
performance
Basic Proficient Distinguished
Integrate knowledge
gained in previous
assignments.
30%
Does not
describe
knowledge
gained in

previous
assignments.
Describes some
of the concepts
or knowledge
gained in
previous
assignments.
Integrates
knowledge
gained in
previous
assignments.
Integrates
knowledge gained
in previous
assignments and
evaluates the

application of the
concepts or
knowledge in this
assignment.
Present to a broad-
based audience of
community leaders a
review of what has
been accomplished.
25%
Does not
present to a
broad-based
audience of
community
leaders a review
of what has
been

accomplished.
Presents to a
broad-based
audience of
community
leaders a partial
review of what
has been
accomplished.
Presents to a
broad-based
audience of
community
leaders a review
of what has
been
accomplished.
Presents to a

broad-based
audience of
community
leaders a review
of what has been
accomplished,
impartially
considering
strengths and
weaknesses in the
process.
Recommend next
steps for both the
Ruby Lake and the
Shoals
neighborhoods.
30%
Does not

describe any
next step for
both the Ruby
Lake and the
Shoals
neighborhoods.
Describes some
next steps for
both the Ruby
Lake and the
Shoals
neighborhoods.
Recommends
next steps for
both the Ruby
Lake and the
Shoals
neighborhoods.

Recommends
next steps for
both the Ruby
Lake and the
Shoals
neighborhoods
and identifies the
criteria behind the
Criteria
Non-
performance
recommendations.
Adhere to APA
formatting standards
and meet the
requirements of

professional written
communications in
the social and
behavioral sciences.
15%
Does not adhere
to APA
formatting
standards or
does not meet
the
requirements of
professional
written
communications
in the social and
behavioral
sciences.

Adheres to APA
formatting
standards and
meets the
requirements of
professional
written
communications
in the social and
behavioral
sciences with
some lapses.
Adheres to APA
formatting
standards and
meets the
requirements of
professional

written
communications
in the social and
behavioral
sciences.
Adheres to APA
formatting
standards, meets
the requirements
of professional
written
communications
in the social and
behavioral
sciences, and cites
at least one article
that supports the
methodology or

conclusions of the
evaluation.
Recurring Steps in the Process Scoring
Guide
Due Date: End of Unit 10.
Percentage of Course Grade: 35%.
Criteria
Non-
performance
Integrate knowledge
gained in previous
assignments.
30%
Does not

describe
knowledge
gained in
previous
assignments.
Describes some
of the concepts
or knowledge
gained in
previous
assignments.
Integrates
knowledge
gained in
previous
assignments.
Integrates
knowledge gained

in previous
assignments and
evaluates the
application of the
concepts or
knowledge in this
assignment.
Present to a broad-
based audience of
community leaders a
review of what has
been accomplished.
25%
Does not
present to a
broad-based
audience of
community

leaders a review
of what has
been
accomplished.
Presents to a
broad-based
audience of
community
leaders a partial
review of what
has been
accomplished.
Presents to a
broad-based
audience of
community
leaders a review
of what has

been
accomplished.
Presents to a
broad-based
audience of
community
leaders a review
of what has been
accomplished,
impartially
considering
strengths and
weaknesses in the
process.
Recommend next
steps for both the
Ruby Lake and the
Shoals

neighborhoods.
30%
Does not
describe any
next step for
both the Ruby
Lake and the
Shoals
neighborhoods.
Describes some
next steps for
both the Ruby
Lake and the
Shoals
neighborhoods.
Recommends
next steps for
both the Ruby

Lake and the
Shoals
neighborhoods.
Recommends
next steps for
both the Ruby
Lake and the
Shoals
neighborhoods
and identifies the
criteria behind the
Criteria
Non-
performance
recommendations.
Adhere to APA

formatting standards
and meet the
requirements of
professional written
communications in
the social and
behavioral sciences.
15%
Does not adhere
to APA
formatting
standards or
does not meet
the
requirements of
professional
written
communications

in the social and
behavioral
sciences.
Adheres to APA
formatting
standards and
meets the
requirements of
professional
written
communications
in the social and
behavioral
sciences with
some lapses.
Adheres to APA
formatting
standards and

meets the
requirements of
professional
written
communications
in the social and
behavioral
sciences.
Adheres to APA
formatting
standards, meets
the requirements
of professional
written
communications
in the social and
behavioral
sciences, and cites

at least one article
that supports the
methodology or
conclusions of the
evaluation.
RIVERBEND
CITY
MISSION
SUMMARY
HPDP
PROGRAM
DESIGN
MISSION
LOCATIONS:
In
this
mission,

you
visited
CEO
Eugene
Pitman’s
office
at
Riverbend
City
Medical
Center.
You
then
visited
two
focus
group
interviews:
one
held
at
a
RCMC
Meeting
room,
and
one
held
at
the
Latimer
Community
Center

in
the
Shoals
neighborhood
of
Riverbend
City—the
neighborhood
that
is
home
to
much
of
the
city’s
Hmong
population.
You
then
visited
the
office
of
Chief
Nursing
Officer
Beatriz
Garcia-‐
Chavez
back
at
RCMC.

MAJOR
THEMES:
In
this
mission,
you
watched
the
beginning
stages
of
the
development
of
a
Health
Promotion
Disease
Prevention
program
(HPDP)
that
is
specifically
targeted
to
a
minority
population.

Embarrassed
by
a
news
report
about
an
altercation
between
the
hospital
and
a
Hmong
family,
Eugene
Pittman
is
prompted
to
investigate
the
feasibility
of
a
Hmong
shaman
training
program.
Similar
to
an
existing

program
in
California,
this
program
would
involve
training
Hmong
shaman
to
bridge
the
communication
gap
between
Hmong
patients
and
the
hospital.
After
an
initial
meeting,
Eugene
sends
Chief

Nursing
Officer
Beatriz
Garcia-‐Chavez
to
interview
nurses
about
their
experiences
with
the
Hmong
community
and
to
ask
them
for
feedback
about
the
proposed
Hmong
shaman
program.
He
also
sends
Nurse
Training
Manager

Shannon
Moe
out
to
conduct
interviews
with
members
of
the
Hmong
community,
which
she
does
with
the
help
of
a
Hmong
community
activist.
After
conducting
the
interviews,
Eugene
decides
the
proposed

HPDP
should
go
forward.
Eugene,
Beatriz,
and
Shannon
discuss
the
correct
steps
for
developing
the
HPDP.
RESOURCES:
The
Free
Press
article
about
the
incident
at
the
hospital.

RIVERBEND CITY MISSION SUMMARY
MSN6012 – COMMUNITY HEALTH MISSION
LOCATIONS:
In this mission you have visited various locations in the Shoals
neighborhood in Riverbend City,
which is home to Riverbend City’s thriving Hmong community.
MAJOR THEMES:
In all the locations you visited, residents talked about various
health issues, such as diabetes and
hypertension. You have also witnessed behaviors that can
contribute to health problems, such as
poor understanding of nutrition and cultural barriers to care.
It is difficult to say exactly how or why health disparities
between ethnic and racial groups arise, but
most research in this area points to three main areas:
• Social determinants of health.
• Barriers encountered by vulnerable populations when trying to
enter into the health care delivery
system; and
• Differences in the quality of health care received by
vulnerable populations.
The effort to eradicate disparities in health and health care
among racial and ethnic groups,
socioeconomic groups, and other classifications such as urban
and rural communities is a major
feature of public health policy, as demonstrated by numerous
health promotion and disease

prevention programs such as Healthy People 2000/2010/2020.
RESOURCES:
There are no downloadable resources associated with this
mission.
Riverbend City: Community Health Mission
Welcome to Riverbend City
Riverbend City has a rich history of diversity, from its early
days as a lumber and milling center to the present day.
While many nationalities and groups are represented in
Riverbend City, the city is home to the second largest
concentration of Hmong in the United States. Within this
community, there are health and wellness issues that are
specific to the Hmong population as well as health and wellness
issues that are seen across the spectrum of the city's
inhabitants.
Instructions
This mission asks you to explore the Shoals neighborhood and
identify health concerns illustrated by the
conversations and dialogues. Go to the Shoals neighborhood,
where you may select the locations in any order. When
you have selected a location, click on the highlighted characters

to hear their conversations.
Shoals Neighborhood
During the Vietnam War, Hmong villagers were recruited by the
U.S. to fight the "Secret War." Afterwards, the
Hmong were persecuted for their anti-communist role.
Thousands of Hmong fled to refugee camps in Thailand. The
Hmong began resettling in the late 70s and Riverbend City was
a main destination. There have been subsequent
waves of immigration from the refugee camps since then.
- Soccer Field
Shoals Neighborhood - Park - Soccer Field
While watching their children playing soccer, several Hmong
parents catch up with each other and discuss various
concerns – including medical issues and frustration with the
medical system.
Kia Sia and Pa Foua chat about diet.
KIA SI MOUA:
Hey, long time, no see.
PA FOUA LEE:

Yeah, I've been in Merced visiting my parents.
KIA SI MOUA:
Didn't they feed you while you were there? You look like you've
lost weight!
PA FOUA LEE:
Ha ha! I ate like crazy, but I'm really working at eating better.
Lots of salads and stir fry and trying to stay away
from the junk.
KIA SI MOUA:
I wish I could say the same. Pheng and the kids go crazy if I
don't have snacks in the house and if that stuff is there,
I eat it.
PA FOUA LEE:
Yeah, I know. But when my cousin found out she had diabetes, I
got really scared. She's been having such a hard
time with it. The doctors tell her one thing; her family, on the
other hand, wants her to just go to a shaman. They
don't understand why she has to change everything she does.
So... I knew I weighed too much, and I thought maybe
I could do something before I get sick myself.

KIA SI MOUA:
Too bad... I guess no more nab vam for you, huh?
PA FOUA LEE:
Not as much, that's for sure.
KIA SI MOUA:
Well, I'll just take your share!
Khai asks Christina about her experience with a new medication
he has been
prescribed.
KHAI VANG:
Has Bee's team been practicing? They look a lot better today.
CHRISTINA CHA:
Yeah, Thao got some help with coaching them and they're
finally starting to understand how to work together.
KHAI VANG:
It looks like it's working. Hey, I have a question for you. Isn't
your father taking medicine for cholesterol? My
doctor says I have to start taking it, but I don't like the idea.
CHRISTINA CHA:

Don't get me started on that. He's supposed to, but he won't ever
take it. The same bottle we bought for him four
months ago is still half full.
KHAI VANG:
He doesn't like taking them?
CHRISTINA CHA:
He thinks they cost too much money, so he cuts them in half and
even then, he won't take them every day.
SHENG XIONG:
My aunt does the same thing. She says they make her legs hurt
all the time.
KHAI VANG:
What did her doctor say about that?
SHENG XIONG:
What-you think she actually told the doctor? She doesn't trust
him a bit. She tried to ask if there was Hmong
medicine she could take instead and he gave her this big lecture
about how she couldn't take any herbal medicine
anymore. She was so mad. She said he talked to her like she
didn't understand English. She hates when they do that.

CHRISTINA CHA:
They're all like that. It's terrible. My parents only go to the
American doctors if they're totally sick. I feel bad. I work
in the hospital and I know they don't mean to come across like
jerks, but most of the doctors just don't understand
anything about Hmong people and our culture. And a lot of
those doctors don't even try to learn. You can tell that
they think so much of how we do things is just dumb.
KHAI VANG:
Yeah, even with me... I mean, I speak English fine, but a lot of
times it seems like Hmong medicine doesn't exist for
them. It's just, "Do this because I said so."
Shoals Neighborhood - Market
Recent immigrants to the United States and Canada are at
increased risk for developing Type 2 diabetes, obesity,
and hypertension. Many factors contribute to this, but one of the
most significant continues to be nutrition and diet.
Gao Na and Bo discuss changing food choices.
GAO NA LOR:
Don't you know anything about Ka Poon? You don't need all
that junk to make curry noodle soup! How many
people are you trying to feed?

BO VANG:
I know! It's crazy isn't it? I just have six people to feed, but it
feels like I am cooking for a whole village. Thao's
mother likes it when I cook traditional foods. Thao and the boys
want fried everything, and Mai has decided she
won't eat meat.
GAO NA LOR:
Thao's mother is right. You should cook more traditional.
BO VANG:
Well, except for the fact that her doctor says she has that
diabetes. They told her she is not supposed to eat rice or
noodles.
GAO NA LOR:
What?
BO VANG:
Yeah, she told me that the doctor said she should stop eating
rice, no fried food, no sweets. She says it's too hard and
she just wants to eat like she did back in Laos.
GAO NA LOR:

I would be the same way. That's ridiculous... no rice? What kind
of doctor tells a Hmong person she can't eat rice?
BO VANG:
I know. I don't know why they won't just give her some
medicine to make it go away. That's what she needs.
GAO NA LOR:
Well, I've always said these doctors don't know what they're
talking about with Hmong people. They make their big
diagnosis and tell us to take their medicines, but I think they
would be happy if we all would just die. I would rather
have traditional medicine to start with. That way I know what I
am taking is the best thing for me.
Shoals Neighborhood - Funeral Home
Hmong people have a culture built on animism and the belief
that after death the soul reincarnates. Hmong funerals
usually last three days or sometimes more, and involve a large
number of family and friends. While the funeral is
one of the most important Hmong rituals, it is also a time to
connect with family and friends.
Pheng and Kao Sua discuss her grandson's recent illness.
PHENG HER:
Kao Sua. I haven't seen you in a long time. How have you been?

KAO SUA FANG:
Not good at all.
DHIA HER:
What's the matter?
KAO SUA FANG:
Oh, my son and his wife are making a big commotion in the
family and everyone is upset.
PHENG HER:
What's the problem?
KAO SUA FANG:
My grandson, Thao, he's sick all the time, but Bee and Mai
Doua have to be the Big Americans. Always at the
hospital or the clinic. We try to tell them they need to have the
shaman come and help, but they don't want anything
to do it that.
PHENG HER:
So many of the young people are like that. I think they make
things worse when they all the time go to the doctor.
They don't even try traditional medicine. Like they're ashamed
of being Hmong.

DHIA HER:
I don't know why they like the hospital so much. No one knows
what they're doing and they just ask you stupid
questions about things that don't matter.
KAO SUA FANG:
I know! I don't tell my son this because it makes them angry,
but I don't think these American doctors are good
people. My sister went to the hospital and they told her that her
heart was very sick so they wanted to cut her open.
She was scared and said "No, no!" But the doctor told her she
would die if she didn't let him fix it. But I think he did
something very bad because after the operation, she still was
very sick and weak. In less than one year she died...
just like the doctor said she would.
DHIA HER:
I'm not surprised. You don't know what those people will do to
you once they have you in that place.
Thao, Kou Lee, and Tony Cha talk about health insurance.
THAO CHA:
How has business been for you this year?

KOU LEE:
Not too bad. We moved to new offices at the beginning of the
year and we've been getting lots of customers. That's
good because costs have gotten very high, so we need all the
business we can get.
TONY CHA:
I was talking to Padou and she was very excited because she
just qualified for benefits.
THAO CHA:
You can offer your employees benefits? Can you afford to do
that?
KOU LEE:
I have to if I'm going to compete for good employees. If I don't
give paid vacation and health insurance, I just end up
training people for some other business that does give them
benefits. Padou is a good example. She has a friend who
works for a bigger company and she's always telling Padou how
good the benefits are. I know she likes working for
family, but if I didn't have health insurance available, she would
have left.
TONY CHA:
Well I know for us, we're lucky that Christina gets it from her
job.

THAO CHA:
Things are so different here. When we lived in Thailand, who
ever thought about insurance and vacation days?
KOU LEE:
Yeah, it's different here, that's for sure.
Sua Zong and Mai Doua discuss Sua's family reaction to her
son's recent illness.
SUA ZONG VUE:
Mai Doua, how have you been? I haven't seen you in so long.
How are Tong Pao and Thao?
MAI DOUA HER:
Oh, I've been crazy busy! Tong Pao is doing really well. He's
working for an investment company in town and they
really like him. After all his work getting through school, it's
nice to see it paying off.
SUA ZONG VUE:
And Thao?
MAI DOUA HER:
Well, he's doing pretty well now but this last year has been
rather hard for him.

SUA ZONG VUE:
Why? What happened?
MAI DOUA HER:
Well it turns out he has asthma, which was hard enough to deal
with, but it caused a lot of conflict with Tong Pao's
family. They're very traditional... even more than my parents...
and it really upset them that we were taking him to
the hospital instead of to a shaman. I mean, we did go to the
shaman, too, but the truth is we are both more
comfortable with Western medicine than traditional ways.
MAI DOUA HER:
Yes, I think you're right. It just can be very difficult to do what
you think is right and at the same time respect your
parents and their ideas.
SUA ZONG VUE:
I think that's true for most Hmong people our age.
SUA ZONG VUE:
But Thao is doing better now?
MAI DOUA HER:
Yes, much better. He's even on the soccer league again. Of

course he has my coordination, but all the medicine in
the world wouldn't help him with that!
Mission Complete!
You have now visited several locations within the Shoals
community and you were able to hear the residents of this
neighborhood talking about their experiences with the medical
community and with the health issues that face them
in their new home. Consider how these experiences reflect real
or potential health disparities. As you prepare your
response for the discussion question in your course room, think
about how this information would relate to the
creation of a health promotion and disease prevention (HPDP)
program.
Download Summary Document
http://media.capella.edu/CourseMedia/RiverbendCity/Missions/
_SummaryDocs/Community_Health_Mission_Summary_MSN60
12.pdf
Credits
Simulation Subject Matter Expert:
Ann Leslie Claesson, PhD, BSN
Subject Matter Expert:

Carol M. Patton, DrPH, RN, FNP-BC, CRNP, CNE; Mary A.
Bemker, DSN, PsyS, RN
Interactive Design:
Matthew Johnson, LaVonne Carlson, Chris Schons, Justin Lee,
Pat Lapinski, Mark Bune, Tara Schiller,
Marc Ashmore
Media Instructional Designer:
Felicity Pearson
Instructional Designer:
Laura Badaracco Amend
Project Manager:
Jesse Rosel, Karen Dodd, Julie Greunke
Editor:
Tom Kapocius
Image Credits:
© iStockphoto.com
Licensed under a Creative Commons Attribution 3.
http://creativecommons.org/licenses/by-nc-nd/3.0/
Program Evaluation: Making

Recommendations based on Program
Evaluations
Presented by Dr. Deb Gilbert, Capella University
Deb Gilbert
Kathie, we are back again and this time I would like to talk to
you about any kind of tips you can give our learners,
regarding, using the information from the program evaluation to
improve student achievement, whether it would be
in their classrooms or whole school, or department. Is there
anything you can tell them to look for or to aid them in
doing this?
Kathie Greer
I think one of the key things that I could say about that, when
you say that it brings to mind. A quote, I heard
recently that I really have kind of attached my brain to, and it
says, "If you want to grow, you must love truth more
than your own opinion, and what that means for me from an
evaluation stand point is that part of what you need to
do when you are looking at programs. Looking at what you are
doing as a practice is that you can not allow yourself
to be so attached to what your own perception is, of what should
work and what you enjoy doing in terms of

intervention and programs
Sometimes, we have our own little favorite things that we want
to do or things that we believe in and just as, I said
earlier, that we had to make a change in terms of what we are
doing with the test prep. I believed of course in that
taking the extra 20 minutes, because it worked extremely well
for me in another school setting, and actually it had
been replicated by some other schools, and it worked very well
for them. So, I believed much it would work here,
but I had to obviously let go of my belief and my perception
that-that was the intervention to be used here, and look
at some other solutions in that regard, because obviously for
this particular setting. It was not the best thing.
So the first thing that you have to do, is to be open and
approach the situation in an open fashion, and not be tied to
any particular train of thought when you are doing that. So you
have to first be open to other possibilities, and not
try to hold on to things that obviously are not being effective,
based on your own personal feelings or view of that
particular thing.
Do not be afraid to fine tune things also, do not be afraid to just
totally scratch something and go back to the
drawing board, because, believe me in a long run, you will save

yourself a whole lot of grief to just sometimes to
start over. I like to say "back up and punt, but to sometimes you
just need to start over, and just recreate something
or create something totally different from what you have done
before, and I had to say, just do not fear that, and a lot
of time we do. Sometimes we do so because we are afraid that
other people might think, "Oh well, they do not know
what they are doing" because you know, it just does not work.
I think that it is much more of something that people can
respect in you if you are willing to say "looked, I recognize
that these is not working, and yeah, it was my idea but it is not
working, so, we need to do something else", and you
will be much more productive by doing that.
The other thing that I would mention is that sometimes you may
have to find your own funding sources and
certainly at the school that we worked at first together, we did
know that because we did not have an endless supply
of resources. Some of the time, we have great ideas that were
not going to be funded by any district initiative or
anything or state, and so, sometimes we have to go out and find
our own funding. The after school grant was a
perfect example of that. It was an idea and something that I felt

was needed and would work but there was nobody in
the district who was willing to fund that, so we had to find our
own funding for that.
Some examples, I guess other examples would be like doing
special scheduling. From one of the things that we do
now is try to tell our master schedule to meet the specific needs
of certain students in our population. Doing bridge
programs and bringing students in at some different levels from
what they normally would be expected to come in as
a ninth grader to try and meet the students where they are, so,
those kinds of things. You have to be creative. You
have to understand that sometimes, you may have to find your
own resources to do things and you have to really
come to it with an open mind.
Credits
Expert Speaker:
Kathie Greer
Interactive Design:
Marc Ashmore
Instructional Design:
Megan Eskola

Project Management:
Jay Her Neish
Licensed under a Creative Commons Attribution 3.0 License.
Program Evaluation: Results and Outcomes
Program Evaluation Outcomes
Michelle Ellis, MA
The results of the CELA test are communicated actually through
the newspaper and the news, that is kind of public
information as to how diverse the school community is, along
with CSAT results also the language proficiency and
population of a school and their English Language Learners is
publicly available, as well. And this is something that
I think we could probably work on because many parents do not
speak English, and we mail the results home with
English, Vietnamese and Spanish letters to each family so that
they will understand their student's abilities in these
four modalities that I spoke of.
And this testing begins in kindergarten and can go all the way
through 12th grade. So conceivably a student might
be receiving ESL services for the entirety of their schooling

career, but typically students and their parents will find
out that they are creeping along and getting better and better at
these four modalities and get to see on a scale of
beginning, early intermediate, intermediate, early advanced and
advanced as to how well they are doing with their
English proficiency.
This helps families to be able to guide their students and you
need to be working, for example, on your reading
skills, because look at your writing score, or look at your
listening and speaking score, those are really doing well,
but your reading needs some work. More often than not a
student's writing score will be behind that of their other
scores, and teachers, parents and the students can have collegial
conversation to try and figure out how to best meet
the needs so that a student's writing skills can improve and be
that of a native speaker of English and a native writer
of English.
At a district level, all of the ESL teachers get together on a
regular basis, we have a monthly meeting and throughout
the year we discuss the previous year's result and the next year's
plan and evaluate between schools or feeder
schools. For example, if a middle school has particular results
and the high school does not seem to be meeting the

needs or progressing at the same level, middle school and high
school teachers will get together and talk about the
specifics that has been implemented for a particular student to
help ensure that that transition between middle and
high school is smooth and that a student does not get lost in the
shuffle.
Though I have some opinions about CELA testing and how
much time it takes away from what we do
instructionally and creatively in our classrooms, I believe that
overall this exam, along with the other exams and
assessments that are done really help us to help students be
successful.
Nathan Long, EdD
The question really arose: Were we happy with the outcome of
the total analysis? And initially, yes. I mean, if we
are looking at the program evaluation of the writing center from
one point in time, it looked really good. However,
this formative Program Evaluation approach was critical,
because in this first year we were able to see what was
working, what patterns we are developing, and it served as a
critical juncture for us in identifying new outcomes for
the program. And herein lies for us the power of program
evaluation at the collegiate level, especially in

understanding our own programs and own services, and what it
means to the development of better indicators for
growth, program improvement and data collection.
So my role in facilitating or participating in the process was to
ensure that the data was collected, that our staff and
myself met and reviewed that data, and analyzed it, and then
came up with some actions for future work. Now as we
presented our results to the stakeholders in the process,
including our faculty and students, they all seemed to nod
their head in agreement, "Yes. Absolutely. The writing center
has great value here. We are glad it is here." But we
really think that these outcomes make better sense.
And those outcomes which we particularly crafted out of that
data, focused specifically on two major components.
Better quantitative indicators needed to show satisfaction or
high satisfaction with writing center staff support. We
also needed to show a stronger correlation between writing
center activities and support and academic achievement.
And looking at what the faculty said—absolutely. Faculty want
to know that the services provided are in fact
helping students, and if we look back historically at
accreditation effort in colleges and universities, as well as K-12
and other educational entities, one of the things we are now

moving towards is a better understanding of how our
students learn, and how we can prove that. What evidence do we
have that our students are actually improving?
And so these new outcomes that we identified, we hope will
better answer those questions, so that when we are up
for accreditation, when we are working on program
improvement, we are able to demonstrate to our community,
stakeholders, our students and faculty, that this program has
value.
Things that take a little longer than expected or anticipated, and
one of the things that came out of this, is we felt we
could summatively evaluate, but we realized we were going to
need to more formatively evaluate, and stretch out
the evaluation process over the next few years. So the data was
not just collected in a one-time vacuum and
informed everything we did, but actually served as sort of a
benchmark in evaluating the work that has being done
by the writing center.
So in the sense, does it make a difference in terms of how long
things took? Absolutely. But did it make a positive
difference? Yes. Because now we are analyzing data more
holistically, we are looking at the program long term as
opposed to short term, and we are using outcomes that we
believe are more measurable and better monitored, and

evaluate the progress of our students and our writing center
staff, in terms of the support they provide.
I do not know that I would have changed anything about the
process related to Program Evaluation at this point. I
think one of the things that most program evaluators realize is
that this process really is organic, especially in
program evaluation, you are really looking at how things
emerge and evolve. Sometimes your best methodological
approach may change as you are going through analysis of the
data. You may find that you need absolutely more
data in order to make better decisions about whether a program
should stay or go, what type of improvements to
make, etcetera.
I might have changed the approach to establishing clear, more
measurable outcomes at the onset, but I think again as
a professional, I also realized that my limitations are that I do
not know everything. And that one of the things that
helped us in terms of the data we received in the program
evaluation that we initially did, was that our outcomes
were developed out of the data. We were able to really pull
those out and have better outcomes as a result of the
initial data collection.
Robert Wang, PhD

Examples of evaluations that I have been involved with:
Probably the most recent was an evaluation on a field-based
environmental science program on the Chippewa/Cree
Reservation in Northern Montana. This program basically
blended Native and Western Knowledge Systems related to
science. And with regard to the Program Evaluation, we
are really looking at mixed methods, and we are specifically
looking at student efficacy and creating an
ethnographic study of the program.
I was really lucky with that particular evaluation, I was serving
as a Principal Investigator and I was able to higher
an excellent evaluation team that pretty much looped with their
participants during the program, and had a very
strong relationship and rapport with the participants.
As far as stakeholders on this particular project, it was really
the funding agency which was the National Science
Foundation, and also the tribe. The tribe members because
obviously things don't go well, then it is more difficult to
get continuing projects like this on that reservation.
Again, my role in that particular project was as a Principal
Investigator which really meant that buck had to stop

with me. So I had a overall supervision of the entire project
including evaluation team. What that translates to is
making sure that evaluation team has all the tools that they need
to do a good evaluation. And when you are working
for example on a reservation that includes logistics, but also
making sure that they have access to for example, tribal
leaders and elders, and folks that they want to sit down and talk
with. And making sure that they knew the norms as
far as how to interact with those individuals.
I guess my general take on that also is to work with the
evaluation team, but make sure that I do not micro-manage
the evaluation team because they are to do an independent
assessment. But I am always balancing that with the fact
that time is money, and I need them, being the evaluation team,
to stay focused and organized, otherwise it is very
easy to blow a budget just on evaluation.
Alright, let us see, go with the outcome of this evaluation?
basically the methodology—the specific methodology we
did, again a mix study where they are looking at quantitative
data, basic demographic information, student test
scores, etcetera. And then they also did small interviews in
focus groups to get that in the graphic information.
Was I happy with the outcome of this? Yes, I was, and I think a
lot of that had to do with the experience of the team

doing the evaluation. I think the most meaningful information
actually came out of the qualitative portion of the
study rather than the quantitative. In this particular case that is
not always true, in that program evaluation.
How did I participate within this program? Again, I guess my
general philosophy is to make sure that the evaluation
team has all the tools they need to do their work, but not to get
too involved in it, or try to prejudice the results in
one way or the other. So once the evaluation team gets going, I
really try to make sure that I am kind of removed
from it since I am also serving as a Principal Investigators.
On this particular project, the stakeholders were pretty positive
as far as the results both folks from the tribe and the
funding agency were happy with the results. And just like many
Program Evaluations, yes, it did take more time
which equates to more dollars having to be spent. But I think
investing that money into evaluation is a real positive
thing in the sense that it ensures that project goals are met.
One thing that could have probably been done better with this
particular project is wider dissemination of results.
That is always an issue, because that usually comes toward the
tail-end of any type of problem evaluation.

Recommendations
Michelle Ellis, MA
The only improvement that I could see that would be really
beneficial is receiving the results for both CELA test and
CSAP test in a more timely manner, prior to the very end of
school or even in some cases over the summer, we are
incapable of actually getting together and making changes to the
next year in a lot of cases. So that would be the
only improvement that I could see.
Overall it has been a real pleasure of mine to get to be a part of
the English Language Department at Bear Creek
High School and over the years, starting out kind of
accidentally in ESL as a science teacher, who was just given an
ESL earth science class, my first year of teaching has grown
into a passion of mine to get to be involved with
English Language Learners and their families, and try and help
develop programs and evaluate assessments that will
be effective in not just meeting a quota or giving a test that is
required, but instead using those exams, those
assessments to effectively help tailor every program to
individual student needs.
Nathan Long, EdD

In terms of general advice, I would say, I think in Program
Evaluation one of the key components is being clear with
your stakeholders, and with the folks that you are working with
to evaluate a particular program. And going back to
that initial discussion about outcomes, see if you can establish
really strong measurable outcomes at the onset, that is
a key component in developing this out. Some other advice that
I would provide—and one piece of advice that I
have been using myself is to have outside evaluators come in.
What we are now doing at our institution, is to bring folks from
other institutions or universities to evaluate the data
that we have collected, and also to provide input. And you
might call that, in Program Evaluation, an external
auditor. That external auditor is vital to providing an outside
bird's eye view of your program and to say, hey, here is
where we think that data is pointing to, and you might want to
consider these other elements. In fact, our writing
center director has enlisted the help of a major university
writing center director. And she is bringing that person in
to review processes that we have in place currently, to see if we
can better improve in meeting our new outcomes.
So with that said, Program Evaluation has a lot of intrinsic and
extrinsic value in serving both the community, the
stakeholders that are involved in terms of your institution or

organization, and can really benefit you in terms of the
long-term goals and objectives that you set forward for your
institution.
Robert Wang
What advice can I give to folks doing evaluation? I think the
key to evaluation is balance. We need to be able to fit
the overall goals of the project. I know a lot of folks involved in
Project Evaluation that want huge percentages of
the budget, and that becomes cumbersome because no funding
agencies every going to fund a project where the
majority of the money is spent toward Program Evaluation.
So I guess one of the things my advice would be for people
doing evaluation to make sure that you have a balanced
perspective that you are sensitive to the overall project goals.
But at the same time, you hold your ground and as far
as making sure that it is quality work. Some conclusions: Just
like traditional qualitative research, many times you
have on emergent design with your research, as far as your
research approach. But at the same time when you are
doing Program Evaluation, you also have to have a clear plan on
what the ultimate goal of the evaluation is, yet need
to balance those two things.

And finally, last but not least, I think just like in business,
communication is key, so if you are involved in Program
Evaluation of your program evaluator to be in contact and
communication with the principal investigators and other
folks on the project team, I think it is pretty critical. So I guess
that is it. I guess that is in a nutshell my experience
with Program Evaluation.
Credits
Heather Miller
Interactive Design:
Marc Ashmore
Nicole Hettwer
Project Management:
Erin Coffey
Licensed under a Creative Commons Attribution 3.0 License.
Program Evaluation: Experiences

Michelle Ellis, MA
For the last three years I have been involved in testing all
English Language Learners. It is a state mandate that
anyone whose language is other than English must receive a test
called the CELA Placement Test and if they pass
that test, wonderful, they just go about their day and go about
their lives taking the mainstream curriculum.
Students who score less than perfect, less than 33 out of 33 on
this Colorado English Language Assessment
Placement Test are what is known as either Non-English
Proficient or NEP or Limited English proficient or LEP.
And a LEP student would perhaps be in an intermediate class or
especially placed with particularly trained teachers
such as English teachers, science teachers and social studies
teachers.
We also have some math teachers that have participated in a
program called ESL Leadership and these teachers have
specific training that gives them some ideas on how to work
with English Language Learners. So a LEP testing
student on this placement test, we will make sure that they are
specifically placed and we evaluate their schedule
based on this to make sure that they get the best possible
services while they are at our school.

A lower-level student would be considered NEP or a Non-
English Proficient and those can be misleading students.
Sometimes the kids have really wonderful oral skills, but not
such great reading or writing skills, because they have
not had literacy at home in this English language. It may have
been in another language but not in English. So NEP
students typically have up to two or three classes with ESL
teachers and then they are very strategically placed in a
very controlled schedule to make sure that they will be
successful and able to grasp our curriculum.
The CELA test is given in January of each school calendar year
to all students who have tested NEP or LEP and it
requires a pretty high score and a fairly fluent proficiency in
both reading and writing skills and then following in
speaking and listening skills. The overall test results will allow
our students to exit our program and then we will
monitor them for up to two years after they have exited and
tested proficient in the English language.
This gives us a real leg-up rather than just being unsure or
unclear about the language proficiency of our students.
We can be very clear about each student's ability in the four
modalities, which can vary, especially between
language group, age group and formal schooling in a student's
first language. We were able to monitor and help

students effectively in speaking, reading, writing and listening
in English.
Nathan Long, EdD
At our college I am going to use a specific example of our
writing center. It is one of the first of its kind in the
country in terms of its presence within a two-year nursing and
health sciences college. And given this important
distinction, we also had to determine how effective the actual
program, the writing center itself, was in terms of
improving student writing processes, faculty and student
support and satisfaction with the services received, as well
as overall academic achievement, if there were any correlations
to writing center services, and how well students did
in the classroom.
The people impacted by our Program Evaluation, what we call
our constituents or stakeholders, were faculty,
students and the community at large. I am just going to describe
each of those groups for you briefly. Faculty
members at our institution consisted of liberal arts and sciences
as well as nursing program faculty. All were initially
concerned about the quality of writing among students.
Transferred students, new students coming into the college,

all had a broad array of writing experiences and capabilities, but
we were finding a deep concern with our
transferred students who were struggling with finding and
stating a thesis, building an argument, and writing
satisfactory conclusions.
All students were and are considered stakeholders as well. The
specific focus for our current Program Evaluation
processes are the students who utilize the services are those
who are stakeholders. Still, we also decided to look at
those not using those services provided by the writing center as
our stakeholders as well. Lastly, community
members consisted of those who benefited from the services our
graduates provided, and of course, the graduates
themselves.
In the former category, community members would essentially
comprise healthcare employers and other nonprofit
healthcare groups. The later group consisted of graduates who
are now employed by these employers or healthcare
groups. My role and responsibility in Program Evaluation was
as the supervisor of the writing center. I basically
supervised the writing center staff, and my key role was and is
to insure programmatic effectiveness in terms of the
following criteria: improving student writing processes; faculty
and student support and satisfaction with the

services received needed to be high.
We had to show some correlation between writing center
support and academic achievement. We also wanted to
make sure expectations of faculty about writing and what
services the writing center provided were fully aligned
with their own individual expectations. An example of that is
related to APA Style. One of the things that faculty
felt a writing center should do, is to review every paper for
APA anomalies or problems.
The writing center staff said, "No," that is not what we do. As
you can tell just in that simple example, there is a
misalignment of expectations. My supervisor, while interested
in the outcome, was not really totally involved and is
not involved in the programmatic evaluations that go on at the
academic service level. What her interest level is, is
how effective is the program and should we continue
maintaining it based on cost, based on benefit to the students
and other stakeholders involved.
Now, when we are talking about Program Evaluation at the
collegiate level, especially when we are talking about
this writing center entity that we are looking at, we had to look
at various data in terms of what we collected, and
how we analyzed it. So I am going to start with ticking off a list

of particular data points that we collected and then
how we analyzed those. Every student who enters our college
takes an English placement exam, and those scores
range from 1 to 15. In those English placement exam scores, we
are able to categorize students by highly proficient,
proficient, and not proficient.
Based on their score, they will either have to enroll in a full-out
remediation course in English, which is a
developmental English course. And in order to transfer credit or
move into English 101, they have to successfully
complete the developmental course. The next option or step in
the EP scoring process is that the students who fall in
the proficient category may still need some remediation related
to their writing skills and process, but that may not
entail enrolling in a developmental course. The writing center
spearheads a tutorial program for those specific
students in terms of how they are identified, what sort of work
they need, and how they can support them in
improving their writing processes.
The highly proficient group is simply identified, as their course
credits will transfer or they will continue on into
English 101. So that is our initial data, and we get all of that
information about their basic writing abilities at the

very start of their work here at the college. We also collected
other basic quantitative data, how many students come
to the writing center, how often, and for what purposes. We
want to find out, are students who are utilizing the
centers struggling in writing, how many are struggling in that
area of writing, writing process, how often are they
meeting with the writing counselor and improving their own
writing processes, and what purposes are they
specifically coming for? Are they coming at the end of a term to
get specific help on a final paper, are they coming
to just simply improve their own writing, and approaches to
writing and communication?
Qualitative indicators of student satisfaction included such
things as open ended questionnaires, as well as basic
anecdotals. These anecdotals range from students talking about
their satisfaction with the writing center staff, to
faculty talking about how well the staff seems to serve the
needs of the students related to writing processes,
support, etcetera.
We also collected qualitative and quantitative survey feedback
from faculty and staff, really looking at the quality of
the program and the level of support provided to students. These
surveys went out to all faculty and staff to try and

find out: Is the writing center meeting the needs of the
institution, is it meeting your understanding of the
expectations of what a writing center does, what kinds of
improvement are you seeing? That type of thing. So we
got the data, and we analyzed that, we put it all together and
discussed it as a collective.
Robert Wang, PhD
Primarily I am going to talk in terms of being a grant recipient
and managing grants. Program evaluation is required.
It is usually specified in fact in the RFP of these grants, as far
as the type of program evaluation that is required.
Factual impacts on these projects include both logistics and also
dollars, as far as having—cannot make sure we
have an adequate budget for program evaluation.
One of my primary responsibilities is to manage the evaluation
team, but not get in the way of evaluation. In other
words make sure there is a separation between evaluation and
the rest of the project, but make sure that the logistics
and everything is in place for the project or the evaluation team
to be successful.
But to bring it up to the one critical issue that I see with
program evaluation as far as these grants, and that is to
make sure that you are hiring experienced evaluators, folks that

really know evaluation. It is all those things that
they do not teach you in evaluation or program evaluation
courses such as, cultural sensitivity, things like that,
which I really think you have to pretty much learn on the job, In
other words, if you are planning and going into any
type of project evaluation really, almost finding a mentor or
doing some type of apprenticeship or internship with
people that have a thorough understanding of project evaluation.
Credits
Heather Miller
Interactive Design:
Marc Ashmore
Nicole Hettwer
Project Management:
Erin Coffey
Licensed under a Creative Commons Attribution 3.0

Program Evaluations: Introductions and
Definitions
Michelle Ellis, MA
My name is Michelle Ellis and I am a teacher and Department
Chair of the Bear Creek High School, English is a
Second Language Department. I have been teaching for 10
years, first of all teaching biology for five-and-a-half
years and most recently I have been the Department Chair in
ESL for the last four.
My official title is ESL Teacher and Department Chair at Bear
Creek and I am the district representative for my
school along with my colleague Mike Flores. We both serve as
liaisons between the district multicultural student
services and our school Bear Creek, and work closely with the
district folks as well as all of the teachers at Bear
Creek High School, especially content area teachers who are
affected by having our ESL students in their classes.
The number one part of my job as far as evaluation, is to see
how well our curriculum aligns with the Curriculum
Alignment Project, which is Jefferson County Public schools'
attempt to be Colorado standards-based and evaluate
how well our ELL, English Language Learners, are performing
and assimilating with the rest of the student

population based on the CSAP and CELA test scores, and CSAT
being the Colorado State Assessment Program that
all ninth- and 10th-graders are required to take as well as the
CELA test which is the Colorado English Language
Assessment.
Both of these tests are taken by our ELLs just to see if they are
indeed learning the English language and becoming
academically proficient with the rest of the student body. As far
as my evaluation role, evaluating our program, we
take daily informal assessments. Mike and I both as teachers,
are constantly doing on-going assessments to make
sure that our students are learning the English language, as well
as after the students take the CELA test and the
CSAT test, we look at their results and compare them with the
mainstream body of ninth, 10th, 11th and 12th
graders to see how they are growing in the English language and
we make changes to our programs based on these
results that we see.
All of my decisions on what I am going to teach are informed
by the data that I look at. Our school is very data-
driven and we use a lot of percentages and cross-curricular data
comparisons to make sure that every content area is
assessing based on standards and also assessing together.

We have a really cool program at Bear Creek High school as
well as in many other Jefferson County High Schools
called collaborative content team. Richter Thor (ph) was sort of
the father of this idea, and the whole program is
based on difference, discipline, meeting together on a weekly
basis preferably to make sure that all of the
assessments and instructions are aligned, so that hopefully all
teachers are teaching the same material to all kids.
And I am proud to say that over the last four years all of my
English Language Learners have been participating in
the Bear Creek High School English program, doing all of the
same assessments and learning all of the same
material that all of our incoming ninth graders are required to
learn, and that is a big step for someone from another
country, English is not their first language. So it is pretty
amazing that they are able to access the same material, it
just has to scaffolded or taught in a little bit different way.
Nathan Long, EdD
My name is Nathan Long. I am an Academic Dean for a small
college in the Midwest here in Cincinnati, Ohio. I am
an academic dean for general education and instructional
support services. My main roles at the institution include

serving as a chief academic officer, the chief information
officer—specifically for technology, supervision of liberal
arts and sciences faculty, as well as academic services support
staff.
I got involved in Program Evaluation by the nature of my job,
by virtue of my role in assessment of student learning
and instructional as well as technological services programs.
Program Evaluation was just a natural outgrowth of
what I do day-to-day, week-to-week, month-to-month, and year-
to-year. Currently, we are engaged in a number of
Program Evaluation activities rooted in assessment of student
learning and programmatic effectiveness.
Robert Wang, PhD
My name is Rob Wang, and I am serving as an Associate
Professor of Science at Colorado Mountain College, in
Glenwood Springs, Colorado, which is located about midway
between Aspen and Vail. As far as my role within
Colorado Mountain College, my primary role is teaching. I
teach Environmental Science, and Outdoor Education.
I am also involved in program development specifically the
development, I have been in Environmental Science and
Environmental Studies Program. And probably, the most
important thing, or relevant thing as far as, Program
Evaluation I also serve as a principal investigator for projects

related to Native American and Alaska Native Science
Education.
This work basically involves tribes from Alaska and then down
through the Rocky Mountain Corridor from
Montana to Arizona. Primarily, it is funded through the
National Science Foundation, but many of these projects are
also funded through organizations such as BIA and private
foundations such as the Gates foundation.
As far as how I initially was exposed to or got involved with
Program Evaluation, initially, I learned about it as a
grad student, when I was a doctoral student. And I always told
myself I will never use this stuff, and it turns out that
it actually—it is very helpful to set a foundation for the work
that I would do later on with grants.
As far as grant work in Program Evaluation, almost all the
projects I am involved with emphasize a strong
evaluation component that is required by the grant, usually this
is throughout the life of the projects both the
formative and summative assessments have to be completed.
This helps to assess project goals, fine-tune the project,
and ultimately disseminate results whether or not the project
goals were met.
Credits

Heather Miller
Interactive Design:
Marc Ashmore
Nicole Hettwer
Project Management:
Erin Coffey
Licensed under a Creative Commons Attribution 3.0 License

Recurring Steps in the Process Scoring Guide Due Da.docx

Recommended

Recommended

More Related Content

Similar to Recurring Steps in the Process Scoring Guide Due Da.docx

Similar to Recurring Steps in the Process Scoring Guide Due Da.docx (20)

More from aryan532920

More from aryan532920 (20)

Recently uploaded

Recently uploaded (20)

Recurring Steps in the Process Scoring Guide Due Da.docx