The document provides guidelines for faculty on promoting best practices for assessing student learning in online courses. It discusses the importance of alignment between learning objectives, assessments, and program competencies. It also outlines steps for assessment planning, collecting and analyzing assessment data, using rubrics and checklists to interpret results, and implementing continuous improvements based on evaluation feedback. The goal is to help faculty design effective online assessments that are learner-focused and promote competency development.

1. Guidelines to Promote Best
Practices, Advice and Information to
an Online Faculty Audience
HE527 Final Project
HE527: Assessing Student Learning
Online
Sharifah Ali

2. Introduction
 Below is a set of guidelines for an online faculty audience. With increased
demand for online learning as well as more institutions of higher learning
striving to provide diverse educational opportunities, online learning
continues to grow as a viable means of providing increased access to a
greater number of students (Keengwe & Kidd, 2010).
 No matter how strong the business case is for creating an online education
program, every college should be ready for faculty apprehension, or even
resistance. Just 31 percent of chief academic officers who responded to the
Babson/College Board survey believe their faculty members accept the
value and legitimacy of online education (Blackboard, 2010). Certain
characteristics found within some faculty cultures can act to impede the
development of an effective assessment system. Some faculty members
are indifferent about the need to have their course outcomes evaluated in
new ways. Some may feel satisfied that the quality of the department’s
program is adequate and that no new assessment system is needed. On the
other hand, some faculty characteristics supportive of a new assessment
system are valuing quality, empowered faculty, desire for certainty, and
respect for industry involvement (Tener, 1999).

3. Introduction
 If program assessment is ever to be taken seriously and used effectively, it
must be systematic and built into the very structure of the program or
department by the faculty themselves (Gigliotti, 2009). The use of program
assessment helps ensure that regular program reviews occur, keeps the focus
on outcomes and competency development, and keeps the process learner
focused. It is a comprehensive means by which courses and programs can be
evaluated on an ongoing basis. The results can be analyzed to determine
areas of the curriculum that are working well, need improvement, or might
be missing (Palloff & Pratt, 2009).

4. Development of Learning Objectives
 Given that a well-designed online course should be learner focused and
centered, it follows that student assessment within that course should be the
same. The reflective process that should be included in an online course
provides the basis for learner-centered assessment. Students should be given
credit for self-reflection, and it should be incorporated into the design and
expectations for the online course. Each collaborative activity should
contain a reflective component. At the very least, students should be asked
to reflect on their participation in the activity and their contributions to the
group (Palloff & Pratt, 2009).

5. Alignment
 Edvantia (2005) defines alignment as “the degree to which expectations
[standards] and assessments are in agreement and serve in conjunction
with one another to guide the system towards students learning what they
are expected to know and do” (p. 3). ”There need to be strong links among
assessment of student performance at the course outcome level, the
competencies developed for the online program of study, and the mission
of the university to create alignment.
 Curriculum alignment includes alignment between and among several
education variables, including state standards, state-mandated
assessments, resources such as textbooks, content of instruction, and
instructional strategies. The extent to which textbooks are aligned with
standards and assessments is important due to the widespread use of
textbooks to guide instruction (Edvantia, 2005). Given that a well
designed online course should be learner focused and centered, it follows
that student assessment within that course should be the same. When
instructors use multiple measures and authentic assessments are based in
real life and not just classroom learning, there is more likelihood that a
true measurement of student competency and performance will result
(Palloff & Pratt, 2009).

6. Alignment
 The process of instructional alignment is labeled as the process between
instruction and assessments. Students can demonstrate mastery, when
instruction is designed to ensure that students perform well on competency
tests (Edvantia, 2005).

7. Assessment Planning
 Before an instructor can embark on the development of good online
activities and assessments, they must have a solid understanding of how
assessment fits into the scheme of course development as well as the
components it comprises. Instructors need to understand learning
outcomes, their importance in the learning process, their development, and
how to achieve them. The components that make up course design are
often referred to as competencies, outcomes, and objectives. There is
different meaning for each term and assessment for each term differs as
well.
 In a learner-centered teaching design, students construct knowledge by
gathering together and synthesizing information by using inquiry,
communication, critical thinking, and problem solving. The instructor’s
role is to facilitate the process, and instructors and students together assess
learning. By this method, teaching and learning are intertwined and the
results are best assessed through papers, projects, performance of
authentic application activities, and portfolios (Palloff & Pratt, 2009).

8. Assessment Planning
 Online assessment is a very contemporary phenomenon that uses the most
current electronic technologies. It involves instructors and instructional
designers who are innovative and who wish to go beyond conventional
approaches to instruction and learning (Oosterhof, Conrad, & Ely, 2008). In
order to conduct effective assessments at the course and program level,
faculty also need be trained on how to create effective assessments by
simply designing their own courses. Many instructors do not write or design
the courses they teach (Palloff & Pratt, 2009). If a faculty member has
created their own course, then an assessment that demonstrates the
instructor’s own objective to test their student’s knowledge will be more
effective.

9. Collect Assessment Data
 Providing feedback directly after an assessment makes it more likely that
students will recall the logic they used when answering items and will take
advantage of that information when reading the feedback.
 The feedback provided at the end of an assessment should match as
closely as possible the reasoning students used during the assessment.
Instead of creating feedback tied to how the student responds, it is better
to prepare just one feedback message for each item that explains the
general logic of the answer and, in the case of multiple- choice items, or
fixed-response items, why the wrong responses are incorrect. This allows
the student to evaluate the logic he or she used to answer the item
(Oosterhof et al., 2008).

10. Collect Assessment Data
 With today’s information technology it is possible to improve both the
construction of tests and the quality of feedback while saving instructor
time. Immediate performance feedback positively affects students, and a
fast and accurate grading process should enhance student learning. By using
information technology to automate the preparation of exam questions,
answer keys, and student feedback, student learning is enhanced by
providing typo-free exam modifications, faster and more accurate grading,
and detailed written explanations for all exam questions. The instructor does
not have to allocate a significant amount of time for exam and answer key
preparation, having to come up with different stories and numbers for each
question version, and making sure that each version is equally challenging
(Ozkul, 2009).

11. Collect Assessment Data
 Consistency in scoring is essential with all item formats. To facilitate
reliable scoring, the scoring plan should incorporate three characteristics in
constructed-response items; the total number of points assigned to the item
based on its importance relative to other items, specific attributes to be
evaluated in students’ responses, and for each attribute, criteria for
rewarding points, including partial credit. Fixed –response assessment
format lends itself to an online administration. Responses can be scored
immediately as each student completes the test, and the student’s records
can be updated automatically without the instructor’s involvement
(Oosterhof et al., 2008).

12. Using Results
 Rubrics have become a useful tool in the assessment of student work online.
They provide clarity in expectations for student performance and assist in
turning subjective assessments of assignments, into more objective
measures of student work (Palloff & Pratt, 2009). Providing examples of
superior work help amplify instructions to students and provide information
about the scoring plan and help to set standards. Checklists are another
option to interpret assessment data. A checklist can structure observations of
a learner in a performance assessment. Checklists can be used in a variety of
settings to establish the presence or absence of a series of conditions. There
are also four references that are widely used for interpreting the
performance of learners. The four references are; ability-referenced, growth-
referenced, norm-referenced, and criterion-referenced. Of these, the norm-
referenced and criterion-referenced interpretations tend to be the most
useful (Oosterhof et al., 2008).

13. Continuous Improvement
 External assessors and their roles vary widely, but they are often most
valued for providing oversight on the complete assessment process. They
comment on the exam questions, they provide feedback to the teachers and
their institution on links between teaching and assessment, they interview
the students and the teachers, and they compare processes and outcomes in
one institution with those in others. External assessors provides impartial
advice to institutions to ensure that students are treated fairly, to enable
teaching teams to understand their standards better, and to improve the
quality of the education which they provide. Evaluation can be achieved on
a group-wide basis and the tools and techniques involve the use of surveys,
questionnaires and semantic differential techniques (Shephard, 2009).

14. References
 Keengwe, J., & Kidd, T. T. (2010). Towards best practices in online
learning and teaching in higher education. Merlot Journal of Online
Learning and Teaching, 6(2), 1-9. Retrieved from
http://jolt.merlot.org/vol6no2/keengwe_0610.pdf
 Oosterhof, A., Conrad, R.M., & Ely, D. (2008). Assessing Learners Online.
New Jersey: Pearson Merrill Prentice Hall
 Ozkul, A. (2009). Using information technology to enhance assessment of
learning: Automating preparation of course exam materials and student
feedback. Information Technology and Performance Journal, 25(1).
Retrieved from http://www.osra.org/itlpj/ozkulspring2009.pdf
 Palloff, R.M., & Pratt, K. (2009). Assessing the Online Learner: Resources
and Strategies for Faculty. San Francisco: Jossey-Bass.
 Shephard, K. (2009). E is for exploration: Assessing hard-to-measure
learning outcomes. British Journal of Educational Technology, 40(2), 386-
398. Retrieved from
http://ehis.ebscohost.com.lib.kaplan.edu/ehost/pdfviewer/pdfviewer?sid
=50f5205a-f447-4546-8e7d-120de776f5ae%40sessionmgr111&vid=6&hid=120

15. References
 Tener, R. K. (1999, January). Outcomes assessment and the faculty
culture: Conflict or congruence. Jornal of Engineering Education.
Retrieved from http://www.jee.org/1999/january/614.pdf
 Educational assessment: designing a system for a more meaningful
approach. (2011, June). Academic Leader. Retrieved from
http://www.facultyfocus.com/free-reports/educational-assessment-
designing-a-system-for-more-meaningful-results/
 Smart Steps to Online Learning. (2012). In Blackboard Career
Colleges. Retrieved from
http://www.blackboard.com/CMSPages/GetFile.aspx?
guid=b4d4d142-7530-4290-a2c2-510a82b7f127
 U.S. Department of Education, Institute of Educational Sciences,
Edvantia. (2006). Aligned curriculum and student achievement
(contract number EDO-01-CO-0016). Retrieved from
http://www.edvantia.org/pdta/pdf/Aligned.pdf