hybrid_learning_program_evaluation

1
Executive summary
Evaluation of UNLV’s Hybrid Learning Program
January 10, 2012
Michael Wilder,
University of Nevada, Las Vegas
Purpose
Hybrid (or blended) learning is an instruction mode that provides some combination of
face-to-face and online instruction. To what degree instruction is divided between these
two modes is often determined by each institution. Thoughtful consideration must be
made regarding how this method of instruction is developed and delivered. Instructors
and support staff need to be aware of institutional policies and procedures in order to be
effective. This program evaluation examines the hybrid learning program at the
University of Nevada, Las Vegas, in light of these issues.
Program evaluation questions
1. What is the current status of hybrid learning at UNLV?
2. What are some defining characteristics of hybrid learning taken by similar institutions
of higher learning?
3. Do members of the UNLV community (faculty and staff) understand the defining
characteristics of hybrid learning?
4. What are the perceived advantages/disadvantages of hybrid learning at UNLV?
5. What are some of the "lessons learned" by faculty that have taught using the hybrid
learning instruction mode?
Data sources
1. University records
Board of Regent and UNLV Office of the Provost policies and guidelines were
examined. Student information system and registration system data also provided
course statistics. Actual course syllabi were examined to determine amount of class
time dedicated to online versus face-to-face instruction. Lastly, various documents
prepared by the Office of Online Education (handouts, presentations, and white
papers) provided essential definitions, pedagogical models, and empirical data
regarding hybrid learning at UNLV.
2. Interviews with UNLV academic professionals
In order to obtain direct feedback regarding UNLV hybrid learning efforts, ten
veteran instructors using this instruction mode were interviewed.
3. Surveys of Non-UNLV academic professionals
Perspectives from ten non-UNLV instructors, administration, and other academic
professionals were gathered via online survey.

2
4. Academic journals and publications
A preliminary review of literature was conducted in order to gain insight from
contemporary academic publications on the subject of hybrid learning,
Findings
1. Between Fall 2010 and Fall 2012, UNLV offered 100 hybrid courses. The number of
hybrid courses being offered annually is increasing.
2. The majority of instructors teaching using the hybrid learning model at UNLV have
misconceptions regarding the working definition of this instruction mode.
3. Most instructors using the hybrid learning instruction mode feel that they have not
received adequate training or preparation.
4. There exists ambiguity and lack of consensus regarding the official characteristics
and definition of different instruction modes at the NSHE and institution level.
Top recommendations
1. Establish clear and unambiguous definitions for different instruction modes
Although definitions vary widely from institution to institution, working definitions of
the various instruction modes should be developed that are clear and unambiguous.
Once identified, instructors and staff should be trained regarding the differences
between these instructional mode designations.
2. Create procedures for development that follow empirical best practices
Courses that are created on the fly, under a short timeframe, or without the full
degree of input between all members of the development team will not rise to the
level of quality required by a first-rate educational institution. Course quality
standards (such as those established by Quality Matters) may be useful. University
procedures may need to be developed and communicated, and these policies need
to be applied consistently.
3. Provide a certificate course in hybrid learning best practices for instructors
Ideally, faculty would participate in an extensive hybrid course that models and
demonstrates best practices and pedagogical approaches appropriate for this
instructional mode. This program could be an essential prerequisite to teaching
hybrid courses at UNLV.
4. Offer ongoing professional development for instructors
In addition to a certificate course, instructors will need continuous professional
development in the form of face-to-face presentations, webinars, tutorial videos,
handouts, job aids, and peer sharing opportunities.

3
Table of contents
Introduction.......................................................................................................................4

Program evaluation questions..........................................................................................8

Evaluator qualifications/expertise.....................................................................................8

Program............................................................................................................................9

Method/data collection ...................................................................................................11

Findings..........................................................................................................................13

Discussion ......................................................................................................................26

Recommendations .........................................................................................................28

References .....................................................................................................................30

Appendices.....................................................................................................................31

4
Introduction
While the conceptual approach of the blended learning model has been present
since the early days of instructional design, the term itself was not coined until the late
nineties (Clark, 2010). Generically speaking, blended (or hybrid) learning involves
combining some measure of face-to-face and distance education. Exact definitions,
however, are elusive and certainly not universally agreed upon. According to Allan and
Seaman (2011), hybrid learning "has between 30 and 80 percent of the course content
delivered online." In a online survey conducted for this evaluation (2012), members of
Penn State’s Institute for Emerging Leadership in Online Learning (consisting of online
learning professionals around the world), defined the hybrid learning mode as “50/50,”
“at least 50% online,” and “25% to 79% online.” In a recent (2012) discussion forum
hosted by the WICHE Cooperative for Educational Technologies (WCET), higher
education professionals defined hybrid or blended learning as "26% to 79% online,"
"30% to 49% in person," and "a course in which a majority--more than 50% but less
than 85%--of the planned instruction occurs when the students and instructors are not in
the same place."
In the absence of a universal formula, working definitions must be established at
each individual institution. While some institutions prefer the term “blended,” UNLV
uses the term “hybrid” to remain consistent with NSHE Board of Regents terminology.
According to a Vice President & Provost policy memorandum (2011), “Hybrid classes

5
must have the majority of instruction delivered at a distance and the face-to-face
component, although reduced, is mandatory and scheduled.”
At UNLV, there are currently no specific requirements as to the mix of classroom and
online resources and activities, although there are typical patterns of use. For example,
an instructor may use class time for lecturing, conducting class discussions of the
material, and assembling work groups as they begin their projects for the semester and
at milestones along the way. The online portion may host course documents, lecture
notes, specific learning activities applying course content, self-tests, unit quizzes, and
discussion boards for use by established work groups.
The effort to take a hybrid approach to teaching and learning at UNLV is still in initial
stages. Many questions need to be answered before informed decisions can be made
and official policies can be established. This goal of this program evaluation is to
assemble essential background information so that such decisions, policies, and
procedures may be made.
About the client
This program evaluation was developed for the UNLV Office of Online Education
(OE). Over 300 fully online or hybrid courses are typically offered at UNLV during the
Fall and Spring semesters each. Online courses are offered in 48 areas including
Hotel Management, Tourism & Convention Administration, Nursing, and Education. In
addition, UNLV offers eight degree programs and two certificate programs.
According to the UNLV Web site:
“UNLV, founded in 1957, is an institution of approximately 27,000 students and
nearly 2,900 faculty and staff located on the southern tip of Nevada minutes from

6
the Las Vegas Strip. Classified by the Carnegie Foundation for the Advancement
of Teaching as a research university with high research activity, UNLV offers
more than 200 undergraduate, graduate and doctoral degree programs including
innovative academic degrees in such fields as gaming management,
entrepreneurship, entertainment engineering and much more. The entertainment
capital of the world, Las Vegas offers students a “living laboratory” for research,
internships, and a wide variety of job opportunities. UNLV is dedicated to
developing and supporting the human capital, regional infrastructure, and
economic diversification that Nevada needs for a sustainable future.”
Among other responsibilities, OE supports the development of quality online courses
through instructional design, graphic design, videography, and instructional
programming. According to the OE Web site:
“The Office of Online Education extends educational outreach and access of
University courses, degree programs, and research opportunities to the local and
global communities in an effort to serve a diverse population of students who
prefer or need alternative methods of delivery. OE is committed to learner needs
and interests by providing exemplary services to the University academic
community. As an entrepreneurial and innovative learning technology field, OE is
committed to collaborating on research initiatives, faculty development, active
learning, measurable outcomes, and using empirically proven pedagogies with
emerging technologies to increase the opportunity for successful learning
outcomes of UNLV students.”
OE also provides professional development (in the form of online courses, webinars,
face-to-face presentations, and handouts), consultation (both over the phone, through
e-mail, and in person) and educational content development (such as learning
management system templates, course organization assistance, video production, and
learning interactive creation).
Needs analysis
In a report summarizing the UNLV data collected during the Spring 2009
administration of the National Survey of Student Engagement (NSSE), a number of
interesting demographics were obtained. This survey compared UNLV data to peers at

7
other national institutions as well as compared data longitudinally with the UNLV Spring
2006 administration of the NSSE. This survey was designed to have students report
how often they participate in activities that represent good educational practice, their
perceptions of the college environment, and how students estimate their own
educational and personal growth (Gittens, 2009).
While this survey provided a wealth of demographic information about UNLV
students, the following facts are relevant to this needs analysis. UNLV students spend
more time than their peers working off campus, providing care for dependents, and
commuting to class. UNLV students spend less time than their peers working with
students outside of class to prepare class assignments, preparing for class, e-mailing
faculty (freshmen only), and reading assigned books, articles, etc. (freshmen only)
The hybrid instruction mode is about access, not convenience. Nevertheless, the
data gathered by the NSSE survey provides support for the idea that the hybrid learning
delivery mode may benefit UNLV students. Since many students must work off campus
to pay for their education and to provide care for dependents, they don’t have as much
time available to come to campus or prepare for. As a result of being off campus,
commuting, and possibly working, many students don’t have as much time to be
physically on campus as their residential peers.

8
Program evaluation questions
This program evaluation focused on examining five major questions related to hybrid
learning at the University of Nevada, Las Vegas. Following are the guiding questions
that guided this program evaluation:
1. What is the current status of hybrid learning at UNLV?
2. What are some defining characteristics of hybrid learning taken by similar
institutions of higher learning?
3. Do members of the UNLV community (faculty and staff) understand the defining
characteristics of hybrid learning?
4. What are the perceived advantages/disadvantages of hybrid learning at UNLV?
5. What are some of the "lessons learned" by faculty that have taught using the
hybrid learning instruction mode?
Evaluator qualifications/expertise
In addition to twenty-four years as an educator (eight years teaching in higher
education) and four years as Learning Technologies Specialist for the UNLV Teaching
and Learning Center, Michael Wilder is currently the Instructional Design Coordinator
for the UNLV Office of Education. Destiny has led Michael on a dual career path of both
education and technology. The result is a hybrid individual with skills and perspective
from both fields. He has worked as teacher, trainer, and Web professional in a variety of
environments, serving, among other positions, as Blackboard administrator, Online
Learning Specialist, and college faculty (Web design, Web-based multimedia,
networking, programming, desktop publishing, essential computer applications, and

9
much more).
Michael specializes in Web-based education and the innovative use of emerging
technologies. He has presented at many major conferences, and has consulted to
schools districts and colleges. Michael received his B.A. in English from UCLA, and
received his master's degree in Educational Leadership, with emphasis on computer-
based education, from Gonzaga University. Michael is currently completing his doctoral
program in Educational Psychology with a focus on learning and technology.
Program
History
Online education, hybrid or otherwise, is still maturing at UNLV. In 1995, the
distance education program was part of the Office of Information Technology, primarily
concerned with providing financial incentives to instructors for development of fully
online courses. Few support services were available for providing instructional or
graphic design, multimedia creation, or professional development. In most cases,
anyone with a desire to teach online was provided the opportunity regardless of
qualifications, training or previous experience (including part-time instructors and
graduate assistants). No hybrid courses were offered at this time.
The online education program at UNLV has come a long way since 1995. Today the
Office of Online Education provides a full range of support to academic units for a
variety of instructional delivery modes including fully Web-based, hybrid, and Web-
managed courses. This office provides financial incentive to departments for subject-
matter expertise, curriculum development collaboration, and participation in specific

10
professional development programs. Subject matter experts also receive support and
consulting from a professional team of instructional designers, instructional artists,
instructional applications programmers, and videographers. In 2010, only twenty-one
courses were delivered using the hybrid model. In 2012, forty-nine courses were
offered using the hybrid learning instruction mode. Clearly this method of course
delivery is gaining popularity at UNLV. It wasn’t until 2011, however, that handouts and
professional development presentations defining hybrid learning were available to
instructors.

11
Method/data collection
An extensive variety of data sources have been analyzed in order to triangulate
findings regarding hybrid programs both at UNLV and other educational institutions.
These methods included gathering quantitative data from existing courses and records
as well as gathering qualitative data from educators, administrators, and development
professionals. Following is a list of data sources and methods used for this evaluation:
University records
Several documents prepared for faculty by the UNLV Office of Online Education
(handouts, presentations, and white papers) provide essential definition, pedagogical
models, and empirical data regarding hybrid learning at UNLV. In addition,
correspondence between statewide distance education administration and Nevada
System of Higher Education (NSHE) officials provide further insight into Nevada and
UNLV standards and processes. Furthermore, OE maintains an online database of
courses that includes such information as course title, term, delivery mode, department,
and instructor. From this database, it was possible to identify number of hybrid courses
delivered, leading departments using this delivery mode, and number of enrollments.
Lastly, syllabi from hybrid courses taught at UNLV were analyzed in order to determine
time allocations (in-class or online).
UNLV academic professionals
In order to obtain direct feedback regarding the status and understanding of the
UNLV hybrid learning efforts, numerous educators and academic support staff were

12
interviewed. Ten UNLV instructors with direct experience teaching hybrid courses were
interviewed (see the appendices for the interview questions). In addition, four
instructional designers for the Office of Online Education were also interviewed. These
interviews yielded perspectives on such topics as lessons learned, accountability, and
educational challenges regarding this delivery mode.
Non-UNLV academic professionals
In order to obtain perspectives from outside the institution, instructors,
administration, and other academic professionals from non-UNLV institutions
participated in an online survey that included questions regarding definition, models,
and pedagogical approaches. For example, members of the Professional and
Organizational Development Network in Higher Education (POD) and members of the
Institute for Emerging Leadership in Online Learning (IELOL) were invited to participate
in an online survey (see appendices for survey questions). Furthermore, conversations
regarding hybrid learning definitions were mined from the WICHE Cooperative for
Educational Technologies (WCET) listserv.
Academic journals and publications
In order to gain insight from contemporary academic publications on the subject of
hybrid learning, a review of literature was conducted.

13
Findings
Current status
Between Fall 2010 and Fall 2012, UNLV offered 100 hybrid courses. Of those
courses, only two were at the graduate level. By far, the English department offered the
most hybrid courses in this time period (39). Nursing courses offered the second most
with 13, followed by Clinical Laboratory Sciences with ten courses.
0

5

10

15

20

25

30

35

40

45

ENG
NURS
CLS
CED
KIN
NUTR
GAM
RLS
TCA
HED
MKT
DAN
ULD

Hybrid
courses
by
department

Fall
2010
-‐
Fall
2012

14
While a comparison of annual totals indicates an increase in popularity between Fall
2010 and Fall 2012, this increase has not been drastic. Hybrid offerings are generally
very highest in the fall and lowest in the summer.
Perceived definition of hybrid learning
Many UNLV faculty members are unclear about the definition of hybrid learning at
UNLV. Interviews with ten veteran instructors teaching hybrid courses indicated an
inconsistent understanding of the current definition of hybrid learning at the University.
Only one instructor was able to express the current definition, that the majority of
instruction should take place online. Three instructors stated that half of instruction
should be delivered online while the other half took place in person using the hybrid
learning model. Two other instructors believed that hybrid learning was defined as
0

5

10

15

20

25

30

Fall
2010
Spring
2011
Summer

2011

Fall
2011
Spring
2012
Summer

2012

Fall
2012

Hybrid
courses
by
year

Fall
2010
-‐
Fall
2012

15
twenty percent of instruction delivered in the classroom while eighty percent was
delivered online. One instructor believed that hybrid learning was defined as forty
percent in person and sixty percent online. Four instructors were unable to provide a
time allocation. For example, one instructor with five years of experience teaching
hybrid courses said, "I don't know of a mandated percentage." Another instructor
commented, "early on there was a specific percentage that was mentioned, but the
guidelines have been a little loose, and I always seem to get a different answer about
what is required." An instructor with one semester teaching hybrid courses stated, "I
thought it was not well defined. I watched the video provided by Distance Ed. It was
generated by a third party that was more of a 'salespitch' than information on how to
define and development a meaningful hybrid experience for the students." Another
instructor with five years of experience teaching hybrid courses said, " I wouldn't try to
put a percentage on the learning because I don't know how much time my students
invest in reading the materials and engaging the course content."
In addition, there is a strong possibility that many face-to-face instructors and
support staff are confused about instructional mode definitions. During the Fall 2012
semester, for example, at least one instructor told students that her course was hybrid
although the registrar’s official designation was in-person delivery. In this fifteen-week
course, six weeks of instruction were taught fully online. In another example during this
same semester, a new administrative assistant designated courses as "Web-managed"
in the student information system when the appropriate designation was hybrid. When

16
asked why the course was designated inappropriately, the administrative assistant
explained that she was unaware of the characteristics of "hi-bread" courses.
Why is this a problem?
There are two major reasons why misperception regarding delivery mode is a
concern. The first reason is pedagogical, and the second is financial.
The way in which a course is organized and facilitated in a face-to-face course may
be very different from the way in which a course is organized and delivered in a fully
online course. The organizational and pedagogical approach to teaching a hybrid
course also requires such special considerations. For example, simply because
instruction time in a face-to-face course may be spent on lecturing does not necessarily
mean that in-person time in a hybrid course should be spent simply lecturing. Teaching
methods and course organization between the three instruction modes can be very
different. Educators and instructional design teams need to be aware of the defining
instruction mode characteristics in order to develop curriculum that is appropriate for
that mode.
Financially, the way in which a course is designated in the UNLV student information
system determines whether additional student fees are assessed. If a course is
designated either hybrid, Web managed, or fully online, for example, students are
assessed an additional thirty-four dollars per credit hour to help defray the cost of
technology and support. If fees are assessed for a course that does not use these
services, then students are being unfairly charged. Conversely, if courses are being

17
offered that use online education resources, but not assessing appropriate fees, then
the University is not being compensated appropriately.
Instructional mode definitions
If faculty are confused about UNLV's definition of hybrid learning, it may be as a
result of unclear definitions at a policy level. Although the University has been offering
online courses since 1995, the exact definitions of the various instructional modes are
still being developed. In 2011, a proposed set of descriptions were prepared by a
statewide consortium of institutional distance education directors, but these descriptions
were never finalized at either the state level nor at the individual institution level. These
instructional mode codes are used in the student information system (SIS) during
registration. In a memo to the Nevada System of Higher Education (NSHE) distance
education directors, dated October 4, 2011, the UNLV Online Education Director stated:
"NSHE DE Directors have a specific interest in ensuring the Instruction Modes
are clear and used appropriately in their accountability to their institution, the
system, and for USDE and state authorization regulations. The group should
finalize our work, including notating Instruction Modes that are “DE” related
regardless if the specific Instruction Mode is used at our respective institution. As
individuals with specific knowledge in our field, we are best equipped to make
this identification."
To date, no statewide-approved set of instruction mode definitions has been
developed.

18
Following is the most current working set of instructional modes and descriptions:
Instruction
Mode
Description Guidelines for Use
CR Correspondence This code should only be used for print-
based Independent Learning courses.
EN Some Web
participation
These “in person” courses have a
WebCampus supplement but face to
face time is not reduced.
FS Field Study These are courses in which the class
meeting place is integral to the learning
experience. (e.g. Geology Field Study)
HY Hybrid These are courses in which face to face
time is reduced and Web participation is
required.
IS Independent Study These are courses with no scheduled
class time.
P In Person These are face to face courses with NO
Web component.
TV Cablecast TV
VR Interactive Video
Receiving Site
These are videoconference classes
taught from a distant site. Typically the
instructor is not at this location.
VS Interactive Video
Sending Site
These are videoconference classes
taught from the origination site.
Typically the instructor is at this
location.
WB Web Based Course These courses are taught entirely online
via the Web with NO required in person
meetings.
WM Web Based
w/some meetings
These are Web-based courses with up
to two required organizational face to
face meetings at the beginning of the
term.
A closer examination of these codes may indicate where confusion exists. The "In
Person" mode is relatively straightforward (representing traditional face-to-face
instruction) as is the "Web Based Course" designation (representing fully online
courses). Misunderstanding may occur in cases where course instruction mode
involves some combination of online and in-person instruction. For example, while
face-to-face time is not reduced, the "Some Web participation" mode indicates some
use of online resources. Since all courses at UNLV, including those meeting completely

19
in person, are granted access to the institutional learning management system, the
distinction between an "In Person" instruction mode and "Some Web Participation"
instruction mode is blurred. Since both students and instructors have access to online
resources, "In Person" courses may in fact involve "Some Web participation."
There is also a discrepancy between the working definition of hybrid learning at
UNLV and the instruction modes identified above. In "Hybrid Learning: A Synopsis," a
handout for faculty, the UNLV Office of Online Education states that "hybrid classes
must have the majority of instruction delivered at a distance and the face-to-face
component, although reduced, is mandatory and scheduled." Guidelines for use for the
"Hybrid" instructional mode, however, states that "These are courses in which face to
face time is reduced and Web participation is required." Notice that the instructional
mode guidelines make no mention of majority or scheduled, mandatory face-to-face
meetings. Interpretation of the handout definition could indicate anywhere from fifty-one
percent to ninety-nine percent online instruction while interpretation of the instructional
mode guidelines indicates anywhere from one percent to ninety-nine percent online.
The inclusion of the "Web Based with some meetings" instructional mode further clouds
the issue. In essence, these courses are online courses with up to two (or three,
depending on the draft of the instructional mode document) organizational meetings,
ostensibly to explain how the course works, with a second meeting later for further
explanation or a proctored exam. This mode has the potential to be a subset of the
"Hybrid" instructional mode. All of these ambiguities and discrepancies may confuse an
administrative assistant scheduling future courses or confuse an instructor who is
developing curriculum.

20
Perceived advantages and disadvantages
UNLV faculty teaching hybrid courses are able to communicate a variety of
perspectives regarding the advantages and disadvantages of using hybrid instructional
mode strategies. All ten instructors interviewed provided feedback in this area.
Following is a representative set of responses:
Advantages
1. "The hybrid format helps students with critical thinking, independent research
and self-reliance, whereas 100% in-person can sometimes foster a dependence
on the professor as 'in-charge' and directive."
2. "Students are more in tune with the course in a hybrid format. They more alert
and more involved."
3. "Most of my students have full-time jobs, families, and outside
commitments. They appreciate not coming to campus as often."
4. "We can delve deeper in our discussions leading to richer experiences."
5. "Students who are good time-managers get to work on their own schedule and
still benefit from face-to-face time rather than entirely online."
6. "Hybrid delivery allows students to accept responsibility for their own
learning."
7. "Hybrid courses reduce (or in some cases, eliminates) the need to photocopy
syllabi, quizzes, and other class materials."
8. "This method gives instructors like me (who are somewhat introverted) a chance
to be more effective in the classroom."

21
9. "Hybrid courses reduce (or in some cases, eliminates) the need to photocopy
syllabi, quizzes, and other class materials."
Disadvantages
1. “Disadvantages include a lack of community feeling between classmates and a
somewhat impersonal experience.”
2. "Because many of my students have not been in college for a while, they
struggle with the technology, become resentful and impede their learning
process in the beginning.”
3. “Some students (and faculty like myself) miss the face time.”
4. “Some students become more anxious in between meeting face to face
regarding feedback.”
5. "Students lose track of time and don't complete the on-line assignments
before the published due dates.”
6. “There are a number of students who resist the use of technology as part of their
university learning experience.”
7. “Some students assume that a hybrid course will somehow be easier than a
face-to-face course. This is a myth I have to debunk every semester.”
8. Some students in a hybrid course think that face-to-face attendance doesn't
matter.”
9. “Getting students to withdraw from a hybrid course can be difficult if they
don't come to class before the drop deadline, they don't respond to e-mail, or
they don't check WebCampus.”

22
Preparation for teaching hybrid courses
Perceptions regarding the amount of professional development provided faculty were
mixed among UNLV instructors using the hybrid learning model. Three of the
instructors interviewed expressed positive comments regarding the amount of training
they had received. One of those instructors, however, attributed positive hybrid learning
training to weekly meetings with departmental mentors rather than opportunities offered
by the Office of Online Education. Three other instructors expressed a lack of hybrid
learning training of any kind. “I don't think I've had any training or support specifically
for this,” stated one instructor. “I taught DE for a long time so it didn't seem necessary.”
Two instructors also claimed to be self-taught. “I would definitely benefit from more
training,” stated another faculty member.
The way in which UNLV instructors approach teaching in a hybrid course may reflect
a lack of professional development. While several were able to indicate the use of such
hybrid learning best practices as “flipped” discussions, group- and project-based
assignments, and asynchronous communication, three instructors were unable to
express any teaching strategies that would benefit students in this mode. One
instructor claimed to be using face-to-face time strictly for lectures. “It’s the way I’ve
always done it,” he stated.

23
Course development process
All new fully online or hybrid courses follow an established development process. To
initiate this process, faculty developers complete an online course development form.
Distinction between fully online and hybrid courses is indicated by the faculty developer
via a checkbox on a form. Once all text fields on the form are completely filled out
(indicating name of the course, course description, instructor, etc.), the form is signed
by the instructor and the department chair, and then delivered to the Office of Online
Education. Following approval by the OE Director, the course is assigned to a
development team consisting of instructional designers, instructional artists, a
videographer, and interactive applications developers. Ideally, the OE development
team meets with the faculty developer/subject matter expert sixteen weeks prior to the
offering of the course to students, at which time the course syllabus and course
objectives are examined and determined. Full course development is outlined, planned
and scheduled at this point.
Realistically, however, a number of major issues exist that affect ideal course
development. First, not all departments initiate course development at least sixteen
weeks prior to offering. Under the mistaken assumption that online course development
takes very little time, academic units occasionally assign new courses to part-time
instructors (or faculty with little prior experience with online education) within only
several weeks of the start of the semester. Generally this results in courses that are
disorganized, lacking in content, and low in quality. Second, instructors are under no
obligation to collaborate with University online education professionals. In some cases,
subject matter experts are simply unaware that trained and educated professionals are

24
available to assist. In other cases, instructors view collaboration as unnecessary
intrusion into their classroom or curriculum. Third, many legacy courses have been
developed prior to the creation of thoughtful development processes or support teams.
In many cases, these legacy courses have been developed in the absence of any basic
standards and created without any essential support. Efforts have been made to
improve the quality of these courses, but this is an ongoing process.
Lessons learned
Instructors of hybrid courses at UNLV provided a wealth of valuable insight both in
terms of advice for future hybrid learning educators as well as suggestions for how the
Office of Online Education can provide additional support. Following are representative
statements from veteran hybrid learning instructors.
Advice for new hybrid learning faculty
1. “My best advice is to allow students time to work on large online projects in
class so they get that shared experience within groups.”
2. "When I receive several e-mails regarding the same issue, I send one
announcement to everyone rather than individually.”
3. “I am specific about how often/when I will be online. This helps some student
who expect responses instantaneously.”
4. “Don't go for too many weeks without seeing your students in-person.
They can actually forget they have the class.”

25
5. "Be absolutely sure that you have a clear idea of which course components
you want to have in your face-to-face classroom and which course
components you want to have online.”
6. “Set guidelines for assignments (including late assignment submission) and
follow them conscientiously.”
7. “Be diligent in checking and responding to all e-mails in a timely fashion."
8. “Extensive class preparation time is still required for both face-to-face and
online time.
9. “Make sure that your syllabus policies are reviewed with students in a face-
to-face meeting.”
10.“Review sample syllabi from other instructors so that you can be familiar with
policies and procedures that work for others.”
11.“Sit in on a hybrid course, 4-5 classes if possible, to see how the instructor
manages her/his time and the material.”
12.“Learn the features of the learning management system in advance so you
don't have to invest as much time in learning the software while teaching a new
course."
Additional support suggestions
1. “Some of the bugs of the Webcampus system could be improved.”
2. “I think that Office of Online Education should develop an online library of
resources devoted to blended/hybrid learning.”

26
3. “I would like the Office of Online Education to offer workshops or training for
the update as soon as possible.”
4. “The Office of Online Education could help students evaluate whether they are
good candidates for hybrid courses. There should be a screening tool that
asks students about their use of the internet, their access to a computer, their
time management skills and so on and then provides a number or score that
says whether they should register for an online course or not.”
5. “I know academic advisors are supposed to help students evaluate their
schedules, but many of my students claim they have never seen an academic
advisor.”
6. “Webinars would be helpful.”
Advice from non-UNLV institutions
In a survey to members of the Professional and Organization Development Network
(POD), valuable advice was provided regarding the development and delivery of hybrid
courses. Following are the representative statements from that survey:
1. “Be clear about your policies around blended/hybrid courses as they can be
interpreted to mean so many different things to different audiences.”
2. “Make sure you set your expectations for your faculty. We had some
problems with faculty who thought assigning extra reading during the online
portion was a hybrid.”

27
3. “I think it helps for faculty to teach fully online first. This helps them break
down their mindset about what works best online and what works best face to
face.”
4. “Offer reliable and consistent technology.”
5. “Do not expect everyone to conform to one pedagogical framework. “
6. “Evaluate services offered to support faculty.“
7. "Get your faculty champions on board quickly. Have them demonstrate their
abilities in this area and document the quality of learning outcomes.”
8. “On-campus time should be used wisely and for activities that are difficult to
accomplish online.”
9. “Pilot and study it first to help get buy-in on campus. Once there are a few
successful models, more are interested.”
Discussion
The Office of Online Education has made many positive achievements in regard to
improving misconceptions about hybrid learning at UNLV. Such achievements include
the creation of a review of literature on hybrid learning, the development of a two-page
handout identifying the characteristics of hybrid learning (including some of the major
models), a face-to-face presentation to the Hotel College, and a webinar offered to the
entire University community.
The data gathered by this report, however, indicate that continued efforts need to be
made in order to overcome some preliminary challenges. Following are a list of
recommendations that may be helpful toward moving past some of these challenges.

28
Recommendations
1. Establish clear and unambiguous definitions for different instruction
modes
Staff and faculty cannot be held accountable for incorrect instructional mode
designations or inappropriate pedagogical approaches until statewide (or at least
institutional) mode definitions are distinct and not overlapping. Once clearly
identified, UNLV administrative assistants and instructors must then be trained
regarding the differences between these designations.
2. Maintain an efficient and consistent course development process
Courses that are created on the fly, under a short timeframe, or without the full
degree of input between all members of the development team will not rise to the
level of quality required by a first-rate educational institution. University policies
(or procedures) may need to be developed and communicated, and these
policies need to be applied consistently.
3. Provide a certificate course in hybrid learning best practices for instructors
Ideally, faculty would participate in an extensive hybrid course that models and
demonstrates best practices and pedagogical approaches appropriate for this
instructional mode. This program could be an essential prerequisite to teaching
hybrid courses at UNLV.
4. Offer ongoing professional development for instructors and staff
In addition to a certificate course, instructors and administrative assistants will
need continuous professional development in the form of face-to-face
presentations, webinars, tutorial videos, handouts, job aids, and peer sharing
opportunities.
5. Develop a strategy for reviewing hybrid course curriculum
In order to maintain quality instruction and accountability, academic units (and
the Office of Online Education) may consider regular review and/or evaluation of
hybrid curriculum. Hybrid courses undergoing informal (or formal) evaluations
should use measures (such as those established by Quality Matters, Chico State,
or Blackboard’s Exemplary Course program) that are specific and appropriate to
hybrid or blended learning. Ideally, such informal (or formal) evaluation will occur
on a regular basis.
6. Establish a system in which course schedules are communicated
Students need to know in advance the exact dates when participants are
expected to meet in person or online in a hybrid course. In addition, these dates
should be communicated to the department or academic unit in advance.
Deviations from a published schedule should be avoided. This schedule should
be communicated at least at the start of the semester through the syllabus or
through initial e-mail/announcements. Ideally, these times will be communicated
to students prior to registration.

29
7. Curate a library of resources for faculty
Hybrid learning instructors could be provided access to a library of current
research in the form of journal articles and publications, reviews of literature, and
conference proceedings. In addition, an online library could contain access to
model courses, example syllabi, faculty testimonials, instructor tips, relevant
slide-based presentations, course rubrics, module overview templates, archived
webinars, and other such resources.
In recent years, the of Office of Online Education has made important strides toward
providing valuable support for courses offered using the hybrid learning model. These
efforts are commendable and should be continued. If national trends are any indication,
use of the hybrid learning model as an effective method of delivering instruction will only
increase. Therefore, continued efforts to increase awareness of the different
characteristics and pedagogical strategies of this mode are a critical element of the
University’s goal to deliver quality education.

30
References
Allan, I.E, & Seaman, J. (2011), Going the Distance: Online Education in the United
States. Babson Survey Research Group, Babson Park, MA.
Bowers, M. (2011). Executive Vice-President & Provost policy memorandum to
Academic Deans (March 18). Distance Education processes.
Clark, R., (2010). Blended Learning. Retrieved from
http://www.nwlink.com/~donclark/hrd/elearning/blended.html
Gitten, S., (2009). UNLV NSSE 2009 Summary Report, UNLV.
U.S. Department of Education (2010). Evaluation of evidence-based practices in online
learning: A meta-analysis and review of online learning studies. Washington,
D.C.

32
Blended Learning Survey
This survey was provided online to learning professionals (administrators and
instructional designers) outside the UNLV community.
Thank you for participating in this survey. UNLV is engaged in a program evaluation
regarding blended (hybrid) learning. Your participation will help us define and develop
our program.
* Required
1. Your institution *
2. Number of blended/hybrid courses at your institution *
3. Number of years using the blended/hybrid learning model *
4. How is blended/hybrid learning defined at your institution? Is there a percentage
formula that determines this definition?
5. What are the primary models of time division for your blended courses? (For
example, alternating days between entirely online and face-to-face meetings, one
week online and one week face-to-face, half semester face to face and half
semester online, etc.).
6. What are the primary pedagogical approaches that have been beneficial to your
blended/hybrid courses. (For example, "flipped" discussions, group collaboration,
project-based assignments, etc.).
7. What are some perceived advantages of the blended/hybrid approach to
teaching and learning?
8. What are some perceived disadvantages of the blended/hybrid approach to
teaching and learning?

33
9. Do your blended/hybrid courses provide students with the opportunity to interact
during the online portion of the course? If yes, how so?
10.How are faculty and students accountable for time either online or in class?
11.How does your institution provide faculty and students with suitable training and
support for courses using the blended/hybrid learning model?
12.Do you have any advice for institutions that are starting to explore the
blended/hybrid model?
13.If I have any additional questions about hybrid/blended learning, may I contact
you? If so, please feel free to share your e-mail address. The privacy of your e-
mail address is extremely important to me. It will be kept in complete
confidentiality, not shared with anyone, and only used for the purposes of
learning more about hybrid/blended learning.

34
Blended Learning interview questions
These interview questions were provided to UNLV faculty who had previously taught
hybrid courses.
Thank you for participating in this interview. UNLV is engaged in a program evaluation
regarding blended (hybrid) learning. Your participation will help us define and develop
our program.
* Required
1. Your name: *
2. Number of years teaching hybrid/blended courses? *
3. How would you define blended/hybrid learning defined at UNLV? Is there a
percentage formula that determines this definition?
4. What are your primary models of time division for your blended courses? (For
example, alternating days between entirely online and face-to-face meetings, one
week online and one week face-to-face, half semester face to face and half
semester online, etc.).
5. What are the primary pedagogical approaches that have been beneficial to your
blended/hybrid courses? (For example, "flipped" discussions, group
collaboration, project-based assignments, etc.).
6. What are some of advantages of the blended/hybrid approach to teaching and
learning?
7. What are some of disadvantages of the blended/hybrid approach to teaching and
learning?

35
8. As an instructor engaged in blended/hybrid teaching and learning, do you have
any "lessons learned" to share? Do you have any advice for instructors that are
starting to explore the blended/hybrid model?
9. Have you been provided with suitable training and support to teach using the
blended/hybrid learning model?
10.How can the Office of Online Education provide additional support for your efforts
to teach blended/hybrid courses?

36
WCET Hybrid Learning Discussion Thread
Northern Virginia CC
·∙ Meets less than 30% of the time in person = online course, managed by central
DL office
·∙ Meets 30% to 49% of the time in person = hybrid course with some oversight by
the central DL office
·∙ Meets 50% or more of the time in person = hybrid course managed by the
campus (f2f) deans
Broward College
·∙ 26 to 79% online portion defined as “blended” class
·∙ 80% or above labeled as “online”
University of Alaska
·∙ 0/Distance = 0% location based
·∙ 1/Distance-based = 1% - 20% location based
·∙ 2/Blended = 21% - 50% location based
·∙ 3/Location-dependent/traditional = 51% or more location based
Eastern Wyoming College
·∙ A hybrid course involves the scheduling of on-campus classroom time along
with the use of any distance delivery method (online, interactive television, and
web conference options). This type of class will typically meet on-campus only
a portion of the time of a standard campus class. “Hybrid” is the use of both
distance and classroom teaching time, with reduced scheduled campus class
meetings.
·∙ A blended course is an on-campus course that includes students participating
synchronously via distance technology. “Blended” is a blend of local and
distance students meeting synchronously.
Utah System of Higher Education
·∙ Technology-Enhanced: Course taught both in-person in a classroom and via
technology. The technology-delivery components include teaching and learning
activities. Use of technology does not reduce the time traditionally spent in the
face-to-face class.
·∙ Hybrid/Blended: Course is taught both in-person, in a classroom and via
technology. The technology-delivered components include teaching and

37
learning activities and reduces the time traditionally spent in the face-to-face
class.
·∙ Online/Internet: Course content is delivered online. While online courses may
require proctored exams, there are no other place-bound requirements and
minimal synchronous (real time) requirements. Regular interactions between
students and instructor are a part of the teaching and learning process.
Washington Community and Technical College System
·∙ Online: A course that uses web-based tools and where 100% of the instruction
and interaction between instructor and student is done online (Proctored exams
still allow for this classification).
·∙ Hybrid: A course that displaces some, but not all, face-to-face time with web-
based tools.
·∙ Web Enhanced: A face-to-face course that does not replace any face-to-face
seat time, and access to web-based tools is required.
Stevenson University
·∙ Blended is any combination of face-to-face regardless of percentages.
·∙ Online is defined as totally online. The exception: 12 or 16 week capstone
courses which occur mostly online and may carry an onsite week or weekend
for completion.
Colorado State University
·∙ Uses 33% as an equivalent to Broward University’s 26% and uses similar
definitions.
The University System of Georgia’s BOR
·∙ F Fully at a distance: All or nearly all of the class sessions are delivered via
technology. The course does not require students to travel to a classroom for
instruction; however, it might require students to travel to a site to attend an
orientation or to take exams. (This is generally equivalent to delivering more
than 95% of sessions via technology.)
·∙ P Partially at a distance: Technology is used to deliver more than 50% of class
sessions, but visits to a classroom (or similar site) are required. Note: if a
course is offered through two-way interactive video, then it should be coded
partially at a distance because students must meet at a designated location.

38
·∙ H Hybrid: Technology used to deliver 50% or less of class sessions, but at least
one class session is replaced by technology.
·∙ T Technology enhanced: Technology is used in delivering instruction to all
students in the section, but no class sessions are replaced by technology.
·∙ Null No Technology: No technology is used in delivering instruction.
Regis College for Professional Studies
·∙ The thoughtful fusion of real-time, synchronous learning experiences and
independently accessed asynchronous learning experiences, developed and
presented in a planned and structured way, to accomplish stated learning
outcomes within the context of a single course.”
Texas Higher Education Coordinating Board
·∙ Distance Education: The formal educational process that occurs when students
and instructors are not in the same physical setting for the majority (more than
50%) of instruction.
·∙ Distance Education Degree or Certificate Program: A program in which a
student may complete a majority (more than 50%) of the credit hours required
for the program through distance education courses.
·∙ Distance Education Course: A course in which a majority (more than 50%) of
the instruction occurs when the student(s) and instructor(s) are not in the same
place. Two categories of distance education courses are defined:
o Fully Distance Education Course: A course which may have mandatory
face-to-face sessions totaling no more than 15% of the instructional time.
Examples of face-to-face sessions include orientation, laboratory, exam
review, or an in-person test.
o Hybrid/Blended Course: A course in which a majority (more than 50% but
less than 85%) of the planned instruction occurs when the student(s) and
instructor(s) are not in the same place.
Minnesota State Colleges and University System
·∙ Media Code 00 - Classroom: All instruction is face-to-face in a classroom
setting. The Internet and or the instructional management system (e.g., D2L)
may be required for components of the course (e.g., grades, assignments,
homework)
·∙ Media Code 03 - Online (Predominantly): Nearly all instruction is delivered
online. No more than two required activities or meetings of the class as a cohort
may occur face to face in the same physical location. The course may have
required proctored exams, in addition to the one or two face to face meetings. A

39
proctored exam is considered one of the face to face meetings if the proctored
exam is given to the entire class at a specified time and location. If the student
has a choice in when and where to get the exam proctored, the proctored exam
is not considered to be one of the face to face meetings. The course may have
required synchronous components: the full class may meet online at a specified
or scheduled time, in addition to the proctored exams and the one or two face to
face meetings.
·∙ Media Code 09 - Blended/Hybrid: A course with reduced classroom seat time
and some instruction delivered online. The course blends online and face-to-
face delivery. Some of the course content is delivered online. The course has
more than two face to face class sessions during the term.
·∙ Media Code 12 - Online (Completely): All instruction for the course is delivered
online. The course has no required face-to-face meetings, no required proctored
exams, and no required synchronous meetings (i.e., there are no meetings of
the class at a predetermined or scheduled time). A student may still have
arranged meetings with small groups of the class or arranged meetings with the
instructor.
·∙ Media Code 13 - Online (With Synchronous Components): All instruction for the
course is delivered online. The course has no required face-to-face meetings
and no required proctored exams. The course has required synchronous online
meetings or activities: the full class meets online at a specified or scheduled
time.

hybrid_learning_program_evaluation

More Related Content

What's hot

Viewers also liked

Similar to hybrid_learning_program_evaluation

More from Michael Wilder

hybrid_learning_program_evaluation