The document provides an overview of the Next Generation Course Redesign project at the University of North Texas. It discusses the goals of improving student learning outcomes, having a university-wide impact, and establishing sustainable and replicable course redesigns. It then describes the process used for course redesign, which involves faculty teams redesigning courses over two years within a community of practice. The document demonstrates the redesign process through examples of a sociology course and a world literature course that were redesigned using blended learning approaches with online, small group, and lecture components.

1. For more information:
“Next Generation Course Redesign” in the Nov/Dec issu
Change
Next Generation Course Redesign
(2010) Peter Lang Publishing
“The Promise of Blended Learning,”
AA&CU News

2. Goals for today
 Share why we thought that it
was important to have a serious
look at how we teach
undergraduates at UNT
 Provide a brief overview of the
blended learning approach
 Describe how the Next
Generation Course Redesign™
Project works at UNT
 Demonstrate two NGen courses
 Challenge you to think about
the optimum blend of
pedagogies for an instructional
unit

3. Why use a blend of pedagogies in the redesign?

4. The “Perfect Storm”
 Bad News
 High DFW rates
 Demographics (higher and more diverse enrollments)
 Financial (tuition cannot keep exceeding CPI)
 Accountability
 Good News
 Knowledge of learning
 Emergence of digital tools

5. What Do We Know About College Students and
How They Learn?
 Brain Research
 The role of neural networks
 Every network can be traced back to an experiential referent
 Our students have “grown up digital”
 Comfortable in a multimedia information-ubiquitous
environment
 Social networking is second nature
 Cognitive development is an important goal
 Most entering freshmen are at Kegan’s 3rd stage
 College experience must encourage a “discipline of inquiry”

6.  We know that, if we
provide an active
learning experience
that allows students to
engage with the
content, each other,
and instructors, they
can and will think
critically and develop
cognitively.

7. Goals of the UNT NGen Project
 To improve student learning outcomes in large
enrollment undergraduate courses
 To have a university-wide impact through the
establishment of a Community of Practice and the
creation of redesign that is both sustainable and
replicable

8. (Translation) Goals:
 Students think, work hard, like what they are doing,
get good grades that mean something, and graduate
 Doesn’t cost more and uses less space
 Faculty have fun

9. UNT’s Transformational QEP Goal

10. Next Generation Course Redesign™ is a Process…
 Faculty teams redesign 4-6 courses each year (two-
year commitment)
 “Choreographed” Process
 Retreats
 Monthly meetings with teams of faculty and staff
 Institution-wide forums
 End-of-pilot meetings
 End-of-project meetings

11. Next Generation Course Redesign™
 The redesign occurs
within an
interdisciplinary
Community of
Practice
 NGen Faculty Fellows
have the option of
being designated
Senior Faculty Fellows

12. Creating a Culture of Reflection
 Promulgate the philosophy of “never ending redesign”
 Schedule small group meetings with the majority of
time devoted to reflection and discussion
 Provide opportunities for ongoing virtual discussion
 Promote assessment-driven change

13. Next Generation Course Redesign™ also has
products. NGen Courses have:
 Clearly articulated
student learning
outcomes that include
outcomes representing
higher level learning.
 Test items that match
the student learning
outcomes.
 An assessment plan that
shows how the test
items map to the
outcomes.

14. Common Assessment Across All
NGen Courses
 Preference for Course
Format Survey (NGen vs.
Traditional)
 Attitude Toward Subject
(pre/post)
 LEP Survey of Cognitive
Development (pre/post)
 Raw score distributions

15. (The Blend) NGen Courses have:
 Large group lectures
(0-30% of contact
hours)
 Small group
experiential learning
(30-60% of contact
hours)
 Media-rich interactive
online environment
(30-50% of contact
hours)
These are the “building blocks” for an
NGen course.

16. In NGen, Lectures are best used to:
 Create interest and motivation and provide assurance
that the students can be successful
 Clarify and expand upon, rather than deliver, content
 Model the acquisition of knowledge in the field, e.g.,
“How does a historian/chemist/sociologist approach a
research question?
 Present the critical lower level concepts to
provide scaffolding for higher level concepts

17. In NGen, Experiential Learning
Activities are best used to:
 Introduce an emotional component (brain-based
learning)
 Analyze, evaluate, and synthesize
 Present and defend newly-acquired hypotheses
 Provide collaborative, cooperative, and academic
controversy activities that encourage thinking
critically from multiple perspectives

18. Dilemma!
 If I use experiential learning in my class, how am I
going to “cover” all of the material?

19. In NGen, Online Learning Activities
are best use to:
 Acquire lower level learning to free up time for in-
class experiential learning
 Chunk content to overcome working memory limits
 Provide low-stakes assessments such as quizzes for
practice and confidence building
 Provide psychomotor experiences
 Provide concrete experiences that are guided and
efficient

20. Next Generation Course
Redesign™
 UNT offers 19 NGen Courses
 Art History, Biology I, Computer Applications, U.S. History I
& II, Human Development, World Literature, Principles of
Language Study, Business Communications, Occupational
Health, Music Appreciation, Developmental Math/College
Algebra, Intro to Communications, Modernism and the
Visual Arts, Organic Chemistry, American Government I,
World Literature Since the Renaissance, and Sociology of
Disasters
 With five more in development (Motor Development, Survey
of Mathematics, Global Marketing Concepts, American
Government II, and Individuals and Society)
 RFP has been issued to select up to eight courses to start the
redesign process in the fall

21. Nicole Dash, PhD
Associate Professor, Sociology
Director, UNT Core Academy

22. Why Redesign?
 Large Enrollment
 No Textbooks
 90% Lecture
 Students struggling to connect what they
learned with real life situations

23. Redesigned Model
 10% Lecture: First two class periods only
 50% Online: 9 Online Modules
 10% Online Activities and Projects
 30% Small Group Experiential Learning
Activities

24. What
the class
looks
like to a
student:

25. Lecture:
 First two face to face classes only
 Introduces how the course will run
 Discusses the schedule and who attends class when
 Focuses on explaining online components
 Introduces subject
 Disaster photos and impact discussions
 Basic definitions
 More complicated theoretical issues

26. Online
 9 online modules
 Exams
 Discussions
 Group Sign Ups
 Wiki Project

27. Experiential Learning
 Small Groups
 Use information learned online
 Develop a Warning Message
 Stratified Monopoly/Disaster Scenario Exercise
 Mock City Commission Meeting

28. Back to Monopoly…
 How does the Monopoly game relate to SLOs?
 Assesses two SLOs:
 6.2 Investigate how social structures (such as race, class, and
gender) generates disaster vulnerability and privilege
 6.5 Assess your own social position and resulting
vulnerability and privilege
 Ties the results of the Monopoly game to a disaster
scenario
 Requires students to step out of their own shoes first,
and then apply that to their own lives

29. Preliminary Findings
 Starting to Analyze Data:
 Majority preferred Next-Gen Model to traditional
model
 57% Overall preferred Next-Gen to F2F
 61% of Successful Students preferred Next Gen
 64% of Unsuccessful Students preferred F2F
 All students who attended class and engaged in activities
passed the class
 Unsuccessful students tended to consistently miss
projects and assignments

30. Next Generation Course Redesign™: Example
 ENGL 2210 World Literature I
Dr. Tracey Gau
NGen Senior Faculty Fellow

31. QEP: The N-Gen
Course Model
 A rigorous outcome-based assessment plan
 Ongoing redesign based on assessment results
 A specific mix of instructional strategies, including:
 Small-group Experiential Activities
 Media rich, Interactive Online Exercises
 Large-group Lectures

32. A Sample N-Gen Course:
World Literature
Course Assessment Blueprint
 Outcomes based assessment plan
 Coordinates SLOs, instruction, and assessment items
 Ensures that what is measured is what is valued
 Shows types of assessment instruments used to measure
each outcome

33. World Literature Test Plan (Blueprint)
©2008 T.M.Gau Dr. Tracey Gau Assessment Plan for N-Gen ENGL2210: World
Literature I Test Items and Difficulty Level
Domain Course Learning Goals L M Hi T C
(Institutional) General Student Learning Outcomes Specific Lesson Learning Outcomes o e ot R
(Departmental) (Classroom) w d
World 1. Demon- 1.1 Recall and recognize the historical 1.1.1 Recognize that the history of 6 2 8
Litera strate an awareness sequence of major literary figures, representative epics are oral or
ture -- and recogni- texts, and movements within the written compositions
Ancient tion of the scope and Ancient, Middle, and Renaissance
through variety of works of periods
Renais-sance literature
1.2 Identify conventional literary 1.2.1 Identify major characters and 6 2 8
genres, elements, and devices figures that appear in more than one
text
1.3 Employ discipline specific 1.3.1 Apply genre characteristics to 4 4 8 1
vocabulary in order to recognize the representative texts
relationship between form and
content
1.4 Relate literary or cultural 1.4.1 Identify and connect literary or 5 3 8
concepts, principles, terms, cultural concepts as they directly
strategies, and styles to a range of relate to representative texts
All course materials © 2009, Dr. T.M. Gau
literature

34. World Literature Test Plan cont’d (Goal 2)
©2008 T.M.Gau Dr. Tracey Gau Assessment Plan for N-Gen ENGL2210: World
Literature I Test Items and Difficulty Level
Domain Course Learning L M Hi T C
Goals General Student Learning Outcomes Specific Lesson Learning Outcomes o e ot R
(Institutional) (Departmental) (Classroom) w d
World 2. Read critically and 2.1 Analyze, evaluate, interpret, 2.1.1 Relate cultural qualities of a 3 2 2 7 1
Litera analytically synthesize representative texts from the hero to a representative character
ture -- Eastern and Western traditions and
Ancient relate them to their literary and cultural
through contexts
Renais-sance
2.2 Compare and contrast major literary 2.2.1 Compare the Eastern depiction 2 3 2 7 1
figures, their situations, decisions of fate, virtue, and heroism to the
Western depiction in representative
texts
2.3 Connect various periods, texts, 2.3.1 Differentiate between 3 2 2 7 1
authors, and characters concepts, such as absolutism and
relativism, using examples from
representative texts
2.4 Evaluate the ideas presented in a 2.4.1 Evaluate characters’ decisions 2 2 3 7 1
text, their implications, and their and actions in the context of their
relationship to ideas beyond the text various cultures and worldviews

35. World Literature Test Plan continued (Goals 3 and 4)
©2008 T.M.Gau Dr. Tracey Gau Assessment Plan for N-Gen ENGL2210: World Literature I
Test Items and Difficulty Level
Domain Course Learning Goals L M H T C
(Institutional) General Student Learning Outcomes Specific Lesson Learning o e i o R
(Departmental) Outcomes w d t
(Classroom)
World 3. Construct 3.1 Formulate a central interpretive idea about Written Composition
Litera informed, organized the texts; (See Constructed Response
ture -- and coherent written 3.2 Develop ideas logically and coherently with Rubric)
Ancient responses to literary adequate supporting textual examples;
through texts 3.3 Present ideas clearly and concisely;
Renais-sance 3.4 Observe the standard conventions of
formatting, citation, grammar, and punctuation
4. Actively discuss 4.1 Present (in discussion posts or small groups)
ideas with others information or conclusions that help other
students summarize, synthesize, and integrate
the conceptual material
4.2 Establish a relevancy between literature and 4.2.1 Relate readings to life and
how it affects one’s life, personal values, ethical academic experiences
behavior, aesthetic judgment, and problem 4.2.2 Make interdisciplinary
solving connections, such as with
history, philosophy, art, rhetoric,
business, politics

36. Structure of N-Gen World Lit
All course materials © 2009, Dr. T.M. Gau

38. SLOs begin every lesson

39. Cognitive Categories
and
Difficulty Levels
 Low: Literal and Factual Knowledge
• Recall, identify, understand
 Med: Conceptual and Procedural Knowledge
• Interpretive and Inferential
• Apply, analyze, interpret
 High: Meta-cognitive Knowledge
• Evaluate, construct, create
All course materials © 2009, Dr. T.M. Gau

40. Formative Assessment:
Online Activities
All course materials © 2009, Dr. T.M. Gau

41. Formative Assessment:
Tools of Mastery

42. Formative Assessment:
Games of Mastery
All course materials © 2009, Dr. T.M. Gau
All course materials © 2009, Dr. T.M. Gau

43. Formative Assessment:
Self Tests
All course materials © 2009, Dr. T.M. Gau

44. Interaction and Collaboration:
Debate
All course materials © 2009, Dr. T.M. Gau

45. Summative Assessment: Constructed
Responses
All course materials © 2009, Dr. T.M. Gau

46. All course materials © 2009, Dr. T.M. Gau

47. All course materials © 2009,

48. Best Practices: Assessments
 Assignments that encourage students to employ critical
thinking strategies
 Rubrics or performance criteria that are made available to
the students
 Scaffolding activities that promote higher cognitive
development
 Providing students with ample opportunities for self
assessment
All course materials © 2009, Dr. T.M. Gau 48

49. Summative Assessment:
Multiple Choice Items
 Offer high degree of reliability & validity
 Vary in difficulty level:
 Low (literal and factual)
 Medium (interpretive and analytical)
 High (metacognitive)
 Correspond to Assessment Plan and SLOs
 Course Goal
 General SLO
 Specific SLO
 Use Item Analysis to improve test items
 Conform to item-writing guidelines
 Match test items to cognitive level of Learning Outcome
 Use Prompts to Address High(er)-Level Cognitive Category
 Use Specific Items Shells in Writing Multiple Choice Item Sets
All course materials © 2009, Dr. T.M. Gau

50. Rate The Item
SLO 2.1
According to Machiavelli, what two animals must a ruler
imitate and what meaning does the analogy represent?
A. Fox and Tiger; in addition to being cunning, a ruler must
protect himself from traps. (LOW .93/.90)
B. Fox and Cougar, in addition to being cunning, a ruler
must be able to act alone.
C. Fox and Lion; in addition to being cunning, a ruler must
use force as well as laws to rule.
1. Low
2. Medium
3. High
0 of 60

51. Rate The Item
SLO 2.1
Which piece of evidence that Iago presents to Othello has the
GREATEST effect of conjuring up doubt about Desdemona’s
fidelity?
A. Cassio’s erotic dream
B. Cassio’s drunken brawl with Roderigo
C. The handkerchief he sees Cassio give to
Bianca (MEDIUM .93/.90)
1. Low
2. Medium
3. High
28 of 60

52. Rate The Item
SLO 2.1
What is the BEST interpretation of Iago’s advice to
Roderigo to “put money in thy purse”?
A. Invest your time, money, and effort in joining Iago in
revenge against Othello (HIGH .63/.64)
B. The marriage between Desdemona and Othello is frail
and can easily be broken
C. Because of woman’s changeable nature, Desdemona
will tire of Othello and seek a new lover
1. Low
2. Medium
3. High
45 of 60

53. Rate The Item
SLO 2.1
Which one of the following is the MOST COMPLETE
description of what Iago represents?
A. Vice Figure
B. Communal Evil (MEDIUM .89/.83)
C. Parasitic Flatterer
1. Low
2. Medium
3. High
41 of 60

54. Item Analysis: Why it’s necessary
to validate m/c items
 Item analysis of summative selected-response (MC) and constructed
(written) response tests due at end of each semester
 Flawed multiple-choice items are up to 15 percentage points
more difficult
than standard items (items that conform to the standard
principles) when testing the same content,
 with median passing rates that are about
3.5 percentage points lower for flawed items as compared to
standard items
(Downing, 2005).

55. Item Prop. Disc. Point Prop. Endorsing Point
No. Correct Index Biser. Alt. Total Low High Biser.
1 .61 .51 .42 A .17 .30 .05 -.27
B .08 .15 .02 -.19
C .61 .35 .86 .42 *
D .14 .19 .06 -.14
2 .21 .18 .18 A .07 .09 .05 -.08
B .22 .26 .18 -.09
C .49 .49 .45 -.02
D .21 .14 .32 .18 *
4 .73 .41 .39 A .73 .50 .91 .39 *
B .02 .06 .00 -.16
C .12 .23 . 04 -.24
D .13 .21 .05 -.19
5 .82 .38 .43 A .02 .05 .01 -.15
B .82 .59 .96 .43 *
C .05 .09 .01 -.16
D .10 .26 .02 -.33

56. CONSTRUCTED RESPONSE STUDENT SCORES
Struc Refer Gram
St Clarity ture Logic Evid Sub tot mar Sent Struct Dev sub tot
1 3 2 2 1 8 0 3 2 7
2 3 3 3 3 12 2 2 2 6
3 4 3 2 3 12 3 2 3 8
4 3 3 2 2 10 2 3 2 7
5 2 2 1 1 6 1 1 2 4
6 2 2 3 3 10 2 2 2 6
7 4 3 3 3 13 3 2 3 8
8 4 4 4 3 15 3 3 3 9
9 3 4 2 2 11 3 3 2 8
…..
…..
38 2 3 2 2 9 2 3 2 7
39 1 2 1 1 5 1 1 2 4
40 2 3 3 3 11 2 2 2 6
M 2.65 2.75 2.28 2.38 10.05 2.13 2.23 2.2 6.60
Mean for content = 10 = 63% Mean for conventions = 6.6 = 55%
Do the average scores for each element make sense in terms of the intent of the
outcomes they are measuring? (validity evidence)
Do the student responses appear to reflect what was taught in terms of the outcomes
they are measuring? (validity evidence)
Did the prompt and the question seem to work? That is, did it produce a high degree of
response?

57. Assessment System Model
Instruction Assessment
Course Design Test Development
VALIDITY
EVIDENCE
Learning Goals
and Outcomes
Scores that are valid for their
intended interpretation
and use

58. SURVEY 1:
Survey of Student Attitude Toward Subject of the
Course
 Survey designed to gather information on how the student feels about the subject
matter of the course the student is beginning, finishing, or has just finished
 Administered Pre and Post
 21 items on a five point scale that are specific to the “subject”
- not teacher effectiveness
- not course effectiveness
 Inference = improved attitude correlates with N-Gen format and course redesign
 13 items positive
 8 items negative
.

59. Specific Results for World Lit:
Student Attitude Toward Subject
Items that increased:
+.190 This subject is worth knowing
+.254 Knowing this subject makes me more employable
+.638 I know a lot about this subject
+.317 This subject is useful to my everyday life
Items that decreased:
-.079 This is a difficult subject for me
-.095 Learning this subject requires a lot of hard work
-.079 This subject is difficult to understand
-.143 This is a complicated subject
-.032 I will have no application of this subject in my
profession
-.032 I am scared by this subject
All course materials © 2009, Dr. T.M. Gau

60. SURVEY 2:
Preference for Course Format:
N-Gen or FTF
his survey consists of one survey question with two answer choices (N-Gen
and FTF) with a short written response in which the student explains the
reason(s) for his/her choice.
nstructor records an S (Successful = A,B,C) or U (Unsuccessful = D,F,W,I) after
end of course. S and U can be typed into the online student- response data
file at end of the course.
wo weeks prior to final exam, spring semester year

61. If you were to start this course over again, would you prefer a traditional
face-to-face format, or would you prefer the N-Gen redesign format you are
experiencing? Please tell why.
Example of student preference for course format (Ngen
vs FTF)
Un-
Total Numb Succ succs Un-
Cours Pref Pref Total Un- prefer Succ prefer succ prefer
N=282 N-gen FTF Numb Succ succ N-Gen prefer FTF N-Gen FTF
Count 172 110 201 81 149 52 51 30
% 61% 39% 71% 29% 74% 37% 63% 37%

62. Categories of Responses
Format Category Typical Comment
N-Gen Pace I liked that I could do most of the work at my own pace when I had time to do it. If I wanted
to go a little slower, I could.
Flexibility
This course allowed me to work out my hectic schedule. Being able to submit quizzes and
stuff online made my life easier.
Learning
I like to learn from a bunch of different sources at once and this course really allowed me
the chance to do that. You got stuff from online sources and you got some face to face
interaction and I think I ended up learning more in this course than I would have otherwise.
Practice I always liked the fact that you could go back and take the quizzes over and over again until
you got them right. I really feel like that helped me a lot in the class.
FTF Manage I prefer a traditional face-to-face lecture because I would often forget about online
assignments and I think my grade suffered. I need more structure in my courses so, I need
to come to campus more often!
Learning I seem to absorb the information better in a traditional class format when I'm taking notes
during a lecture. It just suits my learning style better.
People I can't seem to learn without a teacher lecturing to me and me taking notes. Maybe it
something about the interaction in the communication. I guess I just like being around
people.

63. Final Score Distribution
 To allow a comparison of means from year to year and
pre N-Gen to N-Gen
 Due 1 week after end of each semester year 1 & 2
 Test for statistical significance

64. Success Rates:
World Literature I
ENGL 2210 Large Enrollment Classes
Success Percentages Across Semesters
90%
80% 84%
70%
70% 69% 71%
60% 68% 68%
63% 65%
50%
40%
30%
20%
10%
0%
Fall 04 Spg 05 Fall 05 Spg 06 Fall 06 Spg 07 Fall 07 Spg 08
All course materials © 2009, Dr. T.M. Gau

65. Departmental Advantages
 Addresses problems of course drift, inconsistent learning
experiences, and inefficient use of faculty/course delivery
 Increases the quality of the department’s offerings
 Provides a basic framework that meets university,
departmental, and course objectives
 Offers a pedagogical training for new instructors
 Implements best practices of both face-to-face and online
teaching and learning
 Promotes higher-level learning in introductory courses
 Meets assessment standards of reliability and validity
 Allocates more time for faculty to work on other projects
All course materials © 2009 Dr. T.M. Gau

66. Vision and Objectives after Redesign
 Sustainability and Replication
 Other UNT instructors teaching the course
 Adoption of courseware by outside universities
s World Literature I: Ancient through
Renaissance
World Literature II: Since the Renaissance
 Designer:
 Lead touchpoint resource
 Provides training to instructors
 Updates and maintains course
 Responds to feedback for course improvements
 Cost:
 $90 per student (course fees)
All course materials © 2009 Dr. T.M. Gau

67. Assignment
 See handout with goals and SLO’s
 Select goal and break into groups of 4-6
 Brainstorm combination of the three pedagogies to
reach the SLO’s
 Report

68. Discussion
For more information:
Nov/Dec issue of Change
Next Generation Course Redesign™
Forthcoming from Peter Lang Publishing
Dr. Niki Dash
Nicole.Dash@unt.edu
Dr. Brenda McCoy
Brenda.McCoy@unt.edu
Email to join Dr. Tracey Gau
Course traceygau@comcast.net
Redesign
NING Dr. Phil Turner
Philip.Turner@unt.edu