Currently available course design rubrics can be very valuable tools. However, these rubrics do not address several very important issues related to course quality. We’ll examine five additional areas that should be considered when working to improve the quality of online courses.
Workplace Simulated Courses - Course Technology Computing Conference
Presenter: Angie Rudd & Kelly Hinson, Gaston College
What do our students need to learn to be productive in the workplace, to get a job, what skills do they need? The workplace has changed, leadership has changed, and the future is collaboration. This presentation will discuss the methods and tools used in two online project classes. We will show you how we take our learning outcomes and design online classes to simulate a workplace environment. These courses are designed to give students the most realistic workplace environment that we can in an academic setting. One course teaches Emerging Technologies by using teamwork and collaboration environments. The other course uses the System Development Lifecycle as a guide for students to complete an individual project with feedback and brainstorming from other students. The goals for the session are: demonstrating and discussing collaboration, showing how to include useful teamwork in an online environment, working as a collective team, sharing information and knowledge, encouraging suggestions and ideas, brainstorming, building in frustration on purpose, using peer feedback in projects, enabling team resources, and embracing roles and responsibilities. Attendees will walk away with a template of how to design a course for a workplace environment while meeting the learning objectives of the course.
Professor Michele Pistone, Villanova University, shares her insights on assessment for legal education, including formative and summative assessment. She explains the difference between formative and summative assessments and the components of effective assessment tools. For more information about online learning, visit, Legaledweb.com and You tube/ LegalED.
Presentation given on Dec 13, 2019 at DePaul University for the D2L Connection: Chicago Edition.
Creator and presenter: Barry Dahl, Teaching & Learning Advocate, D2L
Workplace Simulated Courses - Course Technology Computing Conference
Presenter: Angie Rudd & Kelly Hinson, Gaston College
What do our students need to learn to be productive in the workplace, to get a job, what skills do they need? The workplace has changed, leadership has changed, and the future is collaboration. This presentation will discuss the methods and tools used in two online project classes. We will show you how we take our learning outcomes and design online classes to simulate a workplace environment. These courses are designed to give students the most realistic workplace environment that we can in an academic setting. One course teaches Emerging Technologies by using teamwork and collaboration environments. The other course uses the System Development Lifecycle as a guide for students to complete an individual project with feedback and brainstorming from other students. The goals for the session are: demonstrating and discussing collaboration, showing how to include useful teamwork in an online environment, working as a collective team, sharing information and knowledge, encouraging suggestions and ideas, brainstorming, building in frustration on purpose, using peer feedback in projects, enabling team resources, and embracing roles and responsibilities. Attendees will walk away with a template of how to design a course for a workplace environment while meeting the learning objectives of the course.
Professor Michele Pistone, Villanova University, shares her insights on assessment for legal education, including formative and summative assessment. She explains the difference between formative and summative assessments and the components of effective assessment tools. For more information about online learning, visit, Legaledweb.com and You tube/ LegalED.
Presentation given on Dec 13, 2019 at DePaul University for the D2L Connection: Chicago Edition.
Creator and presenter: Barry Dahl, Teaching & Learning Advocate, D2L
Evaluating the Quality of Online Teaching D2L Barry
Presentation by Barry Dahl at the 2018 D2L Connection: London Edition.
This session addresses the development of a process that can be used to evaluate the quality of online teaching. The current evaluation processes at many colleges do not specifically address online instruction, and most academic administrators have limited personal experience with online course design and instruction. Two useful documents will be shared in this webinar. The first is a five-part Pre-Evaluation Worksheet. The second document is a five-category rubric for evaluating teaching performance of online instructors. The rubric could be used as a self-evaluation for faculty, as part of a peer review evaluation, or as part of an academic administrator’s evaluation of faculty. These documents are sharable and editable. Use these documents to start a faculty evaluation process or to build upon your existing process.
Getting started with blended, a presentation for NMSUTanya Joosten
Through an examination of the ten basic questions of blended course redesign, participants will reconceive their traditional face-to-face courses for blended teaching and learning. Participants will follow backwards design principles to design a course module, and will learn techniques for integrating face-to-face and online work, and apply them to their own courses. For experienced teachers, this workshop provides a new approach to design a course in order to overcome "course and a half" syndrome and better manage your workload.
Friday, November 14th, 8:30am-11:30am
Making Connections: How Students Use Feedback to Inform Future LearningBlackboardEMEA
This session will share findings and recommendations from a research study at Sheffield Hallam University aimed at creating a better understanding of the connections students make between their assessment feedback and future assignments, and what impact staff practice has on this process. The presentation will cover each stage of the assessment process for both students and staff, and make specific reference to how identified issues have been addressed through technology elsewhere in the UK Higher Education sector.
The main element of the presentation will be a detailed explanation of the recommendations resulting from the research, and the policy, practice and technology changes required to implement them.
Introducing the PASS program: Pre-arrival success strategies for studentsLearningandTeaching
PASS (Pre-Arrival Success Strategies) is a free online course for students who will be studying with the International College of Manitoba (ICM) in subsequent terms. The course is designed for students who wish to get a head start on their studies. The course is led by ICM alumni who mentor prospective students in navigating the complexities of their new education system.
Units are delivered over five weeks and cover a variety of topics, including overviewing the learning management system (Moodle), connecting students with their future classmates, understanding essential elements of course outlines, planning for weekly and term schedules, introducing concepts of academic integrity, and more. Students have flexibility in choosing which units they wish to complete as they accumulate points for each assessment completed - students who complete various achievements earn various rewards.
In these slides, Robert Daudet reviews the course content, shares lessons learned from its first iteration, and reveals what's next for the course.
This presentation helps you to address the key challenges in remote teaching through various teaching platforms & tools for specific purpose, Effective use of technology using SAMR model. Educators can use this content to manage their own professional growth and well-being.
Needs Assessment
To identify the goals for an instructional project
Identify the gap between the desired goals and the current status
This gap is referred to as a need
So, what is "design"? Officially, one way to define design is: "To formulate a plan for." Thus, design could be thought of more simply as planning. As you probably know, the term design is not exclusive to instructional design. Design is used in many different fields, and even in our everyday lives. For example, architects design buildings, engineers design cars, football coaches design plays, scriptwriters design films, and instructional designers design instruction. In each of these cases the design process takes place before the actual creation, or development, of the product or process of interest. During the design stage the people involved (the designers) create functional specifications for the product or process. Architects create blueprints, engineers create technical drawings, and coaches create playbooks. In the same way, instructional designers create their own collection of design documents.
Designing Competency Structures and Learning ObjectivesD2L Barry
Title: Designing Competency Structures and Learning Objectives.
For a presentation April 21 at Georgia State University.
By Theresa Butori, Univ of North Georgia
More Related Content
Similar to Five Important Things You Won't Find in a Course Quality Rubric - Barry Dahl
Evaluating the Quality of Online Teaching D2L Barry
Presentation by Barry Dahl at the 2018 D2L Connection: London Edition.
This session addresses the development of a process that can be used to evaluate the quality of online teaching. The current evaluation processes at many colleges do not specifically address online instruction, and most academic administrators have limited personal experience with online course design and instruction. Two useful documents will be shared in this webinar. The first is a five-part Pre-Evaluation Worksheet. The second document is a five-category rubric for evaluating teaching performance of online instructors. The rubric could be used as a self-evaluation for faculty, as part of a peer review evaluation, or as part of an academic administrator’s evaluation of faculty. These documents are sharable and editable. Use these documents to start a faculty evaluation process or to build upon your existing process.
Getting started with blended, a presentation for NMSUTanya Joosten
Through an examination of the ten basic questions of blended course redesign, participants will reconceive their traditional face-to-face courses for blended teaching and learning. Participants will follow backwards design principles to design a course module, and will learn techniques for integrating face-to-face and online work, and apply them to their own courses. For experienced teachers, this workshop provides a new approach to design a course in order to overcome "course and a half" syndrome and better manage your workload.
Friday, November 14th, 8:30am-11:30am
Making Connections: How Students Use Feedback to Inform Future LearningBlackboardEMEA
This session will share findings and recommendations from a research study at Sheffield Hallam University aimed at creating a better understanding of the connections students make between their assessment feedback and future assignments, and what impact staff practice has on this process. The presentation will cover each stage of the assessment process for both students and staff, and make specific reference to how identified issues have been addressed through technology elsewhere in the UK Higher Education sector.
The main element of the presentation will be a detailed explanation of the recommendations resulting from the research, and the policy, practice and technology changes required to implement them.
Introducing the PASS program: Pre-arrival success strategies for studentsLearningandTeaching
PASS (Pre-Arrival Success Strategies) is a free online course for students who will be studying with the International College of Manitoba (ICM) in subsequent terms. The course is designed for students who wish to get a head start on their studies. The course is led by ICM alumni who mentor prospective students in navigating the complexities of their new education system.
Units are delivered over five weeks and cover a variety of topics, including overviewing the learning management system (Moodle), connecting students with their future classmates, understanding essential elements of course outlines, planning for weekly and term schedules, introducing concepts of academic integrity, and more. Students have flexibility in choosing which units they wish to complete as they accumulate points for each assessment completed - students who complete various achievements earn various rewards.
In these slides, Robert Daudet reviews the course content, shares lessons learned from its first iteration, and reveals what's next for the course.
This presentation helps you to address the key challenges in remote teaching through various teaching platforms & tools for specific purpose, Effective use of technology using SAMR model. Educators can use this content to manage their own professional growth and well-being.
Needs Assessment
To identify the goals for an instructional project
Identify the gap between the desired goals and the current status
This gap is referred to as a need
So, what is "design"? Officially, one way to define design is: "To formulate a plan for." Thus, design could be thought of more simply as planning. As you probably know, the term design is not exclusive to instructional design. Design is used in many different fields, and even in our everyday lives. For example, architects design buildings, engineers design cars, football coaches design plays, scriptwriters design films, and instructional designers design instruction. In each of these cases the design process takes place before the actual creation, or development, of the product or process of interest. During the design stage the people involved (the designers) create functional specifications for the product or process. Architects create blueprints, engineers create technical drawings, and coaches create playbooks. In the same way, instructional designers create their own collection of design documents.
Designing Competency Structures and Learning ObjectivesD2L Barry
Title: Designing Competency Structures and Learning Objectives.
For a presentation April 21 at Georgia State University.
By Theresa Butori, Univ of North Georgia
Custom Pathways Resources - Kristin Randles.pdfD2L Barry
Presentation by Kristin Randles at the D2L Connection: South Carolina Edition on October 28, 2022 at Piedmont Technical College in Newberry.
Resources:
Carnegie Mellon: https://www.cmu.edu/teaching/designteach/teach/classroomclimate/strategies/choice.html
Cult of Pedagogy: https://www.cultofpedagogy.com/udl-equity/
Novak Education: https://www.novakeducation.com/hubfs/Resources/UDL_FlowChart.pdf
C-BEN: https://www.cbenetwork.org/
Presentation by Denise Huff of Spartanburg Community College at the D2L Connection: South Carolina Edition on October 28, 2022 at Piedmont Technical College in Newberry.
E-Learning Mythbusters Revisited - ITC 2022.pptxD2L Barry
Original presentation was at ITC's eLearning conference in February 2008.
This presentation takes an updated look at some of those e-learning myths in 2022.
Video Captions and Transcripts Made Easy , or at least easierD2L Barry
I share almost everything I do, and typically assign the Creative Commons Attribution type of copyright “protection” which means you are free to use my stuff as you like – but it is nice to get a shout out (or attribution) from time to time.
Resources for the presentation are shared at https://bit.ly/Dahlvideo
Video Captions and Transcripts Made Easy, or at least easierD2L Barry
Presentation by Barry Dahl, Oct 21 at #D2LFusion 2020 (virtual conference).
Finding good captioned videos on YouTube
Editing the auto-captions on YouTube
Creating a transcript for your video
Keyboard shortcuts for YouTube
VTT caption file format
D2L as a Training Platform for Faculty: Lessons LearnedD2L Barry
Webinar:
Date:Apr 7, 2020
Time:3:00 PM ET
Duration:1 hour
Presenters:
Archie L. Williams, Ph.D., Sharee’ Lawrence, Denise Sutton, Dr. Tamara Payne; all of Fort Valley State University
Handout: YuJa, post to a discussion from a mobile deviceD2L Barry
Handout for presentation by Joan Anderssen, Arapahoe Community College at the D2L Connection: 2020 Colorado Edition.
A day of learning, sharing, and fun at Red Rocks Community College in Arvada, Colorado.
Handout for presentation by Joan Anderssen, Arapahoe Community College at the D2L Connection: 2020 Colorado Edition.
A day of learning, sharing, and fun at Red Rocks Community College in Arvada, Colorado.
Yuja the best, Don't just Yuja (D2L Connection Colorado)D2L Barry
Presentation by Joan Anderssen, Arapahoe Community College at the D2L Connection: 2020 Colorado Edition.
A day of learning, sharing, and fun at Red Rocks Community College in Arvada, Colorado.
We all have good and bad thoughts from time to time and situation to situation. We are bombarded daily with spiraling thoughts(both negative and positive) creating all-consuming feel , making us difficult to manage with associated suffering. Good thoughts are like our Mob Signal (Positive thought) amidst noise(negative thought) in the atmosphere. Negative thoughts like noise outweigh positive thoughts. These thoughts often create unwanted confusion, trouble, stress and frustration in our mind as well as chaos in our physical world. Negative thoughts are also known as “distorted thinking”.
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
The Indian economy is classified into different sectors to simplify the analysis and understanding of economic activities. For Class 10, it's essential to grasp the sectors of the Indian economy, understand their characteristics, and recognize their importance. This guide will provide detailed notes on the Sectors of the Indian Economy Class 10, using specific long-tail keywords to enhance comprehension.
For more information, visit-www.vavaclasses.com
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
Five Important Things You Won't Find in a Course Quality Rubric - Barry Dahl
1. Barry Dahl
Teaching & Learning Advocate, D2L
Five Important Things
You Won't Find in a
Course Quality Rubric
2. Brief History of Online Course Design Rubrics
•2002: Cal State Chico developed the ROI (Rubric for
Online Instruction)
• Implemented at Chico in January 2003
•2003: FIPSE Grant won by Maryland Online: "Quality
Matters: Inter-Institutional Quality Assurance in Online
Learning“
• Grant term: Sept 2003 – Aug 2006
3. A Brief Personal History
• Feb. 2004 - QM workshop at ITC eLearning Conference
• May 2004 - Faculty at my college voted to create a peer
review process using V1 of QM rubric as the course design
recommendations
• 2005-present – former college still uses a peer review
process with a rubric adapted from V1 of QM.
• 2010 – began consulting gigs promoting the use of a three-
pronged approach to improving online course and program
quality
• 2012 – began work at D2L – and here we are today
10. Purposes of Instructor Evaluation
Encourage
excellence in
teaching and
learning.
01
Facilitate long-
term quality
improvement by
monitoring
instructional
performance.
02
Provide
constructive
feedback by
identifying
strengths and
areas for
improvement.
03
Inspire
professional
growth and
development.
04
11. Faculty Evaluation Components
• Student Evaluation of Instruction
• Faculty Self-Evaluation
• Professional Development Plan
• Faculty Peer Review
• Faculty Portfolio
• Supervisor’s Evaluation of Faculty
Typically some combination of the above components
12. Barry’s
Thoughts on
Online
Instructor
Evaluation
• It’s not the same
• Classroom observations in
traditional classroom are
mostly for show
• Observers/Evaluators need
to “get” the art of online
teaching
• It CAN be done.
Item
#1
15. Examples of Collegewide Assessments
•Collegewide Outcome #1
• Written and oral
communication
• Practiced extensively,
across the curriculum, in
the context of
progressively more
challenging problems,
projects, and standards
for performance
• Assessment Projects for #1
• Assessments of writing
skills
• Scored by a cross-
disciplinary team of faculty
• Administered to random
selection of students from
across the college
• Segregate writing samples
of online students for
comparison to on-campus
students
18. Why Do Students Take Online
Courses?
Caveat: assuming they aren’t forced to by a
worldwide pandemic or something similar…
19. Ruffalo Noel-Levitz PSOL
•This survey can tell you how satisfiedyour students are and what issues are
reallyimportantto them in the areas of:
•Academic services
•Enrollmentservices
•Institutionalperceptions
•Instructionalservices
•Studentservices
20. PSOL Factors to Enroll in Online
46. Financialassistanceavailable
47. Future employment opportunities
48. Reputation of Institution
49. Work schedule
50. Flexible pacing for completing a program
51. Convenience
52. Distance from campus
53. Program requirements
54. Recommendations from employer
Students rank
each factor on
how important it
is to them.
21. Most Important Factors to Enroll Online
46. Financialassistanceavailable
47. Future employment opportunities
48. Reputation of Institution
49. Work schedule
50. Flexible pacing for completing a program
51. Convenience
52. Distance from campus
53. Program requirements
54. Recommendations from employer
These 3 factors
always rank as
the most
important.
22. Flexibility
•#1 Reason why student take online courses
•Not distance, not cost, not preferred
learning modality
TIME FLEXIBILITY
23. 3. Windows of
Opportunity
What is a Window of
Opportunity?
The amount of time that an online
student is given to complete a
task; such as
• take a quiz,
• complete and submit an
assignment, or
• post to a discussion
• et cetera Item
#3
24. Example: Required Writing Assignment
• Opens Friday morning and closes Sunday night.
• Three days (but only two sleeps) might not seem too short for some people, but it
certainly would for other people,depending on their circumstances.
Piece of cake.
That’s perfect
for me.
No way can I
get that done
in time
27. 4. Student-
friendly Flexible
Design
Course design choices
made for online courses
tend to fall into one of
these categories:
1. Student-friendly
2. Faculty-friendly
3. Both
Item
#4
28. Example of “Friendliness”
• Instructor 1 isgoing to post a schedule for
weekly online office hours.
Step 1: instructor looks at hisweekly schedule
to see what will fit in
Step 2: chooses days/times that fit nicely for
him
Step 3: posts the schedule
• Instructor 2 isgoing to post a schedule for
weekly online office hours.
Step 1: instructor surveys students to see
which days and times they are availableto
meet, if needed
Step 2: chooses days/times that work best for
the most students
Step 3: posts the schedule
30. Beyond Design:
Instructional Procedures that are Student-Friendly
•Again,if the focus is placedsquarelyon the coursedesign,
variousthings will slip through the cracks.
•This next exampleis a caseof instructor behaviors(practices)
and it is unlikelythat someonewould design/planthe coursein
this manner.
31. Example of Untimely Feedback
Monday – Module 1 Essay Questions due date
Thursday – Module 1 Exam due date
Friday – Feedback on Essay Questions received by students
32. Timely Instructor Feedback
•Timely and useful feedback is needed to stay ontrack in online courses.
•Planto set asidesufficientgradingtime soon afterthe assessmentis due.
•Some facultyfind thatthey save time by providingaudioorvideo
feedbackrather than text, whichcan alsoincreaseyour instructorpresence
withonline students.
33. Barry’s
Thoughts on
Student-
Friendly
Design
• Being student-friendly does
not require you to be faculty-
unfriendly.
• Synchronous activity
alternatives?
• Form student groups based
on availability?
• Give option of not working
on weekends or holidays.
Item
#4
34. 5. Continuous
Quality
Improvement
(CQI)
Not “one and done”
Iterations on quality
improvement – don’t eat
the whole elephant.
In and of itself, a course
quality rubric is a very
small cog in an overall
CQI process. Item
#5
35. Barry’s
Thoughts on
Continuous
Quality
Improvement
Some of the important things you
won’t find in a Course Quality
Rubric:
• Institutional support of DE
• Technology support of DE
• Instructional design support
• Online course development
support
• Ongoing PD for online
instructors
• Robust online student services
• Ongoing research and a culture
of evidence
Item
#5
36. Putting it All Together
Continuous
Improvement
Student
Friendly
Windows
Learning
Achieve-
ment
Instruction
Course
Design
Editor's Notes
Hello everyone. Thanks for joining us today
Many of you know me but for those of you who don't, my name is Barry Dahl, and I serve as the Teaching and Learning Advocate at D2L. After spending 27 years working inside Higher Ed, I've spent the last 10 years working with D2L primarily providing professional development opportunities for faculty and support staff who use D2L.
I also had the pleasure of serving on the Board of Directors of the Instructional Technology Council for a few years while still working at a college in Minnesota. This is an organization that's very near and dear to my heart. Let's get started with the next session titled: 5 important things you won't find in of course quality rubric.
Let's start with a brief history of course design rubrics for online courses.
The first one that I know of and can find in the research was from 2002, the Cal State Chico rubric for online instruction or Roi.
This preceded the quality matters rubric by about a year or more.
Quality Matters began in 2003 when they were awarded a substantial FIPSE Grant which covered a three-year term ending in 2006.
There are many other course design rubrics as well, but these two seem to encapsulate the start of the rubrics era
My own history with course design rubric started in February 2004 when I attended the ITC eLearning conference and the people from Maryland Online were doing a workshop about Quality Matters.
Because it was funded by a FIPSE Grant, they needed to share their developments during that Grant period, and they did share version one of their quality matters rubric with the audience at the conference.
That same spring, I brought the rubric back to my College in Minnesota and with a large group of Faculty members we started working on our own process based on an adaptation of the quality matters rubric to create an online course peer review process.
I'm pleased to say that 18 years later that process is still in place and has only grown in quality and acceptance over the years, or so they tell me.
I also spent a fair amount of time working with other colleges as a consultant to take a look at their online course development procedures as well as accreditation efforts for online programs.
In 2012 I started at D2L, and no longer work directly with course quality improvement processes, but I help direct the Course Quality Improvement Affinity Network at our annual Fusion conference and throughout the year.
When I was doing my consulting work, this diagram got a lot of use. Once upon a time I called this my pyramid of e-Quality, and a faculty member quickly pointed out that it’s 2D, so it’s not a pyramid. So, I didn’t call it that anymore.
Part of my message was that there were at minimum three major components of e-learning quality.
As you can see here in this diagram the three different colored triangles represent the quality of learning, of teaching, and of design. I guess you could say that the overall triangle represents the totality of e-learning quality – but that would only be true if lots of other factors were living in that white space in the middle.
My contention then and it's still my contention today is that all three of these components are important when considering the quality of online courses.
The three components can be measured in many ways, but the most common ways are shown on the slide
through assessments we measure the level of learning achieved,
through some sort of performance evaluation process, we measure the level of teaching that occurred,
and through things such as quality matters rubrics and similar instruments we measure the quality of online course design.
Plenty of people do presentations on the course design rubrics, so we are not going to be talking about that today. So, what are we talking about?
We are going to be looking at 1) the quality of teaching, 2) the quality of learning, and then three other items that are not just tangential to the quality of an online course, but I would argue that they are central to it.
Let’s start with the quality of instruction.
One of the issues with relying on a course design rubric is that although it looks at how the course is designed to operate, it is typically evaluated looking at an inactive course. You can't (typically) look at what is actually happening while the course is being taught. In other words, it doesn’t look at how well the course is taught.
For example, a course design rubric typically includes standards related to interaction, such as interactions between students and instructors. However, designing for interaction doesn’t answer the question about whether those interactions actually occurred and at what level they occurred and what value was achieved.
Online Instructor Evaluations are critical to the overall e-Quality process, but they are not without a few potential hurdles or issues.
What are some of the potential issues with conducting an instructor evaluation to determine the “quality” of the instruction?
Here are a few of them:
Many institutions do not have an evaluation instrument appropriate for online teaching
Many institutional leaders do not have the experience with online teaching & learning to be effective evaluators.
Although classroom observations have a long history and are generally accepted practice, observations of the online classroom do not and are not.
However, all the items above have been improving over the past 20 years.
What are the goals or purposes of instructor evaluations?
Things such as:
Encouraging excellence
Monitoring instructional performance
Providing constructive feedback
Inspiring professional growth
Faculty evaluations typically are made up of some combination of the following:
Student Evaluation of Instruction
Faculty Self-Evaluation
Professional Development Plan
Faculty Peer Review
Faculty Portfolio
Supervisor’s Evaluation of Faculty
Some of my thoughts about this process:
Evaluating people who teach online is not the same as evaluating people who teach on-campus (of course many people do both)
Classroom observations in traditional classroom are mostly for show, with the dean or other supervisor sitting in the back of the classroom observing some or all of a single class period.
In online classroom, observation can be very informative and instructive, particularly if the Observers/Evaluators are well-versed in the art of online teaching
Some schools have 20-25 years of experience with this process – so it CAN be done. (I used to often hear the objection of “we just can’t do it!”
Let’s move on to the quality or level of learning that happens in online classes.
There’s really two lenses that I think need to be examined here. Course-level outcomes and college-wide outcomes.
We have a problem if you can’t provide evidence of students successfully completing course outcomes. We also have a more general problem if online students overall are less successful than F2F students in achieving college-wide outcomes.
Let’s start with assessments of collegewide and Program-level learning outcomes.
Do your online students achieve the Collegewide Outcomes and/or Program Outcomes at similar levels as F2F students?
Are you even including (and segregating) online-only students in your collegewide assessments?
Have you been asked by your accrediting bodies for this information? Or have you reported on this info to your accreditors as part of a review process?
An example from my former college
Collegewide Outcome #1
Written and oral communication
Practiced extensively, across the curriculum, in the context of progressively more challenging problems, projects, and standards for performance
Assessment Projects for #1
Assessments of writing skills
Scored by a cross-disciplinary team of faculty
Administered to random selection of students from across the college
Segregate writing samples of online students for comparison to on-campus students
And what about course-level learning outcomes.
Do your online students achieve the expected course Outcomes at similar levels as F2F students in other sections of the same course? Are you tracking Competencies with different groups of learners and comparing?
What can you learn from this? One thing is you can discern your own slice of the pie of the “No Significant Difference” phenomenon and possibly drill into smaller differences in the courses you teach.
What to do about a lower performing group? Possibly remediation for one group or the other. Redesign activities and assessments to help the lower achieving group be more successful
Develop a Culture of Evidence, and this will be part of it.
Not sure if you’ve noticed, but there are still, in 2022, there are still skeptics of online learning. Be prepared to answer them with data.
If you say “but we can’t do that with online students…” then we have a bigger problem to deal with
This was a big deal 15-20 years ago when getting online programs approved by regional accreditors. It still might be a big deal for your accreditor.
Let’s examine this question: Why Do Students Take Online Courses?
How do you answer this question? Why do students take online courses? in normal times, making normal decisions for oneself? In other words, not during a pandemic when remote (online) learning was the only option. I am talking about why students take online courses when they have choices related to the course delivery method.
Many of you are probably familiar with the PSOL, which is the priorities survey of online Learners from Noel Levitz. Back in my college vice-president days, we used the survey 5 times between the years of 2003 and 2009 to gather data about student satisfaction as well as some other factors related to their online learning experiences at the college. I have continued to follow the results of this annual survey since then – partly because I’m a bit of a data nerd – but also because this survey is the one place where I learned the most about our online students and especially how our survey results compared to others who used the survey.
The crux of the survey is satisfaction based, looking at how satisfied online students are with various different services at the college and the quality of instruction received
There's also another portion of the survey that asks about the factors or reasons why they chose to enroll in online courses. You can see the list of nine items on the screen. Three of those nine items consistently have much higher importance levels than all the other items. Which three do you believe would be at the top of their lists?
If you guess the three items in the middle, you are correct. Work schedule, flexible pacing, and convenience, consistently rank as the most important factors.
Almost all of the data that I have seen, both organization specific as well as Nationwide data, point to these three items being by far the most important factors that drive student enrollment in online courses. This is especially true at the undergraduate level. so looking at work schedule, flexible pacing, and convenience, we see a common thread
It seems perfectly logical to me to look at those three items and their similarities and come up with the number one reason why students take online courses. We talk about distance education but it's not really distance. It's not the cost of education because sometimes it's higher than face to face education, in fact usually it is higher. The idea of preferred learning modality is kind of a bunch of hooey.
Instead, those things all relate to the students having time flexibility in their lives. The ability to fit an education into their otherwise full schedules and busy lives.
So, the third factor on our list of five factors today has to do with windows of opportunity. Again this is not something you'll find in a course design rubric, but it's incredibly important for students to manage their time restraints related to coursework.
A window of opportunity is the amount of time that an online student is given to complete a particular task or activity or a series of actions. Faculty members usually make the call on how long the window opportunity will be for each item, which makes sense since they are the ones designing the activities but ensuring that the students have a large enough window of opportunity is a key for student success.
Here's a very simple example, not meant to insult anybody's intelligence, but just to make sure we illustrate what I'm talking about. A writing assignment that the professor posts on Friday morning and the due date is Sunday night so students have three days, in other words, the weekend. For one student that might be awesome. Plenty of time to get it done. For another student that may be incredibly short and they're going to have to disrupt their lives in order to meet the due date.
And what about weekends?
Once upon a time, I took the advice of two students and tried having just one big window for a 16-week Personal Finance course. Everything was available on Day one and everything was due on the last day. There are several reasons that this is not a great idea, unless you’re shooting for having an online correspondence course with very little student interaction – not what most of us are shooting for.
The result was that it worked great for two or three students, and not so great for everybody else. 16 weeks to get the work done, and many students waited until week 14 or 15 to get serious about doing the work. Procrastination is the king of the world.
What would Goldilocks say? How do you know if they’re too long, too short, or just right?
We could make up an answer…but…Common sense goes a long way
Big projects need bigger windows
Student situations are not homogenous – it’s not one size fits all
Consistency is very helpful to online students.
Keep windows similar for similar tasks.
Generally speaking (with exceptions), design choices that are made for online courses tend to fall into one of these categories:
Student-friendly
Faculty-friendly
Both
Here’s a simple example
Instructor 1 is going to post a schedule for weekly online office hours.
Step 1: instructor looks at his weekly schedule to see what will fit in
Step 2: chooses days/times that fit nicely for him
Step 3: posts the schedule
Instructor 2 is going to post a schedule for weekly online office hours.
Step 1: instructor surveys students to see which days and times they are available to meet, if needed
Step 2: chooses days/times that work best for the most students
Step 3: posts the schedule
Instructor 1 is faculty-friendly. Instructor 2 is student-friendly.
Other design considerations to make your course student-friendly
Days and times of the week when due dates are scheduled
Required synchronous activities
Required group work
Modes of contact for online office hours; both the timing of instructor availability and the mode/methods of contact.
Appropriate amount of course content
Extra credit or make-up opportunities for required coursework that is missed
Looking beyond the course design. What about some instructional practices that are Student-Friendly?
Again, if the focus is placed squarely on the course design, various things will slip through the cracks.
This next example is a case of instructor behaviors (practices) and it is unlikely that someone would design/plan the course in this manner.
This is an example of untimely feedback from the instructor to the student.
On Monday the student is supposed to turn in essay question answers for module one. The module one exam is on Thursday of that same week, but the instructor feedback on the answers to the essay questions doesn't get delivered to the students until Friday after the exam.
Again, it's unlikely that someone would design the course with this in mind, but reality doesn't always match design.
Instructor Feedback is one of the items that the PSOL survey shows is super important to students but with which they typically have a low satisfaction rate.
Timely and useful feedback from instructors is important for keeping students on track in online courses.
Plan to set aside sufficient grading time soon after the assessment is due.
Some faculty find that they save time by providing audio or video feedback rather than text, which can also increase your instructor presence with online students.
Being student-friendly does not require you to be faculty-unfriendly.
If synchronous activities are required, have you provided opportunities for students who cannot be in a certain place at a certain time?
If group work is required, have you provided opportunities for students to form groups based on availability?
Are your due dates scheduled in a way, in combination with the length of the windows of opportunity, that provide students with the option of not working on weekends or holidays?
The fifth item was to do with the concept of CQI, or Continuous Quality Improvement.
Don’t treat your Quality Process as a “one and done” operation.
By making it a continuous process, you can make progress over time rather than trying to achieve greatness all at once.
In and of itself, a course quality rubric is a very small cog in an overall CQI process.
Some closing thoughts about Continuous Quality Improvement and the shortcomings of course design rubrics
Some of the important things you won’t find in a Course Quality Rubric:
Institutional support of online education
Technology support for online education
Instructional design support
Online course development support
Ongoing professional development for online instructors
Robust student services for online students
Ongoing research and a culture of evidence
Now comes the easy part. Just take all these moving pieces and create yourself a well-oiled machine of e-Learning greatness.