Before we get started, how many of you have taught online already?How many have designed an online course?
Peer review process to certify quality of course design, not to assess quality of instruction or teaching, just course designThe organization offers subscriptions to colleges and universities. Member institutions can conduct quality improvement process on their online courses. After a course successfully completes the process, it can market itself as a Quality Matters approved course.
Just served on a team that reviewed a psychcourseWe had a psych professor on our teamWe had our first con call with the course instructorThree weeks to review individuallyAnother con call to discuss course and explain items we didn’t passMaster reviewer then called instructor to go over reviewInstructor given about 8 weeks to make changesMaster reviewer then approves or rejectsWhole thing from start to end is 20 weeks
Refer participants to handout and go over a couple of examples from the handout. (Page 5)To meet Quality Matters review expectations a course must: Answer ‘Yes’ to all 3-point Essential Standards AND Earn 72 or more points. Take a minute to read over the rubric and think of examples of how your current or future course either meets or does not meet one of the items.Can anyone give an example of how an online course might meet one of the standards?Any questions about items you think might be difficult to achieve?You can use the rubric not only to evaluate an existing course but also to build a new course.I will use the Quality Matters framework of 8 standards to move through my material this morning.
Here is the front page for a course I teach at CMU about online learning. It has a Getting Started button which gives tips for success as well as links to university resources of interest to students.
Web site organization easy to understand and navigateThe way you organize the site should be logical, not too many clicks. It should allow students to go directly to areas of importance to them: gradebook, discussion board, learning modules.Entire course available at beginning of semesterThis is something that is very different from f2f course but remember, online students often choose this mode of learning because they need to work their learning into an already busy schedule. This helps them manage their time.Opportunities for practice with the technologyEspecially for students who are taking an online course for the first time, give them opportunities in the first week to send email, submit an assignment that is ungraded, access the library, etc.Information about youStudents’ first impression of youEstablish your presence, both professional and personalOpportunities for self-introductionsJust as you would in a f2f class, have students introduce themselves, tell their prior experiences with your subject, describe their goals for the course, etc.
Required readingsTextbooks listed along with some instructions about how to get them (e.g., Amazon)If you will have additional readings, explain how you will provide those to students. Could be readings from Web-sites, course packs sent to students, journal articles from your library, and/or scanned book chapters.Course scheduleSome instructors separate this from the syllabus because it changes from semester to semesterCourse policiesHow do you handle late assignments?Policies on cheating and plagiarismContact info for disability services
Here are the five areas I’ve found in my own research that imply important directives to students.I’ve included a link to an online resource you can refer students to from the Illinois Online Network.Mention requests by Pitt Online students for step by step instructions about accessing library resources.
Ask participants: what is so great about learning objectives?Evidence of these skills, knowledge, and attitudes is what you will look for in every assessment and learning activity you conduct. Your learning objectives should inform your development of practice activities and assessments. Ask participants to contribute examples of skills, knowledge, or attitudes from their coursesSince we have limited time, we will focus on module objectives because it is easier to see how we can link them to activities and assessments.
Many of you are familiar with Bloom’s Taxonomy. It comprises six levels of knowledge or understanding.Words like learn, know, understand are not measurableWords like discuss are activities, not objectivesAlways write the objective at the highest possible level since the higher levels subsume the lower.Handout in your packet provides a longer list of verbs. (Paqe 3)
Some examples here of learning objectivesDiscuss theories of distance education.Replace “discuss” with “explain.”Can we make it at a higher level?Understand the structures and purposes of basic components of prokaryotic and eukaryotic cells.Can we replace “understand” with another verb?Highest level?Describe advantages and disadvantages of electronic health record systems.Verb is good. What about level?Explain the relationship between systems theory and information.Verb is good. What about level?Activity: Have participants use activity sheet to write short description of learning module and develop two or three objectives; leave columns for learning activities and assessments but tell participants not to fill out yet. Use Bloom verbs in packet. Take a minute or two and share some of your learning objectives. (Page 1)
Why objectives are so important. They guide you in developing both your activities and assessments. They describe what you should be looking for in your assessments and what your students should be practicing in their learning activities.If I still haven’t convince you, keep in mind that the Middle States Commission on Higher Education now requires all of its programs to specify learning objectives (they call them outcomes) and provide assessments that demonstrate their achievement:“An accredited institution is expected to possess or demonstrate … clearly articulated statements of expected student learning outcomes … at all levels (institution, degree/program, course) and for all programs that aim to foster student learning and development …”
Pass out pocket instruction design cardBreak here?
Idea is to start out with the end in mind. What do we want our students to be able to do by the end of the course or the end of a module? Those are our course objectives, right? Next question is what is appropriate evidence that our students have accomplished the goals we set for them? These are our assessments. How do we prepare them to be successful in the assessments? Those are the learning activities.Does the perspective of backward design change the way you would write your learning objectives?
The students experience the sequences as Objectives > Activities > Assessments.You write the sequence as Objectives > Assessments > Activities.Does the perspective of backward design change the way you would write your learning objectives?
Two major forces impacting assessment practices in online learning. One is the separation of the student from the instructor. There are concerns about academic integrity as well as how you monitor students’ formative performance.Other driver is one that is also impacting f2f education: quest for more authentic assessmentsAssessments that are situated in meaningful, real world contexts. Giving students the opportunity to demonstrate mastery in such contexts brings them in contact with the environment in which they will eventually practice the skills they learn in your courses.
We know that stakeholders in online learning have concerns about assessment. There is quite a bit of recognition of the issue in the literature although there is not much empirical research.Astudy by Kennedy and others found that both college students and faculty believed that cheating was easier in distance education classes. Vandehey, Diekhoff, and LaBeff say that rates of cheating in college have changed very little in the 20 years between 1984 and 2004. There are virtually no large-scale studies of cheating in online learning. The speculation about it faroutweighs the empircal work. Regardless of the frequency of cheating and plagiarism in DE, the perception of the potential for it warrants special attention.
Wiggins & McTighe talk about a continuum of assessment types. At one end are informal assessments that check for student understanding. At the other end are terminal assessments designed to assess performance mastery.Here is a continuum with four different points, each one corresponding to an example of a type of assessment.Formal assignments like tests, projects, and research papers are usually assigned with a fairly long lead time and once they’re submitted, we normally consider them final. Informal assignmentsenable students to explore new material and also give us a window into their current understanding. Often, they are given with no lead time. They are not expected to be as polished as a formal assignment. They may not even be graded. Because they are not graded, they are often referred to as low-stakes assessments. Conversely, formal assignments are sometimes called high-stakes assessments.Q&A discussion board provides students with a forum to post questions about concepts they don’t understand. Encourage students to answer each other’s questions.Blog– Have students keep a running journal in which they assess their learning for each class and make connections between new and previously-learned material. You can ask them to submit the journals to you in the middle and at the end of the semester. Instead of grading the content, you could assign a percentage that the journal submission contributes to the overall grade.Response paper – For a class with a substantial amount of reading, you could assign two or three 1-2 page response papers throughout the course asking students to respond to an article or book chapter, take a specific position, and relatethe content of the article to other material either in or outside of class.Research Proposal – Because one of the markers of formal assignments is revision, assign papers at this end of the continuum in stages, allowing students to submit portions of the paper for feedback and revision. You could assign a research proposal in three stages: Research question and literature reviewTheoretical framework and methodsComplete proposal In f2f environment, practice activities often not assessed; in online environment, we must attach some point value as incentive to participateIn f2f environment, we can more quickly get a sense of where students are in their understanding of the material; we must be more deliberate online
Two types of assessment strategies: Traditional and authenticTraditional means assessments that are primarily objective, multiple choice, true/false, etc. Your CMS has lots of features to support the administration of either graded or ungraded quizzes. You can create a large pool of quiz items that can be deployed randomly to students so that no two students take the same quiz. You can build your quiz bank over several semesters. When students take a quiz, they can see right away if they’ve gotten the right answer. They can also be shown the right answer. They can be restricted to taking a quiz only once or they may take it multiple times. All of this can be recorded in the CMS’s gradebook. The CMS can also monitor timed tests.Use proctored exams for major summative assessments. Specify what students may have with them during a proctored exam, e.g., no handheld devices. Many proctoring sites are available. Libraries will do it as will many academic institutions. Story about Prague.Plan for technology failure.
Authentic assessments are those which the student accomplishes by engaging in more authentic activities, like writing a research paper or composing a piece of music or writing a computer program. Authentic assessment has been described as being “integrated with the learning experience, not separate from it” (Assessment and Online Teaching, 2002).Field work: interview an expertConcept mapping: Carol’s Fac Dev coursePortfolios: some professional certifications require portfoliosGroup work: many instructors use wikis as project repositories for group projectsOnline discussion: very common practice to grade online discussion
A common practice is that assignments are due more frequently than in f2f classes.
Define cheating and plagiarism so that students know exactly what you mean by those terms.You could even create a quiz students must take to show their understanding.Be specific about what the consequences are.Create a community environment that lessens moral distance.Develop meaningful assessments that gives students opportunities to enhance their learning. Vary assessment types.
Activity: Ask participants to suggest assessments for some sample learning objectivesActivity: Have participants suggest an assessment for one of their learning objectives using module objective worksheet (Page 1)
Learning activities are the bridge between objectives and assessments.
Two broad categories of learning activities: content presentation and active learning. Or you could call them instruction and practice.ReadingsHow can you motivate the students to take responsibility for the reading so that you don’t feel you need to reiterate everything?Use ungraded quizzes to make sure students have done the readings. Electronic resources from textbook publisherMany textbooks provide accompanying instructor resources in electronic format. Look in front or back of book for Web site address.Example from Pharmacology course of animations.TextIn writing your own text, use a writing style midway between formal and informal, similar to how you would communicate in a classroom but remember, it has to be more concise and more clear.VideoYou may want to consider creating a video if you’d like to demonstrate a technique or introduce yourself to your class. Example: Barbara’s four minute intro on YouTubeVideo can also be used to interview guest speakers. Example: Stuart FiskKeep the video short. If you are the sole performer, prepare a script.AudioYou can also use freeware like Audacity to create short audio files in MP3 format that your students can listen to on their computers or MP3 players. Keep it short.Script it. Use strategies to engage your students like asking rhetorical questions. Use the best recording equipment available and reduce ambient noise.Learning objectsA learning object is a Web-based resource that has been created for reuse in an instructional activity. There are a number of repositories on the Web that store and link to learning objects. The advantage of finding learning objects that fit your instructional objectives is that you don’t have to create material from scratch. I’ve included the url for one of the oldest and largest LOR’s: MERLOTCaveats:You have to invest some time to see the LO. It may not be what you want once you get there.Learning object repositories all have different search strategiesLOR’s often have links that go to outdated sites
We all rely on Powerpoint to deliver classroom lectures. Remember that its effect on students is different when they’re viewing it not as a supplement but as the complete information message. Using it this way, you can augment its capacity by doing voiceover or including notes on the Notes page.Break up long presentations into multiple shorter ones according to topical divisions within the presentation. Use font and background colors that provide adequate contrast for reading but keep away from colors that are hard to look at like bright pinks and greens.Use high quality images with appropriate resolution and sizing for the slide. It’s OK to use charts and graphs that convey complex information as long as the data can be read comfortably and labels are used as needed for clarification.Remember that many images you find on the Web and in publications are copyrighted. If you wish to include such images in your slides, you must obtain permission from the publisher. It can take a long time to obtain these permissions. Alternatively, there are sources of copyright free images. I won’t go into that because I know we have a copyright presentation later in the day.In preparation for your voiceover session, you should write a script for each of your slides. Although you may be very familiar with the material you will present, you will find that talking from notes in the recording booth can be time consuming as you try to go back and fix your minor speaking mistakes. Having a script to read from will make the recording session go much faster.
As effective as PowerPoint can be in conveying information that comprises both text and images, there are cases when it may not be the most suitable media. For example, if the material you wish to present is mostly text, it may be better to create Word documents that can be presented to the student in PDF format. Or present your text on HTML pages in your course management system.When your material recapitulates what is in the book, you may not want to present it again in a PowerPoint format. However, to ensure that the students have read the material and help them gauge how well they’ve understood it, you can have them take an ungraded quiz. Activity: What are some other ideas you have for presenting content?
The theory of active learning holds that students learn best when they learn from one another, have some control over their learning experience, and are required to engage in deeper levels of processing new material.Philosophy of practice activities: low stakes opportunities for students to practice skills, knowledge, attitudes but they do need feedbackCulture of sharing of ideas and learning collaboratively, critiquing and being critiqued by other students
What makes a good discussion question?Convergent questions are good for exams but not so good for discussion.Gooddiscussion questions have multiple answers but they require much of the students than simply stating an opinion.
As an instructor, it is your decision as to how much you will participate in the discussion. Some faculty are very hands-off. Others just the opposite. It’s probably good to model effective discussion behavior early on (probing questions, challenging, etc.) and gradually fade as the course progresses. It’s always helpful to students for you to weave and summarize the discussion for them.
Ice breaker activitiesUse these motivational activities at the beginning of the course to help students to get to know one another.Bonk and Dennen describe one activity they call Eight Nouns. Each student must describe themselves using eight nouns and give the reason they chose each one.Creative thinking activitiesThese activities give students an opportunity to develop and articulate insights.Give students a list of scholars, authors, and thinkers you are studying. Have them assume a person of their choice participate in the discussion using that person’s voice. Critical thinking activitiesUse these strategies to support students in analysis, synthesis, and evaluation of course content.You can use case studies in a variety of ways to support these activities.For instance, form students into small groups to discuss different case studies. At the end of the discussion, each team will prepare a summary to post to the entire class.You can also model how to write cases. Have students create their own mini-cases based on their work or field experience. Post cases to class discussion board and have other students react.Collaborative learning activitiesThese strategies allow students to share information, give and receive feedback, and construct knowledge collaboratively.One such strategy is called Structured Controversy. Assign two students the pro side and two students the con side of a debate and have them discuss. Then, switch the roles so that the students must take the opposite position. At the end of the activity, they must write a brief compromise paper together.For more ideas, you can get a copy of the Bonk and Dennen paper online. The url is listed in the Resource Sheet.
Explain how blogs are different from wikis
Blog editing privileges: Multiple options including: 1) anyone in group can contribute entries but no editing of others’ work, 2) only one person can contribute entries but others can read and comment, 3) blog is private between blog author and instructor.
Web site exploration: if you want them to find out about the National Center for Educational Statistics database, give them an assignment that requires them to report on several statistics. For example, ask them to pose a question about changes in higher education enrollments in the last 20 years. Then tell them to find the answer in the data sets. They’ll need to explore multiple databases and learn how to use the various links on the site in order to achieve the goal.
Lot of excitement these days about Web 2.0 tools for education. It’s not a new version of the Web. Rather, it is the current generation of easy to use applications that enable communication and collaboration.Explain what each application does. Talk about Dawn’s wiki. I’ve included a handout pointing you to resources for Web 2.0 (Page 7).
Your level of expertiseComputer literacy and experienceComfort level with technologyCourse workloadDevelopment timeInstitutional support for particular technologySupport staffSoftware licensesEquipmentBudget
Ask students to suggest appropriate technologies. Caveat: not all their objectives will be so obvious.Activity: Suggest learning activities that link your objectives to your assessments using module objective worksheet (Page 1)
Within the field of DE, an issue that continues to receive attention is that of student success in distance courses.Refer participants to Western Interstate Commission for Higher Education Guide to Developing Online Student Services in Resource Sheet
Just as traditional academic programs provide services for students with disabilities, so, too must online programs. Example: Ever notice when you roll your mouse over pictures on Websites that sometimes you’ll see a little text box with a very short description of the picture? That’s an accessibility accommodation for screen readers. People with visual impairments can get screen readers to read the text on a Web page to them. Providing these little text boxes gives the screen reader a way to report what the image is.I’ve included pointers to Web sites on the Resource Sheet that will give you more information about accessible Web sites.
I’ve listed a resource in the Reference list, Effective Workload Management Strategies, you can refer to for more information. Also a book called Keeping Found Things Found that provides solutions for organizing you digital materials.
Pedagogy of Online TeachingIndiana University of PennsylvaniaMay 10, 2010 Lorna Kearns email@example.com
Quality Matters Peer review process to certify quality of online course design Launched in 2003 by MarylandOnline with funding from a FIPSE grant Self-sustaining in 2006 Offers subscription memberships to colleges and universities Institutional members request peer review for their online courses
How It Works Quality Matters is a quality improvement process that is continuous, collegial, and collaborative. Peer review team 3 QM certified peer reviewers including: 1 master reviewer 1 content expert Uses rubric comprising eight standards of quality instructional design
8 Standards Course overview & introduction Learning objectives Assessment & measurement Resources & materials Learner engagement Course technology Learner support Accessibility Critical course components must be aligned
Course Overview and Introduction Provide clear instructions for getting started. Organize your site so that it is easy to understand and navigate. Make the entire course available at beginning of semester. Give your students opportunities for practice with the technology. Provide information about yourself. Ask students to introduce themselves to one another.
Syllabus Course overview and rationale Required readings Assignment descriptions Assignment point distribution Schedule Course policies Tips for success
Online Student Success Manage your time effectively. Cultivate good study habits and practices. Recognize and develop an appropriate set of technology skills. Develop and follow a set of good research practices and policies. Know how to communicate effectively online.
Learning Objectives What will your students be able to do … At the end of the course? At the end of each unit of instruction? Learning objective should be: Observable Measurable What are you observing and measuring? Skills Attitudes Knowledge
Bloom's Taxonomy Higher Order Learning Lower Order Learning
Some Examples Discuss theories of distance education. Understand the structures and purposes of basic components of prokaryotic and eukaryotic cells. Describe advantages and disadvantages of electronic health record systems. Explain the relationship between systems theory and information.
Backward Design “Given a task to be accomplished, how do we get there?” -- Grant Wiggins and Jay McTighe Understanding by Design (1998) Identify desired results. Determine acceptable evidence. Plan learning experiences and instruction.
Assessment for Online Learning Forces impacting assessment practices in online learning: Separation of student from instructor Authentic assessment
Academic Integrity Is cheating easier online? Both college students and faculty believe it is easier in distance learning courses (Kennedy et al. 2000) College student self-reports of cheating for online learning no higher than f2f (Grijalva, Kerkvliet, & Nowell, 2006) Very little change in cheating rates over 20 years (Vandehay, Diekhoff, & LaBeff, 2007)
Assessment Continuum Q&A discussion board Blog Response paper Research paper 2 3 4 1 Formal High-stakes Time for planning Revised & polished Graded Summative Informal Low-stakes Short time frame No revision Not graded Formative
Traditional Assessments Use features of your course management system to make use of: Large pool of test items Randomized selection of items Randomized selection of item response order Timed tests Use proctored exams for major summative assessments. Devise a backup plan for technology failure.
Authentic Assessments Field work Concept mapping Portfolio assessment Group work Online discussion To reduce academic misconduct: Use cumulative assignments that build on one another. Benchmark student writing ability. Employ plagiarism detection strategies. Incorporate personal experiences. Ask for connections to timely, current events.
Content Presentation Readings Electronic resources from textbook publisher Text Video Audio Learning objects PowerPoint
PowerPoint Keep presentations short. Make bulleted items clear. Leave ample white space. Use colors that provide adequate contrast for reading. Use appropriate resolution and sizing for images. Be sure charts and graphs can be read comfortably. Remember copyright issues. Script your narration. Check out Audacity and Jing.
Powerpoint is most effective for conveying information in visual images like charts, graphs, diagrams, photos, schematics.
Choose alternative media when information is primarily text-based.
Use to supplement rather than recapitulate required reading.
Active Learning Discussion board Wikis Blogs Web site exploration Web 2.0 activities
Discussion Board Questions Convergent questions have a right answer Divergent questions require interpretation
Discussion Board Instructions Please post your initial response within the first three days of the week. Respond to at least two of your classmate’s postings. Keep in mind that a meaningful response will usually entail introducing some new information or insight into the discussion. When you make reference to a body of work, please provide the citation. While it is fine to express agreement with a posting or ask for elaboration, a meaningful response must go beyond that type of rejoinder. Your discussion grade will be based on the timeliness of your postings, the substance of your participation, and the degree to which it extends the group discussion.
Web Site Exploration Give students a goal that you know will expose them to what you want them to see. Use principles of scavenger hunt.
Web 2.0 Current generation of World Wide Web tools used to enable social and participatory activity (Anderson, 2007) Sometimes used interchangeably with the terms “social software” and “social media”
Technology Selection Learning objectives should guide technology selection. When multiple technologies can be used to meet learning objectives, use variety. In choosing technologies for your course, consider: Your level of expertise Your students’ level of expertise Students’ access to technology Institutional support for particular technology
Technology Alignment Systems analysis term project with archive of project artifacts Guest presentation about harm reduction for intravenous drug users Pronunciation practice in second language learning Reflection on pre-service teaching experience
Learner Support New student orientation Student advising Course evaluations Student satisfaction surveys Community building activities Course checkpoints like assignment submission and exam performance Technology help desk Library services
Accessibility All course Web pages should be accessible to individuals with disabilities Syllabus includes a statement about how students with disabilities can request accommodations
Managing the Workload Course design takes time. First time teaching is also time consuming. Use these strategies to manage the workload: Complete course design by beginning of semester. Develop rubrics. Make students aware of your policies for responding to email. Create FAQs. Develop naming and organization standards for your digital files.