Webinar Series – Session 2 Outcome-based Learning OpportunitiesMichael Derntl1, Jad Najjar2 and Susanne Neumann1 1 University of Vienna, Austria 2 WU Vienna, Austria
Webinar overviewBackground and motivation for outcome-based educationFormulating learning outcomesLearning outcome based teaching methods and learningdesignsModels and standards for learning outcomes andopportunitiesLearning outcome motivated assessment
Background and motivation Why outcome-oriented teaching and learning? The Bologna Process quality assurance in higher education a unified educational system (mobility, transparency and mutual recognition of qualification) Revise curricula integrate student centeredness and learning outcome orientation Pedagogical issues became central: Alignment of teaching and assessment methods in accordance with learning outcomes
Background and motivation What is your background? Please shortly state in the chat: 1. Your background. (e.g. higher education, PhD student, vocational training, consulting, instructional design, other…) 2. Your position or relation to teaching (if any). 3. Subject(s) you teach / PhD topic.
Formulating learning outcomes Learning outcomes A learning outcome is a statement of what a learner is expected to know, understand or be able to do as a result of a learning process ―European Qualification Framework for Lifelong Learning (EQF) Teaching methods Learning Workload ALIGNMENT outcomes (ECTS) Assessment
Formulating learning outcomes Types of learning outcomes Learning outcomes Competence (personal and professional) The proven ability to use knowledge, skills and personal, social and/or methodological abilities. proven ability to use Skills The ability to apply knowledge and use know- how to complete tasks and solve problems. Personal, social ability and/or methodological to apply abilities Knowledge The outcome of the assimilation of information through learning.
Formulating learning outcomes Intended vs. achieved learning outcomes The teacher defines the intended learning outcomes for a learning opportunity The teacher assesses and learners demonstrate the achievement of intended learning outcomes
Formulating learning outcomes Sentence structure of intended learning outcomes An intended learning outcome is formulated as a sentence that follows a basic structure: Learners are able to Verb Object Describes intended Describes what students (cognitive) process are expected to acquire or constructExample: Learners are able to enumerate the most influential politicians of the 20th century What type of learning outcome is this?
Formulating learning outcomes Examples of learning outcomes and their type Students are able to … … explain the seven layers of the ISO/OSI Reference Model Knowledge … enumerate the most influential politicians of the 20th century … design an interactive website that complies with Web Accessibility Guidelines Skills … filter out relevant literature for a given research problem. … guide their team mates in improving the architectural model of a system. Competence … moderate a virtual task-force meeting
Formulating learning outcomes Guidelines 1. Students are taught about software design flaws → Focus on the learning rather than the teaching! 2. Students are able to identify critical design flaws in a given software system 3. Students complete Unit 1 on basics of academic writing → Focus on the outcome, not the process! 4. Students are able to explain the basic principles of academic writing 5. Students are able to give a good presentation about their semester project → Make sure that achievement can be assessed! 6. Students are able to convey the essential outcomes of their semester project in a five-minute slideshow presentation
Formulating learning outcomes Activity Formulate and write in the chat: If you’re a (PhD) student: a learning outcome of the program or course you‘re taking. If you’re a teacher/tutor: a learning outcome of a course you’re teaching. Indicate in parentheses whether the outcome is related to: knowledge (K) skills (S) or competence (C) Example: Students are able to explain the basic principles of academic writing (K)
Teaching methods and learning designs Problem outline Alignment of outcomes, methods, and workload Here: focus on aligning teaching methods with learning outcomes Goal: facilitating learners in achieving the intended outcomes Teaching methods Learning Workload ALIGNMENT outcomes (ECTS) Assessment
Key concepts ICOPER concept model Sharable Educational Resources is a Assessment TEACHING Method METHOD is a intended defines Learning Learning result of INSTRUCTIONAL roles Facilitator imple- imple- ments Outcome ments MODEL Learner is a is a Assessment LEARNING Design DESIGN involves uses uses instantiation of Learning Content Learning Opportunity instantiation of is a includes Stored in Assessment the OICS Opportunity
Teaching methods and learning designs Examples of Teaching Methods 1/5 Lecture: The teacher is the primary delivery channel for instructional content – often through presentation.
Teaching methods and learning designs Examples of Teaching Methods 2/5 One-Minute Paper: Learners quickly write thoughts to make their level of understanding explicit.
Teaching methods and learning designs Examples of Teaching Methods 3/5 Feedback: Reinforce or correct actions performed by learners.
Teaching methods and learning designs Examples of Teaching Methods 4/5 Think-Pair-Share: To break long presentations, learners first think about a question individually, then discuss their thoughts with a partner, and lastly report back to the teacher.
Teaching methods and learning designs Examples of Teaching Methods 5/5 Role Play: is a “dramatized case-study”. Its purpose is to have members of a group act out roles surrounding a situation, condition, or circumstance to make different perspectives on the issue visible.
Teaching methods and learning designs Description of instructional models ICOPER researched description elements that practitioners require for browsing, selecting and implementing instructional models. General description: title, summary, rationale, subject, learning outcomes, learner characteristics, group size, duration, setting, author/licensing Detailed description: graphical representation, sequence of activities, roles, assessment method, resources, variations, implementations, references Comments: teacher reflections, student feedback, peer review This allows for a structured human-readable description
Teaching methods and learning designs Description of instructional models To describe instructional models in an interoperable machine-readable format, there are several existing standards and specifications, e.g.: IMS Learning Design (LD) SCORM IMS Simple Sequencing (SS) In ICOPER: Instructional models described with IMS LD (other formats allowed), with LOM metadata record(s) on top Storage in the Open ICOPER Content Space (OICS) for OICS/LOM introduction see the replay of Webinar 1; for IMS LD the upcoming Webinar 3; for application demos Webinar 5.
Teaching methods and learning designs Alignment of Learning Outcomes and Assessment Teaching methods Learning Workload ALIGNMENT outcomes (ECTS) Assessment
Teaching methods and learning designs Alignment of Learning Outcomes and Assessment Learning outcome: Students are able to moderate a virtual task-force meeting. Aligned assessment: A tutor observes a task-force meeting where a student moderates, and gives feedback to the student based on pre-defined performance criteria. Misaligned assessment: Students take a multiple-choice test on how to moderate meetings.
Teaching methods and learning designs Alignment of Learning Outcomes and Teaching Methods: General General alignment: What do we mean by “alignment” when talking about teaching methods? Teaching methods Learning Workload ALIGNMENT outcomes (ECTS) Assessment
Teaching methods and learning designs Alignment of Learning Outcomes and Teaching Methods: General Learning outcome: Students are able to moderate a virtual task-force meeting. Teaching methods: Lecture One-minute paper Feedback Think-pair-share Role play
Teaching methods and learning designs Voting Activity Take a vote: Which teaching method(s) do you consider a good/best fit for this learning outcome? “Students are able to moderate a virtual task-force meeting” Teaching methods: Lecture One-minute paper Feedback Think-pair-share Role play
Teaching methods and learning designs Alignment of Learning Outcomes and Teaching Methods: Specific Use taxonomy for alignment, for instance, Anderson & Krathwohl taxonomy of educational objectives Students are able to moderate a virtual task-force meeting. LO: moderationSource: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Blooms Taxonomy of Educational Objectives(Complete ed.). New York: Addison Wesley Longman.
Teaching methods and learning designs Alignment of Learning Outcomes and Teaching Methods: Specific TM: TM: One- TM: TM: Think- TM: Lecture Minute-Paper Feedback Pair-Share Role Play LO: Moderation Assessment: ModerationSource: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Blooms Taxonomy of Educational Objectives(Complete ed.). New York: Addison Wesley Longman.
Teaching methods and learning designs Teaching methods What to look after besides learning outcome alignment when choosing teaching methods? Allow learners to practice and do things themselves; don’t just “talk” about it Plan for social interactions between learners Support learning process (feedback, scaffolding, coaching)
Teaching methods and learning designs Learning Design Building a learning design: Adding content and assessment to the teaching method according to learning outcome. Generic teaching method Specific learning design integrating the teaching method Integration of a whiteboard tool to collect reasons
Models and standards Specifications in Focus Learning Outcomes (IEEE RCD) Describe and manage Learning Outcome Definitions ICOPER LOD AP Learning Opportunities (CEN MLO) Publish metadata of learning opportunities ICOPER MLO APP Personal Achievements (new spec.) Describe and share information on learners/teacher achieved learning outcomes and associated evidence, level and context data Personal Achieved Learning Outcomes (ICOPER PALO) Backbone for a new CEN specification
Models and standards Metadata for Learning Outcomes Describes the characteristics of a learning outcomes Enable storage, referencing, sharing Learning outcomes data
Indexed large set of learning outcomes http://oics.icoper.org/LOD/
Models and standards Sources Learning Outcomes OpenLearn (OUUK) All courses have defined learning outcomes Stored in IEEE LOM metadata classification category IEEE/ACM Computer Science Curricula (Task Force) All course descriptions have learning outcomes PDF file Used worldwide by many universities 21st Century Engineer (NSF project) 36 generic learning outcomes Engineering domain .ppt file European Computer Science Driving License (ECDL) Full list of learning outcomes per topic Excel sheets WU Vienna – Business Education study programme Local XML into LOD XML
Models and standards Learning OpportunitiesExample:Course catalog
Models and standards Metadata for Learning Opportunities Describe an opportunity a learner can follow Enables publishing metadata across universities/countries
Generate MLO Instances Course catalogs Local XML format Transformation into ICOPER MLO XML 1000s instances Want to produce MLO instances from your course catalogs?…Talk to us ;-) http://oics.icoper.org/MLO/
Models and standards Metadata for Personal Achievements Represent information on achieved/required/desired learning outcomes of a learner. Also teachers (taught outcomes) Share data with LMSs, recruitment systems, social applications
Models and standards PALO Example Najjar et al. (2010) A data model for describing and exchanging Personal Achieved Learning Outcomes (PALO). International Journal of IT Standards and Standardization Research, vol. 8, pp. 87-104.
Models and standards Generated PALO Data Data collected across systems ATOM/RSS feeds Shared in iGoogle Facebook LMSs etc.
Learning outcome motivated assessmentEvidence of achievement Successful completion of learning opportunity Evidence that a learning outcome was achieved/attained Each learner achievement has an assessment record Assessment record is a summary result of one or more learning assessments
Learning outcome motivated assessment Approaches Automatic System generates assessment records from assessment results in LMS Submit data to learner profile when course is closed Semi-automatic Student claimes achievement of outcomes submit assessment records to learner profile Teacher approves assessment records data submitted to learner profile
Closing More Resources & Feedback ICOPER Training Infrastructure: Learning Outcome Course: http://training.icoper.org/moodle/mod/resource/view.php?id=269 Instructional Modeling Course: http://training.icoper.org/moodle/mod/resource/view.php?id=270 Your feedback: What do you take home from this webinar? Please share via the chat window or raise your hand to speak. Feedback form: http://tinyurl.com/28gjny2
ICOPER Webinar Series Upcoming Webinars Webinar 3: “eContent for designing and delivering outcome-based learning opportunities” February 3, 2011 (next Thursday) 16.00-17.30 Webinar 4: “Learning outcome-based assessment and applications” February 10, 2011 16.00-17.30 Webinar 5: “Applications for outcome-based learning” February 17, 2011 16.00-17.30