Data Driven ContinuousImprovementJohn Rinderle   @JohnRinderleNorman Bier     @NormanBier
Outcomes for TodayBy the end of this session you will be able to…• Explain the value of a data driven approach.• Implement...
Infinite Points of Light
Infinite Points of Light
Infinite Points of Light
Infinite Points of Light
Infinite ProliferationThe 4 R’sReuseRedistributeReviseRemix
Infinite ProliferationThe 4 R’s           NOT:Reuse               RecreateRedistributeReviseRemix               Add:      ...
The problems of variety• Quality is highly variable• Much duplication of effort• Difficult to choose appropriately• Hard t...
EffectivenessCan be hit or miss
Effectiveness     Demonstrably support students in meeting   articulated, measurable learning outcomes in a               ...
How do we design for effectiveness?
The Course Design Triangle                          Objectives                          Descriptions of what students     ...
Why Focus on Objectives?       1. They communicate our intentions clearly to students          and to colleagues.       2....
Why a “learner-centered”approach? Learning results from what the student does and thinks and only from what the student do...
Learning Dashboard
Let’s Analyze Some ExamplesChecklist: Is the objective…?  •   Student centered (i.e., student should be able to…)  •   Bro...
Scenario #1: Early Enrollment vs.Performance                     Success by Enrollment Date     1   2   3   4   5   6     ...
Data needs to be actionable!                   19
Scenario #2: Psychology Pilot
“liking” ≠ learning               21
Types of Learning Data         Program/Degree          Demographic          Contextual                          What gets ...
Learning Curve AnalysisDataShop: Pittsburgh Science of Learning Center
Other Learning CurveslearnigDataShop: Pittsburgh Science of Learning Center
1.2 10.80.60.4      Activites 1st Try Correct0.2   Activities Eventually Correct      Assessment Correct 0
Other Metrics            Engaged     Didn’t EngageCollege   Pass   Fail   Pass     Fail CFA      87%    13%    36%     64%...
A Virtuous Cycle        Educational        Technology             Data         & Practice                      Theory
Share Alike and Share Data
Upcoming SlideShare
Loading in...5
×

Data Driven Continuous Improvement

1,359

Published on

A powerful feature of online instruction is the ability to embed assessment throughout and capture data on how students are learning. The TAACCCT program calls on institutions to use such data to “continuously assess the effectiveness of their strategies in order to improve their program… and build evidence about effective practice.” In this interactive session, we will share the OLI approach to data driven continuous improvement. Together we will discuss strategies for using learning data to refine course materials by examining examples from our project. We will also present an overview of the learning data and tools available to co-development and Platform+ participants.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,359
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Making assumptions about data is difficult because of variety.
  • Each OER is collecting and capturing different things or not collecting data at all.
  • We need a basis of choice. How do I know if a resource will work with my students?
  • Data needs to be actionable to guide student learning.
  • Data Driven Continuous Improvement

    1. 1. Data Driven ContinuousImprovementJohn Rinderle @JohnRinderleNorman Bier @NormanBier
    2. 2. Outcomes for TodayBy the end of this session you will be able to…• Explain the value of a data driven approach.• Implement design strategies that facilitate meaningful data capture and use.• Avoid commonly pitfalls in the use of learning data.• Share strategies for data driven improvement with your project team.• Explain the data collected and analysis tools available to co-development and platform+.
    3. 3. Infinite Points of Light
    4. 4. Infinite Points of Light
    5. 5. Infinite Points of Light
    6. 6. Infinite Points of Light
    7. 7. Infinite ProliferationThe 4 R’sReuseRedistributeReviseRemix
    8. 8. Infinite ProliferationThe 4 R’s NOT:Reuse RecreateRedistributeReviseRemix Add: Evaluate
    9. 9. The problems of variety• Quality is highly variable• Much duplication of effort• Difficult to choose appropriately• Hard to evaluate• Impossible to improve• Hard to scale success up
    10. 10. EffectivenessCan be hit or miss
    11. 11. Effectiveness Demonstrably support students in meeting articulated, measurable learning outcomes in a given set of contexts
    12. 12. How do we design for effectiveness?
    13. 13. The Course Design Triangle Objectives Descriptions of what students should be able to do at the end of the course Assessments Tasks that providefeedback on students’ knowledge and skills Instructional Activities Contexts and activities that foster 13 students’ active engagement in learning
    14. 14. Why Focus on Objectives? 1. They communicate our intentions clearly to students and to colleagues. 2. They provide a framework for selecting and organizing course content. 3. They guide in decisions about assessment and evaluation methods. 4. They provide a framework for selecting appropriate teaching and learning activities. 5. They give students information for directing their learning efforts and monitoring their own progress. 14Based on A.H. Miller (1987), Course Design for University Lecturers. New York: Nichols Publishing.Also see, C.I. Davidson & S. A. Ambrose (1994), The New Professor’s Handbook: A Guide to Teaching and Research in Engineering and Sciences. Bolton, MA: Anker Publishing Company Inc.
    15. 15. Why a “learner-centered”approach? Learning results from what the student does and thinks and only from what the student does and thinks. The teacher can advance learning only by influencing what the student does to learn (Herb Simon, 2001). It’s not teaching that causes learning. Attempts by the learner to perform cause learning, dependent upon the quality of feedback and opportunities to use it (Grant Wiggins, 1993).
    16. 16. Learning Dashboard
    17. 17. Let’s Analyze Some ExamplesChecklist: Is the objective…? • Student centered (i.e., student should be able to…) • Broken down into component skills (grain size) • Phrased with an action verb • MeasurableHere are some samples: • Understand the U.S. stock market • Recognize logical flaws in a written argument • Appreciate the historical context of the 1940’s 17 • Apply Newton’s Second Law appropriately
    18. 18. Scenario #1: Early Enrollment vs.Performance Success by Enrollment Date 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Days Before Semester Start
    19. 19. Data needs to be actionable! 19
    20. 20. Scenario #2: Psychology Pilot
    21. 21. “liking” ≠ learning 21
    22. 22. Types of Learning Data Program/Degree Demographic Contextual What gets captured Behavioral Interaction Semantic Actions Raw
    23. 23. Learning Curve AnalysisDataShop: Pittsburgh Science of Learning Center
    24. 24. Other Learning CurveslearnigDataShop: Pittsburgh Science of Learning Center
    25. 25. 1.2 10.80.60.4 Activites 1st Try Correct0.2 Activities Eventually Correct Assessment Correct 0
    26. 26. Other Metrics Engaged Didn’t EngageCollege Pass Fail Pass Fail CFA 87% 13% 36% 64% CIT 98% 2% 81% 19% CMU 100% 0% 67% 33% HSS 98% 1% 58% 42% MCS 96% 4% 75% 25% SCS 99% 1% 83% 17% TSB 98% 2% 69% 31% Total 96% 4% 67% 33%
    27. 27. A Virtuous Cycle Educational Technology Data & Practice Theory
    28. 28. Share Alike and Share Data

    ×