Open Education 2011: Openness and
Learning Analytics John Rinderle @johnrinderle Norman Bier @normanbier

Open Learning Initiative Produce and
improve scientifically-based courses and course materials which enact instruction and support instructors Provide open access to these courses and materials Develop communities of use, research and development that enable evaluation and continuous improvement

Introduction: Outcomes Shared understanding of
challenges, tensions and possibilities in learning analytics, around the dimensions of: • Potential of well-used OER in a use-driven design context • Adaptability (Variety)← → Analytics (Coherence) • Analytics Tools and Approach • Data—needs and challenges Describe community-based analytics plans: • Flexible, long-range planning • Useful, short-term steps Commit to action • Identify best existing efforts

Driving Feedback Loops

Infinite Points of Light

Infinite Points of Light

Infinite Points of Light

Infinite Points of Light

Infinite Proliferation The 4 R’s
Reuse Redistribute Revise Remix

Infinite Proliferation The 4 R’s
NOT: Reuse Recreate Redistribute Revise Remix Add: Evaluate

Proliferation isn’t just OER… Intro
to CS @ CMU Statistics @ everywhere Core Statistics Business Statistics Research Statistics Medical Statistics

What drives change in these
scenarios? • Data • Intuition • Market demand • Instructor preferences

The problems of variety •
Quality is highly variable • Much duplication of effort • Difficult to choose appropriately • Hard to evaluate • Impossible to improve • Hard to scale success up

Effectiveness
is hit or miss

Effectiveness What is working in
open education? Why? And how do you know?

Effectiveness Demonstrably support students in
meeting articulated, measurable learning outcomes in a given set of contexts

So why don't we do
this now? • It's hard • It's expensive • Individual faculty can't do it alone • It can be threatening to educators • Disparate systems • How do we measure it? We need enabling processes and systems

Driving Feedback Loops

Great, but: What does it
mean when we get out of the realm of discussion and into the realm of practice? Learning Analytics What are they? How do we create and use them?

What do we mean by
learning analytics? Proxies vs authentic assessment and evaluation

Analytics Definition Data Collection 
Reporting  Decision Making  Intervention  Action Collecting the data is not enough. We also need to make sense of if in ways that are actionable.

Types of analytics • Educational/Academic
Management analytics • Classroom Management analytics • Learning Outcomes analytics

The problem of data collection
1. Agreed upon standards 2. Core collection 3. Space for exploration

The problem of data collection
1. Agreed upon standards 2. Core collection 3. Space for exploration • Ownership • Privacy • Policy

Ideal world •Common data standards
•Analytics-enabled OER •Commonly accepted ownership and privacy approaches •Commitment to measuring effectiveness through assessment

Bring Together What Already Works
1) Data Collection Systems Data Schemas 2) Communities of Evidence 3) Analysis Tools

Learning Dashboard

DataShop

Evidence Hub

Learning Registry

Communities of Evidence

And build new things 1)
Data Collection Systems Data Schemas 2) Communities of Evidence 3) Analysis Tools

Driven by different types of
data Metadata Paradata Synthetic Data Contextual Behavioral Interaction Semantic Raw

Share Alike and Share Data

Community Based Approach

A middle ground? Infinite The
One Variety True Course Communities Coalesce

Can we put these together?
"Full spectrum" analytics to drive different types of decision making, address different feedback loops

Learning Intelligence Systems

What would we be giving
up? This approach forces us to allow our minds to be changed by evidence.

Conclusion: next steps • Innovate
• Standardize • Scale

Conclusion: next steps • Innovate
• Commitment to Assessment • Standardize and Evaluation • Scale • Community Definition of Analytics-enabled OER • Common approach to data • Shared and private analytics platforms

“Improvement in Post Secondary Education
will require converting teaching from a „solo sport‟ to a community based research activity.” —Herbert Simon

Questions • Do you believe
in this approach to analytics-enabled OER? • Can this better address the pedagogy vs. reuse value curve?

A Virtuous Cycle Educational Technology
Data & Practice Theory