3. How do students learn?
What makes a great course?
What courses should we have?
How should we promote our courses?
How are students making their decisions?
Lots of Data Challenges
4. Define standard metrics
Make data easily accessible
Empower and incentivize people
Keys to building a data-driven organization
5. course quality is a constant feedback loop
Measure student feedback
course attributes
Identify which course attributes
are related to positive /
negative student
feedback?
Adjust
standards for publication
guidelines for instructors
feedback pre/post-publication
Udemy is the leading marketplace for online learning and teaching with 5M students, 20,000 courses, 10,000 instructors
Operates on the lifelong learning segment - different from K-12, higher ed. Lifelong learning = Just in Case & Just in Time
content agnostic - offers courses in app development, photography, negotiation etc.
analysis = mix quantitative & qualitative: 1) curate tags, 2) variable selection, 3) curate results
non-linear effects (e.g. audio quality) - utility vs. delight factors
correlation vs. causation
what to do next??
define standard metrics: agreed-upon metrics that drive business decisions; setup data collection if necessary
single source of truth; accessible; summary tables; dumb it down
give access to everyone; train to fish not to ask for data; setup performance goals tied to KPIs
Quality is a constant loop
Does not stop at establishing standards
Big Q: what do you base quality on??
