Adaptive learning systems (ALSs) aim to provide an efficient, effective and customised learning experience for students by dynamically adapting learning content to suit their individual abilities or preferences. Despite consistent evidence of their effectiveness and success in improving student learning over the past three decades, the actual impact and adoption of ALSs in education remain restricted to mostly research projects. In this paper, we provide a brief overview of reflections and lessons learned from developing and piloting an ALS in a course on relational databases. While our focus has been on adaptive learning, many of the presented lessons are also applicable to the development and adoption of educational tools and technologies in general. Our aim is to provide insight for other instructors, educational researchers and developers that are interested in adopting ALSs or are involved in the implementation of educational tools and technologies.

1. Development and Adoption of an Adaptive Learning System
Development and Adoption of an Adaptive Learning
System: Reflections and Lessons Learned
Dr Hassan Khosravi
The University of Queensland
Brisbane, QLD, Australia
h.khosravi@uq.edu.au
Professor Shazia Sadiq
The University of Queensland
Brisbane, QLD, Australia
shazia@itee.uq.edu.au
Professor Dragan Gasevic
Maonash University
Clayton, Vic, Australia
Dragan.Gasevic@monash.edu

2. Development and Adoption of an Adaptive Learning System 2
Overview
Challenges in delivering learner-centered learning at scale Adaptive learning systems provide a potential solution.
But they are expensive to purchase or develop.
RiPPLE: A discipline-agnostic cost-effective crowdsourced
adaptive learning system
Reflections and lessons learned from development and
adoption of RiPPLE.

3. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 3
Developing Cost-Effective ALSs
Adaptive Learning Systems (ALSs)
Reflections and Lessons Learned
Conclusion
The RiPPLE Platform

4. Development and Adoption of an Adaptive Learning System 4
Adaptive Learning Systems
Adaptive Learning Systems make use of data about students, learning processes,
and learning products to provide an efficient, effective and customised learning
experience for students by dynamically adapting learning content to suit their
individual abilities or preferences.
Domain
Model
Learner
Model
Content
Repository
Adaptation Engine

5. Development and Adoption of an Adaptive Learning System 5
Effectiveness of Adaptive Learning Systems
This review found that the effect size of human tutoring is d = 0.79
and the effect size of intelligent tutoring systems was 0.76, so they
are nearly as effective as human tutoring
A meta-analysis of 107 studies on ITSs involving 14,321 participants
found that: ITS were associated with higher achievement relative to
teacher-led large-group instruction, non-ITS computer based-
instruction, and texbooks or workbooks.
This paper conducted a study on 3422 students from 198 offerings
that have used ALEKS reporting significantly higher pass rates
amongst students using ALEKS. Yilmaz
The meta-analysis indicated that Intelligent Tutoring Systems
produced a large effect size on reading comprehension when
compared to traditional instruction (0.86)

6. Development and Adoption of an Adaptive Learning System 6
Developing Adaptive Educational Systems
Publisher Model: designed with pre-existing
content.
• Examples: Pearsons MyLabs, McGraw-Hills
LearnSmart and ALEKS
• Successful in K12 where content is standardized.
• Expensive to use
Platform Model: provides a content-agnostic
system infrastructure that enables instructors to
develop content.
• Examples: Smart Sparrow, Desire2Learn
and edX incorporate adaptive
• Introduce significant overhead for
instructors
• 25 hours of an expert's time for each hour of adaptive instruction (Aleven et al., 2006).
• Both types are very expensive to develop /purchase and challenging to scale across
different domains.

7. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 7
Adaptive Learning Systems (ALSs)
Reflections and Lessons Learned
Conclusion
The RiPPLE Platform
Developing Cost-Effective ALSs

8. Development and Adoption of an Adaptive Learning System 8
Developing Cost-Effective Adaptive Learning Systems

9. Development and Adoption of an Adaptive Learning System 9
Successful Crowdsourcing Stories Outside of Education
Crowdsourcing Knowledge Crowdsourcing information
Crowdsourcing Service Crowdsourcing micro tasks
Crowdsourcing answers
Crowdfunding

10. Development and Adoption of an Adaptive Learning System 10
Successful Crowdsourcing Stories in Education
PeerWise: Crowdsourcing learning resources
Peer grading and evaluation system Crowdy: Interactive, Collaborative, Crowd-
powered Video Learning
Piazza: crowdsources answers
in discussion forums

11. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 13
Adaptive Learning Systems (ALSs)
Reflections and Lessons Learned
Conclusion
Developing Cost-Effective ALSs
The RiPPLE Platform

12. Development and Adoption of an Adaptive Learning System 12
The RiPPLE Platform

13. Development and Adoption of an Adaptive Learning System 13
The RiPPLE Platform
Match students for
peer study sessions
Provide clicker-based
in-class activity
Free and open access to use
See ripplelearning.org

14. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 13
Adaptive Learning Systems (ALSs)
The RiPPLE Platform
Conclusion
Developing Cost-Effective ALSs
Reflections and Lessons Learned

15. Development and Adoption of an Adaptive Learning System 15
Reflections and Lessons Learned

16. Development and Adoption of an Adaptive Learning System 16
1. Harness the Creativity Power of Your Students
Question writing frequency correlated most strongly with summative performance
(Spearman's rank: 0.24, p=<0.001). Only two questions of the 300 'most-answered'
questions analysed had an unacceptable discriminatory value (<0.2)

17. Development and Adoption of an Adaptive Learning System 17
1. Harness the Creativity Power of Your Students
“60% of all explanations classified as being of high or outstanding
quality. Overall, 75% of questions met combined quality criteria”
“People with greater expertise tend to make assumptions about
student learning that turn out to be in conflict with students’ actual
performance and developmental propensities.”
“Crowdsourcing can efficiently yield high-quality assessment items
that meet rigorous judgmental and statistical criteria. ”
Overall, 86% of the examples classified in this multiple cohort
investigation were found to be a ‘High quality question’ by being
coherent, correct, requiring more than just simple factual recall,
possessing a valid solution and reasonable distractors.

18. Development and Adoption of an Adaptive Learning System 18
• Incentivizing students to “eat their broccoli” is challenging!
1. Harness the Creativity Power of Your Students
Frequent reminder of the benefits Inclusion of the activity as part of their
assessment and bonus marks
Availability of gamification aspects Co-development of sample solutions to
questions from past exams

19. Development and Adoption of an Adaptive Learning System 19
2. Provide Rich and Transparent Models of the Learners
(1) Interpretation: easier understand and interpret recommendations
(2) Rationale: easier to understand the rationale behind recommendations
(3) Trust: increases trust in the recommendations
(4) Motivation: increases motivation to study or further use the platform

20. Development and Adoption of an Adaptive Learning System 20
3. Put Instructors in Charge
Provide flexibility in use Adopting co-creation strategies to develop the
platform and features.
Providing small workshops and online help
center.
Comprehensive set of reports to provides rich
analytics on students

21. Development and Adoption of an Adaptive Learning System 21
4. Support Ethical Low Cost Empirical Educational research
Develop guidelines
referring to consent,
transparency and
benevolence
Develop mechanisms
for instructors to run
controlled
experiments.
Provide access to rich
analytics to support
observational studies.
Develop mechanisms for
instructors to run quasi-
experiments.

22. Development and Adoption of an Adaptive Learning System 22
5. Provide Mechanisms to Motivate Students to be Actively Engaged
Leaderboards Awards
Rating Systems

23. Development and Adoption of an Adaptive Learning System 23
6. Usability, Flexibility and Scalability
Usabiity Flexibility Scalability
1. Observing new users
interacting with the
system.
2. Holding focus groups with
students and instructors.
3. Addition of feedback and
survey
4. Adding Google Analytics
to capture users’
interactions and behavior.
1. Started by hosting locally
2. Running stress test
3. Be cloud ready

24. Development and Adoption of an Adaptive Learning SystemDevelopment and Adoption of an Adaptive Learning System 13
Adaptive Learning Systems (ALSs)
The RiPPLE Platform
Reflections and Lessons Learned
Developing Cost-Effective ALSs
Conclusion

25. Development and Adoption of an Adaptive Learning System 25
Conclusion
Educational technologies can help with delivering
learner-centered learning at scale
RiPPLE: A discipline-agnostic cost-effective crowdsourced
adaptive learning system
Reflections and lessons learned from Development and
adoption of RiPPLE.
Dr Hassan Khosravi
The University of Queensland
Brisbane, QLD, Australia
h.khosravi@uq.edu.au