This document discusses using data from formative assessments in team-based learning (TBL) to predict and optimize student outcomes. TBL involves individual and team quizzes with immediate feedback. Studies show that teams outperform individuals, and individuals retain team gains. Data can be used to adapt instruction, such as repeating difficult questions. It can also predict future performance based on past assessments. Overall, TBL generates data that suggests final individual performance is higher than initial team or individual performance.

1. Predictive and adaptive analytics using formative
assessment data in a blended learning classroom
The 2nd Active Learning Conference
University of Sussex
Brighton, United Kingdom
5 June 2018
1
Brian O’Dwyer
Embry-Riddle Aeronautical University Asia (Singapore Campus), Adjunct Faculty
CognaLearn, Executive Chairman and Commercial Founder
Duke-National University of Singapore Medical, Former Entrepreneur-in-Residence

2. Acknowledgements
2
▪ The presenter would like to acknowledge the contribution of Dr Timothy Wertz,
Lecturer at Yale-NUS College to the prediction of student outcomes case study.
▪ The presenter is the Commercial Founder of and has a financial interest in
CognaLearn. CognaLearn is the company that developed InteDashBoard™
www.intedashboard.com, which is TBL software developed in collaboration with
Duke-US Medical School; InteDashBoard™ is one of the technology tools
described in this presentation.
▪ A poster version of this presentation was presented at the 2018 Team-Based
Learning Collaborative Conference 2018, San Diego, United States
Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

3. Objectives
3
After this session participants should be able to:
1. Explain: who I am and why I am here
2. Define: team-based learning (“TBL”)
3. Describe: three examples of how TBL data can be
used to predict and optimize outcomes
Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

4. Team-based learning (“TBL”)
4Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
In class: apply
1. Pre-work 4. Clarify
doubts
2. Quiz 5. Team
applications
3. Team
quiz
In class: theory
Also 360° teammate evaluation

5. 1. Pre-work
5Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

6. 2. Individual Readiness Assurance Test
(“IRAT”)
6Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

7. 3. Team Readiness Assurance Test
(“TRAT”) with immediate feedback
7Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

8. 4. Clarifications
8Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

9. 5. Applications
9Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
Canberra
Sydney
Significant problem
Same problem
Specific choice
Simultaneous report

10. TBL generates data
10Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
1 2 3 4 5 6 7 8 9 10 11 12
Week
Each class will typically generate
Individual test data
Team test data
Team case data

11. TBL data examples
11
1. Individual versus Team versus Final
2. Prediction
3. Adaption
Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

12. Individual versus team: Michaelsen
12Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
Delayed feedback
(end of test):
cumulative team
score averaged 11%
higher than score of
best member
Immediate
feedback:
cumulative team
score averaged
23% higher than
score of best
member
Source: Larry Michaelsen, David Ross Boyd Professor
Emeritus of Management at the University of Oklahoma,
Founder of TBL. Personal communication.
Score of Best
Member
Team Score: Delayed
Feedback
Team Score:
Immediate Feedback
In a study of over 1,500 TEAMS…
+23%
+11%

13. 1. Teams outperform individuals
13Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
76%
Team TestIndividual Test
BEFORE
93%
+22%
Source: Embry-Riddle Aeronautical University, Bachelor of Science in Aviation Business course in Airport Administration and Finance

14. 2. Individuals retain team gains
14Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
76%
Team TestIndividual Test
BEFORE
93%
95%
Individual Final
Exam AFTER
Source: Embry-Riddle Aeronautical University, Bachelor of Science in Aviation Business course in Airport Administration and Finance

15. 3. High-low range narrows
15Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
Team TestIndividual Test
BEFORE
Individual Final
Exam AFTER
High: 80
Low: 54
High: 98 +23%
Low: 92 +70%
Source: Embry-Riddle Aeronautical University, Bachelor of Science in Aviation Business course in Airport Administration and Finance

16. Example 2: simple adaption
16Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
Density altitude
Density altitude Density altitude
Q5 Density
Altitude

17. Example 3: prediction
17Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
Predict final grades in three weeks13 weekly assessments
Source: Dr. Timothy Wertz; Lecturer, Yale-NUS College

18. Caveats
18
▪ Repeated testing impact
▪ Partial credit impact
▪ Small sample sizes
▪ Early stage of analysis
Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution

19. Summary
19
▪ Team-based learning (“TBL”):
type of blended learning with lots
of formative assessment data
▪ TBL data examples suggest:
1. Final > team > individual
2. Repeat hard questions
3. Past performance -> future
performance
Copyright Brian O’Dwyer and CognaLearn Pte Ltd and www.intedashboard.com where applicable.
Slides available from odwyerb@erau.edu and may be used with attribution
Learn more
Brian O’Dwyer
odwyerb@erau.edu
Team-Based Learning
Collaborative
www.teambasedlearning.org
TBL software
www.intedashboard.com