While team work is intrinsic to authentic, collaborative learning in higher education, every student and academic has a horror story to tell about group dysfunction and social loafing. Dysfunctional teams often become apparent too late to rectify the problem. We will showcase a multifaceted approach to supporting students in complex team projects in two large (1200+) First-Year Engineering design and build courses at the University of Queensland. The PETS (Proactively Ensuring Team Success) process systematically utilises learning analytics from in-house LTI tools and Blackboard Learn for strategic intervention to support student teams. Delivered at Innovate and Educate: Teaching and Learning Conference by Blackboard. 24 -27 August 2015 in Adelaide, Australia.

1. PETS - Proactively ensuring team success
Learning analytics aligned with learning design
Authors: Lydia Kavanagh, Carl Reidsema, Esther Fink
Marnie Holt

2. Teamwork and communication are critical
2

3. PETS - a systematic approach to team learning
• a multifaceted approach to
creating effective, productive
and happy student teams.
• tested and evaluated and
continuously improved
since 2002
3
Kavanagh, Lydia, Neil, David and Cokley, John (2011)
Developing and disseminating team skills capacities using interactive online tools
for team formation, learning, assessment and mentoring:
Final report: The Australian Learning and Teaching Council

4. PETS process components + ALTC project aims
4
Author/s: Lydia Kavanagh , David Neil, John Cokley
Lead Institution: The University of Queensland

5. How do analytics fit in?
• Personalised notifications +
reminders (don’t fall behind)
• Constructive, just in time support
• Streamlined, targeted effort
(focus and impact)
• Encourage students to set goals,
self- monitor, reflect
• Visualise progress
5

6. Weaving a support net
• Emphasis: a systemic approach
• Track student progress based on
data from a variety of sources
• Targeted personalised support
• Moving from deficit-based to a
strength-based perspective
6

7. It all starts here – Assessment Design
• Focus on
process not
outcome
• Assessment
aligned with
milestones
7

8. Project features
8
• Individual + shared parts
• Able to be split into tasks
• Enough to do for everyone
Assessment type
Explicit training
Project features
Group allocation

9. Setting teams up for success
9
• A fair chance to succeed
• Team allocation (Team Anneal)
based on specified constraints
e.g. Belbin skills inventory, free-rider/ social
loafer, minority of ESL students, gender
• Import team allocations into Blackboard
groups from csv file via Bulk Group
Manager (BGM) bulding block
Assessment type
Explicit training
Project features
Group allocation

10. Explicit training: Working in Teams
10
• Interactive online learning module
• scheduled for redevelopment
(MOOC /UQx)
Assessment type
Explicit training
Project features
Group allocation

11. Explicit training: In-class workshop
11
• Reflect on Team Roles,
strengths and
weaknesses (Belbin)
• Discuss potential
conflict / team
strategies
• Sign a team charter
(goals, rules etc. project
management/ planning)

12. Preventive action at scale
The challenge:
1100+ students and 185+ teams
12
Prevent team dysfunction by:
•identification of teams with conflict
•identification of social loafing
•identification of leaderless teams
•remedial action through mentoring

13. Peer Evaluation (formative)
• Peer Assessment
with PAF tool (LTI)
• Self Evaluation in relation to
Peer Evaluation
• Indicator for group dysfunction
and social loafing
• adjusted group mark to recognise
individual effort
• Varied staff feedback
Tutor / Mentor / Project leader
aligned with design process
13
Reflection
Peer
Feedback
Self-
Evaluation

14. What’s a PAF?
PAF >1.2 Potential tem dysfunction / “Super leader”
1.1 < PAF < 1.2 Leader
PAF = 1.0 Average team member
0.9 < PAF < 1.0 Slightly under par – team probably taken leader’s points
from here
0.7 < PAF < 0.9 Social loafer
PAF < 0.7 Student in danger of failing/ dropping the course/
alienating team
14
SAPA – Self assessment/Peer Assessment

15. Beyond number crunching…
• Moderation is essential.
Many students use the system and
their input can skew results.
•All students should complete
the assessment. We recommend
making it compulsory.
•Only feedback PAF and SAPA.
Comments +individual scores
should be anonymous.
15

16. Mentoring (and monitoring)
• Discuss problems/ strategies
with teaching team (staff)
• Facilitate team meetings - 20 – 60mins
• Team, time, technical
• Dysfunctions
• Ways forward …
• Monitor teams/ students ‘at risk’
16
Mentor
feedback

17. Peer Evaluation (summative)
• Your criteria:
• Quality of work
• Timeliness of work
• Overall input to project
• Teamwork/ leadership
• …
• Divide 100 points between
members.
• Calculate PAF –
apply to team mark.17
Reflection
Peer
Feedback
Self-
Evaluation

18. eLIPSE
18
EAIT
Sci
ITaLi
ITS
Institutional
Support/Faculty
Consolidation
Prototyping –
efficient translation/pipelining
of matured tools for wider impact
Centre for eLearning and Innovation Partnerships in Science and Engineering
• Faculties of Science,
Engineering,
Architecture and IT
• Institute for Teaching
and Learning
Innovation (ITaLI)
• Information
Technology Services
(ITS)

19. What else?
• Gap-testing
• non-assessable
competencies
quiz for
incoming First
Year students
19

20. Dashboard: What you did /How you did
• Student Facing
• Student participation
– What do they have to say? (Student Voice)
– What are they thinking?
(Reflections/Semantic Analyses)
– What are they doing? (Student interaction,
attendance, participation)
• Engagement as evidence
20