PISA 2015 Results (Volume V): Collaborative Problem Solving, is one of five volumes that present the results of the PISA 2015 survey, the sixth round of the triennial assessment. It examines students’ ability to work with two or more people to try to solve a problem. The volume provides the rationale for assessing this particular skill and describes performance within and across countries. In addition, it highlights the relative strengths and weaknesses of each school system and examines how they are related to individual student characteristics, such as gender, immigrant background and socio-economic status. The volume also explores the role of education in building young people’s skills in solving problems collaboratively.
3. The kind of things that
are easy to teach are
now easy to automate,
digitize or outsource
35
40
45
50
55
60
65
70
1960 1970 1980 1990 2000 2006 2009
Routine manual
Nonroutine manual
Routine cognitive
Nonroutine analytic
Nonroutine interpersonal
Mean task input in percentiles of 1960 task
4. Collaborative problem-solving skills vary across countries,
and are not an automatic by product of disciplinary knowledge
Individual skills explain less than two-thirds of the variation in student
performance on the PISA collaborative problem-solving scale; and only three
quarters of the performance differences among countries on this measure are
explained by the relative standing of countries on the 2012 PISA assessment of
individual problem-solving skills.
5. Singapore
Japan
Hong Kong
(China)Korea
EstoniaCanada FinlandMacao (China)
New Zealand Australia
Chinese Taipei Germany
United StatesDenmark United Kingdom
Netherlands
Sweden Austria
Norway Slovenia Belgium
Czech RepublicIceland PortugalB-S-J-G (China) Spain
France Luxembourg
Latvia
Italy
CroatiaRussia Hungary
Israel Lithuania
Slovak Republic
GreeceChile
Bulgaria
Uruguay Costa Rica
Thailand United Arab
Emirates
Mexico
Colombia
Turkey
Peru
MontenegroBrazil
Tunisia380
400
420
440
460
480
500
520
540
560
Mean score
Figure V.3.3
Mean performance on the PISA
collaborative problem-solving scale
PISA 2015 defines collaborative
problem-solving competency as
the capacity of an individual to
effectively engage in a process
whereby two or more agents
attempt to solve a problem by
sharing the understanding and
effort required to come to a
solution and pooling their
knowledge, skills and efforts to
reach that solution.
7. All countries can make headway
The share of top performers is limited
8. Percentage of low-achieving students and top performers in
collaborative problem solving
Table V.3.1
0
10
20
30
40
50
60
70
80
90
100
Singapore
NewZealand
Canada
Australia
Finland
Japan
UnitedStates
HongKong(China)
Germany
Estonia
UnitedKingdom
Macao(China)
Korea
Netherlands
ChineseTaipei
Sweden
Austria
Denmark
OECDaverage
Belgium
Norway
Luxembourg
France
Iceland
B-S-J-G(China)
Slovenia
Israel
CzechRepublic
Portugal
Spain
Italy
Latvia
Russia
Hungary
SlovakRepublic
Lithuania
Croatia
Bulgaria
Greece
UnitedArabEmirates
Uruguay
Chile
Thailand
Brazil
Colombia
CostaRica
Peru
Mexico
Montenegro
Turkey
Tunisia
Students at Level 4
0
10
20
30
40
50
60
70
80
90
100
Students below Level 2
%
%
An average of only 8% students can solve problem-solving tasks with
fairly high collaboration complexity, maintaining awareness of group
dynamics and taking initiative to overcome obstacles and resolve
disagreements and conflicts
Students below Level 2 can at best complete tasks with low problem
difficulty and limited collaboration complexity. They tend to focus on their
individual role within the group, but with support from team members.
9. Boys are lagging behind
When individual problem-solving skills were at the centre of
PISA in 2012, boys scored higher in most countries. In
contrast, on the 2015 assessment of collaborative problem-
solving girls outperformed boys in in every country
13. Gender differences in performance are
mirrored in attitudes towards collaboration
Girls report more positive attitudes towards relationships, meaning that
they tend to be interested in others’ opinions and want others to
succeed. Boys, on the other hand, are more likely to see the instrumental
benefits of teamwork and how collaboration can help them work more
effectively and efficiently
14. -6
-4
-2
0
2
4
6
I am a good
listener
I enjoy seeing
my classmates
be successful
I take into
account what
others are
interested in
I enjoy
considering
different
perspectives
I prefer working
as part of a
team to working
alone
I find that teams
make better
decisions than
individuals
I find that
teamwork raises
my own
efficiency
I enjoy co-
operating with
peers
Percentage-pointdifference
(boysminusgirls)Gender differences in attitudes towards collaboration
Figure V.5.5
Items comprising the index of valuing relationships Items comprising the index of valuing teamwork
Boys are more likely to
value teamwork
Girls are more likely to
value relationships
15. Attitudes towards collaboration
vary across countries too
If schools foster boys’ appreciation of others and their
interpersonal friendships and relationships, then they may also
see better outcomes among boys in collaborative problem-solving
19. 0
5
10
I am a good
listener
I take into
account what
others are
interested in
I enjoy
considering
different
perspectives
I enjoy seeing
my classmates
be successful
I find that teams
make better
decisions than
individuals
I enjoy co-
operating with
peers
I prefer working
as part of a
team to working
alone
I find that
teamwork raises
my own
efficiency
Score-pointdifference
After accounting for gender and students' and schools' socio-economic profile
Before accounting for gender and students' and schools' socio-economic profile
Attitudes towards collaboration and
relative performance in collaborative problem solving
Figure V.5.8
Higher performance among students who agreed
with any of these statements, even after accounting
for performance in the three core subjects, gender
and students’ and schools’ socio-economic profile
Items comprising the index of valuing relationships Items comprising the index of valuing teamwork
20. Taking into account others’ interests and
relative performance in collaborative problem solving
Figure V.5.9
-5
0
5
10
15
20
25
30
Estonia
NewZealand
Russia
Norway
UnitedKingdom
Belgium
Spain
Canada
CostaRica
Bulgaria
Greece
Italy
Chile
Slovenia
Portugal
Sweden
Finland
Netherlands
OECDaverage
SlovakRepublic
CzechRepublic
Luxembourg
Thailand
B-S-J-G(China)
Japan
Macao(China)
Iceland
Australia
Montenegro
Denmark
Lithuania
Malaysia
Uruguay
Tunisia
Brazil
Turkey
France
Singapore
Korea
UnitedArabEmirates
Germany
Hungary
HongKong(China)
Croatia
Mexico
Austria
Latvia
UnitedStates
Peru
ChineseTaipei
Israel
Colombia
Score-pointdifference
After accounting for gender and students' and schools' socio-economic profile
Before accounting for gender and students' and schools' socio-economic profile
Higher performance among students who agreed/strongly
agreed that they take others’ interests into account, even after
accounting for performance in science, reading and mathematics
21. Finding that teams make better decisions and
relative performance in collaborative problem solving
Figure V.5.10
-10
-5
0
5
10
15
Croatia
Portugal
NewZealand
CzechRepublic
Korea
Norway
Greece
Australia
Slovenia
Russia
Estonia
Sweden
Singapore
Uruguay
Japan
CostaRica
Chile
B-S-J-G(China)
Spain
Canada
SlovakRepublic
UnitedStates
Hungary
OECDaverage
Belgium
Denmark
Montenegro
Finland
Austria
UnitedKingdom
Malaysia
Bulgaria
Brazil
Lithuania
Colombia
Mexico
Macao(China)
Italy
France
Iceland
Latvia
Luxembourg
Thailand
HongKong(China)
ChineseTaipei
Turkey
Netherlands
Germany
Peru
Israel
UnitedArabEmirates
Tunisia
Score-pointdifference
After accounting for gender and students' and schools' socio-economic profile
Before accounting for gender and students' and schools' socio-economic profile
Higher performance among students who
agreed that teams make better decisions, even
after accounting for performance in science,
reading and mathematics
22. Learning environments can shape
attitudes and outcomes in collaboration
PISA asked students about how often they engage in communication-
intensive activities such as explaining one’s ideas in science class;
spending time in the laboratory doing practical experiments; arguing
about science questions; and taking part in class debates about
investigations. The results show a clear relationship between these
activities and positive attitudes towards collaboration
23. -0.30
-0.20
-0.10
0.00
0.10
0.20
0.30
0 1 2 3 4 5 6 7
Mean index All students Boys Girls
Physical exercise and index of valuing relationships, by gender
Figure V.6.3
-0.30
-0.20
-0.10
0.00
0.10
0.20
0.30
0 1 2 3 4 5 6 7
Mean index
Days per week of moderate physical activity Days per week of vigorous physical activity
24. -0.20
-0.10
0.00
0.10
0.20
0 1 2 3 4 5 6 7
Mean index All students Boys Girls
Physical exercise and index of valuing teamwork, by gender
Figure V.6.3
-0.20
-0.10
0.00
0.10
0.20
0 1 2 3 4 5 6 7
Mean index
Days per week of moderate physical activity Days per week of vigorous physical activity
25. -0.30
-0.25
-0.20
-0.15
-0.10
-0.05
0.00
Skipping a whole
day of school
Skipping some
classes
Arriving late for
school
Skipping a whole
day of school
Skipping some
classes
Arriving late for
school
Changeinindex
After accounting for gender and students' and schools' socio-economic profile
Before accounting for gender and students' and schools' socio-economic profile
Student truancy and attitudes towards collaboration
Figure V.6.7
Index of valuing relationships Index of valuing teamwork
Students who play truant are more likely
to show negative attitudes
towards collaboration
26. 0
1
2
3
I am a good
listener
I enjoy seeing
my classmates
be successful
I take into
account what
others are
interested in
I enjoy
considering
different
perspectives
I prefer working
as part of a
team to working
alone
I find that teams
make better
decisions than
individuals
I find that
teamwork raises
my own
efficiency
I enjoy co-
operating with
peers
Percentage-pointdifference
After accounting for gender and students' and schools' socio-economic profile
Before accounting for gender and students' and schools' socio-economic profile
Student interaction in science class
and attitudes towards collaboration
Figure V.6.9
Items comprising the index of valuing relationships Items comprising the index of valuing teamwork
Students who reported that more communication-intensive
activities take place in science class have more positive
attitudes towards collaboration
Students are given opportunities to explain their
ideas; students spend time in the laboratory
carrying out practical experiments; students are
required to argue about science questions; there
is a class debate about investigations
27. -8
-6
-4
-2
0
2
4
6
8
I take into
account what
others are
interested in
I enjoy
considering
different
perspectives
I am a good
listener
I enjoy seeing
my classmates
be successful
I find that
teamwork raises
my own
efficiency
I prefer working
as part of a
team to working
alone
I find that teams
make better
decisions than
individuals
I enjoy co-
operating with
peers
Percentage-pointdifference
(adv.minusdisadv.)Disadvantaged students see the value of teamwork often more clearly
than their advantaged peers
Figure V.5.6
Items comprising the index of valuing relationships Items comprising the index of valuing teamwork
Advantaged students are more likely
to value relationships
Disadvantaged students are more
likely to value teamwork
31. Difference in relative performance in collaborative problem solving
between the top and bottom quarters of the concentration
of immigrant students in school
Table V.4.22
-10
0
10
20
30
40
50
Israel233
UnitedArabEmirates362
Russia015
UnitedStates141
Sweden131
Luxembourg2567
Portugal017
Italy017
Germany133
Canada151
Spain025
Macao(China)4076
Croatia121
Austria234
NewZealand544
OECDaverage123
HongKong(China)1351
France028
Australia043
Slovenia018
Singapore729
Netherlands023
Greece022
CostaRica019
Belgium134
Norway123
Denmark021
UnitedKingdom032
Estonia024
Score-pointdifference
After accounting for gender, and students' and schools' socio-economic profile
Before accounting for gender, and students' and schools' socio-economic profile
Non-immigrant students attending schools with more immigrant students perform
better in collaborative problem solving,
even after accounting for performance in science, reading and mathematics
Topquarter(%)
Bottomquarter(%)
32. Looking beyond school walls
Only a quarter of the performance variation in
collaborative problem-solving skills lies between schools,
much less than is the case in the school disciplines
33. Schools differ less in their performance on collaborative problem-
solving than in performance in academic disciplines
Figure V.4.1
120
100
80
60
40
20
0
20
40
60
Israel122
Bulgaria106
Hungary100
B-S-J-G(China)104
UnitedArabEmirates99
Netherlands103
Belgium107
Austria107
Germany113
Slovenia95
Italy102
SlovakRepublic95
Peru77
Turkey67
Luxembourg110
CzechRepublic91
Thailand77
Brazil84
Uruguay91
Lithuania91
Greece94
Croatia84
Singapore103
OECDaverage100
ChineseTaipei90
Colombia76
Japan79
Chile78
HongKong(China)90
Australia126
UnitedKingdom117
UnitedStates129
Mexico69
Russia94
Macao(China)88
Portugal92
Canada120
NewZealand124
Korea78
Estonia90
CostaRica67
Sweden107
Montenegro69
Tunisia38
Denmark90
Latvia89
Spain86
Norway97
Finland114
Iceland99
% Between-school variation Within-school variation
Total variation as a
proportion of the OECD
average
OECD average 75%
OECD average 24%
36. -40
-30
-20
-10
0
10
Playing video games Meeting friends/talking to friends
on the phone
Working in the household or
taking care of
other family members
Accessing the
Internet/chat/social networks
Score-pointdifference
After accounting for performance in the three core PISA subjects, gender, and students' and schools' socio-economic profile
Before accounting for performance in the three core PISA subjects, gender, and students' and schools' socio-economic profile
Activities before and after school, and performance in
collaborative problem solving
Figure V.6.5
Students who access the internet/chat/use social networks
score higher in collaborative problem solving
Students who play video games score lower in collaborative
problem solving
40. • Strong academic skills do not automatically translate into strong social skills
• Some countries do much better in collaborative problem-solving than their science,
math and reading performance predicts
• Only 8% of students can solve tasks with fairly high collaboration complexity (and
even in top performer Singapore it is only 21%)
• While boys did better in individual problem-solving, girls do better in collaborative
problem-solving in every country, and gender differences in collaborative problem-
solving are mirrored in attitudes towards collaboration
• Learning environments relate to attitudes in collaboration and collaborative skills
• Disadvantaged students see the value of teamwork often more clearly than their
advantaged peers and exposure to diversity tends to be positively related with
collaboration skills
• Frequent playing of video games relates negatively to collaborative problem-solving,
but internet use, chatting and social networks do not
Some key findings
41. • Use the whole range of the curriculum to foster collaboration
• Foster more positive relationships at school and designing learning
environments that benefit students’ collaborative problem-solving
skills and their attitudes towards collaboration.
• Give students ownership over the time, place, path, pace and
interactions of their learning
• Enhance social activities that foster constructive relationships and
school attachment, teacher training on classroom management, a
whole-of-school approach to prevent and address bullying
• Foster parental engagement
Some policy implications
42. 74
74 Thank you
Find out more about our work at www.oecd.org
– All publications
– The complete micro-level database
Email: Andreas.Schleicher@OECD.org
Twitter: SchleicherEDU
and remember:
Editor's Notes
Half of the jobs in the industrialised world are potentially automatable, because the things that are easy to teach and easy to test are also the things that are easy to automate, digitize and outsource.
Labels are performance in collaborative problem solving
In Tunisia, Montenegro, Turkey, Peru, Colombia, Mexico, Thailand, Costa Rica: % of top performers is 0% or near to 0%.
All differences are statistically significant
All differences are statistically significant
Differences expressed as a percentage of the within-country standard deviation in performance
Dark blue diamonds refers to significant gender differences in collaborative problem solving and in science.
In all countries, girls scored significantly higher in collaborative problem solving.
In science, statistically significant gender differences are marked in a darker tone.
In all countries, girls scored significantly higher in collaborative problem solving.
In science, statistically significant gender differences are marked in a darker tone.
So in the figure, dark blue diamonds refers to significant gender differences in collaborative problem solving and in science.
All differences are statistically significant
Goes with next slide : 1) performance 2) relative performance
Same ranking as previous slide, but shows difference in relative performance
All differences for after accounting for gender and students' and schools' socio-economic profile are statistically significant.
All differences are statistically significant.
The index of student interaction in science class is the sum of students' responses to questions about whether their science teachers use the following teaching practices in all lessons or in most lessons: students are given opportunities to explain their ideas; students spend time in the laboratory carrying out practical experiments; students are required to argue about science questions; there is a class debate about investigations. The index ranges from 0 to 4, with all responses weighted equally.
All differences are statistically significant
At the school level, the change in collaborative problem-solving score is per 10 percentage-point increase in the number of schoolmates who reported the above.
At the school level, the change in collaborative problem-solving score is per 10 percentage-point increase in the number of schoolmates who reported the above.
Only countries and economies where the percentage of immigrant students is higher than 6.25% in 2015 are shown.
Only countries and economies where the percentage of immigrant students is higher than 6.25% in 2015 are shown.
Only countries and economies where the percentage of immigrant students is higher than 6.25% in 2015 are shown.
Only countries and economies where the percentage of immigrant students is higher than 6.25% in 2015 are shown.
Countries and economies are ranked in descending order of the score-point difference in collaborative problem solving between first-generation immigrant students and non-immigrant students.
Only countries and economies where the percentage of immigrant students is higher than 6.25% in 2015 are shown.
Difference is immigrant – non-immigrant students, not the usual way!
The intra-class correlation is the variation in student performance between schools, divided by the sum of the variation in student performance between schools and the variation in student performance within schools, and multiplied by 100.
All values are statistically significant for science
All values are statistically significant for science
All differences before accounting for performance in the three core PISA subjects, gender, and students' and schools' socio-economic profile are statistically significant.
Note: UK includes only England, Northern Ireland and Wales
Likelihood that students whose index of self-reported ICT competence is below -1.00 are low performers in CPS compared to those whose index is above -1.00
Score-point difference between students whose index of self-reported ICT competence is above -1.00 and those whose index is below -1.00
Difference between schools in the top and bottom quartiles of the PISA index of economic, social and cultural status
Results based on school principals’ report.
At the school level, the change in collaborative problem-solving score is per 10 percentage-point increase in the number of schoolmates who reported the above.