PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

1. OECD EMPLOYER BRAND Playbook 1 PISA 2012 Creative Problem Solving Students’ skills in tackling real-life problems 1 April 2014 Andreas Schleicher
2. 2 PISA in brief • Over half a million students… – representing 28 million 15-year-olds in 65 countries/economies – Schools and students randomly selected by OECD … took an internationally agreed 2-hour test… – Goes beyond testing whether students can reproduce what they were taught… … to assess students’ capacity to extrapolate from what they know and creatively apply their knowledge in novel situations – Mathematics, reading, science, problem-solving, financial literacy – Total of 390 minutes of assessment material … and responded to questions on… – their personal background, their schools and their engagement with learning and school • Parents, principals and system leaders provided data on… – school policies, practices, resources and institutional factors that help explain performance differences . …the capacity to engage creatively in cognitive processing to understand and resolve problem situations where a method of solution is not immediately obvious (including motivational and affective aspects). Problem Solving: 85 000 students in 44 countries/economies took an additional 40-min test
3. 35 40 45 50 55 60 65 70 1960 1970 1980 1990 2000 2006 2009 Routine manual Nonroutine manual Routine cognitive Nonroutine analytic Nonroutine interpersonal Mean task input in percentiles of 1960 task distribution 3 The case for creative problem-solving Trends in different tasks in occupations (United States) Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Le vy, and Murnane (2003)." MIT Mimeograph, June.
4. 5 TRAFFIC Problem Solving – Sample Question 1 Julio lives in Silver, Maria lives in Lincoln, and Don lives in Nobel. They want to meet in a suburb on the map. No-one wants to travel for more than 15 minutes. Where could they meet? This is an easy item – Level 1 on the problem-solving scale (below baseline) All information required is given at the outset: it is a static problem An embedded calculator ensures the item measures problem solving – not arithmetics This item focuses on students’ ability to monitor and reflect on solutions.
5. 6 TICKETS You plan to take four trips around the city on the subway today. You are a student, so you can use concession fares. Use the ticketing machine to find the cheapest ticket and press BUY. Once you have pressed BUY, you cannot return to the question; Problem Solving – Sample Question 2 This is a harder item – Level 5 on the problem-solving scale Students must engage with the machine, and use the feedback and information uncovered to reach a solution: it is an interactive problem This main demand is exploring and understanding (knowledge acquisition) Sample items can be tried at cbasq.acer.edu.au and www.oecd.org/pisa
6. 7 77 Performance in problem-solving How well do 15-year-olds engage creatively in cognitive processing to understand and resolve problem situations? • Exploring and understanding the information provided with the problem. • Representing and formulating: constructing graphical, tabular, symbolic or verbal representations of the problem situation and formulating hypotheses about the relevant factors and relationships between them. • Planning and executing: devising a plan by setting goals and sub-goals, and executing the sequential steps identified in the plan. • Monitoring and reflecting: monitoring progress, reacting to feedback, and reflecting on the solution, the information provided with the problem, or the strategy adopted.
7. SingaporeKorea Japan Macao-ChinaHong Kong-China Shanghai-ChinaChinese Taipei Canada AustraliaFinland England (U.K.)Estonia France NetherlandsItalyCzech RepublicGermany United States BelgiumAustriaNorway IrelandDenmark Portugal SwedenRussian Fed. Slovak RepublicPoland SpainSlovenia Serbia Croatia Hungary TurkeyIsrael Chile Brazil Malaysia U.A.E Montenegro UruguayBulgaria Colombia 390 400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 550 560 570 Mean score Strong performance in problem solving Low performance in problem solving Average performance of 15-year-olds in problem solving Fig V.2.3 8
8. 99 Excellence in education Top-performers in problem-solving
9. 1010 The rising demand for advanced skills -20 -15 -10 -5 0 5 10 15 20 25 % Evolution of employment in occupational groups defined by PIAAC problem-solving skills Employment of workers with advanced problem-solving skills Employment of workers with poor problem-solving skillsEmployment of workers with medium-low problem-solving skills (PIAAC) Source:PIAAC 2011
10. OECD EMPLOYER BRAND Playbook 12
11. 1414 Strengths and weaknesses in problem-solving Which countries have particular strengths in problem-solving ?
12. 200 300 400 500 600 700 800 200 300 400 500 600 700 800 Patterns of relative performance in problem solving Problem solving performance Mathematics performance Fig V.2.16 Fig V.2.17 Average relationship between problem solving and mathematics performance The United States and England (UK) perform better-than- expected in problem solving. The difference between observed and expected performance is larger among strong performers in mathematics Japan performs better-than-expected in problem solving. The difference between observed and expected performance is larger among low achievers in mathematics Poland’s performance is lower-than-expected in problem solving. The gap between observed and expected performance is similar at all levels of mathematics performance. 15 Spain’s performance is lower-than- expected in problem solving. The gap between observed and expected performance is wider among low achievers in mathematics. Singapore’s performance in problem solving is as high as expected at all levels of mathematics performance
13. Australia Brazil Macao-China England(U.K.) Italy UnitedStates Serbia Japan Korea Austria SlovakRepublic RussianFederation Portugal Sweden Canada CzechRepublic Chile Norway Singapore France Bulgaria Shanghai-China Poland UnitedArabEmirates Hungary Slovenia Israel Uruguay Montenegro Croatia Spain Ireland HongKong-China Netherlands Estonia Turkey Malaysia Germany Denmark Belgium ChineseTaipei Finland OECDaverage Colombia -60 -40 -20 0 20 40 % Relative performance in problem solving Fig V.2.15 Students' performance in problem solving is lower than their expected performance Students' performance in problem solving is higher than their expected performance 16
14. Strengths and weaknesses: interactive and static tasks Fig V.3.10 Better performance on static tasks Better performance on interactive tasks 17 United States Germany Austria France Japan Sweden Australia Israel Canada Ireland Belgium Norway Korea Italy Hong Kong-China Chinese Taipei Macao-China Singapore Shanghai-China Poland England Estonia Finland Slovak Rep. Czech Rep. Turkey Hungary Chile Netherlands Spain Denmark Slovenia Portugal Brazil Uruguay Croatia Bulgaria U.A.E. Montenegro Colombia Malaysia Serbia Russian Fed.
15. Strengths and weaknesses: knowledge-generation and knowledge-utilisation Fig V.3.10 United States England Germany Czech Rep. France Japan Australia Canada Ireland Chile Belgium Spain Portugal Korea Italy Brazil U.A.E. SingaporeColombia Poland Estonia Finland Slovak Rep. AustriaTurkey SwedenHungary Israel NetherlandsDenmark Slovenia Norway Hong Kong-ChinaUruguay Croatia Chinese Taipei Bulgaria Macao-China Montenegro Malaysia Serbia Russian Fed. Shanghai-China Better performance on knowledge- utilisation tasks Better performance on knowledge- generation tasks 18
16. Strengths and weaknesses Fig V.3.10 United States Poland England Estonia Finland Slovak Rep. Germany Austria Czech Rep. France Japan Turkey Sweden Hungary Australia Israel Canada Ireland Chile Belgium Netherlands Spain Denmark Slovenia Portugal Norway Korea Italy Hong Kong-China Brazil Uruguay Croatia Chinese Taipei Bulgaria Macao-China U.A.E. Montenegro Singapore Colombia Malaysia Serbia Russian Fed. Shanghai-China OECD average OECDaverage Better performance on interactive tasks Better performance on static tasks Better performance on knowledge- acquisition tasks Better performance on knowledge- generation tasks Stronger-than-expected performance on interactive items, weaker-than-expected performance on knowledge-acquisition tasks Stronger-than-expected performance on interactive items and on knowledge-acquisition tasks Weaker-than-expected performance on interactive items and on knowledge-acquisition tasks Weaker-than-expected performance on interactive items , stronger-than-expected performance on knowledge-acquisition tasks 19
17. 2020 Student resilience The country where students go to class matters more than what social class students come from
18. 2121 PISA mathematics performance by decile of social background 300325350375400425450475500525550575600625650675 Mexico Chile Greece Norway Sweden Iceland Israel Italy UnitedStates Spain Denmark Luxembourg Australia Ireland UnitedKingdom Hungary Canada Finland Austria Turkey Liechtenstein CzechRepublic Estonia Portugal Slovenia SlovakRepublic NewZealand Germany Netherlands France Switzerland Poland Belgium Japan Macao-China HongKong-China Korea Singapore ChineseTaipei Shanghai-China Source: PISA 2012
19. 0 5 10 15 20 25 30 Macao-China Canada HongKong-China Japan Norway Korea Estonia Italy Sweden Finland UnitedArabEmirates England(United… Spain Denmark Australia Croatia Netherlands ChineseTaipei Montenegro UnitedStates Ireland OECDaverage Austria Singapore Poland RussianFederation Slovenia Colombia France Germany Serbia Israel Belgium Shanghai-China Brazil CzechRepublic Malaysia Turkey Chile Portugal Uruguay Bulgaria Hungary SlovakRepublic Problem solving Mathematics Percentageofvariationinperformance explainedbysocio-economicstatus Relationship between socio-economic background and performance in problem solving and mathematics Fig V.4.9a 22
20. 0 10 20 30 40 50 60 70 80 Macao-China HongKong-China Singapore Korea Japan Shanghai-China ChineseTaipei Canada Italy Estonia Finland Australia England(UK) UnitedStates France Portugal Turkey Netherlands Belgium OECDaverage Spain CzechRepublic Austria Germany Norway Ireland Denmark Sweden Poland RussianFederation Serbia Croatia SlovakRepublic Brazil Slovenia Chile Hungary Colombia Israel Cyprus Malaysia Uruguay Montenegro U.A.E. Bulgaria % Percentage of ‘resilient’ students in problem solving Fig II.2.4 23 Socio-economically disadvantaged students not only score lower in problem solving, they also report lower levels of engagement, drive, motivation and self-beliefs. Resilient students break this link and share many characteristics of advantaged high-achievers. A resilient student is situated in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in the country of assessment and performs in the top quarter of students among all countries, after accounting for socio-economic status.
21. 0 10 20 30 40 50 60 70 Finland Norway Sweden Canada Denmark Netherlands Estonia Montenegro Ireland England(U.K.) Korea Serbia Japan ChineseTaipei Australia Singapore Poland CzechRepublic Croatia OECDaverage Italy RussianFederation Spain Slovenia Israel U.A.E. UnitedStates Macao-China Germany Belgium Turkey Malaysia Austria Portugal HongKong-China Shanghai-China SlovakRepublic Colombia Hungary Brazil Uruguay Bulgaria Chile Problem solving Mathematics PISA index of economic, social and cultural status (ESCS) Proportionofvariationbetweenschools asapercentageoftheoverall(withinandbetweenschool)variation Between-school differences in problem- solving, mathematics and socio-economic status Fig V.2.12 24
22. 2626 Country examples Developing creative problem-solving skills
23. Country examples • Involve employers and parents in developing a vision for education • Make problem-solving competence an overarching goal of the curriculum • Give every student a chance to engage in deep learning through meaningful projects • Support teachers to ensure that project time is learning time Embed learning of 21st century competencies and attitudes such as inquiry-based authentic learning in curricular subjects and co-curricular activities Clear articulation of desired student outcomes to guide schools’ and teachers’ efforts and ensure coherence and alignment of curriculum, pedagogy and assessment. Alberta’s Curriculum Redesign Project Singapore’s 21st Century Competencies Framework Japan’s Zest for Life approach
24. 2828Lessonsfromhighperformers Strong performers and successful reformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins
25. 2929Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence
26. 3030Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence  A commitment to education and the belief that competencies can be learned and therefore all children can achieve  Universal educational standards and personalization as the approach to heterogeneity in the student body… … as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity  Clear articulation who is responsible for ensuring student success and to whom
27. Students and perseverance Percentage of students who reported that the following statements describe someone "very much like me" or "mostly like me" (*) or "not much like me" or "not at all like me" (**) 0 10 20 30 40 50 60 70 Disagree: When confronted with a problem, I give up easily Disagree: I put off difficult problems Agree: I remain interested in the tasks that I start Agree: I continue working on tasks until everything is perfect Agree: When confronted with a problem, I do more than what is expected of me Singapore OECD average Fig III.3.2 31
28. -5 0 5 10 15 20 25 30 35 40 45 Finland Korea Norway NewZealand ChineseTaipei Iceland Sweden Qatar Australia Denmark Portugal U.A.E. France Greece UnitedKingdom Poland Japan Thailand Jordan SlovakRepublic Macao-China Ireland Canada Spain OECDaverage Germany Latvia HongKong-China UnitedStates Liechtenstein Luxembourg Hungary Shanghai-China Lithuania Austria Montenegro Bulgaria Tunisia Malaysia Switzerland Mexico Uruguay Peru Belgium Turkey Italy Singapore Chile CzechRepublic Romania Argentina Brazil Serbia Kazakhstan Slovenia RussianFed. Indonesia VietNam Colombia CostaRica Netherlands Croatia Estonia Israel Albania Score-pointdifference Score-point difference in mathematics associated with one unit of the index of perseverance Average student Change in performance per one unit of the index among lowest-achieving students Change in performance per one unit of the index among highest-achieving students Perseverant students perform better (mathematics)32 Fig III.3.3
29. Openness to problem solving Percentage of students who reported "agree" or "strongly agree" with the following statements: 0 20 40 60 80 100 I can handle a lot of information I am quick to understand things I seek explanation for things I can easily link facts together I like to solve complex problems % Poland Singapore OECD average Fig III.3.4 33
30. -10 0 10 20 30 40 50 60 Korea NewZealand Australia UnitedKingdom Finland Canada CzechRepublic Sweden Lithuania Ireland Denmark ChineseTaipei Norway France Austria Spain Estonia Portugal OECDaverage UnitedStates Latvia Macao-China Liechtenstein Shanghai-China Iceland HongKong-China Greece Slovenia Switzerland Hungary Japan Germany Luxembourg Chile Poland VietNam SlovakRepublic Singapore RussianFed. Italy Mexico Belgium Netherlands CostaRica Uruguay Croatia Turkey Israel Peru U.A.E. Serbia Tunisia Romania Jordan Argentina Bulgaria Malaysia Brazil Qatar Thailand Kazakhstan Indonesia Colombia Montenegro Albania Score-pointdifference Score-point difference in mathematics associated with one unit of the index of students' openness to problem solving Average student Change in performance per one unit of the index among lowest-achieving students Change in performance per one unit of the index among highest-achieving students Students open to problem solving perform better (math)34 Fig III.3.5
31. 3535Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence  Clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems  Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)  High level of metacognitive content of instruction …
32. 3636Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence  Capacity at the point of delivery  Attracting, developing and retaining high quality teachers and school leaders and a work organisation in which they can use their potential  Instructional leadership and human resource management in schools  Keeping teaching an attractive profession  System-wide career development …
33. 3737Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence  Incentives, accountability, knowledge management  Aligned incentive structures For students  How gateways affect the strength, direction, clarity and nature of the incentives operating on students at each stage of their education  Degree to which students have incentives to take tough courses and study hard  Opportunity costs for staying in school and performing well For teachers  Make innovations in pedagogy and/or organisation  Improve their own performance and the performance of their colleagues  Pursue professional development opportunities that lead to stronger pedagogical practices  A balance between vertical and lateral accountability  Effective instruments to manage and share knowledge and spread innovation – communication within the system and with stakeholders around it  A capable centre with authority and legitimacy to act
34. 3838Lessonsfromhighperformers38 School autonomy
35. 39 39 39 Hong Kong-China Brazil Uruguay Albania Croatia Latvia Lithuania Chinese Taipei ThailandBulgaria Jordan Macao-China UAE Argentina Indonesia Kazakhstan Peru Costa Rica Tunisia Qatar Singapore Colombia Malaysia Serbia Romania Viet Nam Shanghai-China USA Poland New Zealand Greece UK Estonia Finland Slovak Rep. Luxembourg Germany Austria Czech Rep. France Japan Turkey Sweden Hungary Australia Israel Canada Chile Belgium Netherlands Spain Denmark Switzerland Iceland Slovenia Portugal Norway Korea Italy R² = 0.13 300 350 400 450 500 550 600 650 -1.5 -1 -0.5 0 0.5 1 1.5 Mathematicsperformance(scorepoints) Index of school responsibility for curriculum and assessment (index points) Countries that grant schools autonomy over curricula and assessments tend to perform better in mathematics Source: PISA 2012
36. No standardised math policy Standardised math policy455 460 465 470 475 480 485 Less school autonomy More school autonomy Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies Score points School autonomy for curriculum and assessment x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials) Fig IV.1.16
37. Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration Teachers don't participate in management Teachers participate in management455 460 465 470 475 480 485 Less school autonomy More school autonomy Score points School autonomy for resource allocation x System's level of teachers participating in school management Across all participating countries and economies Fig IV.1.17
38. Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements School data not public School data public 464 466 468 470 472 474 476 478 Less school autonomy More school autonomy Score points School autonomy for curriculum and assessment x system's level of posting achievement data publicly Fig IV.1.16
39. 0 20 40 60 80 100 Written specification of the school's curriculum and educational goals Written specification of student-performance standards Systematic recording of data, including teacher and student attendance and graduation rates, test results… Internal evaluation/self-evaluation External evaluation Written feedback from students (e.g. regarding lessons, teachers or resources) Teacher mentoring Regular consultation with one or more experts over a period of at least six months with the aim of improving… Implementation of a standardised policy for mathematics % Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement: Singapore OECD average Quality assurance and school improvement Fig IV.4.14 43
40. 4444Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence  Investing resources where they can make most of a difference  Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms)  Effective spending choices that prioritise high quality teachers over smaller classes
41. 4545 Align the resources with the challenges Hong Kong-China Brazil Uruguay Croatia Latvia Chinese Taipei Thailand Bulgaria Jordan Macao-China UAE Argentina Indonesia Kazakhstan Peru Costa Rica Montenegro Tunisia Qatar Singapore Colombia Malaysia Serbia Romania Viet Nam Shanghai-China USA Poland New Zealand Greece UK Estonia Finland Slovak Rep. Luxembourg Germany AustriaFrance Japan Turkey Sweden Hungary Australia Israel Canada Ireland Chile Belgium SpainDenmark Switzerland Iceland Slovenia Portugal Norway Mexico Korea Italy R² = 0.19 300 350 400 450 500 550 600 650 700 -0.500.511.5 Mathematicsperformance(scorepoints) Equity in resource allocation (index points) Greater equityLess equity Adjusted by per capita GDP Countries with better performance in mathematics tend to allocate educational resources more equitably Source: PISA 2012
42. 4646 Adequate resources to address disadvantage Disadvantaged schools reported more teacher shortage Advantaged schools reported more teacher shortage -0.5 -0.3 -0.1 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 Korea Estonia Israel Latvia Slovenia Italy Poland Singapore Argentina Netherlands Portugal Colombia France Finland Tunisia Macao-China Spain Greece Switzerland Norway RussianFed. Japan Austria Montenegro Croatia Canada OECDaverage Germany Denmark Hungary UnitedKingdom Luxembourg HongKong-China Belgium Iceland VietNam Ireland UnitedStates Chile CzechRepublic Serbia Turkey Mexico Indonesia Uruguay Shanghai-China SlovakRepublic Sweden Brazil NewZealand Australia ChineseTaipei Meanindexdifference Difference between socio-economically disadvantaged and socio-economically advantaged schools A shortage of qualified teachers is more of concern in disadvantaged schools
43. -20 0 20 40 60 80 100 120 140 Shanghai-China HongKong-China France SlovakRepublic Macao-China Italy Switzerland Qatar CzechRepublic Israel Thailand Argentina Denmark Belgium VietNam Germany U.A.E. UnitedKingdom Greece Indonesia Spain ChineseTaipei Singapore Japan Finland Uruguay Poland Sweden Australia NewZealand OECDaverage Netherlands Malaysia Austria Luxembourg Bulgaria Mexico Jordan Peru Iceland Portugal Brazil Turkey Romania Canada Norway Tunisia Lithuania Chile Serbia Korea UnitedStates RussianFed. CostaRica Kazakhstan Montenegro Colombia Croatia Slovenia Ireland Latvia Estonia Scorepointdifference before accounting for students' socio-economic status after accounting for students' socio-economic status Difference in mathematics performance, by attendance at pre- primary school Students who attended pre-primary school perform better Fig III.4.12 47
44. 4848Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence  Coherence of policies and practices  Alignment of policies across all aspects of the system  Coherence of policies over sustained periods of time  Consistency of implementation  Fidelity of implementation (without excessive control)
45. 4949Lessonsfromhighperformers Low impact on outcomes High impact on outcomes Low feasibility High feasibility Money pits Must haves Low hanging fruits Quick wins Commitment to universal achievement Gateways, instructional systems Capacity at point of delivery Incentive structures and accountability Resources where they yield most A learning system Coherence
46. 5050Lessonsfromhighperformers Some students learn at high levels All students need to learn at high levels Student inclusion Routine cognitive skills, rote learning Learning to learn, complex ways of thinking, ways of working Curriculum, instruction and assessment Few years more than secondary High-level professional knowledge workers Teacher quality ‘Tayloristic’, hierarchical Flat, collegial Work organisation Primarily to authorities Primarily to peers and stakeholders Accountability What it all means The old bureaucratic system The modern enabling system
47. Thank you ! Find out more about PISA at www.pisa.oecd.org • All national and international publications • The complete micro-level database Email: Andreas.Schleicher@OECD.org Twitter: SchleicherEDU and remember: Without data, you are just another person with an opinion

Editor's Notes

Another way of looking at the evolution of demand for skills is provided by Autor, Levy and Murnane (2003), whoclassify jobs into routine and non-routine tasks. They argue that the share of non-routine analytic and interactive jobtasks (tasks that involve expert thinking and complex communication skills) performed by American workers hasincreased steadily since 1960. The share of routine cognitive and manual tasks began to decline in theearly 1970s and 1980s, respectively – coinciding with the introduction of computers and computerised productionprocesses. These are tasks that are more readily automated and put into formal algorithms. The share of non-routinemanual tasks also declined, but stabilised in the 1990s, possibly due to the fact that they cannot be easily computerisedor outsourced.
In the unit TRAFFIC, students are given a map of a road network with travel times indicated. While this is a unit with static items, because all the information about travel times is provided at the outset, it still exploits the advantages of computer delivery. Students can click on the map to highlight a route, with a calculator in the bottom left corner adding up travel times for the selected route. The context for the items in these units is classified as social and non-technological. In the third item, students have to use a drop-down menu to select the meeting point that satisfies a condition on travel times for all three participants in a meeting. The demand in this third item is classified as a monitoring and reflecting task, because students have to evaluate possible solutions against a given condition.Students who use spatial reasoning in this item – selecting the geometrical midpoint between the three starting positions – are likely to select the correct answer. This makes the item easier than other tasks with similar demands.This taskmeasures anelementarylevel of problemsolvingskills – Level 1 in a scalethat comprises six describedlevels in total. Across the OECD, 21% of students are only able to solvetasksatthislevel of difficulty – if any. At Level 1, students can explore a problem scenario only in a limited way, but tend to do so only when they have encountered very similar situations before. In general, students at Level 1 can solve straightforward problems provided there is only a simple condition to be satisfied and there are only one or two steps to be performed to reach the goal. Level 1 students tend not to be able to plan ahead or set subgoals.
In the unit TICKETS, students are invited to imagine that they have just arrived at a train station that has an automated ticketing machine. The context for the items in these units is classified as social and technological. In this harder task, compared to the previous example, students must use targeted exploration to reach their goal. They are asked to find and buy the cheapest ticket that allows them to take four trips around the city on the subway, within a single day. As students, they can use concession fares. This item is an example of an interactive problem situation: students are required to engage with the unfamiliar machine and to use the machine to satisfy their needs, without having complete instructions and knowledge about the machine at the outset.This item is classified as exploring and understanding because the main (but not the only) demand of the item corresponds to the acquisition of knowledge about how the machine works, and the prices for the available options. It is therefore a knowledge-acquisition task. Indeed, to accomplish the task, students must use a targeted exploration strategy, first generating at least the two most obvious possible alternatives (a daily subway tickets with concession fares, or an individual concession fare ticket with four trips), then verifying which of these is the cheapest ticket. If students visit both screens before buying the cheapest ticket (which happens to be the individual ticket with four trips) they are given full credit. Students who buy one of the two tickets without comparing the prices for the two only earn partial credit. Solving this problem involves multiple steps.Across the OECD, only about one in ninestudents (11%) are able to solveproblemsatLevel 5 on the problem-solvingscale, such as this one. At Level 5, students can systematically explore a complex problem scenario to gain an understanding of how relevant information is structured. When faced with unfamiliar, moderately complex devices, such as vending machines, they respond quickly to feedback in order to control the device. In order to reach a solution, Level 5 problem-solvers think ahead to find the best strategy that addresses all the given constraints. They can immediately adjust their plans or backtrack when they detect unexpected difficulties or when they make mistakes that take them off course.Level 5 problem-solvers, together with students reaching proficiency level 6, are considered problem solving “top performers”.
TEST 2_ordre décroissant
Because problem-solving skills are required in all kinds of occupations, and are not taught as such in school, but rather are nurtured by good instructional practices in every subject, performance in problem solving should not be strongly influenced by such gender-based stereotypes. Problem-solving performance could then be regarded as an overall indicator of gender biases in a country’s education system.The good news is that in most countries, there are no large differences in boys’ and girls’ average performance in problem solving. While boys and girls do not differ markedly in their average performance, the variation in problem-solving performance is larger among boys than among girls. At lower levels of proficiency, there are, in general, equal proportions of boys and girls. But the highest-performing students in problem solving are largely boys – with a few notable exceptions, such as Australia, Finland and Norway, where the proportion of top-performing girls is about the same as the proportion of top-performing boys. In Croatia, Italy and the Slovak Republic, on the other hand, girls are particularly rare among top-performers.Similarly, the Survey of Adults Skills shows that among adults, top-performers in problem solving are mostly men – except in Canada, Australia and Finland. Because advanced problem-solving skills are the key to access leadership positions, a lack of female leadership figures may in turn create biases in society and teachers that limit girls’ ambition to perform at the top – and perpetrate the glass ceiling.Such biases should not run in the way of nurturing each students’ creative dispositions and problem-solving skills, to enable them to live full lives and contribute with their talents and skills to the country’s well-being.
As machines and computers are increasingly replacing humans for performing routine tasks, highly skilled workers, who are capable of applying their unique skills flexibly in a variety of contexts, regulating their own learning, and handling novel situations, are more and more in demand. Knowing the proportion of 15-year-old students who perform at the highest levels in problem solving allows countries to estimate how well they can respond to this demand. Of particular interest is the proportion of students who, in addition to performing at the highest levels in problem solving, also show excellent mastery of specific subjects. These are top performers who combine the mastery of a specific domain of knowledge with the ability to apply their unique skills flexibly, in a variety of contexts.
(more patternscanbeadded)Germany: same as Spain
TEST 1 _ L’ordre des pays est croissant (différent que dans la Figure V.2.15)
Interactive items are central to the PISA problem-solving assessment, and distinguish it from previous attempts at measuring problem-solving skills. They require students to be open to novelty, tolerate doubt and uncertainty, and dare to use intuitions to initiate a solution.
Tasks can also be distinguished by the problem-solving process that constitutes their main cognitive demand. A major distinction is between knowledge-acquisition tasks and knowledge-utilisation tasks.In knowledge-acquisition tasks, the goal is for students to develop or refine their own representation of the problem space. Students need to generate and manipulate the information in a mental representation. The movement is from concrete to abstract, from information to knowledge. The sample item TICKETS is an example. In knowledge-utilisation tasks, the goal is for students to solve a concrete problem. The movement is from abstract to concrete, from knowledge to action. Knowledge-utilisation tasks correspond to the process of “planning and executing”. To ensure that no additional generation or refinement of knowledge about the problem is needed, items targeting “planning and executing” often had the results of “representing and formulating” tasks available.The best-performing countries in problem-solving often excel particularly on knowledge-acquisition tasks that require high levels of reasoning skills and self-directed learning.
Together, the differences in performance according to the nature of the problem situation and the major problem-solving process targeted identify several groups of countries. These groups often overlap with historical and geographical groupings.Six East Asian countries and economies, namely Korea, Singapore, Hong Kong-China, Macao-China, Chinese Taipei and Shanghai-China, stand out for their very high success rates on knowledge-acquisition tasks, compared to their success rates on planning and executing tasks. Within this group, however, there are relatively stark differences in their performance on interactive problems. Students in Korea and Singapore are significantly more at ease with these problems than students in Shanghai-China, Chinese Taipei and Macao-China. Students from Hong Kong-China are in a middle position. While all of these countries and economies are in the top positions for overall performance, this analysis suggests that in Shanghai-China, Chinese Taipei and Macao-China, a focus on students’ skills at dealing with interactive problem situations is required in order to improve further and close the performance gap with Korea and Singapore. In reviewing their curricula, teachers and curriculum developers may want to introduce more opportunities for students to develop and exercise the traits that are linked to success on interactive items, such as curiosity, perseverance and creativity. They may find inspiration in the curricula and teaching practices of their regional neighbours. Among lower-performing countries and economies, the poor performance of several Latin American countries (Brazil, Colombia, Chile and Uruguay) appears to be mainly due to a large performance gap on knowledge-acquisition tasks. These countries have no particular difficulty with interactive tasks – and Brazil even shows a relative strength on such tasks. In these countries, efforts to raise problem-solving competency should concentrate mainly on improving students’ performance on “exploring and understanding” and on “representing and formulating” tasks. These tasks require students to build mental representations of the problem situation from the pieces of information with which they are presented. Moving from the concrete problem scenario to an abstract representation and understanding of it often demands inductive or deductive reasoning skills. Teachers and curriculum experts may question whether current curricula include sufficient opportunities to model these abstract reasoning skills and whether these opportunities are offered in the classroom.In contrast, several countries in Southern and Eastern Europe, namely Bulgaria, Montenegro, Slovenia, Croatia and Serbia, show relatively weak performance both on knowledge-acquisition tasks and on interactive tasks, compared to their performance on “planning and executing” and on static tasks. In these countries, students seem to find it particularly difficult to understand, elaborate on, and integrate information that is not explicitly given to them (in a verbal or visual format), but has to be inferred from experimental manipulation of the environment and careful observation of the effects of that manipulation. Students in these countries may benefit from greater opportunities to learn from hands-on experience.The performance gap between OECD countries in Europe and North America and the top-performing countries in problem solving mainly originates from differences in students’ performance on knowledge-acquisition tasks. In general, the PISA problem-solving assessment shows that there is significant room for improving students’ ability to turn information into useful knowledge, as measured by performance differences on the dimensions of “exploring and understanding” and “representing and formulating” problem situations.Within this group, Ireland and the United States stand out for their strong performance on interactive items, compared, for instance, to the Nordic countries (Sweden, Finland, Norway and Denmark), the Netherlands, and some countries in Central Europe (in particular, Poland, Hungary, the Slovak Republic). Therefore, the analysis also identifies a strong potential for the latter group of countries to improve on their students’ ability to cope with interactive problem situations. To do so, educators may need to foster such dispositions as being open to novelty, tolerating doubt and uncertainty, and daring to use intuition to initiate a solution.Finally, several countries, while performing at different levels, show a similar balance of skill when compared to each other, and one that is close to the OECD average pattern of performance. Italy and Australia, for instance, have a very similar pattern of performance to that observed in Japan, although in terms of overall performance, Japan ranks significantly above Australia, which, in turn, performs better than Italy. These three countries all perform close to their expected level on interactive items (based on the OECD average pattern of performance), and slightly above their expected level on knowledge-acquisition tasks (although the example of Korea and Singapore shows that significant gains are still possible for them). In other countries, such as Spain, England (United Kingdom) and Germany, performance across tasks reflects the balance observed across OECD countries, on average.
While large and significant, the impact of socio-economic disadvantage on problem-solving skills is weaker than it is on performance in mathematics, reading or science. At all levels of the socio-economic ladder, there is more variation in performance in problem solving than there is in mathematics, perhaps because after-school opportunities to develop problem-solving skills are more evenly distributed than opportunities to develop proficiency in mathematics or reading.
Within all countries, problem-solving results vary greatly between schools: differences in problem-solving performance between schools are as large as differences in mathematics performance, indicating that schools have an important role to play in building these skills. The variation in performance between schools is a measure of how big “school effects” are. These school effects may have three distinct explanations: first, they may reflect selection mechanisms that assign students to schools; in addition, they may be the result of differences in policies and practices across schools; finally, they may be the traces of local school cultures, which develop not by design as a result of policies or deliberate practices, but by the interactions among local communities.The between-school variation in student results is therefore not a direct measure of the importance of school policies and practices for student performance in problem solving. However, if the between-school variation is compared across different student characteristics – some sensitive to education policy and practices, such as performance in mathematics, others not, such as socio-economic status – one may infer the extent to which problem-solving results are related to instructional policies and practices.One might expect the proportion of variation in performance observed between schools to be smaller in problem solving than in reading, science, and mathematics. First, the skills required in the PISA assessment of problem solving are not taught as a specific school subject in most countries, in contrast to those required in reading, science, and mathematics. Second, assessments of problem solving are not explicitly used in high-stakes examinations that influence decisions about selecting students for different classes or schools, where these exist. Yet the association ofdifferences in instruction and selection mechanisms with performance in problem solving is as strong as the association with performance in mathematics.The between-school variation, on the other hand, is much larger in student outcome measures – such as reading, mathematics, or indeed problem solving – than in student background factors that influence performance, such as the PISA index of economic, cultural, and social status (ESCS).
Studentswho, at best, are only able to solveproblemssuch as sampletask TRAFFIC.In six partner countries, more thanhalf of all students do not reach the baseline. In Korea, Japan, Macao-China and Singapore, on the other hand, lessthan one in tenstudentis.
Table V.4.23 has the data for Problem Solving – results are very similar
(Fig. II.4.5)
Table V.4.23 has the data for Problem Solving – results are very similar
I want to conclude with what we have learned about successful reform trajectories In the past when you only needed a small slice of well-educated people it was efficient for governments to invest a large sum in a small elite to lead the country. But the social and economic cost of low educational performance has risen substantially and all young people now need to leave school with strong foundation skills.When you could still assume that what you learn in school will last for a lifetime, teaching content and routine cognitive skills was at the centre of education. Today, where you can access content on Google, where routine cognitive skills are being digitised or outsourced, and where jobs are changing rapidly, the focus is on enabling people to become lifelong learners, to manage complex ways of thinking and complex ways of working that computers cannot take over easily.In the past, teachers had sometimes only a few years more education than the students they taught. When teacher quality is so low, governments tend to tell their teachers exactly what to do and exactly how they want it done and they tend to use Tayloristic methods of administrative control and accountability to get the results they want. Today the challenge is to make teaching a profession of high-level knowledge workers. But such people will not work in schools organised as Tayloristic workplaces using administrative forms of accountability and bureaucratic command and control systems to direct their work. To attract the people they need, successful education systems have transformed the form of work organisation in their schools to a professional form of work organisation in which professional norms of control complement bureaucratic and administrative forms of control.

PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

You are reading a preview.

Check these out next

PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

More Related Content

Slideshows for you (20)

Similar to PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems (20)

More from EduSkills OECD (20)

Recently uploaded (20)

PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

Editor's Notes

PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

You are reading a preview.

PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

Recommended

More Related Content

Slideshows for you (20)

Similar to PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems (20)

More from EduSkills OECD (20)

Recently uploaded (20)

PISA 2012 - Creative Problem Solving: Students’ skills in tackling real-life problems

Editor's Notes