This document summarizes a study investigating factors associated with success in technological problem solving among secondary school students. The study defined technological problem solving, developed a conceptual framework, and designed a study involving a well-defined problem task. Data was collected through observation, photographs, and audio recordings of student groups. Analysis identified the most and least successful groups. Overall, more successful groups engaged more in task discussion, demonstrated knowledge verbally and through solutions, spent longer planning conceptually, utilized more positive management, and engaged in more analytical reflection. They also exhibited less tension and were more affected by the competitive task environment.

1. Secondary School Technological
Problem Solving: An Investigation
of Factors Associated with Levels
of Success.
Dr David Morrison-Love (@dmorrisonlove)
BTechEd (hons) PhD
Click for: Profile LinkedIn ResearchGate
18th September 2013

2. Overview
1. Defining Technological Problem Solving
2. Developing a Conceptual Framework
3. Design of the Study & Cohort Identification
4. Analytical Methods: Observation & Artefact
Development
5. Analytical Methods: Verbal Data
6. Findings & Conclusions
7. Questions & Discussion

3. PART 1
PHILOSOPHY, CONCPETUAL FRAMEWORK
& STUDY DESIGN

4. Defining Technological Problem Solving
1. Context of Activity:
Problem Solving ‘In’ vs. ‘Through’ Technology
2. Type of Problem & Resultant ‘Solving’:
Ill-defined vs. well-defined (Explicit vs. Implicit)
3. Form of Solution (and intended use):
Conceptual vs. Tangible Result

5. Defining Technological Problem Solving
Within the context of secondary school technology
classrooms, Technological Problem Solving:
1. Involves solving problems within the intellectual
domain of technology.
2. Involves a broad range of problem types and
associated solver strategies.
3. Results in a shift from conceptual ideation to
tangible analogue (developed for use by others).
4. Does not deny other forms of problem solving taking
place within technology classrooms.

6. Developing a Conceptual Framework
Four modes of problem solving
identified from literature:
1. Well-Defined
2. Ill-defined
3. Discrete Proactive
Troubleshooting
4. Emergent/Reactive
Troubleshooting

7. Developing a Conceptual Framework
Around 20 intellectual processes
identified (e.g. Williams, Halfin).
• Include computing, measuring,
visualising, predicting, modelling
and so forth.
• Stem from analysis of expert
technologists; validated for
classroom context (Hill & Wicklen).

8. Developing a Conceptual Framework
Detailed review of literature allowed
an original ‘Epistemic Model of
Technological Problem Solving’ to be
developed.
Model is transformative in nature
and shown on hand-out 1.
It accounts for types of knowledge
brought to bear and the
development of ‘technological
knowledge through application’.

9. Research Question
“In terms of intellectual processes and knowledge, what
are the differences in the modi operandi between groups
of pupils that produced more and less successful
technological solutions to a well-defined problem?”

10. Design of the Study
Study design addressed the following key areas:
1. Epistemic Stance of Study (Interpretive/P. Positivist).
2. Appropriately targeted participant schools (broad
demographic spread).
3. Design of Problem Solving Task
4. Multiple data gathering instruments (mixed method
- composite representation of reality).
5. Execution of Data Gathering

11. Design of the Study – Sample Identification
• 1 school identified from within 3 socio-demographic
groups (low, average and high poverty).
• Schools placement validated through multiple
demographic measures (SADI, Cartsairs Scores, School
Meal Subsidy).
• Demographic profile compiled for each school
(example profile in hand-out 2).
• S2 Pupils chosen as (relative to a given class) they all
share the same curricular exposure and not made
subject choices. 13 groups in total.

12. Design of the Study – Design of Problem Solving Task
• Well-defined Mode (detailed start state)
• Topic: Structures (Cantilevers) - (Principle shown in
hand-out 3).
• Design and construct one half of a cantilever bridge
using materials provided, and taking account of the
stated restrictions, to exhibit high rigidity and low
deflection.
• Competitive Task Environment
• 2 full periods to complete solutions prior to testing.

13. Design of the Study – Design of Problem Solving Task

14. Design of the Study – Design of Problem Solving Task

15. Design of the Study – Data Gathering Instruments
* Observation in same mode throughout - Temporal Acclimatisation

16. Design of the Study – Data Gathering Instruments
* Observation in same mode throughout - Temporal Acclimatisation

17. Design of the Study – Data Gathering (Hand-out 4)
• Pupils in groups of 4
(as far as possible).
• Single Gender Groups.
• Observation rotates
between groups.
• Audio recording
throughout.
• All groups
photographed at 4
min intervals.

18. PART 1
PHILOSOPHY, CONCPETUAL FRAMEWORK
& STUDY DESIGN
ANY QUESTIONS?

19. PART 2
ANALYTICAL METHODS

20. Selected Analytical Areas
Area 1: Analysis of Photographs of Solution
Development (Bespoke Procedure)
Area 2: Identification of ‘Best’ and ‘Poorest’ groups
(Modified Delphi)
Area 3: Analysis of Verbal Data (Ranked Inductive
Analysis)

21. Analysis of Photographs of Solution
Addition or removal of materials (a development), is
coded against each zone it is physically connected with.

22. Analysis of Photographs of Solution (Example 1)
1. Individual development in Zone D
2. Individual development in Zone A

23. Analysis of Photographs of Solution (Example 2)
Individual development in Zone A & B

24. Analysis of Photographs of Solution (Example 3)
Individual development in Zones A(4), C(2) & D(1).
Not coded for Zone B(3) as it is not physically joined
to the solution.

25. Analysis of Photographs of Solution (Example 4)
All developments were also coded for the level of
functional advantage (‘offering’ or ‘little to no’)
Good Functional Advantage

26. Analysis of Photographs of Solution (Example 4)
All developments were also coded for the level of
functional advantage (‘offering’ or ‘little to no’)
Poor Functional Advantage

27. Analysis of Photographs of Solution
0
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 E 10 11 12 13 14 15 16 17 18 E
LevelofDevelopment
Sample Number
Physical Development of Solution
Group 5 Group 7

28. Analysis of Photographs of Solution

29. Identification of ‘Best’ & ‘Poorest Groups’
• There were 13 groups in total; the study sought a
best cohort or 4 and poorest cohort of 4.
• There was no single correct solution.
• Nature of the models meant that quantifiable
physical testing was not reliable (or possible).
• The criteria for physical testing were relative to a
given class (was 1st place in Class A better or worse
than 1st place in Class B?)
• Researcher had already observed groups creating
solutions – risk that choices fit preconceptions?

30. Identification of ‘Best’ & ‘Poorest Groups’
• Modified Delphi with 8 experts over two rounds.
• Teachers with specific knowledge of structures, not
from participating schools who underwent training.
• Successfully ranked the 13 solutions
• Face validity analysis.
• Combined with physical testing data in WC Matrix.
• Top 4 = Best Task Performance; Bottom 4 = Poorest
Task Performance.
• Researcher removed from decision making process.

31. Analysis of Verbal Data
Group 5
Rank 1 (Best)
Group 7
Rank 1 (Poorest)
Dyad 1
Group 6
Rank 2 (Best)
Group 13
Rank 2 (Poorest)
Dyad 2
Group 12
Rank 3 (Best)
Group 4
Rank 3 (Poorest)
Dyad 3
Group 8
Rank 4 (Best)
Group 2
Rank 4 (Poorest)
Dyad 4
Most
Contrasting
Least
Contrasting
Stage 2
Analysis
Stage 3
Analysis
• Immersion
Approach – initial
themes/areas of
contrast.
• Two most
contrasting cases.
• Notional coding
from C.F.
• Iterative
refinement of
coding (7/8 rnds).

32. Analysis of Verbal Data
• Nvivo – Direct
Coding of
Waveforms
• Not searchable
(as with
transcribed text)
• Group rather
than individual as
unit of study.
• Apx. 2 Hrs/group.

33. PART 2
ANALYTICAL METHODS
ANY QUESTIONS?

34. PART 3
FINDINGS & RESULTS

35. Emerging Frameworks
Analysis of most contrasting cases gave rise to 3
frameworks that described where differences lay.
• Knowledge Differences
• Process Differences (inc. Declarative & Analytical Ref)
• Social & Extrinsic Differences
Frameworks were applied to remaining dyads to
determine the extent to which this was reflected.

36. Emerging Frameworks (Hand Out 5)
Knowledge Differences
Attainment in
Structures Unit
Verbalised
Knowledge During
Activity
The Solution as
Manifest
Knowledge
 Tenison &
Compression
 Triangualtion
 Turning
Moments
 Task
 Concpets &
Principles
 Good Planning
 Results of Poor
Planning
Process Differences
Global Process
Management of
Problem Solving
Process
Process
Engagement
 Pattern of
Solution
Development
 Phases of
Activity
 Fragmented
Vision
 Poor
Involvement
 Roles & Tasks
Group Involvement
Increasing
Efficiency
Planning
 Task Reflection
 Declarative
Reflection
 Analytical
Reflection
Social & Extrinsic
Differences
Group Tension
Competitive
Dynamic
Study Effects
 Positive Effects
 Neutral Effects
 Negative
Effects
 Researcher
 Recorder

37. Concepts of Declarative & Analytical Reflection
Declarative Reflection
That which is close to the observable. A statement
about something already done that does not reveal
reasoning (e.g. “That’s good”, “That doesn’t move”).
Analytical Reflection
A statement about something already done that does
reveal reasoning (e.g. “Aye, that would be better ‘cause
that is stronger than a bit of thread…”)

38. Overall Findings by Cohort
Overall, it was found that higher performing groups:
1. Engaged in more task-related discussion (>21.5%)
2. Verbalise more objective knowledge correctly with
fewer deficits evident in the final artefact.
3. Demonstrate a higher level of tacit-procedural
knowledge (dexterity/fine motor manipulation).

39. Overall Findings by Cohort

40. Overall Findings by Cohort

41. Overall Findings by Cohort
Overall, it was found that higher performing groups:
4. Spent longer in the conceptual phase of problem
solving, prior to commencing construction (18%
longer + all had starting point established).
5. Utilise more positive managerial traits and fewer
negative managerial traits (+ve 252/155; -ve 25/77).
6. Engage in more reflection and, specifically, more
analytical reflection (38% longer gen; 57.5% more AR)

42. Overall Findings by Cohort
Overall, it was found that higher performing groups:
7. Exhibit considerably lower levels of tension between
group members (Hand Out 6). 91% time difference
between high/low cohorts.
8. Are significantly more affected by the competitive
task dynamic (>2x as many negative instances).
9. Are not as affected by influences from the study
itself (65% less duration of instances).

43. ANY QUESTIONS

44. Dr David Morrison-Love
david.morrison-love@glasgow.ac.uk
http://www.linkedin.com/in/davidmorrisonlove
www.dmorrisonlove.net
0141 330 3096