This study analyzed problem-solving behaviors through think-aloud protocols of low-ability soldiers and higher-ability college students solving arithmetic word problems. Soldiers spent less time calculating and more time reading problems verbatim compared to students. Soldiers also made more calculation errors. For the most difficult problems, soldiers spent more time rereading while students restructured problems. In general, both groups spent most time clarifying and calculating, with less time planning strategies and checking. However, strategic behavior was still observed. The findings suggest soldiers may lack specialized reading skills for translating word problems and monitoring calculations.

1. Instructional Science 15: 49- 65 (1986) 49
© Martinus NijhoffPublishers, Dordrecht - Printed in the Netherlands
A prescriptive analysis of low-ability problem-solving behavior
SHARON J. DERRY & ALMINA KELLIS
Department of Psychology, Florida State University, Tallahassee, FL 32306, USA
Abstract. Thispaperdescribesa comparativeanalysisof the thought processesverbalizedbylow-
ability soldiers and higher-abilitycollegestudents who weretrained to talk aloud while solving
arithmetic word problems. We analyzedand comparedthe types of behaviorsexhibited during
problem solving, and what errors were made. The analysis was used as a basis for designing
problem-solvingtraining for the Army'sJob SkillsEducationalProgram (JSEP), whichconsists
of remedialtrainingin basic educationalskillsrelatedto job performance.Our paper includesa
discussion of research findings that led to training prescriptions, describesthe problem-solving
packagethat was developed,and presents data for a formativetryout of the trainingmaterials.
Introduction
The purpose of this study was to develop a prescriptive analysis of the thought
processes verbalized by low-ability soldiers who were trained to talk aloud
while solving arithmetic word problems. The problems employed in this study
resembled items that soldiers encounter on the tests they take for career ad-
vancement, and to some extent, realistic daily situations in which arithmetic
is employed. Our analysis reports the extent to which recruits displayed strate-
gic, systematic behavior during problem solving, what types of errors they
made, and how their problem-solving behavior differed from that of higher-
ability college students. The analysis was used to help design a training mod-
ule in arithmetic problem solving for use by the Army's remedial educational
program. The need for such training is evident, as many recruits test below
ninth grade levels in both reading and math (see, for example, New York Times,
July, 1983).
Numerous researchers (e.g., Baker & Brown, 1984; Palinscar & Brown, 1983;
Resnick & Glaser, 1976) have argued that training programs should be based
on a better understanding of adequate and inadequate performance. In recent
years, the think-aloud protocol study (Ericsson & Simon, 1980) has evolved as
an important methodology for investigating differences between experts and
novices in various problem-solving domains. A review by Greeno and Simon
(1984) reveals, however, that a major thrust in most studies has not been to
guide design of conventional instruction, but rather to develop detailed psy-
chological models that can be programmed to simulate human problem solv-
ing behavior. One practical value of detailed problem-solving models is that
they can be used by intelligent computer-assisted tutors as a basis for sophisti-

2. 50
cated diagnosis of student errors (e.g., Anderson & Reiser, 1985; Johnson &
Soloway, 1985).
There are several impressive modeling studies of problem-solving behavior
in the arithmetic word-problem domain (e.g., Briars & Larkin, 1984; Kintsch
& Greeno, 1985; Riley, Greeno, & Heller, 1983). Unfortunately, these did not
provide adequate guidance for designing the Army problem-solving module.
One reason is that existing models deal with extremely simple problems and
with children's problem-solving processes. Another reason is that our own situ-
ational and contract requirements made artificial-intelligence approaches im-
practical for this project. In attempting to interpret and use the computer
models as foundations for affordable instructional design, we also encoun-
tered some of the difficulties pointed out by Schoenfeld (1982).
The Army problem-solving module was to be a small, supplementary part
of a larger remedial curriculum, and the following parameters were imposed.
First, the entire program was to consume no more than six or seven hours of
the soldier's time. Second, the instructional package was to be delivered partly
with paper, and partly with TICCIT or PLATO hardware. Third, there were
Army regulations restricting the testing of soldiers, which limited our knowl-
edge of the target population. Given these limits, a talk-aloud study using eight
soldiers was planned to help determine, in a general sense, what type of
problem-solving instruction might be beneficial.
Hypotheses of the Study
Three types of problem-solving models provided hypotheses for the talk-aloud
study; a general model, a solution-strategies model, and an integrated model
that incorporates text-comprehension skills. None of these views contradicts
the others, though each places a different emphasis on what skills are likely
to break down in problem solving.
Superordinate Processes
General models distinguish between what Belmont, Butterfield and Ferretti
(1982) call "subordinate" and "superordinate" processes, and focus on the su-
perordinate skills. A superordinate process is the overall executive plan one car-
ries out in dealing with a problem, whereas subordinate processes are more
specific strategiers (e.g., working backward) or operations (e.g., adding and
subtracting) one uses in service of the plan. Many researchers (e.g., Bransford,
1984; Brown, 1978; Hayes, 1981; Polya, 1957) have claimed that skilled prob-
lem solving proceeds according to a trainable general plan involving steps simi-
lar to the following: problem clarification; development of a strategy; carrying

3. 51
out a strategy; checking the results; and revision. Such plans are an important
aspect of "metacognition" (Flavell, 1979), which refers to a subject's knowl-
edge of subordinate processes, as well as the ability to regulate, evaluate, and
direct those processes.
In diagnosing problem-solving deficits of developmentally disabled chil-
dren, numerous commentators (e.g. Torgesen, 1977) have implicated superor-
dinate processing, particularly with respect to breakdowns in maintenance and
transfer of subordinate processes. One hypothesis of our study was that similar
to children, low-ability adults also would exhibit deficits in their superordinate
processing skills. Accordingly, we examined the executive problem-solving be-
haviors of low-ability soldiers and higher-ability college subjects, to determine
their relative fit to a superordinate processing model. Isolation of deficits in
the low-ability group would be interpreted as evidence that training should pro-
vide a general problem-solving plan. The purpose of training would be to help
soldiers improve their abilities to access and orchestrate lower-order processing
capabilities they already may possess. Derry and Murphy (in press) and Silver
(1982) have pointed out that learned general models function as metacognitive
prompts that organize a problem solver's information processing.
Subordinate Processes
In addition to examining protocols for differences in superordinate processing,
we conducted analyses to determine whether soldiers might exhibit deficits in
certain subordinate processing skills required for successful completion of a
general plan. Our search for subordinate processing differences concentrated
on the "clarify" and "carry-out" stages, as initial analyses suggested that these
were most critical.
Clarifyinga problem involves subordinate processes that lead to understand-
ing, which are distinguishable from the processes of solving. Understanding,
as defined by Greeno (1977) and by Simon and Hayes (1976), involves the con-
struction of a problem representation. Kintsch and Greeno (1985) hypothesize
that understanding a word problem requires development of specialized read-
ing strategies that build representations to which arithmetic operations can be
applied. They have embodied this notion within an integrated problem-solving
model that incorporates linguistic and translation skills derived from a more
general model of text comprehension (van Dijk and Kintsch, 1983). Schoenfeld
(1982) has discovered that even advanced math students can misinterpret word
problems, apparently due to lack of these specialized reading skills. We
thought it reasonable that low-ability soldiers might lack these capabilities as
well.
Solution-strategy models also have accounted for differences between suc-
cessful versus unsuccessful problem-solving performance. In a third analysis,

4. 52
we examined protocols for the purpose of detecting critical differences between
good and poor problem solvers in terms of the solution strategies employed
during the "carry out" phase. Newell (1980) lists nine "weak" strategies that
might be employed in problem solving, and Greeno and Simon (1984) have
reviewed work suggesting that for novel problems, model problem solvers ex-
hibit behaviors consistent with planning, backward chaining, and most
predominately, means-end analysis.
Subjects and Procedures
The initial pool of military subjects consisted of 48 enlistees from a southern
Army post, enrolled in the post's remedial educational program. Eight soldiers
with a mean age of 21 from the second quartile of the normal I.Q. distribution
were selected to participate. These participants had performed successfully on
a criterion test of the basic arithmetic operations (double-digit addition, sub-
traction, multiplication and division) that would be required for solving the ex-
perimental word problems. Eight volunteer psychology undergraduates with a
mean age of 18.4 and average or above-average SAT scores were recruited as
a comparison group. College subjects also performed well on the test of
prerequisite arithmetic operations. However, the 8 college subjects scored
statistically significantly higher than the 8 soldiers on several general tests of
math and verbal ability that are utilized by the Army.
Subjects each participated in one individual, non-paid session held in a pri-
vate room with the second author. During training, two talk-aloud training
problems were administered without time pressure in order to familiarize each
subject with thinking aloud while working problems on paper. Then, six ex-
perimental talk-aloud word problems, shown in appendix A, were ad-
ministered.
Subjects were allowed five minutes for each problem, but also were permit-
ted to complete current thoughts when time expired. The task administrator
had a list of brief verbal prompts to be used following 10 seconds of silence
(e.g., "Remember to talk aloud. "). To certain types of questions, the adminis-
trator was allowed to respond "yes" or "no" without further elaboration. Sub-
jects who reached an incorrect answer before the time period expired were told
that the answer was incorrect and that they should continue working. The task
administrator took notes throughout each session, recording the time in
minutes and seconds for particular subject behaviors. A structured interview
was conducted after each session to learn more about each subject's knowledge
of problem solving.
Subjects' verbalizations were recorded on tape. The tapes were re-recorded
with a metronome being used to mark each six-second interval on the tapes.

5. 53
The marked tapes were transcribed so that each interval was indicated on the
protocols. Typed protocols were judged by two trained raters working indepen-
dently. Each six-second interval was coded according to the type of thinking
behavior that was indicated in that interval.
The coding system (Appendix B), similar to one employed by Rowe (1985),
reflected a model of superordinate processing with embedded subordinate
processes. The coding scheme contained 28 categories grouped into several
broader classifications: 1) attempts to clarify the problem; 2) strategic planning
behaviors; 3) attempts to calculate; 4) checking behaviors; and 5) other be-
haviors, including affective statements and silence. Protocols were judged by
two trained raters working independently. Reliability of judgements was esti-
mated by calculating the percentage of interrater agreement for each interval,
with averages exceeding 85%.
Results and Discussion
As expected, college subjects out-performed soldiers on the problem-solving
tasks. The average time in secs for each problem was 152.33 for soldiers and
95.17 for college subjects. Soldiers made 56 errors, compared to 21 for college
subjects. Time was called without correct solution in 11 instances for soldiers,
and in 5 instances for college subjects. The average number of trials per prob-
lem was 1.68 for soldiers, and 1.36 for college subjects.
Proportion of problem-solving time spent by both groups in each major cat-
egory specified by the general model is shown in Table 1. As indicated, both
groups spent at least 80% of their problem-solving time engaged in verbaliza-
tions related to the clarifying and calculating stages, with a much smaller
proportion of their time devoted to strategic planning and checking. However,
a subsequent strategy analysis indicated that even unseccessful problem solving
was generally characterized by planned, organized activity rather than random
calculating, and the structured interview indicated that college subjects be-
lieved that they frequently engaged in self-checking. These findings seem to in-
Table 1. Proportion of problem-solving time, for six word problems, spent in major categories
of a general problem-solving model.
Clarify Choose Carry out Check Other
problem strategy calcs
Army 44 7 36 3 10
(n = 8)
College 41 2 49 3 5
(n=8)

6. 54
dicate that both strategic planning and self-monitoring are covert, automatic
acts rather than deliberate, conscious activities. On the other hand, clarifying
and calculating processes appear to be easily brought to consciousness by a
talk-aloud procedure. In the structured interview, all subjects reported high
confidence that their verbalizations had reflected their actual thoughts, partic-
ularly on later problems in the sequence.
Mann-Whitney U tests revealed one statistically significant difference be-
tween the two groups. Soldiers devoted 36% of their time to calculating,
whereas college subjects devoted more than 49% to carrying out calculations
(p < .05). Results of an error analysis, shown in Fig. 1, also revealed that soldi-
ers made significantlymore calculation errors than did college subjects. Given
that soldiers demonstrated arithmetic competencies outside the word problem
environment, and given that six college subjects, but only two soldiers, report-
ed the belief that they engaged in self-checking, we hypothesized that calcula-
tion errors were due to a self-monitoring deficiency whereby insufficient time
is devoted to covert checking activity.
One evident qualitative difference between groups was that soldiers used
much of their clarifying time reading problem statements verbatim, whereas
college subjects tended to spontaneously restructure, recode, and label problem
components while carrying out a solution. For example, on the two most diffi-
cult problems (problems 5 & 6 in Appendix A), soldiers spent 24.3o/0 of their
problem-solving time reading and rereading, whereas college subjects used
only 9.0% of their time in verbatim reading. Conversely, college subjects spent
I = ARMY A - Misinterpret
Goal
= COLLEGE B - MIslntarprst
Facts
C - Ignore Facts
'°Ii ii-e
Error
z E - Labeling
Error
- Strategic
15 Errors
- Calculation
Error
a:
A B C D E F G
TYPE OF ERROR

7. 55
41.39% of their time restructuring problem statements and focusing on impor-
tant ideas, although soldiers were engaged in these activities only 27.9% of the
time. These findings suggest the importance of certain text translation skills
that may be unique to word problems. Summary findings of the error analysis,
reported in Figure 1, also strongly implicate translation and representation fac-
tors associated with focusing on important aspects of a problem statement.
Poor problem solvers often missed the goal statements, ignored relevant prob-
lem facts, and mislabeled subgoal calculations.
Problem #6 (see appendix A) most clearly discriminated between soldiers
and college subjects in terms of time to solution, number of trials required,
and number of final correct solutions. All college subjects reached a correct
solution within an average of 1.38 trials. Soldiers required an average of
2 trials, and only 3 soldiers out of 8 reached a final solution that was correct.
Protocols for this problem were examined in detail to test the hypothesis that
low-ability problem solving behavior would be characterized by disorganized,
non-strategic calculating.
With minor variations, the dominant problem-solving strategy employed by
successful problem solvers in both groups can be described as follows:
1. Encode cost information for car A.
2. Encode cost information for car B.
3. Encode that car is wanted for one half of vacation (4 days).
*4. Compute number of hours in four days = 96.
*5. Compute cost of car A = (96 hours × $2 per hour) + $10 fee.
*6. Compute cost of car B=(96 hours x $4 fee per hour) + $2 fee.
7. Encode that cheaper car is desired.
8. Eliminate car B as more expensive.
9. Encode goal problem -- compute remainder from $500.
*10. Subtract cost of A from 500.
11. Check answer.
*Major sub-goal calculations
Only one college subject, who generated a full, accurate solution formula
with unknowns, deviated significantly from this strategy. Six college subjects
worked these subgoal calculations on paper, and five of these subjects also
carefully labeled each step as cost of A, cost of B, etc. One college subject who
used the standard strategy rapidly worked the entire problem in here without
aid of pencil and paper. The eighth college subject, who scribbled disorganized
notes and made several careless mistakes, was the only college subject who re-
quired three trials for correct solution. In the final trial, this subject also em-
ployed the standard sub-goal strategy.
Of the three Army subjects who reached a correct solution, all used the stan-

8. 56
dard sub-goal calculation strategy and made organized notes on paper, al-
though only one of these subjects actually labeled problem facts. One soldier
who did not reach a correct solution also employed the standard strategy and
carefully labeled his calculations, but erred by dividing instead of multiplying
at steps 6 and 7. A fifth soldier who employed the standard strategy and cor-
rect arithmetic operations made several calculation and recall errors. This sol-
dier did not utilize pencil and paper, suffered from obvious memory overload,
and reported extreme anxiety associated with working word problems.
One soldier employed a non-standard strategy that led to different sub-goal
calculations, but that could have given a correct solution. However, the sub-
ject's notes were disorganized and unlabeled, and errors occurred for several
calculations that were completed "in the head." Very negative feelings toward
word problems also were reported.
One soldier who made a strategy error appeared to lack relevant world
knowledge regarding car rentals. That is, the soldier did not realize that an ini-
tial car-rental fee was a one-time charge, and thus treated the fee as a daily
charge. Although classified as a strategy error, this mistake did not reflect an
inability to carry out calculations in a systematic manner. However, this sub-
ject's notes were disorganized and unlabeled, and on subsequent trials he did
not hypothesize possible reasons for his mistakes or seek clarification of the
problem statement.
Only one soldier appeared to calculate almost randomly, without a solution
strategy. Notes were messy and unlabeled. An inspection of this soldier's per-
formance on other problems revealed that he did reach correct solutions in five
of the six problems. However, on difficult problems (i.e., problems 4, 5, and
6), he required many more trials than other soldiers, and it took him longer
than average to understand the problem and subsequently organize his think-
ing. For problems 4 and 5, a strategy eventually emerged after light ex-
perimenter prompting. We interpreted these behaviors as indicating superor-
dinate, or metacognitive, processing deficits. This interpretation was verified
somewhat by inspection of the interview data: Oddly, this soldier reported that
he liked math and was "pretty good at it," although he admitted that he was
slow.
We concluded from our analyses that low-ability soldiers can indeed carry
out planned, strategic approaches that are similar in many respects to those
used by college subjects. However, most soldiers did exhibit deficiencies with
problem clarification and representation skills that affect the carrying out of
both simple calculations and solution strategies. For example, soldier perfor-
mance was hindered by the fact that they frequently did not use pencil and pa-
per in an efficacious manner. Pencil notes can help the problem-solver repre-
sent the problem space as a series of sub-goals, recall relevant problem facts
when they are needed, and carry out and check calculations.

9. 57
Training Prescriptions
Having indentified several adult deficiencies in arithmetic problem-solving
that might be corrected through training, we developed an experimental train-
ing module. Although our study primarily implicated subordinate processing
skills associated with problem clarification, we decided to train these skills
within the context of a general processing model, on the assumption that ex-
plicit knowledge of the general model would help soldiers recall and access
trained subordinate processes at appropriate points during problem solving.
Thus, the training is introduced on the computer via a simple, general
problem-solving model called the Four C's: Clarify the problem; Choose a so-
lution strategy; Carry out the solution strategy; and Check results. Associated
with each of these 4 C's are specific information processing skills that should
help soldiers apply the general model within the domain of word problem solv-
ing. The program currently focuses on important text translation and represen-
tation strategies associated with the clarify stage. However, the program is
highly modular, and could be expanded in the future to include more skills as-
sociated with the other stages of problem solving. The following outline sum-
marizes some of the major subskills that are taught.
1. Goal identification and diagramming. This is a self-paced workbook unit
that trains a four-step labeling and diagramming plan: A. Get rid of useless
information; B. Indentify and label the problem goal; C. Identify and label the
problem facts, and D. If you still don't understand the problem, draw an action
picture of the goal and facts.
2. Selecting sub-goals. This also is a self-paced workbook that provides in-
struction, practice, and feedback in analyzing complex problems.
3. Schema identification. As one step in helping students develop problem
representations, they can be trained to recognize that there are different cate-
gories of word problems, each category having similar problem components
and goals. Mayer (1983) reports a study in which untrained judges were able
to sort, into 18 different schemas, arithmetic textbook problems. Mayer (1984)
also has reviewed research supporting the value of schema training as a method
for improving problem-solving ability. We reviewed several job-related tests uti-
lized by the Army's basic skills educational programs and nominated six types
of problems as a basis for a CAI drill-and-practice program that trains soldiers
to classify different types of problems. The problem types currently incorpo-
rated are: conversion problems, percent problems, averaging problems, work
problems, time problems, and distance problems. We have recommended that
the program be expanded in future to include more schemas and instruction
in solution strategies for each schema.
4. Answer Recognition and Self-Checking. This self-paced workbook trains
soldiers to estimate answers as a basis for judging the plausibility of their cal-

10. 58
culations. Soldiers also are taught a self-questioning strategy that systematical-
ly takes them back through each step of the 4 C's and helps them identify er-
rors at each step. The purpose of this unit is to increase the amount of
self-monitoring activity conducted by soldiers, thereby reducing the number of
calculation errors.
The 4 C's and their associated subskills are taught by character Detective
Imus Plan, who was selected as the problem-solving icon following a survey
in which military personnel rated several possible characters. The style of the
instruction is humorous and entertaining, as the excerpts provided in Appen-
dix C indicate. Detective Plan appears not only in these problem-solving les-
sons, but also "pops up" frequently in other Army basic skills training, to re-
mind soldiers that they should continue to use their problem-solving skills.
Preliminary Evaluation
The program has been used experimentally at two Army posts, and data
reported are combined for the two sites. Eighty-five soldiers were randomly as-
signed to either a self-paced group that received the problem-solving training
prior to taking ten prompted basic skills CAI lessons (Group A), or another
self-paced group (Group B) that received the same ten-lesson sequence without
prompts before taking problem-solving training. None of these soldiers had
participated in the talk-aloud study, although they were drawn from the same
population.
The ten-lesson basic skills sequence given to both groups covered various
topics and included lessons that required arithmetic problem solving, although
none of the lessons actually taught problem-solving skills. In the lessons
offered to soldiers in Group A, Detective Plan appeared periodically to remind
those soldiers to use their newly acquired problem-solving skills while taking
the basic skills lessons.
Soldiers in both groups took the same 54-item multiple-choice posttest im-
mediately after completing their last basic skills lesson. Thus, the soldiers in
Group A took the posttest after receiving the problem-solving instruction and
basic skills lessons with prompting, while the soldiers in Group B, who were
not prompted during basic skills lessons, took the posttest prior to receiving
the problem-solving instruction.
The posttest included twenty near-transfer items with arithmetic word prob-
lems not included in the training programs. These items required soldiers to
identify steps and strategies involved in problem solving, classify new problems
according to their categorical types, eliminate irrelevant information from
word problems, select the best diagrammatic representations of problem state-
ments, identify key facts, goals and subgoals in word problems, and find errors

11. 59
in problem-solving protocols of other soldiers. The KR20 reliability coefficient
for the near-transfer items alone was .70 for the entire group.
The treatment group did perform significantly better than the control group
on the near-transfer portion of the posttest. The mean score for soldiers in
treatment group A was 11.54 (sd=2.81), compared to a mean score of 8.95
(sd = 2.96) for group B soldiers, p < .01. Accordingly, we concluded that a num-
ber of subordinate processing skills that underlie problem solving did improve
for soldiers who received the problem-solving treatment.
The far-transfer posttest required soldiers to solve 34 word problems of vari-
ous lengths, difficulties, and types, and included five of the original word
problems that were used in the talk-aloud training study. Soldiers had not
received training on specific solution strategies or formulas associated with any
of the types of word problems that appeared on the far-transfer test. A time
limit of one hour was imposed, although soldiers completed all items. The
score for each soldier was number of correct solutions. The KR20 coefficient
for this test, computed for the entire group, was .77.
For this far-transfer measure, the mean score for group A was 15.79
(sd = 5.37), and for group B was 15.24 (sd= 5.15). Thus, we concluded that the
problem-solving training, which did not include direct instruction on solution
formulas or task-specific solution strategies, did not improve ability to solve
all new types of problems. We note that for five problems actually used in the
talk-aloud procedure and that served as a basis for development of training,
there was a tendency for soldiers in group A to out-perform group B soldiers.
The major differences between these five problems and others that appeared
on the far-transfer test were that the talk-aloud problems were longer and re-
quired inferences based only on common, everyday world knowledge (such as
the number of days in a week). Although other problems on the test were less
complex, they required more specialized formulas or world knowledge than
might have been available to soldiers. For example, several conversion problems
dealt with the metric system. In retrospect, we believe that for these items, sold-
iers needed training in task-specific formulas and knowledge.
All evidence considered, we believe the module is most likely to improve per-
formance for word problems that require multiple calculations but that do not
require specialized formulas or knowledge. The module might be combined
with specialized formula training to produce far transfer to other types of word
problems, or combined with specialized job-skills instruction to improve job
performance. However, these assertions must be tested experimentally.
As Derry and Murphy (1985) point out in a recent review, far transfer is a
problematic goal for strategies training that rarely is achieved even in artificial
experimental settings. In this sense, our findings were similar to those reported
by other researchers, but we were not surprised. Derry (1985) has stated that
improvement of generalized problem-solving skills would require the full JSEP

12. 60
strategies training model, which includes direct training plus prompting over
a lenghty period of time. The short durations of these trials did not permit ade-
quate testing of the transfer hypothesis.
We have analyzed trial data to determine whether or not the embedding of
problem-solving prompts within basic skills lessons might interfere with learn-
ing of the lesson topics. These data were retrievable from one site only, but ini-
tial results indicate that embedded prompts do not interfere. The mean lesson
posttest score was 25.76 (sd = 9.28) for control subjects who were not prompt-
ed, and 27.23 (sd=7.93) for experimentals who took lessons with embedded
prompts.
Soldier ratings of the training package indicated that it was perceived as in-
teresting and useful. The mean Likert rating for the item, I thought the Prob-
lem Solving Skills Module was very useful, was 2.09 (sd = .80) at Army Post 1,
and 2.50 (sd=.86) at Post 2, on a scale where 1 represented "strongly agree,"
and 5 represented "strongly disagree." Again based on this scale, soldiers tend-
ed to like seeing the prompts during actual instruction. The mean score for this
item was 2.65 (sd=l.09) at Post 1, and 2.18 (sd=.05) at Post 2. Most soldiers
agreed that the prompts made them more likely to use their thinking skills. The
mean for "I was more likely to use a skill when ! saw a prompt" was 2.75
(sd=.97) at Post 1, and 2.09 (sd=.43) at Post 2.
Appendix A - Examples of word problems*
1. Three empty jars weight 7 ozs. each. Each jar holds 10 ozs, of water. How much do three full
jars weigh?
2. A repair shop has a measurement chart on the wall that reads: There are 4 cups in a quart
and there are 4 quarts in a gallon. How many cups are there in two gallons?
3. A butcher sells sirloin steak for 4 dollars a pound, chuck steak for 2 dollars a pound, and
hamburger for 1 dollar a pound. One morning the butcher sold 55 pounds of each type of
meat to the cafeteria cook. How much money did the butcher make that morning?
4. Mrs. Jones' son is nine years old. He gets a weekly allowance that equals out to 5 cents per
day. If the boy plans to get a 20 cent raise each year, how much will he get each week when
he is 12 years old?
5. At first of the year a store had 800 dollars worth of goods. In February the store bought
200 dollars worth of goods to add to what they had left over from January. During February
they sold twice as much as they had bought that month. At the first of March, the store bought
the same amount that they had bought in February, but they only sold 100 dollars worth of
goods in March. At the first of April the store burned down. If the manager claims to have
lost 300 dollars worth of goods, how much did the store sell during the month of January?
6. A car rental company rents car A for one 10 dollar fee, plus 2 dollars for every hour you have
* Problems presented to soldiers were identical to these, except that they were stated with mili-
tary terminology. For example, problem six stated that Sgt. James had been working hard and
received an eight-day leave.

13. 61
the car. The same company rents car B, a much sportier model, for a 2 dollar fee plus twice
that much per hour. Mr. James has been working very hard. He receives an eight-day vaca-
tion. He wants to spend half of his vacation days (24 hours each) driving around the country.
If Mr. James has 500 dollars, how much will he have left for food if he rents the car with the
cheapest total cost for four days of his vacation?
Appendix B - Coding categories
I. Clarify
1. Reading problem verbatim.
2. Recoding problem information.
3. Focusing on specific fact.
4. Eliminating irrelevant information.
5. Seeking additional information.
6. Classifying problem type.
7. Identifying goal.
8. Identifying subgoal.
9. Puzzlement related to clarifying.
10. Expressing need to clarify problem.
II. Developing a solution plan
1. Developing complex solution plan or formula.
2. Identifying subgoal operation to be performed.
3. Considering possibility of using particular plan.
4. Recalling specific past plan that might be useful.
5. Expressing need to plan.
6. Puzzlement related to inability to come up with a plan.
III. Carrying out calculations
1. Performing calculations on paper.
2. Performing automatic calculations without paper.
3. Calculating number thought to be final solution.
IV. Checking
1. Spontaneous evaluation.
2. Spontaneous reworking.
3. Searching for errors.
V. Other
1. Silence.
2. Irrelevant and habitual comments.
3. Positive Affect.
4. Negative Affect.
5. Other.

14. 62
Appendix C - Excerpts from training materials
Word problems are not always as
simple as the ones you have seen
these problem solving lessons.
Let us look at some examples of complex problems. Don't try to solve
these problems. They are designed to show you how confusing some
word problems can be! When you finish this lesson you will be able
to solve them. Right now, they may seem confusing.
KEY POINTS: Setting subgoals is simply breaking a big problem into
several smaller ones. You Bre breaking the problem into
parts.
You are changing a long complex problem into several shorter ones.
These shorter ones may not be any easier to solve, but they will be
smaller and easier to think about. The old saying: "don't bite off
more than you can chew" fits with math problem solving. By looking at
only part of the probtem at a time, you are 91vlng yourseff ~ne chance
to take small bites!

15. 63
KEY POINTS:
STEP ONE: get rid of useless information.
% .2,' "' ~.,¢'I -o "
Key Points: There are four steps in labeling and diagramming:
1. Get rid of useless information
2. Identify and label the problem goal
3. tden~ifif and 1abet tbe problem facts
4. Draw an action picture of the goal and facts.
Whenever you work a word problem ALWAYS; do steps 1 through 3. Step 4
should be done when you are still confused about the problem, but I will
teach you more about that later, Right now let's learn about each step.
ANSWERS:
The correct answers are A and B

16. 64
References
Anderson, J. R., & Reiser, B. J. (1985). The LISP Tutor. Byte, 10, 159-179.
Baker, L., & Brown, A. L. (1984). Metacognitive skills and reading. In D. Pearson, R. Barr, M.
Kamil, & P. Mosenthal (Eds.), Handbook of Reading Research (pp. 353-394). New York: Long-
man.
Belmont, J. M., Butterfield, E. C., & Ferretti, R. P. (1982), To secure transfer of training instruct
self-management skills. In D. K. Detterman & R. J. Sternberg (Eds.), How and how much can in-
telligence be increased (pp. 147-154). Norwood, NJ: ABLEX.
Bransford, J. D. (1984). The IDEAL problem solver. New York: Freeman.
Briars, D. J., & Larkin, J. H. (1984). An integrated model of skill in solving elementary word
problems. Cognition and Instruction, 1, 245-296.
Brown, A. L. (1978). Knowing when, where and how to remember: A problem of metacogni-
tion. In R. Glaser (Ed.), Advances in instructionalpsychology (Vol. 1, pp. 77-165). Hillsdale, NJ:
Erlbaum.
Derry, S. J. (1985). Strategy training: An incidental learning model for CAI. Journal of instruc-
tional development, 8, 16-23.
Derry, S. J., & Murphy, D. A. (in press). Designing systems that train learning ability: From the-
ory to practice. Review of Educational Research.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87,
255-251.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of psychological in-
quiry. American Psychologist, 34, 906-911.
Gagne, R. (1966). Human problem solving: Internal and external events. In B. Kleinmuntz (Ed.),
Problem solving: Research, method, and theory. New York: Wiley.
Greeno, J. G. (1977). Processes of understanding in problem solving. In N. J. Castellan, D. B.
Pisoni, & G. R. Potts (Eds.), Cognitive Theory (Vol. 2, pp. 43- 83). Hillsdale, NJ: Erlbaum.
Greeno, J.G., & Simon, H.A. (1984). Problem solving and reasoning. (Tech. Rep. No.
UPTT/LRDC/ONR/APS-14). Pittsburgh, PA: LRDC.
Hayes, J. (1981). The complete problem solver. Philadelphia, PN: Franklin Institute Press.
Hechinger, E M. (July, 1983). Report faults army's teaching of basic skills. New York Times.
Johnson, W.L., & Soloway, Elliot (1985). PROUST. Byte, 10, 179-193.
Kintsch, W., & Greeno, J. G. (1985). Understanding and solving word arithmetic problems. Psy-
chological Review, 92, 109-129.
Mayer, R. E. (1983). Thinking, problem solving, cognition. W. H. Freeman, 1983.
Mayer, R.E. (1984, April). Learnable aspects of problem solving." Some examples. Paper
presented at annual meeting of American Educational Research Association, New Orleans, LA.
Palincsar, A. S., & Brown, A. L. (1983). Reciprocal teaching of comprehension-monitoring ac-
tivities (Tech. Rep. No. 269). Urbana, IL: Center for the Study of Reading, University of Illinois.
Polya, A. (1957). How to solve it. Garden City, NY: Doubleday-Anchor.
Resnick, L. B., & (3laser, R. Problem solving and intelligence. In L. B, Resnick (Ed.), The nature
of intelligence. Hillsdale, NJ: Erlbaum, 1976.
Riley, M. S., Greeno, J. G., & Heller, J. I. (1983). Development of children's problem-solving
ability in arithmetic. In H. P. Ginsburg (Ed.), The development of mathematical thinking. New
York: Academic Press.
Rowe, H. (1985). Problem solving and intelligence. Hillsdale, NJ: Erlbaum.
Schoenfeld, A. H. (1982). Some thoughts on problem-solving research and mathematics educa-
tion. In E K. Lester, Jr. & J. Garofalo (Eds.), Mathematical problem solving: Issues in research
(pp. 25-36). Philadelphia: Franklin Institute Press.

17. 65
Silver, E. D. (1982). Knowledge organization and mathematical problem solving. In F. K. Lester,
Jr. & J. Garofalo (Eds.), Mathematicalproblem solving: lssues in research (pp. 14-25). Philadel-
phia: Franklin Institute Press.
Simon, H. A., & Hayes, J. R. (1976). The understanding process: Problem isomorphs. Cognitive
Psychology, 8, 165-190.
Torgesen, J. K. (1977). The role of nonspecific factors in the task performance of learning disa-
bled children. Journal of Learning Disabilities, 1977, 10, 27-34.
van Dijk, T. A., & Kintsch, W. (1983). Strategies ofcliscourse comprehension. New York: Aca-
demic Press.