Custom Writing Service http://HelpWriting.net/A-Test-Problem-Generator-For-Discrete-L 👈

1. A Test Problem Generator for Discrete
Linear L, Approximation Problems
K.L. HOFFMAN and D.R. SHIER
National Bureau of Standards
Described here are the theoretmal development and computer implementation of a procedure that
generates test problems for L~esth-nationof the linear modely= Xfl + u. The generationprocedure
allows the user flembdlty m specifying the problem dimensions, the LI solution vector fl*, the
distribution of the observed residuals ~, as well as the column rank, row repetitions, and degree of
degeneracyofthe matrix X The user can also specifythe distributional form,mean, and variancefor
each independent variable An lraportant feature of the generator is that any problem it creates is
guaranteed to havea umque solutmn~ wheneverX has fullrank
Key Wordsand Phrases. L1approximation,least absolute dewation, problem generator, test data
CR Categories: 5.13,5.41,5.5
The Algorithm-A Test Problem Generator for Discrete Linear L~ApproximatmnProblems, ACM
Trans.Math. Softw 6, 4 (Dec 1980),615-617
1. INTRODUCTION
Several different algorithms for discrete linear L1 regression have been developed
recently [1, 3-5]. A thorough evaluation of the performance of such algorithms
requires a procedure for generating data representative of a wide variety of L1
regression problems. This paper describes the development and implementation
of a generation procedure having a number of desirable features that permit the
user to specify and vary the data structures to be tested. The procedure incor-
porates many of the characteristics suggested by Holland [9] as useful in testing
robustness of regression programs and has been used in a recent comparison of
L~-approximation codes [8]. It also can produce, as a special case, data structures
similar to those obtained through the L1 generator of Kennedy, Gentle, and
Sposito [10].
This paper is divided into two major sections. The first section describes the
theoretical development of a procedure for generating data sets having a known
and unique L1 solution. The remaining section discusses various options and
characteristics that can be introduced into the generated sets, and describes a
computer implementation of the generation procedure.
2. THEORETICAL DEVELOPMENT
The discrete linear Ll-approximation problem is that of finding parameters
ill, d2 ..... tim to minimize
n m
IV,- ~ fljx,j[,
tzl j=l
Authors' address. Center for AppliedMathematms, National Bureau of Standards, Washington,DC
20234.
© 1980ACM 0098-3500/80/1200-0587$0000
ACMTransactionsonMathematmalSoftware,Vol 6,No.4,December1980,Pages587-593.

2. 588 K.L Hoffman and D.R. Shier
given observations (y,, x,, ..... Xtm), i = 1..... n. Define
m in In
a,= [y,- Y. fljx,j[ =max(y,- ~ fl~x~j,- y, + ~ fljx,j}.
J=l J~l J=l
This Ll-approximation problem can be reformulated as
min ~ a,
or equivalently as
subject to
m
a,>__y,-~fljx,j, t=l .... ,n,
j=l
In
a,>_-y, + ~ fl~x,j, i=l,...,n,
J~l
t t
min Y. a,
In
subject to a, + ~ fljx,j >- y,, i = 1..... n,
J=l
m
Ol,-- ~fljX,j>----y,, i=1 ..... n,
J=l
(i)
(2)
where the decision variables of the above linear programs are a~ .... , an and ill,
.... tim. The dual problem corresponding to (2) is
n ~
max ~ u,y, - v,y,
,ml t~l
subject to u, + v, = 1, i = 1.... , n,
n
(u, - v,)x~j -= O, j = 1,..., m,
t=l
u,, v, -->0, i =- 1.... , n,
(3)
where the corresponding decision variables are u~, ..., Un and v~, ..., Vn.
Suppose that we wish to have the given values fl* ..... fl* represent a solution
of problem (1). Then define
In
e, =y,- ~ flj*x,~, i = 1,..., n,
J=l
and set
Next, using the index sets
~* = IE,
I , , i=1, ..., n.
N + = (i:l'<i<_.n,e,>O},
N- = {i:l <_i <_n,e, <O},
N°= (i:l .< i < n, e, = O),
ACM Transactions on Mathemat,cal Software, Vol 6, No 4, December 1980

3. A Test Problem Generator • 589
define u~ , u~* v~ v* by
u? = l, vT=0 if i E N +,
u* v7=1 if i ~ N ~,
u~'=½, v~'=½ if i~N °.
(4)
Up to this point in the discussion, we have made no assumptions about the
values which the vector y or the matrix X can assume but have defined the
variables el, a~, N +, N-, and N o for any given _y,X pair. We now examine the
conditions that these pairs must satisfy in order forfl~, .. ;,fl*~ to :be a solution.
Consider the primal problem defined in (1) and the dual problem defined in
(3). Clearly the a7 and the fl~ defined above satisfy the constraints ~n (1).
Moreover, the u? and v? defined in (4) satisfy
u? + v? = 1, i = 1.... , n,
u?, v? _> O, i = l, . . . , n.
These values will satisfy the remaining set of constraints in (3) if and only if
xi]= ~ xij, j= 1..... m. (5)
iEN + i~N-
Therefore, if the m relations (5) obtain, then we have found a feasible Solution
a~, .•., an,* fl~, ..., fl*~to the primal problem (1), as well as a feasible solution
u~,.. * v~,. v*
• , Un, •., n to the dual problem (3). Furthermore, by virtue of the
specification (4), these primal and dual solutions satisfy the appropriate comple-
mentary slackness conditions• Namely, if aT > yi - ~, flTxii, then i E N" and
~m o, x theniEN + * ffi
so by (4) u? = 0; similarly, if aT > -yi + ~j=~ pj ij, and thus vi
o. Conversely, if u? > 0, then i E N + or i ~ N °, so that in either case a? = ~ =
Y~ - ~J"~ f17 x~j; likewise, if v7 > 0, then the corresponding primal inequality holds
with equality.
As a result, once the m relations (5) are imposed on the data elements defining
an L,-approximation problem, then the given fl~, ..., fl*~ will in fact be an
optimal solution to problem (1). It is important to note here that each :equation
in (5) represents a single constraint placed on thejth column alone in the "design
matrix" X = (xij). Thus these m constraints can conveniently be enforced
independently of one another, one for each column of the design matrix.
Through the use of (5), a class of L~-approximation problems can be generated
in a straightforward manner. We have chosen to generate the elements of each
column of X from a specified distribution (normal, uniform, or zero-one), subject
to the restriction (5). Further details of this generation process are described in
Section 3. Clearly, the class of problems so generated does not exhaust the
collection of all possible Ll-approximation problems, but it is broader than the
class produced by currently available generators [10]. Specifically,'the generation
procedure of Kennedy et al. produces a matrix X with paired identical rows (one
corresponding to i ~ N ÷, the other corresponding to i ~ N-); thuseq. (5) is
automatically satisfied. The procedure presented here, in contrast, requires only
that the column sums, over N ÷ and N-, be equal.
ACM Transactions on Mathematical Software, Vol. 6, No. 4, December 1980.

4. 590 • K.L. Hoffman and D.R. Shier
The present procedure allows the design matrix to be generated separately
column by column, and this appears to be a useful computational simplification.
The capability for separate column generation also permits the realistic represen-
tation of situations in which the independent variables have different measure-
ment units or in which some variables are quantitative and others qualitative.
Moreover, this facility for separate column generation allows, through proper
scaling, some measure of control over the condition number of the matrix X. For
example, poorly conditioned problems can be created by generating the entries of
column i = 1..... m uniformly over [1, 10'-'].
When using the L,-criterion for curve fitting, the matrix X will contain a
column of l's if a constant term appears in the postulated linear model. Appli-
cation of eq. (5) to this column yields the relation
IN*[ = IN-I. (6)
If we define n + -- IN+I, n- = IN-I, and no= IN°I, then n ffin + + n - + n o and
(6) merely states that the number n+ of positive and the number n- of negative
residuals at the optimal solution are chosen to be equal. This restriction is not
such an unreasonable one, inasmuch as any optimal solution to a linear L,-
approximationproblem satisfies In+- n- [_
_
_m in the absence ofdegeneracy[2];
in the present context, a problem is degenerate only when no > m. Becausem
will, in general, be much smaller than n, this result implies that the number of
positiveand the number ofnegativeresidualsfor any optimal solutionare roughly
equal.
By using (5), we are able to generate Lrapproximation problems having a
known solution fl* ..... fl*m. In the context of evaluating Ll-approximation
computer codes [8], however, it is also desirable to know whether or not this
represents the only solution to the generated problem. As a matter of fact, the
problems generated by the procedure described here will always possess a unique
solution if X is of full column rank m.
To see this, suppose for the moment that the problem is nondegenerate; that
is, n offim (more generally, n o>_m holds at an extreme point solution when X has
full column rank [7]). The primal problem (2) has a unique solutiona~ .... , a*,
fl* .... , fl* if the dual problem (3) is nondegenerate at u~ ..... iu*, v*,..., v*
[6]. Note that problem (3) has
rfn+m
equation constraints. Moreover, by (4), the number of nonzero variables u*, v*
is precisely
n + + n- + 2n ° ffi n + n o = r,
using the fact that n o --- m. Thus, the optimal basis defined in (4) for the dual
problem must contain only positive u,* and v*, and so is nondegenerate. This
establishes that the primal solution is in fact unique.
This result also holds when n o > m. Indeed, suppose that a given primal
problem P, (with n o> m) is generated with a specified solution vector ft. We wish
to show that this solution vector is unique. Derive problem P2 from problem P,
by removing those constraints in (2) corresponding to i E S, where S is an
arbitrary set of n o - m indices i chosen from N °. Define ~ ffi N - S, where N =
ACM Transactions on Mathematwal Software, Vol. 6, No. 4, December 1980

5. A Test Problem Generator .. 591
{1, 2..... n}. Thus the objective function value zdfl) in P1 for parametex vector
fl can be written as ~
= Z_~(P)+ Z a~(~)
iES iES
----Z2(fl) "4- ~ ai(fl), (7)
iES
where z2(fl) denotes the objective function value for fl in problem P2.
Note that the given fl is assured to be optimal for P1, because (u?, v~: i E N}
are constructed to satisfy (3) and to ensure complementary slackness. It follows
that {u?, v? : i ~ S} are feasible and obey complementary slackness with respect
to fl in problem P2; in fact, since S C N °, relation (5) still obtains in P2. As a
result, fl is also optimal for problem P2. Using the definition of S and (7), it is
seen that z2(fl ) ffi zl(fl ).
Let flbe any other solution of P1, so that zl(fl) ffi zl(fl). Then from (7)
z,(fl) ffi z2(fl) + G(fl),
with G(fl) _>0. If G(fl) > 0, then z2(fl) < z,(fl) = z,(fl) = z2(fl), a contradiction
to the above observation that fl is optimal for problem P2. Therefore, G(~) ffi 0
and z2(~) ffi zl(~) ffi zl(fl) ffi z2(fl). But in problem P2 we have n o ffi m,. and by
our previous result this problem has a unique solution. Hence ~ ffi fl, and so
problem P1 has a unique solution as weU.
We conclude then that the generated problem P1 also has a unique solution in
the case n o> m. Thus, even in the presence of degeneracy, the generated problem
is guaranteed to possess a unique solution.
3. COMPUTER IMPLEMENTATION
This section describes a computer implementation (L1GNR) for generating the
data sets described in the previous section. In order to encompass a wide range
of problems, a number of characteristics of the generated problem are available
for specification by the user. The following input parameters to L1GNR can be
varied to produce such data sets:
(1) The number of observations n and the number of parameters m for the
problem.
(2) The solution vector fl--either specified by the user or randomly generated
from a distributional form (uniform, normal) specified by the user (together
with the mean and variance of that distribution).
(3) The distributional form (uniform, normal, binomial) and particular distribu-
tion parameters for each column of the matrix X.
(4) The number of rows outside the LP basis (the constraints that define the
solution vector) having a residual value equal to zero, corresponding to the
degree ofproblem degeneracy. If X has rank r, then this is the excess number
n o - r of observation points lying on the estimated regression surface in
r-space.
ACM Transactions on Mathematical Software Vol. 6, No. 4, December 1980.

6. 592 • K.L. Hoffman and D.R. Shier
(5) The distributional form (normal, uniform) and particular distribution param-
eters for the remainder of the observed residuals ~.
(6) The number of dependent columns in the matrix X or degree of rank loss.
(7) Row repetitions, indicating the number of times each row is replicated in the
X matrix.
Once the input parameters have been specified, a set of pseudorandom problems
having these characteristics can be generated. The sequence of steps followed in
L1GNR to produce such problems is described next.
The overall strategy of L1GNR can be divided into two phases. The first phase
generates the vector/~, if not specified, and a "skeleton" matrix that would
represent the final matrix of a problem without rank loss, degeneracy, or row
repetitions. The second phase expands the matrix to allow those options and
generates the corresponding vectors • and y.
In the first phase, the number of basic variables NBAS is set equal to the
number of parameters minus the degree of rank loss. The number of distinct rows
is next set equal to the number of observations divided by the number NREP of
repetitions (the procedure delineated here requires divisibility without a remain-
der). Then the submatrix of X defined by these distinct rows is generated one
column at a time (from the appropriate user-specified column distribution) for
each of the NBAS columns, completing the first phase of the generation process.
The second phase begins with a block replication of rows in the current matrix
NREP times. The first NBAS rows are designated as basic and remain fixed to
ensure that the final basis is invertible. To provide a greater measure of random-
ness in the process, however, all other rows are permuted using a pseudorandom
permutation of the integers 1 to n-NBAS. Next, m-NBAS "dependent" columns
are adjoined. These columns are constructed by summing NIND independent
columns, with the user specifying NIND. The first dependent column equals the
sum of the first NIND independent columns, the second column is the sum of the
independent columns 2 through NIND + 1, and so forth.
The matrix X is now in final form, except that the columns generated may not
satisfy the constraint (5)
x,j -- ~, x,,, j=l ..... m.
,EN + tEN-
To satisfy this constraint, the nonbasic rows are separated into three parts: the
first part contains those n°-NBAS rows corresponding to zero-valued residuals.
The remaining rows are divided into two equal parts, one designated as the rows
i having positive residuals (i ~ N +) and the other designated as the rows i having
negative residuals (i ~ N-). The last row of the negative residual group is then
replaced using the entries
x,,= E x,j- E x,,, j=l,...,m.
tEN + lEN-
t#r,
All columns of X now satisfy eq. (5) and the matrix is in final form.
During the next step the residuals • and observations y are generated. For all
rows i not in either N + or N-, E,= 0. For i E N + or i E N-, a value e, is generated
ACM Transactions on Mathematical Software, Vol 6, No 4, December 1980

7. A Test Problem Generator • 593
from the specified distribution of residuals. After calculating z = Xfl, the proce-
dure sets (for i = 1, ..., n)
{z,+[~,l if i~N +,
y,= z,-[e, I if i~N-,
z, otherwise.
The matrix X together with the vectors y and/~ have now been generated. In
the present form, the first NBAS rows form the optimal basis. Since certain L1-
approximation algorithms are dependent on the order in which rows appear in
the matrix, the last operation of this procedure applies a random permutation to
the rows of X.
An American National Standard Fortran version of this procedure appears in
this issue of the ACM Transactions on Mathematical Software. While this
particular implementation has not incorporated such features as nearly dependent
columns or nearly degenerate rows, it would not be difficult to modify this
procedure using the results of Section 2 to include these characteristics (a slight
perturbation in each element of a dependent column would result in a column
with near-dependency).
ACKNOWLEDGMENT
The authors wish to thank Christoph Witzgall for his valuable contributions to
this work.
REFERENCES
1. ABDELMALEK, N.N. An efficient method for the discrete linear L1 approximation problem
Math Comput 29, 131 (July 1975),844-850.
2. APPA, G, AND SMITH, C. On L1 and Chebyshev estmmtlon.Math Program. 5, 1 (Aug.1973),
73-87.
3. ARMSTRONG, R D., AND FROME, E.L. A comparison of two algonthms for absolute deviation
curve fitting J. Am. Stat. Assoc. 71, 354 (June 1976),328-330
4. BARRODALE, I., AND ROBERTS, F.D K. An Improved algorithm for discrete L~ linear approxi-
mation. SIAM J Numer Anal. 10, 5 (Oct. 1973),839-848.
5. BARTELS, R H, CONN,A.R, AND SINCLAIR,J W. Mmim~zationtechniques for piecewisediffer-
entlable functions' The l~solutionto an overdeterminedlinear system.SIAM J. Numer. Anal. 15,
2 (April 1978),224-241.
6. GAVER, D.P., AND THOMPSON, G.L Programming and Probability Models ~n Operations
Research Brooks-Cole,Monterey, Calff, 1973,p. 198
7. GENTLE, J.E, KENNEDY, W J., ANDSPOSITO,V A Onleast absolutevaluesestimation.Commun.
Stat. A, 6, 9 (1977),839-845.
8. GILSINN, J., HOFFMAN, K, JACKSON, R.H.F., LEYENDECKER, E., SAUNDERS, P., AND SHIER,
D Methodologyand analysis for comparingdiscrete linear L~approximation codes. Commun.
Stat. B, 6, 4 (1977),399-413
9 HOLLAND, P.W. Robust test problems for robust regression programs. In Proc. 9th Interface
Syrup. Computer Science and Statistics (April1976),99-105.
10 KENNEDY, W J, GENTLE, J E., AND SPOSITO, V.A A computer ormnted method for generating
test problems for L1regression. Commun Stat B, 6, 1 (1977),21-27.
Recewed July 1978;revised August 1979;accepted December 1979
ACM Transactions on Mathematical Software, Vol.6, No. 4, December 1980.