A Regularization Approach to the Reconciliation of Constrained Data Sets


Published on

A new iterative solution to the statistical adjustment of constrained data sets is derived in this paper. The method is general and may be applied to any weighted least squares problem containing nonlinear equality constraints. Other methods are available to solve this class of problem, but are complicated when unmeasured variables and model parameters are not all observable and the model constraints are not all independent. Of notable exception however are the methods of Crowe (1986) and Pai and Fisher (1988), although these implementations require the determination of a matrix projection at each iteration which may be computationally expensive. An alternative solution is proposed which makes the pragmatic assumption that the unmeasured variables and model parameters are known with a finite but equal uncertainty. We then re-formulate the well known data reconciliation solution in the absence of these unknowns to arrive at our new solution; hence the regularization approach. Another procedure for the classification of observable and redundant variables is also given which does not require the explicit computation of the matrix projection. The new algorithm is demonstrated using three illustrative examples previously used in other studies.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

A Regularization Approach to the Reconciliation of Constrained Data Sets

  1. 1. A Regularization Approach to the Reconciliation of Constrained Data Sets Jeffrey Dean Kelly1 1 Industrial Algorithms LLC., 15 St. Andrews Road, Toronto, Ontario, Canada, M1P 4C3 E-mail: jdkelly@industrialgorithms.ca
  2. 2. 2 Abstract A new iterative solution to the statistical adjustment of constrained data sets is derived in this paper. The method is general and may be applied to any weighted least squares problem containing nonlinear equality constraints. Other methods are available to solve this class of problem, but are complicated when unmeasured variables and model parameters are not all observable and the model constraints are not all independent. Of notable exception however are the methods of Crowe (1986) and Pai and Fisher (1988), although these implementations require the determination of a matrix projection at each iteration which may be computationally expensive. An alternative solution is proposed which makes the pragmatic assumption that the unmeasured variables and model parameters are known with a finite but equal uncertainty. We then re-formulate the well known data reconciliation solution in the absence of these unknowns to arrive at our new solution; hence the regularization approach. Another procedure for the classification of observable and redundant variables is also given which does not require the explicit computation of the matrix projection. The new algorithm is demonstrated using three illustrative examples previously used in other studies. Keywords Data Reconciliation, Projection Matrix, Regularization, Generalized Inverses, Observability and Redundancy.
  3. 3. 3 Introduction The issue of reconciling process measurements subject to equality constraints has been widely studied in the past several decades in the chemical engineering literature with a recent and comprehensive review given by Crowe (1996). The basic premise of data reconciliation is to statistically adjust or smooth the process measurements consistent with known conservation laws such as steady-state material, energy and component balances where the measurements are assumed to be subject to at worst random errors (i.e., to have an expected value of zero and known variance). Unfortunately, these measurements may be unknowingly distorted by systematic or gross errors such as instrument malfunctions or miscalibrations, non-representative sampling and even variability caused by the process itself. These gross errors consequently eliminate any realistic conclusions that can be drawn from the results even if just one of these anomalies is present in the data set. Although the objective of this article is to highlight a different approach to the data reconciliation solution, it must be understood that all methods are negatively influenced by the presence of gross errors and techniques such as those presented by Tong (1995) and Tong and Crowe (1995) are strongly recommended to aid in the identification and removal of such aberrants. The general data reconciliation problem, shown below, is essentially a weighted least squares minimization of the measurement adjustments subject to nonlinear constraints involving reconciled, unmeasured and fixed variables. min w.r.t. s.t. J T T        (x x) Q (x x) x Q x x f(x, y,z) 0 m 1 m a 1 a [1] Using the above nomenclature, the objective function J is known to be distributed as a Chi-square statistic with degrees of freedom equal to the number of constraints minus the number of independent unmeasured variables (Knepper and Gorman, 1980). The general equality constraint functions, denoted by f(x,y,z)of size ng x 1, may contain any combination of reconciled x variables of size nx x 1 (i.e., known with uncertainty) , unmeasured y variables or model parameters of size ny x 1 (i.e., unknown) and fixed or constant z variables of size nz x 1 (i.e., known without uncertainty). Segmenting the variables into three categories is useful in representing known conservation laws and other strictures that may be appropriate to model the underlying system. The vector xm contains the measurements, xa contains the vector of adjustments to the measurements where the symmetric and positive definite matrix Q contains the variance-covariance elements of the measurement errors and thus quantifies the uncertainty in each measured value. The solution to the above problem has been well studied by many researchers with one of the first solutions, presented in the chemical engineering literature, given by Kuehn and Davidson (1961) although their formulation did not include the existence
  4. 4. 4 of unmeasured variables. It is the estimation of the measurement adjustments, unmeasured variables and model parameters that poses the greatest difficulty where recent literature suggests that successful estimation cannot be achieved unless it is indeed performed simultaneously (MacDonald and Howat, 1988; Crowe, 1996). Other workers in the field of applied statistics have also studied these problems in depth with useful reviews given by Reilly and Patino-Leal (1981), Ricker (1984) and Dovi and Paladino (1989) and concern the problem of estimating parameters when there is error in all the variables with a recent application using this technique found in Weiss et. al. (1996). However, it wasn’t until the fundamental work of Britt and Luecke (1973) that a somewhat general method for problem [1] was derived. They used the method of Lagrange multipliers to arrive at the following iterative solution for the nonlinear data reconciliation problem  x x QA (A QA ) f A (x x ) B (y y )k 1 m k k k 1 k k m k k k 1 k         T T [2]    y y B (A QA ) B B (A QA ) f A (x x )k 1 k k k k 1 k 1 k k k 1 k k m k        T T T T [3] where in their derivation the functions fk are successively linearized about xk, yk and z at iteration k in a Taylor series expansion yielding A (x x ) B (y y ) f 0k k 1 k k k 1 k k       [4] and the notation of f(xk,yk,z) is simplified to fk. The matrices Ak and Bk are the first-order derivatives or Jacobian of the constraints with respect to the reconciled and unmeasured variables respectively where representative starting values are required to initiate the solution.       A f x k k kij i j    [5]       B f y k k kij i j    [6] These derivatives can be calculated analytically or by perturbing each variable individually and recording the change in the specific function (i.e., finite difference). However, it is well known that the numerical derivative estimates are computationally expensive, hence Jacobian update techniques (Pai and Fisher, 1988; Press et.al., 1994) and optimal Jacobian partitioning
  5. 5. 5 algorithms (Coleman et. al., 1984) have been developed to reduce the calculational load and are especially useful when solving large sparse systems. A similar result to equations [2] and [3] was presented by Knepper and Gorman (1980), where through the use of generalized inverses, were the first to recognize that the solution of the unmeasured variables are actually estimated through the normal equations of the unconstrained least squares problem. Unfortunately, both these methods cannot be used for truly general data reconciliation problems since the two inverses found in equations [2] and [3] may not exist i.e., (A QA )k kT 1 [7]  B (A QA ) Bk k k 1 kT T  1 [8] Two situations arise which make these solutions infeasible, for example, the matrix shown in expression [7] will not have an inverse if a constraint involves only unmeasured variables thus causing a zero row in the matrix (i.e., Ak is row rank deficient). The second situation arises when not all of the unmeasured variables are determinable because of a deficiency in the number of measurements (i.e., Bk is column rank deficient). Interestingly, a similar expression to equation [3] presented by Mah et. al. (1976) to determine the unmeasured variables, is also not appropriate due to the above mentioned conditions. Furthermore, these two restrictions concerning the usefulness of equations [2] and [3] is not unknown, Britt and Luecke (1973) presented necessary conditions on the ranks of Ak and Bk though no insight into the identification of the dependent constraints and unmeasured variables were given. In fact, Knepper and Gorman (1980) discuss the case when all of the constraints may not be independent due to modeling relatively large problems but no discussion on how to identify these dependent equations was elaborated. In addition, other solutions such as the “Error-In-Variables Model” presented by Reilly and Patino-Leal (1981) also suffer from these rank deficiencies where no discussion nor attempt is made to identify and remove these dependent vectors. Further, a method for solving large flowsheet applications was presented by Stephenson and Shewchuk (1986) who estimated the adjustments, unmeasured variables and Lagrange multipliers simultaneously by solving a full-space successive quadratic program (SQP) structured problem with only equality constraints considered. However no treatment on the identification of dependent unmeasured variables was given and consequently no classification of variables was reviewed. And similarily, Kim et. al. (1997) suggest using a nonlinear programming algorithm, presumably employing an SQP, to perform “robust” data reconciliation and gross error detection based on the MIMT serial elimination technique of Serth and Heenan (1986) and on the work of Liebman and Edgar (1988) who impose explicit inequality constraints on both the reconciled and unmeasured variable estimates; although determining realistic and consistent upper and lower bounds is a nontrivial exercise.
  6. 6. 6 Notwithstanding, Crowe et. al. (1983) and Crowe (1986) presented an elegant method which identified dependent unmeasured variables and eliminated dependent constraints whereby it was possible to coaptate (Mah et. al., 1976) or decouple the problem into two sub-problems, one containing only reconciled variables and the other involving unmeasured variables. They achieved this result by using a projection matrix which effectively spanned the null space of BkT however finding this matrix projection involves some effort. Later, Pai and Fisher (1988) published an algorithm to solve the general nonlinear data reconciliation problem shown below    x x QA U U A QA U U f A (x x )k 1 m k k k k k k k k k m k      T T T T1 [9]     y y B I 0 f A (x x )k 1 k 1 k k k k 1 k       1 [10] where  I 0 is a matrix of size ny x ng if there are no dependent unmeasured variables and Uk is the matrix projection determined by first partitioning Bk into B B B B B k 1 k 3 k 2 k 4 k       [11] and then computing its elements as   U B B Ik 2 k 1 k 1    T [12] which satisfies the requirement that UkT Bk = 0. Careful attention to the row and column orderings in equation [11] is required where the existence of dependent columns in Bk can indicate non-observable unmeasured variables (Crowe, 1989) and consequently the unmeasured variables estimated from equation [10] are for the re-ordered independent unmeasured variables only; the dependent unmeasured variables are set to zero (Crowe, 1986). For a complete description of the re- orderings, and the determination of the matrix projection Uk , the reader is referred the work of Kelly (1998). Interestingly, Bagajewicz (1997) points out that if Ak and Bk are in canonical form (Madron, 1992), that is if Bk is of full column rank and Ak is of full row rank (i.e., equations [7] and [8] are realizable) then this is equivalent to the matrix projection procedure introduced by Crowe et. al. (1983). Consequently, the determination of this matrix projection provides the means to obtain a canonical representation of the data reconciliation problem and hence establishes an equivalence with equations [2] and [3]. However, the task of obtaining an alternative to this canonical representation is proposed in this article.
  7. 7. 7 Since the matrix projection must be computed at each kth iteration, which requires additional computation although it can significantly reduce the size of the matrix to be inverted in equation [9], it seems reasonable to ponder if another method is possible which alleviates the two restrictions imposed by equations [2] and [3] without explicitly computing Uk. In fact, other researchers such as Swartz (1989) and Sanchez and Romagnoli (1996) have proposed different methods to solve the general data reconciliation problem, though they use the fundamental concept of a projection matrix to decouple the problem but compute its elements using QR factorization. And recently, Albuquerque and Biegler (1996) solved the problem of dynamic data reconciliation using an SQP where they introduced a LU decomposition strategy to obtain an estimate of the matrix projection in order to classify the measured and unmeasured variables. The focus of this paper is to introduce a regularization approach to solve the data reconciliation problem described in equation [1] which negates the explicit use of a matrix projection whilst being able to arrive at a minimized solution. Further, this article offers an alternative method to classify unmeasured variables into observable and non-observable and reconciled variables into redundant and non-redundant which also does not require the matrix projection. In addition, the variance-covariance structures of the measurement adjustments, unmeasured variables and constraint residuals are presented in order to complete the method for the very important post analysis of the final results (i.e., gross error detection and identification). A Regularized Solution Approach By solving a Lagrange multiplier problem, assuming that we have no unmeasured variables, we may use the solution presented by Kuehn and Davidson (1961) to solve problem [1] as  x x QA (A QA ) f A (x x )k 1 m k k k 1 k k m k      T T [13] If we add the unmeasured variables to the problem by arbitrarily making the assumption that they have a sufficiently large but equal uncertainty defined by the non-negative real number k , which may change at every iteration, we may re-write equation [13] similar to equation [2] as  x x QA K f A (x x ) B (y y )k 1 m k k k k m k k k 1 k         T ( ) 1 [14] where Kk is defined as the kernel matrix of the reconciliation solution and is expressed below as    K A B Q 0 0 I A B A QA B Bk k k y k k k k k k                   k T T T k T [15]
  8. 8. 8 Consequently, this solution minimizes the original objective function value J found in problem [1] plus a second term (i.e., 1/ ( ) ( )k T y y y yk 1 k k 1 k    ) since the unmeasured variables are now included as measured quantities. If for the moment we assume that all of the unmeasured variables are observable (i.e., Bk is of full column rank), then by using the method of generalized inverses discussed in Knepper and Gorman (1980) and sometimes referred to as the Moore-Penrose pseudo inverse (Golub and Van Loan, 1983), we may solve for the unmeasured variables found in equation [4] as an unconstrained parameter estimation problem at each iteration i.e.,  y y (B B ) B f A (x x )k 1 k k k k k k k 1 k       T T1 [16] Then by substituting the above into equation [14] we arrive at, after little manipulation, a new solution to the data reconciliation problem      x x D QA K E f A (x x )k 1 m k k k k k k m k       1 1T [17] with the matrices Dk and Ek being both symmetric and defined as    D I QA K I E Ak x k k g k k    T 1 [18a]  E I B B B Bk g k k k k   T T1 [18b] where if Bk were square and of full rank i.e., has as many unmeasured variables as equations (ng = ny), then Ek would be zero and no data reconciliation would be possible. Although equation [17] is an approximation, it is surprisingly insensitive to the choice of k as we shall see in the examples. This is obviously an important and desirable property to possess since including the matrices Dk and Ek can increase the calculations substantially, especially for large problems. Moreover, it should also be noticed that the inclusion of k and B Bk kT into equation [15] resembles the method of linear regularization described in Press et. al. (1994). Regularization methods have found success in many practical and diverse problems when it is impossible to invert the kernel matrix due to degeneracy (i.e., equation [7]). Hence, the above formulation adopts this methodology for the solution of data reconciliation problems in order to circumvent the degeneracy problem without computing the matrix projection (which uses the null space of BkT to produce a canonical or minimum order kernel matrix i.e., the inverse found in equation [9]). However, this approach relies on the proper selection of k which is a major factor in the success of the method and consequently guidelines for its choice will be discussed in detail later.
  9. 9. 9 Interestingly, the solution for the unmeasured variables in terms of the normal equations of the least squares minimization shown in equation [16] is identical to Britt and Luecke (1973)’s linear solution for the unmeasured variables (i.e., Bagajewicz, (1997)’s equation [5] and very similar to our equation [3]) if Bagajewicz (1997)’s equation [4] for the reconciled variables is substituted into the linear version of equation [16] (i.e., with ( )Ax Czm  as the response vector and yk removed). Consequently, equation [16] is numerically simpler to implement than equation [3] even when it is formulated into canonical form and similarly, equation [10] requires full knowledge of the independent constraints of B in order to compute the inverse of B1k . And in line with the regularization approach, it is known that by adding a diagonally dominant matrix to BkT Bk i.e.,  B B Ik k y T k  [19] with  k chosen as a positive scalar, we can also increase the numerical stability of the iterative solution especially when solving ill-conditioned systems. This technique has proven useful in fitting nonlinear and time dependent transfer function models and is sometimes referred to as the Levenberg-Marquardt method (Box and Jenkins, 1976; Press et. al., 1994) or ridge regression (Golub and Van Loan, 1983). Nevertheless, if  k is non-zero at the converged iteration, the unmeasured variables are biased since the above modification imposes equality constraints on the parameters from their starting values in the least squares sense. Though this technique was not used in this study, it may prove useful in other problems to aid the data reconciliation practitioner in obtaining a converged solution since it is relatively straightforward to implement. As well, other numerical stability techniques such as line searches and trust regions have been shown to be useful in many other nonlinear programming solutions (Lucia and Xu, 1990 and 1994) and can be easily augmented in the same manner as that of Pai and Fisher (1988). Several Implementation Issues Discussed Before we can proceed to the estimation of the variance-covariance matrices and to the classification of the variables which completes the method, five important issues need to be addressed. The first issue concerns the identification of dependent columns of the matrix Bk (i.e., equation [8]) and relates to the existence of a solution for equations [3] and [16]. We can determine an independent set of unmeasured variables which simplifies the partition of equation [11] into the following two sub-matrices    B P P B Bk 12 C,k 34 C,k 12 k 34 k  [20]
  10. 10. 10 where P12C,k and P34C,k are column permutation matrices containing the column re-orderings and equations [16], [17] and [18] can be re-written with Bk replaced by B12k. This partitioning may be performed at every iteration however it needs only be initiated when the inverse matrix in equation [16] is not positive definite (i.e., singular). If it is, then any identified dependent columns must be added to the matrix B34k. Dependent columns can be easily detected if a modified Cholesky factorization is used to solve for the independent unmeasured variables where the Cholesky factor for the identified independent columns can easily be extracted from the results (cf. Kelly, 1998). As an aside, the presence of dependent columns in Bk will not affect the existence of an inverse for the kernel matrix and therefore BkBk T can remain unchanged in equation [15]. If there do exist dependent columns (i.e., BkT Bk is not positive definite) then the vector of unmeasured variables can be comprised of a re-ordered set expressed as  y P P y y k 12 C,k 34 C,k 12 k 34 k       [21] Here, y k is the vector of original unmeasured variables of arbitrary order, and y12k and y34k are the vectors of independent and dependent unmeasured variables respectively where y34k variables are arbitrarily set to zero (Golub and Van Loan, 1983; Crowe, 1986). For the y12k variables, which are the solution to equation [16], we show the modified solution in full as    y P P y 0 (B B ) B f A (x x ) 0 k 1 12 C,k 34 C,k 12 k 12 k 12 k 12 k k k k 1 k                             T T1 [22] The second issue is the existence of an inverse for the kernel matrix shown in equation [15]. Since we have included the Jacobian or topology matrix of the unmeasured variables (which is itself degenerate), its existence eliminates the problem associated with expression [7] provided k is not zero requiring the specified equality constraints to be functions of at least the measured or unmeasured variables or both. Notwithstanding, we may encounter constraints which are linearly dependent on other or combinations of other constraints, this may occur when we are developing large and diverse problems. These situations will cause singularities and thus these linear dependent constraints must be identified, then removed from the valid constraint set, which will typically be performed once at the start of the reconciliation. Since Cholesky factorization is recommended to construct the kernel matrix’s inverse (Reilly and Patino-Leal, 1981; Forrest, 1991; Fang and Puthenpura, 1993) dependent constraints can be easily detected at each iteration as mentioned previously. The third issue concerns the choice of k , which as mentioned, can be considered as a regularization parameter and is sometimes referred to as a shrinkage factor in ridge regression. Given that k from a practical viewpoint is an overall
  11. 11. 11 estimate of the uncertainty for all of the unmeasured variables, a balance or compromise must be struck between the reality of the solution in terms of minimizing the adjustments to the measured variables and the speed of execution. If for example we increase k , so that the elements of (Kk)-1Bk tend toward zero (which is possible since any norm of Kk will increase with k ), then equation [17] would numerically collapse to    x x QA K f A (x x )k 1 m k k k k m k      T 1 [23] since (Kk)-1Ek would approach (Kk)-1 and Dk would approach the identity matrix thus avoiding the extra computations and decreasing the execution speed. Unfortunately, as k is increased substantially we find that the closure of the constraints fk using equation [23] becomes more difficult since less emphasis is placed on reconciling the measurements. Conversely, if k is not chosen large enough, where the norm of (Kk)-1Bk does not tend toward zero, then an accurate solution may also not be obtained using equation [23] and we will not be able to avoid including Dk and Ek to ensure a realistic solution. Therefore, a concession must be made concerning the choice of k if we are to simplify the computing requirements. Fortunately, estimating k using the largest trace (i.e., matrix diagonal sum) between AkQAkT and ny x BkBkT has been used successfully as we shall observe in the examples to follow i.e.,     k T y T tr n tr max ,A QA B Bk k k k [24] This expression provides a numerically acceptable compromise which has been modified from a similar approach of Press et. al. (1994) who suggest the ratio of the traces to impose an equal concern regularization in ridge regression types of parameter estimation techniques (i.e., in the absence of equality constraints). Given that it is well known that if the uncertainty of a measured variable is substantially larger than that compared to the other measured variables’ uncertainties, then it is expected that the reconciled result for that variable will be close to that if it were to be explicitly made unmeasured. Essentially, this is our proposed approach using a suitably chosen k though by re-deriving the reconciliation solution to obtain equations [17] and [18], has delineated how choosing different k ’s will allow us to simplify the solution (i.e., equation [23]). This type of comprise is not new nor unexpected in regularization type solutions. Press et. al. (1994) articulate that these problem methods involve a trade-off between two optimizations: agreement between data and solution and smoothness or stability of the solution where the regularization parameter must adjudicate a delicate compromise between the two. Therefore, in order to further aid in the determination of a suitable k , the following relationship has been derived and is detailed in the appendix.
  12. 12. 12   tr n n T g yA QA Kk k k     1 12 0, ; k  [25] This expression implies that as k becomes large, the trace found above approaches the difference between the number of equality constraints and the number of independent unmeasured variables found in Bk (i.e., ny,12). It has been observed that a suitable k is reached when this computed trace nears the value (ng - ny,12) which is demonstrated in the examples. Thus equation [24] can be used as an initial guess which can be easily checked as being suitable if expression [25] is sufficiently satisfied (i.e., within 1% of (ng - ny,12) for example) else it can be increased accordingly. It should be noted that this type of a priori determination of k has not been found in the literature for other regularization approaches and does not necessarily require adjustment at each iteration. Moreover, it is known that these regularization methods have the ability to smooth or stabilize a solution. Though this may be true, no benefit is claimed nor has been investigated in order to exploit such a property, but may in the future be the subject of further research. The fourth issue, which is somewhat concomitant with the choice of k , is to scale the constraints and reconciled and unmeasured variables so as to give comparable importance to each in the solution. Nguyen et. al. (1988) presented a brief but relevant review of several scaling methods however their focus was on optimally scaling multivariable steady-state gain matrices when analysing model-based process controllers. The basic premise of scaling is to somehow “balance” the required matrices via pre and post multiplying by appropriate diagonal scaling matrices in order to increase the numerical robustness or stability of the solution. Hence at each iteration we may choose to scale the system thereby solving the following successive linearization sub-problem min w.r.t. s.t. (x x ) S Q S (x x ) x S A S x S B S y S f 0 m k x k 1 x k m k k f k k x k k f k k y k k f k k             T T [26] where  x S xk x k k  1 ,  y S yk y k k  1 and Sf k, Sx k and Sy k are the diagonal scaling matrices for the constraints, reconciled and unmeasured variables respectively. And from Nguyen et. al. (1988), we can compute these matrices at each iteration using the Euclidean norms of the rows and columns of the augmented Jacobian matrix [Ak Bk] as follows     S A B f k k kii i gi n   1 1 2  [27]
  13. 13. 13     S A B x k k kjj j xj n   1 1 2  [28] and     S A B y k k kjj j x x yj n n n     1 1 2 ( ) ( ) [29] where the authors also recommend that more than one pass at computing the scaling matrices may be required to improve the balancing of the scaled matrices Ak and Bk at each iteration. Although we did not employ nor require scaling of any kind for our examples, scaling may in some cases prove to be very beneficial and has been included here to provide an avenue for the data reconciliation practitioner pending numerical difficulties. The fifth issue concerns the computational effort required to arrive at the solution using the method defined by equations [16] and [23]. Since the majority of the effort for these problems is concentrated in the factorization of the matrices requiring inversion (Crowe, 1986), this will be the focus of our argument. Given that the proposed method’s order of operations to factorize the kernel matrix is circa ng 3 , its factorization can be costly since the method of Pai and Fisher (1988) can reduce the dimension of the inverse inside its kernel matrix to a size of (ng-ny,12) x (ng-ny,12). Notwithstanding, in order to determine the partitions found in equation [11], we require some technique, and possibly a matrix factorization technique, to identify how the matrix Bk will be partitioned. If we assume the technique is Cholesky factorization, then we must first decompose BkT Bk then BkBkT to determine and remove any dependent vectors found in these matrices. Since the factorization of BkT Bk is also required by the proposed method (i.e., equation [16]), there appears to be no advantage nor disadvantage. Concerning BkBkT , which will always be of the same order as the proposed method’s kernel matrix Kk, its factorization will effectively require the same amount of calculations (ignoring a sparse matrix implementation). However, the row and column permutation matrices required to partition Bk in equation [11] may not change from iteration to iteration and consequently the factorization of BkBkT may not be necessary at each iteration though this has never been mentioned as an efficiency improvement in previous works. Once the factorizations have occurred, then to complete the solution of the proposed method, forward and backward substitutions are then required using the determined factors (i.e., for both inverses found in equations [16] and [23]). As for the matrix projection method, equation [9]’s kernel matrix must be factored and solved at each iteration where equations [10] and [12] require factorization and forward and backward solving involving B1k. Therefore from the brief discussion, it appears that even though the matrix projection can in some cases substantially reduce the size of
  14. 14. 14 the problem as noted by Crowe (1986), its computation does not come without cost and for some problems, may require at least conceptually, as many arithmetic operations as the proposed method if BkBkT must be factored at each iteration. Unfortunately, the relative execution speed of one method over the other for sparse flowsheets is problem specific due to differences in the sparsity patterns of the individual matrices, and thus a realistic and general comparison is not possible. It should also be mentioned that for large problems the technique known as iterative improvement of a solution (Press et. al., 1994; Golub and Van Loan, 1983) may be required for both methods to improve the solution estimates. Due to accumulating round-off errors occurring in both the solution vector and the Cholesky factors, iterative improvement may be necessary to improve the numerical accuracy of the final results. As a matter of insight, other methods are available to solve these class of problems besides the direct factorization methods mentioned. These techniques, called iterative solution methods (Axelsson, 1996) such as the congugate gradient method, may prove to be highly efficient alternatives for large sparse problems however more research is required to assess their benefit. Estimating the Variance-Covariance Matrices Using expression [17] for the reconciled variables and expression [16] for the unmeasured variables, we can approximate the following matrices at the converged solution to be H D QA K EAQA EK AQDa 1 1     1 1T T [30] and        H B B B I AD QA K E AQA I AD QA K E B B By 1 g T g 1          T T T T T T1 1 1 1 [31] where we have neglected the superscript k since we are dealing with the final solution. Here, Ha and Hy are the adjusted and unmeasured variables’ variance-covariance matrices respectively. And, in the same manner as before, if dependent columns in B are found, then B is replaced by B12 where the variance-covariance matrix for the original unmeasured variances can be approximated by  H P P H 0 0 I P P y 12 C 34 C y,12 y,34 12 C 34 C                T T [32] and the variances for the dependent unmeasured variables are arbitrarily set equal to the final converged value of  . Furthermore when all of the non-observable variables have been determined, using the technique to be presented in the next section, their variances must be reset to  as well. From a computational point of view, full computation of the variance- covariances is not explicitly required since a lower triangular Cholesky factor of Q can be found i.e.,
  15. 15. 15 Q R RQ Q T [33] where the matrix square roots of Ha and Hy are straightforward to obtain. In fact, when more sophisticated gross error detection methods are used such as principle component analysis (Tong and Crowe, 1995), the matrix square roots can be used directly to determine the eigenvalues and eigenvectors of the variance-covariance matrices. Similar to Ha and Hy, the variance-covariance matrix of the constraint residuals is formulated as a function of the adjusted variables’ variance-covariance matrix H AH Ag a T [34] where these residuals are defined as g f(x , y,z) Ax By Cz Axm m a     [35] at the final converged solution. The matrix C is the Jacobian of the constraints with respect to the fixed variables z and is computed in the same way A and B are formed; however C is not actually required and only shown for insight. The above variance-covariance matrices may then be used in the detection and identification of gross errors which were briefly discussed in the Introduction. Furthermore, since we are only proposing to use equation [23] discussed above, these matrices can be simplified to yield very tractable estimates of Ha and Hy as H QA K AQa  T 1 [36] and  H I B K By y     T 1 [37] These expressions can be easily derived by re-formulating the solution at convergence as   x y Q 0 0 I A B K Ax Cz a y m                      T T 1 [38] where as a matter of insight, the estimation of the unmeasured variables at each iteration may be approximated by    y y B K f A (x x )k 1 k k k k m kk     k T 1 [39] Interestingly, since no inversion of BT B is required in equations [37] and [39] above, and since the kernel matrix changes negligibly even if there exist dependent columns in B, these expressions are readily available without partitioning (i.e., equation
  16. 16. 16 [20]). However, as mentioned previously, if there are any identified non-observable variables then their variances must be reset to  and their covariances reset to zero for lack of any other a priori information. In addition, it may also be useful to know the variance-covariance matrix of the reconciled variables; this can be easily computed as Q minus Ha. And further, given that any data reconciliation problem employing a quadratic objective function is well suited to a SQP implementation, the task of computing estimates of the variance-covariances and classifying the measured and unmeasured variables must still be performed external to its solution (Albuquerque and Biegler, 1996; Kim et. al., 1997). Yet, with our’s and Crowe’s methods, a substantial amount of the required computations are performed already in the path to the solution (i.e., the inverse of the kernel matrix) in addition to the computations necessary to compute the check found in [25]. Finally, in order to assess when the solution has converged, a suitable stopping criterion must be defined. One approach is to compute the Euclidean norm of the linearized constraint residuals of equation [4] which should be less than a specified tolerance where its choice is dependent on the order of magnitude of the constraints (cf. Buzzi-Ferraris and Tronconi, 1993, for an interesting discussion on convergence acceptance in the solution to nonlinear sets of equations). Another Approach to the Classification of Variables After the reconciliation has completed, it is important to classify the unmeasured and measured variables into observable and redundant variables respectively. A comprehensive description and review has been given by Crowe (1989) who discusses the observability and redundancy terms in detail. The intent of this section is to describe another technique to classify these variables without employing the use of the matrix projection. Since the efficacy of the proposed method depends on not computing the matrix projection, the techniques shown below allow the user to classify the variables using matrices already available from the final reconciliation solution. We first discuss the classification of the unmeasured variables. Stated succinctly, an unmeasured variable is observable if it can be uniquely determined from the available measurements and model constraints. If it is not, then any calculated value cannot be trusted and it is defined to be non-observable. In order to identify these variables, we can re-write equation [4] as  Ax B B y y Cz 012 34 12 34          [40] using the partitions of B and the product C times z which combined above should be near zero upon convergence. If we again use the method of generalized inverses, we can solve for y12 yielding    y B B B Ax Cz B y12 12 12 12 34 34    T T1 [41]
  17. 17. 17 Thus, if any row of the matrix  B B B B12 12 12 34 T T1 contains non-zero elements, then the unmeasured variables associated with y12 are declared non-observable since they are functions of the other dependent non-observable variables contained in y34. This result is similar to those presented by Crowe (1989), Sanchez and Romagnoli (1996) and Albuquerque and Biegler (1996), however we do not require the identification of dependent constraints in the matrix B as required by the other authors. Further, the matrix contained in the above equation is partially determined since we already have the Cholesky factors of B12 T B12 from the solution of equation [16]. The next step is to determine redundant and non-redundant variables. By definition, redundant variables are those measurements which could be uniquely calculated from the remaining measurements if they were not measured. Non- redundant variables are simply those measurements which are not redundant. With the aid of the matrix projection, identifying redundant variables is quite easy since zero columns in UT A correspond to non-redundant measurements. This is a direct result of the fact that non-redundant variables would translate into dependent columns in B if a measurement were not available. Unfortunately, the matrix projection is not available in this implementation and thus another strategy is required. If we use the definition of a redundant reconciled variable literally, then by incrementally testing one column of A at a time (i.e., similar to identifying dependent unmeasured variables) we can use the technique of matrix inversion by partition found in Press et. al. (1994) to identify measurement redundancy. Hence, equation [42] below describes the matrix that would be required to be inverted in equation [16] if the j th measurement were not available i.e., B B B A A B A A 12 12 12 12 T T j j T j T j [ ] [ ] [ ] [ ]         [42] We know that if the j th column of A, denoted as [A]j , is linear dependent on the other columns contained in B12, then by applying the updated inversion technique to the above, we will obtain for the inverse           B B B B B A A B B B s B B B A s A B B B s s 12 12 12 12 12 12 12 12 12 12 12 12 12 12 T T T j j T T j T T j j j T T j j                 1 1 1 1 1 1 [ ] [ ] /[ ] [ ] /[ ] [ ] /[ ] /[ ] [43] where a value near or equal to zero for  [ ] [ ] [ ] [ ] [ ]s A A A B B B B A12 12 12 12j j T j j T T T j  1 [44]
  18. 18. 18 can indicate a singular update. Thus the j th reconciled variable can be sufficiently declared as non-redundant if [s]j is effectively zero. Equation [44] is referred to as the Schur complement and is an underlying structure found in many direct linear algebra solution methods (Golub and Van Loan, 1983). In essence, checking [s]j is identical to ensuring that the determinant of equation [42] will not be zero (i.e., indicating a singularity) since this determinant is equal to the determinant of B12 T B12 times [s]j (Kelly, 1998). In addition, if we re-formulate equation [44] into matrix format, the vector s may be expressed as      s A I B B B B A A EAg 2 12 12 12    diag diagT T T T 1 1 [45] which is interesting since E is a projection matrix in its own right being symmetric and idempotent (Householder, 1975; Axelsson, 1996) where if B is a maximal square matrix then s is zero for all measured variables. This is consistent since no reconciliation is possible and all measured variables are non-redundant. In order to minimize the accumulation of numerical round-off errors, we again recommend the use of Cholesky factorization to compute equation [44] (and equation [41]) instead of the direct inversion of B12 T B12 as shown above. This can be accomplished easily and efficiently by forward solving for [V]j in R V B AB,12 12[ ] [ ]j T j [46] since RB,12 is the lower triangular Cholesky factor of B12 T B12 where [s]j may now be re-written as [ ] [ ] [ ] [ ] [ ]s A A V Vj j T j j T j  [47] Finally, the above analysis to classify unmeasured and reconciled variables as observable and redundant has used the final converged set of Jacobian estimates A and B. Unfortunately for nonlinear problems , these matrices are only point estimates of the underlying state of the modeled process at the converged solution and may be further degenerate by their numerical update strategy. Therefore, careful interpretation of the results is necessary which may even involve performing an ad hoc sensitivity analysis on the point estimates if needed (i.e., perturbing x and y slightly, re-computing A and B then performing the above analysis to verify the stationarity of the results), though for large problems this may be impractical. Fortunately for linear systems, where the Jacobians are constant, the results will be invariate and not dependent on the tested operating space of the process.
  19. 19. 19 Illustrative Examples Presented below are three small examples illustrating the proposed method. The first 2 examples are both nonlinear (in fact they are bilinear) and the third involves a linear mass balance. The product MATLAB® Version 4.2c.1 from The MathWorks Inc. was used to program the examples where MATLAB®’s standard matrix inversion routines were used to solve for the linear affine equations. Example 1. This example involves the reconciliation of copper and zinc concentration measurements from a mineral floatation circuit presented by Smith and Ichiyen (1973). The measurement values were taken from Tong (1995) who solved the problem using the bilinear reconciliation method of Crowe (1986). Figure 1 describes the process flowsheet and Table 1 details the measured, unmeasured and fixed variables. There are a total of 8 streams in the process with 4 unit operations where the number of measured variables is 14 and the number of unmeasured variables is 9. There is 1 fixed or constant variable which represents the total flow rate of stream one and in total there are 12 functions or equalities. A separate table is also presented, Table 2, which details the constraint functions and their associated Jacobian elements where the variance- covariance matrix Q is diagonal with elements corresponding to a relative standard deviation of 6.56%. In addition, the initial conditions for the x and y variables were their measured values and a vector of ones respectively. Table 1 Variable Definitions for Example 1. X y z 1 Copper in Stream 1 Copper in Stream 8 Flow of Stream 1 2 Copper in Stream 2 Zinc in Stream 8 3 Copper in Stream 3 Flow of Stream 2 4 Copper in Stream 4 Flow of Stream 3 5 Copper in Stream 5 Flow of Stream 4 6 Copper in Stream 6 Flow of Stream 5 7 Copper in Stream 7 Flow of Stream 6 8 Zinc in Stream 1 Flow of Stream 7 9 Zinc in Stream 2 Flow of Stream 8 10 Zinc in Stream 3 11 Zinc in Stream 4 12 Zinc in Stream 5 13 Zinc in Stream 6 14 Zinc in Stream 7 Table 2 Function and Jacobian Definitions for Example 1. Function A B C f(1) = x(1)z(1)-x(2)y(3)-x(5)y(6) A(1,1) = z(1), A(1,2) = -y(3), A(1,5) = -y(6) B(1,3) = -x(2), B(1,6) = -x(5) C(1,1) = x(1) f(2) = x(8)z(1)-x(9)y(3)-x(12)y(6) A(2,8) = z(1), A(2,9) = -y(3), A(2,12) = -y(6) B(2,3) = -x(9), B(2,6) = -x(12) C(2,1) = x(8) f(3) = z(1)-y(3)-y(6) - B(3,3) = -1, B(3,6) = -1 C(3,1) = 1 f(4) = x(2)y(3)-x(3)y(4)-y(1)y(9) A(4,2) = y(3), A(4,3) = -y(4) B(4,1) = -y(9), B(4,3) = x(2), -
  20. 20. 20 B(4,4) = -x(3), B(4,9) = -y(1) f(5) = x(9)y(3)-x(10)y(4)-y(2)y(9) A(5,9) = y(3), A(5,10) = -y(4) B(5,2) = -y(9), B(5,3) = x(9), B(5,4) = -x(10), B(5,9) = -y(2) - f(6) = y(3)-y(4)-y(9) - B(6,3) = 1, B(6,4) = -1, B(6,9) = -1 - f(7) = x(3)y(4)-x(4)y(5)-x(7)y(8) A(7,3) = y(4), A(7,4) = -y(5), A(7,7) = -y(8) B(7,4) = x(3), B(7,5) = -x(4), B(7,8) = -x(7) - f(8) = x(10)y(4)-x(11)y(5)- x(14)y(8) A(8,10) = y(4), A(8,11) = -y(5), A(8,14) = -y(8) B(8,4) = x(10), B(8,5) = -x(11), B(8,8) = -x(14) - f(9) = y(4)-y(5)-y(8) - B(9,4) = 1, B(9,5) = -1, B(9,8) = -1 - f(10) = x(5)y(6)- x(6)y(7)+y(1)y(9) A(10,5) = y(6), A(10,6) = -y(7) B(10,1) = y(9), B(10,6) = x(5), B(10,7) = -x(6) - f(11) = x(12)y(6)- x(13)y(7)+y(2)y(9) A(11,12) = y(6), A(11,13) = -y(7) B(11,2) = y(9), B(11,6) = x(12), B(11,7) = -x(13), B(11,9) = y(2) - f(12) = y(6)-y(7)+y(9) - B(12,6) = 1, B(12,7) = -1, B(12,9) = 1 - Table 3 below, shows the objective function value J as a function of  starting from 1 to 10 million for the simplified solution using equations [16] and [23] and for the more rigorous solution using equations [16] and [17] (note that  was fixed for each run for illustrative purposes only though it may be allowed to change at each iteration as described in equation [24]). The term “flops” is MATLAB®’s estimation of the floating point operations performed for the simplified solution and k refers to the number of iterations required for both solutions to satisfy a equality constraint norm tolerance of 10-6. The final item in the table records the objective function value, the number of flops and iterations required for Pai and Fisher (1988)’s method (i.e., equations [9] and [10] without using their line search strategy). For this example, only the first iteration involved determining the matrix projection permutation matrices PC and PR using the Cholesky factorization identification method of Kelly (1998) where a cut-off tolerance (i.e., machine precision estimate) of 10-9 for determining dependencies was used. Though the matrix projection was computed at each iteration using MATLAB®’s matrix inversion routine to find U in equation [12], PC and PR were only required to be determined once at the start of the solution. Table 3 Effect of  on the Solution Results for Example 1.  J Eq [16,23] flops k J Eq [16,17] k Eq [25] 1 20.62137302 - - 17.9616397 22 4.95273 10 18.00556583 - - 17.9616397 22 4.64172 100 17.9616423 1217772 51 17.9616397 22 3.82171 1,000 17.9616400 620872 26 17.9616397 22 3.17081 10,000 17.9616399 525368 22 17.9616397 22 3.01942 49,500 17.9616400 525368 22 17.9616397 22 3.00397 100,000 17.9616392 525638 22 17.9616397 22 3.00196 1,000,000 17.9616424 525638 22 17.9616414 19 3.00019 10,000,000 17.9616191 1217772 51 17.9614510 34 3.00002 J Eq [9,10] 17.9619397 534384 22 - - - 2 Tolerance was relaxed to 10-3 to allow convergence. 3 Tolerance was relaxed to 10-4 to allow convergence.
  21. 21. 21 From the above table we observe that the more rigorous method is very insensitive to the choice of  and converges easily, however for small  ’s, this is not the case for the simplified method. The shaded row in the table identifies the trace value found in equation [24] having an initial nominal value of 49,500 and shows a consistent value for the objective function. In addition, as can be seen from the results, the objective function values for all three solutions are reasonably close indicating that no solution bias exists. We also notice from the table that the flop count for the simplified method is slightly smaller than that for the matrix projection method. In fact, when the standard sparse matrix option is used in MATLAB® we arrive at a flop count of 64,332 for the simplified method and 86,970 for our implementation of the matrix projection method which translates into a 26% reduction in the required arithmetic operations. The last column of Table 3 follows the path of expression [25]. Given 12 equality constraints and 9 identified independent unmeasured variables, the last column values indeed approach 3 as  is increased in light of the fact that A is not of full row rank. Table 4 Solution Values for Example 1 with  =49,500. xm x [Ha]jj ½ [s]j y [Hy]jj ½ Eq. [41] [RB]jj2 1 1.930 1.917 5.730e-03 1.226e-03 34.104 1.069e+01 0 1.560e-04 2 0.450 0.451 2.945e-04 1.075e-03 -13.147 8.500e+00 0 1.560e-04 3 0.130 0.126 6.721e-03 8.325e-01 0.925 8.244e-03 0 3.146e+01 4 0.090 0.092 2.960e-03 7.027e-01 0.916 7.342e-03 0 2.953e+01 5 19.860 19.959 5.330e-02 7.099e-06 0.842 8.907e-03 0 8.940e-01 6 21.440 21.447 3.020e-02 4.736e-09 0.075 8.244e-03 0 5.548e+02 7 0.510 0.515 8.409e-03 5.502e-03 0.084 7.343e-03 0 1.585e+02 8 3.810 4.603 1.888e-01 5.024e-02 0.075 5.776e-03 0 7.255e+02 9 4.920 4.408 2.684e-01 6.350e-02 0.009 2.852e-03 0 1.331e+01 10 5.360 4.578 3.022e-01 1.843e-02 11 0.410 0.410 5.069e-04 4.786e-05 12 7.090 7.004 4.539e-02 4.193e-04 13 4.950 4.885 2.421e-02 1.544e-04 14 52.100 51.681 7.243e-01 3.748e-07 Table 4 details the solution values for a  of 49,500 where three significant digits were used for xm, x and y for the sole purpose of presentation and if substituted in the equations may not satisfy the specified tolerance (i.e., 10-6). In this case no dependent unmeasured variables were detected, as indicated by the square of the diagonal of RB and the value of expression [25], where all of the reconciled variables are redundant as indicated by s (i.e., equation [47]). These results are consistent with those presented by the previous authors however reconciled variable number 6 is close to being declared non-redundant if we were to assume a cut-off tolerance greater than 10-9. In addition, we also notice a negative zinc concentration in stream 8 as highlighted by the shading for unmeasured variable number 2; this was also pointed out by Crowe (1986) who detailed this
  22. 22. 22 example as well. Given the variance of that estimate, derived from equation [37], we notice that this value would not pass a 95% confidence interval (i.e., +/- 2 x 8.500) and consequently its value must not be trusted even though it is uniquely determinable from the measurements. From this, we could proceed to a more detailed gross error detection and identification study to determine the cause of the negative aberrant, however our focus is on the solution of the problem and this will not be pursued further. Example 2. The second example is taken from Swartz (1989) who studied a small heat exchanger circuit which is presented in Figure 2 and has also been detailed previously by Tjoa and Biegler (1991) and Albuquerque and Biegler (1996). We use the same convergence and cut-off tolerances as those of the first example however the initial maximum trace is estimated to be 78.4 million which could be reduced if the previously mentioned scaling technique were to be employed. Tables 5 and 6 describe the variables, functions and Jacobian elements where no fixed or constant variables exist in this example. The initial conditions for the reconciled variables are the measured values where default values of 500 for the unmeasured variables are used as suggested by Swartz (1989). The measurement variance-covariance matrix is diagonal with standard deviations for temperatures of 0.75 absolute and 2% of the measured value for flows. Table 5 Variable Definitions for Example 2. x y 1 Flow of Stream A1 Flow of Stream A2 2 Temperature of Stream A1 Temperature of Stream A2 3 Flow of Stream A3 Flow of Stream A4 4 Temperature of Stream A3 Flow of Stream A5 5 Temperature of Stream A4 Temperature of Stream A6 6 Temperature of Stream A5 Flow of Stream A7 7 Flow of Stream A6 Flow of Stream A8 8 Temperature of Stream A7 Flow of Stream B2 9 Temperature of Stream A8 Temperature of Stream B2 10 Flow of Stream B1 Flow of Stream B3 11 Temperature of Stream B1 Temperature of Stream B3 12 Flow of Stream C1 Flow of Stream C2 13 Temperature of Stream C1 Temperature of Stream C2 14 Temperature of Stream D1 Flow of Stream D1 15 Flow of Stream D2 16 Temperature of Stream D2 Table 6 Variable Definitions for Example 2. Function A B f(1) = x(1)-y(1)) A(1,1) = 1 B(1,1) = -1 f(2) = y(8)-y(10) - B(2,8) = 1, B(2,10) = -1 f(3) = x(1)(y(2)-x(2))-y(8)(y(9)-y(11)) A(3,1) = (y(2)-x(2)), A(3,2) = -x(1) B(3,2) = x(1), B(3,8) = -(y(9)-y(11)), B(3,9) = -y(8), B(3,11) = y(8) f(4) = y(1)-x(3)-x(7) A(4,3) = -1, A(4,7) = -1 B(4,1) = 1 f(5) = y(2)-x(4) A(5,4) = -1 B(5,2) = 1
  23. 23. 23 f(6)= y(2)-y(5) - B(6,2) = 1, B(6,5) = -1 f(7) = x(3)-y(3) A(7,3) = 1 B(7,3) = -1 f(8) = x(10)-y(8) A(8,10) = 1 B(8,8) = -1 f(9) = x(3)(x(5)-x(4))-x(10)(x(11)-y(9)) A(9,3) = (x(5)-x(4)), A(9,4) = -x(3), A(9,5) = x(3), A(9,10) = -(x(11)-y(9)), A(9,11) = -x(10) B(9,9) = x(10) f(10) = y(3)-y(4) - B(10,3) = 1, B(10,4) = -1 f(11) = x(12)-y(12) A(11,12) = 1 B(11,12) = -1 f(12) = y(3)(x(6)-x(5))-x(12)(x(13)-y(13)) A(12,5) = -y(3), A(12,6) = y(3), A(12,12) = -(x(13)-y(13)), A(12,13) = -x(12) B(12,3) = (x(6)-x(5)), B(12,13) = x(12) f(13) = x(7)-y(6) A(13,7) = 1 B(13,6) = -1 f(14) = y(14)-x(15) A(14,15) = -1 B(14,14) = 1 f(15) = x(7)(x(8)-y(5))-y(14)(x(14)-x(16)) A(15,7) = (x(8)-y(5)), A(15,8) = x(7), A(15,14) = -y(14), A(15,16) = y(14) B(15,5) = -x(7), B(15,14) = -(x(14)-x(16)) f(16) = y(4)+y(6)-y(7) - B(16,4) = 1, B(16,6) = 1, B(16,7) = -1 f(17) = y(4)x(6)+y(6)x(8)-y(7)x(9) A(17,6) = y(4), A(17,8) = y(6), A(17,9) = -y(7) B(17,4) = x(6), B(17,6) = x(8), B(17,7) = -x(9) Table 7 Effect of  on the Solution Results for Example 2.  J Eq [16,23] flops k J Eq [16,17] k Eq. [25] 1 3230.931044 - - 14.5861656 5 10.439026 10 229.5708045 - - 14.5861656 5 8.294295 100 14.5861740 1858238 33 14.5861656 5 5.437646 1,000 14.5861656 563177 10 14.5861656 5 3.512451 10,000 14.5861656 337949 6 14.5861656 5 3.059837 100,000 14.5861656 281642 5 14.5861656 5 3.006092 1,000,000 14.5861656 281642 5 14.5861656 5 3.000610 10,000,000 14.5861656 281642 5 14.5861656 5 3.000061 78,400,000 14.5861656 281642 5 14.5861656 5 3.000008 100,000,000 14.5861661 281642 5 14.5861656 5 3.000006 1,000,000,000 14.5861608 563177 10 14.5861653 8 3.000001 J Eq [9,10] 14.5861656 304163 5 - - - In Table 7, which is similar to Table 3, the flop count and the number of iterations for the simplified method are recorded. As well, the objective function values are shown for all three methods. The shaded row in the table highlights the solution using the initial maximum trace, where as before, the objective function values and iteration counts are almost identical for each method though the first 2 solutions for the simplified method did not converge at the specified tolerance as noted. Further, we notice that the flop count for the simplified method is lower than the matrix projection method and when standard sparse storage and computations are used, the flop count for the simplified method is 16,482 and 17,742 for the matrix projection method with the same number of iterations indicating a 7% reduction. However, similar to Example 1, only the first 4 iterations were found to be sufficient to correctly identify the permutations matrices required for the computation of the 4 Tolerance was relaxed to 0.5 to allow convergence. 5 Tolerance was relaxed to 10-1 to allow convergence.
  24. 24. 24 matrix projection, which had this improvement not be taken, the flop count would have been higher. In addition, Swartz (1989) quoted a iteration count of 4 (as opposed to 5 in our example) however the tolerance used by Swartz (1989) was greater than 10-6. With regard to the last column of Table 7, it is apparent that expression [25] converges to 3 which is not surprising since there are 17 constraints and 14 independent unmeasured variables and again A is not of full row rank. Interestingly, when expression [25] is within 1% of its expected value, we observe that a  of 100,000 seems stable to provide a stable solution. Table 8 presents the results using the initial maximum trace solution where the numerical results do not match exactly with those found by Swartz (1989). Unfortunately no information was given by Swartz (1989) regarding the heat capacities of the streams hence defaults of unity for each heat capacity were used throughout our example. Table 8 Solution Values for Example 2 with  =78,400,000. xm x [Ha]jj½ [s]j y [Hy]jj½ Eq. [41] [RB]jj2 1 1000.00 964.19 1.730e+01 5.000e-01 964.19 1.004e+01 0 2.000e+00 2 466.33 466.33 1.872e-06 2.328e-10 481.91 7.364e-01 0 9.297e+05 3 401.70 407.77 2.787e+00 5.000e-01 407.77 7.531e+00 0 7.298e+03 4 481.78 481.91 1.421e-01 4.905e-01 407.77 7.531e+00 0 3.789e+05 5 530.09 530.09 1.103e-06 5.821e-11 481.91 7.364e-01 0 3.096e+05 6 616.31 615.51 2.591e-01 4.370e-01 556.42 7.826e+00 0 2.007e+00 7 552.70 556.42 7.801e+00 5.293e-01 964.19 1.004e+01 0 5.003e-01 8 619.00 617.78 3.682e-01 1.304e+00 253.20 5.060e+00 0 2.008e+00 9 614.92 616.82 6.127e-01 2.444e+00 540.51 2.821e+00 0 6.411e+04 10 253.20 253.20 5.637e-05 2.729e-12 253.20 5.060e+00 0 5.019e-01 11 618.11 618.11 6.787e-07 1.455e-11 481.19 4.894e+00 0 1.369e-01 12 308.10 308.10 4.780e-05 0 308.10 6.160e+00 0 1.000e+00 13 694.99 694.99 3.463e-07 -1.455e-11 581.94 3.448e+00 0 1.301e+01 14 667.84 668.00 1.758e-01 7.507e-01 688.42 9.996e+00 0 1.020e+00 15 680.10 688.42 9.222e+00 1.910e-02 16 558.34 558.18 1.758e-01 7.507e-01 From the table, as indicated by the shaded cells, we notice that there are six non-redundant variables and no non-observable variables. These non-redundant variables match exactly to those found by Swartz (1989) where it should be noted that if non- redundant variables are identified then their estimated adjustment variances (i.e., diag(Ha)) should be set to zero. From a gross error detection perspective, Swartz (1989) performed a second simulation which deleted the temperatures in streams A4 and A7 in order to reduce the overall Chi-square statistic (i.e., the objective function) to within an acceptable level. Table 9 displays our results for this reconciliation where two new variables were appended to the vector of unmeasured variables. Specifically, stream A4’s temperature is now unmeasured variable number 15 and stream A7’s temperature is now unmeasured variable number 16 where the reconciled variables are displaced accordingly. Table 9 Solution Values when Temperatures in Streams A4 and A7 are Deleted (J=3.5515).
  25. 25. 25 xm x [Ha]jj½ [s]j y [Hy]jj½ Eq. [41] [RB]jj2 1 1000.00 969.30 1.720e+01 5.000e-01 969.30 1.020e+01 0 2.000e+00 2 466.33 466.33 2.630e-06 0 481.77 7.372e-01 1.072e-14 9.395e+05 3 401.70 406.65 2.784e+00 5.000e-01 406.65 7.532e+00 4.108e-12 3.798e+05 4 481.78 481.77 1.381e-01 3.094e-01 406.65 7.532e+00 2.348e-12 3.798e+05 5 616.31 616.30 9.978e-02 1.616e-01 481.77 7.372e-01 1.326e-14 3.166e+05 6 552.70 562.65 7.553e+00 5.173e-01 562.65 8.065e+00 8.718e-17 1.992e+00 7 614.92 614.94 2.378e-01 9.182e-01 969.30 1.020e+01 9.052e-12 4.997e-01 8 253.20 253.20 2.214e-04 2.328e-10 253.20 5.060e+00 1.219e-18 2.007e+00 9 618.11 618.11 1.375e-06 7.276e-12 1391.80 7.491e+00 1.606e+00 6.411e+04 10 308.10 308.10 3.139e-04 0 253.20 5.060e+00 1.212e-17 5.019e-01 11 694.99 694.99 1.567e-06 0 1332.70 6.840e+00 1.606e+00 1.355e-01 12 667.84 667.83 1.667e-01 4.511e-01 308.10 6.160e+00 0 1.000e+00 13 680.10 679.38 8.833e+00 1.171e-02 -118.44 1.560e+01 1.320e+00 2.499e-01 14 558.34 558.35 1.667e-01 4.511e-01 679.38 1.034e+01 1.9146e-15 1.019e+00 15 0 5.600e+06 - 6.905e-11 16 613.95 1.312e+00 1.858e-15 1.328e+00 By deleting these two variables, the objective function is now 3.5515 which is in line with the threshold value, however since stream A4’s temperature was a non-redundant variable, we observe from Table 9’s last column that unmeasured variable 15 is identified as being dependent (and non-observable). The shaded cells of Table 9 are exactly consistent with the results of Swartz (1989) which indicate non-redundant and non-observable variables. The three shaded cells in the second last column were estimated by finding the largest absolute value in each row of the matrix found in equation [41] which identify the independent although non-observable unmeasured variables. Example 3. The third and final example can be found in Sanchez and Romagnoli (1996) and is a linear material balance consisting of 29 measured variables, 34 unmeasured variables and 31 balances of which only 8 are redundant (and confirmed by expression [25]). This example is included to further verify the accuracy of the classification of variables technique which does not rely on the matrix projection to be computed. Table 10 highlights the results which are exact when compared to the values presented by the above authors where we have used the same stream number designations. The final objective function value is computed at 26.1236 using a  of 2,300 which was found to be adequate where equations [16] and [23] were used.
  26. 26. 26 Table 10 Solution Values for Example 3 with  =2,300 (J=26.1236). Stream xm x [s]j Stream y Eq. [41] [RB]jj2 1 1 70.490 70.104 0.3333 3 0.0000 1.000 2.000e+00 2 2 7.103 7.096 0.3333 4 70.104 1.000 1.500e+00 3 7 13.040 11.980 0.8333 5 0 - 0 4 8 35.380 35.540 0.8333 6 0 - 0 5 9 53.210 53.640 1.333 10 -11.980 1.000 2.000e+00 6 12 23.900 23.560 0.5000 11 0 - 0 7 13 0 -0.0241 1.000 34 41.881 0 2.000e+00 8 14 0.0765 -0.1515 1.000 35 18.876 0 1.500e+00 9 15 54.590 53.816 1.200 36 13.235 0 1.333e+00 10 16 12.780 11.934 1.200 24 4.703 0 1.250e+00 11 17 23.420 23.005 1.200 63 0.0000 1.000 2.000e+00 12 18 0.2378 0.2278 1.200 61 24.203 1.000 1.500e+00 13 19 8.657 8.618 1.000 38 0.0000 1.000 1.333e+00 14 20 5.087 5.413 1.200 55 24.203 1.000 1.250e+00 15 21 1.740 1.787 0.2000 54 0.0000 1.000 1.200e+00 16 22 0.0255 0.0262 1.200 48 24.203 1.000 1.167e+00 17 23 3.113 3.178 1.200 56 0.0000 1.000 1.143e+00 18 25 5.407 5.354 1.200 57 0 - 2.220e-16 19 26 2.898 2.890 1.200 58 0.0000 1.000 1.125e+00 20 27 11.830 13.239 1.000 60 0.0000 1.000 1.111e+00 21 28 8.197 8.907 1.059 59 0 - 4.441e-16 22 29 1.364 1.393 1.059 62 0.0000 1.000 1.100e+00 23 30 20.940 19.872 2.000 47 4.595 1.000 1.091e+00 24 31 1.051 1.069 1.059 49 19.608 0 1.083e+00 25 32 12.580 13.465 1.059 44 0 - 0 26 33 4.999 5.338 1.059 45 4.595 1.000 1.077e+00 27 37 5.730 5.969 0.0588 51 -19.608 1.000 1.071e+00 28 46 4.250 4.595 0.0588 50 0.0000 1.000 1.067e+00 29 53 16.340 19.608 0.0588 52 0 - 0 30 42 0.0000 1.000 1.063e+00 31 43 0 - 0 32 41 0 - 4.441e-16 33 40 0 - 2.220e-16 34 39 0 - 8.882e-16 As we notice from the table, there are no non-redundant variables and of the 34 unmeasured variables only 23 are independent as indicated by not being shaded. However, of those independent unmeasured variables, we find that only five of these variables are actually observable (i.e., stream flows 24, 34, 35, 36 and 49) as determined by equation [41] which are identical to the unmeasured variables deemed observable by Sanchez and Romagnoli (1996). Conclusions In this article we have presented a new regularized solution to the reconciliation of process flowsheets, although making some practical assumptions in its derivation to mitigate the two limitations outlined in the Introduction, has proven successful in
  27. 27. 27 three examples. The new approach has been shown to be a viable and numerically efficient alternative to the matrix projection method presented by Pai and Fisher (1988), however it depends on the proper choice of the regularization parameter  . Yet, two useful guidelines were given to aid in its selection i.e., equations [24] and [25]. A detailed discussion on other issues which affect its success were also included which at least conceptually addresses the major concerns of the technique. In addition, estimates of the variance-covariance matrices for the adjustments, unmeasured variables and constraint residuals have been derived which can be used in the critical post analysis of the solution to detect significant gross errors if they exist. Further, an alternative technique for the classification of variables into observable and redundant has been detailed using readily available matrix factorizations highlighting the use of another projection matrix which is independent on the choice of  . Nomenclature A Jacobian or topology matrix for measured variables B Jacobian or topology matrix for unmeasured variables B ,B ,B B1 2 3 4& partitions of B defined by Crowe et. al.(1983) B B12 3& 4 column partitions of B C Jacobian topology matrix for fixed variables D symmetric and positive matrix found in equation [18a] E idempotent matrix found in equation [18b] f k vector of function values evaluated at iteration k g vector of constraint residuals found in equation [34] Ha variance-covariance matrix of measurement adjustments Hg variance-covariance matrix of constraint residuals Hy variance-covariance matrix of unmeasured variables i typically a row index Ig identity matrix of size ng x ng Ix identity matrix of size nx x nx Iy identity matrix of size ny x ny j typically a column index J value of quadratic objective function k iteration count K symmetric and positive definite kernel matrix found in equation [15]
  28. 28. 28 n g number of constraints nx number of measured variables n y number of unmeasured variables n y,12 number of unmeasured variables which are independent nz number of fixed variables PC column permutation matrix P P12 C 34 C & column permutation sub-matrices PR row permutation matrix Q matrix of measurement error covariance’s RB lower triangular Cholesky factor of BT B RQ lower triangular Cholesky factor of Q s vector of numbers indicating redundancy found in equation [44] Sf row or constraint diagonal scaling matrix Sx column or measured variable diagonal scaling matrix Sy column or unmeasured variable diagonal scaling matrix U matrix projection V matrix found in equation [46] xa vector of adjustments for measured variables xm vector of measurements for measured variables x vector of reconciled values for measured variables y vector of original unmeasured variables z vector of fixed variables Greek Letters  estimated uncertainty for the unmeasured variables in the kernel matrix  regularization parameter found in equation [19]  difference or delta Superscripts C column k iteration R row T transpose Subscripts a adjusted
  29. 29. 29 m measured 1 partition of B with independent columns and rows 2 partition of B with independent columns and dependent rows 3 partition of B with dependent columns and independent rows 4 partition of B with dependent columns and dependent rows 12 column partition of B with independent columns 34 column partition of B with dependent columns Operators  scaled variable indicated  i denotes the i th row of a matrix  j denotes the j th column of a matrix 2 Euclidean norm of a vector  tr trace of a matrix References Albuquerque, J.S. and Biegler, L.T., Data Reconciliation and Gross-Error Detection for Dynamic Systems, AIChE Journal, 42, 10, 1996. Axelsson, O., Iterative Solution Methods, Cambridge University Press, New York, 1996. Bagajewicz, M.J., Design and Retrofit of Sensor Networks in Process Plants, AIChE Journal, 43, 9, 1997. Box, G.E.P., and Jenkins, G.M., Time Series Analysis: Forecasting and Control, Revised Edition, Holden-Day, Oakland, California, 1976. Britt, H.L. and Luecke, R.H., The Estimation of Parameters in Nonlinear, Implicit Models, Technometrics, 15, 2, 1973. Buzzi-Ferraris, G. and Tronconi, E., An Improved Convergence Criterion in the Solution of Nonlinear Algebraic Equations, Computers chem. Engng., 17, 10, 1993. Coleman, T.F., Garbow, B.S. and More, J.J., Algorithm 618, ACM-Trans. Math. Software, 10, 3, 1984. Crowe, C.M., Garcia, Y.A., and Hrymak, A., Reconciliation of Process Flow Rates by Matrix Projection, Part I: Linear Case, AIChE Journal, 29, 6, 1983. Crowe, C.M., Reconciliation of Process Flow Rates by Matrix Projection, Part II: The Nonlinear Case, AIChE Journal, 32, 4, 1986. Crowe, C.M., Observability and Redundancy of Process Data for Steady-State Reconciliation, Chem. Eng. Sci., 44, 12, 1989. Crowe, C.M., Data Reconciliation - Progress and Challenges, J. Proc. Cont., 6, 2, 1996. Dovi, V.G. and Paladino, O., Fitting of Experimental Data to Implicit Models Using a Constrained Variation Algorithm, Computers chem. Engng, 13, 6, 1989. Fang, S-C and Puthenpura, S., Linear Optimization and Extensions: Theory and Algorithms, Prentice-Hall International, 1993. Forrest,J., The Current Status of IBM’s Optimization Subroutine Library (OSL) and Future Research Directions, NPRA Computer Conference, 1991. Golub, G.H., and Van Loan, C.F., Matrix Computations, The Johns Hopkins University Press, Baltimore, Maryland, 1983.
  30. 30. 30 Householder, A.S., The Theory of Matrices in Numerical Analysis, Dover Publications Inc., New York, New York, 1975. Kelly, J.D., On Finding the Matrix Projection in the Data Reconciliation Solution, Computers chem. Engngg, In Press, 1998. Kim, I-W, Kang, M.S., Park, S., and Edgar, T.F., Robustness Data Reconciliation and Gross Error Detection: The Modified MIMT Using NLP, Computers chem. Engng, 21, 7, 1997. Knepper, J.C., and Gorman, J.W., Statistical Analysis of Constrained Data Sets, AIChE Journal, 26, 2, 1980. Kuehn, D.R., and Davidson, H., Computer Control II Mathematics of Control, Chem. Eng. Prog., 57, 44, 1961. Liebman, M.J., and Edgar, T.F., Data Reconciliation for Nonlinear Processes, AIChE Annual Meeting, Washington, DC., 1988. Lucia, A., and Xu, J., Chemical Process Optimization Using Newton-Like Methods, Computers chem. Engng, 14, 2, 1990. Lucia, A., and Xu, J., Methods of Successive Quadratic Programming, Computers chem. Engng, 18, Suppl., 1994. MacDonald, R.J. and Howat, C.S., Data Reconciliation and Parameter Estimation in Plant Performance Analysis, AIChE Journal, 34, 1, 1988. Mah, R.S., Stanley, G.M. and Downing, D.M., Reconciliation and Rectification of Process Flow and Inventory Data, Ind. Eng. Chem. Process Des. Dev., 15,1, 1976. Madron, F., Process Plant Performance. Measurement and Data Processing for Optimization and Retrofits, Ellis Horwood Ltd., Chichester, England, 1992. Nguyen, T.C., Barton, G.W., Perkins, J.D., and Johnston, R.D., A Condition Number Scaling Policy for Stability Robustness Analysis, AIChE Journal, 34, 7, 1988. Pai, C.C.D. and Fisher, G.D., Application of Broyden’s Method to Reconciliation of Nonlinearly Constrained Data, AIChE Journal, 34, 5, 1988. Press, W.H., Vetterling, W.T., Teukolsky,S.A. and Flannery, B.P., Numerical Recipes in FORTRAN: The Art of Scientific Computing, Second Edition, Cambridge University Press, 1994. Ricker, N.L., Comparison of Methods for Nonlinear Parameter Estimation, Ind. Eng. Chem. Process Des. Dev., 23, 2, 1984. Reilly, P.M and Patino-Leal, H., A Bayesian Study of the Error-In-Variables Model, Technometrics, 23, 3, 1981. Sanchez, M., and Romagnoli, J., Use of Orthogonal Transformations in Data Classification-Reconciliation, Computers chem. Engng, 20, 5, 1996. Serth, R.W., and Heenan, W.A., Gross Error Detection and Data Reconciliation in Steam-Metering Systems, AIChE Journal, 32, 5, 1986. Smith, H.W., and Ichiyen, N., Computer Adjustment of Metallurgical Balances, Can. Inst. Mining Metall. (C.I.M.) Bull., 66, 97, 1973. Stephenson, G.R., and Shewchuk, C.F., Reconciliation of Process Data with Process Simulation, AIChE Journal, 32, 2, 1986. Swartz, C.L.E., Data Reconciliation for Generalized Flowsheet Applications, Amer. Chem. Society National Meeting, Dallas, Texas, 1989. Tjoa, I.B., and Biegler, L.T., Simultaneous Strategies for Data Reconciliation and Gross Error Detection of Nonlinear Systems, Computers chem. Engng, 15, 10, 1991. Tong, H., Studies in Data Reconciliation Using Principle Component Analysis, Ph.D. Dissertation, McMaster University, Hamilton, Ontario, Canada, 1995. Tong, H., and Crowe, C.M., Detection of Gross Errors in Data Reconciliation by Principle Components, AIChE Journal, 41, 7, 1995. Weiss, G.H., Romagnoli, J.A., and Islam, K.A., Data Reconciliation - An Industrial Case Study, Computers chem. Engng, 20, 12, 1996.
  31. 31. 31 Appendix The following presents the derivation of expression [25]. For the sake of argument, assume that A is of full row rank and B is of full column rank, then it is possible to expand the kernel matrix of equation [15] using the Sherman-Morrison-Woodbury formula (Golub and Van Loan, 1983) as         K AQA AQA B I B AQA B B AQAy         1 1 1 1 1 1T T T T T T   [A1] By pre-multiplying [A1] by AQAT , we arrive at     I B I B AQA B B AQAg y             T T T T1 1 1 [A2] Since  will be chosen to be sufficiently large, the inverse containing Iy can be approximated by      1 1 1 B AQA BT T [A3] Substituting [A3] into [A2] and taking its trace, we have     tr T T T T I B B AQA B B AQAg          1 1 1 [A4] which can be easily simplified to       tr tr T T T T I B AQA B B AQA Bg          1 1 1 [A5] Upon further simplification of the second term, which is equivalent to Iy, we can now write    tr tr n ng yI Ig y   [A6] If B is not of full column rank then it is required to replace B with B12 thus replacing ny with ny,12 in [A6]. Given that for realistic problems A will most likely not possess full row rank, the inverse AQAT will never presumably be realizable. While strictly speaking this is true, where the above derivation fancifully describes a hypothetical situation (i.e., a schematic representation), it has been observed in practice without exception that expression [25] is indeed approached as  becomes large. Consequently, it seems appropriate to declare expression [25] as a sensible guideline to select the regularization parameter  .
  32. 32. 32 1 2 3 4 8 5 6 7 1, 2 & 3 4, 5 & 6 7, 8 & 9 10, 11 & 12 Figure 1 Floatation Circuit Flowsheet of Smith and Ichiyen (1973)
  33. 33. 33 1, 2 & 3 A2 A3 A6 A1 B1 A5 A7 A8 C1 C2B2 D1 D2B3 A4 4, 5 & 6 7, 8 & 9 10, 11 & 12 13, 14 & 15 16 & 17 Figure 2 Heat Exchanger Flowsheet of Swartz (1989)