Norman Morrison - 0.1 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters Typesetter: When typesetting this book Please include a blank line wherever I do Please indent wherever I do Please do not indent if I do not indent Please set an item ‘landscape’ whenever I do ---------------------------------------I have included typesetting requests from time to time in red Please follow my requests Sincerely Norman Morrison
Norman Morrison - 0.2 - Tracking Filter Engineering The Gauss-Newton and Polynomial FiltersTracking Filter Engineering The Gauss-Newton and Polynomial Filters Norman Morrison Department of Electrical Engineering, University of Cape Town, South Africa Formerly Bell Labs Ballistic Missile Defense, Whippany, New Jersey, USA
Norman Morrison - 0.3 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters For Adriénne Juliet Folb without whose presence in my lifethis book would never have been written.
Norman Morrison - 0.4 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters Behold, I will cause breath to enter into you, and ye shall live.And I will lay sinews upon you, and I will bring up flesh upon you, and cover you with skin, and put breath in you, and ye shall live. Ezekiel XXXVII
Norman Morrison - 0.5 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters AcknowledgementsMany people made this book possible.Dr. Richard Lord, formerly of the University of Cape Town and now of the Karroo ArrayTelescope (KAT), stepped forward to bring his remarkable creativity and outstandingprogramming skills to bear on our tracking problems. I will always be grateful to him.Johan Kannemeyer, an extremely inventive software engineer at Reutech Radar Systems,programmed and tested many of the algorithms. He also made many suggestions, all ofwhich were adopted. It was Johan who came up with the ratio matrix that is discussed inChapter 7.Professors Linda Haines and Tim Dunne of the University of Cape Town’s Departmentof Statistical Sciences kept their doors open for me and made available whatever time ittook to discuss my problems. They also read parts of the manuscript, found errors andmade suggestions, all of which were adopted. Without their help this book would nothave been written.Professors Daya Reddy and Ronnie Becker of the University of Cape Town’sDepartment of Mathematics and Applied Mathematics read parts of the manuscript,found errors and offered many excellent suggestions.Professor Mike Inggs of the University of Cape Town’s Department of ElectricalEngineering posed a problem to me that ultimately led to the writing of this book. He alsoread parts of the manuscript and made suggestions. It was Mike who connected me withthe IET and with the University of Cape Town’s Open Access Database.Professor Pieter Willem van der Walt, formerly Dean of Engineering at the University ofStellenbosch and now Technology Executive of Reutech Radar Systems, read an earlyversion of the manuscript and set me on the right path.Pieter-Jan Wolfaardt, Technology Executive of Reutech Radar Systems, constantlychallenged me with his brilliant mind and his fresh ideas. It was Pieter-Jan who broughtme into their radar project, and I will always be beholden to him for his support andencouragement.Ivan Gibbons, Chief Engineer of Denel Aerospace Systems, and Pieter Reyneke, Denelsoftware engineer, quickly caught on to the merits of Gauss-Newton. Ivan encouraged meto keep on going, and Pieter gave up his time and programming skills to derive neededresults.
Norman Morrison - 0.6 - Tracking Filter Engineering The Gauss-Newton and Polynomial FiltersDrs. Yossi Etzion and Michael Maidanik of IAI also quickly caught on to the merits ofGauss-Newton. Yossi made a number of suggestions, all of which were adopted, andMichael’s enthusiastic understanding of the advantages of Gauss-Newton served as aconstant reason for me to keep on going against the odds.Jean-Paul de Conçeaçao, a graduate student at the University of Cape Town, asked aquestion that caused me to rewrite the entire book. In his Master’s thesis, Jean-Paulsuccessfully implemented a Gauss-Newton filter in an FPGA.Roaldje Nadjiasngar, also a graduate student at the University of Cape Town, read themanuscript, programmed a number of the algorithms, found errors that we fixed and thenverified that the algorithms were correct. Roaldje has since made use of Gauss-Newton inhis doctoral thesis with great success.Professor Stephen Hodgart of the University of Surrey revived my interest in trackingfilters during our walks on Table Mountain, after I had been away from them for manyyears. Thank you, Stephen.Professor Dov Hazony, my doctoral thesis adviser at Case Western Reserve University inCleveland, Ohio, taught me, almost fifty years ago, what the word ‘complete’ means. Ican only hope that I have succeeded.The late Professor Richard Duffin of the University of Pittsburgh was an enduringinspiration to so many us. While writing this book I thought often of him, and also of thelate Professor Brian Hahn of the University of Cape Town who died under tragiccircumstances, much too long before his time. I learnt a great deal from both Dick andBrian. They were good teachers and good friends, and I miss them both.I was privileged, almost half a century ago, to work with many brilliant people at BellLabs in New Jersey who first introduced me to tracking filters, among them Drs. MarvinEpstein, Paul Buxbaum, Alphonse Claus and Jack Riordan. While writing this book,memories of those friends and what they taught me were constantly in my mind.My very special thanks to Professor Hugh Griffiths of University College London, IETSeries Editor, and to Nicki Dennis and Paul Deards, both IET Commissioning Editors.Hugh and Nicki gave me the encouragement I needed at a most critical time and Paulhelped with the shortening and reorganization of the typescript. My thanks also to HelenLangley, IET Production Controller, for her seemingly endless patience, and to the manyothers at the IET who participated in the publication of this book ◄►A number of people found errors and omissions in the manuscript of this book and I havemade every effort to correct them and to ensure that it is error-free. However, I assumefull responsibility for any and all errors, omissions and incorrect statements that mightstill remain.
Norman Morrison - 0.7 - Tracking Filter EngineeringThe Gauss-Newton and Polynomial Filters The Greek alphabet
Norman Morrison - 0.8 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters Preface Why would anyone write a book about Gauss-Newton tracking filters? There are many reasons. Here are just a few. ◊ The Kalman filter has become an extremely complex topic with a bewildering 8-pt number of mutations. By contrast, Gauss-Newton filters are easy to understand anddiamonds easy to implement. What we are offering here is a simpler, more practical approach to tracking filter engineering that will appeal to both experienced practitioners and to newcomers. ◊ Gauss-Newton filters work, and they work extremely well. We and our colleagues have established that – by using them in a variety of operational hardware as well as in simulations. ◊ Once launched, Kalman filters1 have only one type of memory – expanding – something that is impossible to change. By contrast, Gauss-Newton filters can have fixed length, variable length or expanding length memories and any of these can be changed at will during filter operation. Gauss-Newton filter models can also be changed during filter operation – if need be from one cycle to the next. This ability to change their memory lengths and filter models opens up powerful new ways by which to track manoeuvring targets, ways that are difficult, if not impossible, to implement with Kalman filters. ◊ Gauss-Newton filters can be used when the observations are stage-wise correlated, something that cannot be done with Kalman filters, and while it is true that observations in filter engineering are often stage-wise uncorrelated, that is not always the case. ◊ Perhaps most important of all, Gauss-Newton filters are not vulnerable to the instability problems that plague the extended Kalman filters – in which, unpredictably, the estimation errors become inconsistent with the filter’s covariance matrix. Gauss-Newton filters therefore do not require what is known in Kalman parlance as ‘tuning’ – searching for and including a Q matrix that one hopes will prevent 1 The words ‘Kalman filter’ mean different things to different people. When we use them here we are referring specifically to the mainstream version appearing in References 3, 6, 22, 23, 24, 25, 30, 31, 48, 50, 53, 54, 55, 56, 57, 173 and 174. The equations of the mainstream version appear in Appendix 11.1.
Norman Morrison - 0.9 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters something that is both unpredictable and almost inexplicable, and in the process degrading the accuracy of the Kalman estimate. This in turn means that in practice Gauss-Newton filters are more accurate, and that they are easier to implement, require shorter project time, are extremely robust and, best of all, unconditionally stable. And what can one say about execution time? Using Gauss-Newton on a 1958-era machine, the extraction of a satellite’s orbital parameters from a set of radar observations would have added about one hour of processing time after the observations had been obtained. However, on today’s readily available desk-top or lap-top machines, that hour has shrunk to a few hundred milliseconds, and as computer performance and other technologies such as FPGA continue to advance, it is safe to say that in the not too distant future it will have shrunk to tens of milliseconds or even less. ◄► As envisaged here, tracking filters are computer programs that extract estimates of the states of dynamic processes in near real-time from sequences of discrete real observations. Tracking filter engineering is the collection of techniques, knowledge and activities that we employ when we create and operate such filters. The following is a partial list of fields in which tracking filters are used: Air-traffic control, artificial intelligence, astronomy, atmospheric re-entry, ballistic missile defence, chemical engineering, civil engineering, control engineering, econometrics, electrical engineering, GPS and WAAS, industrial engineering, inertial navigation, mechanical engineering, missile engineering, neural networks, physics, pilotless-aircraft engineering, radar and tracking, robotics, satellite engineering, space navigation, statistics, telecommunications, telescope engineering and wind-power engineering. This book will be of interest to three groups of people: 8-pt ◊ Practitioners working in the above or similar fields.diamonds ◊ Graduate-level newcomers wishing to learn about Gauss-Newton and polynomial filters, and how they can be used in filter engineering. ◊ University lecturers who might wish to include material on Gauss-Newton and polynomial filters in graduate-level courses on tracking filter engineering.
Norman Morrison - 0.10 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters ◄►Tracking filter engineering came into existence around 1958 at the start of theSatellite Era.2Engineers and scientists had long been fitting curves to data by least squares, but this wasdifferent. Now there were huge tracking radars making observations on artificialsatellites, from which estimates of orbital parameters – known as Keplerians – were to beextracted, and the filtering algorithms for doing so were far more sophisticated thansimple least squares.Astronomers had been estimating Keplerians of objects in orbit since the time of Gauss,using what was known as the Gauss-Newton algorithm. However, such calculations weredone by hand and it took many months to obtain results, whereas with artificial satellitesthe need was for the observations to be processed by computers and for the estimates tobe available in near real-time.Running Gauss-Newton on a 1958-era machine would have taken roughly an hour toextract a satellite’s Keplerians from a set of radar observations after the latter had beenobtained. Such a delay was clearly too far removed from real-time, and so Gauss-Newtoncould not be used.In its place two new filters were devised – both related to Gauss-Newton butcomputationally different. The first of these was published by Swerling in 1958 3 and thesecond by Kalman4 and Bucy5 in 1960/61, both of which could extract Keplerians in nearreal-time on the existing machines.6Starting from the extraction of Keplerians, the use of the Kalman filter spread rapidly tothe many other fields listed above where today it occupies the dominant place in filterengineering. The Kalman filter has also come to occupy the dominant place in filter-engineering curricula throughout the academic world. ◄►But while all of this was taking place, something else was also happening.2 Sputnik-1 was placed in orbit by the Soviet Union on October 4th, 1957. See Ref. 40.3 Peter Swerling (American mathematician and radar theoretician), 1929 – 2000. See References 6, 7, 8, 9,91.4 Rudolf Emil Kalman (Hungarian-born American scientist), 1930 – . See References 1, 2, 6, 92.5 Richard S. Bucy (American mathematician), 1935 – . See Reference 93.6 See Chapter 11 for what we mean by ‘Swerling filter’ and ‘Kalman filter’.
Norman Morrison - 0.11 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters Since 1958, computer technology has made advances that are difficult to comprehend, and the end is still nowhere in sight. Consider for example the following item that appeared not long ago the Intel corporate website: “Intel remains at the forefront of Moore’s Law. Our 22 nanometer technology-based Intel microprocessors will enable never-before-seen levels of performance, capability, and energy-efficiency in a range of computing devices.” And so, while it may have been appropriate to reject Gauss-Newton in 1958, it is no longer appropriate to do so today. In this book we accordingly do something different. Many books have been written about the Kalman filter – the author is aware of at least fifteen. However, to our knowledge nobody has yet written a book about the Gauss- Newton filters and their remarkable fit to certain important areas of filter engineering. This then is our attempt to do so, and to present those filters in a readable and self- contained way. ◄► The Gauss-Newton filters possess attributes that make them particularly well-suited for use in tracking filter engineering, among them the following: 10-pt ◊ Ideal for tracking both manoeuvring and non-manoeuvring targets.diamonds ◊ Can be used when the observations are stage-wise correlated. ◊ Are not vulnerable to the instability phenomena that plague both the Kalman and Swerling extended filters, and hence require no tuning. ◊ Require no initialization in the all-linear case and very little in the three nonlinear cases. ◊ Possess total flexibility with regard to ◊ Memories that can be configured as fixed length, variable length or 8-pt expanding length – and if need be, reconfigured cycle by cycle. diamonds ◊ Filter models that can also be reconfigured, if need be, from cycle by cycle. 10-pt ◊ Offer immediate access to the residuals which can then be used todiamond run goodness-of-fit tests.7 7 Goodness-of-fit tests are used by the Master Control Algorithms (MCA’s) to control a Gauss filter’s memory length and filter model when tracking manoeuvring targets, in such a way that its performance is
Norman Morrison - 0.12 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters 10-pt ◊ Unconditionally stable, meaning that they are alwaysdiamonds error/covariance-matrix consistent.8 ◊ Cramér-Rao consistent, meaning that their accuracy is the best that it can possibly be. ◊ Can employ nonlinear models of almost any complexity. ◊ Nonrecursive, and hence devoid of internal dynamics. ◊ Easy to implement. ◊ Extremely robust. The techniques proposed in this book have all been tested in simulations, and many of them have been used in operational hardware built by industrial organizations with which the author has been associated. The engineers who created the filters for those applications have made it known how easy it was to do that, and how robust were the resulting filters. ◄► Regarding Gauss-Newton execution time on today’s machines: We have carried out simulator-based timing studies in which Gauss-Newton filters were used to track a mix of manoeuvring targets such as high-performance military jets, cruise missiles and helicopters. We found that readily available modern computers – either desk-top or lap-top – can readily handle the load imposed by these filters while tracking many such targets, including all of the overhead calculations, namely track formation, track deletion and plot-to-track allocation using the auction algorithm. Indeed, depending on the level of radar clutter, our studies have suggested that concurrent tracking of a mix of possibly as many as fifty manoeuvring targets of this kind can be accomplished on such computers using these filters, and as processing power continues to increase, the conservative figure of fifty will continue to rise. And we would add the following: The execution time of the overhead calculations in the above studies far outweighed that of the Gauss-Newton filters, and the overhead calculations would also have to be performed if Kalman filters were used. Thus under similar circumstances, the number of targets that could be tracked using Kalman filters would have been almost the same. ◄► always optimal. The MCA’s are discussed in Chapter 10. 8 Error/covariance-matrix (ECM) consistency is discussed in Chapter 1. When a Kalman or Swerling filter becomes unstable then it is ECM consistency that is absent, a condition which is fatal.
Norman Morrison - 0.13 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters Our discussion of filter engineering centres on two concepts mentioned in the above list: ◊ Error/covariance-matrix (ECM) consistency, meaning that a filter’s 10-pt covariance matrix is an accurate representation of the actual covariance matrix of thediamonds estimation errors, something that we examine in depth and whose absence in a filter is fatal.9 ◊ Cramér-Rao (CR) consistency, meaning that the filter produces estimates of the highest possible accuracy. In this regard we discuss six tests: ◊ Three to determine if a filter is ECM consistent. ◊ Three to determine if it is CR consistent. A number of other concepts are also discussed, among them the following: ◊ Tracking of manoeuvring targets using what we call the Master Control Algorithms (MCA’s) that dynamically control a Gauss-Newton filter’s memory length and filter model in such a way that its performance is always optimal. ◊ A goodness-of-fit test which determines if a Gauss-Newton filter’s estimate is an unbiased fit to the observations. ◊ A real-time test which monitors the observation errors. ◊ Prefiltering, in which the polynomial filters form a high-data-rate front end for the main tracking filters such as Gauss-Newton, Kalman or Swerling, which then extract the desired estimates at their low data rates. We include fourteen computer programs that were created as research tools over a period of many years, and which serve to demonstrate almost every aspect of our discussion: ◊ In most of them we have implemented one of the Gauss-Newton filters. ◊ In almost all of them we have implemented one or more of the eight tests mentioned above. 9 ECM consistency of a filter is by no means guaranteed, and as is well known and mentioned above, both the Kalman and Swerling filters can spontaneously lose their ECM consistency – known in Kalman parlance as ‘becoming unstable’ – prevention of which is accomplished by the use of what is known as a Q matrix for ‘tuning’ the filter, but which also degrades their accuracy. We examine all of this in Chapter 11.
Norman Morrison - 0.14 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters 10-pt ◊ In two of them we have implemented prefiltering using the polynomial filters,diamond and in which the main filters are Gauss-Newton, Kalman and Swerling – all three of which process precisely the same observations. These two programs 8-pt ◊ Demonstrate how prefiltering works. diamonds ◊ Enable you to witness instability in both the Kalman and Swerling filters 10 and the fact that Gauss-Newton, operating on exactly the same data, is always stable. ◊ Make it possible to time the executions of the three main filters. ◊ Enable you to compare their performance in a way that is ‘apples-to- apples’. ◄► The polynomial filters discussed in this book were first devised when we worked at Bell Labs Ballistic Missile Defense in Whippany New Jersey from 1964 to 1968. They have been written up twice before – once in our first textbook (see Reference 6) and again in Reference 24. However, over the succeeding years we have learned a great deal more about them and continue to do so11 and in this book we provide a complete discussion which includes much of that new material. ◄► We have tried to keep the book as generic as possible, i.e. applicable to multiple fields. However, in some places we were forced to place it in a specific context, and the field that we selected was that of radar and tracking. Some of our discussion is thus slanted towards target motion sensing using two types of radar – those with constantly rotating antennas (track-while-scan or TWS radars) and those with steerable antennas (tracking radars). In this regard we beg forgiveness from those readers who use filters in other fields. However, we assume very little knowledge of radar and tracking and instead place our primary emphasis on filtering techniques, and so we hope that the book can be read and used by practitioners and students who are interested in using tracking filters in almost any field. 10 The Swerling filter becomes unstable whenever the Kalman filter does, and in all regards their performances are essentially identical. 11 See References 112 and 118.
Norman Morrison - 0.15 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters ◄►The material in the book has been taught on six occasions on three continents to a mix ofpracticing filter engineers, graduate students and university lecturers.It is well suited for a complete single-semester graduate course of 24 lectures or else forpartial incorporation into other graduate courses, and contains more than a sufficiency ofexaminable material that such courses require. The fourteen computer programs and theend-of-chapter problems and projects will enable students to envisage and apply theconcepts that are discussed in the chapters.We have made every effort to keep the book readable and friendly. We can only hope thatwe have succeeded.Norman MorrisonCape Town, July 2012
Norman Morrison - 0.16 - Tracking Filter Engineering The Gauss-Newton and Polynomial Filters Viewing the Track-While-Scan Video ClipsIn order to see, first hand, how well-suited these filters are to tracking manoeuvringtargets, we suggest at this point that you turn on your computer and retrieve the down-loadable material. (See downloading instructions at the start of Chapter 1.)Once that is done, please read the following document so that you understand fully whatyou will be viewing: Video_ClipsTWSDocumentsReadme.pdfThen please take a few minutes to view the track-while-scan video clips that arecontained in the folder Video_ClipsTWSFlights.This will enable you to see how effectively the Gauss-Newton filters under MCA-1control (Master Control Algorithm, Version-1) are able to perform when trackingmanoeuvring targets. Words containing the letter ZWe lived in the USA for thirty years and were under the impression that the use of theletter z in words like maximize and minimize was a distinctly American thing, and that onour side of the Atlantic one would spell them as maximise and minimise.Imagine our surprise when we consulted the Shorter Oxford Dictionary and found that italso uses the letter z in many such cases. Throughout the book we have attempted tofollow the usage in that dictionary.
Norman Morrison - 0.17 - Tracking Filter EngineeringThe Gauss-Newton and Polynomial Filters Contents