The visual intelligence iris one airborne digital camera systems petrie geoinformatics-6-2012

  • 269 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
269
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Article Based on its iOne Sensor Tool Kit Architecture Visual Intelligence’s Iris One The Visual Intelligence company has developed a family of modular and scaleable airborne digital camera systems that are based on its Iris One Sensor Tool Kit Architecture (iOne STKA). This architecture utilizes multiple camera modules that can be equipped with a variety of CCD formats and lenses to provide extended ground coverage in various alternative configurations. The article reviews the several different systems that have been developed, including both their hardware and software, and describes the calibration procedures that are used to establish their metric properties. The company’s planned future developments are also outlined. By G ordon Petrie I - Introduction & Background Multiple camera systems comprising various combinations of vertical and oblique pointing cameras have long been part of the aerial photographic scene. The use of “fans” of such cameras has been popular for many years, especially in aerial reconnaissance operations where they provide extended cross-track coverage, in some cases, from horizon-to-horizon. The usual arrangement consists of a single nadir pointing camera in combination with either two (or four) oblique pointing cameras. The shutters of all three (or five) cameras in the “fan” are triggered to fire simultaneously. Thus all the resulting photographs in a set will have been acquired from a single position in the air. Older examples (from many) of such “fans” that utilized film cameras are (i) the Bagley three-lens camera systems manufactured by Bausch & Lomb and used by the U.S. Army Air Service during the 1920s [Fig.1]; and (ii) the Tri-Metrogon system [Fig. 2] employing three Fairchild K-17 standard-format (23 x 23 cm) metric cameras that was used extensively by the U.S. Army Air Force for aeronautical mapping and charting purposes during World War II (WW-II). Fig. 3 – The Carl Zeiss KRb 8/24 Tri-lens camera has been used in numerous NATO manned and UAV aircraft. (Source: Carl Zeiss) Fig. 1 – A Bagley T-1A three-lens camera system with focal lengths of f = 125 mm for the central (nadir-pointing) lens and f = 178 mm for the two wing (oblique-pointing) lenses. (Source: Smithsonian National Air & Space Museum) Turning to more modern times, the Carl Zeiss KRb 8/24 tri-lens film camera series [Fig.3] with up to 143° cross-track coverage have been used extensively by Canadian, German and French air forces, including a version that has been used in the Canadair CL-289 drone (UAV) aircraft. Also the TriMetrogon configuration has continued to be used, for example in the German Tupolev Fig. 2 – An example of a Tri-Metrogon camera installation comprising three separate Fairchild K-17 metric film cameras with f = 152 mm lenses as mounted in a Boeing B17 bomber aircraft and used during World War II. (Source: USAF) 30 Fig. 4 – The three Carl Zeiss Jena LMK photogrammetric film cameras mounted in a Tri-Metrogon configuration in the German Tupolev Tu-154M Open Skies aircraft. (Source: IGI) Tu-154M Open Skies aircraft, employing three Carl Zeiss Jena LMK 15/23 mapping cameras [Fig. 4]. Furthermore the later Carl Zeiss KS-153 reconnaissance cameras have been produced in both three and five lens forms [Fig. 5] and have been used widely by NATO air forces, including the U.S. Navy (on F-14 aircraft) and the German Luftwaffe (on Tornado aircraft). Indeed a number of these cameras are still in operational service. All of the above examples have utilized film cameras. However the “fan” configuration has remained popular in the digital camera era. An example is the DLR-3k camera system [Fig. 6] employing three smallformat cameras, which is in use by the German Space Agency (DLR). Currently both IGI (with its Triple-DigiCam system) and Leica Geosystems within its RCD30 multihead range offer similar three-camera “fan” configurations, but using medium-format cameras instead of the small-format cameras used in the DLR-3k system. There is even an echo of the WW-II Tri-Metrogon system with the advent recently of the Russian Tupolev Tu-214 ON Open Skies aircraft, which is being equipped with three Intergraph Z/I DMC-II140 large-format digital cameras [Fig. 7] in a Tri-Metrogon configuration to September 2012
  • 2. Article Airborne Camera Systems [a] Fig. 5 – (a) A Carl Zeiss KS-153 Tri-lens camera installed in a pod mounted on a German Tornado aircraft. (b) & (c) Diagrams of the Carl Zeiss KS-153 reconnaissance camera showing the lens design and ground coverage in its Tri-lens form [in (b)]; and in its Penta-lens 53 form [in (c)], providing fans of three and five photographs respectively in the cross-track direction. (Source: Carl Zeiss) [a] [b] [b] Fig. 6 – The DLR-3k three-camera system makes use of a “fan” of three Canon EOS small-format digital cameras with one pointing vertically and two obliquely to provide wide cross-track coverage. (Source: DLR) [c] Fig. 7 – The Tri-Metrogon style arrangement of the three Intergraph Z/I DMC-II140 large-format cameras that have been installed in the Russian Tupolev Tu-214ON Open Skies aircraft. (Source: Z/I Imaging) Fig. 8 – (a) This photo shows the rigid ARCA or “bridge” into which an array or “fan” of three or five small-format cameras can be inserted. (b) The geometric arrangement of an ARCA configuration. (Source: Visual Intelligence) camera system that is based on the so-called ARCA (Arched Retina Camera Array) technology on which Visual Intelligence holds a number of patents. The system features a rigid ARCA or “bridge” into which an array or “fan” of three or five digital cameras can be fitted with a choice of lenses and CCD formats [Fig. 8 (a)]. This configuration allows the optical axis of each individual camera in the array to intersect, passing through a single perspective centre [Fig. 8 (b)]. This particular arrangement has been used in the Iris One 50 system (renamed the Iris One Ortho 19 kps (1) system), which is the specific model that has been awarded ‘Digital Aerial Sensor Certification’ by the USGS in 2009. II – The iOne STKA & ARCA [Note (1): In this particular context, Dr. Guevara, provide the required wide-angle coverage of the ground during its monitoring flights. Latest News? Visit www.geoinformatics.com (a) Single ARCA Configuration The latest additions to this multiple camera “fan” type of aerial camera system are the Iris One systems that have been developed by the Visual Intelligence company based on its iOne Sensor Tool Kit Architecture (iOne STKA). The company – which is located in Houston, Texas in the U.S.A. – was originally called M7 Visual Intelligence prior to the sale of its sister aircraft component and repair business (M7 Aerospace) in 2010, after which, the company assumed its present title. The Iris One is a modular and scaleable digital aerial 31 CEO of Visual Intelligence, has coined the term “kps” (kilo pixel swath), where 1 kps is a swath that is 1,000 pixels wide. Different ARCA configurations with different lenses, angular coverages and CCD array formats allow systems to be constructed with swath widths of 11 kps up to 60 kps.] (b) Double ARCA Configurations A further and quite distinctive feature of the scaleable ARCA technology is that an additional ARCA can be added in parallel to the first with the two ARCAs or “bridges” being fitted precisely together [Fig. 9 (a)]. This allows the co-registration of all the images September 2012
  • 3. Article [a] [b] Fig. 9 – (a) Showing two ARCAs or “bridges” fitted together in parallel in a Lego-like arrangement. (b) Showing how the images produced by the five colour (RGB) cameras fitted to the first ARCA are co-registered with the five near infra-red (NIR) images produced by the cameras fitted to the second ARCA. (Source: Visual Intelligence) produced by the two fans – as required, for example, in a multi-spectral version of the Iris One system. This allows the set of colour RGB images that have been recorded by the cameras mounted in the first ARCA or “bridge” to be co-registered with and superimposed on the corresponding set of near infra-red (NIR) images that have been produced by the cameras mounted on the second ARCA or “bridge” [Fig. 9 (b)]. Visual [a] Intelligence calls this its “CoCo” (Co-mounted and Co-registered) system. Another possibility with the double ARCA arrangement is that the two sets of images can be offset with respect to one another, allowing them to be interlaced in order to widen the field of view and the cross-track coverage that can be achieved from a single flight. This particular configuration has been adopted in the Iris One Hi5 system [Fig. 10 (a)]. Depending on the focal length of the lenses that have been selected and fitted to the camera modules and the CCD format size, this system can provide different swath widths over the ground from a given flying height. As shown in the two diagrams [Figs. 10 (b) & (c)], the former provides a 10 km wide swath (using 11 Megapixel camera modules), while the latter provides a 12 km wide swath (using 16 Megapixel camera modules) from a flying height (H) of 12 km (using a jet aircraft!). Fig.11 is a sample colour RGB frame image (described as a single ARCA Virtual Frame) that has been acquired by an Iris One system. (c) Triple ARCA Configurations three ARCAs or “bridges” of the system – with their 9 varying-format camera modules – can be oriented either in the cross-track or the along-track direction. The pattern of the resulting ground coverage for each of these two directions is shown in Figs. 12 (b) & (c). When each camera module is fitted with a CCD array having a format size of 6,576 x 4,384 pixels = 29 Megapixels, then the total coverage of a single set of the 9 images is 19,000 x 12,500 pixels. Using cameras fitted with f = 85 mm lenses, this produces an angular coverage of 46° (crosstrack) x 70° (along-track) (12,500 x 19,000 pixels) with the cameras oriented in the along-track direction [Fig. 12 (b)]. The 60 % longitudinal overlap along the flight line that is produced when the cameras are programmed to expose overlapping sets of colour (RGB) images in a stereo convergent configuration gives a base:height ratio of 0.6. When the system is rotated by 90 degrees into the cross-track direction (19,000 x 12,500 pixels) [Fig. 12 (c)], this yields a base:height ratio of 0.34, similar to that achieved by the overlapping stereo- Fig. 11 – A sample wide-swath colour RGB image frame that has been acquired by an Iris One camera system. (Source: Visual Intelligence) With the addition of a third ARCA or “bridge” (interlocked and butted together) in parallel to the previous two ARCAs, still more versatile configurations can be envisaged. One of these is the Iris One Stereo camera system [Fig. 12 (a)]. With this, the [b] [b] Fig. 10 – (a) An Iris One camera system. [N.B. The same enclosure cabinet is used for the Ortho, MS, Oblique, Stereo and Hi-5 models]. (b) Diagrams showing the alternative ground coverage patterns with interlaced angular coverages that are produced by the Iris One Hi5 system using cameras having different focal lengths and CCD sizes (11 Mpix or 16 Mpix). (Source: Visual Intelligence) [a] [b] Fig. 12 – (a) An Iris One Stereo camera system showing the 9 cameras mounted on three parallel ARCAs or “bridges”. (b) The pattern of ground coverage that is produced by the 9 camera array when set in the along-track direction. (c) The pattern of the ground coverage that is produced by the 9 camera array in the cross-track direction. (Source: Visual Intelligence) 32 September 2012
  • 4. Article images that are produced by conventional large-format digital mapping cameras. lapping stereo-images that are produced by conventional large-format (non-digital) mapping cameras. images can then be handled by any standard third-party photogrammetric software solution. III – Camera & System Calibration The radiometric calibration that is also carried out in the laboratory (outside under natural light conditions) for the Iris One system is a relative calibration (not an absolute calibration) and includes colour and tonal balancing, shading and aperture correction, and smearing removal. The resulting data is incorporated into a radiometric correction software module for use during later image processing operations. The module can also separate the colour RGB images into their individual component red, green and blue images and interpolate the missing pixel values produced by the Bayer mosaic filter. Another possible arrangement of the flexi- [a] [b] One of the main applications of the various Iris One systems is the topographic mapping field. Thus the metric side and, in particular, the calibration of (i) each individual camera in an ARCA or “bridge”; and (ii) that of the overall camera system as a whole is obviously a matter of prime concern. The geometric calibration of each individual camera is carried out in a laboratory using a calibration cage [Fig. 14] that is fitted out with a set of well distributed and highly reflective coded targets, whose coordinates have been determined to a precision of ±1mm in X and Y and ±5 mm in Z. Photographs of this target field are taken from a number of different positions and orientations with each individual camera, so providing a strong convergent geometry. The images of all the targets that have been [b] Fig. 13 – (a) An Iris One Stereo camera system showing the 15 cameras mounted on three parallel ARCAs or “bridges”. (b) The pattern of the ground coverage that is produced by the 15 camera array oriented in the along-track direction. (c) The 60 % longitudinal overlap that is produced by two successive sets of exposures from the Iris One Stereo system gives a base: height ratio of 0.6. (Source: Visual Intelligence) ble three ARCAs or “bridges” of the Iris One Stereo camera system is to employ 15 small-format cameras [Fig. 13 (a)], as set out in a paper given by Dr. Hwangbo of Visual Intelligence at the recent ASPRS 2012 Conference. The pattern of the ground coverage that results if the cameras are oriented in the along-track direction is shown in Fig. 13 (b). When each camera is fitted with a CCD array having a 4,008 x 2,672 pixels = 11 Megapixel format, then the total coverage of a single set of 15 images is 21,460 x 7,438 pixels. Using cameras fitted with f = 135 mm lenses, this produces an angular coverage of 27° (cross-track) x 70° (along-track). The 60 % longitudinal overlap along the flight line that is utilized to produce overlapping sets of colour (RGB) images in a stereo convergent configuration gives a base:height ratio of 0.6 [Fig. 13 (c)]. This is similar to that achieved by the over- Latest News? Visit www.geoinformatics.com Fig. 14 – The Visual Intelligence calibration cage. (Source: Visual Intelligence) recorded on each photograph are then measured. From this information, the value of the focal length of the lens and the position of the principal point of the frame image are then determined for each individual camera, together with the lens distortion values or parameters. After each individual camera has been calibrated, the relative position and orientation of all the cameras within a complete ARCA array are then determined with respect to one other, again using the known coordinates of the targets in the target field. After this second stage of the calibration process, a single composite “virtual” frame image with a single perspective centre can be defined using the measured data from all the component images that have been generated by the array of modular cameras mounted on the ARCA or “bridge”. These “virtual” frame 33 This laboratory calibration is supplemented by a further in-flight geometric calibration that is carried out over a field of signalized ground control points laid out in a test area within the Houston metropolitan region. This test area is overflown by an aircraft at a flying height (H) of 1 km. The Iris One camera system is programmed to expose its images with a large (80%) longitudinal and (60%) lateral overlap. Crossstrips are flown as well as the parallel strips of the main block of aerial photography covering the test area. Automatic image matching of the target and tie points is carried out on all the overlapping photographs that have been exposed during the flight. A bundle aerial triangulation and block adjustment employing self-calibration is then implemented using the BINGO software from GIP in Aalen, Germany to generate the final coordinate values and their residuals at the ground control points. Needless to say, the lever arm corrections that relate the positions of the GPS antenna and the IMU to the perspective centre of the “virtual” frame photo will also be determined. A large number of test flights of the Iris One system have been undertaken by various aerial photographic and mapping companies in the U.S.A. Numerous flights have been undertaken for test purposes over a range of flying heights by Aerial Viewpoint, which is based locally in the Houston area. Further extensive testing has taken place in cooperation with Techmap, a mapping company that is based in Peachtree City, Georgia. Among the larger American aerial mapping companies, Sanborn, Fugro Horizons & EarthData and Northrop Grumman 3001 Inc. have all conducted trial flights with Iris One systems. Further test flights have also September 2012
  • 5. Article been undertaken by INEGI, the Mexican national mapping agency. Purchasers of Iris One systems include Aviation Supplies, a leading supplier of aircraft solutions in China. In the U.S.A., McKim & Creed, a surveying and mapping company based in Raleigh, North Carolina – has acquired several iOne Infrastructure Mapping Systems (iOne IMS). IV – System Software Fig. 15 – An Iris One camera system (at left) with its compact MaxCube data collection, camera control and data processing server (at right) – with its laptop controller located on top of the main cabinet. (Source: Visual Intelligence) The Iris One system is of course driven and controlled by software. The main components that [a] carry out the in-flight data acquisition operations are provided through the so-called Visual Navigator software. This features three modules – (i) for image data acquisition; (ii) for flight line management and camera control; and (iii) for data management respectively. A second software system, called Isis, carries out the subsequent image data processing, including the radiometric correction software mentioned above. It features two modules. (i) The Isis Sky module carries out the processing of the acquired imagery in conjunction with the DGPS and IMU data that has been collected simultaneously in-flight. If an existing DEM is available, e.g. from USGS in the United States, this module can also carry out the generation of an orthophoto in-flight. (ii) The Isis Earth module is used to carry out a more accurate ortho-rectification using ground control points and more refined GPS/IMU data in a post-flight processing operation that is carried out later on the ground. Within this context, Visual Intelligence has teamed up with the MaxVision company, based in Madison, Alabama, to create an image processing system that will satisfy the rather demanding requirements of the Visual Navigator and Isis Sky software modules, especially in respect of implementing its inflight image processing capability. The computer that is being used for the purpose is the compact MaxCube II mobile “super server” [Fig. 15]. This ruggedized computer can be supplied with two Intel Xeon processor units providing up to twelve powerful 64bit CPUs and up to eight removable hard drives, each with a 3 terabyte storage capacity. All of this processing power and storage capacity is contained in a cabinet that is close to one cubic foot in size and has a low power consumption. [b] V - Current & Future Developments A recent development has been the introduction of the Iris One Infrastructure Metric-Mapping System – having the titular acronym iOne IMS. This comprises a single rigid ARCA or “bridge” into which is inserted a pair of nadir-pointing camera Fig. 16 – (a) Showing the ARCA or “bridge” on which the twin nadirpointing RGB and NIR cameras and the two oblique-pointing cameras are mounted – as used in the Iris One Infrastructure Metric-Mapping System [iOne IMS]. (b) Showing an iOne IMS system with the protective cover in place over the cameras. (c) Showing the arrangement of the four lenses – two nadir-pointing and two oblique-pointing – of an iOne IMS camera system from the underside. {Source Visual Intelligence) 34 September 2012
  • 6. Article [a] can utilize the standard camera hole on a fixed-wing aircraft. Another closely related development has been the co-mounting of an Iris One camera system with a RIEGL VQ-580 airborne topographic laser scanner [Fig. 17]. The two devices are mounted closely together on a common base plate fitted with anti-vibration dampeners. The spatial relationship of the two devices, including their relationship to the accompanying GPS/IMU sub-system, is determined very precisely through the measurement of the lever arm offset during the system calibration. Visual Intelligence has also developed the concept of a five- or nine-camera system [Fig. 18] that it calls its “360° orthostereo-oblique” system. This generates ground coverage in the form of a “Maltese Cross” that is similar to the systems developed by Pictometry and Track’Air, but is based on the ARCA technology. Further developments of this architecture are currently focussed on the development, miniaturization and production of very light weight and compact camera systems [Fig. 19] that are fully metric and can be utilized in UAVs; in ground vehicles; and in mobile devices. [b] VI – Conclusion Fig. 17 – (a) The Iris One and RIEGL VQ-580 combination mounted side-by-side on a common base plate which is placed on a set of anti-vibration dampers – as viewed from the side at left and from above at right. (b) An iOne IMS camera system mounted alongside a RIEGL VQ-580 laser scanner. (Source: Visual Intelligence) modules of varying-format sitting side-by side; the one a colour RGB camera; the other an NIR camera. These are flanked by a pair of oblique-pointing cameras, that are pointing in opposite directions [Fig. 16]. Essentially this arrangement is similar to the Tri-Metrogon configuration discussed above. Since the system is designed specifically for the corridor mapping of infrastructure, the ARCA or “bridge” on which the cameras are mounted is oriented in the along-track (flight) direction. The system can be installed in a specially constructed pod that is fitted to the underside of a helicopter or it Fig. 18 – The “360° ortho-stereo-oblique” system (Source: Visual Intelligence) Visual Intelligence has already developed a most interesting series of airborne digital frame camera systems based on its scaleable and modular iOne Sensor Tool Kit Architecture. For the future, it will be very interesting to follow the concepts that are currently in the research and development stage and to see them come to fruition. Fig. 19 – A prototype of a three-camera ARCA or “bridge” made of carbon-fibre. (Source: Visual Intelligence) 36 Gordon Petrie is Emeritus Professor of Topographic Science in the School of Geographical & Earth Sciences of the University of Glasgow, Scotland, U.K. E-mail – Gordon.Petrie@glasgow.ac.uk; Web Site – http://web2.ges.gla.ac.uk/~gpetrie September 2012