Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The i one ims v-iisual intelligence technical white paper - armando guevara


Published on

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

The i one ims v-iisual intelligence technical white paper - armando guevara

  1. 1. © The iOne IMS Visual Intelligence -- LiDAR News 1The iOneTMInfrastructure Metric-Mapping SystemA Paradigm Shift on Co-Mounting and Co-Registering Geoimaging Sensors with LiDARDr. J. Armando GuevaraVisual Intelligence LP - President and CEOarmando.guevara@visualintell.comABSTRACTIntegration efforts of geoimaging sensors with LiDAR have demonstrated the need tosystematically improve the co registration of the imagery with the LiDAR data such that errors in theimagery collected are greatly reduced by the sensors being rigidly mounted, share the geopositionalmetadata and are registered to each other in a rigorously calibrated metric configuration.At Visual Intelligence (“VI”) a committed pursuit with our customers is to increasingly enablethem with our ubiquitous metric geoimaging sensor technology to collect more, do more, for less; thiswhilst abating the increasingly speed at which digital devices become obsolete. In this pursuit VI hasdeveloped the iOneTMSensor Tool Kit Architecture (or iOne STKATM) from which the iOne InfrastructureMetric-Mapping System has been developed. The iOne IMS is a modular and scalable co mounted and coregistered (“CoCoTM”) geoimaging sensor with LiDAR that can readily, efficiently and economically beconfigured to fit a variety of infrastructure surveying applications. This paper describes first the iOneSTKA and the iOne IMS, its core design and the operational efficiencies it provides.KEYWORDS: iOne, STKA, Iris One, Digital camera, Camera calibration, Co mounting and Coregistration of Sensors, infrastructure mapping and surveying1. INTRODUCTIONFounded in 1997, Visual Intelligence (VI) has focused on research and development (R&D) to providea multipurpose metric digital geoimaging sensor technology with scalable sensor imaging arrays forautomated high-accuracy metric geoimaging for mapping, surveillance, ground and mobile applications.The sensor architecture is designed to be economical (lowest cost of ownership), light, small, highcollection, high resolution, and fast in deployment. The multi-year R&D has resulted with various grantedpatents that have provided the foundation to generate a Virtual Frame (VF) camera systems comprised ofmultiple COTS camera modules arranged at certain angles to achieve flexible and rapid configurations asdifferent and distinct (sometimes conflicting) mission requirements may mandate.The patents awarded along with the USGS Aerial Digital Sensor Type Certification received in 2009for the Iris One 50 (now called the Iris One Ortho 19 kps); validate the uniqueness of VI’s intellectualproperty, technological foundation, and its forthcoming potential transforming role in the digitalgeoimaging industry. The current and evolving portfolio of VI IP has been casted into a sensor tool kitarchitecture called the iOne Sensor Tool Kit Architecture or iOne STKA. The scalability of the sensorsbuilt from the iOne STKA are based on the Angular Retinal Camera Array (ARCA); this scalabilityproperty allows for both functional and collection scalability. Functionally the sensors can be configured tohave only or many features such as ortho, stereo, oblique, full 3D as well as CoCo -(the co mounting and coregistering of sensors; e.g. imagery fusion- such as LiDAR, thermal, SWIR, FLIR, multispectral andhyperspectral among other types of passive (e.g. electro-optical) and active (e.g. LiDAR, radar) sensors.The modularity of the iOne STKA allows flexibility and scalability to meet various customer needs andapplications within one single base sensor system (hence “iOne”).Visual Intelligence among its sensor family built from the iOne STKA has brought to market the IrisOne Ortho 19 kps; the Iris One MS; the Iris One Stereo and recently the iOne Infrastructure MappingSystem or iOne IMS –the subject matter of this paper. The iOne family of digital geoimaging sensorsystems in general, and in particular the iOne Stereo, is designed to achieve and exceed the performance ofthe film aerial cameras, in collection capacity and metric accuracy. For a detailed description of the IrisOne family of sensors created to-date from the iOne STKA please see Petrie, Gordon 2012.
  2. 2. © The iOne IMS Visual Intelligence -- LiDAR News 22. iOne STKA ARCA BASED DESIGNSThe iOne STKA is an optimal set of software, hardware, methods and procedures geoimagingcomponents (”Lego®-like) that include advanced imaging processing algorithms for radiometric andgeometric accuracy, pixelgrammetry processing and image analysis (feature extraction, point cloudgeneration); it is a flexible and modular set of components, all solid state that are all integrated by software.The iOne STKA is based on the ARCA (Guevara, 2009), a patented angular cross eyed imaging arraythat allows the system to be small, light and scalable in collection capacity, resolution, and functionallyusing the same base architecture –i.e. One system for all applications. An advantage of the ARCA is thatcamera modules are configured in a linear arrangement with an “hour glass imaging effect”, giving it theadvantage of imaging a larger swath while looking through a small aperture; the optical axis of eachindividual CM in the array to intersect, passing through a single perspective center. The patented ARCAdesign uses synchronously operating camera module heads to form a single virtual central-perspectiveimage.With multiple ARCAs, Iris One system can be easily configured as multi-spectral sensor (doubleARCAs) and stereo system (triple ARCAs). The multi-spectral version of Iris One system allows the colorRGB images recorded by camera modules mounted on one ARCA to be co-registered with andsuperimposed on the corresponding near infra-red (NIR) images collected by the cameras mounted on thesecond ARCA. The Iris One stereo system, with three ARCAs, can be oriented either in the cross-track orthe along-track direction. Vary-format camera modules and different types of lens produce various groundcoverage. There are two typical settings for the Iris One stereo system. The 60 % longitudinal overlap alongthe flight line that is produced when the cameras are programmed to expose overlapping sets of images in astereo convergent configuration gives a base: height ratio of 0.6 –unique in the industry. When the systemis equipped with 9x29 MPIx camera modules and 135mm lenses and rotated by 90 degrees into the cross-track position, the system yields a 0.34 base: height ratio, similar to that achieved by the overlapping stereoimages that are produced by conventional large-format digital mapping cameras.(a) (b)Figure 1- (a) This figure shows the ARCA(s) into which varying-format camera modules can be inserted.(b) The geometric arrangement of an ARCA configuration; each ARCA or set of ARCAs generates a singlemetric frame ingestable by any 3rd party photogrammetric workflow.As such the iOne STKA allows to configure combinations of camera modules (CMs) that yield crosstrack collection efficiencies from 20,000 pixels upward of 60,000 pixels cross track, as well as frame sizesalong track in excess of 19,000 pixels (depending on how the arrays are aligned). Table 1 depicts part of thedeveloped Iris One systems family. For a detailed technical description of the ARCA and its functional andcollection efficiencies please see Petrie, Gordon 2012.
  3. 3. © The iOne IMS Visual Intelligence -- LiDAR News 3Iris One Ortho/MS 19 kps A highly agile and robust system for ortho wide area collectionIris One Stereo withconfigurable B/H .6 or .3depending on ARCA crosstrack or along trackorientationBased on B/H .6, only sensor in the industry with engineering qualityimagery equivalent or superior to film. The same system can be “rotated”to achieve higher collection efficiencies whilst achieving .3 B/H.Iris One InfrastructureMapping System (IMS)A powerful, very light and compact sensor that provides ortho, multi-spectral, backward and forward oblique, all in one passIsis Earth™ Software Post processing software that is integrated with Iris One sensors used togenerate accurate ortho images.Isis Sky™ Software Near real-time onboard (in-flight) ortho processing software that isintegrated with Iris One sensors.Table 1- The Iris One family of sensors and software based on the iOne STKA is capable of handlingnumerous collection scenarios.For engineering-quality metric application, high accuracy and resolution requirement in both geometricand radiometric aspects must be met (Cramer, 2006). The geometric accuracy of the Iris One digitalimaging sensor systems is achieved from laboratory calibration of each camera module, the arrays set, aswell as calibration flight using BINGO (Kruck, 2010; Hwangbo, 2012). With highly accurately determinedgeometric properties for the complete camera module set in the ARCA array(s), the Iris One system is ableto produce virtual frames from each ARCA CM set which is defined as single central perspective,distortion-free image. The simple geometry of the ARCA Virtual Frame image makes it compatible withthe traditional workflow of any photogrammetric software. Moreover, radiometric calibration explorescamera’s radiometric properties to naturally link image data with actual scene for high-quality imageryproduction. For further in depth technical details on both geometric and radiometric calibration procedures,please refer to Hwangbo 2012 and Guevara-Wang 2012).Figure 2- VI’s geometric calibration facilities: a). Distribution of control points used for laboratorycalibration (red dots are located on a 2D calibration wall and blue stars are located on a 3D calibrationframe); b). Calibration field with distribution of ground control points.3. INTRODUCING THE CONCEPT OF KPSA kps is the number of pixels across track covered by a sensor on the ground, in other words, thenumber of pixels in the swath. 1 kps = one thousand pixel wide swath. Therefore the number of pixelsallowed by an iOne sensor to collect is defined as kps, or kilo pixel swath. So depending of the applicationthe sensor can be as small as 7 kps to as large as it is required by the mission of the sensor, all using thesame base architecture –feature that yields very fast sensor deployment time.(a) (b)
  4. 4. © The iOne IMS Visual Intelligence -- LiDAR News 4Why kps? It is often difficult to differentiate the competing claims of different digital aerialcamera manufacturers when it comes to efficiency. There are many aspects that contribute to theefficiency, but one simple measure is for the same aircraft speed, how much area is collected per hour offlying?There are many factors that go into the design of a successful aerial metric geoimaging project,including but not limited to camera focal length, CCD size and flying height. For digital cameras, theproject is designed for a specific nominal ground sample distance or GSD. No matter what flying height,focal length, or CCD size, the amount of area covered is the nominal GSD times the number of pixels in theselected width of the CCD array (x or y orientation – typically the largest number of pixels on the CCD isin x so if more depth is desired the CCD can be rotated with x becoming y).Normal block collection of a project will factor in a 30% side lap between flight lines. Examples of pixelsswath with “CCD configurations” of:A. 7 kps will have an effective swath 5051 wideB. 11 kps will collect a swath 8,043 wideC. 19 kps will collect a swath 13,280 wideD. …and so on.kps Width30% sidelap7 5,051 2,16511 8,043 3,44719 13,280 5,691In a project example with a flying speed of 150 miles an hour times 5,280 feet in a mile, the aircraft travelsapproximately 792,000 feet an hour. Efficiency results for the above kps example are shown on thefollowing chart:TraveledDistance/hour Width sq feet sq miles EfficiencyA 792,000 5,051 4,000,550,400 144 1.00B 792,000 8,043 6,370,056,000 228 1.59C 792,000 13,280 10,517,522,400 377 2.63Figure 3- Impact of kps in collecting at 1 foot pixels (30 cm)
  5. 5. © The iOne IMS Visual Intelligence -- LiDAR News 54. INTRODUCING CoCoIn 1998 VI designed, built and operated its first generation of digital aerial imaging and mappingsensor. In 2001 VI acquired its first LiDAR. Since then, VI has been at the leading edge of innovation byimproving the operational use of LiDAR technology tightly coupled (co mounted and co registered) withaerial digital cameras with different kps and FOV according to mission. Recently VI integrated LiDARtechnology with its 3rd generation imaging sensor technology, the iOne IMS (infrastructure metric-mapping system).CoCo is a patented vehicle based data collection and processing system and imaging sensorsystem and methods thereof. The apparatus’ and methods optimize the co-mounting and co-registering oftwo or more sensors, for example, an EO camera system with a LiDAR rigidly mounted on a single platewith IMU. The claims were directed towards the incorporation of the co-mounted and co-registered natureinto VI’s system for terrain mapping which obtained unprecedented sensor integration performance forlarge scale and large geographic area mapping.The integration of CoCo begins with mounting the two sensors together. The sensors must berigidly mounted on the same base plate so that aircraft flex is minimized. This flex needs to be less than100th. The sensors must be mounted as close as possible and best practices have the two sensors mountedover the same aperture. This is possible because the Iris One’s ARCA can operates without the need for agyro-stabilized mount. The nearness of the sensors is needed so that they can share the same ABGPS/IMU.This is only possible also because the Iris One family of sensors have a light, compact and rigid design thatcan be Co-mounted with out the need for an additional standard camera hole.Figure 4- CoCo System DiagramThe lever arm measurements must be taken for each sensor so that when processed each sensorhas its own offsets. By sharing the IMU and GPS, all sensors are calibrated substantially the same epoch,using the same GPS signal, ground targets and under the same atmospheric conditions. Basically thismeans, each image pixel and each LiDAR pixel are referenced from the same location. This greatly reducescompounded errors realized when calibrating each separately, using different GPS signals or underatmospheric conditions. Separate sensor holes (with no co-mounting) with a shared IMU is not the same, asthe aircraft has its own flex that can change or move independent to each other which causes positionalerrors which may work depending on accuracies needed. Separate IMUs is another approach but each IMUwill have its own biases and cause differences in yaw, pitch, and roll data.
  6. 6. © The iOne IMS Visual Intelligence -- LiDAR News 64. INTRODUCING iOne IMSTMThe Iris One Infrastructure Metric-Mapping System (iOne IMS) camera system based on the iOneSTKA, integrated with a LiDAR system in CoCo mode, is an efficient, economical, one-pass, all featuredigital infrastructure capture system that can support numerous image data requirements using a helicopteror fixed wing aircraft. From a single control interface, operators can capture and monitor imagery fromhigh-resolution oblique, wide swath multi-spectral, and optional thermal and video cameras referenced to asingle GPS/IMU reference system for common picture and overlapping display.With its single pass, full capture capability, the system produces accurate, high-quality imagerysaving 50%-75% over less efficient aerial collection cost. With its multi-sensor ARCA based platformarchitecture, the system can grow to accommodate additional sensors thereby further increasinginformation value with little added data collection cost.The iOne IMS configuration includes the following benefits:• Two camera oblique images (fore/aft) assures full coverage• Single wide-field NADIR cameras for full swath 4-Band coverage with high positional accuracy• Single operator control with on-board Quick-Look quality review• B/H = 32% at 700 ft AGL to support future stereo/corridor-wide DTMFigure 5- (a) the iOne IMS dimensions –a small, light compact system that can be flown on rotary or fixedwing aircrafts (b) iOne IMS capability to collect ortho, near infrared, backward and forward oblique –all inone pass.The iOne IMS sensor is designed to collect orthophotos (RGB and Near Infrared) and obliqueimagery (forward and aft) simultaneously with a LiDAR sensor. The collection of laser and imagery data isbeing conducted to be able to support, for example, the generation of surveys of power line transmissioncorridors.The iOne IMS generated surveys support the following categories of analysis:• Inspection of transmission hardware mounted on towers• Right -of -Way Analysis• Inspection of power and relay substations• Power line sag and tension analysis• Encroachment of man-made and natural vegetation (e.g. trees)
  7. 7. © The iOne IMS Visual Intelligence -- LiDAR News 7This system will support the same analysis functions in other transmission corridors such as railways,pipelines and others as required. The nominal parameters for the oblique sensor system are defined below:Figure 6- (a) Minimum Oblique Collection Geometry(Nominal altitude = 700 ft, Range, 850 ft, GSD approx. 0.5 inches) (b) Nadir Ortho Collection GeometryFeatures Analysis FactorsInsulators/Conductors Size, Texture, Condition, Nominal (0.44 inch) MaxGSD (1 inch)Transformers Size, Condition, Nominal (0.44 inch) Max GSD (1inch)Transmission Lines Condition, Sag, Sway Distance, Nominal (0.44inch) Max GSD (1 inch)Transmission Towers Condition, Size Envelope, Max GSDGround Vegetation Location, Distance to Towers, Max GSD, BandsTrees/Foliage Location, Distance to Sway Line Limit, BandsFences LocationAccuracy standards if processed with solid ground controlof Manmade Structures within EasementLocationRoads/Access to Easement LocationExamples of Electrical Infrastructure Feature Types that can be Collected- Sizes and CharacteristicsNominal Distance between camera and target ft 850 ftEstimated tower height (minimum) 60 ftEstimated tower height, (maximum) 200 ftEstimated tower height, (average) 120 ftTower covers % of image height (minimum) 75Desired resolution of tower, inches/pixel 0.44
  8. 8. © The iOne IMS Visual Intelligence -- LiDAR News 8Fig. 7- (a) iOne IMS Oblique sample image (b) Oblique image and automated extraction offeatures of interest for further detailed analysis.The operational envelope of the iOne IMS is as follows:Corridor (or Coverage) Min/Max for Nadir Ortho Camera• Minimum Swath: 600 feet• Nominal Swath: 750 feet• Desired Swath: 900 feetCorridor (or Coverage) Min/Max for Oblique Camera• Minimum Swath: 100% of infrastructure feature surveyed Width & Height• Nominal Swath: Minimum Swath to Support feature collected
  9. 9. © The iOne IMS Visual Intelligence -- LiDAR News 9GSD Min/Max• Oblique Camera Minimum GSD: 0.44”• Oblique Camera Maximum GSD: 1.0”• Nadir Cameras Minimum GSD: 6”• Nadir Cameras Maximum GSD: +- 10%iOne IMS sensor operation• Single Operator• Minimal Operations Workload; Flight technician-level skillsAltitudes (MSL) and (AGL) min/max• Minimum Altitude (AGL): 600 ft• Nominal Operations Altitude (AGL): 700 ft• Maximum Altitude (AGL): 3,000 ftEnvironment/Sun Angle/Time of Day• Operations within +/- 20 deg sun angle range such that features are interpreted (detected)in shadows.• Platform Angle Range• +/- Roll : system controlled• +/- Pitch: system controlled• +/- Yaw: system controlled• Ground Speed Velocity and Tolerance: system controlled• Features are interpretable in shadows.Typical Mission Day• Pre-mission calibration (boresight)• 4-6 hours of collection• Camera system operation is automated. No more than 5 minutes/hour operationsrequired for system operations monitoring• 2TB on-Board SSD Data Storage (1-1.5 TB typical)• 1 Hour Post Mission Data Quality Analysis• On-site Imagery Review with Post-flight previewer softwareThe Iris One IMS will produce multiple image products for use in utility corridor status analysis andasset assessment. These products are summarized as:Image RGB Oblique of full features (e.g. Towers) -100% coverage- front/back (image pair)o GeoTiff with lat/long location center, image scaleo One image per feature structure is generated – iOne IMS creates an oblique virtual frameif the feature appears in two or more images. Feature virtual frames are minimized.o Provides KML with camera orientation during exposure (meta data option)o Google Earth KMZ file with image and camera locations (meta data option)Provides Multispectral Image -Four-Band Orthos- (metrically co registered at 1:1 resolutionRGB+NIR)The images are color balanced across for the mosaicing workflow.o Oblique Images maintain color saturation, intensity, and hue across the missiono Ortho images support color balancing and mosaicing methods
  10. 10. © The iOne IMS Visual Intelligence -- LiDAR News 10The iOne IMS was designed for ease of use for the operator and minimal training requirements tobe proficient at operating the system. VI includes a flight planning software tool called TopoFlightNavigator that is bundled with the iOne Isis Earth orthophoto system.TopoFlight Navigator is used to navigate the aircraft for image acquisition flights. A predefinedflight plan (e.g. provided using TopoFlight) is used as base data. The camera is triggered at the pre-definedpositions. The interface for every camera can be delivered or can be implemented by the operator. Thesystem consists of different modules to provide the capability to combine the actual TopoFlight Navigatorwith any available GPS, IMU and camera system.(a) (b)Fig. 8 (a) The iOne IMS and RIEGL VQ-480 combination mounted side-by-side on a common base platewhich is placed on a set of anti-vibration dampers – as viewed from the side at left and from above at right.(b) The iOne IMS system mounted in an Aerocommander aircraft.5. OPERATIONAL EFFICIENCIESFor many projects collecting LiDAR and Imagery together can lead to less flight time or eliminatethe need for an additional aircraft with separate sensors. Over the years since VI established the CoCoapproach, we have created for our users innovative ways to collect the data simultaneously. Some examplesfollow.5.1 ForestryCoCo collection was used in several forestry projects where the Imagery FOV was 70° and theLiDAR FOV of 45° (due to point density needed). Since the LiDAR was the limiting factor, VI devised forits customer the flight plans to maximize each sensor FOV. To solve for this the imagery and LiDAR werecollected on even numbered flight lines during prime sun angle and LiDAR only was collected on oddnumbered lines during times of less than optimal sun angle. This improved overall collection time by usingthe least number of lines and only one aircraft with one flight and ground crew. This approach allowed forthe imagery to be flown with the minimal amount of flight lines, which relates to less data to process, fasterdeliverables and overall operational savings.5.2 Infrastructure Corridor MappingCoCo was used in a corridor mapping project where the LiDAR was flown to create more accurateDEMs for the ortho imagery. Sample case project collected 500 miles of pipeline. The customer wanted a 1mile swath of imagery and 3500ft swath of LiDAR. Instead of using two aircrafts, one with a LiDAR andthe other with a digital camera, the projects was flown with both simultaneously, and by using an Iris One19 kps the project was flown more efficiently- having both sensors collecting concurrently reduced costsby 50%.
  11. 11. © The iOne IMS Visual Intelligence -- LiDAR News 116. CONCLUSIONSThis paper has described the iOne STKA, the CoCo technology and its new embodiment the IrisOne Infrastructure Metric-Mapping System or iOne IMS, an efficient, economical, one-pass, all featuredigital infrastructure capture system that can support numerous image data requirements using a helicopteror fixed wing aircraft. The operational improvements (data collection, time, cost) obtained by flying intandem the iOne IMS in CoCo mode can lead to great operational efficiencies and cost savings such as lessflight time and/or eliminate the need for an additional aircraft with separate sensors.With the iOne STKA VI has created a robust and solid software and hardware Lego®-likefoundation to design and deploy any type of EO sensor, and if required, fused (“CoCo” - co mounted andco registered) with any other passive or active sensor type in the most effective and efficient manner, e.g.LiDAR, thermal, video, UV. The iOne STKA is backed by numerous patents and IP (methods, proceduresand software) that yield a very powerful plug-and-play sensor foundation. Methods and procedures includebut are not limited to robust geometric and radiometric calibration; very large virtual frame generation thatis ingestible by any traditional photogrammetric workflow (the ARCA array set behaves like one singlecamera); ortho direct positioning onboard processing software that is the platform for event driven reportgeneration and more.REFERENCESCramer, M., 2006. Calibration and validation of airborne cameras. Proceedings ISPRS Commission ISymposium “From Sensor to Imagery”, Paris – Marne Le Valle, July 4-6, 2006.Guevara, A., 2009. The ARCA of Iris: a new modular & scalable digital aerial imaging sensor architecture.ASPRS 2009 Annual Conference, Baltimore, March 9-13, 2012.Guevara, A.; Wang, W 2013. The iOne STKA Foundation for the Iris One Sensor Family. ASPRS 2013Annual Conference.Hwangbo, J, 2012. Iris One Stereo System, ASPRS 2012 Annual Conference, Sacramento, March 19-23,2012.Kruck, E., 2010. Developments and challenges in bundle triangulation, ASPRS 2010 Annual Conference,San Diego, April 26-30, 2010.Petrie, Gordon 2012. Visual Intelligence’s Iris One Airborne Camera Systems - Based on its iOne SensorTool Kit Architecture. Emeritus Professor of Topographic Science in the School of Geographical & EarthSciences of the University of Glasgow, Scotland, U.K. E-mail – ; Web Site – - Geoinformatics Magazine (September 2012 issue no. 6/2012