IGARSS presentation WKLEE.pptx


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

IGARSS presentation WKLEE.pptx

  1. 1. An Efficient Automatic Geo-registration Technique <br />for High Resolution Spaceborne SAR Image Fusion<br />IGARSS 2011<br />28/July 2011<br />Woo-Kyung Lee and A.R. Kim<br />Korea Aerospace University<br />wklee@kau.ac.kr<br />
  2. 2. Motivation<br /><ul><li> As the resolution level improves, </li></ul> * the unique feature of the radar imaging becomes prominent and the task of image fusion with optical image becomes complicated, <br /> * the number of pixels increases and the amount of resources for calculation such as memory and time consumption escalates exponentially. <br />To relieve the burden of the work and make it done in real time.<br />Efficient image matching in both rural and urban regions<br />Simple approach to the SAR image registration and fusion<br />Let the machine do the rest of the job <br />in almost real time<br />One click<br />
  3. 3. SAR SensorandGeometricCharacteristic<br /><ul><li>Image processing depends on thesurface characteristics and structures
  4. 4. Radar images suffer from unrealistic distortions
  5. 5. Non-linear distortions along range, Shortening from shadow region
  6. 6. Inaccurate Doppler parameter estimation leads to geocoding errors
  7. 7. Unstability in internal system clock and orbit parameters</li></ul>System error <br />Side-looking Observation<br />SAR vs. optics images<br />Image acquisition<br />
  8. 8. SAR SensorandGeometricCharacteristic<br />Effect of Error<br />Error Source<br />SAR sensor<br /> - Electronic Time Delay<br /> - Slant Range Error<br /> - Incidence Angle Estimation<br /> - PRF Fluctuation<br />Effect of Error<br /> - Range Location<br /> - Range Scale <br /> - Azimuth Scale<br />Error correction method<br />- Geometric Calibration<br />- Deskew<br /> - Ground Projection<br /> - Image Rotation<br /> - Terrain Correction<br />Earth<br /> - Azimuth Skew<br /> - Range Non-Linearity<br /> - Foreshortening, Layover, <br /> Shadowing<br />Earth<br /> - Earth Rotation<br /> - Side-looking<br /> - Target Height<br />Source of SAR geocoding errors <br />Platform<br /> - Image Orientation Error<br /> - Squint Angle<br /> - Doppler Centroid<br />Platform<br /> - Inclination Angle<br /> - Yaw Angel Error<br /> - Pitch Angle Error<br />
  9. 9. SAR SensorandGeometricCharacteristic<br /><ul><li>Mismatch between SAR and Optical images</li></ul>Optics<br />SAR<br />Geometrical distortions in SAR images<br />(a) Azimuth Distortion<br />(b) Non linear Range Error<br />(c) Deskew<br />
  10. 10. SAR Geo-correction with satellite internal data<br /><ul><li>Slant range function has non linear scale variations</li></ul>Azimuth<br />Slant range image<br />Range<br />Sland based<br />Ground range image<br />Ground projection example<br />Reference image<br />(EO image)<br />Ground Based<br /><ul><li>Discrepancy exist compared with the reference image
  11. 11. Distortion between SAR and EO are case-sensitive</li></li></ul><li>GCP based geometry registration<br /><ul><li>A reference image is chosen to be used for geometric correction or fusion
  12. 12. Multiple GCP(Ground Center Point)s are selected and directly applied to individual position error calculation and correction i
  13. 13. Based on the selected GCPs, image transforml function is characterized that best describes the discrepancy between the images
  14. 14. Original image is re-sampled and re-arranged by the generated transform function
  15. 15. To perform geometrical calibration and restore distortion, the GCPs in the SAR images would be re-arranged into the true ground positions
  16. 16. It becomes most essential to pick up the best candidates of GCPs</li></ul>Basic Principle<br />Choice of GCP<br /><ul><li>In general, distinctive features such as road, building, bridges, reflectors are chosen that are easily discriminated for convenience
  17. 17. Manually? Or Automatically?? Who will chose what points??</li></li></ul><li>Selection of GCPs within SAR images<br /><ul><li> Speckle noise inherence in SAR image makes it difficult to guarantee to pinpoint precise positions that correspond to the reference points.
  18. 18. A human work of manual GCP selection is never reliable
  19. 19. The number of available points are case-sensitive and still limited by the existence of the distinctive features
  20. 20. The precision of the GCP location is not fully guaranteed and the error variance may increase in coarse resolution images. </li></ul>Difficulty of GCP choice<br />SAR image GCP<br />Optical image GCP<br />
  21. 21. Methodology<br /><ul><li>Speeded-Up Robust Features (SURF) is known to have highly robust performance
  22. 22. Scale, rotation and illumination-invariant feature descriptor.
  23. 23. Adaptive for noisy environment and mutll-scale images</li></ul>- Only summing operation is involved in producing integral image to match the scale and calculation is speeded up<br /><ul><li> Selection of matching points(GCP) are performed using feature vectors described by Hessian Matrix
  24. 24. The size of the constructed Hessian matix can be varied and can be increased to multiple dimensions as desired
  25. 25. The number of dimensions is limited by the complexity, time consumption and precision of the image matching.- case sensitive
  26. 26. Parameters required for the decision algorithms is set intuitively
  27. 27. This work is motivated to find the decision parameters automatically compromising the performance and the complexity</li></ul>SURF algorithm<br />Selection of GCP and matching<br />
  28. 28.                                            <br />Block diagram for GCP pair selection <br />
  29. 29. Integral image generation<br /><ul><li>For the given image, an integral set of points are summed together
  30. 30. The size is variable depending on the scale and complexity of the image
  31. 31. Simple summations of intensity levels are performed over two dimensional domain</li></ul>: A +B +C + D<br />
  32. 32. GCP candidate generation<br /><ul><li>The Hessian matrix , corresponding to each pixel, is simplified by summation with adjacent cells
  33. 33. The image scale is varied and the simplified Hessian matrix is obtained for each scale space
  34. 34. Harr-wavelet responses are calculated and the feature descriptor is generated
  35. 35. The polarity of the image intensity variation is investigated and stored</li></ul>Harr wavelet<br />X, Y direction<br />X direction<br />Y direction<br />
  36. 36. GCP selection <br /><ul><li> Find a pair of feature descriptors that best fit to each other
  37. 37. One-by-one comparison is straightforward but time-consuming and does not guarantee successful matching due to increased ambiguity
  38. 38. Construct a look-up table for the feature descriptor
  39. 39. Each feature descriptor is indexed depending on their size, variation rate, orientation
  40. 40. For a given GCP , a “search process” is performed within other look-up table generated from reference image and the best matching pair is selected
  41. 41. Nearest neighbor search is adopted to find the correct matching pair</li></ul>Principle<br />
  42. 42. GCP pair selection <br />- Among the selected GCPs, the Euclidian distance (x, y) – (x’, y’) are estimated to find the nearest points with similar feature descriptor<br /><ul><li>The rates of intensity variations along orientations (denoted as Aand B) are considered as weighting factors
  43. 43. Distance multiplication is performed
  44. 44. The number of orientation can be increased in order to reduce ambiguity and avoid wrong decision.
  45. 45. Appropriate threshold level is required to compare with the distance multiplication and make a decision
  46. 46. The GCP match is confirmed when the distance multiplication is less than the threshold level
  47. 47. Image Projective Transform function is deduced from the matching GCPs</li></ul>Define threshold level<br />
  48. 48. Overall procedure diagram for image matching<br />
  49. 49. GCP extraction demonstration <br /><ul><li>ScanSAR image is exposed to higher noise level and composed of extended resolution cells.
  50. 50. GCP candidates are extracted from both images using the same Hessian matrix structure
  51. 51. The number of GCP points appear to be close to each other despite the gap in the image quality</li></ul>GCP extraction from SAR images <br />(a) Stripmap image<br />(b) Scan image<br />
  52. 52. Geo-registration demonstration<br /><ul><li>Original SAR image is corrected using GCP matching and transformation
  53. 53. Strip mode SAR image over Vancouver, Canada is geo-registered using the reference image in Radarsat-1 SSG format
  54. 54. The threshold level is set to be zero for convenience </li></ul>881<br />557<br />Time consumption vs Th. Level<br />GCP # vs. Threshold level<br />GCP selection for raw image<br />Raw<br />Reference<br />912<br />544<br />Time consumption vs Th. Level<br />GCP # vs. Threshold level<br />GCP selection for reference image<br />GCP selection<br />
  55. 55. Image Matching Demonstration<br /><ul><li>Original image is geo-corrected </li></ul>Fusion image<br />
  56. 56. Measure of registration errors<br /><ul><li>RMSE(Root Mean Square Error) is calculated for the selected GCPs</li></ul>Corrected <br />Reference<br />The average deviation is about one point pixel size<br />
  57. 57. Application to higher resolution images<br /><ul><li>Reference image of TerraSARin EEC format, Toronto,
  58. 58. The number of GCP increases consistently when the level of correlation between the two images are high</li></ul>As the similarity of the images are high, the GCP increases consistently as the “Threshold Level” decreases<br />1.72<br />1595<br />0.81<br />252<br />Original<br />Reference<br />3.21<br />2680<br />GCP variation rate<br />1.47<br />404<br />GCP selection<br />
  59. 59. Image Matching Demonstration<br />After fusion<br />
  60. 60. Mismatch Error Estimate<br />Corrected <br />Reference<br />The average position error is less than one pixel<br />The performance of the matched GCP selection is affected by the image resolution<br />Mismatch error is reduced as the image resolution improves <br />
  61. 61. Image fusion of SAR over EO <br /><ul><li>This case is where SAR image constitute a small portion of the EO image
  62. 62. GCPs from the two images are distinguished </li></ul>- The matching GCPs are easily identified by the nearest search algorithm <br />(b) LANDSAT EO image<br />(a) JERS SAR image<br />
  63. 63. Automated threshold level selection<br />There exist a turning point, where <br />further reduction of the threshold level stops affecting the number of GCP matching points<br />Computer traces the variation of the available GCP matches and find the turning point<br /><ul><li> Threshold variance</li></ul>Threshold 500, Matching image (14 points)<br />Threshold 200, Matching image (15 points)<br />Threshold 350, Matching image (15 points)<br />
  64. 64. GCP matching and Image transformation <br /><ul><li>Matching GCP selection process stops automatically and image transform function is obtained from the selected points</li></ul> 15 points extraction<br />Feature points extraction<br />Find equation<br />Transform equation<br />
  65. 65. Result of the geo-registration<br />Overlaid image<br /><ul><li>RMSE is affected by the resolution discrepancy and inherent image property. - Here it is given as 1.26 pixel </li></li></ul><li>Automated Geo-registration software<br /><ul><li>Usually, the threshold level is manually set-up by user looking into the complexity of the images and resultant fusion performance
  66. 66. This procedure is replaced by compute search algorithm, where the threshold level is traced to find the turning point
  67. 67. Total elapsed time is within several minutes and will be further reduced by adaptive search algorithm</li></ul>Original<br />Reference<br />Corrected<br />GCP selection and matching <br />Fusion<br />
  68. 68. Performance analysis vs. Resolution<br /><ul><li>The number of total GCP is not affected by modes(Stripmap and Scan) </li></ul>(a) Stripmap image<br />(b) Scan image<br /><ul><li>However, the RMSE is measured as 0.94 for ScanSAR mode while it is 0.34 for stripmap mode
  69. 69. It appears that the performance improves as the resolution improves</li></li></ul><li>Insufficient information for SAR geometry<br /><ul><li>Internal data within SAR instrument fails to retrieve shadow region
  70. 70. There is non-linear discrepancy between slant and ground ranges
  71. 71. Generate Errors in geometrical coordinate
  72. 72. Need external references to retrieve broken information and correct errors in ground range allocations
  73. 73. foreshortening, layover, shadowing</li></ul>Limited information<br />Shadowing<br />Layover<br />Foreshortening<br />
  74. 74. Impact of the ground characteristics<br /><ul><li>Diverse ground geometry becomes a source of matching errors
  75. 75. Mountain areas are severely distorted from the EO case
  76. 76. Need to adopt separate transform functions within the image</li></ul>After correction<br />After correction<br />Coast line area<br />Mountain area<br />Mountain area fusion<br />Coast line fusion<br />
  77. 77. Matching Performance Comparison <br /><ul><li> Image distortion is not compensated for by the simple GCP transformation
  78. 78. Need to divide blocks and adopt modified transform functions separately</li></ul>Coast line<br />Mountain<br />
  79. 79. Modified Transform functions<br /><ul><li>Image is divided into blocks according to the geographical properties</li></ul>Average 1.8 RMSE error<br />Mountain Area<br />Average0.67 RMSE Error<br />Urban Area<br />Application of separate transform function leads to the reduction of RMSE<br />
  80. 80. Summary <br /><ul><li>SURF provides an efficient tool to perform SAR image geo-registration
  81. 81. A choice of threshold level is required to perform efficient of GCP matching and it can be automated by tracing its variation curve
  82. 82. The image matching algorithm works with various SAR and EO images and the average RMSE is measured to be around 1 pixel.
  83. 83. Image blocks containing mountain areas need separate GCP matching and transform function to compensate for image distortion
  84. 84. Need to develop optimized transfer function for different type of ground characteristics</li></ul> - Indexing of GCP is performed based on their intensity and variation vector <br /><ul><li>With the introduction of adaptive indexing table for selected GCP, the automated image matching is expected to be carried out in real time</li></ul>Conclusion<br />Further work<br />