Buildings Recognition and Camera Localization Using Image Texture Description  SULEIMAN Wassim 1 ,  JOLIVEAU Thierry 1 , F...
Objective Find  a building in an image SIG 3D 3D  GIS Locate the camera that took the image by using the location of the b...
Methodology <ul><li>  Enhancing the GIS database with information which can describe the building </li></ul><ul><li>unique...
How to isolate the building facade in the image? <ul><li>Manual method (long) </li></ul>Sourimant   2009 <ul><li>Automatic...
Work plan <ul><li>GIS database enhancement with building texture information </li></ul><ul><li>Facade recognition </li></u...
Work plan <ul><li>Enhancing GIS databases with building texture information </li></ul><ul><li>Facade recognition </li></ul...
Enhancing GIS databases with building texture information Texture analyses (SIFT) Finding the interest points with their l...
Enhancing GIS databases with building texture information Finding the (x,y,z) of the interest points Homography constraint...
Enhancing GIS databases with building texture information The texture descriptor  : list of interest points with their loc...
Work plan <ul><li>Enhancing GIS databases with building texture information </li></ul><ul><li>Facade recognition </li></ul...
Facade recognition False matching because of the locality of the descriptor
Facade recognition <ul><li>Eliminate the false matching using the homography constraints </li></ul><ul><li>Select the best...
Facade recognition The facade in the 3D GIS
Work plan <ul><li>Enhancing the GIS database with building texture information </li></ul><ul><li>Facade recognition </li><...
Camera Geolocation Association of the interest points with the 3D position of the matched points in the GIS databases
Camera geolocation 4 points non-collinear (Yang & al. 2009)  Real position Measured position
Camera Geolocation <ul><li>Error for distance (20-100)m and angle (0-30°) camera direction and facade normal : </li></ul><...
<ul><li>Error because the texture description is not an affine function </li></ul>4 points  non-coplanaires   SOFTPOSIT (D...
Work plan <ul><li>Enhancing the GIS database with building texture information </li></ul><ul><li>Facade recognition </li><...
Possible Applications <ul><li>Management of photos taken in urban areas </li></ul>Link GIS
Possible Applications <ul><li>Navigation systems support in an urban environment </li></ul><ul><ul><li>Satellites visibili...
Possible applications <ul><li>Initial phase in the (2D/3D) registration </li></ul>Sourimant   2009
Work plan <ul><li>Enhancing the GIS database with building texture information </li></ul><ul><li>Facade recognition </li><...
Limits <ul><li>Angle between camera direction and facade normal has to be less than 30°  </li></ul>
Limits <ul><li>Distance between camera and facade has to be less than 200 m </li></ul>
Limits <ul><li>Identical facades :  </li></ul>
Limits <ul><li>Glass facade which reflects the sky and other buildings </li></ul>
Thank you For your attention Suleiman wassim [email_address]
Upcoming SlideShare
Loading in...5
×

Buildings Recognition and Camera Localization Using Image Texture Description

425

Published on

3D GIS model/2D image registration called much attention in the recent years because of its vast range of potential applications in real and virtual navigation. However, automatic registration remains until now a challenge. This paper presents a methodology for enhancing and complementing a GIS database of buildings with a descriptor of their texture by using information extracted from a reference images. This descriptor is used to locate any other image by searching similar texture in the image. Then the absolute position and orientation of the camera of the new image can be computed if the camera parameters (like focal length) are known. The paper proposes a technique that can be used for achieving the identification of the facade in the photograph, calibrated camera geolocation and discusses the quality of the results.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
425
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • SURF is faster than SIFT descriptor. But our tests show that the SIFT descriptor gives better results when the angle of view vary between two images SIFT code of VLFEAT and SURF code from OPENSURF. http://www.vlfeat.org/~vedaldi/code/sift.html http://www.mathworks.com/matlabcentral/fileexchange/28300
  • SURF is faster than SIFT descriptor. But our tests show that the SIFT descriptor gives better results when the angle of view vary between two images SIFT code of VLFEAT and SURF code from OPENSURF. http://www.vlfeat.org/~vedaldi/code/sift.html http://www.mathworks.com/matlabcentral/fileexchange/28300
  • SURF is faster than SIFT descriptor. But our tests show that the SIFT descriptor gives better results when the angle of view vary between two images SIFT code of VLFEAT and SURF code from OPENSURF. http://www.vlfeat.org/~vedaldi/code/sift.html http://www.mathworks.com/matlabcentral/fileexchange/28300
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • The level of the recognition based on the number of the interest points and their discriminative descriptor
  • The level of the recognition based on the number of the interest points and their discriminative descriptor
  • The level of the recognition based on the number of the interest points and their discriminative descriptor
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • With four points (three non collinear scene points or non collinear image points). (Yang et al. 2009) is applied. It is based on the homography matrix between the front wall in the query image and the 3D model. The best result is obtained when the camera is located just in front of the building at a distance less than 100 meters. The quality falls down steeply also if the distance between the camera and the building exceeds 70 m (low resolution), and the view angle is above 30 degrees (non affine descriptor).
  • With four points (three non collinear scene points or non collinear image points). (Yang et al. 2009) is applied. It is based on the homography matrix between the front wall in the query image and the 3D model. The best result is obtained when the camera is located just in front of the building at a distance less than 100 meters. The quality falls down steeply also if the distance between the camera and the building exceeds 70 m (low resolution), and the view angle is above 30 degrees (non affine descriptor).
  • We need at least 4 non coplanar points. The SOFTPOSIT algorithm (David et al. 2004) is used which has a high efficiency If one of the discovered facades shows an angle above 40 degrees, this facade will cause more error in the computation and contribute to the drop down of accuracy in the position of the camera
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • Approximate position and orientation of the camera are the first step of an automatic registration process Urban environment with low GPS precision (Li et al. 2006) (Bioret et al. 2009). Affine descriptor (Mikolajczyk &amp; Schmid 2004) for the facade texture. This could greatly ameliorate the result for multiple facades detection If the method should be applied on Smartphones with continue video data, the SURF descriptor could perform better than SIFT because it is faster ( http://www.kooaba.com/ )
  • Approximate position and orientation of the camera are the first step of an automatic registration process Urban environment with low GPS precision (Li et al. 2006) (Bioret et al. 2009). Affine descriptor (Mikolajczyk &amp; Schmid 2004) for the facade texture. This could greatly ameliorate the result for multiple facades detection If the method should be applied on Smartphones with continue video data, the SURF descriptor could perform better than SIFT because it is faster ( http://www.kooaba.com/ )
  • Approximate position and orientation of the camera are the first step of an automatic registration process Urban environment with low GPS precision (Li et al. 2006) (Bioret et al. 2009). Affine descriptor (Mikolajczyk &amp; Schmid 2004) for the facade texture. This could greatly ameliorate the result for multiple facades detection If the method should be applied on Smartphones with continue video data, the SURF descriptor could perform better than SIFT because it is faster ( http://www.kooaba.com/ )
  • The results are not always good, because of the error in the 3D model, the complexity of urban environment and the collusion Sourimant et al., 2009. GPS, GIS AND VIDEO REGISTRATION FOR BUILDING RECONSTRUCTION
  • Buildings Recognition and Camera Localization Using Image Texture Description

    1. 1. Buildings Recognition and Camera Localization Using Image Texture Description SULEIMAN Wassim 1 , JOLIVEAU Thierry 1 , FAVIER Eric 2 1 ISTHME-ISIG CNRS/UMR EVS, Université Jean Monnet - Saint-Etienne. 2 DIPI EA 3719 École Nationale d'Ingénieurs de Saint-Etienne [email_address] [email_address] [email_address] 25th International Cartographic Conference (Sageo) – 8 july 2011 – Palais de congrès Paris
    2. 2. Objective Find a building in an image SIG 3D 3D GIS Locate the camera that took the image by using the location of the building
    3. 3. Methodology <ul><li> Enhancing the GIS database with information which can describe the building </li></ul><ul><li>unique information </li></ul><ul><li>quantifiable information </li></ul><ul><ul><li>The texture signature </li></ul></ul>Texture signature
    4. 4. How to isolate the building facade in the image? <ul><li>Manual method (long) </li></ul>Sourimant 2009 <ul><li>Automatic method : (3D SIG model/2D image) registration (simple building) </li></ul>
    5. 5. Work plan <ul><li>GIS database enhancement with building texture information </li></ul><ul><li>Facade recognition </li></ul><ul><li>Camera geolocation </li></ul><ul><li>Possible applications </li></ul><ul><li>Limits </li></ul>
    6. 6. Work plan <ul><li>Enhancing GIS databases with building texture information </li></ul><ul><li>Facade recognition </li></ul><ul><li>Camera Geolocation </li></ul><ul><li>Possible Applications </li></ul><ul><li>Limits </li></ul>
    7. 7. Enhancing GIS databases with building texture information Texture analyses (SIFT) Finding the interest points with their local descriptor
    8. 8. Enhancing GIS databases with building texture information Finding the (x,y,z) of the interest points Homography constraints 3D GIS model
    9. 9. Enhancing GIS databases with building texture information The texture descriptor : list of interest points with their local descriptor and their 3D position
    10. 10. Work plan <ul><li>Enhancing GIS databases with building texture information </li></ul><ul><li>Facade recognition </li></ul><ul><li>Camera Geolocation </li></ul><ul><li>Possible Applications </li></ul><ul><li>Limits </li></ul>
    11. 11. Facade recognition False matching because of the locality of the descriptor
    12. 12. Facade recognition <ul><li>Eliminate the false matching using the homography constraints </li></ul><ul><li>Select the best matching score between the current image and the stored descriptor in the databases </li></ul>
    13. 13. Facade recognition The facade in the 3D GIS
    14. 14. Work plan <ul><li>Enhancing the GIS database with building texture information </li></ul><ul><li>Facade recognition </li></ul><ul><li>Camera geolocation </li></ul><ul><li>Possible applications </li></ul><ul><li>Limits </li></ul>
    15. 15. Camera Geolocation Association of the interest points with the 3D position of the matched points in the GIS databases
    16. 16. Camera geolocation 4 points non-collinear (Yang & al. 2009) Real position Measured position
    17. 17. Camera Geolocation <ul><li>Error for distance (20-100)m and angle (0-30°) camera direction and facade normal : </li></ul><ul><ul><li>Position : 1 - 3 m </li></ul></ul><ul><ul><li>Orientation : 5 - 10° </li></ul></ul>
    18. 18. <ul><li>Error because the texture description is not an affine function </li></ul>4 points non-coplanaires SOFTPOSIT (David et al. 2004) Camera Geolocation Real position Measured position
    19. 19. Work plan <ul><li>Enhancing the GIS database with building texture information </li></ul><ul><li>Facade recognition </li></ul><ul><li>Camera geolocation </li></ul><ul><li>Possible applications </li></ul><ul><li>Limits </li></ul>
    20. 20. Possible Applications <ul><li>Management of photos taken in urban areas </li></ul>Link GIS
    21. 21. Possible Applications <ul><li>Navigation systems support in an urban environment </li></ul><ul><ul><li>Satellites visibility </li></ul></ul><ul><ul><li>Multipath </li></ul></ul>
    22. 22. Possible applications <ul><li>Initial phase in the (2D/3D) registration </li></ul>Sourimant 2009
    23. 23. Work plan <ul><li>Enhancing the GIS database with building texture information </li></ul><ul><li>Facade recognition </li></ul><ul><li>Camera geolocation </li></ul><ul><li>Possible applications </li></ul><ul><li>Limits </li></ul>
    24. 24. Limits <ul><li>Angle between camera direction and facade normal has to be less than 30° </li></ul>
    25. 25. Limits <ul><li>Distance between camera and facade has to be less than 200 m </li></ul>
    26. 26. Limits <ul><li>Identical facades : </li></ul>
    27. 27. Limits <ul><li>Glass facade which reflects the sky and other buildings </li></ul>
    28. 28. Thank you For your attention Suleiman wassim [email_address]
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×