PyCon PH 2014 - GeoComputation

1,454 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,454
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
41
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

PyCon PH 2014 - GeoComputation

  1. 1. GEOCOMPUTATION Engr. Ranel O. Padon PyCon PH 2014 | ranel.padon@gmail.com http://www.gctours.net/product_images/uploaded_images/grand-canyon-hd720.jpg
  2. 2. ABOUT ME  Full-Time Drupal Developer (CNN Travel)  Lecturer, UP DGE (Java/Python OOP Undergrad Courses)  Lecturer, UP NEC (Web GIS Training Course)  BS Geodetic Engineering in UP  MS Computer Science in UP (25/30 units)  Involved in Java, Python, and Drupal projects.
  3. 3. ABOUT MY TOPIC The role of Python in implementing a rapid and mass valuation of lots along the Pasig River tributaries. This is the story of what we have done.
  4. 4. TOPIC FLOW I • PRTSAS BACKGROUND II • VALUATION COMPONENT III • AHP MODELING IV • RECOMMENDATIONS
  5. 5. OF FLOOD AND MEN http://www.reynaelena.com/wp-content/uploads/2009/09/ondoy-aftermath-by-wenzzo-pancho.jpg http://1.bp.blogspot.com/-sdUQ_XBc5o8/TnfOuNASgjI/AAAAAAAAAug/u-OQ1Cv5oEg/s1600/Ondoymissionhospital.jpg http://filsg.com/download/ondoy16.jpg
  6. 6. GIL SCOTT-HERON Man is a complex being: he makes deserts bloom - and lakes die. http://i.dailymail.co.uk/i/pix/2011/05/28/article-0-0C4E40E200000578-673_468x301.jpg http://d2tq98mqfjyz2l.cloudfront.net/image_cache/1254443971159430.jpeg
  7. 7. PASIG RIVER | BEFORE http://ourss14blog.blogspot.com/2011/10/article-xii-national-economy-and.html
  8. 8. PASIG RIVER | AFTER http://ourss14blog.blogspot.com/2011/10/article-xii-national-economy-and.html
  9. 9. BACKGROUND | PRTSAS PRTSAS = Pasig River Tributaries Survey and Assessment Study PRTSAS = PRRC + UP TCAGP Aims to gather baseline information on the physical characteristics of major and minor tributaries of the Pasig River. The gathered information will be used to properly manage the river and correctly steer its rehabilitation.
  10. 10. BACKGROUND | PRTSAS | PRRC “To transform Pasig River and its environs into a showcase of a new quality of urban life.” http://www.prrc.gov.ph/
  11. 11. BACKGROUND | PRTSAS | PRRC Restore the Pasig River to its historically pristine condition by applying bio-eco engineering and attain a sustainable socio-economic development. Relocation of formal and informal settlers. Regulate the 3-m easement.
  12. 12. BACKGROUND | PRTSAS | UP TCAGP http://dge.upd.edu.ph/dge/about/about-tcagp/
  13. 13. BACKGROUND | PRTSAS | UP TCAGP Research and extension arm of UP DGE. Large-Scale Projects:  DREAM (DOST NOAH)  PRTSAS  PRS 92 R&D and Implementation Support
  14. 14. BACKGROUND | PRTSAS | COMP. PRTSAS has 5 major components:  Parcel/As-Built Survey  Hydrographic Component  Water Quality/Environmental Impact  Easement and Adjoining Lots Valuation  Web GIS
  15. 15. BACKGROUND | PRTSAS | COVERAGE
  16. 16. BACKGROUND | PRTSAS | COVERAGE
  17. 17. VALUATION | DUTIES  To perform individual valuation work of the PRRC proposed relocation sites.  To perform a rapid appraisal of the 3-meter easements and adjoining lots for all tributary locations.  To develop and perform an automated GIS-assisted valuation of the lots adjoining all tributaries.
  18. 18. VALUATION | THE TEAM
  19. 19. VALUATION | OVERVIEW Develop a GIS-assisted valuation model and perform automated valuation of lots adjoining the tributaries.
  20. 20. VALUATION | EASEMENT CONDITION Fully-Developed Partially-Developed Undeveloped/ Depreciated
  21. 21. VALUATION | MARKET VALUE  determined by the highest price a property can command if put up for sale in an open market  determinations are made from market evidence or transactions and found on published market listings or information from market participants.
  22. 22. VALUATION | MARKET VALUE  The ultimate question is: how do you value a land?  And how do you value lands with huge coverage rapidly? http://blog.melvinpereira.com/wp-content/uploads/2011/04/man-thinking.jpg http://e.peruthisweek.e3.pe//ima/0/0/0/1/5/15908/624x468.png
  23. 23. GENERAL PROCESS FLOW AHP Model Formulation Geospatial Data Buildup Market Value Geoprocessing ArcPy http://ithelp.port.ac.uk/images/SPSS-logo-32F23C8B51-seeklogo.png http://www.lic.wisc.edu/training/Images/arcgis.gif http://www.logilab.org/ Market Value Map
  24. 24. AHP Analytic Hierarchy Process is a decision-making method based on mathematics and psychology developed by Prof. Thomas L. Saaty in the 1970s. The input can be obtained from actual measurements such as price, weight, etc. and from subjective opinion such as satisfaction feelings and preferences. http://www.nae.edu/File.aspx?id=41107
  25. 25. AHP  used in scientific and business contexts  useful in situation with scarce, but high-quality or highimportance data  80/20 Principle: essential information (80%) could be expressed by just a small but important set of data (20%)  unlike the case of face recognition problem which requires voluminous data to be stable http://www.nae.edu/File.aspx?id=41107
  26. 26. AHP | CHOOSING A LEADER http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process
  27. 27. AHP | CHOOSING A LEADER BRAIN http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process
  28. 28. AHP | CHOOSING A PARTNER 1. Parameters II. Weights of Parameters
  29. 29. AHP | MURPHY’S LAW OF LOVE BRAIN B· B· A = k BEAUTY AVAILABILITY
  30. 30. AHP | I. PARAMETERS Intelligence Values Humor Beauty Wealth Religion Choosing a partner Health Interests Sports Zodiac Sign and so on
  31. 31. AHP | I. PARAMETERS Use statistical software to evaluate if some factors could be eliminated, values to watch out: 1.) Kaiser-Meyer-Olkin (KMO) Coefficient – tests whether the partial correlations among variables are small 2.) Barlett’s Test for Sphericity (BTS) – tests whether the correlation matrix is an identity matrix Choosing a partner
  32. 32. AHP | I. PARAMETERS Why Dimensionality Reduction?  To simplify data structures  Conserve computing and/or storage resources Examples: Face Recognition, MP3 and JPEG file formats, Douglas-Peucker Algorithm
  33. 33. AHP | I. PARAMETERS Dimensionality Reduction | EigenFaces  Principal vectors used in the problem of human face recognition http://cognitrn.psych.indiana.edu/nsfgrant/FaceMachine/faceMachine.html
  34. 34. AHP | I. PARAMETERS Dimensionality Reduction/Factor Analysis  Is the strength of the relationships among variables large enough?  Is it a good idea to proceed a factor analysis for the data? Choosing a partner
  35. 35. AHP | II. WEIGHTS OF PARAMETERS Possible major components after Factor Extraction 1. Humor 2. Beauty 3. Intelligence Choosing a partner
  36. 36. AHP | II. WEIGHTS OF PARAMETERS Sample Preference Matrix (3 Parameters) Criteria More Important Intensity A 5 A Humor B Beauty Humor Intelligence A 7 Beauty Intelligence A 3 Choosing a partner
  37. 37. AHP | II. WEIGHTS OF PARAMETERS Choosing a partner
  38. 38. AHP | II. WEIGHTS OF PARAMETERS As you might observed, we need to reduce the number of parameters so that the respondents/evaluators will just have to evaluate the smallest preference matrix possible. Choosing a partner
  39. 39. AHP | FINAL PARAMETERS’ WEIGTHS Apply the AHP algorithm to compute the relative weights, possible result: 0.60 Humor 0.25 Beauty 0.15 Intelligence Choosing a partner
  40. 40. AHP | FINAL PARAMETERS’ WEIGTHS Optimum Partner (among alternatives/suitors) = 0.60 Humor + 0.25 Beauty + 0.15 Intelligence Choosing a partner
  41. 41. AHP | VALUING A LAND 1. Parameters II. Weights of Parameters III. Weights of Sub-Categories http://i.domainstatic.com.au/b432bfa9-1e06-4d69-812e-ea14e22d0112/domain/20108120961pio04192711
  42. 42. AHP | I. PARAMETERS Lot Shape Topography Easement Condition Neighborhood Classification Accessibility to Main Roads Corner Influence Land-Use Type Proximity to Commercial Area Proximity to Churches Proximity to Markets Proximity to School Proximity to LGUs Existing Improvements Public Utilities and so on Obtaining the optimal land value
  43. 43. AHP | I. PARAMETERS
  44. 44. AHP | I. PARAMETERS We used SPSS for computing the KMO and BTS Coefficients. 1.) KMO > 0.5 2.) BTS < 0.001 SPSS also provides validation values that could be used when we decide to automate the process in pure Python later. Choosing a partner
  45. 45. AHP | I. PARAMETERS  Factor Analysis (18 raw & unordered variables)
  46. 46. AHP | I. PARAMETERS  Extracted Factors Land-Use Accessibility Lot Size Lot Shape Neighborhood
  47. 47. AHP | II. WEIGHTS OF PARAMETERS Sample Preference Matrix (4 Parameters) Criteria More Important Intensity A 3 A Cost B Safety Cost Cost Safety Safety Style Capacity Style Capacity A A A A 7 3 9 1 Style Capacity B 7 Choosing a car: 4 Params, 6 Comparisons
  48. 48. AHP | II. WEIGHTS OF PARAMETERS Actual Data Obtaining the Optimal Value : 5 Params, 10 Comparisons
  49. 49. AHP | II. WEIGHTS OF PARAMETERS The CSV File
  50. 50. AHP | II. WEIGHTS OF PARAMETERS AHP Algorithms (Ishizaka & Lusti, 2006) 1. The Eigenvalue Approach (Power Method) 2. The Geometric Mean 3. The Mean of Normalized Values
  51. 51. AHP | II. WEIGHTS OF PARAMETERS 3. The Mean of Normalized Values
  52. 52. AHP | II. WEIGHTS OF PARAMETERS
  53. 53. AHP | II. WEIGHTS OF PARAMETERS
  54. 54. AHP | II. WEIGHTS OF PARAMETERS Effective AHP parameters Parameter Weight Land Use 0.372 Location/Accessibility 0.276 Lot Size 0.125 Lot Shape 0.111 Neighborhood Classification 0.116
  55. 55. AHP | II. WEIGHTS OF PARAMETERS Some issues for the computation of our AHP parameters: 1.) Assumes all respondents have consistent preference matrices 2.) Uses the arithmetic mean for computing the effective parameter weights across all the respondents.
  56. 56. AHP | II. WEIGHTS OF PARAMETERS consistency means that if A>B and B>C then A>C, where A, B, and C, refer to the criteria/parameters of the land value. It also means that if A > 2*B and B > 3*C then A > 6*C, as the number of criteria increases, it's more difficult to be consistent
  57. 57. AHP | II. WEIGHTS OF PARAMETERS We have implemented the proposed Saaty's Consistency Measure of the preference matrix of the respondents but we have found it to be too limiting.
  58. 58. AHP | II. WEIGHTS OF PARAMETERS Pelaez and Lamata (2002) proposed a new way of computing the Consistency Index and that is by using the concept of determinants. We implemented their paper using Python and NumPy and we obtained a better filtering for the consistent survey answers.
  59. 59. AHP | II. WEIGHTS OF PARAMETERS
  60. 60. AHP | II. WEIGHTS OF PARAMETERS
  61. 61. AHP | II. WEIGHTS OF PARAMETERS However, [Aragon, et al (2012)], shown that it is better to use the geometric mean than the arithmetic mean of the AHP parameters' weights. We re-implemented the effective parameters' weights using the geometric mean of all weights across all respondents.
  62. 62. AHP | II. WEIGHTS OF PARAMETERS
  63. 63. AHP | II. WEIGHTS OF PARAMETERS
  64. 64. AHP | II. WEIGHTS OF PARAMETERS There are two approaches [Aragon, et al (2012)] for solving the effective parameters: (1) EIW: Effective Individual Weights computes the individual parameters' weights and get their geometric mean (2) WEPM: Weights of the Effective Preference Matrix get the geometric mean of all the preference matrices and compute the parameters' weights.
  65. 65. AHP | II. WEIGHTS OF PARAMETERS We implemented both approaches in combination with the 3 AHP algorithms for comparison and validation.
  66. 66. AHP | II. WEIGHTS OF PARAMETERS Finally, we will use the following result (using the Weights of the Effective Preference Matrix of the Mean of Normalized Values AHP Algorithm)
  67. 67. AHP | III. SUBCATEGORY WEIGHTS AHP allows hierarchies/subcategories Phase III for gathering the sub-categorical weights or adjustment factors
  68. 68. AHP | III. SUBCATEGORY WEIGHTS
  69. 69. AHP | III. SUBCATEGORY WEIGHTS Geometric Mean of all survey data
  70. 70. AHP | FINAL PARAMS AND WEIGHTS (Context is Per Estero) Computed Unit Market Value = Average Market Value * ( Land-Use * (Commercial|Industrial|Residential…) + Accessibility *(Proximity to POIs and Access to Roads) + Lot Area * (Preferred|Not-Preferred) + Lot Shape * (Quadrilateral|NonQuadrilateral) + Neighborhood Classification * (Formal|Informal) )
  71. 71. AHP | FINAL PARAMS AND WEIGHTS (Context is Per Estero) Computed Unit Market Value = Average Market Value * ( 0.4287 * (1.5148l|1.1308|1.1288|1.0080|1.0000) + 0.2809 *(0..1) + 0.1119 * (1.5599|0.3338) + 0.0988 * (1.3831|0.5997) + 0.0797 * (1.4082|0.5696) )
  72. 72. AHP | GIS http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Key_aspects_of_GIS/00v20000000r000000/
  73. 73. AHP | ArcPy http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Working_with_geometry_in_Python/002z0000001s000000/
  74. 74. AHP | ArcPy
  75. 75. AHP | ArcPy http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Working_with_geometry_in_Python/002z0000001s000000/
  76. 76. AHP | ArcPy
  77. 77. AHP | ArcPy
  78. 78. AHP | ArcPy
  79. 79. AHP | ArcPy
  80. 80. AHP | ArcPy
  81. 81. AHP | ArcPy
  82. 82. AHP | ArcPy
  83. 83. AHP | ArcPy
  84. 84. AHP | ArcPy
  85. 85. AHP | ArcPy
  86. 86. AHP | ArcPy
  87. 87. AHP | VALIDATION
  88. 88. AHP | WELCH’S TEST
  89. 89. AHP | MARKET VALUE MAP
  90. 90. AHP | MARKET VALUE MAP
  91. 91. AHP | MARKET VALUE MAP
  92. 92. RECOMMENDATIONS Pure Python Pipeline: SPSS <= RPy2 or Pandas (Python Data Analysis Library) ArcGIS (ArcPy) <= QGIS (PyQGIS)
  93. 93. GENERAL PROCESS FLOW AHP Model Formulation Geospatial Data Buildup Market Value Geoprocessing ArcPy http://ithelp.port.ac.uk/images/SPSS-logo-32F23C8B51-seeklogo.png http://www.lic.wisc.edu/training/Images/arcgis.gif http://www.logilab.org/ Market Value Map
  94. 94. RECOMMENDATIONS AHP Model Formulation Geospatial Data Buildup Market Value Geoprocessing PyQGIS http://pandas.pydata.org/ http://rpy.sourceforge.net/rpy2/doc-dev/html/index.html http://trac.osgeo.org/qgis/chrome/site/qgis-icon.png Market Value Map
  95. 95. RECOMMENDATIONS | BOOKS http://locatepress.com/
  96. 96. RECOMMENDATIONS | MASHUP This comprehensive article demonstrates the tight integration of Python’s data analysis and geospatial libraries:          IPython Pandas Numpy Matplotlib Basemap Shapely Fiona Descartes PySAL
  97. 97. MICHAEL STANIER There are two types of expertise. One is the type you already know – content expertise, immersing yourself deeper and deeper in a subject, practicing for 10,000 hours and all of that. But I think there’s a connection expertise too. That comes from going horizontal rather than vertical. It’s about knowing a little about a lot, and finding wisdom in how things connect in new and different ways. http://www.speakers.ca/wp-content/uploads/2012/12/Michael-Bungay-Stanier_Feb2-760x427.jpg
  98. 98. END NOTE Python could be a valuable tool for expanding your knowledge vertically, as well as horizontally. And, it’s a must have tool for connectionist experts.
  99. 99. http://fc01.deviantart.net/fs25/i/2009/022/1/a/inject_knowledge_question_mark_by_CHIN2OFF.jpg
  100. 100. REFERENCES Aragon,T., et al (2012). Deriving Criteria Weights for Health Decision Making: A Brief Tutorial, http://www.academia.edu. Forman, E. & Selly, M. (2001). Decision By Objectives: How to Convince Others That You Are Right. World Scientific Publishing Co. Pte. Ltd. Singapore. Griffiths, D. (2009). Head First Statistics. O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. USA. Ishizaka, A. & Lusti, M. (2006). How to Derive Priorities in AHP: A Comparative Study. Central European Journal of Operations Research,Vol. 14-4, pp. 387-400. Lamata, M. & Pelaez, J. (2002). A Method for Improving the Consistency of Judgements. International Journal of Uncertainty, Fuzziness, and Knowledge-Based Systems. Vol. 10, No.6, pp. 677-686. World Scientific Publishing Company. Pelaez, J. & Lamata, M. (2002). A New Measure of Consistency for Positive Reciprocal Matrices. Computers and Mathematics with Applications, 46 (8), pp. 1839-1849. Pornasdoro, K. & Redo, R. S. (2011). GIS-Assisted Valuation Using Analytic Hierarchy Process and Goal Programming: Case Study of the UP Diliman Informal Settlement Areas (Undergraduate Thesis). Uysal, M. P. (2010). Analytic Hierarchy Process Approach to Decisions on Instructional Software. 4th International Computer & Instructional Technologies Symposium, Selçuk University, Konya, Turkey, pp. 1035-1040.
  101. 101. Thank You!

×