Introducing PostGIS
What is Geospatial
• Most common use
– Narrow a search within a distance from a point of
origin
– Sort by that distance
Distance Formulas
• Flat-surface formulae
– Spherical Earth projected to a plane
– Ellipsoidal Earth projected to a plane
– Polar coordinate flat-Earth formula
• Spherical-surface formulae
– Haversine
– Tunnel distance
• Ellipsoidal-surface formulae
– Lambert’s formula for long lines
– Bowring’s method for short lines
Here’s one example
What you usually care about
• Given 2 latitude and longitude points
– Calculate the distance so we can find what’s close
• This is the common capability of databases with
“geospatial” support
A few options…
• Do the math in a query
• Here’s what it looks like in SQL
– (ACOS(least(1,COS(0.607198022186895)*COS(-
1.4410239410591847)*COS(RADIANS(latitude))*COS(RAD
IANS(longitude)) + COS(0.607198022186895)*SIN(-
1.4410239410591847)*COS(RADIANS(latitude))*SIN(RAD
IANS(longitude)) +
SIN(0.607198022186895)*SIN(RADIANS(latitude))))*39
63.19)) <= 25)
– Calculating within 25 miles of a point of origin based on an origin point
– You can do that in any database (it’s just math)
– Very intense query that slows as your dataset grows
– This…is not using an index
Speed that up
• Distance query a subset instead
• Use the lat/lng to create query a box
around the boundary
• Queried with a numerical index
• Then check the distance from center
for the subset
• NOTE: Doing this as part of another
query is a lot easier with multi-index
queries
And here’s what THAT looks like
WHERE (
latitude > 34.42845936786603
AND latitude < 35.15130863213399
AND longitude > -83.00467808012291
AND longitude < -82.12450191987708
)
AND
( (ACOS(least(1,COS(0.607198022186895)*COS(-
1.4410239410591847)*COS(RADIANS(latitude))*COS(RADIANS(longitude))+
COS(0.607198022186895)*SIN(-
1.4410239410591847)*COS(RADIANS(latitude))*SIN(RADIANS(longitude))+
SIN(0.607198022186895)*SIN(RADIANS(latitude))))*3963.19)
) <= 25)
Simple, am I right?
Also, you need to drop that distance formula into the ORDER BY clause too.
There’s a Gem for that!
Geokit
https://github.com/geokit/geokit
• Distance calculations between 2 points
– Multiple formulas and units of measure
• Multiple providers for Geocoding
different data
– Addresses
• Yahoo
• Geocoder.us/.ca
• Geonames
• Bing
• Yandex
• MapQuest
• Geocode.io
• Mapbox
• Google
• FCC
• Open Street Map
– IP Address
• hostip.info
• Geoplugin.net
• RIPE
• MaxMind (HIGHLY RECOMMEND)
• freegeoip.net
Geokit Rails (any database)
https://github.com/geokit/geokit-rails
• Premium Rails Integration
• ActiveRecord distance finders
• IP based location lookup
• Cookie based user location tracking
• Scopes: within, beyond, in_range, in_bounds,
closest, farthest, by_distance
• Generate the SQL
• Auto-Geocoding
• Mixin
class Location < ActiveRecord::Base
acts_as_mappable :default_units => :miles,
:default_formula => :sphere,
:distance_field_name => :distance,
:lat_column_name => :lat,
:lng_column_name => :lng
end
Actually, there’s several…
RGeo
• Adapters for
– MySQL
– SQLite
– PostGIS
• Process GeoJSON
• Read shapefiles
• Uses C++ extensions for
processing
Geocoder
• Object Geocoding
– IP
– Address
• Reverse Geocoding
– Address from lat/lng or IP
• Center of Multiple Locations
• Geographic Queries
– Near / nearby
– Distance
– Direction
• Easier to use
• Seems to be some dispute about
accuracy of data
What about PostgreSQL?
• Provide database functions
to handle this calculations
– Distance in straight line or
great circle
– Convert lat/lng to point
– Get lat/lng from a point
– Calculate containment
within a cube
• Point datatype
– Operator to get distance
between
Earth Distance extension
http://www.postgresql.org/docs/9.2/s
tatic/earthdistance.html
Cube based
Point based
SO WHAT DOES POSTGIS DO THEN?
Funny you should ask…
EVERYTHING
Seriously…it does pretty much
everything
Import and Export Data
• Import CSV
• Import/export with GDAL (Geospatial Abstraction
Library)
– Shapefiles
– OGR
– Geospatial vector data files
– OpenStreetMap data (openstreetmap.org)
– Raster datasets
• Import OGR files
• ETL migration tools
– GeoKettle
GIS Desktop Software
Open Source
• OpenJump
• QuantumGIS
• uDig
• GvSig
• OrbisGIS
• PostGIS Viewer
• pgAdmin plugin
Commercial
• CadCorp SIS
• Manifold.net
• MapInfo Professional
• AutoCAD
• ESRI ArcGIS
Mapping Servers Integration
• Mapserver
• GeoServer
• Deegree
• QGIS Server
• MapGuide
Data: OpenStreetMap
Data: US Census
Data: WorldClim
US Geological Survey
Natural Earth
MaxMind GeoIP
Web Services / API
Constantly Updated Databases
Free and Paid Versions
Check your IP to try it out
https://www.maxmind.com/en/locate
_my_ip
Working with Data
• Datatypes
– Geography
• Ellipsoidal spatial data
– Geometry
• Planar spatial data
• Indexes
• Topology
• Spatial Joins
• 3D
– Mapping (building, etc)
– Image generation
• pgRoute
– Generate driving route
• Generate raster images from SQL
queries
• Spatial Relationship Functions
– ST_Contains
– ST_Covers
– ST_Crosses
– ST_DWithin
– ST_Intersects
– ST_Distance
• Clustered Queries for Huge
Datasets
Fun data tricks
• Use a TRIGGER to populate a geographic data
column for transparent indexing
• Create a Geospatial View
• Use table inheritance to centralize
commonality among different data types
– Geographic Data
– Search Data
AR PostGIS Adapter
Migrations
create_table :locations do |t|
t.column :shape1, :geometry
t.geometry :shape2
t.line_string :path, :srid => 3785
t.point :lonlat, :geographic => true
t.point :lonlatheight, :geographic => true, :has_z => true
t.index :lonlat, :spatial => true
end
Datatypes
:geometry -- Any geometric type
:point -- Point data
:line_string -- LineString data
:polygon -- Polygon data
:geometry_collection -- Any collection type
:multi_point -- A collection of Points
:multi_line_string -- A collection of LineStrings
:multi_polygon -- A collection of Polygons
ActiveRecord
Location.where(:lonlat => 'POINT(-122 47)').first
Location.where("ST_Distance(latlon,
'POINT(-122.330779 47.604828)') < 10000")
scope :distance_from, ->(lat, lon, dist) do
where(“ST_Distance(latlon, ‘POINT(? ?)’) < ?, lat, lon, dist)
end
Location.where("ST_Intersects(latlon,
'POLYGON((
-122.19 47.68,
-122.2 47.675,
-122.19 47.67,
-122.19 47.68))')")
https://github.com/rgeo/active
record-postgis-adapter
Features
• Uses Rgeo gem
• Spatial Migrations
• Spatial Datatype
• Spatial Queries
• Create / Update PostGIS DB
• Support central schema
Assignment 2
• Refine and improve your application
• Add authentication
• Add authors (or some type of user) to created data
• Display author info on your data
• Implement simple_form somewhere
• Add a new validation rule to a model
• Add a page to show related data for an author
• Use slim (or haml) to create one of your views
• Add a BASIC geographic capability (optional)
– Use your own discretion for how deep you go
– Suggestions / Ideas
• Geocode user IP addresses to get city/state/zip info
• Add address fields to your data type to make it geographically relevant
• Use a distance filter in search

Day 6 - PostGIS

  • 1.
  • 2.
    What is Geospatial •Most common use – Narrow a search within a distance from a point of origin – Sort by that distance
  • 3.
    Distance Formulas • Flat-surfaceformulae – Spherical Earth projected to a plane – Ellipsoidal Earth projected to a plane – Polar coordinate flat-Earth formula • Spherical-surface formulae – Haversine – Tunnel distance • Ellipsoidal-surface formulae – Lambert’s formula for long lines – Bowring’s method for short lines
  • 4.
  • 5.
    What you usuallycare about • Given 2 latitude and longitude points – Calculate the distance so we can find what’s close • This is the common capability of databases with “geospatial” support
  • 6.
    A few options… •Do the math in a query • Here’s what it looks like in SQL – (ACOS(least(1,COS(0.607198022186895)*COS(- 1.4410239410591847)*COS(RADIANS(latitude))*COS(RAD IANS(longitude)) + COS(0.607198022186895)*SIN(- 1.4410239410591847)*COS(RADIANS(latitude))*SIN(RAD IANS(longitude)) + SIN(0.607198022186895)*SIN(RADIANS(latitude))))*39 63.19)) <= 25) – Calculating within 25 miles of a point of origin based on an origin point – You can do that in any database (it’s just math) – Very intense query that slows as your dataset grows – This…is not using an index
  • 7.
    Speed that up •Distance query a subset instead • Use the lat/lng to create query a box around the boundary • Queried with a numerical index • Then check the distance from center for the subset • NOTE: Doing this as part of another query is a lot easier with multi-index queries
  • 8.
    And here’s whatTHAT looks like WHERE ( latitude > 34.42845936786603 AND latitude < 35.15130863213399 AND longitude > -83.00467808012291 AND longitude < -82.12450191987708 ) AND ( (ACOS(least(1,COS(0.607198022186895)*COS(- 1.4410239410591847)*COS(RADIANS(latitude))*COS(RADIANS(longitude))+ COS(0.607198022186895)*SIN(- 1.4410239410591847)*COS(RADIANS(latitude))*SIN(RADIANS(longitude))+ SIN(0.607198022186895)*SIN(RADIANS(latitude))))*3963.19) ) <= 25) Simple, am I right? Also, you need to drop that distance formula into the ORDER BY clause too.
  • 9.
    There’s a Gemfor that! Geokit https://github.com/geokit/geokit • Distance calculations between 2 points – Multiple formulas and units of measure • Multiple providers for Geocoding different data – Addresses • Yahoo • Geocoder.us/.ca • Geonames • Bing • Yandex • MapQuest • Geocode.io • Mapbox • Google • FCC • Open Street Map – IP Address • hostip.info • Geoplugin.net • RIPE • MaxMind (HIGHLY RECOMMEND) • freegeoip.net Geokit Rails (any database) https://github.com/geokit/geokit-rails • Premium Rails Integration • ActiveRecord distance finders • IP based location lookup • Cookie based user location tracking • Scopes: within, beyond, in_range, in_bounds, closest, farthest, by_distance • Generate the SQL • Auto-Geocoding • Mixin class Location < ActiveRecord::Base acts_as_mappable :default_units => :miles, :default_formula => :sphere, :distance_field_name => :distance, :lat_column_name => :lat, :lng_column_name => :lng end
  • 10.
    Actually, there’s several… RGeo •Adapters for – MySQL – SQLite – PostGIS • Process GeoJSON • Read shapefiles • Uses C++ extensions for processing Geocoder • Object Geocoding – IP – Address • Reverse Geocoding – Address from lat/lng or IP • Center of Multiple Locations • Geographic Queries – Near / nearby – Distance – Direction • Easier to use • Seems to be some dispute about accuracy of data
  • 11.
    What about PostgreSQL? •Provide database functions to handle this calculations – Distance in straight line or great circle – Convert lat/lng to point – Get lat/lng from a point – Calculate containment within a cube • Point datatype – Operator to get distance between Earth Distance extension http://www.postgresql.org/docs/9.2/s tatic/earthdistance.html Cube based Point based
  • 12.
    SO WHAT DOESPOSTGIS DO THEN? Funny you should ask…
  • 13.
  • 14.
    Import and ExportData • Import CSV • Import/export with GDAL (Geospatial Abstraction Library) – Shapefiles – OGR – Geospatial vector data files – OpenStreetMap data (openstreetmap.org) – Raster datasets • Import OGR files • ETL migration tools – GeoKettle
  • 15.
    GIS Desktop Software OpenSource • OpenJump • QuantumGIS • uDig • GvSig • OrbisGIS • PostGIS Viewer • pgAdmin plugin Commercial • CadCorp SIS • Manifold.net • MapInfo Professional • AutoCAD • ESRI ArcGIS
  • 16.
    Mapping Servers Integration •Mapserver • GeoServer • Deegree • QGIS Server • MapGuide
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
    MaxMind GeoIP Web Services/ API Constantly Updated Databases Free and Paid Versions Check your IP to try it out https://www.maxmind.com/en/locate _my_ip
  • 23.
    Working with Data •Datatypes – Geography • Ellipsoidal spatial data – Geometry • Planar spatial data • Indexes • Topology • Spatial Joins • 3D – Mapping (building, etc) – Image generation • pgRoute – Generate driving route • Generate raster images from SQL queries • Spatial Relationship Functions – ST_Contains – ST_Covers – ST_Crosses – ST_DWithin – ST_Intersects – ST_Distance • Clustered Queries for Huge Datasets
  • 24.
    Fun data tricks •Use a TRIGGER to populate a geographic data column for transparent indexing • Create a Geospatial View • Use table inheritance to centralize commonality among different data types – Geographic Data – Search Data
  • 25.
    AR PostGIS Adapter Migrations create_table:locations do |t| t.column :shape1, :geometry t.geometry :shape2 t.line_string :path, :srid => 3785 t.point :lonlat, :geographic => true t.point :lonlatheight, :geographic => true, :has_z => true t.index :lonlat, :spatial => true end Datatypes :geometry -- Any geometric type :point -- Point data :line_string -- LineString data :polygon -- Polygon data :geometry_collection -- Any collection type :multi_point -- A collection of Points :multi_line_string -- A collection of LineStrings :multi_polygon -- A collection of Polygons ActiveRecord Location.where(:lonlat => 'POINT(-122 47)').first Location.where("ST_Distance(latlon, 'POINT(-122.330779 47.604828)') < 10000") scope :distance_from, ->(lat, lon, dist) do where(“ST_Distance(latlon, ‘POINT(? ?)’) < ?, lat, lon, dist) end Location.where("ST_Intersects(latlon, 'POLYGON(( -122.19 47.68, -122.2 47.675, -122.19 47.67, -122.19 47.68))')") https://github.com/rgeo/active record-postgis-adapter Features • Uses Rgeo gem • Spatial Migrations • Spatial Datatype • Spatial Queries • Create / Update PostGIS DB • Support central schema
  • 26.
    Assignment 2 • Refineand improve your application • Add authentication • Add authors (or some type of user) to created data • Display author info on your data • Implement simple_form somewhere • Add a new validation rule to a model • Add a page to show related data for an author • Use slim (or haml) to create one of your views • Add a BASIC geographic capability (optional) – Use your own discretion for how deep you go – Suggestions / Ideas • Geocode user IP addresses to get city/state/zip info • Add address fields to your data type to make it geographically relevant • Use a distance filter in search