What is spatial sql

WHAT IS SPATIAL SQL?Peter “Shawty” Shaw

WHO AM I• I‟m the UK & Europe group manager for the Linked.NET Users group (Lidnug) a global .NET user group based on the Linked-In platform.• I‟ve been doing I.T & Computing in one form or another since the early 1980‟s and I pride myself on being one of the UK‟s original back bedroom developers.• I now own & run Digital Solutions UK, offering consultancy & other I.T related services in the North East of England.• I‟m also Chief Technical Person for a GPS/GIS fleet management solutions provider based in the North East of England called Enablecom.

HOW TO CONTACT ME....• Twitter : @shawty_ds• Skype : shawty_ds• Emails peter.shaw@lidnug.org peter.shaw@digital-solutions.me.uk• Lidnug http://www.linkedin.com/groups/LinkedNET- users-group-LIDNUG-43315

A QUICK SPONSORS PLUG Syncfusion are generously sponsoring this session today with the „Succinctly Series‟ of free .NET developer E-books. In the next few weeks my book „GIS Succinctly‟ (GIS for the .NET developer will be available along with all the other great titles.You can download the E-Books free from :

INTRODUCTIONAka What will we be covering in this session.

WHAT WILL WE BE COVERING• The Definition of Spatial SQL. (What makes it different from normal SQL)• Why we have a need for Spatial SQL.• What problem is Spatial SQL designed to solve?• OGC Standards. (The rulebook of everything GIS)• Spatial Specifics Part 1 – SRID‟s & Co-Ordinate systems.

WHAT WILL WE BE COVERING• Spatial Specifics Part 2 – Geometries & Attributes• Software choices.• Spatial SQL Examples.

DEFINING SPATIAL SQL

WHAT EXACTLY DOESSPATIAL SQL LOOK LIKE?• Just the same as normal SQL:

SO IT MUST BE MORECOMPLICATED THEN?• If it‟s not so different to regular SQL that is....It CAN be.... But no more so than some regular SQL I‟ve seen:SELECT roadType,approachSpeed,recordID,Distance(Transform(GeomFromText(POINT(-1.5924443 54.7865551),4326),27700),PointN(Transform(geometry,27700),NumPoints(geometry)) ) AS DistanceToEnd,Distance(Transform(GeomFromText(POINT(-1.5924443 54.7865551),4326),27700),PointN(Transform(geometry,27700),1) ) AS DistanceFromStart,Distance(Transform(GeomFromText(POINT(-1.5924443 54.7865551),4326),27700),Transform(geometry,27700)) AS Distance FROM roads WHERE Distance(GeomFromText(POINT(-1.5924443 54.7865551),4326),geometry) < 0.0002 ORDER BY Distance(GeomFromText(POINT(-1.5924443 54.7865551),4326),geometry) LIMIT 1

SO IT MUST HANDLE SOMETHINGDATABASES NORMALY DONT?• YES, that‟s to $1000 dollar answer.• We have databases with specific spatial functionality, because the extra data that needs to be processed, typically needs to be processed in a highly specialised way.• In order to understand that, we need to answer one more question.......

WHAT DOES SPATIAL DATALOOK LIKE?• In it‟s raw form spatial data is generally stored in a BLOB• For spatial data to be useful it also needs attributes.• Thus in a standard non spatial database a spatial data record is no more than a Binary Object with accompanying normal database columns containing supporting data.

WHAT DOES SPATIAL DATALOOK LIKE?

BUT THERE‟S NOTHING SPECIALABOUT BLOBS IS THERE?• In the case of spatial yes, very much so.• If you where able to see on the previous slide, the blobs shown where all actually quite uniform.• The data is used to hold things like the geometry type, the spatial reference ID and the actual definition of the points that make the geometry up.

SO CAN‟T I JUST STORE ALL THATINFO IN REGULAR COLUMNS?• Yes, if you want to there‟s nothing stopping you.• For example, there‟s nothing stopping you keeping 2 decimal columns labelled X & Y and filling them with the Longitude & Latitude values you obtain from your users logging onto your app with a mobile phone.• In reality though, you wouldn‟t want to.......

THERE‟S ONE MORE PART TOTHE GIS STORY.....• GIS Databases have highly optimized functions added that understand the different specialist types of BLOB.• These functions understand how to use GIS specific indexes (such as GIST indexes) to speed up location searches.• They also understand how to store multiple arrays of many co-ordinates in ONE binary field and still perform well.• And they know how to output that binary data into many different formats we can read.

THERE‟S ONE MORE PART TOTHE GIS STORY.....• Putting your data into normal non GIS columns means that you cannot use ANY of these specialist functions to perform searches, transformations, composition and many other tasks that GIS systems do routinely.• It means that in the long run, you‟ll most likely end up writing your own code, inside your own app to do many of the tasks that you can do in simple SQL statements.

WHY DO WE HAVE A NEEDFOR SPATIAL SQL?

TODAY‟S MODERN WORLDRELIES ON MOBILE• More and more we are using devices that we carry with us.• Most devices these days are fitted with some kind of location awareness.• Most vehicles have built in sat-navs which can provide location awareness and web browsing capabilities.• Social media sites such as Twitter & Facebook can do clever things if they know a persons location.

ALL OF THESE SOURCESPRODUCE TERABYTES OFSPATIAL DATA.• That‟s FAR too much specialist data to be handled in an ordinary way using standard SQL database techniques.• Imagine for example doing a lookup for point‟s of interest in a given area from a mobile app, in the centre of London?• Or how about a company such as the AA wanting to keep track of where every one of their vehicles deployed in the UK are?

WHAT PROBLEM IS SPATIALSQL DESIGNED TO SOLVE?

PUT SIMPLY....• Spatial SQL is designed to make easier the handling of masses of specialist formatted data, that‟s stored using it‟s own binary format.• It‟s designed to allow us as humans to NOT need know anything about the maths behind it all (Who remembers trigonometry classes from school?)• It‟s designed to output this specialist data, no matter what it‟s input format as something we can read and understand easily.

PUT SIMPLY....• Most importantly however......• It‟s designed to do ALL OF THE ABOVE, very, very fast.• After all. What‟s the use of having this data if your web app for tourists in London takes over a minute to find points of interest?

THERE‟S ALSO THE TRICKYSUBJECT OF ACTUAL LOCATION• As we‟ll see in just a moment, your location on the earth isn‟t always what you might think it is.• The maths to make sure this location is as precise as it can be would make the average developers hair go white instantly.

OGC STANDARDSThe rulebook of everything GIS

GROAN!! NOT MORESTANDARDS COMPLIANCE...• Unfortunately YES, but... NOT for you as a GIS developer.• The Open Geospatial Consortium (OGC) are responsible for making sure that anyone who makes any kind of Geospatial software, make it so that pretty much everything can make use of it.• This means that any software designed to work with a GIS database for example can do so without a problem.• Which leads me onto...........

A SPOT OF SQL-SERVERBASHING.....• My favourite phrase for this part of my talk is:“The Most Compatible, Incompatible Spatial SQL implementation, I’ve ever seen”

WHY IS SQL SERVER SO BADFOR GIS?The Good points:• It supports 75% of the over 300 functions the OGC recommend.• It supports both geometry and geography data types.• It works seamlessly with the tools we as .NET developers are used too.

WHY IS SQL SERVER SO BADFOR GIS?The Bad points:• It‟s implementation is CLR based and therefore an object model that doesn‟t scale well and cannot do normal SQL set based operations.• It understands SRIDs in it‟s own way but doesn‟t create standard meta data about spatial co-ordinate systems preventing use with standard GIS software.

WHY IS SQL SERVER SO BADFOR GIS?The Bad points:• It generally needs specialist adapter technologies (such as ODBC) to translate calls for standard Spatial SQL statements (Due to it‟s CLR nature)• It does not understand a lot of the intricies involved in Spatial SQL, it‟s designed for high speed searching of flat geometry via the Entity Framework classes supplied in EF4.5

WHY IS SQL SERVER SO BADFOR GIS?The Bad points:• Specialist tools are needed to load data into the system.• I‟ll stop here  because there are many more.

BUT AS .NET DEV‟S WERESTUCK WITH SQL-SERVER?• If you absolutley have to use SQL-Server, there are a number of things you can do to become more standards based.• You can use toolkits such as “Dot Spatial”, “Sharp Map” and the “.NET Topology Suite”• All of these are available via NuGet and / or Codeplex.

BUT AS .NET DEV‟S WERESTUCK WITH SQL-SERVER?• However......• I strongly recommend that if possible you use a database made for the job such as “PostgreSQL” and it‟s companion “PostGIS”• Both are free and you can get an all in one installer from “Enterprise DB” at: http://www.enterprisedb.com/products- services-training/pgdownload

BUT AS .NET DEV‟S WERESTUCK WITH SQL-SERVER?• If you can‟t install and / or use a completely different server.• You can install “System.Data.SQLite” from NuGet, to install the SQLite single file database.• Then you can download the “Spatialite” extensions for SQLite from http://www.gaia-gis.it/gaia-sins/And with a simple SQL command load it in to the DB to give you local spatial facilities.

ONCE YOUR OGCCOMPLIANT LIFE IS GOOD• You can data load using standard tools, such as the excellent free GIS package “Quantum GIS”• Everything else I show you here, will just work.• Index, Geometry, Functions and Scripts are 99.5% compatible across all platforms.• There is a heap of documentation, and websites out there (Such as BostonGIS.com and gis.stackexchange.com) that will help you go further.

SPATIAL SPECIFICS PART 1SRID‟s & Co-Ordinate systems

WHAT‟S IN A NUMBER?• When it comes to spatial SQL numbers are everything• There is one overall number you MUST know if you want to do anything useful, that number is the SRID• The 2 most common SRID‟s you‟ll use in the UK are 4326 (Known as WGS84) and 27700 (Known as OSGB36)

WHY DO WE HAVE DIFFERENTCO-ORDINATE SYSTEMS?• In a word....• Because the Earth is round!!!• Translating from a spherical co-ordinate system to a flat planar model needed for a map is actually a very tricky thing to do.• Local co-ordinate systems where invented to allow more accuracy over a smaller area when used on a flat map than if using a global system.

HOW MANY CO-ORDINATESYSTEMS ARE THERE?• At last count, well over 900• Most of the co-ordinate systems are defined by the EPSG (European Petroleum Survey Group), a full list can be found on their site at http://www.epsg.org/• If your dealing with world wide positioning, most of the time you can just use WGS84 and take a hit on the accuracy.• If your working in a specific territory however, then you really do need to use an SRID for that locale.

MS-SQL AND SRID‟S• An OGC compliant database will have a meta table called „spatial_ref_sys‟• This table will hold a list of ALL the SRID‟s known by that DB, allowing you to transform co-ordinates on the fly in standard SQL.• MS-SQL DOES NOT by default create or use this table. Instead, each geometry instance holds a „property‟ in it‟s CLR object letting anything referencing the object know what SRID it was defined in, which is 0 by default.

SPATIAL SPECIFICS PART 2Geometries & Attributes

ATTRIBUTES• Attributes as we‟ve already seen are discrete items of data that mean something only when used with a specific bit of geometry• Attributes are just regular data columns like any other data type in a normal DBMS.• Attributes can also be virtual, EG: ST_Area() used on a polygon object will compute and return the overall area of an object defined in it‟s local co-ordinate system.

GEOMETRIES• There are 3 base geometry types: POINT LINE POLYGON• As well as these three base types there are also „MULTI‟ versions (Think of them as being similar to .NET collections) MULTIPOINT, MULTILINE, MULTIPOLYGON

GEOMETRIES• One other meta data table that is managed by the DB is a table called „geometry_columns‟ which holds details of the geometry type and SRID assigned to it.• This is the most important of the 2 tables as most GIS software will use this to locate spatial data in your database.• MS-SQL does NOT create and / or populate this table automatically, other OGC compliant DB‟s do

GEOMETRIES• An OGC compliant DB will provide access functions to transform these geometry blobs between different formats that we as humans can understand.• OGC Compliant formats are known as WKB, EWKB, WKT and EWKT• Input / output functions will Take these geometry objects and transfer them between GeoJSON, KML, GML and many other regular text formats.

WHAT DOES WKT LOOK LIKE?• Very simple, it‟s the human readable version of the binary blob. POINT(-1.23456 54.123456)• Or as an EWKT SRID=4326;POINT(-1.23456 54.123456)• Lines and polygons are similar but with multiple points LINE(1 2,3 4,5 6) or POLYGON((1 2,3 4,5 6))

ADDING WKT TO YOURDATABASE• You can use the WKT format to add data to the binary object in your DB by using the ST_GeomFromText() function. INSERT INTO table VALUES(ST_GeomFromText(„POINT(1 2)‟,27700)• The first parameter is the text the second is the SRID to use.

ADDING WKT TO MS-SQL• Yes, you guessed it... MS does it differently INSERT INTO mydb VALUES(geometry::STGeomFromText( „POINT(1 2)‟).SRID = 27700;• No surprise there... 

WHAT ABOUT READINGGEOMETRY BACK?• Again very simple. SELECT ST_AsText(myGeometry)• Will give you the WKT format SELECT ST_AsGeoJSON(myGeometry) SELECT ST_AsKml(myGeometry)• Will give you GeoJSON & KML as expected

READING IT BACK IN MS-SQL• And once again.... SELECT geometry::myGeometryColumn.AsText()• To the best of my knowledge only GeoJSON and GML outputs are supported on top of WKT in MS-SQL

SOFTWARE CHOICES

DEVELOPMENT KITS• Dot Spatial• Sharp Map• .NET Topology Suite• Telerik RAD Controls• DEV Express controls

MANAGMENT TOOLS• Jump GIS• Quantum GIS• Arc GIS• Map Info• Open GEO

DATA UP-LOADERS• Geo Kettle• Safe FME• SharpMap MS-SQL Tools• Quantum GIS Plugin (Spit)• GDAL

SPATIAL SQL EXAMPLES

FINDING POINTS IN APOLYGON• Given a source polygon, return a list of geometric points that are contained within it SELECT * FROM mytable WHERE ST_Within(points,ST_GeomFromText( „POLYGON( ..... )‟, 27700) Where 27700 MUST be the same SRID as the „points‟ column in your database

FINDING POINTS IN A POLYGON(THE MS-SQL WAY)• Given a source polygon, return a list of geometric points that are contained within it SELECT * FROM mytable WHERE geometry::STGeomFromText(„POLYGON(...)‟).S RID = 27700.STWithin(geometry::points) Where 27700 MUST be the same SRID as the „points‟ column in your database

FINDING LINES IN A GIVENRADIUS• Given a table of „LINES‟ find which lines are within a 10 meter radius of the supplied point. SELECT * FROM mylinetable WHERE ST_Dwithin(geometrycolumn,ST_GeomFromTex t(„POINT(1 2)‟,27700),10) Where 27700 is the SRID of the point your querying and of the lines in your database so that the „10‟ represents meters.

FINDING LINES IN A GIVENRADIUS (THE MS-SQL WAY)• Given a table of „LINES‟ find which lines are within a 10 meter radius of the supplied point. SELECT * FROM mylinetable WHERE geometry::geometrycolumn.STDwithin(geome try::STGeomFromText(„POINT(1 2)‟).SRID=27700,10) Where 27700 is the SRID of the point your querying and of the lines in your database so that the „10‟ represents meters.

LISTING POLYGONS INORDER OF SIZE• Given a table of „POLYGONS‟ list the top 10 largest , ordering by the biggest at the top. SELECT * FROM mypolytable ORDER BY ST_Area(geometrycolumn) LIMIT 10

LISTING POLYGONS INORDER OF SIZE (MS-SQL)• Given a table of „POLYGONS‟ list the top 10 largest , ordering by the biggest at the top. SELECT TOP 10 * FROM mypolytable ORDER BY geometrycolumn.STArea()

FINDING THE DISTANCEBETWEEN TWO POINTS• Given a table of „POINTS‟ show the distance in fractions of a degree between that point and a provided point, listing the top 10 closest. SELECT ST_Distance(geometrycolumn,ST_GeomFromT ext(„POINT(1 2)‟,4326) FROM mypointtable ORDER BY ST_Distance(geometrycolumn,ST_GeomFromT ext(„POINT(1 2)‟,4326) LIMIT 10

FINDING THE DISTANCE BETWEENTWO POINTS (MS-SQL)• Given a table of „POINTS‟ show the distance in fractions of a degree between that point and a provided point, listing the top 10 closest. SELECT geometry::geometrycolumn.STDistance(ST_Ge omFromText(„POINT(1 2)‟).SRID=4326) FROM mypointtable ORDER BY geometry::geometrycolumn.STDistance(ST_Ge omFromText(„POINT(1 2)‟).SRID=4326)

SUMMARY

WHAT DID WE LEARN• Spatial SQL is noting special.• Spatial SQL is pretty much like normal SQL with a lot of specialist functions and datatypes for handling specialist data.• Spatial SQL is governed by a worldwide standard to ensure that most software that is used to manage it all works in harmony.• Spatial Data is everywhere around us weather we realise it or not.

WHAT DID WE LEARN• Spatial SQL cam produce huge volumes of specialist data that is difficult to manage.• SRID‟s are extremely important if you want to represent real world locations.• If you mix your SRID‟s your heading for trouble.• MS-SQL spatial may be useful and interesting, but of your serious about handling spatial data MS-SQL should not be your first choice.

THANK YOUFor watching....

What is spatial sql

by shawty_ds on Oct 14, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

What is spatial sql Presentation Transcript