MySQL 5.7 GIS
•Download as PPTX, PDF•
7 likes•1,430 views
This presentation was given by Pavan Naik in Open Source India (OSI) 2014 even held in Nimans Convention Centre, Bangalore. It talks about GIS features in MySQL 5.7. Presentation covers following topics : 1. Introduction to GIS 2. Common Terms and Concepts 3. What's new in MySQL 5.7 4. A Real World Example 5. What's next for MySQL GIS
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MySQL 5.7 GIS
Similar to MySQL 5.7 GIS (20)
Recently uploaded
Recently uploaded (20)
MySQL 5.7 GIS
- 2. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Mapping with MySQL: GIS in MySQL 5.7 Pavan Naik MySQL Eng Team
- 3. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 3
- 4. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | GIS in MySQL 5.7 : Agenda 1 2 3 4 5 Introduction to GIS Common terms and concepts What’s new in MySQL 5.7 A real world example What’s next for MySQL GIS 4
- 5. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Introduction to GIS 5
- 6. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | What is GIS? • Geographic Information Systems – Features : graphic spatial representations of real-world physical features • Generally a map of some sort – Attributes : non-spatial data describing the features • Name/value pairs used to describe a location and to allow for grouping of data • Data formats – Vector data : points, lines and polygons • Generally what’s used with an RDBMS, such as MySQL – Raster data : grid matrix containing cells • Generally used for aerial and satellite imagery 6
- 7. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | What Would I Use it for? • Location services – Where is something? – How do I get from Point A to Point B? – What are the closest <thing>s to me? – What are the relevant details of each location or Point? 7 * Source: ESRI
- 8. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | How Would I Use It? 8 StoreCollect AnalyzeVisualize • Collect spatial data – Free (OSM) • Store the data – Within MySQL tables • Analyze the data – SQL queries are used to analyze the data to derive meaningful relationships • Visualize the data – Provide maps containing the resulting attributes and relationships
- 9. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Common Terms and Concepts 9
- 10. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Common Terms • Coordinates – x,y,z coordinates in planar space (4D is m or measure) – MySQL currently only supports x,y(2D) coordinates • Projection – Allows a spheroidal surface to be represented in planar format – Necessary for creating “flat” or 2D maps from locations on a spheroid • Coordinate reference system (CRS/SRS/SRID) – Defines where a POINT—represented by a longitude and latitude coordinate pair—is located on the physical earth and defines its relationship to other POINTs 10
- 11. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Data Formats • Vector – Points, lines, and polygons – Position (x,y) is relative in a coordinate system – Generally used by database servers – Includes .OSM, .KML, .GeoJSON, … • Raster – Cells in a grid matrix, tied to an anchor (e.g. the {1,1} cell) – Generally used in aerial, satellite, and other imagery – Includes .jpg, .gif and other pixel based formats 11
- 12. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Data Formats 12
- 13. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Geometry Data Types • POINT • LINESTRING • POLYGON • MULTIPOINT • MULTILINESTRING • MULTIPOLYGON • GEOMETRYCOLLECTION 13
- 14. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | What’s New in MySQL 5.7 14
- 15. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Integrating Boost.Geometry • Replaced custom code – For geometry representations – For geometry comparisons • Provides OGC compliance – With improved performance • Boost.Geometry contains – Field and domain experts – Bustling and robust community • We’re also Boost.Geometry contributors! – Two full-time developers contributing upstream 15
- 16. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Spatial Indexes for InnoDB • R-tree based – Full transactional support – Minimum bounding rectangle • Small and compact – Currently only supports 2D data • We would like to add 3D support in the future 16
- 17. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Additional Features • GeoHash – B-tree indexes on the generated hash values – Quick lookups for exact matches • GeoJSON – Open standard for encoding geometric/geographical features 17 { "type": "Feature", "geometry": { "type": "Point", "coordinates": [125.6, 10.1] }, "properties": { "name": "Dinagat Islands" } } GeoJSON Example
- 18. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | A Real World Example 18
- 19. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Starting Point • My apartment in Brooklyn, NY – 33 Withers Street Brooklyn, NY 11211 – POINT(<LONG>,<LAT>) • -73.951353,40.716914 19 https://www.google.com/maps/place/33+Withers+St,+Brooklyn,+NY+11211/@40.7169144,-73.9513538
- 20. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | The Application Use Case • I’m hungry and in the mood for Thai food – What Thai restaurants are around me? – What’s the closest one? – How would I get there? 20
- 21. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Getting Some Data In • Download a NYC OSM extract: – http://osm-extracted-metros.s3.amazonaws.com/new-york.osm.bz2 • Import the data using a customized OsmDB.pm Perl module – http://wiki.openstreetmap.org/wiki/OsmDB.pm (original) – https://www.dropbox.com/s/l17vj3wf9y13tee/osmdb-scripts.tar.gz (customized) • Creates a spatial index on the ‘geom’ column 21 mysql -e "create database nyosm" bunzip2 new-york.osm.bz2 ./bulkDB.pl new-york.osm nyosm
- 22. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | The Generated Schema • http://wiki.openstreetmap.org/wiki/Elements 22 mysql> show tables; +-----------------+ | Tables_in_nyosm | +-----------------+ | nodes | | nodetags | | relationmembers | | relations | | relationtags | | waynodes | | ways | | waytags | +-----------------+ – We’ll focus on nodes and nodetags for our examples – Nodes • A point or location – Nodetags • Metadata about each location • name/value pairs
- 23. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | De-normalizing the Tag Data • Greatly simplify our query • Allow for the use of a full-text index – Also improves performance 23 mysql> alter table nodes add column tags text, add fulltext index(tags); mysql> update nodes set tags=(SELECT group_concat(concat(k, "=", v) SEPARATOR ';') from nodetags where nodetags.id=nodes.id group by nodes.id);
- 24. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Final Nodes Table 24 mysql> show create table nodesG *************************** 1. row *************************** Table: nodes Create Table: CREATE TABLE `nodes` ( `id` bigint(20) DEFAULT NULL, `geom` geometry NOT NULL, `user` varchar(50) DEFAULT NULL, `version` int(11) DEFAULT NULL, `timestamp` varchar(20) DEFAULT NULL, `uid` int(11) DEFAULT NULL, `changeset` int(11) DEFAULT NULL, `tags` text, UNIQUE KEY `i_nodeids` (`id`), SPATIAL KEY `i_geomidx` (`geom`), FULLTEXT KEY `tags` (`tags`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
- 25. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Creating a Distance Calculation Function • Various great circle (orthodrome) distance formulas – Haversine, Spherical Law of Cosines (my choice), … – http://en.wikipedia.org/wiki/Great-circle_distance – Necessary for calculating distances between two Geometries • Need goes away when we support Geography Type 25 mysql> CREATE FUNCTION slc (lat1 double, lon1 double, lat2 double, lon2 double) RETURNS double RETURN 6371 * acos(cos(radians(lat1)) * cos(radians(lat2)) * cos(radians(lon2) - radians(lon1)) + sin(radians(lat1)) * sin(radians(lat2)));
- 26. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Creating a Bounding Box For Our Search • Utilize the r-tree index by limiting area – Need to use some additional geographic formulas • http://www.movable-type.co.uk/scripts/latlong.html 26 ${origlon} = -73.951368 ${origlat} = 40.716743 ${lon1} = ${origlon} + (${distance_in_km}/111) ${lat1} = ${origlat} + (${distance_in_km}/111) ${lon2} = ${origlon} - (${distance_in_km}/111) ${lat2} = ${origlat} - (${distance_in_km}/111)
- 27. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Calculating the Results • Our final query, searching within ~ 10km radius 27 mysql> SELECT id, slc(40.716743, -73.951368, y(geom), x(geom))*1000 as distance_in_meters, tags, ST_AsText(geom) FROM nodes WHERE ST_Contains(ST_Envelope(linestring(point((-73.951368+(10/111)), (40.716743+(10/111))), point((-73.951368-(10/111)), (40.716743-(10/111))))), geom) AND match(tags) against ("+thai +restaurant" IN BOOLEAN MODE) ORDER BY distance_in_metersG
- 28. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Examining the Results 28 *************************** 1. row *************************** id: 888976948 distance_in_meters: 614.4973960877276 tags: addr:street=Bedford Avenue;amenity=restaurant;name=Tai Thai;addr:housenumber=206;phone=7185995556 astext(geom): POINT(-73.958637 40.717174) *************************** 2. row *************************** id: 2178443635 distance_in_meters: 2780.87697408101 tags: microbrewery=no;website=http://www.onemorethai.net/;name=One More Thai;amenity=restaurant;opening_hours=12:00-22:30;cuisine=thai;phone=(212) 228-8858 astext(geom): POINT(-73.983871 40.7210541) *************************** 3. row *************************** …
- 29. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Mapping the Results 29 • From my place – -73.951353,40.716914 • To Tai Thai – -73.958637,40.717174 https://www.google.com/maps/dir/40.716914,+-73.951353/40.717174,+-73.958637
- 30. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | What’s Next for MySQL GIS 30
- 31. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Additional Features • R-tree enhancements – 3D support • Improved storage – Fixed length storage when possible • Geography types • Projections • Additional performance optimizations • What else would you like to see…? Let us know..!!! 31
- 32. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Appendix : Additional Resources • Manual – http://dev.mysql.com/doc/refman/5.7/en/spatial-extensions.html • Community forum – http://forums.mysql.com/list.php?23 • Boost.Geometry – http://www.boost.org/libs/geometry • Report GIS bugs and submit feature requests – http://bugs.mysql.com/ 32
- 33. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Q&A 33
Editor's Notes
- This is a Safe Harbor Front slide, one of two Safe Harbor Statement slides included in this template. One of the Safe Harbor slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy To learn more about this policy, e-mail: Revrec-americasiebc_us@oracle.com For internal communication, Safe Harbor Statements are not required. However, there is an applicable disclaimer (Exhibit E) that should be used, found in the Oracle Revenue Recognition Policy for Future Product Communications. Copy and paste this link into a web browser, to find out more information. http://my.oracle.com/site/fin/gfo/GlobalProcesses/cnt452504.pdf For all external communications such as press release, roadmaps, PowerPoint presentations, Safe Harbor Statements are required. You can refer to the link mentioned above to find out additional information/disclaimers required depending on your audience.
- GIS: A computer-based system that stores geographically referenced data layers (features) and links it with non-graphic data tables (attributes) allowing for a wide range of information processing, including manipulation, analysis, and modeling. A GIS also allows for map display and production.
- We’re going to focus on simple location services in our examples.
- X or northing which will typically be a longitude value, Y or easting which will typically be a latitude value, Z or height (optionally a true geodetic value), and M or measure (which can be used for Time, for example).
- University Consortium for Geographic Information Science USGS or United States Geological Survey
- Started out as the Generic Geometry Library by OSGeo. Now it’s of course part of Boost.
- R-trees are the most common index type used for spatial data. It’s a wide search tree, with some similarities with B-trees. It’s similar in that of course it’s a search tree, and it has root nodes, branch nodes, and leaf nodes. The main differences are: 1. R-trees use pages for each level in the tree, and the page can contain X number of nodes. So the search is not binary at any level. Each level in the tree can have up to some maximum number of nodes. The maximum being set by the specific implementation. 2. You search by bounding box. If the search box overlaps with the MBR stored in a node, then you continue to search down that path, moving to the next child node.
- You can set the SRID of geometries in MySQL to any 32 bit unsigned integer, and we will refuse to mix geometries of different SRIDs in the same operation. In calculations, everything will be treated as SRID 0, which in MySQL is a Cartesian system without units (what you get if you don't specify an SRID).
- Common alternatives are a fixed address like this, or a GPS location from your mobile device.
- I’m using the spherical law of cosines formula because it’s simpler, and thus faster, than Haversine; while also giving us virtually the same accuracy (Haversine is generally more accurate though, particularly for short distances). That’s generally why Haversine is the most common method that you’ll see used. It’s possible that this need goes away later in 5.7 too. We’re looking into possibly adding an ST_Distance_Sphere() function that returns the great-circle earth distance calculation, in meters, between two geometries (which are points containing X,Y or LON/LAT coordinate pairs). ST_Distance_Sphere — Returns minimum distance in meters between two lon/lat geometries. Uses a spherical earth and radius of 6,370,986 meters.
- We’ll simply use the average distance between longitude and latitude degrees in our coming example. We don’t need the envelope to be too accurate as we’re simply using it to pass down to the spatial index, and we’re later calculating the actual distance using our new SLC function. The need for this may go away later in 5.7 too. We’re considering adding an ST_MakeEnvelope() function that takes a Geometry parameter (again a POINT containing a LON/LAT coordinate pair) and an integer to create an envelope that would contain all points within the specified number of meters from the Geometry. ST_MakeEnvelope(point, distance) --- Make a rectangle around a point(x,y) so that the distance between the point and the rectangle's boundaries is 'distance' units (km) away. This is done on an abstract cartesian plane and simply creates an envelope that contains all points within a radius of approximately <distance> km around the <point>.
- In our example query we’re simplifying the bounding box calculation and using <km>/111, 111 being the average distance in km between longitude and latitude degrees. It’s not very accurate for longitude, in particular the further you get away from the equator, but it doesn’t have to be very accurate for our bounding box as we’re calculating the actual distance with our SLC function. The bounding box is just to pass down to the spatial index so that we can weed out all of the irrelevant points.