Successfully reported this slideshow.

Where the %$#^ Is Everybody? Geospatial Solutions For Oracle APEX

0

Share

1 of 31
1 of 31

Where the %$#^ Is Everybody? Geospatial Solutions For Oracle APEX

0

Share

Download to read offline

Geospatial use cases are common – closest coffeeshop, efficient delivery routing, other stores near me – and I’ll show you how to use Oracle’s Spatial and Graph feature set to tackle them within a simple-to-build APEX application.

Geospatial use cases are common – closest coffeeshop, efficient delivery routing, other stores near me – and I’ll show you how to use Oracle’s Spatial and Graph feature set to tackle them within a simple-to-build APEX application.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Where the %$#^ Is Everybody? Geospatial Solutions For Oracle APEX

  1. 1. Where the %$#^ Is Everybody? Geospatial Solutions For Oracle APEX Jim Czuprynski @JimTheWhyGuy Zero Defect Computing, Inc. May 14, 2021
  2. 2. My Credentials • 40 years of database-centric IT experience • Oracle DBA since 2001 • Oracle 9i, 10g, 11g, 12c OCP and ADWC • Oracle ACE Director since 2014 • ODTUG Database Committee Lead • Editor of ODTUG TechCeleration • Oracle-centric blog (Generally, It Depends) • Regular speaker at Oracle OpenWorld, COLLABORATE, KSCOPE, and international and regional OUGs E-mail me at jczuprynski@zerodefectcomputing.com Follow me on Twitter (@JimTheWhyGuy) Connect with me on LinkedIn (Jim Czuprynski)
  3. 3. Jim Czuprynski Liron Amitzi https://www.beyondtechskills.com The podcast that talks about everything tech – except tech.TM
  4. 4. Converged Database: A Vision for the Future, 21c and Beyond Personal / External Datasets Enterprise Applications Data Integration OAC, OML, APEX, and Graph Studio Ad hoc, Batch or Scheduled Business Leaders Analysts Data Scientists Developers OAC Dataflow, Manual or ETL Data Management ADW Business Analytics ERP CRM HCM Self-sufficient, encrypted, secured data storehouse Self-service analytics via ML REST-Enabled External APIs IoT and Edge Computing ATP AJD AGD The new kid on the block: Autonomous JSON Database Recently announced: Autonomous Database for Graph Studio New APEX As A Service offering makes it even easier to create APEX environments for immediate use
  5. 5. Why Have Geographic Information Systems (GIS) Become So Important? COVID-19 vaccination, mitigation, and tracking Electrical Vehicles (EVs) and effective ranges to charging stations Smart Cities: Scooters, Bicycles, and Traffic Flow More efficient use of agricultural resources & eliminating over-watering Autonomous delivery vehicles Improved worker safety for occupations most affected by climate change
  6. 6. Geospatial Basics: GeoCoding, POI, Routing, and Directions Finding points on a map is relatively easy – we just need a longitude/latitude pair But finding point(s) within the range of a specific location is quite a bit harder and requires some serious math And determining a route between multiple points on a map on known roads takes quite a bit of sophistication And then figuring out the best route between points when the traveler is outside the range of that route? That’s the subject of well-written PhD dissertations
  7. 7. Geolocation and the SDO_GEOMETRY Datatype INSERT INTO user_sdo_geom_metadata VALUES ( 'T_SMARTMETERS' ,'SM_GEOLOCATION' ,SDO_DIM_ARRAY( SDO_DIM_ELEMENT('Longitude', -180, 180, 0.5) ,SDO_DIM_ELEMENT('Latitude', -90, 90, 0.5) ) ,8307); COMMIT; This registers the table and its corresponding SDO_GEOMETRY column as eligible for GIS handling using the Longitude / Latitude WGS 84 coordinate system, thus permitting the creation of a spatial index later . . . UPDATE simiot.t_smartmeters SET sm_geolocation = SDO_GEOMETRY( 2001 ,8307 ,SDO_POINT_TYPE( sm_lng ,sm_lat ,NULL) ,NULL ,NULL ); . . . 2001 indicates this is a 2 - dimensional single point 8307 says we’re using the Longitude / Latitude WGS 84 coordinate system These are the columns in T_SMARTMETERS containing the Longitude + Latitude pair These remaining settings are NULL because they’re not applicable to this type of geometry
  8. 8. Jeff Kemp’s Google Maps APEX Plug-In (JK64) Jeff Kemp has continued to refine this APEX plug-in’s feature set to include sophisticated, in-demand features Just about any feature of Google Maps can be exploited, including geolocation, distancing, routing, and directions Get Jeffrey Kemp’s latest Google Map Plug-ins here: https://github.com/jeffreykemp/jk64-plugin- reportmap/releases-latest
  9. 9. Oracle GeoSpatial (OGS) APEX Plug-In Full details here: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/75461594 You can experiment with Oracle’s built-in geolocation services by adding addresses … … and one click on the Geocode button captures and retains latitude & longitude values
  10. 10. JK64: Showing Locations On a Map We’re showing the location and some attributes of the top 25 consumers of solar power for a specific time frame SELECT sm_lat AS lat ,sm_lng AS lng ,sm_name AS name ,sm_id AS id ,sm_business_type as info ,[add your preferred icon file or URL reference]' as icon FROM simiot.t_smartmeters SM ,(SELECT smr_id ,ROUND(AVG(smr_solar_kwh) / AVG(smr_kwh_used) ,2) pct_solar FROM t_meter_readings WHERE smr_timestamp BETWEEN TO_TIMESTAMP('2021-01-23 09:00', 'yyyy-mm-dd hh24:mi’) AND TO_TIMESTAMP('2021-01-23 09:45', 'yyyy-mm-dd hh24:mi') GROUP BY smr_id HAVING ROUND(AVG(smr_solar_kwh) / AVG(smr_kwh_used) ,2) >= 10.00 ORDER BY smr_id FETCH FIRST 25 ROWS ONLY ) SMR WHERE SMR.smr_id = SM.sm_id; Just place a query that returns the proper result set within the source of the plug-in’s region
  11. 11. JK64: GeoLocation Capabilities We can also use the Google Maps API to capture and retain longitude and latitude for a specific address or location We can also change the Map Zoom Level for the resulting map to zoom into the surrounding area at a predefined focus
  12. 12. JK64: Finding Closest Points of Interest Here’s a more complex scenario: Locating closest Points of Interest to a specific location This result set is returned based on the screening criteria supplied in the topmost region The resulting map shows up to three (3) Points of Interest closest to the selected location based on the Miles Away specified. Some JavaScript in the dynamic event for the Lookup Results region redraws the map whenever we select a new location
  13. 13. Prerequisites to POI: Creating Spatial Indexes DROP INDEX simiot.dispatch_centers_spidx FORCE; CREATE INDEX simiot.dispatch_centers_spidx ON simiot.t_dispatch_centers(dc_geolocation) INDEXTYPE IS MDSYS.SPATIAL_INDEX_V2; DROP INDEX simiot.smartmeters_spidx FORCE; CREATE INDEX simiot.smartmeters_spidx ON simiot.t_smartmeters(sm_geolocation) INDEXTYPE IS MDSYS.SPATIAL_INDEX_V2; Create system-managed domain indexes on the contents of each table’s SDO_GEOMETRY column SELECT index_name ,ityp_name ,domidx_status ,domidx_opstatus FROM user_indexes WHERE index_type = 'DOMAIN'; Domain Index Operation Index Name Index Type Status Status ---------------------- ------------------ --------- --------- DISPATCH_CENTERS_SPIDX SPATIAL_INDEX_V2 VALID VALID SMARTMETERS_SPIDX SPATIAL_INDEX_V2 VALID VALID Verifying the status of Spatial Indexes -- Underlying query from OGS Points of Interest map plug-in SELECT TO_CHAR(dc_id) AS id ,dc_geolocation AS geometry ,'Dispatch Center: ' || dc_name AS infotext ,dc_address || ', ' || dc_city || ', IL ' || dc_zipcode AS infotip ,'blue' AS style FROM simiot.t_dispatch_centers WHERE SDO_NN( dc_geolocation ,(SELECT sm_geolocation FROM simiot.t_smartmeters WHERE sm_id = :P230_SMID ) ,'sdo_num_res=3, distance=5, unit=mile', 1) = 'TRUE' UNION ALL SELECT TO_CHAR(sm_id) AS id ,sm_geolocation AS geometry ,'Customer: ' || sm_name AS infotext ,sm_address || ', ' || sm_city || ', IL ' || sm_zipcode AS infotip ,'red' AS icon FROM simiot.t_smartmeters WHERE sm_id = :P230_SMID This qualifies the maximum number of points to return within a specific distance
  14. 14. JK64: Routing and Directions It’s also possible to take direct advantage of Google Maps Routing and Directions API features We’re limited to a starting point, an ending point, and eight (8) other locations … … including detailed directions between each point on the map
  15. 15. OGS: Showing Locations On a Map Here’s the same map we originally built using the Google Maps plug-in, again showing the location and some attributes of the top 25 consumers of solar power for a specific time frame SELECT sm_id AS id ,sm_geolocation AS geometry ,sm_name || ' ' || sm_address || ', ' || sm_city || ', IL ' || sm_zipcode AS infotip ,sm_business_type AS infotext ,’red' AS style FROM simiot.t_smartmeters SM ,(SELECT smr_id ,ROUND(AVG(smr_solar_kwh) / AVG(smr_kwh_used) ,2) pct_solar FROM t_meter_readings WHERE smr_timestamp BETWEEN TO_TIMESTAMP('2021-01-23 09:00', 'yyyy-mm-dd hh24:mi’) AND TO_TIMESTAMP('2021-01-23 09:45', 'yyyy-mm-dd hh24:mi') GROUP BY smr_id HAVING ROUND(AVG(smr_solar_kwh) / AVG(smr_kwh_used) ,2) >= 10.00 ORDER BY smr_id FETCH FIRST 25 ROWS ONLY ) SMR WHERE SMR.smr_id = SM.sm_id; Again, all that’s required is to place a query that returns the expected result set within the source of the plug-in’s region Note that we don’t need to supply longitude and latitude values – OGS captures those attributes directly from the SDO_GEOMETRY column
  16. 16. OGS: GeoLocation Capabilities The OGS plug-in also offers Geolocation capabilities, including metrics on how successful and accurate the geolocation operation was It’s also possible to perform controlled zoom in / out operations, or allow mouse wheel zoom control if it’s preferrable
  17. 17. OGS: Finding Closest Points of Interest With only a few changes to the underlying query that the OGS map plug-in requires, we’re able to produce a map with virtually identical features Here’s a closer look at the selected Customer site (red) and the nearest three Dispatch Centers (blue)
  18. 18. OGS: Routing and Directions Unfortunately, without access to the appropriate web services, we can’t provide detailed routing … … nor can we provide detailed directions for the route itself
  19. 19. Comparison: JK64 Google Maps Plug-in vs. OGS Plug-in Feature JK64 Plug-In OGS Plug-In (as of March 17, 2021) Mapping • Easy to set up via simple SQL query Uses longitude / latitude pairs to identify points • Easy to set up via simple SQL query Uses SDO_GEOMETRY datatype to identify points Geolocation • Uses Google Maps API calls to locate points Potentially costly charges tied to Google API key • Uses calls to OGS native methods to locate points • Geolocation limits more generous than Google API Points of Interest • Uses Google Maps API calls to process Area of Interest queries • Potentially costly charges tied to Google API key • Uses calls to OGS native methods to identify pertinent Areas of Interest • Limits appear to be more generous than Google API Directions & Routing • Easy to display with new Show Directions DA • Uses Google Maps API calls to build features • Potentially costly charges tied to Google API key • Requires licensing appropriate Routing Engine features, a part of Spatial Web Services • Need to set up a separate Web Logic Server to host Routing Engine services Note: The Oracle APEX Roadmap projects a new Native Map Component coming soon: https://apex.oracle.com/en/learn/resources/roadmap/
  20. 20. APEX Mapping Plug-Ins: Cost Comparisons Services and Features Google Maps1 Oracle Spatial & Graph2 API Key Needed? Yes No Geocoding and Geolocation $5 / 1000 requests1 Included with DB license2 Static Mapping $2 / 1000 requests1 Included with DB license2 Dynamic Mapping $7 / 1000 requests1 Included with DB license2 Points Of Interest (POI) $7 / 1000 requests1 Included with DB license2 Distances and Travel Times $5 / 1000 requests1 Included with DB license2 Routing and Directions $5 / 1000 requests1 Included with DB license2 1Google pricing: https://cloud.google.com/maps-platform/pricing. The first $200 per month of consumption is free. 2Oracle Spatial & Graph pricing details: https://blogs.oracle.com/oraclespatial/spatial-now-free-with-all-editions-of-oracle-database
  21. 21. REST Data Sources: Leave the Data Where It Lives! REST Data Sources let you access vast amounts of external data via REST API calls 1 Here we’re using the GeoCodio API to retrieve Latitude, Longitude, and other specific GIS information for a large batch of addresses 2 The REST API call requires an API key as well as a compressed, single- field version for each address, but it returns a wealth of information, including longitude / latitude pairs 3
  22. 22. Complex REST API Calls? APEX Packages to the Rescue! DECLARE -- Processing variables: SQLERRNUM INTEGER := 0; SQLERRMSG VARCHAR2(255); -- CLOBs for input and output: sent_clob CLOB; recv_clob CLOB; -- JSON parsing variables: recv_values APEX_JSON.T_VALUES; mbr_count PLS_INTEGER; vid VARCHAR2(4000); lat NUMBER(9,6); lng NUMBER(9,6); -- Process each set of next 40 smart meters at one time CURSOR curNeedLatLng IS SELECT TO_CHAR(sm_id) AS sm_id ,sm_address || sm_city || sm_state || sm_zipcode AS formatted_address FROM t_smartmeters WHERE s,_lat IS NULL AND sm_lng IS NULL AND rownum <= 40; . . . Set up variables for processing and gathering 3rd-party Geocoding data for entries missing it . . . BEGIN APEX_JSON.INITIALIZE_CLOB_OUTPUT; APEX_JSON.OPEN_OBJECT; FOR i IN curNeedLatLng LOOP APEX_JSON.WRITE(i.sm_id, i.formatted_address); END LOOP; APEX_JSON.CLOSE_OBJECT; sent_clob := APEX_JSON.GET_CLOB_OUTPUT; APEX_JSON.FREE_OUTPUT; APEX_WEB_SERVICE.G_REQUEST_HEADERS.DELETE(); APEX_WEB_SERVICE.G_REQUEST_HEADERS(1).name := 'Content-Type'; APEX_WEB_SERVICE.G_REQUEST_HEADERS(1).value := 'application/json’; recv_clob := APEX_WEB_SERVICE.MAKE_REST_REQUEST( p_url => 'https://api.geocod.io/v1.6/geocode?api_key=ofe…^$#@%^&$#@...ee5' ,p_http_method => 'POST' ,p_body => sent_clob); . . . Call the corresponding APEX web service to retrieve the Geocoding values . . . -- Populate the received JSON output into a CLOB, and then process the results APEX_JSON.PARSE( p_values => recv_values ,p_source => recv_clob); mbr_count := APEX_JSON.GET_COUNT(p_path => 'results', p_values => recv_values); FOR i IN 1 .. mbr_count LOOP sid := (APEX_JSON.GET_MEMBERS(p_path => 'results', p_values => recv_values)(i)); lat := APEX_JSON.GET_VARCHAR2(p_path => 'results.’|| sid || '.response.results[1].location.lat', p_values => recv_values); lng := APEX_JSON.GET_VARCHAR2(p_path => 'results.’|| sid || '.response.results[1].location.lng', p_values => recv_values); UPDATE t_smartmeters SET sm_lat = lat, sm_lng = lng WHERE sm_id = TO_NUMBER(sid); END LOOP; COMMIT; . . . Apply each Long/Lat pair to its corresponding entry in the table . . . UPDATE t_smartmeters SET sm_geopoint = SDO_GEOMETRY( 2001 ,8307 ,SDO_POINT_TYPE(sm_lng, sm_lat, NULL) ,NULL ,NULL); COMMIT; END; / Refresh all updated Long/Lat data in the underlying SDO_GEOMETRY data type of the table
  23. 23. 3rd-Party Alternatives for Routing and Directions Build a list of the 24 SmartMeter customer sites within a range of 20 miles from a particular Dispatch Center After a little creative editing, we’ll pass this list of longitude / latitude coordinate pairs to OpenRouteService’s free routing and directions engine OpenRouteService provides a relatively simple map showing the results graphically … … but routing and direction information can be downloaded in GeoJSON format, which means we can use it as a layer on any existing map!
  24. 24. APEX 21.1: Announcing Native Map Regions The latest version of APEX (21.1) offers a new Native Map region that lets you build maps to visualize location data on a map, without any plug-in required! • Five different layer types: Point, Line, Polygon, Heat Map, and Extruded (3D) Polygon • All background maps are sourced directly from the Oracle Maps Cloud • Fully interactive map features, including Drag, Zoom, and ToolTips • Plot map features from: • Simple numeric columns (Longitude and Latitude) • SDO_GEOMETRY datatype • GeoJSON documents • Leverages Oracle Spatial features, including Spatial Indexes and Coordinate Transformations (if they are available in your data model) • Visualize data from SQL queries against local database, REST-Enabled SQL, or from REST Data Sources • No API Key required!
  25. 25. APEX 21.1: Using a Native Map Region (1) Select the new Map Region … … name the new Page … … and set up the new page’s Navigation entry
  26. 26. APEX 21.1: Using a Native Map Region (2) Choose a target table … … pick either a column with an SDO_GEOMETRY datatype, or select a pair of Longitude / Latitude columns for map positioning Let’s also add a Faceted Search region for the new page
  27. 27. APEX 21.1: Using a Native Map Region (3) Here’s the results of the Create Page dialog, including the Faceted Search region The Native Map region offers excellent control over the appearance, navigation, and initial focus of the map that’s displayed
  28. 28. APEX 21.1: Using a Native Map Region (4) Here’s an example of filtering map points for just a single ZIP Code via the Search region … … and now filtering for only map points containing the phrase “archer”
  29. 29. Useful Resources and Documentation Jeffrey Kemp’s Google Maps APEX Plug-ins: https://jeffkemponoracle.com/tag/apex-plugins/ https://github.com/jeffreykemp/jk64-plugin-reportmap/wiki/SQL-Query-Examples Articles on Leveraging APEX For ML, Mapping and Geolocation: https://www.odtug.com/p/bl/et/blogaid=966 https://www.odtug.com/p/bl/et/blogaid=981 APEX_DATA_PARSER and APEX_JSON Implementation Examples: https://blogs.oracle.com/apex/super-easy-csv-xlsx-json-or-xml-parsing-about-the-apex_data_parser-package https://jsao.io/2015/07/relational-to-json-with-apex_json/
  30. 30. Useful Resources and Documentation APEX Sample Geolocation Showcase: Documentation: https://cloudmarketplace.oracle.com/marketplace/en_US/listing/75461594 YouTube Video Demo: https://www.youtube.com/watch?v=QCh8nAUxDBg&feature=youtu.be Oracle Spatial and Graph Developers Guide: https://docs.oracle.com/en/database/oracle/oracle-database/19/spatl/ Finding the Closest Items of Interest: https://docs.oracle.com/en/database/oracle/oracle-database/19/spatl/routing-engine-concepts.html Using Oracle Routing Engine Features: https://docs.oracle.com/en/database/oracle/oracle-database/19/spatl/routing-engine-concepts.html OpenRouteService Open-Source GIS Tools: https://openrouteservice.org/
  31. 31. Useful Resources and Documentation APEX 21.1 General Availability Announcement: https://blogs.oracle.com/apex/announcing-oracle-apex-211 APEX 21.1 Native Map Regions: https://docs.oracle.com/en/database/oracle/application-express/21.1/htmdb/creating-maps.html Managing Faceted Search: https://docs.oracle.com/en/database/oracle/application-express/21.1/htmdb/managing-faceted-search.html

×