Data managing and Exchange GDB


Published on

Presentation on Esri European User Conference 2011 by Jaime Nieves and Isaac Medel.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Data managing and Exchange GDB

  1. 1. esri European User Conference Madrid 2011 Data managing and Data Exchange: Geodatabases and Interoperability Jaime Nieves Isaac Medel Esri Spain
  2. 2. Session highlights • An Overview of the Esri Geodatabase - The basics about databases in ArcGIS - Data model and object types • Geodatabase Workflows & Editing - Geodatabase levels - Versioning, replication and historics • Database Administration - Database creation and managing • Data storage and connection - Connection approaches: Clients and databases - Spatial types and interoperability
  3. 3. Geodatabase basic concepts
  4. 4. Geodatabase • Geographic Data Store - Scalable, multi-user • Core ArcGIS Data model Networks Survey Addresses Vectors Annotations • Transactional model for 3D Objects Attributes GIS workflows Dimensions Topology • COM Components for Terrain Parcels data accesibility CAD Cartography Raster
  5. 5. Data management approach • Geodatabase built on a table structure - Spatial Data Types & SQL - Relational Integrity - Scalability, Reliability, Flexibility - Support for continous, large datasets • Based on the simple feature model - Points, Lines, Polygons - Open Access
  6. 6. Data on Geodatabases • Esri System Tables - Advanced Business Logic System Tables User Data • Consolidated at 10.0 in 6 single tables. - Improved stability & fast data access - Uses XML columns - Same structure for all GDBs SQL XML type • Sharing of data model - Schema & Data to XML
  7. 7. Features and Feature Classes • Builds on the Relational Model • A feature is a spatial object • A feature is an instance of a feature class • Extended the relational model - Geometry attribute types A feature class is a table of rows, where each row has a geographic column
  8. 8. Geodatabase Elements Tables Feature dataset Feature Classes Spatial reference Raster Datasets Polygon Route Additional geodatabase elements Line Dimension Parcel fabrics Terrain datasets Point Annotation Representations Locators Relationship classes Toolboxes Tool Model Script Geometric networks Topology Behavior Attribute defaults Connectivity rules Network datasets Attribute domains Relationship rules Split/merge policy Topology rules
  9. 9. Complex data in the Geodatabase • Multi-part features - Many parts, one record in the table • 3D Objects - Vertical lines 1 3/8" - Multipatches Placerville • Measurements Sacramento - M-Coordinate Topanga Canyon - Dimensions • Cartographic representations • Geodatabase attachments
  10. 10. Complex data in the Geodatabase • Elevation data - Dynamic TINs (Terrain Datasets). - LiDAR datasets • Raster data - Imagery and continuous data - Mosaic Datasets - Image processing & filtering - Raster catalogs - Raster files inside a table field
  11. 11. Complex behavior in the Geodatabase • Validation Rules - Attribute, connectivity and relationships • Domains - Ranged or coded values • Subtypes - Defined as the class level • Relationship classes - Association between objects on different classes
  12. 12. Complex behavior in the Geodatabase • Geometric Networks - Connectivity rules • Network datasets - Multi-modal networks - Turns, restrictions, costs • Topology - Set of rules that define spatial behavior - Ensure data integrity • Parcel fabrics
  13. 13. Behavior: Summary • You can: - Control the default value and acceptable values for any attribute (Domains) - Partition the objects into like groups (Subtypes) - Instantiate classes with predefined behavior (Dimensions and Annotation) - Control the general and network relationships in which an object can participate (Relationship Classes) • Out of the Box in ArcGIS! - Configurable, no programming required
  14. 14. Geodatabase workflows Geodatabase editing
  15. 15. Geodatabase levels ArcGIS Server ArcSDE ArcGIS Desktop Enterprise Workgroup Desktop Personal File Capacity Number of Users - Storage Microsoft Access Files RDBMS - Capacity 2 GB 1 to 256 TB per table Variable - Platform Windows Multi-platform Variable- Users editing Single User Single User Multi-user
  16. 16. Geographic data management ArcGIS Server ArcSDE Enterprise Workgroup Desktop Capacity Number of users Esri technology for managing and accessing geographic data on Relational Database Systems
  17. 17. Geographic data management ArcGIS Server ArcSDE Enterprise Workgroup Desktop Capacity Number of users DB2, Informix, Netezza Oracle, PostgreSQL & RDBMS SQL Server Express SQL Server Express SQL Server Capacity 10 Gb Max 10 Gb Max Unlimited Platorm Windows Windows Multi-platform Users/Editing 4/1 10 / 10 Unlimited
  18. 18. Editing Geodatabases • ArcGIS datasets in the geodatabase are editable - Modify building footprints in parcel management - Add water mains to a water network - Update land owners information stored in a table - Etc… • Transaction model for editing in ArcGIS - Edits are performed in an edit session - Open session – edit – save edits / don’t save edits - A series of edit operations constitutes a transaction - The transaction is either committed or rolled back
  19. 19. Editing Geodatabases… • Personal Geodatabases - Mainly single user editing on small datasets - Multiple readers - Editing locks at geodatabase level - Two editors cannot edit within the same geodatabase at same time • File Geodatabase - Mainly single user editing small to very large datasets - Multiple readers - Editing locks at the dataset level - Multiple editors cannot edit the same table or stand-alone feature class at the same time - Multiple editors cannot edit feature classes in the same feature dataset at the same time
  20. 20. Editing Geodatabases… • Enterprise Geodatabases - Extend the transaction model with Versions - Multiuser editing without locking - Unique isolated view of the geodatabase • Benefits of versioned editing - Multiple editors, editing over long periods of time - Undo / Redo - Archiving - Replication
  21. 21. Versioning • Technology that allows multiple users to edit and view data at the same time - Appears to users as if they have their own copy of a table - Does not apply locks or duplicate data Default Readers Version1 Version2 Editor 1 Editor 2
  22. 22. Workflow in a multi-user editing environment Version creation Connect to the version Editing Reconcile: ¿conflicts? Conflict resolution Post Remove & Compress
  23. 23. Replicas ArcSDE GDB • Geodatabase replication extends the DFLT concept of versioning to multiple Parent VR1 VR2 databases. Child check out check in DFLT ArcSDE GDB V1 V2 File GDB DFLT Parent Child ArcSDE GDB VR1 VR2 Two One way way - Versions in parent and child geodatabases are used as replica versions. File GDB DFLT DFLT - Changes in this replica versions are transferred during synchronization V1 V2 ArcSDE GDBArcSDE GDB
  24. 24. Replication types • Checkout/Check-in replication: One time - Edits in child replica can be syncronized one single time with Parent Child parent replica. SDE - Uses: Disconected editing; File GDB as a requisite. SDE PGDB FGDB • One-way Replication: - Allows data changes to be sent multiple times in a single direction (parent-to-child or child-to-parent) Development Production - Data in origin GDB are editable; data in destination GDB are Persist read-only. Edit Read - Changes in destination GDB are overwritten during syncronization if edits in origin GDB exist. SDE (child-to-parent) SDE PGDB - Uses: development-staging-production models; data publishing FGDB • Two-way replication: Office A Office B - Allows data changes to be sent multiple times, both ways. Persist - If the same row is edited in both replica geodatabases, it is detected as a conflict when the replicas are syncronized. Edit A Edit B SDE Conflict SDE resolution
  25. 25. New workflows • New replication scenarios: - File GDBs can be used as one-way and SDE FGDB check-in/check-out replicas. - One-way child-to-parent replicas - Multiple sendings from child to parent - Editable data on child GDB; read-only on parent GDB. - One-way replicas using archiving - Improved performance and easier management compared to versioning.
  26. 26. Resolving syncronization conflicts • Conflict resolving using Replica Manager - Enables user to decide the geometry and attributes to keep in case of a conflict. - Uses a visual environment - Merge geometries if there is a conflict on different areas on the same Shape.
  27. 27. Historical versions • Geodatabase archiving. - Built on Geodatabase versions. - Enables recording and accessing changes made to a subset of data. - No need for storing snapshots for the entire database. - Archiving lives as long as the Version is alive. - Introduces a historical version. - Creation of historical markers possible.
  28. 28. Geodatabase compress • Maintain database performance and health. - Delete all states that do not participate within a versions lineage. - Collapse any candidate lineage of states into one state. - Move rows from the delta tables into the business tables. Before After 0 0 v2 v2 v1 DEFAULT v1 v4 v4 DEFAULT
  29. 29. Database administration
  30. 30. Geodatabase Data Management • Schema is defined in ArcCatalog - Define feature classes, datasets, relationships, etc - Catalog window in ArcMap @ 10 • Import and convert data from other formats - Shapefile - Coverage - CAD - Raster • Copy and Paste • - Import basemaps and layers from galleries, groups… • Use an esri Data Model or Downloadable Template - Industry specific data models available - Download a geodatabase template from the resource centers
  31. 31. Create an enterprise geodatabase • Enabling the advanced capabilities of the geodatabase in: - IBM DB2 - Informix - Microsoft SQL Server - Oracle - PostgreSQL • Enable Enterprise Geodatabase Tool. - Adds the geodatabase to an existing database and authorizes it • Create Enterprise Geodatabase Tool. - Creates database and geodatabase in PostgreSQL and SQL Server • Python script
  32. 32. Geodatabase administration in 10.1 • New Geodatabase administration dialog box - Manage versions - View and remove connections - View and manage locks - Block new connections • For system tables: - Rebuild indexes - Update statistics
  33. 33. Managing data in ArcGIS 10.1 • View and manage privileges • View locks on a version or specific dataset • Rebuild indexes on multiple datasets • Update database statistics on multiple datasets • Enable editor tracking • One-click spatial table registering
  34. 34. Database administration in 10.1 Isaac Medel
  35. 35. Database storage and connection
  36. 36. ArcSDE ArcGIS Desktop From Esri Geodatabases To Esri Clients
  37. 37. Direct connect the database • Client-DBMS connection with an SDE service in the way. • Process load on the client side, not the server side - Improved CPU & Memory usage on the server. - Higher scalability. - Backwards compatibility. - No need for ArcSDE service Application Server Direct Connect administrattion. (three-tier) (two-tier) - Enables operating system authentication GIS GIS for your ArcGIS Connections client client in Oracle geodatabases. Direct connect driver AppServer Geodatabase
  38. 38. Database compatibility
  39. 39. Geodata • Geodatabase access through a web connection. • Exposes remote capabilities for: - Replication. - Copy (data extraction). - Query. Enterprise GDB Enterprise GDB Geodata 2-way Local Service GDB File GDB LAN SQL Server Internet Express SQL Server File GDB Express
  40. 40. Forward compatibility in 10.1 • Updates in 10.1 datasets: - Python scripting with annotation & dimensions - Editor tracking in tables & feature classes - Netwok datasets improvements - Improved mosaic datasets • Creation of new datasets in 10.0 geodatabases - No upgrade required - Still accesible by 10.0 clients. - 10.0 SP2 recommended 10.0 10.1
  41. 41. Query Layers • A layer or stand alone table defined by a SQL query • Provides direct, read-only access to spatial data independent of where it is stored • Data does not need to be registered with the geodatabase • Full power of SQL to create feature classes and Tables: - Joins, Spatial Types, Aggregation, Field Derivation • New tool in 10.1: - Make Query Layer SELECT Name, City, ShapePoint FROM Hospitals
  42. 42. Query Layers Isaac Medel
  43. 43. Anywhere ArcGIS Desktop From Any Database To Esri client
  44. 44. Work with databases in ArcGIS 10.1 • Database connection from Desktop - Work with databases without enterprise geodatabases - Copy & paste between databases and geodatabases - Create & query tables and features in a database - Create views in a database - Publish data to ArcGIS Server • Database connection dialog box simplified - Less inputs required - Sharing connection files - Create database connection geoprocessing tool
  45. 45. Connection to databases Isaac Medel
  46. 46. Anywhere Anything From Any Database To Any Client
  47. 47. Feature Services • Web access to feature geometry • RESTful capabilities: - Geometry & attribute query - Database editing - Access related records and non-spatial tables Mobile Feature Service URL Web Desktop / Database ArcGIS Server Web Desktop
  48. 48. Spatial Data Server (SDS) • Low-memory web server. • Exposes geometries, attributes, symbols and templates from vector data in a database or geodatabase. • Uses feature services to access spatial data. • Available for IIS andJava Spatial Data Server Database
  49. 49. Spatial Data Server Isaac Medel
  50. 50. ArcSDE Anything From Esri Geodatabases To Any client
  51. 51. Third-party clients concurrent editing • Non-versioned Multi-user editing - Table-to-table edition SDE FGDB - Short transactions - Simple features • Versioned and non-versioned editing on the same data. - “Move edits to base tables” option - Edit from ArcGIS and third-party applications concurrently. - DEFAULT version stores changes in base tables.
  52. 52. Database accessing from third-party clients GDB APIs C# JavaArcSDE GDB Client
  53. 53. GDB developing environments • Geodatabase ArcObjects: - Application-level logic. - Geometry behavior and integrity. - Programming: C++, Java, .Net (C#, Java), Python • ArcSDE SDK: - A set of independent libraries for database accessing from third-party clients. - Programming: C, Java • Features: - Connections and transactions. - Geometry operations - Table management. - Raster management. - Change or create the Schema. - SQL access to Esri’s spatial type. - Versioning. - XML storage and search.
  54. 54. File Geodatabase API • No ArcObjects (no licence). • Access to File GDB from third-party clients. • C++ API which enables developers to: - Create, access and delete FGDBs. - Read FGDB Schema - Create Schemas for simple objects. - Read/write data. - Perform alphanumeric and spatial queries - Navigate through relationships. - Work with geodatabase attachments.
  55. 55. Database accessing from third-party clients GDB APIs C# JavaArcSDE GDB Client ST_GEOMETRY SQL SFS (OGC) ISO 19125
  56. 56. About ST_Geometry • UDT available for storage in all databases. • OGC * and ISO** definition: • SFS (Binary Geometry) 1.1, SFS (Types and Functions) 1.1 • Implemented as OGC Compliant in ArcGIS since 2006. • Install the ST_Geometry storage type in Oracle or PostgreSQL • Create Spatial Type tool* OpenGIS Implementation Specification for Geographic information - Simple feature access - Part 2: SQL option** ISO/IEC 13249-3 SQL multimedia and application packages - Part 3: Spatial ISO 19125-2:2004 Geographic information -- Simple feature access -- Part 2: SQL option
  57. 57. OGC and Standards OGC OGC Client Non-GIS Client
  58. 58. OGC support in ArcGIS OGC Category Standards Map WMS 1.0, WMS 1.1.1, WMS 1.3.0, WMtS 1.0.0, KML 2.1.0, KML 2.2.0, GML 2.1.2, GML 3.0 Feature WFS 1.0.0, WFS 1.1.0, GMLsf 1.1.0, WFS(T) 1.0.0 Metadata CAT CS/W 2.0.1, CAT 1.0, CAT 1.1.1, CAT 2.0.2, GeoRSS 1.0.0, CAT2 AP eRIM 1.0.0, CAT2 ISO AP 1.0.0, CAT2 ebRIM part2 1.0.0 SQL SFS(BG) 1.1, SFS(TF) 1.1 Raster WCS 1.0.0, WCS 1.1.0, WCS 1.1.1 Any Symbology SLD 1.0 client Processing WPS 1.0.0 Encoding Filter 1.0, Filter 1.1
  59. 59. OGC support in ArcGIS
  60. 60. ArcGIS supports IT standards OGC Client REST / HTTP XML UNICODE SQL Javascript Android IPv6 Flash iOS Silverlight WP7 C++ .Net (C#, VB) Geocoding Network analysis Java Editing Image processing Python Routing Spatial Analysis WMWare Querying Spatial Statistics Windows MS Hyper-V UNIX Amazon EC2 Linux MS Azure
  61. 61. Data for everyone Isaac Medel
  62. 62. Thank You