Kalyan K. Janakiraman
Land and Property Information / Macquarie University
kalyankj@gmail.com
Prof. Mehmet A Orgun
Department of Computing
Macquarie University, NSW
mehmet@comp.mq.edu.au
Dr. Abhaya Nayak
Department of Computing
Macquarie University, NSW
Abhaya@comp.mq.edu.au
Relevance
Geospatial Applications are becoming main
stream decision support Systems
Growing demand for accurate current geographic
information in critical sectors
Expensive and difficult to update and maintain
To explore for technical solution that could increase geo-spatial
data legal compliance, currency, accuracy, and efficiency of
updates as well as address system scalability
Objective:
LPI’sSpatialInformationExchange(SIX)portal
http://lite.maps.nsw.gov.au/
Tries to provide
current
information for
business and
Government of
NSW
DataandJurisdictionalDomains
NSW Land & Property Authority (LPI)
- Cadastral Lots feature class
- Address points feature class
- Elevation contours feature class
RTA / Dept of transport
- Road centre lines
- Road Corridors
- Roads under repair/restricted
-Traffic signals, operations, Bus routes
County / Dept of Planning
- Land-use Planning zones
-Planning height control/FSR restrictions
County / local Government Authority
- Suburbs boundaries
- State/Federal electoral boundaries
- LGA Boundaries
Data that makeup what we see is from many data layers and is distributed vertically
among many government agencies and horizontally among many different administrative
domains.
GeographicData–Transactionalperspective
• Typically, many DBMS transactions per geospatial
transactionLarge Atomicity:
• commonly takes many days to prepare and commit one
geospatial transaction
Multi-session,
Multi-user, long
transaction
• Direct – no concern for geospatial integrity
• Versioned– commonly used in most production
environments. 4 states: create version, edit, fix
conflicts (reconcile) and commit (Post)
• Local session -- OpenStreetMap uses
Versioned
Updating
GeospatialTransactionexample
A case of a release of a portion of forest-land for residential development.
Managed by State Forest Authority
Managed by LPI
Managed by Dept of Transport
To update consistently, 3 layers held by 3
departments needs to be modified as a
single geospatial transaction.
Version beginning:
Changes to State Forest Layer
Changes to Cadastral Layer
Changes toTransport Layer
Version end
GeographicTransactionsinNSW–2month
snapshotfromLPI
▪ 7756 geospatial transactions were recorded in LPI comprising 60 layers.
▪ One layer geospatial transaction: Only111occurrences
▪ Average: 9 layers, 118 changes per geospatial transaction.
▪ Maximum: 40 layers per geospatial transaction
▪ About 25% of edits (230,000 edits) - other agency data (unauthorized / redundant).
0
500
1000
1500
2000
2500
3000
3500
1
2to9
10-19
20-29
30-39
40-49
50-59
60-69
70-79
80-89
90-99
100-109
110-119
120-129
130-139
140-149
150-159
160-169
170-179
180-189
190-199
200-209
210-219
220-229
230-239
240-249
250-259
260-269
270-279
280-289
290-299
300+
Frequency
database transactions per geospatial transaction
LPI
State Forest Department
PresentUpdatingWorkflow(Silo)
Create
Version
Edit Reconcile Post
Dept of Transport
Each Agency is a silo. Each may be doing the same unaware of the other
agency’s activity
Serialized geospatial transaction distribution
▪ Each geospatial transaction captured and applied to other repositories serially
▪ Serializability property does not hold. Each geospatial transaction need to be
applied in the same order as they occurred in the source repository.
PostedVersion
PostedVersion
PostedVersion
Tn
Agent Handler
Posted
VersionPosted
VersionPosted
Version Tn-3
http-soap/
MQ/ftp transport
Time
Node1 Node2
Janakiraman, K. K; Hansen, L, Orgun, M; "Failed-tuple blocking strategy for near-real time geospatial database
replication," Proceedings of the First conference on Computing for Geospatial Research & Application InternationalConference
on , vol., no., pp.__, 21-23 June 2010.
Failure handling:
LPI
Versionedfederatedgeo-databaseworkflow
State Forest Dept
Create
Version
Edit Reconcile Request
Post
Dept ofTransport
Verify /
Change
Verify /
Change
Accept
Post
Accept
Post
Accept
Post
State Forest Dept
LPI
VersionedFederatedgeo-database(Crowdsourced)
Create
Version
Edit Reconcile Request Post
Dept of Transport
Verify / Change Accept
Post
Accept
Post
Accept
Post
{Public residents}
Verify /
Change
Accept
Post
Verify / Change
Geo-database scaling:
PostedVersion
PostedVersion
PostedVersion
Tn
Agent Handler
http-soap/
MQ/ftp transport
Time
Node-1
Node-2 Node…
Node-n
Handler
Handler
Handler-n
- Node synchronisation by serialised geospatial transaction propagation
- Will be accurate to last but one Committed geospatial transaction
replicated.
Lot
Boundary
WS
Cloudgeo-databaseDataaccessarchitecture
Roads
Centreline
WS
REST Style
Web-services (one for each layer)
Forest
Boundary
WS
Version
Controller
WS
Get
Get
Get
Create/
Reconcile/
RequestPost
Put/
post/
deleteCharacteristics:
• Data access only through web-services
• WS stateless
• WS rapidly scalable
• WS simple and pure REST style
Notification feed
Queue
@Entity
@Table(name = “suburb")
public class Suburb {
@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)
@Column(name = "gid")
private Long id;
@Column(name = "name")
private String name;
@Column(name = "the_geom")
@Type(type =
"org.hibernatespatial.GeometryUserType")
private MultiPolygon geometry;
public MultiPolygon getPolygon() {
return geometry;
}
}
GET
Envelope box = //Window rectangle;
List<RLSuburb> stnList = (List<RLSuburb>)RLSuburbDao.get(box);
for (RLSuburb event : stnList) {
System.out.println(event);
}
GET OPERATION – EXAMPLE FOR SUBURB LAYER
Output:
MULTIPOLYGON (((30 20, 10 40, 45 40, 30 20)),…))
…
Cloudgeo-database:Versionmanagement
Version_id Version_name
Version_id layer Feature_id timestamp Owner_decision timestamp
Unique
ActiveVersions
Active changes
Version
Post Background
Agent
Commit is initiated when OWNER_DECISION
For all the changed feature for a version = POST
{RequestPost, notified, Reject, Post}
OverallCloudGeo-databaseArchitecture
▪ Cloud as a Database Service – agencies subscribe.
▪ one unique URL per geospatial Layer.
▪ Write access to custodian Agency.
▪ Read access to all subscribing agency.
▪ versioning managed by cloud.
▪ Any layer can participate in versioned updating.
▪ Non-authorised updates would propagate to the responsible
authority for consideration for update.
▪ Scaling through node synchronisation by serialised geospatial
transaction propagation.
▪ Authentication/Access control for agency+user combination

Up to-date geographic data management

  • 1.
    Kalyan K. Janakiraman Landand Property Information / Macquarie University kalyankj@gmail.com Prof. Mehmet A Orgun Department of Computing Macquarie University, NSW mehmet@comp.mq.edu.au Dr. Abhaya Nayak Department of Computing Macquarie University, NSW Abhaya@comp.mq.edu.au
  • 2.
    Relevance Geospatial Applications arebecoming main stream decision support Systems Growing demand for accurate current geographic information in critical sectors Expensive and difficult to update and maintain To explore for technical solution that could increase geo-spatial data legal compliance, currency, accuracy, and efficiency of updates as well as address system scalability Objective:
  • 3.
  • 4.
    DataandJurisdictionalDomains NSW Land &Property Authority (LPI) - Cadastral Lots feature class - Address points feature class - Elevation contours feature class RTA / Dept of transport - Road centre lines - Road Corridors - Roads under repair/restricted -Traffic signals, operations, Bus routes County / Dept of Planning - Land-use Planning zones -Planning height control/FSR restrictions County / local Government Authority - Suburbs boundaries - State/Federal electoral boundaries - LGA Boundaries Data that makeup what we see is from many data layers and is distributed vertically among many government agencies and horizontally among many different administrative domains.
  • 5.
    GeographicData–Transactionalperspective • Typically, manyDBMS transactions per geospatial transactionLarge Atomicity: • commonly takes many days to prepare and commit one geospatial transaction Multi-session, Multi-user, long transaction • Direct – no concern for geospatial integrity • Versioned– commonly used in most production environments. 4 states: create version, edit, fix conflicts (reconcile) and commit (Post) • Local session -- OpenStreetMap uses Versioned Updating
  • 6.
    GeospatialTransactionexample A case ofa release of a portion of forest-land for residential development. Managed by State Forest Authority Managed by LPI Managed by Dept of Transport To update consistently, 3 layers held by 3 departments needs to be modified as a single geospatial transaction. Version beginning: Changes to State Forest Layer Changes to Cadastral Layer Changes toTransport Layer Version end
  • 7.
    GeographicTransactionsinNSW–2month snapshotfromLPI ▪ 7756 geospatialtransactions were recorded in LPI comprising 60 layers. ▪ One layer geospatial transaction: Only111occurrences ▪ Average: 9 layers, 118 changes per geospatial transaction. ▪ Maximum: 40 layers per geospatial transaction ▪ About 25% of edits (230,000 edits) - other agency data (unauthorized / redundant). 0 500 1000 1500 2000 2500 3000 3500 1 2to9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99 100-109 110-119 120-129 130-139 140-149 150-159 160-169 170-179 180-189 190-199 200-209 210-219 220-229 230-239 240-249 250-259 260-269 270-279 280-289 290-299 300+ Frequency database transactions per geospatial transaction
  • 8.
    LPI State Forest Department PresentUpdatingWorkflow(Silo) Create Version EditReconcile Post Dept of Transport Each Agency is a silo. Each may be doing the same unaware of the other agency’s activity
  • 9.
    Serialized geospatial transactiondistribution ▪ Each geospatial transaction captured and applied to other repositories serially ▪ Serializability property does not hold. Each geospatial transaction need to be applied in the same order as they occurred in the source repository. PostedVersion PostedVersion PostedVersion Tn Agent Handler Posted VersionPosted VersionPosted Version Tn-3 http-soap/ MQ/ftp transport Time Node1 Node2 Janakiraman, K. K; Hansen, L, Orgun, M; "Failed-tuple blocking strategy for near-real time geospatial database replication," Proceedings of the First conference on Computing for Geospatial Research & Application InternationalConference on , vol., no., pp.__, 21-23 June 2010. Failure handling:
  • 10.
    LPI Versionedfederatedgeo-databaseworkflow State Forest Dept Create Version EditReconcile Request Post Dept ofTransport Verify / Change Verify / Change Accept Post Accept Post Accept Post
  • 11.
    State Forest Dept LPI VersionedFederatedgeo-database(Crowdsourced) Create Version EditReconcile Request Post Dept of Transport Verify / Change Accept Post Accept Post Accept Post {Public residents} Verify / Change Accept Post Verify / Change
  • 12.
    Geo-database scaling: PostedVersion PostedVersion PostedVersion Tn Agent Handler http-soap/ MQ/ftptransport Time Node-1 Node-2 Node… Node-n Handler Handler Handler-n - Node synchronisation by serialised geospatial transaction propagation - Will be accurate to last but one Committed geospatial transaction replicated.
  • 13.
    Lot Boundary WS Cloudgeo-databaseDataaccessarchitecture Roads Centreline WS REST Style Web-services (onefor each layer) Forest Boundary WS Version Controller WS Get Get Get Create/ Reconcile/ RequestPost Put/ post/ deleteCharacteristics: • Data access only through web-services • WS stateless • WS rapidly scalable • WS simple and pure REST style Notification feed Queue
  • 14.
    @Entity @Table(name = “suburb") publicclass Suburb { @Id @GeneratedValue(strategy=GenerationType.IDENTITY) @Column(name = "gid") private Long id; @Column(name = "name") private String name; @Column(name = "the_geom") @Type(type = "org.hibernatespatial.GeometryUserType") private MultiPolygon geometry; public MultiPolygon getPolygon() { return geometry; } } GET Envelope box = //Window rectangle; List<RLSuburb> stnList = (List<RLSuburb>)RLSuburbDao.get(box); for (RLSuburb event : stnList) { System.out.println(event); } GET OPERATION – EXAMPLE FOR SUBURB LAYER Output: MULTIPOLYGON (((30 20, 10 40, 45 40, 30 20)),…)) …
  • 15.
    Cloudgeo-database:Versionmanagement Version_id Version_name Version_id layerFeature_id timestamp Owner_decision timestamp Unique ActiveVersions Active changes Version Post Background Agent Commit is initiated when OWNER_DECISION For all the changed feature for a version = POST {RequestPost, notified, Reject, Post}
  • 16.
    OverallCloudGeo-databaseArchitecture ▪ Cloud asa Database Service – agencies subscribe. ▪ one unique URL per geospatial Layer. ▪ Write access to custodian Agency. ▪ Read access to all subscribing agency. ▪ versioning managed by cloud. ▪ Any layer can participate in versioned updating. ▪ Non-authorised updates would propagate to the responsible authority for consideration for update. ▪ Scaling through node synchronisation by serialised geospatial transaction propagation. ▪ Authentication/Access control for agency+user combination