Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

1Spatial: FME World Tour London: Postal address clean-up


Published on

Presentation given at the FME WT London

Published in: Technology
  • Login to see the comments

  • Be the first to like this

1Spatial: FME World Tour London: Postal address clean-up

  1. 1. Postal address clean-up Andrew Zolnai, Richard Mosley
  2. 2. Andrew Zolnai Support Specialist, Lead Development at Safe Software helped with this. Richard Mosley Geologist turned to GIS for over 30 years, covered all aspects of geo-data, project and business management in petroleum and volunteered geography.
  3. 3. Two non-spatial Workbench uses: 1) postal address clean-up 2) & electoral list updates
  4. 4. Help French 2017 presidential / parliamentary election campaign Using for a London UK based campaign Liste Electorale Consulaire is structured but inconsistent Ergo normalise 4 address columns into a schema The premise
  5. 5. Regex on Steroids 1Spatial FME WT 2015: scrape 4 years worth of playlists and tracks off the StrayFM website and categorise and rank the most played artists and tracks Inspired usage here:  StringSearcher search address components  AttributeSplitter split them into similar parts  AttributeManager re- order into one schema
  6. 6. Address normalization
  7. 7. Clean Rinse Repeat Get the first matches of address strings in the 4 address fields If string is empty then assign the next address string to it Country name is constant last string Build normalised string sets backward from it 01 02 04 03
  8. 8. Address schema ready to upload
  9. 9. But that’s not all… What about on-going updates to Electoral Lists?
  10. 10. FeatureMerger non-spatial too! Initial load:  find rejected addresses  repeat the procedure if possible Ongoing updates:  find the new entries as updated lists are received  repeat the procedure on the “delta” only
  11. 11. Refresh mega-data lists
  12. 12. Place your screenshot here Electoral lists for campaigns
  13. 13. Metadata is king Data never come clean Create schemas is key Automate? Web service? Tame mega lists
  14. 14. Thank you!