Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

FME Delicacies: Tasty Examples from the City of St. Albert


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

FME Delicacies: Tasty Examples from the City of St. Albert

  1. 1. FME Delicacies:Tasty Examples from the City of St. Albert<br />Tammy Kobliuk<br />GIS Coordinator<br />
  2. 2. On the Menu:<br />Appetizer: 2010 Municipal Census<br />Salad Course: Business Licensing<br />Main Course: Fire Services 9-1-1 Dispatch<br />Dessert: Questions? (you’re all on a diet)<br />
  3. 3. Appetizer<br />Municipal Census<br />
  4. 4. Appetizer: 2010 Municipal Census<br />The challenge for the kitchen:<br />City-conducted municipal census<br />All residences within City boundaries<br />First on-line census<br />No previous spatial census-base layer available<br />No spatial address point layer available<br />No single address database available<br />All residences require a secure PIN number<br />All residences will be mailed their PIN number<br />A complete residential address listing is required<br />
  5. 5. Available Ingredients<br />Census Landbase<br />Postal Codes<br />Enumeration Areas<br />Enumeration<br />Districts<br />
  6. 6. Desired Outputs<br />Primary and secondary PIN numbers for each address<br />A mail merge database including postal codes and PIN numbers<br />A list of addresses requiring in person PIN letter delivery<br />A list of addresses requiring institutional enumeration<br />A final GIS census polygon landbase including assigned PIN numbers<br />
  7. 7. Finding the Right Recipe<br />The goals for the PIN numbers:<br />No less than 5 characters long<br />No more than 8 characters long<br />Primary and secondary PINs<br />No duplicate PINs<br />Repeatable method<br />No confusing characters (e.g. 0 vs O)<br />
  8. 8. Trial 1: Using Existing Info<br />The City of Airdrie did it so why can’t we?<br />Available Info:<br />Street names<br />Legallot ID numbers<br />Centroid X/Y coordinates<br />Neighbourhood names<br />Strip them down, mash them together, in short make an unproductive mess…<br />
  9. 9. Sampling Ingredients<br />1. Input Landbase<br />5. Generate random 9-digit number<br />2. Fetch CentroidCoodinates<br />6. Generate single random number<br />3. Trim off decimal places<br />7. Extract first letters<br /> of street names<br />4. Replace decimal<br />
  10. 10. And It Tastes Like?<br />Ummm…best not to be sampled:<br />Generated duplicates<br />Complicated at best<br />Seemed like it would take more work than it was worth to get it to work<br />Like Plan B would be better: why use existing ingredients when you can just create some tasty new ones?<br />
  11. 11. Plan B: Just Random<br />Use individually generated random numbers that match the ASCII character designations for capital letters A to Z.<br />Stick with just letters instead of mixing in numbers<br />Generate 7 individual letters<br />Combine in order for the initial 7-digit PIN<br />Recombine to create an 8-digit PIN<br />Little chance of generating duplicates as there are now 26 choices, instead of 10, for each character<br />
  12. 12. Appetizer: The Model<br />1. Generate<br />Random<br />Letters<br />2. Create first PIN<br />3. Check for duplicates<br />4. Create second PIN<br />5. GIS Output<br />
  13. 13. Appetizer: The Model (more yummies)<br />7. Address merge<br />file for Admin staff<br />(Access table)<br />6. Addresses<br />must be sorted<br />In ascending order of:<br /><ul><li>Enumeration Area
  14. 14. Enumeration District
  15. 15. Street
  16. 16. House number
  17. 17. Suite number</li></ul>8. Addresses for<br />hand delivery of<br />PIN letters<br />(Access table)<br />9. Addresses for<br />City staff enumeration<br />(Access table)<br />11. Blank database<br />for census contractor<br />(CSV file)<br />10. Get rid of <br />offending commas<br />
  18. 18. Appetizer in Review<br />We went with simple instead of fancy for PIN number generation.<br />We chose to optimize our development time.<br />We had difficulties in joining postal codes to address points in FME so ended up doing that in ArcGIS as pre-processing.<br />Overall development time was approximately 1.5 days, including initial sandbox time.<br />Very happy with final result.<br />
  19. 19. SALAD COURSE<br />Mapping Home-Based Businesses<br />
  20. 20. Salad Course:Mapping Business Licenses<br />The challenge for the kitchen:<br />To map the home-based business licenses<br />The license database is not spatial<br />The license database is address-based<br />We don’t have a spatial point address layer<br />We do have a census landbase<br />Address format structure does not match between the two databases<br />We wish to have an automated process for producing updated business license datasets<br />
  21. 21. Available Ingredients<br />Census Landbase<br />Business License<br />Database<br />NAICS Code<br />Lookup Table<br />
  22. 22. Desired Outputs<br />GIS point layer of home-based businesses<br />Business trade name<br />Address (full and parsed)<br />License number<br />License category/subcategory<br />NAICS code<br />NAICS code full description<br />NAICS sector<br />NAICS class<br />
  23. 23. Salad Course: The Model<br />Create address points;<br />Match to BL data<br />Create address<br />join string<br />Merge with NAICS<br />lookup table<br />
  24. 24. Resulting Delicacy<br />
  25. 25. Salad Course in Review<br />Overall development time was approximately 5 days, including initial sandbox time.<br />Very happy with final result. Have already updated the data with the latest monthly data dump.<br />Challenges:<br />Understanding the tools<br />Chaining multiple models together<br />Parameterizing the model for flexibility<br />
  26. 26. Main Course<br />Fire Services 9-1-1 Dispatch<br />
  27. 27. Main Course:Fire Services 9-1-1 Dispatch<br />The challenge for the kitchen:<br />9-1-1 calls come in from Telus<br />Telus feed has specifically formatted addresses<br />9-1-1 addresses must be structured to match Telus feed data as matching is automated<br />9-1-1 system requires address point dataset<br />All non-relevant addresses are to be stripped<br />Each address must have a unique ID, repeatable through each data load<br />Addresses cannot be duplicated<br />The City does not have a spatial address dataset<br />
  28. 28. Available Ingredients<br />Parcel Polygons<br />Townhouse Polygons<br />Apartment/Condo<br />Suites<br />Additional Address<br />Points<br />
  29. 29. Desired Outputs<br />GIS address file<br />Shapefile format<br />Point feature type<br />Containing only addresses that will be referenced by Telus<br />Containing only those fields required by CriSys<br />Formatted to match Telus feed<br />
  30. 30. Challenge for the Kitchen<br />Must be repeatable<br />Must be automated as much as possible<br />Must be able to easily QC for duplicate IDs and Addresses<br />Must be fast<br />Must be flexible to deal with changing inputs<br />
  31. 31. Main Course: The Model<br />
  32. 32. Building the Crust<br />Convert polygons to points<br />Filter out non-relevant addresses<br />Filter out Null addresses/IDs – multiple methods required<br />
  33. 33. Building the Crust<br />Convert UPPER CASE street names to Sentence Case<br />Fix McKenney Avenue (inner capital letter)<br />Adjust spelling of key road names to match CriSys/Telus<br />
  34. 34. Building the Crust<br />Checking for duplicate IDs<br />Filtering out or dealing with duplicate IDs<br />Filter out specific addresses for replacement<br />
  35. 35. Building the Crust<br />Converting LegallotID value to CriSys ID<br />Automated removal of duplicate addresses<br />Output QC datasets for parcel addresses and duplicates<br />
  36. 36. Making the Filling<br />Convert townhouse polygons to points<br />Change UPPER CASE street names to Sentence Case<br />Fix McKenney Ave case<br />
  37. 37. Making the Filling<br />Filter for addresses with/without suite numbers<br />Create unique ID numbers from LegallotID, suite and house numbers<br />Deal with two specific problem properties<br />Output QC dataset<br />
  38. 38. Making the Topping<br />Converting multi-story suite polygons to points<br />Change UPPER CASE street names to Sentence Case<br />Fix McKenney Ave case<br />
  39. 39. Making the Topping<br />Create final ID string from LegallotID, house number and suite number (when applicable)<br />Output QC dataset<br />
  40. 40. Putting It All Together<br />Adding in the extra points<br />Streaming all the outputs into a single final address point file<br />
  41. 41. Main Course: The Model<br />
  42. 42. Resulting Delicacy<br />
  43. 43. Main Course in Review<br />This was actually our first (and most complex) FME project<br />This is the second iteration of the model<br />Taking the course helped in refining the model (thanks!)<br />It’s a live model and is expected to have to constantly evolve to deal with data oddities<br />The model is currently pretty robust<br />Challenges were mostly related to discovering and dealing with data oddities<br />
  44. 44. A Quick Desert<br />Recap and Thoughts<br />
  45. 45. Recap and Thoughts<br />We LOVE FME!<br />The software is intuitive<br />It’s also powerful and complicated<br />It definitely meets our needs<br />We see it playing a key role in data integration<br />We use it from soup to nuts – everything from simple field rearranging to complex translation and integration models<br />We see it being useful for more than just spatial information<br />
  46. 46. Thank You!<br />Questions?<br />For more information:<br />Tammy Kobliuk<br />City of St Albert<br />Enter other resources<br />