To meet increasing and diversified user needs for geographic information, Natural Resources Canada (NRCan) must produce and maintain geographic data at multiple scales. To automate the generalization process NRCan is using an approach based on FME and MetaAlgorithms.

1. Using FME for Topographical
Data Generalization at Natural
Resources Canada
Daniel Pilon
Senior project officer at NRCan

2. Overview
 Map generalization at NRCan
 Principles of good map generalization
 ETL generalization patterns

3. Map Generalization
The process of reducing detail on a map
Very complex: manually or automatically

4. Generalization Paradigm Shift
Art Craft
Process
Commodity
Process
 Manually
 Comply to very strict specifications
 Prepackage products 250K, 1M, 7.5M
 Finality on its own
 Automatically (faster turnaround)
 Less complex specifications
 On demand scale production
 Integrated into other products

5. Generalization Paradigm Shift
Example WMS
Web Mapping Service
• 1 : 30 000 000
• 1 : 7 500 000
• 1 : 1 000 000
• 1 : 250 000
• 1 : 50 000

6. Generalization Paradigm Shift
Example WMS
1 : 40 000 000

7. Generalization Paradigm Shift
Example WMS
1 : 5 000 000

8. Generalization Paradigm Shift
Example WMS
1 : 1 000 000

9. Generalization Paradigm Shift
Example WMS
1 : 119 000 (serving 1: 50 000) 1 : 121 000 (serving 1: 250 000)
Threshold: 1: 120 000

10. FME for Generalization…Why?
 Few generalization solutions
 No complete generalization solution
 Some European solutions
 Very complex software to use…
 Tackle European problems

11. FME for Generalization…Why?

12. FME for Generalization…Why?
 Few generalization solutions
 No complete generalization solution
 Some European solutions
 Very complex software to use…
 Tackle European problems
 FME as a generalization tool…
 General purpose spatial manipulation engine
 Reliable and fast
 Easily add extensions (python)

13. Overview
 Map generalization at NRCan
 Principles of good map generalization
 ETL generalization patterns

14. The principles of Good
Genralization Automation
 Simple and efficient decomposition of the
tasks performed by the cartographers
 Objective tools to characterize map objects
 Tools to edit modify the spatial objects
 Reactive control on the map object’s state
during the process
 Unambiguous rules in order to guide the
process
Generalization
cases
Measure
Behaviour
pattern
Operator
Constraint

15. Generalization Case
 It is a formalization of the cartographer’s
knowledge (generalization rules) and a
communication tool between the cartographer
and the software engineer
 Generalization cases do not tell us what
algorithms or what parameters to use

16. Generalization Case example
 For the Vegetation
 For all the Vegetation which breaks the constraint
Dimension: Minimum area
 If the Vegetation breaks the constraint: Position proximity
 Aggregate the area
 Else
 Eliminate the area

17. The principles of Good
Genralization Automation
 Simple and efficient decomposition of the
tasks performed by the cartographers
 Objective tools to characterize map objects
 Tools to edit modify the spatial objects
 Reactive control on the map object’s state
during the process
 Unambiguous rules in order to guide the
process
Generalization
cases
Measure
Behaviour
pattern
Operator
Constraint

18. Measures
 Measures are used:
 To calculate the characteristics of map objects (or
group of map objects )
 Before the generalization: to know how to
generalize
 After the generalization: to assess the success of
the applied generalization operations

19. Measure Examples
Unary measures
Length Area/Perimeter Circularity ratio
Bounding box
Oriented
Bounding box
Triangulation

20. Measure Examples
 Binary measures
Spatial relationship Distance Triangulation

21. Measure Examples
 N-ary mesaures H1
S1
H1
S1
H3
S1
H2
S1
H1
S1
H1
S1
H1
S1
H2
S1
H1
S1
H3
S3
H3
S3
H3
S2
H2
S2
H3
S2
H2
S2
H2
S2
Stream ordering (Strahler or Horton)

22. The principles of Good
Genralization Automation
 Simple and efficient decomposition of the
tasks performed by the cartographers
 Objective tools to characterize map objects
 Tools to edit the spatial objects
 Reactive control on the map object’s state
during the process
 Unambiguous rules in order to guide the
process
Generalization
cases
Measure
Behaviour
pattern
Operator
Constraint

23. Generalization Operators
 Generalization operators are typical transformations
applied on spatial objects. They allow the decomposition
of the generalization process into several sub-problems in
order to manage complexity
 Generalization operator are implemented using
different generalization algorithms. Ex.: For line
simplification: Douglas & Peucker, Lang, Sherbend

24. Generalization Operator
Examples
 Selection
 Simplification
 Smoothing
 Collapse

25. Generalization Operators
Examples
 Aggregation
 Typification
 Displacement

26. The principles of Good
Genralization Automation
 Simple and efficient decomposition of the
tasks performed by the cartographers
 Objective tools to characterize map objects
 Tools to edit modify the spatial objects
 Reactive control on the map object’s state
during the process
 Unambiguous rules in order to guide the process
Generalization
cases
Measure
Behaviour
pattern
Operator
Constraint

27. Constraints
 Constraints are rules applied to data in order to comply
with requirements of the target map specifications (ex:
An area must respect the minimum size threshold)
 Constraint are used:
 To trigger a generalization operation
 To assess the result of a generalization operation
 To achieve generalization, a feature must fulfill several
constraints

28. The principles of Good
Genralization Automation
 Simple and efficient decomposition of the
tasks performed by the cartographers
 Objective tools to characterize map objects
 Tools to edit modify the spatial objects
 Reactive control on the map object’s state
during the process
 Unambiguous rules in order to guide the
process
Generalization
cases
Measure
Behaviour
pattern
Operator
Constraint

29. Generalization Behaviour
 Provides the link between constraints, generalization
operator and measures
 Implements mechanism for choosing algorithms and
parameters
 Implements mechanism for choosing alternate
scenarios in case of constraint failure

30. Generalization Behaviour
Example for Line Simplification
Measures on
the feature
Determine the
algorithm to use
Asses the results
through a set of
constraints
Success
Apply the
algorithm on the
feature
Reset to the original geometry
Determine an alternate
algorithm and/or parameters
Failed
ETL Generalization
Patterns

31. Overview
 Map generalization at NRCan
 Principles of good map generalization
 ETL generalization patterns

32. ETL Generalization Patterns
 General and reusable solution to a commonly
occurring problem in generalization when used
in an ETL context
 Three stages pattern
 Meta-algorithm pattern

33. Three Stages Pattern
 Used when generalization operator(s) are
applied on different part of the same feature
 Solution:
1) Create a pseudo-object representing each
generalization operation to be done on the real
objects
2) Evaluate the constraint against the pseudo-objects
created in 1
3) Apply on the real objects the pseudo-objects that
meet the constraint evaluated in 2

34. Three Stages Pattern Example
 Example with amalgamator operator
 If you implement a one stage process

35. Three Stages Pattern Example
 Example with amalgamator operator
 If you implement a three stages process
 First create the amalgamation zones
 Second you remove unwanted zone
 Dissolve the area

36. Meta-Algorithm Pattern
 Used when it is too complex to program the
behaviour and the constraint in an ETL
 Solution:
1) Select the algorithm(s)
2) Select the constraint(s) to implement
3) Select a behaviour pattern
4) Implement the solution in a high level language
5) Wrap the solution in a transformer

37. Meta-Algorithm Pattern Example
 Example with Sherbend
 Implementation of the Wang algorithm

38. Line simplification
Basic generalization
operation
We all thought of the
Douglas-Peucker
algorithm to solve that
problem…
Does it simulate the work
of the cartographer well
enough?
An example with
contours
Meta-Algorithm Pattern Example

39. Contour line
simplification
Red: Original contours
Black: Contours with DP
50m
Meta-Algorithm Pattern Example

40. Contour line
simplification
Black: Contours with DP
50m
The results are
unacceptable
Does it simulate the
work of the
cartographer?
No
Douglas-Peucker line
simplification is not doing
a good job of
generalizing natural
features
Meta-Algorithm Pattern Example

41. Contour line
simplification
Red: Original contours
Black: Contours with
Sherbend algorithm with
a parameter of 150m
The Sherbend algorithm
will remove the bends
below the tolerance and
keep the ones above it
Meta-Algorithm Pattern Example

42.  Example with Sherbend
 Implementation of the Wang algorithm
 Implementation of constraints
 Self intersection
 Line crossing
 Sidedness
Meta-Algorithm Pattern Example

43.  Implementation of constraints
 Self intersection
 Line crossing
 Sidedness
Meta-Algorithm Pattern Example

44.  Example with Sherbend
 Implementation of the Wang algorithm
 Implementation of constraints
 Self intersection
 Line crossing
 Sidedness
 Implementation of a scenario to resolve conflicts
(behaviour pattern)
Meta-Algorithm Pattern Example

45.  Scenario to resolve conflitcs
 Implementation of an iterative process to
resolve constraints
Meta-Algorithm Pattern Example

46.  Example with Sherbend
 Implementation of the Wang algorithm
 Implementation of constraints
 Self intersection
 Line crossing
 Sidedness
 Implementation of a scenario to resolve conflicts
(behaviour pattern)
 Creation of a new transformer
Meta-Algorithm Pattern Example

47. Meta-Alorithm Pattern
 Meta-Algrorithms can be developed by user in FME
 FME Python extension (Python caller)
 Open source libraries
 Shapely for spatial manipulation and spatial
relationship (http://pypi.python.org/pypi/Shapely)
 RTree for spatial indexing (http://pypi.python.org/pypi/Rtree/)
 Already implemented the following meta-algorithms
 Douglas Peucker
 Convex
 Spike
 Smoothing
 Sherbend

48. Future Works
 Develop new generalization patterns applied to ETL
 Develop new meta-algorithms
 Start the generalization of the hydrographic
network

49. Thank You!
 Questions?
 For more information:
 Daniel Pilon
 Natural Resources Canada
 dpilon@nrcan.gc.ca
 WWW.GEOGRATIS.GC.CA
 WWW.GEOBASE.CA