Download to read offline
Uploaded bySafe Software
Extracting Hierarchies with Recursive Tree Traversal Using FME
The document discusses the challenges and solutions in extracting hierarchies from a complex KMZ file using FME tools, specifically focusing on recursive SQL techniques and the InlineQuerier feature in FME 2018. It highlights the importance of preserving semantic information during the conversion process and demonstrates a structured approach to managing deep folder hierarchies. The document also includes examples and references for utilizing recursive queries effectively.
More Related Content
Extracting Hierarchies with Recursive Tree Traversal Using FME
- 1. Extracting Hierarchies withRecursive Tree Traversal using FME Reed Whittington
- 2. FLIGHT PLAN ● Introductions ●Dallas - We have a problem!! ● FME 2018 to the Rescue ● Trees are everywhere ● SQL WITH RECURSIVE made easy ● With the FME 2018 InlineQuerier ● Extracting name/values from <tables> ● Demo – Q/A
- 3. Deep FME History •Safe Integration Partners since 2006 • 5+ FME Server Developers, 20+ FME Users • FME Internals Plugin development • Packaged FME Server Solutions/Projects • FME Custom Transformers • FME Custom Formats • Advanced ETL workflows • Master Data Management • App/Data Integrations • Professional Staff
- 4. Dallas we havea Problem yes, this happens You have been given a very large (~ 400MB) kmz file to convert to shapefile(s) to be loaded into GIS. The kmz source has been aggregated with the Google Earth “My Places” tools. This kmz file has ~4000 folders and ~400 kml documents with a very deep document/folder hierarchy; over 7 levels deep in some places. The folder hierarchy contains important semantic information that needs to be saved as an attribute to be used for folder creation or feature classification.
- 5. Read Read the SourceTree Data structure KML doc/folders are just an example. Trees are everywhere Recursion is a natural solution Recursive SQL used to be tricky. inlineQuerier A Path To Success railroads/crossings/bridges railroads/crossings/underground railroads/tracks railroads/switches Sometimes we need a path string Traverse This approach Scales The forest of trees has been broken down to manageable parts all within one .fmw Process FME 2018 to the Rescue Hierarchical (parent/child) relationships are very common, and can be a challenge to extract paths without custom or brute force approach.
- 6. Trees Are Everywhere Ageneral approach to extracting and constructing parent-child-hierarchies is needed. A directory (folder) tree: railroads/crossings/bridges railroads/crossings/underground railroads/tracks railroads/switches Org Charts/Graphs: Single Level (queue): …
- 7.
- 8. parent-child-hierarchy data structure idparent_id seq name url_path 1 1root 2 1 1Diagram diagram 3 1 2My models my_models 4 1 3Share requests share 5 1 4My account account 6 1 5Invite people invite 7 1 6Help help 8 7 1Documentation doc 9 7 2FAQ faq 10 7 3Ask a question ask 11 7 4Request a feature feature 12 7 5Report a problem problem 13 7 6Keyboard shortcuts shortcuts 14 8 1Section 1 section1 15 8 2Section 2 section2 16 8 3Section 3 section3 fanout results Dir> tree Input table
- 9. InlineQuerier CodePath SELECT WITH RECURSIVE <CTE> CTE: Common Table Expression InputTable Recursive iteration Terminal condition level==0
- 10. Extracting name/values fromhtml tables
- 11.
- 12. Tips/Tricks Powerful tools inyour FME toolbox • SQL/inlineQuerierRegular Expressions • CTs/Python inlineQuerier A lot of power packed in one transformer AND ANOTHER CTE: Common Table Expressions Extend the power of the inlineQuery with materialized tables and recursion
- 13. REFERENCES ● https://www.safe.com/transformers/inline-querier/ ● http://www.sqlite.org/lang_select.html ●https://www.essentialsql.com/recursive-ctes-explained/ ● https://academy.vertabelo.com/blog/do-it-in-sql-recursive-tree-traversal/ ● http://www.sqlfiddle.com All product names, logos, and brands are property of their respective owners.
- 14.