FME WT 2014: (ENG) FME 2014 in action

823 views

Published on

Presentation on FME World Tour 2014 in Mechelen hosted by GIM of Ken Bragg (Safe Software)
- Introduction to FME and FME2014
- Database Updating and Deleting
- Automation with FME Server
- Advanced Workflows
(DUTCH)

Published in: Software, Technology
  • Be the first to comment

  • Be the first to like this

FME WT 2014: (ENG) FME 2014 in action

  1. 1. CONNECT. TRANSFORM. AUTOMATE. FME World Tour 2014 Come Along on a Grand FME Adventure #FMEWT March 27 – May 23 Ken Bragg European Services Manager Safe Software
  2. 2. CONNECT. TRANSFORM. AUTOMATE. Let’s Go!  Introduction to FME and FME 2014  Database Updating and Deleting  Automation with FME Server  Advanced Workflows
  3. 3. CONNECT. TRANSFORM. AUTOMATE. Why we do what we do.  Free your data  Your data shouldn‟t be locked into systems, applications, or formats.  Your time is too valuable to be wasted fighting with data. FME is all about Freedom. Freedom to get the most value from data everywhere.
  4. 4. CONNECT. TRANSFORM. AUTOMATE. An explosion of formats! Always more Always faster.
  5. 5. CONNECT. TRANSFORM. AUTOMATE. Key Capability: Data Transformation ▪ Restructure and manipulate data contents
  6. 6. CONNECT. TRANSFORM. AUTOMATE. FME 2014
  7. 7. CONNECT. TRANSFORM. AUTOMATE.Best Practice FME Desktop Training 2012 FormatsFormats
  8. 8. CONNECT. TRANSFORM. AUTOMATE. New formats and systems – FME 2014  AIXM 5.x (Aeronautical Information Exchange Model GML version 5.x) (R/W)  Amazon DynamoDB (R/W)  Amazon Redshift (Tech Preview) (R/W)  Autodesk Revit (R)  CARIS Spatial Archive (CSAR) (R/W)  Esri ArcGIS Online Feature Service (R)  Esri Geodatabase: (ArcSDE Geodb Raster Catalog) (R)  Esri Geodatabase: (File Geodb Raster Catalog) (R)  Golden Software Surfer Binary Grid (R/W)  Google BigQuery (Tech Preview) (R/W)  Google Cloud SQL Non-spatial (R/W)  Google Cloud SQL Spatial (R/W)  Google Maps Engine (GME) Tables (R/W)  Google WebP (R/W)  INSPIRE GML (R/W)  Landmark Z-Map Grid (R/W)  Landmark Z-Map Vector (R)  OGP P1/90 Post Plot Positioning Data (R/W)  SeabedML (GML) (R/W)  Trimble SketchUp (R/W)  Virtual Reality Modeling Language (VRML97) (R)  Windows Azure SQL Database Non-spatial (R/W)  Windows Azure SQL Database Spatial (R/W)
  9. 9. CONNECT. TRANSFORM. AUTOMATE.Best Practice FME Desktop Training 2012 TransformTransform
  10. 10. CONNECT. TRANSFORM. AUTOMATE. New transformers (highlights).  MapnikRasterizer  WebSocketSender, WebSocketReceiver  DatabaseUpdater, DatabaseDeleter  GeographicBufferer  RasterAspectCalculator, RasterSlopeCalculator  S3Uploader, S3Downloader  VertexCreator
  11. 11. CONNECT. TRANSFORM. AUTOMATE. New transformers.  DatabaseDeleter  DatabaseUpdater  GeographicBufferer  JSONFragmenter  MapnikRasterizer  NullAttributeMapper  PointCloudComponentAdder  PointCloudComponentCopier  PointCloudComponentKeeper  PointCloudComponentRemover  PointCloudComponentRenamer  PointCloudComponentTypeCoercer  PointCloudExtractor  PointCloudReplacer  PointCloudTransformationApplier  RasterAspectCalculator  RasterSlopeCalculator  S3Downloader  S3Uploader  SNSSender  SQSReceiver  SQSSender  VertexCreator  WebSocketReceiver  WebSocketSender
  12. 12. CONNECT. TRANSFORM. AUTOMATE. Beginners Guide to Database Updating and Deleting
  13. 13. CONNECT. TRANSFORM. AUTOMATE. Writing to Databases - Inserts  Easy to INSERT data  Automatically INSERTs new data  Creates a table if it doesn‟t exist  Appends to an existing table  Simple settings for:  Drop Table or  Truncate Table
  14. 14. CONNECT. TRANSFORM. AUTOMATE. Writing to Databases - Updating/Deleting  Updating/Deleting existing data isn‟t so easy  Requires multiple parameter settings  Uses format attributes and writer „modes‟  Might involve multiple transformers  Can take some effort to get working  Prerequisites to incremental updates…  You need to know which features to insert / update / delete !
  15. 15. CONNECT. TRANSFORM. AUTOMATE. New Transformers  DatabaseUpdater  Features enter the transformer and it updates existing features  DatabaseDeleter  Feature(s) enter the transformer and it deletes existing features.
  16. 16. CONNECT. TRANSFORM. AUTOMATE. Formats Supported  Google Fusion  Access  SQL Server  Oracle  PostgreSQL/Postgis  SQL Lite  Windows Azure
  17. 17. CONNECT. TRANSFORM. AUTOMATE. Parameters  Attributes to Join on  Where clause option  Columns to Update
  18. 18. CONNECT. TRANSFORM. AUTOMATE. Pros  Very easy to configure  Great for relatively simple updates:  a few feature types  a small number of fields  Great for deleting based on an attribute value
  19. 19. CONNECT. TRANSFORM. AUTOMATE. Cautions  No Geodatabase support  No Transactioning or Commit/Rollback handling  Not so useful for updating lots of attributes  Not appropriate for Dynamic Workflows
  20. 20. CONNECT. TRANSFORM. AUTOMATE. Other Approaches  Use feature based updates  fme_db_operation  fme_where  See the KnowledgeBase article:  http://fmepedia.safe.com/articles/How_To/Using- Feature-Based-Attributes-for-Database-updates
  21. 21. CONNECT. TRANSFORM. AUTOMATE. FME Server Automation & Validation
  22. 22. CONNECT. TRANSFORM. AUTOMATE. Why Automate?
  23. 23. CONNECT. TRANSFORM. AUTOMATE. What is Automation Event Action
  24. 24. CONNECT. TRANSFORM. AUTOMATE. Scheduling
  25. 25. CONNECT. TRANSFORM. AUTOMATE. Directory Watch Demo Automatically convert DWG to Shape and email a link to download
  26. 26. CONNECT. TRANSFORM. AUTOMATE. Database Triggers
  27. 27. CONNECT. TRANSFORM. AUTOMATE. Email Trigger
  28. 28. CONNECT. TRANSFORM. AUTOMATE. Validation Workflows on FME Server
  29. 29. CONNECT. TRANSFORM. AUTOMATE. What is Real-Time Data? Data is delivered for processing as it is collected. 2 Classifications On-Demand Real-Time Complex Event Processing Low Data Velocity, Low-Mid Data Volume Notification Based Technology Continuous Real-Time Continuous stream of data High Data Velocity, Low-High Data Volume Dedicated Engine Technology
  30. 30. CONNECT. TRANSFORM. AUTOMATE. Real-Time Trigger Amazon SQS Amazon SNSWeb Sockets REST APIJava Messaging Service (JMS) UDP TCP/IP
  31. 31. CONNECT. TRANSFORM. AUTOMATE. Workflow Management for Multiple Workspaces
  32. 32. CONNECT. TRANSFORM. AUTOMATE. Workflow Management What are we talking about?
  33. 33. CONNECT. TRANSFORM. AUTOMATE. Workflow Management What are we talking about? Using the power of FME Workbench to design workflows  Moving and Copying Files  Data Transformation  Quality Control  Database Updates  Notification (Emails, Mobile Alerts)  Etc.
  34. 34. CONNECT. TRANSFORM. AUTOMATE. Workflow Management Why do we care?  Because many of you are doing this already!  Users have noticed that FME Workbench is a powerful process control tool  Many of you running complex workflows with multiple workspace already  FME Server Survey 2012 #1 Enhancement Request was “Automated Workflows”
  35. 35. CONNECT. TRANSFORM. AUTOMATE. Workflow Management: 2 Case Studies 1. “Call Before you Dig” Client (FME Desktop based) 2. Norwegian Environment Agency (thanks to VAR Norkart) Please note: Demos are very loosely based on these real projects
  36. 36. CONNECT. TRANSFORM. AUTOMATE. Workflow Management - Key Tools  Directory and File Pathnames Reader  File Copy Writer  WorkspaceRunner  FMEServerJobSubmitter  FMEServerNotifier  Notification Service
  37. 37. CONNECT. TRANSFORM. AUTOMATE. Case Study #2 Norwegian EA Goal: automatically add valid data from the SOSI-format to the ArcSDE operational database • geometry must be validated and inside the database before running any other database updates • split between geometryhandling, attributehandling, databasehandling • easily add other procedures. Currently working new one to take attributes from the features and stores these in another database. View the Application here: http://innsyn.naturbase.no/
  38. 38. CONNECT. TRANSFORM. AUTOMATE. Case Study #2 Norwegian EA SOS I-file ArcSDE Operational DB MSSQL Logging etc SOSI- file, PDF- report, text- report 1 2 3 4 Verfiy Data and Database updates Generate paths/published parameters based on attributes in SOSI-file To Do all checks: - Geometry - SOSI - Attributename/values - Exisiting Ids Create logfiles (in SOSI, txt, 2D / 3D PDF) and add geometry to ArcSDE All data verified and all logs updated To Do: - Wrap-up files from previous FMW- files into one package and send by e-mail to supplier
  39. 39. CONNECT. TRANSFORM. AUTOMATE. Case Study #2 Norwegian EA  Controller Workspace Running two consecutive tasks 1. Quality Control Workspace 2. Database Update Workspace  Notification at all stages (Success and Failure)
  40. 40. CONNECT. TRANSFORM. AUTOMATE. Where have we been!  Introduction to FME and FME 2014  Database Updating and Deleting  Automation with FME Server  Advanced Workflows
  41. 41. CONNECT. TRANSFORM. AUTOMATE. Thank You for Attending! For more information:  www.fmepedia.com  www.safe.com Contact: ken.bragg@safe.com FME Partner: We value your opinion.

×