FME is a feature manipulation engine that can be used for crossing different databases and file formats, data validation and analysis, cleaning processes, geocoding, report generation, and synchronization processes. It supports over 367 data formats and can be used to clean data by standardizing fields, detecting patterns and errors, and loading results into various file formats and databases. The document provides examples of using FME to cross Excel, CSV, and database files, identify merged and unmatched records, clean address data by normalizing case and accents, and synchronize data between databases.
4. Possibility of uses for IT
Crossing different databases
Crossing different types of files and databases
Validation and analysis of results
Cleaning processes
Validations standards
Process loads
Geocoding
Notifications
Report generation
Provision of services
Synchronization processes
Data export in different formats
7. Databases crossing with errors
Inconsistency at the intersection of database may
happen in these cases, you can identify these
points and treats them in a specific way, for
example:
Outputs:
• Merger - Records that were found in the two
bases;
• Not Merged - no records found
• Referenced / Unreferenced - unused records
in both analyzed database.
VIDEO DEMO CLICK HERE!
8. Crossing three databases
Based on Excel with São José dos Campos addresses
Database CSV with Jacarei addresses
• Facility at the intersection of several databases.
• Cross between tables and files.
• Crossing by direct or composite key key.
VIDEO DEMO CLICK HERE!
9. Problems with accents and standard
Data analysis
Streets and types without names
• Pattern Identification.
• Clerical errors.
• Null attributes.
• Different forms of writing.
• Different data models.
What to do
10. Data cleaning
Problems found in the address base:
• No pattern of uppercase and lowercase letters;
• accent errors;
• Full address is required for the division:
• Title;
• Address;
• City;
• State;
Before
After
11. • No pattern of uppercase and lowercase letters;
• Accent errors;
Data cleaning
VIDEO DEMO CLICK HERE!
12. Validation of data
• Nulls addresses;
• Validation type of addresses, for example, Avenue
Road, etc ...
VIDEO DEMO CLICK HERE!
13. Standardization
• Full address is required for the division:
• Title;
• Address;
• City;
• State;
VIDEO DEMO CLICK HERE!
14. Data load
The results of operations can be recorded more than 367
formats supported by FME
• Database
• Files
• services
15. The results will be stored in Postgres
Data load
VIDEO DEMO CLICK HERE!
16. Process base synchronism
FME allows after the completion of loading, the timing between the bases
with the help of some Transformers.
• Change Detector
• Sql Executor
• Matcher
• FeatureMerger
Specific functions of the Reader and Writer, assist in sync process
• Sql after and Before the translation;
• Where clause;
• Mode operation;