Brief Introduction to ETL, showing: - its advantages - differences from normal applications The Live Talend Demo was given at Software Niagara's DevTricks on Oct. 17, 2016. If you want to learn more about ETL, please contact me.

1. Introduction to ETL
by
Maira Bay de Souza

2. About the Author
Maira Bay de Souza, BSc. Comp.
Science
Working with
● software testing
● software development
since 2001
Talend ETL developer and tester since
2013
IBM, HP, SunLife, small businesses

3. What is ETL?
● Extract, Transform, Load
● Sequence of operations on the same dataset
● Sometimes joining datasets together in T
● Simple Transformations may be done in E, L

4. Extract
Read any kind of static data source:
● Extract data from a website (HTML, JSON,
RSS, etc)
● Read files from a server (FTP, SCP, etc)
● Query a RESTful API
● Read from a database
● Read from a cloud storage unit: GoogleDrive,
GoogleStorage, AWS, DropBox, etc
● Read data from common business applications:
SAP, SalesForce, SugarCRM, etc

5. Transform
Make operations on data as a whole:
● Split names into first, middle, last
● Filter out people with blank addresses
● Sort employees by % of sales target achived
● Join data from an excel file and a database
● Find duplicate names using Levenstein
● Normalize or denormalize list of addresses
● Split postal code based on Regex
● Validate XML with XSD

6. Load
Output data in any kind of format:
● Save a CSV, XML, etc
● Insert or Update a table in a database
● Send a file in an email
● Make a JSON available through a RESTful API
● Save data on a cloud storage unit:GoogleDrive,
GoogleStorage, AWS, DropBox, etc
● Save data on common business applications:
SAP, SalesForce, SugarCRM, etc

7. Example applications
● Find twitter followers who are not facebook
followers and make their names and logins
available on a JSON via RESTful API
● Join employee names from HR database with
sales records from CRM and send weekly
email to CMO with names and progress
towards sales target

8. Difference between ETL and
WebApp
WebApp
● Reads one or more user inputs
or actions:
– forms filled
– button clicked
– etc
● Produces a result:
– page updated
– page loaded
– etc
ETL
● Reads one or more data inputs:
– table from database
– pages from RSS feed
– etc
● Produces another data output or
action:
– send email
– create Jasper Report
– etc

9. Tools
Tools:
1)Talend Open Studio
2)Pentaho Spoon/PDI (previously Kettle)
Features:
1)Free
2)Easy-to-use
3)Powerful

10. Demo
Live Demo:
creating a Talend ETL job

11. Questions/Requests
Questions???
I'd be glad to answer any of your ETL questions and requests. Click
here to schedule a complimentary 15-min skype voice call.

12. Licensing
This presentation is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivatives 4