The METL process extracts meta data from a source system, transforms it to describe a different database structure, and loads it into a target system. This allows data to be accessed across systems with different structures without changing the source. Specifically, it extracts meta data from a legacy banking system, transforms it to work with a business intelligence tool, and loads it so the tool can query the legacy data through an SQL interface without knowing the source's structure. The METL process facilitates data sharing across different systems in a non-invasive way to prolong legacy systems' lives and provide access to production data.

1. The METL Process inThe METL Process in
Investment BankingInvestment Banking
An Introduction byAn Introduction by
Antony BenzingAntony Benzing

2. TheThe METLMETL ProcessProcess
What exactly is this?What exactly is this?
In short,In short, MMeta Dataeta Data EExtractionxtraction TTransformation andransformation and LLoad Process.oad Process.
Meta DataMeta Data is the data that defines the “shape” and structure of a database. Ais the data that defines the “shape” and structure of a database. A
Relational Database Management System (RDBMS) uses meta data in order to buildRelational Database Management System (RDBMS) uses meta data in order to build
the physical database on the server/mainframe. Typically it is held within a meta datathe physical database on the server/mainframe. Typically it is held within a meta data
repository, which may take many different forms.repository, which may take many different forms.
The METL Process starts with theThe METL Process starts with the extractionextraction of meta data from the meta dataof meta data from the meta data
repository of the “source” system.repository of the “source” system.
Secondly, the METL Process contains software thatSecondly, the METL Process contains software that transformstransforms the extracted metathe extracted meta
data which has the effect of producing new meta data which describes a databasedata which has the effect of producing new meta data which describes a database
with a different shape and structure from the original. However, the new databasewith a different shape and structure from the original. However, the new database
structure still refers to the data contained in the original.structure still refers to the data contained in the original.
Thirdly,Thirdly, loadingloading this new meta data into the meta data repository of a differentthis new meta data into the meta data repository of a different
“target” application gives that new application an “understanding” of the original“target” application gives that new application an “understanding” of the original
application’s database and access to it’s data.application’s database and access to it’s data.

3. The METL ProcessThe METL Process
Why do we need to do it?Why do we need to do it?
Because heritage systems running on proprietary and orBecause heritage systems running on proprietary and or
legacy; hardware, operating system and databaselegacy; hardware, operating system and database
cannot easily supply structured data to modern SQLcannot easily supply structured data to modern SQL
based Application Development and or Managementbased Application Development and or Management
Information/Business Intelligence tools.Information/Business Intelligence tools.
The METL process produces both a backward andThe METL process produces both a backward and
forward facing non invasive SQL interface layer thatforward facing non invasive SQL interface layer that
means neither system need have any knowledge of themeans neither system need have any knowledge of the
other’s database structure yet a full dialogue is possible.other’s database structure yet a full dialogue is possible.

4. Uses and OpportunitiesUses and Opportunities
Data and system migration.Data and system migration.
Outbound interfaces to down stream systems.Outbound interfaces to down stream systems.
Data extraction for Management Information andData extraction for Management Information and
Business Intelligence Systems.Business Intelligence Systems.
Data reconciliation between separate systems.Data reconciliation between separate systems.

5. Possible source/target systemsPossible source/target systems
ACBS (Advanced Commercial Banking System)ACBS (Advanced Commercial Banking System)
Loan IQLoan IQ
SummitSummit
Business ObjectsBusiness Objects
Microsoft Reporting ServicesMicrosoft Reporting Services
SASSAS
InformaticaInformatica
ORACLE FinancialsORACLE Financials
JDE financialsJDE financials
NOTE:
The entire point of the METL Process is to facilitate the passing of data between any systems that
have gone through METL Processing.
This means that systems can be both a source or target system depending upon your system
architecture.

6. Technical DetailTechnical Detail
By way of example let us consider the problem of building a ManagementBy way of example let us consider the problem of building a Management
Information System using Business Objects over ACBS.Information System using Business Objects over ACBS.
Data isData is “shaped”“shaped” by the database structures that contain it.by the database structures that contain it.
Consider a source system such as ACBS. Typically it runs on the iSeriesConsider a source system such as ACBS. Typically it runs on the iSeries
using the OS/400 proprietary DBMS and in certain areas is not fullyusing the OS/400 proprietary DBMS and in certain areas is not fully
compliant with standard database design methods (un-normalised). Thecompliant with standard database design methods (un-normalised). The
shape of this database can be considered as being slightlyshape of this database can be considered as being slightly “square”“square”..
Business Objects runs over standard relational SQL databases which weBusiness Objects runs over standard relational SQL databases which we
shall generalise as beingshall generalise as being “round”.“round”.
In essence then, the technical challenge is inIn essence then, the technical challenge is in ““trying to fit a square peg intotrying to fit a square peg into
a round hole”.a round hole”.

7. Technical DetailTechnical Detail
How does the METL process work?How does the METL process work?

8. Meta Data ExtractionMeta Data Extraction
Using the METL tool set the basicUsing the METL tool set the basic
database definitions (shape) of the sourcedatabase definitions (shape) of the source
system are extracted.system are extracted.

10. Meta Data TransformationMeta Data Transformation
Using the METL toolset together with the manual intervention of aUsing the METL toolset together with the manual intervention of a
designated design authority this meta data is altered.designated design authority this meta data is altered.
This enables remodelling of the design together with the creation ofThis enables remodelling of the design together with the creation of
new data types, long table names, long column names and foreignnew data types, long table names, long column names and foreign
key constraints to be imposed on the source system without alteringkey constraints to be imposed on the source system without altering
it.it.
This non invasive process has now transformed the meta data andThis non invasive process has now transformed the meta data and
describes a database with an all together different shape from thedescribes a database with an all together different shape from the
source.source.
Using the METL tool set This transformed meta data is used toUsing the METL tool set This transformed meta data is used to
create an SQL interface layer that faces both backwards into thecreate an SQL interface layer that faces both backwards into the
source system and forwards to the target system.source system and forwards to the target system.

12. Meta Data LoadMeta Data Load
The transformed meta data may now beThe transformed meta data may now be
loaded into the target system’s own metaloaded into the target system’s own meta
data repository.data repository.

14. The ResultThe Result
What does this achieve?What does this achieve?
In the case of loading into Business Objects it willIn the case of loading into Business Objects it will
generate a Business Objects Universe that allows thegenerate a Business Objects Universe that allows the
end user to query data multi dimensionally.end user to query data multi dimensionally.
Business Objects sends SQL requests to the interfaceBusiness Objects sends SQL requests to the interface
layer that in turn resolves these requests back to thelayer that in turn resolves these requests back to the
source system and forwards again to Business Objectssource system and forwards again to Business Objects
for end user presentation.for end user presentation.
Business Objects has no knowledge of the true shape ofBusiness Objects has no knowledge of the true shape of
the source system.the source system.

16. METL AdvantagesMETL Advantages
Non invasive.Non invasive.
Real time access to production legacy data.Real time access to production legacy data.
Provides single SQL interface layer that can be accessed byProvides single SQL interface layer that can be accessed by
multiple bi and application development tools.multiple bi and application development tools.
Places the workload on the back end server.Places the workload on the back end server.
Prolongs life of legacy systems.Prolongs life of legacy systems.
User friendly database view for developers and end users.User friendly database view for developers and end users.