Your SlideShare is downloading. ×
0
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Effective capture of metadata using ca e rwin data modeler 09232010
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Effective capture of metadata using ca e rwin data modeler 09232010

1,353

Published on

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • http://dbmanagement.info/Tutorials/Erwin.htm
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
1,353
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
65
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Effective Capture of Metadata UsingCA ERwin Data ModelerMetadata –Data gets meaning.
  • 2. Abstract• In any data centric environment either Data warehouse or an OLTP data environment ,vital information anybody looks for is the purpose and content of the tables and columns. Its metadata of the data, provides more insight about the structure. In many organization, the lack of metadata has lead to redundant definition of table and columns , ignorance of real capability of your data centric system, inability to define standards and build the knowledge layer for the business. In the case of data warehouse its vital to capture the source and transformation rules along with dimensional model as it will help fixing incorrect mappings early in the life cycle, effective communication to ETL team and store the ETL rules close to the data model. Without metadata ,it will lead to individuals interpretation of data just like blind folded touching the elephant. In this webinar we will discuss about the various flexible features provided by CA ERwin Data Modeler for data warehouse and relational model. PAGE 2
  • 3. Speaker Bio• Sampath Kumar brings 11 years of experience in implementing small, medium and large scale data centric environments (both relational and data warehouse ) using CA ERwin Modeling suite of products. He is currently working for Infosys Technologies Limited as Technology Architect in their DW/BI practice group. Prior to that he was working with American Express Credit Cards as Sr System Analyst for their Worldwide Risk Information Management group. In all his experience he has worked extensively in various database products ,BI tools ,data modeling and related products offered by CA such as CA ERwin Data Modeler, CA ERwin Model Validator,CA ERwin Model Manager and CA ERwin Data Profiler. PAGE 3
  • 4. Agenda• Not going to focus on the known fundamentals and data jargons• Effective capture of metadata in Data warehouse environment – Case study using Customer_Dim• Other Flexible Options to capture metadata into data model – Using an example. PAGE 4
  • 5. Effective capture of metadata in Datawarehouse environmentCase study using Customer_Dim
  • 6. Problem Statement In any data warehouse development project, some of the major challenges include• Effective capture of metadata information in data model such as data source ,transformation, enrichment and data synchronization rules etc.• Keeping data model in synch with changing ETL rules and vice versa i.e. keeping ETL rules close to DW Data model (blueprint of your DW data)• Early identification of incorrect ETL mappings in the complete lifecycle. PAGE 6
  • 7. Problem Statement contd…• Effective communication of captured metadata information by data modeler to other teams such as ETL PAGE 7
  • 8. Background3 Important pieces of information:• Source of data• Transformation rules-The method in which the data is getting extracted, transformed and loaded• Frequency: The frequency and timing of data warehouse updates. PAGE 8
  • 9. CA ERwin FeaturesCA ERwin Data Modeler supports the following salient features to capture the metadata information effectively.• Data warehouse Sources Dialog• Columns Editor• Data Movement Rules Editor PAGE 9
  • 10. Customer_Dim Snapshot Communication – customer_SKID Email Address – snapshot_Begin_Date Phone – snapshot_End_Date Fax – current_ind Segmentation • Basic Information Shopping – Customer name Behavior – Customer Date of Birth – Driving License • Address – Mailing Address – Physical Address PAGE 10
  • 11. Capturing Data Source PAGE 11
  • 12. Creating Customer_Dim PAGE 12
  • 13. Make it Dimensional Model PAGE 13
  • 14. Make it Dimensional Model… PAGE 14
  • 15. Data Source Enabled… PAGE 15
  • 16. Data warehouse Source Selector PAGE 16
  • 17. Data warehouse Source PAGE 17
  • 18. Data warehouse SourcesThe “Import other” provides three options to import the table structure• Flat File• Database/Script• Model Manager PAGE 18
  • 19. Importing table from CA ERwin® Model Manager Customer_Address customer_id (FK) mailing_address_line1 mailing_address_line2 mailing_city mailing_state mailing_county mailing_country physical_address_line1 Customer physical_address_line2 physical_county customer_id physical_city customer_first_name physical_state Opens Model Mart customer_last_name physical_country dob Library driving_license_nbr driving_license_state Behavioral_Segment behavioral_segment_nbr behavioral_segment_name Customer_Segmentation customer_id (FK) behavioral_segment_nbr (FK) Shopping_Segment shopping_segment_nbr (FK) shopping_segment_nbr shopping_segment_name PAGE 19
  • 20. Import source tables PAGE 20
  • 21. Source Tables Populated PAGE 21
  • 22. Data warehouse Source Selector. Multiple sources can be added Transformation and business rules can be PAGE 22 added here.
  • 23. Source as Flat File PAGE 23
  • 24. ETL Mapping Template-using Data Browser PAGE 24
  • 25. Report Template Builder PAGE 25
  • 26. Customize the Report PAGE 26
  • 27. Generate the Metadata Report PAGE 27
  • 28. ETL Mapping Sheet PAGE 28
  • 29. Data Movement Rules PAGE 29
  • 30. Documenting the rule PAGE 30
  • 31. Attaching table to rule. PAGE 31
  • 32. Export as Report PAGE 32
  • 33. Other Flexible Options to capturemetadata into data modelUsing simple example
  • 34. Import from MS ExcelUsing simple example
  • 35. Metadata Capture from MS Excel• When it would be useful – Import the definitions available already into data model – Import the definitions from business stakeholders for key columns to avoid wrong interpretation.• Step 1: Store the model locally in the hard disk• Step 2: Use the excel sheet “Import Definitions” or VBA macro provided by CA .• Step 3 :Import the metadata into the model by running VBA code. PAGE 35
  • 36. Person Table PAGE 36
  • 37. Import Definitions PAGE 37
  • 38. In this format PAGE 38
  • 39. Final StepOpen the first sheet and click on “Update Entity Defns” which will update the definitions written for that particular table into the data model. Similarly click on the “Update Attribute Defns” which will update the attribute definitions.Note:• Keep the data model closed otherwise you will get error that it’s open.• Make sure table and column names are exactly same as in the data model.• It’s not only for business people but also for the data modelers who can enter the definitions in MS Excel and get the approval from the business or data management team, then it can uploaded separately using this utility. PAGE 39
  • 40. Capture metadata in Data Browser.
  • 41. Metadata Capture using Data Browser PAGE 41
  • 42. Conclusion• The metadata information such as “Data Source”, “Transformations rules” and “Data Movement rules” are very important for any Data warehousing efforts and it’s very critical to capture the correct information.• Metadata from data management standpoint , reduces considerable amount of time while consolidating the attributes or entities or databases during acquisition or merger.• Knowing the importance of metadata for the data model ,CA ERwin has provided these flexible options which can be leveraged to make the data model & data more meaningful. PAGE 42
  • 43. Questions? In case of any additional questions you can reach me in Sampath_Kumar01@infosys.com Sampath.k.kumar@gmail.com PAGE 43

×