The document discusses metadata in data warehousing and business intelligence contexts. Some key points:
1. Metadata provides information about data in a data warehouse or warehouse components like data marts. It describes data structures, attributes, transformations and more.
2. Metadata is important for tasks like ETL processing, querying, reporting and overall data management. It helps users understand what data is available and how to access and analyze it.
3. There are different types of metadata including technical metadata about data storage and processes, and business metadata that provides business definitions and rules. Maintaining accurate and consistent metadata is vital for a successful data warehouse.
This is the 3- Tier architecture of Data Warehouse. This is the topic under Data Mining subject. Data mining is extracting knowledge from large amount of data.
This is the 3- Tier architecture of Data Warehouse. This is the topic under Data Mining subject. Data mining is extracting knowledge from large amount of data.
History, definition, need, attributes, applications of data warehousing ; difference between data mining, big data, database and data warehouse ; future scope
The seminar is about Data warehousing, in here we are gonna discuss about what is data warehousing, comparison b/w database and data warehouse, different data warehouse models.about Data mart, and disadvantages of data warehousing.
Data Warehousing is a data architecture that separates reporting and analytics needs from operational transaction systems. This presentation is an introduction into traditional data warehousing architectures and how to determine if your environment requires a data warehouse.
Know different types of tips about Importance of dataware housing, Data Cleansing and Extracting etc . For more details visit: http://www.skylinecollege.com/business-analytics-course
This lecture gives various definitions of Data Mining. It also gives why Data Mining is required. Various examples on Classification , Cluster and Association rules are given.
History, definition, need, attributes, applications of data warehousing ; difference between data mining, big data, database and data warehouse ; future scope
The seminar is about Data warehousing, in here we are gonna discuss about what is data warehousing, comparison b/w database and data warehouse, different data warehouse models.about Data mart, and disadvantages of data warehousing.
Data Warehousing is a data architecture that separates reporting and analytics needs from operational transaction systems. This presentation is an introduction into traditional data warehousing architectures and how to determine if your environment requires a data warehouse.
Know different types of tips about Importance of dataware housing, Data Cleansing and Extracting etc . For more details visit: http://www.skylinecollege.com/business-analytics-course
This lecture gives various definitions of Data Mining. It also gives why Data Mining is required. Various examples on Classification , Cluster and Association rules are given.
We offer online IT training with placements, project assistance in different platforms with real time industry consultants to provide quality training for all it professionals, corporate clients and students etc. Special features by InformaticaTrainingClasses are Extensive Training will be in both Informatica Online Training and Placement. We help you in resume preparation and conducting Mock Interviews.
Emphasis is given on important topics which are essential and mostly used in real time projects. Informatica training Classes is an Online Training Leader when it comes to high-end effective and efficient I.T Training. We have always been and still are focusing on the key aspects which are providing utmost effective and competent training to both students and professionals who are eager to enrich their technical skills.
Training Features at Informatica training classes:
We believe that online training has to be measured by three major aspects viz., Quality, Content and Relationship with the Trainer and Student. Not only our online training classes are important but apart from that the material which we provide are in tune with the latest IT training standards, so a student has not to worry at all whether the training imparted is outdated or latest.
Course content:
• Basics of data warehousing concepts
• Power center components
• Informatica concepts and overview
• Sources
• Targets
• Transformations
• Advanced Informatica concepts
Please Visit us for the Demo Classes, we have regular batches and weekend batches.
Informatica online training classes
Phone: (404)-900-9988
Email: info@informaticatrainingclasses.com
Web: http://www.informaticatrainingclasses.com
Data Warehousing is a topic on Management of Information Technology that would help students on their subject matter and as reference for their assigned report.
Data Mining is defined as extracting information from huge sets of data. In other words, we can say that data mining is the procedure of mining knowledge from data.
According to Inmon, a data warehouse is a subject oriented,
integrated, time-variant, and non-volatile collection of data. He defined the terms
in the sentence as follows:
this is the ppt this contains definition of data ware house , data , ware house, data modeling , data warehouse architecture and its type , data warehouse types, single tire, two tire, three tire .
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
2. • In some data warehouse implementations, a
data mart is a miniature data warehouse;
• In others, it is just one segment of the data
warehouse.
• Data marts are often used to provide
information to functional segments of the
organization.
www.notesvillage.com
3. When Data Mart Appropiate
• Data marts are sometimes designed as complete individual data warehouses and
contribute to the overall organization as a member of a distributed data
warehouse.
• In other designs, data marts receive data from a master data warehouse through
periodic updates, in which case the data mart functionality is often limited to
presentation services for clients.
• Data Marts are created for the following reasons
– To speed up work by reducing the volume of data scanned
– To structure data for a user access tool
– To partition data in order to impose access control strategies
– To segment data into different hardware platform
www.notesvillage.com
4. DESIGN OF DATA MART
• Regardless of the functionality provided by data marts, they must be designed as
components of the master data warehouse so that data organization, format, and schemas
are consistent throughout the data warehouse.
• Inconsistent table designs, update mechanisms, or dimension hierarchies can prevent data
from being reused throughout the data warehouse, and they can result in inconsistent
reports from the same data
• Example:
– it is unlikely that summary reports produced from a finance department data mart that
organizes the sales force by management reporting structure will agree with summary
reports produced from a sales department data mart that organizes the same sales force
by geographical region.
Before designing for data mart we must confirm that data mart solution is appropiate for the
solution
– Identify whether there is a natural functional split within the organization
– Identify whether there is a natural split of data
– Data marts should be designed from the perspective that they are components of the
data warehouse regardless of their individual functionality or construction
– This provides consistency and usability of information throughout the organization.
www.notesvillage.com
5. IDENTIFY FUNCTIONAL SPLIT
• We must see if the split will help the organisational benefit or not
• Example
– athe retail sales in a organisation in which merchant is responsible for sales.Their berief
could be to maximize the sales by ensuring adequate sales.
• In practice the information would be of value of:
– Sales transaction on a daily level or to monitor actual sales
– Sales forecast on weekly basis
– Stock position daily basis
– Stock movement on a daily basis .
www.notesvillage.com
6. Importance of Data Mart
• Easy access to frequently needed data
• Creates collective view by a group of users
• Improves end-user response time
• Ease of creation
• Lower cost than implementing a full Data
warehouse
• Potential users are more clearly defined than
in a full Data warehouse
www.notesvillage.com
7. META DATA
• Metadata is loosely defined as data about data.
• Metadata is a concept that applies mainly to electronically archived or presented
data and is used to describe the
– a) definition,
– b) structure and
– c) administration of data files with all contents in context to ease the use of
the captured and archived data for further use.
– example: a web page may include metadata specifying what language it's
written in, what tools were used to create it, where to go for more on the
subject and so on
www.notesvillage.com
8. What is Meta data
• Metadata (meta data, or sometimes metainformation) is "data about other data", of any
sort in any media. An item of metadata may describe an individual datum, or content item, or
a collection of data including multiple content items and hierarchical levels, such as a
database schema. In data processing, metadata provides information about, or
documentation of, other data managed within an application or environment. This commonly
defines the structure or schema of the primary data.
– metadata would document data about data elements or attributes, (name, size, data type, etc) and data about
records or data structures (length, fields, columns, etc) and data about data (where it is located, how it is associated,
ownership, etc.). Metadata may include descriptive information about the context, quality and condition, or
characteristics of the data. It may be recorded with high or low granularity
Definition:
Metadata contains information about that data or other data
Metadata is structured, encoded data that describe characteristics of information-bearing entities to aid in
the identification, discovery, assessment, and management of the described entities
www.notesvillage.com
9. Why Metadata is important
• Assume that the project team has completed successfully the development of first
data mart.But the user can have several things in mind:
– Are the predefined queries I look at
– What are the various elements in data warehouse
– Is there information about unit sales and unit costs by product
– How can I browse and see what is available
– From where did they get the data for data warehouse? From which source
system
– How old are data warehouse
– When is the last time fresh data was brought in
– Are there summaries by months and product
www.notesvillage.com
10. • We can define meta data in terms data
warehousing which includes:
– Data about data
– Table of content for data
– Catalog for data
– Data warehouse roadmap
– Data warehouse directory
www.notesvillage.com
11. Applications of Metadata
• Libraries
• Metadata has been used in various forms as a means of cataloging archived
information.
• Photographs
• Metadata may be written into a digital photo file that will identify who owns it,
copyright & contact information, what camera created the file, along with
exposure information and descriptive information such as keywords about the
photo, making the file searchable on the computer and/or the Internet
• Web pages
• Web pages often include metadata in the form of meta tags. Description and
keywords meta tags are commonly used to describe the Web page's content. Most
search engines use this data when adding pages to their search index.
www.notesvillage.com
12. Critical Need of Data warehouse
• Meta data is absolute need in building datawarehouse i.e
– For Using data warehouse:
• To run adhoc queries and formatting reports users need to know
about the data in data warehouse.
• The users should gain maximum from data ware house and
ignorance of data should not give them wrong conclusion
– For building the data warehouse:
• For data extraction we must know the source system
• Structures and content will help in determining mapping
• As a Role of DBA if one needs to know about metadata for physical loading and
staging.
– Data Administration
• Data Administration is not possible knowing the metadata
• Metadata is absoultely necessary for building datawarehouse
www.notesvillage.com
13. Data warehouse Metadata
• Metadata systems in data warehouse are
sometimes separated into two sections:
1.back room metadata that are used for Extract,
transform, load functions to get OLTP data into a
data warehouse
2.front room metadata that are used to label
screens and create reports
www.notesvillage.com
14. Business Intelligence metadata
• Business Intelligence is the process of analyzing large amounts of corporate data,
usually stored in large databases such as a Data Warehouse, tracking business
performance, detecting patterns and trends, and helping enterprise business users
make better decisions. Business Intelligence metadata describes how data is
queried, filtered, analyzed, and displayed in Business Intelligence software tools,
such as Reporting tools, OLAP tools, Data Mining tools.
– Examples:
• Data Mining metadata: The descriptions and structures of Data Sets, Algorithms, Queries
• OLAP metadata: The descriptions and structures of Dimensions, Cubes, Measures (Metrics),
Hierarchies, Levels, Drill Paths
• Reporting metadata: The descriptions and structures of Reports, Charts, Queries, Data Sets,
Filters, Variables, Expressions
www.notesvillage.com
15. Building the data warehouse
• To build the metadata when need the data for data
warehouse extracted,the programmer needs to know
– the source system,data structure
– The data content
– How to handle data
• For DBA
– Incremental loading
– Last Compared data
– Populating tables
www.notesvillage.com
16. Administrating of Data warehouse
• Add new summary table
• Expand storage
• Add information delivery to the users
• When to schedule back ups
• How o maintain security system
• How to keep data definition up to date
• How o verify external data ongoing basis
www.notesvillage.com
17. Metadata used for Transformation
and Load
• Metadata may be used during data transformation and load to describe data any
changes made to data.
• The greater the difference in source the greater the requirement of metadata.
• The advantages of storing metadata is any transformation takes place as source
data changes it can be captured by metadata.
• For source data the following information required
– Source field(needs to be uniquely identified
• Unique Identifier
• Name
• Type
• Location
– System
– Object
www.notesvillage.com
18. Data management
• Meta data is required to describe the data as it resides in the data warehouse.
• This is needed for warehouse manager to track and control all data movement.
• Metadata is needed for all these things
– Tables
• Columns
• Name
• Type
– Indexes
• Columns
– Name
– Type
– Views
• Columns
– Name
– Type
– Constraints(name,type,tables
www.notesvillage.com
19. Data management
• For each table the information stored are:
– Table name(should be name in data dictionary
– Columns
• Column name
• Reference identifier
• Aggregation to be stored in the way table is stored with aggregation name
and columns .
• Similarly partition also need information like partition key and data range
inside the table
www.notesvillage.com
20. Data E T L
• How to handle data changes
• How to include new sources
• Where to cleanse the data
• How to change data cleansing
method
• How to switch to new data
transformation technique
• How to add new external data
source
• How to drop external data source
• How merging and acquisition takes
place
Data Warehouse
• How to add new summary table
• How to expand storage
• How to add new information tools for
users
• How to continue ongoing training
• How to improve adhoc queries
• When to schedule back ups
• How to maintain security systems
• How to monitor load distribution
www.notesvillage.com
21. Why Metadata for vital end users
• Meta data helps user to know the complexity of data and how it should be
transformed into the information.
• In a company when a business analyst analyses the reason for loss or profit ,he
sees the following things:
• Are the sales stored in individual transactions or summary totals.
• Can sales be analyzed by product , promotion ,store and month.
• Can the current month sales be compared to previous month sales
• From where the sales come from , what is the source system.
• How old are sales system and how does it get updated.
– If the analyst is not sure of data he can not anlayze perfectly.
– It would be perfect for a anlyst if he has a perfect road map of
metadata.
www.notesvillage.com
22. Metadata Vital for End users
• Data Content
• Summary Data
• Business Dimensions
• Business metrics
• Navigation paths
• Source systems
• External data
• Last update data
• Report formats
• OLAP data
www.notesvillage.com
23. Who needs Metadata
IT Professionals POWER USERS CASUAL USERS
Information
discovery
Database,Tables,col
umns ,server
Databases,tables,co
lumns
Queries ,reports
Meaning of Data Data structures
,data definations
Cleanising functions
Cleansing functions
Transformations
rules
Data owners,filters
Information Access SQL,3GL,4GL, Query tools Authorization
requests,
Information
retreival
www.notesvillage.com
24. Query Generation
• Meta data is required by the query manager
to enable generate queries.
• The query manager generate metadata about
the queries it has run
• The metadata can be used build a history of all
queries run and generate query profile.
www.notesvillage.com
25. Query Generation
• The metadata that is required for each query are:
– Query
• Tables accessed
– Columns accessed
» Name
» Reference identifier
• Restriction applied
– Column name
– Table name
– Reference identifier
– Restriction
• Join criteria applied
– Column name
– Table name
– Reference identifier www.notesvillage.com
26. Why Metadata is essential for IT
• Beginning from data extraction to information delivery metadata is crucial.
• The following is the need for IT to process data:
– Source of data structures
– Source platforms
– Data extraction methods
– External data
– Data transformation rules
– Data cleansing rules
– Staging area structures
– Dimensional models
– OLAP Sytems
– Query/report Design
www.notesvillage.com
27. Automation of datawarehouse
tasks
• Tools performs major functions of data warehouse
• Tools enables data movement ,transformation accordingly etc.
• While designing data warehouse we must at the beginning see to create tool for
metadata.
• In backend processes each tools record it’s own metadata.
– Source data structure definition
– Data extraction
– Initial Reformatting/merging
– Preliminary data cleansing
– Data transformation
– Validation
– Data warehouse structure definition
– Load Merge creation
www.notesvillage.com
28. Classification of Metadata types
• Classification of metadata types by functional
areas:
– Data acquisition
– Data storage
– Information delivery
www.notesvillage.com
29. • Acquisition process:
– Data Extraction
– Data transformation
– Data cleansing
– Data Integration
– Data staging
• Metadata Types:
– Source system platforms
– Source structure definition
– Data extraction method
– Data transformation rules
– Data cleansing rules
– External data sructures
– External data definition
– Summerization rules
– Target physical and logical
models
www.notesvillage.com
30. Data Storage
• The metadata used recorded by the process in
data storage area is used for development
,administration and for user.
• User would like to see what is the last time
previous data loaded.
• DBA will use the metadata for processes
backup and incremental loads.
www.notesvillage.com
31. Information Delivery
• Information delivery
– Report generation
– Query processing
– Complex Analysis
• Metadata types:
– Source systems
– Source data definitions
– Data extraction tools
– Query templates
– Preformatted reports
– OLAP content
www.notesvillage.com
32. Technical Metadata
• Technical Metadata:
– data about the processes, the tool sets, the
repositories, the physical layers of data under the
covers. Data about run-times, performance
averages, table structures, indexes, constraints;
data about relationships, sources and targets, up-
time, system failure ratios, system resource
utilization ratios, performance numbers
www.notesvillage.com
33. Technical Metadata
• List of questions Technical Metadat can answer
– What database and tables exists
– What are column for each table
– What are keys and indexes
– What are physical files
– What load refresh schedules
– What type aggregations are available
– What is source to target mapping in data warehouse.
www.notesvillage.com
34. Business Metadata
• Better understand metadata by looking at a list of example:
– Source systems
– Source to target mapping
– Data transformation business rules
– Data transformation
– Attributes and business definition
– Query reporting tools
– Predefined tools
– Predefine reports
– Report distribution information
– Currency OLAP Report
– Rules for analysis using OLAP report
www.notesvillage.com
35. Behaviour of Business Metadata
• How can I sign onto Metadata
• Which part of data warehouse I can access.
• What are part of definition I need on my part for query.
• What are types of aggregation available for my metrics.
• How Old are OLAP data. Should I wait for next update.
• Benificaries:
– Managers
– Business analyst
– Regular users
www.notesvillage.com
36. Business MetaData:
In IT, Business Metadata is adding additional text or statement
around a particular word that adds value to data. Business
Metadata is about creating definitions, business rules. For
example, when tables and columns are created the following
business metadata would be more useful for generating reports to
functional and technical team. The advantage is of this business
metadata is whether they are technical or non-technical,
everybody would understand what is going on within the
organization.
Table’s Metadata: While creating a table, metadata for definition
of a table, source system name, source entity names, business
rules to transform the source table, and the usage of the table in
reports should be added in order to make them available for
taking metadata reports.
Column’s Metadata: Similarly for columns, source column name
(mapping), business rules to transform the source column name,
and the usage of the column in reports should be added for taking
metadata reports.
www.notesvillage.com
37. Business rules In dataware house
• In the course of designing and populating a data warehouse, some key questions must be
answered about the data being incorporated in the warehouse. More often than not, many
of these answers are not known at the outset of the project, but must be established if the
data warehouse is to succeed. Interestingly, these for the most part represent the same
contextual information about the data that business users of the warehouse will need to
know to be able to fully understand the information provided, and to trust in its reliability.
The questions include:
• What are the valid values for the attributes of the data warehouse?
• What are the valid data sources for the data warehouse?
• When the data’s life cycle, in the operational world, should it be captured and sent to the
data warehouse?
• What are the “cleansing rules” for the source data?
• What are the transformation rules to move the source data to the target database?
• How was the data calculated in the operational database
www.notesvillage.com
39. Difference between Technical
metadata and business metadata
• Metadata into technical (the tool-specific metadata used by IT and vendors)
and business metadata (what a businessperson needs to know about what
data represents).
• The technology person thinks about a data column - how it's defined in a
database, represented in a data model, mapped and transformed in the ETL
tool and defined in the BI report. All of this, however, is very much related to
how the tools store and process the data. The primary challenge is
gathering and integrating the metadata across tools.
• The businessperson thinks about where the data came from, its associated
data quality level, how it was filtered from its source and what types of
business rules and algorithms were applied to it. Most of this metadata is
either not stored in the tools or needs some serious translation from
technical terms to business language.
www.notesvillage.com
40. METADATA MANAGEMENT
• The requirements for Metadata management are:
– Capturing and storing business
• Changes of algorithm methodology occurs when data for several years stores.
• Versioning must be maintained
– Variety of Metadata sources
• Different sources metadata available
– Metadata integration
• To be unified,merge to give a meaning to the end user.
– Metadata standardization
• Storage all the metadata should be in the same manner
– Rippling through revisions
• Revisions will occur as business rules changes
– Metadata Exchange
• End user should be able to exchange one meta data to another meta data.
– Support for end user
• Meat data must provide simple graphical and tabular representation to make-it
easy to browse through.
www.notesvillage.com
41. Challenges
• Major challenges for Metadata management are:
– Each software tool has it’s own propiriey of metadata.If we
are using several tools ,how can we reconcile it.
– No industry wide accepted standards exist for metadata
formats
– Preserving metadata version control uniformity in data
warehouse is very much difficult.
– Unifying data sources are very much difficult , since we
have to deal with conflicting standards, formats , data
naming conventions , units and measures.
www.notesvillage.com
42. META DATA REPOSITORY
• Metadata repository may be thought of two
distinct information queries:
– Technical Metadata
– Business Metadata
www.notesvillage.com
Editor's Notes
www.notesvillage.com-free any university tutorials,lectures,notes,results.universities supported mgu,kerala,anna,cusat,annamalai,calicut...and more.file types of pdf,ppt,txt,doc,etc....