This document discusses denormalization techniques used in data warehousing to improve query performance. It explains that while normalization is important for databases, denormalization can enhance performance in data warehouses where queries are frequent and updates are less common. Some key denormalization techniques covered include collapsing tables, splitting tables horizontally or vertically, pre-joining tables, adding redundant columns, and including derived attributes. Guidelines for when and how to apply denormalization carefully are also provided.
After completing this module, you will be able to:
List and describe the major components of the Teradata architecture.
Describe how the components interact to manage incoming and outgoing data.
List 5 types of Teradata database objects.
After completing this module, you will be able to:
List and describe the major components of the Teradata architecture.
Describe how the components interact to manage incoming and outgoing data.
List 5 types of Teradata database objects.
P REFIX - BASED L ABELING A NNOTATION FOR E FFECTIVE XML F RAGMENTATIONijcsit
XML is
gradually
emplo
yed as
a standard of data exchange
in
web
environment
since its inception
in the
90s
until
present
.
It
serves
as a data exchange between system
s
and other application
s
.
Meanwhile t
he data
volume has grown substantially
in the web and
thus effective methods
of
storing and retrieving
these
data
is
essential
.
One recommended way is
p
hysically or virtually
fragments
the large chunk of data
and
distributes
the fragments
into different nodes.
F
ragmentation design
of XML document
contains of two
parts: fragmentat
ion operation and fragmentation method. The
three
fragmentation o
peration
s are
Horizontal, Vertical
and Hybrid. It
determines how the XML should be fragmented.
This
paper
aims
to give
an overview on the fragmentation design consideration
and
subsequently,
propose a
fragmentation
technique
using
number addressing
.
Whenever you make a list of anything – list of groceries to buy, books to borrow from the library, list of classmates, list of relatives or friends, list of phone numbers and so o – you are actually creating a database.
An example of a business manual database may consist of written records on a paper and stored in a filing cabinet. The documents usually organized in chronological order, alphabetical order and so on, for easier access, retrieval and use.
Computer database are those data or information stored in the computer. To arrange and organize records, computer databases rely on database software
Microsoft Access is an example of database software.
Data Warehouse Physical Design,Physical Data Model, Tablespaces, Integrity Constraints, ETL (Extract-Transform-Load) ,OLAP Server Architectures, MOLAP vs. ROLAP, Distributed Data Warehouse ,
An introduction to database architecture, design and development, its relation to Object Oriented Analysis & Design in software, Illustration with examples to database normalization and finally, a basic SQL guide and best practices
P REFIX - BASED L ABELING A NNOTATION FOR E FFECTIVE XML F RAGMENTATIONijcsit
XML is
gradually
emplo
yed as
a standard of data exchange
in
web
environment
since its inception
in the
90s
until
present
.
It
serves
as a data exchange between system
s
and other application
s
.
Meanwhile t
he data
volume has grown substantially
in the web and
thus effective methods
of
storing and retrieving
these
data
is
essential
.
One recommended way is
p
hysically or virtually
fragments
the large chunk of data
and
distributes
the fragments
into different nodes.
F
ragmentation design
of XML document
contains of two
parts: fragmentat
ion operation and fragmentation method. The
three
fragmentation o
peration
s are
Horizontal, Vertical
and Hybrid. It
determines how the XML should be fragmented.
This
paper
aims
to give
an overview on the fragmentation design consideration
and
subsequently,
propose a
fragmentation
technique
using
number addressing
.
Whenever you make a list of anything – list of groceries to buy, books to borrow from the library, list of classmates, list of relatives or friends, list of phone numbers and so o – you are actually creating a database.
An example of a business manual database may consist of written records on a paper and stored in a filing cabinet. The documents usually organized in chronological order, alphabetical order and so on, for easier access, retrieval and use.
Computer database are those data or information stored in the computer. To arrange and organize records, computer databases rely on database software
Microsoft Access is an example of database software.
Data Warehouse Physical Design,Physical Data Model, Tablespaces, Integrity Constraints, ETL (Extract-Transform-Load) ,OLAP Server Architectures, MOLAP vs. ROLAP, Distributed Data Warehouse ,
An introduction to database architecture, design and development, its relation to Object Oriented Analysis & Design in software, Illustration with examples to database normalization and finally, a basic SQL guide and best practices
Architectural anti-patterns for data handlingGleicon Moraes
Now with three more anti patterns and a new required listening. This is the Discipline release, all hail to King Crimson and Fripp's care with details.
This White Paper on Spool Space in Teradata was presented by Nazir Iqbal at Wipro where he works at present.
TERADATA Spool Space is unused Perm Space that it used for running queries.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
CIS 515 discussion post responses.There are two discussions he.docxsleeperharwell
CIS 515 discussion post responses.
There are two discussions here that need to be responded to thoroughly. Responses must be on APA format 150+words 1-2 legitimate verifiable sources per response.
Respond to the colleagues posts regarding:
• Many legacy systems require normalization. Identify at least two factors that should be considered in order to produce an optimal normalized set of tables when performing normalization. Include in your discussion a detailed example on how each factor would eliminate data redundancy.
• Optimal database design recognizes proper organization of table structures and relationships. Suggest at least two methods that can be used to improve the design of a database system.
JM post states the following:
Factors to Consider for an Optimal Normalization
Normalization refers to the process of organizing data on a database to reduce redundancy when designing a database. One of the factors to consider during normalization is naming conventions. The names used should describe the type of information contained in the database for easy data identification. Abbreviations and obscure acronyms should be avoided for names because they can easily obfuscate the meaning and can as well be challenging to remember. The naming system used across the entire database should be consistent and memorable.
It should as well provide information on users, filenames, and other related objects. Another factor to consider is the nature of the database. Tables should incur divided to ensure that data are well isolated to allow additions, deletion, and modifications without interfering with the entire database. Each field should have different information, which should be independent of the information on other grounds. It will reduce redundancy by eliminating the possibility of data duplication (Dennis et al. 12).
Method of improving the design of the database system
A properly designed database system enhance easy access to recent and accurate information. The method process of strengthening the design of database design includes normalization and storing binary data externally. Standardization ensures that data engage appropriately organized within the database. Normalization can incur achieved by using primary keys to ensure that each raw within the database system is unique. Using external data storage is also another method of improving database system design. It enhances the performance of the data and ensures easy access to the data. Refining primary keys as required for data granularity is also another method of improving database system design. The entity of using the granularity of low keys, atomic value ensures that the database is more detailed and specific (Coronel, 2016).
Coronel, Carlos, and Steven Morris. Database systems: design, implementation, & management.
Cengage Learning, 2016.
Dennis, Alan, Barbara Haley Wixom, and Roberta M. Roth. Systems analysis and design. John
Wiley & sons, 2018
LP.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Data Warehousing and Business Intelligence is one of the hottest skills today, and is the cornerstone for reporting, data science, and analytics. This course teaches the fundamentals with examples plus a project to fully illustrate the concepts.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
The French Revolution Class 9 Study Material pdf free download
Intro to Data warehousing Lecture 04
1. Data Warehousing
De-normalization
1
Ch Anwar ul Hassan (Lecturer)
Department of Computer Science and Software
Engineering
Capital University of Sciences & Technology, Islamabad
Pakistan
anwarchaudary@gmail.com
2. 2
Striking a balance between “good” & “evil”
Flat Table
Data Lists
Data Cubes 1st Normal Form
2nd Normal Form
3rd Normal Form
4+ Normal Forms
NormalizationDe-normalization
One big flat file
Too many tables
3. 3
What is De-normalization?
It is not chaos, more like a “controlled crash”
with the aim of performance enhancement
without loss of information.
Normalization is a rule of thumb in DBMS,
but in DSS ease of use is achieved by way of
denormalization.
De-normalization comes in many flavors,
such as combining tables, splitting tables,
adding data etc., but all done very carefully.
4. 4
Why De-normalization In DSS?
Bringing “close” dispersed but related data
items.
Query performance in DSS significantly
dependent on physical data model.
Very early studies showed performance
difference in orders of magnitude for different
number de-normalized tables and rows per table.
The level of de-normalization should be
carefully considered.
5. 5
How De-normalization improves performance?
De-normalization specifically improves
performance by either:
Reducing the number of tables and hence the
reliance on joins, which consequently speeds up
performance.
Reducing the number of joins required during
query execution
Reducing the number of rows to be retrieved from
the Primary Data Table.
6. 6
4 Guidelines for De-normalization
1. Carefully do a cost-benefit analysis
(frequency of use, additional storage,
join time).
2. Do a data requirement and storage
analysis.
3. Weigh against the maintenance issue
of the redundant data (triggers used).
4. When in doubt, don’t denormalize.
7. 7
Areas for Applying De-Normalization Techniques
Dealing with the abundance of star schemas.
Fast access of time series data for analysis.
Fast aggregate (sum, average etc.) results and
complicated calculations.
Multidimensional analysis (e.g. geography) in a complex
hierarchy.
Dealing with few updates but many join queries.
De-normalization will ultimately affect the database size and
query performance.
8. Star Schema, the center of the star can have one fact table and a
number of associated dimension tables. It is known as star schema
as its structure resembles a star. The star schema is the simplest
type of Data Warehouse schema. It is also known as Star Join
Schema and is optimized for querying large data sets.
8
Star Schema
9. Snowflake Schema is an extension of a Star Schema,
and it adds additional dimensions. It is called snowflake
because its diagram resembles a Snowflake.
9
Star Schema
10. 10
Five principal De-normalization techniques
1. Collapsing Tables.
- Two entities with a One-to-One relationship.
- Two entities with a Many-to-Many relationship.
2. Splitting Tables (Horizontal/Vertical Splitting).
3. Pre-Joining.
4. Adding Redundant Columns (Reference Data).
To eliminate joins for many queries
5. Derived Attributes (Age, Total, Balance etc).
12. 12
Collapsing Tables
ColA ColB
ColA ColC
normalized
ColA ColB ColC
denormalized
Reduced storage space.
Reduced update time.
Does not changes business view.
13. 13
Splitting Tables
ColA ColB ColC
Table
Vertical Split
ColA ColB ColA ColC
Table_v1 Table_v2
ColA ColB ColC
Horizontal split
ColA ColB ColC
Table_h1 Table_h2
14. 14
Splitting Tables: Horizontal splitting…
Breaks a table into multiple tables based upon
common column values. Example: Campus specific
queries.
GOAL
Spreading rows for exploiting parallelism.
Grouping data to avoid unnecessary query load in
WHERE clause.
15. 15
Splitting Tables: Horizontal splitting
ADVANTAGE
Enhance security of data.
Organizing tables differently for different queries.
Graceful degradation of database in case of table
damage.
Fast data retrieval.
16. 16
Splitting Tables: Vertical Splitting
Infrequently accessed columns become extra
“baggage” thus degrading performance.
Very useful for rarely accessed large text columns
with large headers.
Header size is reduced, allowing more rows per
block, thus reducing I/O.
Splitting and distributing into separate files with
repeating primary key.
For an end user, the split appears as a single table
through a view.
17. 17
Pre-joining …
Identify frequent joins and append the tables
together in the physical data model.
Generally used for 1:M such as master-
detail.
Additional space is required as the master
information is repeated in the new header
table.
19. 19
Pre-Joining: Typical Scenario
Typical of Market basket query
Join ALWAYS required
Tables could be millions of rows
Squeeze Master into Detail
Repetition of facts. How much?
Detail 3-4 times of master
21. 21
Adding Redundant Columns…
Columns can also be moved, instead of making them
redundant. Very similar to pre-joining as discussed
earlier.
EXAMPLE
Frequent referencing of code in one table and
corresponding description in another table.
A join is required.
22. 22
Derived Attributes: Example
Age is also a derived attribute, calculated as Current_Date
– DoB (calculated periodically).
GP (Grade Point) column in the data warehouse data
model is included as a derived value. The formula for
calculating this field is Grade*Credits.
#SID
DoB
Degree
Course
Grade
Credits
Business Data Model
#SID
DoB
Degree
Course
Grade
Credits
GP
Age
DWH Data Model
Derived attributes
Calculated once
Used Frequently
DoB: Date of Birth