SlideShare a Scribd company logo
1 of 31
Download to read offline
Query Processing and
Optimization
Introduction
• In this chapter we shall discuss the techniques
used by a DBMS to process, optimize and execute
high-level queries.
• The techniques used to split complex queries into
multiple simple operations and methods of
implementing these low-level operations.
• The query optimization techniques are used to
chose an efficient execution plan that will
minimize the runtime as well as many other types
of resources such as number of disk I/O, CPU
time and so on.
Query Processing
• Query Processing is a procedure of transforming a high-level
query (such as SQL) into a correct and efficient execution plan
expressed in low-level language.
• When a database system receives a query for update or
retrieval of information, it goes through a series of
compilation steps, called execution plan.
• Query processing goes through various phases:
• first phase is called syntax checking phase, the system parses
the query and checks that it follows the syntax rules or not.
• It then matches the objects in the query syntax with the view
tables and columns listed in the system table.
• In second phase the SQL query is translated in to an algebraic
expression using various rules.
• So that the process of transforming a high-level SQL query into a
relational algebraic form is called Query Decomposition.
• The relational algebraic expression now passes to the query
optimizer.
• In third phase optimization is performed by substituting equivalent
expression depends on the factors such that the existence of
certain database structures, whether or not a given file is stored,
the presence of different indexes & so on.
• Query optimization module work in tandem with the join manager
module to improve the order in which joins are performed.
• At this stage the cost model and several other
estimation formulas are used to rewrite the query.
• The modified query is written to utilize system
resources so as to bring the optimal performance.
• The query optimizer then generates an action plan also
called a execution plan.
• This action plans are converted into a query codes that
are finally executed by a run time database processor.
• The run time database processor estimate the cost of
each action plan and chose the optimal one for the
execution.
Query Analyzer
• The syntax analyzer takes the query from the
users, parses it into tokens and analyses the
tokens and their order to make sure they
follow the rules of the language grammar.
• Is an error is found in the query submitted by
the user, it is rejected and an error code
together with an explanation of why the query
was rejected is return to the user.
Query Decomposition
• In query decomposition the query processing aims are to transfer
the high-level query into a relational algebra query and to check
whether that query is syntactically and semantically correct.
• Thus the query decomposition is start with a high-level query and
transform into query graph of low-level operations, which satisfy
the query.
• The SQL query is decomposed into query blocks (low-level
operations), which form the basic unit.
• Hence nested queries within a query are identified as separate
query blocks.
• The query decomposer goes through five stages of processing for
decomposition into low-level operation and translation into
algebraic expressions.
Query Analysis
• During the query analysis phase, the query is
syntactically analyzed using the programming
language compiler (parser).
• A syntactically legal query is then validated,
using the system catalog, to ensure that all
data objects (relations and attributes) referred
to by the query are defined in the database.
• The type specification of the query qualifiers
and result is also checked at this stage.
• Example:
SELECT emp_nm FROM EMPLOYEE WHERE
emp_desg>100
This query will be rejected because the comparison
“>100” is incompatible with the data type of
emp_desg which is a variable character string.
• At the end of query analysis phase, the high-level query
(SQL) is transformed into some internal representation
that is more suitable for processing.
• This internal representation is typically a kind of query
tree.
• A Query Tree is a tree data structure that
corresponds expression.
• A Query Tree is also called a relational algebra
tree.
– Leaf node of the tree, representing the base
input relations of the query.
– Internal nodes result of applying an operation in
the algebra.
– Root of the tree representing a result of the query.
• SELECT (P.proj_no, P.dept_no, E.name, E.add,
E.dob)
• FROM PROJECT P, DEPARTMENT D, EMPLOYEE
E
• WHERE P.dept_no = D.d_no AND D.mgr_id =
E.emp_id AND P.proj_loc = ‘Mumbai’ ;
• The three relations PROJECT, DEPARTMENT,
EMPLOYEE are represent as a leaf nodes P, D
and E, while the relational algebra operations
of the represented by internal tree nodes.
• Same SQL query can have man different
relational algebra expressions and hence
many different query trees.
• The query parser typically generates a
standard initial (canonical) query tree.
Query Normalization
• The primary phase of the normalization is to avoid
redundancy.
• The normalization phase converts the query into a
normalized form that can be more easily manipulated.
• In the normalization phase, a set of equivalency rules
are applied so that the projection and selection
operations included on the query are simplified to
avoid redundancy.
• The projection operation corresponds to the SELECT
clause of SQL query and the selection operation
correspond to the predicate found in WHERE clause.
• The equivalency transformation rules that are applied.
Semantic Analyzer
• The objective of this phase of query processing is to reduce the number of
predicates.
• The semantic analyzer rejects the normalized queries that are incorrectly
formulated.
• A query is incorrectly formulated if components do not contribute to the
generation of result.
• This happens in case of missing join specification.
• A query is contradictory if its predicate cannot satisfy by any tuple in the relation.
• The semantic analyzer examine the relational calculus query (SQL) to make sure it
contains only data objects that is table, columns, views, indexes that are defined in
the database catalog.
• It makes sure that each object in the query is referenced correctly according to its
data type.
• In case of missing join specifications the components do not contribute to the
generation of the results, and thus, a query may be incorrect formulated.
Query Simplifier
• The objectives of this phase are to detect redundant
qualification, eliminate common sub-expressions and
transform sub-graph to semantically equivalent but
more easy and efficiently computed form.
• Why to simplify?
– Commonly integrity constraints, view definitions and
access restrictions are introduced into the graph at this
stage of analysis so that the query must be simplified as
much as possible.
– Integrity constraints defines constants which must holds
for all state of database, so any query that contradict an
integrity constraints must be avoid and can be rejected
without accessing the database.
Query Restructuring
• In the final stage of the query decomposition,
the query can be restructured to give a more
efficient implementation.
• Transformation rules are used to convert one
relational algebra expression into an
equivalent form that is more efficient.
• The query can now be regarded as a relational
algebra program, consisting of a series of
operations on relation.
Query Optimization
• The primary goal of query optimization is of
choosing an efficient execution strategy for
processing a query.
• The query optimizer attempts to minimize the
use of certain resources (mainly the number of
I/O and CPU time) by selecting a best execution
plan (access plan).
• A query optimization start during the validation
phase by the system to validate the user has
appropriate privileges.
• Now an action plan is generate to perform the
query.
Block Diagram of Query Optimization
• Relational algebra query tree generated by the query
simplifier module of query decomposer.
• Estimation formulas used to determine the cardinality
of the intermediate result table.
• A cost Model.
• Statistical data from the database catalogue.
The output of the query optimizer is the execution plan
in form of optimized relational algebra query.
A query typically has many possible execution
strategies, and the process of choosing a suitable one
for processing a query is known as Query Optimization.
The basic issues in Query Optimization
• How to use available indexes?
• How to use memory to accumulate
information and perform immediate steps
such as sorting?
• How to determine the order in which joins
should be performed?
Objective of query optimization
• The term query optimization does not mean
giving always an optimal (best) strategy as the
execution plan.
• It is just a responsibly efficient strategy for
execution of the query.
• The decomposed query block of SQL is
translating into an equivalent extended
relational algebra expression and then
optimized.
Techniques for Query Optimization
• The first technique is based on Heuristic Rules
for ordering the operations in a query
execution strategy.
• The second technique involves the systematic
estimation of the cost of the different
execution strategies and choosing the
execution plan with the lowest cost.
• Semantic query optimization is used with the
combination with the heuristic query
transformation rules.
• It uses constraints specified on the database
schema such as unique attributes and other
more complex constraints, in order to modify
one query into another query that is more
efficient to execute.
Heuristic Rules
• The heuristic rules are used as an optimization
technique to modify the internal representation
of query.
• Usually, heuristic rules are used in the form of
query tree of query graph data structure, to
improve its performance.
• One of the main heuristic rule is to apply SELECT
operation before applying the JOIN or other
BINARY operations.
• This is because the size of the file resulting from a
binary operation such as JOIN is usually a multi-
value function of the sizes of the input files.
Heuristic Rules
• The SELECT and PROJECT reduced the size of
the file and hence, should be applied before
the JOIN or other binary operation.
• Heuristic query optimizer transforms the
initial (canonical) query tree into final query
tree using equivalence transformation rules.
• This final query tree is efficient to execute.
General Transformation Rules
• Cascade of σ :-
• σ c1 AND c2 AND …AND cn (R) = σ c1 (σ c2 (…(σ cn (R))…))
• Commutativity of σ :-
• σ C1 (σ C2 (R)) = σ C2 (σ C1 (R))
Cascade of Л :-
• Л List1 (Л List2 (…(Л List n (R))…)) = Л List1 (R)
• Commuting σ with Л :-
• Л A1,A2,A3…An (σ C (R) ) = σ C (Л A1,A2,A3…An (R))
General Transformation Rules
Commutativity of ⋈ AND x :-
R ⋈ c S = S ⋈ c R
R x S = S x R
• Commuting σ with ⋈ or x :-
If all attributes in selection condition c involved only attributes of one of the relation
schemas (R).
σ c (R ⋈ S) = (σ c (R) ) ⋈ S
Alternatively, selection condition c can be written as (c1 AND c2) where condition c1
involves only attributes of R and condition c2 involves only attributes of S then :
σ c (R ⋈ S) = (σ c1 (R) ) ⋈ (σ c2 (S) )
• Commuting Л with ⋈ or x :-
The projection list L = {A1,A2,..An,B1,B2,…Bm}.
A1…An attributes of R and B1…Bm attributes of S.
Join condition C involves only attributes in L then :
ЛL ( R ⋈ c S ) = ( ЛA1,…An (R) ) ⋈c ( ЛB1,…Bm(S) )
General Transformation Rules
• Commutative of SET Operation :-
R ⋃ S = S ⋃ R
R ⋂ S = S ⋂ R
Minus (R-S) is not commutative.
• Associatively of ⋈, x, ⋂, and ⋃ :-
If ∅ stands for any one of these operation throughout the expression then :
(R ∅ S) ∅ T = R ∅ (S ∅ T)
• Commutativity of σ with SET Operation :-
If ∅ stands for any one of three operations (⋃,⋂,and-) then :
σ c (R ∅ S) = (σ c (R)) ⋃ (σ c (S))
Л c (R ∅ S) = (Л c (R)) ⋃ (Лc (S))
• The Л operation comute with ⋃ :-
Л L (R ⋃ S) = (Л L(R)) ⋃ (Л L(S))
• Converting a (σ,x) sequence with ⋃
(σ c (R x S)) = (R ⋈ c S)
THANK YOU

More Related Content

What's hot

Distribution transparency and Distributed transaction
Distribution transparency and Distributed transactionDistribution transparency and Distributed transaction
Distribution transparency and Distributed transactionshraddha mane
 
Query Decomposition and data localization
Query Decomposition and data localization Query Decomposition and data localization
Query Decomposition and data localization Hafiz faiz
 
Replication in Distributed Database
Replication in Distributed DatabaseReplication in Distributed Database
Replication in Distributed DatabaseAbhilasha Lahigude
 
Distributed Query Processing
Distributed Query ProcessingDistributed Query Processing
Distributed Query ProcessingMythili Kannan
 
Lock based protocols
Lock based protocolsLock based protocols
Lock based protocolsChethanMp7
 
Overview of Concurrency Control & Recovery in Distributed Databases
Overview of Concurrency Control & Recovery in Distributed DatabasesOverview of Concurrency Control & Recovery in Distributed Databases
Overview of Concurrency Control & Recovery in Distributed DatabasesMeghaj Mallick
 
Query processing-and-optimization
Query processing-and-optimizationQuery processing-and-optimization
Query processing-and-optimizationWBUTTUTORIALS
 
Object Relational Database Management System(ORDBMS)
Object Relational Database Management System(ORDBMS)Object Relational Database Management System(ORDBMS)
Object Relational Database Management System(ORDBMS)Rabin BK
 
database recovery techniques
database recovery techniques database recovery techniques
database recovery techniques Kalhan Liyanage
 
Concurrency Control in Database Management System
Concurrency Control in Database Management SystemConcurrency Control in Database Management System
Concurrency Control in Database Management SystemJanki Shah
 
DISTRIBUTED DATABASE WITH RECOVERY TECHNIQUES
DISTRIBUTED DATABASE WITH RECOVERY TECHNIQUESDISTRIBUTED DATABASE WITH RECOVERY TECHNIQUES
DISTRIBUTED DATABASE WITH RECOVERY TECHNIQUESAAKANKSHA JAIN
 

What's hot (20)

Recovery techniques
Recovery techniquesRecovery techniques
Recovery techniques
 
Distribution transparency and Distributed transaction
Distribution transparency and Distributed transactionDistribution transparency and Distributed transaction
Distribution transparency and Distributed transaction
 
Query Decomposition and data localization
Query Decomposition and data localization Query Decomposition and data localization
Query Decomposition and data localization
 
Replication in Distributed Database
Replication in Distributed DatabaseReplication in Distributed Database
Replication in Distributed Database
 
Distributed Query Processing
Distributed Query ProcessingDistributed Query Processing
Distributed Query Processing
 
View of data DBMS
View of data DBMSView of data DBMS
View of data DBMS
 
Database recovery
Database recoveryDatabase recovery
Database recovery
 
Lock based protocols
Lock based protocolsLock based protocols
Lock based protocols
 
Query processing
Query processingQuery processing
Query processing
 
DDBMS
DDBMSDDBMS
DDBMS
 
Overview of Concurrency Control & Recovery in Distributed Databases
Overview of Concurrency Control & Recovery in Distributed DatabasesOverview of Concurrency Control & Recovery in Distributed Databases
Overview of Concurrency Control & Recovery in Distributed Databases
 
Query processing-and-optimization
Query processing-and-optimizationQuery processing-and-optimization
Query processing-and-optimization
 
Distributed database
Distributed databaseDistributed database
Distributed database
 
Object Relational Database Management System(ORDBMS)
Object Relational Database Management System(ORDBMS)Object Relational Database Management System(ORDBMS)
Object Relational Database Management System(ORDBMS)
 
Concurrency control
Concurrency controlConcurrency control
Concurrency control
 
database recovery techniques
database recovery techniques database recovery techniques
database recovery techniques
 
DBMS - RAID
DBMS - RAIDDBMS - RAID
DBMS - RAID
 
Concurrency Control in Database Management System
Concurrency Control in Database Management SystemConcurrency Control in Database Management System
Concurrency Control in Database Management System
 
DISTRIBUTED DATABASE WITH RECOVERY TECHNIQUES
DISTRIBUTED DATABASE WITH RECOVERY TECHNIQUESDISTRIBUTED DATABASE WITH RECOVERY TECHNIQUES
DISTRIBUTED DATABASE WITH RECOVERY TECHNIQUES
 
Distributed DBMS - Unit 5 - Semantic Data Control
Distributed DBMS - Unit 5 - Semantic Data ControlDistributed DBMS - Unit 5 - Semantic Data Control
Distributed DBMS - Unit 5 - Semantic Data Control
 

Viewers also liked

Cost estimation for Query Optimization
Cost estimation for Query OptimizationCost estimation for Query Optimization
Cost estimation for Query OptimizationRavinder Kamboj
 
Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...
Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...
Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...Beat Signer
 
13. Query Processing in DBMS
13. Query Processing in DBMS13. Query Processing in DBMS
13. Query Processing in DBMSkoolkampus
 
Query processing and Query Optimization
Query processing and Query OptimizationQuery processing and Query Optimization
Query processing and Query OptimizationNiraj Gandha
 
Normalization of Data Base
Normalization of Data BaseNormalization of Data Base
Normalization of Data BaseRavinder Kamboj
 
Data Warehousing and Data Mining
Data Warehousing and Data MiningData Warehousing and Data Mining
Data Warehousing and Data Miningidnats
 
Lecture 1&2(rdbms-ii)
Lecture 1&2(rdbms-ii)Lecture 1&2(rdbms-ii)
Lecture 1&2(rdbms-ii)Ravinder Kamboj
 
비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...
비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...
비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...成 金
 
PROMPERU - guia de mercado Ecuador
PROMPERU - guia de mercado EcuadorPROMPERU - guia de mercado Ecuador
PROMPERU - guia de mercado Ecuadoragroalimentaria.pe
 
Le molecole biologiche - didattica differenziata
Le molecole biologiche - didattica differenziataLe molecole biologiche - didattica differenziata
Le molecole biologiche - didattica differenziataIacopo Pappalardo
 
Lecture 5 6_7 - divide and conquer and method of solving recurrences
Lecture 5 6_7 - divide and conquer and method of solving recurrencesLecture 5 6_7 - divide and conquer and method of solving recurrences
Lecture 5 6_7 - divide and conquer and method of solving recurrencesjayavignesh86
 
Event Driven Automation Meetup May 14/2015
Event Driven Automation Meetup May 14/2015Event Driven Automation Meetup May 14/2015
Event Driven Automation Meetup May 14/2015Dmitri Zimine
 

Viewers also liked (17)

Cost estimation for Query Optimization
Cost estimation for Query OptimizationCost estimation for Query Optimization
Cost estimation for Query Optimization
 
Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...
Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...
Query Processing and Optimisation - Lecture 10 - Introduction to Databases (1...
 
13. Query Processing in DBMS
13. Query Processing in DBMS13. Query Processing in DBMS
13. Query Processing in DBMS
 
Query processing and Query Optimization
Query processing and Query OptimizationQuery processing and Query Optimization
Query processing and Query Optimization
 
Normalization of Data Base
Normalization of Data BaseNormalization of Data Base
Normalization of Data Base
 
Data Warehousing and Data Mining
Data Warehousing and Data MiningData Warehousing and Data Mining
Data Warehousing and Data Mining
 
Lecture 1&2(rdbms-ii)
Lecture 1&2(rdbms-ii)Lecture 1&2(rdbms-ii)
Lecture 1&2(rdbms-ii)
 
비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...
비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...
비아그라 판매 =<7cc.kr>=비아그라 정품 판매~비아그라판매±비아그라 정품판매∏비아그라 인터넷구입,프릴리지 인터넷구입,흥분제 인터...
 
PROMPERU - guia de mercado Ecuador
PROMPERU - guia de mercado EcuadorPROMPERU - guia de mercado Ecuador
PROMPERU - guia de mercado Ecuador
 
MINAGRI - siea 2015
MINAGRI - siea 2015MINAGRI - siea 2015
MINAGRI - siea 2015
 
MINAGRI - agroindustrial 2015
MINAGRI - agroindustrial 2015MINAGRI - agroindustrial 2015
MINAGRI - agroindustrial 2015
 
Le molecole biologiche - didattica differenziata
Le molecole biologiche - didattica differenziataLe molecole biologiche - didattica differenziata
Le molecole biologiche - didattica differenziata
 
Bitacoras
BitacorasBitacoras
Bitacoras
 
Sandra m lenguaje señas
Sandra m lenguaje señasSandra m lenguaje señas
Sandra m lenguaje señas
 
Lecture 5 6_7 - divide and conquer and method of solving recurrences
Lecture 5 6_7 - divide and conquer and method of solving recurrencesLecture 5 6_7 - divide and conquer and method of solving recurrences
Lecture 5 6_7 - divide and conquer and method of solving recurrences
 
BIS05 Introduction to SQL
BIS05 Introduction to SQLBIS05 Introduction to SQL
BIS05 Introduction to SQL
 
Event Driven Automation Meetup May 14/2015
Event Driven Automation Meetup May 14/2015Event Driven Automation Meetup May 14/2015
Event Driven Automation Meetup May 14/2015
 

Similar to Query processing and optimization (updated)

LECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptx
LECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptxLECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptx
LECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptxAthosBeatus
 
Lecture 5.pptx
Lecture 5.pptxLecture 5.pptx
Lecture 5.pptxShafii8
 
Query optimization
Query optimizationQuery optimization
Query optimizationPooja Dixit
 
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cPresentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cRonald Francisco Vargas Quesada
 
UNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data baseUNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data baseshindhe1098cv
 
Models of Operational research, Advantages & disadvantages of Operational res...
Models of Operational research, Advantages & disadvantages of Operational res...Models of Operational research, Advantages & disadvantages of Operational res...
Models of Operational research, Advantages & disadvantages of Operational res...Sunny Mervyne Baa
 
Software Engineering Lec 4-requirments
Software Engineering Lec 4-requirmentsSoftware Engineering Lec 4-requirments
Software Engineering Lec 4-requirmentsTaymoor Nazmy
 
Overview of query evaluation
Overview of query evaluationOverview of query evaluation
Overview of query evaluationavniS
 
Testing Frameworks
Testing FrameworksTesting Frameworks
Testing FrameworksMoataz Nabil
 
Query handlingbytheserver
Query handlingbytheserverQuery handlingbytheserver
Query handlingbytheserversqlserver.co.il
 
DS M1 full - KQB KtuQbank.pdf
DS M1 full - KQB KtuQbank.pdfDS M1 full - KQB KtuQbank.pdf
DS M1 full - KQB KtuQbank.pdfMidhunM83
 

Similar to Query processing and optimization (updated) (20)

LECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptx
LECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptxLECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptx
LECTURE_06_DATABASE PROCESSING & OPTIMAZATION.pptx
 
Lecture 5.pptx
Lecture 5.pptxLecture 5.pptx
Lecture 5.pptx
 
Query optimization
Query optimizationQuery optimization
Query optimization
 
Query optimization
Query optimizationQuery optimization
Query optimization
 
module 1.pptx
module 1.pptxmodule 1.pptx
module 1.pptx
 
Query optimization
Query optimizationQuery optimization
Query optimization
 
Mc seminar
Mc seminarMc seminar
Mc seminar
 
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cPresentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12c
 
Measures of query cost
Measures of query costMeasures of query cost
Measures of query cost
 
Statistics
StatisticsStatistics
Statistics
 
Oracle query optimizer
Oracle query optimizerOracle query optimizer
Oracle query optimizer
 
UNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data baseUNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data base
 
Development Guideline
Development GuidelineDevelopment Guideline
Development Guideline
 
Models of Operational research, Advantages & disadvantages of Operational res...
Models of Operational research, Advantages & disadvantages of Operational res...Models of Operational research, Advantages & disadvantages of Operational res...
Models of Operational research, Advantages & disadvantages of Operational res...
 
Software Engineering Lec 4-requirments
Software Engineering Lec 4-requirmentsSoftware Engineering Lec 4-requirments
Software Engineering Lec 4-requirments
 
Overview of query evaluation
Overview of query evaluationOverview of query evaluation
Overview of query evaluation
 
Testing Frameworks
Testing FrameworksTesting Frameworks
Testing Frameworks
 
dd presentation.pdf
dd presentation.pdfdd presentation.pdf
dd presentation.pdf
 
Query handlingbytheserver
Query handlingbytheserverQuery handlingbytheserver
Query handlingbytheserver
 
DS M1 full - KQB KtuQbank.pdf
DS M1 full - KQB KtuQbank.pdfDS M1 full - KQB KtuQbank.pdf
DS M1 full - KQB KtuQbank.pdf
 

More from Ravinder Kamboj

More from Ravinder Kamboj (9)

Data warehouse,data mining & Big Data
Data warehouse,data mining & Big DataData warehouse,data mining & Big Data
Data warehouse,data mining & Big Data
 
Architecture of dbms(lecture 3)
Architecture of dbms(lecture 3)Architecture of dbms(lecture 3)
Architecture of dbms(lecture 3)
 
Sql fundamentals
Sql fundamentalsSql fundamentals
Sql fundamentals
 
Java script
Java scriptJava script
Java script
 
File Management
File ManagementFile Management
File Management
 
HTML Forms
HTML FormsHTML Forms
HTML Forms
 
DHTML
DHTMLDHTML
DHTML
 
CSA lecture-1
CSA lecture-1CSA lecture-1
CSA lecture-1
 
Relational database management system (rdbms) i
Relational database management system (rdbms) iRelational database management system (rdbms) i
Relational database management system (rdbms) i
 

Recently uploaded

Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Osopher
 
Narcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfNarcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfPrerana Jadhav
 
Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Celine George
 
Jason Potel In Media Res Media Component
Jason Potel In Media Res Media ComponentJason Potel In Media Res Media Component
Jason Potel In Media Res Media ComponentInMediaRes1
 
4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptxmary850239
 
Indexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdfIndexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdfChristalin Nelson
 
DiskStorage_BasicFileStructuresandHashing.pdf
DiskStorage_BasicFileStructuresandHashing.pdfDiskStorage_BasicFileStructuresandHashing.pdf
DiskStorage_BasicFileStructuresandHashing.pdfChristalin Nelson
 
How to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineHow to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineCeline George
 
Objectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptxObjectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptxMadhavi Dharankar
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesVijayaLaxmi84
 
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFEPART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFEMISSRITIMABIOLOGYEXP
 
Sarah Lahm In Media Res Media Component
Sarah Lahm  In Media Res Media ComponentSarah Lahm  In Media Res Media Component
Sarah Lahm In Media Res Media ComponentInMediaRes1
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQuiz Club NITW
 
MS4 level being good citizen -imperative- (1) (1).pdf
MS4 level   being good citizen -imperative- (1) (1).pdfMS4 level   being good citizen -imperative- (1) (1).pdf
MS4 level being good citizen -imperative- (1) (1).pdfMr Bounab Samir
 
Farrington HS Streamlines Guest Entrance
Farrington HS Streamlines Guest EntranceFarrington HS Streamlines Guest Entrance
Farrington HS Streamlines Guest Entrancejulius27264
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
The Emergence of Legislative Behavior in the Colombian Congress
The Emergence of Legislative Behavior in the Colombian CongressThe Emergence of Legislative Behavior in the Colombian Congress
The Emergence of Legislative Behavior in the Colombian CongressMaria Paula Aroca
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationdeepaannamalai16
 

Recently uploaded (20)

Teaching Critical AI Literacies - Maha Bali
Teaching Critical AI Literacies - Maha BaliTeaching Critical AI Literacies - Maha Bali
Teaching Critical AI Literacies - Maha Bali
 
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
Healthy Minds, Flourishing Lives: A Philosophical Approach to Mental Health a...
 
Narcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfNarcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdf
 
Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17
 
Jason Potel In Media Res Media Component
Jason Potel In Media Res Media ComponentJason Potel In Media Res Media Component
Jason Potel In Media Res Media Component
 
Spearman's correlation,Formula,Advantages,
Spearman's correlation,Formula,Advantages,Spearman's correlation,Formula,Advantages,
Spearman's correlation,Formula,Advantages,
 
4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx
 
Indexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdfIndexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdf
 
DiskStorage_BasicFileStructuresandHashing.pdf
DiskStorage_BasicFileStructuresandHashing.pdfDiskStorage_BasicFileStructuresandHashing.pdf
DiskStorage_BasicFileStructuresandHashing.pdf
 
How to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineHow to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command Line
 
Objectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptxObjectives n learning outcoms - MD 20240404.pptx
Objectives n learning outcoms - MD 20240404.pptx
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their uses
 
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFEPART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
PART 1 - CHAPTER 1 - CELL THE FUNDAMENTAL UNIT OF LIFE
 
Sarah Lahm In Media Res Media Component
Sarah Lahm  In Media Res Media ComponentSarah Lahm  In Media Res Media Component
Sarah Lahm In Media Res Media Component
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
 
MS4 level being good citizen -imperative- (1) (1).pdf
MS4 level   being good citizen -imperative- (1) (1).pdfMS4 level   being good citizen -imperative- (1) (1).pdf
MS4 level being good citizen -imperative- (1) (1).pdf
 
Farrington HS Streamlines Guest Entrance
Farrington HS Streamlines Guest EntranceFarrington HS Streamlines Guest Entrance
Farrington HS Streamlines Guest Entrance
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
 
The Emergence of Legislative Behavior in the Colombian Congress
The Emergence of Legislative Behavior in the Colombian CongressThe Emergence of Legislative Behavior in the Colombian Congress
The Emergence of Legislative Behavior in the Colombian Congress
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentation
 

Query processing and optimization (updated)

  • 2. Introduction • In this chapter we shall discuss the techniques used by a DBMS to process, optimize and execute high-level queries. • The techniques used to split complex queries into multiple simple operations and methods of implementing these low-level operations. • The query optimization techniques are used to chose an efficient execution plan that will minimize the runtime as well as many other types of resources such as number of disk I/O, CPU time and so on.
  • 3. Query Processing • Query Processing is a procedure of transforming a high-level query (such as SQL) into a correct and efficient execution plan expressed in low-level language. • When a database system receives a query for update or retrieval of information, it goes through a series of compilation steps, called execution plan. • Query processing goes through various phases: • first phase is called syntax checking phase, the system parses the query and checks that it follows the syntax rules or not. • It then matches the objects in the query syntax with the view tables and columns listed in the system table.
  • 4. • In second phase the SQL query is translated in to an algebraic expression using various rules. • So that the process of transforming a high-level SQL query into a relational algebraic form is called Query Decomposition. • The relational algebraic expression now passes to the query optimizer. • In third phase optimization is performed by substituting equivalent expression depends on the factors such that the existence of certain database structures, whether or not a given file is stored, the presence of different indexes & so on. • Query optimization module work in tandem with the join manager module to improve the order in which joins are performed.
  • 5. • At this stage the cost model and several other estimation formulas are used to rewrite the query. • The modified query is written to utilize system resources so as to bring the optimal performance. • The query optimizer then generates an action plan also called a execution plan. • This action plans are converted into a query codes that are finally executed by a run time database processor. • The run time database processor estimate the cost of each action plan and chose the optimal one for the execution.
  • 6. Query Analyzer • The syntax analyzer takes the query from the users, parses it into tokens and analyses the tokens and their order to make sure they follow the rules of the language grammar. • Is an error is found in the query submitted by the user, it is rejected and an error code together with an explanation of why the query was rejected is return to the user.
  • 7. Query Decomposition • In query decomposition the query processing aims are to transfer the high-level query into a relational algebra query and to check whether that query is syntactically and semantically correct. • Thus the query decomposition is start with a high-level query and transform into query graph of low-level operations, which satisfy the query. • The SQL query is decomposed into query blocks (low-level operations), which form the basic unit. • Hence nested queries within a query are identified as separate query blocks. • The query decomposer goes through five stages of processing for decomposition into low-level operation and translation into algebraic expressions.
  • 8.
  • 9. Query Analysis • During the query analysis phase, the query is syntactically analyzed using the programming language compiler (parser). • A syntactically legal query is then validated, using the system catalog, to ensure that all data objects (relations and attributes) referred to by the query are defined in the database. • The type specification of the query qualifiers and result is also checked at this stage.
  • 10. • Example: SELECT emp_nm FROM EMPLOYEE WHERE emp_desg>100 This query will be rejected because the comparison “>100” is incompatible with the data type of emp_desg which is a variable character string. • At the end of query analysis phase, the high-level query (SQL) is transformed into some internal representation that is more suitable for processing. • This internal representation is typically a kind of query tree.
  • 11. • A Query Tree is a tree data structure that corresponds expression. • A Query Tree is also called a relational algebra tree. – Leaf node of the tree, representing the base input relations of the query. – Internal nodes result of applying an operation in the algebra. – Root of the tree representing a result of the query.
  • 12. • SELECT (P.proj_no, P.dept_no, E.name, E.add, E.dob) • FROM PROJECT P, DEPARTMENT D, EMPLOYEE E • WHERE P.dept_no = D.d_no AND D.mgr_id = E.emp_id AND P.proj_loc = ‘Mumbai’ ;
  • 13.
  • 14. • The three relations PROJECT, DEPARTMENT, EMPLOYEE are represent as a leaf nodes P, D and E, while the relational algebra operations of the represented by internal tree nodes. • Same SQL query can have man different relational algebra expressions and hence many different query trees. • The query parser typically generates a standard initial (canonical) query tree.
  • 15. Query Normalization • The primary phase of the normalization is to avoid redundancy. • The normalization phase converts the query into a normalized form that can be more easily manipulated. • In the normalization phase, a set of equivalency rules are applied so that the projection and selection operations included on the query are simplified to avoid redundancy. • The projection operation corresponds to the SELECT clause of SQL query and the selection operation correspond to the predicate found in WHERE clause. • The equivalency transformation rules that are applied.
  • 16. Semantic Analyzer • The objective of this phase of query processing is to reduce the number of predicates. • The semantic analyzer rejects the normalized queries that are incorrectly formulated. • A query is incorrectly formulated if components do not contribute to the generation of result. • This happens in case of missing join specification. • A query is contradictory if its predicate cannot satisfy by any tuple in the relation. • The semantic analyzer examine the relational calculus query (SQL) to make sure it contains only data objects that is table, columns, views, indexes that are defined in the database catalog. • It makes sure that each object in the query is referenced correctly according to its data type. • In case of missing join specifications the components do not contribute to the generation of the results, and thus, a query may be incorrect formulated.
  • 17. Query Simplifier • The objectives of this phase are to detect redundant qualification, eliminate common sub-expressions and transform sub-graph to semantically equivalent but more easy and efficiently computed form. • Why to simplify? – Commonly integrity constraints, view definitions and access restrictions are introduced into the graph at this stage of analysis so that the query must be simplified as much as possible. – Integrity constraints defines constants which must holds for all state of database, so any query that contradict an integrity constraints must be avoid and can be rejected without accessing the database.
  • 18. Query Restructuring • In the final stage of the query decomposition, the query can be restructured to give a more efficient implementation. • Transformation rules are used to convert one relational algebra expression into an equivalent form that is more efficient. • The query can now be regarded as a relational algebra program, consisting of a series of operations on relation.
  • 19. Query Optimization • The primary goal of query optimization is of choosing an efficient execution strategy for processing a query. • The query optimizer attempts to minimize the use of certain resources (mainly the number of I/O and CPU time) by selecting a best execution plan (access plan). • A query optimization start during the validation phase by the system to validate the user has appropriate privileges. • Now an action plan is generate to perform the query.
  • 20. Block Diagram of Query Optimization
  • 21. • Relational algebra query tree generated by the query simplifier module of query decomposer. • Estimation formulas used to determine the cardinality of the intermediate result table. • A cost Model. • Statistical data from the database catalogue. The output of the query optimizer is the execution plan in form of optimized relational algebra query. A query typically has many possible execution strategies, and the process of choosing a suitable one for processing a query is known as Query Optimization.
  • 22. The basic issues in Query Optimization • How to use available indexes? • How to use memory to accumulate information and perform immediate steps such as sorting? • How to determine the order in which joins should be performed?
  • 23. Objective of query optimization • The term query optimization does not mean giving always an optimal (best) strategy as the execution plan. • It is just a responsibly efficient strategy for execution of the query. • The decomposed query block of SQL is translating into an equivalent extended relational algebra expression and then optimized.
  • 24. Techniques for Query Optimization • The first technique is based on Heuristic Rules for ordering the operations in a query execution strategy. • The second technique involves the systematic estimation of the cost of the different execution strategies and choosing the execution plan with the lowest cost.
  • 25. • Semantic query optimization is used with the combination with the heuristic query transformation rules. • It uses constraints specified on the database schema such as unique attributes and other more complex constraints, in order to modify one query into another query that is more efficient to execute.
  • 26. Heuristic Rules • The heuristic rules are used as an optimization technique to modify the internal representation of query. • Usually, heuristic rules are used in the form of query tree of query graph data structure, to improve its performance. • One of the main heuristic rule is to apply SELECT operation before applying the JOIN or other BINARY operations. • This is because the size of the file resulting from a binary operation such as JOIN is usually a multi- value function of the sizes of the input files.
  • 27. Heuristic Rules • The SELECT and PROJECT reduced the size of the file and hence, should be applied before the JOIN or other binary operation. • Heuristic query optimizer transforms the initial (canonical) query tree into final query tree using equivalence transformation rules. • This final query tree is efficient to execute.
  • 28. General Transformation Rules • Cascade of σ :- • σ c1 AND c2 AND …AND cn (R) = σ c1 (σ c2 (…(σ cn (R))…)) • Commutativity of σ :- • σ C1 (σ C2 (R)) = σ C2 (σ C1 (R)) Cascade of Л :- • Л List1 (Л List2 (…(Л List n (R))…)) = Л List1 (R) • Commuting σ with Л :- • Л A1,A2,A3…An (σ C (R) ) = σ C (Л A1,A2,A3…An (R))
  • 29. General Transformation Rules Commutativity of ⋈ AND x :- R ⋈ c S = S ⋈ c R R x S = S x R • Commuting σ with ⋈ or x :- If all attributes in selection condition c involved only attributes of one of the relation schemas (R). σ c (R ⋈ S) = (σ c (R) ) ⋈ S Alternatively, selection condition c can be written as (c1 AND c2) where condition c1 involves only attributes of R and condition c2 involves only attributes of S then : σ c (R ⋈ S) = (σ c1 (R) ) ⋈ (σ c2 (S) ) • Commuting Л with ⋈ or x :- The projection list L = {A1,A2,..An,B1,B2,…Bm}. A1…An attributes of R and B1…Bm attributes of S. Join condition C involves only attributes in L then : ЛL ( R ⋈ c S ) = ( ЛA1,…An (R) ) ⋈c ( ЛB1,…Bm(S) )
  • 30. General Transformation Rules • Commutative of SET Operation :- R ⋃ S = S ⋃ R R ⋂ S = S ⋂ R Minus (R-S) is not commutative. • Associatively of ⋈, x, ⋂, and ⋃ :- If ∅ stands for any one of these operation throughout the expression then : (R ∅ S) ∅ T = R ∅ (S ∅ T) • Commutativity of σ with SET Operation :- If ∅ stands for any one of three operations (⋃,⋂,and-) then : σ c (R ∅ S) = (σ c (R)) ⋃ (σ c (S)) Л c (R ∅ S) = (Л c (R)) ⋃ (Лc (S)) • The Л operation comute with ⋃ :- Л L (R ⋃ S) = (Л L(R)) ⋃ (Л L(S)) • Converting a (σ,x) sequence with ⋃ (σ c (R x S)) = (R ⋈ c S)