SlideShare a Scribd company logo
1 of 11
Dynamic SQL for efficient
      searching
        Ian Meade
Ian Meade
• Working with databases and applications for
  20 years.
• Working with SQL Server since 2001
• Certifications: MCM for SQL
  2008, MCITP, MCSD.NET
What is dynamic SQL
• Dynamic doesn’t mean exciting or
  interesting, it simply means that the SQL
  queries are created at run time.
• There are two main use cases for dynamic
  SQL.
  – DBAs to automate maintenance activates.
  – Used for complex and unpredictable queries
Why use dynamic SQL?
• Can lead to incredible performance and
  scalability increases.
  – Reductions in query execution time, memory
    used, IO required
  – Lead to reductions in locking / blocking, flushing
    memory buffers and contention with other parts
    of work load.
• Necessary as data sets grow to medium and
  larger sizes.
BUT….
• Dynamic SQL will not improve every type of
  query
• Issues around:
  – more difficult to write and test and to change over
    time
  – security concerns that need to be resolved
  – memory issues
  – does not always fit easily into development
    frameworks like ORM tools
Scary?
• Sounds like something to avoid?
• Yes – you should avoid dynamic SQL where
  possible
  – Only use it where it can help.
  – Be careful with coding, testing and security.
Alternatives..
• Live with the problem
  – Suitable for smaller datasets and possibly batch
    operations
• Hard code the query
  – Example
• Push functionality down to the client
• Hardcode parts of the query
Which queries to make dynamic
• Customer search is a good example.
• In most DBs, it usually has:
  – Large number of rows
  – Wide tables
  – Composed of several tables joined together
  – Unpredictable and complex search criteria
• Could be other parts of the DB.
  – People / employees, Products other domain
    specific entities
Implementing dynamic SQL
• Break query into parts
• Assemble the required parts
• Execute
  – Use sp_executesql to parameterise query
• Security concerns

• Example
Extending 1…
• Make the search a re-usable function

• Allows:
  – Re-use of search screen
  – Auditing by recording search results
  – Recording “as of” result sets
  – Extending search rules – eg security.

• Example
Extending 2…
• Making search meta-data driven
• Re-using complex SQL “framework”
• Suitable for applications with many similar
  search screens.

• Example

More Related Content

Similar to Dynamic sql for efficient searching

Kb 40 kevin_klineukug_reading20070717[1]
Kb 40 kevin_klineukug_reading20070717[1]Kb 40 kevin_klineukug_reading20070717[1]
Kb 40 kevin_klineukug_reading20070717[1]
shuwutong
 
Handling Massive Writes
Handling Massive WritesHandling Massive Writes
Handling Massive Writes
Liran Zelkha
 
Чурюканов Вячеслав, “Code simple, but not simpler”
Чурюканов Вячеслав, “Code simple, but not simpler”Чурюканов Вячеслав, “Code simple, but not simpler”
Чурюканов Вячеслав, “Code simple, but not simpler”
EPAM Systems
 
Storage Systems For Scalable systems
Storage Systems For Scalable systemsStorage Systems For Scalable systems
Storage Systems For Scalable systems
elliando dias
 
A data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madisonA data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madison
Terry Bunio
 
Collaborate 2011-tuning-ebusiness-416502
Collaborate 2011-tuning-ebusiness-416502Collaborate 2011-tuning-ebusiness-416502
Collaborate 2011-tuning-ebusiness-416502
kaziul Islam Bulbul
 
Ds03 data analysis
Ds03   data analysisDs03   data analysis
Ds03 data analysis
DotNetCampus
 

Similar to Dynamic sql for efficient searching (20)

Scaling apps for the big time
Scaling apps for the big timeScaling apps for the big time
Scaling apps for the big time
 
Kb 40 kevin_klineukug_reading20070717[1]
Kb 40 kevin_klineukug_reading20070717[1]Kb 40 kevin_klineukug_reading20070717[1]
Kb 40 kevin_klineukug_reading20070717[1]
 
Handling Massive Writes
Handling Massive WritesHandling Massive Writes
Handling Massive Writes
 
Чурюканов Вячеслав, “Code simple, but not simpler”
Чурюканов Вячеслав, “Code simple, but not simpler”Чурюканов Вячеслав, “Code simple, but not simpler”
Чурюканов Вячеслав, “Code simple, but not simpler”
 
Storage Systems For Scalable systems
Storage Systems For Scalable systemsStorage Systems For Scalable systems
Storage Systems For Scalable systems
 
AppSphere 15 - Is the database affecting your critical business transactions?
AppSphere 15 - Is the database affecting your critical business transactions?AppSphere 15 - Is the database affecting your critical business transactions?
AppSphere 15 - Is the database affecting your critical business transactions?
 
SQL Server Wait Types Everyone Should Know
SQL Server Wait Types Everyone Should KnowSQL Server Wait Types Everyone Should Know
SQL Server Wait Types Everyone Should Know
 
SQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data ArchitectureSQL, NoSQL, BigData in Data Architecture
SQL, NoSQL, BigData in Data Architecture
 
Designing, Building, and Maintaining Large Cubes using Lessons Learned
Designing, Building, and Maintaining Large Cubes using Lessons LearnedDesigning, Building, and Maintaining Large Cubes using Lessons Learned
Designing, Building, and Maintaining Large Cubes using Lessons Learned
 
Performance Optimization of Cloud Based Applications by Peter Smith, ACL
Performance Optimization of Cloud Based Applications by Peter Smith, ACLPerformance Optimization of Cloud Based Applications by Peter Smith, ACL
Performance Optimization of Cloud Based Applications by Peter Smith, ACL
 
ECMDay2015 - Kent Agerlund – Configuration Manager 2012 – A Site Review
ECMDay2015 - Kent Agerlund – Configuration Manager 2012 – A Site ReviewECMDay2015 - Kent Agerlund – Configuration Manager 2012 – A Site Review
ECMDay2015 - Kent Agerlund – Configuration Manager 2012 – A Site Review
 
A data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madisonA data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madison
 
Remote DBA Experts SQL Server 2008 New Features
Remote DBA Experts SQL Server 2008 New FeaturesRemote DBA Experts SQL Server 2008 New Features
Remote DBA Experts SQL Server 2008 New Features
 
Collaborate 2011-tuning-ebusiness-416502
Collaborate 2011-tuning-ebusiness-416502Collaborate 2011-tuning-ebusiness-416502
Collaborate 2011-tuning-ebusiness-416502
 
Ds03 data analysis
Ds03   data analysisDs03   data analysis
Ds03 data analysis
 
Pr dc 2015 sql server is cheaper than open source
Pr dc 2015 sql server is cheaper than open sourcePr dc 2015 sql server is cheaper than open source
Pr dc 2015 sql server is cheaper than open source
 
From Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@LohikaFrom Pilot to Product - Morning@Lohika
From Pilot to Product - Morning@Lohika
 
Roman Rehak: 24/7 Database Administration + Database Mail Unleashed
Roman Rehak: 24/7 Database Administration + Database Mail UnleashedRoman Rehak: 24/7 Database Administration + Database Mail Unleashed
Roman Rehak: 24/7 Database Administration + Database Mail Unleashed
 
Scaling a High Traffic Web Application: Our Journey from Java to PHP
Scaling a High Traffic Web Application: Our Journey from Java to PHPScaling a High Traffic Web Application: Our Journey from Java to PHP
Scaling a High Traffic Web Application: Our Journey from Java to PHP
 
Scaling High Traffic Web Applications
Scaling High Traffic Web ApplicationsScaling High Traffic Web Applications
Scaling High Traffic Web Applications
 

Dynamic sql for efficient searching

  • 1. Dynamic SQL for efficient searching Ian Meade
  • 2. Ian Meade • Working with databases and applications for 20 years. • Working with SQL Server since 2001 • Certifications: MCM for SQL 2008, MCITP, MCSD.NET
  • 3. What is dynamic SQL • Dynamic doesn’t mean exciting or interesting, it simply means that the SQL queries are created at run time. • There are two main use cases for dynamic SQL. – DBAs to automate maintenance activates. – Used for complex and unpredictable queries
  • 4. Why use dynamic SQL? • Can lead to incredible performance and scalability increases. – Reductions in query execution time, memory used, IO required – Lead to reductions in locking / blocking, flushing memory buffers and contention with other parts of work load. • Necessary as data sets grow to medium and larger sizes.
  • 5. BUT…. • Dynamic SQL will not improve every type of query • Issues around: – more difficult to write and test and to change over time – security concerns that need to be resolved – memory issues – does not always fit easily into development frameworks like ORM tools
  • 6. Scary? • Sounds like something to avoid? • Yes – you should avoid dynamic SQL where possible – Only use it where it can help. – Be careful with coding, testing and security.
  • 7. Alternatives.. • Live with the problem – Suitable for smaller datasets and possibly batch operations • Hard code the query – Example • Push functionality down to the client • Hardcode parts of the query
  • 8. Which queries to make dynamic • Customer search is a good example. • In most DBs, it usually has: – Large number of rows – Wide tables – Composed of several tables joined together – Unpredictable and complex search criteria • Could be other parts of the DB. – People / employees, Products other domain specific entities
  • 9. Implementing dynamic SQL • Break query into parts • Assemble the required parts • Execute – Use sp_executesql to parameterise query • Security concerns • Example
  • 10. Extending 1… • Make the search a re-usable function • Allows: – Re-use of search screen – Auditing by recording search results – Recording “as of” result sets – Extending search rules – eg security. • Example
  • 11. Extending 2… • Making search meta-data driven • Re-using complex SQL “framework” • Suitable for applications with many similar search screens. • Example