The document provides tips for speeding up SQL queries and database performance, including avoiding SELECT *, using indexes appropriately, normalizing tables, parameterizing queries, and optimizing stored procedures. Specific suggestions include explicitly selecting columns, using memory tables for frequently accessed lookup tables, and increasing query timeouts for long running reports.
This presentation discusses performance and scalability testing and optimization for Drupal websites. It covers capacity planning, tools for testing and analysis, common bottlenecks, caching, database optimization, and Apache configuration tips. Specific technologies and modules mentioned include Varnish, APC, MySQL query cache, and Drupal caching.
Building a Fast, Reliable SQL Server for kCura RelativityBrent Ozar
You want Relativity to run as fast as possible, but you don't have an unlimited budget. Microsoft Certified Master Brent Ozar will give you real-life stories from shops with high-performance environments. You’ll learn how to choose between virtual and physical servers, understand different high-availability methods, and discover why clustering is such a no-brainer.
You host Relativity data in SQL Server, and you can't just go buy a new server. You need to figure out where your server's bottleneck is, and find the easiest workaround to make it go faster. Microsoft Certified MasterBrent Ozar will show you how to check your SQL Server's vital stats with free tools, and then determine whether you need indexes, memory, or different config settings.
What I Learned About SQL Server at Ignite 2015Brent Ozar
SQL Server 2016 will include improvements to availability groups, columnstore indexes, security features, and performance tuning capabilities. Some key updates include:
- Availability groups will add support for distributed transaction coordination and faster failover capabilities.
- Columnstore indexes will gain update capabilities and the ability to create regular indexes on top of clustered columnstore indexes to improve concurrency.
- New security features include row-level security, dynamic data masking, and always encrypted for encrypting sensitive data in the client.
- Performance tuning demonstrations showed how to identify and address waits like PAGEIOLATCH through features like columnstore indexes or trace flags to improve query processing speed.
This presentation explains how to get started with Apache Cassandra to provide a scale out, fault tolerant backend for inventory storage on OpenSimulator.
You’re a developer or a DBA, and you’re comfortable writing queries to get the data you need. You’re much less comfortable trying to design the right indexes for your database server. In this series of videos, you’ll learn how the SQL Server engine looks at your indexes and builds your query results.
Introduction to SQL Server Internals: How to Think Like the EngineBrent Ozar
When you pass in a query, how does SQL Server build the results? Time to role play: Brent will be an end user sending in queries, and you will play the part of the SQL Server engine. Using simple spreadsheets as your tables, you will learn how SQL Server builds execution plans, uses indexes, performs joins, and considers statistics.
This session is for DBAs and developers who are comfortable writing queries, but not so comfortable when it comes to explaining nonclustered indexes, lookups, and sargability.
Dynamic SQL: How to Build Fast Multi-Parameter Stored ProceduresBrent Ozar
This document discusses best practices for building fast stored procedures that accept multiple parameters. It begins by describing a common business need to build a search page and outlines four approaches to building the stored procedure: 1) using OR conditions, 2) COALESCE, 3) dynamic SQL, 4) combining OR and dynamic SQL. Each approach is demonstrated with examples and limitations are discussed. While dynamic SQL allows for different execution plans per parameter combination, it can lead to bloating the plan cache and potential parameter sniffing issues. The document recommends techniques for troubleshooting dynamic SQL like using comments and debug variables.
This presentation discusses performance and scalability testing and optimization for Drupal websites. It covers capacity planning, tools for testing and analysis, common bottlenecks, caching, database optimization, and Apache configuration tips. Specific technologies and modules mentioned include Varnish, APC, MySQL query cache, and Drupal caching.
Building a Fast, Reliable SQL Server for kCura RelativityBrent Ozar
You want Relativity to run as fast as possible, but you don't have an unlimited budget. Microsoft Certified Master Brent Ozar will give you real-life stories from shops with high-performance environments. You’ll learn how to choose between virtual and physical servers, understand different high-availability methods, and discover why clustering is such a no-brainer.
You host Relativity data in SQL Server, and you can't just go buy a new server. You need to figure out where your server's bottleneck is, and find the easiest workaround to make it go faster. Microsoft Certified MasterBrent Ozar will show you how to check your SQL Server's vital stats with free tools, and then determine whether you need indexes, memory, or different config settings.
What I Learned About SQL Server at Ignite 2015Brent Ozar
SQL Server 2016 will include improvements to availability groups, columnstore indexes, security features, and performance tuning capabilities. Some key updates include:
- Availability groups will add support for distributed transaction coordination and faster failover capabilities.
- Columnstore indexes will gain update capabilities and the ability to create regular indexes on top of clustered columnstore indexes to improve concurrency.
- New security features include row-level security, dynamic data masking, and always encrypted for encrypting sensitive data in the client.
- Performance tuning demonstrations showed how to identify and address waits like PAGEIOLATCH through features like columnstore indexes or trace flags to improve query processing speed.
This presentation explains how to get started with Apache Cassandra to provide a scale out, fault tolerant backend for inventory storage on OpenSimulator.
You’re a developer or a DBA, and you’re comfortable writing queries to get the data you need. You’re much less comfortable trying to design the right indexes for your database server. In this series of videos, you’ll learn how the SQL Server engine looks at your indexes and builds your query results.
Introduction to SQL Server Internals: How to Think Like the EngineBrent Ozar
When you pass in a query, how does SQL Server build the results? Time to role play: Brent will be an end user sending in queries, and you will play the part of the SQL Server engine. Using simple spreadsheets as your tables, you will learn how SQL Server builds execution plans, uses indexes, performs joins, and considers statistics.
This session is for DBAs and developers who are comfortable writing queries, but not so comfortable when it comes to explaining nonclustered indexes, lookups, and sargability.
Dynamic SQL: How to Build Fast Multi-Parameter Stored ProceduresBrent Ozar
This document discusses best practices for building fast stored procedures that accept multiple parameters. It begins by describing a common business need to build a search page and outlines four approaches to building the stored procedure: 1) using OR conditions, 2) COALESCE, 3) dynamic SQL, 4) combining OR and dynamic SQL. Each approach is demonstrated with examples and limitations are discussed. While dynamic SQL allows for different execution plans per parameter combination, it can lead to bloating the plan cache and potential parameter sniffing issues. The document recommends techniques for troubleshooting dynamic SQL like using comments and debug variables.
SQL Server Wait Types Everyone Should KnowDean Richards
Many people use wait types for performance tuning, but do not know what some of the most common ones indicate. This presentation will go into details about the top 8 wait types I see at the customers I work with. It will provide wait descriptions as well as solutions.
Cassandra Community Webinar | Data Model on FireDataStax
Functional data models are great, but how can you squeeze out more performance and make them awesome? Let's talk through some example Cassandra 2.0 models, go through the tuning steps and understand the tradeoffs. Many time's just a simple understanding of the underlying Cassandra 2.0 internals can make all the difference. I've helped some of the biggest companies in the world do this and I can help you. Do you feel the need for Cassandra 2.0 speed?
The document discusses 14 ways to kill MySQL performance, including not using EXPLAIN for queries, choosing arbitrary data types, using persistent connections, not getting the right storage engine, and not profiling or benchmarking. It provides suggestions for improving performance such as using EXPLAIN to optimize queries, choosing smaller data types, avoiding heavy database abstraction layers, using the proper storage engine for your needs, and profiling and benchmarking your database.
This document provides an overview of NoSQL databases. It begins by defining NoSQL as non-relational databases that are distributed, open source, and horizontally scalable. It then discusses some of the limitations of relational databases that led to the rise of NoSQL, such as issues with scalability and the need for flexible schemas. The document also summarizes some key NoSQL concepts, including the CAP theorem, ACID versus BASE, and eventual consistency. It provides examples of use cases for NoSQL databases and discusses some common NoSQL database types and how they address scalability.
Design Patterns for Distributed Non-Relational Databasesguestdfd1ec
The document discusses design patterns for distributed non-relational databases, including consistent hashing for key placement, eventual consistency models, vector clocks for determining history, log-structured merge trees for storage layout, and gossip protocols for cluster management without a single point of failure. It raises questions to ask presenters about scalability, reliability, performance, consistency models, cluster management, data models, and real-life considerations for using such systems.
This document discusses operational and performance concerns with large node Cassandra deployments prior to version 1.2, and improvements made in versions 1.2 through 2.1 to better support large nodes. Memory structures like bloom filters and compression metadata that previously grew with data size are now stored off-heap. The number of token ranges per node was increased from 1 to 256 with virtual nodes. Disk I/O was improved through "JBOD" support and failure policies. Repair and compaction algorithms were enhanced. These changes alleviate many issues with large Cassandra nodes.
SQL Server instances allow separate deployments of Database Engine, Analysis Services, and Reporting Services on the same computer. Each instance has its own program files, directory, and security configuration. Multiple instances enable deploying SQL Server multiple times without extra operating system deployments, with SQL Server 2012 supporting up to 50 instances. Instance-level settings like memory allocation apply only to the configured instance, not other instances on the host computer.
This document discusses best practices for migrating database workloads to Azure Infrastructure as a Service (IaaS). Some key points include:
- Choosing the appropriate VM series like E or M series optimized for database workloads.
- Using availability zones and geo-redundant storage for high availability and disaster recovery.
- Sizing storage correctly based on the database's input/output needs and using premium SSDs where needed.
- Migrating existing monitoring and management tools to the cloud to provide familiarity and automating tasks like backups, patching, and problem resolution.
SQL Server ASYNC_NETWORK_IO Wait Type ExplainedConfio Software
When a SQL Server session waits on the async network io event, it may be encountering issues with the network or with aclient application not processing the data quickly enough. If the wait times for "async network io" are high, review the client application to see if large results sets are being sent to the client. If they are, work with the developers to understand if all the data is needed and reduce the size of result set if possible. Learn tips and techniques for decreasing decrease waits for async_network_io in this presentation.
The document discusses scalable storage systems and key-value stores as an alternative to traditional databases. It provides an overview of vertical and horizontal scalability. Traditional databases are not well-suited for scalable systems due to their complexity, wasted features, and multi-step query processing. Key-value stores offer simpler data models and interfaces that are designed from the start for scaling across hundreds of machines. Performance comparisons show key-value stores significantly outperforming traditional databases. The document also outlines how key-value storage systems work at the aggregation and storage layers.
This document provides an overview of how to successfully migrate Oracle workloads to Microsoft Azure. It begins with an introduction of the presenter and their experience. It then discusses why customers might want to migrate to the cloud and the different Azure database options available. The bulk of the document outlines the key steps in planning and executing an Oracle workload migration to Azure, including sizing, deployment, monitoring, backup strategies, and ensuring high availability. It emphasizes adapting architectures for the cloud rather than directly porting on-premises systems. The document concludes with recommendations around automation, education resources, and references for Oracle-Azure configurations.
Enterprise NoSQL: Silver Bullet or Poison PillBilly Newport
Enterprise NoSQL Silver bullet or poison pill? discusses the pros and cons of NoSQL databases compared to SQL databases. While SQL databases will remain prevalent, NoSQL databases offer alternative data storage options with different tradeoffs. NoSQL systems typically relax constraints of SQL like schema rigidity in exchange for implementation flexibility, but this comes at the cost of features like joins and global indexes. NoSQL also shifts the system of record away from a single database, requiring applications to handle consistency and creating multiple copies of data to scale.
I'm from California and often the only database administrator (DBA) that tech startups in the area will consult with. This document discusses the use of multiple database technologies at different companies, including MySQL, Oracle, Hadoop, and various NoSQL databases. It argues that while a relational database can solve most problems, different specialized databases are needed for the remaining 10% of problems, and that database experts should expand their skills beyond a single database.
Building scalable application with sql serverChris Adkin
Chris Adkin has 15 years of IT experience and 14 years of experience as a DBA working with various sectors. He has over 10 years of experience with SQL Server from version 2000. He provides his email and Twitter contact information. The document then discusses various topics related to database design and performance including OLTP vs OLAP characteristics, data modeling best practices, indexing strategies, query optimization techniques, and concurrency control methods.
MySQL 8 Tips and Tricks from Symfony USA 2018, San FranciscoDave Stokes
This document discusses several new features in MySQL 8 including:
1. A new transactional data dictionary that stores metadata instead of files for improved simplicity and crash safety.
2. The addition of histograms to help the query optimizer understand data distributions without indexes for better query planning.
3. Resource groups that allow assigning threads to groups with specific CPU and memory limits to control resource usage.
4. Enhancements to JSON support like in-place updates and new functions for improved flexibility with semi-structured data.
1) The document discusses using decision tables to drive the development of data-driven applications. Decision trees are converted to tables stored in a database to define application logic and behavior.
2) Storing decision tables in the database allows changing application behavior by altering the tables. It also enables fast queries by searching for results matching conditions.
3) The author finds decision table-driven development helps testing, maintains application logic separately from code, and eases maintenance through the lifetime of an application.
A Web architect who doesn't usually use Microsoft technologies explains Microsoft's cloud offering, Azure, in terms that hopefully other UNIX/open source people will understand.
World-class Data Engineering with Amazon RedshiftLars Kamp
These are the slides used in the Redshift training by intermix.io. This class introduces you to strategies and best practices for designing a data platform using Amazon Redshift.
For a link to the video, please contact nikola@intermix.io.
10 sql tips to speed up your database cats whocode.comKaing Menglieng
This article provides 10 tips for optimizing and speeding up a SQL database:
1. Carefully design the database structure with clear table and column naming.
2. Use the EXPLAIN statement to understand query performance and identify areas for optimization.
3. Implement query caching to avoid repeatedly running the same queries.
4. Only select the necessary columns rather than using "*" to reduce overhead.
5. Use LIMIT to restrict large result sets when only a subset is needed.
6. Avoid running queries in loops which significantly increases load.
7. Prefer joins over subqueries when possible for better performance.
8. Use wildcards judiciously as full table scans are slow
SQL Server Wait Types Everyone Should KnowDean Richards
Many people use wait types for performance tuning, but do not know what some of the most common ones indicate. This presentation will go into details about the top 8 wait types I see at the customers I work with. It will provide wait descriptions as well as solutions.
Cassandra Community Webinar | Data Model on FireDataStax
Functional data models are great, but how can you squeeze out more performance and make them awesome? Let's talk through some example Cassandra 2.0 models, go through the tuning steps and understand the tradeoffs. Many time's just a simple understanding of the underlying Cassandra 2.0 internals can make all the difference. I've helped some of the biggest companies in the world do this and I can help you. Do you feel the need for Cassandra 2.0 speed?
The document discusses 14 ways to kill MySQL performance, including not using EXPLAIN for queries, choosing arbitrary data types, using persistent connections, not getting the right storage engine, and not profiling or benchmarking. It provides suggestions for improving performance such as using EXPLAIN to optimize queries, choosing smaller data types, avoiding heavy database abstraction layers, using the proper storage engine for your needs, and profiling and benchmarking your database.
This document provides an overview of NoSQL databases. It begins by defining NoSQL as non-relational databases that are distributed, open source, and horizontally scalable. It then discusses some of the limitations of relational databases that led to the rise of NoSQL, such as issues with scalability and the need for flexible schemas. The document also summarizes some key NoSQL concepts, including the CAP theorem, ACID versus BASE, and eventual consistency. It provides examples of use cases for NoSQL databases and discusses some common NoSQL database types and how they address scalability.
Design Patterns for Distributed Non-Relational Databasesguestdfd1ec
The document discusses design patterns for distributed non-relational databases, including consistent hashing for key placement, eventual consistency models, vector clocks for determining history, log-structured merge trees for storage layout, and gossip protocols for cluster management without a single point of failure. It raises questions to ask presenters about scalability, reliability, performance, consistency models, cluster management, data models, and real-life considerations for using such systems.
This document discusses operational and performance concerns with large node Cassandra deployments prior to version 1.2, and improvements made in versions 1.2 through 2.1 to better support large nodes. Memory structures like bloom filters and compression metadata that previously grew with data size are now stored off-heap. The number of token ranges per node was increased from 1 to 256 with virtual nodes. Disk I/O was improved through "JBOD" support and failure policies. Repair and compaction algorithms were enhanced. These changes alleviate many issues with large Cassandra nodes.
SQL Server instances allow separate deployments of Database Engine, Analysis Services, and Reporting Services on the same computer. Each instance has its own program files, directory, and security configuration. Multiple instances enable deploying SQL Server multiple times without extra operating system deployments, with SQL Server 2012 supporting up to 50 instances. Instance-level settings like memory allocation apply only to the configured instance, not other instances on the host computer.
This document discusses best practices for migrating database workloads to Azure Infrastructure as a Service (IaaS). Some key points include:
- Choosing the appropriate VM series like E or M series optimized for database workloads.
- Using availability zones and geo-redundant storage for high availability and disaster recovery.
- Sizing storage correctly based on the database's input/output needs and using premium SSDs where needed.
- Migrating existing monitoring and management tools to the cloud to provide familiarity and automating tasks like backups, patching, and problem resolution.
SQL Server ASYNC_NETWORK_IO Wait Type ExplainedConfio Software
When a SQL Server session waits on the async network io event, it may be encountering issues with the network or with aclient application not processing the data quickly enough. If the wait times for "async network io" are high, review the client application to see if large results sets are being sent to the client. If they are, work with the developers to understand if all the data is needed and reduce the size of result set if possible. Learn tips and techniques for decreasing decrease waits for async_network_io in this presentation.
The document discusses scalable storage systems and key-value stores as an alternative to traditional databases. It provides an overview of vertical and horizontal scalability. Traditional databases are not well-suited for scalable systems due to their complexity, wasted features, and multi-step query processing. Key-value stores offer simpler data models and interfaces that are designed from the start for scaling across hundreds of machines. Performance comparisons show key-value stores significantly outperforming traditional databases. The document also outlines how key-value storage systems work at the aggregation and storage layers.
This document provides an overview of how to successfully migrate Oracle workloads to Microsoft Azure. It begins with an introduction of the presenter and their experience. It then discusses why customers might want to migrate to the cloud and the different Azure database options available. The bulk of the document outlines the key steps in planning and executing an Oracle workload migration to Azure, including sizing, deployment, monitoring, backup strategies, and ensuring high availability. It emphasizes adapting architectures for the cloud rather than directly porting on-premises systems. The document concludes with recommendations around automation, education resources, and references for Oracle-Azure configurations.
Enterprise NoSQL: Silver Bullet or Poison PillBilly Newport
Enterprise NoSQL Silver bullet or poison pill? discusses the pros and cons of NoSQL databases compared to SQL databases. While SQL databases will remain prevalent, NoSQL databases offer alternative data storage options with different tradeoffs. NoSQL systems typically relax constraints of SQL like schema rigidity in exchange for implementation flexibility, but this comes at the cost of features like joins and global indexes. NoSQL also shifts the system of record away from a single database, requiring applications to handle consistency and creating multiple copies of data to scale.
I'm from California and often the only database administrator (DBA) that tech startups in the area will consult with. This document discusses the use of multiple database technologies at different companies, including MySQL, Oracle, Hadoop, and various NoSQL databases. It argues that while a relational database can solve most problems, different specialized databases are needed for the remaining 10% of problems, and that database experts should expand their skills beyond a single database.
Building scalable application with sql serverChris Adkin
Chris Adkin has 15 years of IT experience and 14 years of experience as a DBA working with various sectors. He has over 10 years of experience with SQL Server from version 2000. He provides his email and Twitter contact information. The document then discusses various topics related to database design and performance including OLTP vs OLAP characteristics, data modeling best practices, indexing strategies, query optimization techniques, and concurrency control methods.
MySQL 8 Tips and Tricks from Symfony USA 2018, San FranciscoDave Stokes
This document discusses several new features in MySQL 8 including:
1. A new transactional data dictionary that stores metadata instead of files for improved simplicity and crash safety.
2. The addition of histograms to help the query optimizer understand data distributions without indexes for better query planning.
3. Resource groups that allow assigning threads to groups with specific CPU and memory limits to control resource usage.
4. Enhancements to JSON support like in-place updates and new functions for improved flexibility with semi-structured data.
1) The document discusses using decision tables to drive the development of data-driven applications. Decision trees are converted to tables stored in a database to define application logic and behavior.
2) Storing decision tables in the database allows changing application behavior by altering the tables. It also enables fast queries by searching for results matching conditions.
3) The author finds decision table-driven development helps testing, maintains application logic separately from code, and eases maintenance through the lifetime of an application.
A Web architect who doesn't usually use Microsoft technologies explains Microsoft's cloud offering, Azure, in terms that hopefully other UNIX/open source people will understand.
World-class Data Engineering with Amazon RedshiftLars Kamp
These are the slides used in the Redshift training by intermix.io. This class introduces you to strategies and best practices for designing a data platform using Amazon Redshift.
For a link to the video, please contact nikola@intermix.io.
10 sql tips to speed up your database cats whocode.comKaing Menglieng
This article provides 10 tips for optimizing and speeding up a SQL database:
1. Carefully design the database structure with clear table and column naming.
2. Use the EXPLAIN statement to understand query performance and identify areas for optimization.
3. Implement query caching to avoid repeatedly running the same queries.
4. Only select the necessary columns rather than using "*" to reduce overhead.
5. Use LIMIT to restrict large result sets when only a subset is needed.
6. Avoid running queries in loops which significantly increases load.
7. Prefer joins over subqueries when possible for better performance.
8. Use wildcards judiciously as full table scans are slow
PASS Spanish Recomendaciones para entornos de SQL Server productivosJavier Villegas
This document summarizes an upcoming webinar on recommendations for productive SQL Server environments. The webinar will cover topics like index rebuilds, updating statistics, sp_configure settings, tempdb optimization, the model database, and optimizing backup and restore times. It provides details on the speaker Javier Villegas and moderator Kenneth Urena. It also lists upcoming PASS events on Azure SQL Database and SQL Server Analysis Services and encourages users to stay connected through the PASS social media channels.
The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication, and data federation. The overall goal of application architecture is to scale traffic and data while maintaining high availability and performance. Horizontal scaling by adding more servers is preferable to vertical scaling of buying larger servers.
The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication and sharding, and asynchronous queuing to distribute workloads across multiple servers. The goal of these patterns is to scale traffic, data size, and maintainability through horizontal expansion rather than just vertical upgrades.
The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication and sharding, and asynchronous queuing to distribute workloads across multiple servers. The goal of these patterns is to scale traffic, data size, and maintainability through horizontal expansion rather than just vertical upgrades.
UnConference for Georgia Southern Computer Science March 31, 2015Christopher Curtin
I presented to the Georgia Southern Computer Science ACM group. Rather than one topic for 90 minutes, I decided to do an UnConference. I presented them a list of 8-9 topics, let them vote on what to talk about, then repeated.
Each presentation was ~8 minutes, (Except Career) and was by no means an attempt to explain the full concept or technology. Only to wake up their interest.
Quick And Easy Guide To Speeding Up MySQL for web developersJonathan Levin
This document provides a quick guide to speeding up MySQL for website developers, covering topics such as monitoring MySQL performance, choosing the right data types and storage engines, understanding some MySQL internals like key buffers and query caches, using replication for reads and writes, and improving application performance through techniques like de-normalizing data, caching query results, and storing more data instead of logic in the database. The goal is to help developers optimize MySQL so that it works better for their applications with only basic tuning when necessary.
White Paper for OMG! Identifying and Refactoring Common SQL...Jeff Jacobs
The document discusses common SQL anti-patterns that lead to poor database performance. It identifies issues like overuse of inline views, unnecessary outer joins, and updating primary keys. The document proposes techniques called the "OMG Method" to refactor SQL and fix performance problems, such as replacing IN with EXISTS, using subquery factoring with WITH, and properly applying index hints. The goal is to help developers write more efficient SQL without requiring advanced privileges.
The document discusses various techniques for scaling databases and applications, including caching, replication, functional partitioning, sharding, batching, buffering, queuing, and background processing. It provides examples of when and how to implement these techniques, as well as considerations around caching policies, data distribution strategies, and managing asynchronous replication. The goal is to optimize performance and scalability through techniques that reduce round trips, parallelize operations, and distribute load across servers and databases.
This document provides an overview of NoSQL databases and key-value stores. It discusses why NoSQL databases were created, examples of different NoSQL categories like key-value stores and document stores. It then focuses on key-value stores like Memcached and MemcacheDB. Memcached is an in-memory key-value store while MemcacheDB provides persistence. Both use the BerkeleyDB for storage with MemcacheDB.
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson
The document discusses common patterns and approaches for scaling web architectures. It covers topics like load balancing, caching, database scaling through replication and sharding, high availability, and storing large files across multiple servers and data centers. The overall goal is to discuss how to architect systems that can scale horizontally to handle increasing traffic and data sizes.
What is your sql server backup strategy tech_republicKaing Menglieng
The document discusses various SQL Server backup strategies including full backups, differential backups, transaction log backups, and file group backups. It explains how to implement each strategy using either SQL Server Enterprise Manager graphical user interface or Transact SQL commands. Full backups create a complete picture of the database, differential backups back up only changed data since the last full backup, transaction log backups back up transactions and allow point-in-time recovery, and file group backups allow backing up individual files or groups.
Using sql server 2008's merge statement tech republicKaing Menglieng
The document discusses SQL Server 2008's new MERGE statement, which allows inserting, updating, or deleting data in one table based on conditions in another table in a single statement. It provides an example using the MERGE statement to load sales data from a staging table into a reporting table, handling inserting new records as well as updating existing records with new sales amounts and counts. The MERGE statement combines the logic of separate insert, update, and delete statements into one statement for efficiently modifying data between tables.
Using object dependencies in sql server 2008 tech republicKaing Menglieng
This document discusses how SQL Server 2008 improves on tracking object dependencies by name rather than ID. It introduces two new dynamic management functions and a view for tracking dependencies. An example creates database objects with dependencies and queries the new functions and view to identify referenced and referencing objects. The conclusion states advantages of tracking dependencies by name include finding objects referencing those that don't exist.
Using hash fields in sql server tech republicKaing Menglieng
The document discusses using hash fields in SQL Server. It explains that hash fields can be used for auditing data changes and capturing data for data warehouses. It shows how to create a hash field using the BINARY_CHECKSUM function on multiple fields. This creates a hash value that changes when the fields change, allowing detection of data modifications. It demonstrates adding a hash field to a sample SalesHistory table and discusses how hash fields can simplify change data capture for data warehousing.
Using grouping sets in sql server 2008 tech republicKaing Menglieng
This article discusses the new GROUPING SETS clause in SQL Server 2008, which allows specifying combinations of field groupings in queries to see different levels of aggregated data. It provides an example showing how GROUPING SETS can return aggregated data at both the product level and grand total level. Another example shows returning aggregates by product, sales tier, and grand total. The article notes this functionality enhances reporting by returning multiple aggregations in one statement rather than separate queries.
Understand when to use user defined functions in sql server tech-republicKaing Menglieng
User-defined functions (UDFs) in SQL Server allow users to define custom functions that can accept parameters and return values. There are two main types of UDFs - table-valued functions that return results in a table that can be queried, and scalar-valued functions that return a single value. The document provides examples of creating both types of UDFs and using them to return sales data from a sample SalesHistory table based on input parameters.
Sql server indexed views speed up your select queries part 1 - code-projecKaing Menglieng
This document discusses indexed views in SQL Server, which allow for optimizing select queries. Indexed views store clustered index data to provide another location for the query optimizer to find data. They are best suited for read-heavy databases like data warehouses. The document covers how to create indexed views, including using schema binding and unique clustered indexes, and constraints like deterministic queries. It provides an example of creating a view and index on a table to optimize queries against that data.
This document discusses how to optimize SQL queries by removing bookmark lookups, RID lookups, and key lookups. These lookups occur when a query uses a non-clustered index to seek on columns in the WHERE clause, but must then perform further lookups to return non-key columns in the SELECT clause. The document demonstrates how to remove these lookups using two methods: 1) creating a non-clustered index that covers all columns in the query, or 2) creating an included column non-clustered index. Removing lookups in this way improves query performance.
Sql server common interview questions and answersKaing Menglieng
The document discusses common interview questions and answers related to SQL Server. It provides 6 questions and answers about topics like the TCP/IP port SQL Server uses, the differences between clustered and non-clustered indexes, index configurations for tables, collation sensitivity types, what OLTP is, and the differences between primary and unique keys.
Sql server common interview questions and answers page 6Kaing Menglieng
This document discusses common SQL Server interview questions and answers. It defines BCP as a tool used to copy large amounts of data between tables or views without copying structures. It also describes how to implement one-to-one, one-to-many, and many-to-many relationships in table design. Finally, it explains that an execution plan shows the query optimization methods chosen by SQL Server and can be viewed within Query Analyzer to understand query performance.
Sql server common interview questions and answers page 5Kaing Menglieng
This document contains 30 common interview questions and answers about SQL Server. Some of the questions covered include: what is a NOT NULL constraint, how to get @@ERROR and @@ROWCOUNT at the same time, what are the advantages of using stored procedures, what is a table called if it has neither a cluster nor non-cluster index, and if SQL Server can be linked to other servers like Oracle.
Sql server common interview questions and answers page 4Kaing Menglieng
This document discusses differences between local and global temporary tables in SQL Server. A local temporary table exists only for the duration of a connection, while a global temporary table remains in the database permanently but the rows only exist within a given connection. It also explains the STUFF and REPLACE functions, where STUFF overwrites existing characters at a given start position and length, while REPLACE replaces all occurrences of a search string. Primary keys, unique keys, foreign keys and check constraints are also summarized. Primary keys uniquely identify rows, unique keys enforce uniqueness of column values, foreign keys prevent orphaned rows, and check constraints limit column values.
Sql server common interview questions and answers page 2Kaing Menglieng
The document discusses differences between the DELETE and TRUNCATE commands in SQL Server. DELETE removes rows one by one and can be rolled back, while TRUNCATE removes all rows faster but cannot be rolled back. It also discusses the use of UPDATE_STATISTICS to update indexes after large data changes, and the differences between the HAVING and WHERE clauses. Finally, it defines SQL Profiler as a tool to monitor SQL Server events and lists the two authentication modes in SQL Server (Windows and Mixed) that can be changed in the SQL Server Configuration Manager.
Sql server – 2008 – hardware and software requirements for installing sql seKaing Menglieng
The document provides the minimum hardware and software requirements for installing SQL Server 2008. It lists requirements such as the .NET Framework 3.5, Windows Installer 4.5, Internet Explorer 6 SP1, and various supported Windows operating systems. It also provides specific requirements for installing different editions of SQL Server 2008 such as the Enterprise edition requiring a minimum 1 GHz processor and 512 MB of RAM.
Speeding up queries with semi joins and anti-joinsKaing Menglieng
- Semi-joins return rows from the first table where matches are found in the second table, but return each matching row from the first table only once. Anti-joins return rows from the first table where no matches are found in the second table.
- Semi-joins can be written using EXISTS or IN clauses, while anti-joins use NOT EXISTS or NOT IN. This allows Oracle to use more efficient semi-join and anti-join access paths.
- Oracle has improved at optimizing queries with EXISTS and IN clauses over time, and in Oracle 9i and later the choice is usually not important as Oracle will generate efficient plans for both.
Speed up sql server apps - visual studio magazineKaing Menglieng
This article provides 10 tips for improving the performance of SQL Server applications. Some of the tips include using EXISTS instead of COUNT(*) when checking for existence, being careful when using WHERE IN and WHERE NOT IN clauses, randomizing result sets with NEWID(), and increasing the default packet size for transferring large data fields.
See sql server graphical execution plans in action tech republicKaing Menglieng
Tim Chapman identifies a few basic things to look for in SQL Server graphical execution plans to understand how indexes are used. He creates a sample database with different indexes and queries it in various ways to demonstrate index seeks, scans, and lookups. The article shows how indexes can improve query performance when columns have high selectivity but may not help when columns have few distinct values.
Reviewing sql server permissions tech republicKaing Menglieng
The document reviews SQL Server permissions. It discusses reviewing login information using the sys.server_principals view, determining database users using sys.database_principals, viewing roles assignments with other system views, and identifying object permissions with sys.database_permissions. Examples are provided to test adding a login, user, and role membership. The document aims to help administrators understand permissions on their SQL Server instance.
Query optimization how to search millions of record in sql table faster -Kaing Menglieng
The document is a question posted on Stack Overflow asking how to search millions of records in a SQL table faster. The question details searching a table with 50 million records and 5 columns to find records containing "lifeis". Indexing did not help speed up the LIKE query. Respondents recommend using a full-text index, which supports faster LIKE queries for patterns beginning with a fixed string. While an index on the searched column could provide some help, a full-text index is likely the best approach given the large number of records.
Optimize sql server queries with these advanced tuning techniques tech repuKaing Menglieng
The document discusses several advanced techniques for optimizing SQL Server queries:
1. Using JOIN statements instead of subqueries to improve performance.
2. Using explicit transactions around data manipulation statements to reduce writes to the transaction log.
3. Using UNION ALL instead of UNION to avoid removing duplicate records.
4. Using EXISTS instead of COUNT(*) when checking for conditions to return results faster.
5. Using STATISTICS IO to determine the number of logical reads of a query to optimize performance.
Optimize sql server queries with these advanced tuning techniques tech repu
Speed up sql
1. Speed Up SQL
CodeCharge Studio
Web Reporting
Visually create Web Reports in PHP, ASP, .NET, Java, Perl and ColdFusion.
CodeCharge.com
YesSoftware Forums -> CodeCharge Studio -> Tips & Solutions
Speed Up SQL
Author Message
feha Posted: 05/21/2006, 2:59 AM
googling i found this:
Quote :
Avoid making SELECT * calls, instead explicitly name all the columns you want to retrieve, and if you are using
MySQL, test your queries using the EXPLAIN statement. To optimize loops, consider using duplicated code
instead of low iteration loops; also use as many static values, such as count($array) values, inside the loop as
Posts: 711 you can, generating their values before the loop once.
_________________
Regards
feha
www.vision.to
feedpixel.com
marcwolf Posted: 05/30/2006, 11:54 PM
Depending on your database you can REALLY speed things up using memory tables.
We use MySQL and have a fairly large code lookup table. On some display we cah hit this tabel 10 times or more.
its often the only way to achieve normalisation on very large records.
We noted that the system was getting slower regardless of howmany indexes and optimisations we used.
Posts: 361
In the end we made a duplicat of the code table and assigned it the type of HEAP.
Now - as the code table is updaed in only one place we just tell MySQl to do a replace from the real table into the
HEAP one is there are any changes.
Speed increase was fantastic
With a little bit of work the above can be applied to any grouping of small code value tables like Country, State,
etc that takes a lot of hits.
Take Care
Dave
_________________
' Coding Coding Coding
Keep Those Keyboards Coding.
Raw Code!!!!!!!
mamboBROWN Posted: 05/31/2006, 7:49 AM
Upgrading MySQL
Also, keep in mind that upgrading MySQL to 5.x version also will give you some features that could aid in speeding
up your applications. Such as views and stored procedures.
http://forums.yessoftware.com/posts.php?post_id=75008[08/29/2012 4:20:12 PM]
2. Speed Up SQL
If that's not an option checkout possible configuration changes or purchase the MySQL book that gives you pointers
and hints on improving speed and response time of your MySQL database(s).
Posts: 1699
Andres Posted: 06/27/2006, 6:11 AM
Rormoser
SQL Server Speed UP Tips
1. Normalize your tables
There are two common excuses for not normalizing databases: performance and
pure laziness. You'll pay for the second one sooner or later; and, about
performance, don't optimize what's not slow. And, more frequent than the
inverse, the resulting design is slower. DBMS's were designed to be used
with normalized databases and SQL Server is no exception, so design with
normalization in mind.
2. Avoid using cursors
Use cursors wisely. Cursors are fundamentally evil. They force the database
engine to repeatedly fetch rows, negotiate blocking, manage locks, and
transmit results. They consume network bandwidth as the results are
transmitted back to the client, where they consume RAM, disk space, and
screen real estate. Consider the resources consumed by each cursor you build
and multiply this demand by the number of simultaneous users. Smaller is
better. And good DBAs, most of the time, know what they are doing. But, if
you are reading this, you are not a DBA, right?
Having said this the other question that comes is, If I were to use cursors
then .... ? Well here are my 20Cents on cursor usage. Use the appropriate
cursors for the job in hand.
· Don't use scrollable cursors unless required
· Use readonly cursors if you donot intend to update. This would be
90% of the situations.
· Try to use Forward Only cursor when using cursors
· Don't forget to close and deallocate the cursors used.
· Try to reduce the number of columns and records fetched in a
cursor
3. Index Columns
Create Index on columns that are going to be highly selective. Indexes are
vital to efficient data access; however, there is a cost associated with
creating and maintaining an index structure. For every insert, update and
delete, each index must be updated. In a data warehouse, this is acceptable,
but in a transactional database, you should weigh the cost of maintaining an
index on tables that incur heavy changes. The bottom line is to use
effective indexes judiciously. On analytical databases, use as many indexes
as necessary to read the data quickly and efficiently.
Now a classic example is DONOT index an column like "Gender". This would
have a selectivity of 50% and if your table is having 10 Million records,
you can be least assured that using this index you may have to travel half
the number of rows ... Hence maintaining such indexes can slow your
performance.
4. Use transactions
Use transaction judiciously. This will save you when things get wrong.
Working with data for some time you'll soon discover some unexpected
situation which will make your stored procured crash. See that the
transaction starts as late as possible and ends as early as possible. This
would reduce the requirement to lock down the resources while accessing. In
http://forums.yessoftware.com/posts.php?post_id=75008[08/29/2012 4:20:12 PM]
3. Speed Up SQL
short,
5. Analyze deadlocks
Access your tables on the same order always. When working with stored
procedures and transactions, you may find this soon. Any SQL programmer /
database analyst would have come across this problem. If the order changes
then there wold be a cyclic wait for resources to be released and the users
would experience a permanent hang in the application. Deadlocks can be
tricky to find if the lock sequence is not carefully designed. To summarize,
Deadlock occurs when two users have locks on separate objects and each user
is trying to lock the other user's objects. SQL Server automatically detects
and breaks the deadlock. The terminated transaction will be automatically
rolled back and an error code 1205 will be issued.
6. GOTO Usage
Avoid using the infamous GOTO. This is a time-proven means of adding
disorder to program flow. There are some cases where intelligent use of GOTO
is preferable to dogmatically refusing to use it. On the other hand,
unintelligent use of GOTO is a quick ticket to unreadable code.
7. Increase timeouts
When querying a database, the default timeout is often low, like 30 seconds.
Remember that report queries may run longer than this, specially when your
database grows. Hence increase this value to an acceptable value.
8. Avoid NULLable columns
When possible, normalize your table and separate your nullable columns. They
consume an extra byte on each NULLable column in each row and have more
overhead associated when querying data. It will be more flexible and faster,
and will reduce the NULLable columns. I'm not saying that NULLs are the evil
incarnation. I believe they can simplify coding when "missing data" is part
of your business rules.
9. TEXT datatype
Unless you are using it for really large data. The TEXT datatype is not
flexible to query, is slow and wastes a lot of space if used incorrectly.
Sometimes a VARCHAR will handle your data better. You can also look at the
"text in row" feature with the table options for SQL Server 2000. But still
I would stick to the first statement, Avoid using them on first place.
10. SELECT * Usage
Its very difficult to get out of this habit, but believe me this is very
essential. Please DONOT use this syntax. Always qualify the full list of
columns. Using all columns increases network traffic, requires more buffers
and processing, and could prove error prone if the table or view definition
changes.
11. Temporary tables usage
Unless strictly necessary. More often than not a subquery can substitute a
temporary table. In SQL Server 2000, there are alternatives like the TABLE
variable datatype which can provide in-memory solutions for small tables
inside stored procedures too. If I were to recollect some of the advantages
of using the same:
· A table variable behaves like a local variable. It has a
well-defined scope, which is the function, stored procedure, or batch in
which it is declared. Within its scope, a table variable may be used like a
regular table.
· However, table may not be used in the following statements: INSERT
INTO table_variable EXEC stored_procedure SELECT select_list INTO
table_variable statements.
· Table variables are cleaned up automatically at the end of the
function, stored procedure, or batch in which they are defined.
· Table variables used in stored procedures result in fewer
http://forums.yessoftware.com/posts.php?post_id=75008[08/29/2012 4:20:12 PM]
4. Speed Up SQL
recompilations of the stored procedures than their counterparts temporary
tables.
· Transactions involving table variables last only for the duration
of an update on the table variable. Thus, table variables require less
locking and logging resources
12. Using UDF
UDF can replace stored procedures. But be careful in their usage. Sometimes
UDFs can take a toll on your applications performance. And UDFs have to
prefixed with the owners name. This is not a drawback but a requirement. I
support usage of SPs more than UDFs.
13. Multiple User Scenario
Sometimes two users will edit the same record at the same time. While
writing back, the last writer wins and some of the updates will be lost.
It's easy to detect this situation: create a timestamp column and check it
before you write. Code for these practical situations and test your
application for these scenarios.
14. Use SCOPE_IDENTITY
Dont do SELECT max(ID) from MasterTable when inserting in a Details table.
This is a common mistake, and will fail when concurrent users are inserting
data at the same instance. Use one of SCOPE_IDENTITY or IDENT_CURRENT. My
choice would be SCOPE_IDENTITY as this would give you the identity value
from the current context in prespective.
15. Analyze Query Plans
The SQL Server query analyzer is a powerful tool. And surely is your friend,
and you'll learn a lot of how it works and how the query and index design
can affect performance through it. Understand the execution plan that the
execution plan window shows for potential bottlenecks.
16. Parameterized queries
Parameterize all your queries using the sp_executesql. This would help the
optimzer to chace the execution plans and use the same when requested teh
second time. You can cache-in the time required to parse, compile and place
the execution plan. Avoid using of D-SQL as much as possible.
17. Keep Procedures Small
Keep SPs small in size and scope. Two users invoking the same stored
procedure simultaneously will cause the procedure to create two query plans
in cache. It is much more efficient to have a stored procedure call other
ones then to have one large procedure.
18. Bulk INSERT
Use DTS or the BCP utility and you'll have both a flexible and fast
solution. Try avoiding use of Insert statement for the Buld loading feature,
they are not efficent and are not designed for the same.
19. Using JOINS
Make sure that there are n-1 join criteria if there are n tables.
Make sure that ALL tables included in the statement are joined. Make sure
that only tables that
· Have columns in the select clause
· Have columns referenced in the where clause
· Allow two unrelated tables to be joined together are included.
20. Trap Errors
Make sure that the @@ERROR global variable is checked after every statement
which causes an update to the database (INSERT, UPDATE, DELETE). Make sure
http://forums.yessoftware.com/posts.php?post_id=75008[08/29/2012 4:20:12 PM]
5. Speed Up SQL
that rollbacks (if appropriate) are performed prior to inserting rows into
an exception table
21. Small Result Set
Retrieving needlessly large result sets (for example, thousands of rows) for
browsing on the client adds CPU and network I/O load, makes the application
less capable of remote use, and limits multi-user scalability. It is better
to design the application to prompt the user for sufficient input so queries
are submitted that generates modest result sets.
22. Negative Arguments
Minimize the use of not equal operations, <> or !=. SQL Server has to scan a
table or index to find all values to see if they are not equal to the value
given in the expression. Try rephrasing the expression using ranges:
WHERE KeyColumn < 'TestValue' AND KeyColumn > 'TestValue'
23. Date Assumption
Prevent issues with the interpretation of centuries in dates, do not specify
years using two digits. Assuming dates formats is the first place to break
an application. Hence avoid making this assumption.
24. SP_ Name
DONOT start the name of a stored procedure with SP_. This is because all the
system related stored procedures follow this convention. Hence a valid
procedure today may clash with the naming convention of a system procedure
that gets bundled with a Service pack / Security patch tomorrow. Hence do
not follow this convention.
25. Apply the latest Security Packs / Service Packs
Even though this point applies to the network and the database
administrators, it is always better to keep up-to date on the software's.
With the "slammer" virus and many more still outside, it is one of the best
practices to be up-to date on the same. Consider this strongly.
26. Using Count(*)
The only 100 percent accurate way to check the number of rows in a table is
to use a COUNT(*) operation. The statement might consume significant
resources if your tables are very big because scanning a large table or
index can consume a lot of I/O. Avoid these type of queries to the maximum.
Use short circuting methods as EXISTS etc. Here is one other way you can
find the total number of rows in a table. SQL Server Books Online (BOL)
documents the structure of sysindexes; the value of sysindexes.indid will
always be 0 for a table and 1 for a clustered index. If a table doesn't have
a clustered index, its entry in sysindexes will always have an indid value
of 0. If a table does have a clustered index, its entry in sysindexes will
always have an indid value of 1.
SELECT object_name(id) ,rowcnt
FROM sysindexes
WHERE indid IN (1,0) AND OBJECTPROPERTY(id, 'IsUserTable') = 1
27. Ownership Chaining
Try using this feature (available from SQL Server 2000 SP3), for permission
management within a single database. Avoid using this feature to manage
permissions across database.
28. SQL Injection
Security has been a prime concern for everyone. Hence validate all the
incoming parameters at all levels of the application. Limit the scope of
possible damage by permitting only minimally privileged accounts to send
user input to the server. Adding to it, run SQL Server itself with the least
necessary privileges.
29. Fill-factor
http://forums.yessoftware.com/posts.php?post_id=75008[08/29/2012 4:20:12 PM]
6. Speed Up SQL
The 'fill factor' option specifies how full SQL Server will make each index
page. When there is no free space to insert new row on the index page, SQL
Server will create new index page and transfer some rows from the previous
page to the new one. This operation is called page splits. You can reduce
the number of page splits by setting the appropriate fill factor option to
reserve free space on each index page. The fill factor is a value from 1
through 100 that specifies the percentage of the index page to be left
empty. The default value for fill factor is 0. It is treated similarly to a
fill factor value of 100, the difference in that SQL Server leaves some
space within the upper level of the index tree for FILLFACTOR = 0. The fill
factor percentage is used only at the time the index is created. If the
table contains read-only data (or data that very rarely changed), you can
set the 'fill factor' option to 100. When the table's data modified very
often, you can decrease the 'fill factor' option to 70 percent, for example.
Having explained page splits in detail I would warn you in over looking at
this point because more free space means that SQL Server has to traverse
through more pages to get the same amount of data. Hence try to strike a
balance and arrive at an appropriate value.
30. Start-up Procedures
Verify all the stored procedures for safety reasons.
31. Analyze Blocking
More often than not any implementers nightmare would be to see a blocking
process. Blocking occurs when a process must wait for another process to
complete. The process must wait because the resources it needs are
exclusively used by another process. A blocked process will resume operation
after the resources are released by the other process. Sometimes this can
become cyclic and the system comes to a stand still. The only solution is to
analyze your indexing strategy and table design. Consider these points
strongly.
32. Avoid Un-necessary Indexes
Avoid creating un-necessary indexes on table thinking they would improve
your performance. Understand that creating Indexes and maintaining them are
overheads that you incur. And these surely do reduce the throughput for the
whole application. You can create a simple test on a large table and find it
for yourself how multiple indexes on the same column decrease performance.
33. Consider Indexed Views
Sometimes we would require an view to be indexed. This feature is bundled
with SQL Server 2000. The result set of the indexed view is persist in the
database and indexed for fast access. Because indexed views depend on base
tables, you should create indexed views with SCHEMABINDING option to prevent
the table or column modification that would invalidate the view. Hence using
them can reduce a lot of load on the base tables but increases the
maintainability.
34. WITH SORT_IN_TEMPDB Option
Consider using this option when you create an index and when tempdb is on a
different set of disks than the user database. This is more of a tuning
recommendation. Using this option can reduce the time it takes to create an
index, but increases the amount of disk space used to create an index. Time
is precious, disk is cheaper.
35. Reduce Number of Columns
Try to reduce the number of columns in a table. The fewer the number of
columns in a table, the less space the table will use, since more rows will
fit on a single data page, and less I/O overhead will be required to access
the table's data. This should be considered strongly by applications that
talk across different machines. More the unwanted data passed more is the
network latency observed.
"feha" <feha@forum.codecharge> wrote in message
http://forums.yessoftware.com/posts.php?post_id=75008[08/29/2012 4:20:12 PM]