Objectives Familiarization with main SQL Server 2008 DB engine enhancements. Reviewing Integration Services enhancements. Reviewing Reporting Services enhancements. Reviewing Analysis Services enhancements. Upgrading to 2008 recommended method.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Data Compression (Enterprise Edition) Compression in row or page format. Affect tables, indexes or each partition separately. Stores fix length type as variable length. An int value of 1: No compression 4 bytes + 2 bytes metadata overhead. With Compression 1 byte + 4 bits metadata overhead. Reduce storage requirements and memory usage. Less number of pages in cache less number of latches Higher buffer hit rates better memory utilization with a bit of extra CPU time. Significant performance improvement for large I/O bound workload, like DWH. Main goal Shrink fact tables (2X to 7X compression ratio for real DW fact data). Secondary goal improve query performance.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Backup Compression (Enterprise Edition) Require less I/O Increase backup speed. Less storage required to keep backups online (expect 4:1 ratio). Increase CPU usage significantly can be set to low priority session by resource governor. Average Performance for a Set of Customer Databases:
SQL Server 2008 EnhancementsDB Engine Main Enhancements Resource Governor (Enterprise Edition) Allow defining resource limit & priorities for different workloads. Govern queries base on group/ pool. Limits the amount of CPU & memory that incoming applications requests can use. Resource limits can be configured in real time with minimal impact. Limited to DB engine cannot be used for SSAS, SSIS, SSRS. DWH usage – handling unpredictable workloads execution, like 2 un-insolated DWH applications or a mix of OLTP & DWH applications.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Filtered Indexes & Statistics (Enterprise Edition) Non clustered index only, that support WHERE condition and includes only matching rows. Improved query performance and execution plan quality it’s smaller then full table non clustered index and have smaller and more accurate statistics. Reduce index maintenance cost DML statements is not always affecting the index. Reduce index storage cost Replacing a full table non clustered index with multiple filtered index without significantly increasing storage requirements. CREATE INDEX #IX_Filtered ON Production.WorkOrderRouting (ActualEndDate) WHERE ActualStartDate > 2002-01-01 AND ActualStartDate < 2002-12-31‘
SQL Server 2008 EnhancementsDB Engine Main Enhancements Parallel Query Processing on Partitioned Objects (Enterprise Edition) Multiple processes work simultaneously to process a single SQL Statement. In SQL Server 2005 one thread was created per partition, regardless the number of processing cores. In SQL Server 2008 all processors can be used to satisfy the query requirements. If threads number > partitions number, query processor allocates equal number of threads number to each partition.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Change Data Capture - CDC (Enterprise Edition) Capturing the fact that DML changes were made + the actual metadata that was changed. Changes are captured and placed in changes tables. Log-based capture (by jobs) with minimal impact on source. Load delta only instead of full refresh – Efficient for updates of data warehouses or similar. ETL process can call functions to retrieve changes into DWH or similar. Work on schema changes as well – changes to column structure are stored in cdc.ddl _history table. Built-in automatic cleanup (by job) – by default 3 days of data is retained.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Change Tracking – CT Captures the fact that rows in a table were changed, but does not capture the data that was changed (only PK value is saved). Lightweight Tracks time & context of most recent row change. Get latest row data directly from user table limited historical data less storage overhead than CDC. Crawl Transaction Log Asynchronous High Performance. Provide functions to obtain changes info. Best for Occasionally Connected Systems like Mobile Sync Apps or Remote Dial Up Apps.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Auditing (Enterprise Edition) SQL Server instance & database level audit specifications. Available auditing destinations: Binary file. Windows application log. Windows security log (not available in Windows XP). Audit specifications defines what to audit Login, Logout, DDL/DML changes, Restarts, etc. 3 categories of actions for audit: Server level – server operations, such as management changes and logon and logoff operations. Database level – DMLs and DDLs operations. Audit level – include actions in the auditing process (CREATE AUDIT, ALTER AUDIT etc). Based on Extended Events architecture – can be correlated with windows events to resolve resources problems.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Data Collector Data warehouse mechanism. Running all the time or on a user-defined schedule, and collecting different sets of data. Collected data stored in the management data warehouse. System data collection sets including: Server Activity. Disk Usage. Query Statistics. Supports dynamic tuning for data collection extensible through its API. Integrated reporting & analyzing.
SQL Server 2008 EnhancementsDB Engine Main Enhancements FILESTREAM Storage Improved performance: Break 2GB limit of BLOB storage. NTFS caching & streaming. Keep BLOBs out of SQL memory pool. Transaction file I/O over NTFS. Stores BLOB data on NTFS – for VARBINARY(MAX) only. It’s recommended for BLOB objects that are larger than 1MB. BLOB are stored in SQL-managed folder: Folder managed as a file group. Nesting of folders not ok. Limit folder ACLs to SQL service account.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Policy Based Management Enforce compliance on database objects: Asserts a condition against one or more targets. Repair some violations automatically. Run on schedule or on demand. Server Group Management Creating Central Management Server. Supports targeting multiple servers for: Execution T-SQL queries. Managing policies.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Merge Statement Perform multiple DML (insert/ update/ delete) once and perform better than separate DMLs. Operations based on: WHEN MATCHED. WHEN [TARGET] NOT MATCHED. WHEN SOURCE NOT MATCHED. MERGE statement is transactional - No explicit transaction required. Multiple WHEN clauses possible on: MATCHED and SOURCE NOT MATCHED. Only one WHEN clause for TARGET NOT MATCHED. One Pass Through Tables. At most a full outer join. Matching rows = when matched. Left-outer join rows = when target not matched. Right-outer join rows = when source not matched.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Merge Statement (2) Usage Example: MERGE dbo.FactBuyingHabits AS Target USING (SELECT CustomerID, ProductID, PurchaseDate FROM dbo.Purchases) AS Source ON (Target.ProductID = Source.ProductID AND Target.CustomerID = Source.CustomerID) WHEN MATCHED THEN UPDATE SET Target.LastPurchaseDate = Source.PurchaseDate WHEN NOT MATCHED BY TARGET THEN INSERT (CustomerID, ProductID, LastPurchaseDate) VALUES (Source.CustomerID, Source.ProductID, Source.PurchaseDate) OUTPUT $action, Inserted.*, Deleted.*;
SQL Server 2008 EnhancementsDB Engine Main Enhancements Programmability Enhancements New separate DATE & TIME types. User-Defined table type: Declared globally. Used as argument to stored procedures and functions. Great for INSERT/ UPDATE procedures. Eliminates connection isolation. Declaration example: CREATE TYPE LocationTableType AS TABLE (LocationName varchar(50), CostRate int); DECLARE @tbl_LocationTableType AS LocationTableType; Sparse Columns – Null values are not stored Ideal for property/ value storage. HIRARCHYID Data type – Enables DB applications to model tree structures Implemented as a CLR UDT. Grouping Sets – allow multiple GROUP BY clauses in a single SQL statement. GROUP BY GROUPING SETS ((), (a), (a, b, c))
SQL Server 2008 EnhancementsDB Engine Main Enhancements Programmability Enhancements (2) T-SQL Row constructors – allow multiple value inserts within a single INSERT statement. INSERT INTO dbo.Departments VALUES (1, Human Resources, Margheim),(2, Sales, Byham), (3, Finance, Gill),(4, Purchasing, Barber), (5, Manufacturing, Brewer); Compound Operators – SET @x += 2 Query editor enhancements: Transact-SQL Debugger – similar to the Visual Studio debuggers including breakpoints, view call stack and watches. IntelliSense – word completion, error underlining, parameter help, colorizing, quick Info and more. Database Engine Error List Window – displays the syntax and semantic errors.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Transaction Logging Enhancements Usually Inserts written to disk twice: once to log + once to database pages. Minimally logged INSERTs In SQL Server 2005 – simple or bulk-logged recovery model only. In SQL Server 2008 – also in full recovery model. Performance improvement of large scale INSERT By reducing the number of log records to be written. By reducing the amount of log space required. Performance boost – 3X to 5X.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Spatial Data, Types and Indexes Spatial data Info about the physical location and shape of geometric objects – Point, locations or complex objects such as countries, roads, or lakes. SELECT * FROM roads WHERE roads.geom.Intersects(@ms)=1 Spatial data types: Geometry – Flat earth data (X/Y coordinates). Geography – Round earth data (GPS latitude & longitude). Spatial data objects supported by spatial data types Point, MultiPoint, LineString, MultiLineString, Poligon, MultiPolygon, GeometryCollection. Visual mapping tool spatial data results can be viewed in in Query Editor. Spatial index extended index that allows indexing a spatial column based on B trees.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Mirroring Enhancements Automatic page repair get fresh copy of corrupted pages from mirroring partner. Log performance enhancements: Compress outgoing log stream at least 12.5 percent compression ratio. Write-ahead on the incoming log stream on the mirror server incoming log records are written to disk asynchronously. Reduces manual failover downtime mirror does not restart to recover uncommitted log. Enhanced supportability includes additional performance counters & DMVs. For example: sys.dm_db_mirroring_auto_page_repair - tracks last 100 corrupted pages from mirroring sessions.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Service Broker Enhancements New Conversation Priorities – Ensures that low priority work does not block higher priority work. CREATE BROKER PRIORITY PremierCustomer FOR CONVERSATION SET (CONTRACT_NAME = premier, LOCAL_SERVICE_NAME = sender, REMOTE_SERVICE_NAME = receiver, PRIORITY_LEVEL = 7); New Diagnostic Utility – ssbdiagnose utility analyzes the configuration between two Service Broker services. New system monitor object and counters.
SQL Server 2008 EnhancementsDB Engine Main Enhancements Star Join Improvements (enterprise edition) Improved recognition of star join pattern. While joining a fact table with several dimensions. Improved costing of star join plans. Significantly quicker query processing for DWH. While processing a significant fraction of fact table rows. Based on bitmap filters Allows eliminate non-qualifying fact table rows from further processing. Saves a considerable amount of CPU time.
SQL Server 2008 EnhancementsIntegration Services Main Enhancements Lookup Performance A lookup tests whether each row in a stream of rows has a matching row in another dataset (like JOIN operation). In SQL Server 2005: Each Lookup component would cache its own copy. Cache is reloaded every time it’s used. In SQL Server 2008: Cache can be shared between multiple lookup components in the same package or different package. Cache can be saved to virtual memory or permanent file storage New miss-cache feature – load to cache key values that have no matching entries in the reference dataset.
SQL Server 2008 EnhancementsIntegration Services Main Enhancements Data Profiling Task and Data Profile Viewer The information provided by the profile helps you identify potential problems with data quality: The distribution of lengths in the column values. The percentage of null values. The distribution of values in the column.
SQL Server 2008 EnhancementsReporting Services Main Enhancements Report Authoring Enhanced Chart Data Region. New Gauge Data Region. New Tablix Data Region. New Report Builder 2.0 offers many new features, like enhanced data layout, visualization and text formatting, and on-demand rendering.
SQL Server 2008 EnhancementsReporting Services Main Enhancements Report Scalability Can render much larger reports – renders each page of a report as you view it. Can create reports with hundreds or thousands of pages. Server Scalability Dependency in IIS removed – includes native support for HTTP.SYS and ASP.NET. Can manage its own memory and memory limits.
SQL Server 2008 EnhancementsAnalysis Services Main Enhancements MDX Query Performance Improve block computation – No time is wasted evaluating null cells. Query and Write back Performance In SQL Server 2005: SSAS required write back partitions to have ROLAP storage. In SQL Server 2008: Allow write back partitions with MOLAP storage. Compressed MOLAP format is much faster than querying the relational data source. Enhanced Backup less disk I/O is required. Scalable Shared Database (SSD) Can scale out your OLAP query workload across many small servers with just a single copy of the database.
SQL Server 2008 EnhancementsRecommended Upgrade Method Support upgrade from: SQL Server 2000 SP4. SQL Server 2005 SP2. Considerations: Reporting services – IIS was replaced with HTTP.SYS Upgrade an existing instance and not databases divide and rule. Existing database retain their compatibility level until application is ready.
SQL Server 2008 EnhancementsFeatures by edition Enterprise StandardComponent Group Feature Description Edition EditionDatabase Scalability & Data Compression Reduce storage requirementsEngine Performance Backup Compression Less storage required to keep backups online Filtered Indexes and Index that support WHERE Statistics conditions Resource Governor To define resource limits & priorities for different workloads Parallel Query Processing Using multiple processors in on Partitioned Objects parallel to process query Star join improvements Better plan + better raw query execution performance Performance Data Collecting storing and analyzing Collector performance data Logging Enhancements Minimally logged INSERT INTO, in full recovery model DB Manageability Policy-Based Management Enables efficient management of multiple instances from a single location Server Group Enables execute of T-SQL queries Management against multiple servers SQL Server Extended Can be correlated with Windows Events events to resolve resources problems
SQL Server 2008 EnhancementsFeatures by edition (Part 2) Enterprise StandardComponent Group Feature Description Edition EditionDatabase Manageability Auditing Create & manage database engineEngine events via DDL Change Data Capture Data changes are captured and (CDC) placed in changes tables Change Tracking (CT) Allow applications to obtain incremental changes to user table Transact-SQL Debugger Including break points, data values watches, call stack and more Database Engine Error Display syntax & semantic errors List Window IntelliSense Word completion, error underlining, parameter help and more Security Transparent Database The ability to encrypt the entire Encryption database files External Key management Storing keys separate from the data Programmability MERGE Statement Enable INSERT UPDATE & DELETE data with a single statement New Date and Time Data Separate DATE and TIME types, Types DARETINEOFFSET, DATETIME2 hierarchyid Data Type Enables to model tree structure
SQL Server 2008 EnhancementsFeatures by edition (Part 3) Enterprise StandardComponent Group Feature Description Edition EditionDatabase Programmability Spatial Data Types GEOMETRY for storing flat-earthEngine data and GEOGRAPHY for round- earth data such as GPS User-Defined Table Type Support table structure as a parameter for stored procedures and functions User Defined Types (UDT) Maximum size of UDT has been increased to 2 GB Compound Operators for Example: SET @x += 2 Enhanced CONVERT Allow conversions between binary & characters hexadecimal values Grouping Sets Enable defining multiple grouping in the same query Table-Valued Parameters Provide easier way to define table type that can be passed to stored procedures and functions FILESTREAM Storage Enables storing of documents and images on the file system Sparse Columns Optimizing storage format for NULL values Spatial Data Storage Improved spatial indexes. Results can be viewed in query editor
SQL Server 2008 EnhancementsFeatures by edition (Part 4) Enterprise StandardComponent Group Feature Description Edition EditionDatabase Programmability Wide Tables A table that has defined column set.Engine Up to 8019 bytes size. Integrated Full Text Catalogs integrated into the Search database instead of file system Mirroring Automatic Page Repair Get fresh copy of corrupted pages from mirroring partner Improved Performance Compress outgoing log stream Enhanced Supportability Includes additional performance counters & DMVs Service Broker New Conversation Priority Ensures that low priority work does not block higher priority work New Diagnostic Utility ssbdiagnose utility analyzes configuration between two service broker services Replication Enhanced Support for Enable execution of SWITCH Partitioned tables PARTITION on publication databaseIntegration Components Enhanced Performance Includes faster cache loading andServices and Caching for the more efficient lookup operations Lookup Transformation New ADO.NET ADO NET source component & ADO Components NET destination component
SQL Server 2008 EnhancementsFeatures by edition (Part 5) Enterprise StandardComponent Group Feature Description Edition EditionIntegration Components New data profiling task Help identify data quality problemsServices within individual columns and with column relationships New integration services Create package that contains connections project connection information wizard New scrip environment BI Development studio integrates with MS visual studio for applications (VSTA) Data Management Enhanced data type Provides additional info & options handling in Import and related to date type conversions Export wizard New Date and Time data DT_DBTIME2, DT_DBTIMESTAMP2 types DT_DBTIMESTAMPOFFSET Enhanced SQL statements New merge statement, get changes to data source, Improved performance of bulk load Performance Change data capture Data changes are captured and Troubleshooting placed in changes tables New debug dump files Creates dump files(.mdmp & .tmp) to provide info when package runsAnalysis Designers improvements New Aggregation designer. TheServices (cube, dimension, cube wizard has been simplified aggregation) and enhanced. New Attribute Relationship designer.
SQL Server 2008 EnhancementsFeatures by edition (Part 6) Enterprise StandardComponent Group Feature Description Edition EditionAnalysis Backup and restore Due improved backup compressionServices improved performance less disk I/O is required.Reporting Report Authoring Enhanced Chart Data Enhanced Chart Data Region thatServices Region includes a redesigned chart control that supports many new chart types. New Gauge Data Region New Tablix Data Region New Tablix Data Region for Table, for Table, Matrix, and List Matrix, and List, which combines table, list, and matrix structures into a single structure. New Report Builder 2.0 New Report Builder 2.0 offers many new features, like enhanced data layout, visualization and text formatting, and on-demand rendering. Report New and Enhanced New and Enhanced Rendering Processing and Rendering Extensions Extensions includes new and Rendering enhanced rendering extensions for Word, Excel, and CSV. New On-Demand Report New On-Demand Report Processing Processing is a processing enhancement that renders each page of a report as you view it.
SQL Server 2008 EnhancementsFeatures by edition (Part 7) Enterprise StandardComponent Group Feature Description Edition EditionReporting Report Server New Report Server Redesigned server architecture thatServices Architecture and Architecture removes the dependency on IIS and Tools includes native support for HTTP.SYS and ASP.NET. Enhanced Toolset for Support for specific tasks have Report Server been added to some tools and Configuration and removed from others. Management New Support for Data- New Support for Data-driven driven Subscriptions and Subscriptions and Job Management Job Management in in SharePoint Integrated Mode by SharePoint Integrated using the new subscription Mode definition pages that are provided by the SSRS Add-in for SharePoint. Report Server Report Definition New Report Definition Programmability Customization Extension Customization extension (RDCE) that you can use to dynamically customize a report definition before it is passed to the processing engine. New Methods for the New methods for that endpoint ReportService2006 Class provide support for data-driven subscriptions and job management.