The document discusses indexes in SQL Server. It describes internal and external fragmentation that can occur in indexes. Internal fragmentation is unused space between records within a page, while external fragmentation is when page extents are not stored contiguously on disk. It provides examples of identifying fragmentation using system views and the dm_db_index_physical_stats dynamic management function. It also covers best practices for index types, such as numeric and date fields making good candidates while character fields are less efficient. Composite indexes, fill factor, and rebuilding vs. reorganizing indexes are also discussed.

1. MSSQL Server Indexes
Ram Kedem

2. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Internal fragmentation
•When records are stored non-contiguously inside the page, then it is called internal fragmentation.
•In other words, internal fragmentation is said to occur if there is unused space between records in a page.
•This fragmentation occurs through the process of data modifications (INSERT, UPDATE, and DELETE statements) that are made against the table and therefore, to the indexes defined on the table.
•As these modifications are not equally distributed among the rows of the table and indexes, the fullness of each page can vary over time.
•This unused space causes poor cache utilization and more I/O, which ultimately leads to poor query performance.

3. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
External Fragmentation
•When on disk, the physical storage of pages and extents is not contiguous.
•When the extents of a table are not physically stored contiguously on disk, switching from one extent to another causes higher disk rotations.

4. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Identify Fragmentation – Demo 1
-- 1. Identify the object using sys.indexes and sys.objects
USE Northwind;
GO
SELECT * FROM sys.objects WHERE name = 'Products'
SELECT * FROM sys.indexes
WHERE object_id = object_id('products')

5. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Identify Fragmentation – Demo 1
-- 2. Query the index physical stats DMV using sys.dm_db_index_physical_stats
-- The procedure gets the following parameters
-- 1. Database ID
-- 2. Object ID (the table)
-- 3. Index ID
-- 4. Partition ID (if needed)
-- 5. Scope of detail (LIMITED , SAMPLED , DETAILED)
SELECT index_id,index_type_desc,
index_level,
avg_fragmentation_in_percent AS 'external fragmentation',
avg_page_space_used_in_percent AS 'internal fragmentation',
page_count
FROM sys.dm_db_index_physical_stats(DB_ID(),533576939,NULL,NULL,'DETAILED');

6. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Identify Fragmentation – Demo 1
-- or
DECLARE @object_id int
SELECT @object_id = object_id FROM sys.objects WHERE name = 'Products'
SELECT index_id,index_type_desc,
index_level,
avg_fragmentation_in_percent AS 'external fragmentation',
avg_page_space_used_in_percent AS 'internal fragmentation',
page_count
FROM sys.dm_db_index_physical_stats(DB_ID(),@object_id,NULL,NULL,'DETAILED');

7. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Identify Fragmentation – Demo 1
SELECT name
FROM sys.indexes
WHERE object_id = object_id('products')
AND index_id = 2
ALTER INDEX CategoriesProducts ON products REBUILD

8. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
External / Internal Fragmentation
•External Fragmentation - Aim lower than 10%
•avg_fragmentation_in_percent: This is a percentage value that represents external fragmentation.
•The lower this value, the better it is. If this value is higher than 10%, some corrective action should be taken.
•Internal Fragmentation - Aim higher than 75%
•avg_page_space_used_in_percent: This is an average percentage use of pages that represents to internal fragmentation.
•Higher the value, the better it is. If this value is lower than 75%, some corrective action should be taken.
•If an index is below a certain size, defragmentation doesn't end up doing anything.
•In some cases the index is tiny, so fragmentation doesn't matter and won't get defragmented anyway.

9. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Numeric Index Data
•Numeric
•Indexes with numeric keys work efficiently
•Integer types are the most efficient (Exact numeric types)
•Approximate data types (float and real) much less efficient
CREATE TABLE test_tbl
(numeric_col int)
CREATE INDEX numeric_col_ix ON test_tbl(numeric_col)
DROP INDEX test_tbl.numeric_col_ix
DROP TABLE test_tbl

10. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Date Index Data
•Date-Related Index Data
•Date data types are generally good candidates for index keys
•Only slightly less efficient than integer data
•date (Accuracy of 1 day) more efficient than datetime (Accuracy of 3.33 milliseconds)
CREATE TABLE test_tbl
(date_col date)
CREATE INDEX date_co_ix ON test_tbl(date_col)
DROP INDEX test_tbl.date_co_ix
DROP TABLE test_tbl

11. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Character Index Data
•Character data types are much less efficient when used in index keys
•Character values tend to be much larger than numeric values
CREATE TABLE test_tbl
(char_col int)
CREATE INDEX char_col_ix ON test_tbl(char_col)
DROP INDEX test_tbl.char_col_ix
DROP TABLE test_tbl

12. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
GUID Index Data
CREATE TABLE test_tbl
(product_id UNIQUEIDENTIFIER ,
product_name varchar(25))
CREATE INDEX product_id_ix ON test_tbl(product_id)
DROP INDEX test_tbl.product_id_ix
DROP TABLE test_tbl

13. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Computed Column and Indexes
DROP TABLE computed_tbl ;
GO
CREATE TABLE computed_tbl
(id int,
-- date_col AS RAND() PERSISTED
-- Since it's not a deterministic function it cannot be PERSISTED
rand_col AS RAND(),
id_col AS (id+1) PERSISTED ,
decimal_col DECIMAL(3, 1));
GO
INSERT INTO computed_tbl (id, decimal_col)
VALUES (1 , 2.1),(2, 3.4),(NULL, 4.7)
-- Values computed every select
SELECT * FROM computed_tbl
-- The expressions must be deterministic
----------------------------------------
CREATE INDEX test_ix ON computed_tbl(rand_col)
-- Works
CREATE INDEX test_ix ON computed_tbl(id_col)

14. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Single Column vs. Composite Indexes
-- Single Column
-----------------
DROP TABLE test_tbl
SELECT productID , CategoryID , UnitPrice
INTO test_tbl
FROM northwind.dbo.products
INSERT INTO test_tbl SELECT CategoryID , UnitPrice FROM test_tbl
SELECT COUNT(*) FROM test_tbl
SELECT * FROM test_tbl
CREATE INDEX prod_id_ix ON test_tbl (productID)
CREATE INDEX prod_cat_ix ON test_tbl (CategoryID)
SELECT CategoryID FROM test_tbl ORDER BY CategoryID
SELECT CategoryID FROM test_tbl ORDER BY CategoryID , productID

15. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Single Column vs. Composite Indexes
-- Composite Indexes
--------------------
DROP TABLE test_tbl
SELECT productID , CategoryID , UnitPrice
INTO test_tbl
FROM northwind.dbo.products
DECLARE @counter INT
SET @counter = 1
WHILE (@counter <=12)
BEGIN
INSERT INTO test_tbl SELECT CategoryID , UnitPrice FROM test_tbl
SET @counter = @counter + 1
END
SELECT COUNT(*) FROM test_tbl
CREATE INDEX prod_id_ix ON test_tbl (CategoryID, productID)
SELECT CategoryID, productID FROM test_tbl ORDER BY CategoryID , productID
SELECT CategoryID, productID FROM test_tbl ORDER BY productID , CategoryID

16. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Ascending vs. Descending Indexes
DROP INDEX test_tbl.prod_id_ix
CREATE INDEX prod_id_ix ON test_tbl(productID DESC)
SELECT productID FROM test_tbl ORDER BY productID
SELECT productID FROM test_tbl ORDER BY productID DESC
DROP INDEX test_tbl.prod_id_ix
CREATE INDEX prod_id_ix ON test_tbl (CategoryID DESC, productID)
SELECT CategoryID FROM test_tbl ORDER BY CategoryID DESC, productID
SELECT CategoryID FROM test_tbl ORDER BY CategoryID , productID

17. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Forwarding Points
USE tempdb;
GO
DROP TABLE dbo.PhoneLog
-- Step 1: Create a table as a heap
CREATE TABLE dbo.PhoneLog
( PhoneLogID int IDENTITY(1,1) NOT NULL,
LogRecorded datetime2 NOT NULL,
PhoneNumberCalled nvarchar(100) NOT NULL,
CallDurationMs int NOT NULL
);
GO
-- Step 2: Query sys.indexes to view the structure
SELECT index_id,
index_type_desc,
forwarded_record_count,
page_count
FROM sys.dm_db_index_physical_stats(DB_ID(),object_id('dbo.PhoneLog'),NULL,NULL,'SAMPLED');

18. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Forwarding Points
-- Step 3: Query sys.indexes to view the structure
SELECT index_id,
index_type_desc,
forwarded_record_count,
page_count
FROM sys.dm_db_index_physical_stats(DB_ID(),object_id('dbo.PhoneLog'), NULL,NULL,'SAMPLED');
-- Step 4: Insert some data into the table
SET NOCOUNT ON;
DECLARE @Counter int = 0;
WHILE @Counter < 1000 BEGIN
INSERT dbo.PhoneLog (LogRecorded, PhoneNumberCalled, CallDurationMs)
VALUES(SYSDATETIME(),'999-9999',CAST(RAND() * 1000 AS int));
SET @Counter += 1;
END;
GO

19. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Forwarding Points
SELECT * FROM dbo.PhoneLog
SELECT index_id,
index_type_desc,
forwarded_record_count,
page_count
FROM sys.dm_db_index_physical_stats(DB_ID(),object_id('dbo.PhoneLog'),NULL,NULL,'SAMPLED');

20. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Forwarding Points
-- Step 5: Modify the data in the table using replicate
SET NOCOUNT ON;
DECLARE @Counter int = 0;
WHILE @Counter < 1000 BEGIN
UPDATE dbo.PhoneLog SET PhoneNumberCalled = REPLICATE('9',CAST(RAND() * 100 AS int))
WHERE PhoneLogID = @Counter;
IF @Counter % 100 = 0 PRINT @Counter;
SET @Counter += 1;
END;
GO
-- Step 6: Check the level of fragmentation via sys.dm_db_index_physical_stats
SELECT * FROM dbo.PhoneLog
SELECT index_id,
index_type_desc,
forwarded_record_count,
page_count
FROM sys.dm_db_index_physical_stats(DB_ID(),object_id('dbo.PhoneLog'),NULL,NULL,'SAMPLED');
-- Step 8: Rebuild the table
ALTER TABLE dbo.PhoneLog REBUILD;
GO

21. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Create Clustered Indexes
-- 1. Can be created by specifying PRIMARY KEY on table, when creating it
CREATE TABLE dbo.Article
( ArticleID int IDENTITY(1,1) PRIMARY KEY,
ArticleName nvarchar(50) NOT NULL,
PublicationDate date NOT NULL
);
-- 2. Can be created by specifying PRIMARY KEY on table, when altering it
CREATE TABLE dbo.LogData
( LogID int IDENTITY(1,1),
LogData xml NOT NULL
);
ALTER TABLE dbo.LogData
ADD CONSTRAINT PK_LogData
PRIMARY KEY (LogId);

22. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Create Clustered Indexes
-- 3. You can add the term NONCLUSTERED to the definition of the PRIMARY KEY
-- to avoid this behavior if you wish
CREATE TABLE emps1
(empid int PRIMARY KEY NONCLUSTERED ,
empName varchar(30))
-- 3. Can be created directly
CREATE TABLE emps2
(empid int,
empName varchar(30))
CREATE CLUSTERED INDEX CL_empid
ON dbo.emps2(empName);
-- View the different table types
SELECT tab.object_id , tab.name AS 'TableName' , i.type_desc , i.name AS 'IndexName'
FROM sys.indexes i, sys.tables tab
WHERE i.object_id = tab.object_id

23. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Drop Clustered Indexes
DROP INDEX CL_empid ON dbo.emps2;
ALTER TABLE dbo.Article
DROP CONSTRAINT PK__Article__9C6270C8C831D6B7;

24. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Altering a Clustered Index
•Index Rebuild : This process drops the existing Index and Recreates the index.
•Index Reorganize : This process physically reorganizes the leaf nodes of the index.
•Recommendation:
•Index should be rebuild when index fragmentation is great than 40%.
•Index should be reorganized when index fragmentation is between 10% to 40%.
•Index rebuilding process uses more CPU and it locks the database resources.
•SQL Server development version and Enterprise version has option ONLINE, which can be turned on when Index is rebuilt. ONLINE option will keep index available during the rebuilding.

25. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Altering a Clustered Index
-- Rebuild
ALTER INDEX PK_LogData ON dbo.LogData
REBUILD;
-- Disable
ALTER INDEX PK_LogData ON dbo.LogData
DISABLE;
SELECT name , is_disabled FROM sys.indexes
WHERE name = 'PK_LogData'
-- Reorginize / Rebuild
ALTER INDEX PK_LogData ON dbo.LogData
REORGANIZE;
ALTER INDEX PK_LogData ON dbo.LogData
REBUILD;

26. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Fill Factor
DROP TABLE emps3
GO
CREATE TABLE emps3
(empid int CONSTRAINT emp3_id_pk PRIMARY KEY WITH (FILLFACTOR = 70),
empname varchar(25))
ALTER INDEX emp3_id_pk ON emps3
REBUILD WITH (FILLFACTOR = 80)
GO

27. Heap Table Vs. Clustered Index In Depth

28. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Heap Table
DROP TABLE dummyTable1
CREATE TABLE DummyTable1
(EmpId Int,
EmpName Varchar(8000))
Insert Into DummyTable1 Values (4, Replicate ('d',2000))
Insert Into DummyTable1 Values (6, Replicate ('f',2000))
Insert Into DummyTable1 Values (1, Replicate ('a',2000))
Insert Into DummyTable1 Values (3, Replicate ('c',2000))
Insert Into DummyTable1 Values (10, Replicate ('j',2000))
Insert Into DummyTable1 Values (2, Replicate ('b',2000))
Insert Into DummyTable1 Values (5, Replicate ('e',2000))
Insert Into DummyTable1 Values (8, Replicate ('h',2000))
Insert Into DummyTable1 Values (9, Replicate ('i',2000))
Insert Into DummyTable1 Values (7, Replicate ('g',2000))
Select EmpID From DummyTable1
GO
-- Allow DBCC return info back to your session
DBCC TRACEON(3604)
GO

29. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Heap Table
-- 2. View page allocations
Declare @DBID Int, @TableID Int
Select @DBID = db_id(), @TableID = object_id('DummyTable1')
DBCC ind(@DBID, @TableID, -1)
GO
--PagePID is the physical page numbers used to store the table. In this case, three pages are
--currently used to store the data.
--IndexID is the type of index,Where:
-- 0 – Datapage
-- 1 – Clustered Index
-- 2 – Greater and equal to 2 is an Index page (Non-Clustered Index and ordinary index),
-- PageType tells you what kind of data is stored in each database, Where:
-- 10 – IAM (Index Allocation MAP)
-- 1 – Datapage
-- 2 – Index page
-- View each page contents
Declare @DBID Int
Select @DBID = db_id()
DBCC page(@DBID, 1, 304, 3)
GO

30. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Nonclustred index
CREATE UNIQUE NONCLUSTERED INDEX DummyTable1_empid
ON DummyTable1 (empid)
GO
Declare @DBID Int, @TableID Int
Select @DBID = db_id(), @TableID = object_id('DummyTable1')
DBCC ind(@DBID, @TableID, -1)
GO
-- View index page contents
Declare @DBID Int
Select @DBID = db_id()
DBCC page(@DBID, 1, 312, 3)
GO

31. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Clustered index
DROP TABLE dummyTable2
CREATE TABLE DummyTable2
(EmpId Int Primary Key,
EmpName Varchar(8000))
Insert Into DummyTable2 Values (4, Replicate ('d',2000))
Insert Into DummyTable2 Values (6, Replicate ('f',2000))
Insert Into DummyTable2 Values (1, Replicate ('a',2000))
Insert Into DummyTable2 Values (3, Replicate ('c',2000))
Insert Into DummyTable2 Values (10, Replicate ('j',2000))
Insert Into DummyTable2 Values (2, Replicate ('b',2000))
Insert Into DummyTable2 Values (5, Replicate ('e',2000))
Insert Into DummyTable2 Values (8, Replicate ('h',2000))
Insert Into DummyTable2 Values (9, Replicate ('i',2000))
Insert Into DummyTable2 Values (7, Replicate ('g',2000))
Select EmpID From DummyTable2
GO

32. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Clustered index
Declare @DBID Int, @TableID Int
Select @DBID = db_id(), @TableID = object_id('DummyTable2')
DBCC ind(@DBID, @TableID, -1)
GO
-- Possible to use this undocumented command to see page contents
Declare @DBID Int
Select @DBID = db_id()
DBCC page(@DBID, 1, 317, 3)
GO
Declare @DBID Int, @TableID Int
Select @DBID = db_id(), @TableID = object_id('DummyTable1')
DBCC ind(@DBID, @TableID, -1)
GO
Declare @DBID Int
Select @DBID = db_id()
DBCC page(@DBID, 1, 286, 3)
GO

33. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Clustered index (with secondary index)
DROP TABLE dummyTable2
CREATE TABLE DummyTable2
(EmpId Int Primary Key,
EmpName Varchar(8000),
emp_lName varchar(10))
Insert Into DummyTable2 Values (4, Replicate ('d',2000), Replicate ('d',3))
Insert Into DummyTable2 Values (6, Replicate ('f',2000), Replicate ('f',3))
Insert Into DummyTable2 Values (1, Replicate ('a',2000), Replicate ('a',3))
Insert Into DummyTable2 Values (3, Replicate ('c',2000), Replicate ('c',3))
Insert Into DummyTable2 Values (10, Replicate ('j',2000), Replicate ('j',3))
Insert Into DummyTable2 Values (2, Replicate ('b',2000), Replicate ('b',3))
Insert Into DummyTable2 Values (5, Replicate ('e',2000), Replicate ('e',3))
Insert Into DummyTable2 Values (8, Replicate ('h',2000), Replicate ('h',3))
Insert Into DummyTable2 Values (9, Replicate ('i',2000), Replicate ('i',3))
Insert Into DummyTable2 Values (7, Replicate ('g',2000), Replicate ('g',3))
CREATE UNIQUE NONCLUSTERED INDEX DummyTable2_lname
ON DummyTable2 (emp_lName)
GO

34. Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent
Clustered index (with secondary index)
Select EmpID From DummyTable2
GO
Declare @DBID Int, @TableID Int
Select @DBID = db_id(), @TableID = object_id('DummyTable2')
DBCC ind(@DBID, @TableID, -1)
GO
-- Possible to use this undocumented command to see page contents
Declare @DBID Int
Select @DBID = db_id()
DBCC page(@DBID, 1, 292, 3)
GO
UPDATE DummyTable2 SET EmpId = 100 WHERE EmpId = 1