SlideShare a Scribd company logo
1 of 16
Is “the bigger the better” valid in the
database world?
Best practices around VLDBs
HP FastTrack Solution
Tips, tricks and gotchas with multi-TB databases
About me
 Ivan Donev, MCT
 SQL Server Technology Team Lead, HP




2 |
Agenda
     Introduction
     Best practices in configuration
     HP FastTrack Reference Architecture 4.0
     Share the experience




3 |
Introduction

  More data
         Needs larger hardware
         Needs optmization
         Needs time
         Requires more responsiblities
  More responsibilities
       Need more attention
       Require more knowledge
       Give you more headaches




4 |
Best practices - IO

     Hardware - trust your vendor
     Follow the Storage Top 10 Best Practices
     HBA Queue Depth
     TEST, TEST, TEST




5 |
Best practices – SQL Server

  Startup options
       -E and T1117
       T834?

  MaxServerMemory
  MaxDop – never leave it to 0
  Local policy
       Lock pages in memory
       Perform Volume maintenance tasks
  Resource governor!




6 |
Best practices – Inside SQL Server

  TempDB
       Microsoft rules – 1 file per CPU
       Experience
     Partitioning
     Compression – be careful!
     Column Store index
     Maintenance, maintenance, maintenance!!!




7 |
HP FastTrack 4.0

  Prebuild Hardware for best
   performance
  Configuration Guide
  Performance




8 |
Let’s talk about reality

  The hardware - HP DL980 G7
       2TB RAM
       8x8-core Intel Xeon => 128 logical CPUs
       Storage – dedicated P2000 and XP24000
  The Software
       Windows Server 2008 R2 with SP1
       SQL Server 2008 R2 Enterprise with CU7




9 |
10 |
Let’s talk about reality - servicing

   T834 flag or why the engine is not booting up
   Patching and ugrade
        Upgrade from SQL Server 2005 to 2008 R2
           Approach
           Licensing
           Bugs




11 |
Let’s talk about reality - maintenance

   Examine fragmentation
   Do not forget about statistics
   Check the integrity
        Size matters
   Autogrowth and proportional fill




12 |
Let’s talk about reality - compression

   Estimate compression gain
   Compress and keep compressing
   How do we reclaim the space?
        Moving object to new file
        Shrinking




13 |
IS THE BIGGER THE BETTER?




14 |
15 |
Sponsors

More Related Content

What's hot

Spectrum Scale final
Spectrum Scale finalSpectrum Scale final
Spectrum Scale final
Joe Krotz
 
Nimble storage
Nimble storageNimble storage
Nimble storage
dvmug1
 
EarthLink Business Cloud Server Backup
EarthLink Business Cloud Server BackupEarthLink Business Cloud Server Backup
EarthLink Business Cloud Server Backup
Mike Ricca
 
1 rh storage - architecture whitepaper
1 rh storage - architecture whitepaper1 rh storage - architecture whitepaper
1 rh storage - architecture whitepaper
Accenture
 

What's hot (20)

PernixData FVP Overview
PernixData FVP OverviewPernixData FVP Overview
PernixData FVP Overview
 
DataCore Executive Brief
DataCore Executive BriefDataCore Executive Brief
DataCore Executive Brief
 
Capacity Efficiency: Identifying the Right Solutions for the Right Challenge
Capacity Efficiency: Identifying the Right Solutions for the Right ChallengeCapacity Efficiency: Identifying the Right Solutions for the Right Challenge
Capacity Efficiency: Identifying the Right Solutions for the Right Challenge
 
IBM general parallel file system - introduction
IBM general parallel file system - introductionIBM general parallel file system - introduction
IBM general parallel file system - introduction
 
ICEOTOPE & OCF: Performance for Manufacturing
ICEOTOPE & OCF: Performance for Manufacturing ICEOTOPE & OCF: Performance for Manufacturing
ICEOTOPE & OCF: Performance for Manufacturing
 
Présentation NUTANIX - ÉVÉNEMENT TOUR D’ARGENT : ACROPOLIS - NUTANIX - SIPART...
Présentation NUTANIX - ÉVÉNEMENT TOUR D’ARGENT : ACROPOLIS - NUTANIX - SIPART...Présentation NUTANIX - ÉVÉNEMENT TOUR D’ARGENT : ACROPOLIS - NUTANIX - SIPART...
Présentation NUTANIX - ÉVÉNEMENT TOUR D’ARGENT : ACROPOLIS - NUTANIX - SIPART...
 
In-Memory Computing: Myths and Facts
In-Memory Computing: Myths and FactsIn-Memory Computing: Myths and Facts
In-Memory Computing: Myths and Facts
 
Why Government Agencies Should Consider Dell 13th Gen Servers
Why Government Agencies Should Consider Dell 13th Gen ServersWhy Government Agencies Should Consider Dell 13th Gen Servers
Why Government Agencies Should Consider Dell 13th Gen Servers
 
V3700 infographic final
V3700 infographic finalV3700 infographic final
V3700 infographic final
 
Get significantly better transactional database performance for less with val...
Get significantly better transactional database performance for less with val...Get significantly better transactional database performance for less with val...
Get significantly better transactional database performance for less with val...
 
Spectrum Scale final
Spectrum Scale finalSpectrum Scale final
Spectrum Scale final
 
Netbackup advantages features and benefits Netbackup classes may help hands o...
Netbackup advantages features and benefits Netbackup classes may help hands o...Netbackup advantages features and benefits Netbackup classes may help hands o...
Netbackup advantages features and benefits Netbackup classes may help hands o...
 
PHD Virtual: Optimizing Backups for Any Storage
PHD Virtual: Optimizing Backups for Any StoragePHD Virtual: Optimizing Backups for Any Storage
PHD Virtual: Optimizing Backups for Any Storage
 
Nimble storage
Nimble storageNimble storage
Nimble storage
 
Inter connect2016 yss1841-cloud-storage-options-v4
Inter connect2016 yss1841-cloud-storage-options-v4Inter connect2016 yss1841-cloud-storage-options-v4
Inter connect2016 yss1841-cloud-storage-options-v4
 
EarthLink Business Cloud Server Backup
EarthLink Business Cloud Server BackupEarthLink Business Cloud Server Backup
EarthLink Business Cloud Server Backup
 
GPFS Solution Brief
GPFS Solution BriefGPFS Solution Brief
GPFS Solution Brief
 
1 rh storage - architecture whitepaper
1 rh storage - architecture whitepaper1 rh storage - architecture whitepaper
1 rh storage - architecture whitepaper
 
The Cloud as a means of Recovery
The Cloud as a means of RecoveryThe Cloud as a means of Recovery
The Cloud as a means of Recovery
 
Microsoft private cloud with Cisco and Netapp - Flexpod solution
Microsoft private cloud with Cisco and Netapp -  Flexpod solutionMicrosoft private cloud with Cisco and Netapp -  Flexpod solution
Microsoft private cloud with Cisco and Netapp - Flexpod solution
 

Viewers also liked

Get the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMsGet the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMs
Ivan Donev
 

Viewers also liked (6)

Welcome to the nightmare of locking, blocking and isolation levels!
Welcome to the nightmare of locking, blocking and isolation levels!Welcome to the nightmare of locking, blocking and isolation levels!
Welcome to the nightmare of locking, blocking and isolation levels!
 
Get the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMsGet the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMs
 
Securing SQL Azure DB? How?
Securing SQL Azure DB? How?Securing SQL Azure DB? How?
Securing SQL Azure DB? How?
 
The Nightmare of Locking, Blocking and Isolation Levels
The Nightmare of Locking, Blocking and Isolation LevelsThe Nightmare of Locking, Blocking and Isolation Levels
The Nightmare of Locking, Blocking and Isolation Levels
 
Tips and tricks to optimiza SQL Server Backup and Restore
Tips and tricks to optimiza SQL Server Backup and RestoreTips and tricks to optimiza SQL Server Backup and Restore
Tips and tricks to optimiza SQL Server Backup and Restore
 
Data science & data scientist
Data science & data scientistData science & data scientist
Data science & data scientist
 

Similar to Is "the bigger the beter" valid in the database world

Similar to Is "the bigger the beter" valid in the database world (20)

Performance tuning intro
Performance tuning introPerformance tuning intro
Performance tuning intro
 
Performance Tuning intro
Performance Tuning introPerformance Tuning intro
Performance Tuning intro
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestAdding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance Test
 
Hadoop-DS: Which SQL-on-Hadoop Rules the Herd
Hadoop-DS: Which SQL-on-Hadoop Rules the HerdHadoop-DS: Which SQL-on-Hadoop Rules the Herd
Hadoop-DS: Which SQL-on-Hadoop Rules the Herd
 
S de0882 new-generation-tiering-edge2015-v3
S de0882 new-generation-tiering-edge2015-v3S de0882 new-generation-tiering-edge2015-v3
S de0882 new-generation-tiering-edge2015-v3
 
S sy0883 smarter-storage-strategy-edge2015-v4
S sy0883 smarter-storage-strategy-edge2015-v4S sy0883 smarter-storage-strategy-edge2015-v4
S sy0883 smarter-storage-strategy-edge2015-v4
 
Sql server tips from the field
Sql server tips from the fieldSql server tips from the field
Sql server tips from the field
 
Management and Automation of MongoDB Clusters - Slides
Management and Automation of MongoDB Clusters - SlidesManagement and Automation of MongoDB Clusters - Slides
Management and Automation of MongoDB Clusters - Slides
 
Splice machine-bloor-webinar-data-lakes
Splice machine-bloor-webinar-data-lakesSplice machine-bloor-webinar-data-lakes
Splice machine-bloor-webinar-data-lakes
 
Virtual Storage Center
Virtual Storage CenterVirtual Storage Center
Virtual Storage Center
 
Sql server 2016 it just runs faster sql bits 2017 edition
Sql server 2016 it just runs faster   sql bits 2017 editionSql server 2016 it just runs faster   sql bits 2017 edition
Sql server 2016 it just runs faster sql bits 2017 edition
 
Architecting with power vm
Architecting with power vmArchitecting with power vm
Architecting with power vm
 
The benefits of IBM FlashSystems
The benefits of IBM FlashSystemsThe benefits of IBM FlashSystems
The benefits of IBM FlashSystems
 
Informix warehouse accelerator update
Informix warehouse accelerator updateInformix warehouse accelerator update
Informix warehouse accelerator update
 
Best Practices for a Complete Postgres Enterprise Architecture Setup
Best Practices for a Complete Postgres Enterprise Architecture SetupBest Practices for a Complete Postgres Enterprise Architecture Setup
Best Practices for a Complete Postgres Enterprise Architecture Setup
 
HPC DAY 2017 | HPE Storage and Data Management for Big Data
HPC DAY 2017 | HPE Storage and Data Management for Big DataHPC DAY 2017 | HPE Storage and Data Management for Big Data
HPC DAY 2017 | HPE Storage and Data Management for Big Data
 
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based HardwareRed hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
 
Using IBM Rational Change as an Enterprise-Wide Error Management Solution – ...
 Using IBM Rational Change as an Enterprise-Wide Error Management Solution – ... Using IBM Rational Change as an Enterprise-Wide Error Management Solution – ...
Using IBM Rational Change as an Enterprise-Wide Error Management Solution – ...
 
Delivering High Availability and Performance with SQL Server 2014 (Silviu Nic...
Delivering High Availability and Performance with SQL Server 2014 (Silviu Nic...Delivering High Availability and Performance with SQL Server 2014 (Silviu Nic...
Delivering High Availability and Performance with SQL Server 2014 (Silviu Nic...
 
Presentation oracle optimized solutions
Presentation   oracle optimized solutionsPresentation   oracle optimized solutions
Presentation oracle optimized solutions
 

More from Ivan Donev

Self-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewSelf-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerView
Ivan Donev
 

More from Ivan Donev (9)

Discovery Day 2019 Sofia - Big data clusters
Discovery Day 2019 Sofia - Big data clustersDiscovery Day 2019 Sofia - Big data clusters
Discovery Day 2019 Sofia - Big data clusters
 
Discovery Day 2019 Sofia - What is new in SQL Server 2019
Discovery Day 2019 Sofia - What is new in SQL Server 2019Discovery Day 2019 Sofia - What is new in SQL Server 2019
Discovery Day 2019 Sofia - What is new in SQL Server 2019
 
Bi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stackBi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stack
 
Power bi - enterprise cloud reporting platform Azure Bootcamp 19
Power bi - enterprise cloud reporting platform Azure Bootcamp 19Power bi - enterprise cloud reporting platform Azure Bootcamp 19
Power bi - enterprise cloud reporting platform Azure Bootcamp 19
 
Develop your database with Visual Studio
Develop your database with Visual StudioDevelop your database with Visual Studio
Develop your database with Visual Studio
 
Windows Azure Bootcamp - Microsoft BI in Azure VMs
Windows Azure Bootcamp - Microsoft BI in Azure VMsWindows Azure Bootcamp - Microsoft BI in Azure VMs
Windows Azure Bootcamp - Microsoft BI in Azure VMs
 
Building your first AS solution
Building your first AS solutionBuilding your first AS solution
Building your first AS solution
 
Sql server consolidation and virtualization
Sql server consolidation and virtualizationSql server consolidation and virtualization
Sql server consolidation and virtualization
 
Self-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewSelf-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerView
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern Enterprise
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 

Is "the bigger the beter" valid in the database world

  • 1. Is “the bigger the better” valid in the database world? Best practices around VLDBs HP FastTrack Solution Tips, tricks and gotchas with multi-TB databases
  • 2. About me Ivan Donev, MCT SQL Server Technology Team Lead, HP 2 |
  • 3. Agenda  Introduction  Best practices in configuration  HP FastTrack Reference Architecture 4.0  Share the experience 3 |
  • 4. Introduction  More data  Needs larger hardware  Needs optmization  Needs time  Requires more responsiblities  More responsibilities  Need more attention  Require more knowledge  Give you more headaches 4 |
  • 5. Best practices - IO  Hardware - trust your vendor  Follow the Storage Top 10 Best Practices  HBA Queue Depth  TEST, TEST, TEST 5 |
  • 6. Best practices – SQL Server  Startup options  -E and T1117  T834?  MaxServerMemory  MaxDop – never leave it to 0  Local policy  Lock pages in memory  Perform Volume maintenance tasks  Resource governor! 6 |
  • 7. Best practices – Inside SQL Server  TempDB  Microsoft rules – 1 file per CPU  Experience  Partitioning  Compression – be careful!  Column Store index  Maintenance, maintenance, maintenance!!! 7 |
  • 8. HP FastTrack 4.0  Prebuild Hardware for best performance  Configuration Guide  Performance 8 |
  • 9. Let’s talk about reality  The hardware - HP DL980 G7  2TB RAM  8x8-core Intel Xeon => 128 logical CPUs  Storage – dedicated P2000 and XP24000  The Software  Windows Server 2008 R2 with SP1  SQL Server 2008 R2 Enterprise with CU7 9 |
  • 10. 10 |
  • 11. Let’s talk about reality - servicing  T834 flag or why the engine is not booting up  Patching and ugrade  Upgrade from SQL Server 2005 to 2008 R2  Approach  Licensing  Bugs 11 |
  • 12. Let’s talk about reality - maintenance  Examine fragmentation  Do not forget about statistics  Check the integrity  Size matters  Autogrowth and proportional fill 12 |
  • 13. Let’s talk about reality - compression  Estimate compression gain  Compress and keep compressing  How do we reclaim the space?  Moving object to new file  Shrinking 13 |
  • 14. IS THE BIGGER THE BETTER? 14 |
  • 15. 15 |

Editor's Notes

  1. Some clarifications: we are talking about multiterabyte databases, running on high-end hardware and primarily DATAWAREHOUSE LOAD
  2. The SQL Server environment is just like a garden – the more you want, the more you need to invest
  3. HARDWARE – use the vendors best practices for configuration; every vendor has a guide for SQL Server Best IO performanceTEST – use SQLIO to see if you have the performance needed RAID 10 for disks;Split data from log LUNs1 data file per LUN (use single LOG file) http://technet.microsoft.com/en-us/library/cc966534.aspx
  4. MAX Server Memory – Total Physical Memory – 4-8 GBs for the OSWith the -E startup option, an administrator can increase the number of extents allocated per file in the filegroup during each iteration of the round robin cycle used to distribute extents across files
  5. Lining up the number of data files with CPU’s has scalability advantages for allocation intensive workloads. It is recommended to have .25 to 1 data files (per filegroup) for each CPU on the host server. This is especially true for TEMPDB where the recommendation is 1 data file per CPU.Dual core counts as 2 CPUs; logical procs (hyperthreading) do not.
  6. HARDWARE - DL G7 Bades with MSA P2000 storage
  7. SIZE MATTERS – after 2008R2 SP1 you can take advantage from faster DBCC PHYSICAL_ONLY
  8. SIZE MATTERS – after 2008R2 SP1 you can take advantage from faster DBCC PHYSICAL_ONLY
  9. Do you need big?Do you want big?Can you handle big?