Vld bs conquering the giant
•Download as PPTX, PDF•
0 likes•410 views
This document discusses strategies for managing very large databases (VLDBs). It defines a VLDB as having over 1 terabyte of data or billions of rows. It then covers challenges like rapid data growth, large backup sizes, and slow queries. Potential solutions presented include partitioning tables, backup compression, dropping unused indexes, and using tools like SQL Server's Resource Governor to prioritize workloads. The document recommends Quest software products for capacity planning and compression to help tackle issues in VLDB management.
Report
Share
Report
Share
Recommended
Delphix
Delphix turns database infrastructure into software that operates using less space while maintaining full functionality and performance. It pioneers data de-duplication techniques that are more efficient for databases compared to traditional hypervisor approaches. Delphix's crossflow engine maps database streams into a virtual format called TimeFlow, allowing full database copies to be provisioned instantly without consuming additional hardware resources and with no impact on production performance. This enables faster application development and testing along with reduced infrastructure costs.
Delphix
Delphix turns database hardware into software that runs in a fraction of the space, preserving full functionality and performance. It aims to consolidate non-production databases, automatically refresh and provision them, cutting capital expenses by 50% and slashing operating expenses by 90%. Delphix's solution provides instant virtual copies of databases with no impact to production for accelerated application development, real-time reporting, compliance, and more.
What is Delphix
1. Delphix is a data virtualization platform that creates virtual copies of database environments for development, testing, and analytics.
2. It collects changes incrementally from the source database and stores them in a compressed format, allowing for quick provisioning of clones with near-zero storage.
3. Use cases for Delphix include accelerating development by enabling self-service cloning, facilitating forensics and testing by rewinding databases to past states, and improving BI performance through fast refresh of data warehouses.
Denver devops : enabling DevOps with data virtualization
This document discusses how data constraints can limit DevOps efforts and proposes a solution using virtual data and thin cloning. It notes that moving and copying production data is challenging due to storage, personnel and time requirements. This typically results in bottlenecks, long wait times for environments, code check-ins and production bugs. The solution presented is to use a data virtualization platform that can take thin clones of production data using file system snapshots, compress the data and share it across environments through a centralized cache. This allows self-service provisioning of database environments and accelerates DevOps processes.
Data Virtualization: revolutionizing database cloning
This document discusses how data virtualization can help address the challenges posed by data being a constraint in IT. It describes how data flooding infrastructure strains resources and costs companies huge amounts of money. Typical database architectures result in a "triple data tax". The document proposes data virtualization as a solution, allowing for unlimited clones of database environments using thin cloning. This enables faster development and testing, quicker disaster recovery, and real-time business intelligence. Case studies show benefits like doubling development throughput, 9x faster analytics refreshes, and cutting project timelines in half.
12 best practices for virtualizing active directory DCs
The document outlines 12 best practices for virtualizing Active Directory domain controllers (AD DCs). It discusses why AD DCs are different from other workloads and should be virtualized carefully. The best practices include ensuring high availability of virtualized DCs, avoiding snapshots and clones of DCs, using backups that can restore objects and forests, preventing clock drift, and having anti-affinity rules and disaster recovery plans for DCs. Implementing these practices can make virtualizing DCs safer while gaining benefits of virtualization like high availability and migration capabilities.
Accelerate Develoment with VIrtual Data
This document summarizes best practices for application development using data virtualization to remove data as a constraint. It discusses how data management currently does not scale with agile development and is a major bottleneck. The solution presented is using a data virtualization appliance to create thin clones from production data for development, QA, and test environments. This allows for self-service provisioning of environments and parallel development. It provides use cases showing how virtual data improves development throughput, shifts testing left to find bugs earlier, and enables continuous delivery of features to production.
Exploring Windows XP to 7 Migration Options
In this speech for the Chicago Windows User Group, I review six different technologies that both individuals and enterprise IT managers can use to migrate XP desktops to Windows 7.
Recommended
Delphix
Delphix turns database infrastructure into software that operates using less space while maintaining full functionality and performance. It pioneers data de-duplication techniques that are more efficient for databases compared to traditional hypervisor approaches. Delphix's crossflow engine maps database streams into a virtual format called TimeFlow, allowing full database copies to be provisioned instantly without consuming additional hardware resources and with no impact on production performance. This enables faster application development and testing along with reduced infrastructure costs.
Delphix
Delphix turns database hardware into software that runs in a fraction of the space, preserving full functionality and performance. It aims to consolidate non-production databases, automatically refresh and provision them, cutting capital expenses by 50% and slashing operating expenses by 90%. Delphix's solution provides instant virtual copies of databases with no impact to production for accelerated application development, real-time reporting, compliance, and more.
What is Delphix
1. Delphix is a data virtualization platform that creates virtual copies of database environments for development, testing, and analytics.
2. It collects changes incrementally from the source database and stores them in a compressed format, allowing for quick provisioning of clones with near-zero storage.
3. Use cases for Delphix include accelerating development by enabling self-service cloning, facilitating forensics and testing by rewinding databases to past states, and improving BI performance through fast refresh of data warehouses.
Denver devops : enabling DevOps with data virtualization
This document discusses how data constraints can limit DevOps efforts and proposes a solution using virtual data and thin cloning. It notes that moving and copying production data is challenging due to storage, personnel and time requirements. This typically results in bottlenecks, long wait times for environments, code check-ins and production bugs. The solution presented is to use a data virtualization platform that can take thin clones of production data using file system snapshots, compress the data and share it across environments through a centralized cache. This allows self-service provisioning of database environments and accelerates DevOps processes.
Data Virtualization: revolutionizing database cloning
This document discusses how data virtualization can help address the challenges posed by data being a constraint in IT. It describes how data flooding infrastructure strains resources and costs companies huge amounts of money. Typical database architectures result in a "triple data tax". The document proposes data virtualization as a solution, allowing for unlimited clones of database environments using thin cloning. This enables faster development and testing, quicker disaster recovery, and real-time business intelligence. Case studies show benefits like doubling development throughput, 9x faster analytics refreshes, and cutting project timelines in half.
12 best practices for virtualizing active directory DCs
The document outlines 12 best practices for virtualizing Active Directory domain controllers (AD DCs). It discusses why AD DCs are different from other workloads and should be virtualized carefully. The best practices include ensuring high availability of virtualized DCs, avoiding snapshots and clones of DCs, using backups that can restore objects and forests, preventing clock drift, and having anti-affinity rules and disaster recovery plans for DCs. Implementing these practices can make virtualizing DCs safer while gaining benefits of virtualization like high availability and migration capabilities.
Accelerate Develoment with VIrtual Data
This document summarizes best practices for application development using data virtualization to remove data as a constraint. It discusses how data management currently does not scale with agile development and is a major bottleneck. The solution presented is using a data virtualization appliance to create thin clones from production data for development, QA, and test environments. This allows for self-service provisioning of environments and parallel development. It provides use cases showing how virtual data improves development throughput, shifts testing left to find bugs earlier, and enables continuous delivery of features to production.
Exploring Windows XP to 7 Migration Options
In this speech for the Chicago Windows User Group, I review six different technologies that both individuals and enterprise IT managers can use to migrate XP desktops to Windows 7.
VLDB Administration Strategies
This document discusses very large database (VLDB) configurations and maintenance. It begins by defining a VLDB as a database occupying more than 1 terabyte or containing several billion rows. It then covers various configuration topics like operating system settings, instance memory allocation including the importance of tempdb configuration, and database file configuration. The document also discusses maintenance best practices such as disaster recovery planning, partitioning data to aid restores, compressing backups and data, purging or archiving old data, and performing regular index maintenance and integrity checks.
Vldb Yaron Moshe
This document discusses SQL Server 2008 as a platform for managing petabyte-scale data. It defines what a very large database (VLDB) is and notes the challenges in managing massive amounts of data insertion, queries and high availability. The key design philosophy is to partition large databases into smaller, more manageable components rather than having a single large database. Methods for partitioning include by server, instance, database and tables. Partitioning can be by data, such as by month or state, or by function, such as sales vs manufacturing data. Table partitioning commonly uses a "sliding window" approach. Scalability is achieved through clustering while cost is reduced via compression, smaller partitions and moving historical data to cheaper storage.
נועם
SQL Server allows for parallel processing to improve query performance. It uses multiple threads or workers that can run tasks such as reading, writing, and calculating in parallel across multiple CPU cores. The degree of parallelism (DOP) or number of threads used can be configured through settings like MAXDOP. Determining the optimal DOP requires testing different values and monitoring performance and waits to find the best setting for each workload type.
Flashy prefetching for high performance flash drives
The document describes a technique called "flashy prefetching" that aims to improve data prefetching performance for high-performance flash drives such as SSDs. It recognizes that traditional prefetching is not optimized for flash drives and their characteristics. Flashy prefetching dynamically controls prefetching aggressiveness based on drive performance, prefetching performance, and enabling prefetching for multiple accesses. An implementation called "Prefetchd" achieved on average 20% speedup on large file benchmarks and 65-70% prefetching accuracy.
Vldb 2010 event processing tutorial
The document provides an overview of event processing, including its history, current state, and future trends. It discusses event processing concepts like events, patterns, context, and languages. It also outlines some challenges like ordering events in distributed systems and dealing with uncertainty. The future trends highlighted include expanding applications of event processing, diversifying platforms and quality of service needs, standardizing aspects of event processing, and making it more accessible to non-programmers.
Big data processing systems research
This document summarizes several systems for big data processing that extend or improve upon the MapReduce programming model. It discusses systems for iterative processing like HaLoop, stream processing like Muppet, improving performance through caching and indexing like Incoop and HAIL, and automatic optimization of MapReduce programs like MANIMAL and SkewTune. The document also briefly introduces broader distributed data processing frameworks beyond MapReduce like Dryad, SCOPE, Spark, Nephele/PACTs, and the ASTERIX scalable data platform.
SQL Server 2008 R2 - Implementing High Availabilitty
SQL Server 2008 R2 provides several techniques for achieving high availability including failover clustering, database mirroring, replication, and log shipping. Failover clustering uses redundant hardware and allows for active-active or active-passive configurations with automatic failover. Database mirroring maintains a synchronous up-to-date copy of a database on a secondary server and supports automatic or manual failover. Replication copies and distributes database changes across multiple servers but does not provide built-in failover. Log shipping backs up and restores transaction logs between servers periodically but is not real-time and disconnects users during restores. Each technique has advantages depending on requirements for redundancy, performance, and downtime prevention.
Daniel Abadi: VLDB 2009 Panel
The document discusses the database research community's addiction to relational database technology and outlines steps to address it, acknowledging that relational databases do not scale well for modern web-scale data needs. It notes that companies are turning to systems like Hadoop instead and that the highest impact work is now being done outside traditional database conferences. It suggests a multi-step approach for the community to admit relational databases have scaling problems, embrace new systems like MapReduce, inventory limitations, focus research on fault tolerance and scheduling, and work to remove shortcomings.
2016 VLDB - Messing Up with Bart: Error Generation for Evaluating Data-Cleani...
We study the problem of introducing errors into clean databases for the purpose of benchmarking data-cleaning algorithms. Our goal is to provide users with the highest possible level of control over the error-generation process, and at the same time develop solutions that scale to large databases. We show in the paper that the error-generation problem is surprisingly challenging, and in fact, NP-complete. To pro- vide a scalable solution, we develop a correct and efficient greedy algorithm that sacrifices completeness, but succeeds under very reasonable assumptions. To scale to millions of tuples, the algorithm relies on several non-trivial optimizations, including a new symmetry property of data quality constraints. The trade-off between control and scalability is the main technical contribution of the paper.
Performance Issues on Hadoop Clusters
The document discusses performance issues on Hadoop clusters and proposes three solutions:
1) Data placement - Distributing data across heterogeneous cluster nodes according to their computing capabilities to reduce data transfer time.
2) Prefetching - Improving performance by preloading required data before tasks are assigned to reduce CPU stall time.
3) Preshuffling - Minimizing data shuffling during reduce by pipelining intermediate data between map and reduce tasks.
Hadoop admin
This document provides an overview of Hadoop and MapReduce concepts. It discusses:
- HDFS architecture with NameNode and DataNodes for metadata and data storage. HDFS provides reliability through block replication across nodes.
- MapReduce framework for distributed processing of large datasets across clusters. It consists of map and reduce phases with intermediate shuffling and sorting of data.
- Hadoop was developed based on Google's papers describing their distributed file system GFS and MapReduce processing model. It allows processing of data in parallel across large clusters of commodity hardware.
Case Study with Answers.com on Scaling with Memcached and MySQL
The document discusses Answers.com's experience scaling their website and databases. They used several techniques including adding solid state drives to MySQL servers, implementing a Memcached caching tier with Gear6 for redundancy and failover, using Lighttpd to monitor database servers, and hardware load balancers. Gear6 provided an alternative to developing a custom Memcached wrapper that mirrored data and improved reliability over the standard Memcached implementation. The presentation provides examples of how these techniques helped Answers.com scale their infrastructure to support over 50 million monthly users.
Kscope 14 Presentation : Virtual Data Platform
This document discusses how data constraints impact IT and businesses. It presents virtual data as a solution to alleviate these constraints. Some key points made include:
1) Data constraints strain IT resources, have a huge price tag, and many companies are unaware of their impact. Moving and managing data is difficult and expensive.
2) Virtual data platforms use thin cloning and compression to reduce storage needs and provision environments quickly. This helps development, QA, recovery scenarios and enables faster business intelligence refreshes.
3) Customers using virtual data have seen benefits like doubling development throughput, slashing financial close times by 10x, and cutting surgical recovery times by 8x. Virtual data helps optimize the "factory floor of IT
DevOps, Databases and The Phoenix Project UGF4042 from OOW14
This document discusses using a virtual data platform to address data constraints in IT. It begins by explaining how data flooding infrastructure strains IT resources and costs companies huge sums. Most companies are unaware of these data costs. The solution presented is a virtual data appliance that can clone database environments from snapshots to provision dev/test environments quickly without consuming large amounts of physical storage. Key benefits outlined include unlimited, full-sized, self-service database environments for development and QA teams as well as fast rollback capabilities and A/B testing for QA.
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
This document discusses worst practices for database administrators (DBAs). It covers issues related to disaster recovery planning, backups and restores, change control, storage configuration, file configuration, indexing, and administration techniques. Specifically, it highlights not having service level agreements, not testing disaster recovery plans, relying only on backups without verifying them, insufficient test environments, no performance baselines, capacity-centric rather than performance-centric storage design, manual rather than automated administration, and not defining alerts or checklists. The goal is to help DBAs avoid these pitfalls and instead follow best practices for database management and high availability.
Data Virtualization: Revolutionizing data cloning
This document discusses data virtualization and its use in DevOps. It begins by explaining that data virtualization, also known as copy data management, is becoming more common. It then discusses how data virtualization enables DevOps practices like continuous integration by allowing fast provisioning of full database environments.
The document outlines some of the typical challenges with traditional database architectures, including long setup times, lack of parallel environments, and high storage costs due to many full database copies. It presents data virtualization as a solution, allowing instant provisioning of thin clones from a production database. Finally, it provides examples of how data virtualization can help with development/QA, production support, and business intelligence use cases.
SQL Server On SANs
How to configure and test SQL Server on storage area networks (SANs) with tools like multipathing and SQLIO.
BGOUG "Agile Data: revolutionizing database cloning'
The document discusses using data virtualization to address the constraint of data in DevOps workflows. It describes how traditional database cloning methods are inefficient and consume significant resources. The solution presented uses thin cloning technology to take snapshots of production databases and provide virtual copies for development, QA, and other environments. This allows for unlimited, self-service virtual databases that reduce bottlenecks and waiting times compared to physical copies.
Apache Storm
This document provides an introduction to Apache Storm and real-time big data analytics. It discusses how Storm can be used for processing real-time streaming data, as batch processing is not suitable. The three key points are:
1) Storm is an open source framework for real-time processing of streaming data. It uses spouts and bolts to distribute work across a cluster.
2) Real-time analytics are needed in many domains like social media, SaaS, and financial services to take immediate actions.
3) The Lambda architecture combines batch and real-time processing to always serve the latest data through merging batch and real-time views.
Data Vault 2.0 Demystified: East Coast Tour
Dan Linstedt, inventor of Data Vault 2.0, explained why many see Data Vault as the trend of the future for Data Warehousing.
During the event, attendees heard how Data Vault 2.0 can help their teams:
- Manage and enforce compliance to Sarbanes-Oxley, HIPPA, and BASIL II in your Enterprise Data Warehouse
- Spot business problems that were never visible previously
- Rapidly reduce business cycle time for implementing changes
- Merge new business units into the organization rapidly
- Achieve rapid ROI and delivery of information to new Star Schemas
- Consolidate disparate data stores, tackling Master Data Management
- Implement and deploy Service-Oriented Architecture fast
- Scale efficiently to hundreds of Terabytes or Petabytes
- Reach SEI CMM Level 5 compliance (repeatable, consistent, redundant architecture)
- More easily trace all of your data back to the source system
Kscope 2013 delphix
This document discusses database virtualization and instant cloning technologies. It begins by outlining the challenges businesses face with growing databases and increasing demands for copies from developers, reporting teams, etc. It then covers three main parts:
1) Cloning technologies including physical cloning, thin provision cloning using file system snapshots, and database virtualization.
2) How these technologies can accelerate businesses by enabling faster development, testing, recovery and reporting.
3) Specific use cases like development acceleration through frequent, full clones; branching for rapid QA; recovery and testing capabilities; and enabling fast data refreshes for reporting.
6212883126866262792 performance testing_cloud
1) The document discusses performance testing in the cloud for Oracle Database upgrades, utilities, cloud migrations, and patching. It provides an overview of common testing challenges and how to address them when testing in the cloud.
2) Tools like SQL Performance Analyzer, Database Replay, and Real Application Testing are included with some cloud database offerings and can help with testing in the cloud. Data subsetting techniques and using snapshot standbys are also discussed.
3) Repeatable testing is important, and restoring to guaranteed restore points or using snapshot standbys allows restoring the database to a known state before and after tests. Statistics need to be refreshed after restoring to ensure accurate optimizer statistics.
More Related Content
Viewers also liked
VLDB Administration Strategies
This document discusses very large database (VLDB) configurations and maintenance. It begins by defining a VLDB as a database occupying more than 1 terabyte or containing several billion rows. It then covers various configuration topics like operating system settings, instance memory allocation including the importance of tempdb configuration, and database file configuration. The document also discusses maintenance best practices such as disaster recovery planning, partitioning data to aid restores, compressing backups and data, purging or archiving old data, and performing regular index maintenance and integrity checks.
Vldb Yaron Moshe
This document discusses SQL Server 2008 as a platform for managing petabyte-scale data. It defines what a very large database (VLDB) is and notes the challenges in managing massive amounts of data insertion, queries and high availability. The key design philosophy is to partition large databases into smaller, more manageable components rather than having a single large database. Methods for partitioning include by server, instance, database and tables. Partitioning can be by data, such as by month or state, or by function, such as sales vs manufacturing data. Table partitioning commonly uses a "sliding window" approach. Scalability is achieved through clustering while cost is reduced via compression, smaller partitions and moving historical data to cheaper storage.
נועם
SQL Server allows for parallel processing to improve query performance. It uses multiple threads or workers that can run tasks such as reading, writing, and calculating in parallel across multiple CPU cores. The degree of parallelism (DOP) or number of threads used can be configured through settings like MAXDOP. Determining the optimal DOP requires testing different values and monitoring performance and waits to find the best setting for each workload type.
Flashy prefetching for high performance flash drives
The document describes a technique called "flashy prefetching" that aims to improve data prefetching performance for high-performance flash drives such as SSDs. It recognizes that traditional prefetching is not optimized for flash drives and their characteristics. Flashy prefetching dynamically controls prefetching aggressiveness based on drive performance, prefetching performance, and enabling prefetching for multiple accesses. An implementation called "Prefetchd" achieved on average 20% speedup on large file benchmarks and 65-70% prefetching accuracy.
Vldb 2010 event processing tutorial
The document provides an overview of event processing, including its history, current state, and future trends. It discusses event processing concepts like events, patterns, context, and languages. It also outlines some challenges like ordering events in distributed systems and dealing with uncertainty. The future trends highlighted include expanding applications of event processing, diversifying platforms and quality of service needs, standardizing aspects of event processing, and making it more accessible to non-programmers.
Big data processing systems research
This document summarizes several systems for big data processing that extend or improve upon the MapReduce programming model. It discusses systems for iterative processing like HaLoop, stream processing like Muppet, improving performance through caching and indexing like Incoop and HAIL, and automatic optimization of MapReduce programs like MANIMAL and SkewTune. The document also briefly introduces broader distributed data processing frameworks beyond MapReduce like Dryad, SCOPE, Spark, Nephele/PACTs, and the ASTERIX scalable data platform.
SQL Server 2008 R2 - Implementing High Availabilitty
SQL Server 2008 R2 provides several techniques for achieving high availability including failover clustering, database mirroring, replication, and log shipping. Failover clustering uses redundant hardware and allows for active-active or active-passive configurations with automatic failover. Database mirroring maintains a synchronous up-to-date copy of a database on a secondary server and supports automatic or manual failover. Replication copies and distributes database changes across multiple servers but does not provide built-in failover. Log shipping backs up and restores transaction logs between servers periodically but is not real-time and disconnects users during restores. Each technique has advantages depending on requirements for redundancy, performance, and downtime prevention.
Daniel Abadi: VLDB 2009 Panel
The document discusses the database research community's addiction to relational database technology and outlines steps to address it, acknowledging that relational databases do not scale well for modern web-scale data needs. It notes that companies are turning to systems like Hadoop instead and that the highest impact work is now being done outside traditional database conferences. It suggests a multi-step approach for the community to admit relational databases have scaling problems, embrace new systems like MapReduce, inventory limitations, focus research on fault tolerance and scheduling, and work to remove shortcomings.
2016 VLDB - Messing Up with Bart: Error Generation for Evaluating Data-Cleani...
We study the problem of introducing errors into clean databases for the purpose of benchmarking data-cleaning algorithms. Our goal is to provide users with the highest possible level of control over the error-generation process, and at the same time develop solutions that scale to large databases. We show in the paper that the error-generation problem is surprisingly challenging, and in fact, NP-complete. To pro- vide a scalable solution, we develop a correct and efficient greedy algorithm that sacrifices completeness, but succeeds under very reasonable assumptions. To scale to millions of tuples, the algorithm relies on several non-trivial optimizations, including a new symmetry property of data quality constraints. The trade-off between control and scalability is the main technical contribution of the paper.
Performance Issues on Hadoop Clusters
The document discusses performance issues on Hadoop clusters and proposes three solutions:
1) Data placement - Distributing data across heterogeneous cluster nodes according to their computing capabilities to reduce data transfer time.
2) Prefetching - Improving performance by preloading required data before tasks are assigned to reduce CPU stall time.
3) Preshuffling - Minimizing data shuffling during reduce by pipelining intermediate data between map and reduce tasks.
Hadoop admin
This document provides an overview of Hadoop and MapReduce concepts. It discusses:
- HDFS architecture with NameNode and DataNodes for metadata and data storage. HDFS provides reliability through block replication across nodes.
- MapReduce framework for distributed processing of large datasets across clusters. It consists of map and reduce phases with intermediate shuffling and sorting of data.
- Hadoop was developed based on Google's papers describing their distributed file system GFS and MapReduce processing model. It allows processing of data in parallel across large clusters of commodity hardware.
Viewers also liked (11)
Flashy prefetching for high performance flash drives
Flashy prefetching for high performance flash drives
SQL Server 2008 R2 - Implementing High Availabilitty
SQL Server 2008 R2 - Implementing High Availabilitty
2016 VLDB - Messing Up with Bart: Error Generation for Evaluating Data-Cleani...
2016 VLDB - Messing Up with Bart: Error Generation for Evaluating Data-Cleani...
Similar to Vld bs conquering the giant
Case Study with Answers.com on Scaling with Memcached and MySQL
The document discusses Answers.com's experience scaling their website and databases. They used several techniques including adding solid state drives to MySQL servers, implementing a Memcached caching tier with Gear6 for redundancy and failover, using Lighttpd to monitor database servers, and hardware load balancers. Gear6 provided an alternative to developing a custom Memcached wrapper that mirrored data and improved reliability over the standard Memcached implementation. The presentation provides examples of how these techniques helped Answers.com scale their infrastructure to support over 50 million monthly users.
Kscope 14 Presentation : Virtual Data Platform
This document discusses how data constraints impact IT and businesses. It presents virtual data as a solution to alleviate these constraints. Some key points made include:
1) Data constraints strain IT resources, have a huge price tag, and many companies are unaware of their impact. Moving and managing data is difficult and expensive.
2) Virtual data platforms use thin cloning and compression to reduce storage needs and provision environments quickly. This helps development, QA, recovery scenarios and enables faster business intelligence refreshes.
3) Customers using virtual data have seen benefits like doubling development throughput, slashing financial close times by 10x, and cutting surgical recovery times by 8x. Virtual data helps optimize the "factory floor of IT
DevOps, Databases and The Phoenix Project UGF4042 from OOW14
This document discusses using a virtual data platform to address data constraints in IT. It begins by explaining how data flooding infrastructure strains IT resources and costs companies huge sums. Most companies are unaware of these data costs. The solution presented is a virtual data appliance that can clone database environments from snapshots to provision dev/test environments quickly without consuming large amounts of physical storage. Key benefits outlined include unlimited, full-sized, self-service database environments for development and QA teams as well as fast rollback capabilities and A/B testing for QA.
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
This document discusses worst practices for database administrators (DBAs). It covers issues related to disaster recovery planning, backups and restores, change control, storage configuration, file configuration, indexing, and administration techniques. Specifically, it highlights not having service level agreements, not testing disaster recovery plans, relying only on backups without verifying them, insufficient test environments, no performance baselines, capacity-centric rather than performance-centric storage design, manual rather than automated administration, and not defining alerts or checklists. The goal is to help DBAs avoid these pitfalls and instead follow best practices for database management and high availability.
Data Virtualization: Revolutionizing data cloning
This document discusses data virtualization and its use in DevOps. It begins by explaining that data virtualization, also known as copy data management, is becoming more common. It then discusses how data virtualization enables DevOps practices like continuous integration by allowing fast provisioning of full database environments.
The document outlines some of the typical challenges with traditional database architectures, including long setup times, lack of parallel environments, and high storage costs due to many full database copies. It presents data virtualization as a solution, allowing instant provisioning of thin clones from a production database. Finally, it provides examples of how data virtualization can help with development/QA, production support, and business intelligence use cases.
SQL Server On SANs
How to configure and test SQL Server on storage area networks (SANs) with tools like multipathing and SQLIO.
BGOUG "Agile Data: revolutionizing database cloning'
The document discusses using data virtualization to address the constraint of data in DevOps workflows. It describes how traditional database cloning methods are inefficient and consume significant resources. The solution presented uses thin cloning technology to take snapshots of production databases and provide virtual copies for development, QA, and other environments. This allows for unlimited, self-service virtual databases that reduce bottlenecks and waiting times compared to physical copies.
Apache Storm
This document provides an introduction to Apache Storm and real-time big data analytics. It discusses how Storm can be used for processing real-time streaming data, as batch processing is not suitable. The three key points are:
1) Storm is an open source framework for real-time processing of streaming data. It uses spouts and bolts to distribute work across a cluster.
2) Real-time analytics are needed in many domains like social media, SaaS, and financial services to take immediate actions.
3) The Lambda architecture combines batch and real-time processing to always serve the latest data through merging batch and real-time views.
Data Vault 2.0 Demystified: East Coast Tour
Dan Linstedt, inventor of Data Vault 2.0, explained why many see Data Vault as the trend of the future for Data Warehousing.
During the event, attendees heard how Data Vault 2.0 can help their teams:
- Manage and enforce compliance to Sarbanes-Oxley, HIPPA, and BASIL II in your Enterprise Data Warehouse
- Spot business problems that were never visible previously
- Rapidly reduce business cycle time for implementing changes
- Merge new business units into the organization rapidly
- Achieve rapid ROI and delivery of information to new Star Schemas
- Consolidate disparate data stores, tackling Master Data Management
- Implement and deploy Service-Oriented Architecture fast
- Scale efficiently to hundreds of Terabytes or Petabytes
- Reach SEI CMM Level 5 compliance (repeatable, consistent, redundant architecture)
- More easily trace all of your data back to the source system
Kscope 2013 delphix
This document discusses database virtualization and instant cloning technologies. It begins by outlining the challenges businesses face with growing databases and increasing demands for copies from developers, reporting teams, etc. It then covers three main parts:
1) Cloning technologies including physical cloning, thin provision cloning using file system snapshots, and database virtualization.
2) How these technologies can accelerate businesses by enabling faster development, testing, recovery and reporting.
3) Specific use cases like development acceleration through frequent, full clones; branching for rapid QA; recovery and testing capabilities; and enabling fast data refreshes for reporting.
6212883126866262792 performance testing_cloud
1) The document discusses performance testing in the cloud for Oracle Database upgrades, utilities, cloud migrations, and patching. It provides an overview of common testing challenges and how to address them when testing in the cloud.
2) Tools like SQL Performance Analyzer, Database Replay, and Real Application Testing are included with some cloud database offerings and can help with testing in the cloud. Data subsetting techniques and using snapshot standbys are also discussed.
3) Repeatable testing is important, and restoring to guaranteed restore points or using snapshot standbys allows restoring the database to a known state before and after tests. Statistics need to be refreshed after restoring to ensure accurate optimizer statistics.
Kubernetes Failure Stories - KubeCon Europe Barcelona
Talk given on 2019-05-21 at KubeCon Barcelona: https://kccnceu19.sched.com/event/MPcM/kubernetes-failure-stories-and-how-to-crash-your-clusters-henning-jacobs-zalando-se
Bootstrapping a Kubernetes cluster is easy, rolling it out to nearly 200 engineering teams and operating it at scale is a challenge. In this talk, we are presenting our approach to Kubernetes provisioning on AWS, operations and developer experience for our growing Zalando developer base. We will walk you through our horror stories of operating 100+ clusters and share the insights we gained from incidents, failures, user reports and general observations. Our failure stories will be sourced from recent and past incidents, so the talk will be up-to-date with our latest experiences.
Most of our learnings apply to other Kubernetes infrastructures (EKS, GKE, ..) as well. This talk strives to reduce the audience's unknown unknowns about running Kubernetes in production.
VMware vFabric Data Director for DB as a Service
vFabric Data Director is a software solution that enables enterprises to provide self-service database provisioning and management capabilities to users on demand. It allows developers to quickly provision fully configured databases with automated backup and restore. It also provides administrators with fine-grained access controls and tools to enforce IT policies and optimize resource utilization. The solution is built on VMware vSphere and uses a vSphere-optimized PostgreSQL database to deliver benefits like elastic resource allocation and automated high availability.
Data as a Service
This document discusses using data virtualization to accelerate application projects by 50%. It outlines some common problems with physical data copies, such as bottlenecks, bugs due to old data, difficulty creating subsets, and delays. The document then introduces the concept of using a data virtualization appliance to take snapshots of production data and create thin clones for development and testing environments. This allows for fast, full-sized, self-service clones that can be refreshed quickly. Use cases discussed include improved development and testing workflows, faster production support like recovery and migration, and enabling continuous business intelligence functions.
Business_Continuity_Planning_with_SQL_Server_HADR_options_TechEd_Bangalore_20...
The document discusses several high availability and disaster recovery options for SQL Server including failover clustering, database mirroring, log shipping, and replication. It provides examples of how different companies have implemented these technologies depending on their requirements. Key factors that influence architecture choices are downtime tolerance, deployment of technologies, and operational procedures. The document also covers SQL Server upgrade processes and how to move databases to a new datacenter while maintaining high availability.
DOs and DONTs on the way to 10M users
The document summarizes Wix's evolution from its initial architecture to a more distributed and scalable architecture over time. Some key lessons learned include:
- The initial architecture worked well for a startup but needed replacing within 2 years as needs changed.
- Architect for gradual rewrites and separating concerns as understanding evolves.
- Caching should be introduced selectively to address real performance needs.
- Separating the editor and public segments improved reliability and release cycles.
- Immutable data and GUID keys improved scalability of the database.
- A content delivery network improved media file performance significantly.
- Automated testing, continuous integration, and DevOps practices improved release quality and frequency.
Denver SQL Saturday The Next Frontier
The document discusses opportunities for using cloud and virtualization technologies. It describes how data virtualization can help optimize copy data management tasks like provisioning databases and refreshing environments. Virtualization allows creating many virtual copies of databases that take up minimal storage space since they share data blocks. This simplifies tasks like patching databases. The document also discusses best practices for migrating databases to the cloud and using data virtualization to enhance the migration process.
Virtualization and SAN Basics for DBAs
This document discusses best practices for virtual machines (VMs), storage area networks (SANs), and SQL Server. It provides three "nevers" for VMs: never overallocate virtual CPUs, never use automatic settings, and never assume VMs are alone. It also gives three "always" for SANs: always know your neighbors on the SAN, always test storage performance first with SQLIO before SQL Server, and always be checking performance metrics. Key metrics discussed include processor queue length, SQL Server memory page life expectancy, and physical disk read/write average time. The document emphasizes testing storage, understanding competition from other workloads, and monitoring for subtle performance changes.
Stating the obvious - 121 Test Automation Day, Dublin, 2018
121 Test Automation Day
Dublin, May 23rd 2018
https://1point21gws.com/testingsummit/dublin/
Stating the obvious: adding performance and scalability tests to a Continuous Integration pipeline
Performance and scalability are core quality attributes of any system; unit testing, integration testing, UI testing, they all focus on functional requirements. Good performances mean happy users, less resource usage which translates to lower running costs (power, cloud bills) and customer retention.
In this session we will recall some basic concept of performance testing and demonstrate some of the many tools available in the cloud.
Things learned from OpenWorld 2013
The document discusses the history and evolution of Oracle Database from its beginnings in 1977 through version 12c. It describes how early versions introduced SQL and basic reliability features, and how subsequent versions added capabilities like distributed processing, transactions, PL/SQL, and Real Application Clusters. It also summarizes how new pluggable database and in-memory technologies in version 12c allow for more efficient consolidation of databases and management of storage.
Similar to Vld bs conquering the giant (20)
Case Study with Answers.com on Scaling with Memcached and MySQL
Case Study with Answers.com on Scaling with Memcached and MySQL
DevOps, Databases and The Phoenix Project UGF4042 from OOW14
DevOps, Databases and The Phoenix Project UGF4042 from OOW14
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
BGOUG "Agile Data: revolutionizing database cloning'
BGOUG "Agile Data: revolutionizing database cloning'
Kubernetes Failure Stories - KubeCon Europe Barcelona
Kubernetes Failure Stories - KubeCon Europe Barcelona
Business_Continuity_Planning_with_SQL_Server_HADR_options_TechEd_Bangalore_20...
Business_Continuity_Planning_with_SQL_Server_HADR_options_TechEd_Bangalore_20...
Stating the obvious - 121 Test Automation Day, Dublin, 2018
Stating the obvious - 121 Test Automation Day, Dublin, 2018
Recently uploaded
Infrastructure Challenges in Scaling RAG with Custom AI models
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Mind map of terminologies used in context of Generative AI
Mind map of common terms used in context of Generative AI.
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Best 20 SEO Techniques To Improve Website Visibility In SERP
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Recently uploaded (20)
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Vld bs conquering the giant
- 1. Conquering the Giants:Very Large Databases For audio, call the phone number in your meeting invite. Audio will not come through the computer speakers.
- 2. SQL Server Expertfor Quest Software Editor-in-Chief at SQLServerPedia.com Former SQL DBA Managed data warehouses,>80tb SAN storage for $7b company
- 3. Conquering the Giants: Agenda What’s a VLDB? Ginormous Growth Big Backups Tremendous Tables Slow SQL Statements Your Wingman: Quest Resources Photo Licensed with Creative Commons from http://www.flickr.com/photos/pagedooley/2172001078/
- 4. What Makes a VLDB? 1 terabyte Billion rows But Not:User Qty,Criticality,Transactions Per Second 4 Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053095394
- 6. Default File Growth Settings 6
- 7. Starts Out Small… 7
- 9. Instant File Init in Secpol.msc 9
- 10. VLDBs = Big Backups Time Problem Nightly Load Windows Time to Recover Dev, QA, Test, DR 10
- 11. Giant Monstrous Backups 11
- 12. Cutting Them Down to Size 12
- 13. Next Weapon: SAN Snapshots 13
- 14. Post-Snapshot 14
- 15. Making the Snap Available 15
- 16. Weapon: Backup Compression 16 Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053045338/
- 17. Or – Compress Everything! 17
- 18. VLDB Problem: Giant Tables 18
- 19. Hunt Down and Kill Unused Indexes 19
- 20. Breaking Giants Into Pieces 20
- 21. The New Way: Partitioning 21
- 22. Partitioning Basics Create filegroups Create files in the filegroups Design partition function Design partition scheme Alter table’s clustered index 22
- 23. Sliding Window Loads Partition by day Create newtable similarto Sales table,on same filegroup Load one day’sdata into empty table Swap out a partition for that table 23
- 24. Giant Drawbacks Not easy to change Requires lots of experimentation Partition elimination doesn’t always work Requires familiarity with queries, data Storage skills help Hardware-dependent 24 Photo Licensed with Creative Commons from http://www.flickr.com/photos/cellphonesusie/3224335140/
- 25. SQLCAT MAXDOP FAQ 25
- 26. The New-New Way: 2008 R2 26
- 27. Slow SQL Statements 27
- 28. Your Shovel: Resource Governor Slows zombie queries down Throttle CPU, memory Can throttle jobs, backups too Doesn’t throttle storage (yet) 28 Photo Licensed with Creative Commons from http://www.flickr.com/photos/archiemcphee/3612278108/
- 29. Implementing The Governator Workload groups: ETL processes Reports AdHoc Resource pool: ETL Reports Classification function based on: Login name Application name 29 Photo Licensed with Creative Commons from http://www.flickr.com/photos/shaunwong/2467881648/
- 30. End Results in SSMS 2008 30
- 31. Resource Governor Caveats Ask the network team about QoS Get manager consensus on SLAs Keep it absolutely transparent Avoid false alarms about performance problems Teach managers how to “see” it working 31
- 32. Your Wingman: Quest Software Capacity Manager v3.0 Capacity Planning Index Defragmentation Partitioning GUI LiteSpeed v5.1 SmartDiffs Throttled Compression 32
- 33. Showing the Giant Who’s Boss Use Instant File Initialization Consider Differential Backups Compress Data Drop Baggage Partitioning FTW Resource Governator Check Out QuestCapacity Manager Photo Licensed with Creative Commons from http://www.flickr.com/photos/keeg/1661401660/
- 34. VLDB Resources Presentation Resources:http://sqlserverpedia.com/wiki/VLDB Quest Software:http://www.quest.com/sql-server/ Upcoming Webcasts:http://www.quest.com/events
Editor's Notes
- Production, Dev, QA, DRWhen files are this big, even the simplest thing is time-consuming – think growing a file.
- Runs faster tooFrees up nightly windowDoesn’t ease recovery time, though. Makes recovery time LONGER.Let’s look at a solution that can ease recovery time too.
- Do your backups off the snapshotHold multiple snapshots over timeNot everybody can afford this level of recoverability though.Let’s look at something that Microsoft included with SQL 2008 Enterprise Edition.
- New in SQL 2008Enterprise Edition onlyInclude DR, development boxesNot configurable
- New in SQL 2008Enterprise Edition onlyInclude DR, development boxesNot inherited – tables, indexesScript to make it easier at the SSP article
- Note index size on 2nd rowEvery row has to be backed up, DBCC’d. Harder to get DBCC time on VLDBs.Every index has to be backed up and reorganized
- Dropping indexes gives us less of them, but they’re still huge.How can we break them into smaller pieces?
- Easier because it’s transparent to the application.
- Happens nearly instantaneouslyOnly a metadata change
- Zombie ad-hoc queries run forever. User closes web browser, but the query runs on headlessly