5 SQL Server Indexing Myths
•
1 like•1,194 views
There are many "best practices" around that help people decide how to index databases. Having these practices can help alleviate the time that it takes to design an indexing strategy for a database. These practices can be of great use, except when they are wrong. Join us in this session, as we discuss some common myths associated with indexes and then dive into the myths to demonstrate how they can be debunked. At the end of the session, you'll know a few more things about indexes and leave armed with scripts that can help you debunk these myths on your own. http://pragmaticworks.com/LearningCenter/FreeTrainingWebinars/PastWebinars.aspx?ResourceId=654
Report
Share
Report
Share
Download to read offline
Recommended
Getting Started with Windows Azure and SQL Databases
The Microsoft Windows Azure platform provides service-based offerings for your data needs. These include services from file storage to SQL databases that are scalable to your specific needs. Regardless of the size of your data platform, Windows Azure has options that fit to your business needs. In this session, we'll discuss the Windows Azure platform and discuss how the SQL Database and other services enhance and extend your data platform. By session end, you'll understand how Windows Azure is able to fit into your environment and be ready to start using Windows Azure.
http://pragmaticworks.com/LearningCenter/FreeTrainingWebinars/PastWebinars.aspx?ResourceId=639
Introduction to Clustered Indexes and Heaps
It's the age old question, do I put a clustered index on the table or leave it as a heap? While a simple question, this can have serious impacts on the performance of the table in queries. In this session, we'll review the basics of both clustered indexes and heaps; identifying key differences and cases where you might choose one over another.
Leveraging Cloud for the Modern SQL Developer
In the past, developing a database was a simple affair where you built and deployed the database for a server down the hall in your datacenter, or in some cases to a server under someone's desk. While those good ole days were a much more simple time, they were quite limiting in how a database could be deployed and there were much fewer options for ensuring stability and performance. Today, SQL Developers have many more options for deployment including the services provided through Windows Azure. In this session, we'll dive into what you need to know to start developing databases with Windows Azure and discuss the value that cloud delivery can provide to your database development.
http://pragmaticworks.com/LearningCenter/FreeTrainingWebinars/PastWebinars.aspx?ResourceId=640
The Plan Cache Whisperer - Performance Tuning SQL Server
Execution plans tell SQL Server how to execute queries. If you listen closely, execution plans can also tell you when performance-tuning opportunities exist in your environment. By listening to your queries, you can understand how SQL Server is operating and gain insight into how your environment is functioning. In this session, learn how to use XQuery to browse and search the plan cache, enabling you to find potential performance issues and opportunities to tune your queries. In addition, learn how a performance issue on a single execution plan can be used to find similar issues on other execution plans, enabling you to scale up your performance tuning effectiveness. You can use this information to help reduce issues related to parallelism, shift queries from using scans to using seek operations, or discover exactly which queries are using which indexes. All this and more is readily available through the plan cache.
Tools and Tips For Data Warehouse Developers (SQLGLA)
Tools and Tips For Data Warehouse Developers (Presented at SQLGLA in Glasgow, Scotland on September 14th, 2018)
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (Presented for 24 Hours of PASS: Summit Preview on June 13th 2018)
Automating Data Quality Processes at Reckitt
Reckitt is a fast-moving consumer goods company with a portfolio of famous brands and over 30k employees worldwide. With that scale small projects can quickly grow into big datasets, and processing and cleaning all that data can become a challenge. To solve that challenge we have created a metadata driven ETL framework for orchestrating data transformations through parametrised SQL scripts. It allows us to create various paths for our data as well as easily version control them. The approach of standardising incoming datasets and creating reusable SQL processes has proven to be a winning formula. It has helped simplify complicated landing/stage/merge processes and allowed them to be self-documenting.
But this is only half the battle, we also want to create data products. Documented, quality assured data sets that are intuitive to use. As we move to a CI/CD approach, increasing the frequency of deployments, the demand of keeping documentation and data quality assessments up to date becomes increasingly challenging. To solve this problem, we have expanded our ETL framework to include SQL processes that automate data quality activities. Using the Hive metastore as a starting point, we have leveraged this framework to automate the maintenance of a data dictionary and reduce documenting, model refinement, testing data quality and filtering out bad data to a box filling exercise. In this talk we discuss our approach to maintaining high quality data products and share examples of how we automate data quality processes.
Misusing MLflow To Help Deduplicate Data At Scale
At Intuit, we have a lot of data – and a lot of duplicate data collected over decades. So we built a rule-based, self-serve tool to identify and merge duplicate records. It takes experimentation and iteration to get deduplication just right for 100s of millions of records, and spreadsheet-based tracking just wasn’t enough. We now use MLflow to automatically capture execution notes, rule settings, weights, key validation metrics, etc., all without requiring end-user action. In this talk, we’ll talk about our use case and why MLflow is useful outside its traditional ML Ops use cases.
Recommended
Getting Started with Windows Azure and SQL Databases
The Microsoft Windows Azure platform provides service-based offerings for your data needs. These include services from file storage to SQL databases that are scalable to your specific needs. Regardless of the size of your data platform, Windows Azure has options that fit to your business needs. In this session, we'll discuss the Windows Azure platform and discuss how the SQL Database and other services enhance and extend your data platform. By session end, you'll understand how Windows Azure is able to fit into your environment and be ready to start using Windows Azure.
http://pragmaticworks.com/LearningCenter/FreeTrainingWebinars/PastWebinars.aspx?ResourceId=639
Introduction to Clustered Indexes and Heaps
It's the age old question, do I put a clustered index on the table or leave it as a heap? While a simple question, this can have serious impacts on the performance of the table in queries. In this session, we'll review the basics of both clustered indexes and heaps; identifying key differences and cases where you might choose one over another.
Leveraging Cloud for the Modern SQL Developer
In the past, developing a database was a simple affair where you built and deployed the database for a server down the hall in your datacenter, or in some cases to a server under someone's desk. While those good ole days were a much more simple time, they were quite limiting in how a database could be deployed and there were much fewer options for ensuring stability and performance. Today, SQL Developers have many more options for deployment including the services provided through Windows Azure. In this session, we'll dive into what you need to know to start developing databases with Windows Azure and discuss the value that cloud delivery can provide to your database development.
http://pragmaticworks.com/LearningCenter/FreeTrainingWebinars/PastWebinars.aspx?ResourceId=640
The Plan Cache Whisperer - Performance Tuning SQL Server
Execution plans tell SQL Server how to execute queries. If you listen closely, execution plans can also tell you when performance-tuning opportunities exist in your environment. By listening to your queries, you can understand how SQL Server is operating and gain insight into how your environment is functioning. In this session, learn how to use XQuery to browse and search the plan cache, enabling you to find potential performance issues and opportunities to tune your queries. In addition, learn how a performance issue on a single execution plan can be used to find similar issues on other execution plans, enabling you to scale up your performance tuning effectiveness. You can use this information to help reduce issues related to parallelism, shift queries from using scans to using seek operations, or discover exactly which queries are using which indexes. All this and more is readily available through the plan cache.
Tools and Tips For Data Warehouse Developers (SQLGLA)
Tools and Tips For Data Warehouse Developers (Presented at SQLGLA in Glasgow, Scotland on September 14th, 2018)
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (Presented for 24 Hours of PASS: Summit Preview on June 13th 2018)
Automating Data Quality Processes at Reckitt
Reckitt is a fast-moving consumer goods company with a portfolio of famous brands and over 30k employees worldwide. With that scale small projects can quickly grow into big datasets, and processing and cleaning all that data can become a challenge. To solve that challenge we have created a metadata driven ETL framework for orchestrating data transformations through parametrised SQL scripts. It allows us to create various paths for our data as well as easily version control them. The approach of standardising incoming datasets and creating reusable SQL processes has proven to be a winning formula. It has helped simplify complicated landing/stage/merge processes and allowed them to be self-documenting.
But this is only half the battle, we also want to create data products. Documented, quality assured data sets that are intuitive to use. As we move to a CI/CD approach, increasing the frequency of deployments, the demand of keeping documentation and data quality assessments up to date becomes increasingly challenging. To solve this problem, we have expanded our ETL framework to include SQL processes that automate data quality activities. Using the Hive metastore as a starting point, we have leveraged this framework to automate the maintenance of a data dictionary and reduce documenting, model refinement, testing data quality and filtering out bad data to a box filling exercise. In this talk we discuss our approach to maintaining high quality data products and share examples of how we automate data quality processes.
Misusing MLflow To Help Deduplicate Data At Scale
At Intuit, we have a lot of data – and a lot of duplicate data collected over decades. So we built a rule-based, self-serve tool to identify and merge duplicate records. It takes experimentation and iteration to get deduplication just right for 100s of millions of records, and spreadsheet-based tracking just wasn’t enough. We now use MLflow to automatically capture execution notes, rule settings, weights, key validation metrics, etc., all without requiring end-user action. In this talk, we’ll talk about our use case and why MLflow is useful outside its traditional ML Ops use cases.
How to Build a Data-Driven Company: From Infrastructure to Insights
Companies like Buffer, SeatGeek, and Asana aren’t just talking about the value of data, they’re building data infrastructure that can actually deliver it. Join this 45-minute webinar to learn why these companies are investing in data and what you need to know to keep up.
Building the Artificially Intelligent Enterprise
Mike Ferguson is Managing Director of Intelligent Business Strategies Limited and specializes in business intelligence/analytics and data management. He discusses building the artificially intelligent enterprise and transitioning to a self-learning enterprise. Some key challenges discussed include the siloed and fractured nature of current data and analytics efforts, with many tools and scripts in use without integration. He advocates sorting out the data foundation, implementing DataOps and MLOps, creating a data and analytics marketplace, and integrating analytics into business processes to drive value from AI.
Code Once Use Often with Declarative Data Pipelines
The document discusses using declarative data pipelines to code data workflows once and reuse them easily. It describes Flashfood, a company dealing with food waste data. The problem of maintaining many pipelines across different file types and clouds is presented. Three attempts at a solution showed that too little automation led to boilerplate code while too much automation caused unexpected behavior. The solution was to define YAML configuration files that jobs could be run against, allowing flexibility while enforcing DRY principles. This approach reduced maintenance overhead and allowed anyone to create similar jobs. Lessons included favoring parameters over inference and reusing extract and load code. Future work may involve programmatically adding new configurations and a Spark YAML grammar.
Brokering Data: Accelerating Data Evaluation with Databricks White Label
As the data-as-a-service ecosystem continues to evolve, data brokers are faced with an unprecedented challenge – demonstrating the value of their data. Successfully crafting and selling a compelling data product relies on a broker’s ability to differentiate their product from the rest of the market. In smaller or static datasets, measures like row count and cardinality can speak volumes. However, when datasets are in the terabytes or petabytes though – differentiation becomes much difficult. On top of that “data quality” is a somewhat ill-defined term and the definition of a “high quality dataset” can change daily or even hourly.
This breakout session will describe Veraset’s partnership with Databricks, and how we have white labeled Databricks to showcase and accelerate the value of our data. We’ll discuss the challenges that data brokers have faced to date and some of the primitives of our businesses that have guided our direction thus far. We will also actively demo our white label instance and notebook to show how we’ve been able to provide key insights to our customers and reduce the TTFB of data onboarding.
Strategies for SQL Server Index Analysis
Properly managing indexes for a database is a common use in many environments. When reviewing an index, should it have a single or multiple columns? Should the table be a heap, a clustered index or clustered columnstore index? These considerations are often at the forefront when analyzing your indexes. In this session, we'll look at easy methods for identifying new indexes, we'll review patterns for index consolidation, and discuss how and when to remove indexes. At the end, you'll have a strategy that helps design indexes to improve performance.
Stored Procedure Superpowers: A Developer’s Guide
Stored procedures often get a bad rap, but this webinar argues that many of the things people hate about stored procedures are more about the implementation, rather than the concept. Watch this webinar presented by John Hugg, Founding Engineer, VoltDB to understand how stored procedure “superpowers” can make your application faster, simpler and safer.
A2B Data™ Brochure
A healthcare data warehouse manager discussed how their organization uses A2B Data software to process updates across over 3,100 tables in development, QA, and production environments daily. They also quickly converted hundreds of ETL processes to A2B Data in minutes. The manager stated that for data integration, A2B Data is their first choice. The document then described how A2B Data eliminates the need for developers and time typically required for data staging by extracting and loading source data quickly and efficiently.
Model Monitoring at Scale with Apache Spark and Verta
For any organization whose core product or business depends on ML models (think Slack search, Twitter feed ranking, or Tesla Autopilot), ensuring that production ML models are performing with high efficacy is crucial. In fact, according to the McKinsey report on model risk, defective models have led to revenue losses of hundreds of millions of dollars in the financial sector alone. However, in spite of the significant harms of defective models, tools to detect and remedy model performance issues for production ML models are missing.
Based on our experience building ML debugging and robustness tools at MIT CSAIL and managing large-scale model inference services at Twitter, Nvidia, and now at Verta, we developed a generalized model monitoring framework that can monitor a wide variety of ML models, work unchanged in batch and real-time inference scenarios, and scale to millions of inference requests. In this talk, we focus on how this framework applies to monitoring ML inference workflows built on top of Apache Spark and Databricks. We describe how we can supplement the massively scalable data processing capabilities of these platforms with statistical processors to support the monitoring and debugging of ML models.
Learn how ML Monitoring is fundamentally different from application performance monitoring or data monitoring. Understand what model monitoring must achieve for batch and real-time model serving use cases. Then dig in with us as we focus on the batch prediction use case for model scoring and demonstrate how we can leverage the core Apache Spark engine to easily monitor model performance and identify errors in serving pipelines.
2016 09 measurecamp - event data modeling
Presentation by Christophe Bogaert to Measurecamp London September 2016. Christophe discussed what makes consuming and analysing event-streams difficult, and outlined a number of techniques for overcoming those obstacles.
Embedding Data & Analytics With Looker
Delivering data and analytics to your customers should be straightforward. The Looker Data Platform allows for easy access to data through a robust API and embeddable charts, tables and dashboards.
Learn how Looker can help you:
- Embed charts, tables and dashboards into applications
- Use the Looker API to deliver data to applications, including Slack
- Build a customer portal to deliver more value to your customers
- Design the above with version control through Git, along with clustering and multi-server setup as needed
FlorenceAI: Reinventing Data Science at Humana
Humana strives to help the communities we serve and our individual members achieve their best health – no small task in the past year! We had the opportunity to rethink our existing operations and reimagine what a collaborative ML platform for hundreds of data scientists might look like. The primary goal of our ML Platform, named FlorenceAI, is to automate and accelerate the delivery lifecycle of data science solutions at scale. In this presentation, we will walk through an end-to-end example of how to build a model at scale on FlorenceAI and deploy it to production. Tools highlighted include Azure Databricks, MLFlow, AppInsights, and Azure Data Factory.
We will employ slides, notebooks and code snippets covering problem framing and design, initial feature selection, model design and experimentation, and a framework of centralized production code to streamline implementation. Hundreds of data scientists now use our feature store that has tens of thousands of features refreshed in daily and monthly cadences across several years of historical data. We already have dozens of models in production and also daily provide fresh insights for our Enterprise Clinical Operating Model. Each day, billions of rows of data are generated to give us timely information.
We already have examples of teams operating orders of magnitude faster and at a scale not within reach using fixed on-premise resources. Given rapid adoption from a dozen pilot users to over 100 MAU in the first 5 months, we will also share some anecodotes about key early wins created by the platform. We want FlorenceAI to enable Humana’s data scientists to focus their efforts where they add the most value so we can continue to deliver high-quality solutions that remain fresh, relevant and fair in an ever changing world.
Tableau API
The document discusses the Tableau API and why a developer may want to use it. The Tableau API includes the Document API, Extract API, JS API, REST API/TSC library, and Web Data Connector. It allows developers to extend Tableau's functionality, automate tasks, customize visualizations, and integrate visualizations into other applications. The APIs provide programmatic access to create and modify Tableau reports, create extracts, embed visualizations, and manage Tableau Server.
A taste of Snowplow Analytics data
An introduction to Snowplow Analytics and the problems it solves for analysts - presented to Web Analytics Wednesday Melbourne. 15th May 2014
Beyond Data Discovery: The Value Unlocked by Modern Data Modeling
In this webinar we will discuss Looker’s novel approach to data modeling and how it powers a data exploration environment with unprecedented depth and agility.
Some topics we will cover:
-A new architecture beyond direct connect
-Language-based, git-integrated data modeling
-Abstractions that make SQL more powerful and more efficient
Democratizing Data Quality Through a Centralized Platform
Bad data leads to bad decisions and broken customer experiences. Organizations depend on complete and accurate data to power their business, maintain efficiency, and uphold customer trust. With thousands of datasets and pipelines running, how do we ensure that all data meets quality standards, and that expectations are clear between producers and consumers? Investing in shared, flexible components and practices for monitoring data health is crucial for a complex data organization to rapidly and effectively scale.
At Zillow, we built a centralized platform to meet our data quality needs across stakeholders. The platform is accessible to engineers, scientists, and analysts, and seamlessly integrates with existing data pipelines and data discovery tools. In this presentation, we will provide an overview of our platform’s capabilities, including:
Giving producers and consumers the ability to define and view data quality expectations using a self-service onboarding portal
Performing data quality validations using libraries built to work with spark
Dynamically generating pipelines that can be abstracted away from users
Flagging data that doesn’t meet quality standards at the earliest stage and giving producers the opportunity to resolve issues before use by downstream consumers
Exposing data quality metrics alongside each dataset to provide producers and consumers with a comprehensive picture of health over time
Portable Scalable Data Visualization Techniques for Apache Spark and Python N...
Python Notebooks are great for communicating data analysis & research but how do you port these data visualizations between the many available platforms (Jupyter, Databricks, Zeppelin, Colab,…). Also learn about how to scale up your visualizations using Spark
When and Where to Embed Business Intelligence
Watch the recorded webinar at http://bit.ly/1MeX7QK
Everywhere you look, companies are using external-facing analytics to maximize the value derived from their data assets, by moving customers up the value chain, increasing stickiness, and offering a more competitive product on the marketplace.
Listen to learn about how to bring an external-facing data product to market by embedding BI software, and what that can add to your offering.
Presentation covers:
-Top uses cases for embedding business intelligence software
Case studies from different companies currently embedding BI
-Build vs buy considerations
-Evaluating ROI
Tableau Customer Presentation
This document discusses extending Splunk data access through ODBC connectivity with Tableau. It provides an overview of Tableau software for visualizing and exploring data, and how Tableau can now connect to Splunk data sources through ODBC. The presentation demonstrates how Splunk data can be analyzed in Tableau dashboards and visualizations, highlighting use cases across different teams at Tableau including security, operations, development, marketing and executives.
Data modeling trends for Analytics
Do we still need a star schema? Should we have a huge data warehouse? This session answers those questions.
Understanding event data
This document discusses event data and the Snowplow data pipeline. It notes that 3 years ago, analyzing user behavior and engagement using tools like Google Analytics was difficult. The Snowplow data pipeline was created to collect and analyze event-level data at scale using open source big data technologies. The pipeline has expanded to encompass different types of digital event data by developing a schema for structured JSON events and a versioning system. A real-time version of the pipeline is also being built to feed event data into applications in addition to batch processing. Developing a semantic model and standard framework for describing events is discussed as being important for enabling downstream applications to consume structured event data.
Geek Sync | SQL Server Indexing Basics
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/fKTi50A5q2l
We’d like to thank you for signing up to view the latest Geek Sync, SQL Server Indexing Basics, with Mindy Curnutt. Mindy focused on the foundational components of strategic SQL Server index design and creation. Whether you missed the live webinar or would like to view it again, it is now available on our site.
Writing optimal queries
Writing optimal queries requires considering things like using functions properly, avoiding data type mismatches, ensuring join predicates and statistics are defined, and addressing issues like clustered GUIDs. Developers should also avoid anti-patterns like using temp tables unnecessarily, putting filters in HAVING that could go in WHERE, and handling multiple aggregates carefully. The document provides tips for writing efficient queries from Sourabh Agarwal, a Microsoft premier field engineer who blogs and is active on Facebook and Twitter regarding SQL Server topics.
More Related Content
What's hot
How to Build a Data-Driven Company: From Infrastructure to Insights
Companies like Buffer, SeatGeek, and Asana aren’t just talking about the value of data, they’re building data infrastructure that can actually deliver it. Join this 45-minute webinar to learn why these companies are investing in data and what you need to know to keep up.
Building the Artificially Intelligent Enterprise
Mike Ferguson is Managing Director of Intelligent Business Strategies Limited and specializes in business intelligence/analytics and data management. He discusses building the artificially intelligent enterprise and transitioning to a self-learning enterprise. Some key challenges discussed include the siloed and fractured nature of current data and analytics efforts, with many tools and scripts in use without integration. He advocates sorting out the data foundation, implementing DataOps and MLOps, creating a data and analytics marketplace, and integrating analytics into business processes to drive value from AI.
Code Once Use Often with Declarative Data Pipelines
The document discusses using declarative data pipelines to code data workflows once and reuse them easily. It describes Flashfood, a company dealing with food waste data. The problem of maintaining many pipelines across different file types and clouds is presented. Three attempts at a solution showed that too little automation led to boilerplate code while too much automation caused unexpected behavior. The solution was to define YAML configuration files that jobs could be run against, allowing flexibility while enforcing DRY principles. This approach reduced maintenance overhead and allowed anyone to create similar jobs. Lessons included favoring parameters over inference and reusing extract and load code. Future work may involve programmatically adding new configurations and a Spark YAML grammar.
Brokering Data: Accelerating Data Evaluation with Databricks White Label
As the data-as-a-service ecosystem continues to evolve, data brokers are faced with an unprecedented challenge – demonstrating the value of their data. Successfully crafting and selling a compelling data product relies on a broker’s ability to differentiate their product from the rest of the market. In smaller or static datasets, measures like row count and cardinality can speak volumes. However, when datasets are in the terabytes or petabytes though – differentiation becomes much difficult. On top of that “data quality” is a somewhat ill-defined term and the definition of a “high quality dataset” can change daily or even hourly.
This breakout session will describe Veraset’s partnership with Databricks, and how we have white labeled Databricks to showcase and accelerate the value of our data. We’ll discuss the challenges that data brokers have faced to date and some of the primitives of our businesses that have guided our direction thus far. We will also actively demo our white label instance and notebook to show how we’ve been able to provide key insights to our customers and reduce the TTFB of data onboarding.
Strategies for SQL Server Index Analysis
Properly managing indexes for a database is a common use in many environments. When reviewing an index, should it have a single or multiple columns? Should the table be a heap, a clustered index or clustered columnstore index? These considerations are often at the forefront when analyzing your indexes. In this session, we'll look at easy methods for identifying new indexes, we'll review patterns for index consolidation, and discuss how and when to remove indexes. At the end, you'll have a strategy that helps design indexes to improve performance.
Stored Procedure Superpowers: A Developer’s Guide
Stored procedures often get a bad rap, but this webinar argues that many of the things people hate about stored procedures are more about the implementation, rather than the concept. Watch this webinar presented by John Hugg, Founding Engineer, VoltDB to understand how stored procedure “superpowers” can make your application faster, simpler and safer.
A2B Data™ Brochure
A healthcare data warehouse manager discussed how their organization uses A2B Data software to process updates across over 3,100 tables in development, QA, and production environments daily. They also quickly converted hundreds of ETL processes to A2B Data in minutes. The manager stated that for data integration, A2B Data is their first choice. The document then described how A2B Data eliminates the need for developers and time typically required for data staging by extracting and loading source data quickly and efficiently.
Model Monitoring at Scale with Apache Spark and Verta
For any organization whose core product or business depends on ML models (think Slack search, Twitter feed ranking, or Tesla Autopilot), ensuring that production ML models are performing with high efficacy is crucial. In fact, according to the McKinsey report on model risk, defective models have led to revenue losses of hundreds of millions of dollars in the financial sector alone. However, in spite of the significant harms of defective models, tools to detect and remedy model performance issues for production ML models are missing.
Based on our experience building ML debugging and robustness tools at MIT CSAIL and managing large-scale model inference services at Twitter, Nvidia, and now at Verta, we developed a generalized model monitoring framework that can monitor a wide variety of ML models, work unchanged in batch and real-time inference scenarios, and scale to millions of inference requests. In this talk, we focus on how this framework applies to monitoring ML inference workflows built on top of Apache Spark and Databricks. We describe how we can supplement the massively scalable data processing capabilities of these platforms with statistical processors to support the monitoring and debugging of ML models.
Learn how ML Monitoring is fundamentally different from application performance monitoring or data monitoring. Understand what model monitoring must achieve for batch and real-time model serving use cases. Then dig in with us as we focus on the batch prediction use case for model scoring and demonstrate how we can leverage the core Apache Spark engine to easily monitor model performance and identify errors in serving pipelines.
2016 09 measurecamp - event data modeling
Presentation by Christophe Bogaert to Measurecamp London September 2016. Christophe discussed what makes consuming and analysing event-streams difficult, and outlined a number of techniques for overcoming those obstacles.
Embedding Data & Analytics With Looker
Delivering data and analytics to your customers should be straightforward. The Looker Data Platform allows for easy access to data through a robust API and embeddable charts, tables and dashboards.
Learn how Looker can help you:
- Embed charts, tables and dashboards into applications
- Use the Looker API to deliver data to applications, including Slack
- Build a customer portal to deliver more value to your customers
- Design the above with version control through Git, along with clustering and multi-server setup as needed
FlorenceAI: Reinventing Data Science at Humana
Humana strives to help the communities we serve and our individual members achieve their best health – no small task in the past year! We had the opportunity to rethink our existing operations and reimagine what a collaborative ML platform for hundreds of data scientists might look like. The primary goal of our ML Platform, named FlorenceAI, is to automate and accelerate the delivery lifecycle of data science solutions at scale. In this presentation, we will walk through an end-to-end example of how to build a model at scale on FlorenceAI and deploy it to production. Tools highlighted include Azure Databricks, MLFlow, AppInsights, and Azure Data Factory.
We will employ slides, notebooks and code snippets covering problem framing and design, initial feature selection, model design and experimentation, and a framework of centralized production code to streamline implementation. Hundreds of data scientists now use our feature store that has tens of thousands of features refreshed in daily and monthly cadences across several years of historical data. We already have dozens of models in production and also daily provide fresh insights for our Enterprise Clinical Operating Model. Each day, billions of rows of data are generated to give us timely information.
We already have examples of teams operating orders of magnitude faster and at a scale not within reach using fixed on-premise resources. Given rapid adoption from a dozen pilot users to over 100 MAU in the first 5 months, we will also share some anecodotes about key early wins created by the platform. We want FlorenceAI to enable Humana’s data scientists to focus their efforts where they add the most value so we can continue to deliver high-quality solutions that remain fresh, relevant and fair in an ever changing world.
Tableau API
The document discusses the Tableau API and why a developer may want to use it. The Tableau API includes the Document API, Extract API, JS API, REST API/TSC library, and Web Data Connector. It allows developers to extend Tableau's functionality, automate tasks, customize visualizations, and integrate visualizations into other applications. The APIs provide programmatic access to create and modify Tableau reports, create extracts, embed visualizations, and manage Tableau Server.
A taste of Snowplow Analytics data
An introduction to Snowplow Analytics and the problems it solves for analysts - presented to Web Analytics Wednesday Melbourne. 15th May 2014
Beyond Data Discovery: The Value Unlocked by Modern Data Modeling
In this webinar we will discuss Looker’s novel approach to data modeling and how it powers a data exploration environment with unprecedented depth and agility.
Some topics we will cover:
-A new architecture beyond direct connect
-Language-based, git-integrated data modeling
-Abstractions that make SQL more powerful and more efficient
Democratizing Data Quality Through a Centralized Platform
Bad data leads to bad decisions and broken customer experiences. Organizations depend on complete and accurate data to power their business, maintain efficiency, and uphold customer trust. With thousands of datasets and pipelines running, how do we ensure that all data meets quality standards, and that expectations are clear between producers and consumers? Investing in shared, flexible components and practices for monitoring data health is crucial for a complex data organization to rapidly and effectively scale.
At Zillow, we built a centralized platform to meet our data quality needs across stakeholders. The platform is accessible to engineers, scientists, and analysts, and seamlessly integrates with existing data pipelines and data discovery tools. In this presentation, we will provide an overview of our platform’s capabilities, including:
Giving producers and consumers the ability to define and view data quality expectations using a self-service onboarding portal
Performing data quality validations using libraries built to work with spark
Dynamically generating pipelines that can be abstracted away from users
Flagging data that doesn’t meet quality standards at the earliest stage and giving producers the opportunity to resolve issues before use by downstream consumers
Exposing data quality metrics alongside each dataset to provide producers and consumers with a comprehensive picture of health over time
Portable Scalable Data Visualization Techniques for Apache Spark and Python N...
Python Notebooks are great for communicating data analysis & research but how do you port these data visualizations between the many available platforms (Jupyter, Databricks, Zeppelin, Colab,…). Also learn about how to scale up your visualizations using Spark
When and Where to Embed Business Intelligence
Watch the recorded webinar at http://bit.ly/1MeX7QK
Everywhere you look, companies are using external-facing analytics to maximize the value derived from their data assets, by moving customers up the value chain, increasing stickiness, and offering a more competitive product on the marketplace.
Listen to learn about how to bring an external-facing data product to market by embedding BI software, and what that can add to your offering.
Presentation covers:
-Top uses cases for embedding business intelligence software
Case studies from different companies currently embedding BI
-Build vs buy considerations
-Evaluating ROI
Tableau Customer Presentation
This document discusses extending Splunk data access through ODBC connectivity with Tableau. It provides an overview of Tableau software for visualizing and exploring data, and how Tableau can now connect to Splunk data sources through ODBC. The presentation demonstrates how Splunk data can be analyzed in Tableau dashboards and visualizations, highlighting use cases across different teams at Tableau including security, operations, development, marketing and executives.
Data modeling trends for Analytics
Do we still need a star schema? Should we have a huge data warehouse? This session answers those questions.
Understanding event data
This document discusses event data and the Snowplow data pipeline. It notes that 3 years ago, analyzing user behavior and engagement using tools like Google Analytics was difficult. The Snowplow data pipeline was created to collect and analyze event-level data at scale using open source big data technologies. The pipeline has expanded to encompass different types of digital event data by developing a schema for structured JSON events and a versioning system. A real-time version of the pipeline is also being built to feed event data into applications in addition to batch processing. Developing a semantic model and standard framework for describing events is discussed as being important for enabling downstream applications to consume structured event data.
What's hot (20)
How to Build a Data-Driven Company: From Infrastructure to Insights
How to Build a Data-Driven Company: From Infrastructure to Insights
Code Once Use Often with Declarative Data Pipelines
Code Once Use Often with Declarative Data Pipelines
Brokering Data: Accelerating Data Evaluation with Databricks White Label
Brokering Data: Accelerating Data Evaluation with Databricks White Label
Model Monitoring at Scale with Apache Spark and Verta
Model Monitoring at Scale with Apache Spark and Verta
Beyond Data Discovery: The Value Unlocked by Modern Data Modeling
Beyond Data Discovery: The Value Unlocked by Modern Data Modeling
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
Portable Scalable Data Visualization Techniques for Apache Spark and Python N...
Portable Scalable Data Visualization Techniques for Apache Spark and Python N...
Viewers also liked
Geek Sync | SQL Server Indexing Basics
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/fKTi50A5q2l
We’d like to thank you for signing up to view the latest Geek Sync, SQL Server Indexing Basics, with Mindy Curnutt. Mindy focused on the foundational components of strategic SQL Server index design and creation. Whether you missed the live webinar or would like to view it again, it is now available on our site.
Writing optimal queries
Writing optimal queries requires considering things like using functions properly, avoiding data type mismatches, ensuring join predicates and statistics are defined, and addressing issues like clustered GUIDs. Developers should also avoid anti-patterns like using temp tables unnecessarily, putting filters in HAVING that could go in WHERE, and handling multiple aggregates carefully. The document provides tips for writing efficient queries from Sourabh Agarwal, a Microsoft premier field engineer who blogs and is active on Facebook and Twitter regarding SQL Server topics.
Slideshare Directions
This document provides instructions for uploading a PowerPoint presentation to Slideshare and embedding the presentation on a Wiki page. It outlines the steps to log in to Slideshare, upload a PowerPoint file, fill in metadata fields, generate an embed code, and copy the code to embed the presentation on a Wiki page. The document also provides username and password credentials for two homeroom classes that can be used to log in.
Giving directions
This document provides guidance on using prepositions of place and movement when giving directions. It outlines the proper uses of "in", "at", "to", and "on" with examples. It also gives phrases for providing directions, such as "take the first road on the left" or "turn right at the end." Landmarks that can be used as points of reference like roundabouts and junctions are also listed.
Giving directions
The document provides examples of how to use prepositions like "on", "next to", "across from", "between", and "on the corner of" to give directions and describe locations. It gives sample sentences like "The school is on Fifth Street" and "The bank is across from the post office." It then asks questions to prompt using these prepositions, such as "Where is the post office?" and provides possible answers using the prepositions, like "It is on Main Street. It's across from the bank." The document teaches how to use these common prepositions to effectively give and understand directions.
Introduction of sql server indexing
SQL Server supports two main types of indexes - clustered and nonclustered. A clustered index physically orders the data on disk based on the index key. Only one clustered index is allowed per table. A nonclustered index contains key values and row locators but does not determine the physical order of data. SQL Server supports up to 999 nonclustered indexes per table. The choice of index depends on the query patterns against the table and the desired performance characteristics.
Database index(sql server)
This slide is introduced What is index in database system .
It's very simple to let you understand what is index pros and cons.
there has chinese voice version youtube :)
https://www.youtube.com/watch?v=BOBZAMfQfrQ&feature=youtu.be
https://www.youtube.com/watch?v=hkl5CcmZ4OI&feature=youtu.be
Direction and Bearings
There are eight cardinal directions that can be expanded to include intermediate directions like southeast and northwest. Occasionally sixteen directions are used, or bearings measured in degrees from 0 to 360 can specify an exact direction. Magnetic compasses and protractors are tools used to determine directions and calculate bearings, although GPS is more commonly used today.
FOLLOWING DIRECTIONS
This blog is aimed at 5th grade students to reinforce class topics through independent learning. It provides resources like powerpoint presentations on grammar and prepositions, videos on address vocabulary and reading maps, and games like crosswords and soup letters to practice the material. Quizzes are also included to test comprehension.
Asking for and giving Directions
The PowerPoint presetation can be used to teach asking for and giving directions in classrooms where English is studied as a foreing or a second language. Depending on the learners' needs, an EFL or ESL teacher can change the contents to suit his or her leaners who specifically want to improve oral fluency in the target language
Bearings lesson
Bearings are used to describe direction and position more accurately than compass points alone. They are always three figures measured clockwise from North. To measure the bearing from point A to point B, draw a line from A to B and measure the angle clockwise from the North line at A. To measure the bearing from B to A, do the same but measure the exterior angle clockwise from the North line at B, and add 180 degrees if the measurement is less than 180. Practice examples are provided to help understand how to measure bearings.
SQL Server 2016 SSRS and BI
This document discusses the new features in SQL Server 2016 related to business intelligence (BI). Key highlights include:
- Power BI integration, allowing paginated report items to be pinned to Power BI dashboards.
- Enhancements to SQL Server Reporting Services, including modern paginated reports with updated tools, mobile reports optimized for mobile devices, and a new web portal to consume both report types.
- The ability to export paginated reports to PowerPoint, pin report items to Power BI dashboards, and create interactive mobile reports accessed through a single mobile app.
Giving directions
To get to a location, one can ask "Excuse me, where is the...?" or "How can I get to...?" and will likely receive directions involving taking the first or second street on the left or right. The document provides examples of asking for and giving directions using basic vocabulary.
Asking and giving directions
This document provides expressions for asking for and giving directions. To ask for directions, one can say "How can I get to...", "Where is the...", or "How far is the..." and be sure to use polite expressions like "excuse me" or "Can you tell me...". When giving directions, one can say "Go to the corner of...", "Turn left", "Turn right", "Pass the...", or "Go straight ahead on..." followed by transitions like "After that", "Then", or "When you get to..." to connect ideas clearly. An example uses these expressions to give directions from a current location to a hospital.
Asking and giving directions
The document contains a series of directions for navigating a map with various landmarks such as a theatre, cinema, school, park, bank, church, hotel, museum, restaurant, hospital, post office, cafes, and streets. At each step, the directions provide instructions to move between the landmarks and streets, with occasional questions about the current location. The overall document plays out like a game with the goal of navigating from one location to another by following the text-based directions.
Asking and giving directions by ajarn wheng
This document provides guidance on asking for and giving directions. It includes sample dialogues and key language for directions involving turns (left, right), landmarks, transportation (bus stops), and locations (in front of, near). Directions should use landmarks and a "go to, then" structure (e.g. go to Main St, then turn left). Clarification may be needed when asking or giving directions.
Asking & giving directions
The document provides guidance on asking for and giving directions, including common expressions to use when asking like "How can I get to..." and when giving directions like "Turn left" and "Go straight ahead". It also gives an example of how to give directions from one location to a hospital, including specific places to pass along the way. It recommends checking additional webpages for more practice with directions.
Asking and giving direction
The document provides guidance on asking for and giving directions. It lists common phrases used when asking for directions such as "Excuse me, where is..." and "Could you tell me how to get to...". When giving directions, it recommends using prepositions of place and phrases like "go along this street until..." and "take the first/second turning". It then provides a sample conversation where one person asks for directions to a gas station and the other person provides clear directions using the recommended language and structure. The document concludes with an assignment asking students to practice asking for and giving directions between different locations on a sample map.
Asking giving directions
This document provides examples of how to ask for and give directions. When asking for directions, common questions include "How do I get to..." or "Where is..." When giving directions, prepositions like "next to" and "behind" can be used to describe locations. Expressions like "go straight", "turn right/left", and "take [road name]" are also used. Estimating distances and travel times helps provide additional context.
Giving directions presentation_oseiasa_modelo
The document provides instructions for giving directions and using prepositions of place. It includes examples of directional terms like left, straight ahead, right, as well as prepositions like between, next to, across from. It then has practice exercises matching objects with their locations using prepositions. There are also sample dialogues where one person asks for directions and the other provides them. At the end is an invitation with directions to a party that asks the reader to identify the location being described.
Viewers also liked (20)
Similar to 5 SQL Server Indexing Myths
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
This document discusses 7 common database mistakes and how to avoid them. It begins by emphasizing the importance of proper backups and being able to restore data. It stresses having documentation and training others on restoration processes. The document also recommends keeping software updated for security reasons. It advises monitoring databases to understand performance and ensure uptime. Other mistakes covered include having inconsistent user permissions, not understanding indexing best practices, and not optimizing queries. The document concludes by promoting the benefits of using JSON columns in databases.
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
As a developer using PostgreSQL one of the most important tasks you have to deal with is modeling the database schema for your application. In order to achieve a solid design, it’s important to understand how the schema is then going to be used as well as the trade-offs it involves.
As Fred Brooks said: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.”
In this talk we're going to see practical normalisation examples and their benefits, and also review some anti-patterns and their typical PostgreSQL solutions, including Denormalization techniques thanks to advanced Data Types.
ADV Slides: 2021 Trends in Enterprise Analytics
The document discusses trends in enterprise advanced analytics for 2021 and beyond. Some key trends include remote work continuing, strong tech spending rebound led by cloud capabilities, leading organizations increasing focus on AI/ML with model deployment taking center stage, more edge AI, rise of data lakes, new technology stacks focusing on data fabrics and AI pipelines, increased automation, open source becoming more prevalent, Kubernetes becoming the standard analytics stack, and general AI beginning to emerge. Winning approaches for 2021 include cloud, AI, data lakes, data warehousing, MDM, agile development, Kubernetes, automation, data quality, and DevOps/MLOps.
Don't Drop ACID (July 2021)
NoSQL document databases provide unique capabilities of scaling, flexibility, and performance for a wide variety of use cases. However, many developers from relational backgrounds are understandably nervous (for a variety of reasons) about using NoSQL in their next project. This session will address one of those reasons: ACID transactions (or lack thereof). This session will start with some background about why NoSQL databases didn’t (initially) have full ACID capabilities. Next, we’ll look at why lack of ACID may not be a big deal and some of the data modeling and querying techniques to use instead. Finally, we’ll look at the more recent trend of document databases adding distributed multi-document ACID capabilities and show a live demo of a NoSQL transaction. You’ll leave this session with a better understanding of how ACID works and when to use it.
Old code doesn't stink - Detroit
I've seen projects with shiny, new code render into unmaintainable big balls of mud within 2-3 years. Multiple times. But regardless of whether it's the code base as a whole that's rotten, or whether it's just the UI and User Experience that needs a major overhaul: the question on rewrite vs refactoring will come up sooner or later. Based on years of experience, and a plethora of bad decisions cumulating into epic failures, I'll share my experience on how to have a code base that stays maintainable - even after years. After this talk, you'll have more insight into whether you should refactor or rewrite, and how to do it right from now on.
Database Refactoring With Liquibase
This document discusses agile database development. It promotes refactoring databases using evolutionary modeling and automated testing to allow for continuous integration of database changes. Sandboxes are used to test changes before migrating them to production. Changes are tracked using tools like Liquibase that version database schemas and manage changesets and migrations. This helps break the "curse" of databases that are difficult to change by allowing agile practices to be applied to database development.
Agile Database Development with Liquibase
Learn to apply well-understood agile development practices to the database, using the open-source Liquibase tool to refactor your database schema in a controlled, incremental fashion.
Big Data - Big Pitfalls.
Big Data technologies promise lot of advantages both from business and engineering point of view. But nothing comes for free. 2 years on this path led to lot of pitfalls discovered in term of infrastructure approaches, solutions complexity balance and technology stack selection approaches. Now we intend to warn you and have some fun.
Java/Scala Lab: Роман Никитченко - Big Data - Big Pitfalls.
Big Data technologies promise lot of advantages both from business and engineering point of view.
But nothing comes for free. 2 years on this path led to lot of pitfalls discovered in term of infrastructure approaches, solutions complexity balance and technology stack approaches.
Now we intend to warn you and have some fun.
Alchemy of Data Elements - Top Down Meets Bottom Up
This document discusses data elements, which are the basic building blocks of information systems such as fields, attributes and cells. It notes that while data elements are important, their definitions and handling are often removed from business understanding and left to low-level tasks. The document advocates mapping different labels for the same data element, such as social security number and SSN. It provides references for further information on data dictionaries and semantically representing data elements.
Performance By Design
This document discusses principles for designing systems and databases for high performance. It emphasizes that:
1. Performance requirements should be specified upfront and the architecture should support meeting those requirements.
2. The database design impacts performance, so logical and physical designs like normalization, indexing, partitioning, and caching should be considered.
3. Application architecture also affects performance through techniques like minimizing network traffic and SQL parsing overhead.
Top 100 .NET Interview Questions and Answers
Top 100 câu hỏi phỏng vấn về .NET Framework dành cho lập trình viên. Top 100 .NET Interview Questions and Answers
Hybrid Transactional/Analytics Processing with Spark and IMDGs
This document discusses hybrid transactional/analytical processing (HTAP) with Apache Spark and in-memory data grids. It begins by introducing the speaker and GigaSpaces. It then discusses how modern applications require both online transaction processing and real-time operational intelligence. The document presents examples from retail and IoT and the goals of minimizing latency while maximizing data analytics locality. It provides an overview of in-memory computing options and describes how GigaSpaces uses an in-memory data grid combined with Spark to achieve HTAP. The document includes deployment diagrams and discusses data grid RDDs and pushing predicates to the data grid. It describes how this was productized as InsightEdge and provides additional innovations and reference architectures.
Discovering the plan cache (sql sat175)
This document discusses exploring the SQL Server plan cache. It begins with an overview of the plan cache and how it stores execution plans. It then covers various methods for viewing the plan cache, including using dynamic management objects. The document demonstrates how to interpret plan cache contents, including the showplan XML structure. It shows scenarios and demos related to missing indexes, parameter sniffing, physical operators, and key lookups. The document emphasizes that the plan cache contains valuable performance information and techniques for accessing it.
Kill mysql-performance
If you need to move some serious data or deal with massive scale, you need to think about the ecosystem in which MySQL lives.
5 Amazing Reasons DBAs Need to Love Extended Events
Extended events provide DBAs with a powerful tool that can be used to troubleshoot and investigate SQL Server. Throughout this session, you’ll walk through five great reasons, with demos. By the end of the webcast, you’ll be itching to grab the scripts from the demos to start building your own extended event sessions today.
Big Data: fall seven times, stand up eight!
More than year of extremely intensive Big Data technologies development with Hadoop, HBase, MapReduce and ZooKeeper as key technologies. New company that has established infrastructure which grows pretty fast. Lot of experience in networking and distributed systems but completely new enterprise solutions world. What tasks does this bring? What issue and traps? What lessons were learned and what is considered as near future tasks? How embedded developer can enter this new world and what advantages he or she has? What challenges should you be ready to face?
Documenting serverless architectures could we do it better - o'reily sa con...
The document discusses documenting serverless architectures. It introduces serverless architecture and some of its benefits and challenges, including the lack of clear guidelines around choosing different serverless computing options. It proposes using several views - use case view, logical view, process view, implementation view, and deployment view - based on the 4+1 architectural view model to document serverless architectures. Examples of using sequence diagrams and collaboration diagrams for the logical view and process view are provided to illustrate how different views can capture various aspects of the system architecture.
Spark SQL Adaptive Execution Unleashes The Power of Cluster in Large Scale wi...
Spark SQL is a very effective distributed SQL engine for OLAP and widely adopted in Baidu production for many internal BI projects. However, Baidu has also been facing many challenges for large scale including tuning the shuffle parallelism for thousands of jobs, inefficient execution plan, and handling data skew. In this talk, we will explore Intel and Baidu’s joint efforts to address challenges in large scale and offer an overview of an adaptive execution mode we implemented for Baidu’s Big SQL platform which is based on Spark SQL. At runtime, adaptive execution can change the execution plan to use a better join strategy and handle skewed join automatically. It can also change the number of reducer to better fit the data scale. In general, adaptive execution decreases the effort involved in tuning SQL query parameters and improves the execution performance by choosing a better execution plan and parallelism at runtime.
We’ll also share our experience of using adaptive execution in Baidu’s production cluster with thousands of server, where adaptive execution helps to improve the performance of some complex queries by 200%. After further analysis we found that several special scenarios in Baidu data analysis can benefit from the optimization of choosing better join type. We got 2x performance improvement in the scenario where the user wanted to analysis 1000+ advertisers’ cost from both web and mobile side and each side has a full information table with 10 TB parquet file per-day. Now we are writing probe jobs to detect more scenarios from current daily jobs of our users. We are also considering to expose the strategy interface based on the detailed metrics collected form adaptive execution mode for the upper users.
Does it only have to be ML + AI?
The document discusses machine learning and artificial intelligence applications inside and outside of Snowflake's cloud data warehouse. It provides an overview of Snowflake and its architecture. It then discusses how machine learning can be implemented directly in the database using SQL, user-defined functions, and stored procedures. However, it notes that pure coding is not suitable for all users and that automated machine learning outside the database may be preferable to enable more business analysts and power users. It provides an example of using Amazon Forecast for time series forecasting and integrating it with Snowflake.
Similar to 5 SQL Server Indexing Myths (20)
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Java/Scala Lab: Роман Никитченко - Big Data - Big Pitfalls.
Java/Scala Lab: Роман Никитченко - Big Data - Big Pitfalls.
Alchemy of Data Elements - Top Down Meets Bottom Up
Alchemy of Data Elements - Top Down Meets Bottom Up
Hybrid Transactional/Analytics Processing with Spark and IMDGs
Hybrid Transactional/Analytics Processing with Spark and IMDGs
5 Amazing Reasons DBAs Need to Love Extended Events
5 Amazing Reasons DBAs Need to Love Extended Events
Documenting serverless architectures could we do it better - o'reily sa con...
Documenting serverless architectures could we do it better - o'reily sa con...
Spark SQL Adaptive Execution Unleashes The Power of Cluster in Large Scale wi...
Spark SQL Adaptive Execution Unleashes The Power of Cluster in Large Scale wi...
More from Jason Strate
Accelerating Business Intelligence Solutions with Microsoft Azure pass
Business Intelligence (BI) solutions need to move at the speed of business. Unfortunately, roadblocks related to availability of resources and deployment often present an issue. What if you could accelerate the deployment of an entire BI infrastructure to just a couple hours and start loading data into it by the end of the day. In this session, we'll demonstrate how to leverage Microsoft tools and the Azure cloud environment to build out a BI solution and begin providing analytics to your team with tools such as Power BI. By end of the session, you'll gain an understanding of the capabilities of Azure and how you can start building an end to end BI proof-of-concept today.
Introduction to Columnstore Indexes
Columnstore indexes were first introduced in SQL Server 2012. Columnstore indexes provide the ability to flip-over your indexes from column-based to row-based, leading to dramatic changes in the storage and performance of your data. In this session, we'll review the basic concepts of columnstore indexes and look at the changes coming to them in SQL Server 2014
The Flavors of Non-Clustered Indexes
A non-clustered index is just another way to get to data. Add in a couple columns and off you go. Not too much to think about, just fire and forget. But it’s not really that easy. Non-clustered indexes have a number of various ways in which they can be built and designed that can have significant impacts on performance. In this session we’ll look at what can go into a non-clustered index. We will explore what you can put into a non-clustered index and what you can expect back out from performance.
Necessary Evils, Building Optimized CRUD Procedures
Every developer loves them and a lot of DBAs hate them. But there are many and valid reasons for creating generic SELECT, INSERT, UPDATE, and DELETE procedures. In this session, we’ll go through designing CRUD procedures that utilize new and existing SQL features to create CRUD procedures that are optimized for performance.
How Do Non-Clustered Indexes Improve Performance?
We've all heard the mantra about the importance of clustered indexes, but what about non-clustered indexes? Do you really need non-clustered indexes or are they just something that consultants recommend to increase their billable hours? In this session, we'll investigate the various flavors of non-clustered indexes and how they can be used to provide significant performance improvements.
Choosing Your Clustered Index
We often hear that you should always have clustered indexes on your tables. What's left off of this discussion is the how to choose the columns for this index. Should it be the primary key? Or some other column or set of columns? Having the wrong key column(s) in the clustered index can lead to performance problems, possibly worse than having no clustered index on the table at all. In this session, we'll review common patterns for selecting clustered indexes and how to determine which pattern you need. Also, we'll look at how to analyze tables to help select the best clustered index for every table.
http://www.jasonstrate.com/go/20131010-IX/
BuildingSecurity Audits with Extended Events
The document discusses using Extended Events in SQL Server to perform security audits. It describes the components of Extended Events including events, actions, predicates, targets, and packages. It then provides examples of how to build audits to track logins and queries, demonstrating how to capture useful security information without invasive monitoring. Templates are shown for login and query audits that can be directed to file, ring buffer, or event stream targets.
5 ways to improve performance through indexing
The document discusses 5 ways to improve database performance through indexing: 1) addressing lookup imbalance by moving clustered indexes or adding included columns, 2) reducing scans on large indexes by adding alternative indexes, 3) using filtered indexes to improve performance on queries involving a subset of rows, 4) addressing missing index recommendations to optimize queries, and 5) indexing foreign keys to avoid performance issues during DML operations.
The Side Effect of NOLOCK
This document discusses the side effects of using NOLOCK hints in SQL queries. It notes that NOLOCK is not an acceptable way to fix locking issues and risks corrupting data. The document recommends researching SNAPSHOT ISOLATION as an alternative and provides contact information for Jason Strate, a database architect, who discusses locking and isolation levels on his website.
Introduction to SQL Server Security
This document discusses securing Microsoft SQL Server. It covers securing the SQL Server installation, controlling access to the server and databases, and validating security. Key points include using least privilege for service accounts, controlling access through logins, roles and permissions, auditing with SQL Server Audit and Policy Based Management, and services available from Pragmatic Works related to SQL Server security, training and products.
What are you waiting for? (#SQLSat211)
The document discusses wait statistics and wait types in SQL Server. It provides an overview of defining, viewing, collecting and reviewing wait data using dynamic management views and wait statistics. It also simplifies the SQL Server execution model using a grocery store checkout line analogy. Specific wait types are explained such as CXPACKET, OLEDB, PAGEIOLATCH_X, PAGELATCH_X and LATCH_X. Guidelines and potential resolutions are provided for analyzing and addressing top wait events.
Discovering the plan cache (#SQLSat211)
The document discusses discovering and analyzing the plan cache in SQL Server. It covers viewing the plan cache, the structure of showplan XML, and demonstrates scenarios and techniques for querying the plan cache including identifying missing indexes, parameter sniffing issues, and physical operator details. The goal is to help attendees leverage the plan cache to troubleshoot performance issues and tune queries.
Discovering the Plan Cache (#SQLSat 206)
The document discusses viewing and analyzing the plan cache in SQL Server. It covers viewing the plan cache, exploring the structure of showplan XML, and demonstrating methods to query the plan cache through demos. The goals are to discuss the plan cache, explore it, demonstrate querying methods, and demonstrate performance tuning concepts.
A Function by Any Other Name is a Function
This document discusses user defined functions in SQL Server and their performance impacts. It identifies purposes for creating user defined functions, discusses the different types including scalar, inline, and multi-statement functions. It demonstrates the performance impact of each type of function through examples and benchmarks. Scalar functions have linear performance effects while inline and multi-statement functions allow for more complex logic but can introduce duplication of access. In summary, functions provide benefits like consolidation and reuse but their performance impact depends on the type and how they are used.
What are you waiting for
This document discusses wait statistics and wait analysis in SQL Server. It defines waits as occurring when a process requests a resource that is unavailable. It provides an overview of wait types categorized as background, external, or queue waits. The document demonstrates how to view wait statistics using DMVs and third party tools. It provides translations and guidelines for some common wait types like CXPACKET, PAGEIOLATCH, and LATCH. It concludes with additional resources for learning about waits.
More from Jason Strate (15)
Accelerating Business Intelligence Solutions with Microsoft Azure pass
Accelerating Business Intelligence Solutions with Microsoft Azure pass
Necessary Evils, Building Optimized CRUD Procedures
Necessary Evils, Building Optimized CRUD Procedures
Recently uploaded
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Building RAG with self-deployed Milvus vector database and Snowpark Container...
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentationsHow to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Recently uploaded (20)
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
5 SQL Server Indexing Myths
- 1. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM 5 SQL Server Indexing Myths
- 2. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM2 Introduction Jason Strate e: jstrate@pragmaticworks.com e: jasonstrate@gmail.com b: www.jasonstrate.com t: StrateSQL Resources www.jasonstrate.com/go/indexing
- 3. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM3 Unicorns aren’t real!
- 4. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM4 I want to believe!
- 5. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM5 But husky rhinos are!
- 6. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM6 Primary Keys Are Always Clustered
- 7. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM7 Primary Keys Are Always Clustered Primary keys are clustered by default Common point of confusion Often the desired behavior Is the primary key a good clustered index? Static and stable Ever increasing
- 8. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Primary Keys Are Always Clustered Demo
- 9. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM9 Index Column Order Doesn’t Matter
- 10. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM10 Index Column Order Doesn’t Matter Using and seeking an index are not the same Read execution plans carefully Validate scans versus seeks Indexes are sorted left to right All columns are not required Scans aren’t necessarily bad
- 11. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM11 Index Column Order Doesn’t Matter Col1 Col2 A 1 A 2 A 3 B 1 B 2 B 3 C 1 C 2 C 3 A 1
- 12. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Index Column Order Doesn’t Matter Demo
- 13. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM13 Clustered Indexes Are Physically Ordered
- 14. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM14 Clustered Indexes Are Physically Ordered Clustered indexes sort data within the index Accurately find specific values A before B, B before C Physical sorted at the tree level Path data access directly to the required page Logically sorted at the page level Minimize page rewrites on DML
- 15. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Clustered Indexes Are Physically Ordered Demo
- 16. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM16 Fill Factor Affects Insert Operations
- 17. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM17 Fill Factor Affects Insert Operations Fill factor leaves space on data pages Why do we do this? Leave space for DML operations Mitigate page splits during DML operations Provides a buffer for data variability
- 18. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM18 Fill Factor Affects Insert Operations
- 19. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM19 Fill Factor Affects Insert Operations
- 20. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Fill Factor Affects Insert Operations Demo
- 21. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM21 Every Table Should Have a Heap/Clustered Index
- 22. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM22 Every Table Should Have a XXX Index Blanket statements suck Doesn’t account for all situations Or even most situations Not 50/50 Not 90/10 Is there a safe answer?
- 23. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM23 Heap / Clustered Indexes Heap • Non-selective access • Bulk Loaded tables • Fragmentation prone with high inserts • Staging tables Clustered Index • Highly modified data • Singleton access • Range scans • Key ordered results
- 24. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM 5 SQL Server Indexing Myths Questions?
- 25. Products Improve the quality, productivity, and performance of your SQL Server and BI solutions. Services Speed development through training and rapid development services from Pragmatic Works. Foundation Helping those who don’t have the means to get into information technology and to achieve their dreams. Name: Jason Strate Email: jstrate@pragmaticworks.com Blog: www.jasonstrate.com Resource: www.jasonstrate.com/go/indexing Need Help? jasonstrate.com/go/vmdba