This session is for DBAs and developers who are comfortable writing queries, but not so comfortable when it comes to explaining nonclustered indexes, lookups, sargability, fill factor, and corruption detection.
Presentation review best ways to accomplish database load testing and analysis of database performance. Presentation targeting major RDBMS:systems - Oracle and SQL Server as well as tolls necessary for database load testing, Oracle performance tuning, SQL Server performance tuning, Windows and Linux performance optimization
Once the ‘Backup Database’ command executed, SQL Server automatically does few ‘Checkpoint’ to reduce the recovery time and also it makes sure that at point of command execution there is no dirty pages in the buffer pool. After that SQL Server creates at least three workers as ‘Controller’, ‘Stream Reader’ and ‘Stream Writer’ to read and buffer the data asynchronously into the buffer area (Out of buffer pool) and write the buffers into the backup device.
Presentation review best ways to accomplish database load testing and analysis of database performance. Presentation targeting major RDBMS:systems - Oracle and SQL Server as well as tolls necessary for database load testing, Oracle performance tuning, SQL Server performance tuning, Windows and Linux performance optimization
Once the ‘Backup Database’ command executed, SQL Server automatically does few ‘Checkpoint’ to reduce the recovery time and also it makes sure that at point of command execution there is no dirty pages in the buffer pool. After that SQL Server creates at least three workers as ‘Controller’, ‘Stream Reader’ and ‘Stream Writer’ to read and buffer the data asynchronously into the buffer area (Out of buffer pool) and write the buffers into the backup device.
SQL Server 2016 New Features and EnhancementsJohn Martin
SQL Server 2016 new features session that I delivered at SQL Relay 2015 at; Reading, London, Cardiff and Birmingham.
Looking at some of the new features currently slated for inclusion in the next version of Microsoft SQL Server 2016.
Demo Code can be found at: http://1drv.ms/1PC5smY
SQL Server R Services: What Every SQL Professional Should KnowBob Ward
SQL Server 2016 introduces a new platform for building intelligent, advanced analytic applications called SQL Server R Services. This session is for the SQL Server Database professional to learn more about this technology and its impact on managing a SQL Server environment. We will cover the basics of this technology but also look at how it works, troubleshooting topics, and even usage case scenarios. You don't have to be a data scientist to understand SQL Server R Services but you need to know how this works so come upgrade you career by learning more about SQL Server and advanced analytics.
Introduction to SQL Server Internals: How to Think Like the EngineBrent Ozar
When you pass in a query, how does SQL Server build the results? Time to role play: Brent will be an end user sending in queries, and you will play the part of the SQL Server engine. Using simple spreadsheets as your tables, you will learn how SQL Server builds execution plans, uses indexes, performs joins, and considers statistics.
This session is for DBAs and developers who are comfortable writing queries, but not so comfortable when it comes to explaining nonclustered indexes, lookups, and sargability.
Hekaton is the original project name for In-Memory OLTP and just sounds cooler for a title name. Keeping up the tradition of deep technical “Inside” sessions at PASS, this half-day talk will take you behind the scenes and under the covers on how the In-Memory OLTP functionality works with SQL Server.
We will cover “everything Hekaton”, including how it is integrated with the SQL Server Engine Architecture. We will explore how data is stored in memory and on disk, how I/O works, how native complied procedures are built and executed. We will also look at how Hekaton integrates with the rest of the engine, including Backup, Restore, Recovery, High-Availability, Transaction Logging, and Troubleshooting.
Demos are a must for a half-day session like this and what would an inside session be if we didn’t bring out the Windows Debugger. As with previous “Inside…” talks I’ve presented at PASS, this session is level 500 and not for the faint of heart. So read through the docs on In-Memory OLTP and bring some extra pain reliever as we move fast and go deep.
This session will appear as two sessions in the program guide but is not a Part I and II. It is one complete session with a small break so you should plan to attend it all to get the maximum benefit.
Brk3288 sql server v.next with support on linux, windows and containers was...Bob Ward
SQL Server is bringing its world-class RDBMS to Linux and Windows with SQL Server v.Next. In this session you will learn what´s next for SQL Server on Linux and how application developers and IT architects can now leverage the enterprise class features of SQL Server in every edition on Linux, Windows and containers.
You’re a developer or a DBA, and you’re comfortable writing queries to get the data you need. You’re much less comfortable trying to design the right indexes for your database server. In this series of videos, you’ll learn how the SQL Server engine looks at your indexes and builds your query results.
SQL Server 2016 New Features and EnhancementsJohn Martin
SQL Server 2016 new features session that I delivered at SQL Relay 2015 at; Reading, London, Cardiff and Birmingham.
Looking at some of the new features currently slated for inclusion in the next version of Microsoft SQL Server 2016.
Demo Code can be found at: http://1drv.ms/1PC5smY
SQL Server R Services: What Every SQL Professional Should KnowBob Ward
SQL Server 2016 introduces a new platform for building intelligent, advanced analytic applications called SQL Server R Services. This session is for the SQL Server Database professional to learn more about this technology and its impact on managing a SQL Server environment. We will cover the basics of this technology but also look at how it works, troubleshooting topics, and even usage case scenarios. You don't have to be a data scientist to understand SQL Server R Services but you need to know how this works so come upgrade you career by learning more about SQL Server and advanced analytics.
Introduction to SQL Server Internals: How to Think Like the EngineBrent Ozar
When you pass in a query, how does SQL Server build the results? Time to role play: Brent will be an end user sending in queries, and you will play the part of the SQL Server engine. Using simple spreadsheets as your tables, you will learn how SQL Server builds execution plans, uses indexes, performs joins, and considers statistics.
This session is for DBAs and developers who are comfortable writing queries, but not so comfortable when it comes to explaining nonclustered indexes, lookups, and sargability.
Hekaton is the original project name for In-Memory OLTP and just sounds cooler for a title name. Keeping up the tradition of deep technical “Inside” sessions at PASS, this half-day talk will take you behind the scenes and under the covers on how the In-Memory OLTP functionality works with SQL Server.
We will cover “everything Hekaton”, including how it is integrated with the SQL Server Engine Architecture. We will explore how data is stored in memory and on disk, how I/O works, how native complied procedures are built and executed. We will also look at how Hekaton integrates with the rest of the engine, including Backup, Restore, Recovery, High-Availability, Transaction Logging, and Troubleshooting.
Demos are a must for a half-day session like this and what would an inside session be if we didn’t bring out the Windows Debugger. As with previous “Inside…” talks I’ve presented at PASS, this session is level 500 and not for the faint of heart. So read through the docs on In-Memory OLTP and bring some extra pain reliever as we move fast and go deep.
This session will appear as two sessions in the program guide but is not a Part I and II. It is one complete session with a small break so you should plan to attend it all to get the maximum benefit.
Brk3288 sql server v.next with support on linux, windows and containers was...Bob Ward
SQL Server is bringing its world-class RDBMS to Linux and Windows with SQL Server v.Next. In this session you will learn what´s next for SQL Server on Linux and how application developers and IT architects can now leverage the enterprise class features of SQL Server in every edition on Linux, Windows and containers.
You’re a developer or a DBA, and you’re comfortable writing queries to get the data you need. You’re much less comfortable trying to design the right indexes for your database server. In this series of videos, you’ll learn how the SQL Server engine looks at your indexes and builds your query results.
Dynamic SQL: How to Build Fast Multi-Parameter Stored ProceduresBrent Ozar
ou're comfortable writing T-SQL, and you've built a lot of stored procedures that have a bunch of parameters. For example, you have that "product search" stored proc with parameters for product category, name, price range, sort order, etc, and you have to accept any of 'em.
So how do we make those go fast? And how can we get 'em to use indexes?
In one all-demo hour, performance tuner Brent Ozar will show you several ways that fail in comically bad ways. You'll learn how to write dynamic SQL that's easy to tune, manage, and troubleshoot.
SQLSaturday is a training event for SQL Server professionals and those wanting to learn about SQL Server. This event will be held Jun 13 2015 at Hochschule Bonn-Rhein-Sieg, Grantham-Allee 20, St. Augustin, Rheinland, 53757, Germany. Admittance to this event is free, all costs are covered by donations and sponsorships. Please register soon as seating is limited, and let friends and colleagues know about the event.
###
Maintenance Plans for Beginners (but not only) | Each of experienced administrators used (to some extent) what is called Maintenance Plans - Plans of Conservation. During this session, I'd like to discuss what can be useful for us to provide functionality when we use them and what to look out for. Session at 200 times the forward-300, with the opening of the discussion.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
3. I know, I hate killing trees.
But having these next 3 pages in your hand will help a lot
as we talk through the demos.
Print this 3-page PDF to follow along:
http://u.BrentOzar.com/engine.pdf
3
4. Brent Ozar
Consultant, Brent Ozar Unlimited
I make SQL Server faster and more reliable.
I created sp_Blitz® and the SQL Server First
Responder Kit, and I loves sharing knowledge at
BrentOzar.com. I hold a bunch of certifications and
awards including the rare Microsoft Certified Master.
You don’t care about any of that though.
Download the PDF: BrentOzar.com/go/enginepdf
/brentozar @brento brentozar
5. Agenda
When you pass in a query, how does SQL Server build the results? Time
to role play: Brent will be an end user sending in queries, and you will
play the part of the SQL Server engine. Using simple spreadsheets as
your tables, you will learn how SQL Server builds execution plans,
uses indexes, performs joins, and considers statistics.
This session is for DBAs and developers who are comfortable writing
queries, but not so comfortable when it comes to explaining
nonclustered indexes, lookups, sargability, fill factor, and corruption
detection.
27. Let’s add a sort.
SELECT Id
FROM dbo.Users
WHERE LastAccessDate > ‘2014/07/01’
ORDER BY LastAccessDate
28. Your execution plan
1. Shuffle through all of the pages,
writing down fields __________ for each record,
if their LastAccessDate > ‘2014/07/01’.
2. Sort the matching records by LastAccessDate.
31. Cost is up ~4x
We needed space to
write down our results,
so we got a memory grant
Order By:
32.
33. Memory is set when the query starts,
and not revised.
SQL Server has to assume other people
will run queries at the same time as you.
Your memory grant can change with
each time that you run a query.
You can’t always get
what you want.
35. Let’s get all the fields.
SELECT *
FROM dbo.Users
WHERE LastAccessDate > ‘2014/07/01’
ORDER BY LastAccessDate
36. Your execution plan
1. Shuffle through all of the pages,
writing down fields __________ for each record,
if their LastAccessDate > ‘2014/07/01’.
2. Sort the matching records by LastAccessDate.
38. Why does it suck?
Do we work harder to read the data?
Do we work harder to write the data?
Do we work harder to sort the data?
Do we work harder to output the data?
43. Let’s run it a few times.
SELECT *
FROM dbo.Users
WHERE LastAccessDate > ‘2014/07/01’
ORDER BY LastAccessDate;
GO 5
44. Your execution plan
1. Shuffle through all of the pages,
writing down all the fields for each record,
if their LastAccessDate > ‘2014/07/01’.
2. Sort the matching records by LastAccessDate.
3. Keep the output so you could reuse it the next
time you saw this same query?
48. Nonclustered indexes: copies.
Stored in the order we want
Include the fields we want
CREATE INDEX
IX_LastAccessDate_Id
ON dbo.Users(LastAccessDate, Id)
49. Let’s go simple again.
SELECT Id
FROM dbo.Users
WHERE LastAccessDate > ‘2014/07/01’
ORDER BY LastAccessDate;
50. Your execution plan
1. Grab IX_LastAccessDate and seek to 2014/07/01.
2. Read the Id’s out in order.
59. It covers the fields we need in this query.
But if we change the query…
That’s a covering index.
60. Let’s add a couple of fields.
SELECT Id, DisplayName, Age
FROM dbo.Users
WHERE LastAccessDate > ‘2014/07/01’
ORDER BY LastAccessDate;
61. One execution plan
1. Grab IX_LastAccessDate_Id, seek to 2014/07/01.
2. Write down the Id and LastAccessDate of
matching records.
3. Grab the clustered index (white pages), and look
up each matching row by their Id to get
DisplayName and Age.
63. For simplicity, I told you I created this index with the Id.
SQL Server always includes your clustering keys whether
you ask for ‘em or not because it has to join indexes.
That’s why SQL Server includes the key
64. Key lookup is required
when the index doesn’t
have all the fields we need.
Hover your mouse over the
key lookup and look for the
OUTPUT fields.
Small? Frequently used?
Add ‘em to the index.
DO NOT ADD A NEW INDEX.
Classic index
tuning sign
69. Decide which index to use
What order to process tables/indexes in
Whether to do seeks or scans
Guess how many rows will match your query
How much memory to allocate for the query
Statistics help SQL Server:
74. Automatic stats updates aren’t enough. Consider:
• http://Ola.Hallengren.com
• http://MinionWare.net/reindex
Typical strategy: weekly statistics updates
Updated statistics on an index invalidate query plans that
involve that index
• Affects your plan cache analysis
• Can cause unpredictable query plan changes
Keep statistics updated.
92. Clustered indexes hold all the fields*
Nonclustered indexes are light-weight* copies of the table
NC indexes reduce not just reads, but also CPU work
SQL Server caches raw data pages, not query output
Statistics drive seek vs scan, index choice, memory
Statistics aren’t the only part: cardinality estimation matters
Includes and seeks aren’t magically delicious
What we learned
We’re using the StackOverflow.com database as an example.
To get download it, go to http://BrentOzar.com/go/querystack.
These screenshots are from the 2016/03 export, which is ~100GB.
If you use a newer or older export, your numbers of pages may vary.
This session focuses on the Users table
Id – primary key, clustered index. It’s an identity, starts at 1 and goes into the millions.
The white paper you’re holding in your hands – that’s the clustered index.
It includes all of the fields on the table – sort of.
Notice the About Me? It’s an NVARCHAR(MAX), and may not fit on a row. SQL Server may store that off-row, on other pages, if people get really wordy in their about-me field. We’re not going to touch on off-row data here, but I just want you to know there’s an overhead to that. Same thing with XML, JSON.
For old-school tables, everything is stored in 8KB pages.
These pages are the same whether they’re in memory or on disk.
It’s the smallest unit of data SQL Server works with.
(Things are different for Hekaton and columnstore indexes, but we’re focusing on old-school tables today.)