SlideShare a Scribd company logo
Morne Blake
Cloud Solution Architect
Microsoft
Azure SQL Database Insert Performance
Why are the inserts so slow?
10 DTU SQL Database – 10 000 rows
• ForEach loop 15 minutes
• Throw the whole insert – time out and about 4 000 rows written
100 DTU SQL Database – 100 000 rows
• ForEach loop 10 minutes
• Throw the whole insert – time out and about 7 000 rows written
Is Azure SQL database just slow?
No!
Let’s have a look at some code!
Written by a SQL guy ;)
What about updates?
• Same patterns can be applied
• Use table value parameters in a stored procedure
• Use Bulk Insert
Merge
MERGE Employee AS target
USING
(SELECT [FirstName], [LastName], [SocialSecurityNumber] FROM @employees) AS source
([FirstName], [LastName], [SocialSecurityNumber])
ON
(target.[SocialSecurityNumber] = source.[SocialSecurityNumber])
WHEN MATCHED THEN UPDATE SET
target.FirstName = source.FirstName, target.LastName = source.LastName
WHEN NOT MATCHED THEN INSERT
([FirstName], [LastName], [SocialSecurityNumber]) VALUES (source.[FirstName],
source.[LastName], source.[SocialSecurityNumber]);
using (var command = new SqlCommand("", sqlConn))
{
command.CommandText =
"CREATE TABLE #TmpDiskPrice (" +
"[Name] [nvarchar] (400) NOT NULL," +
"[Type] [nvarchar] (400) NOT NULL," +
//Some More things go here
);";
command.ExecuteNonQuery();
using (var bc = new SqlBulkCopy(sqlConn))
{
bc.DestinationTableName = "#TmpDiskPrice";
bc.ColumnMappings.Add("Name", "Name");
bc.ColumnMappings.Add("Type", "Type");
//Some More things go here
bc.WriteToServer(dt2);
}
command.CommandTimeout = 300;
command.CommandText = "MERGE dbo.apiManagedDisks AS Target " +
"USING (SELECT [Name], [RegionName],[currency],[Transactions], " +
"[PremiumSnapshot], [StandardSnapshotLrs], [StandardSnapshotZrs], [Size], [Speed], " +
"[Iops], [Price], [Type] FROM dbo.#TmpDiskPrice) AS Source " +
"ON (Target.Name = Source.Name AND Target.regionName = Source.regionName " +
"AND Target.currency = Source.currency AND Target.Type = Source.Type) " +
"WHEN MATCHED THEN " +
"UPDATE SET Target.Transactions = Source.Transactions, " +
"Target.PremiumSnapshot = Source.PremiumSnapshot," +
"Target.StandardSnapshotLrs = Source.StandardSnapshotLrs, " +
"Target.StandardSnapshotZrs = Source.StandardSnapshotZrs, " +
"Target.Size = Source.Size, " +
"Target.Speed = Source.Speed, " +
"Target.Iops = Source.Iops, " +
"Target.Price = Source.Price " +
"WHEN NOT MATCHED BY TARGET THEN " +
"INSERT([Name], [Type], [regionName],[currency],[Transactions], [StandardSnapshotLrs], [StandardSnapshotZrs]" +
", [PremiumSnapshot], [Size], [Speed], [Iops], [Price]) " +
"VALUES(Source.Name, Source.Type, Source.regionName, Source.currency, " +
"Source.Transactions, Source.StandardSnapshotLrs, Source.StandardSnapshotZrs," +
"Source.PremiumSnapshot ,Source.Size,Source.Speed,Source.Iops, Source.Price); " +
"DROP TABLE #TmpDiskPrice; ";
command.ExecuteNonQuery();

More Related Content

What's hot

Include
IncludeInclude
Include
Shi Chakep
 
Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)
Jonathan Katz
 
Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
Peter Eisentraut
 
Murano docker demo
Murano docker demoMurano docker demo
Murano docker demo
Henar Muñoz Frutos
 
Qtp Scripts
Qtp ScriptsQtp Scripts
Qtp Scripts
G.C Reddy
 
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Ruby Meditation
 
Class
ClassClass
Sbt for mere mortals
Sbt for mere mortalsSbt for mere mortals
Sbt for mere mortals
Ivan Porto Carrero
 
No More Deadlocks; Asynchronous Programming in .NET
No More Deadlocks; Asynchronous Programming in .NETNo More Deadlocks; Asynchronous Programming in .NET
No More Deadlocks; Asynchronous Programming in .NET
Filip Ekberg
 
RxJS 5 in Depth
RxJS 5 in DepthRxJS 5 in Depth
RxJS 5 in Depth
C4Media
 
Asynchronous programming
Asynchronous programmingAsynchronous programming
Asynchronous programming
Filip Ekberg
 
Actor Based Asyncronous IO in Akka
Actor Based Asyncronous IO in AkkaActor Based Asyncronous IO in Akka
Actor Based Asyncronous IO in Akka
drewhk
 
Async Redux Actions With RxJS - React Rally 2016
Async Redux Actions With RxJS - React Rally 2016Async Redux Actions With RxJS - React Rally 2016
Async Redux Actions With RxJS - React Rally 2016
Ben Lesh
 
VMUGIT UC 2013 - 07c Veeam
VMUGIT UC 2013 - 07c VeeamVMUGIT UC 2013 - 07c Veeam
VMUGIT UC 2013 - 07c Veeam
VMUG IT
 
Swift Sequences & Collections
Swift Sequences & CollectionsSwift Sequences & Collections
Swift Sequences & Collections
CocoaHeads France
 
PostgreSQLからMongoDBへ
PostgreSQLからMongoDBへPostgreSQLからMongoDBへ
PostgreSQLからMongoDBへ
Basuke Suzuki
 
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRealtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Rick Copeland
 
GPars For Beginners
GPars For BeginnersGPars For Beginners
GPars For Beginners
Matt Passell
 

What's hot (18)

Include
IncludeInclude
Include
 
Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)
 
Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
 
Murano docker demo
Murano docker demoMurano docker demo
Murano docker demo
 
Qtp Scripts
Qtp ScriptsQtp Scripts
Qtp Scripts
 
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
 
Class
ClassClass
Class
 
Sbt for mere mortals
Sbt for mere mortalsSbt for mere mortals
Sbt for mere mortals
 
No More Deadlocks; Asynchronous Programming in .NET
No More Deadlocks; Asynchronous Programming in .NETNo More Deadlocks; Asynchronous Programming in .NET
No More Deadlocks; Asynchronous Programming in .NET
 
RxJS 5 in Depth
RxJS 5 in DepthRxJS 5 in Depth
RxJS 5 in Depth
 
Asynchronous programming
Asynchronous programmingAsynchronous programming
Asynchronous programming
 
Actor Based Asyncronous IO in Akka
Actor Based Asyncronous IO in AkkaActor Based Asyncronous IO in Akka
Actor Based Asyncronous IO in Akka
 
Async Redux Actions With RxJS - React Rally 2016
Async Redux Actions With RxJS - React Rally 2016Async Redux Actions With RxJS - React Rally 2016
Async Redux Actions With RxJS - React Rally 2016
 
VMUGIT UC 2013 - 07c Veeam
VMUGIT UC 2013 - 07c VeeamVMUGIT UC 2013 - 07c Veeam
VMUGIT UC 2013 - 07c Veeam
 
Swift Sequences & Collections
Swift Sequences & CollectionsSwift Sequences & Collections
Swift Sequences & Collections
 
PostgreSQLからMongoDBへ
PostgreSQLからMongoDBへPostgreSQLからMongoDBへ
PostgreSQLからMongoDBへ
 
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRealtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
 
GPars For Beginners
GPars For BeginnersGPars For Beginners
GPars For Beginners
 

Similar to Azure sql insert perf

FMDB - SLC-Cocoaheads
FMDB - SLC-CocoaheadsFMDB - SLC-Cocoaheads
FMDB - SLC-Cocoaheads
Dave Stevenson
 
QB Into the Box 2018
QB Into the Box 2018QB Into the Box 2018
QB Into the Box 2018
Ortus Solutions, Corp
 
Beg sql
Beg sqlBeg sql
Beg sql
KPNR Jan
 
Beg sql
Beg sqlBeg sql
Master tuning
Master   tuningMaster   tuning
Master tuning
Thomas Kejser
 
Ss dotnetcodexmpl
Ss dotnetcodexmplSs dotnetcodexmpl
Ss dotnetcodexmpl
rpravi12
 
Owasp Indy Q2 2012 Advanced SQLi
Owasp Indy Q2 2012 Advanced SQLiOwasp Indy Q2 2012 Advanced SQLi
Owasp Indy Q2 2012 Advanced SQLi
owaspindy
 
Python with MySql.pptx
Python with MySql.pptxPython with MySql.pptx
Python with MySql.pptx
Ramakrishna Reddy Bijjam
 
Fast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on OracleFast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on Oracle
Raimonds Simanovskis
 
Ruby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRuby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrāde
Raimonds Simanovskis
 
Shrug2017 arcpy data_and_you
Shrug2017 arcpy data_and_youShrug2017 arcpy data_and_you
Shrug2017 arcpy data_and_you
SHRUG GIS
 
When to NoSQL and when to know SQL
When to NoSQL and when to know SQLWhen to NoSQL and when to know SQL
When to NoSQL and when to know SQL
Simon Elliston Ball
 
Python database access
Python database accessPython database access
SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?
SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?
SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?
PyungHo Yoon
 
ShmooCon 2009 - (Re)Playing(Blind)Sql
ShmooCon 2009 - (Re)Playing(Blind)SqlShmooCon 2009 - (Re)Playing(Blind)Sql
ShmooCon 2009 - (Re)Playing(Blind)Sql
Chema Alonso
 
groovy databases
groovy databasesgroovy databases
groovy databases
Paul King
 
Protractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applicationsProtractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applications
Ludmila Nesvitiy
 
Controlling The Cloud With Python
Controlling The Cloud With PythonControlling The Cloud With Python
Controlling The Cloud With Python
Luca Mearelli
 
Java OOP Programming language (Part 8) - Java Database JDBC
Java OOP Programming language (Part 8) - Java Database JDBCJava OOP Programming language (Part 8) - Java Database JDBC
Java OOP Programming language (Part 8) - Java Database JDBC
OUM SAOKOSAL
 
Slickdemo
SlickdemoSlickdemo
Slickdemo
Knoldus Inc.
 

Similar to Azure sql insert perf (20)

FMDB - SLC-Cocoaheads
FMDB - SLC-CocoaheadsFMDB - SLC-Cocoaheads
FMDB - SLC-Cocoaheads
 
QB Into the Box 2018
QB Into the Box 2018QB Into the Box 2018
QB Into the Box 2018
 
Beg sql
Beg sqlBeg sql
Beg sql
 
Beg sql
Beg sqlBeg sql
Beg sql
 
Master tuning
Master   tuningMaster   tuning
Master tuning
 
Ss dotnetcodexmpl
Ss dotnetcodexmplSs dotnetcodexmpl
Ss dotnetcodexmpl
 
Owasp Indy Q2 2012 Advanced SQLi
Owasp Indy Q2 2012 Advanced SQLiOwasp Indy Q2 2012 Advanced SQLi
Owasp Indy Q2 2012 Advanced SQLi
 
Python with MySql.pptx
Python with MySql.pptxPython with MySql.pptx
Python with MySql.pptx
 
Fast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on OracleFast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on Oracle
 
Ruby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRuby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrāde
 
Shrug2017 arcpy data_and_you
Shrug2017 arcpy data_and_youShrug2017 arcpy data_and_you
Shrug2017 arcpy data_and_you
 
When to NoSQL and when to know SQL
When to NoSQL and when to know SQLWhen to NoSQL and when to know SQL
When to NoSQL and when to know SQL
 
Python database access
Python database accessPython database access
Python database access
 
SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?
SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?
SQL 쿼리를 AWS DynamoDB에서 (CLI)로 사용해 볼까요?
 
ShmooCon 2009 - (Re)Playing(Blind)Sql
ShmooCon 2009 - (Re)Playing(Blind)SqlShmooCon 2009 - (Re)Playing(Blind)Sql
ShmooCon 2009 - (Re)Playing(Blind)Sql
 
groovy databases
groovy databasesgroovy databases
groovy databases
 
Protractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applicationsProtractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applications
 
Controlling The Cloud With Python
Controlling The Cloud With PythonControlling The Cloud With Python
Controlling The Cloud With Python
 
Java OOP Programming language (Part 8) - Java Database JDBC
Java OOP Programming language (Part 8) - Java Database JDBCJava OOP Programming language (Part 8) - Java Database JDBC
Java OOP Programming language (Part 8) - Java Database JDBC
 
Slickdemo
SlickdemoSlickdemo
Slickdemo
 

Recently uploaded

June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
flufftailshop
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 

Recently uploaded (20)

June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 

Azure sql insert perf

  • 1. Morne Blake Cloud Solution Architect Microsoft Azure SQL Database Insert Performance
  • 2. Why are the inserts so slow? 10 DTU SQL Database – 10 000 rows • ForEach loop 15 minutes • Throw the whole insert – time out and about 4 000 rows written 100 DTU SQL Database – 100 000 rows • ForEach loop 10 minutes • Throw the whole insert – time out and about 7 000 rows written
  • 3. Is Azure SQL database just slow? No!
  • 4. Let’s have a look at some code! Written by a SQL guy ;)
  • 5. What about updates? • Same patterns can be applied • Use table value parameters in a stored procedure • Use Bulk Insert
  • 6. Merge MERGE Employee AS target USING (SELECT [FirstName], [LastName], [SocialSecurityNumber] FROM @employees) AS source ([FirstName], [LastName], [SocialSecurityNumber]) ON (target.[SocialSecurityNumber] = source.[SocialSecurityNumber]) WHEN MATCHED THEN UPDATE SET target.FirstName = source.FirstName, target.LastName = source.LastName WHEN NOT MATCHED THEN INSERT ([FirstName], [LastName], [SocialSecurityNumber]) VALUES (source.[FirstName], source.[LastName], source.[SocialSecurityNumber]);
  • 7. using (var command = new SqlCommand("", sqlConn)) { command.CommandText = "CREATE TABLE #TmpDiskPrice (" + "[Name] [nvarchar] (400) NOT NULL," + "[Type] [nvarchar] (400) NOT NULL," + //Some More things go here );"; command.ExecuteNonQuery(); using (var bc = new SqlBulkCopy(sqlConn)) { bc.DestinationTableName = "#TmpDiskPrice"; bc.ColumnMappings.Add("Name", "Name"); bc.ColumnMappings.Add("Type", "Type"); //Some More things go here bc.WriteToServer(dt2); } command.CommandTimeout = 300; command.CommandText = "MERGE dbo.apiManagedDisks AS Target " + "USING (SELECT [Name], [RegionName],[currency],[Transactions], " + "[PremiumSnapshot], [StandardSnapshotLrs], [StandardSnapshotZrs], [Size], [Speed], " + "[Iops], [Price], [Type] FROM dbo.#TmpDiskPrice) AS Source " + "ON (Target.Name = Source.Name AND Target.regionName = Source.regionName " + "AND Target.currency = Source.currency AND Target.Type = Source.Type) " + "WHEN MATCHED THEN " + "UPDATE SET Target.Transactions = Source.Transactions, " + "Target.PremiumSnapshot = Source.PremiumSnapshot," + "Target.StandardSnapshotLrs = Source.StandardSnapshotLrs, " + "Target.StandardSnapshotZrs = Source.StandardSnapshotZrs, " + "Target.Size = Source.Size, " + "Target.Speed = Source.Speed, " + "Target.Iops = Source.Iops, " + "Target.Price = Source.Price " + "WHEN NOT MATCHED BY TARGET THEN " + "INSERT([Name], [Type], [regionName],[currency],[Transactions], [StandardSnapshotLrs], [StandardSnapshotZrs]" + ", [PremiumSnapshot], [Size], [Speed], [Iops], [Price]) " + "VALUES(Source.Name, Source.Type, Source.regionName, Source.currency, " + "Source.Transactions, Source.StandardSnapshotLrs, Source.StandardSnapshotZrs," + "Source.PremiumSnapshot ,Source.Size,Source.Speed,Source.Iops, Source.Price); " + "DROP TABLE #TmpDiskPrice; "; command.ExecuteNonQuery();