Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLBits XIV)

3,060 views

Published on

Tools and Tips: From Accidental to Efficient Data Warehouse Developer (Presented at SQLBits XIV on March 6th 2015)

Published in: Data & Analytics

Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLBits XIV)

  1. 1. Tools and Tips for Data Warehouse Developers Cathrine Wilhelmsen - SQLBits XIV
  2. 2. Session description You have probably heard about the Accidental DBA, but what about the Accidental Data Warehouse developer? We stumbled into the world of data warehousing, learned dimensional modeling and work with T-SQL and SSIS daily. We're masters of googling solutions to our problems and make sure our complex ETL processes run without errors. We deliver data to business users... but we don't deliver data as fast as we want. You might not be able to rewrite your entire data warehouse or change your team's processes over night, but there are many things you can do to increase your own productivity and become a more efficient data warehouse developer. In this session I will show you some of what I've learned and discovered that has made me burst out "Oh wow! Why did I not know this yesterday!?" including query improvements, free tools and scripts, SSMS features and even a couple of things I used to think were only useful for those scary DBAs.
  3. 3. Cathrine Wilhelmsen @cathrinew cathrinewilhelmsen.net Data Warehouse Architect Business Intelligence Developer
  4. 4. you 1-3 years? T-SQL? SSIS?
  5. 5. once upon a time...
  6. 6. how I felt...
  7. 7. how I want to be...
  8. 8. what? SSMS Queries Biml for SSIS
  9. 9. Tip #1: Visual Information
  10. 10. Connection Colors
  11. 11. Status Bar and Tab Text
  12. 12. Results in Separate Tab
  13. 13. Tab Groups - Vertical
  14. 14. Tab Groups - Horizontal
  15. 15. Split –one query in two windows
  16. 16. Tip #2: Shortcuts
  17. 17. Query Shortcuts
  18. 18. Keyboard Shortcuts Assign shortcuts you frequently use Remove shortcuts you accidentally click (no more "ooops") msdn.microsoft.com/en-us/library/ms174205.aspx
  19. 19. Magic keys! HOME END PG UP PG DNCTRL ALT SHIFT TAB
  20. 20. Show / Hide Query Results CTRL R
  21. 21. Toggle Full Screen ALTSHIFT ENTER
  22. 22. Cycle through windows TABCTRL
  23. 23. Change database while writing query CTRL U
  24. 24. Column / Multi-Line Editing SHIFTALT
  25. 25. Comment / Uncomment CTRL K CTRL C Comment Line CTRL K CTRL U Uncomment Line
  26. 26. Tip #3: Search in SSMS
  27. 27. Free Tool: Redgate SQL Search red-gate.com/products/sql-development/sql-search/
  28. 28. Free Tool: Redgate SQL Search
  29. 29. Tip #4: Templates and Snippets
  30. 30. Templates Template Browser Drag & Drop Templates Create Templates CTRL ALT T
  31. 31. Template Parameters Replace Template Parameters with actual values CTRL SHIFT M
  32. 32. Snippets CTRL K CTRL X Insert Snippet CTRL K CTRL S Surround With Snippet
  33. 33. Advanced Snippets and Formatting Redgate SQL Prompt (Licensed) ApexSQL Complete / Refactor SSMS Tools Pack (Licensed) SSMS Boost Poor Man's T-SQL Formatter dbForge SQL Complete (Licensed) red-gate.com apexsql.com ssmstoolspack.com ssmsboost.com poorsql.com devart.com/dbforge
  34. 34. Redgate SQL Prompt Demo
  35. 35. Tip #5: Registered Servers and Multiserver Queries
  36. 36. Registered Servers Save and group servers Is the server running? Multiserver Queries View Registered Servers CTRL ALT G
  37. 37. Manage services from SSMS
  38. 38. Multiserver Queries
  39. 39. Multiserver Queries
  40. 40. Tip #6: SARGable Queries
  41. 41. SARGable queries "The query can efficiently seek using an index to find the correct rows searched for in WHERE or JOIN clauses" Compare it to finding a person in a phone book (We'll pretend we still use phone books)
  42. 42. Adama, Lee Adama, William Agathon, Karl Baltar, Gaius Dualla, Anastasia Gaeta, Felix Henderson, Cally Roslin, Laura Thrace, Kara Tigh, Saul Tyrol, Galen Valerii, Sharon SARGable queries Find all rows where Name starts with "T"
  43. 43. Adama, Lee Adama, William Agathon, Karl Baltar, Gaius Dualla, Anastasia Gaeta, Felix Henderson, Cally Roslin, Laura Thrace, Kara Tigh, Saul Tyrol, Galen Valerii, Sharon SARGable queries Find all rows where Name starts with "T"
  44. 44. Non-SARGable queries "The query has to scan each row in the table to find the correct rows searched for in WHERE or JOIN clauses" Compare it to finding a person in a phone book (We'll keep pretending we still use phone books)
  45. 45. Adama, Lee Adama, William Agathon, Karl Baltar, Gaius Dualla, Anastasia Gaeta, Felix Henderson, Cally Roslin, Laura Thrace, Kara Tigh, Saul Tyrol, Galen Valerii, Sharon Non-SARGable queries Find all rows where Name contains "al"
  46. 46. Adama, Lee Adama, William Agathon, Karl Baltar, Gaius Dualla, Anastasia Gaeta, Felix Henderson, Cally Roslin, Laura Thrace, Kara Tigh, Saul Tyrol, Galen Valerii, Sharon Non-SARGable queries Find all rows where Name contains "al"
  47. 47. WHERE Name LIKE '%al%' WHERE Name LIKE 'T%' WHERE LEFT(Name,1,1) = 'T' SARGable or Non-SARGable?
  48. 48. WHERE Name LIKE '%al%' WHERE Name LIKE 'T%' WHERE LEFT(Name,1,1) = 'T' SARGable or Non-SARGable?
  49. 49. WHERE CAST(EpisodeDate AS DATE) = '20050114' WHERE CONVERT(CHAR(6), EpisodeDate, 112) = '200501' WHERE YEAR(EpisodeDate) = 2005 WHERE EpisodeDate >= '20050101' AND EpisodeDate < '20060101' SARGable or Non-SARGable?
  50. 50. WHERE CAST(EpisodeDate AS DATE) = '20050114' WHERE CONVERT(CHAR(6), EpisodeDate, 112) = '200501' WHERE YEAR(EpisodeDate) = 2005 WHERE EpisodeDate >= '20050101' AND EpisodeDate < '20060101' SARGable or Non-SARGable?
  51. 51. WHERE Survivors < 40000 WHERE @Survivors BETWEEN Survivors-1000 AND Survivors+1000 WHERE Survivors BETWEEN @Survivors-1000 AND @Survivors+1000 SARGable or Non-SARGable?
  52. 52. WHERE Survivors < 40000 WHERE @Survivors BETWEEN Survivors-1000 AND Survivors+1000 WHERE Survivors BETWEEN @Survivors-1000 AND @Survivors+1000 SARGable or Non-SARGable?
  53. 53. sqlbits.com/Sessions/Event7/Understanding_SARGability_to_make_your_queries_run_faster
  54. 54. Tip #7: Query Analysis
  55. 55. Execution Plans Display Estimated Execution Plan CTRL L Include Actual Execution Plan CTRL M
  56. 56. Execution Plans See how a query was or will be executed:
  57. 57. Details in Tooltips
  58. 58. Details in Properties
  59. 59. Free Tool: SQL Sentry Plan Explorer sqlsentry.com/products/plan-explorer
  60. 60. Free Tool: SQL Sentry Plan Explorer answers.sqlperformance.com
  61. 61. Free Book: SQL Server Execution Plans by Grant Fritchey red-gate.com/community/books
  62. 62. Tip #8: Query Statistics
  63. 63. Statistics IO SET STATISTICS IO OFF; SET STATISTICS IO ON;
  64. 64. Statistics Time SET STATISTICS TIME OFF; SET STATISTICS TIME ON;
  65. 65. Free Tool: Statistics Parser statisticsparser.com
  66. 66. Client Statistics Include Client Statistics SHIFT SALT
  67. 67. Client Statistics Compare multiple query executions:
  68. 68. Tip #9: Activity Monitoring
  69. 69. Free Script: sp_WhoIsActive by Adam Machanic sqlblog.com/blogs/adam_machanic
  70. 70. Free Script: sp_WhoIsActive by Adam Machanic
  71. 71. Tip #10: Automatically Generate SSIS Packages with Biml
  72. 72. Business Intelligence Markup Language Easy to read and write XML dialect Generate SSIS packages from metadata
  73. 73. What do I need? Free add-in for BIDS / SSDT-BI bidshelper.codeplex.com
  74. 74. How does it work? Create many SSIS packages from one Biml file
  75. 75. …what do you need me to do after lunch? Of course I can create 200 SSIS Packages!
  76. 76. Biml syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Connections> <OleDbConnection Name="Source" ConnectionString="…" /> </Connections> <Packages> <Package Name="EmptyPackage"> … </Package> </Packages> </Biml>
  77. 77. From Biml to SSIS
  78. 78. From Biml to SSIS
  79. 79. The magic is in the Extend Biml with C# or VB.NET code blocks Import database structure and metadata Loop over tables and columns Add expressions to replace static values
  80. 80. BimlScript syntax <#@ import namespace="Varigence.Hadron.CoreLowerer.SchemaManagement" #> <# var conAW2014 = SchemaManager.CreateConnectionNode("AW2014", "..."); #> <# var AW2014DB = conAW2014.ImportDB("","", ImportOptions.ExcludeViews); #> <Packages> <# foreach (var table in AW2014DB.TableNodes) { #> <Package Name="Load_<#=table.Name#>"> … </Package> <# } #> </Packages>
  81. 81. Biml for SSIS demo
  82. 82. …BimlBreak the rest of the week  Biml on Monday…
  83. 83. cathrinewilhelmsen.net/efficient Not enough details? Too fast? Don't worry! Slide deck, links and resources:
  84. 84. Thank you! @cathrinew cathrinewilhelmsen.net no.linkedin.com/in/cathrinewilhelmsen contact@cathrinewilhelmsen.net cathrinewilhelmsen.net/efficient

×