1. Tools and Tips:
From Accidental to Efficient
Data Warehouse Developer
Cathrine Wilhelmsen - SQLSaturday Lisbon 2015

2. Session description
You have probably heard about the Accidental DBA, but what about the Accidental Data
Warehouse developer? We stumbled into the world of data warehousing, learned dimensional
modeling and work with T-SQL and SSIS daily. We're masters of googling solutions to our
problems and make sure our complex ETL processes run without errors. We deliver data to
business users... but we don't deliver data as fast as we want.
You might not be able to rewrite your entire data warehouse or change your team's processes
over night, but there are many things you can do to increase your own productivity and become
a more efficient data warehouse developer.
In this session I will show you some of what I've learned and discovered that has made me
burst out "Oh wow! Why did I not know this yesterday!?" including query improvements, free
tools and scripts, SSMS features and even a couple of things I used to think were only useful for
those scary DBAs.

3. Thank you to the main sponsors!

4. Say thank you to organizers and volunteers!
They spend their FREE time to give you this event
Because they are crazy 
Because they want YOU to learn from the BEST IN
THE WORLD

5. Paulo Matos:

6. Pedro Simões:

7. André Batista:

8. Paulo Borges:

9. André Melancia:

10. Murilo Miranda:

11. Quilson Antunes:

12. Important Activities
Sponsor Sessions with Raffles
• 15:10 - Rumos, BI4ALL, Bold Int, CozyRoc, Pythian
WIT (Women in Technology)
• 15:10 at BizSpark Room (Ground Floor)
SQLClinic Challenges
• 17:00 BI (Steph Locke)

13. Cathrine Wilhelmsen
@cathrinew
cathrinewilhelmsen.net
Data Warehouse Architect
Business Intelligence Developer

14. you
1-3 years?
T-SQL?
SSIS?

15. once upon a time...

17. how I felt...

19. how I want to be...

21. what?
SSMS
Queries
Biml for SSIS

22. Tip #1: Visual Information

23. Connection Colors

24. Status Bar and Tab Text

25. Results in Separate Tab

26. Tab Groups - Vertical

27. Tab Groups - Horizontal

28. Split –one query in two windows

29. Tip #2: Shortcuts

30. Query Shortcuts

31. Oh, by the way...
SELECT SUM(rows)
FROM sys.partitions
WHERE index_id IN (0, 1)
AND object_id =
OBJECT_ID('Sales');
...querying sys.partitions can be better and faster than COUNT(*)

32. Keyboard Shortcuts
Assign shortcuts
you frequently use
Remove shortcuts
you accidentally click
(no more "ooops")
msdn.microsoft.com/en-us/library/ms174205.aspx

33. Magic keys!
HOME END
PG UP PG DNCTRL ALT
SHIFT TAB

34. Show / Hide Query Results
CTRL R

35. Toggle Full Screen
ALTSHIFT ENTER

36. Cycle through windows
TABCTRL

37. Change database while writing query
CTRL U

38. Column / Multi-Line Editing
SHIFTALT

39. Comment / Uncomment
CTRL K CTRL C
Comment Line
CTRL K CTRL U
Uncomment Line

40. Tip #3: Search in SSMS

41. Free Tool: Redgate SQL Search
red-gate.com/products/sql-development/sql-search/

42. Free Tool: Redgate SQL Search

43. Tip #4: Templates and Snippets

44. Templates
Template Browser
Drag & Drop Templates
Create Templates
CTRL ALT T

45. Template Parameters
Replace Template Parameters with actual values
CTRL SHIFT M

46. Snippets
CTRL K CTRL X
Insert Snippet
CTRL K CTRL S
Surround With Snippet

47. Advanced Snippets and Formatting
Redgate SQL Prompt (Licensed)
ApexSQL Complete / Refactor
SSMS Tools Pack (Licensed)
SSMS Boost
Poor Man's T-SQL Formatter
dbForge SQL Complete (Licensed)
red-gate.com
apexsql.com
ssmstoolspack.com
ssmsboost.com
poorsql.com
devart.com/dbforge

48. Redgate SQL Prompt Demo

49. Tip #5: Registered Servers and Multiserver Queries

50. Registered Servers
Save and group servers
Is the server running?
Multiserver Queries
View Registered Servers
CTRL ALT G

51. Manage services from SSMS

52. Multiserver Queries

53. Multiserver Queries

54. Tip #6: SARGable Queries

55. SARGable queries
"The query can efficiently seek using an index to find the
correct rows searched for in WHERE or JOIN clauses"
Compare it to finding a person in a phone book
(We'll pretend we still use phone books)

56. Adama, Lee
Adama, William
Agathon, Karl
Baltar, Gaius
Dualla, Anastasia
Gaeta, Felix
Henderson, Cally
Roslin, Laura
Thrace, Kara
Tigh, Saul
Tyrol, Galen
Valerii, Sharon
SARGable queries
Find all rows where Name starts with "T"

57. Adama, Lee
Adama, William
Agathon, Karl
Baltar, Gaius
Dualla, Anastasia
Gaeta, Felix
Henderson, Cally
Roslin, Laura
Thrace, Kara
Tigh, Saul
Tyrol, Galen
Valerii, Sharon
SARGable queries
Find all rows where Name starts with "T"

58. Non-SARGable queries
"The query has to scan each row in the table to find the
correct rows searched for in WHERE or JOIN clauses"
Compare it to finding a person in a phone book
(We'll keep pretending we still use phone books)

59. Adama, Lee
Adama, William
Agathon, Karl
Baltar, Gaius
Dualla, Anastasia
Gaeta, Felix
Henderson, Cally
Roslin, Laura
Thrace, Kara
Tigh, Saul
Tyrol, Galen
Valerii, Sharon
Non-SARGable queries
Find all rows where Name contains "al"

60. Adama, Lee
Adama, William
Agathon, Karl
Baltar, Gaius
Dualla, Anastasia
Gaeta, Felix
Henderson, Cally
Roslin, Laura
Thrace, Kara
Tigh, Saul
Tyrol, Galen
Valerii, Sharon
Non-SARGable queries
Find all rows where Name contains "al"

61. WHERE Name LIKE '%al%'
WHERE Name LIKE 'T%'
WHERE LEFT(Name,1,1) = 'T'
SARGable or Non-SARGable?

62. WHERE Name LIKE '%al%'
WHERE Name LIKE 'T%'
WHERE LEFT(Name,1,1) = 'T'
SARGable or Non-SARGable?

63. WHERE CAST(EpisodeDate AS DATE) = '20050114'
WHERE CONVERT(CHAR(6), EpisodeDate, 112) = '200501'
WHERE YEAR(EpisodeDate) = 2005
WHERE EpisodeDate >= '20050101'
AND EpisodeDate < '20060101'
SARGable or Non-SARGable?

64. WHERE CAST(EpisodeDate AS DATE) = '20050114'
WHERE CONVERT(CHAR(6), EpisodeDate, 112) = '200501'
WHERE YEAR(EpisodeDate) = 2005
WHERE EpisodeDate >= '20050101'
AND EpisodeDate < '20060101'
SARGable or Non-SARGable?

65. WHERE Survivors < 40000
WHERE @Survivors BETWEEN Survivors-1000
AND Survivors+1000
WHERE Survivors BETWEEN @Survivors-1000
AND @Survivors+1000
SARGable or Non-SARGable?

66. WHERE Survivors < 40000
WHERE @Survivors BETWEEN Survivors-1000
AND Survivors+1000
WHERE Survivors BETWEEN @Survivors-1000
AND @Survivors+1000
SARGable or Non-SARGable?

67. sqlbits.com/Sessions/Event7/Understanding_SARGability_to_make_your_queries_run_faster

68. Tip #7: Query Analysis

69. Execution Plans
Display Estimated Execution Plan
CTRL L
Include Actual Execution Plan
CTRL M

70. Execution Plans
See how a query was or will be executed:

71. Details in Tooltips (ok)

72. Details in Properties (better)

73. Free Tool: SQL Sentry Plan Explorer
sqlsentry.com/products/plan-explorer

74. Free Tool: SQL Sentry Plan Explorer
answers.sqlperformance.com

75. Free Book: SQL Server Execution Plans
by Grant Fritchey
red-gate.com/community/books

76. Tip #8: Query Statistics

77. Statistics IO
SET STATISTICS IO OFF;
SET STATISTICS IO ON;

78. Statistics Time
SET STATISTICS TIME OFF;
SET STATISTICS TIME ON;

79. Free Tool: Statistics Parser by Richie Rump
statisticsparser.com

80. Client Statistics
Include Client Statistics
SHIFT SALT

81. Client Statistics
Compare multiple query executions:

82. Tip #9: Activity Monitoring

83. Free Script: sp_WhoIsActive by Adam Machanic
sqlblog.com/blogs/adam_machanic

84. Free Script: sp_WhoIsActive by Adam Machanic

85. Tip #10: Generate SSIS Packages with Biml

86. Business Intelligence Markup Language
Easy to read and write XML dialect
Generate SSIS packages from metadata

87. What do I need?
Free add-in for BIDS / SSDT-BI
bidshelper.codeplex.com

88. How does it work?
Create many SSIS packages from one Biml file

89. …what do you need me to do after lunch?
Of course I can create 200 SSIS Packages!

90. Biml syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Connections>
<OleDbConnection Name="Source" ConnectionString="…" />
</Connections>
<Packages>
<Package Name="EmptyPackage">
…
</Package>
</Packages>
</Biml>

91. From Biml to SSIS

92. From Biml to SSIS

93. The magic is in the
Extend Biml with C# or VB.NET code blocks
Import database structure and metadata
Loop over tables and columns
Add expressions to replace static values

94. BimlScript syntax
<#@ import namespace="Varigence.Hadron.CoreLowerer.SchemaManagement" #>
<# var conAW2014 = SchemaManager.CreateConnectionNode("AW2014", "..."); #>
<# var AW2014DB = conAW2014.ImportDB("","", ImportOptions.ExcludeViews); #>
<Packages>
<# foreach (var table in AW2014DB.TableNodes) { #>
<Package Name="Load_<#=table.Name#>">
…
</Package>
<# } #>
</Packages>

95. Biml for SSIS demo

96. …BimlBreak the rest of the week 
Biml on Monday…

97. Bonus Tip: Be social in your community!
#SQLHelp

98. cathrinewilhelmsen.net/efficient
Not enough details? Too fast? Don't worry!
Slide deck, links and resources:

99. Go talk to the sponsors!

100. Thank you! Enjoy your lunch 
@cathrinew
cathrinewilhelmsen.net
no.linkedin.com/in/cathrinewilhelmsen
contact@cathrinewilhelmsen.net
cathrinewilhelmsen.net/efficient