Your SlideShare is downloading. ×
1   extreme performance - part i
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

1 extreme performance - part i

802
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
802
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • No escalation from Row to PageEach lock structure is 100 bytesNumber of locks acquired on a rowset > 5000 - Periodically after that
  • Self-explanatory. See Books Online for more details about partitioning. Also see http://msdn2.microsoft.com/en-us/library/ms345146.aspx
  • Query 1 is updating partition 1. It hits the escalation threshold and escalates to a TABLE X lock. Query 2 tries to update partition 3 and can’t because of the table X lock, even though nothing in partition 3 is being altered by query 1.
  • Instead of escalating to the table level and blocking all other queries on the table, you can now choose to have escalation to the partition level, which allows concurrent queries on different partitions.
  • אחד האהובים עלי
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Database applications work in much slower times. For complex event processing applications, the latencies involved are much smaller.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Transcript

    • 1. Extreme Performance for DBAs
      מאיר דודאי| SQL Server MVP| ואלינור
    • 2. All transaction isolation level
      Differences between Serializeable and Repateable Read
      RangeX-S locking vsRangeX-U
      Deep dive into bookmark lookup
      Page internals
      Indexes on Sparse columns
    • 3. All transaction isolation level
      Differences between Serializeable and Repateable Read
      RangeX-S locking vsRangeX-U
      Deep dive into bookmark lookup
      Page internals
      Indexes on Sparse columns
    • 4. לפני קצת יותר מ-10 שנים...
    • 5. לפני קצת יותר מ-10 שנים...
    • 6. לפני קצת יותר מ-10 שנים...
    • 7. מטרות
      היכרות עם תהליך P&T
      כלים חדשים ולא מוכרים לניהול Performance
      הבנת יישום נכון של Compression
      היכרות עם יכולות Scale out בגירסאות הנוכחיות וב-Denali
    • 8. מי אני?
      מאיר דודאי:
      SQL Server MVP
      יועץ בתחום בסיסי נתונים
      ניסיון של למעלה מ-10 שנים ב-SQL Server
      עורך האתר:
    • 9. ואלינור
      החברה המובילה בישראל במתן שירותים מקצועיים בתחום בסיסי נתונים: SQL Server, Oracle, MySQL
      צוות יועצים מוביל המתמחה בכלל הרבדים שלבסיסי הנתונים
      שותפים של Microsoft ו- Oracle
      נציגי מוצרים מובילים בתחום ה-DB
      מעל 300 לקוחות בכל המגזרים
    • 10. My Assumptions About You
      You may be a part-time or full-time DBA
      You are familiar with SQL Server in previous versions
      You would like to learn about new features in SQL Server 2008&R2&2011 that can help you to manage and improve performance
      You are interested in good lunch, two days free from work, and some giveaways from advertisers
    • 11.
    • 12.
    • 13.
    • 14. ארגז הכלים החדש שלכם
    • 15. Agenda
      • Utility Control Point
      • 16. Detecting bottlenecks
      • 17. New Performance Monitor in Windows 2008/R2
      • 18. Performance Data Collector
    • Agenda
      • Utility Control Point
      • 19. Detecting bottlenecks
      • 20. New Performance Monitor in Windows 2008/R2
      • 21. Performance Data Collector
    • Database evolution
      Underutilized hardware
      Hardware computing capacity
      Overburdened Administrators
      Number of database apps
      Number of DBAs
      1990
      2000
      2010
    • 22. Having fun with deployment
    • 23. DAC Concepts
      Data-tier Application Component
      • .dacpac= unit of deployment (data-tier application + developer intent)
      • 24. Data-tier Application Component
      • 25. Improves collaboration between developer and DBA
      • 26. Moves developers from a procedural model to a declarative model
      • 27. V1 is targeted at small apps
      Schema
      LOGICAL
      Tables, Views, Procs, UDFs
      PHYSICAL
      Users, Logins, Indexes
      DAC Deployment Profile
      Deployment Requirements, Management Policies
    • 28. Key Concept: Utility Control Point
      • Provides unified view of an organization’s SQL Server usage
      • 29. Shows health of
      • 30. SQL Server instances
      • 31. Data-tier applications
      • 32. Database files, filegroups and volumes
      • 33. Provides resource monitoring
      • 34. CPU utilization
      • 35. Storage space
      SQL02
      SQL01
      SQL03
      SQL05
      SQL04
      DBA
      UCP
      Managed Instances
      Management Studio
    • 36. DEVELOP
      DEPLOY
      MANAGE
      SQL Server Management Studio
      FinApp
      Dev DB
      1
      Deploy /
      Upgrade DAC
      Reverse Engineer DAC
      2
      SQL Server Management Studio
      9
      4
      8
      Visual Studio 2010
      Manage, Register, Uninstall, Extract, Upgrade DAC
      Control
      Point
      Create policies
      DBA
      DBA
      7
      3
      Deploy /
      Upgrade DAC
      HR
      .dacpac
      Hand-off to DBA
      Compile
      + Build
      FinApp
      Prod DB
      6
      Developer
      SALES
      5
      Managed Instances
    • 37. Utility Control Point
    • 38. UCP
    • 39. UCP
    • 40. UCP Reports
    • 41. UCP Reports
      http://bit.ly/j8xMwH
    • 42. UCP Reports
    • 43. What’s missing?
      Supports only SQL Server 2008 R2 (or later)
      Better management
      More counters can be useful…
      Monitors only DAC
      Or is it?
    • 44. Fake DAC
    • 45. What’s next?
      Denali’s Contained Databases…
    • 46. Agenda
      • Utility Control Point
      • 47. Detecting bottlenecks
      • 48. New Performance Monitor in Windows 2008/R2
      • 49. Performance Data Collector
    • Performance Tools
      DTA
      SSMS
      Built-in Reports
      Activity Monitor (or what’s left of it)
      Profiler assistants:
      ClearTrace
      Trace Analyzer
      UCP
      BPA
      Extended Events
    • 50. SSMS Activity Monitor
    • 51. Performance Reports
    • 52. Using Profiler to Detect Blocks
    • 53. Clear Trace & Trace Analyzer
    • 54. Agenda
      • Utility Control Point
      • 55. Detecting bottlenecks
      • 56. New Performance Monitor in Windows 2008/R2
      • 57. Performance Data Collector
    • Windows Performance Tools
      Performance Monitor
      Reliability Monitor
      Resource Monitor
      PAL (Performance Analysis of Logs)
    • 58. Performance & Resource Monitor
    • 59. New performance counters
      Database Mirroring
      Compression stats
      Transactions volume
      Mirroring performance
      Custom counters
    • 60. Custom counters
    • 61. Agenda
      • Utility Control Point
      • 62. Detecting bottlenecks
      • 63. New Performance Monitor in Windows 2008/R2
      • 64. Performance Data Collector
    • Monitor with Insight
      • Performance data collection
      • 65. OS perf indicators
      • 66. Performance data
      • 67. Extensible collection
      • 68. Central repository
      • 69. Consolidate information
      • 70. Open interface
      • 71. Management reports
      • 72. Canned reports
      • 73. Extensible
    • Open Your Mind!
    • 74. Agenda
      Magic settings
      RCSI
      Partition-level lock escalation
      Architectural design
      Bulk Load
      StreamInsight
      Caching
      Offloading Traffic
      Denali HADR
    • 75. GreenRoad Case Study:RCSI
      GreenRoad is an Israeli startup improving driving behavior for fleets and consumers
      Works as SAAS
      More than 400 inserts/sec
      Users running queries and reports frequently
      Method used to increase concurrency:
      RCSI: Read Committed Snapshot Isolation
    • 76. Tran2 (Select)
      Tran1 (Update)
      Reader Writer Blocking
      Data Page
      S-Lock
      Blocked
      Row-1
      Row-1
      X-Lock
    • 77. Read Committed Snapshot
      New “flavor” of read committed
      Turn ON/OFF on a database
      Readers see committed values as of beginning of statement
      Writers do not block Readers
      Readers do not block Writers
      Writers do block writers
      Can greatly reduce locking / deadlocking without changing applications
    • 78. Demo
    • 79. Lock Escalation
      HOBT
      T1: IX
      T1: X
      Page
      Page
      Page
      T1: X
      T1: X
      T1: X
      T1: X
      T1: X
      Row
      Row
      Row
      T1: X
      T1: X
      T1: X
      Lock Escalation
      T1: IX
    • 80. Lock Escalation
      Converting finer-grain locks to coarse grain locks.
      Row to Table
      Page to Table.
      Benefits
      Reduced locking overhead
      Reduces Memory requirement
      Triggered when
      Number of locks acquired on a rowset > 5000
      Memory pressure
    • 81. Partitioned Tables and Indexes
      SQL Server 2005 introduced partitioning, which some customers use to scale a query workload
      Another common use is to streamline maintenance and enable fast range inserts and removals from tables
      Partitioned
      Table
      FG1
      FG2
      FG3
      Partition 2
      Partition 3
      Partition 1
    • 82. Lock Escalation: The Problem
      Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions
      Only way to solve this in SQL Server 2005 is to disable lock escalation
      Query 1
      Query 2
      Partitioned
      Table
      IX
      X
      ESCALATE
      update
      update
      Partition 2
      Partition 1
      Partition 3
      FG1
      FG2
      FG3
    • 83. Lock Escalation: The Solution
      SQL Server 2008 allows lock escalation to the partition level, allowing concurrent access to other partitions
      Escalation to partition level does not block queries on other partitions
      Query 1
      Query 2
      Partitioned
      Table
      IX
      ESCALATE
      update
      update
      X
      Partition 2
      Partition 1
      Partition 3
      FG1
      FG2
      FG3
    • 84. Demo
    • 85. Filtered Indexes
      An index with a WHERE clause to specify a criteria
      Essentially index only a subset of a the table
      Query optimizer most likely to use when WHERE clause matches that of the filtered index
    • 86. Using Filtered Indexes
      Advantages of Filtered Indexes
      Improve query performance, partly by enhancing execution plan quality
      Smaller index maintenance costs
      Less disk storage
    • 87. Using Filtered Indexes
      Scenarios for Filtered Indexes
      Sparse columns, where most data is null
      Columns with categories of values
      Columns with distinct ranges of values
    • 88. Filtered indexes
    • 89. New server option in SQL Server 2008
      Only a stub is cached on first execution
      Full plan cached after second execution
      SP_CONFIGURE 'show advanced options',1RECONFIGUREGO
      SP_CONFIGURE 'optimize for ad hoc workloads',1RECONFIGUREGO
      Optimize for ad hoc workloads
    • 90. “I would not use it on some DWH systems”
      “Why is this not on by default?”
      “I can’t think of a reason why opt for adhoc workloads would ever really be a problem”
      “I definitely recommend it!”
      “Best movie of 2011! Breathtaking!”
      “But paintball guns are also not very pleasant!”
      How good is it?
    • 91. Codename Lyngby Case Study: Power of bulk inserts
      Security/audit system
      50GB of data inserted daily
      Data is kept for one year and should be available for queries and reports at any time
      Method used to increase concurrency:
      Bulk inserts
    • 92. Benefits of Bulk Inserts
      Improved concurrency
      Allow application-database decoupling
      In Lyngby case – 50GB a day!
      http://www.tapuz.co.il/blog/userBlog.asp?FolderName=Hygge
    • 93. Bulk Inserts & RCSI: SQLCAT Benchmark
      What happens when you query a table while bulk inserts are running?
    • 94. Bulk Inserts & RCSI: SQLCAT Benchmark
      And with RCSI?
      Bulk loading does not affect the size of the version store in tempdb under RCSI
      Keep in mind: RCSI adds 14 noncompressible bytes into every row in every table
      Bottom line: RCSI works extremely well with bulk inserts and achieves improved concurrency
      More details about this benchmark:
      http://sqlcat.com/technicalnotes/archive/2009/04/06/bulk-loading-data-into-a-table-with-concurrent-queries.aspx
    • 95. StreamInsight:Complex Event Processing
      SQL Server 2008 R2 feature
      Processing and querying of event data streams
      Data queried while “in flight”
      May involve multiple concurrent event sources
      Works with high data rates
      Aims for near-zero latency
    • 96. Isn’t This Just a Database Application?
      Event
      request
      output stream
      input stream
      response
    • 97. Caching
      Your database can get some rest...
    • 98. Microsoft AppFabric
    • 99. Other solutions
      Memcache
      Requires application change
      3rd party
      Usually transparent
    • 100. Tapuz Case Study: Offloading Traffic
      Tapuz is the leading user content based website in Israel
      1,200 forums
      74,000 blogs
      500,000 visitors each month
      More than 4000 queries/sec
      Method used to increase concurrency:
      Offloading using Transactional Replication
    • 101. Compression
    • 102. As data volume grows…
    • Solution – Compress Data Files?
      No benefit in-memory
      Need to compress larger chunks
      Does not scale for random access
    • 107. Solution – SQL Server 2008 / R2
      Store data efficiently in the row/page
      (+) More data can fit in memory
      (+) Better Performance for I/O bound workload
      (-) Performance degradation for CPU bound workload
    • 108. Data Compression
      SQL Server 2008
      ROW and PAGE compression
      Backup Compression
      SQL Server 2008 R2
      Unicode compression
    • 109. Enabling Compression Examples
      Latest partition uncompressed
      Uncompressed
      PAGE Compressed
      ROW Compressed
      Jan-Mar
      Apr-June
      July-Sept
      Oct-Dec
    • 110. Should it be so complex?
      In real life – usually compress the entire large tables using page compression…
    • 111. Summary – Compression
      Can reduce size of database significantly
      Lower total cost of ownership (TCO)
      Easy to enable/disable
      No application changes
      Performance gains!
    • 112. Control Your Resources
    • 113. Resource Hogs
      What are some of the biggest resource hogs a DBA runs into?
      Bulk Load Processes
      Data Archiving
      Poorly Coded Queries (Runaways)
      Reporting…
    • 114. Reporting in Production
      Is this a best practice???
    • 115. Reporting In Production
      NO!
    • 116. Reporting In Production
      NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
    • 117. Reporting In Production
      OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!
    • 118. Resource Governor
      Similar to a governor you would find on a car
      It limits the “speed” that a SQL Server session can run at
    • 119. Resource Governor – Workloads
      Ability to differentiate workloads based on connection property
      e.g. app_name, login, user, db name
      Per-request limits
      Max memory %
      Max CPU time
      Grant timeout
      Max Requests