1 extreme performance - part i

  • 780 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
780
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • No escalation from Row to PageEach lock structure is 100 bytesNumber of locks acquired on a rowset > 5000 - Periodically after that
  • Self-explanatory. See Books Online for more details about partitioning. Also see http://msdn2.microsoft.com/en-us/library/ms345146.aspx
  • Query 1 is updating partition 1. It hits the escalation threshold and escalates to a TABLE X lock. Query 2 tries to update partition 3 and can’t because of the table X lock, even though nothing in partition 3 is being altered by query 1.
  • Instead of escalating to the table level and blocking all other queries on the table, you can now choose to have escalation to the partition level, which allows concurrent queries on different partitions.
  • אחד האהובים עלי
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Database applications work in much slower times. For complex event processing applications, the latencies involved are much smaller.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.
  • Complex event processing involves the processing and querying of data when the data is in flight. The data needs to be processed before there is time to persist it.Such processing involves very high data rates and often deals with multiple concurrent event sources and multiple output data.The aim is to process with as little latency as possible.

Transcript

  • 1. Extreme Performance for DBAs
    מאיר דודאי| SQL Server MVP| ואלינור
  • 2. All transaction isolation level
    Differences between Serializeable and Repateable Read
    RangeX-S locking vsRangeX-U
    Deep dive into bookmark lookup
    Page internals
    Indexes on Sparse columns
  • 3. All transaction isolation level
    Differences between Serializeable and Repateable Read
    RangeX-S locking vsRangeX-U
    Deep dive into bookmark lookup
    Page internals
    Indexes on Sparse columns
  • 4. לפני קצת יותר מ-10 שנים...
  • 5. לפני קצת יותר מ-10 שנים...
  • 6. לפני קצת יותר מ-10 שנים...
  • 7. מטרות
    היכרות עם תהליך P&T
    כלים חדשים ולא מוכרים לניהול Performance
    הבנת יישום נכון של Compression
    היכרות עם יכולות Scale out בגירסאות הנוכחיות וב-Denali
  • 8. מי אני?
    מאיר דודאי:
    SQL Server MVP
    יועץ בתחום בסיסי נתונים
    ניסיון של למעלה מ-10 שנים ב-SQL Server
    עורך האתר:
  • 9. ואלינור
    החברה המובילה בישראל במתן שירותים מקצועיים בתחום בסיסי נתונים: SQL Server, Oracle, MySQL
    צוות יועצים מוביל המתמחה בכלל הרבדים שלבסיסי הנתונים
    שותפים של Microsoft ו- Oracle
    נציגי מוצרים מובילים בתחום ה-DB
    מעל 300 לקוחות בכל המגזרים
  • 10. My Assumptions About You
    You may be a part-time or full-time DBA
    You are familiar with SQL Server in previous versions
    You would like to learn about new features in SQL Server 2008&R2&2011 that can help you to manage and improve performance
    You are interested in good lunch, two days free from work, and some giveaways from advertisers
  • 11.
  • 12.
  • 13.
  • 14. ארגז הכלים החדש שלכם
  • 15. Agenda
    • Utility Control Point
    • 16. Detecting bottlenecks
    • 17. New Performance Monitor in Windows 2008/R2
    • 18. Performance Data Collector
  • Agenda
    • Utility Control Point
    • 19. Detecting bottlenecks
    • 20. New Performance Monitor in Windows 2008/R2
    • 21. Performance Data Collector
  • Database evolution
    Underutilized hardware
    Hardware computing capacity
    Overburdened Administrators
    Number of database apps
    Number of DBAs
    1990
    2000
    2010
  • 22. Having fun with deployment
  • 23. DAC Concepts
    Data-tier Application Component
    • .dacpac= unit of deployment (data-tier application + developer intent)
    • 24. Data-tier Application Component
    • 25. Improves collaboration between developer and DBA
    • 26. Moves developers from a procedural model to a declarative model
    • 27. V1 is targeted at small apps
    Schema
    LOGICAL
    Tables, Views, Procs, UDFs
    PHYSICAL
    Users, Logins, Indexes
    DAC Deployment Profile
    Deployment Requirements, Management Policies
  • 28. Key Concept: Utility Control Point
    • Provides unified view of an organization’s SQL Server usage
    • 29. Shows health of
    • 30. SQL Server instances
    • 31. Data-tier applications
    • 32. Database files, filegroups and volumes
    • 33. Provides resource monitoring
    • 34. CPU utilization
    • 35. Storage space
    SQL02
    SQL01
    SQL03
    SQL05
    SQL04
    DBA
    UCP
    Managed Instances
    Management Studio
  • 36. DEVELOP
    DEPLOY
    MANAGE
    SQL Server Management Studio
    FinApp
    Dev DB
    1
    Deploy /
    Upgrade DAC
    Reverse Engineer DAC
    2
    SQL Server Management Studio
    9
    4
    8
    Visual Studio 2010
    Manage, Register, Uninstall, Extract, Upgrade DAC
    Control
    Point
    Create policies
    DBA
    DBA
    7
    3
    Deploy /
    Upgrade DAC
    HR
    .dacpac
    Hand-off to DBA
    Compile
    + Build
    FinApp
    Prod DB
    6
    Developer
    SALES
    5
    Managed Instances
  • 37. Utility Control Point
  • 38. UCP
  • 39. UCP
  • 40. UCP Reports
  • 41. UCP Reports
    http://bit.ly/j8xMwH
  • 42. UCP Reports
  • 43. What’s missing?
    Supports only SQL Server 2008 R2 (or later)
    Better management
    More counters can be useful…
    Monitors only DAC
    Or is it?
  • 44. Fake DAC
  • 45. What’s next?
    Denali’s Contained Databases…
  • 46. Agenda
    • Utility Control Point
    • 47. Detecting bottlenecks
    • 48. New Performance Monitor in Windows 2008/R2
    • 49. Performance Data Collector
  • Performance Tools
    DTA
    SSMS
    Built-in Reports
    Activity Monitor (or what’s left of it)
    Profiler assistants:
    ClearTrace
    Trace Analyzer
    UCP
    BPA
    Extended Events
  • 50. SSMS Activity Monitor
  • 51. Performance Reports
  • 52. Using Profiler to Detect Blocks
  • 53. Clear Trace & Trace Analyzer
  • 54. Agenda
    • Utility Control Point
    • 55. Detecting bottlenecks
    • 56. New Performance Monitor in Windows 2008/R2
    • 57. Performance Data Collector
  • Windows Performance Tools
    Performance Monitor
    Reliability Monitor
    Resource Monitor
    PAL (Performance Analysis of Logs)
  • 58. Performance & Resource Monitor
  • 59. New performance counters
    Database Mirroring
    Compression stats
    Transactions volume
    Mirroring performance
    Custom counters
  • 60. Custom counters
  • 61. Agenda
    • Utility Control Point
    • 62. Detecting bottlenecks
    • 63. New Performance Monitor in Windows 2008/R2
    • 64. Performance Data Collector
  • Monitor with Insight
    • Performance data collection
    • 65. OS perf indicators
    • 66. Performance data
    • 67. Extensible collection
    • 68. Central repository
    • 69. Consolidate information
    • 70. Open interface
    • 71. Management reports
    • 72. Canned reports
    • 73. Extensible
  • Open Your Mind!
  • 74. Agenda
    Magic settings
    RCSI
    Partition-level lock escalation
    Architectural design
    Bulk Load
    StreamInsight
    Caching
    Offloading Traffic
    Denali HADR
  • 75. GreenRoad Case Study:RCSI
    GreenRoad is an Israeli startup improving driving behavior for fleets and consumers
    Works as SAAS
    More than 400 inserts/sec
    Users running queries and reports frequently
    Method used to increase concurrency:
    RCSI: Read Committed Snapshot Isolation
  • 76. Tran2 (Select)
    Tran1 (Update)
    Reader Writer Blocking
    Data Page
    S-Lock
    Blocked
    Row-1
    Row-1
    X-Lock
  • 77. Read Committed Snapshot
    New “flavor” of read committed
    Turn ON/OFF on a database
    Readers see committed values as of beginning of statement
    Writers do not block Readers
    Readers do not block Writers
    Writers do block writers
    Can greatly reduce locking / deadlocking without changing applications
  • 78. Demo
  • 79. Lock Escalation
    HOBT
    T1: IX
    T1: X
    Page
    Page
    Page
    T1: X
    T1: X
    T1: X
    T1: X
    T1: X
    Row
    Row
    Row
    T1: X
    T1: X
    T1: X
    Lock Escalation
    T1: IX
  • 80. Lock Escalation
    Converting finer-grain locks to coarse grain locks.
    Row to Table
    Page to Table.
    Benefits
    Reduced locking overhead
    Reduces Memory requirement
    Triggered when
    Number of locks acquired on a rowset > 5000
    Memory pressure
  • 81. Partitioned Tables and Indexes
    SQL Server 2005 introduced partitioning, which some customers use to scale a query workload
    Another common use is to streamline maintenance and enable fast range inserts and removals from tables
    Partitioned
    Table
    FG1
    FG2
    FG3
    Partition 2
    Partition 3
    Partition 1
  • 82. Lock Escalation: The Problem
    Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions
    Only way to solve this in SQL Server 2005 is to disable lock escalation
    Query 1
    Query 2
    Partitioned
    Table
    IX
    X
    ESCALATE
    update
    update
    Partition 2
    Partition 1
    Partition 3
    FG1
    FG2
    FG3
  • 83. Lock Escalation: The Solution
    SQL Server 2008 allows lock escalation to the partition level, allowing concurrent access to other partitions
    Escalation to partition level does not block queries on other partitions
    Query 1
    Query 2
    Partitioned
    Table
    IX
    ESCALATE
    update
    update
    X
    Partition 2
    Partition 1
    Partition 3
    FG1
    FG2
    FG3
  • 84. Demo
  • 85. Filtered Indexes
    An index with a WHERE clause to specify a criteria
    Essentially index only a subset of a the table
    Query optimizer most likely to use when WHERE clause matches that of the filtered index
  • 86. Using Filtered Indexes
    Advantages of Filtered Indexes
    Improve query performance, partly by enhancing execution plan quality
    Smaller index maintenance costs
    Less disk storage
  • 87. Using Filtered Indexes
    Scenarios for Filtered Indexes
    Sparse columns, where most data is null
    Columns with categories of values
    Columns with distinct ranges of values
  • 88. Filtered indexes
  • 89. New server option in SQL Server 2008
    Only a stub is cached on first execution
    Full plan cached after second execution
    SP_CONFIGURE 'show advanced options',1RECONFIGUREGO
    SP_CONFIGURE 'optimize for ad hoc workloads',1RECONFIGUREGO
    Optimize for ad hoc workloads
  • 90. “I would not use it on some DWH systems”
    “Why is this not on by default?”
    “I can’t think of a reason why opt for adhoc workloads would ever really be a problem”
    “I definitely recommend it!”
    “Best movie of 2011! Breathtaking!”
    “But paintball guns are also not very pleasant!”
    How good is it?
  • 91. Codename Lyngby Case Study: Power of bulk inserts
    Security/audit system
    50GB of data inserted daily
    Data is kept for one year and should be available for queries and reports at any time
    Method used to increase concurrency:
    Bulk inserts
  • 92. Benefits of Bulk Inserts
    Improved concurrency
    Allow application-database decoupling
    In Lyngby case – 50GB a day!
    http://www.tapuz.co.il/blog/userBlog.asp?FolderName=Hygge
  • 93. Bulk Inserts & RCSI: SQLCAT Benchmark
    What happens when you query a table while bulk inserts are running?
  • 94. Bulk Inserts & RCSI: SQLCAT Benchmark
    And with RCSI?
    Bulk loading does not affect the size of the version store in tempdb under RCSI
    Keep in mind: RCSI adds 14 noncompressible bytes into every row in every table
    Bottom line: RCSI works extremely well with bulk inserts and achieves improved concurrency
    More details about this benchmark:
    http://sqlcat.com/technicalnotes/archive/2009/04/06/bulk-loading-data-into-a-table-with-concurrent-queries.aspx
  • 95. StreamInsight:Complex Event Processing
    SQL Server 2008 R2 feature
    Processing and querying of event data streams
    Data queried while “in flight”
    May involve multiple concurrent event sources
    Works with high data rates
    Aims for near-zero latency
  • 96. Isn’t This Just a Database Application?
    Event
    request
    output stream
    input stream
    response
  • 97. Caching
    Your database can get some rest...
  • 98. Microsoft AppFabric
  • 99. Other solutions
    Memcache
    Requires application change
    3rd party
    Usually transparent
  • 100. Tapuz Case Study: Offloading Traffic
    Tapuz is the leading user content based website in Israel
    1,200 forums
    74,000 blogs
    500,000 visitors each month
    More than 4000 queries/sec
    Method used to increase concurrency:
    Offloading using Transactional Replication
  • 101. Compression
  • 102. As data volume grows…
  • Solution – Compress Data Files?
    No benefit in-memory
    Need to compress larger chunks
    Does not scale for random access
  • 107. Solution – SQL Server 2008 / R2
    Store data efficiently in the row/page
    (+) More data can fit in memory
    (+) Better Performance for I/O bound workload
    (-) Performance degradation for CPU bound workload
  • 108. Data Compression
    SQL Server 2008
    ROW and PAGE compression
    Backup Compression
    SQL Server 2008 R2
    Unicode compression
  • 109. Enabling Compression Examples
    Latest partition uncompressed
    Uncompressed
    PAGE Compressed
    ROW Compressed
    Jan-Mar
    Apr-June
    July-Sept
    Oct-Dec
  • 110. Should it be so complex?
    In real life – usually compress the entire large tables using page compression…
  • 111. Summary – Compression
    Can reduce size of database significantly
    Lower total cost of ownership (TCO)
    Easy to enable/disable
    No application changes
    Performance gains!
  • 112. Control Your Resources
  • 113. Resource Hogs
    What are some of the biggest resource hogs a DBA runs into?
    Bulk Load Processes
    Data Archiving
    Poorly Coded Queries (Runaways)
    Reporting…
  • 114. Reporting in Production
    Is this a best practice???
  • 115. Reporting In Production
    NO!
  • 116. Reporting In Production
    NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
  • 117. Reporting In Production
    OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!
  • 118. Resource Governor
    Similar to a governor you would find on a car
    It limits the “speed” that a SQL Server session can run at
  • 119. Resource Governor – Workloads
    Ability to differentiate workloads based on connection property
    e.g. app_name, login, user, db name
    Per-request limits
    Max memory %
    Max CPU time
    Grant timeout
    Max Requests