OSMC 2019 | How to improve database Observability by Charles JudithNETWAYS
Delivering a database service is not a simple job but to ensure that everything is working correctly your platform needs to be observable. In this talk, I’ll talk about how we make the MySQL/MariaDB databases observable. We’ll talk about the RED, USE methods, and the golden signals. You’ll discover how we dealt with the following questions “We think the database is slow”. This talk will allow you to make your databases discoverable with open source solutions.
Improve your skills with one of the hot innovations in the industry: Natural Language Processing. Apache cTAKES bridges the gap between humans and machines by transforming clinical thinking from provider notes into semi-structured data that can be processed by machines. This talk will also comprise of a demo analyzing and visualizing healthcare records using Apache cTAKES , running using Docker. From A-Z in 5 minutes.
This talk was presented during Apache Roadshow [1]
[1] - https://apachecon.com/usroadshowdc19/schedule.html
Presented at Apache Roadshow Washington DC 2019:
https://apachecon.com/usroadshowdc19/schedule.html
Abstract: Improve your skills with one of the hot innovations in the industry: Natural Language Processing. Apache cTAKES bridges the gap between humans and machines by transforming clinical thinking from provider notes into semi-structured data that can be processed by machines. This talk will also comprise of a demo analyzing and visualizing healthcare records using Apache cTAKES , running using Docker. From A-Z in 5 minutes.
Please find more details on:
http://ctakes.apache.org/
SQL Saturday 511 - Troubleshooting made easier using extended eventsAmit Banerjee
Extended Events in SQL Server provides a huge opportunity to debug and analyze complex problems with ease. I had presented this session at SQL Saturday 511 - Redmond to show the diagnostics enhancements available in SQL Server database engine in SQL Server 2012 Service Pack 3 and above. You will be able to leverage the diagnostics to troubleshoot and mitigate issues quickly in mission-critical environments and simplify troubleshooting experience for common SQL Server scenarios
Testing real-time Linux. What to test and how Chirag Jog
This paper describes testing of the real-time (CONFIG_PREEMPT_RT) Linux kernel. It explains how testing the real-time kernel is different from testing the mainline Linux kernel and provides some tips and guidelines about writing test cases for the real-time kernel. It illustrates real-time tests in the Linux Test Project (LTP) suite using examples. It also briefly covers real-time tests that are not part of LTP.
OSMC 2019 | How to improve database Observability by Charles JudithNETWAYS
Delivering a database service is not a simple job but to ensure that everything is working correctly your platform needs to be observable. In this talk, I’ll talk about how we make the MySQL/MariaDB databases observable. We’ll talk about the RED, USE methods, and the golden signals. You’ll discover how we dealt with the following questions “We think the database is slow”. This talk will allow you to make your databases discoverable with open source solutions.
Improve your skills with one of the hot innovations in the industry: Natural Language Processing. Apache cTAKES bridges the gap between humans and machines by transforming clinical thinking from provider notes into semi-structured data that can be processed by machines. This talk will also comprise of a demo analyzing and visualizing healthcare records using Apache cTAKES , running using Docker. From A-Z in 5 minutes.
This talk was presented during Apache Roadshow [1]
[1] - https://apachecon.com/usroadshowdc19/schedule.html
Presented at Apache Roadshow Washington DC 2019:
https://apachecon.com/usroadshowdc19/schedule.html
Abstract: Improve your skills with one of the hot innovations in the industry: Natural Language Processing. Apache cTAKES bridges the gap between humans and machines by transforming clinical thinking from provider notes into semi-structured data that can be processed by machines. This talk will also comprise of a demo analyzing and visualizing healthcare records using Apache cTAKES , running using Docker. From A-Z in 5 minutes.
Please find more details on:
http://ctakes.apache.org/
SQL Saturday 511 - Troubleshooting made easier using extended eventsAmit Banerjee
Extended Events in SQL Server provides a huge opportunity to debug and analyze complex problems with ease. I had presented this session at SQL Saturday 511 - Redmond to show the diagnostics enhancements available in SQL Server database engine in SQL Server 2012 Service Pack 3 and above. You will be able to leverage the diagnostics to troubleshoot and mitigate issues quickly in mission-critical environments and simplify troubleshooting experience for common SQL Server scenarios
Testing real-time Linux. What to test and how Chirag Jog
This paper describes testing of the real-time (CONFIG_PREEMPT_RT) Linux kernel. It explains how testing the real-time kernel is different from testing the mainline Linux kernel and provides some tips and guidelines about writing test cases for the real-time kernel. It illustrates real-time tests in the Linux Test Project (LTP) suite using examples. It also briefly covers real-time tests that are not part of LTP.
A brief overview of linux scheduler, context switch , priorities and scheduling classes as well as new features. Also provides an overview of preemption models in linux and how to use each model. all the examples are taken from http://www.discoversdk.com
Oracle ACE Director Dan Morgan presented those slides about migrating to database 12c and how to get it right. For more information, visit www.perftuning.com
Between 2015 and 2017 a large percentage of Oracle's existing customer base will be upgrading their existing databases to the new version 12cR1. Most of the time when upgrades happen the only benefits organizations receive are the satisfaction of having survived the upgrade unscathed. In general, the new database, other than having a new version number, provides little in the way of tangible benefits.
With the re-architecture that can come with a 12cR1 upgrade it is, for the first time, possible to plan for and receive substantial measurable benefits, and possible to make costly mistakes that could create substantial liabilities that are both business and financial.
Oracle ACE Director and industry veteran Dan Morgan, in a presentation targeted to IT/IS management explores both the benefits and the risks and provide a guideline for "getting it right."
This Performance Tuning's Lunch & Learn event focuses on management, planning, and budgeting, not features and technology, and provides you and your management teams the information they need to perform the next database upgrade or migration cycle.
A Backup Today Saves Tomorrow is a presentation from Percona Live 2013 that provides insight into planning and the tools used today to capture MySQL backups.
Delivered at the FISL13 conference in Brazil: http://www.youtube.com/watch?v=K9w2cipqfvc
This talk introduces the USE Method: a simple strategy for performing a complete check of system performance health, identifying common bottlenecks and errors. This methodology can be used early in a performance investigation to quickly identify the most severe system performance issues, and is a methodology the speaker has used successfully for years in both enterprise and cloud computing environments. Checklists have been developed to show how the USE Method can be applied to Solaris/illumos-based and Linux-based systems.
Many hardware and software resource types have been commonly overlooked, including memory and I/O busses, CPU interconnects, and kernel locks. Any of these can become a system bottleneck. The USE Method provides a way to find and identify these.
This approach focuses on the questions to ask of the system, before reaching for the tools. Tools that are ultimately used include all the standard performance tools (vmstat, iostat, top), and more advanced tools, including dynamic tracing (DTrace), and hardware performance counters.
Other performance methodologies are included for comparison: the Problem Statement Method, Workload Characterization Method, and Drill-Down Analysis Method.
What No One Tells You About Writing a Streaming App: Spark Summit East talk b...Spark Summit
So you know you want to write a streaming app but any non-trivial streaming app developer would have to think about these questions:
How do I manage offsets?
How do I manage state?
How do I make my spark streaming job resilient to failures? Can I avoid some failures?
How do I gracefully shutdown my streaming job?
How do I monitor and manage (e.g. re-try logic) streaming job?
How can I better manage the DAG in my streaming job?
When to use checkpointing and for what? When not to use checkpointing?
Do I need a WAL when using streaming data source? Why? When don’t I need one?
In this talk, we’ll share practices that no one talks about when you start writing your streaming app, but you’ll inevitably need to learn along the way.
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelDaniel Coupal
MongoDB presentation from Silicon Valley Code Camp 2015.
Walkthrough developing, deploying and operating a MongoDB application, avoiding the most common pitfalls.
SQL Server Wait Types Everyone Should KnowDean Richards
Many people use wait types for performance tuning, but do not know what some of the most common ones indicate. This presentation will go into details about the top 8 wait types I see at the customers I work with. It will provide wait descriptions as well as solutions.
A brief overview of linux scheduler, context switch , priorities and scheduling classes as well as new features. Also provides an overview of preemption models in linux and how to use each model. all the examples are taken from http://www.discoversdk.com
Oracle ACE Director Dan Morgan presented those slides about migrating to database 12c and how to get it right. For more information, visit www.perftuning.com
Between 2015 and 2017 a large percentage of Oracle's existing customer base will be upgrading their existing databases to the new version 12cR1. Most of the time when upgrades happen the only benefits organizations receive are the satisfaction of having survived the upgrade unscathed. In general, the new database, other than having a new version number, provides little in the way of tangible benefits.
With the re-architecture that can come with a 12cR1 upgrade it is, for the first time, possible to plan for and receive substantial measurable benefits, and possible to make costly mistakes that could create substantial liabilities that are both business and financial.
Oracle ACE Director and industry veteran Dan Morgan, in a presentation targeted to IT/IS management explores both the benefits and the risks and provide a guideline for "getting it right."
This Performance Tuning's Lunch & Learn event focuses on management, planning, and budgeting, not features and technology, and provides you and your management teams the information they need to perform the next database upgrade or migration cycle.
A Backup Today Saves Tomorrow is a presentation from Percona Live 2013 that provides insight into planning and the tools used today to capture MySQL backups.
Delivered at the FISL13 conference in Brazil: http://www.youtube.com/watch?v=K9w2cipqfvc
This talk introduces the USE Method: a simple strategy for performing a complete check of system performance health, identifying common bottlenecks and errors. This methodology can be used early in a performance investigation to quickly identify the most severe system performance issues, and is a methodology the speaker has used successfully for years in both enterprise and cloud computing environments. Checklists have been developed to show how the USE Method can be applied to Solaris/illumos-based and Linux-based systems.
Many hardware and software resource types have been commonly overlooked, including memory and I/O busses, CPU interconnects, and kernel locks. Any of these can become a system bottleneck. The USE Method provides a way to find and identify these.
This approach focuses on the questions to ask of the system, before reaching for the tools. Tools that are ultimately used include all the standard performance tools (vmstat, iostat, top), and more advanced tools, including dynamic tracing (DTrace), and hardware performance counters.
Other performance methodologies are included for comparison: the Problem Statement Method, Workload Characterization Method, and Drill-Down Analysis Method.
What No One Tells You About Writing a Streaming App: Spark Summit East talk b...Spark Summit
So you know you want to write a streaming app but any non-trivial streaming app developer would have to think about these questions:
How do I manage offsets?
How do I manage state?
How do I make my spark streaming job resilient to failures? Can I avoid some failures?
How do I gracefully shutdown my streaming job?
How do I monitor and manage (e.g. re-try logic) streaming job?
How can I better manage the DAG in my streaming job?
When to use checkpointing and for what? When not to use checkpointing?
Do I need a WAL when using streaming data source? Why? When don’t I need one?
In this talk, we’ll share practices that no one talks about when you start writing your streaming app, but you’ll inevitably need to learn along the way.
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelDaniel Coupal
MongoDB presentation from Silicon Valley Code Camp 2015.
Walkthrough developing, deploying and operating a MongoDB application, avoiding the most common pitfalls.
SQL Server Wait Types Everyone Should KnowDean Richards
Many people use wait types for performance tuning, but do not know what some of the most common ones indicate. This presentation will go into details about the top 8 wait types I see at the customers I work with. It will provide wait descriptions as well as solutions.
Linux /proc filesystem for MySQL DBAs - FOSDEM 2021Valeriy Kravchuk
Tools and approaches based on /proc sampling (like 0x.tools by Tanel Poder or ad hoc scripts) allow to measure individual thread level activity in MySQL server on Linux, like thread sleep states, currently executing system calls and kernel wait locations. If needed you can drill down into CPU usage of any thread or the system as a whole. Historical data can be captured for post factum analysis, without much impact on the system and no need to install or change anything in its configuration. In this presentation I am going to summarize what's possible with /proc and show useful examples for MySQL DBAs.
• We sleeping well. And our mobile ringing and ringing. Message: DISASTER! In this session (on slides) we are NOT talk about potential disaster (such BCM); we talk about: And what NOW? New version old my old well-known session updated for whole changes which happened in DBA World in last two-three years.
• So, from the ground to the Sky and further - everything for surviving disaster. Which tasks should have been finished BEFORE. Is virtual or physical SQL matter? We talk about systems, databases, peoples, encryption, passwords, certificates and users.
• In this session (on few demos) I'll show which part of our SQL Server Environment are critical and how to be prepared to disaster. In some documents I'll show You how to be BEST prepared.
A preview into SQL Server 2019 from Bob, Asad and my presentation at PASS Summit 2018 (Nov '18). We provided insights into what our public preview builds for SQL Server 2019 had in November.
Azure database services for PostgreSQL and MySQLAmit Banerjee
The slide deck that Rachel and I had used to present on an overview of the managed PostgreSQL and MySQL service on Azure at SQL Saturday Redmond, 2018. This is part of the Azure Database family.
A talk from SQL Saturday Oregon where I described the methodology and tools used by our Tier-1 customers to upgrade their SQL Server instances to a recent release of SQL Server with minimal downtime and risk.
SQL PASS 2017 - Building one million predictions per second using SQL Server ...Amit Banerjee
Using the power of OLTP and data transformation in SQL 2016 and advanced analytics in Microsoft R Server, various industries that really push the boundary of processing higher number of transaction per second (tps) for different use cases. In this talk, we will walk through the use case of predicting loan charge off (loan default) rate, architecture configuration that enable this use case, and rich visual dashboard that allow customer to do what-if analysis. Attend this session to find out how SQL + R allows you to build an “intelligent data warehouse”.
This is a talk which builds on my previous talk on how SQL Server 2016 helps build an intelligent data warehouse.
Building 1 million predictions per second using SQL-RAmit Banerjee
Using the power of OLTP and data transformation in SQL 2016 and advanced analytics in Microsoft R Server, various industries really push the boundary of processing higher number of transaction per second (tps) for different use cases. In this talk, we walked through the use case of predicting loan charge off (loan default) rate, architecture configuration that enable this use case, and rich visual dashboard that allow customer to do what-if analysis. Find out how SQL + R allows you to build an “intelligent datawarehouse”.
Troubleshooting common scenarios with Always On - A Dress RehearsalAmit Banerjee
Two of the most common issues that Microsoft Support is contacted related to Always On Availability Groups are associated with Latency and Failovers. During this session, we will demonstrate walkthroughs for latency analysis and failover root cause using enhancements in SQL Server and Management Studio. After this session, you will be able to troubleshoot such issues easily, quickly and efficiently.
Enhancements to High Availability, Disaster Recovery and ReplicationAmit Banerjee
SQL Server 2012 Service Pack 3, SQL Server 2014 Service Pack 2 and SQL Server 2016 bring about a number of enhancements to Always On Availability Groups and Replication. Join us to understand the enhancements available which will make your SQL Server instances work better if you have a dependency on Always On or Replication. We have enhancements for various scenarios like diagnostics, replicating between an on-premise SQL Server instance and AzureDB and troubleshooting common issues.
Recent changes to security compliance is driving the initiative in various IT environments to disable all security protocols apart from TLS 1.2. This has wide ranging impact on SQL Server installations from startup failures to connectivity issues. In this session, we will talk about the changes available in SQL Server 2008 and above to support TLS 1.2 and the changes required on the server and SQL Server configuration to support TLS 1.2.
All the script samples are available on GitHub at https://github.com/Microsoft/tigertoolbox/tree/master/tls1.2
Always On Availability Group Troubleshooting Ninja Amit Banerjee
SQL BITS Presentation
Session Objectives:
1.Troubleshoot failovers in your Availability Group easily
2.Determine the reason for connectivity loss and timeouts
3.Understand which part of your topology is the reason for latency
Debugging made easier with extended eventsAmit Banerjee
SQL BITS presentation
Session Objectives:
1.Understand the diagnostics enhancements available in SQL Server database engine in SQL Server 2012 Service Pack 3 and above
2.Leverage the diagnostics to troubleshoot and mitigate issues quickly in mission-critical environments
3.Simplify troubleshooting experience for common SQL Server scenarios
In this PASS VC PowerShell session, I had discussed about some best practice checks which can be automated for SQL Server instances running on your on-premise data centers and on Azure Virtual Machines.
All the scripts are available on https://github.com/amitmsft/SqlOnAzureVM
Performance Demystified for SQL Server on Azure Virtual MachinesAmit Banerjee
The slide deck for the session at TechEd which gives an overview of how to harness the power of Microsoft Azure Virtual Machines for SQL Server database. A few tips and tricks is all it requires to keep a SQL Server database healthy on an Azure Virtual Machine. What’s more, we will even slip in few tricks to automate the entire deployment with the performance best practices enabled.
Backup enhancements with SQL Server 2014Amit Banerjee
The presentation that I had used at the SQL Bangalore UG meeting for introducing new enhancements in SQL Server 2014 for Backup and Restore. I talked about three new concepts: Backup to URL, Encrypted Backups and Managed Backups
Troubleshooting SQL Server using Power Pivot and Power ViewAmit Banerjee
The presentation used at SQLBangalore User Group's Annual Day event at Bangalore to present on the topic "Troubleshooting SQL Server using PowerPivot and PowerView". This session talked about leveraging Power View and Power Pivot for monitoring SQL Server related issues.
The presentation used during my session at the SQL Bangalore UG Meeting on 24th August, 2013. The session talked about creating visualizations for monitoring the SQL Server instance using Power Pivot and Power View features in Excel 2013.
3. • What is a Deadlocked Scheduler?
• What are my options?
• Common Reasons
• Proactive action
• Reactive action
4. Scheduler • A scheduler is a logical CPU that is used by SQL
Server workers
Worker thread • Only a single worker can own the scheduler
Reason • Out of worker threads to process incoming
requests
• When a deadlocked scheduler issue is detected
17888 and over 50 percent of the workers are waiting on
a common resource type
5. DAC Notifications Post-mortem
• Collect DMV • Windows Event • Analyze the
outputs Log alerts logs
• Alerts on new • Debug the
memory dump memory dump
files
6. All schedulers have encountered a 17883
condition
All workers are blocked on a critical
resource. Eg. Spinlocks
All workers are executing a long-running
query: Parallelism and Lock Blocking
Open Transactions