This document provides a guide to MySQL performance tuning. It discusses identifying performance bottlenecks, measuring system resources like I/O, memory, and CPU, tuning MySQL settings like the key buffer size and InnoDB buffer pool size, and changing application queries and indexes to improve performance. Key steps include finding slow queries, enabling the slow query log, and profiling queries to identify optimization opportunities.
This document discusses nonsteroidal anti-inflammatory drugs (NSAIDs) and their cardiovascular risks. It summarizes several major clinical trials that compared NSAIDs to placebos or other NSAIDs. The VIGOR trial found rofecoxib (Vioxx) increased heart attack risk compared to naproxen. Subsequent trials had conflicting results on COX-2 inhibitors and traditional NSAIDs. Meta-analyses found both COX-2 inhibitors and traditional NSAIDs, except naproxen, modestly increase cardiovascular risk compared to placebos. Aspirin was the only NSAID found to reduce cardiovascular risk.
The Linux booting process begins when the user turns on the computer. The BIOS loads and runs a power-on self-test before finding the bootable devices and loading the boot sector from the master boot record (MBR). The MBR then loads the boot loader, such as GRUB or LILO, which loads the Linux kernel into memory and passes control to it. The kernel initializes essential system components and starts the init process, which launches other processes according to the runlevel configuration to complete the system startup.
A tutorial for beginners who are curious to learn about the Linux boot process. If you have any more doubts, you can contact me through my email given in the slide, or through my blog: mastro77.blogspot.in
Original N-Channel Mosfet CS13N50 13N50 500V 13A TO-220 NewAUTHELECTRONIC
The document provides specifications for the CS13N50 A8H silicon N-channel power MOSFET, including its electrical characteristics, thermal properties, maximum ratings and applications. Key features include low on-resistance, fast switching speeds, and high single pulse avalanche energy rating. The MOSFET is intended for use in power switching circuits for applications such as adaptor and charger power switches.
Integrando Redis en aplicaciones Symfony2Ronny López
Sus múltiples casos de usos y su excepcional rendimiento hacen que Redis sea hoy una pieza clave en la arquitectura de aplicaciones altamente dinámicas.
En la charla se expone de forma práctica cómo puede integrarse Redis en una aplicación Symfoy y cómo pueden implementarse varias de las características de las aplicaciones usando Redis, como por ejemplo: Session storage, Monolog logging handlers, Doctrine caching, SwiftMailer spooling, Profiler storage, Data Collector for Symfony2 Profiler.
Además de estos casos de uso generales, se comentan otros casos de usos específicos de aplicaciones que por su naturaleza no pueden beneficiarse de una capa de cache y se requiere por tanto una herramienta eficiente y escalable para resolver ciertos tipos de problemas.
A real-time system must respond to external stimuli within a finite time period. The correctness of real-time computations depends on both logical results and timeliness. Real-time systems require substantial design effort to ensure task deadlines are met. There are two types of real-time systems: hard where missing deadlines causes damage, and soft where missing deadlines is undesirable. Scheduling algorithms like earliest deadline first (EDF) and rate monotonic analysis (RMA) are used to ensure tasks meet deadlines in real-time systems.
An RTOS differs from a common OS in that it allows direct access to the microprocessor and peripherals, helping to meet deadlines. An RTOS provides mechanisms for multitasking like scheduling, context switching, and IPC. It uses techniques like priority-based preemptive scheduling and memory management with separate stacks and heaps for tasks. Common RTOS services include timing functions, synchronization, resource protection and communication between processes.
This document discusses nonsteroidal anti-inflammatory drugs (NSAIDs) and their cardiovascular risks. It summarizes several major clinical trials that compared NSAIDs to placebos or other NSAIDs. The VIGOR trial found rofecoxib (Vioxx) increased heart attack risk compared to naproxen. Subsequent trials had conflicting results on COX-2 inhibitors and traditional NSAIDs. Meta-analyses found both COX-2 inhibitors and traditional NSAIDs, except naproxen, modestly increase cardiovascular risk compared to placebos. Aspirin was the only NSAID found to reduce cardiovascular risk.
The Linux booting process begins when the user turns on the computer. The BIOS loads and runs a power-on self-test before finding the bootable devices and loading the boot sector from the master boot record (MBR). The MBR then loads the boot loader, such as GRUB or LILO, which loads the Linux kernel into memory and passes control to it. The kernel initializes essential system components and starts the init process, which launches other processes according to the runlevel configuration to complete the system startup.
A tutorial for beginners who are curious to learn about the Linux boot process. If you have any more doubts, you can contact me through my email given in the slide, or through my blog: mastro77.blogspot.in
Original N-Channel Mosfet CS13N50 13N50 500V 13A TO-220 NewAUTHELECTRONIC
The document provides specifications for the CS13N50 A8H silicon N-channel power MOSFET, including its electrical characteristics, thermal properties, maximum ratings and applications. Key features include low on-resistance, fast switching speeds, and high single pulse avalanche energy rating. The MOSFET is intended for use in power switching circuits for applications such as adaptor and charger power switches.
Integrando Redis en aplicaciones Symfony2Ronny López
Sus múltiples casos de usos y su excepcional rendimiento hacen que Redis sea hoy una pieza clave en la arquitectura de aplicaciones altamente dinámicas.
En la charla se expone de forma práctica cómo puede integrarse Redis en una aplicación Symfoy y cómo pueden implementarse varias de las características de las aplicaciones usando Redis, como por ejemplo: Session storage, Monolog logging handlers, Doctrine caching, SwiftMailer spooling, Profiler storage, Data Collector for Symfony2 Profiler.
Además de estos casos de uso generales, se comentan otros casos de usos específicos de aplicaciones que por su naturaleza no pueden beneficiarse de una capa de cache y se requiere por tanto una herramienta eficiente y escalable para resolver ciertos tipos de problemas.
A real-time system must respond to external stimuli within a finite time period. The correctness of real-time computations depends on both logical results and timeliness. Real-time systems require substantial design effort to ensure task deadlines are met. There are two types of real-time systems: hard where missing deadlines causes damage, and soft where missing deadlines is undesirable. Scheduling algorithms like earliest deadline first (EDF) and rate monotonic analysis (RMA) are used to ensure tasks meet deadlines in real-time systems.
An RTOS differs from a common OS in that it allows direct access to the microprocessor and peripherals, helping to meet deadlines. An RTOS provides mechanisms for multitasking like scheduling, context switching, and IPC. It uses techniques like priority-based preemptive scheduling and memory management with separate stacks and heaps for tasks. Common RTOS services include timing functions, synchronization, resource protection and communication between processes.
Embedded Linux provides a standardized operating system solution for embedded systems through the Linux kernel. The Linux kernel abstracts the underlying hardware and provides drivers to interface with hardware peripherals. This allows application developers to focus on their code without needing to manage low-level hardware interactions. A bootloader initializes the hardware and loads the Linux kernel from memory. The kernel then loads and runs programs stored in the filesystem. Cross-compilers allow the same source code to target different processor architectures. Libraries and drivers help share code and resources across applications and hardware.
The document provides an overview of the Linux kernel, including its architecture, startup process, functionality, configuration, and compilation. It discusses the differences between micro and monolithic kernels. It also explains the Linux kernel architecture with user space and kernel space separated by a system call interface. Key aspects covered include process management, memory management, device management, and the kernel build system.
A teaching slide set describing the mechanisms of action and clinical use of local anaesthetics. This session is a basic introduction to the pharmacodynamics and pharmacokinetics of local anaesthetics. It is aimed at preclinical medical or dental students, or students in the early years of a pharmacology degree.
Memory management in operating system | Paging | Virtual memoryShivam Mitra
This document discusses memory management techniques in operating systems. It begins by covering contiguous memory allocation approaches like fixed and variable partitioning. It then discusses non-contiguous techniques like paging and segmentation. Key concepts covered include logical vs physical addresses, page tables, translation lookaside buffers, demand paging, and virtual memory. The document provides examples and links to detailed video explanations of these important OS memory management topics.
Processes and threads are fundamental concepts in Windows Vista. A process contains the virtual address space, threads, and resources for program execution. Each process has a process environment block (PEB) and can create multiple threads, each with their own thread environment block (TEB). Threads are the unit of CPU scheduling and each process must have at least one thread. Interprocess communication (IPC) allows processes to communicate and share data using various methods like pipes, mailslots, sockets, and shared memory. Synchronization objects like mutexes, events, and semaphores coordinate access to shared resources between threads.
Absorbed dose is the amount of energy imparted to matter by ionizing radiation. The larger the absorbed dose, the greater the hazard. Early limits on radiation exposure were based on preventing obvious effects like skin damage, but later considered delayed effects like cancer. The most radiation-sensitive organs are those with high cellular replication rates like bone marrow, lung and thyroid. As hazards were discovered, researchers published injuries and set early limits, with the first recommended dose of 10 rad per day in 1902. Monitoring devices like film badges and dosimeters now help detect radiation exposure and reduce risk through limiting time, increasing distance and improving shielding.
Linux kernel has a special feature called Watchdog timer which would reset the system in case of any software faults | system hangs | or any application crashes after a timeout has reached.
For Complete Learning: http://www.thelearnet.com/
Overview
Multithreading Models
Threading Issues
Pthreads
Solaris 2 Threads
Windows 2000 Threads
Linux Threads
Java Threads
- Photoacoustics involves using the photoacoustic effect to generate ultrasonic waves through the absorption of pulsed laser light in a material.
- The photoacoustic effect results from the thermoelastic expansion that occurs when optical energy is absorbed and converted to heat, resulting in localized pressure increases.
- The photoacoustic wave equation can be derived from the acoustic wave equation and the thermal diffusion equation, and has an inhomogeneous term representing the optical energy deposition.
- The general solution to the photoacoustic wave equation can be obtained using Green's function approaches and Cauchy contour integration in spherical coordinates to model the resulting spherical acoustic wave propagation.
The document discusses limitations of the Von Neumann architecture and opportunities for parallel processing through non-Von Neumann architectures. It provides background on the Von Neumann model and its key components. The document then explains how increasing demand for faster computers is driving researchers to explore parallel processing approaches like SIMD and MIMD that can solve larger, more complex problems by distributing work across multiple processors. It aims to describe non-Von Neumann architectures and parallel algorithms.
This document discusses calcium channel blockers (CCBs), including their mechanism of action, classification, therapeutic uses, adverse effects, and drug interactions. CCBs work by blocking calcium channels, leading to vasodilation and reduced heart rate and contractility. They are classified into dihydropyridines, benzothiazepines, and phenylalkylamines. Common uses include hypertension, angina, and arrhythmias. Side effects reflect their pharmacological actions and include flushing, constipation, and bradycardia. CCBs can interact dangerously with beta blockers due to combined cardio-depressant effects.
Here are some key things to practice with regular expressions using grep, sed and awk:
- Use grep to search files for patterns using basic regex metacharacters like ^, $, ., *, [, ], etc.
- Use sed for search and replace operations on files using regex to match patterns
- Use awk to parse files into fields based on delimiters like space, comma etc and perform operations on the fields stored in variables $1, $2 etc.
- Write simple awk scripts to print, filter and manipulate data from files. For example print certain fields, filter rows, perform calculations etc.
- Learn about more advanced regex features supported by tools like extended regex in grep, backreferences in sed etc
Local anesthetics are drugs that cause reversible loss of sensation, especially pain, in a restricted area of the body. They work by interfering with sodium channel function and action potential generation in neurons. Common local anesthetics include lidocaine, bupivacaine, and tetracaine. Adding vasoconstrictors like epinephrine prolongs the duration of anesthesia by slowing absorption from the injection site. Local anesthetics are used for surface anesthesia, injections, and spinal anesthesia. Adverse effects include central nervous system toxicity, cardiovascular effects, and allergic reactions at high doses.
The document discusses different types of computer systems and operating systems. It describes the main components of a computer system including hardware, operating system, application programs, and users. It then covers different types of operating systems such as mainframe systems, batch systems, time-sharing systems, desktop systems, parallel systems, distributed systems, real-time systems, and handheld systems. The document also discusses hardware protection mechanisms used by operating systems, including dual-mode operation, I/O protection, memory protection, and CPU protection.
Contiguous allocation is an early method for allocating main memory between the operating system and user processes. Each process is contained within a single contiguous section of memory. Relocation and limit registers are used to protect processes from each other and allow transient changes to the operating system. Multiple-partition allocation allocates variable sized partitions to processes out of memory holes. Storage allocation approaches like first-fit and best-fit are used to allocate partitions from the holes. Fragmentation can occur both externally and internally, taking up space that cannot be used.
Recent advances in ischemic heart diseasessaachslides15
This document provides an overview of ischemic heart diseases and recent advances in treatment. It discusses the types of angina pectoris and myocardial infarction. Current treatment strategies include nitrates, beta blockers, calcium channel blockers, and potassium channel openers. Newer agents that are discussed include trimetazidine, ranolazine, and ivabradine which lower cardiac oxygen demand and improve symptoms of angina. Combination therapies are also used to maximize antianginal effects while minimizing side effects.
MySQL High-Availability and Scale-Out architecturesFromDual GmbH
This document discusses MySQL high availability and scale-out architectures. It covers topics like scale-up vs scale-out, MySQL replication, high availability solutions, MySQL Cluster, application partitioning, and other architectural considerations. The document provides examples of MySQL replication configurations and cluster architectures to achieve goals like high availability, performance and scale-out. It also discusses emerging technologies like SSD that could disrupt database architectures.
The document provides best practices for performance tuning MySQL databases. It discusses benchmarking and profiling concepts, sources of performance problems like inefficient schemas and indexes, and SQL coding practices. It also recommends tuning server parameters and provides tools for benchmarking, profiling, and optimizing performance.
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)Aurimas Mikalauskas
Is my MySQL server configured properly? Should I run Community MySQL, MariaDB, Percona or WebScaleSQL? How many innodb buffer pool instances should I run? Why should I NOT use the query cache? How do I size the innodb log file size and what IS that innodb log anyway? All answers are inside.
Aurimas Mikalauskas is a former Percona performance consultant and architect currently writing and teaching at speedemy.com. He's been involved with MySQL since 1999, scaling and optimizing MySQL backed systems since 2004 for companies such as BBC, EngineYard, famous social networks and small shops like EstanteVirtual, Pine Cove and hundreds of others.
Additional content mentioned in the presentation can be found here: http://speedemy.com/17
This document discusses InnoDB locking mechanisms in MySQL, including:
- Row-level locks are used to prevent conflicts between transactions accessing the same rows.
- Different isolation levels determine which types of locks are used and what kinds of conflicts are allowed. Read committed is the default isolation level.
- Deadlocks can occur when multiple transactions try to acquire locks in incompatible orders. InnoDB detects and rolls back transactions involved in deadlocks.
This document provides an overview of performance tuning the MySQL server. It discusses where to find server configuration and status information, how to analyze what the database is doing using status variables, and which configuration variables can be tuned for optimization, including global, per-session, and storage engine variables. Key areas covered include memory usage, query analysis, indexing strategies, and tuning storage engines like InnoDB and MyISAM.
The document discusses tuning MySQL server settings for performance. Some key points covered include:
- Settings are workload-specific and depend on factors like storage engine, OS, hardware. Tuning involves getting a few settings right rather than maximizing all settings.
- Monitoring tools like SHOW STATUS, SHOW INNODB STATUS, and OS tools can help evaluate performance and identify tuning opportunities.
- Memory allocation and settings like innodb_buffer_pool_size, key_buffer_size, query_cache_size are important to configure based on the workload and available memory.
Embedded Linux provides a standardized operating system solution for embedded systems through the Linux kernel. The Linux kernel abstracts the underlying hardware and provides drivers to interface with hardware peripherals. This allows application developers to focus on their code without needing to manage low-level hardware interactions. A bootloader initializes the hardware and loads the Linux kernel from memory. The kernel then loads and runs programs stored in the filesystem. Cross-compilers allow the same source code to target different processor architectures. Libraries and drivers help share code and resources across applications and hardware.
The document provides an overview of the Linux kernel, including its architecture, startup process, functionality, configuration, and compilation. It discusses the differences between micro and monolithic kernels. It also explains the Linux kernel architecture with user space and kernel space separated by a system call interface. Key aspects covered include process management, memory management, device management, and the kernel build system.
A teaching slide set describing the mechanisms of action and clinical use of local anaesthetics. This session is a basic introduction to the pharmacodynamics and pharmacokinetics of local anaesthetics. It is aimed at preclinical medical or dental students, or students in the early years of a pharmacology degree.
Memory management in operating system | Paging | Virtual memoryShivam Mitra
This document discusses memory management techniques in operating systems. It begins by covering contiguous memory allocation approaches like fixed and variable partitioning. It then discusses non-contiguous techniques like paging and segmentation. Key concepts covered include logical vs physical addresses, page tables, translation lookaside buffers, demand paging, and virtual memory. The document provides examples and links to detailed video explanations of these important OS memory management topics.
Processes and threads are fundamental concepts in Windows Vista. A process contains the virtual address space, threads, and resources for program execution. Each process has a process environment block (PEB) and can create multiple threads, each with their own thread environment block (TEB). Threads are the unit of CPU scheduling and each process must have at least one thread. Interprocess communication (IPC) allows processes to communicate and share data using various methods like pipes, mailslots, sockets, and shared memory. Synchronization objects like mutexes, events, and semaphores coordinate access to shared resources between threads.
Absorbed dose is the amount of energy imparted to matter by ionizing radiation. The larger the absorbed dose, the greater the hazard. Early limits on radiation exposure were based on preventing obvious effects like skin damage, but later considered delayed effects like cancer. The most radiation-sensitive organs are those with high cellular replication rates like bone marrow, lung and thyroid. As hazards were discovered, researchers published injuries and set early limits, with the first recommended dose of 10 rad per day in 1902. Monitoring devices like film badges and dosimeters now help detect radiation exposure and reduce risk through limiting time, increasing distance and improving shielding.
Linux kernel has a special feature called Watchdog timer which would reset the system in case of any software faults | system hangs | or any application crashes after a timeout has reached.
For Complete Learning: http://www.thelearnet.com/
Overview
Multithreading Models
Threading Issues
Pthreads
Solaris 2 Threads
Windows 2000 Threads
Linux Threads
Java Threads
- Photoacoustics involves using the photoacoustic effect to generate ultrasonic waves through the absorption of pulsed laser light in a material.
- The photoacoustic effect results from the thermoelastic expansion that occurs when optical energy is absorbed and converted to heat, resulting in localized pressure increases.
- The photoacoustic wave equation can be derived from the acoustic wave equation and the thermal diffusion equation, and has an inhomogeneous term representing the optical energy deposition.
- The general solution to the photoacoustic wave equation can be obtained using Green's function approaches and Cauchy contour integration in spherical coordinates to model the resulting spherical acoustic wave propagation.
The document discusses limitations of the Von Neumann architecture and opportunities for parallel processing through non-Von Neumann architectures. It provides background on the Von Neumann model and its key components. The document then explains how increasing demand for faster computers is driving researchers to explore parallel processing approaches like SIMD and MIMD that can solve larger, more complex problems by distributing work across multiple processors. It aims to describe non-Von Neumann architectures and parallel algorithms.
This document discusses calcium channel blockers (CCBs), including their mechanism of action, classification, therapeutic uses, adverse effects, and drug interactions. CCBs work by blocking calcium channels, leading to vasodilation and reduced heart rate and contractility. They are classified into dihydropyridines, benzothiazepines, and phenylalkylamines. Common uses include hypertension, angina, and arrhythmias. Side effects reflect their pharmacological actions and include flushing, constipation, and bradycardia. CCBs can interact dangerously with beta blockers due to combined cardio-depressant effects.
Here are some key things to practice with regular expressions using grep, sed and awk:
- Use grep to search files for patterns using basic regex metacharacters like ^, $, ., *, [, ], etc.
- Use sed for search and replace operations on files using regex to match patterns
- Use awk to parse files into fields based on delimiters like space, comma etc and perform operations on the fields stored in variables $1, $2 etc.
- Write simple awk scripts to print, filter and manipulate data from files. For example print certain fields, filter rows, perform calculations etc.
- Learn about more advanced regex features supported by tools like extended regex in grep, backreferences in sed etc
Local anesthetics are drugs that cause reversible loss of sensation, especially pain, in a restricted area of the body. They work by interfering with sodium channel function and action potential generation in neurons. Common local anesthetics include lidocaine, bupivacaine, and tetracaine. Adding vasoconstrictors like epinephrine prolongs the duration of anesthesia by slowing absorption from the injection site. Local anesthetics are used for surface anesthesia, injections, and spinal anesthesia. Adverse effects include central nervous system toxicity, cardiovascular effects, and allergic reactions at high doses.
The document discusses different types of computer systems and operating systems. It describes the main components of a computer system including hardware, operating system, application programs, and users. It then covers different types of operating systems such as mainframe systems, batch systems, time-sharing systems, desktop systems, parallel systems, distributed systems, real-time systems, and handheld systems. The document also discusses hardware protection mechanisms used by operating systems, including dual-mode operation, I/O protection, memory protection, and CPU protection.
Contiguous allocation is an early method for allocating main memory between the operating system and user processes. Each process is contained within a single contiguous section of memory. Relocation and limit registers are used to protect processes from each other and allow transient changes to the operating system. Multiple-partition allocation allocates variable sized partitions to processes out of memory holes. Storage allocation approaches like first-fit and best-fit are used to allocate partitions from the holes. Fragmentation can occur both externally and internally, taking up space that cannot be used.
Recent advances in ischemic heart diseasessaachslides15
This document provides an overview of ischemic heart diseases and recent advances in treatment. It discusses the types of angina pectoris and myocardial infarction. Current treatment strategies include nitrates, beta blockers, calcium channel blockers, and potassium channel openers. Newer agents that are discussed include trimetazidine, ranolazine, and ivabradine which lower cardiac oxygen demand and improve symptoms of angina. Combination therapies are also used to maximize antianginal effects while minimizing side effects.
MySQL High-Availability and Scale-Out architecturesFromDual GmbH
This document discusses MySQL high availability and scale-out architectures. It covers topics like scale-up vs scale-out, MySQL replication, high availability solutions, MySQL Cluster, application partitioning, and other architectural considerations. The document provides examples of MySQL replication configurations and cluster architectures to achieve goals like high availability, performance and scale-out. It also discusses emerging technologies like SSD that could disrupt database architectures.
The document provides best practices for performance tuning MySQL databases. It discusses benchmarking and profiling concepts, sources of performance problems like inefficient schemas and indexes, and SQL coding practices. It also recommends tuning server parameters and provides tools for benchmarking, profiling, and optimizing performance.
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)Aurimas Mikalauskas
Is my MySQL server configured properly? Should I run Community MySQL, MariaDB, Percona or WebScaleSQL? How many innodb buffer pool instances should I run? Why should I NOT use the query cache? How do I size the innodb log file size and what IS that innodb log anyway? All answers are inside.
Aurimas Mikalauskas is a former Percona performance consultant and architect currently writing and teaching at speedemy.com. He's been involved with MySQL since 1999, scaling and optimizing MySQL backed systems since 2004 for companies such as BBC, EngineYard, famous social networks and small shops like EstanteVirtual, Pine Cove and hundreds of others.
Additional content mentioned in the presentation can be found here: http://speedemy.com/17
This document discusses InnoDB locking mechanisms in MySQL, including:
- Row-level locks are used to prevent conflicts between transactions accessing the same rows.
- Different isolation levels determine which types of locks are used and what kinds of conflicts are allowed. Read committed is the default isolation level.
- Deadlocks can occur when multiple transactions try to acquire locks in incompatible orders. InnoDB detects and rolls back transactions involved in deadlocks.
This document provides an overview of performance tuning the MySQL server. It discusses where to find server configuration and status information, how to analyze what the database is doing using status variables, and which configuration variables can be tuned for optimization, including global, per-session, and storage engine variables. Key areas covered include memory usage, query analysis, indexing strategies, and tuning storage engines like InnoDB and MyISAM.
The document discusses tuning MySQL server settings for performance. Some key points covered include:
- Settings are workload-specific and depend on factors like storage engine, OS, hardware. Tuning involves getting a few settings right rather than maximizing all settings.
- Monitoring tools like SHOW STATUS, SHOW INNODB STATUS, and OS tools can help evaluate performance and identify tuning opportunities.
- Memory allocation and settings like innodb_buffer_pool_size, key_buffer_size, query_cache_size are important to configure based on the workload and available memory.
MySQL Webinar 2/4 Performance tuning, hardware, optimisationMark Swarbrick
This document summarizes a webinar on installing, configuring, and tuning MySQL for performance. It discusses hardware specifications for MySQL servers, setting up replication between a master and slave servers, and techniques for performance tuning. The webinar agenda covers hardware specifications, setting up replication, and performance tuning. It also provides an overview of MySQL support across various hardware platforms and operating systems.
In deploying MySQL, scale-out techniques can be used to scale out reads, but for scaling out writes, other techniques have to be used. To distribute writes over a cluster, it is necessary to shard the database and store the shards on separate servers. This session provides a brief introduction to traditional MySQL scale-out techniques in preparation for a discussion on the different sharding techniques that can be used with MySQL server and how they can be implemented with PHP. You will learn about static and dynamic sharding schemes, their advantages and drawbacks, techniques for locating and moving shards, and techniques for resharding.
The document provides an overview of how to summarize and interpret information from MySQL server status and variable outputs to understand server performance and optimize configuration. It explains that status variables show current server activity levels, while global and session variables display configuration settings. Comparing status outputs over time calculates rates like queries/second. Key metrics help identify bottlenecks like a small key buffer size if the key read cache miss rate is high.
9 DevOps Tips for Going in Production with Galera Cluster for MySQL - SlidesSeveralnines
Galera is a MySQL replication technology that can simplify the design of a high availability application stack. With a true multi-master MySQL setup, an application can now read and write from any database instance without worrying about master/slave roles, data integrity, slave lag or other drawbacks of asynchronous replication.
And that all sounds great until it’s time to go into production. Throw in a live migration from an existing database setup and devops life just got a bit more interesting ...
So if you are in devops, then this webinar is for you!
Operations is not so much about specific technologies, but about the techniques and tools you use to deploy and manage them. Monitoring, managing schema changes and pushing them in production, performance optimizations, configurations, version upgrades, backups; these are all aspects to consider – preferably before going live.
Let us guide you through 9 key tips to consider before taking Galera Cluster into production.
This document discusses MySQL performance tuning. It covers high availability solutions, MySQL system and status variables, memory usage, storage engines like InnoDB and MyISAM, monitoring tools, and the Performance Schema. Tuning levers include configuring variables related to caching, buffers, threads, and storage engines. Monitoring helps identify optimization opportunities.
The document discusses methods for sharding MySQL databases. It begins with an introduction to sharding and the different types of sharding methods. It then provides details on building a large database cluster using the Palomino Cluster Tool, which utilizes configuration management tools like Ansible, Chef and Puppet. The document concludes with a section on administering the large database cluster using the open source tool Jetpants.
Join-fu is the art of performance-tuning your application's SQL. Join Jay in a fun, irreverent look at the common ways application developers misuse and abuse their database.
MySQL Indexing - Best practices for MySQL 5.6MYXPLAIN
This document provides an overview of MySQL indexing best practices. It discusses the types of indexes in MySQL, how indexes work, and how to optimize queries through proper index selection and configuration. The presentation emphasizes understanding how MySQL utilizes indexes to speed up queries through techniques like lookups, sorting, avoiding full table scans, and join optimizations. It also covers new capabilities in MySQL 5.6 like index condition pushdown that provide more flexible index usage.
This document provides an overview of MySQL server performance tuning. It discusses laying the foundation for performance tuning by examining the server, OS, network and filesystem. It also covers examining current server settings and status variables, and tuning various aspects of MySQL like InnoDB, MyISAM, queries and session settings. The document aims to provide guidance on areas to optimize to improve MySQL server performance.
The technology has almost written off MySQL as a database for new fancy NoSQL databases like MongoDB and Cassandra or even Hadoop for aggregation. But MySQL has a lot to offer in terms of 'ACID'ity, performance and simplicity. For many use-cases MySQL works well. In this week's ShareThis workshop we discuss different tips & techniques to improve performance and extend the lifetime of your MySQL deployment.
Adrian Hardy's slides from PHPNW08
Once you have your query returning the correct results, speed becomes an important factor. Speed can either be an issue from the outset, or can creep in as your dataset grows. Understanding the EXPLAIN command is essential to helping you solve and even anticipate slow queries.
Associated video: http://blip.tv/file/1791781
Administering a Hadoop cluster isn't easy. Many Hadoop clusters suffer from Linux configuration problems that can negatively impact performance. With vast and sometimes confusing config/tuning options, it can can tempting (and scary) for a cluster administrator to make changes to Hadoop when cluster performance isn't as expected. Learn how to improve Hadoop cluster performance and eliminate common problem areas, applicable across use cases, using a handful of simple Linux configuration changes.
MySQL Optimization from a Developer's point of viewSachin Khosla
Optimization from a developer's point of view. Optimization is not only the duty of a DBA but its should be done by all those who are involved in the ecosystem
The document discusses best practices for running MySQL on Linux, covering choices for Linux distributions, hardware recommendations including using solid state drives, OS configuration such as tuning the filesystem and IO scheduler, and MySQL installation and configuration options. It provides guidance on topics like virtualization, networking, and MySQL variants to help ensure successful and high performance deployment of MySQL on Linux.
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...PostgreSQL-Consulting
Input-output performance problems are on every day agenda for DBAs since the databases exist. Volume of data grows rapidly and you need to get your data fast from the disk and moreover - fast to the disk. For most databases there is a more or less easy to find checklist of recommended Linux settings to maximize IO throughput. In most cases that checklist is good enough. But it is always better to understand how it works, especially if you run into some corner-cases. This talk is about how IO in Linux works, how database pages travel from disk level to database own shared memory and back and what kind of mechanisms exist to control this. We will discuss memory structures, swap and page-out daemons, filesystems, schedulers and IO methods. Some fundamental differences in IO approaches between PostgreSQL, Oracle and MySQL will be covered
The document discusses best practices for deploying MongoDB including sizing hardware with sufficient memory, CPU and I/O; using an appropriate operating system and filesystem; installing and upgrading MongoDB; ensuring durability with replication and backups; implementing security, monitoring performance with tools, and considerations for deploying on Amazon EC2.
This document discusses using Hadoop and Elasticsearch for real-time analytics. It provides an overview of Elasticsearch, including how it is document-oriented, schema-free, distributed and fast. It also demonstrates indexing, retrieving, updating and deleting documents from Elasticsearch. The demo portion involves extracting data from a SQL database using Hive, transforming it with Hadoop/Hive, and loading it into Elasticsearch to run queries. Lessons learned focus on concurrency, filtering, field data caching and JVM memory usage.
Managing Security At 1M Events a Second using ElasticsearchJoe Alex
The document discusses managing security events at scale using Elasticsearch. Some key points:
- The author manages security logs for customers, collecting, correlating, storing, indexing, analyzing, and monitoring over 1 million events per second.
- Before Elasticsearch, traditional databases couldn't scale to billions of logs, searches took days, and advanced analytics weren't possible. Elasticsearch allows customers to access and search logs in real-time and perform analytics.
- Their largest Elasticsearch cluster has 128 nodes indexing over 20 billion documents per day totaling 800 billion documents. They use Hadoop for long term storage and Spark and Kafka for real-time analytics.
This document discusses various MySQL consulting issues including:
1) Locality of reference and how data storage and retrieval patterns impact performance. Two examples of datasets with non-sequential access patterns are described.
2) The commit_demo.pl script which can test transaction performance on different storage configurations. Graphs of runtime vs transactions/commit are shown.
3) An overview of InnoDB status variables and what they indicate about the InnoDB buffer pool, logs, pages, rows and other areas. RAM disks are also discussed as a solution for temporary tables that exceed in-memory limits.
The document discusses key-value stores as options for scaling the backend of a Facebook game. It describes Redis, Cassandra, and Membase and evaluates them as potential solutions. Redis is selected for its simplicity and ability to handle the expected write-heavy workload using just one or two servers initially. The game has since launched and is performing well with the Redis implementation.
The document provides an overview of how to perform basic troubleshooting and tuning for MySQL performance issues as a "5 Minute DBA". It recommends first checking server versions, hardware resources like CPU, memory, and I/O using tools like sar, iostat, vmstat, and top. If issues persist, it suggests reviewing configuration settings and database queries/indexes before further optimizing the database. The document stresses that quick fixes may not always be optimal and assumes a Linux/Unix environment.
- The document provides guidance on deploying MongoDB including sizing hardware, installing and upgrading MongoDB, configuration considerations for EC2, security, backups, durability, scaling out, and monitoring. Key aspects discussed are profiling and indexing queries for performance, allocating sufficient memory, CPU and disk I/O, using 64-bit OSes, ext4/XFS filesystems, upgrading to even version numbers, and replicating for high availability and backups.
Linux performance tuning & stabilization tips (mysqlconf2010)Yoshinori Matsunobu
This document provides tips for optimizing Linux performance and stability when running MySQL. It discusses managing memory and swap space, including keeping hot application data cached in RAM. Direct I/O is recommended over buffered I/O to fully utilize memory. The document warns against allocating too much memory or disabling swap completely, as this could trigger the out-of-memory killer to crash processes. Backup operations are noted as a potential cause of swapping, and adjusting swappiness is suggested.
This document summarizes Dan van der Ster's experience scaling Ceph at CERN. CERN uses Ceph as the backend storage for OpenStack volumes and images, with plans to also use it for physics data archival and analysis. The 3PB Ceph cluster consists of 47 disk servers and 1,128 OSDs. Some lessons learned include managing latency, handling many objects, tuning CRUSH, trusting clients, and avoiding human errors when managing such a large cluster.
Haytham ElFadeel presented on next-generation storage systems and key-value stores. He began with an overview of scalable systems and the need for both vertical and horizontal scalability. He discussed the limitations of traditional databases in scaling, including complexity, wasted features, and multi-step query processing. Key-value stores were presented as an alternative, offering simple interfaces and designs optimized for scaling across hundreds of machines. Performance comparisons showed key-value stores significantly outperforming databases. Systems discussed included Amazon Dynamo, Facebook Cassandra, and Redis.
Top 10 lessons learned from deploying hadoop in a private cloudRogue Wave Software
This document discusses lessons learned from deploying Hadoop in a private cloud. Some key lessons include: choosing the right hardware with sufficient CPU, RAM, and bandwidth; understanding that configuration is critical for Hadoop, HBase, and Solr; expecting failures and bugs to occur; and realizing that big data projects take a long time to complete. Public clouds are expensive for long-term big data storage needs, so a private cloud may be more cost effective despite requiring infrastructure management. Open source tools like Hadoop have advanced to enable organizations to tackle "big data" challenges.
The document provides guidance on deploying MongoDB in production environments. It discusses sizing hardware requirements for memory, CPU, and disk I/O. It also covers installing and upgrading MongoDB, considerations for cloud platforms like EC2, security, backups, durability, scaling out, and monitoring. The focus is on performance optimization and ensuring data integrity and high availability.
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?DATAVERSITY
Wordnik migrated from a MySQL relational database to the non-relational MongoDB database for 5 key reasons: speed, stability, scaling, simplicity, and fitting their object model better. They tested MongoDB extensively, iteratively improving their data mapping and access patterns. The migration was done without downtime by switching between the databases. While inserts were much faster in MongoDB, updates could be slow due to disk I/O. Wordnik addressed this through optimizations like pre-fetching on updates and moving to local storage. Overall, MongoDB was a better fit for Wordnik's large and evolving datasets.
OSDC 2016 - Tuning Linux for your Database by Colin CharlesNETWAYS
Many operations folk know that performance varies depending on using one of the many Linux filesystems like EXT4 or XFS. They also know of the schedulers available, they see the OOM killer coming and more. However, appropriate configuration is necessary when you're running your databases at scale.
Learn best practices for Linux performance tuning for MariaDB/MySQL (where MyISAM uses the operating system cache, and InnoDB maintains its own aggressive buffer pool), as well as PostgreSQL and MongoDB (more dependent on the operating system). Topics that will be covered include: filesystems, swap and memory management, I/O scheduler settings, using and understanding the tools available (like iostat/vmstat/etc), practical kernel configuration, profiling your database, and using RAID and LVM.
There is a focus on bare metal as well as configuring your cloud instances in.
Learn from practical examples from the trenches.
Network support for resource disaggregation in next-generation datacentersSangjin Han
Presented at the 12th ACM Workshop on Hot Topics in Networks (HotNets XII)
Datacenters have traditionally been a collection of individual servers, each of which aggregates a fixed amount of compute, memory, storage, and network resources as an independent physical entity. Extrapolating from recent trends, we envisage that future datacenters will be architected in a drastically different manner: all computational resources within a server will be disaggregated into standalone blades, and the datacenter network will directly interconnect them. This is what we call "Disaggregated Datacenters".
This presentation briefly sketches why and how this transition will happen. In particular, we focus on the role of network fabric in such disaggregated datacenters, as it will face a set of challenges brought by the new datacenter architecture.
The paper can be found here: http://www.eecs.berkeley.edu/~sangjin/static/pub/hotnets2013_ddc.pdf
This document provides guidance on considerations for deploying MongoDB in production environments. It covers sizing hardware requirements for memory, CPU and I/O; installing and configuring MongoDB; using MongoDB on Amazon EC2; implementing security, backups, and durability; upgrading MongoDB versions; scaling deployments by sharding; and monitoring MongoDB performance.
MariaDB/MySQL pitfalls - And how to come out again...FromDual GmbH
During the last conferences the audience was asking for more war stories than just new features.
In this presentation we have a look at the most often seen problems as a MariaDB/MySQL consultant in field.
MariaDB / MySQL tripping hazard and how to get out again?FromDual GmbH
The document discusses common pitfalls and mistakes when using MariaDB/MySQL databases and how to avoid or recover from them, including issues related to different versions and forks of MariaDB and MySQL not being fully compatible, keeping implementations simple to avoid unnecessary complexity, and problems that can occur from table locking, disk space usage, and other operational concerns.
In this presentation we discuss the New Features of MariaDB 10.4. First we give a short overview of the MariaDB Branches and Forks. Then we talk about the announced IPO. Technically we cover topics like Authentication, Accounts, InnoDB, Optimizer improvements, Application-Time Period Tables the new Backup Stage Galera 4 and other changes...
MariaDB 10.4 became General Available (GA = ready for production) this summer. So it is time to look at the new Features in MariaDB 10.4. After a short intro about history we look for the reason of broad usage of MariaDB nowadays. Most important improvements where in User Authentication, InnoDB improvements, and Optimizer enhancements. A completely New Feature is Application-Time Period Tables. Backup got a new Locking behaviour so LVM snapshots are possible and officially supported now. And last but not least MariaDB 10.4 comes with Galera 4.
MariaDB 10.2 New Features for Developers, Administrators and DevOps. Window Functions, Common Table Expressions, Check Constraints, GeoJSON, GIS, JSON, Oracle compatibility and MariaDB Connectors
Der Datenbank-Backup ist gemacht - was nun?FromDual GmbH
* Datenbank-Backup – welcher Zweck?
* Tauglichkeit des Backup, Verifikation
* Echtdaten vollständig nutzen
* Dem Datenschutz genügen
* Material für die Entwicklung
* Automatisierung
Weltweite Produktionsdatenverwaltung mit MySQL-ReplikationFromDual GmbH
Weltweite Produktionsdatenverwaltung mit MySQL
* Ausgangslage
* Probleme die sich stellen
* Wer darf welche Daten kriegen?
* Wie werden Daten verteilt?
* Produktionsdaten zurück?
* MySQL multi-Source Replikation
* Bertriebsverantwortung
* Hochverfügbarkeit
* Sensitive Daten
* Wer darf welche Daten sehen?
* MySQL Row Filterung
* Nachträgliche Forderungen
MySQL Performance Tuning für Oracle-DBA'sFromDual GmbH
MySQL Performance Tuning
* Was ist Performance?
* Was kostet Performance?
* Tuning Massnahmen
* MySQL Konfiguration
* Wo schauen?
* Langsame Abfragen finden
* Optimiere das Query!
* Monitoring
Reading MySQL fingerprints
Every MySQL Database has its typical fingerprints. They can be seen with SHOW GLOBAL STATUS;
How you interprete them you will find in this presentation.
This document provides an overview of Galera Cluster for MySQL. It discusses how Galera Cluster allows for highly available, multi-master replication across multiple nodes through synchronous replication. It also covers topics like installation, configuration, operations including rolling restarts, load balancing, and catching node state changes. The document includes demonstrations of these concepts.
This document discusses high availability with Galera Cluster for MySQL. It provides an overview of Galera Cluster, including its advantages like active-active multi-master clusters and automatic node membership control. It also covers Galera Cluster installation, configuration, operations like initial cluster start and rolling restarts, load balancing, and online schema upgrades.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
OpenID AuthZEN Interop Read Out - AuthorizationDavid Brossard
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfTechgropse Pvt.Ltd.
In this blog post, we'll delve into the intersection of AI and app development in Saudi Arabia, focusing on the food delivery sector. We'll explore how AI is revolutionizing the way Saudi consumers order food, how restaurants manage their operations, and how delivery partners navigate the bustling streets of cities like Riyadh, Jeddah, and Dammam. Through real-world case studies, we'll showcase how leading Saudi food delivery apps are leveraging AI to redefine convenience, personalization, and efficiency.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Choosing The Best AWS Service For Your Website + API.pptx
MySQL Performance Tuning
1. MySQL Performance Tuning
A practical guide
Oli Sennhauser
Senior Consultant
osennhauser@mysql.com
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 1
2. Introduction
• Who we are?
• What we want?
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 2
3. Table of Contents
• Find the problem
• MySQL architecture
• Database settings
• Detect and eliminate slow queries
• Table tuning
• Application tuning
• Alternatives
• Prevention
• Dirty tricks and other stuff
• Now it's up to you...
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 3
4. DBA: We have a problem!
• What does performance mean to you?
• How does it look like?
– DB is (suddenly!?) slow.
– No historical data (or not the one we need).
– “Screw something on the DB!”
– We are short before going life and much too slow!!!
• We have a problem. And what now?
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 4
5. Efficiency of tuning measurements
• Application/Architecture/Design
➳ No way! For what ever reason :-(
• So on the DBA side: Memory, DB settings, I/O,
Indexes, etc.
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 5
6. Find the problem / the bottleneck
• No history data!?! :-(
• Best if:
– you can simulate it
– it happens predictable and/or periodically
• Your friends are:
– vmstat / dstat
– iostat
– top
– any graphical history of values
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 6
7. Tuning means ...
• The tuning life cycle:
• Only one change at a time
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 7
8. Measure
• Find the bottleneck / limiting resource:
– I/O
– Memory
– CPU
– Network bandwidth
• But how?
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 8
9. Measure I/O
• vmstat
# vmstat 1
procs swap io cpu
r b si so bi bo us sy id wa
0 0 3 3 94 143 21 21 56 2
0 0 0 0 0 4 9 37 54 0
• iostat (--> sysstat package)
# iostat x 1
avgcpu: %user %nice %system %iowait %steal %idle
5.88 0.00 34.31 2.94 0.00 56.86
Device: r/s w/s rkB/s wkB/s await svctm %util
hda 0.00 0.00 0.00 0.00 0.00 0.00 0.00
hdc 0.00 2.94 0.00 23.53 14.67 12.00 3.53
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 9
10. Measure memory
• ps
# ps eo user,pid,%cpu,%mem,vsz,rsz,comm sort vsz |
egrep 'mysql|COMMAND'
USER PID %CPU %MEM VSZ RSZ COMMAND
mysql 1361 0.0 1.5 108368 16444 mysqld
mysql 1210 0.0 0.1 4536 1956 bash
mysql 1289 0.0 0.1 4060 1444 safe_mysqld
mysql 1204 0.0 0.1 4048 1404 su
• free / top:
#free
total used free shared buffers cached
Mem: 1036016 983864 52152 0 35484 547432
/+ buffers/cache: 400948 635068
swap: 4202112 96148 4105964
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 10
11. Measure CPU
• top
• vmstat
• dstat
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 11
13. Think
• I/O
– Who does it?
– Is it read or write?
– Is it random I/O or sequential I/O?
• Memory
– Easy to find!
– DB sizing
– Is it somebody else?
• CPU
– Easy to find!
– Who is “burning” CPU?
• Network bandwidth
– Who does it?
– Sniff traffic?
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 13
14. Change
• What could be changed?
• Hardware -> I/O system (RAID5), RAM, CPU, NW
• O/S -> do not touch (kernel upgrade)
• DB -> my.cnf
• Application -> Queries!!!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 14
15. Change Hardware
• More RAM helps more!!!
• Faster CPU if it is the bottleneck (not more!)
• More expensive I/O system:
– RAID5 is bad for databases!!!
– RAID10 is good.
– Many spindles
– Battery buffered I/O system cache???
• 1 Gbit Network?
• Forget about virtualization (VMware etc.)!!!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 15
16. Change O/S
• Use mainstream O/S -> for MySQL this means
SLES/RHEL!
• Use 64-bit architecture (> 4 GB RAM)
• Use recent kernel (>= 2.6.12)
• Use mainstream file system -> ext3 and xfs
• Take what you are most familiar with!
--> But on O/S you cannot change much. They are
already optimal! :-)
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 16
18. Change MySQL: Performance Features
• The magic of caching: “Do as little work as
possible: Be lazy!!!”
• Performance features:
– Thread cache
– Query cache
– Prepared statements
– Stored Procedures (see “the SP trap!”)
– delayed INSERT (MyISAM only)
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 18
19. Change MySQL: database settings
• “The big 3!”
– key_buffer_size
– innodb_buffer_pool_size
– innodb_log_file_size
• Some others: query_cache_size, thread_cache_size
• My approach:
– use defaults (or templates)
– add: “the big 3” + 2 (see above)
– do NOT change except it was proved and measured
that is useful!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 19
20. Change MySQL
• Where to change?
– my.cnf (Caution: many possible places!!!)
• Where to measure?
– SHOW /*!50000 GLOBAL */ STATUS;
• Where to cheat?
– http://dev.mysql.com/doc/refman/5.0/en/index.html
– 5.2.3. System Variables
– 5.2.5. Status Variables
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 20
21. The big 3
• MyISAM
key_buffer_size = 2533% of RAM
Key_blocks_unused > actual value
Key_blocks_used > high water mark
Key_read_requests / Key_reads > >= 99% ideally
• InnoDB
innodb_buffer_pool_size = 80% of RAM
Innodb_buffer_pool_pages_free
Innodb_buffer_pool_read_requests /
Innodb_buffer_pool_reads > >= 99% ideally
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 21
22. The big 3
• InnoDB
innodb_log_file_size = 32 128 Mbyte
Innodb_os_log_pending_fsyncs > ???
> hiccups!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 22
23. Query cache & thread cache
• Query cache
query_cache_size = 32 128 Mbyte (caution: 512!)
Qcache_total_blocks
Qcache_free_blocks
Qcache_free_memory > Fragmentation
Qcache_hits
Qcache_inserts > Hit ratio, ideally >> 2 : 1
Qcache_lowmem_prunes > too small or too fragmented
• Thread cache
thread_cache_size = 8 128
Threads_cached
Threads_created > should not grow much over time
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 23
24. Some more...
• That's it! :-)
• Avoid any kind of I/O: logging!
sync_binlog > 0 !!!
#log > Not on production!!!
#log_bin > Where we do NOT need it!!!
log_slow_queries > is OK, we do not have such :)
• Try to avoid sync writing:
innodb_flush_log_at_trx_commit = 2
➔ Simulates MyISAM behaviour for InnoDB. But caution!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 24
25. Some more...
• Table cache
table_cache = 64 2048
Open_tables > <= table_cache
Opened_tables > should change moderately
• Other InnoDB settings:
innodb_additional_mem_pool_size
➔ Do NOT change this! > 20 Mbyte is non sense!
innodb_flush_method
➔ Sometimes O_DIRECT or O_DSYNC can help. But test
before!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 25
26. Change Application!
• Transaction log and binlog cache:
Binlog_cache_disk_use > increase binlog_cache_size
Innodb_log_waits > increase innodb_log_buffer_size
➔ Too big transactions???
• Temporary results:
max_heap_table_size = 16 256 Mbyte
tmp_table_size = 32 512 Mbyte
Created_tmp_disk_tables > changes often
➔ Too big temporary results?
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 26
27. Change Application!
• Sort buffer:
sort_buffer_size = 2 16 Mbyte
Sort_merge_passes > sort buffer too small
➔ Too big sorts???
● Application or Network problems:
Aborted_clients
Aborted_connects
• Network traffic:
Bytes_received
Bytes_sent
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 27
28. Change Application!
• Locking:
Table_locks_immediate
Table_locks_waited
➔ Too high concurrency or too slow queries! -> Optimize
queries or try InnoDB.
Innodb_row_lock_current_waits
Innodb_row_lock_time
Innodb_row_lock_time_avg
Innodb_row_lock_time_max
Innodb_row_lock_waits
➔ InnoDB locking! Optimize queries or think about
changing the application.
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 28
29. Change Application!
• Missing Indexes:
Select_full_join
Select_range_check > should both be zero!!!
➔ Missing Index!
• Full-Table-Scan:
Select_scan
Handler_read_rnd_next
Sort_scan
➔ Find the queries! :-)
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 29
30. Find the slow queries!
• Quick:
SHOW [FULL] PROCESSLIST;
• Proper: Enable the slow query log!
# my.cnf
log_slow_queries = slow_query.log
long_query_time = 1
log_queries_not_using_indexes = 1
➔ And now??? Thousands of queries!!!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 30
31. Find the slow queries!
• Profile the slow query log:
# mysqldumpslow s t slowquery.log > slow_query.profile
• That's how the profile looks like:
Count: 4498 Time=212.72s (956824s) Lock=0.04s (198s) Rows=0.0 (0)
create table TMP.SS_temp2_36 select l.initlot,s.lot,s.wafer,s.x,s.y,
Count: 810 Time=121.74s (98610s) Lock=0.30s (245s) Rows=0.0 (0)
insert into TOD.row_descr select l.initlot,w.lot,w.wafer,'S' dataset,'S'
Count: 477 Time=149.99s (71547s) Lock=0.01s (4s) Rows=2.7 (1284)
SELECT l.lot,count(h.MFG_STEP_NAME) cnt FROM DB1.lot_7000 l left join
Count: 92 Time=573.43s (52756s) Lock=0.00s (0s) Rows=325.6 (29958)
SELECT ps.X, ps.Y, SUM(N*ps.PARVALUE)/COUNT(ps.PARVALUE) PARMEAN FROM
➔ Start working now! EXPLAIN ...
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 31
32. MySQL EXPLAIN
• Generate an execution plan:
EXPLAIN
SELECT i.number, l.answer
FROM poll_item i
JOIN poll_item_l l ON (l.poll_id = i.poll_id
AND l.number = i.number)
WHERE i.poll_id = '4'
AND l.language_id = '2'
ORDER BY i.number ASC;
+++++++++++
| id | select | tab | type | pos_keys | key | k_len | ref | rows | Extra |
+++++++++++
| 1 | SIMPLE | i | ref | PRIMARY | PRIMARY | 2 | const | 5 | Using where; Using index |
| 1 | SIMPLE | l | eq_ref | PRIMARY | PRIMARY | 5 | const,... | 1 | Using where |
+++++++++++
• Rewrite DML into SELECT.
• Be cautious with Subqueries! They are executed!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 32
33. MySQL visual explain
• http://mysqltoolkit.sourceforge.net/
./mysqlvisualexplain test.exp
JOIN
+ Filter with WHERE
| + Bookmark lookup
| + Table
| | table l
| | possible_keys PRIMARY
| + Unique index lookup
| key l>PRIMARY
| possible_keys PRIMARY
| key_len 5
| ref const,topodb.i.number,const
| rows 1
+ Filter with WHERE
+ Index lookup
key i>PRIMARY
possible_keys PRIMARY
key_len 2
ref const
rows 5
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 33
34. Table tuning
• Indexing
➔ See above.
➔ What should be indexed and how?
• Data type tuning
● mysqldump –alldatabases nodata
• Table design
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 34
35. Table tuning – Indexing
• What should be indexed?
– All attributes where you JOIN
– All attributes where you filter (WHERE)
– All attributes where you ORDER or GROUP BY
– All attributes where you want to do an Index Scan
instead of a Table scan.
– NOT on attributes with an evenly distributed low
cardinality.
• How should be indexed?
– Indexes can only be used from left to right.
– Keep them short.
– Compound indexes: INDEX(a, b).
– Prefixed indexes: INDEX(a, b(10)).
– Do not function-cover indexed attributes
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 35
36. Table tuning – data type tuning
• Idea behind: Increase the data density!
• How to get: mysqldump --no-data
CREATE TABLE betatesters (
user_id bigint(20) NOT NULL,
nick varchar(255) NOT NULL,
registerdate varchar(30) NOT NULL,
daysregistered int(11) NOT NULL,
value double default NULL,
timestamp_data bigint(15) default NULL,
ip varchar(16) default NULL
PRIMARY KEY (`nick`),
UNIQUE KEY user_id (`user_id`)
KEY (`user_id`, `nick`)
) DEFAULT CHARSET=utf8;
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 36
37. Table tuning – table design
• Normalization versus de-normalization
– Joins are expensive --> CPU
– Denormalized is big --> high redundancy --> RAM -->
Disk --> Slow
➔ Find the trade-off!
➔ Bring everything in 3rd NF --> then start denormalizing
if necessary.
• vertical and horizontal partitioning:
split for static and dynamic data
split for
example
by time
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 37
38. Locality of Reference
• In theory: We should not care how data are
stored internally.
• In practice: It is sometimes good to know!
• Why?
• 2 examples from the last 9 months:
– wind mills
– vehicle tracking for parcel delivery
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 38
39. Example 1
• Several 100 wind mills
• 50 measured values per
wind mill
• Every 5-15 minutes
• Up to 10 years
• Dozens of GB of data
• Record size up to 2k!
• Search pattern: Give me
value x from wind mill
#13 in this time range!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 39
40. Example 2
• Several 100 vehicles
• 24 h/d
• Every 2 min position
• Status/position per
vehicle, later per
parcel!!!
• Dozens of GB of data
• Record size 400 bytes
• Search pattern: Give
me all positions of
vehicle #13 from the
last 24 hours.
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 40
41. Locality of Reference
• These 2 examples have one behaviour in
common:
• Delivery of data is completely different than
search pattern.
– Usually data are delivered sorted by time and also
(more or less) retrieved by time.
– In this cases time has a secondary influence!
• But what happens???
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 41
42. Locality of Reference
• Block size is 16k/4k
• PK is AUTO_INCREMENT
• Synthetical PK are sometimes dangerous!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 42
43. Locality of Reference
• What to do???
➔ PK on (vehicle_id, ts) for example or
➔ PK on (windmill_id, data, ts)
➔ Can be up to 100 times more efficient (not
necessarily faster)
• What about MyISAM?
• What about Falcon? (Mail from Ann can be
provided).
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 43
44. Change Application
• Where are we now?
• What else can we do?
➔ Avoid – reduce – optimize
● Do not!
➔ Put more intelligence into your application!
● Reduce!
➔ Do only once. Cache!
● Do it better!
➔ Tune the statement, tune the code, tune the logic!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 44
45. Change Application
• Clean up first, before you invest into new
hardware or even a redesign.
– New hardware brings maybe a factor of 2x
– Clean up can bring factors up to 10x
– Sometimes new hardware is cheaper :-(
• Read issues are a caching problem.
➔ Try to cache!
• Write issues are a batching problem.
➔ Try to batch!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 45
46. commit_demo.pl
20
15
run time [s]
normal
10 bin_log
5
0
1 2 4 8 16 32 64 128 256
INSERT/COMMIT
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 46
47. Alternatives when exhausted
• See this afternoon! :-)
• Change architecture.
– Scale-Out?
• Tricks like Materialized Views?
• Application partitioning
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 47
48. Prevention
• What can we do to
prevent Performance
problems?
– Do load testing.
– Do benchmarking.
– Collect historical data
and make predictions.
• An then: Measure and
monitor...
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 48
49. Measure
• top, vmstat, iostat,
dstat, mstat, free, ...
• mytop, innotop, mtop
• Nagios, MySQL AR,
MySQL Administrator,
Cacti, MRTG, RRD,
Munin, Moodds, Big
Sister, MySQLStat,
Zabbix, Hobbit, Monit,
...
http://www.shinguz.ch/MySQL/mysql_monitoring.html
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 49
51. RAM disks (I)
• ORDER BY, GROUP BY, DISTINCT --> temp tables
– bigger than:
tmp_table_size = 32M
max_heap_table_size = 16M
– BLOB/TEXT
• Will be written into:
tmpdir = /tmp/
• Can be seen in:
Created_tmp_disk_tables 0
Created_tmp_tables 20
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 51
52. RAM disk (II)
• Both counters are increased!
• Solutions?
– Change your statement/requirements
– Optimize your Query
– Reduce size of result set
– Avoid BLOB/TEXT
• And if you cannot?
--> Use a RAM disk!
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 52
53. RAM disk (III)
• RAM disk is a disk in RAM :-) --> So you need
much RAM (8 Gbyte on 32-bit systems?)!
• Can use your SWAP (we do not want that)!
• More info:
/usr/src/linux/Documentation/filesystems
# cat /proc/filesystems
# mount tmpfs t tmpfs /mnt o size=100m
# mount
• Bug in 5.0.4x!!! :-(
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 53
54. Now it's up to you...
• Output of: SHOW /*!50000 GLOBAL */ STATUS;
• Output of: SHOW GLOBAL VARIABLES;
• Slow query log.
• Slow queries
• Execution plans (EXPLAIN SELECT ...)
• Output of "vmstat 1" during peak time.
Copyright 2007 Oli Sennhauser The World’s Most Popular Open Source Database 54