There are many poorly designed database application in use today. There are several reasons including lack of expertise, or lack of experience in MySQL specifically.
This presentation will aim to educate an attendee for the top 20 things every database design should have. As a consultant I see these common mistakes regularly, some of which can easily be corrected without any code changes.
Some topics include:
Why is normalization important? When do you optimize your normalized schema? Why NOT NULL is important? Why VARCHAR is bad? Disk = Memory = Performance? MySQL has 9 different Numeric data types, Oracle has 1. Why? Think transactions, think strict data integrity. Leverage the power of covering indexes. Query Cache, friend or enemy.
Every website wants to become successful. Few websites however undertake the basic and fundamental steps to build a rock solid foundation to ensure a scalable
Designing an extensible, flexible schema that supports user customization is a common requirement, but it's easy to paint yourself into a corner.
Examples of extensible database requirements:
- A database that allows users to declare new fields on demand.
- Or an e-commerce catalog with many products, each with distinct attributes.
- Or a content management platform that supports extensions for custom data.
The solutions we use to meet these requirements is overly complex and the performance is terrible. How should we find the right balance between schema and schemaless database design?
I'll briefly cover the disadvantages of Entity-Attribute-Value (EAV), a problematic design that's an example of the antipattern called the Inner-Platform Effect, That is, modeling an attribute-management system on top of the RDBMS architecture, which already provides attributes through columns, data types, and constraints.
Then we'll discuss the pros and cons of alternative data modeling patterns, with respect to developer productivity, data integrity, storage efficiency and query performance, and ease of extensibility.
- Class Table Inheritance
- Serialized BLOB
- Inverted Indexing
Finally we'll show tools like pt-online-schema-change and new features of MySQL 5.6 that take the pain out of schema modifications.
Group Replication went Generally Available end of 2016, it introduces a 'synchronous' active:active multi-master eplication, in addition to asynchronous and semi-synchronous replication, the latter 2 being available in in MySQL for longtime.
As with any new feature, and especially with introducing active:active multi-master replication, it takes a while before companies are adopting the software in production database environment.
For example, even though MySQL 5.7 has been GA for more than a year, adoption is only starting to increase recently.
We can, and should, expect the same from Group Replication. As with every release, bugs will be found, and with new features, best practises still need to formed out of practical experience.
After giving a short introduction on what Group Replication is, I will cover my experience so far in evaluating Group Replication.
As a developer, it is important to understand MySQL storage engines and how they can impact performance. The key factors to consider include the type of data being stored, concurrency needs, and requirements for transactions. The storage engine chosen affects aspects like locking granularity, indexing support, and performance for queries, inserts, and updates. Explain statements help analyze query execution plans and identify opportunities to improve performance through proper indexing.
These are the *updated* slides (InnoDB clusters and MySQL Enterprise Monitor 3.4 are now GA) from the following webinar, which you can now watch on demand:
https://www.mysql.com/news-and-events/web-seminars/why-mysql-high-availability-matters/
-----------------------------------------------------
MySQL high availability matters because your data matters. If your database goes down, whether due to human error, catastrophic network failure, or planned maintenance, the accessibility and accuracy of your data can be compromised with disastrous results. We'll examine the critical elements of a high availability solution, including:
- Data redundancy
- Data consistency
- Automatic fault detection and resolution
- No single point of failure
And how you can achieve these things more easily than ever before using MySQL's new native HA solution.
Every website wants to become successful. Few websites however undertake the basic and fundamental steps to build a rock solid foundation to ensure a scalable
Designing an extensible, flexible schema that supports user customization is a common requirement, but it's easy to paint yourself into a corner.
Examples of extensible database requirements:
- A database that allows users to declare new fields on demand.
- Or an e-commerce catalog with many products, each with distinct attributes.
- Or a content management platform that supports extensions for custom data.
The solutions we use to meet these requirements is overly complex and the performance is terrible. How should we find the right balance between schema and schemaless database design?
I'll briefly cover the disadvantages of Entity-Attribute-Value (EAV), a problematic design that's an example of the antipattern called the Inner-Platform Effect, That is, modeling an attribute-management system on top of the RDBMS architecture, which already provides attributes through columns, data types, and constraints.
Then we'll discuss the pros and cons of alternative data modeling patterns, with respect to developer productivity, data integrity, storage efficiency and query performance, and ease of extensibility.
- Class Table Inheritance
- Serialized BLOB
- Inverted Indexing
Finally we'll show tools like pt-online-schema-change and new features of MySQL 5.6 that take the pain out of schema modifications.
Group Replication went Generally Available end of 2016, it introduces a 'synchronous' active:active multi-master eplication, in addition to asynchronous and semi-synchronous replication, the latter 2 being available in in MySQL for longtime.
As with any new feature, and especially with introducing active:active multi-master replication, it takes a while before companies are adopting the software in production database environment.
For example, even though MySQL 5.7 has been GA for more than a year, adoption is only starting to increase recently.
We can, and should, expect the same from Group Replication. As with every release, bugs will be found, and with new features, best practises still need to formed out of practical experience.
After giving a short introduction on what Group Replication is, I will cover my experience so far in evaluating Group Replication.
As a developer, it is important to understand MySQL storage engines and how they can impact performance. The key factors to consider include the type of data being stored, concurrency needs, and requirements for transactions. The storage engine chosen affects aspects like locking granularity, indexing support, and performance for queries, inserts, and updates. Explain statements help analyze query execution plans and identify opportunities to improve performance through proper indexing.
These are the *updated* slides (InnoDB clusters and MySQL Enterprise Monitor 3.4 are now GA) from the following webinar, which you can now watch on demand:
https://www.mysql.com/news-and-events/web-seminars/why-mysql-high-availability-matters/
-----------------------------------------------------
MySQL high availability matters because your data matters. If your database goes down, whether due to human error, catastrophic network failure, or planned maintenance, the accessibility and accuracy of your data can be compromised with disastrous results. We'll examine the critical elements of a high availability solution, including:
- Data redundancy
- Data consistency
- Automatic fault detection and resolution
- No single point of failure
And how you can achieve these things more easily than ever before using MySQL's new native HA solution.
Inno db internals innodb file formats and source code structurezhaolinjnu
This document discusses the goals, architecture, and on-disk format of the InnoDB storage engine for MySQL. InnoDB aims to provide transaction support, reliability, and scalability. Its architecture includes buffering, locking, recovery, and efficient I/O mechanisms. The on-disk format is designed for durability, performance through techniques like compression, and compatibility through file format management. Source code is organized into subdirectories corresponding to major subsystems.
A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.
A video recording of this presentation is available from Ustream: http://ustre.am/fyLk
MySQL Group Replication - HandsOn TutorialKenny Gryp
Group Replication is a plugin for MySQL that provides multi-master replication. It works by having each node send write transactions to other nodes through a group communication system. The writes are certified locally in an asynchronous manner to ensure total order of transactions across all nodes. Group Replication uses optimistic locking where local locks are released right after commit, and conflict detection happens during certification rather than at the start of transactions.
MySQL Replication Performance Tuning for Fun and Profit!Vitor Oliveira
MySQL Replication, in addition to bringing high-availability, is the foundation to build high-performance MySQL database systems. Using read scale-out and sharding one can design systems that go from the capacity of a single server to supporting the largest internet sites. But to design and operate high-performance, efficient, manageable and reliable deployments requires knowing the intricacies of the underlying technologies.
This session will provide insights on the main factors that affect the performance of Asynchronous Replication and Group Replication, and how to configure them to make the most out of the underlying computing system. It will also show the latest developments in MySQL 5.7 and 8.0, in areas spanning from group communication to the multi-threaded slave applier, and how effective they are in helping meet the performance requirements in terms of throughput, latency and durability to support the most demanding workload types.
MySQL InnoDB Cluster - A complete High Availability solution for MySQLOlivier DASINI
MySQL InnoDB Cluster provides a complete high availability solution for MySQL. It uses MySQL Group Replication, which allows for multiple read-write replicas of a database to exist with synchronous replication. MySQL InnoDB Cluster also includes MySQL Shell for setup, management and orchestration of the cluster, and MySQL Router for intelligent connection routing. It allows databases to scale out writes across replicas in a fault-tolerant and self-healing manner.
This document discusses troubleshooting an error of "Too many open files" when running MySQL. It describes how the open_files_limit parameter was not taking effect due to MySQL setting the file descriptor limit internally based on other factors like max_connections. Through debugging with GDB and tracing the code, it was found that MySQL was calling setrlimit() to set the limit based on the lower of the configured open_files_limit or the OS file limit. The solution involved configuring the ulimit directly via mysqld_safe to set the limit before MySQL started.
MySQL High Availability with Group ReplicationNuno Carvalho
MySQL Group Replication is a multi-master update everywhere replication plugin that provides high availability. It removes the need for handling server failover, provides fault tolerance, and automates group reconfiguration. Transactions are replicated to all group members, with conflicts detected and resolved using a first committer wins rule. Failed members automatically rejoin the group and synchronize with the others transparently. Group Replication uses the standard MySQL and InnoDB architecture, so existing users will feel familiar. It also supports features like auto-increment handling, GTIDs, secure connections, and a new single primary mode.
The document discusses testing HBase source code. It describes the required JAR files for a simple test, inserts test data into HBase tables, and measures the insertion performance. It also summarizes the entry points for HMaster and HRegionServer source code, and describes some common HBase operations like building tables, handling DDL exceptions, flushing, compacting, and splitting regions.
Group Replication: A Journey to the Group Communication CoreAlfranio Júnior
This document discusses the journey of MySQL Group Replication from using a third-party group communication system (Corosync) to its own built-in group communication engine (XCOM) based on Paxos algorithms. It describes the key components of MySQL Group Replication including the group communication interface, XCOM engine, and optimizations in XCOM like pipelining and batching. It also provides performance results of MySQL Group Replication with multiple writers.
This document provides an overview and agenda for a presentation on MySQL best practices for DBAs and developers. The presentation covers essential MySQL configuration practices like server SQL mode and storage engines. It also discusses improving SQL, user security, schema optimizations, instrumentation, and monitoring. Specific topics include comment SQL, formatting SQL, future proofing SQL, and analyzing SQL. The document provides examples and recommendations for each topic.
This document provides an overview of advanced MySQL replication techniques. It begins with an introduction to replication basics like principles, setup, and monitoring. It then discusses challenges with replication including master switchovers during failures and slaves falling behind. The document outlines various replication power techniques and features that can help address these issues like row-based and semi-synchronous replication. It also covers tools for replication and techniques for replication outside of the traditional setup.
This document provides an overview of techniques for capturing and analyzing SQL queries in MySQL databases. It discusses built-in MySQL options like the slow query log, general query log, and binary log. It also covers other techniques like using MySQL Proxy, TCP/IP capture, Dtrace/SystemTap, and application management. Specific examples are provided for slow query log output, the general query log from WordPress, and the binary log and processlist.
The document discusses high availability and scalability in MySQL. It describes various techniques for achieving high availability including replication, clustering, and shared storage solutions. It also discusses different approaches for scaling MySQL including replication, sharding, and clustering. MySQL replication is described as asynchronous with a single master and multiple read-only slaves. MySQL Cluster provides synchronous replication across nodes and automatic failover for high availability.
MySQL InnoDB Cluster and Group Replication - OSI 2017 BangaloreSujatha Sivakumar
The document discusses MySQL InnoDB Cluster and Group Replication. It provides an introduction and overview of InnoDB Cluster, outlining the key features and how to get an InnoDB Cluster up and running in 3 steps: deploying instances, creating a cluster, and adding more instances. It also covers setting up and starting a router. For Group Replication, it discusses the concept of replicating writes across multiple servers for high availability and read scaling. It shows how Group Replication achieves consensus on membership, message delivery and state updates across the group.
This document provides an overview of troubleshooting replication in MySQL and MariaDB databases. It discusses typical errors seen with the slave IO thread and slave SQL thread, such as replication stopping, the slave lagging behind the master, and the master increasing in resource usage. It also covers replication concepts like the master-slave architecture, binary logging formats, global transaction identifiers, and tools for monitoring replication like SHOW SLAVE STATUS and Performance Schema.
Just about anyone can write a basic SQL query for a table. Not everyone can write a good query though - that takes practice and knowing how to understand what the optimizer is doing with the query. Learn the basics of query optimization so you keep your application engaging the user rather then showing the progress bar as they wait on the database.
"Disaster is inevitable" and "To move forward you must first backup" should be known to all software developers. This presentation will discuss all the options for your valuable data assets in MySQL, and highlight how to maintain site reliability of your data
The History and Future of the MySQL ecosystemRonald Bradford
This document provides an overview of the history and ecosystem of MySQL. It discusses how MySQL originated in Sweden in the 1990s and was later acquired by Sun Microsystems and Oracle. It describes major versions and features added over time. It also summarizes the various distributions, storage engines, patches, and alternatives that have emerged, including Percona Server, MariaDB, MySQL Cluster, and NoSQL options. The document concludes with predictions that MySQL 5.7 may make InnoDB the only storage engine and notes MariaDB will continue to diverge from Oracle MySQL.
Inno db internals innodb file formats and source code structurezhaolinjnu
This document discusses the goals, architecture, and on-disk format of the InnoDB storage engine for MySQL. InnoDB aims to provide transaction support, reliability, and scalability. Its architecture includes buffering, locking, recovery, and efficient I/O mechanisms. The on-disk format is designed for durability, performance through techniques like compression, and compatibility through file format management. Source code is organized into subdirectories corresponding to major subsystems.
A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.
A video recording of this presentation is available from Ustream: http://ustre.am/fyLk
MySQL Group Replication - HandsOn TutorialKenny Gryp
Group Replication is a plugin for MySQL that provides multi-master replication. It works by having each node send write transactions to other nodes through a group communication system. The writes are certified locally in an asynchronous manner to ensure total order of transactions across all nodes. Group Replication uses optimistic locking where local locks are released right after commit, and conflict detection happens during certification rather than at the start of transactions.
MySQL Replication Performance Tuning for Fun and Profit!Vitor Oliveira
MySQL Replication, in addition to bringing high-availability, is the foundation to build high-performance MySQL database systems. Using read scale-out and sharding one can design systems that go from the capacity of a single server to supporting the largest internet sites. But to design and operate high-performance, efficient, manageable and reliable deployments requires knowing the intricacies of the underlying technologies.
This session will provide insights on the main factors that affect the performance of Asynchronous Replication and Group Replication, and how to configure them to make the most out of the underlying computing system. It will also show the latest developments in MySQL 5.7 and 8.0, in areas spanning from group communication to the multi-threaded slave applier, and how effective they are in helping meet the performance requirements in terms of throughput, latency and durability to support the most demanding workload types.
MySQL InnoDB Cluster - A complete High Availability solution for MySQLOlivier DASINI
MySQL InnoDB Cluster provides a complete high availability solution for MySQL. It uses MySQL Group Replication, which allows for multiple read-write replicas of a database to exist with synchronous replication. MySQL InnoDB Cluster also includes MySQL Shell for setup, management and orchestration of the cluster, and MySQL Router for intelligent connection routing. It allows databases to scale out writes across replicas in a fault-tolerant and self-healing manner.
This document discusses troubleshooting an error of "Too many open files" when running MySQL. It describes how the open_files_limit parameter was not taking effect due to MySQL setting the file descriptor limit internally based on other factors like max_connections. Through debugging with GDB and tracing the code, it was found that MySQL was calling setrlimit() to set the limit based on the lower of the configured open_files_limit or the OS file limit. The solution involved configuring the ulimit directly via mysqld_safe to set the limit before MySQL started.
MySQL High Availability with Group ReplicationNuno Carvalho
MySQL Group Replication is a multi-master update everywhere replication plugin that provides high availability. It removes the need for handling server failover, provides fault tolerance, and automates group reconfiguration. Transactions are replicated to all group members, with conflicts detected and resolved using a first committer wins rule. Failed members automatically rejoin the group and synchronize with the others transparently. Group Replication uses the standard MySQL and InnoDB architecture, so existing users will feel familiar. It also supports features like auto-increment handling, GTIDs, secure connections, and a new single primary mode.
The document discusses testing HBase source code. It describes the required JAR files for a simple test, inserts test data into HBase tables, and measures the insertion performance. It also summarizes the entry points for HMaster and HRegionServer source code, and describes some common HBase operations like building tables, handling DDL exceptions, flushing, compacting, and splitting regions.
Group Replication: A Journey to the Group Communication CoreAlfranio Júnior
This document discusses the journey of MySQL Group Replication from using a third-party group communication system (Corosync) to its own built-in group communication engine (XCOM) based on Paxos algorithms. It describes the key components of MySQL Group Replication including the group communication interface, XCOM engine, and optimizations in XCOM like pipelining and batching. It also provides performance results of MySQL Group Replication with multiple writers.
This document provides an overview and agenda for a presentation on MySQL best practices for DBAs and developers. The presentation covers essential MySQL configuration practices like server SQL mode and storage engines. It also discusses improving SQL, user security, schema optimizations, instrumentation, and monitoring. Specific topics include comment SQL, formatting SQL, future proofing SQL, and analyzing SQL. The document provides examples and recommendations for each topic.
This document provides an overview of advanced MySQL replication techniques. It begins with an introduction to replication basics like principles, setup, and monitoring. It then discusses challenges with replication including master switchovers during failures and slaves falling behind. The document outlines various replication power techniques and features that can help address these issues like row-based and semi-synchronous replication. It also covers tools for replication and techniques for replication outside of the traditional setup.
This document provides an overview of techniques for capturing and analyzing SQL queries in MySQL databases. It discusses built-in MySQL options like the slow query log, general query log, and binary log. It also covers other techniques like using MySQL Proxy, TCP/IP capture, Dtrace/SystemTap, and application management. Specific examples are provided for slow query log output, the general query log from WordPress, and the binary log and processlist.
The document discusses high availability and scalability in MySQL. It describes various techniques for achieving high availability including replication, clustering, and shared storage solutions. It also discusses different approaches for scaling MySQL including replication, sharding, and clustering. MySQL replication is described as asynchronous with a single master and multiple read-only slaves. MySQL Cluster provides synchronous replication across nodes and automatic failover for high availability.
MySQL InnoDB Cluster and Group Replication - OSI 2017 BangaloreSujatha Sivakumar
The document discusses MySQL InnoDB Cluster and Group Replication. It provides an introduction and overview of InnoDB Cluster, outlining the key features and how to get an InnoDB Cluster up and running in 3 steps: deploying instances, creating a cluster, and adding more instances. It also covers setting up and starting a router. For Group Replication, it discusses the concept of replicating writes across multiple servers for high availability and read scaling. It shows how Group Replication achieves consensus on membership, message delivery and state updates across the group.
This document provides an overview of troubleshooting replication in MySQL and MariaDB databases. It discusses typical errors seen with the slave IO thread and slave SQL thread, such as replication stopping, the slave lagging behind the master, and the master increasing in resource usage. It also covers replication concepts like the master-slave architecture, binary logging formats, global transaction identifiers, and tools for monitoring replication like SHOW SLAVE STATUS and Performance Schema.
Just about anyone can write a basic SQL query for a table. Not everyone can write a good query though - that takes practice and knowing how to understand what the optimizer is doing with the query. Learn the basics of query optimization so you keep your application engaging the user rather then showing the progress bar as they wait on the database.
"Disaster is inevitable" and "To move forward you must first backup" should be known to all software developers. This presentation will discuss all the options for your valuable data assets in MySQL, and highlight how to maintain site reliability of your data
The History and Future of the MySQL ecosystemRonald Bradford
This document provides an overview of the history and ecosystem of MySQL. It discusses how MySQL originated in Sweden in the 1990s and was later acquired by Sun Microsystems and Oracle. It describes major versions and features added over time. It also summarizes the various distributions, storage engines, patches, and alternatives that have emerged, including Percona Server, MariaDB, MySQL Cluster, and NoSQL options. The document concludes with predictions that MySQL 5.7 may make InnoDB the only storage engine and notes MariaDB will continue to diverge from Oracle MySQL.
Lessons Learned Managing Large AWS EnvironmentsRonald Bradford
How to you optimize management of 500+ AWS servers? In this presentation I share my experiences using Amazon Web Servers covering techniques for webscale. Learn how to optimized your cost, handle security, automate and be prepared for handling failure.
Monitoring your technology stack with New RelicRonald Bradford
There is no excuse to not have monitoring of your LAMP stack, NoSQL database like MongoDB/Redis/Cassandra/Memcache, Cloud services and much more when you can use the popular New Relic tool for free. As the MySQL plugin author I can offer the following link will give you access to free monitoring http://j.mp/newrelic-mysql There can never be an excuse to not know how your application is performing, from 1 server to 100+ servers.
This document summarizes best practices for configuring and using MySQL. It discusses recommended configuration settings like using the InnoDB storage engine and strict SQL modes. It also covers topics like application user security through limited grants, logging and formatting SQL for analysis, and future-proofing SQL. The document recommends tools like the general query log to help developers analyze and optimize SQL performance.
This document discusses common MySQL scalability mistakes and provides solutions. It begins with problems like a slow or unresponsive website and large database size. For each problem, it provides the likely cause such as table locking, storing large files in the database, and unnecessary queries. It then offers solutions such as using a transactional storage engine, compressing large files, and optimizing SQL. Throughout it provides examples and explains why addressing these issues is important for performance and scalability. It also offers guidance on monitoring, automation, high availability and integrating scalability principles from the start.
This document discusses several MySQL idiosyncrasies related to data integrity, transactions, and variable scope that can cause unexpected behavior for developers. Specifically, it notes that MySQL does not enforce data integrity by default, transactions are not supported with all storage engines, and variable settings apply at the session level rather than globally by default. The document provides examples of each issue and recommends configurations like stricter SQL modes and using InnoDB to avoid potential problems.
This document discusses some idiosyncrasies in MySQL that can cause issues if not properly understood. It covers three main topics:
1. Data integrity - MySQL does not enforce data integrity by default like other RDBMS. Values outside ranges are inserted with warnings instead of errors. Settings like STRICT_ALL_TABLES can help enforce integrity.
2. Transactions - Non-transactional storage engines like MyISAM do not support transactions fully. Changes may not roll back as expected. InnoDB is recommended.
3. Variable scope - Changing global variables like storage_engine does not affect existing tables. New tables will still use prior default unless DB is restarted. Variable changes only affect new connections.
This document discusses principles for successful MySQL scalability. It covers the need for integrated monitoring and instrumentation to enable preemptive analysis. Automated server deployment is also discussed to ensure consistent configuration. The document stresses that disaster recovery planning is crucial, as disaster is inevitable. It advocates supporting different levels of data availability and scalability principles to meet business needs.
1. The document discusses MySQL idiosyncrasies related to data integrity, transactions, and variable scope. It provides examples of unexpected behavior when the default storage engine is MyISAM rather than InnoDB, and the use of SQL modes to enforce data integrity.
2. Transactions examples show data is not rolled back as expected when tables are MyISAM rather than InnoDB.
3. Changing global variables like storage engine does not affect existing or new tables, demonstrating variable scope issues in MySQL.
While MySQL is a popular and widely used RDBMS, some default features and settings are very foreign in comparison with other commercial RDBMS products. In this discussion, Ronald Bradford will discuss some of the MySQL defaults including a non-transactional state, silent data truncations, date management, and transaction isolation options. These are all critical for data integrity and consistency. He will cover in-depth topics including SQL_MODE that saves the day. He will also cover character sets and collations and the best practices to ensure your UTF8 is stored and retrieved correctly.
This document describes 10 steps to achieve a 10x performance improvement for a MySQL database for a social media website. The steps include monitoring the database, identifying slow SQL queries, analyzing problem queries, improving indexes, offloading read load to slaves, improving SQL, using optimal storage engines, and implementing caching. Key actions include installing monitoring tools, using mk-query-digest to analyze SQL, adding indexes, configuring InnoDB, converting tables to InnoDB, and caching query results and content with Memcache. The goal is to optimize the database infrastructure and queries to handle the load of a growing web 2.0 application.
Only after a successful preparation covered in IGNITION can you be ready for the implementation and management of a MySQL ecosystem and a successful launch of your product.
We Discuss:
* Escape Options – Before and after backup and recovery situations
* Good to Go – Knowing and confirming your MySQL environment is ready
* Full Throttle – Understanding and Improving MySQL database performance
* A Green Dashboard – Monitoring for Success
* The Human Factor – Nobody is perfect, dealing with the design changes
* Propellant – The murky mess of MySQL versions, patches and variants
* Best Practices – Proven techniques for consistency, automation and reproducibility
Note: This is volume 2 of a two part series
This document outlines the agenda for a MySQL training camp for Oracle DBAs. The training will cover topics such as translating SQL statements between MySQL and Oracle, installing and protecting MySQL instances, database mechanics like transactions and storage engines, setting up redundancy using replication, and checklists for tasks like security and application configuration. It also lists resources that will be used during the training like the MySQL sandbox and online documentation.
This presentation discusses the current state of the Drizzle database project including the principles behind this leading open source project, and the healthy and growing community. For more information visit http://drizzle.org
There has been significant movement in recent times towards less structured approaches of storing and retrieving data. No longer the realm of Relational Databases, there is a new crop of structured key/value pair stores and unstructured data offerings. This closing panel debate at Open SQL Camp 2009 discussed the SQL v NoSQL topic.
Example section on MySQL for the Oracle DBA 1 day bootcamp.
In object management we look at the key SQL objects including what differs with Oracle and what is Oracle specific functionality.
We also look at the MySQL data dictionary, the INFORMATION_SCHEMA
Know Your Competitor - Oracle 10g Express EditionRonald Bradford
The document provides an overview of Oracle 10g Express Edition (XE) for MySQL developers. It discusses why developers should be familiar with Oracle as the largest RDBMS provider, describes what XE is and how to install and configure it. The key processes involved in running an XE database are also outlined, including the log writer, database writer, checkpoint process, and memory manager.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
A Comprehensive Guide to DeFi Development Services in 2024Intelisync
DeFi represents a paradigm shift in the financial industry. Instead of relying on traditional, centralized institutions like banks, DeFi leverages blockchain technology to create a decentralized network of financial services. This means that financial transactions can occur directly between parties, without intermediaries, using smart contracts on platforms like Ethereum.
In 2024, we are witnessing an explosion of new DeFi projects and protocols, each pushing the boundaries of what’s possible in finance.
In summary, DeFi in 2024 is not just a trend; it’s a revolution that democratizes finance, enhances security and transparency, and fosters continuous innovation. As we proceed through this presentation, we'll explore the various components and services of DeFi in detail, shedding light on how they are transforming the financial landscape.
At Intelisync, we specialize in providing comprehensive DeFi development services tailored to meet the unique needs of our clients. From smart contract development to dApp creation and security audits, we ensure that your DeFi project is built with innovation, security, and scalability in mind. Trust Intelisync to guide you through the intricate landscape of decentralized finance and unlock the full potential of blockchain technology.
Ready to take your DeFi project to the next level? Partner with Intelisync for expert DeFi development services today!
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Top 20 Design Tips for MySQL Data Architects
1. The Top 20 Design Tips
The Top 20 Design Tips
For MySQL Enterprise Data
Architects
Ronald Bradford
Principal
42SQL
April 2008
Presented By: Ronald Bradford
Version 1.1 10.Apr.2008 www.ronaldbradford.com
2. The Top 20 Design Tips
1. Know Your Technology Tools
❖ Generics are inefficient
❖ Product expertise in a different RDBMS is
not enough
❖ You have chosen MySQL
❖ Maximize it's strengths
❖ Minimize it's weaknesses
Presented By: Ronald Bradford www.ronaldbradford.com
3. The Top 20 Design Tips
Overview
❖ Table Structure
❖ SQL
❖ Indexes
❖ Enterprise Approaches
Presented By: Ronald Bradford www.ronaldbradford.com
4. The Top 20 Design Tips
1. Know Your Technology Tools
❖ Maximize MySQL strengths
❖ Scale out / HA Options
❖ Different Storage Engines
❖ Query Cache
❖ Minimize MySQL weaknesses
❖ No Online Alter
❖ Backup Strategies
❖ Instrumentation
Presented By: Ronald Bradford www.ronaldbradford.com
5. The Top 20 Design Tips
2. Know Your Disk Footprint
Disk = Memory = Performance
❖ Every single byte counts
❖ Average 25% - 30% saving on engagements
❖ Better 60% (200GB System)
❖ Best 78% (8GB per master with 12 masters)
Less disk accesses and more data in memory
Presented By: Ronald Bradford www.ronaldbradford.com
6. The Top 20 Design Tips
3. Choose Your Numeric Data Type
❖ MySQL has 9 numeric data types
❖ Oracle for example has only 1
Presented By: Ronald Bradford www.ronaldbradford.com
7. The Top 20 Design Tips
3. Choose Your Numeric Data Type
❖ Integer: TINYINT, SMALLINT,
MEDIUMINT, INT, BIGINT
❖ Floating Point: FLOAT, DOUBLE
❖ Fixed Point: DECIMAL
❖ Other: BIT, (ENUM maybe)
Presented By: Ronald Bradford www.ronaldbradford.com
8. The Top 20 Design Tips
3. Choose Your Numeric Data Type
❖ Favorite signs of poor design
❖ INT(1)
❖ BIGINT AUTO_INCREMENT
❖ no UNSIGNED used
❖ DECIMAL(31,0)
Presented By: Ronald Bradford www.ronaldbradford.com
9. The Top 20 Design Tips
3. Choose Your Numeric Data Type
❖ INT(1) - 1 does not mean 1 digit
❖ (1) represents client output display format only
❖ INT is 4 Bytes, TINYINT is 1 Byte
❖ TINYINT UNSIGNED can store from 0 – 255
❖ BIT is even better when values are 0 - 1
Presented By: Ronald Bradford www.ronaldbradford.com
10. The Top 20 Design Tips
3. Choose Your Numeric Data Type
❖ BIGINT is not needed for AUTO_INCREMENT
❖ INT UNSIGNED stores 4.3 billion values
❖ You should be partitioning when at billions of rows
❖ BIGINT is applicable for some columns
❖ e.g. summation of values
Presented By: Ronald Bradford www.ronaldbradford.com
11. The Top 20 Design Tips
3. Choose Your Numeric Data Type
Best
Practice
❖ Best Practice
❖ All integer columns UNSIGNED unless there is a
reason otherwise
❖ Adds a level of data integrity for negative values
Presented By: Ronald Bradford www.ronaldbradford.com
12. The Top 20 Design Tips
4. Other Data Type Efficiencies
❖ TIMESTAMP v DATETIME
❖ Suitable for EPOCH only values
❖ TIMESTAMP is 4 bytes
❖ DATETIME is 8 bytes
❖ FYI: DATE is 3 bytes, TIME is 3 bytes = 6 Bytes???
Presented By: Ronald Bradford www.ronaldbradford.com
13. The Top 20 Design Tips
4. Other Data Type Efficiencies
❖ CHAR(n)
❖ Use VARCHAR(n) for variable values
❖ e.g. CHAR(128) when storing ~10 bytes
Presented By: Ronald Bradford www.ronaldbradford.com
14. The Top 20 Design Tips
5. Application Data Type Efficiencies
❖ Using Codes or ENUM
❖ A description is a presentation layer function
❖ e.g. 'M', 'F' instead of 'Male', 'Female'
❖ e.g. 'A', 'I' instead of 'Active', 'Inactive'
❖ BINARY(16/20) v CHAR(32/40)
❖ MD5() or HASH() Hex value with twice the length
❖ INT UNSIGNED for IPv4 address
❖ VARCHAR(15) results in average 12 bytes v 4 bytes
Presented By: Ronald Bradford www.ronaldbradford.com
15. The Top 20 Design Tips
6. NOT NULL
❖ Saves up to a byte per column per row of data
❖ Double benefit for indexed columns
❖ Don't use frameworks or tools
❖ NOT NULL DEFAULT '' is bad design
Best
Always use NOT NULL unless Practice
there is a reason why not
Presented By: Ronald Bradford www.ronaldbradford.com
16. The Top 20 Design Tips
7. Know about character sets
❖ Default in MySQL 5 is UTF8
❖ Can be defined at database, schema, table
or column level
❖ Only define columns that need UTF8
❖ e.g. Codes, MD5 Value, web address
❖ MySQL internal buffers are fixed width
❖ e.g. VARCHAR(255) utf8 is 765 bytes to store just
1 byte
Presented By: Ronald Bradford www.ronaldbradford.com
17. The Top 20 Design Tips
8. When VARCHAR Is Bad
❖ VARCHAR(255)
❖ Poor Design - No understanding of underlying
data
❖ Old Design - ( 4.x limitation, now 3-4 years old)
❖ Disk usage may be efficient
❖ MySQL internal memory usage is not
Presented By: Ronald Bradford www.ronaldbradford.com
18. The Top 20 Design Tips
8. When VARCHAR is bad
CREATE TABLE `XXX` (
`orderHandle` varchar(255) NOT NULL default '',
`personName` varchar(255) default NULL,
`addressLines` varchar(255) default NULL,
`city` varchar(255) default NULL,
`state` varchar(255) default NULL,
`postalCode` varchar(255) default NULL,
`countryCode` varchar(255) default NULL,
`phone` varchar(255) default NULL,
`email` varchar(255) default NULL,
`shipMethod` varchar(255) default NULL,
`shipTo` varchar(255) default NULL,
`receiveByDate` date default NULL,
`currency` varchar(3) default NULL,
`price` varchar(255) default NULL,
`flags` int(11) default '0',
`lastUpdateTime` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
`creationTime` timestamp NOT NULL default '0000-00-00 00:00:00',
PRIMARY KEY (`orderHandle`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
Presented By: Ronald Bradford www.ronaldbradford.com
19. The Top 20 Design Tips
9. Be Wary of TEXT/BLOB
❖ Using SELECT *
❖ MySQL Internal Temporary table will force Temp
Disk Table
❖ Internal storage (e.g. Innodb)
❖ Stores first 768 bytes, then a separate 16k data
page per row per TEXT/BLOB field
Presented By: Ronald Bradford www.ronaldbradford.com
20. The Top 20 Design Tips
10. Know Every SQL Statement
❖ Developers don't write proper SQL statements
❖ SQL statements will directly affect your performance
❖ For Example
❖ Repeating SQL statements for no benefit
❖ 1000 very quick small unnecessary queries is worse
then 1 slow query
Presented By: Ronald Bradford www.ronaldbradford.com
21. The Top 20 Design Tips
10. Know Every SQL Statement
❖ Data collection options
❖ Incomplete Options
❖ Slow Query Log
❖ SHOW PROCESSLIST
❖ Application level logging
❖ Impractical Options
❖ General Log
Presented By: Ronald Bradford www.ronaldbradford.com
22. The Top 20 Design Tips
10. Know Every SQL Statement
❖ Data collection options
❖ MySQL Proxy
❖ See histogram.lua
❖ Firewall forwarding rules
Presented By: Ronald Bradford www.ronaldbradford.com
23. The Top 20 Design Tips
11. Monitor Every SQL Statement
❖ Review Query Execution Plan (QEP)
❖ EXPLAIN
❖ Time queries
❖ Row Count / Affected rows
❖ Result Set Size Best
Practice
Review over time, things change
Presented By: Ronald Bradford www.ronaldbradford.com
24. The Top 20 Design Tips
12. The Impact Of Indexes
❖ Good
❖ Dramatic performance improvements
❖ Improves memory usage
❖ Data Integrity
❖ Bad
❖ Slows performance for writes
❖ Wastes disk space for unused, duplicate or
ineffective indexes
❖ In-effective usage of memory
Presented By: Ronald Bradford www.ronaldbradford.com
25. The Top 20 Design Tips
13. Index Types For Design
❖ Concatenated Indexes
❖ (col1, col2)
❖ Partial Indexes
❖ (name(20))
❖ Covering Indexes
❖ Full Text Indexes
❖ No function based indexes
Presented By: Ronald Bradford www.ronaldbradford.com
26. The Top 20 Design Tips
14. Minimizing internal MySQL processing
❖ Correctly design tables, indexes and SQL to
eliminate
❖ Using temporary table
❖ Using filesort
Presented By: Ronald Bradford www.ronaldbradford.com
27. The Top 20 Design Tips
15. Transactions
❖ Always design for transactions
❖ Always use transactions
❖ Use a transactional storage engine
Presented By: Ronald Bradford www.ronaldbradford.com
28. The Top 20 Design Tips
16. Data Integrity is Key
❖ MySQL historically has been very lax
❖ Warnings (e.g. Truncations) are rarely every
caught
❖ SQL_MODE=STRICT_ALL_TABLES
❖ Within Schema
❖ NOT NULL
❖ ENUM
❖ UNSIGNED
Presented By: Ronald Bradford www.ronaldbradford.com
29. The Top 20 Design Tips
17. Leverage The Query Cache
❖ Query Cache can be a great benefit
❖ Deterministic v Non Deterministic SQL
Best
Practice
MySQL Query Cache is not the only type
of caching you should consider
Presented By: Ronald Bradford www.ronaldbradford.com
30. The Top 20 Design Tips
17. Leverage The Query Cache
Presented By: Ronald Bradford www.ronaldbradford.com
31. The Top 20 Design Tips
17. Leverage The Query Cache
❖ SHOW PROFILE Path
❖ Simple SELECT
❖ No Query Cache 17 steps
❖ With Query Cache 5 steps
❖ Does not perform parse
❖ Does not perform optimize
Presented By: Ronald Bradford www.ronaldbradford.com
33. The Top 20 Design Tips
18. Create Objects Appropriately
❖ Using 1 table instead of ʻnʼ for same column
structure
❖ e.g. Code table for each type of code
❖ Splitting tables for optimal storage
❖ e.g. Placing optional TEXT/BLOB columns in
second table
❖ Use permanent tables instead of
TEMPORARY tables
Presented By: Ronald Bradford www.ronaldbradford.com
34. The Top 20 Design Tips
19. Naming Standards
❖ Name all Primary Keyʼs Uniquely
❖ e.g. customer_id, order_id not id
❖ Use Data Dictionary SQL to verify data types
❖ Data Types & Lengths
❖ Be Descriptive
❖ e.g. invoice_date not just date
❖ Avoid Reserved Words
❖ e.g. date, time, timestamp
Presented By: Ronald Bradford www.ronaldbradford.com
35. The Top 20 Design Tips
20. Testing, Testing, Testing
❖ You must have a testing environment
❖ Testing on a Production server is not an
option
Best
Practice
The goal of a testing environment
is not to test your software,
it is to break your software.
Presented By: Ronald Bradford www.ronaldbradford.com
36. The Top 20 Design Tips
Executive Summary
❖ Learn and know MySQL specifics
❖ Disk = Memory = Performance
❖ If you don't know your SQL you don't know
your application. Log, Review & Monitor all
SQL
❖ Know all benefits of different indexes
❖ You must test to failure in a dedicated test
environment
Presented By: Ronald Bradford www.ronaldbradford.com
37. The Top 20 Design Tips
Professional Help is Available
❖ PrimeBase Technologies
❖ Technology Experts
❖ Solution Experts
❖ 2 decades Expertise & Experience in
Enterprise RDBMS Data Architecture
❖ 9 years in MySQL
www.ronaldbradford.com/contact
Presented By: Ronald Bradford www.ronaldbradford.com