Get the best out of MySQL Cluster, presentation covers:
- Tuning and optimization to exploit the auto-sharded, distributed design of MySQL Cluster
- Using Adaptive Query Localization to scale cross-shard JOINs
- Data access patterns, schema and query optimizations
- Recommended tuning parameters
Tune in to the on-demand webinar: http://www.mysql.com/news-and-events/on-demand-webinars/display-od-719.html
MySQL Group Replication - Ready For Production? (2018-04)Kenny Gryp
At the end of 2016, Oracle released a new Plugin called MySQL Group Replication, which is a new MySQL replication method that aims to provide better High Availability, and built-in failover with consistency guarantees.
I evaluated the initial GA versions back in early 2017. I presented my initial findings with several best practices and concerns with the current implementation which made me state that Group Replication was not quite ready yet.
(https://www.slideshare.net/Grypyrg/my-sql-group-replication)
(Un)lucky as I was, a large part of the attendees were Oracle developers and the months after this, many of these bugs and missing features were implemented in both MySQL 8.0 as well as backported to MySQL 5.7. (Thank you!)
This is a followup presentation on my previous analysis, where I will look into the changes since and re-evaluate the readiness of Group Replication for production usage and provide my insights and opinion on the state of GR.
EFK Stack이란 ElasticSearch, Fluentd, Kibana라는 오픈소스의 조합으로, 방대한 양의 데이터를 신속하고 실시간으로 수집/저장/분석/시각화 할 수 있는 솔루션입니다. 특히 컨테이너 환경에서 로그 수집을 위해 주로 사용되는 기술 스택입니다.
Elasitc Stack에 대한 소개와 EFK Stack 설치 방법에 대해 설명합니다.
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11Kenny Gryp
Oracle's MySQL solutions make it easy to setup various database architectures and achieve high availability with the introduction MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet meeting various high availability requirements. MySQL InnoDB ClusterSet provides a popular disaster recovery solution.
Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business critical applications.
In this presentation the various database architecture solutions for high availability and disaster recovery will be covered and help you choose the right solutions based on your business requirements.
MySQL InnoDB Cluster HA Overview & DemoKeith Hollman
Take a look at the High Availability option that you can use with your out-of-the-box MySQL: MySQL InnoDB Cluster. With MySQL Server 8.0, MySQL Shell & MySQL Router you can convert from single-primary to multi-primary and back again, in a single command. Want to know how?
MySQL Group Replication - Ready For Production? (2018-04)Kenny Gryp
At the end of 2016, Oracle released a new Plugin called MySQL Group Replication, which is a new MySQL replication method that aims to provide better High Availability, and built-in failover with consistency guarantees.
I evaluated the initial GA versions back in early 2017. I presented my initial findings with several best practices and concerns with the current implementation which made me state that Group Replication was not quite ready yet.
(https://www.slideshare.net/Grypyrg/my-sql-group-replication)
(Un)lucky as I was, a large part of the attendees were Oracle developers and the months after this, many of these bugs and missing features were implemented in both MySQL 8.0 as well as backported to MySQL 5.7. (Thank you!)
This is a followup presentation on my previous analysis, where I will look into the changes since and re-evaluate the readiness of Group Replication for production usage and provide my insights and opinion on the state of GR.
EFK Stack이란 ElasticSearch, Fluentd, Kibana라는 오픈소스의 조합으로, 방대한 양의 데이터를 신속하고 실시간으로 수집/저장/분석/시각화 할 수 있는 솔루션입니다. 특히 컨테이너 환경에서 로그 수집을 위해 주로 사용되는 기술 스택입니다.
Elasitc Stack에 대한 소개와 EFK Stack 설치 방법에 대해 설명합니다.
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11Kenny Gryp
Oracle's MySQL solutions make it easy to setup various database architectures and achieve high availability with the introduction MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet meeting various high availability requirements. MySQL InnoDB ClusterSet provides a popular disaster recovery solution.
Completely built in-house and supported by Oracle, many enterprises large and small have adopted these solutions into business critical applications.
In this presentation the various database architecture solutions for high availability and disaster recovery will be covered and help you choose the right solutions based on your business requirements.
MySQL InnoDB Cluster HA Overview & DemoKeith Hollman
Take a look at the High Availability option that you can use with your out-of-the-box MySQL: MySQL InnoDB Cluster. With MySQL Server 8.0, MySQL Shell & MySQL Router you can convert from single-primary to multi-primary and back again, in a single command. Want to know how?
2021년 11월 18일(목)
- 14:00 ~ 15:00 MySQL Operator for Kubernetes
: Kubernetes 환경에서 MySQL에 대한 더 쉬운 운영
- 15:00 ~ 15:15 MySQL HA and Auto-Failover
: MySQL replication과 오픈소스 MHA를 통한 고가용성 확보
29回勉強会資料「PostgreSQLのリカバリ超入門」
See also http://www.interdb.jp/pgsql (Coming soon!)
初心者向け。PostgreSQLのWAL、CHECKPOINT、 オンラインバックアップの仕組み解説。
これを見たら、次は→ http://www.slideshare.net/satock/29shikumi-backup
This is a presentation at Bengaluru TechDay -October2019 for Oracle Database Admin and Architects presented by Karthik P R ( CEO Mydbops ). He explains the possible High Availability options in MySQL ecosystem.
https://www.meetup.com/All-India-Oracle-Users-Group-Bangalore-Chapter/events/265252214/
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
Introduction to memcached, a caching service designed for optimizing performance and scaling in the web stack, seen from perspective of MySQL/PHP users. Given for 2nd year students of professional bachelor in ICT at Kaho St. Lieven, Gent.
PostgreSQL is a very popular and feature-rich DBMS. At the same time, PostgreSQL has a set of annoying wicked problems, which haven't been resolved in decades. Miraculously, with just a small patch to PostgreSQL core extending this API, it appears possible to solve wicked PostgreSQL problems in a new engine made within an extension.
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
This presentation by Krzysztof Książek at Percona Live 2017 in Santa Clara, California gives detailed descriptions and comparisons of the leading open source database load balancing technologies
This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
사례로 알아보는 MariaDB 마이그레이션
현대적인 IT 환경과 애플리케이션을 만들기 위해 우리는 오늘도 고민을 거듭합니다. 최근 들어 오픈소스 DB가 많은 업무에 적용되고 검증이 되면서, 점차 무거운 상용 데이터베이스를 가벼운 오픈소스 DB로 전환하는 움직임이 대기업의 미션 크리티컬 업무까지로 확산하고 있습니다. 이는 클라우드 환경 및 마이크로 서비스 개념 확산과도 일치하는 움직임입니다.
상용 DB를 MariaDB로 이관한 사례를 통해 마이그레이션의 과정과 효과를 살펴 볼 수 있습니다.
MariaDB로 이관하는 것은 어렵다는 생각을 막연히 가지고 계셨다면 본 자료를 통해 이기종 데이터베이스를 MariaDB로 마이그레이션 하는 작업이 어렵지 않게 수행될 수 있다는 점을 실제 사례를 통해 확인하시길 바랍니다.
웨비나 동영상
https://www.youtube.com/watch?v=xRsETZ5cKz8&t=52s
MySQL InnoDB Cluster and Group Replication in a Nutshell: hands-on tutorialFrederic Descamps
If you are trying to migrate from asynchronous Master-Slave environment to a full HA solution using MySQL InnoDB Cluster 8.0, this tutorial shows you how to proceed and how to manage and monitor your cluster.
This presentation shortly describes key features of Apache Cassandra. It was held at the Apache Cassandra Meetup in Vienna in January 2014. You can access the meetup here: http://www.meetup.com/Vienna-Cassandra-Users/
MySQL InnoDB Cluster provides a complete, high-availability solution for MySQL. Learn how with a few easy-to-use Shell commands, how to set up a MySQL database architecture.
My MySQL and NoSQL presentation from the NoSQL Search event in Copenhagen: http://nosqlroadshow.com/nosql-cph-2013/speaker/Ted+Wennmark
MySQL offers solutions to implement NoSQL concepts like auto-sharding, key-value access or asynchronous operations. This adds all known solutions from the SQL world to the NoSQL space.
The combined approach of SQL and NoSQL gives developers the choice to select whatever features from both worlds they need.
In this talk we take a deeper look at key-value access to MySQL and MySQL Cluster, auto-sharding and scalability of MySQL Cluster, mapping of schemaless key value access to a relational data model and the performance of NoSQL access to MySQL.
2021년 11월 18일(목)
- 14:00 ~ 15:00 MySQL Operator for Kubernetes
: Kubernetes 환경에서 MySQL에 대한 더 쉬운 운영
- 15:00 ~ 15:15 MySQL HA and Auto-Failover
: MySQL replication과 오픈소스 MHA를 통한 고가용성 확보
29回勉強会資料「PostgreSQLのリカバリ超入門」
See also http://www.interdb.jp/pgsql (Coming soon!)
初心者向け。PostgreSQLのWAL、CHECKPOINT、 オンラインバックアップの仕組み解説。
これを見たら、次は→ http://www.slideshare.net/satock/29shikumi-backup
This is a presentation at Bengaluru TechDay -October2019 for Oracle Database Admin and Architects presented by Karthik P R ( CEO Mydbops ). He explains the possible High Availability options in MySQL ecosystem.
https://www.meetup.com/All-India-Oracle-Users-Group-Bangalore-Chapter/events/265252214/
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
Introduction to memcached, a caching service designed for optimizing performance and scaling in the web stack, seen from perspective of MySQL/PHP users. Given for 2nd year students of professional bachelor in ICT at Kaho St. Lieven, Gent.
PostgreSQL is a very popular and feature-rich DBMS. At the same time, PostgreSQL has a set of annoying wicked problems, which haven't been resolved in decades. Miraculously, with just a small patch to PostgreSQL core extending this API, it appears possible to solve wicked PostgreSQL problems in a new engine made within an extension.
MySQL Load Balancers - Maxscale, ProxySQL, HAProxy, MySQL Router & nginx - A ...Severalnines
This presentation by Krzysztof Książek at Percona Live 2017 in Santa Clara, California gives detailed descriptions and comparisons of the leading open source database load balancing technologies
This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
사례로 알아보는 MariaDB 마이그레이션
현대적인 IT 환경과 애플리케이션을 만들기 위해 우리는 오늘도 고민을 거듭합니다. 최근 들어 오픈소스 DB가 많은 업무에 적용되고 검증이 되면서, 점차 무거운 상용 데이터베이스를 가벼운 오픈소스 DB로 전환하는 움직임이 대기업의 미션 크리티컬 업무까지로 확산하고 있습니다. 이는 클라우드 환경 및 마이크로 서비스 개념 확산과도 일치하는 움직임입니다.
상용 DB를 MariaDB로 이관한 사례를 통해 마이그레이션의 과정과 효과를 살펴 볼 수 있습니다.
MariaDB로 이관하는 것은 어렵다는 생각을 막연히 가지고 계셨다면 본 자료를 통해 이기종 데이터베이스를 MariaDB로 마이그레이션 하는 작업이 어렵지 않게 수행될 수 있다는 점을 실제 사례를 통해 확인하시길 바랍니다.
웨비나 동영상
https://www.youtube.com/watch?v=xRsETZ5cKz8&t=52s
MySQL InnoDB Cluster and Group Replication in a Nutshell: hands-on tutorialFrederic Descamps
If you are trying to migrate from asynchronous Master-Slave environment to a full HA solution using MySQL InnoDB Cluster 8.0, this tutorial shows you how to proceed and how to manage and monitor your cluster.
This presentation shortly describes key features of Apache Cassandra. It was held at the Apache Cassandra Meetup in Vienna in January 2014. You can access the meetup here: http://www.meetup.com/Vienna-Cassandra-Users/
MySQL InnoDB Cluster provides a complete, high-availability solution for MySQL. Learn how with a few easy-to-use Shell commands, how to set up a MySQL database architecture.
My MySQL and NoSQL presentation from the NoSQL Search event in Copenhagen: http://nosqlroadshow.com/nosql-cph-2013/speaker/Ted+Wennmark
MySQL offers solutions to implement NoSQL concepts like auto-sharding, key-value access or asynchronous operations. This adds all known solutions from the SQL world to the NoSQL space.
The combined approach of SQL and NoSQL gives developers the choice to select whatever features from both worlds they need.
In this talk we take a deeper look at key-value access to MySQL and MySQL Cluster, auto-sharding and scalability of MySQL Cluster, mapping of schemaless key value access to a relational data model and the performance of NoSQL access to MySQL.
An outline on why the MySQL 8 release is viewed as a gamechanger with a look at some of the new features like CTEs, Window Functions, MySQL InnoDB Cluster, Enterprise Data Masking, and more
Breakthrough performance with MySQL Cluster (2012)Frazer Clement
Presentation from the MySQL Connect conference in San Francisco 2012.
Describes cluster architecture and impacts on performance, benchmarking, analysing and techniques for improving performance.
200 million qps on commodity hardware : Getting started with MySQL Cluster 7.4Frazer Clement
MySQL Cluster 7.4 has been benchmarked executing over 200 million queries per second on commodity hardware. This presentation from Oracle OpenWorld 2015 describes MySQL Cluster's architecture and gives some detail on how this benchmark was achieved, as well as some tips on getting started with MySQL Cluster 7.4.
Slides from the MySQL Cluster 8.0 tutorial given at MySQL preFOSDEM days in Brussels, January 2020.
There is an accompanying text file with commands and example output.
Moodle Moot Spain: Moodle Available and Scalable with MySQL HA - InnoDB Clust...Keith Hollman
When we start off with Moodle and the MySQL instance, we're working and it's all going smoothly. But, then, a pandemic hits, and everything goes remote .This means that our online systems are more stressed, and especially Moodle. Here we hope to help you understand how MySQL can help, the difference between InnoDB Cluster and InnoDB Replicaset, the components that make up each solution, and therefore, decide on which of the 2 solutions will help Moodle cope with the new load and keep your business running as usual, or better. All this, with no changes to the application, whether it's Cluster or Replicaset. Completely transparent.
OpenStack Days East -- MySQL Options in OpenStackMatt Lord
In most production OpenStack installations, you want the backing metadata store to be highly available. For this, the de facto standard has become MySQL+Galera. In order to help you meet this basic use case even better, I will introduce you to the brand new native MySQL HA solution called MySQL Group Replication. This allows you to easily go from a single instance of MySQL to a MySQL service that's natively distributed and highly available, while eliminating the need for any third party library and implementations.
If you have an extremely large OpenStack installation in production, then you are likely to eventually run into write scaling issues and the metadata store itself can become a bottleneck. For this use case, MySQL NDB Cluster can allow you to linearly scale the metadata store as your needs grow. I will introduce you to the core features of MySQL NDB Cluster--which include in-memory OLTP, transparent sharding, and support for active/active multi-datacenter clusters--that will allow you to meet even the most demanding of use cases with ease.
Need to dive into #MySQL suddenly and find out, briefly, what can be done with MySQL technology? NoSQL, MySQL 8.0, Highly Available, InnoDB Cluster & MySQL Cluster both Community & Enterprise Edition. It's all here.
Presentation from Bosch Connected World, providing an overview of Blockchain, applications within the IoT, and how to get started evaluating the potential benefits
10-Step Methodology to Building a Single View with MongoDBMat Keep
MongoDB has been used in many single view projects across enterprises of all sizes and industries. In this session, we will share the best practices we have observed and institutionalized over the years. By attending the webinar, you will learn:
- A repeatable, 10-step methodology to successfully delivering a single view
- The required technology capabilities and tools to accelerate project delivery
- Case studies from customers who have built transformational single view applications on MongoDB.
An overview of how MongoDB and Apache Spark are enabling IoT innovators to create new business models.
Presented at the Business of IoT meetup in London on Jan 13th 2016
Overview of security best practices and the new security architecture of MongoDB, including latest features in MongoDB 2.6, including user-defined roles, LDAP, PKI integration, field level redaction, auditing and more
This presentation is designed to assist Developers, Architects and DBAs in navigating the complex waters of HA. It presents:
- Methodology for selecting the right HA solution to meet SLAs;
Tour of the leading certified HA solutions for MySQL;
Operational best practices to implement and support HA.
MySQL Cluster 7.2 added support for the Memcached API, enabling web services to directly query MySQL Cluster using the Memcached API, and adding a persistent, scalable, fault tolerant backend to Memcached.
The slides take you through the design concepts and introduce a sample social media app built using memcached and MySQL Cluster
Replication enhancements in MySQL 5.6, including GTIDs, HA / Self-healing, multi-threaded slaves and more. Slides over design rationale, implementation and how to get started using these new capabilities
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
1. MySQL Cluster : Delivering Breakthrough Performance
26th July 2012
Andrew Morgan Mat Keep
Senior Product Manager – MySQL HA Senior Product Manager – MySQL HA
andrew.morgan@oracle.com mat.keep@oracle.com
clusterdb.com
2. The presentation is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
Copyright 2012 Oracle Corporation - 26th July 2012 2
3. Session Agenda
• Introduction to MySQL Cluster
• Where does MySQL Cluster fit?
• Benchmarks
• WHILE (cluster.measurePerformance() < target) {
cluster.optimize();
}
• Boosting performance
• Scaling out
• Further resources
Copyright 2012 Oracle Corporation - 26th July 2012 3
4. MySQL Cluster – Users & Applications
Extreme Scalability, Availability and Affordability
• Web
• High volume OLTP
• eCommerce
• On-Line Gaming
• Digital Marketing
• User Profile Management
• Session Management & Caching
• Content Management
• Telecoms
• Service Delivery Platforms
• VAS: VoIP, IPTV & VoD
• Mobile Content Delivery
• Mobile Payments
• LTE Access http://www.mysql.com/customers/cluster/
Copyright 2012 Oracle Corporation - 26th July 2012 4
5. MySQL Cluster Architecture
JPA REST
Application Nodes
Node Group 1 Node Group 2
F1 F2
Node 1
Node 3
Cluster Cluster
Mgmt F3 F4 Mgmt
F3 F4
Node 4
Node 2
F1 F2
Data Nodes
Copyright 2012 Oracle Corporation - 26th July 2012 5
6. When to Consider MySQL Cluster
l What are the consequences of downtime or failing to meet
performance requirements?
l How much effort and $ is spent in developing and managing HA in
your applications?
l Are you considering sharding your database to scale write
performance? How does that impact your application and
developers?
l Do your services need to be real-time?
l Will your services have unpredictable scalability demands,
especially for writes ?
l Do you want the flexibility to manage your data
with more than just SQL ?
Copyright 2012 Oracle Corporation - 26th July 2012 6
7. Where would I not Use MySQL Cluster?
• “Hot” data sets >3TB
• Replicate cold data to InnoDB
• Long running transactions
• Large rows, without using BLOBs
• Foreign Keys
• Check out MySQL Cluster 7.3 Early Access: http://labs.mysql.com/
• Many full table scans
• Geo-Spatial indexes
• In these scenarios; InnoDB storage engine would be the
right choice
MySQL Cluster Evaluation Guide
http://mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php
Copyright 2012 Oracle Corporation - 26th July 2012 7
8. General Design Considerations
• MySQL Cluster is designed for
– Short transactions
– Many parallel transactions
• Utilize Simple access patterns to fetch data
– Use efficient scans and batching interfaces
• Analyze what your most typical use cases are
– optimize for those
Overall design goal
Minimize network roundtrips for your
most important requests!
Copyright 2012 Oracle Corporation - 26th July 2012 8
9. Servicing the Most Performance-Intensive Workloads
Copyright 2012 Oracle Corporation - 26th July 2012 9
10. writes
Servicing the Most Performance-Intensive Workloads
Copyright 2012 Oracle Corporation - 26th July 2012 10
11. Comparing MySQL Cluster Performance
8x Higher Performance per Node
20
18
Reads per Second (Millions)
16
14
12
10
8
6
4
2
0
MySQL Cluster 7.1 MySQL Cluster 7.2
• 1 Billion+ Reads per Minute, 8 node Intel Xeon cluster
• Multi-Threaded Data Node Extensions
• NoSQL C++ API, flexaSynch benchmark
Copyright 2012 Oracle Corporation - 26th July 2012 11
12. 1.2 Billion UPDATEs per Minute
25
Millions of UPDATEs per Second
20
15
10
5
0
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
MySQL Cluster Data Nodes
• 30 x Intel E5-2600 Intel Servers
• NoSQL C++ API, flexaSynch benchmark
• ACID Transactions, with Synchronous Replication
Copyright 2012 Oracle Corporation - 26th July 2012 12
13. WHILE (cluster.measurePerformance() < target)
• Don’t optimize for the sake of it
• Wastes effort
• Introduces unnecessary compromises/complications
• Forget about database benchmarks
• You care about the end-to-end performance of your
application on your database
• If possible drive your application to drive the database
• Measurements need to be based on
representative traffic
• Measurements need to be repeatable
• Easily see impact of each optimization
Copyright 2012 Oracle Corporation - 26th July 2012 13
14. Where has the time gone?
• Enable the slow query log
– set global slow_query_log=1;
– set global long_query_time=3; //3 seconds
– set global log_queries_not_using_indexes=1;
– Slow queries will be written in the slow query log:
mysql> show global variables like 'slow_query_log_file';
+---------------------+------------------------------+
| Variable_name | Value |
+---------------------+------------------------------+
| slow_query_log_file | /data1/mysql/mysqld-slow.log |
+---------------------+------------------------------+
• Queries will be written in plain text
Copyright 2012 Oracle Corporation - 26th July 2012 14
15. Query Analyzer in MySQL Enterprise
Monitor (take the easy option!)
Copyright 2012 Oracle Corporation - 26th July 2012 15
16. Simple database traffic generation
create.sql:
CREATE TABLE sub_name (sub_id INT NOT NULL PRIMARY KEY, name
VARCHAR(30)) engine=ndb;
CREATE TABLE sub_age (sub_id INT NOT NULL PRIMARY KEY, age INT)
engine=ndb;
INSERT INTO sub_name VALUES (1,'Bill'),(2,'Fred'),(3,'Bill'),
(4,'Jane'),(5,'Andrew'),(6,'Anne'),(7,'Juliette'),(8,'Awen'),
(9,'Leo'),(10,'Bill');
INSERT INTO sub_age VALUES (1,40),(2,23),(3,33),(4,19),(5,21),
(6,50),(7,31),(8,65),(9,18),(10,101);
query.sql:
SELECT sub_age.age FROM sub_name, sub_age WHERE
sub_name.name='Bill' AND sub_name.sub_id=sub_age.sub_id;
Copyright 2012 Oracle Corporation - 26th July 2012 16
17. Simple database traffic generation
shell> mysqlslap --concurrency=5 --
iterations=100 --query=query.sql --
create=create.sql
Benchmark
Average number of seconds to run all queries: 0.132 seconds
Minimum number of seconds to run all queries: 0.037 seconds
Maximum number of seconds to run all queries: 0.268 seconds
Number of clients running queries: 5
Average number of queries per client: 1
Copyright 2012 Oracle Corporation - 26th July 2012 17
18. What on Earth is it doing?
EXPLAIN is your friend
mysql> EXPLAIN <query>;
mysql> EXPLAN PARTITIONS <query>;
mysql> EXPLAIN EXTENDED <query>;
mysql> SHOW WARNINGS;
Copyright 2012 Oracle Corporation - 26th July 2012 18
20. Before you do anything else, ANALYZE!
• New for MySQL Cluster 7.2
• Lets the MySQL optimizer figure out how to best use
indexes etc.
• Instantly speed up queries by many times
mysql> ANALYZE TABLE <tab-name>;
• Repeat after changing schema, adding/removing
indexes or making major data changes
• Only needs running on one mysqld in the cluster
Copyright 2012 Oracle Corporation - 26th July 2012 20
21. Access patterns
• Primary key reads/writes -> O(1)
• Independent of database size and number of nodes
• Index searched -> O(log n)
• n = number of tuples
• BLOBs are stored in second table -> take longer to
access
• JOINs massively faster in MySQL Cluster 7.3
• Partition pruning
• By allowing a query to be satisfied with a single data node,
reduce resource consumption -> greater throughput
• If result sets not large, will also reduce latency
Copyright 2012 Oracle Corporation - 26th July 2012 21
22. Adaptive Query Localization
Scaling Distributed Joins 70x
More
Performance
• Perform Complex Queries
mysqld across Shards
• JOINs pushed down to data nodes
A Data Nodes
• Executed in parallel
Q • Returns single result set to MySQL
L
• Opens Up New Use-Cases
• Real-time analytics
• Recommendations engines
mysqld
• Analyze click-streams
Data Nodes DON’T COMPROMISE
FUNCTIONALITY TO SCALE-OUT !!
Copyright 2012 Oracle Corporation - 26th July 2012 22
23. MySQL Cluster 7.2 AQL Test Query
Web-Based Content Management System
MySQL
Server
Data Data
Node1 Node2
Copyright 2012 Oracle Corporation - 26th July 2012 23
24. Web-Based CMS
Query Execution Time Seconds
70x
More
100
Performance
90
80
70
87.23 seconds
60
50
40
30
20
10 1.26 seconds
0
MySQL Cluster 7.1 MySQL Cluster 7.2
Must Analyze tables for best results
mysql> ANALYZE TABLE <tab-name>;
Copyright 2012 Oracle Corporation - 26th July 2012 24
25. Did I mention ANALYZE TABLE?
Copyright 2012 Oracle Corporation - 26th July 2012 25
26. AQL – How to Use it
• Activated when ndb_join_pushdown is on (default)
• Rules for a Join to be pushed down:
1. Joined columns use identical types
2. No reference to BLOB or TEXT columns
3. No explicit lock
4. Child tables in the Join must be accessed using one of the ref, eq_ref, or
const
5. Tables not explicitly partitioned by [LINEAR] HASH, LIST, or RANGE
6. Query plan doesn’t select ‘Using join buffer'
7. If root of Join is an eq_ref or const, child tables must be joined by eq_ref
• Run ANALYZE TABLE <tab-name> on each table once
• Use EXPLAIN to see what components are being pushed down:
• Extra: Child of 'd' in pushed join@1
• EXPLAIN EXTENDED <query>;SHOW WARNINGS;
Copyright 2012 Oracle Corporation - 26th July 2012 26
27. Distribution Aware Apps
SELECT SUM(population) FROM towns • Partition selected using hash on
WHERE country=“UK”;
Partition Key
Partition Key
Primary Key
• Primary Key by default
town country population • User can override in table definition
Maidenhead UK 78000 • MySQL Server (or NDB API) will
Paris France 2193031 attempt to send transaction to the
Boston UK 58124
correct data node
Boston USA 617594
• If all data for the transaction are in
SELECT SUM(population) FROM towns the same partition, less messaging -
WHERE town=“Boston”; > faster
Partition Key • Aim to have all rows for high-running
Primary Key queries in same partition
town country population
Maidenhead UK 78000
Paris France 2193031
Boston UK 58124
Boston USA 617594
Copyright 2012 Oracle Corporation - 26th July 2012 27
28. Distribution Aware – Multiple Tables
Partition Key
Primary Key
sub_id age gender • Extend partition awareness over
19724 25 male multiple tables
84539 43 female
• Same rule – aim to have all data for
19724 16 female
instance of high running
74574 21 female
transactions in the same partition
Partition Key
Primary Key ALTER TABLE service_ids
service sub_id svc_id PARTITION BY KEY(sub_id);
twitter 19724 76325732
twitter 84539 67324782
facebook 19724 83753984 EXPLAIN PARTITIONS <query>;
facebook 73642 87324793
Copyright 2012 Oracle Corporation - 26th July 2012 28
29. Validate if “partition pruning” is working
mysql> SHOW GLOBAL STATUS LIKE 'ndb_pruned_scan_count';
+-----------------------+-------+
| Variable_name | Value |
+-----------------------+-------+
| Ndb_pruned_scan_count | 12 |
+-----------------------+-------+
mysql> SELECT * FROM services WHERE sub_id=1;
+--------+--------------+--------------+
| sub_id | service_name | service_parm |
+--------+--------------+--------------+
| 1 | IM | 878 |
| 1 | ssh | 666 |
| 1 | Video | 654 |
+--------+--------------+--------------+
mysql> SHOW GLOBAL STATUS LIKE 'ndb_pruned_scan_count';
+-----------------------+-------+
| Variable_name | Value |
+-----------------------+-------+
| Ndb_pruned_scan_count | 13 |
+-----------------------+-------+
Copyright 2012 Oracle Corporation - 26th July 2012 29
30. Batching
• MySQL Cluster allows batching on
• Inserts, index scans (when not part of a JOIN), PK reads, PK deletes,
and PK (most) updates.
• Batching means that one network round trip is used to read/modify a
number of records → less ping-pong!
• If you can batch - do it!
• Example – Insert 1M records
• No batching:
• INSERT INTO t1(data) VALUES (<data>);
• 765 seconds to insert 1M records
• Batching (batches of 16):
• INSERT INTO t1(<columns>) VALUES (<data>),
(<data>)...
• 50 seconds to insert 1M records
Copyright 2012 Oracle Corporation - 26th July 2012 30
31. Batching, more examples
SELECT * FROM t1 WHERE userid=1 AND serviceid
IN (1,2,3,4,5,7,8,9,10);
SET transaction_allow_batching=1; //must be set
on the connection
BEGIN;
INSERT INTO t1 ....;
INSERT INTO t2 ....;
INSERT INTO t3 ....;
INSERT INTO t4 ....;
DELETE FROM t5 ....;
UPDATE t1 SET value='new value' WHERE id=1;
COMMIT;
Copyright 2012 Oracle Corporation - 26th July 2012 31
32. Optimizing schema - denormalization
userid
voip_data
userid
bb_data
SELECT * FROM bb,voip WHERE 1
<data>
1
<data>
2
<data>
2
<data>
bb.userid=voip.userid AND 3
<data>
3
<data>
bb.userid=1; 4
<data>
4
<data>
voip bb
userid
voip_data
bb_data
1
<data>
<data>
mysql> SELECT * FROM 2
<data>
<data>
voip_bb WHERE userid=1; 3
<data>
<data>
4
<data>
<data>
1.7x improvement
voip_bb
Copyright 2012 Oracle Corporation - 26th July 2012 32
33. Connection Pools
• Network hops increase latency (e.g.
Compared with InnoDB read of cached
data)
App thread
App thread
App thread
App thread
App thread
App thread
• Increase throughput by sending in lots of
parallel operations
• Multiple client connections (sessions) to each
MySQL Server
mysqld mysqld
• Multiple MySQL Servers
NDB API NDB API
• Connection pooling between MySQL
Servers and data nodes
• Set ndb-cluster-connection-pool > 1
in my.cnf
• Ensure enough [api] sections in
Data Nodes
config.ini
• Don’t assign hostnames!
Copyright 2012 Oracle Corporation - 26th July 2012 33
34. Multi-threaded data nodes
• Scaling out on commodity
hardware is the standard
Application Nodes way to increase
performance
• Add more data nodes and
Node 3
API nodes as required
• MySQL Cluster 7.2
Node 1
increases the ability to also
scale-up each data node
• Increases maximum
number of utilised threads
from 8 to 59
Node 2
Node 4
Node Group 1 Node Group 2
Copyright 2012 Oracle Corporation - 26th July 2012 34
35. Multi-threaded data nodes
• Threads:
• ldm: 1,2,4,8 or 12 Local
Application Nodes
Query Handler threads
• tc: typically ldm/4
Transaction
Coordinator threads
• send: ~2-3 Send
Data Node 1
recv send main threads
• recv: ~2-4 Receive
threads
• main: 1 Main thread
• rep: 1 Replication
tc ldm rep io thread
• io: 1 I/O thread
Copyright 2012 Oracle Corporation - 26th July 2012 35
36. Multi-threaded data nodes
• Applies to ntbdmtd only
• Configure through either:
• MaxNoOfExecutionThreads
• Single value for number of threads
• System will allocate these threads to blocks in a reasonable
way
• ThreadConfig
• Specify explicitly how many threads for each block type
• Lock threads to CPUs for further performance gains
• Threadconfig=main={cpubind=0},ldm=
{count=4,cpubind=1,2,5,6},io=
{count=2,cpubind=3,4}
Copyright 2012 Oracle Corporation - 26th July 2012 36
37. Multi-threaded data nodes – starting
point for ThreadConfig
24 threads 32 threads 40 threads 48 thread
ldm 8 12 16 16
tc 4 6 8 12
recv 3 3 4 5
send 3 3 4 4
rep 1 1 1 1
Note that some threads are left for other data node
blocks as well as the OS
Copyright 2012 Oracle Corporation - 26th July 2012 37
39. Hardware
• High bandwidth, low latency network
• Turn off firewalls if you can
• Use multiple disks
• Checkpoints
• Log files
• Table spaces
• SSDs
• Biggest benefit for Table spaces
• Refer to MySQL Cluster Evaluation Guide for
more details
Copyright 2012 Oracle Corporation - 26th July 2012 39
40. MySQL Query Cache
• Don't enable the Query Cache!
• It is very expensive to invalidate over multiple MySQL servers
• A write on one server will force the others to purge their cache.
• If you have tables that are read only (or change very
seldom):
my.cnf:
query_cache_size=1000000
query_cache_type=2 (ON DEMAND)
mysql> SELECT SQL_CACHE <cols> .. FROM table;
• SQL_CACHE tells (demands) MySQL to cache the results
from this SELECT
• This can be good for STATIC data
Copyright 2012 Oracle Corporation - 26th July 2012 40
41. Non-Durable tables
• Some types of tables account for a lot of WRITEs, but do not
need to be recovered (e.g, Session tables)
• Unnecessary to persist such tables - no REDO LOGs or
CHECKPOINTs
• Create these tables as 'NO LOGGING' tables:
mysql> set ndb_table_no_logging=1;
mysql> create table session_table
(..) engine=ndb;
mysql> set ndb_table_no_logging=0;
• After system restart table will be there, but empty!
Copyright 2012 Oracle Corporation - 26th July 2012 41
42. More optimisation tips
• When using auto-increment columns, increase ndb-
autoincrement-prefetch-sz
• Set RedoBuffer=32-64M
• Disk-based tables:
• Increase UNDO_BUFFER for write-intensive apps
• Increase DiskIOThreadPool
• Increase DiskPageBufferMemory for better caching in the data
nodes; Monitor effectiveness using NDBINFO / MEM
• FragmentLogFileSize=256M
• NoOfFragmentLogFiles= 6 x DataMemory (in
MB) / (4x 256MB)
• Use OPTIMIZE TABLE and perform rolling restarts if
memory fragmentation is an issue
Copyright 2012 Oracle Corporation - 26th July 2012 42
43. Scaling out with MySQL Cluster Manager
client
mysqld mysqld
ndb_mgmd ndb_mgmd
agent agent
192.168.0.10 192.168.0.11
ndbd ndbd
ndbd ndbd
agent agent
192.168.0.12 192.168.0.13
Copyright 2012 Oracle Corporation - 26th July 2012 43
47. Further resources
• MySQL Cluster Performance white paper:
http://www.mysql.com/why-mysql/white-papers/
mysql_wp_cluster_performance.php
• MySQL Cluster Forum:
http://forums.mysql.com/list.php?25
• MySQL Cluster Evaluation guide:
http://www.mysql.com/why-mysql/white-papers/
mysql_cluster_eval_guide.php
• MySQL Cluster in Web-Scale Architectures:
http://www.mysql.com/why-mysql/white-papers/
mysql_cluster_eval_guide.php