The document discusses several technologies for using MySQL as a NoSQL database system by bypassing SQL overhead: HandlerSocket allows direct access to indexes; the NDB API provides a NoSQL interface to MySQL Cluster; the BLOB Streaming Engine improves BLOB handling; the Handler Interface skips parsing and optimization; and the OQGRAPH storage engine enables graph queries and computations. Benchmarks show these techniques providing significant performance gains over standard SQL.
Indexing with MySQL. What indexes are, where to use them and what to care about.
Indexierung mit MySQL. Was Indizes sind, wo man sie nutzen sollte und worauf man achten sollte.
This tutorial covers all parallel replication implementation in MariaDB 10.0 and 10.1 and MySQL 5.6, 5.7 and 8.0 (including how it works in Group Replication).
MySQL and MariaDB have different types of parallel replication. In this tutorial, we present the different implementations that allow us to understand their limitations and tuning parameters. We cover how to make parallel replication faster and what to avoid for maximizing its benefits. We also present tests from Booking.com workloads.
Some of the subjects that are covered are group commit and optimistic parallel replication in MariaDB, the parallelism interval of MySQL and its Write Set optimization, and the ?slowing down the master to speed up the slave? optimization.
After this tutorial, you will know everything you need to implement and tune parallel replication in your environment. But more importantly, we will show how you can test parallel replication benefit in a non-disruptive way before deployment.
Since 5.7.2, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK (DATABASE based parallel replication is also implemented in 5.6 but this is not covered in this talk). In early 5.7 versions, parallel replication was based on group commit (like MariaDB) and 5.7.6 changed that to intervals.
Intervals are more complicated but they are also more powerful. In this talk, I will explain in detail how they work and why intervals are better than group commit. I will also cover how to optimize parallel replication in MySQL 5.7 and what improvements are coming in MySQL 8.0.
MySQL Parallel Replication: inventory, use-case and limitationsJean-François Gagné
In the last 24 months, MySQL/MariaDB replication speed has improved a lot thanks to parallel replication. MySQL and MariaDB have different types of parallel replication; in this talk, I present the different implementations, with their limitations and the corresponding tuning parameters. I cover what to do to make parallel replication faster and what to avoid for maximizing parallel replication benefits. I also present benchmark results from real Booking.com workloads. Finally, I discuss some deployments at Booking.com that take advantage of parallel replication speed improvements.
Indexing with MySQL. What indexes are, where to use them and what to care about.
Indexierung mit MySQL. Was Indizes sind, wo man sie nutzen sollte und worauf man achten sollte.
This tutorial covers all parallel replication implementation in MariaDB 10.0 and 10.1 and MySQL 5.6, 5.7 and 8.0 (including how it works in Group Replication).
MySQL and MariaDB have different types of parallel replication. In this tutorial, we present the different implementations that allow us to understand their limitations and tuning parameters. We cover how to make parallel replication faster and what to avoid for maximizing its benefits. We also present tests from Booking.com workloads.
Some of the subjects that are covered are group commit and optimistic parallel replication in MariaDB, the parallelism interval of MySQL and its Write Set optimization, and the ?slowing down the master to speed up the slave? optimization.
After this tutorial, you will know everything you need to implement and tune parallel replication in your environment. But more importantly, we will show how you can test parallel replication benefit in a non-disruptive way before deployment.
Since 5.7.2, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK (DATABASE based parallel replication is also implemented in 5.6 but this is not covered in this talk). In early 5.7 versions, parallel replication was based on group commit (like MariaDB) and 5.7.6 changed that to intervals.
Intervals are more complicated but they are also more powerful. In this talk, I will explain in detail how they work and why intervals are better than group commit. I will also cover how to optimize parallel replication in MySQL 5.7 and what improvements are coming in MySQL 8.0.
MySQL Parallel Replication: inventory, use-case and limitationsJean-François Gagné
In the last 24 months, MySQL/MariaDB replication speed has improved a lot thanks to parallel replication. MySQL and MariaDB have different types of parallel replication; in this talk, I present the different implementations, with their limitations and the corresponding tuning parameters. I cover what to do to make parallel replication faster and what to avoid for maximizing parallel replication benefits. I also present benchmark results from real Booking.com workloads. Finally, I discuss some deployments at Booking.com that take advantage of parallel replication speed improvements.
MySQL Parallel Replication: inventory, use-case and limitationsJean-François Gagné
In the last 24 months, MySQL replication speed has improved a lot thanks to implementing parallel replication. MySQL and MariaDB have different types of parallel replication; in this talk, I present in details the different implementations, with their limitations and the corresponding tuning parameters. I also present benchmark results from real Booking.com workloads. Finally, I discuss some deployments at Booking.com that benefits from parallel replication speed improvements.
MySQL/MariaDB Parallel Replication: inventory, use-case and limitationsJean-François Gagné
In the last 24 months, MySQL/MariaDB replication speed has improved a lot, thanks to parallel replication. MySQL and MariaDB Server have different types of parallel replication; in this talk, I present the different implementations which will allow us to understand their limitations and tuning parameters. I am covering how to make parallel replication faster and what to avoid for maximizing its benefits. I also present benchmark results from Booking.com workloads. Finally, I discuss some deployments at Booking.com that take advantage of parallel replication speed improvements.
MySQL Parallel Replication: inventory, use-cases and limitationsJean-François Gagné
In the last 24 months, MySQL replication speed has improved a lot thanks to implementing parallel replication. MySQL and MariaDB have different types of parallel replication; in this talk, I present in detail the different implementations, with their limitations and the corresponding tuning parameters (covering MySQL 5.6, MariaDB 10.0, MariaDB 10.1 and MySQL 5.7). I also present benchmark results from real Booking.com workloads. Finally, I discuss some deployments at Booking.com that benefits from parallel replication speed improvements.
Riding the Binlog: an in Deep Dissection of the Replication StreamJean-François Gagné
Binary Logs are the cornerstone of MySQL Replication, but is it fully understood ? To start apprehending this, we can think of the binary logs as a transport for a Stream of Transactions. Traveling from master to slave, sometimes via Intermediate Masters, this stream evolves: it can shrink by the application of filters, can grow by the addition of slave-local transactions, and two streams can merge by the usage of multi-source replication. After presenting the binary logs Stream Model, the different MySQL use-cases will be mapped to the model, which can serve as a validation of the model. After this validation, the model will be used to make prediction on new use-cases/features that could emerge in the future.
MariaDB Server on macOS - FOSDEM 2022 MariaDB DevroomValeriy Kravchuk
Current MariaDB Server GA versions are formally not supported (and probably not even regularly built or tested) on macOS 10.x and 11.y. But it's relatively easy to set up the environment and build MariaDB Server from current 10.2 - 10.8 GitHub sources, with few minor issues to resolve in the process, depending on macOS and major server version used.
This talk is a summary of my related experience on 10.13 High Sierra that I had a chance to work on recently, with additional quick review of related fixed and open bugs, as well as some unique features like DTrace support that one may benefit from on macOS. Actually, studying DTrace in context of MariaDB Server troubleshooting and performance tuning was one of the goals why I started to use macOS again.
MySQL Parallel Replication (LOGICAL_CLOCK): all the 5.7 (and some of the 8.0)...Jean-François Gagné
Since 5.7.2, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK (DATABASE based parallel replication is also implemented in 5.6 but this is not covered in this talk). In early 5.7 versions, parallel replication was based on group commit (like MariaDB) and 5.7.6 changed that to intervals.
Intervals are more complicated but they are also more powerful. In this talk, I will explain in detail how they work and why intervals are better than group commit. I will also cover how to optimize parallel replication in MySQL 5.7 and what improvements are coming in MySQL 8.0. I will also explain why Group Replication is replicating faster than standard asynchronous replication.
Come to this talk to get all the details about MySQL 5.7 Parallel Replication.
More on bpftrace for MariaDB DBAs and Developers - FOSDEM 2022 MariaDB DevroomValeriy Kravchuk
bpftrace is a relatively new open source tracer for modern Linux (kernels 5.x.y) that may help to troubleshoot performance issues in production as well as to get insights on how software really works. I use it for a couple of years and would like to present more details on how to do it efficiently, including but not limited to adding user probes to different lines of the code inside functions, checking values of local variables and using bpftrace as a code coverage tool.
OpenNebulaConf2015 2.02 Backing up your VM’s with Bacula - Alberto GarcíaOpenNebula Project
How to use Bacula and live snapshot’s capabilities on OpenNebula to make backups of your virtual machines and store them.
Author Biography
Automate all the things! I love using any tool to make things to work automagically.
MySQL Parallel Replication: inventory, use-case and limitationsJean-François Gagné
In the last 24 months, MySQL replication speed has improved a lot thanks to implementing parallel replication. MySQL and MariaDB have different types of parallel replication; in this talk, I present in details the different implementations, with their limitations and the corresponding tuning parameters. I also present benchmark results from real Booking.com workloads. Finally, I discuss some deployments at Booking.com that benefits from parallel replication speed improvements.
MySQL/MariaDB Parallel Replication: inventory, use-case and limitationsJean-François Gagné
In the last 24 months, MySQL/MariaDB replication speed has improved a lot, thanks to parallel replication. MySQL and MariaDB Server have different types of parallel replication; in this talk, I present the different implementations which will allow us to understand their limitations and tuning parameters. I am covering how to make parallel replication faster and what to avoid for maximizing its benefits. I also present benchmark results from Booking.com workloads. Finally, I discuss some deployments at Booking.com that take advantage of parallel replication speed improvements.
MySQL Parallel Replication: inventory, use-cases and limitationsJean-François Gagné
In the last 24 months, MySQL replication speed has improved a lot thanks to implementing parallel replication. MySQL and MariaDB have different types of parallel replication; in this talk, I present in detail the different implementations, with their limitations and the corresponding tuning parameters (covering MySQL 5.6, MariaDB 10.0, MariaDB 10.1 and MySQL 5.7). I also present benchmark results from real Booking.com workloads. Finally, I discuss some deployments at Booking.com that benefits from parallel replication speed improvements.
Riding the Binlog: an in Deep Dissection of the Replication StreamJean-François Gagné
Binary Logs are the cornerstone of MySQL Replication, but is it fully understood ? To start apprehending this, we can think of the binary logs as a transport for a Stream of Transactions. Traveling from master to slave, sometimes via Intermediate Masters, this stream evolves: it can shrink by the application of filters, can grow by the addition of slave-local transactions, and two streams can merge by the usage of multi-source replication. After presenting the binary logs Stream Model, the different MySQL use-cases will be mapped to the model, which can serve as a validation of the model. After this validation, the model will be used to make prediction on new use-cases/features that could emerge in the future.
MariaDB Server on macOS - FOSDEM 2022 MariaDB DevroomValeriy Kravchuk
Current MariaDB Server GA versions are formally not supported (and probably not even regularly built or tested) on macOS 10.x and 11.y. But it's relatively easy to set up the environment and build MariaDB Server from current 10.2 - 10.8 GitHub sources, with few minor issues to resolve in the process, depending on macOS and major server version used.
This talk is a summary of my related experience on 10.13 High Sierra that I had a chance to work on recently, with additional quick review of related fixed and open bugs, as well as some unique features like DTrace support that one may benefit from on macOS. Actually, studying DTrace in context of MariaDB Server troubleshooting and performance tuning was one of the goals why I started to use macOS again.
MySQL Parallel Replication (LOGICAL_CLOCK): all the 5.7 (and some of the 8.0)...Jean-François Gagné
Since 5.7.2, MySQL implements parallel replication in the same schema, also known as LOGICAL_CLOCK (DATABASE based parallel replication is also implemented in 5.6 but this is not covered in this talk). In early 5.7 versions, parallel replication was based on group commit (like MariaDB) and 5.7.6 changed that to intervals.
Intervals are more complicated but they are also more powerful. In this talk, I will explain in detail how they work and why intervals are better than group commit. I will also cover how to optimize parallel replication in MySQL 5.7 and what improvements are coming in MySQL 8.0. I will also explain why Group Replication is replicating faster than standard asynchronous replication.
Come to this talk to get all the details about MySQL 5.7 Parallel Replication.
More on bpftrace for MariaDB DBAs and Developers - FOSDEM 2022 MariaDB DevroomValeriy Kravchuk
bpftrace is a relatively new open source tracer for modern Linux (kernels 5.x.y) that may help to troubleshoot performance issues in production as well as to get insights on how software really works. I use it for a couple of years and would like to present more details on how to do it efficiently, including but not limited to adding user probes to different lines of the code inside functions, checking values of local variables and using bpftrace as a code coverage tool.
OpenNebulaConf2015 2.02 Backing up your VM’s with Bacula - Alberto GarcíaOpenNebula Project
How to use Bacula and live snapshot’s capabilities on OpenNebula to make backups of your virtual machines and store them.
Author Biography
Automate all the things! I love using any tool to make things to work automagically.
A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.
A video recording of this presentation is available from Ustream: http://ustre.am/fyLk
MySQL Cluster (NDB) - Best Practices Percona Live 2017Severalnines
This presentation by Johan Andersson at Percona Live 2017 in Santa Clara, California gives detailed information on all you need to know to effectively deploy and manage MySQL Cluster technology in your environment.
Introducing Galera Cluster & the Codership Team
Galera Cluster in a nutshell:
True multi-master:
Read & write to any node
* Synchronous replication
* No slave lag
* No integrity issues
* No master-slave failovers or VIP needed
* Multi-threaded slave, no performance penalty
* Automatic node provisioning
Elastic:
Easy scale-out & scale-in, all nodes read-write
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007eLiberatica
This is a presentation held at eLiberatica 2007.
http://www.eliberatica.ro/2007/
One of the biggest events of its kind in Eastern Europe, eLiberatica brings community leaders from around the world to discuss about the hottest topics in FLOSS movement, demonstrating the advantages of adopting, using and developing Open Source and Free Software solutions.
The eLiberatica organizational committee together with our speakers and guests, have graciously allowed media representatives and all attendees to photograph, videotape and otherwise record their sessions, on the condition that the photos, videos and recordings are licensed under the Creative Commons Share-Alike 3.0 License.
Presentation given at the GoSF meetup on July 20, 2016. It was also recorded on BigMarker here: https://www.bigmarker.com/remote-meetup-go/GoSF-EVCache-Peripheral-I-O-Building-Origin-Cache-for-Images
MariaDB/MySQL pitfalls - And how to come out again...FromDual GmbH
During the last conferences the audience was asking for more war stories than just new features.
In this presentation we have a look at the most often seen problems as a MariaDB/MySQL consultant in field.
MariaDB / MySQL tripping hazard and how to get out again?FromDual GmbH
MySQL is NOT MariaDB, KISS, MyISAM Table Locking, big ibdata1, Noisy Neighbours, No BLOBs in a RDBMS, Disk full!, Locking, Crash, Out of Memory, Oom, Restart, Crash with Stacktrace, Tables without a Primary Key, Server has gone away! NUMA, Slave Lag, Sending data, Too many connections, Simple Query Tuning, Galera Cluster, Progress bar
In this presentation we discuss the New Features of MariaDB 10.4. First we give a short overview of the MariaDB Branches and Forks. Then we talk about the announced IPO. Technically we cover topics like Authentication, Accounts, InnoDB, Optimizer improvements, Application-Time Period Tables the new Backup Stage Galera 4 and other changes...
MariaDB 10.4 became General Available (GA = ready for production) this summer. So it is time to look at the new Features in MariaDB 10.4. After a short intro about history we look for the reason of broad usage of MariaDB nowadays. Most important improvements where in User Authentication, InnoDB improvements, and Optimizer enhancements. A completely New Feature is Application-Time Period Tables. Backup got a new Locking behaviour so LVM snapshots are possible and officially supported now. And last but not least MariaDB 10.4 comes with Galera 4.
MariaDB 10.2 New Features for Developers, Administrators and DevOps. Window Functions, Common Table Expressions, Check Constraints, GeoJSON, GIS, JSON, Oracle compatibility and MariaDB Connectors
Der Datenbank-Backup ist gemacht - was nun?FromDual GmbH
* Datenbank-Backup – welcher Zweck?
* Tauglichkeit des Backup, Verifikation
* Echtdaten vollständig nutzen
* Dem Datenschutz genügen
* Material für die Entwicklung
* Automatisierung
Weltweite Produktionsdatenverwaltung mit MySQL-ReplikationFromDual GmbH
Weltweite Produktionsdatenverwaltung mit MySQL
* Ausgangslage
* Probleme die sich stellen
* Wer darf welche Daten kriegen?
* Wie werden Daten verteilt?
* Produktionsdaten zurück?
* MySQL multi-Source Replikation
* Bertriebsverantwortung
* Hochverfügbarkeit
* Sensitive Daten
* Wer darf welche Daten sehen?
* MySQL Row Filterung
* Nachträgliche Forderungen
MySQL Performance Tuning für Oracle-DBA'sFromDual GmbH
MySQL Performance Tuning
* Was ist Performance?
* Was kostet Performance?
* Tuning Massnahmen
* MySQL Konfiguration
* Wo schauen?
* Langsame Abfragen finden
* Optimiere das Query!
* Monitoring
Reading MySQL fingerprints
Every MySQL Database has its typical fingerprints. They can be seen with SHOW GLOBAL STATUS;
How you interprete them you will find in this presentation.
MySQL indexing is one of the areas where you can achieve the most performance gains. In this presentation we have a short look how we can improve MySQL performance with indexing...
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
1. HandlerSocket and similar Technologies -
NoSQL for MySQL
FOSDEM, Brussels
5./6. February 2011
Oli Sennhauser
Senior MySQL Consultant at FromDual
oli.sennhauser@fromdual.com
www.fromdual.com 1
3. About FromDual
● We provide neutral and vendor independent:
● Consulting (on-site and remote)
● Remote-DBA / MySQL Operation Services
● Support
● Training (DBA, Performance Tuning, Scale-Out, High
Availability, MySQL Cluster)
● We are consulting partner of the Open Database
Alliance (ODBA.org)
● Oracle Silver Partner (OPN)
http://www.fromdual.com
www.fromdual.com 3
5. What is the problem?
Storage Engine part
SQL overhead
● SQL overhead is up to 70-80% for simple queries (~ 1 ms)!
● with NO SQL → we could be up to 5 times faster
● SQL is made for complex queries
● NoSQL typically solves simple queries
www.fromdual.com 5
6. Where does this overhead come
from?
Application / Client
Thread Connection mysqld
Cache Manager Parser
Optimizer
User Au-
thentication
Access Control
Command Table Open
Logging
Dispatcher Cache (.frm, fh)
Table Manager
Query Query Cache Table Definition
Cache Module Cache (tbl def.)
Handler Interface
MyISAM InnoDB Memory NDB PBXT Aria XtraDB Federated-X ...
www.fromdual.com 6
7. What can we do about?
● HandlerSocket (2010)
● NDB-API (1997!)
● PrimeBase Streaming Engine (2008)
● Handler Interface (2001/2011)
● OQGRAPH SE (2009)
www.fromdual.com 7
8. HandlerSocket
● October 20, 2010, Yoshinori Matsunobu:
Using MySQL as a NoSQL - A story for ex-
ceeding 750,000 qps on a commodity server
www.fromdual.com 8
10. Infos
● Compile yourself (easy!)
● 7.5 times more throughput?!?
● Works with 5.5.8 and MariaDB
● Faster than mem-
cached!?!
● In Percona-Server
12.3
www.fromdual.com 10
11. Features / functionality
● Different query patterns (see also handler interface later)
● Lots of concurrent connections
● Highly performant (200 – 700%)
● No duplicate cache (MySQL and memcached)
● No data inconsistency (MySQL and memcache)
● Crash-safe
● SQL access as well (for complex queries like reporting)
● No need to modify/rebuild MySQL ?!?
● Theoretically works for other storage engines as well (I did not
test it).
www.fromdual.com 11
12. NDB-API
● 1997, Mikael Ronström: The NDB Cluster – A
parallel data server for telecommunications
applications
● November 25, 2008, Jonas Oreland: 950'000
reads per second on 1 datanode
www.fromdual.com 12
13. MySQL Cluster
Application Application Application Application Application
NDB-API NDB-API
Load balancer
SQL Node 1 SQL Node 2 SQL Node 3
...
Mgm Node 1
Mgm Node 2
Data Node 1 Data Node 2
Sw.
Sw.
Data Node 3 Data Node 4
www.fromdual.com 13
16. Benchmarks and numbers
● ./flexAsynch -ndbrecord -temp -con 4 -t 16
-p 312 -l 3 -a 2 -r 2
● From the MySQL Cluster test suite
(src/storage/ndb/test/ndbapi)
● 16 number of concurrent threads (-t) ● 4 concurrent connections (-con)
312 number of parallel operation per thread
●
● 2 number of records (-r ?)
● 3 iterations (-l)
● 1 32-bit word per attribute
● 2 attributes per table (8 bytes) (-a)
● 1 ndbmtd (1 NoOfRepl.)
insert average: 506506/s min: 497508/s max: 522613/s stddev: 2%
update average: 503664/s min: 495533/s max: 507833/s stddev: 1%
delete average: 494225/s min: 474705/s max: 518272/s stddev: 3%
read average: 980386/s min: 942242/s max: 1028006/s stddev: 2%
www.fromdual.com 16
17. Learnings
● Observations
● CPU's not maxed out. "Somewhere" is potential !?!
● When you overdo: CPU is maxed out and performance
drops to 14%!
● The fakes:
● Look at the parameters!
● All other setups: I got worse throughput
● IP instead of localhost: 89% throughput
● Learnings:
● Do not trust benchmarks you did not fake yourself!
www.fromdual.com 17
18. BLOB Streaming Project
● April 2008, Paul McCullagh: Introduction to
the BLOB Streaming Project
● March 5 2010, Barry Leslie: Upload 1000+
BLOB's per second!
www.fromdual.com 18
20. Advantages of BLOB's in the
database
● old: RDBMS are not fast in storing BLOB's
→ do NOT store BLOB's in the databases
● new: With NoSQL technologies it becomes much
better!
● With PBSE: atomic transactions → No “dangling”
references.
● BLOB's in the normal database Backup !?!
● BLOB's can be replicated
● BLOB's in the DB will scale better. Most file systems
perform poorly when the number of files exceeds 2
million ?
www.fromdual.com 20
21. The Handler Interface
● October 2001, MySQL manual: A new
HANDLER interface to MyISAM tables
● December 27, 2010, Stephane Varoqui:
Using MySQL as a NoSQL: a story for
exceeding 450'000 qps with MariaDB
● January 10, 2011, Stephane Varoqui: 20% to
50% improvement in MariaDB 5.3 Handler
Interface using prepared statement
www.fromdual.com 21
23. HANDLER Example
# MySQL
# SELECT * FROM family;
HANDLER tbl OPEN
HANDLER family OPEN;
HANDLER tbl READ idx (..., ..., …)
HANDLER family
WHERE ... LIMIT ...
READ `PRIMARY` = (id)
WHERE id = 1;
HANDLER tbl READ idx FIRST
HANDLER family CLOSE;
WHERE ... LIMIT ...
HANDLER tbl READ idx NEXT
WHERE ... LIMIT ...
# With MariaDB 5.3
HANDLER tbl READ idx PREV
WHERE ... LIMIT ...
HANDLER family OPEN;
HANDLER tbl READ idx LAST
PREPARE stmt
WHERE ... LIMIT ...
FROM 'HANDLER family
READ `PRIMARY` = (id)
HANDLER tbl READ FIRST
WHERE id = ?';
WHERE ... LIMIT ...
set @id=1;
HANDLER tbl READ NEXT
EXECUTE stmt USING @id;
WHERE ... LIMIT ...
DEALLOCATE PREPARE stmt;
HANDLER family CLOSE;
HANDLER tbl CLOSE
Use persistent connections!!!
www.fromdual.com 23
24. Characteristics of the Handler
Interface
● HANDLER is faster than SELECT:
● Less parsing involved
● No optimizer overhead
● Less query-checking overhead
● The table does not have to be locked between two handler
requests
● No consistent look of the data (dirty reads are permitted)
● Some optimizations possible that SELECT does not allow
● Traverse the database in a manner that is difficult (or even
impossible) to accomplish with SELECT
www.fromdual.com 24
25. A Graph Storage Engine
● May 5, 2009, Arjen Lentz: OQGRAPH
Computation Engine for MySQL, MariaDB &
Drizzle
● It's included in MariaDB 5.1 ff.
● And available for MySQL 5.0 ff.
www.fromdual.com 25
26. How does it feel?
● It is similar to MEMORY SE (persistency, locking, trx)
● We talk in: Node
● Node/Item/vertex and
● Edge/connection/link
● Edges have a direction
Edge
● We can do networks and hierarchies
● (family relations, friends of friends, fastest way from a to b)
● To talk to the OQGRAPH SE we use the latches (which
algorithm to use)
● It is a computational engine not a storage engine!
www.fromdual.com 26
27. Simple example: My family
SELECT f1.name AS parent, f2.name AS child
INSERT INTO family VALUES FROM relation AS r
(1, 'Grandgrandma') JOIN family f1 ON f1.id = r.origid
, (2, 'Grandma') JOIN family f2 ON f2.id = r.destid;
, (3, 'Granduncle')
, (4, 'Grandaunt') +++
, (5, 'Grandpa') | parent | child |
, (6, 'Mother') +++
, (7, 'Uncle 1') | Grandgrandma | Grandma |
, (8, 'Uncle 2') | Grandgrandma | Granduncle |
, (9, 'Father') | Grandgrandma | Grandaunt |
, (10, 'Me') | Grandma | Mother |
, (11, 'Sister'); | Grandma | Uncle 1 |
| Grandma | Uncle 2 |
INSERT INTO relation (origid, destid) | Grandpa | Mother |
VALUES | Grandpa | Uncle 1 |
(1, 2), (1, 3), (1, 4) | Grandpa | Uncle 2 |
, (2, 6), (2, 7), (2, 8) | Mother | Me |
, (5, 6), (5, 7), (5, 8) | Mother | Sister |
, (6, 10), (6, 11) | Father | Me |
, (9, 10), (9, 11); | Father | Sister |
+++
www.fromdual.com 27
28. Network queries
SELECT r.weight, r.seq, f.name
FROM relation AS r
SELECT GROUP_CONCAT(f.name SEPARATOR ' > ') JOIN family AS f ON (r.linkid = f.id)
AS path
AS path WHERE r.latch = 2
FROM relation AS r
FROM relation AS r
AND r.destid = 10;
JOIN family AS f ON (r.linkid = f.id)
JOIN family AS f ON (r.linkid = f.id)
WHERE latch = 1
AND origid = 1
AND origid = 1
AND destid = 10
AND destid = 10 ++++
ORDER BY seq; | weight | seq | name |
++++
| 3 | 6 | Grandgrandma |
++ | 2 | 5 | Grandpa |
| path | | 2 | 4 | Grandma |
++ | 1 | 3 | Father |
| Grandgrandma > Grandma > Mother > Me | | 1 | 2 | Mother |
++
| 0 | 1 | Me |
++++
latch = 1: Find shortest path (Dijkstra)
latch = 1: Find shortest path (Dijkstra)
latch = 2: Find originating nodes
(Breadthfirst search)
www.fromdual.com 28
29. Resume
● SQL is good for complex queries
● NoSQL is typically for simple queries
● Be careful with performance numbers!
● Architecture / coding becomes more
complex
● You can gain better performance
● But it is interesting like hell!
www.fromdual.com 29
30. Literature
● Using MySQL as a NoSQL - A story for exceeding 750,000 qps on a commodity server:
http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
● 950k reads per second on 1 datanode: http://jonasoreland.blogspot.com/2008/11/950k-reads-
per-second-on-1-datanode.html
● Scalable BLOB Streaming Infrastructure for MySQL and Drizzle:
http://www.blobstreaming.org/
● HandlerSocket: Why did our version not take off?
http://pbxt.blogspot.com/2010/12/handlersocket-why-did-out-version-did.html
● Using MySQL as a NoSQL: a story for exceeding 450000 qps with MariaDB:
http://varokism.blogspot.com/2010/12/using-mysql-as-nosql-story-for_27.html
● HANDLER Syntax: http://dev.mysql.com/doc/refman/5.5/en/handler.html
● GRAPH Computation Engine – Documentation: http://openquery.com/graph/doc
www.fromdual.com 30
31. Q&A
Questions ?
Discussion?
We have some slots free to provide
you personal consulting services...
www.fromdual.com 31