Scaling MySQL using Fabric

•Download as PPT, PDF•

1 like•3,686 views

This is my presentation on MySQL Fabric in LSPE meet up ( Large Scale Production Engineering ) held at Yahoo! India on 13-09-2014.

Technology

Scaling Persistent Store Using
MySQL FABRIC
P.R.KARTHIK
MySQL DBA

About US
Karthik.P.R
• 4 + years of Industry experience as MySQL DBA.
• 1+ year at Yahoo! Administrating MySQL servers.
• Area of Focus MySQL HA and MySQL Sharding
• Student : M.Sc ( FOSS)
• Blogger : remotemysqldba.blogspot.in

PROGRAM AGENDA
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric

Scaling MySQL
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
• Handling Shards

Scaling MySQL
• Starts with a single node

Scaling MySQL
• As reads and writes grows
Scale Vertically.
• Increase system resources
• Memory
• CPU
• Hard Disk

Scaling MySQL
Scaling reads using replication.
• Replication is the best solution.
• Replication is Async.
• Failover has to be set.
• Single writer
Write Queries Read Queries

Scaling MySQL
Scaling reads using Galera
• Read/ Write on any node
• Synchronous Replication
• A good HA solution.
• Easy to add nodes.
Write / Read
Queries

Scaling MySQL
Scaling Writes
• Replication can’t scale writes
• Partitioning is needed.
• Distributes the writes
• Horizontal scaling or Sharding.
ID 1 - 1000 ID 1001 - 2000

Scaling MySQL
Components .
•Shard Key
– Range
– Hash
– List
•Meta data store
•Managing data set
•High Availability of shards.
•Database and schema changes.

Scaling MySQL
Sharding Architecture
Proxy layer
State Store
Shard A
Shard B
Shard C
Application

Scaling MySQL
Advantages of Sharding
•Improved Performance
•Smaller Data set on local node.
•Handling large data set
•Scale well horizontally.
•Only small data set is affected on a node failure.
Disadvantages of Sharding
•Managing the shards
•Code change is need at some cases.
•Splitting Shards
•Maintaining the HA

Available Sharding Tools
• Sharding is used widely in Large Scale MySQL Deployments.
• Large organization built their own tools for sharding.
Popular MySQL Shards,
 Facebook
• Twitter
• Tumblr
• Flickr
• Youtube
• Dropbox

Available Sharding Tools
Sharding Tools.
1) Saclebase ( Closed Source)
2) Jetpants (Tumblr )
3) Vitess ( Youtube )
4) MySQL Fabric ( Oracle MySQL )

MySQL FABRIC
• MySQL Fabric is the frame work to maintain shards and High Availability.
• Downloaded from MySQL Utilities.
• Good CLI commands.
• Works on MySQL > 5.6.10
• XML RPC for Python , php and Java.
• Minimized downtime of shard.

MySQL FABRIC ARCHITECTURE
Application XML-RPC
Connector
XML-RPC
FABRIC
NODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store

MySQL FABRIC
FABRIC Components.
• Fabric node.
• Fabric aware connectors ( XML RPC )
• Server Groups.
– Global Group
– Shard Group
• Back Store

MySQL FABRIC
Fabric node
• Stores Fabric Config
• Shard Moving
• Shard splitting
• High Availability monitoring
Back Store
• Global tables info
• Shard key info and mapping
• Shard monitoring
Fabric Node Back store

MySQL FABRIC
Fabric aware connectors
• Supports Python/ php / Java Connectors
• Fetch Shard info
• Local cache
• Avoids the proxy hop.
Application XML-RPC
Connector

MySQL FABRIC
Server Groups
• Stores actual data
– Global Group
– Shard Group
• Primary – Read / Write
• Secondary – Reads ( failover)
• Spare ( On request )
Server Group
Primary Node Secondary Node
Spare Node

MySQL FABRIC
Server Groups ( Global Group )
• Stores Global Tables and schema
• Schema changes
• Replicates to Shard groups
Global Group Shard Group
Global Writes
Fabric Node

MySQL FABRIC ARCHITECTURE ( Again )
Application XML-RPC
Connector
XML-RPC
FABRIC
NODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store

MySQL FABRIC
Advantages:
• Ease of Failover with GTID.
• Shard Split is easy
• RPC avoid network hops.
Dis Advantages
• Fabric node will be single point of failure
• No Cross Shard . ( Shard query can help here)
• Limited Connectors.

What if you were asked to support a database platform that you had never worked with before? First you would probably say no, but after you lost that fight, then what? That is exactly how I came to support MySQL. Over the last year my team has worked to learn MySQL, architect a production environment, and figure out how to support it alongside our other platforms (Microsoft SQL Server and Oracle). Along the way, I have also come to appreciate the unique offering of this platform and see it as an important part of our environment going forward. To make things even more challenging, our first MySQL databases were the backend for a critical, web based application that needed to be highly available across multiple data centers. This meant that we did not have the luxury of standing up a simpler environment to start with and building confidence there. Our final architecture ended up using a five node Percona XtraDB Cluster spread across three data centers. This session will focus on lessons learned along the way, as well as challenges related to supporting more than one database platforms. It should be interesting to anyone who is new to MySQL, anyone who is being asked to support more than one database platform, or anyone who wants to see how an outsider views the platform.

Using MySQL in Automated TestingMorgan Tocker

Solr cloud the 'search first' nosql database extended deep dive

lucenerevolution

Presented by Mark Miller, Software Engineer, Cloudera As the NoSQL ecosystem looks to integrate great search, great search is naturally beginning to expose many NoSQL features. Will these Goliath's collide? Or will they remain specialized while intermingling – two sides of the same coin. Come learn about where SolrCloud fits into the NoSQL landscape. What can it do? What will it do? And how will the big data, NoSQL, Search ecosystem evolve. If you are interested in Big Data, NoSQL, distributed systems, CAP theorem and other hype filled terms, than this talk may be for you.

State transfer With Galera

Mydbops

The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale

Colin Charles

As proxies (and database routers) go, the first one I ever used was the now deprecated MySQL Proxy. Since then, I've managed to use MariaDB MaxScale quite a bit (including its fork AirBnB MaxScale), played around with ProxySQL in recent time, and also started taking a look at MySQL Router. In this quick 20-minute overview, we'll discuss why these three exist, a feature comparison, and reasons when to use the right tool for the job.

Best practices for MySQL/MariaDB Server/Percona Server High Availability - presented at Percona Live Amsterdam 2016. The focus is on picking the right High Availability solution, discussing replication, handling failure (yes, you can achieve a quick automatic failover), proxies (there are plenty), HA in the cloud/geographical redundancy, sharding solutions, how newer versions of MySQL help you, and what to watch for next.

Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...

rhatr

You’ve got your Hadoop cluster, you’ve got your petabytes of unstructured data, you run mapreduce jobs and SQL-on-Hadoop queries. Something is still missing though. After all, we are not expected to enter SQL queries while looking for information on the web. Altavista and Google solved it for us ages ago. Why are we still requiring SQL or Java certification from our enterprise bigdata users? In this talk, we will look into how integration of SolrCloud into Apache Bigtop is now enabling building bigdata indexing solutions and ingest pipelines. We will dive into the details of integrating full-text search into the lifecycle of your bigdata management applications and exposing the power of Google-in-a-box to all enterprise users, not just a chosen few data scientists.

Ansible for large scale deployment

Karthik .P.R

Percona tool kit for MySQL DBA's

Karthik .P.R

Capacity planning for your data stores

Colin Charles

Databases in the hosted cloud

Colin Charles

Today you can use hosted MySQL/MariaDB/Percona Server in several "cloud providers" in what is considered using it as a service, a database as a service (DBaaS). You can also use hosted PostgreSQL and MongoDB thru various service providers. Learn the differences, the access methods, and the level of control you have for the various public cloud offerings: - Amazon RDS for MySQL and PostgreSQL - Google Cloud SQL - Rackspace OpenStack DBaaS - The likes of compose.io, MongoLab and Rackspace's offerings around MongoDB The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include: * Different backup strategies * Planning for multiple data centres for availability * Where do you host your application? * How do you get the most performance out of the solution? * What does this all cost? Growth topics include: * How do you move from one DBaaS to another? * How do you move all this from DBaaS to your own hosted platform? Questions like this will be demystified in the talk. This talk will benefit experienced database administrators (DBAs) who now also have to deal with cloud deployments as well as application developers in startups that have to rely on "managed services" without the ability of a DBA.

MySQL 5.6 - Operations and Diagnostics ImprovementsMorgan Tocker

MySQL in the Hosted Cloud - Percona Live 2015

Colin Charles

Stream your Operational Data with Apache Spark & Kafka into Hadoop using Couc...

Data Con LA

Abstract:- Tracking user events as they happen can challenge anyone providing real time user interaction. It can demand both huge scale and a lot of processing to support dynamic adjustment to targeting products and services. As the operational data store Couchbase data services are capable of processing tens of millions of updates a day. Streaming through systems such as Apache Spark and Kafka into Hadoop, information about these key events can be turned into deeper knowledge. We will review Lambda architectures deployed at sites like PayPal, Live Person and LinkedIn that leverage a Couchbase Data Pipeline. Bio:- Justin Michaels. With over 20 years experience in deploying mission critical systems, Justin Michaels industry experience covers capacity planning, architecture and industry vertical experience. Justin brings his passion for architecting, implementing and improving Couchbase to the community as a Solution Architect. His expertise involves both conventional application platforms as well as distributed data management systems. He regularly engages with existing and new Couchbase customers in performance reviews, architecture planning and best practice guidance.

MHA: Getting started & moving past quirks percona live santa clara 2013Colin Charles

Scaling MySQL in Amazon Web Services

Laine Campbell

MySQL Live Migration - Common Scenarios

Mydbops

Apache Kafka at LinkedIn

Guozhang Wang

MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup

Colin Charles

Cassandra Introduction & Features

Phil Peace

MySQL HA Sharding-FabricAbdul Manaf

Using MySQL Fabric for High Availability and Scaling Out

OSSCube

MariaDB 10: The Complete Tutorial

Colin Charles

Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #5: Oracle’s InnoDB Cluster

Continuent

Oracle’s InnoDB Cluster vs. Continuent Tungsten Clusters for MySQL Building a Geo-Distributed, Multi-Region and Highly Available MySQL Cloud Back-End This is the fifth of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution. InnoDB Cluster uses MySQL’s group replication to handle the replication. It’s also known as semi-synchronous replication. Learn about this and more in this webinar! You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons! AGENDA - Goals for the High Noon Webinar Series - High Noon Series: Tungsten Clustering vs Others - Oracle InnoDB Cluster - Key Characteristics - Certification-based Replication - InnoDB Cluster Multi-Site Requirements - Limitations Using InnoDB Cluster - How to do better MySQL HA / DR / Geo-Distribution? - InnoDB Cluster vs Tungsten Clustering - About Continuent & Its Solutions PRESENTER Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.

What's hot

Backup and Recovery in MySQL Cluster

priyanka_sangam

Scalablity and benchmark in mysql performance

Amrendra Kumar

MySQL: From Single Instance to Big Data

Morgan Tocker

Best practices for MySQL/MariaDB Server/Percona Server High Availability

Colin Charles

Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...

rhatr

Ansible for large scale deployment

Karthik .P.R

Percona tool kit for MySQL DBA's

Karthik .P.R

Capacity planning for your data stores

Colin Charles

Databases in the hosted cloud

Colin Charles

MySQL 5.6 - Operations and Diagnostics ImprovementsMorgan Tocker

MySQL in the Hosted Cloud - Percona Live 2015

Colin Charles

Stream your Operational Data with Apache Spark & Kafka into Hadoop using Couc...

Data Con LA

MHA: Getting started & moving past quirks percona live santa clara 2013Colin Charles

Scaling MySQL in Amazon Web Services

Laine Campbell

MySQL Live Migration - Common Scenarios

Mydbops

Apache Kafka at LinkedIn

Guozhang Wang

MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup

Colin Charles

Cassandra Introduction & Features

Phil Peace

What's hot (18)

Backup and Recovery in MySQL Cluster

Scalablity and benchmark in mysql performance

MySQL: From Single Instance to Big Data

Best practices for MySQL/MariaDB Server/Percona Server High Availability

Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...

Ansible for large scale deployment

Percona tool kit for MySQL DBA's

Capacity planning for your data stores

Databases in the hosted cloud

MySQL 5.6 - Operations and Diagnostics Improvements

MySQL in the Hosted Cloud - Percona Live 2015

Stream your Operational Data with Apache Spark & Kafka into Hadoop using Couc...

MHA: Getting started & moving past quirks percona live santa clara 2013

Scaling MySQL in Amazon Web Services

MySQL Live Migration - Common Scenarios

Apache Kafka at LinkedIn

MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup

Cassandra Introduction & Features

Similar to Scaling MySQL using Fabric

MySQL HA Sharding-FabricAbdul Manaf

Using MySQL Fabric for High Availability and Scaling Out

OSSCube

MariaDB 10: The Complete Tutorial

Colin Charles

Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #5: Oracle’s InnoDB Cluster

Continuent

The Complete MariaDB Server tutorial

Colin Charles

Massively sharded my sql at tumblr presentation

kriptonium

Evan Ellis "Tumblr. Massively Sharded MySQL"

Alexey Mahotkin

Scaling with sync_replication using Galera and EC2

Marco Tusa

The Complete MariaDB Server Tutorial - Percona Live 2015

Colin Charles

Getting started with Riak in the Cloud

Ines Sombra

MySQL Options in OpenStack

Tesora

Optimizing MySQL for Cascade Server

hannonhill

MySQL NDB Cluster 8.0

Ted Wennmark

MariaDB: in-depth (hands on training in Seoul)

Colin Charles

OpenStack Days East -- MySQL Options in OpenStack

Matt Lord

In most production OpenStack installations, you want the backing metadata store to be highly available. For this, the de facto standard has become MySQL+Galera. In order to help you meet this basic use case even better, I will introduce you to the brand new native MySQL HA solution called MySQL Group Replication. This allows you to easily go from a single instance of MySQL to a MySQL service that's natively distributed and highly available, while eliminating the need for any third party library and implementations. If you have an extremely large OpenStack installation in production, then you are likely to eventually run into write scaling issues and the metadata store itself can become a bottleneck. For this use case, MySQL NDB Cluster can allow you to linearly scale the metadata store as your needs grow. I will introduce you to the core features of MySQL NDB Cluster--which include in-memory OLTP, transparent sharding, and support for active/active multi-datacenter clusters--that will allow you to meet even the most demanding of use cases with ease.

MySQL: Scale Through Consolidation Webinar

NetApp

Maria db 10 and the mariadb foundation(colin)

kayokogoto

How does Apache Pegasus (incubating) community develop at SensorsData

acelyc1112009

Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL

Continuent

MS Azure Database for MySQL vs. Continuent Tungsten Clusters Building a Geo-Scale, Multi-Region and Highly Available MySQL Cloud Back-End This is the third of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution. Azure Database for MySQL is a managed database cluster within Microsoft Azure Cloud that runs MySQL community edition. There are really two deployment options: “Single Server” and “Flexible Server (Preview).” We will look at the Flexible Server version, even though it is still preview, because most enterprise applications require failover, so this is the relevant comparison for Tungsten Clustering. You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons! Enjoy the webinar! AGENDA - Goals for the High Noon Webinar Series - High Noon Series: Tungsten Clustering vs Others - Microsoft Azure Database for MySQL - Key Characteristics - Certification-based Replication - Azure MySQL Multi-Site Requirements - Limitations Using Azure MySQL - How to do better MySQL HA / DR / Geo-Scale? - Azure MySQL vs Tungsten Clustering - About Continuent & Its Solutions PRESENTER Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.

The MySQL Server ecosystem in 2016

sys army

Similar to Scaling MySQL using Fabric (20)

MySQL HA Sharding-Fabric

Using MySQL Fabric for High Availability and Scaling Out

MariaDB 10: The Complete Tutorial

Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #5: Oracle’s InnoDB Cluster

The Complete MariaDB Server tutorial

Massively sharded my sql at tumblr presentation

Evan Ellis "Tumblr. Massively Sharded MySQL"

Scaling with sync_replication using Galera and EC2

The Complete MariaDB Server Tutorial - Percona Live 2015

Getting started with Riak in the Cloud

MySQL Options in OpenStack

Optimizing MySQL for Cascade Server

MySQL NDB Cluster 8.0

MariaDB: in-depth (hands on training in Seoul)

OpenStack Days East -- MySQL Options in OpenStack

MySQL: Scale Through Consolidation Webinar

Maria db 10 and the mariadb foundation(colin)

How does Apache Pegasus (incubating) community develop at SensorsData

Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL

The MySQL Server ecosystem in 2016

Recently uploaded

Video Streaming: Then, Now, and in the Future

Alpen-Adria-Universität

In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.

Enhancing Performance with Globus and the Science DMZ

Globus

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

UiPath Community Day Dubai: AI at Work..

UiPathCommunity

Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking. 📕 Curious on our agenda? Wait no more! 10:00 Welcome note - UiPath Community in Dubai Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank 10:20 A UiPath cross-region MEA overview Ashraf El Zarka, VP and Managing Director MEA, UiPath 10:35: Customer Success Journey Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank 11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more Boris Krumrey, Global VP, Automation Innovation, UiPath 12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services. Brendan Lingam, Director of Sales and Business Development, Marc Ellis

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Aggregage

A tale of scale & speed: How the US Navy is enabling software delivery from l...

sonjaschweigert1

Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved: - Reduction in onboarding time from 5 weeks to 1 day - Improved developer experience and productivity through actionable findings and reduction of false positives - Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO) Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production. We will cover: - How to remove silos in DevSecOps - How to build efficient development pipeline roles and component templates - How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence) - How to streamline operations with automated policy checks on container images

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

Elizabeth Buie - Older adults: Are we really designing for our future selves?

Nexer Digital

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

RESUME BUILDER APPLICATION Project for students

KAMESHS29

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Jen Stirrup

The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives? How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Albert Hoitingh

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

By Design, not by Accident - Agile Venture Bolzano 2024

Pierluigi Pugliese

Recently uploaded (20)

Video Streaming: Then, Now, and in the Future

Enhancing Performance with Globus and the Science DMZ

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

UiPath Community Day Dubai: AI at Work..

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Generative AI Deep Dive: Advancing from Proof of Concept to Production

A tale of scale & speed: How the US Navy is enabling software delivery from l...

FIDO Alliance Osaka Seminar: Overview.pdf

Elizabeth Buie - Older adults: Are we really designing for our future selves?

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

RESUME BUILDER APPLICATION Project for students

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

Removing Uninteresting Bytes in Software Fuzzing

GraphRAG is All You need? LLM & Knowledge Graph

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Epistemic Interaction - tuning interfaces to provide information for AI support

By Design, not by Accident - Agile Venture Bolzano 2024

Scaling MySQL using Fabric

1. Scaling Persistent Store Using MySQL FABRIC P.R.KARTHIK MySQL DBA

2. About US Karthik.P.R • 4 + years of Industry experience as MySQL DBA. • 1+ year at Yahoo! Administrating MySQL servers. • Area of Focus MySQL HA and MySQL Sharding • Student : M.Sc ( FOSS) • Blogger : remotemysqldba.blogspot.in

3. PROGRAM AGENDA • Scaling MySQL • Available Sharding Tools • MySQL Fabric

4. Scaling MySQL • Scaling MySQL • Available Sharding Tools • MySQL Fabric • Handling Shards

5. Scaling MySQL • Starts with a single node

6. Scaling MySQL • As reads and writes grows Scale Vertically. • Increase system resources • Memory • CPU • Hard Disk

7. Scaling MySQL Scaling reads using replication. • Replication is the best solution. • Replication is Async. • Failover has to be set. • Single writer Write Queries Read Queries

8. Scaling MySQL Scaling reads using Galera • Read/ Write on any node • Synchronous Replication • A good HA solution. • Easy to add nodes. Write / Read Queries

9. Scaling MySQL Scaling Writes • Replication can’t scale writes • Partitioning is needed. • Distributes the writes • Horizontal scaling or Sharding. ID 1 - 1000 ID 1001 - 2000

10. Scaling MySQL Components . •Shard Key – Range – Hash – List •Meta data store •Managing data set •High Availability of shards. •Database and schema changes.

11. Scaling MySQL Sharding Architecture Proxy layer State Store Shard A Shard B Shard C Application

12. Scaling MySQL Advantages of Sharding •Improved Performance •Smaller Data set on local node. •Handling large data set •Scale well horizontally. •Only small data set is affected on a node failure. Disadvantages of Sharding •Managing the shards •Code change is need at some cases. •Splitting Shards •Maintaining the HA

13. Available Sharding Tools • Sharding is used widely in Large Scale MySQL Deployments. • Large organization built their own tools for sharding. Popular MySQL Shards,  Facebook • Twitter • Tumblr • Flickr • Youtube • Dropbox

14. Available Sharding Tools Sharding Tools. 1) Saclebase ( Closed Source) 2) Jetpants (Tumblr ) 3) Vitess ( Youtube ) 4) MySQL Fabric ( Oracle MySQL )

15. MySQL FABRIC

16. MySQL FABRIC • MySQL Fabric is the frame work to maintain shards and High Availability. • Downloaded from MySQL Utilities. • Good CLI commands. • Works on MySQL > 5.6.10 • XML RPC for Python , php and Java. • Minimized downtime of shard.

17. MySQL FABRIC ARCHITECTURE Application XML-RPC Connector XML-RPC FABRIC NODE Global Group Group 1 Group 2 Group 3 Group 4 Back Store

18. MySQL FABRIC FABRIC Components. • Fabric node. • Fabric aware connectors ( XML RPC ) • Server Groups. – Global Group – Shard Group • Back Store

19. MySQL FABRIC Fabric node • Stores Fabric Config • Shard Moving • Shard splitting • High Availability monitoring Back Store • Global tables info • Shard key info and mapping • Shard monitoring Fabric Node Back store

20. MySQL FABRIC Fabric aware connectors • Supports Python/ php / Java Connectors • Fetch Shard info • Local cache • Avoids the proxy hop. Application XML-RPC Connector

21. MySQL FABRIC Server Groups • Stores actual data – Global Group – Shard Group • Primary – Read / Write • Secondary – Reads ( failover) • Spare ( On request ) Server Group Primary Node Secondary Node Spare Node

22. MySQL FABRIC Server Groups ( Global Group ) • Stores Global Tables and schema • Schema changes • Replicates to Shard groups Global Group Shard Group Global Writes Fabric Node

23. MySQL FABRIC ARCHITECTURE ( Again ) Application XML-RPC Connector XML-RPC FABRIC NODE Global Group Group 1 Group 2 Group 3 Group 4 Back Store

24. MySQL FABRIC Advantages: • Ease of Failover with GTID. • Shard Split is easy • RPC avoid network hops. Dis Advantages • Fabric node will be single point of failure • No Cross Shard . ( Shard query can help here) • Limited Connectors.

25. THANK YOU

Scaling MySQL using Fabric

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to Scaling MySQL using Fabric

Similar to Scaling MySQL using Fabric (20)

Recently uploaded

Recently uploaded (20)

Scaling MySQL using Fabric