The document provides an overview of NewSQL databases. It discusses how traditional SQL databases do not scale well for modern large datasets and high concurrency requirements. NewSQL databases aim to provide SQL interfaces with ACID transactions while also scaling horizontally. They use techniques like shared-nothing architectures, non-locking concurrency control, and in-memory storage. Example NewSQL databases mentioned include NuoDB, VoltDB, ClustrixDB, FoundationDB, and MemSQL. Each takes different approaches to balancing consistency, availability, and partitioning.
This document provides an overview of Oracle 9i Real Application Clusters (RAC) on Linux. It discusses the benefits of RAC such as scalability, high availability, and transparent expansion. Key components of RAC are described including cache fusion, global cache management, and resource coordination. Failure detection and recovery processes are also summarized. The document concludes with information on configuring Oracle 9i RAC and Linux kernel parameters on Linux systems.
This document discusses Red Hat's distributed cache and data grid platform, Infinispan. It provides an overview of Infinispan, describing it as a distributed in-memory key/value data grid and cache that is highly available, elastic, and manageable. It outlines Infinispan's architecture as both a standalone library and clustered server, its clustering and persistence capabilities, support for transactions and querying, and use cases such as session clustering and big data processing.
Skalierbarkeit mit MariaDB und MaxScale - MariaDB Roadshow Summer 2014 Hambur...MariaDB Corporation
Skalierbarkeit mit MariaDB und MaxScale
Presented by Ralf Gebhardt at the MariaDB Roadshow Germany: 4.7.2014 in Hamburg, 8.7.2014 in Berlin and 11.7.2014 in Frankfurt.
The document provides an overview of NewSQL databases. It discusses how traditional SQL databases do not scale well for modern large datasets and high concurrency requirements. NewSQL databases aim to provide SQL interfaces with ACID transactions while also scaling horizontally. They use techniques like shared-nothing architectures, non-locking concurrency control, and in-memory storage. Example NewSQL databases mentioned include NuoDB, VoltDB, ClustrixDB, FoundationDB, and MemSQL. Each takes different approaches to balancing consistency, availability, and partitioning.
This document provides an overview of Oracle 9i Real Application Clusters (RAC) on Linux. It discusses the benefits of RAC such as scalability, high availability, and transparent expansion. Key components of RAC are described including cache fusion, global cache management, and resource coordination. Failure detection and recovery processes are also summarized. The document concludes with information on configuring Oracle 9i RAC and Linux kernel parameters on Linux systems.
This document discusses Red Hat's distributed cache and data grid platform, Infinispan. It provides an overview of Infinispan, describing it as a distributed in-memory key/value data grid and cache that is highly available, elastic, and manageable. It outlines Infinispan's architecture as both a standalone library and clustered server, its clustering and persistence capabilities, support for transactions and querying, and use cases such as session clustering and big data processing.
Skalierbarkeit mit MariaDB und MaxScale - MariaDB Roadshow Summer 2014 Hambur...MariaDB Corporation
Skalierbarkeit mit MariaDB und MaxScale
Presented by Ralf Gebhardt at the MariaDB Roadshow Germany: 4.7.2014 in Hamburg, 8.7.2014 in Berlin and 11.7.2014 in Frankfurt.
The document describes different database system architectures including centralized systems, client-server systems, and server system architectures. Centralized systems run on a single computer while client-server systems have client systems that make requests to server systems. Server systems can be transaction servers, where clients send requests to servers to execute transactions, or data servers, where data is shipped to client systems to perform processing. The document discusses the components and processes involved in transaction server architectures.
Skype uses PostgreSQL for its databases and has over 100 database servers handling over 10,000 transactions per second across 200 databases storing billions of records. The databases are split both vertically and horizontally to scale with load. All database access is through stored procedures for security and flexibility. Replication and remote procedure calls connect the databases. Tools like pgBouncer, PL/Proxy, PgQ, and Londiste help partition, connect, and load balance the databases to form a large and complex but manageable distributed database architecture.
ProxySQL - High Performance and HA Proxy for MySQLRené Cannaò
High Availability proxy designed to solve real issues of MySQL setups from small to very large production environments.
Presentation at Percona Live Amsterdam 2015
This is the presentation delivered by Karthik.P.R at MySQL User Camp Bangalore on 09th June 2017. ProxySQL is a high performance MySQL Load Balancer Designed to scale database servers.
Presentation shows how ProxySQL can improve the HA in solution like MySQL async and sync replication without the need to increase the platform complexity.
The database world is undergoing a major upheaval. NoSQL databases such as MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offering significantly better scalability and performance. But these databases have a very different and unfamiliar data model and APIs as well as a limited transaction model. Moreover, the relational world is fighting back with so-called NewSQL databases such as VoltDB, which by using a radically different architecture offers high scalability and performance as well as the familiar relational model and ACID transactions. Sounds great but unlike the traditional relational database you can't use JDBC and must partition your data.
In this presentation you will learn about popular NoSQL databases - MongoDB, and Cassandra - as well at VoltDB. We will compare and contrast each database's data model and Java API using NoSQL and NewSQL versions of a use case from the book POJOs in Action. We will learn about the benefits and drawbacks of using NoSQL and NewSQL databases.
An Elastic Metadata Store for eBay’s Media PlatformMongoDB
In order to build a robust, multi-tenant, highly available storage services that meet the business’ SLA your databases has to be sharded. But if your service has to scale continuously through the incremental additions of storage without service interruption or human intervention, basic static sharding is not enough. At eBay, we are building MStore to solve this problem, with MongoDB as the storage engine. In this presentation, we will dive into the key design concepts of this solution.
This document introduces Infinispan, an open source distributed in-memory data grid. It provides high availability and elasticity through data replication across nodes. The API allows developers to treat Infinispan as a distributed concurrent hash map. Key features include transactions, persistence, querying, and map/reduce capabilities. Infinispan can be used as a local cache, cluster of caches, or autonomous data store accessed via protocols like REST and Memcached. Future plans include improvements to transactions, non-blocking state transfers, and support for cross-datacenter replication and eventual consistency.
MongoDB WiredTiger Internals: Journey To TransactionsMydbops
MongoDB has adapted transaction feature (ACID Properties) in MongoDB 4.0. This talk focuses on the internals of how MongoDB adapted the ACID properties with Weird Tiger Engine. Weird tiger offers more future possibilities for MongoDB. This tech talk was presented at Mydbops Database Meetup on 27-04-2019 by Manosh Malai Senior Devops/NoSQL Consultant with Mydbops and Ranjith Database Administrator with Mydbops.
gRPC is a modern high performance RPC (Remote Procedure Call) framework that can run in any environment. gRPC is based on HTTP/2, Protocol Buffers and other modern standard-based technologies. Together with excellent tooling, it helps you create high throughput, low latency, real-time services.
Policy based cluster management in oracle 12c Anju Garg
Oracle Grid Infrastructure 12c enhances the use of server pools by introducing server attributes e.g. memory, CPU_count etc. which can be associated with each server. Server pools can be configured so that their members belong to a category of servers, which share a particular set of attributes. Moreover, administrators can maintain a library of policies and switch between them as required rather than manually reallocating servers to various server pools based on workload. This paper discusses in detail the new features of policy based cluster management in 12c.
How to JavaOne 2016 - Generate Customized Java 8 Code from Your Database [TUT...Malin Weiss
The best code is the one you never need to write. Using code generation and automated builds, you can minimize the risk of human error when developing software, but how do you maintain control over code when large parts of it are handed over to a machine? In this tutorial, you will learn how to use open source software to create and control code automation. You will see how you can generate a completely object-oriented domain model by automatically analyzing your database schemas. Every aspect of the process is transparent and configurable, giving you, as a developer, 100 percent control of the generated code. This will not only increase your productivity but also help you build safer, more maintainable Java applications and is a perfect solution for Microservices.
This document outlines the agenda for a training on Oracle RDBMS 12c new features. The training will cover 6 chapters: introduction, multitenant architecture, upgrade features, Flex Cluster, Global Data Service, and an overview of RDBMS features. The agenda provides a high-level overview of topics to be discussed in each chapter, including multitenant architecture concepts, upgrade options and tools, Flex Cluster configurations, Global Data Service components, and new features such as temporary undo and multiple indexes on the same columns.
MariaDB MaxScale is a database proxy that provides scalability, high availability, and data streaming capabilities for MariaDB and MySQL databases. It acts as a load balancer and router to distribute queries across database servers. MaxScale supports services like read/write splitting, query caching, and security features like selective data masking. It can monitor replication lag and route queries accordingly. MaxScale uses a plugin architecture and its core remains stateless to provide flexibility and high performance.
- MySQL HA can be achieved with solutions like shared storage (DRBD), replication, MySQL Cluster, or Linux HA/Pacemaker.
- Linux HA/Pacemaker provides high availability by managing resources across nodes and ensuring that services are running on an available node if one fails.
- It uses a central configuration (CIB) to define resources, constraints between them, and monitor their status to determine the optimal placement of resources across nodes.
Cassandra Summit 2014: Huge Online Genealogical Database Driven By CassandraDataStax Academy
Presenter: Software Developer at Family Search
FamilySearch hosts a collaborative family tree with over a billion editable records. The tree currently serves as many as 10,000 concurrent users at peak weekly load. These users come from across the globe and collectively maintain and enhance the tree around the clock. Recent efforts to port the tree from a relational database to Cassandra have resulted in drastically improved performance and scalability. The database consists of more than 5 billion records in journaled form, and we anticipate having over 10TB of live data available for user view & edit, with that data size growing significantly as our user base grows. The dataset has resisted sharding in the past, so the port involved rethinking the core data model. The model we chose retains the consistency that our users demand, and is able to be implemented without requiring ACID transactions. Specifically, the consistency model we chose combined a Convergent and Commutative Replicated Data Type (CvRDT and CmRDT) with Cassandra's atomic batch implementation to form the basis for a consistency model that met the demanding needs of the family tree application.
Since its inception, the LibX browser plugin has been adopted by over 500 libraries to provide access to their services at the user's point of need. We are now developing LibX 2.0, a community platform that allows anybody to create, share, and deploy library services in a distributed and decentralized fashion. We'll describe the technology used in LibX 2.0, with a particular emphasis on the developer API and the deployment infrastructure facilitating this community engagement. -- Godmar Back, Virginia Tech
MongoDB San Francisco 2013: Basic Sharding in MongoDB presented by Brandon Bl...MongoDB
Sharding allows you to distribute load across multiple servers and keep your data balanced across those servers. This session will review MongoDB’s sharding support, including an architectural overview, design principles, and automation.
This document provides an overview of Couchbase, a NoSQL database. It begins with defining what NoSQL is and how it differs from SQL databases in being non-relational, schema-less, and scaling horizontally. It then discusses Couchbase specifically, describing its architecture as a distributed key-value store that partitions data into buckets and virtual buckets. The document outlines how Couchbase provides high availability, easy scaling, and flexible data modeling. It also covers Couchbase SDKs, queries, and replication capabilities.
Data Stream Processing for Beginners with Kafka and CDCAbhijit Kumar
This document provides an overview of data stream processing using Apache Kafka and change data capture (CDC). It defines CDC as identifying and capturing changes made to a database. CDC is useful for applications like data replication, microservice architectures, and caching. The document discusses how Kafka and Kafka Connect can be used to capture change data from databases logs and send them as a stream of events. It also introduces Debezium, an open-source CDC connector for databases like MySQL, PostgreSQL and MongoDB. Finally, it mentions that a live demo will be shown of capturing change events from a MySQL database using Debezium and viewing them in a Kafka topic.
This document provides an overview of Membase, including:
- Membase is a distributed database that allows applications and data to scale independently. It uses the Memcached protocol and architecture.
- Membase can be deployed in various ways, including using the built-in Memcached caching layer or standalone proxies. It also supports secure multitenant buckets.
- The document demonstrates Membase's use cases with examples from large companies and discusses its architecture, including clustering, data access protocols, and a future NodeCode capability.
The document describes different database system architectures including centralized systems, client-server systems, and server system architectures. Centralized systems run on a single computer while client-server systems have client systems that make requests to server systems. Server systems can be transaction servers, where clients send requests to servers to execute transactions, or data servers, where data is shipped to client systems to perform processing. The document discusses the components and processes involved in transaction server architectures.
Skype uses PostgreSQL for its databases and has over 100 database servers handling over 10,000 transactions per second across 200 databases storing billions of records. The databases are split both vertically and horizontally to scale with load. All database access is through stored procedures for security and flexibility. Replication and remote procedure calls connect the databases. Tools like pgBouncer, PL/Proxy, PgQ, and Londiste help partition, connect, and load balance the databases to form a large and complex but manageable distributed database architecture.
ProxySQL - High Performance and HA Proxy for MySQLRené Cannaò
High Availability proxy designed to solve real issues of MySQL setups from small to very large production environments.
Presentation at Percona Live Amsterdam 2015
This is the presentation delivered by Karthik.P.R at MySQL User Camp Bangalore on 09th June 2017. ProxySQL is a high performance MySQL Load Balancer Designed to scale database servers.
Presentation shows how ProxySQL can improve the HA in solution like MySQL async and sync replication without the need to increase the platform complexity.
The database world is undergoing a major upheaval. NoSQL databases such as MongoDB and Cassandra are emerging as a compelling choice for many applications. They can simplify the persistence of complex data models and offering significantly better scalability and performance. But these databases have a very different and unfamiliar data model and APIs as well as a limited transaction model. Moreover, the relational world is fighting back with so-called NewSQL databases such as VoltDB, which by using a radically different architecture offers high scalability and performance as well as the familiar relational model and ACID transactions. Sounds great but unlike the traditional relational database you can't use JDBC and must partition your data.
In this presentation you will learn about popular NoSQL databases - MongoDB, and Cassandra - as well at VoltDB. We will compare and contrast each database's data model and Java API using NoSQL and NewSQL versions of a use case from the book POJOs in Action. We will learn about the benefits and drawbacks of using NoSQL and NewSQL databases.
An Elastic Metadata Store for eBay’s Media PlatformMongoDB
In order to build a robust, multi-tenant, highly available storage services that meet the business’ SLA your databases has to be sharded. But if your service has to scale continuously through the incremental additions of storage without service interruption or human intervention, basic static sharding is not enough. At eBay, we are building MStore to solve this problem, with MongoDB as the storage engine. In this presentation, we will dive into the key design concepts of this solution.
This document introduces Infinispan, an open source distributed in-memory data grid. It provides high availability and elasticity through data replication across nodes. The API allows developers to treat Infinispan as a distributed concurrent hash map. Key features include transactions, persistence, querying, and map/reduce capabilities. Infinispan can be used as a local cache, cluster of caches, or autonomous data store accessed via protocols like REST and Memcached. Future plans include improvements to transactions, non-blocking state transfers, and support for cross-datacenter replication and eventual consistency.
MongoDB WiredTiger Internals: Journey To TransactionsMydbops
MongoDB has adapted transaction feature (ACID Properties) in MongoDB 4.0. This talk focuses on the internals of how MongoDB adapted the ACID properties with Weird Tiger Engine. Weird tiger offers more future possibilities for MongoDB. This tech talk was presented at Mydbops Database Meetup on 27-04-2019 by Manosh Malai Senior Devops/NoSQL Consultant with Mydbops and Ranjith Database Administrator with Mydbops.
gRPC is a modern high performance RPC (Remote Procedure Call) framework that can run in any environment. gRPC is based on HTTP/2, Protocol Buffers and other modern standard-based technologies. Together with excellent tooling, it helps you create high throughput, low latency, real-time services.
Policy based cluster management in oracle 12c Anju Garg
Oracle Grid Infrastructure 12c enhances the use of server pools by introducing server attributes e.g. memory, CPU_count etc. which can be associated with each server. Server pools can be configured so that their members belong to a category of servers, which share a particular set of attributes. Moreover, administrators can maintain a library of policies and switch between them as required rather than manually reallocating servers to various server pools based on workload. This paper discusses in detail the new features of policy based cluster management in 12c.
How to JavaOne 2016 - Generate Customized Java 8 Code from Your Database [TUT...Malin Weiss
The best code is the one you never need to write. Using code generation and automated builds, you can minimize the risk of human error when developing software, but how do you maintain control over code when large parts of it are handed over to a machine? In this tutorial, you will learn how to use open source software to create and control code automation. You will see how you can generate a completely object-oriented domain model by automatically analyzing your database schemas. Every aspect of the process is transparent and configurable, giving you, as a developer, 100 percent control of the generated code. This will not only increase your productivity but also help you build safer, more maintainable Java applications and is a perfect solution for Microservices.
This document outlines the agenda for a training on Oracle RDBMS 12c new features. The training will cover 6 chapters: introduction, multitenant architecture, upgrade features, Flex Cluster, Global Data Service, and an overview of RDBMS features. The agenda provides a high-level overview of topics to be discussed in each chapter, including multitenant architecture concepts, upgrade options and tools, Flex Cluster configurations, Global Data Service components, and new features such as temporary undo and multiple indexes on the same columns.
MariaDB MaxScale is a database proxy that provides scalability, high availability, and data streaming capabilities for MariaDB and MySQL databases. It acts as a load balancer and router to distribute queries across database servers. MaxScale supports services like read/write splitting, query caching, and security features like selective data masking. It can monitor replication lag and route queries accordingly. MaxScale uses a plugin architecture and its core remains stateless to provide flexibility and high performance.
- MySQL HA can be achieved with solutions like shared storage (DRBD), replication, MySQL Cluster, or Linux HA/Pacemaker.
- Linux HA/Pacemaker provides high availability by managing resources across nodes and ensuring that services are running on an available node if one fails.
- It uses a central configuration (CIB) to define resources, constraints between them, and monitor their status to determine the optimal placement of resources across nodes.
Cassandra Summit 2014: Huge Online Genealogical Database Driven By CassandraDataStax Academy
Presenter: Software Developer at Family Search
FamilySearch hosts a collaborative family tree with over a billion editable records. The tree currently serves as many as 10,000 concurrent users at peak weekly load. These users come from across the globe and collectively maintain and enhance the tree around the clock. Recent efforts to port the tree from a relational database to Cassandra have resulted in drastically improved performance and scalability. The database consists of more than 5 billion records in journaled form, and we anticipate having over 10TB of live data available for user view & edit, with that data size growing significantly as our user base grows. The dataset has resisted sharding in the past, so the port involved rethinking the core data model. The model we chose retains the consistency that our users demand, and is able to be implemented without requiring ACID transactions. Specifically, the consistency model we chose combined a Convergent and Commutative Replicated Data Type (CvRDT and CmRDT) with Cassandra's atomic batch implementation to form the basis for a consistency model that met the demanding needs of the family tree application.
Since its inception, the LibX browser plugin has been adopted by over 500 libraries to provide access to their services at the user's point of need. We are now developing LibX 2.0, a community platform that allows anybody to create, share, and deploy library services in a distributed and decentralized fashion. We'll describe the technology used in LibX 2.0, with a particular emphasis on the developer API and the deployment infrastructure facilitating this community engagement. -- Godmar Back, Virginia Tech
MongoDB San Francisco 2013: Basic Sharding in MongoDB presented by Brandon Bl...MongoDB
Sharding allows you to distribute load across multiple servers and keep your data balanced across those servers. This session will review MongoDB’s sharding support, including an architectural overview, design principles, and automation.
This document provides an overview of Couchbase, a NoSQL database. It begins with defining what NoSQL is and how it differs from SQL databases in being non-relational, schema-less, and scaling horizontally. It then discusses Couchbase specifically, describing its architecture as a distributed key-value store that partitions data into buckets and virtual buckets. The document outlines how Couchbase provides high availability, easy scaling, and flexible data modeling. It also covers Couchbase SDKs, queries, and replication capabilities.
Data Stream Processing for Beginners with Kafka and CDCAbhijit Kumar
This document provides an overview of data stream processing using Apache Kafka and change data capture (CDC). It defines CDC as identifying and capturing changes made to a database. CDC is useful for applications like data replication, microservice architectures, and caching. The document discusses how Kafka and Kafka Connect can be used to capture change data from databases logs and send them as a stream of events. It also introduces Debezium, an open-source CDC connector for databases like MySQL, PostgreSQL and MongoDB. Finally, it mentions that a live demo will be shown of capturing change events from a MySQL database using Debezium and viewing them in a Kafka topic.
This document provides an overview of Membase, including:
- Membase is a distributed database that allows applications and data to scale independently. It uses the Memcached protocol and architecture.
- Membase can be deployed in various ways, including using the built-in Memcached caching layer or standalone proxies. It also supports secure multitenant buckets.
- The document demonstrates Membase's use cases with examples from large companies and discusses its architecture, including clustering, data access protocols, and a future NodeCode capability.
Databus is a change data capture system developed by LinkedIn in 2005 to provide a scalable and highly available system for capturing data changes from databases and delivering them to applications in a consistent, low-latency manner. It uses triggers in the source database to capture changes and stores the changes in event buffers. The Databus relay encapsulates the change capture logic and streams change events to multiple consumers like search indexes, applications, and read replicas in a multi-tenant way. The system provides a consistent view of data across applications through capturing committed database transactions.
Autonomous Database is Oracle's fully managed cloud database service. It provides automated operations including provisioning, tuning, patching and backup. Autonomous Database can be deployed in serverless or dedicated modes. Serverless provides elastic scaling on shared infrastructure, while dedicated runs on a customer's dedicated Exadata cloud service. Both support Autonomous Data Warehouse and Autonomous Transaction Processing databases. Autonomous Database handles all database administration tasks, allowing users to focus on running SQL queries without managing infrastructure.
VoltDB is a high performance database for real-time analytics that can be deployed on SoftLayer cloud infrastructure. The document outlines the process to install and run VoltDB on SoftLayer, including unpacking the VoltDB distribution, installing Java, exporting the VoltDB binaries to the path, and running VoltDB using the run.sh script. It also discusses how VoltDB enables real-time analytics by ingesting and exporting data to Netezza for deeper historical analysis in a closed loop system.
This document provides an overview of SQLite, including:
- SQLite is an embedded SQL database that is not a client-server system and stores the entire database in a single disk file.
- It supports ACID transactions for reliability and data integrity.
- SQLite is used widely in applications like web browsers, Adobe software, Android, and more due to its small size and not requiring a separate database server.
- The Android SDK includes classes for managing SQLite databases like SQLiteDatabase for executing queries, updates and deletes.
The document provides an overview of 14 topics related to Oracle Autonomous Database. It begins with how to get started with the Autonomous Database free tier and Oracle Machine Learning. It then discusses cross region data guard, exporting data as JSON to object storage, wallet rotation, partitions with external tables in cloud, set patch level when cloning, performance monitoring, data safe audit retention time increase, change concurrency limits via console, SQL monitor report, ASH analytics in performance hub, workload metrics on performance hub, and customer managed keys.
Using a Fast Operational Database to Build Real-time Streaming AggregationsVoltDB
Simplicity, accuracy, speed; these are three things everyone wants from their data architecture. Join this webinar presented by Behzad Pirvali, Performance Architect at MaxCDN and Peter Vescuso, CMO at VoltDB to learn how MaxCDN used VoltDB, the world’s fastest operational DB with fast data pipeline, to reduce the number of managed environments by 2/3 times with 1/10th of CPU cycles required with alternative solutions. All while achieving 100% billing accuracy on 32 TB of daily web server data. The fill recording of this webinar is also available here: http://learn.voltdb.com/WRMaxCDN.html
Whats new in Autonomous Database in 2022Sandesh Rao
This session covers the new features and happenings in the autonomous database world and will help answer more questions DBAs and Developers will have on the Autonomous Database, from provisioning to backups, troubleshooting, tips and tricks, security and HA. This is a good introduction for on-prem DBAs who want to learn how this works quickly without spending too much time on it. Questions like what does the free tier cover, how do I do backup or if it's automated, how do I manage it, how to scale up and down, how to secure their environment, how to use mtls, how to use tools like SQLDeveloper and SQLModeler, performance tuning all in a quick 45-minute session which might take weeks to pick up reading documentation or spanning several presentations
This document discusses Java Database Connectivity (JDBC) which provides Java applications with an API for accessing databases. It describes the four types of JDBC drivers: Type 1 uses JDBC-ODBC bridge, Type 2 uses native database APIs, Type 3 communicates through a middle-tier server, and Type 4 communicates directly via sockets. The document also outlines the basic steps to use JDBC for database connectivity including loading a driver, establishing a connection, creating statements, executing SQL, and processing result sets.
Hadoop & no sql new generation database systemsramazan fırın
This document provides a summary of a presentation about Hadoop, NoSQL databases, and graph databases. It discusses how big data is driving new database technologies like Hadoop and NoSQL to process large datasets. Specific NoSQL databases mentioned include MongoDB, Cassandra, Redis, CouchDB, HBase, and Neo4j. Use cases for telecommunications companies are discussed, like using big data to prevent customer churn, target customized campaigns, and gain more customers. A demo of these technologies is also included on the agenda.
IBM Informix - The Ideal Database for Internet of Things
Exclusive luncheon at IBM World of Watson 2016. Informix is the best fit for IoT sensor data analytics at the edge and in the cloud.
This document discusses real-time web applications and technologies. It defines real-time apps as allowing bi-directional communication between clients and servers so that users receive information as soon as it is published. Examples include chat, social media, gaming and notifications. Key implementation methods discussed are HTTP polling, streaming and WebSockets. The document also surveys popular real-time libraries for publish/subscribe, data syncing and hybrid approaches.
MongoDB World 2018: Breaking the Mold - Redesigning Dell's E-Commerce PlatformMongoDB
This document discusses Dell's redesign of its e-commerce platform by moving from a relational database to MongoDB. It outlines the advantages of a resource-oriented architecture and document stores. It then provides details of Dell's DCQO application that was migrated from an XML-based relational database to MongoDB, including the architecture, schema design, data migration strategy, and operations management tools.
MongoDB has taken a clear lead in adoption among the new generation of databases, including the enormous variety of NoSQL offerings. A key reason for this lead has been a unique combination of agility and scalability. Agility provides business units with a quick start and flexibility to maintain development velocity, despite changing data and requirements. Scalability maintains that flexibility while providing fast, interactive performance as data volume and usage increase. We'll address the key organizational, operational, and engineering considerations to ensure that agility and scalability stay aligned at increasing scale, from small development instances to web-scale applications. We will also survey some key examples of highly-scaled customer applications of MongoDB.
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...confluent
RocksDB is the default state store for Kafka Streams. In this talk, we will discuss how to improve single node performance of the state store by tuning RocksDB and how to efficiently identify issues in the setup. We start with a short description of the RocksDB architecture. We discuss how Kafka Streams restores the state stores from Kafka by leveraging RocksDB features for bulk loading of data. We give examples of hand-tuning the RocksDB state stores based on Kafka Streams metrics and RocksDB’s metrics. At the end, we dive into a few RocksDB command line utilities that allow you to debug your setup and dump data from a state store. We illustrate the usage of the utilities with a few real-life use cases. The key takeaway from the session is the ability to understand the internal details of the default state store in Kafka Streams so that engineers can fine-tune their performance for different varieties of workloads and operate the state stores in a more robust manner.
Access Data from XPages with the Relational ControlsTeamstudio
Did you know that Domino and XPages allows for the easy access of relational data? These exciting capabilities in the Extension Library can greatly enhance the capability of your applications and allow access to information beyond Domino. Howard and Paul will discuss what you need to get started, what controls allow access to relational data, and the new @Functions available to incorporate relational data in your Server Side JavaScript programming.
This document discusses various techniques for scaling web applications, including horizontal scaling by adding more servers behind a load balancer, using a session store like Redis for shared sessions, centralized logging, and continuous integration to deploy updates. It also covers load balancing with HAProxy, monitoring with Zabbix, caching with Varnish, database scaling with master-slave replication or sharding in MongoDB, and using queues like RabbitMQ. The key is to think of the application as independent workers that can run on multiple servers rather than a single instance.
Similar to 140116 max scale-for_madrid_meetup (20)
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Trusted Execution Environment for Decentralized Process MiningLucaBarbaro3
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Dive into the realm of operating systems (OS) with Pravash Chandra Das, a seasoned Digital Forensic Analyst, as your guide. 🚀 This comprehensive presentation illuminates the core concepts, types, and evolution of OS, essential for understanding modern computing landscapes.
Beginning with the foundational definition, Das clarifies the pivotal role of OS as system software orchestrating hardware resources, software applications, and user interactions. Through succinct descriptions, he delineates the diverse types of OS, from single-user, single-task environments like early MS-DOS iterations, to multi-user, multi-tasking systems exemplified by modern Linux distributions.
Crucial components like the kernel and shell are dissected, highlighting their indispensable functions in resource management and user interface interaction. Das elucidates how the kernel acts as the central nervous system, orchestrating process scheduling, memory allocation, and device management. Meanwhile, the shell serves as the gateway for user commands, bridging the gap between human input and machine execution. 💻
The narrative then shifts to a captivating exploration of prominent desktop OSs, Windows, macOS, and Linux. Windows, with its globally ubiquitous presence and user-friendly interface, emerges as a cornerstone in personal computing history. macOS, lauded for its sleek design and seamless integration with Apple's ecosystem, stands as a beacon of stability and creativity. Linux, an open-source marvel, offers unparalleled flexibility and security, revolutionizing the computing landscape. 🖥️
Moving to the realm of mobile devices, Das unravels the dominance of Android and iOS. Android's open-source ethos fosters a vibrant ecosystem of customization and innovation, while iOS boasts a seamless user experience and robust security infrastructure. Meanwhile, discontinued platforms like Symbian and Palm OS evoke nostalgia for their pioneering roles in the smartphone revolution.
The journey concludes with a reflection on the ever-evolving landscape of OS, underscored by the emergence of real-time operating systems (RTOS) and the persistent quest for innovation and efficiency. As technology continues to shape our world, understanding the foundations and evolution of operating systems remains paramount. Join Pravash Chandra Das on this illuminating journey through the heart of computing. 🌟
4. MaxScale
-‐
Objectives
•Highly
Scalable
•Extendible
•Lightweight,
with
small
footprint
•Must
provide
authentication
•Minimum
possible
latency
•Highly
available
•Must
be
transparent
to
the
application
!
4
5. MaxScale
Core
•Event
driven
network
I/O
processor
•Polling,
event
driven
mechanism
that
is
responsible
for
dispatching
events
to
the
various
modules
that
make
up
the
MaxScale
application
•Events
in
MaxScale
=
Network
requests,
such
as:
•Handling
an
incoming
connection
on
a
listener
socket
•Incoming
data
for
a
client
connection
•Data
arriving
on
a
connection
from
a
backend
database
server
•A
socket
error
on
one
of
the
client
or
database
connections
•A
socket
closure
•The
availability
of
connections
to
receive
more
data
5
6. MaxScale
Polling
•Linux
epoll
mechanism
to
poll
for
activity
on
the
file
descriptors
•The
epoll
mechanism
allows
a
file
descriptor
to
be
registered
along
with
some
user
data
•MaxScale
makes
use
of
this
user
data
to
register
a
descriptor
control
block
for
each
descriptor
6
7. Descriptor
Control
Block
-‐
DCB
•The
main
source
of
state
information
that
is
used
within
the
event
processing
core
of
MaxScale.
•On
an
event
occurring
for
a
descriptor
the
corresponding
DCB
is
used
to
determine
how
that
event
should
be
handled.
•DCB
Members:
•State
-‐
used
to
determine
where
in
its
lifecycle
the
DCB
currently
is.
•Func
-‐
set
of
function
pointers
for
routines
that
are
called
when
particular
events
occur
or
when
MaxScale
wishes
to
perform
certain
operations.
It
allows
MaxScale
to
be
extended
to
support
new
protocols
without
the
need
to
alter
the
MaxScale
core
itself.
•Session
-‐
a
pointer
to
the
session
to
which
this
connection
belongs.
•Service
-‐
the
service
that
this
connection
belongs
to.
•Writeq
-‐
a
pointer
to
a
chain
of
GWBUF
structures
that
contain
data
waiting
to
be
written.
•Delayq
-‐
similar
to
writeq,
it
buffer
data
that
has
to
be
held
due
to
some
other
activity.
7
8. testroute | Router | V1.0.0
debugcli | Router | V1.0.0
readconnroute | Router | V1.0.0
(gdb)
Descriptor
Control
Block
-‐
DCB
Descriptor Control Blocks
The Descriptor Control Block (DCB) is an important concept within MaxScale since it is this
block that is passed to the polling system, when an event occurs it is that structure that is
available and from this structure it must be possible to navigate to all other structures that
contain state regarding the session and protocol in use.
!
gw_dcb_state2string (int state) {!
! switch(state) {!
! ! case DCB_STATE_ALLOC:!
! ! ! return "DCB Allocated";!
! ! case DCB_STATE_POLLING:!
! ! ! return "DCB in the polling loop";!
! ! case DCB_STATE_LISTENING:!
! ! ! return "DCB for listening socket";!
! ! case DCB_STATE_DISCONNECTED:!
! ! ! return "DCB socket closed";!
! ! case DCB_STATE_FREED:!
! ! ! return "DCB memory could be freed";!
! ! case DCB_STATE_ZOMBIE:!
! ! ! return "DCB Zombie";!
! ! default:!
! ! ! return "DCB (unknown)";!
! }!
}
Similar print routines exist for the DCB
(gdb) call printAllDCBs()
DCB: 0x60ead0
DCB state:
DCB for listening socket
Queued write data:
0
8
9. Descriptor
Control
Block
-‐
DCB
!
DCB: 0xfb1650!
! DCB state:
! Service:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
DCB: 0x1050bc0!
! DCB state:
! Service:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
DCB: 0xfae370!
! DCB state:
! Service:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
DCB: 0x107ff50!
! DCB state:
! Service:
! Connected to:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
!
DCB for listening socket!
RW Split Router!
0!
0!
Writes: 0!
1324!
DCB for listening socket!
HTTPD Router!
0!
0!
Writes: 0!
0!
DCB for listening socket!
Debug Interface!
0!
0!
Writes: 0!
1!
DCB: 0x10ed970!
! DCB state:
! Service:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
DCB: 0x10a0640!
! DCB state:
! Service:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
DCB: 0x10a6e40!
! DCB state:
! Service:
! Connected to:
! Statistics:!
! ! No. of Reads:
! ! No. of Writes:
! ! No. of Buffered
! ! No. of Accepts:
DCB in the polling loop!
RW Split Router!
4!
3!
Writes: 0!
0!
DCB in the polling loop!
RW Split Router!
3!
2!
Writes: 0!
0!
DCB in the polling loop!
Debug Interface!
127.0.0.1!
12!
205!
Writes: 0!
0!
DCB in the polling loop!
RW Split Router!
192.168.0.20!
3!
3!
Writes: 0!
0
9
12. Module
•A
separate
code
entity
that
may
be
loaded
dynamically
into
MaxScale
to
increase
the
available
functionality.
•Modules
are
implemented
as
run-‐
time
loadable
shared
objects
!
MaxScale> show modules!
!
Module Name
| Module Type | Version!
----------------+-------------+---------!
MySQLBackend
| Protocol
| V2.0.0!
telnetd
| Protocol
| V1.0.1!
HTTPD
| Protocol
| V1.0.1!
MySQLClient
| Protocol
| V1.0.0!
mysqlmon
| Monitor
| V1.0.0!
readwritesplit | Router
| V1.0.2!
testroute
| Router
| V1.0.0!
debugcli
| Router
| V1.0.1
12
13. Server
•An
individual
database
server
to
which
a
client
can
be
connected
via
MaxScale
!
MaxScale> show servers!
Server 0xfae860!
! Server:
192.168.0.21!
! Status:
Master, Running!
! Protocol:! !
MySQLBackend!
! Port:! ! !
3306!
! Number of connections: 1320!
! Current no. of connections:!1!
Server 0xfb15c0!
! Server:
192.168.0.22!
! Status:
Slave, Running!
! Protocol:
MySQLBackend!
! Port:
3306!
! Number of connections: 298!
! Current no. of connections:!0!
Server 0xfb1530!
! Server:
192.168.0.23!
! Status:
Slave, Running!
! Protocol:
MySQLBackend!
! Port:
3306!
! Number of connections: 365!
! Current no. of connections:!0!
Server 0xfb14a0!
! Server:
192.168.0.24!
! Status:
Slave, Running!
! Protocol:
MySQLBackend!
! Port:
3306!
! Number of connections: 365!
! Current no. of connections:!0!
Server 0xfb1410!
! Server:
192.168.0.25!
! Status:
Slave, Running!
! Protocol:
MySQLBackend!
! Port:
3306!
! Number of connections: 292!
! Current no. of connections:!1!
MaxScale>
13
14. Monitor
•A
module
that
can
be
executed
within
MaxScale
to
monitor
the
state
of
a
set
of
database.
•The
use
of
an
internal
monitor
is
optional,
monitoring
may
be
performed
externally
to
MaxScale.
!
MaxScale> show monitors!
Monitor: 0x104ddc0!
! Name:! ! MariaDB10 Monitor!
! Monitor running!
! Monitored servers:!192.168.0.21:3306, 192.168.0.22:3306, 192.168.0.23:3306, 192.168.0.24:3306, 192.168.0.25:3306
14
15. Router
•A
module
within
MaxScale
that
will
route
client
request
to
the
various
database
server
which
MaxScale
provides
a
service
interface
to
15
16. Connection
Routing
•A
method
of
handling
requests
in
which
MaxScale
will
accept
connections
from
a
client
and
route
data
on
that
connection
to
a
single
database
using
a
single
connection.
•Connection
based
routing
sets
the
connection
to
a
database
and
it
does
not
examine
requests
on
the
connection.
•Once
the
connection
with
a
database
is
established,
it
cannot
be
moved.
16
17. Statement
Routing
•A
method
of
handling
requests
in
which
each
request
within
a
connection
will
be
handled
individually.
•Requests
may
be
sent
to
one
or
more
servers
and
connections
may
be
dynamically
added
or
removed
from
the
session.
17
18. Protocol
•A
software
module
that
is
used
to
communicate
with
another
software
entity
within
the
system.
•MaxScale
supports
the
dynamic
loading
of
protocol
modules.
18
19. Listener
•The
network
endpoint
that
is
used
to
listen
for
connections
to
MaxScale
from
the
client
applications.
•A
listener
is
associated
to
a
single
service,
however
a
service
may
have
many
listeners.
19
20. Service
•MaxScale
offers
a
set
of
services
to
the
clients
•Each
service
is
a
protocol/port
pair
+
a
query
routing
module
+
backends
•The
protocol
that
should
be
used
•The
port
on
which
to
listen
for
incoming
client
requests
•The
query
routing
module
defines
the
choice
of
backed
server
used
to
execute
the
statement
received
from
the
client
•The
backends
are
the
servers
that
statements
may
be
sent
to
as
part
of
this
service
•A
service
may
have
more
than
one
protocol/port
associated
with
it
!
MaxScale> show services!
Service 0x104cd40!
! Service:! ! RW Split Router!
! Router:! ! ! readwritesplit (0x7ff2a1921d40)!
! Number of router sessions:
! 1320!
! Current no. of router sessions:
! 1!
! Number of queries forwarded:
!
34308!
! Number of queries forwarded to master:!
26691!
! Number of queries forwarded to slave: !
6189!
! Number of queries forwarded to all:
!
1428!
! Started:! ! Wed Jan 1 11:15:17 2014!
! Backend databases!
! ! 192.168.0.25:3306 Protocol: MySQLBackend!
! ! 192.168.0.24:3306 Protocol: MySQLBackend!
! ! 192.168.0.23:3306 Protocol: MySQLBackend!
! ! 192.168.0.22:3306 Protocol: MySQLBackend!
! ! 192.168.0.21:3306 Protocol: MySQLBackend!
! Users data:
! 0x104dd00!
! Total connections:! 1321!
! Currently connected:!2!
Service 0xfaf380!
! Service:! ! HTTPD Router!
! Router:! ! ! testroute (0x7ff2a1b277c0)!
! Started:! ! Wed Jan 1 11:15:17 2014!
! Backend databases!
! ! 192.168.0.25:3306 Protocol: MySQLBackend!
! ! 192.168.0.24:3306 Protocol: MySQLBackend!
! ! 192.168.0.23:3306 Protocol: MySQLBackend!
! ! 192.168.0.22:3306 Protocol: MySQLBackend!
! ! 192.168.0.21:3306 Protocol: MySQLBackend!
! Users data:
! 0xfaf740!
! Total connections:! 1!
! Currently connected:!1!
Service 0xfae8f0!
! Service:! ! Debug Interface!
! Router:! ! ! debugcli (0x7ff2a1d31500)!
! Started:! ! Wed Jan 1 11:15:17 2014!
! Backend databases!
! Users data:
! 0xfaf1b0!
! Total connections:! 2!
! Currently connected:!2
20
21. Connection
Failover
•When
a
connection
currently
being
used
between
MaxScale
and
the
database
server
fails
a
replacement
will
be
automatically
created
to
another
server
by
MaxScale
without
client
intervention
21
25. Set
the
Environment
Variables
and
Start
MaxScale
!
[root@Sky6 ~]# MAXSCALE_HOME=/usr/local/skysql/maxscale/MaxScale!
[root@Sky6 ~]# LD_LIBRARY_PATH=/usr/local/skysql/maxscale/lib!
[root@Sky6 ~]# /usr/local/skysql/maxscale/bin/maxscale!
!
!
SkySQL MaxScale! Tue Dec 31 14:59:16 2013!
------------------------------------------------------!
Info : MaxScale will be run in a daemon process.!
! See the log from the following log files :!
!
Error log
Message log
Trace log
Debug log
:!
:!
:!
:!
/usr/local/skysql/maxscale/MaxScale/log/skygw_err1.log!
/usr/local/skysql/maxscale/MaxScale/log/skygw_msg1.log!
/usr/local/skysql/maxscale/MaxScale/log/skygw_trace1.log!
/usr/local/skysql/maxscale/MaxScale/log/skygw_debug1.log!
!
Listening MySQL connections at 0.0.0.0:4004!
Listening http connections at 0.0.0.0:6444!
Listening telnet connections at 0.0.0.0:4444
…or
create
a
3
lines
script
25
26. Checking
MaxScale
!
[root@Sky6 ~]# telnet localhost 4444!
Trying ::1...!
telnet: connect to address ::1: Connection refused!
Trying 127.0.0.1...!
Connected to localhost.!
Escape character is '^]'.!
Welcome the SkySQL MaxScale Debug Interface (V1.0.1).!
Type help for a list of available commands.!
!
MaxScale login: admin!
Password:!
!
MaxScale>
UID/PWD:
admin
/
skysql
!
MaxScale> help!
Available commands:!
add user!
clear server!
remove user!
restart monitor!
restart service!
set server!
show dcbs!
show dcb!
show dbusers!
show epoll!
show modules!
show monitors!
show server!
show servers!
show services!
show session!
show sessions!
show users!
shutdown maxscale!
shutdown monitor!
shutdown service!
reload config!
reload dbusers!
enable log!
disable log
26
28. Configuration
Template
!
[root@Sky6 etc]# cat MaxScale_template.cnf!
#!
# Example MaxScale.cnf configuration file!
#!
# Number of server threads!
# Valid options are:!
# ! hreads=<number of threads>!
t
!
[maxscale]!
threads=1!
…!
!
[Monitor Module]!
…!
!
[R/W Split Module]!
…!
!
[Read Connection Router]!
…!
!
[HTTPD Router]!
…!
!
[Debug Interface]!
…
!
# Listener definitions for the services!
#!
# Valid options are:!
#!
# ! ervice=<name of service defined elsewhere>!
s
# ! rotocol=<name of protocol module with which to listen>!
p
# ! ort=<Listening port>!
p
!
[RW Split Listener]!
…!
!
[Read Connection Listener]!
…!
!
[Debug Listener]!
…!
!
[HTTPD Listener]!
…!
!
# Definition of the servers!
!
[server1]!
…!
!
[serverN]!
…
28
29. Database
Settings
•Create
a
valid
MaxScale
user
•Allow
access
from
the
MaxScale
node
!
MariaDB [test]> create user maxuser identified by 'maxpwd';!
Query OK, 0 rows affected (0.01 sec)!
!
MariaDB [test]> grant all on *.* to maxuser@‘192.168.0.26’;!
Query OK, 0 rows affected (0.00 sec)
29
30. Global
Settings
!
•Settings
used
to
configure
the
core
of
MaxScale
[maxscale]!
threads=1!
•Options:
•threads
-‐
number
of
user
threads
that
poll
for
network
traffic
• Start
with
the
smallest
number
and
try
to
increase
to
check
the
impact
on
high
workload
• This
parameter
does
not
include
the
number
of
“internal
threads”
30
31. Server
Settings
•Settings
used
to
configure
the
backend
•type
-‐
the
server
type
•Options:
•address
-‐
The
IP
address
used
by
MaxScale
to
connect
to
the
server
•port
-‐
The
TCP
port
used
by
MaxScale
to
connect
to
the
server
•Protocol
-‐
The
DB
protocol.
• MySQLBackend
is
the
native
MySQL
protocol
•MonitorUser
-‐
The
DB
user
used
by
MaxScale
to
connect
with
the
Monitoring
module
• When
MonitorUser
is
not
present,
MaxScale
uses
the
credentials
set
in
the
Monitor
section
•MonitorPW
-‐
The
DB
user
password
!
[max1]!
type=server!
address=192.168.0.21!
port=3006!
protocol=MySQLBackend!
!
[max2]!
type=server!
address=192.168.0.22!
port=3006!
protocol=MySQLBackend!
!
…!
!
[max5]!
type=server!
address=192.168.0.25!
port=3006!
protocol=MySQLBackend
31
32. Monitor
Settings
•Settings
used
to
configure
the
monitor
module
for
MySQL/MariaDB
•type
-‐
the
monitor
type
•Options:
•module
-‐
The
loadable
module,
mysqlmon
for
MySQL/
MariaDB
•servers
-‐
Comma
separated
list
of
the
servers
to
monitor
• Servers
must
be
of
the
same
type
(e.g.
MySQL/MariaDB
servers)
•user
-‐
DB
user
used
to
connect
to
the
server
• Used
when
the
credentials
are
not
available
in
the
server
section
•passwd
-‐
DB
user
password
!
[MariaDB10 Monitor]!
type=monitor!
module=mysqlmon!
servers=max1,!
max2,!
max3,!
max4,!
max5!
user=maxuser!
passwd=maxpwd
32
33. SLB
with
Read/Write
Split
Service
Settings
•Settings
used
to
configure
the
service
module
for
statement-‐based
load
balancing
(with
read/write
splitting)
•type
-‐
the
service
type
•Options:
•router
-‐
The
loadable
module,
readwritesplit
for
MySQL/
MariaDB
R/W
splitting
with
MySQL
Replication
•servers
-‐
Comma
separated
list
of
the
servers
to
monitor
•user
-‐
DB
user
used
to
connect
to
the
server
to
extract
the
list
of
the
database
users
to
allow
local
authentication
•passwd
-‐
DB
user
password
!
[RW Split Router]!
type=service!
router=readwritesplit!
servers=max1,!
max2,!
max3,!
max4,!
max5 !
user=maxuser!
passwd=maxpwd
33
34. CLB
Service
Settings
•Settings
used
to
configure
the
service
module
for
statement-‐
based
load
balancing
(with
read/write
splitting)
•type
-‐
the
service
type
•Options:
•router
-‐
The
loadable
module,
readconnroute
for
connection
load
balancing
with
MySQL
Replication
•router_options
-‐
options
passed
to
the
module.
• slave
means
that
the
load
balancing
is
applied
only
to
the
slave
servers
• master,slave
means
that
the
load
balancing
is
applied
master
and
slave
servers
•servers
-‐
Comma
separated
list
of
the
servers
to
monitor
•user
-‐
DB
user
used
to
connect
to
the
server
to
extract
the
list
of
the
database
users
to
allow
local
authentication
•passwd
-‐
DB
user
password
!
[Read Connection Router]!
type=service!
router=readconnroute!
router_options=slave!
servers=max1,!
max2,!
max3,!
max4,!
max5!
user=maxuser!
passwd=maxpwd
34
35. Listener
Settings
-‐
SLB
!
•Set
the
Protocol
and
Service
pair.
This
is
used
to
offer
the
SLB
with
Read/Write
Splitting
•type
-‐
the
listener
type
[RW Split Listener]!
type=listener!
service=RW Split Router!
protocol=MySQLClient!
port=4006!
•Options:
•service
-‐
The
service
associated
to
the
listener
(e.g.
RW
Split
Router)
•protocol
-‐
The
protocol
paired
to
the
service
(e.g.
MySQLClient)
•port
-‐
The
TCP
port
used
to
listen
to
the
client
requests
35
36. Listener
Settings
-‐
Debug
&
HTTP
!
•These
listeners
are
used
to
offer
a
debug
and
a
HTTP
interface
to
MaxScale
•type
-‐
the
listener
type
•Options:
•service
-‐
DBAs
can
connect
to
MaxScale
using
telnet
or
via
a
restful
API
•protocol
-‐
The
protocol
paired
to
the
service
(e.g.
telnetd
and
httpd
)
•port
-‐
The
TCP
port
used
to
listen
to
the
client
requests
[Debug Listener]!
type=listener!
service=Debug Interface!
protocol=telnetd!
port=4444!
!
[HTTPD Listener]!
type=listener!
service=HTTPD Router!
protocol=HTTPD!
port=6444
36
37. Final
Configuration
!
[root@Sky6 etc]# cat MaxScale.cnf!
#!
# Number of server threads!
# Valid options are:!
# !
threads=<number of threads>!
[maxscale]!
threads=1!
!
#!
# Define a monitor that can be used to determine the state!
# and role of the servers.!
#!
# Valid options are:!
#!
# !
module=<name of module to load>!
# !
servers=<server name>,<server name>,...!
# !
user =<user name - must have slave replication and!
#
slave client privileges>!
# !
passwd=<password of the above user,
#
plain text currently>!
[MariaDB10 Monitor]!
type=monitor!
module=mysqlmon!
servers=max1,max2,max3,max4,max5!
user=maxuser!
passwd=maxpwd
!
#!
# A series of service definition!
#!
# Valid options are:!
#!
# !
router=<name of router module>!
# !
servers=<server name>,<server name>,...!
# !
user=<User to fetch password inforamtion with>!
# !
passwd=<Password of the user, plain text currently>!
#!
# Valid router modules currently are:!
# !
readwritesplit, readconnroute and debugcli!
[RW Split Router]!
type=service!
router=readwritesplit!
servers=max1,max2,max3,max4,max5!
user=maxuser!
passwd=maxpwd!
!
[HTTPD Router]!
type=service!
router=testroute!
servers=max1,max2,max3,max4,max5!
!
[Debug Interface]!
type=service!
router=debugcli
37
38. Final
Configuration
!
#!
# Listener definitions for the services!
#!
# Valid options are:!
#!
# !
service=<name of service defined elsewhere>!
# !
protocol=<name of protocol module with which to listen>!
# !
port=<Listening port>!
[RW Split Listener]!
type=listener!
service=RW Split Router!
protocol=MySQLClient!
port=4004!
!
[Debug Listener]!
type=listener!
service=Debug Interface!
protocol=telnetd!
port=4444!
!
[HTTPD Listener]!
type=listener!
service=HTTPD Router!
protocol=HTTPD!
port=6444!
!
# Servers Definition!
[max1]!
type=server!
address=192.168.0.21!
port=3306!
protocol=MySQLBackend!
!
[max2]!
type=server!
address=192.168.0.22!
port=3306!
protocol=MySQLBackend!
!
[max3]!
type=server!
address=192.168.0.23!
port=3306!
protocol=MySQLBackend!
!
[max4]!
type=server!
address=192.168.0.24!
port=3306!
protocol=MySQLBackend!
!
[max5]!
type=server!
address=192.168.0.25!
port=3306!
protocol=MySQLBackend
38
39. Password
Encryption
•Used
to
remove
plain
password
from
the
config
file
•Passwords
are
stored
in
a
readonly
file
(default
$MAXSCALE_HOME/etc/.secrets)
•Plain
passwords
in
the
config
file
can
be
replaced
with
the
encrypted
password
!
[root@Sky6 ~]# /usr/local/skysql/maxscale/bin/
maxkeys /usr/local/skysql/maxscale/MaxScale/
etc/.secrets!
!
[root@Sky6 ~]# MAXSCALE_HOME=/usr/local/skysql/
maxscale/MaxScale!
[root@Sky6 ~]# /usr/local/skysql/maxscale/bin/
maxpasswd maxpwd!
14AE17C29AE7E6DB94EA5E6068D9833D!
!
[root@Sky6 ~]# vi /usr/local/skysql/maxscale/
MaxScale/etc/MaxScale.cnf!
!
…!
[MariaDB10 Monitor]!
type=monitor!
module=mysqlmon!
servers=max1,max2,max3,max4,max5!
user=maxuser!
passwd=14AE17C29AE7E6DB94EA5E6068D9833D!
…
39
40. Checking
MaxScale
SLB
R/W
Splimer
(for
the
users)
RESTful
API
(under
development)
Telnet
Interface
(for
debugging
and
DBAs)
!
MaxScale> show services!
Service 0x8e1d40!
! Service:! ! RW Split Router!
! Router:! ! ! readwritesplit (0x7ff7dd4d0d40)!
! Number of router sessions:
! 0!
! Current no. of router sessions:
! 0!
! Number of queries forwarded:
! 0!
! Number of queries forwarded to master:! 0!
! Number of queries forwarded to slave: ! 0!
! Number of queries forwarded to all:
! 0!
! Started:! ! Tue Dec 31 17:36:13 2013!
! Backend databases!
! ! 192.168.0.25:3006 Protocol: MySQLBackend!
! ! 192.168.0.24:3006 Protocol: MySQLBackend!
! ! 192.168.0.23:3006 Protocol: MySQLBackend!
! ! 192.168.0.22:3006 Protocol: MySQLBackend!
! ! 192.168.0.21:3006 Protocol: MySQLBackend!
! Users data:
!0x8e2d00!
! Total connections:! 1!
! Currently connected:! 1!
Service 0x844380!
! Service:! ! HTTPD Router!
! Router:! ! ! testroute (0x7ff7dd6d67c0)!
! Started:! ! Tue Dec 31 17:36:13 2013!
! Backend databases!
! ! 192.168.0.25:3006 Protocol: MySQLBackend!
! ! 192.168.0.24:3006 Protocol: MySQLBackend!
! ! 192.168.0.23:3006 Protocol: MySQLBackend!
! ! 192.168.0.22:3006 Protocol: MySQLBackend!
! ! 192.168.0.21:3006 Protocol: MySQLBackend!
! Users data:
!0x844740!
! Total connections:! 1!
! Currently connected:! 1!
Service 0x8438f0!
! Service:! ! Debug Interface!
! Router:! ! ! debugcli (0x7ff7dd8e0500)!
! Started:! ! Tue Dec 31 17:36:13 2013!
! Backend databases!
! Users data:
!0x8441b0!
! Total connections:! 2!
! Currently connected:! 2
40
41. Troubleshooting
!
2013 12/31 17:47:24 Error : access for secrets file [/usr/local/skysql/maxscale/MaxScale/etc/.secrets]
failed. Error 2, No such file or directory.
MaxScale
cannot
fine
the
encrypted
password
file
!
[root@Sky6 bin]# /usr/local/skysql/maxscale/MaxScale/bin/maxkeys /usr/local/skysql/maxscale/MaxScale/
etc/.secrets!
41
42. Troubleshooting
!
[root@Sky0 ~]# mysql -uroot -h 192.168.0.26 -P4004 -p!
Enter password:!
ERROR 1045 (2800): Authorization failed!
[root@Sky0 ~]#
root
is
not
loaded
to
the
available
users
to
connect
via
MaxScale
!
<<<< User [root] was not found!
<<< CLIENT AUTH FAILED for user [root]!
!
[root@Sky0 ~]# mysql -umaxuser -h 192.168.0.26 -P4004 -pmaxpwd!
Welcome to the MariaDB monitor. Commands end with ; or g.!
Your MySQL connection id is 1535!
Server version: 5.5.22-SKYSQL-0.1.0 MariaDB Server!
!
Copyright (c) 2000, 2013, Oracle, Monty Program Ab and others.!
!
Careful
with
this!
Type 'help;' or 'h' for help. Type 'c' to clear the current
input statement.!
!
MySQL [(none)]>
42
43. Troubleshooting
From
MaxScale
You
see
the
Max1
process
list
!
MySQL [(none)]> show processlist;!
+-----+---------+-------------------+------+-------------+-------+----------------------------------------+------------------+----------+!
| Id | User
| Host
| db
| Command
| Time | State
| Info
| Progress |!
+-----+---------+-------------------+------+-------------+-------+----------------------------------------+------------------+----------+!
|
5 | root
| skycluster3:33830 | NULL | Binlog Dump | 82453 | Master has sent all binlog to slave; … | NULL
|
0.000 |!
|
6 | root
| skycluster4:43403 | NULL | Binlog Dump | 82389 | Master has sent all binlog to slave; … | NULL
|
0.000 |!
|
7 | root
| skycluster5:46878 | NULL | Binlog Dump | 82382 | Master has sent all binlog to slave; … | NULL
|
0.000 |!
|
8 | root
| skycluster2:54912 | NULL | Binlog Dump | 82376 | Master has sent all binlog to slave; … | NULL
|
0.000 |!
| 100 | root
| skycluster0:41977 | NULL | Sleep
|
0 |
| NULL
|
0.000 |!
| 101 | root
| skycluster0:41997 | NULL | Sleep
|
60 |
| NULL
|
0.000 |!
| 129 | maxuser | skycluster6:55557 | NULL | Sleep
|
5 |
| NULL
|
0.000 |!
| 132 | maxuser | skycluster6:55565 | NULL | Query
|
0 | init
| show processlist |
0.000 |!
| 135 | maxuser | skycluster6:55571 | NULL | Sleep
|
41 |
| NULL
|
0.000 |!
+-----+---------+-------------------+------+-------------+-------+----------------------------------------+------------------+----------+!
9 rows in set (0.00 sec)
This
is
directly
from
Max2
!
MariaDB [(none)]> show processlist;!
+----+-------------+-------------------+------+---------+-------+----------------------------------+------------------+----------+!
| Id | User
| Host
| db
| Command | Time | State
| Info
| Progress |!
+----+-------------+-------------------+------+---------+-------+----------------------------------+------------------+----------+!
| 4 | system user |
| NULL | Connect | 82804 | Waiting for master to send event | NULL
|
0.000 |!
| 5 | system user |
| NULL | Connect | 73387 | Slave has read all relay log; wa…| NULL
|
0.000 |!
| 14 | root
| skycluster0:35933 | NULL | Sleep
|
0 |
| NULL
|
0.000 |!
| 15 | root
| skycluster0:35936 | NULL | Sleep
|
216 |
| NULL
|
0.000 |!
| 44 | maxuser
| skycluster6:46518 | NULL | Sleep
|
3 |
| NULL
|
0.000 |!
| 46 | root
| localhost
| NULL | Query
|
0 | init
| show processlist |
0.000 |!
+----+-------------+-------------------+------+---------+-------+----------------------------------+------------------+----------+!
6 rows in set (0.00 sec)
43
45. MaxScale
with
MySQL
Replication
using
Connection
Load
Balancing
For
applicaLons
that
are
“ReplicaLon
aware”,
i.e.
they
can
use
2
connecLons,
one
for
the
master
and
one
for
the
slave
servers
MaxScale
Each
applicaeon
server
uses
2
conneceon:
1
Read/Write,
1
Read/Only
Conneceon
load
balancing
is
fast,
lightweight
and
it
provides
high
availability
Database
Database
Database
Database
Database
MaxScale
connects
the
R/W
client
conneceon
to
the
master
and
it
load
balances
the
R/O
client
conneceon
to
one
of
the
slaves
MHA
provides
the
automaec
failover
in
case
of
fault
of
the
master
45
46. MaxScale
with
MySQL
Replication
using
Connection
Load
Balancing
[Write Listener]
type=listener
service=Write Service
protocol=MySQLClient
port=4007
[Read Listener]
type=listener
service=Read Service
protocol=MySQLClient
port=4006
MaxScale
[Write Service]
type=service
router=readconnroute
router_options=master
servers=s1,s2,s3,s4,s5
user=maxuser
auth=maxpwd
[Read Service]
type=service
router=readconnroute
router_options=slave
servers=s1,s2,s3,s4,s5
user=maxuser
auth=maxpwd
Database
Database
Database
Database
Database
46
47. MaxScale
with
Galera
using
Connection
Load
Balancing
Each
applicaeon
server
uses
only
1
conneceon
For
applicaLons
that
can
use
the“all-‐
master”
capabiliLes
of
Galera
MaxScale
MaxScale
monitors
the
state
of
each
Galera
Cluster
and
it
applies
load
balancing
only
on
synced
nodes
MaxScale
load
balances
the
client
conneceons
to
the
Galera
nodes
Database
Database
Database
47
48. MaxScale
with
Galera
using
Connection
Load
Balancing
[Read Listener]
type=listener
service=Read Service
protocol=MySQLClient
port=4006
MaxScale
[Galera Service]
type=service
router=readconnroute
router_options=synced
servers=s1,s2,s3,s4,s5
user=maxuser
auth=maxpwd
Database
Database
Database
48
49. MaxScale
with
MySQL
Replication
using
Read/Write
Splitting
Each
applicaeon
server
uses
only
1
conneceon
For
applicaLons
that
have
been
designed
to
work
with
a
single
server
and
require
read
scalability
MaxScale
MaxScale
monitors
the
state
of
each
node
and
it
applies
load
balancing
only
on
the
available
slaves
Database
Database
Database
Database
Database
MaxScale
creates
2
conneceons,
one
for
R/W
on
the
master
node
and
one
R/O
load
balanced
on
the
slave
nodes
49
50. How
does
MaxScale
compare
to…
•HA
Proxy
•MySQL
Proxy
•Sharding
solutions
-‐
ScaleBase,
DBShards,
Parelastic,
etc.
50
51. JSON
over
http
and
BSON
Strictly
Confidential
-‐
SkySQL
Internal
Use
Only
51
53. Where
To
Find
More
Material
GitHub
Project
hUps://github.com/skysql/MaxScale
Tarball
hUps://downloads.skysql.com/files/SkySQL/MaxScale
Documentaeon
hUps://github.com/skysql/MaxScale/tree/master/DocumentaLon
53