Introducing TiDB - Percona Live Frankfurt
•
0 likes•355 views
TiDB is an open-source distributed SQL database developed by PingCAP that is compatible with MySQL. It provides horizontal scalability, high availability, and consistent distributed transactions. Mobike, which has 200 million users and 9 million bikes, uses TiDB to handle over 30 TB of data per day. While TiDB aims to be compatible with MySQL, some features like stored procedures work differently or are still in development.
Report
Share
Report
Share
Download to read offline
Recommended
TiDB Introduction - Boston MySQL Meetup Group
This document provides an overview and summary of TiDB, an open-source distributed SQL database inspired by Google's Spanner and F1. The summary includes:
1. TiDB is a distributed SQL database that is compatible with MySQL and provides horizontal scalability, high availability, and strong consistency with a hybrid OLTP/OLAP architecture.
2. It consists of TiDB, TiKV, and PD components where TiDB is the frontend MySQL compatible database layer, TiKV is the distributed key-value storage layer, and PD is the placement driver for metadata management.
3. TiDB is being used by over 300 companies including Mobike for applications such as real-time analytics, high concurrency
TiDB Introduction - San Francisco MySQL Meetup
This document provides an overview and agenda for introducing TiDB, an open source distributed SQL database inspired by Google's Spanner and F1 projects. The summary includes:
- TiDB is a distributed SQL database that is compatible with MySQL and provides horizontal scalability, high availability, and strong consistency with its key components TiDB, TiKV, and PD.
- The agenda covers an introduction to PingCAP, the company behind TiDB, a technical walkthrough of the TiDB architecture, and a use case example with Mobike, one of TiDB's customers with over 200 million users.
- A live demo of running TiDB on Google Kubernetes Engine is also included on the agenda along with discussions of
TiDB Introduction
This document provides an overview and summary of TiDB, an open-source distributed SQL database compatible with MySQL. It discusses TiDB's architecture which includes TiDB for the SQL layer, TiKV for storage, and PD for placement driving. TiDB provides features like horizontal scalability, distributed transactions, and high availability. Example use cases are also presented, like Mobike's use of TiDB for locking/unlocking bikes and real-time analytics of bike usage data across 200 cities in China.
FOSDEM MySQL and Friends Devroom
TiDB is a distributed, horizontally scalable SQL database that is compatible with MySQL. It separates processing and storage into independent scalable components - the TiDB SQL layer and the TiKV storage foundation. TiDB uses a multi-version concurrency control approach based on Google's Spanner/F1 databases. It has been used in large-scale production deployments containing over 30 TB of data per day. Benchmarks show it can scale linearly with additional nodes. While aiming to be compatible with MySQL features, it does not support some like stored procedures and triggers.
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
This presentation was delivered at the NYC SQL meetup on September 27, 2018. It provided a technical overview of the TiDB Platform, a deep dive into TiDB's MySQL compatible layer and MySQL ecosystem tools, use case of Mobike, and appendix with detail materials on coprocessor and transaction model.
The Dark Side Of Go -- Go runtime related problems in TiDB in production
Ed Huang, CTO of PingCAP, talked at Go System Conference about dealing with the typical and profound issues related to Go’s runtime as your systems become more complex. Taking TiDB as an example, he demonstrated how these problems can be reproduced, located, and analyzed in production.
Presto talk @ Global AI conference 2018 Boston
Presented at Global AI Conference in Boston 2018:
http://www.globalbigdataconference.com/boston/global-artificial-intelligence-conference-106/speaker-details/kamil-bajda-pawlikowski-62952.html
Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale in a variety of use cases at Facebook, Airbnb, Netflix, Uber, Twitter, LinkedIn, Bloomberg, and FINRA, Presto experienced an unprecedented growth in popularity in both on-premises and cloud deployments in the last few years. Presto is really a SQL-on-Anything engine in a single query can access data from Hadoop, S3-compatible object stores, RDBMS, NoSQL and custom data stores. This talk will cover some of the best use cases for Presto, recent advancements in the project such as Cost-Based Optimizer and Geospatial functions as well as discuss the roadmap going forward.
My First 90 days with Vitess
This document summarizes the author's first 90 days of experience with Vitess, an open source database proxy. It provides an overview of Vitess, including that it sits between applications and MySQL to provide routing, query consolidation, and other features. It also discusses Vitess terminology, questions about MySQL compatibility, consistency models, and other quirks and features. The document concludes with a discussion of the best use cases for Vitess and areas where it could be improved.
Recommended
TiDB Introduction - Boston MySQL Meetup Group
This document provides an overview and summary of TiDB, an open-source distributed SQL database inspired by Google's Spanner and F1. The summary includes:
1. TiDB is a distributed SQL database that is compatible with MySQL and provides horizontal scalability, high availability, and strong consistency with a hybrid OLTP/OLAP architecture.
2. It consists of TiDB, TiKV, and PD components where TiDB is the frontend MySQL compatible database layer, TiKV is the distributed key-value storage layer, and PD is the placement driver for metadata management.
3. TiDB is being used by over 300 companies including Mobike for applications such as real-time analytics, high concurrency
TiDB Introduction - San Francisco MySQL Meetup
This document provides an overview and agenda for introducing TiDB, an open source distributed SQL database inspired by Google's Spanner and F1 projects. The summary includes:
- TiDB is a distributed SQL database that is compatible with MySQL and provides horizontal scalability, high availability, and strong consistency with its key components TiDB, TiKV, and PD.
- The agenda covers an introduction to PingCAP, the company behind TiDB, a technical walkthrough of the TiDB architecture, and a use case example with Mobike, one of TiDB's customers with over 200 million users.
- A live demo of running TiDB on Google Kubernetes Engine is also included on the agenda along with discussions of
TiDB Introduction
This document provides an overview and summary of TiDB, an open-source distributed SQL database compatible with MySQL. It discusses TiDB's architecture which includes TiDB for the SQL layer, TiKV for storage, and PD for placement driving. TiDB provides features like horizontal scalability, distributed transactions, and high availability. Example use cases are also presented, like Mobike's use of TiDB for locking/unlocking bikes and real-time analytics of bike usage data across 200 cities in China.
FOSDEM MySQL and Friends Devroom
TiDB is a distributed, horizontally scalable SQL database that is compatible with MySQL. It separates processing and storage into independent scalable components - the TiDB SQL layer and the TiKV storage foundation. TiDB uses a multi-version concurrency control approach based on Google's Spanner/F1 databases. It has been used in large-scale production deployments containing over 30 TB of data per day. Benchmarks show it can scale linearly with additional nodes. While aiming to be compatible with MySQL features, it does not support some like stored procedures and triggers.
Introducing TiDB [Delivered: 09/27/18 at NYC SQL Meetup]
This presentation was delivered at the NYC SQL meetup on September 27, 2018. It provided a technical overview of the TiDB Platform, a deep dive into TiDB's MySQL compatible layer and MySQL ecosystem tools, use case of Mobike, and appendix with detail materials on coprocessor and transaction model.
The Dark Side Of Go -- Go runtime related problems in TiDB in production
Ed Huang, CTO of PingCAP, talked at Go System Conference about dealing with the typical and profound issues related to Go’s runtime as your systems become more complex. Taking TiDB as an example, he demonstrated how these problems can be reproduced, located, and analyzed in production.
Presto talk @ Global AI conference 2018 Boston
Presented at Global AI Conference in Boston 2018:
http://www.globalbigdataconference.com/boston/global-artificial-intelligence-conference-106/speaker-details/kamil-bajda-pawlikowski-62952.html
Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale in a variety of use cases at Facebook, Airbnb, Netflix, Uber, Twitter, LinkedIn, Bloomberg, and FINRA, Presto experienced an unprecedented growth in popularity in both on-premises and cloud deployments in the last few years. Presto is really a SQL-on-Anything engine in a single query can access data from Hadoop, S3-compatible object stores, RDBMS, NoSQL and custom data stores. This talk will cover some of the best use cases for Presto, recent advancements in the project such as Cost-Based Optimizer and Geospatial functions as well as discuss the roadmap going forward.
My First 90 days with Vitess
This document summarizes the author's first 90 days of experience with Vitess, an open source database proxy. It provides an overview of Vitess, including that it sits between applications and MySQL to provide routing, query consolidation, and other features. It also discusses Vitess terminology, questions about MySQL compatibility, consistency models, and other quirks and features. The document concludes with a discussion of the best use cases for Vitess and areas where it could be improved.
Presentation at SF Kubernetes Meetup (10/30/18), Introducing TiDB/TiKV
This deck was presented at the SF Kubernetes Meetup held at Microsoft's downtown SF office, introducing the architecture of TiDB and TiKV (a CNCF project), key use cases, a user story with Mobike (one of the largest bikesharing platforms in the world), and how TiDB is deployed across different cloud environment using TiDB Operator.
Presto Summit 2018 - 07 - Lyft
Presto and you are there (Nishant Rayan, Lyft)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Presto Summit 2018 - 09 - Netflix Iceberg
Iceberg: a modern table format for big data (Ryan Blue & Parth Brahmbhatt, Netflix)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Introducing Scylla Open Source 4.0
Since its inception, Scylla has offered a compelling alternative to Apache Cassandra, providing better performance for a lower cost of ownership.
With Scylla Open Source 4.0 we continue to extend our CQL interface features and capabilities and also now provide an open source alternative to DynamoDB, allowing you to run your workloads anywhere, on any cloud provider, or on premises.
Join ScyllaDB co-founders, CTO Avi Kivity and CEO Dor Laor, for a look at the new features in Scylla Open Source 4.0, and architectural and cost comparisons with the coming Cassandra 4.0.
Topics will include:
Improved consistency with our new Lightweight Transactions
Scylla Operator for Kubernetes
How we stack up against Apache Cassandra 4.0
Our “run anywhere” DynamoDB alternative
Reactive database access with Slick3
The document discusses Reactive Slick, a new version of the Slick database access library for Scala that provides reactive capabilities. It allows parallel database execution and streaming of large query results using Reactive Streams. Reactive Slick is suitable for composite database tasks, combining async tasks, and processing large datasets through reactive streams.
Presto Summit 2018 - 02 - LinkedIn
LinkedIn’s Presto adventure (Mark Wagner, LinkedIn)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Mongo DB Monitoring - Become a MongoDB DBA
This presentation was presented by Art van Scheppingen at Percona Live 2017 in Santa Clara CA and covers what you need to know to effectively monitor MongoDB
Introducing TiDB [Delivered: 09/25/18 at Portland Cloud Native Meetup]
This deck introduces TiDB, an open source distributed NewSQL database, to the Portland Cloud Native meetup on September 25, 2018. It includes materials on technical architecture, core features, using TiDB Operator to deploy in any cloud environment, and appendix on transaction model and join support.
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
In this talk AWS’ Ken Krupa, Head of Specialized Solutions Architecture, will describe the architecture and capabilities of two new AWS EC2 instance types perfect for data-intensive storage and IO-heavy workloads like ScyllaDB: the Intel-based I4i and the Graviton2-based I4g series.
The Intel Xeon Ice Lake-based I4i series provides unparalleled raw horsepower for your most demanding workloads. Meanwhile, the Graviton2-powered I4g instances provide lower cost per storage on a power-efficient platform to deploy your cloud-native applications.
Ken will also describe the AWS Nitro SSD, a new form of high-speed NVMe storage with a Flash Translation Layer built with Nitro controllers, which powers both of these instance families.
ScyllaDB VP of Product Tzach Livyatan will then share benchmarking results showing how ScyllaDB behaves under load on these two instance types, providing maximum system utility and efficiency.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Running a DynamoDB-compatible Database on Managed Kubernetes Services
With the release of Alternator, Scylla’s DynamoDB-compatible API, you can now take your locked-in DynamoDB workloads and run them anywhere. Scylla provides a cost-effective open source alternative to Amazon’s DynamoDB, deployable wherever a user would want: on-premises, on other public clouds like Microsoft Azure or Google Cloud Platform, still on AWS (such as the high-density i3en instances) or as a fully managed DBaaS.
In this session, we will cover:
- Scylla Alternator: Scylla’s Amazon DynamoDB-compatible API
- Scylla Operator: Running Scylla Alternator on Kubernetes
- Demo Alternator - Demo and explain DynamoDB on GKE
Introducing TiDB Operator
This document discusses PingCAP's Kubernetes operator for TiDB, an open source distributed SQL database. It provides a brief history of PingCAP and the TiDB community. It then gives a technical overview of TiDB's architecture before explaining how the TiDB operator works. The operator allows users to deploy and manage TiDB clusters on Kubernetes through custom resources that are controlled by custom controllers. This provides capabilities like automated scaling, updates, and failover for stateful applications running on Kubernetes. The operator is open source and TiDB is also available as a managed service on GCP Marketplace.
Webinar 2017. Supercharge your analytics with ClickHouse. Alexander Zaitsev
Alexander Zaitsev presented on LifeStreet's experience implementing ClickHouse for their ad analytics platform. Some key points:
- LifeStreet processes over 10 billion events per day from their ad exchange and needed a high performance analytics solution.
- They tried various databases but migrated fully to ClickHouse due to its performance for analytics workloads.
- Major challenges included designing an efficient schema, sharding and replication strategy, and reliable data ingestion.
- ClickHouse's dictionary feature allowed them to implement normalized dimensions tables while supporting updates, improving storage efficiency and query performance.
Lookout on Scaling Security to 100 Million Devices
The massive increase of security-related data requires companies to respond with new approaches to ingestion. Learn how Lookout has changed its approach for ingesting telemetry to meet their goal of growing from 1.5 million devices to 100 million devices and beyond, using Kafka Connect and switching from AWS DynamoDB to Scylla.
TiDB for Big Data
Shen Li, VP engineering at PingCAP, shares the slides about TiDB with the Big Data Ecosystem. Enjoy~
TiDB, an open source distributed HTAP database. Inspired by Google Spanner/F1, PingCAP develops TiDB, an open source distributed Hybrid Transactional/Analytical Processing (HTAP) database. TiDB features infinite horizontal scalability, strong consistency, and high availability. The goal of TiDB is to serve as a one-stop solution for online transactions and analysis.
Webinar slides: Migrating to Galera Cluster for MySQL and MariaDB
This document provides an overview of online and offline migration strategies for migrating from a standalone MySQL or MySQL master-slave setup to a Galera Cluster. It discusses preparation steps like database schema checks and compatibility. It then outlines the process for offline migration using backups and restore, as well as online migration using MySQL replication to sync data between the existing and new Galera clusters before cutting over. Testing strategies like A/B testing in read-only mode are also presented.
Introduction to Data Engineer and Data Pipeline at Credit OK
The document discusses the role of data engineers and data pipelines. It begins with an introduction to big data and why data volumes are increasing. It then covers what data engineers do, including building data architectures, working with cloud infrastructure, and programming for data ingestion, transformation, and loading. The document also explains data pipelines, describing extract, transform, load (ETL) processes and batch versus streaming data. It provides an example of Credit OK's data pipeline architecture on Google Cloud Platform that extracts raw data from various sources, cleanses and loads it into BigQuery, then distributes processed data to various applications. It emphasizes the importance of data engineers in processing and managing large, complex data sets.
Webinar slides: Designing Open Source Databases for High Availability
It is said that if you are not designing for failure, then you are heading for failure. How do you design a database system from the ground up to withstand failure? This can be a challenge as failures happen in many different ways, sometimes in ways that would be hard to imagine. This is a consequence of the complexity of today’s database environments.
At Severalnines we’re big fans of high availability databases and have seen our fair share of failure scenarios across the thousands of database deployments we enable every year.
In this webinar replay, we’ll look at the different types of failures you might encounter and what mechanisms can be used to address them. We will also look at some of popular HA solutions used today, and how they can help you achieve different levels of availability.
AGENDA
- Why design for High Availability?
- High availability concepts
- CAP theorem
- PACELC theorem
- Trade offs
- Deployment and operational cost
- System complexity
- Performance issues
- Lock management
- Architecting databases for failures
- Capacity planning
- Redundancy
- Load balancing
- Failover and switchover
- Quorum and split brain
- Fencing
- Multi datacenter and multi-cloud setups
- Recovery policy
- High availability solutions
- Database architecture determines Availability
- Active-Standby failover solution with shared storage or DRBD
- Master-slave replication
- Master-master cluster
- Failover and switchover mechanisms
- Reverse proxy
- Caching
- Virtual IP address
- Application connector
SPEAKER
Ashraf Sharif is System Support Engineer at Severalnines. He was previously involved in hosting world and LAMP stack, where he worked as principal consultant and head of support team and delivered clustering solutions for large websites in the South East Asia region. His professional interests are on system scalability and high availability.
Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...
ScyllaDB is a distributed database designed to scale horizontally and vertically — in theory. What about in practice? ScyllaDB’s Benny Halevy, Director, Software Engineering, will take you through the process and results of benchmarking our NoSQL database at the petabyte level, showing how you can use advanced features like workload prioritization to control priorities of transactional (read-write) and analytic (read-only) queries on the same cluster with smooth and predictable performance.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Virtual training Intro to the Tick stack and InfluxEnterprise
In this webinar, we will provide an introduction to the components of the TICK Stack and a review the features of InfluxEnterprise and InfluxCloud. We also demo how to install the TICK stack.
Challenges in Building a Data Pipeline
The document discusses challenges in building a data pipeline including making it highly scalable, available with low latency and zero data loss while supporting multiple data sources. It covers expectations around real-time vs batch processing and streaming vs batch data. Implementation approaches like ETL vs ELT are examined along with replication modes, challenges around schema changes and NoSQL. Effective implementations should address transformations, security, replays, monitoring and more. Reference architectures like Lambda and Kappa are briefly outlined.
Introducing TiDB @ SF DevOps Meetup
This document introduces TiDB, an open source distributed SQL database developed by PingCAP. It provides a 3-part summary:
1) TiDB is a hybrid transactional/analytical database inspired by Google Spanner/F1 that provides horizontal scalability, MySQL compatibility, and ACID transactions. It consists of TiDB, TiKV, and Placement Driver.
2) Mobike, a bike sharing platform with 200 million users, uses TiDB to power operations like bike locking/unlocking tracking and real-time analytics to handle high concurrency and permanent storage needs.
3) Over 200 companies use TiDB for two major uses - MySQL scalability and hybrid OLTP/OLAP architecture
Introducing TiDB Operator [Cologne, Germany]
This slide was delivered at the Kubernetes/Docker meetup in Cologne, Germany, hosted by Giant Swarms on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
More Related Content
What's hot
Presentation at SF Kubernetes Meetup (10/30/18), Introducing TiDB/TiKV
This deck was presented at the SF Kubernetes Meetup held at Microsoft's downtown SF office, introducing the architecture of TiDB and TiKV (a CNCF project), key use cases, a user story with Mobike (one of the largest bikesharing platforms in the world), and how TiDB is deployed across different cloud environment using TiDB Operator.
Presto Summit 2018 - 07 - Lyft
Presto and you are there (Nishant Rayan, Lyft)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Presto Summit 2018 - 09 - Netflix Iceberg
Iceberg: a modern table format for big data (Ryan Blue & Parth Brahmbhatt, Netflix)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Introducing Scylla Open Source 4.0
Since its inception, Scylla has offered a compelling alternative to Apache Cassandra, providing better performance for a lower cost of ownership.
With Scylla Open Source 4.0 we continue to extend our CQL interface features and capabilities and also now provide an open source alternative to DynamoDB, allowing you to run your workloads anywhere, on any cloud provider, or on premises.
Join ScyllaDB co-founders, CTO Avi Kivity and CEO Dor Laor, for a look at the new features in Scylla Open Source 4.0, and architectural and cost comparisons with the coming Cassandra 4.0.
Topics will include:
Improved consistency with our new Lightweight Transactions
Scylla Operator for Kubernetes
How we stack up against Apache Cassandra 4.0
Our “run anywhere” DynamoDB alternative
Reactive database access with Slick3
The document discusses Reactive Slick, a new version of the Slick database access library for Scala that provides reactive capabilities. It allows parallel database execution and streaming of large query results using Reactive Streams. Reactive Slick is suitable for composite database tasks, combining async tasks, and processing large datasets through reactive streams.
Presto Summit 2018 - 02 - LinkedIn
LinkedIn’s Presto adventure (Mark Wagner, LinkedIn)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Mongo DB Monitoring - Become a MongoDB DBA
This presentation was presented by Art van Scheppingen at Percona Live 2017 in Santa Clara CA and covers what you need to know to effectively monitor MongoDB
Introducing TiDB [Delivered: 09/25/18 at Portland Cloud Native Meetup]
This deck introduces TiDB, an open source distributed NewSQL database, to the Portland Cloud Native meetup on September 25, 2018. It includes materials on technical architecture, core features, using TiDB Operator to deploy in any cloud environment, and appendix on transaction model and join support.
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
In this talk AWS’ Ken Krupa, Head of Specialized Solutions Architecture, will describe the architecture and capabilities of two new AWS EC2 instance types perfect for data-intensive storage and IO-heavy workloads like ScyllaDB: the Intel-based I4i and the Graviton2-based I4g series.
The Intel Xeon Ice Lake-based I4i series provides unparalleled raw horsepower for your most demanding workloads. Meanwhile, the Graviton2-powered I4g instances provide lower cost per storage on a power-efficient platform to deploy your cloud-native applications.
Ken will also describe the AWS Nitro SSD, a new form of high-speed NVMe storage with a Flash Translation Layer built with Nitro controllers, which powers both of these instance families.
ScyllaDB VP of Product Tzach Livyatan will then share benchmarking results showing how ScyllaDB behaves under load on these two instance types, providing maximum system utility and efficiency.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Running a DynamoDB-compatible Database on Managed Kubernetes Services
With the release of Alternator, Scylla’s DynamoDB-compatible API, you can now take your locked-in DynamoDB workloads and run them anywhere. Scylla provides a cost-effective open source alternative to Amazon’s DynamoDB, deployable wherever a user would want: on-premises, on other public clouds like Microsoft Azure or Google Cloud Platform, still on AWS (such as the high-density i3en instances) or as a fully managed DBaaS.
In this session, we will cover:
- Scylla Alternator: Scylla’s Amazon DynamoDB-compatible API
- Scylla Operator: Running Scylla Alternator on Kubernetes
- Demo Alternator - Demo and explain DynamoDB on GKE
Introducing TiDB Operator
This document discusses PingCAP's Kubernetes operator for TiDB, an open source distributed SQL database. It provides a brief history of PingCAP and the TiDB community. It then gives a technical overview of TiDB's architecture before explaining how the TiDB operator works. The operator allows users to deploy and manage TiDB clusters on Kubernetes through custom resources that are controlled by custom controllers. This provides capabilities like automated scaling, updates, and failover for stateful applications running on Kubernetes. The operator is open source and TiDB is also available as a managed service on GCP Marketplace.
Webinar 2017. Supercharge your analytics with ClickHouse. Alexander Zaitsev
Alexander Zaitsev presented on LifeStreet's experience implementing ClickHouse for their ad analytics platform. Some key points:
- LifeStreet processes over 10 billion events per day from their ad exchange and needed a high performance analytics solution.
- They tried various databases but migrated fully to ClickHouse due to its performance for analytics workloads.
- Major challenges included designing an efficient schema, sharding and replication strategy, and reliable data ingestion.
- ClickHouse's dictionary feature allowed them to implement normalized dimensions tables while supporting updates, improving storage efficiency and query performance.
Lookout on Scaling Security to 100 Million Devices
The massive increase of security-related data requires companies to respond with new approaches to ingestion. Learn how Lookout has changed its approach for ingesting telemetry to meet their goal of growing from 1.5 million devices to 100 million devices and beyond, using Kafka Connect and switching from AWS DynamoDB to Scylla.
TiDB for Big Data
Shen Li, VP engineering at PingCAP, shares the slides about TiDB with the Big Data Ecosystem. Enjoy~
TiDB, an open source distributed HTAP database. Inspired by Google Spanner/F1, PingCAP develops TiDB, an open source distributed Hybrid Transactional/Analytical Processing (HTAP) database. TiDB features infinite horizontal scalability, strong consistency, and high availability. The goal of TiDB is to serve as a one-stop solution for online transactions and analysis.
Webinar slides: Migrating to Galera Cluster for MySQL and MariaDB
This document provides an overview of online and offline migration strategies for migrating from a standalone MySQL or MySQL master-slave setup to a Galera Cluster. It discusses preparation steps like database schema checks and compatibility. It then outlines the process for offline migration using backups and restore, as well as online migration using MySQL replication to sync data between the existing and new Galera clusters before cutting over. Testing strategies like A/B testing in read-only mode are also presented.
Introduction to Data Engineer and Data Pipeline at Credit OK
The document discusses the role of data engineers and data pipelines. It begins with an introduction to big data and why data volumes are increasing. It then covers what data engineers do, including building data architectures, working with cloud infrastructure, and programming for data ingestion, transformation, and loading. The document also explains data pipelines, describing extract, transform, load (ETL) processes and batch versus streaming data. It provides an example of Credit OK's data pipeline architecture on Google Cloud Platform that extracts raw data from various sources, cleanses and loads it into BigQuery, then distributes processed data to various applications. It emphasizes the importance of data engineers in processing and managing large, complex data sets.
Webinar slides: Designing Open Source Databases for High Availability
It is said that if you are not designing for failure, then you are heading for failure. How do you design a database system from the ground up to withstand failure? This can be a challenge as failures happen in many different ways, sometimes in ways that would be hard to imagine. This is a consequence of the complexity of today’s database environments.
At Severalnines we’re big fans of high availability databases and have seen our fair share of failure scenarios across the thousands of database deployments we enable every year.
In this webinar replay, we’ll look at the different types of failures you might encounter and what mechanisms can be used to address them. We will also look at some of popular HA solutions used today, and how they can help you achieve different levels of availability.
AGENDA
- Why design for High Availability?
- High availability concepts
- CAP theorem
- PACELC theorem
- Trade offs
- Deployment and operational cost
- System complexity
- Performance issues
- Lock management
- Architecting databases for failures
- Capacity planning
- Redundancy
- Load balancing
- Failover and switchover
- Quorum and split brain
- Fencing
- Multi datacenter and multi-cloud setups
- Recovery policy
- High availability solutions
- Database architecture determines Availability
- Active-Standby failover solution with shared storage or DRBD
- Master-slave replication
- Master-master cluster
- Failover and switchover mechanisms
- Reverse proxy
- Caching
- Virtual IP address
- Application connector
SPEAKER
Ashraf Sharif is System Support Engineer at Severalnines. He was previously involved in hosting world and LAMP stack, where he worked as principal consultant and head of support team and delivered clustering solutions for large websites in the South East Asia region. His professional interests are on system scalability and high availability.
Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...
ScyllaDB is a distributed database designed to scale horizontally and vertically — in theory. What about in practice? ScyllaDB’s Benny Halevy, Director, Software Engineering, will take you through the process and results of benchmarking our NoSQL database at the petabyte level, showing how you can use advanced features like workload prioritization to control priorities of transactional (read-write) and analytic (read-only) queries on the same cluster with smooth and predictable performance.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Virtual training Intro to the Tick stack and InfluxEnterprise
In this webinar, we will provide an introduction to the components of the TICK Stack and a review the features of InfluxEnterprise and InfluxCloud. We also demo how to install the TICK stack.
Challenges in Building a Data Pipeline
The document discusses challenges in building a data pipeline including making it highly scalable, available with low latency and zero data loss while supporting multiple data sources. It covers expectations around real-time vs batch processing and streaming vs batch data. Implementation approaches like ETL vs ELT are examined along with replication modes, challenges around schema changes and NoSQL. Effective implementations should address transformations, security, replays, monitoring and more. Reference architectures like Lambda and Kappa are briefly outlined.
What's hot (20)
Presentation at SF Kubernetes Meetup (10/30/18), Introducing TiDB/TiKV
Presentation at SF Kubernetes Meetup (10/30/18), Introducing TiDB/TiKV
Introducing TiDB [Delivered: 09/25/18 at Portland Cloud Native Meetup]
Introducing TiDB [Delivered: 09/25/18 at Portland Cloud Native Meetup]
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
Running a DynamoDB-compatible Database on Managed Kubernetes Services
Running a DynamoDB-compatible Database on Managed Kubernetes Services
Webinar 2017. Supercharge your analytics with ClickHouse. Alexander Zaitsev
Webinar 2017. Supercharge your analytics with ClickHouse. Alexander Zaitsev
Lookout on Scaling Security to 100 Million Devices
Lookout on Scaling Security to 100 Million Devices
Webinar slides: Migrating to Galera Cluster for MySQL and MariaDB
Webinar slides: Migrating to Galera Cluster for MySQL and MariaDB
Introduction to Data Engineer and Data Pipeline at Credit OK
Introduction to Data Engineer and Data Pipeline at Credit OK
Webinar slides: Designing Open Source Databases for High Availability
Webinar slides: Designing Open Source Databases for High Availability
Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...
Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...
Virtual training Intro to the Tick stack and InfluxEnterprise
Virtual training Intro to the Tick stack and InfluxEnterprise
Similar to Introducing TiDB - Percona Live Frankfurt
Introducing TiDB @ SF DevOps Meetup
This document introduces TiDB, an open source distributed SQL database developed by PingCAP. It provides a 3-part summary:
1) TiDB is a hybrid transactional/analytical database inspired by Google Spanner/F1 that provides horizontal scalability, MySQL compatibility, and ACID transactions. It consists of TiDB, TiKV, and Placement Driver.
2) Mobike, a bike sharing platform with 200 million users, uses TiDB to power operations like bike locking/unlocking tracking and real-time analytics to handle high concurrency and permanent storage needs.
3) Over 200 companies use TiDB for two major uses - MySQL scalability and hybrid OLTP/OLAP architecture
Introducing TiDB Operator [Cologne, Germany]
This slide was delivered at the Kubernetes/Docker meetup in Cologne, Germany, hosted by Giant Swarms on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
Scale Relational Database with NewSQL
This is the speech Shen Li gave at the 3rd NEXTCON: Cloud+Data NEXT Conference Seattle on September 16th, 2017.
"Smooth Operator" [Bay Area NewSQL meetup]
This slide was delivered at the Bay Area NewSQL meetup in California on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
TiDB as an HTAP Database
TiDB is a NewSQL database that provides horizontal scalability, ACID transactions, high availability, and SQL support. It aims to be an HTAP (Hybrid Transactional/Analytical Processing) database by supporting both OLTP and OLAP workloads on the same database using the same SQL interface.
TiDB achieves horizontal scalability through its distributed architecture with the TiKV storage engine and PD for metadata management. It supports ACID transactions through MVCC and Raft consensus. The database is available through replication of regions across nodes. TiDB also supports real-time analytics on the same dataset as transactions through its cost-based optimizer and distributed query processing engine.
Spark can run queries directly against the
A Brief Introduction of TiDB (Percona Live)
TiDB is an open-source distributed SQL database that supports high availability, horizontal scalability, and consistent distributed transactions. It provides a MySQL compatible API and seamless online expansion. TiDB uses Raft for consensus and implements the MVCC model to support high concurrency. It also provides distributed transactions through a two-phase commit protocol. The architecture consists of a stateless SQL layer (TiDB) and a distributed transactional key-value storage (TiKV).
When Apache Spark Meets TiDB with Xiaoyu Ma
During the past 10 years, big-data storage layers mainly focus on analytical use cases. When it comes to analytical cases, users usually offload data onto Hadoop cluster and perform queries on HDFS files. People struggle dealing with modifications on append only storage and maintain fragile ETL pipelines.
On the other hand, although Spark SQL has been proven effective parallel query processing engine, some tricks common in traditional databases are not available due to characteristics of storage underneath. TiSpark sits directly on top of a distributed database (TiDB)’s storage engine, expand Spark SQL’s planning with its own extensions and utilizes unique features of database storage engine to achieve functions not possible for Spark SQL on HDFS. With TiSpark, users are able to perform queries directly on changing / fresh data in real time.
The takeaways from this two are twofold:
— How to integrate Spark SQL with a distributed database engine and the benefit of it
— How to leverage Spark SQL’s experimental methods to extend its capacity.
TiDB + Mobike by Kevin Xu (@kevinsxu)
This presentation provides an overview of the architecture and technology of TiDB, an open-source distributed NewSQL database, and how it helps Mobike, one of the largest dockless bikeshare platform, scale its infrastructure to achieve hyper-growth.
TiDB DevCon 2020 Opening Keynote
At TiDB DevCon 2020, Max Liu, CEO at PingCAP, gave a keynote speech. He believes that today’s database should be more real-time, more flexible, and easier to use, and TiDB, an elastic, cloud-native, real-time HTAP database, is exactly that kind of database.
TiDB vs Aurora.pdf
TiDB and Amazon Aurora can be combined to provide analytics on transactional data without needing a separate data warehouse. TiDB Data Migration (DM) tool allows migrating and replicating data from Aurora into TiDB for analytics queries. DM provides full data migration and incremental replication of binlog events from Aurora into TiDB. This allows joining transactional and analytical workloads on the same dataset without needing ETL pipelines.
Keynote -- Percona Live Europe 2018
This deck was the keynote speech delivered by Kevin Xu (GM of Global Strategy at Operations) and Shen Li (VP of Engineering at PingCAP) on TiDB architecture, tools and migration path, and TiDB Cloud fully-managed offering at Percona Live Europe 2018 in Frankfurt, Germany.
DPDK summit 2015: It's kind of fun to do the impossible with DPDK
The document discusses using Lagopus software-defined networking (SDN) switches to demonstrate an SDN internet exchange (IX) at the Interop Tokyo 2015 technology show. Key points:
- Two Lagopus SDN switches were deployed as the core switches in an SDN IX to enable automated provisioning of inter-autonomous system layer 2 connectivity and on-demand packet filtering between internet service providers.
- The Lagopus switches achieved an average throughput of 2Gbps with no packet drops over a week during the show, demonstrating the potential for software switches in next-generation SDNs.
- Previous work to optimize the Lagopus switch performance through techniques like hardware offloading to FPGAs helped enable its
DPDK Summit 2015 - NTT - Yoshihiro Nakajima
DPDK Summit 2015 in San Francisco.
NTT presentation by Yoshihiro Nakajima.
For additional details and the video recording please visit www.dpdksummit.com.
Data-at-scale-with-TIDB Mydbops Co-Founder Kabilesh PR at LSPE Event
Explore the world of TiDB with Kabilesh PR, Co-Founder of Mydbops, as he unveils the potential of this open-source distributed SQL database. Dive into the architecture, scalability solutions, and production readiness of TiDB, and discover how it addresses MySQL scalability bottlenecks through sharding. Gain insights into its stateless SQL interface, transactional storage with TiKV, and analytical capabilities with TiFlash. Learn about TiDB's native sharding features, use cases across various industries, and its readiness for production environments. Delve into its limitations and discover how TiDB can transform your data management landscape.
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...
Discover how Mydbops achieved an impressive 80% cost savings and ensured uninterrupted availability through a transformative MySQL database case study. Join Vinoth Kanna RS, Co-Founder of Mydbops, as he shares insights into optimizing infrastructure, enhancing observability, and navigating critical technology decisions. Learn from real-world challenges, innovative solutions, and valuable takeaways for your own database management endeavors.
Titan and Cassandra at WellAware
An overview of the graph database ecosystem followed by an introduction to Titan and how it integrates with Cassandra.
The Nitty Gritty of Advanced Analytics Using Apache Spark in Python
Apache Spark is the next big data processing tool for Data Scientist. As seen on the recent StackOverflow analysis, it's the hottest big data technology on their site! In this talk, I'll use the PySpark interface to leverage the speed and performance of Apache Spark. I'll focus on the end to end workflow for getting data into a distributed platform, and leverage Spark to process the data for advanced analytics. I'll discuss the popular Spark APIs used for data preparation, SQL analysis, and ML algorithms. I'll explain the performance differences between Scala and Python, and how Spark has bridged the gap in performance. I'll focus on PySpark as the interface to the platform, and walk through a demo to showcase the APIs.
Talk Overview:
Spark's Architecture. What's out now and what's in Spark 2.0Spark APIs: Most common APIs used by Spark Common misconceptions and proper techniques for using Spark.
Demo:
Walk through ETL of the Reddit dataset. SparkSQL Analytics + Visualizations of the Dataset using MatplotLibSentiment Analysis on Reddit Comments
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo Mazzaferro
Those of us who use TensorFlow often focus on building the model that's most predictive, not the one that's most deployable. So how to put that hard work to work? In this talk, we'll walk through a strategy for taking your machine learning models from Jupyter Notebook into production and beyond.
Presto @ Zalando - Big Data Tech Warsaw 2020
A cloud journey for Europe’s leading online fashion retailer. Zalando Data Lake evolution and role of Starburst Presto in the Data Lake's landscape.
SQREAM DB on IBM Power9
GPU data warehouse Sqream DB provides a massively parallel processing engine powered by GPUs that is faster and more efficient than CPU-based systems. It can ingest terabytes of data per hour onto a single GPU and handle petabytes of data stored in a compact 2U server. With familiar SQL queries and connectors, Sqream DB accelerates analytics by 100x over traditional warehouses through its GPU-accelerated processing and columnar storage.
Similar to Introducing TiDB - Percona Live Frankfurt (20)
DPDK summit 2015: It's kind of fun to do the impossible with DPDK
DPDK summit 2015: It's kind of fun to do the impossible with DPDK
Data-at-scale-with-TIDB Mydbops Co-Founder Kabilesh PR at LSPE Event
Data-at-scale-with-TIDB Mydbops Co-Founder Kabilesh PR at LSPE Event
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...
MySQL Transformation Case Study: 80% Cost Savings & Uninterrupted Availabilit...
The Nitty Gritty of Advanced Analytics Using Apache Spark in Python
The Nitty Gritty of Advanced Analytics Using Apache Spark in Python
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo Mazzaferro
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo Mazzaferro
More from Morgan Tocker
Introducing Spirit - Online Schema Change
This document discusses Spirit, an online schema change utility for MySQL 8.0. It begins by covering the state of DDL operations in MySQL and how Spirit works to perform schema changes without blocking reads or writes. It then discusses optimizations Spirit uses and features like checkpointing. Finally, it outlines some feature requests to make more operations instant or inplace in MySQL to reduce the need for Spirit in many cases.
MySQL Usability Guidelines
The document outlines 10 usability guidelines for MySQL:
1) All features should be possible through SQL for consistency and discoverability.
2) Features, configurations, and errors should be intuitively obvious and discoverable without reading manuals cover-to-cover.
3) Too many similar configuration options without clear use cases can be paralyzing; only add options if use cases are known.
4) New configuration options must allow the effect to be measured through observability.
5) Features should work consistently across contexts for orthogonality.
6) The system should be safe to script against and avoid duplicate processing.
7) Extend functionality to match common use cases.
8) Preserve the ability to
MySQL 8.0 Optimizer Guide
The document is an introduction to the MySQL 8.0 optimizer guide. It includes a safe harbor statement noting that the guide outlines Oracle's general product direction but not commitments. The agenda lists 25 topics to be covered related to query optimization, diagnostic commands, examples from the "World Schema" sample database, and a companion website with more details.
MySQL Server Defaults
The document discusses proposed changes to MySQL Server 8.0 and replication defaults. Some key areas discussed include changing the default character set to UTF8MB4, turning on the event scheduler by default, increasing some session buffer sizes, enabling security defaults, and enabling replication features like binary logging and GTIDs by default. The document seeks feedback from users on the proposed changes.
MySQL Cloud Service Deep Dive
The document discusses Oracle's MySQL Cloud Service which provides MySQL as a database service on Oracle Public Cloud. Key features include automated backups, patching, monitoring, elastic scaling, high availability, security features from MySQL Enterprise Edition, and tools for data access, migration and restoration. The service runs MySQL 5.7 Enterprise Edition with an optimized configuration for the cloud environment.
MySQL 5.7 + JSON
MySQL 5.7 introduced native support for JSON data with a new JSON data type and JSON functions. The JSON type allows efficient storage and access of JSON documents compared to traditional text storage. JSON functions allow querying and manipulating JSON data through operations like extraction, search, and generation of JSON values. Developers now have more flexibility to work with hierarchical and unstructured data directly in MySQL.
Using MySQL in Automated Testing
This document discusses using MySQL in automated testing. It covers various tools that can be used to automate and manage database deployments as part of testing, including pt-online-schema-change, MySQL Sandbox, SYS, Outbrain Propagator, Liquibase, ORM migrations, and libeatmydata. It also discusses considerations for different MySQL versions, such as online DDL support being introduced in MySQL 5.6. The document aims to demonstrate that databases can and should be automated and treated as first-class citizens in testing environments.
Upcoming changes in MySQL 5.7
The document discusses upcoming changes and new features in MySQL 5.7. Key points include:
- MySQL 5.7 development has focused on performance, scalability, security and refactoring code.
- New features include online DDL support for additional DDL statements, InnoDB support for spatial data types, and cost information added to EXPLAIN output.
- Benchmarks show MySQL 5.7 providing significantly higher performance than previous versions, with a peak of 645,000 queries/second on some workloads.
MySQL Query Optimization
This document discusses query optimization in MySQL. It provides an introduction to how the MySQL query optimizer works to determine the most efficient execution plan for a SQL query. Several examples are shown using the EXPLAIN statement to analyze queries against sample data in the World Schema. Indexes are added and analyzed to demonstrate how they can improve query performance in different scenarios. The document also discusses some general strategies and rules of thumb used by the query optimizer.
MySQL Performance Metrics that Matter
This document discusses various MySQL performance metrics that are important to measure from within the database, operating system, and application. It outlines key InnoDB internal structures like the buffer pool and log system. Specific metrics that provide insight into buffer pool usage, page churn, and log writes are highlighted. Optimizing the working set size and ensuring sufficient free space in the log files are important factors for performance.
MySQL For Linux Sysadmins
This document provides an overview of MySQL for Linux system administrators. It discusses MySQL architecture including storage engines, memory usage, the MySQL server process, and InnoDB transaction processing. It also covers topics like backups and replication, and the agenda includes performance and capacity planning. The goal is to help system administrators understand and manage MySQL databases.
MySQL: From Single Instance to Big Data
The document discusses various MySQL database architectures for different usage needs, from single server setups to high availability configurations. It begins with traditional single server and web/database tier setups. It then covers high availability options using MySQL replication, shared storage, and MySQL Cluster. Popular topologies include master-slave replication for scaling reads, read-write splitting between master and slaves, and using slaves for reporting queries to improve performance. Considerations like network latency, failure handling, and limitations of read-write splitting are also discussed.
MySQL NoSQL APIs
The document discusses NoSQL APIs in MySQL. It provides an overview of the memcached caching system and the history of the HandlerSocket protocol. It then describes the NoSQL interface introduced in MySQL 5.6, which allows for memcached-style operations on MySQL data. It notes that MySQL 5.7 further improved the performance and scalability of this interface.
MySQL 5.7: Core Server Changes
The document outlines changes and new features in MySQL versions 5.7 through upcoming releases. Key points include:
- MySQL 5.7 development follows a milestone release process to stabilize new features before general availability. Four development milestone releases have been completed so far.
- Notable 5.7 features include statement timeouts, change replication without stopping SQL threads, and performance improvements like optimized UNION ALL queries.
- Some existing functionality will change in 5.7, like making replication more durable by default and producing errors for queries with only partial GROUP BY clauses.
- Ongoing efforts include refactoring and improving InnoDB, the optimizer, and other components for better performance and scalability. New features in development
MySQL 5.6 - Operations and Diagnostics Improvements
This document discusses MySQL 5.6 and its improvements to operational and diagnostic capabilities. Key enhancements include online DDL operations that do not block reads or writes, buffer pool dump and restore for faster startup, import/export of partitioned tables, and transportable tablespaces. Diagnostic tools were improved with EXPLAIN showing more details, the ability to EXPLAIN updates and deletes, optimizer tracing, and the performance schema providing detailed query level instrumentation and monitoring by default.
Locking and Concurrency Control
The document discusses locking and concurrency control in databases, demonstrating how table locks, row locks, and multi-version concurrency control work through examples of a database being backed up while concurrent changes are made. It shows how different locking strategies, like those used in MyISAM and InnoDB, allow for concurrent access to data while maintaining consistency and isolation. A live demo then highlights deadlocks and lock waits that can occur with concurrent access and how they are handled.
The InnoDB Storage Engine for MySQL
The document provides an overview of the InnoDB storage engine used in MySQL. It discusses InnoDB's architecture including the buffer pool, log files, and indexing structure using B-trees. The buffer pool acts as an in-memory cache for table data and indexes. Log files are used to support ACID transactions and enable crash recovery. InnoDB uses B-trees to store both data and indexes, with rows of variable length stored within pages.
My sql 5.7-upcoming-changes-v2
MySQL 5.7 proposes several changes to improve performance and consistency including:
1. Making replication durable by default by setting sync_binlog and repository options.
2. Deprecating features like INNODB monitor tables and ALTER IGNORE TABLE in favor of newer standards.
3. Simplifying and restricting SQL modes to encourage stricter querying and remove ambiguous options. Explanations for errors and modes will also be improved.
Mysql 57-upcoming-changes
Upcoming Changes in MySQL 5.7
Morgan Tocker, MySQL Community Manager
The presentation outlines several proposed changes and deprecations in MySQL 5.7, including making replication more durable by default, replacing some InnoDB monitor tables with performance schema instrumentation, simplifying SQL modes, and deprecating features like ALTER IGNORE TABLE, the \N NULL synonym, and the Federated storage engine. Feedback is sought on these proposed changes to help guide MySQL's future development.
Optimizing MySQL
This document discusses various methods for optimizing performance of MySQL databases, including upgrading hardware and software, optimizing configuration settings, optimizing queries, and optimizing database schemas. It provides an example of using EXPLAIN plans and adding indexes to optimize queries on a database table to improve performance. The author recommends focusing on query optimization as the best method, using profilers and slow query logs to identify queries to optimize.
More from Morgan Tocker (20)
MySQL 5.6 - Operations and Diagnostics Improvements
MySQL 5.6 - Operations and Diagnostics Improvements
Recently uploaded
The Rising Future of CPaaS in the Middle East 2024
Explore "The Rising Future of CPaaS in the Middle East in 2024" with this comprehensive PPT presentation. Discover how Communication Platforms as a Service (CPaaS) is transforming communication across various sectors in the Middle East.
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation with Azure DevOps Build & Release Pipelines
Automating end-to-end (e2e) test for Android and iOS native apps, and web apps, within Azure build and release pipelines, poses several challenges. This session dives into the key challenges and the repeatable solutions implemented across multiple teams at a leading Indian telecom disruptor, renowned for its affordable 4G/5G services, digital platforms, and broadband connectivity.
Challenge #1. Ensuring Test Environment Consistency: Establishing a standardized test execution environment across hundreds of Azure DevOps agents is crucial for achieving dependable testing results. This uniformity must seamlessly span from Build pipelines to various stages of the Release pipeline.
Challenge #2. Coordinated Test Execution Across Environments: Executing distinct subsets of tests using the same automation framework across diverse environments, such as the build pipeline and specific stages of the Release Pipeline, demands flexible and cohesive approaches.
Challenge #3. Testing on Linux-based Azure DevOps Agents: Conducting tests, particularly for web and native apps, on Azure DevOps Linux agents lacking browser or device connectivity presents specific challenges in attaining thorough testing coverage.
This session delves into how these challenges were addressed through:
1. Automate the setup of essential dependencies to ensure a consistent testing environment.
2. Create standardized templates for executing API tests, API workflow tests, and end-to-end tests in the Build pipeline, streamlining the testing process.
3. Implement task groups in Release pipeline stages to facilitate the execution of tests, ensuring consistency and efficiency across deployment phases.
4. Deploy browsers within Docker containers for web application testing, enhancing portability and scalability of testing environments.
5. Leverage diverse device farms dedicated to Android, iOS, and browser testing to cover a wide range of platforms and devices.
6. Integrate AI technology, such as Applitools Visual AI and Ultrafast Grid, to automate test execution and validation, improving accuracy and efficiency.
7. Utilize AI/ML-powered central test automation reporting server through platforms like reportportal.io, providing consolidated and real-time insights into test performance and issues.
These solutions not only facilitate comprehensive testing across platforms but also promote the principles of shift-left testing, enabling early feedback, implementing quality gates, and ensuring repeatability. By adopting these techniques, teams can effectively automate and execute tests, accelerating software delivery while upholding high-quality standards across Android, iOS, and web applications.
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
原版定制【微信:bwp0011】《(hull学位证书)英国赫尔大学毕业证硕士文凭》【微信:bwp0011】成绩单 、雅思、外壳、留信学历认证永久存档查询，采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信bwp0011】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信bwp0011】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
Building API data products on top of your real-time data infrastructure
This talk and live demonstration will examine how Confluent and Gravitee.io integrate to unlock value from streaming data through API products.
You will learn how data owners and API providers can document, secure data products on top of Confluent brokers, including schema validation, topic routing and message filtering.
You will also see how data and API consumers can discover and subscribe to products in a developer portal, as well as how they can integrate with Confluent topics through protocols like REST, Websockets, Server-sent Events and Webhooks.
Whether you want to monetize your real-time data, enable new integrations with partners, or provide self-service access to topics through various protocols, this webinar is for you!
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
原版一模一样【微信：741003700 】【(sdsu毕业证书)圣地亚哥州立大学毕业证成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理(sdsu毕业证书)圣地亚哥州立大学毕业证【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理(sdsu毕业证书)圣地亚哥州立大学毕业证【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理(sdsu毕业证书)圣地亚哥州立大学毕业证【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理(sdsu毕业证书)圣地亚哥州立大学毕业证【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar
June. 18, 2024
For more Alluxio Events: https://www.alluxio.io/events/
Speaker:
- Jianjian Xie (Staff Software Engineer, Alluxio)
As Trino users increasingly rely on cloud object storage for retrieving data, speed and cloud cost have become major challenges. The separation of compute and storage creates latency challenges when querying datasets; scanning data between storage and compute tiers becomes I/O bound. On the other hand, cloud API costs related to GET/LIST operations and cross-region data transfer add up quickly.
The newly introduced Trino file system cache by Alluxio aims to overcome the above challenges. In this session, Jianjian will dive into Trino data caching strategies, the latest test results, and discuss the multi-level caching architecture. This architecture makes Trino 10x faster for data lakes of any scale, from GB to EB.
What you will learn:
- Challenges relating to the speed and costs of running Trino in the cloud
- The new Trino file system cache feature overview, including the latest development status and test results
- A multi-level cache framework for maximized speed, including Trino file system cache and Alluxio distributed cache
- Real-world cases, including a large online payment firm and a top ridesharing company
- The future roadmap of Trino file system cache and Trino-Alluxio integration
Boost Your Savings with These Money Management Apps
A money management app can transform your financial life by tracking expenses, creating budgets, and setting financial goals. These apps offer features like real-time expense tracking, bill reminders, and personalized insights to help you save and manage money effectively. With a user-friendly interface, they simplify financial planning, making it easier to stay on top of your finances and achieve long-term financial stability.
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
ACE - Team 24 Wrapup event at ahmedabad.
Atlassian Team 24 event wrap-up Ahmedabad. Recap of Atlassian team24 event.
14 th Edition of International conference on computer vision
About the event
14th Edition of International conference on computer vision
Computer conferences organized by ScienceFather group. ScienceFather takes the privilege to invite speakers participants students delegates and exhibitors from across the globe to its International Conference on computer conferences to be held in the Various Beautiful cites of the world. computer conferences are a discussion of common Inventions-related issues and additionally trade information share proof thoughts and insight into advanced developments in the science inventions service system. New technology may create many materials and devices with a vast range of applications such as in Science medicine electronics biomaterials energy production and consumer products.
Nomination are Open!! Don't Miss it
Visit: computer.scifat.com
Award Nomination: https://x-i.me/ishnom
Conference Submission: https://x-i.me/anicon
For Enquiry: Computer@scifat.com
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Moving to a more digitally focused era, the importance of software is rapidly increasing. Software tools are crucial for upgrading life standards, enhancing business prospects, and making a smart world. The smooth and fail-proof functioning of the software is very critical, as a large number of people are dependent on them.
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Closing talk for the Performance Engineering track at Community Over Code EU (Bratislava, Slovakia, June 5 2024) https://eu.communityovercode.org/sessions/2024/why-apache-kafka-clusters-are-like-galaxies-and-other-cosmic-kafka-quandaries-explored/ Instaclustr (now part of NetApp) manages 100s of Apache Kafka clusters of many different sizes, for a variety of use cases and customers. For the last 7 years I’ve been focused outwardly on exploring Kafka application development challenges, but recently I decided to look inward and see what I could discover about the performance, scalability and resource characteristics of the Kafka clusters themselves. Using a suite of Performance Engineering techniques, I will reveal some surprising discoveries about cosmic Kafka mysteries in our data centres, related to: cluster sizes and distribution (using Zipf’s Law), horizontal vs. vertical scalability, and predicting Kafka performance using metrics, modelling and regression techniques. These insights are relevant to Kafka developers and operators.
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
Ensuring the optimal performance of your audio-visual (AV) equipment is crucial for delivering exceptional experiences. AV performance validation is a critical process that verifies the quality and functionality of your AV setup. Whether you're a content creator, a business conducting webinars, or a homeowner creating a home theater, validating your AV performance is essential.
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
Visual testing plays a vital role in ensuring that software products meet the aesthetic requirements specified by clients in functional and non-functional specifications. In today's highly competitive digital landscape, users expect a seamless and visually appealing online experience. Visual testing, also known as automated UI testing or visual regression testing, verifies the accuracy of the visual elements that users interact with.
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0
🏎️Tech Transformation: DevOps Insights from the Experts 👩💻
Connect with fellow Trailblazers, learn from industry experts Glenda Thomson (Salesforce, Principal Technical Architect) and Will Dinn (Judo Bank, Salesforce Development Lead), and discover how to harness DevOps tools with Salesforce.
Superpower Your Apache Kafka Applications Development with Complementary Open...
Kafka Summit talk (Bangalore, India, May 2, 2024, https://events.bizzabo.com/573863/agenda/session/1300469 )
Many Apache Kafka use cases take advantage of Kafka’s ability to integrate multiple heterogeneous systems for stream processing and real-time machine learning scenarios. But Kafka also exists in a rich ecosystem of related but complementary stream processing technologies and tools, particularly from the open-source community. In this talk, we’ll take you on a tour of a selection of complementary tools that can make Kafka even more powerful. We’ll focus on tools for stream processing and querying, streaming machine learning, stream visibility and observation, stream meta-data, stream visualisation, stream development including testing and the use of Generative AI and LLMs, and stream performance and scalability. By the end you will have a good idea of the types of Kafka “superhero” tools that exist, which are my favourites (and what superpowers they have), and how they combine to save your Kafka applications development universe from swamploads of data stagnation monsters!
Recently uploaded (20)
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
Building API data products on top of your real-time data infrastructure
Building API data products on top of your real-time data infrastructure
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
🏎️Tech Transformation: DevOps Insights from the Experts 👩💻
🏎️Tech Transformation: DevOps Insights from the Experts 👩💻
Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...
Introducing TiDB - Percona Live Frankfurt
- 1. Introducing TiDB For those coming from MySQL...
- 2. Agenda ● History and Community ● Technical Walkthrough ● Use Case with Mobike ● MySQL Compatibility
- 3. A little about me ● Senior Product / Community Manager ● ~15 years MySQL Experience ○ MySQL AB, Sun, Percona, Oracle ● Previously Product Manager for MySQL Server
- 4. A little about PingCAP ● Founded in April 2015 by 3 infrastructure engineers ● Offices in China and North America ● Remote Friendly! ○ I work from here ➡
- 6. PingCAP.com Our Product is the TiDB Platform ● TiDB Platform (Ti = Titanium) ○ TiDB (SQL Layer) ○ TiKV (Storage) ○ TiSpark (Apache Spark plugin to TiKV) ● Open source from Day 1 ○ GA 1.0: October 2017 ○ GA 2.0: April 2018
- 7. PingCAP.com TiDB is a NewSQL Database RDBMS NoSQL NewSQL 1970s 2010 2015 MySQL PostgreSQL Oracle DB2... Redis HBase Cassandra MongoDB Present Google Spanner Google F1 TiDB
- 8. Common Use Cases 1. MySQL Scalability 2. Hybrid OLTP/OLAP Architecture 3. Unifying Data Storage/Management
- 9. PingCAP.com Architecture (Simplified) TiDB TiDB TiDB Applicationvia MySQLProtocol TiKV TiKV TiKV TiKV TiKV TiKV ... ... PD Cluster DistSQL API KV API PD PD PD Metadata TSO / Data Location
- 10. PingCAP.com Community Stars ● TiDB: 15,000+ ● TiKV: 3700+ Contributors ● TiDB: 200+ ● TiKV: 100+
- 11. PingCAP.com Sneak Peek! Early Sign-up: https://www.pingcap.com/tidb-academy/
- 13. PingCAP.com Platform Architecture TiDB TiDB TiDB Applicationvia MySQLProtocol TiKV TiKV TiKV TiKV TiKV TiKV Worker Worker Worker Spark Driver ... ... ... SparkSQL Spark Cluster PD Cluster DistSQL API KV API DistSQL API PD PD PD Metadata TSO / Data Location Data Location
- 14. PingCAP.com TiKV: The Storage Foundation RocksDB Raft Transaction Txn KV API Coprocessor API RocksDB Raft Transaction Txn KV API Coprocessor API RocksDB Raft Transaction Txn KV API Coprocessor API Raft Group Client gRPC TiKV Instance TiKV Instance TiKV Instance gRPC gRPC PD Cluster
- 15. PingCAP.com TiDB: The SQL Layer Node1 Node2 Node3 Node4 MySQL Network Protocol SQL Parser Cost-based Optimizer Distributed Executor (Coprocessor) ODBC/JDBC MySQL Client Any ORM which supports MySQL TiDB TiKV
- 18. Common Use Cases 1. MySQL Scalability 2. Hybrid OLTP/OLAP Architecture 3. Unifying Data Storage/Management
- 19. PingCAP.com Mobike + TiDB ● 200 million users ● 200 cities ● 9 million smart bikes ● ~30 TB / day
- 21. PingCAP.com Summary ● Compatibility with MySQL 5.7 ○ Joins, subqueries, DML, DDL etc. ● On the roadmap: ○ Views, CTEs, Window Functions, GIS ● Missing: ○ Stored Procedures, Triggers, Events, Fulltext pingcap.com /docs/sql/mysql-compatibility/
- 22. PingCAP.com Nuanced ● Some features work differently ○ Auto Increment ○ Optimistic Locking ● TiDB works better with smaller transactions ○ Recommended to batch updates, deletes, inserts to 5000 rows pingcap.com /docs/sql/mysql-compatibility/