Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

M|18 Intel and MariaDB: Strategic Collaboration to Enhance MariaDB Functionality, Performance and TCO

435 views

Published on

M|18 Intel and MariaDB: Strategic Collaboration to Enhance MariaDB Functionality, Performance and TCO

Published in: Data & Analytics
  • Be the first to comment

M|18 Intel and MariaDB: Strategic Collaboration to Enhance MariaDB Functionality, Performance and TCO

  1. 1. Strategic collaboration to enhance MariaDB functionality, performance & TCO
  2. 2. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at intel.com. No computer system can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit http://www.intel.com/performance. Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate. © 2018 Intel Corporation. Intel, the Intel logo, and Intel Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as property of others. 2 Notices & Disclaimers
  3. 3. • Collaborating to transform MariaDB transaction performance and cost • Enabling reliable, real-time web-scale applications through distributed log capabilities • Accelerating MariaDB with Intel® FPGAs 3 Agenda
  4. 4. 4 Collaborating to Transform MariaDB transaction performance and cost
  5. 5. Cloud & DATA Center Things & Devices MEM ORY FPGA Cloud and Data Center, AI, SD Infrastructure, Big Data Analytics Manufacturing, Memory, FPGA, Software, Security Intel Capital leads $20 million investment to grow MariaDB Open source database gets funding to boost adoption and develop product range Intel® Xeon® Processor Scalable Family 3D XPointTM Intel® OptaneTM SSD DC P4800X Intel® Stratix® 10 FPGA Virtuous Cycle of Growth
  6. 6. Intel’s Software Group: MariaDB Team 6 Shanghai Intel DCG database performance lab (Intel® Xeon®) Moscow MariaDB database performance Early Access lab London Database performance tools & customer engagements Munich MariaDB - Intel Global Relationship Management
  7. 7. 7 Working together to optimize the combination of: • MariaDB* and MyRocks* storage engine • Intel® Xeon® Scalable Platform • Intel® Optane™ SSDs Co-Marketing Collaborations MariaDB Intel Collaboration Early access to Intel pre-production H/W GoToMarket support and customer Trusted Advisors Joint performance analysis and tuning S/W development tools Full tech. support Long-term development project collaboration MyRocks
  8. 8. delivers 1.65x average performance boost over prior Generation1 Intel® Xeon® Scalable platform http://www.intel.com/performance Perform ance Agilit y Securi tyPervasive through compute, storage and network Pervasive data security with no performance overhead Rapid service delivery The Foundation of Data Center Innovation: Agile & Trusted Infrastructure
  9. 9. MariaDB OLTP Performance on Intel® Xeon® Scalable PLATFORM 9 0 175000 350000 525000 700000 Xeon E5-2697v3 Xeon E5-2699v4 Xeon Platinum 8180 NOPM 1.26X 1.34X 1.7X HammerDB test provides a complex OLTP workload with write/write locking contention to simulate real-world applications and compare performance across database platforms http://www.intel.com/performance MariaDB OLTP Performance on Intel® Xeon® Scalable PLATFORM MariaDB 10.2 InnoDB OLTP Workload
  10. 10. 10 0. 0.3 0.6 0.9 1.2 1.5 MariaDB Commercial $ 0 0.35 0.7 1.05 1.4 MariaDB Commercial $ 0. 0.25 0.5 0.75 1. MariaDB Commercial $ 32X 41X 42X • Software Total Cost of Ownership (TCO) over 3 years MariaDB and commercial database • ‘Cost per Transaction’ means cost for the same unit of work • MariaDB becomes more compelling with each CPU generation $0.038 $1.20 $1.23 $0.030 $0.92 $0.022 http://www.intel.com/performance Lower Cost Lower Cost Lower Cost MariaDB OLTP Cost on Intel® Xeon® Scalable Platform 2014 Cost per Transaction E5-2697v3 2016 Cost per Transaction E5-2699v4 2017 Cost per Transaction Platinum 8180
  11. 11. A Glimpse Inside the Intel® Xeon® Scalable platform SSD Intel® Optane™ SSD DC P4800X Complementary Intel® FPGA Workload optimized frameworks Integrated Options Fabric Intel® Omni-Path Architecture Networking Intel® Ethernet Accelerators Intel® QuickAssist Intel® AVX-512 Advancing virtually every aspect: Brand New core, cache, on-die interconnects, memory controller & more performa nce Agilitysecurity
  12. 12. 12 Combining the attributes of memory and storage with low latency, high endurance, outstanding QoS and high throughput+ Intel® optane™ SSD DC P4800X series is delivering on data center needs Enabling a new data tier to accelerate applications for fast caching & storage and expanded memory pools, to increase scale per server and reduce transaction cost Bringing synergy with Intel® Xeon® processors to enable bigger and more affordable datasets to gain new insights from larger memory pools
  13. 13. 13 MySQL* 5.7 Sysbench internal testing Memory DRAM 256GB DRAM 256GB DRAM 256GB CPU Intel® Xeon® v4 (44 core) Intel® Xeon™ Intel® Xeon® v4 (44 core) Intel® Xeon™ Intel® Xeon® v4 (44 core) Intel® Xeon™ Storage Intel® SSD DC S3520 Intel® SSD DC P4500 Intel® Optane™ SSD Transactions Per Second 17,235 5,834 3,535 TPS1 4x better up to 8x better up to Latency1 Intel® Optane™ ssd vs. “good” solution P99 Latency 12 ms 51 ms 100 ms 1.System configuration:Server Intel® Server System R2208WT2YS, 2x Intel® Xeon® E5 2699v4, 384 GB DDR4 DRAM, boot drive- 1x Intel® SSD DC S3710 Series (400 GB), database drives- 1x Intel® SSD DC P3700 Series (400 GB) and 1x Intel® Optane™ SSD DC P4800X Series (140 GB prototype),CentOS 7.2, MySQL Server 5.7.14, Sysbench 0.5 configuredfor 70/30 Read/WriteOLTP transactionsplit using a 100GB database.Cost per transactiondetermined by totalMSRP for each configurationdivided by the transactionsper second. Estimatedresultswereobtainedpriortoimplementationofrecentsoftwarepatchesandfirmwareupdatesintendedtoaddressexploitsreferredtoas"Spectre"and "Meltdown".Implementationoftheseupdatesmaymaketheseresultsinapplicabletoyourdeviceorsystem. *Other names and brands names may be claimed as thepropertyof others
  14. 14. 14 Xeon E5-2699v4 Xeon Platinum 8180 1.78X MariaDB*: Sysbench OLTP workload. OS: Ubuntu Server 17.04 x64. Sysbench 0.4.12, MariaDB 10.2.9 GIT snapshot 2017-09-12. Testing by Intel, September 2017. BASELINE: 2S Intel® Xeon® E5-2699 v4, 2.2GHz, 22 cores, turbo and HT on, BIOS 63.R00, 64GB total memory, 1600 MT/s / DDR4 LRDIMM, 1TB SATA HDD, Intel® SSD DC P3700 Series 2TB . NEW: 2S Intel® Xeon® Platinum 8180 processor, 2.5 GHz, 28 cores, turbo and HT on, BIOS 1.00.0412, 192GB total memory, 12 slots / 16GB / 2666 MT/s / DDR4 LRDIMM, 1 TB SATA HDD, Intel® SSD DC P3700 Series 2TB. http://www.intel.com/performance “MyRocks is a storage engine that adds the RocksDB LSM flash storage-optimized database to MariaDB. Even while under development, it shows good performance and scalability -- close to mature storage engines like InnoDB. Running our pre-release MariaDB 10.2.9 with a built-in MyRocks engine under a Sysbench RO multi-table test on the newest Intel® Xeon® Platinum 8180 processors, we saw up to 1.78x more throughput with a 1.18x reduction of average response time vs. previous-gen Intel® Xeon® processor E5-2699 v4 performance.” Michael Widenius, CTO, MariaDB Corporation Ab MYROCKS* NEXT-GENERATION STORAGE ENGINE ON Intel® Xeon® Scalable PlatformMariaDB 10.2.9 Sysbench OLTP Workload
  15. 15. 15 Enabling reliable, real-time web-scale applications through distributed log capabilities
  16. 16. Database Engine Shared Storage Clustering Software Oracle RAC Oracle Automatic Storage Management Oracle Clusterware* The Stack Hardware x86 servers interconnected via Infiniband* (or Ethernet) Oracle RAC* as the “Gold Standard”
  17. 17. 17 1.AWS Relational Database Service (RDS) 2.AWS Aurora (RDS) “Data Center Networks Are in My Way!” Using AWS’s RDS* vis-à-vis their RDS/Aurora* Log-Structured Storage as the example
  18. 18. 18 Matsunobu, "MyRocks: Space and write optimized OLTP database at Facebook," 2016 https://atscaleconference.com/videos/myrocks-space-and-write-optimized-oltp-database-at-facebook/ Facebook’s MyRocks* Comes Out from Behind the Curtain Major FEATURES in MyRocks Similar Feature Sets as InnoDB* TRANSAC TIONS ONLINE BACKUP• Atomicity • Non-locking consistent reads • Read committed • Repeatable read • Crash-safe slave and master • Logical backup by mysqldump • Binary backup by myrocks_hotbackup
  19. 19. 19 Customer “Pain Points” The Downside of Becoming a Data Company Companies that have pivoted to AI and analytics in order to monetize their data are experiencing accelerating data growth rates Disaggregating Storage This data growth challenge makes disaggregating storage from compute attractive because the company can scale their storage capacity to match their data growth, independent of compute. Cloud-Like, Open-Source, and AI/Analytics-Ready Many of these companies are big Oracle RAC*/Exadata* users. They expressed a desire to move off this platform to something more cloud-like, open source, and readily integrated into their AI/analytics investments. 1 2 3
  20. 20. 20 Physical Cluster ToR Switch Spine Switch JBOF Storage Servers Compute Servers 100GbsLinks 50GbsLinks Resource Mgmt Service Mgmt Services A Solution to Address These Pain Points CLUSTER SOFTWARE
  21. 21. 21 Revisiting Cluster Computing in the Era of Cloud Rack-Scale Design and Disaggregated Storage 1.Rack-Centric, Physical Cluster 2.Cluster-Wide Service Management (k8s) 3.Database-as-a- Service on Log- Structured Storage website1 website2 ELB Service (Load balancer) Pod ingress controller Ingress Service (Node Port) Service (Node Port) … Service (Node Port) Pod (website1) Pod (website2) Pod (default backend) Kubernetes
  22. 22. • Support for the persistence requirements of distributed databases • Fast recovery and cloning through an automated recovery process - Replicated log service • Little or no interruptions to the transaction processing - High-throughput and low-latency - Durability, replication and strong consistency - Optimizes both performance and TCO for transaction-centric workloads 22 Advantages of Log- Structured Storage
  23. 23. 23 A Stack Comparison A Stack Comparison Enterprise (Oracle) Cloud (Amazon Aurora*) Intel Solution (MariaDB*-based Solution) Database Engine Proprietary (Oracle RAC*) Open Source (MySQL, Postgres) Open Source MariaDB with the MaxScale Layer-7 Proxy Shared Storage Proprietary Oracle Automatic Storage Management (ASM) with POSIX File System interface Open Source +Proprietary (Distributed Log, S3) Open Source MyRocks* compiled with Rockset’s RocksDB-Cloud* library Clustering Software Proprietary (Oracle Clusterware*) Proprietary AWS RDS* Open Source Kubernetes* Hardware Proprietary x86 servers interconnected via Infiniband* (or Ethernet) Commodity x86 servers interconnected via Ethernet Fabric Commodity x86 servers interconnected via 100Gbs Ethernet Fabric
  24. 24. Flush sst file to local SSD writes memtable cache Persistent read cache on SSD Flush sst file to cloud storage Cloud Storage Bucket Cloud Application RocksDB-Cloud* block cache reads Queries Updates Cloud Log Storage, Kafka* Topic 24 Provisioning a Database Instance WAL
  25. 25. read write Cloned Server RocksDB-Cloud read Cloud Bucket A Server RocksDB-Cloud* Clones are read-only replicas 25 fast cloning Queries Updates Queries Cloud Bucket B write read Cloud Log Storage, Kafka* WAL writes WAL tailer
  26. 26. • Uses MyRocks w/Rockset’s RocksDB-Cloud library • Requires Support for Lossless-SemiSync Protocol and GTIDs in the MaxLog Binlog Server (“Log-Tailer”) • Need a “Lambda” function to kickoff “Switchover” • Need Automation for carrying out Switchover • Can use either InnoDB* or MyRocks* w/RocksDB* library • Requires Support for Lossless-SemiSync Protocol and GTIDs in the Slaves • Need a “Lambda” function to kickoff “Switchover” • Need Automation for carrying out Switchover 26 Time0 Time1 Time0 Time1 Database Instance within a Single Cluster1.Master-Slave (Shared-Nothing) 2.Master-Only (Shared)
  27. 27. 27 What’s Next? Enabling the use of Nonvolatile Memory (NVM)
  28. 28. 28 Improve real-time data analytics and data warehousing TCO with Intel® FPGAs
  29. 29. Intel® Big Data Analytics Frameworks Accelerate innovation in big data analytics with frameworks built on software-defined Infrastructure with open-standard building blocks. Intel® Frameworks & Libraries Integrated with FPGAs Run unmodified customer applications, use runtime orchestration with both Intel® Xeon® processor and FPGA support, and leverage end-to-end virtualization and security. Accelerate Relational, NoSQL and Unstructured FPGA data access, networking and algorithm acceleration options with a single FPGA for highly structured, semi- structured, and unstructured data for better TCO, flexibility and future proofing. Analytics Landscape and Scaling Accelerate Big Data Analytics with Existing Interfaces and FPGAs 1 2 3
  30. 30. Microsoft Scale-Out FPGA Multi-Function Accelerator • “Diversity of cloud workloads and … rapid … change” (weekly or monthly) – Search, SmartNIC, Machine Learning, Encrypt, Compress, Big Data Analytics… • Bing Search: 2X server level perf, 29% latency reduction • Networking Virtualization: 10X latency improvement • Machine Learning: Stratix 10 capable of 90 TFLOPs 8-bit floating-point Single FPGA Algorithm, Networking & Data Access Acceleration
  31. 31. 32 Data Network Streaming Data Integrate to Intel® Frameworks and APIs – Run unmodified customer applications – Orchestration run-time advantage: Xeon® or FPGA – End-to-end security and virtualization framework Moderate Acceleration is Common – PCIe* look-aside acceleration (two data copies) Significant Acceleration requires FPGA – Multifunction and inline with a single FPGA FPGAs Offer Unique Value for Analytics & Streaming Offloads algorithm, networking and data access processing Single Multifunction Accelerator
  32. 32. 34 Acceleration Overview • 20X+ single table inserts/s for real time data analytics - With modest tuning, 15M INSERT/s1 • 2X+ optimized queries for data warehousing - Using industry standard TPC-DS benchmark • 3X+ storage compression - Data and tables managed by Swarm64* Swarm64 Relational Database Acceleration Two Workloads: Traditional Data Warehousing, Real-Time Data Analytics Database acceleration with a plugin
  33. 33. 35 Swarm64 Relational Database Acceleration Scale-Up Data Warehousing, Real-Time Data Analytics, and Storage Compression Database acceleration with a plugin Overview • No customer application change - Storage engine plugin - Query Engine accelerates INSERT, SELECT, … • Optimized indexing • More I/O bandwidth, memory depth from compression Significant Acceleration √ Data access acceleration √ Compression, filtering, replication … √ Memory-mapped acceleration, data cache √ “Optimized Columns” indexing
  34. 34. 37 FPGACPU User Application and Libraries CPU FPGA Interface Manager (FIM) Intel® Acceleration Engine with OPAE1 Technology Accelerator Function (Developer created or provided by Intel) UPI2/PCIe* HSSI3 Hypervisor & OS Optimized and simplified hardware and software APIs provided by IntelOPAE FPGA Acceleration Environment Common Developer Interface for Intel® FPGA Data Center Products Accelerator Function Interfaces
  35. 35. 38 Start developing for Intel® FPGAs with OPAE today: http://github.com/OPAE • Consistent API across product generations and platforms - Abstraction for hardware-specific FPGA resource details • Designed for minimal software overhead and latency - Lightweight user-space library (libfpga) • Open ecosystem for industry and developer community - License: FPGA API (BSD), FPGA driver (GPLv2) • FPGA driver being upstreamed into Linux* kernel • Supports both virtual machines and bare metal platforms • Faster development and debugging of Accelerator Functions with the included AFU Simulation Environment (ASE)** • Includes guides, command-line utilities and sample code FPGA Hardware + Interface Manager FPGA Driver (Physical Function – PF) FPGA API (C) (Enumeration, Management, Access) Applications, Frameworks, Intel® Acceleration Libraries Bare Metal Virtual Machine FPGA Driver (Virtual Function - VF) Hypervisor FPGA Driver (Common – AFU, Local Memory) OS Open Programmable Acceleration Engine (OPAE) Technology Simplified FPGA Programming Model for Application Developers
  36. 36. • Data analytics acceleration with no change to application required - Relational, NoSQL and Spark*/Hadoop* • Single workload with FPGA multifunction acceleration • Swarm64* accelerates big-and-fast data or traditional data warehousing • Potential to accelerate proxy access with partner rENIAC • Intel® FPGA card with framework for interfaces, end-to-end security and virtualization - Production in 2Q18 SUMMARY

×