More Related Content
What's hot
What's hot (12)
Similar to openGauss - The evolution route of openGauss' AIcapabilities
Similar to openGauss - The evolution route of openGauss' AIcapabilities (20)
Recently uploaded
Recently uploaded (20)
openGauss - The evolution route of openGauss' AIcapabilities
- 1. The Evolution Route of openGauss' AI Capabilities Gauss Lab, Huawei Tianqing Wang (王天庆) wangtianqing2@huawei.com
- 3. 3 The Different Stages of AI combining with DB External AI Module Internal AI Module Embedding AI Module AI Kernel Servicing Full Scenario Intelligence
- 4. AI Framework for Database (overview) AI Framework AI Algorithms AI in SQL AI in DBMS AI in Kernel Full Stack AI/ML/DL Full Scenario SQL/DBMS/Kernel Analyst /DBA/Developer Deep Learning Platform Machine Learning Library Optimization Algorithms In-DB Analytics Knobs Tuning Learned Optimizer Workload Forecasting View/ Partition Advisor Workload Scheduling Index Advisor Self Diagnosis/ Healing Learned Index/Storage Self Assembler In-DB Optimization In-DB Operator Self Security Heterogeneous computing KunPeng X86 Ascend GPU DB4AI AI4DB Self Monitor
- 5. LOGO PART 2 AI Capability Introduction
- 6. AI in DBMS：Improve database performance and help operation and maintenance 2 Self-Diagnosis，Always Online 1 Self-Tuning 3 Self-Assembly Support diversified workloads • For rich industry applications, the existing product development model cannot meet customized delivery Componentized self-assembly • Database componentization: Easy to use search algorithms or rules customize according to business scenarios (e.g., append-only or in-place update storage engine) Self-Diagnosis • Diagnosis Engine：Trace-based intelligent diagnosis system that supports the combination of AI and rule engine Self-Healing • Try to repair: According to the self-diagnosis results, try to call the optimization toolkit to repair the fault • Fault warning：Predict software & hardware failures, and the failure detection time is less than 5s Intelligence Tuning on Huawei Cloud • Knobs Tuning：Reduce operating time and improve performance • Index Recommendation：Give advices by single SQL statement or workload Workload Forecasting on Huawei Cloud • Workload Forecasting：Accurately predict workload performance, support users to efficiently orchestrate services and intelligent workload scheduling Knobs Tuning Workload Forecasting Self-diagnosis architecture One-Stack-Fit-All architecture Ref: XuanYuan: An AI-Native Database
- 7. The overall architecture of AI in DBMS in cloud scenarios Control Server AI Server Directly operate the database to obtain instant information workload OS DB time-series metrics Offline Calculation Service Zone POD Zone Request Agent … KVM DB DB DB DB KVM DB DB DB DB JDBC protocol Automatic disk expansion mount /dev/sdb Storage Prediction Knobs Tuning SQL visualization SQL Diagnosis Index Recommendation Features: ① Intelligent: define the level of autonomy and provide self-service and intelligent services ② Emphasize the characteristics of "prediction": the ability to provide foresight ③ Mainly oriented to workload-level tuning: knob tuning, index recommendation and other functions focus on workload Resource quota management Intelligent Operation and Maintenance Platform … DBMind: A typical architecture of an intelligent management platform
- 8. AI in DBMS: Knob tuning and diagnosis Tuning knob list: preset according to different scenarios, users can also configure according to experience; Tuning method summary: combining deep reinforcement learning (DDPG) and global optimization algorithms (PSO, Bayes optimization), fine-grained tuning for different types of knobs. Tuning effect evaluation: Observe the benchmark results, the preset benchmarks are rich, minimalist and extensible. Ref: QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning Diagnosis and Recommendation Result
- 9. • The execution time of a certain SQL statement can be predicted by the cost value given by the database optimizer. • Using deep learning (Tree-LSTM) models to predict the execution time of SQL statements can often achieve better results than linear models. • For more information, please refer to our published papers. TPC-H JOB TPC-DS Cost TBCNN R-LSTM Cost TBCNN R-LSTM Cost TBCNN R-LSTM Training time (s) 590 79 1990 425 31204.5 477 Avg accuracy 5.90E+06 1.9(1.8) 1.4(1.3) 1.20E+064.2(4.0) 10.6(4.1) 1.30E+076.9(4.3) 28.9(7.8) <10 acc 74% 99% 99% 62% 93% 91% 53% 74% 74% <2 acc 42% 77% 91% 43% 53% 66% 12% 44% 29% TPC-H JOB TPC-DS TPC-H JOB TPC-DS Data Collection Feature Extraction Training/ Prediction Multi-task learning cost rows AI in Kernel：query performance prediction based on online learning Experimental results: Ref: An End-to-End Learning-based Cost Estimator Query Performance Prediction for Concurrent Queries using Graph Embe Learned Cardinality Estimation for Similarity Queries.
- 10. THANKS！ 谢 谢 观 看 扫 码 添 加 微 信 公 众 号 ， 微 信 小 助 手 THANKS！ 谢 谢 观 看 10