Presenter: David Wang
Alibaba Cloud Product Specialist
Leveraging ApsaraDB to
Deploy Business Data
on the Cloud
Outline
ApsaraDB Introduction Countering Challenges Functional Solutions Typical Customers
Introduction to ApsaraDB
Business
Applications
Redis
Memcache
MySQL
SQL Server
PostgreSQL
MongoDB
PPAS (Oracle)
POLARDB
Hbase
HybridDB for MySQL(PetaData)
HybridDB for PG(GreenPlum)
Product Relationships
Cache Fast Storage Long-term Analysis Big
Covers over 70% of key online database engines
Product Relationships
ApsaraDB
RDS
MySQL
SQL Server
PostgreSQL
PPAS(Oracle)
NoSQL
Redis
MongoDB
Memcahce
Data
Warehousing
HBase
HTAP
ADS
Tools
DTS
DMS
HybridDB for MySQL(PetaData)
HybridDB for PostgreSQL(GreenPlum)
Product Features
High
Availability
SSDs used as storage, verifiable backup
restorability at any time
The first RCPIS Grade III
cloud database in China
Master-Slave backup nodes & disaster tolerance
solutions to ensure business continuity
Automated management & monitoring
Top database expert team to provide
customer support
Raft algorithm guaranteed consistency
Free migration of full or incremental
data without downtime
ApsaraDB Introduction Countering Challenges Functional Solutions Typical Customers
ApsaraDB Usage
Poor Database
Performance Weak O&M Platform
Sensitive Data
Difficulties Recruiting
DBAs
Current Issues
Poor Database Performance
Performance & Capacity
- 60 CPUs + 470 GB memory
(exclusive physical machines)
- 2TB SSD+50000 IOPS
- 100,000–150,000 QPS
- No-downtime configuration changes
Source Code Improvements
- MySQLPostgreSQLRedisMongoDB
- Locks, transaction optimization, and
optimized master/slave syncing to improve
performance 3–5 times over
Sensitive Data
Security Features
VPC: Isolated network environments
IP Whitelist: Controlled client sources
SSL: Encrypted network traffic
TDE: Transparent data encryption
SQL audit tracking: Detailed access records
Weak O&M Platform
Automated O&M
• Fault self-recovery
• Backup self-verification
• High-frequency monitoring
• Webpage operations
• Comprehensive analysis for resources, SQL databases,
and engines
- CPU resources
- Memory resources
- Storage resources
- Connection count IOPS
- Statement consumption
summary and ranking
- Execution plan analysis
- Table structure analysis
- Database lock analysis
- Transaction optimization analysis
- Deadlock rollback
Difficulties Recruiting DBAs
Expert Services
• Implementation of data migration to the cloud
• Database emergency support
• Database health diagnostics
• Business escort service for high-traffic events
• Customized open-source database source code
]
Invoice
Invoice_ID
Price
Tax
Date
Due Date
Total
]
]
]
]
]
Product
Product_ID
Material_ID
Type
Availability
Stock
Subcontractor_ID
Subcontractor
Subcontractor_ID
Name
Address
Postal Code
EmailMaterial
Material_ID
Material_Type
Availability
Stock
Subcontractor
Order
Order_ID
Order_Type
Product_Type
Product_Location
Product_ID
Event
Event_ID
Location
Date
Address_ID
ApsaraDB Introduction Countering Challenges Functional Solutions Typical Customers
Functions List
Function ApsaraDB ECS User-created Database
Service availability
Master/Backup architecture ✔️ Hot standby ✖️
Local disaster tolerance ✔️ Multiple zones ✖️
Remote disaster tolerance ✔️ DR instances ✖️
Fault drills ✔️ Master/Backup switchover ✖️
Data reliability
Online storage redundancy ✔️ Local RAID ✔️ Ultra/SSD cloud disks
Offline long-term backup ✔️ Up to 730 days ✖️
Time point recovery ✔️ Instance clone ✖️
Data replication ✔️ Asynchronous/Semisynchronous ✖️
Data security
Network isolation ✔️ White list groups ✔️ ECS security groups
Audit logs ✔️ SQL audit/keyword filtering ✖️
Network encryption ✔️ SSL ✖️
Encrypted storage ✔️ TDE ✖️
Monitoring and alarms
Resource monitoring ✔️ CPU/MEM/DISK/IOPS ✔️
Engine monitoring ✔️ QPS/TPS/etc ✖️
Seconds-level monitoring ✔️ 300s/60s ✔️ 300s
Cloud Monitor alarms ✔️ Resource alarms/Availability alarms ✖️
Parameter management
Parameter templates ✔️ Almost 100 optional parameters ✖️
Change history ✔️ Change record tracing ✖️
Performance optimization
Overhead cost analysis ✔️ SQL consumption statistics ✖️
Optimization recommendations ✔️ Missing index analysis ✖️
All-in-one service
Data management ✔️ DMS visual user interface ✖️
Data synchronization ✔️ DTS online data migration/subscription ✖️
Disaster Tolerance Solution
East China
Zone A
Intranet
• Disaster tolerance within a data center: Two nodes in different racks
• Dual-data center local disaster tolerance: Log latency no longer than 3ms. Select "Multi-zone" during instance activation.
• Coming soon: Dual-data center, tri-copy local disaster tolerance
• Remote disaster tolerance: Log transmission delay of shorter than 1 second between two data centers located 300 km to
1000 km apart
Master
South China
Zone A
Leased
Line
M S
Master
Zone B
Provides 3 levels of high-availability (disaster tolerance) architectures for selection
Read/Write Splitting
Provides single-point "read scaling" capabilities
• 10 read-only nodes
• Independent connection address
• Adjustable configurations
Data Sync in
MillisecondsRead-only Requests
Loading
Read/Write
Service
Heterogeneous Data Storage
• Provides “hotspot data” capabilities
• Distributed memory cluster
• Up to 512 GB for a single instance
• Simple protocols (MemCache/Redis)
Hotspot Data
Persistent Data
Access
Common
Transactions
Data Storage Scaling
(For Relational Databases)
Distributed, low cost,
PB-level storage
• Designed for IoT data storage and log
data storage scenarios
• Provides 3–5 years' capacity planning
capabilities to enterprises
• 7–10x compression capabilities
(100 GB –> 15 GB)
• Improves Insert performance by 10x
• Over 70% compatible with MySQL
Application
Adaptation Costs HighNone
• 60 Core, 470 GB mem
• 2TB, 20000 IOPS
• Configuration change as desired
• 10 read-only nodes
• Independent connection address
• Adjustable configurations
• PetaData
• Simultaneous read/write scaling
• Data volumes from 8 TB to 1 PB
PolarDB, 100% compatible with MySQL, 100 TB
Data Storage Scaling
• Built-in distributed clusters
• Application integration with 0
code modification
• 512 GB max storage for a
single Redis instance
Memcache Redis MongoDB
• MongoDB Sharding, unlimited
memory and storage
(For NoSQL Databases)
Big Data Computing
Access to the Data Technology (DT) age:
renewed focus on how to make good use of d
ata
BI & Analytics
IoT & Real-time
Analytics
In-
house
Cloud Backup
Leased Line Access
ApsaraDB Introduction Countering Challenges Functional Solutions Typical Customers
Typical Customers
After Alipay launched its Yu'E Bao business,
Tianhong's accounting computation needs
surged.
Using RDS cluster instances for parallel
computing, the company's revenue accounting
time fell from 24 hours to 4 hours.
Alipay
Tianhong
Securities Yu’er Bao
Typical Customers
Chai Jing's documentary "Under the Dome"
released a pollution map app
The surge in users and traffic caused the app's
offline database to crash.
Using Alibaba Cloud’s Expert
Service, the database was
migrated to a read-only RDS
instance.
There, it was able to withstand
dozens of times more access
pressure.
Community Organization:
Pollution Map
Learn More About ApsaraDB for RDS
www.alibabacloud.com/tc (Chinese)
www.alibabacloud.com (English)
Leveraging ApsaraDB to Deploy Business Data on the Cloud

Leveraging ApsaraDB to Deploy Business Data on the Cloud

  • 1.
    Presenter: David Wang AlibabaCloud Product Specialist Leveraging ApsaraDB to Deploy Business Data on the Cloud
  • 2.
    Outline ApsaraDB Introduction CounteringChallenges Functional Solutions Typical Customers Introduction to ApsaraDB
  • 3.
    Business Applications Redis Memcache MySQL SQL Server PostgreSQL MongoDB PPAS (Oracle) POLARDB Hbase HybridDBfor MySQL(PetaData) HybridDB for PG(GreenPlum) Product Relationships Cache Fast Storage Long-term Analysis Big Covers over 70% of key online database engines
  • 4.
  • 5.
    Product Features High Availability SSDs usedas storage, verifiable backup restorability at any time The first RCPIS Grade III cloud database in China Master-Slave backup nodes & disaster tolerance solutions to ensure business continuity Automated management & monitoring Top database expert team to provide customer support Raft algorithm guaranteed consistency Free migration of full or incremental data without downtime
  • 6.
    ApsaraDB Introduction CounteringChallenges Functional Solutions Typical Customers
  • 7.
    ApsaraDB Usage Poor Database PerformanceWeak O&M Platform Sensitive Data Difficulties Recruiting DBAs Current Issues
  • 8.
    Poor Database Performance Performance& Capacity - 60 CPUs + 470 GB memory (exclusive physical machines) - 2TB SSD+50000 IOPS - 100,000–150,000 QPS - No-downtime configuration changes Source Code Improvements - MySQLPostgreSQLRedisMongoDB - Locks, transaction optimization, and optimized master/slave syncing to improve performance 3–5 times over
  • 9.
    Sensitive Data Security Features VPC:Isolated network environments IP Whitelist: Controlled client sources SSL: Encrypted network traffic TDE: Transparent data encryption SQL audit tracking: Detailed access records
  • 10.
    Weak O&M Platform AutomatedO&M • Fault self-recovery • Backup self-verification • High-frequency monitoring • Webpage operations • Comprehensive analysis for resources, SQL databases, and engines - CPU resources - Memory resources - Storage resources - Connection count IOPS - Statement consumption summary and ranking - Execution plan analysis - Table structure analysis - Database lock analysis - Transaction optimization analysis - Deadlock rollback
  • 11.
    Difficulties Recruiting DBAs ExpertServices • Implementation of data migration to the cloud • Database emergency support • Database health diagnostics • Business escort service for high-traffic events • Customized open-source database source code ] Invoice Invoice_ID Price Tax Date Due Date Total ] ] ] ] ] Product Product_ID Material_ID Type Availability Stock Subcontractor_ID Subcontractor Subcontractor_ID Name Address Postal Code EmailMaterial Material_ID Material_Type Availability Stock Subcontractor Order Order_ID Order_Type Product_Type Product_Location Product_ID Event Event_ID Location Date Address_ID
  • 12.
    ApsaraDB Introduction CounteringChallenges Functional Solutions Typical Customers
  • 13.
    Functions List Function ApsaraDBECS User-created Database Service availability Master/Backup architecture ✔️ Hot standby ✖️ Local disaster tolerance ✔️ Multiple zones ✖️ Remote disaster tolerance ✔️ DR instances ✖️ Fault drills ✔️ Master/Backup switchover ✖️ Data reliability Online storage redundancy ✔️ Local RAID ✔️ Ultra/SSD cloud disks Offline long-term backup ✔️ Up to 730 days ✖️ Time point recovery ✔️ Instance clone ✖️ Data replication ✔️ Asynchronous/Semisynchronous ✖️ Data security Network isolation ✔️ White list groups ✔️ ECS security groups Audit logs ✔️ SQL audit/keyword filtering ✖️ Network encryption ✔️ SSL ✖️ Encrypted storage ✔️ TDE ✖️ Monitoring and alarms Resource monitoring ✔️ CPU/MEM/DISK/IOPS ✔️ Engine monitoring ✔️ QPS/TPS/etc ✖️ Seconds-level monitoring ✔️ 300s/60s ✔️ 300s Cloud Monitor alarms ✔️ Resource alarms/Availability alarms ✖️ Parameter management Parameter templates ✔️ Almost 100 optional parameters ✖️ Change history ✔️ Change record tracing ✖️ Performance optimization Overhead cost analysis ✔️ SQL consumption statistics ✖️ Optimization recommendations ✔️ Missing index analysis ✖️ All-in-one service Data management ✔️ DMS visual user interface ✖️ Data synchronization ✔️ DTS online data migration/subscription ✖️
  • 14.
    Disaster Tolerance Solution EastChina Zone A Intranet • Disaster tolerance within a data center: Two nodes in different racks • Dual-data center local disaster tolerance: Log latency no longer than 3ms. Select "Multi-zone" during instance activation. • Coming soon: Dual-data center, tri-copy local disaster tolerance • Remote disaster tolerance: Log transmission delay of shorter than 1 second between two data centers located 300 km to 1000 km apart Master South China Zone A Leased Line M S Master Zone B Provides 3 levels of high-availability (disaster tolerance) architectures for selection
  • 15.
    Read/Write Splitting Provides single-point"read scaling" capabilities • 10 read-only nodes • Independent connection address • Adjustable configurations Data Sync in MillisecondsRead-only Requests Loading Read/Write Service
  • 16.
    Heterogeneous Data Storage •Provides “hotspot data” capabilities • Distributed memory cluster • Up to 512 GB for a single instance • Simple protocols (MemCache/Redis) Hotspot Data Persistent Data Access Common Transactions
  • 17.
    Data Storage Scaling (ForRelational Databases) Distributed, low cost, PB-level storage • Designed for IoT data storage and log data storage scenarios • Provides 3–5 years' capacity planning capabilities to enterprises • 7–10x compression capabilities (100 GB –> 15 GB) • Improves Insert performance by 10x • Over 70% compatible with MySQL Application Adaptation Costs HighNone • 60 Core, 470 GB mem • 2TB, 20000 IOPS • Configuration change as desired • 10 read-only nodes • Independent connection address • Adjustable configurations • PetaData • Simultaneous read/write scaling • Data volumes from 8 TB to 1 PB PolarDB, 100% compatible with MySQL, 100 TB
  • 18.
    Data Storage Scaling •Built-in distributed clusters • Application integration with 0 code modification • 512 GB max storage for a single Redis instance Memcache Redis MongoDB • MongoDB Sharding, unlimited memory and storage (For NoSQL Databases)
  • 19.
    Big Data Computing Accessto the Data Technology (DT) age: renewed focus on how to make good use of d ata BI & Analytics IoT & Real-time Analytics In- house Cloud Backup Leased Line Access
  • 20.
    ApsaraDB Introduction CounteringChallenges Functional Solutions Typical Customers
  • 21.
    Typical Customers After Alipaylaunched its Yu'E Bao business, Tianhong's accounting computation needs surged. Using RDS cluster instances for parallel computing, the company's revenue accounting time fell from 24 hours to 4 hours. Alipay Tianhong Securities Yu’er Bao
  • 22.
    Typical Customers Chai Jing'sdocumentary "Under the Dome" released a pollution map app The surge in users and traffic caused the app's offline database to crash. Using Alibaba Cloud’s Expert Service, the database was migrated to a read-only RDS instance. There, it was able to withstand dozens of times more access pressure. Community Organization: Pollution Map
  • 23.
    Learn More AboutApsaraDB for RDS www.alibabacloud.com/tc (Chinese) www.alibabacloud.com (English)

Editor's Notes

  • #13 X86 based&Heterogeneous Network 10 GE -> 25 GE CPU from boardwell -> skylark Deep dive on the different instance families will come soon, on performance, and target business scenarios X-Dragon
  • #21 X86 based&Heterogeneous Network 10 GE -> 25 GE CPU from boardwell -> skylark Deep dive on the different instance families will come soon, on performance, and target business scenarios X-Dragon