Clous

1. Cloud Data Management
20ITS01

2. Cloud Data Fundamentals
• Online Transaction Processing RDS and DRDS – POLARDB - NoSQL –
Redis - OLAP – Analytic DB - Utilities – DTS -MySQL Database
deployment – Database High Availability - database backup and
recovery - database monitoring and maintenance.

3. Unit Fundamental
• Data is the backbone of modern applications (Efficient storage,
retrieval, and processing are essential for businesses)
• Ensures system reliability, security, and performance (OLTP)
• Supports scalability & real-time analytics (NoSQL, OLAP, and
AnalyticDB handle growing data needs for decision-making.)
• Enables disaster recovery & smooth data migration (Backup, recovery,
and DTS ensure data protection)

4. Technologies
• OLTP (Online Transaction Processing) – Handles real-time
transactions like banking, e-commerce, and ticket booking.
• NoSQL (Not Only SQL) – A flexible, scalable database for handling
unstructured or semi-structured data (e.g., MongoDB, Redis).
• OLAP (Online Analytical Processing) – Used for complex data analysis
and business intelligence (e.g., sales trends, financial reports).
• AnalyticDB – A cloud-based OLAP database for high-speed big data
analysis.
• DTS (Data Transmission Service) – A tool for database migration,
synchronization, and disaster recovery.

5. Online Transaction Processing (OLTP)
• OLTP (Online Transaction Processing) is a system that handles real-
time, high-volume transactions.
• Data is updated continuously and transactions are processed instantly
such as order processing, payments, or inventory updates.
• These systems are designed to quickly process many small
transactions at once, ensuring that data is accurate and consistent.

6. Example
• You select the T-shirt and press 'Buy Now'.
• The system checks if it's available in the warehouse.
• Your payment is processed through a secure payment system.
• The stock is updated to show that the T-shirt has been bought.

7. Key Features of OLTP
• Real-time processing: Every transaction is processed as soon as it
happens (e.g., you buy something and the stock is updated instantly).
• Short transactions: The system handles many small transactions
quickly (like adding items to your shopping cart).
• Data consistency: OLTP systems ensure that the data is accurate at all
times (e.g., you can't buy something if it's out of stock).
• Frequent updates: OLTP systems often involve updating and querying
databases (e.g., checking your bank balance).

8. Examples
• Banking systems: When you transfer money, the transaction happens
immediately.
• Online shopping: When you place an order, the system checks
availability, processes payment, and updates stock.
• Airline ticket booking: When you book a flight, the seats are updated
in real-time.

9. RDS (Relational Database Service)
• It’s a service provided by companies like AWS or Alibaba Cloud that
helps store, manage, and organize your data without you having to do
everything manually.
• It takes care of things like backups, updates, and security so you
don’t have to worry about them.
• It’s like having a super-efficient assistant who helps you manage all
your data easily.
• It’s simple and effective for managing the data of small-to-medium-
sized online stores. So, no need to worry about database
management since RDS handles it.

10. Example
• Small online store or a simple e-commerce platformuses RDS to keep
track of things like:
• Product details (name, price, description).
• Customer information (name, email, shipping address).
• Order history (what the customer bought, when they bought it).

11. DRDS (Distributed Relational Database Service)
DRDS is like Amazon's massive platform, where data is too large to fit
into a single database.
Imagine millions of customers shopping on Amazon at the same time.
The product catalog, customer orders, and reviews need to be split
across multiple databases for faster access and to handle heavy traffic.
Key Features
• Scalability: Handles large-scale applications by distributing the load.
• High Availability: Data is replicated and always available.
• Improved Performance: Allows handling millions of users at the same
time.

12. Why Use RDS and DRDS?
RDS: Perfect for small to medium-sized applications that don’t require
massive amounts of data.
DRDS: Ideal for large-scale applications (like e-commerce platforms or
social media apps) where there is huge data that needs to be split into
different parts for better management and performance.

13. POLARDB
POLARDB is a Cloud-Native database that combines high performance,
high availability, and scalability for both transactional and analytical
workloads.
Key Features:
• Fully managed by Alibaba Cloud.
• Supports MySQL, PostgreSQL, and Oracle compatibility.
• Designed for modern cloud applications.

14. How Does POLARDB Work
• Architecture:
• POLARDB uses a shared storage architecture, which means that multiple
database instances can access the same storage layer.
• Storage Layer: The data is stored in a cloud-based, highly scalable storage system.
• Compute Layer: This layer is responsible for handling queries and processing
data. You can have multiple compute nodes.
• Example:
• Think of POLARDB like a book library:
• The books (data) are stored in a centralized library storage.
• Multiple librarians (compute nodes) work together to serve the readers (users), allowing
faster access to books (data).

15. Features of POLARDB
• High Availability:
• POLARDB offers multi-AZ deployment (across different availability zones) to ensure
that the database stays online even during failures.
• Elastic Scalability:
• Easily scale your database's compute and storage independently based on the
demand.
• Compatibility:
• Supports MySQL, PostgreSQL, and Oracle databases.
• Allows for easy migration from other cloud databases (like RDS).
• Backup & Recovery:
• Automatic backups are taken and can be restored with point-in-time recovery.

16. Use Cases of POLARDB
• E-Commerce:
• POLARDB can handle large-scale transactional workloads such as online orders
and payments.
• Gaming:
• For games that require quick access to large amounts of data (like user profiles,
scores, and game history).
• Financial Applications:
• For handling complex transactions, with the ability to scale as the business grows.
• Web Applications:
• POLARDB is well-suited for web apps that need to serve a large number of users
simultaneously, such as social media platforms or streaming services.

17. POLARDB vs Traditional Databases
Feature POLARDB Traditional Databases
Cloud-Native Yes No
Scalability Elastic (scale up/down easily) Limited (requires hardware upgrades)
Cost Pay-as-you-go Fixed infrastructure costs
High Availability Multi-AZ support Requires manual setup
Performance Distributed architecture (fast) Limited by server capacity

18. Real-Time Example (Amazon)
• Amazon uses POLARDB for its backend data management to handle millions
of orders, customer details, and product catalogs.
• POLARDB's elastic scalability allows Amazon to increase storage and compute
resources based on high demand during sales events like Black Friday or
Prime Day.

19. POLARDB - NoSQL

20. PolarDB
• PolarDB is a powerful and modern database service developed by Alibaba Cloud.
• It’s built to handle large amounts of data efficiently and flexibly, especially for
cloud-based applications.
• Think of it as a "next-gen" database that is fast, reliable, and able to grow easily as
your data needs increase.
• Imagine you’re running a large online store with millions of customers. You need a
database that can handle all the transactions, product data, and customer
information without slowing down, especially during busy times (like big sales
events).
• PolarDB is made for this kind of workload.

21. Key Features of PolarDB
1.Separation of Computing and Storage:
• In most traditional databases, storage and computing (processing power) are
tightly linked together.
• When you need more storage, you also need to upgrade the computing power.
• In PolarDB: These two are separate, so you can easily add more storage or
computing power without impacting the other.
• This makes it scalable and flexible.
Example: Imagine you have an online store, and during the holiday season, the
number of customers browsing increases. With PolarDB, you can quickly add more
computing power to handle the traffic while keeping your storage needs separate.

22. 2.Large Storage Capacity:
• PolarDB can store huge amounts of data—up to 500 TB.
• Example: Let’s say your business starts with 10GB of data, but over the years,
you collect lots of customer data, transactions, product info, etc.
• PolarDB can grow with you, allowing you to store petabytes of data without
needing to worry about running out of space.
3.Cost-Effective:
• With PolarDB, you’re only charged for the computing power you use (when you
add extra nodes), not for the storage you use.
• In traditional databases, you pay for both computing and storage.

23. 4.Elastic Scaling (Fast Scaling):
• What it means: You can scale up (add more computing resources) or scale
down (remove resources) in just a few minutes, without causing any downtime or
interruptions.
• Example: During a huge sale, you can quickly scale your database to handle
thousands of transactions per second. After the sale ends, you can scale back
down to save costs.
5.Read Consistency:
• What it means: PolarDB ensures that the data you read is up-to-date, even when
there are multiple copies (replicas) of the database.
• Example: Imagine you’re checking the stock availability of a product. PolarDB
ensures that no matter which server (replica) you access, you always see the most
recent information.

24. 6.Millisecond-level Latency:
• Data changes made on the primary node are reflected almost instantly on other
copies of the database.
• This helps to avoid delays, even when you’re making changes to large tables (e.g.,
adding new fields or indexes).
Example: When you update the price of an item, that change is instantly visible to
customers browsing the website, ensuring consistency in real-time.
7.Data Backup in Seconds:
• PolarDB can back up large amounts of data (even in terabytes) very quickly
without locking the database, ensuring minimal impact on performance.
Example: If you’re running a financial application, PolarDB can back up your
entire database without interrupting your users or slowing down the service.

25. Architecture of PolarDB for MySQL and PolarDB for PostgreSQL

26. PolarDB Components
PolarProxy:
• PolarProxy is a layer between your application and the database. It directs requests from the
application to the correct database node.
• Example: If your application asks for product data, PolarProxy decides which database node
should handle the request (whether it’s the main node or a read-only copy).
Compute Nodes:
• These are the servers that handle the processing (computing) of data. There’s one primary node
(where data is written) and several read-only nodes (which handle the reading of data).
• Example: When a customer places an order, the primary node records the transaction. When
customers browse products, the read-only nodes serve those requests.
Shared Storage:
• All nodes in a cluster share the same storage, which means they can access the same data at the
same time.
• Example: Whether it’s the main server or a backup server, every node can access the same data,
ensuring consistency across the system.

27. Three Types of PolarDB Engines:
• PolarDB for MySQL – 100% compatible with MySQL. It’s perfect for MySQL
users who want to scale and improve performance.
• PolarDB for PostgreSQL – 100% compatible with PostgreSQL, and highly
compatible with Oracle databases.
• PolarDB-X – A more advanced version for very large applications that need to
handle huge amounts of data and very high traffic.

28. SQL vs NoSQL
• SQL (Structured Query Language) databases are relational and store data in
tables (rows and columns).
• They use predefined schemas and support complex queries.
Example:
• Banking transactions (money transfer, account balance tracking).
• E-commerce orders (product inventory, order history).

29. • ApsaraDB for RDS (Relational Database Service) → Supports MySQL,
PostgreSQL, SQL Server.
• PolarDB → A next-generation cloud-native relational database designed for
high performance, scalability, and compatibility with MySQL, PostgreSQL, and
Oracle.
• Example Use Case
A banking system needs to ensure every transaction is 100% accurate (e.g.,
transferring 500 from Account A to Account B).
₹
• This requires ACID compliance, which SQL databases provide.

30. NoSQL
• NoSQL (Not Only SQL) databases are non-relational, meaning they store
unstructured or semi-structured data.
• They support flexible schemas and are designed for high-speed performance.
Example:
• Social media posts (Twitter, Facebook).
• IoT sensor data (Smart Home, Weather monitoring).
Alibaba Cloud NoSQL Solutions:
• ApsaraDB for MongoDB → A document-based NoSQL database (stores JSON
data).
ApsaraDB for Redis → A key-value NoSQL database (fast data retrieval).
Table Store (Lindorm) → A column-based NoSQL database (big data processing).

31. Example Use Case:
• A social media app needs to store millions of tweets/posts every second, and
users should see new posts instantly.
• A SQL database would be too slow and difficult to scale.
• Instead, MongoDB (NoSQL) stores posts as documents (JSON format), making
retrieval fast and scalable.

32. Choosing the Right Database on Alibaba Cloud
Application Type Best Choice
Banking & Finance (Strict Accuracy) ApsaraDB for RDS (SQL)
Social Media, IoT, Gaming (Scalable & Fast) ApsaraDB for MongoDB (NoSQL)
Real-time Caching (Fast Reads/Writes) ApsaraDB for Redis (NoSQL)
E-commerce (Mix of Transactions & Unstructured
Data)
PolarDB + MongoDB

33. SQL or NoSQL on Alibaba Cloud
• Use SQL when: You need structured data, strict consistency, and complex
queries (e.g., Banking, Transactions).
• Use NoSQL when: You need scalability, flexibility, and fast performance (e.g.,
Social Media, IoT, Big Data).

34. Redis
• Redis is a super-fast memory-based database.
• It stores data in RAM (not on disk), so retrieving data is lightning fast.
• Imagine you are using Flipkart or Amazon and searching for a mobile phone.
• Without Redis: Every time you search, the website queries the database, making it
slow.
• With Redis: Product details are cached (stored temporarily in Redis), so the page
loads instantly!
• Think of Redis as a super-fast notebook where frequently used data is stored for
quick access!

35. OLAP
• OLAP (Online Analytical Processing) is used for analyzing large amounts of data.
• It helps in decision-making, reporting, and business intelligence (BI).
• Used for analytics and reports (e.g., sales trends, customer behavior).
• Big companies use OLAP to analyze millions of records quickly.
• Alibaba Cloud Solution – Alibaba provides AnalyticDB, a cloud-based OLAP
solution.

36. Real-Time Example (How Amazon Uses OLAP)
• Real-Time Example (How Amazon Uses OLAP)
• Amazon analyzes past 6 months' sales data to understand:
• Which products sold the most?
• What time of the year had the highest sales?
• Which city had the most orders?
• OLAP helps Amazon make better business decisions by analyzing
huge datasets.

37. Data Analytics
Data analytics is the process of examining raw data to find trends, patterns, and
useful information. It helps businesses make informed decisions.
Example: Netflix analyzes your watch history to recommend shows you might
like.
Real-Time Data Analytics
• Real-time analytics means processing and analyzing data immediately as it is
generated.
• It is used when quick decision-making is important.
Example: Google Maps live traffic updates – It constantly tracks vehicles and
updates routes instantly.

38. Traditional Databases vs. Cloud Databases

39. Example – E-Commerce Sales Tracking
• Amazon or Flipkart during a Big Sale (Diwali Sale, Black Friday, etc.).
Problems Faced by Flipkart:
• Millions of people are buying products at the same time.
• They need to track sales instantly (Example: How many iPhones are sold
every second?).
• Suggest products based on what customers are buying.
• Handle massive amounts of transactions without slowing down.

40. AnalyticDB
• AnalyticDB is a cloud-based, real-time data warehouse developed by
Alibaba Cloud.
• It is designed for fast data processing and analytics, making it useful for big
data applications, financial transactions, and business intelligence.
Why is AnalyticDB Needed?
• Traditional databases are slow for large datasets.
• Businesses need real-time analytics for decision-making.
• Scalability is crucial as data grows over time.

41. Key Features of AnalyticDB
• Real-Time Data Processing – Handles petabytes of data instantly.
• High-Speed Queries – Faster than traditional databases.
• Supports SQL – Works like MySQL and PostgreSQL.
• Distributed Architecture – Uses multiple servers for better performance.
• Auto Scaling – Increases or decreases resources based on usage.

42. SQL Query in AnalyticDB
Flipkart wants to track the top 5 best-selling mobile brands in real
time.
SQL Query in AnalyticDB:

43. Utilities in AnalyticDB
AnalyticDB provides several built-in utilities to help manage data efficiently.
These utilities assist in:
• Monitoring performance
• Managing queries
• Importing/exporting data
• Optimizing storage

44. DTS (Data Transmission Service)
• DTS (Data Transmission Service) is a cloud-based service in Alibaba Cloud
that helps to migrate, synchronize, and replicate data between different
databases in real-time.
• Migrate data from one database to another without downtime (e.g., MySQL →
AnalyticDB).
• Sync data in real-time between two databases (e.g., keeping a backup database
updated).
• Stream changes from a database to an application (e.g., real-time stock price
updates).

45. How DTS Works
DTS works by connecting a source database to a destination database and
continuously transferring data between them.
DTS Components:
• Source Database – Where the data comes from.
• Destination Database – Where the data is copied or synced to.
• DTS Engine – The system that processes and moves the data.
DTS Supports:
• Migration – Moving data permanently from one database to another.
• Synchronization – Keeping two databases updated in real time.
• Subscription – Streaming database changes for analytics or applications.

46. Real-Life Examples of DTS Use Cases
Example 2: E-commerce Sales Data Sync
• Problem: A company has an order database in MySQL and needs real-time
analytics.
• Solution: DTS syncs MySQL orders → AnalyticDB for live reports.
Example 2: Cloud Migration for Banking System
• Problem: A bank wants to move customer records from on-premise SQL
Server to Alibaba Cloud without downtime.
• Solution: DTS migrates SQL Server → MySQL on Alibaba Cloud while
keeping services running.

47. Advantages
• Fast & Real-Time – Keeps databases updated without delay.
• Cross-Platform – Works with MySQL, PostgreSQL, SQL Server, etc.
• Secure – Uses encryption for data transfer.

48. MySQL Database Deployment in Alibaba Cloud
• MySQL deployment means setting up and running a MySQL database on a server to
store and manage data.
• It can be deployed on-premise (your own hardware) or on the cloud (Alibaba Cloud,
AWS, etc.)
Why Deploy MySQL on Alibaba Cloud?
• No Need to Manage Hardware – Alibaba Cloud handles security, backups,
scaling.
• High Availability – MySQL runs without downtime using Alibaba Cloud’s
infrastructure.
• Auto Scaling – The database can expand automatically based on traffic.
• Security – Built-in firewalls, encryption, and monitoring.

49. Database High Availability (HA) in Alibaba Cloud
• High Availability (HA) means that a database remains accessible and
operational even if a failure occurs.
• HA databases automatically recover from failures with minimal downtime.
Example:
• A banking application must be available 24/7.
• If the primary database server fails, another server takes over immediately to
prevent downtime.

50. Why is High Availability Important?
• Prevents Downtime: Ensures continuous business operations.
• Data Safety: Protects against data loss due to failures.
• Better Performance: Distributes workload across multiple servers.
• Disaster Recovery: Automatically switches to a backup server.

51. How Alibaba Cloud Ensures High Availability for
Databases?
Alibaba Cloud provides three key solutions for database HA:
ApsaraDB for MySQL High-Availability Edition
1.Primary-Standby Architecture: If the primary database fails, a standby database
takes over.
2.Automatic Failover: Switches to standby in seconds.
3.Backups & Recovery: Automatic snapshots to prevent data loss.

52. Read/Write Splitting with Read Replicas
• Read replicas help in load balancing by handling read requests separately.
• Improves database performance for applications with heavy traffic.
Alibaba Cloud DTS (Data Transmission Service) for Disaster Recovery
• DTS replicates data across regions to prevent data loss in case of disasters.
• Used for cross-region backups and live database migration.

53. Real-Time Example
Example: HA for an Online Food Delivery App
• Users place food orders 24/7 from different locations.
• The app uses ApsaraDB for MySQL (High-Availability Edition) to ensure
zero downtime.
• If the primary database fails, the standby database takes over automatically.
• Read replicas improve performance by handling multiple user requests
efficiently.

54. Database Backup and Recovery
• Database backup is a copy of the database stored in a safe location.
• It is used to restore data in case of failure, corruption, or accidental deletion.
Example:
A company takes a daily backup of its sales database. If data is lost, they can restore
the last backup to recover it.
• Database recovery means restoring the database from a backup after data loss or
failure.
• It ensures minimum downtime and prevents business loss.
Example:
If a cyberattack deletes important files, the company can recover the database from the
backup and continue operations.

55. Backup Methods in Alibaba Cloud
Automatic Backups (Scheduled Backups)
• Alibaba Cloud automatically creates backups at regular intervals.
• Available in ApsaraDB for MySQL, AnalyticDB, and ECS databases.
Manual Backups (On-Demand Backups)
• Users can manually trigger a backup before making big changes.
• Helps to restore a previous version if something goes wrong.
Backup Storage:
• Backups are stored in Alibaba Cloud OSS (Object Storage Service).
• They can be encrypted for security.

56. Database Monitoring
• Database monitoring means tracking the performance, health, and security of a
database.
• It helps detect slow queries, high CPU usage, and connection issues before they cause
downtime.
Example:
If an e-commerce website experiences slow order processing, monitoring tools can check
if the database is overloaded and fix the issue.
• Ensures High Performance – Keeps response time fast.
• Prevents Failures – Detects issues before they cause downtime.
• Improves Security – Identifies unauthorized access.
• Reduces Costs – Optimizes resource usage.

57. Monitoring Tools in Alibaba Cloud
• Alibaba Cloud provides built-in monitoring tools for database health tracking:
CloudMonitor
• Tracks CPU, memory, disk usage, and slow queries.
• Sends alerts if something goes wrong.
Database Autonomy Service (DAS)
• AI-powered monitoring that automatically optimizes performance.
• Detects slow SQL queries and suggests improvements.
Logs & Audit Service
• Stores database logs to track changes and troubleshoot issues.

58. Database Maintenance
• Database maintenance means keeping the database clean, updated, and
running efficiently.
Common Database Maintenance Tasks
Index Optimization – Speeds up searches by improving indexes.
Cleaning Old Data – Deletes unnecessary data to save space.
Updating Security Patches – Protects against cyber threats.
Checking Backup Integrity – Ensures that backups work properly.
Performance Tuning – Adjusts settings to improve speed.