Chaowlert Chaisrichalermpol outlines his experience with Azure storage services, emphasizing cost-effective solutions like blob and table storage for high-volume applications. He details the performance metrics and scalability targets of various storage options, along with considerations for partition design and workload management. The document also compares Azure's SQL and NoSQL offerings while providing insights into optimizing data access and job processing within Azure's architecture.

1. Chaowlert Chaisrichalermpol
Senior Developer
Jetabroad

2.  Blob
 Table
 Queue
 Storage services

3. Chaowlert Chaisrichalermpol
Azure Storage experiences
 Architect of Samsung Gift & AIS Privilege
 Monthly Active Users > 400,000
 Daily Active Users > 150,000
 Experienced migrate from Sql Azure to Table Storage

5.  Store files
 Files
 Media files
 Logs, raw data (consider using Azure Hadoop to analyze)
 Backup
 Any!
 Cost is insanely cheap! (low as 24$/mth for 1TB!)
 This price include fault tolerance & API
 48$/mth for geo replication

6.  Movie provider

7.  Performance
 Point to Blob directly to reduce web role load
 Gzip text file before put into blob
 Resume download by Accept-Ranges header
 Scalability
 Use CDN
 Scalability target per blob is 64M/sec
 Upload blob simultaneously is faster

9.  NoSql key-value store
 NoSql = No join + no grouping + no order by + no paging + no
index + no transaction + no view + no stored proc + no function +
no constraint + no default + no trigger + no fancy data type + no
backup restore + much more!!!

10. Azure Table SQL Azure
Features Basic Full sql features
Performance 20,000 rows/sec
Higher for cross accounts
180 concurrent connections
Higher for premium services
Scalability Automatic Sql Federation
Availability Geo redundant Sql Data Sync
Cost (150 GB) 20$ (include 120M
storage transactions)
226$ (no premium services,
no federation, no data sync)
Usage Insert intensive data All others

11.  Table Load Test
loader.io
20 web roles
Azure
Table
SQL
Azure

12. Azure Table Sql Azure
Insert (DateTime Key) 2,113 tx/sec 588 tx/sec
Insert (Guid key) MAX!! 595 tx/sec
Update to single item 58 tx/sec 74 tx/sec
Read single MAX!! MAX!!
Read 1,000 179 tx/sec 860 tx/sec

13.  Review PartitionKey design
 Batch: Insert, update, delete up to 100 entities
 Ordering: Data always sorted by PartitionKey and RowKey
 Scalability
 Scalability target for single partition is 2,000 rows/sec
 Avoid prepend/append pattern on PartitionKey
 Avoid hotspot on PartitionKey
 Performance
 Use Json over AtomPub (70% bandwidth reduction)
 Use SCL 3.0+ (higher performance + use Json as default)

15.  Persistent job processing
 Able to process up to 32 messages at a time
 Not for log computing
 Message size is up to 64kb
 May have to keep data in blob

16. Azure Queue Service Bus
Max Age 7 days Unlimited
Protocols REST REST, SBMP, AMQP
Queue access Polling Polling, Long polling,
Push
Feature Simple queue Session, Transaction,
Duplicate detect, Auto
dead lettering
Cost 0.05$ / million tx 1$ / million messages
Usage Job Messaging

17. Queue
Web Role
1. Add message
Worker Role
Message
2. Get message
& lease
3. Delete
message

18.  Workload leveling
 Image resizing
 Order processing
 Workload distribution
 Email sending
 Sending push notification
 Temporal decoupling
 Night report jobs

19.  Ticket sell
 300,000 users in 1 minute!
 Only 3,000 tickets available

20.  Scalability
 Scalability target per queue is 2,000 message /sec
 Consider use multiple queues for higher scalability
 Workload scaling
 AutoScaling by using MessageCount on queue
 Cost of queue is worker role
 99.9% worker role, 0.1% storage
 If usage is low, use web role as worker
 Use AutoScaling

22. LRS ZRS GRS RA-GRS
Store 3 repl single
faculty
3 repl on
multiple
faculties
6 repl on 2
regions
(3 repl each)
6 repl on 2
regions
(3 repl each)
Durability Disk/node
failure
Zone failure
(fire)
Regional
disaster
Regional
disaster
Services Blob, Table,
Queue
Block Blob
only
Blob, Table,
Queue
Blob, Table,
Queue
Failover to
secondary
region
- - Auto within 15
mins
Failover read
access by app
Price (first TB)
/GB/month
$0.024 $0.030 $0.048 $0.061

23. Understand single partition unit
 Blob
 Single Blob = Account + Container + Blob name
 Single Blob scalability target is 60MB/sec
 Table
 Single Table Partition = Account + Table name + Partition Key
 Scalability target is 2,000 entities (1kb) / sec
 Queue
 Single queue = Account + Queue name
 Single queue scalability target is 2,000 small message (1kb)/sec

24.  Scalability Target per account
 Capacity: up to 200TB
 Transaction: 20,000 messages (1kb) / second
 Network
 Local redundant: Ingress 10Gibps/ Egress 15Gibps
 Geo redundant: Ingress 5Gibps/ Egress 10Gibps
 Scale beyond single storage account
User Location Account
Lee Hong Kong Storage East
John London Storage USA
Amm Thailand Storage SE

25.  Storage & Web Role should be in the same region
 Client can access storage directly (Public or SAS)
 Use latest SDK
 Net setting
 ServicePointManager.UseNagleAlgorithm = false;
 ServicePointManager.Expect100Continue = false;
 ServicePointManager.DefaultConnectionLimit = 100; (Or More)

26.  Public access (blob only)
 Direct access
 CDN access
 Shared Access Signature (SAS)
 For client applications (mobile, web)
 Restricted access by permission: list, read, add, update, delete
 Restricted access at data level
 Invalidate when regenerate full access key
 Full access key
 For trusted services

27.  Todo list app (Share Access Signature )
Client
Web Role
1. Request token
Azure Table
2. Access Data