This presentation provides architecture overview of Azure storage services including blob, table, queue and best practices of storage services.

1. Chaowlert Chaisrichalermpol
Senior Developer
Jetabroad

2.  Blob
 Table
 Queue
 Storage services

3. Chaowlert Chaisrichalermpol
Azure Storage experiences
 Architect of Samsung Gift & AIS Privilege
 Monthly Active Users > 400,000
 Daily Active Users > 150,000
 Experienced migrate from Sql Azure to Table Storage

5.  Store files
 Files
 Media files
 Logs, raw data (consider using Azure Hadoop to analyze)
 Backup
 Any!
 Cost is insanely cheap! (low as 24$/mth for 1TB!)
 This price include fault tolerance & API
 48$/mth for geo replication

6.  Movie provider

7.  Performance
 Point to Blob directly to reduce web role load
 Gzip text file before put into blob
 Resume download by Accept-Ranges header
 Scalability
 Use CDN
 Scalability target per blob is 64M/sec
 Upload blob simultaneously is faster

9.  NoSql key-value store
 NoSql = No join + no grouping + no order by + no paging + no
index + no transaction + no view + no stored proc + no function +
no constraint + no default + no trigger + no fancy data type + no
backup restore + much more!!!

10. Azure Table SQL Azure
Features Basic Full sql features
Performance 20,000 rows/sec
Higher for cross accounts
180 concurrent connections
Higher for premium services
Scalability Automatic Sql Federation
Availability Geo redundant Sql Data Sync
Cost (150 GB) 20$ (include 120M
storage transactions)
226$ (no premium services,
no federation, no data sync)
Usage Insert intensive data All others

11.  Table Load Test
loader.io
20 web roles
Azure
Table
SQL
Azure

12. Azure Table Sql Azure
Insert (DateTime Key) 2,113 tx/sec 588 tx/sec
Insert (Guid key) MAX!! 595 tx/sec
Update to single item 58 tx/sec 74 tx/sec
Read single MAX!! MAX!!
Read 1,000 179 tx/sec 860 tx/sec

13.  Review PartitionKey design
 Batch: Insert, update, delete up to 100 entities
 Ordering: Data always sorted by PartitionKey and RowKey
 Scalability
 Scalability target for single partition is 2,000 rows/sec
 Avoid prepend/append pattern on PartitionKey
 Avoid hotspot on PartitionKey
 Performance
 Use Json over AtomPub (70% bandwidth reduction)
 Use SCL 3.0+ (higher performance + use Json as default)

15.  Persistent job processing
 Able to process up to 32 messages at a time
 Not for log computing
 Message size is up to 64kb
 May have to keep data in blob

16. Azure Queue Service Bus
Max Age 7 days Unlimited
Protocols REST REST, SBMP, AMQP
Queue access Polling Polling, Long polling,
Push
Feature Simple queue Session, Transaction,
Duplicate detect, Auto
dead lettering
Cost 0.05$ / million tx 1$ / million messages
Usage Job Messaging

17. Queue
Web Role
1. Add message
Worker Role
Message
2. Get message
& lease
3. Delete
message

18.  Workload leveling
 Image resizing
 Order processing
 Workload distribution
 Email sending
 Sending push notification
 Temporal decoupling
 Night report jobs

19.  Ticket sell
 300,000 users in 1 minute!
 Only 3,000 tickets available

20.  Scalability
 Scalability target per queue is 2,000 message /sec
 Consider use multiple queues for higher scalability
 Workload scaling
 AutoScaling by using MessageCount on queue
 Cost of queue is worker role
 99.9% worker role, 0.1% storage
 If usage is low, use web role as worker
 Use AutoScaling

22. LRS ZRS GRS RA-GRS
Store 3 repl single
faculty
3 repl on
multiple
faculties
6 repl on 2
regions
(3 repl each)
6 repl on 2
regions
(3 repl each)
Durability Disk/node
failure
Zone failure
(fire)
Regional
disaster
Regional
disaster
Services Blob, Table,
Queue
Block Blob
only
Blob, Table,
Queue
Blob, Table,
Queue
Failover to
secondary
region
- - Auto within 15
mins
Failover read
access by app
Price (first TB)
/GB/month
$0.024 $0.030 $0.048 $0.061

23. Understand single partition unit
 Blob
 Single Blob = Account + Container + Blob name
 Single Blob scalability target is 60MB/sec
 Table
 Single Table Partition = Account + Table name + Partition Key
 Scalability target is 2,000 entities (1kb) / sec
 Queue
 Single queue = Account + Queue name
 Single queue scalability target is 2,000 small message (1kb)/sec

24.  Scalability Target per account
 Capacity: up to 200TB
 Transaction: 20,000 messages (1kb) / second
 Network
 Local redundant: Ingress 10Gibps/ Egress 15Gibps
 Geo redundant: Ingress 5Gibps/ Egress 10Gibps
 Scale beyond single storage account
User Location Account
Lee Hong Kong Storage East
John London Storage USA
Amm Thailand Storage SE

25.  Storage & Web Role should be in the same region
 Client can access storage directly (Public or SAS)
 Use latest SDK
 Net setting
 ServicePointManager.UseNagleAlgorithm = false;
 ServicePointManager.Expect100Continue = false;
 ServicePointManager.DefaultConnectionLimit = 100; (Or More)

26.  Public access (blob only)
 Direct access
 CDN access
 Shared Access Signature (SAS)
 For client applications (mobile, web)
 Restricted access by permission: list, read, add, update, delete
 Restricted access at data level
 Invalidate when regenerate full access key
 Full access key
 For trusted services

27.  Todo list app (Share Access Signature )
Client
Web Role
1. Request token
Azure Table
2. Access Data