Windows Azure Storage – Architecture View

853 views

Published on

This presentation provides architecture overview of Azure storage services including blob, table, queue and best practices of storage services.

Published in: Software, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
853
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Windows Azure Storage – Architecture View

  1. 1. Chaowlert Chaisrichalermpol Senior Developer Jetabroad
  2. 2.  Blob  Table  Queue  Storage services
  3. 3. Chaowlert Chaisrichalermpol Azure Storage experiences  Architect of Samsung Gift & AIS Privilege  Monthly Active Users > 400,000  Daily Active Users > 150,000  Experienced migrate from Sql Azure to Table Storage
  4. 4.  Store files  Files  Media files  Logs, raw data (consider using Azure Hadoop to analyze)  Backup  Any!  Cost is insanely cheap! (low as 24$/mth for 1TB!)  This price include fault tolerance & API  48$/mth for geo replication
  5. 5.  Movie provider
  6. 6.  Performance  Point to Blob directly to reduce web role load  Gzip text file before put into blob  Resume download by Accept-Ranges header  Scalability  Use CDN  Scalability target per blob is 64M/sec  Upload blob simultaneously is faster
  7. 7.  NoSql key-value store  NoSql = No join + no grouping + no order by + no paging + no index + no transaction + no view + no stored proc + no function + no constraint + no default + no trigger + no fancy data type + no backup restore + much more!!!
  8. 8. Azure Table SQL Azure Features Basic Full sql features Performance 20,000 rows/sec Higher for cross accounts 180 concurrent connections Higher for premium services Scalability Automatic Sql Federation Availability Geo redundant Sql Data Sync Cost (150 GB) 20$ (include 120M storage transactions) 226$ (no premium services, no federation, no data sync) Usage Insert intensive data All others
  9. 9.  Table Load Test loader.io 20 web roles Azure Table SQL Azure
  10. 10. Azure Table Sql Azure Insert (DateTime Key) 2,113 tx/sec 588 tx/sec Insert (Guid key) MAX!! 595 tx/sec Update to single item 58 tx/sec 74 tx/sec Read single MAX!! MAX!! Read 1,000 179 tx/sec 860 tx/sec
  11. 11.  Review PartitionKey design  Batch: Insert, update, delete up to 100 entities  Ordering: Data always sorted by PartitionKey and RowKey  Scalability  Scalability target for single partition is 2,000 rows/sec  Avoid prepend/append pattern on PartitionKey  Avoid hotspot on PartitionKey  Performance  Use Json over AtomPub (70% bandwidth reduction)  Use SCL 3.0+ (higher performance + use Json as default)
  12. 12.  Persistent job processing  Able to process up to 32 messages at a time  Not for log computing  Message size is up to 64kb  May have to keep data in blob
  13. 13. Azure Queue Service Bus Max Age 7 days Unlimited Protocols REST REST, SBMP, AMQP Queue access Polling Polling, Long polling, Push Feature Simple queue Session, Transaction, Duplicate detect, Auto dead lettering Cost 0.05$ / million tx 1$ / million messages Usage Job Messaging
  14. 14. Queue Web Role 1. Add message Worker Role Message 2. Get message & lease 3. Delete message
  15. 15.  Workload leveling  Image resizing  Order processing  Workload distribution  Email sending  Sending push notification  Temporal decoupling  Night report jobs
  16. 16.  Ticket sell  300,000 users in 1 minute!  Only 3,000 tickets available
  17. 17.  Scalability  Scalability target per queue is 2,000 message /sec  Consider use multiple queues for higher scalability  Workload scaling  AutoScaling by using MessageCount on queue  Cost of queue is worker role  99.9% worker role, 0.1% storage  If usage is low, use web role as worker  Use AutoScaling
  18. 18. LRS ZRS GRS RA-GRS Store 3 repl single faculty 3 repl on multiple faculties 6 repl on 2 regions (3 repl each) 6 repl on 2 regions (3 repl each) Durability Disk/node failure Zone failure (fire) Regional disaster Regional disaster Services Blob, Table, Queue Block Blob only Blob, Table, Queue Blob, Table, Queue Failover to secondary region - - Auto within 15 mins Failover read access by app Price (first TB) /GB/month $0.024 $0.030 $0.048 $0.061
  19. 19. Understand single partition unit  Blob  Single Blob = Account + Container + Blob name  Single Blob scalability target is 60MB/sec  Table  Single Table Partition = Account + Table name + Partition Key  Scalability target is 2,000 entities (1kb) / sec  Queue  Single queue = Account + Queue name  Single queue scalability target is 2,000 small message (1kb)/sec
  20. 20.  Scalability Target per account  Capacity: up to 200TB  Transaction: 20,000 messages (1kb) / second  Network  Local redundant: Ingress 10Gibps/ Egress 15Gibps  Geo redundant: Ingress 5Gibps/ Egress 10Gibps  Scale beyond single storage account User Location Account Lee Hong Kong Storage East John London Storage USA Amm Thailand Storage SE
  21. 21.  Storage & Web Role should be in the same region  Client can access storage directly (Public or SAS)  Use latest SDK  Net setting  ServicePointManager.UseNagleAlgorithm = false;  ServicePointManager.Expect100Continue = false;  ServicePointManager.DefaultConnectionLimit = 100; (Or More)
  22. 22.  Public access (blob only)  Direct access  CDN access  Shared Access Signature (SAS)  For client applications (mobile, web)  Restricted access by permission: list, read, add, update, delete  Restricted access at data level  Invalidate when regenerate full access key  Full access key  For trusted services
  23. 23.  Todo list app (Share Access Signature ) Client Web Role 1. Request token Azure Table 2. Access Data

×