Windows Azure Storage – Architecture View
Upcoming SlideShare
Loading in...5
×
 

Windows Azure Storage – Architecture View

on

  • 262 views

This presentation provides architecture overview of Azure storage services including blob, table, queue and best practices of storage services.

This presentation provides architecture overview of Azure storage services including blob, table, queue and best practices of storage services.

Statistics

Views

Total Views
262
Views on SlideShare
260
Embed Views
2

Actions

Likes
0
Downloads
9
Comments
0

1 Embed 2

http://www.slideee.com 2

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Windows Azure Storage – Architecture View Presentation Transcript

  • 1. Chaowlert Chaisrichalermpol Senior Developer Jetabroad
  • 2.  Blob  Table  Queue  Storage services
  • 3. Chaowlert Chaisrichalermpol Azure Storage experiences  Architect of Samsung Gift & AIS Privilege  Monthly Active Users > 400,000  Daily Active Users > 150,000  Experienced migrate from Sql Azure to Table Storage
  • 4.  Store files  Files  Media files  Logs, raw data (consider using Azure Hadoop to analyze)  Backup  Any!  Cost is insanely cheap! (low as 24$/mth for 1TB!)  This price include fault tolerance & API  48$/mth for geo replication
  • 5.  Movie provider
  • 6.  Performance  Point to Blob directly to reduce web role load  Gzip text file before put into blob  Resume download by Accept-Ranges header  Scalability  Use CDN  Scalability target per blob is 64M/sec  Upload blob simultaneously is faster
  • 7.  NoSql key-value store  NoSql = No join + no grouping + no order by + no paging + no index + no transaction + no view + no stored proc + no function + no constraint + no default + no trigger + no fancy data type + no backup restore + much more!!!
  • 8. Azure Table SQL Azure Features Basic Full sql features Performance 20,000 rows/sec Higher for cross accounts 180 concurrent connections Higher for premium services Scalability Automatic Sql Federation Availability Geo redundant Sql Data Sync Cost (150 GB) 20$ (include 120M storage transactions) 226$ (no premium services, no federation, no data sync) Usage Insert intensive data All others
  • 9.  Table Load Test loader.io 20 web roles Azure Table SQL Azure
  • 10. Azure Table Sql Azure Insert (DateTime Key) 2,113 tx/sec 588 tx/sec Insert (Guid key) MAX!! 595 tx/sec Update to single item 58 tx/sec 74 tx/sec Read single MAX!! MAX!! Read 1,000 179 tx/sec 860 tx/sec
  • 11.  Review PartitionKey design  Batch: Insert, update, delete up to 100 entities  Ordering: Data always sorted by PartitionKey and RowKey  Scalability  Scalability target for single partition is 2,000 rows/sec  Avoid prepend/append pattern on PartitionKey  Avoid hotspot on PartitionKey  Performance  Use Json over AtomPub (70% bandwidth reduction)  Use SCL 3.0+ (higher performance + use Json as default)
  • 12.  Persistent job processing  Able to process up to 32 messages at a time  Not for log computing  Message size is up to 64kb  May have to keep data in blob
  • 13. Azure Queue Service Bus Max Age 7 days Unlimited Protocols REST REST, SBMP, AMQP Queue access Polling Polling, Long polling, Push Feature Simple queue Session, Transaction, Duplicate detect, Auto dead lettering Cost 0.05$ / million tx 1$ / million messages Usage Job Messaging
  • 14. Queue Web Role 1. Add message Worker Role Message 2. Get message & lease 3. Delete message
  • 15.  Workload leveling  Image resizing  Order processing  Workload distribution  Email sending  Sending push notification  Temporal decoupling  Night report jobs
  • 16.  Ticket sell  300,000 users in 1 minute!  Only 3,000 tickets available
  • 17.  Scalability  Scalability target per queue is 2,000 message /sec  Consider use multiple queues for higher scalability  Workload scaling  AutoScaling by using MessageCount on queue  Cost of queue is worker role  99.9% worker role, 0.1% storage  If usage is low, use web role as worker  Use AutoScaling
  • 18. LRS ZRS GRS RA-GRS Store 3 repl single faculty 3 repl on multiple faculties 6 repl on 2 regions (3 repl each) 6 repl on 2 regions (3 repl each) Durability Disk/node failure Zone failure (fire) Regional disaster Regional disaster Services Blob, Table, Queue Block Blob only Blob, Table, Queue Blob, Table, Queue Failover to secondary region - - Auto within 15 mins Failover read access by app Price (first TB) /GB/month $0.024 $0.030 $0.048 $0.061
  • 19. Understand single partition unit  Blob  Single Blob = Account + Container + Blob name  Single Blob scalability target is 60MB/sec  Table  Single Table Partition = Account + Table name + Partition Key  Scalability target is 2,000 entities (1kb) / sec  Queue  Single queue = Account + Queue name  Single queue scalability target is 2,000 small message (1kb)/sec
  • 20.  Scalability Target per account  Capacity: up to 200TB  Transaction: 20,000 messages (1kb) / second  Network  Local redundant: Ingress 10Gibps/ Egress 15Gibps  Geo redundant: Ingress 5Gibps/ Egress 10Gibps  Scale beyond single storage account User Location Account Lee Hong Kong Storage East John London Storage USA Amm Thailand Storage SE
  • 21.  Storage & Web Role should be in the same region  Client can access storage directly (Public or SAS)  Use latest SDK  Net setting  ServicePointManager.UseNagleAlgorithm = false;  ServicePointManager.Expect100Continue = false;  ServicePointManager.DefaultConnectionLimit = 100; (Or More)
  • 22.  Public access (blob only)  Direct access  CDN access  Shared Access Signature (SAS)  For client applications (mobile, web)  Restricted access by permission: list, read, add, update, delete  Restricted access at data level  Invalidate when regenerate full access key  Full access key  For trusted services
  • 23.  Todo list app (Share Access Signature ) Client Web Role 1. Request token Azure Table 2. Access Data