Cloud storage azureblob_s3
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Cloud storage azureblob_s3

  • 397 views
Uploaded on

On June 27, I had the privilege of speaking at the Boston Azure User Group in Boston, MA. The topic covered a in-depth discussion of Azure Blob Storage and Amazon S3. It was an very interactive......

On June 27, I had the privilege of speaking at the Boston Azure User Group in Boston, MA. The topic covered a in-depth discussion of Azure Blob Storage and Amazon S3. It was an very interactive session with lots of great questions from the attendees. The topics that we covered

Storage
Overview
Azure Blob Storage
Amazon S3
Comparing Azure Blob and Amazon S3
Benchmark comparisons for performance, stability, and scalability
CDN
Shared Access Signature
Best Practice
Resource

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
397
On Slideshare
397
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Cloud Storage Azure Blob and Amazon S3 Udaiappa Ramachandran ( Udai ) @nhcloud
  • 2. Who am I? • Udaiappa Ramachandran ( Udai ) • Chief Architect, Ektron, Inc., • Azure Insider • New Hampshire Cloud User Group (//www.nashuaug.org ) • Focus on Cloud Computing • Windows Azure and Amazon Web Services • //cloudycode.wordpress.com • @nhcloud
  • 3. Agenda • Storage • Overview • Azure Blob Storage • Amazon S3 • Comparing Azure Blob and Amazon S3 • Benchmark comparisons for performance, stability, and scalability • CDN • Shared Access Signature • Best Practice • Resource • QA
  • 4. Storage
  • 5. Overview • Evaluation • Network Storage • Hosted Storage • Cloud Based Storage • Benefits • Highly Available with Strong Consistency • Provides access to data in face of failures/partitioning • Durability • Replicate data several times within and across data centers • Scalability • Need to scale to Exabyte’s and beyond • Provide a global namespace to access data around the world • Automatically load balance data to meet peak traffic demands • Easy setups and low cost • Pay as you go • API Accessible, Emulator, Web Portal
  • 6. Azure Blob 1 of 3 • Blobs • File with a metadata • Blob Types • Block Blob • Targeted for streaming workloads • Each blob consists of a sequence of blocks, Each block is identified by a Block ID targeted for streaming workloads • Size limit 200GB per blob • Optimistic concurrency via ETags • Page Blob • Targeted for random read/write workloads • Each blob consists of an array of pages. Each page is identified by its offset from the start of the blob • Size limit 1TB per blob • Optimistic or Pessimistic (locking) concurrency via leases • Drives • Durable NTFS volumes for Windows Azure applications to use • Based on Blobs • Drive access across multiple instance(s) possible thru SMB
  • 7. Azure Blob 2 of 3 • Storage End Point • http://<account>.blob.core.windows.net/<container>/<blobname> • [Account][Container][Blobs][Pages/Blocks] • Mock Storage Emulator • Can CDN Enable Account • Blobs delivered via 24 global CDN nodes • Can co-locate storage account with compute account • Explicitly or using affinity groups • Accounts have two independent 512 bit shared secret keys • 100 TBs per account • Geo-Replication • Storage Analytics • Logs: Provide trace of executed requests for your storage accounts • Metrics: Provide summary of key capacity and request statistics for Blobs, Tables, and Queues • HTTP headers for Blobs • RESTful and Client API support
  • 8. Azure Blob 3 of 3 • Associate Metadata With blob • Standard HTTP metadata/headers (Cache-Control, Content-Encoding, Content-Type, etc) • Metadata is <name, value> pairs, up to 8KB per blob • Either as part of PutBlob or independently • Blob always accessed by name • Can include ‘/‘ or other delimiters in name e.g. /<container>/myblobs/blob.jpg • Basic Blob Operations • GetBlob • DeleteBlob • CopyBlob • SnapshotBlob • LeaseBlob • ListBLobs
  • 9. Azure Drive • Durable NTFS volume for Windows Azure Instances • Use existing NTFS APIs to access a network attached durable drive • Use System.IO from .NET • Benefits • Move existing apps using NTFS more easily to the cloud • Durability and survival of data on instance recycle • A Windows Azure Drive is an NTFS VHD Page Blob • Mounts Page Blob over the network as an NTFS drive • Local cache on instance for read operations • All flushed and un buffered writes to drive are made durable to the Page Blob • A Windows Azure Drive is a Page Blob formatted as a NTFS single volume Virtual Hard Drive (VHD) • Drives can be up to 1TB
  • 10. Azure Drive • A Page Blob can be mounted: • On one instance at a time for read/write • Using read-only snapshots to multiple instances at once • An instance can dynamically mount up to 16 drives • Remote Access via standard Blob UI • Can’t remotely mount drive • Can upload the VHD to a Page Blob using the blob interface, and then mount it as a Drive • Can download the VHD to a local file and mount locally • Operations performed via Drive API not REST Calls • Operations on Drives • Create Drive • Mount / UnMount Drive • Get Mounted Drives • Snapshot Drive
  • 11. Amazon S3 1 of 2 • Simple Storage Service • Storage Endpoints • Virtual Hosted Style: http[s]://<bucketname>.s3-website-[us-east-1].amazonaws.com/object • Path-Style: http[s]://s3.amazonaws.com/<bucketname>/object • Associate Metadata With blob • Standard HTTP metadata/headers (Cache-Control, Content-Encoding, Content-Type, etc) • Metadata is <name, value> pairs, up to 8KB per blob • Either as part of PutObject or independently • Blob always accessed by key • Can include ‘/‘ or other delimiter in name but folder must end with ‘/’ e.g. /<bucketname>/myblobs/blob.jpg • 1-5 TB per blob • Unlimited Size per accounts • RESTful/API support • Storage Analytics • Ability to host static template from blob • Bittorrent protocol support
  • 12. Amazon S3 2 of 2 • Choose region to avoid latency • Elastic Block Store • Data availability from replication across an Availability Zone • Data persistence independent of the life of the instance • The ability to create snapshots and incremental backups • Upto 1000 IOPS • Free monitoring • S3 Operations • Put • Get • Delete • Copy • List
  • 13. Comparing Azure Blob, Amazon S3
  • 14. Comparing Azure Blob, Amazon S3
  • 15. CDN
  • 16. CDN • Types • Download • Streaming • Azure • <random>.vo.msecnd.net • custom domain support • Amazon • <random>.cloudfront.net • Custom domain support • Custom behaviors • Ability to Bill the end user per download
  • 17. Shared Access Signature
  • 18. Signed URL Scenario
  • 19. Signing the URL • Azure • Customizable • In-built API • Smooth streaming support • Format • st=starttime • se=expiring time • sr=blob|container ---resource • sp=rwd --permission • sig=signed hash • Amazon • Customizable • Uses the OpenSSL • RTMP (Adobe's Real-Time Message Protocol) support for video • Download signing is different than streaming • Distribution Format • ?Expires=<expires>&Signature=<signature>&Key-Pair-Id=<keypairid> • ?Policy=<policy>&Signature=<signature>&Key-Pair-Id=<keypairid> • Download: http://d111111abcdef8.cloudfront.net/images/image.jpg • Streaming: videos/mediafile.flv
  • 20. Best Practices • Avoid lock-in to specific provider • Choose the location closest to your customer • Use only lower cases for container/key name • Always remove public access from container/bucket. Instead grant public access to keys if required. • Avoid unnecessary request (such a call to check if blob exists instead use the response) • Use compression for large file if possible • Enable CDN
  • 21. Resources • //microsoft.com/azure • //blogs.msdn.com/b/windowsazurestorage/archive/2010/12/30/wind ows-azure-storage-architecture-overview.aspx • //msdn.microsoft.com/en-us/library/windowsazure/ee395415.aspx • //blog.smarx.com/posts/smooth-streaming-with-windows-azure- blobs-and-cdn • //channel9.msdn.com/shows/Cloud+Cover/Cloud-Cover-Episode-8- Shared-Access-Signatures/ • //aws.amazon.com/articles • //docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ WorkingWithStreamingDistributions.html • //www.longtailvideo.com/jw-player/wizard%23rtmp
  • 22. Q & A
  • 23. Thank you for attending Boston Azure Usergroup