Download to read offline
![Build,SkillandEnable
There are two options available at this stage:
1. you can either connect with the storage account name and storage
account key
[everyone who gets the name/key credentials has full access to all operations in
your storage account. Those people can use that access to add and delete queues,
read and write messages, and even access blob and table storages]
2. With a temporary token.
[create a temporary token that is valid for a particular queue, and limited by
functionality and time. Obviously, this way you can give someone restricted access
(e.g. read-only) for a specific time period (e.g. 5 minutes from now), and maintain
much tighter control over who has access to your application.]
Storage QUEUE- Security](https://image.slidesharecdn.com/azure-datastorewithstorageandcosmodb-190304123708/75/Azure-data-store-with-storage-and-cosmo-db-46-2048.jpg)
Uploaded byBipeen Sinha
Azure data store with storage and cosmo db
This document discusses Azure storage options including managed disks, unmanaged disks, storage accounts, blobs, tables, queues, and page blobs. It provides details on managed vs unmanaged disks, different storage account types, performance tiers and replication options, and how various Microsoft services use Azure storage. Key points covered include that managed disks simplify storage management, storage accounts provide namespaces for objects, and blobs can store large unstructured data files for applications.
More Related Content
Similar to Azure data store with storage and cosmo db
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Azure data store with storage and cosmo db
- 1. Build,SkillandEnableBuild,SkillandEnable Microsoft Azure DataStorage Bipin Sinhaa Bipeensinha.com Innovate, Collaborate & Accelerate
- 2. Build,SkillandEnable Topics • Azure Storage– Introduction • VM(Disk) Storage • Bob Storage • Table Storage • CosmoDB • Queue • Page Storage • Storage Encryption
- 3.
- 4.
- 5.
- 6.
- 7. Build,SkillandEnable Azure Managed Disksis an elastic disk storage system that is offered as an alternative to the current storage account-based system. Managed Disks eliminates the complexity of storage accounts and offers a simple scalable and highly available solution for creating and managing disks .It store the .VHD files in Page Blobs. Simple: Abstract storage accounts from customers Better Performance: Storage accounts limits does nit apply Big Scale : Up to 20,000 disk per region per subscription. Encryption: Support VM level disk encryption, secure data at rest Secure- No public access to underlying blob Azure Managed Disk for VM Storage
- 8. Build,SkillandEnable Unmanaged disk alsostore the .VHD files in Page Blobs. Management : Disk created and managed by customers Proper Sizing Storage accounts limited to 20,000 IOPs Sparse Storage (Standard): Only pay for actual data then the size Manage by customer Unmanage Storage Accounts Azure Unmanaged Disk for VM Storage
- 9.
- 10.
- 11. Build,SkillandEnable General-purpose v2 accounts General-purposev2 storage accounts support the latest Azure Storage features and incorporate all of the functionality of general-purpose v1 and Blob storage accounts. General-purpose v2 storage accounts support these Azure Storage services: •Blobs (all types: Block, Append, Page) •Files •Disks •Queues •Tables General-purpose v1 accounts General-purpose v1 accounts provide access to all Azure Storage services, but may not have the latest features or the lowest per gigabyte pricing. General-purpose v1 storage accounts support these Azure Storage services: •Blobs (all types) •Files •Disks •Queues •Tables Blob storage accounts A Blob storage account is a specialized storage account for storing unstructured object data as block blobs. Blob storage accounts provide the same durability, availability, scalability, and performance features that are available with general-purpose v2 storage accounts. Blob storage accounts support storing block blobs and append blobs, but not page blobs. Types of Storage Accounts •Storage account names must be between 3 and 24 characters in length and may contain numbers and lowercase letters only. •Your storage account name must be unique within Azure. No two storage accounts can have the same name
- 12. Build,SkillandEnable Storage account type Supported services Supported performance tiers Supported access tiers Replication options Deployment model1Encryption2 General- purpose V2 Blob, File, Queue, Table, and Disk Standard, Premium Hot, Cool, Archive 3 LRS, ZRS 4 , GRS, RA-GRS Resource Manager Encrypted General- purpose V1 Blob, File, Queue, Table, and Disk Standard, Premium N/A LRS, GRS, RA- GRS Resource Manager, Classic Encrypted Blob storage Blob (block blobs and append blobs only) Standard Hot, Cool, Archive 3 LRS, GRS, RA- GRS Resource Manager Encrypted • All storage accounts are encrypted using Storage Service Encryption (SSE) • The Archive tier is available at level of an individual blob only, not at the storage account level. Only block blobs and append blobs can be archived • Zone-redundant storage (ZRS) is available only for standard general-purpose v2 storage accounts. Types of Storage Accounts- Comparison
- 13. Build,SkillandEnable Performance tiers General-purpose storage accountsmay be configured for either of the following performance tiers: •A standard performance tier for storing blobs, files, tables, queues, and Azure virtual machine disks. •A premium performance tier for storing unmanaged virtual machine disks only. Replication/Redundancy LRS- Local Redundant Storage – 3 Copies in single DC ZRS- Zonal Redundant Storage – 3 copies in different DCs in same region or different region GRS- Geo Redundant Storage – 6 Copies . 3 in LRS and 3 in DCs in different geo regions. Geo- redundant storage (GRS) replicates your data to a secondary region that is hundreds of miles away from the primary region. If your storage account has GRS enabled, then your data is durable even in the case of a complete regional outage or a disaster in which the primary region is not recoverable. RA GRS- Read Access Geo Redundant Storage – 6 copies same as GRS but 3 copies in different geo DC also have read access. Read-access geo-redundant storage (RA-GRS) maximizes availability for your storage account, by providing read-only access to the data in the secondary location, in addition to the replication across two regions provided by GRS. Storage Accounts- Performance tiers and Replication
- 14. Build,SkillandEnable Storage account endpoints Astorage account provides a unique namespace in Azure for your data. Every object that you store in Azure Storage has an address that includes your unique account name. The combination of the account name and the Azure Storage service endpoint forms the endpoints for your storage account. For example, if your general-purpose storage account is named mystorageaccount, then the default endpoints for that account are: Blob storage: http://mystorageaccount.blob.core.windows.net Table storage: http://mystorageaccount.table.core.windows.net Queue storage: http://mystorageaccount.queue.core.windows.net Azure Files: http://mystorageaccount.file.core.windows.net Storage Account Endpoints The URL for accessing an object in a storage account is constructed by appending the object's location in the storage account to the endpoint. For example, a blob address might have this format: http://mystorageaccount.blob.core.windows.net/mycontainer/myblob.
- 15. Build,SkillandEnable Blobs Simple Interface to storeand retrieve large amount of unstructured data like files, images, videos, etc Tables No SQLDatastore Used for storing structured, non- relational data. Queues A queue contains many messages. Decouple frontend role from backend. Disks Persistent standard and premium storage for IaaS VM built on PageBlobs . Files SMB 3.0 access to azure storage. Map to file share . Lift and shift legacy apps. Azure Import/Export Ship data in hard disk drive to azure datacentre AZCopy High Performance data uploads , downloads and copy Azure Storage Offerings
- 16. Build,SkillandEnable Abstraction of Storage •Blobs – Simple Interface to store and retrieve files in cloud. • Data Sharing : Share documents , picture, video , music etc. • Big Data : Store rwa data/logs • Backups-Data and device Backup • Disks– Network mounted durables disks for VMs in Azure • Data Sharing : Share documents , picture, video , music etc. • Mounted disks and VHDs stored in Azure Blobs • Move on-premised applications to cloud • Tables– Massively scalable and extremely easy to use NoSQL system that auto scales • Store users information, device information and any type of metadata for your service. • Queues– Reliable messaging system • Decouple components/roles • Web role to worker role communication • Allow roles to scale independently • Implement scheduling of asynchronous tasks • Building process/workflows
- 17.
- 18. Build,SkillandEnable Accessing Windows StorageVia Explorer Cloud storage studio by Cerebrata Azure Storage Explorer by Neudistic CloudXplorer by ClumsyLeaf CloudBerry Explorer by Cloudberry Lab
- 19. Build,SkillandEnable How the AzureStorage Used by Microsoft Xbox: Uses Blobs, Tables and Queues for cloud game saves, Xbox Music, Xbox live etc Skype: Uses Blobs, Tables and Queues for Skype video messages and to keep metadata to allow skype clients to connect with each other. Bing: Uses Blobs, Tables and Queues to provide a near real time ingestion engine that consume Twitter and Facebook feeds, indexes them which is the folded inti Bing search SkyDrive: Uses Blobs to store pictures , documents , video, files etc.
- 20.
- 21. Build,SkillandEnable Blob Storage Simple Interfaceto store and retrieve large amount of unstructured data like files, images, videos, etc . A single blob can be hundreds of gigabytes in size, and a single storage account can contain up to 200TB of blobs. Azure Storage supports three types of blobs: Block blobs store text and binary data, up to about 4.7 TB. Block blobs are made up of blocks of data that can be managed individually. Append blobs are made up of blocks like block blobs, but are optimized for append operations. Append blobs are ideal for scenarios such as logging data from virtual machines. Page blobs store random access files up to 8 TB in size. Page blobs store the virtual hard drive (VHD) files serve as disks for Azure virtual machines
- 22. Build,SkillandEnable Blob Storage isdesigned for : • Serving images or documents directly to a browser. • Storing files for distributed access. • Streaming video and audio. • Writing to log files. • Storing data for backup and restore, disaster recovery, and archiving. • Storing data for analysis by an on- premises or Azure-hosted service. Blob Storage
- 23. Build,SkillandEnable Blob Storage Resources Blobstorage offers three types of resources: •The storage account- A storage account provides a unique namespace in Azure for your data. Every object that you store in Azure Storage has an address that includes your unique account name. An account can have many Blob Containers •A container in the storage account-A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs. Sharing policies are set at the container level, where a container can be set to private or to be publicly accessible •A blob in a container
- 24. Build,SkillandEnable The Hotaccess tier, which is optimized for frequent access of objects in the storage account. Accessing data in the Hot tier is most cost-effective, while storage costs are somewhat higher. The Cool access tier, which is optimized for storing large amounts of data that is infrequently accessed and stored for at least 30 days. Storing data in the Cool tier is more cost-effective, but accessing that data may be somewhat more expensive than accessing data in the Hot tier. The Archive tier, which is available only for individual block blobs. The Archive tier is optimized for data that can tolerate several hours of retrieval latency and will remain in the Archive tier for at least 180 days. The Archive tier is the most cost-effective option for storing data, but accessing that data is more expensive than accessing data in the Hot or Cool tiers. Access tiers for block blob data
- 25. Build,SkillandEnable Archive Access tiersfor block blob data
- 26.
- 27. Build,SkillandEnable Table Storage • AzureTable storage is a service that stores structured NoSQL data in the cloud, providing a key/attribute store with a schemaless design. • Because Table storage is schemaless it's easy to adapt your data as the needs of your application evolve. • Access to Table storage data is fast and cost-effective for many types of applications, and is typically lower in cost than traditional SQL for similar volumes of data
- 28. Build,SkillandEnable Table Storage Components StorageAccount – All access to Windows Azure Storage is done through a storage account. This is the highest level of the namespace for accessing tables. An account can have many Tables Table: A table is a collection of entities. Tables don't enforce a schema on entities, which means a single table can contain entities that have different sets of properties. Entity: An entity is a set of properties, similar to a database row. An entity in Azure Storage can be up to 1MB in size. An entity contains a set of properties. Each table has two properties, “PartitionKey and RowKey”, which form the unique key for the entity Properties: A property is a name-value pair. An entity can hold up to 255 properties. Each entity also has three system properties that specify a partition key, a row key, and a timestamp.
- 29. Build,SkillandEnable PartitionKey – Thefirst key property of every table. The system uses this key to automatically distribute and load balance the table’s entities over many servers. RowKey – A second key property for the table. This is the unique ID of the entity within the partition it belongs to. The PartitionKey combined with the RowKey uniquely identifies an entity in a table. Timestamp – Every entity has a version maintained by the system. This is used for optimistic concurrency. Table Storage Properties
- 30.
- 31.
- 32.
- 33. Build,SkillandEnable CosmoDB- NoSQL No datais is born relational, it’s born dirty and messy, in whatever shape or structure it’s created. NoSQL databases used by companies such as Facebook, Google and Amazon as they working with big data problems and/or real time web applications. This set of technology is coined “Not Only SQL” since they might support a SQL syntax for retrieving data. The Azure Tables are extremely scalable and cheap, but if you start querying on any other attributes then you start running into problems.
- 34. Build,SkillandEnable NoSQL - HowIt Store the data
- 35. Build,SkillandEnable CosmoDB- Introduction Azure CosmosDB is a globally distributed, multi-model database service that supports document, key- value, wide- column, and graph databases.
- 36. Build,SkillandEnable CosmoDB -Turnkey globaldistribution
- 37. Build,SkillandEnable The Database ishosted in US region while the users are in different regions . This add latency and delay . In application like stock market it is big problem. One possible solutions is to use the read only copy to different regions and aloo the users to read it, but this also lead to inconsistency because of sync dely. Another solutions is to use the eventual commit where data cannot be read until it commit to all database. CosmoDB Consistency- A situational scenario
- 38. Build,SkillandEnable No single machine bottleneck. ServerSide partition Management GB to PB Storage selectable throughput Automatic Document expiration Pay by the hour , customized throughput cost Support for request per second or minute CosmoDB Abstract
- 39. Build,SkillandEnable Turnkey global distribution •Cosmos DB enables you to build highly responsive and highly available applications worldwide. • Cosmos DB transparently replicates your data wherever your users are, so your users can interact with a replica of the data nearest to them. • Cosmos DB allows you to add or remove any of the Azure regions to your Cosmos account at any time, with a click of a button. Always On Cosmos DB provides 99.999% high availability for both reads and writes. Cosmos DB also provides you with the ability to programmatically (or via Portal) invoke the regional failover of your Cosmos account. This capability helps you ensure that while a Cosmos database may automatically failover if there is a regional disaster. Guaranteed low latency worldwide Cosmos DB guarantees less than 10- ms latencies for both, reads and (indexed) writes at the 99th percentile, all around the world. Cosmos DB is available in all Azure regions worldwide, including 54+ regions in public cloud, Azure China 21Vianet, Azure Germany, Azure Government, and Azure Government for Department of Defense (DoD). CosmoDB Advantage
- 40.
- 41. Build,SkillandEnable It stores thedata in JSON. It’s massively scalable but fully query-able by all parts of JSON tree. CosmoDB - Structure
- 42. Build,SkillandEnable The Azure portalhas three explorers that can be used to manipulate collection. Document Explorer – It allows you to add, update and delete documents from a collection. Query Explorer - Allows you to select document data from the collection using a SQL like syntax. Script Explorer- Allows you to define JavaScript objects to support advanced functionality CosmoDB –Azure Portal Explorer
- 43.
- 44. Build,SkillandEnable It’s all abouta collection that has two primary methods: enqueue (to insert new items to the queue at the bottom) and dequeue (to get the top most item from the queue). Storage QUEUE Microsoft Azure Queues are a ready-to-use service that loosely connects components or applications through the cloud. Azure Queues are a part of the Storage Service that unites Blobs, Tables and Queues under one storage account. There are two queue technologies available in Microsoft Azure: Azure Queues and Service Bus Queues. The latter is newer and has a focus on publish/subscribe mechanisms (i.e. the ability to receive messages without having to poll the queue) and integration with the WCF communication stack. On the other hand, Azure Queues are faster and more applicable when storing more than 5 GB of messages in a queue.
- 45. Build,SkillandEnable Azure Queues arequeues located in the Microsoft cloud which you can use for exchanging messages between components either in the cloud or on premise. A message typically represents a task created by someone (“producer”) that has to be processed by someone else (“consumer”). Each message has a small body and some attributes, such as time-to-live, which you can use for configuring your service. As Azure ensures that a dequeued message is invisible to other listeners, you could imagine many producers and many consumers as well as one-to-one scenarios. The main architectural benefit is loose coupling Storage QUEUE- Architecture
- 46. Build,SkillandEnable There are twooptions available at this stage: 1. you can either connect with the storage account name and storage account key [everyone who gets the name/key credentials has full access to all operations in your storage account. Those people can use that access to add and delete queues, read and write messages, and even access blob and table storages] 2. With a temporary token. [create a temporary token that is valid for a particular queue, and limited by functionality and time. Obviously, this way you can give someone restricted access (e.g. read-only) for a specific time period (e.g. 5 minutes from now), and maintain much tighter control over who has access to your application.] Storage QUEUE- Security
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53. Build,SkillandEnable Azure VM DiskEncryption
- 54.
- 55. Build,SkillandEnable Index B- AzureData Store
- 56.