Azure data store with storage and cosmo db

Build,SkillandEnableBuild,SkillandEnable
Microsoft Azure Data Storage
Bipin Sinhaa
Bipeensinha.com
Innovate, Collaborate & Accelerate

Build,SkillandEnable
Topics
• Azure Storage – Introduction
• VM(Disk) Storage
• Bob Storage
• Table Storage
• CosmoDB
• Queue
• Page Storage
• Storage Encryption

Introduction to Store Storage

Azure VM Storage

VM Storage Type

Azure Managed Disks is an elastic disk storage
system that is offered as an alternative to the
current storage account-based system. Managed
Disks eliminates the complexity of storage accounts and
offers a simple scalable and highly available solution for
creating and managing disks .It store the .VHD files in
Page Blobs.
Simple: Abstract storage accounts from customers
Better Performance: Storage accounts limits does nit apply
Big Scale : Up to 20,000 disk per region per subscription.
Encryption: Support VM level disk encryption, secure data
at rest
Secure- No public access to underlying blob
Azure Managed Disk for VM Storage

Unmanaged disk also store the .VHD files in
Page Blobs.
Management : Disk created and managed by
customers
Proper Sizing Storage accounts limited to 20,000
IOPs
Sparse Storage (Standard): Only pay for actual
data then the size
Manage by customer
Unmanage Storage Accounts
Azure Unmanaged Disk for VM Storage

Managed Vs Unmanaged Disk

Storage Accounts

General-purpose v2 accounts
General-purpose v2 storage accounts
support the latest Azure Storage
features and incorporate all of the
functionality of general-purpose v1 and
Blob storage accounts. General-purpose
v2 storage accounts support these Azure
Storage services:
•Blobs (all types: Block, Append, Page)
•Files
•Disks
•Queues
•Tables
General-purpose v1 accounts
General-purpose v1 accounts provide
access to all Azure Storage services,
but may not have the latest features
or the lowest per gigabyte pricing.
General-purpose v1 storage accounts
support these Azure Storage services:
•Blobs (all types)
•Files
•Disks
•Queues
•Tables
Blob storage accounts
A Blob storage account is a
specialized storage account for
storing unstructured object data
as block blobs. Blob storage
accounts provide the same
durability, availability, scalability,
and performance features that are
available with general-purpose v2
storage accounts. Blob storage
accounts support storing block
blobs and append blobs, but not
page blobs.
Types of Storage Accounts
•Storage account names must be between 3 and 24 characters in length and may contain numbers and lowercase letters only.
•Your storage account name must be unique within Azure. No two storage accounts can have the same name

Storage
account type
Supported
services
Supported
performance
tiers
Supported
access tiers
Replication
options
Deployment
model1 Encryption2
General-
purpose V2
Blob, File,
Queue, Table,
and Disk
Standard,
Premium
Hot, Cool,
Archive
3
LRS, ZRS
4
,
GRS, RA-GRS
Resource
Manager
Encrypted
General-
purpose V1
Blob, File,
Queue, Table,
and Disk
Standard,
Premium
N/A LRS, GRS, RA-
GRS
Resource
Manager,
Classic
Encrypted
Blob storage Blob (block
blobs and
append blobs
only)
Standard Hot, Cool,
Archive
3
LRS, GRS, RA-
GRS
Resource
Manager
Encrypted
• All storage accounts are encrypted using Storage Service Encryption (SSE)
• The Archive tier is available at level of an individual blob only, not at the storage account level. Only block blobs and append blobs can be archived
• Zone-redundant storage (ZRS) is available only for standard general-purpose v2 storage accounts.
Types of Storage Accounts- Comparison

Performance tiers
General-purpose storage
accounts may be
configured for either of
the following
performance tiers:
•A standard
performance tier for
storing blobs, files,
tables, queues, and
Azure virtual machine
disks.
•A premium
performance tier for
storing unmanaged
virtual machine disks
only.
Replication/Redundancy
LRS- Local Redundant Storage – 3 Copies in single DC
ZRS- Zonal Redundant Storage – 3 copies in different DCs in same region or different region
GRS- Geo Redundant Storage – 6 Copies . 3 in LRS and 3 in DCs in different geo regions. Geo-
redundant storage (GRS) replicates your data to a secondary region that is hundreds of miles
away from the primary region. If your storage account has GRS enabled, then your data is
durable even in the case of a complete regional outage or a disaster in which the primary
region is not recoverable.
RA GRS- Read Access Geo Redundant Storage – 6 copies same as GRS but 3 copies in
different geo DC also have read access. Read-access geo-redundant storage (RA-GRS)
maximizes availability for your storage account, by providing read-only access to the data in
the secondary location, in addition to the replication across two regions provided by GRS.
Storage Accounts- Performance tiers and Replication

Storage account endpoints
A storage account provides a unique namespace in Azure for your data. Every object that you store in Azure Storage
has an address that includes your unique account name. The combination of the account name and the Azure
Storage service endpoint forms the endpoints for your storage account.
For example, if your general-purpose storage account is named mystorageaccount, then the default endpoints for
that account are:
Blob storage: http://mystorageaccount.blob.core.windows.net
Table storage: http://mystorageaccount.table.core.windows.net
Queue storage: http://mystorageaccount.queue.core.windows.net
Azure Files: http://mystorageaccount.file.core.windows.net
Storage Account Endpoints
The URL for accessing an object in a storage account is constructed by appending the object's location in the storage account
to the endpoint. For example, a blob address might have this format:
http://mystorageaccount.blob.core.windows.net/mycontainer/myblob.

Blobs
Simple Interface to
store and retrieve
large amount of
unstructured data
like files, images,
videos, etc
Tables
No SQLDatastore
Used for storing
structured, non-
relational data.
Queues
A queue contains
many messages.
Decouple frontend
role from backend.
Disks
Persistent standard
and premium storage
for IaaS VM built on
PageBlobs .
Files
SMB 3.0 access to
azure storage. Map
to file share . Lift and
shift legacy apps.
Azure Import/Export
Ship data in hard disk drive to azure datacentre
AZCopy
High Performance data uploads , downloads
and copy
Azure Storage Offerings

Abstraction of Storage
• Blobs – Simple Interface to
store and retrieve files in
cloud.
• Data Sharing : Share
documents , picture, video ,
music etc.
• Big Data : Store rwa
data/logs
• Backups-Data and device
Backup
• Disks– Network mounted
durables disks for VMs in
Azure
• Data Sharing : Share documents
, picture, video , music etc.
• Mounted disks and VHDs
stored in Azure Blobs
• Move on-premised applications
to cloud
• Tables– Massively scalable and
extremely easy to use NoSQL
system that auto scales
• Store users information,
device information and any
type of metadata for your
service.
• Queues– Reliable messaging
system
• Decouple components/roles
• Web role to worker role
communication
• Allow roles to scale
independently
• Implement scheduling of
asynchronous tasks
• Building process/workflows

Accounts
Container
Tables
Queue
Blobs
Entities
Messages
https://<account>.blob.core.windows.net/<container>
https://<account>.table.core.windows.net/<table>
https://<account>.queue.core.windows.net/<queue>
Abstraction of Storage

Accessing Windows Storage Via Explorer
Cloud storage studio by Cerebrata
Azure Storage Explorer by Neudistic
CloudXplorer by ClumsyLeaf
CloudBerry Explorer by Cloudberry Lab

How the Azure Storage Used by Microsoft
Xbox: Uses Blobs, Tables and Queues for cloud game saves, Xbox Music, Xbox live
etc
Skype: Uses Blobs, Tables and Queues for Skype video messages and to keep
metadata to allow skype clients to connect with each other.
Bing: Uses Blobs, Tables and Queues to provide a near real time ingestion engine
that consume Twitter and Facebook feeds, indexes them which is the folded inti Bing
search
SkyDrive: Uses Blobs to store pictures , documents , video, files etc.

Blob Storage

Blob Storage
Simple Interface to store and retrieve large amount of
unstructured data like files, images, videos, etc .
A single blob can be hundreds of gigabytes in size, and
a single storage account can contain up to 200TB of
blobs.
Azure Storage supports three types of blobs:
Block blobs store text and binary data, up to about 4.7
TB. Block blobs are made up of blocks of data that can
be managed individually.
Append blobs are made up of blocks like block blobs,
but are optimized for append operations. Append
blobs are ideal for scenarios such as logging data from
virtual machines.
Page blobs store random access files up to 8 TB in size.
Page blobs store the virtual hard drive (VHD) files serve
as disks for Azure virtual machines

Blob Storage is designed for :
• Serving images or documents
directly to a browser.
• Storing files for distributed access.
• Streaming video and audio.
• Writing to log files.
• Storing data for backup and
restore, disaster recovery, and
archiving.
• Storing data for analysis by an on-
premises or Azure-hosted service.
Blob Storage

Blob Storage Resources
Blob storage offers three types of resources:
•The storage account- A storage account provides a unique
namespace in Azure for your data. Every object that you store in
Azure Storage has an address that includes your unique account
name. An account can have many Blob Containers
•A container in the storage account-A container organizes a set
of blobs, similar to a directory in a file system. A storage account
can include an unlimited number of containers, and a container
can store an unlimited number of blobs. Sharing policies are set
at the container level, where a container can be set to private or
to be publicly accessible
•A blob in a container

 The Hot access tier, which is optimized for frequent access of objects in the storage account. Accessing data in the
Hot tier is most cost-effective, while storage costs are somewhat higher.
 The Cool access tier, which is optimized for storing large amounts of data that is infrequently accessed and stored
for at least 30 days. Storing data in the Cool tier is more cost-effective, but accessing that data may be somewhat
more expensive than accessing data in the Hot tier.
 The Archive tier, which is available only for individual block blobs. The Archive tier is optimized for data that can
tolerate several hours of retrieval latency and will remain in the Archive tier for at least 180 days. The Archive tier
is the most cost-effective option for storing data, but accessing that data is more expensive than accessing data in
the Hot or Cool tiers.
Access tiers for block blob data

Archive Access tiers for block blob data

Table Storage

Table Storage
• Azure Table storage is a service that stores structured NoSQL data in the
cloud, providing a key/attribute store with a schemaless design.
• Because Table storage is schemaless it's easy to adapt your data as the
needs of your application evolve.
• Access to Table storage data is fast and cost-effective for many types of
applications, and is typically lower in cost than traditional SQL for
similar volumes of data

Table Storage Components
Storage Account – All access to Windows Azure Storage is done through a storage
account. This is the highest level of the namespace for accessing tables. An account
can have many Tables
Table: A table is a collection of entities. Tables don't enforce a schema on entities,
which means a single table can contain entities that have different sets of properties.
Entity: An entity is a set of properties, similar to a database row. An entity in Azure
Storage can be up to 1MB in size. An entity contains a set of properties. Each table
has two properties, “PartitionKey and RowKey”, which form the unique key for the
entity
Properties: A property is a name-value pair. An entity can hold up to 255 properties.
Each entity also has three system properties that specify a partition key, a row key,
and a timestamp.

PartitionKey – The first key property of every table.
The system uses this key to automatically distribute
and load balance the table’s entities over many
servers.
RowKey – A second key property for the table. This is
the unique ID of the entity within the partition it
belongs to. The PartitionKey combined with the
RowKey uniquely identifies an entity in a table.
Timestamp – Every entity has a version maintained by
the system. This is used for optimistic concurrency.
Table Storage Properties

How to Partition

Schema of Table

CosmoDB- NoSQL
No data is is born relational, it’s born
dirty and messy, in whatever shape or
structure it’s created.
NoSQL databases used by companies
such as Facebook, Google and
Amazon as they working with big data
problems and/or real time web
applications. This set of technology is
coined “Not Only SQL” since they
might support a SQL syntax for
retrieving data.
The Azure Tables are
extremely scalable
and cheap, but if you
start querying on
any other attributes
then you start
running into
problems.

NoSQL - How It Store the data

CosmoDB- Introduction
Azure Cosmos DB
is a globally
distributed,
multi-model
database service
that supports
document, key-
value, wide-
column, and
graph databases.

CosmoDB -Turnkey global distribution

The Database is hosted in US
region while the users are in
different regions . This add
latency and delay . In
application like stock market it is
big problem.
One possible solutions is
to use the read only copy
to different regions and
aloo the users to read it,
but this also lead to
inconsistency because of
sync dely.
Another solutions is
to use the eventual
commit where data
cannot be read until
it commit to all
database.
CosmoDB Consistency- A situational scenario

No single machine
bottleneck.
Server Side
partition
Management
GB to PB Storage
selectable
throughput
Automatic
Document
expiration
Pay by the hour ,
customized
throughput cost
Support for
request per
second or minute
CosmoDB Abstract

Turnkey global distribution
• Cosmos DB enables you to build highly
responsive and highly available
applications worldwide.
• Cosmos DB transparently replicates your
data wherever your users are, so your
users can interact with a replica of the data
nearest to them.
• Cosmos DB allows you to add or remove
any of the Azure regions to your Cosmos
account at any time, with a click of a
button.
Always On
Cosmos DB provides 99.999% high
availability for both reads and
writes. Cosmos DB also provides you
with the ability to programmatically
(or via Portal) invoke the regional
failover of your Cosmos account.
This capability helps you ensure that
while a Cosmos database may
automatically failover if there is a
regional disaster.
Guaranteed low latency worldwide
Cosmos DB guarantees less than 10-
ms latencies for both, reads and
(indexed) writes at the 99th
percentile, all around the world.
Cosmos DB is available in all Azure
regions worldwide, including 54+
regions in public cloud, Azure China
21Vianet, Azure Germany, Azure
Government, and Azure
Government for Department of
Defense (DoD).
CosmoDB Advantage

CosmoDB Components

It stores the data in JSON. It’s massively scalable but
fully query-able by all parts of JSON tree.
CosmoDB - Structure

The Azure portal has three explorers that can be
used to manipulate collection.
Document Explorer – It allows you to add, update
and delete documents from a collection.
Query Explorer - Allows you to select document
data from the collection using a SQL like syntax.
Script Explorer- Allows you to define JavaScript
objects to support advanced functionality
CosmoDB –Azure Portal Explorer

Azure Queue

It’s all about a collection that has two primary
methods: enqueue (to insert new items to the queue at
the bottom) and dequeue (to get the top most item from
the queue).
Storage QUEUE
Microsoft Azure Queues are a ready-to-use service
that loosely connects components or applications
through the cloud.
Azure Queues are a part of the Storage Service that
unites Blobs, Tables and Queues under one storage
account.
There are two queue technologies
available in Microsoft Azure: Azure
Queues and Service Bus Queues. The
latter is newer and has a focus on
publish/subscribe mechanisms (i.e. the
ability to receive messages without
having to poll the queue) and integration
with the WCF communication stack. On
the other hand, Azure Queues are faster
and more applicable when storing more
than 5 GB of messages in a queue.

Azure Queues are queues located in the Microsoft cloud which you can use
for exchanging messages between components either in the cloud or on
premise.
A message typically represents a task created by someone (“producer”)
that has to be processed by someone else (“consumer”).
Each message has a small body and some attributes, such as time-to-live,
which you can use for configuring your service. As Azure ensures that a
dequeued message is invisible to other listeners, you could imagine many
producers and many consumers as well as one-to-one scenarios.
The main architectural benefit is loose coupling
Storage QUEUE- Architecture

There are two options available at this stage:
1. you can either connect with the storage account name and storage
account key
[everyone who gets the name/key credentials has full access to all operations in
your storage account. Those people can use that access to add and delete queues,
read and write messages, and even access blob and table storages]
2. With a temporary token.
[create a temporary token that is valid for a particular queue, and limited by
functionality and time. Obviously, this way you can give someone restricted access
(e.g. read-only) for a specific time period (e.g. 5 minutes from now), and maintain
much tighter control over who has access to your application.]
Storage QUEUE- Security

Storage QUEUE- Example

Azure FileShare

Azure FileShare -Advantages

Azure Storage Encryption

Storage Service Encryption (SSE)

Azure VM Disk Encryption

Index A- Sample Diagram

Index B- Azure Data Store

Thank You
Innovate, Collaborate & Accelerate

Azure data store with storage and cosmo db

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Azure data store with storage and cosmo db

Similar to Azure data store with storage and cosmo db (20)

More from Bipeen Sinha

More from Bipeen Sinha (7)

Recently uploaded

Recently uploaded (20)

Azure data store with storage and cosmo db