Azure Cosmos DB
Globally-distributed, multi-model database
service
Matías Quaranta - @ealsur
James Whittaker - @docjamesw
“We are reducing the world to data”
Developing planet-scale
apps comes with
planet-scale challenges
Scaling both throughput and storage based on global
demand
Delivering highly-responsive experiences
Ensuring an always-on system
Write accurate, globally distributed apps
Managing and versioning complex schemas
Balancing the needs for strong and eventual
consistency
Azure Cosmos DB
Globally-distributed, multi-model database
service
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Automatically replicate all your data around the world – across more
regions than Amazon and Google combined
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Multi-model + multi API
Use key-value, graph, and document with a schema-agnostic service that
doesn’t require any schema or secondary indexes
KEY-VALUE COLUMN-FAMILY
DOCUMENT GRAPH
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Multi-model + multi API
Elastic scale-out
Independently and elastically scale storage and throughput across
regions
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Multi-model + multi API
Elastic scale-out
Choice of consistency
Choose from five defined consistency levels for low latency and high
availability
Strong Bounded-stateless Session Consistent prefix Eventual
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Multi-model + multi API
Elastic scale-out
Choice of consistency
Serve <10 ms read and <15 ms write requests at the 99th percentile from
the nearest region while delivering data globally
Guaranteed single-digit latency
Read < 1 ms
Writes < 6 ms
Read < 10 ms
Writes < 15 ms
99%50%
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Multi-model + multi API
Elastic scale-out
Choice of consistency
Guaranteed single-digit latency
Only service with financially-backed SLAs for millisecond latency at the
99th percentile, 99.99% HA and guaranteed throughput and consistency
99.99%
HA
Throughput
Guaranteed
Consistency
Guaranteed
Enterprise-level SLAs
<10ms
Latency
99th
percentile
Introducing Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service
Global distribution
Multi-model + multi API
Elastic scale-out
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLAs
No Problem
No Schema
DocumentDB API
Resource hierarchy
Demo time!
Programming tools
API REST
Azure Cosmos DB
Geospatial query support
MongoDB API
Keep using the same tools
MongoDB wire protocol
Azure Cosmos DB
API for Mongo
Leverage Cosmos DB
Demo time!
Premium Tables API
(Preview)
Tables API in Azure Cosmos DB
Premium experience (low latency, well-defined consistency)
Globally Distributed
Secondary Indexes for user-defined queries
Millisecond latency, Guaranteed throughput
Azure Cosmos DB:
Table API
Azure Storage:
Standard Table API
Azure Storage SDKs
100% Backwards compatible, Seamless experience
Azure Cosmos DB:
Table API
Azure Storage SDKs
Coming Soon : Update for standard Tables, optimized for storage
Seamless migration
Demo time!
Graph API (Preview)
Model the real world
Relationship as first-class entities
Optimized for graph storage & traversal
Gremlin standard
Gremlin API in Azure Cosmos DB
Azure Cosmos DB:
Graph API
Globally distributed, elastically scalable, low
latency, auto-indexed service
Independently scalable graph engine (using
Tinkerpop framework)
Gremlin and SQL query languages
Native Graph Processing
Demo time!
Billed hourly similar to RU/sec
Get 10X more RU refills
Use only when needed
Guaranteed low latency for spiky workloads
Up-to 73% reduction in cost observed
Request Unit per minute (RU /m)
Predictable performance for unpredictable needs
Request Unit per minute (RU /m)
Predictable performance for unpredictable needs
Change Feed
Logging and change-tracking patterns
Security & Compliance
Enterprise grade security
Encryption at Rest By default Certified
Always encrypted at rest
and in motion
Data, index, backups, and
attachments encrypted
No impact on
performance, throughput
or availability
Transparent to your
application
ISO 27001, ISO 27018,
EUMC, HIPAA, PCI
SOC1, SOC2, FedRAMP, IRS
1075, IL2, HITRUST (coming
soon)
Powering global solutions
Field-tested by Microsoft’s planet-scale services and industry-leading enterprises apps
Globally-distributed
mission-critical apps
IoT
Accommodate bursts
of traffic and deliver
low-latency multiplayer
experiences
Personalization Retail and
e-commerce
Gaming
Guarantee uptime to
users worldwide with
high-availability and
low-latency
Scale instantly for
uncertain IoT workloads
without sacrificing
performance
Generate personalized
service through
low-latency and tunable
consistency settings
Support queries over
product catalogs,
traffic spikes, and rapidly
changing inventory
Key benefits
• Cosmos DB can scale elastically
without operational overhead of
MongoDB
• Perform fast queries over events to
deliver recommended services, safety
notices to vehicles
• Perform staged migration via
MongoDB APIs
Business need
• Need to ingest massive
volumes of diagnostic data
from vehicles and take real-
time actions as part of
connected car platform
• Management and operations of
database infrastructure to
handle exponential growth of
data
8TB
Vehicle Telemetry
250K
Lexus Cars
Toyota drives connected car push
forward with Azure Cosmos DB
Azure Cosmos DBAzure HDInsight
Storm
Azure Storage
(archival)
Business need
• Process Ms of retail transactions
per second in milliseconds in
inventory pipeline during peak
(“Black Friday”)
• Fast development cycles and
loosely coupled micro-services to
keep up with a competitive
marketplace
Key benefits
• Cosmos DB provides elastic scalability
from 1-10M requests per second
• Improved reliability, and faster order
processing times than previous OSS
solution
• Reduced development time and
operational overhead
Azure Cosmos DB
Azure Service Fabric
Pricing Service
Azure Cosmos DB
Azure Service Fabric
Inventory Service
E-commerce challenger eyes the top spot,
runs on Azure Cosmos DB
12TB
Provisioned
10M
Peak RPS
64
Databases
Business need
• Handle millions of players on
Day 1 due to popularity of the
TV series
• Match-making of players for
competitive and lag-free
experience
• Provide new content weekly,
and iterate on social
functionality
•
Key benefits
• Cosmos DB provides elastic scalability
for millions of users and flexible
schema to support social features and
gameplay
• Global distribution allows for low
latency for players spread worldwide
• Automatic indexing used to build real-
time leaderboards
Performance at massive scale allows
millions to play mobile game
Azure Traffic
Manager
Azure API Aps
(game backend)
Azure CDN
Azure Cosmos DB
Azure Functions
Azure Notification
Hubs (push
notifications)
Azure Storage
(game files)
1M
Peak Active
#1
iOS App Store
1B
Daily Queries
Capabilities Azure Cosmos DB AWS DynamoDB Google Cloud Spanner
Global distribution A single entity can span any number of
Azure regions (30+).
No No, currently single region. Multi-region
support is coming later in 2017.
Horizontal scalability of both storage and
throughput worldwide
Yes, across any number of Azure regions No No
Multi-homing APIs Yes No No
Guaranteed low read/write latency worldwide Yes, at 99th percentile worldwide No No
Consistency models 5 (Strong, Eventual, Bounded Staleness,
Session, Consistent-Prefix)
2 (Strong, Eventual) 1 (Strong)
Multi-model + multi-API Yes Yes No
Schema management Schema agnostic; no management required Index management required Schema and index management required
Automatic indexing Yes No No
Comprehensive SLAs Yes (latency, consistency, throughput,
high availability)
No No (availability only)
Azure Cosmos DB stacks up
Thank you!
askcosmosdb@microsoft.com
@AzureCosmosDB

Azure Cosmos DB - NET Conf AR 2017 - English

  • 1.
    Azure Cosmos DB Globally-distributed,multi-model database service Matías Quaranta - @ealsur
  • 2.
    James Whittaker -@docjamesw “We are reducing the world to data”
  • 3.
    Developing planet-scale apps comeswith planet-scale challenges Scaling both throughput and storage based on global demand Delivering highly-responsive experiences Ensuring an always-on system Write accurate, globally distributed apps Managing and versioning complex schemas Balancing the needs for strong and eventual consistency
  • 4.
    Azure Cosmos DB Globally-distributed,multi-model database service
  • 5.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Automatically replicate all your data around the world – across more regions than Amazon and Google combined
  • 6.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Multi-model + multi API Use key-value, graph, and document with a schema-agnostic service that doesn’t require any schema or secondary indexes KEY-VALUE COLUMN-FAMILY DOCUMENT GRAPH
  • 7.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Multi-model + multi API Elastic scale-out Independently and elastically scale storage and throughput across regions
  • 8.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Multi-model + multi API Elastic scale-out Choice of consistency Choose from five defined consistency levels for low latency and high availability Strong Bounded-stateless Session Consistent prefix Eventual
  • 9.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Multi-model + multi API Elastic scale-out Choice of consistency Serve <10 ms read and <15 ms write requests at the 99th percentile from the nearest region while delivering data globally Guaranteed single-digit latency Read < 1 ms Writes < 6 ms Read < 10 ms Writes < 15 ms 99%50%
  • 10.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Multi-model + multi API Elastic scale-out Choice of consistency Guaranteed single-digit latency Only service with financially-backed SLAs for millisecond latency at the 99th percentile, 99.99% HA and guaranteed throughput and consistency 99.99% HA Throughput Guaranteed Consistency Guaranteed Enterprise-level SLAs <10ms Latency 99th percentile
  • 11.
    Introducing Azure CosmosDB A globally distributed, massively scalable, multi-model database service Global distribution Multi-model + multi API Elastic scale-out Choice of consistency Guaranteed single-digit latency Enterprise-level SLAs
  • 12.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
    Keep using thesame tools MongoDB wire protocol Azure Cosmos DB API for Mongo
  • 21.
  • 22.
  • 23.
  • 24.
    Tables API inAzure Cosmos DB Premium experience (low latency, well-defined consistency) Globally Distributed Secondary Indexes for user-defined queries Millisecond latency, Guaranteed throughput Azure Cosmos DB: Table API Azure Storage: Standard Table API Azure Storage SDKs 100% Backwards compatible, Seamless experience Azure Cosmos DB: Table API Azure Storage SDKs Coming Soon : Update for standard Tables, optimized for storage Seamless migration
  • 25.
  • 26.
  • 27.
    Model the realworld Relationship as first-class entities Optimized for graph storage & traversal Gremlin standard Gremlin API in Azure Cosmos DB Azure Cosmos DB: Graph API
  • 28.
    Globally distributed, elasticallyscalable, low latency, auto-indexed service Independently scalable graph engine (using Tinkerpop framework) Gremlin and SQL query languages Native Graph Processing
  • 29.
  • 30.
    Billed hourly similarto RU/sec Get 10X more RU refills Use only when needed Guaranteed low latency for spiky workloads Up-to 73% reduction in cost observed Request Unit per minute (RU /m) Predictable performance for unpredictable needs
  • 31.
    Request Unit perminute (RU /m) Predictable performance for unpredictable needs
  • 32.
    Change Feed Logging andchange-tracking patterns
  • 33.
    Security & Compliance Enterprisegrade security Encryption at Rest By default Certified Always encrypted at rest and in motion Data, index, backups, and attachments encrypted No impact on performance, throughput or availability Transparent to your application ISO 27001, ISO 27018, EUMC, HIPAA, PCI SOC1, SOC2, FedRAMP, IRS 1075, IL2, HITRUST (coming soon)
  • 34.
    Powering global solutions Field-testedby Microsoft’s planet-scale services and industry-leading enterprises apps Globally-distributed mission-critical apps IoT Accommodate bursts of traffic and deliver low-latency multiplayer experiences Personalization Retail and e-commerce Gaming Guarantee uptime to users worldwide with high-availability and low-latency Scale instantly for uncertain IoT workloads without sacrificing performance Generate personalized service through low-latency and tunable consistency settings Support queries over product catalogs, traffic spikes, and rapidly changing inventory
  • 35.
    Key benefits • CosmosDB can scale elastically without operational overhead of MongoDB • Perform fast queries over events to deliver recommended services, safety notices to vehicles • Perform staged migration via MongoDB APIs Business need • Need to ingest massive volumes of diagnostic data from vehicles and take real- time actions as part of connected car platform • Management and operations of database infrastructure to handle exponential growth of data 8TB Vehicle Telemetry 250K Lexus Cars Toyota drives connected car push forward with Azure Cosmos DB Azure Cosmos DBAzure HDInsight Storm Azure Storage (archival)
  • 36.
    Business need • ProcessMs of retail transactions per second in milliseconds in inventory pipeline during peak (“Black Friday”) • Fast development cycles and loosely coupled micro-services to keep up with a competitive marketplace Key benefits • Cosmos DB provides elastic scalability from 1-10M requests per second • Improved reliability, and faster order processing times than previous OSS solution • Reduced development time and operational overhead Azure Cosmos DB Azure Service Fabric Pricing Service Azure Cosmos DB Azure Service Fabric Inventory Service E-commerce challenger eyes the top spot, runs on Azure Cosmos DB 12TB Provisioned 10M Peak RPS 64 Databases
  • 37.
    Business need • Handlemillions of players on Day 1 due to popularity of the TV series • Match-making of players for competitive and lag-free experience • Provide new content weekly, and iterate on social functionality • Key benefits • Cosmos DB provides elastic scalability for millions of users and flexible schema to support social features and gameplay • Global distribution allows for low latency for players spread worldwide • Automatic indexing used to build real- time leaderboards Performance at massive scale allows millions to play mobile game Azure Traffic Manager Azure API Aps (game backend) Azure CDN Azure Cosmos DB Azure Functions Azure Notification Hubs (push notifications) Azure Storage (game files) 1M Peak Active #1 iOS App Store 1B Daily Queries
  • 38.
    Capabilities Azure CosmosDB AWS DynamoDB Google Cloud Spanner Global distribution A single entity can span any number of Azure regions (30+). No No, currently single region. Multi-region support is coming later in 2017. Horizontal scalability of both storage and throughput worldwide Yes, across any number of Azure regions No No Multi-homing APIs Yes No No Guaranteed low read/write latency worldwide Yes, at 99th percentile worldwide No No Consistency models 5 (Strong, Eventual, Bounded Staleness, Session, Consistent-Prefix) 2 (Strong, Eventual) 1 (Strong) Multi-model + multi-API Yes Yes No Schema management Schema agnostic; no management required Index management required Schema and index management required Automatic indexing Yes No No Comprehensive SLAs Yes (latency, consistency, throughput, high availability) No No (availability only) Azure Cosmos DB stacks up
  • 39.

Editor's Notes

  • #3 View: https://www.youtube.com/watch?v=ewKfzZi9kyU
  • #6 Worldwide presence Automatic multi-region replication Multi-homing APIs Manual and automatic failovers Latency, throughput, consistency, and availability guarantees
  • #7 First party and popular third-party OSS APIs Key-value, Document, Columnar, and Graph DocumentDB (SQL and JavaScript), MongoDB, Table, and Gremlin Supported across number of programming languages More APIs to be added
  • #8 Pay by the hour, only what you need No partition management, no limits Change throughput at any time, effective in seconds Scale from 100 to 100s of millions of requests/sec across any number of regions Automatically indexed SSD storage Global distribution across Azure regions Automatic expiration via TTL Now supports requests/min to handle spikes cost-effectively
  • #9 While most database services force you to choose between strong or eventual consistency, Azure Cosmos DB provides multiple well-defined, intuitive consistency choices – so you can select just the right one for your app.   Every globally distributed databases need to navigate the CAP theorem. This is a fact of life, how ever anyone else markets it. We take a unique approach to it by allowing our developers to define the consistency levels their application needs. With well-defined consistency model, you have the choice and control but also have predictability in the trade offs you are making. As a globally distributed database which is designed from the ground up, Cosmos DB has taken a unique approach to navigate the CAP theorem to provide well-defined behavior between the tradeoffs between latency/high availability and consistency.
  • #10 Globally distributed with reads and writes served from local region Write optimized, latch-free database engine designed for SSDs Synchronous and automatic indexing at sustained ingestion rates Single-digit millisecond latency at any scale
  • #11 Only service with financially-backed SLAs for millisecond latency at the 99th percentile, 99.99% HA and guaranteed throughput and consistency
  • #13 Multi-model nature is based on a schema-less storage
  • #14 <Explain BW-TREE indexing as lightly and clear as posible>
  • #17 Showcasing CosmosDB Data Explorer & Queries
  • #21 Migrate your MongoDB data to CosmosDB and keep using the same SDKs and apps; your applications will work just by changing your connection string
  • #22 Benefit from Azure CosmosDB’s global distribution and scaling features while using Mongo in your application
  • #23 Showcasing Mongo client with CosmosDB connection string
  • #25 Today, I have the privilege of announcing that Azure Cosmos DB will support Azure Tables API for a premium experience for Tables customers. You can now use Azure tables and get the same millisecond latency that Azure Cosmos DB offers and you can control the consistency level for your app. Your Azure Tables can automatically be distributed across any Azure region. With automatic indexing, you do not need secondary indexes and all of this is 100% backwards compatible. As a developer your experience is seamless – All it takes is to change a connection string. With support for Tables API, we are brining to you most of the top 10 asks. By GA, We will be bringing an update for existing Azure Tables and seamlessly migrating all of the current Azure Tables workloads to have the same functionality that we are announcing today with the simplicity and low cost of Azure Tables.
  • #26 Showcasing Azure Storage Tables SDK with CosmosDB connection string & Azure Portal Data Explorer
  • #28 Graph database provides a highly intuitive data model and allows us to to reflect how the world really operates. Graph databases and traversal allows us to find the relational connections between objects and data. In many scenarios that our customers use Azure Cosmos DB, one of the next steps often identified is the need to find these relationships and operate on them. They need to interact with the graph model. With Gremlin API support, we allow you to work with Azure Cosmos DB using the most popular graph language and standards out there.
  • #30 Showcasing Azure Portal Data Explorer Graph queries
  • #31 Azure Cosmos DB allows for provisioned throughput and our model reserves the throughput at a second granularity. But not all workloads are predictable. Our customers wanted predictable performance for their unpredictable workloads. Today, we announce the preview of RU/m. With RU/m you can spread your provisioned throughput across the whole minute without worrying about rate limiting for spiky workloads. This allows you to control your budget while maintaining low latency even in the event of bursts or spikes. We have customers who have been using the feature in private preview and we have observed upto 73% cost savings for some customers when compared to provisioning for their peak spikes.
  • #32 Azure Cosmos DB allows for provisioned throughput and our model reserves the throughput at a second granularity. But not all workloads are predictable. Our customers wanted predictable performance for their unpredictable workloads. Today, we announce the preview of RU/m. With RU/m you can spread your provisioned throughput across the whole minute without worrying about rate limiting for spiky workloads. This allows you to control your budget while maintaining low latency even in the event of bursts or spikes. We have customers who have been using the feature in private preview and we have observed upto 73% cost savings for some customers when compared to provisioning for their peak spikes.
  • #34 As a foundational Azure Service, Security and Compliance is an important aspect of Azure Cosmos DB. As I previously mentioned, several high value Microsoft services that have stringent compliance and security requirements are built on Azure Cosmos DB. All of the data is encrypted at rest and it is enabled by default. We made sure that we continue to provide the performance guarantees even with encryption enabled and this is transparent to your application. The service is HIPAA , PCI, ISO certified and other compliance certifications are in the process with all of them estimated to be completed by next month.