The document discusses a presentation by Radu Vunvulea about using Azure Cosmos DB to improve various solutions. It begins with an introduction and then provides an overview of what Azure Cosmos DB is, including its key features like global distribution, support for multiple data models, elastic scaling, choice of consistency levels, and latency and availability guarantees. The presentation then demonstrates how Azure Cosmos DB can be used to improve the performance and scalability of command management, command tracking, payload metadata storage, payload assignment, device topology storage, and payload delivery status aspects of a transport platform solution compared to previous implementations that used multiple Azure data services. It concludes by thanking the audience.
4. 16 – 17 November, SofiaISTACON.ORG
Purpose
Understand where and how Azure Cosmos
DB can improve our solutions
5. 16 – 17 November, SofiaISTACON.ORG
What is Azure Cosmos DB?
6. 16 – 17 November, SofiaISTACON.ORG
Azure Cosmos DB
Global distribution
Multi-model
Elastic scale-up
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLA
7. 16 – 17 November, SofiaISTACON.ORG
Azure Cosmos DB
Global distribution
Multi-model
Elastic scale-up
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLA
Key Value
Column
Family
Document
Graph
8. 16 – 17 November, SofiaISTACON.ORG
Azure Cosmos DB
Global distribution
Multi-model
Elastic scale-up
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLA
Cross
regions
Independ
and elastic
scale
9. 16 – 17 November, SofiaISTACON.ORG
Azure Cosmos DB
Global distribution
Multi-model
Elastic scale-up
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLA
Strong
Bounded Staleness
Session
Consistent Prefix
Eventual
10. 16 – 17 November, SofiaISTACON.ORG
Consistency levels
Strong
• Reads are
guaranteed to
return the
most recent
version of an
item.
Bounded
Staleness
• Consistent
Prefix. Reads
lag behind
writes by k
prefixes or t
interval
Session
• Consistent
Prefix.
Monotonic
reads,
monotonic
writes, read-
your-writes,
write-follows-
reads
Consistent
Prefix
• Updates
returned are
some prefix of
all the
updates, with
no gaps
Eventual
• Out of order
reads
11. 16 – 17 November, SofiaISTACON.ORG
Azure Cosmos DB
Global distribution
Multi-model
Elastic scale-up
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLA
Read <10ms
Write <15ms
99%
12. 16 – 17 November, SofiaISTACON.ORG
Azure Cosmos DB
Global distribution
Multi-model
Elastic scale-up
Choice of consistency
Guaranteed single-digit latency
Enterprise-level SLA
Guaranteed
Consistency
&Throughput
HA
99.99%
Latency
<10ms
15. Business Scenario
• Transport platform between
devices and business systems
• Device management
• Command handling
• Payload assignment
• Payload delivery
17. 16 – 17 November, SofiaISTACON.ORG
Command Management
Azure Table for each device
High throughput on R/W
Millions of commands per
second
Multiple storage accounts
18. 16 – 17 November, SofiaISTACON.ORG
Command Management
Table used to store commands
No more cross storage
partitioning
Reduce deployment complexity
Removed storage resolver
20. 16 – 17 November, SofiaISTACON.ORG
Command Tracking
Each command has multiple
states
High throughput on write
operations
Low latency
Timeout detection
Reporting capabilities
21. 16 – 17 November, SofiaISTACON.ORG
Command Tracking
Table used to store commands
Reduce the load on Azure SQL
Scale more easily
Timeout detection using Spark
Generate reports using
Hadoop and PowerBI
23. 16 – 17 November, SofiaISTACON.ORG
Payload Metadata
Multiple Azure Tables
High number of read
operations
Never delete a metadata
Replicate across Azure Region
Custom logic for replication
Risk of inconsistency
24. 16 – 17 November, SofiaISTACON.ORG
Payload Metadata
Table used to stored payload
metadata information
Replication across regions
Removed custom
syncronization layer
Reduce load on internal
Service Bus instances
26. 16 – 17 November, SofiaISTACON.ORG
Payload assignment
Real time calculation
Result caching is not allowed
Requiring computation power
Relies on indexes and virtual
tables
27. 16 – 17 November, SofiaISTACON.ORG
Payload assignment
Doesn’t make sense to migrate
SQL engine is optimized for
this kind of operations
Azure SQL can scale pretty
nice using partitioning
29. 16 – 17 November, SofiaISTACON.ORG
Device Topology
Documents inside MongoDB
There are arround 700.000 different
combinations
New devices appear every month
Synchronization cross region
Risk of inconsistency after a few
years
Writes operations in all nodes
30. 16 – 17 November, SofiaISTACON.ORG
Device Topology
Documents inside Cosmos DB
Remove the custom
synchronization layer
No more inconsistency risk
Support for future devices
31. 16 – 17 November, SofiaISTACON.ORG
Payload Delivery Status
32. 16 – 17 November, SofiaISTACON.ORG
Payload Delivery Status
One Azure Table per device
Multiple storage accounts
Low throughput
No synchronization cross region
Reporting capabilities (hot
discussions)
Storage account resolver
capabilities
33. 16 – 17 November, SofiaISTACON.ORG
Payload Delivery Status
Tables inside Cosmos DB
No need for multiple storage
accounts anymore
Reduce deployment complexity
Good reporting capabilities
Replicate all content to a central
location
34. 16 – 17 November, SofiaISTACON.ORG
Final thoughts