SlideShare a Scribd company logo
Architecting petabyte-scale analytics by
scaling out Postgres on Azure with Citus
Alicja Kucharczyk
EMEA Global Blackbelt OSS Data Tech Specialist
DevOps Meetup Zürich, 2020-03-04
Nothing
Compares To
VACUUM/The
Ballad of Bloat
The naming
thing
Hyperscale
(Citus)
Hyperscale (Citus)
• Open source extension
• Pure Postgres, not a fork
• Turns Postgres into distributed, sharded database
• All the benefits of Postgres, without worry about
scale
Hyperscale (Citus)
• Open source extension
• Pure Postgres, not a fork
• Turns Postgres into distributed, sharded database
• All the benefits of Postgres, without worry about
scale
Why Microsoft
likes Hyperscale
(Citus)?
How do you know if the next update
to your software is ready for
hundreds of millions of customers?
Internal RQV analytics dashboard
RQV analytics dashboard is a critical tool
for Windows engineers, program managers,
and execs.
The short story
Min Wei, Principal Engineer at
Microsoft
discovered the open source Citus
extension to Postgres by listening
to a recorded conference talk on
his drive home
Impressed with the early results,
he transitioned the project from a
proof of concept into an official
project.
A few months later Microsoft had
acquired Citus Data.
Measuring
the quality of
Windows
“Release Quality View” (RQV) dashboard
tracks 20,000 diagnostic and quality metrics
over 800M unique devices monthly
supports over 6 million queries per day
hundreds of concurrent users
1000s of monthly active users
100s of dashboard pages
Production
database
cluster
2816 Cores, 18TB DRAM,
1PB Azure Premium Storage,
Multi-PB Azure Blob Storage - for the staging
queue and raw Windows event data
• 2 Physical clusters behind a query router (Azure
Web Service and Azure Redis Service)
• Ingest and delete ~5TB data per day
• P75 query latency ~90ms/200ms (response times
for 75 percent of queries are less than 200
milliseconds)
• Support long running queries up to 4 mins.
• Support batch scheduled jobs that can run up for
2hours
Run Anywhere
On-Premises
In the Cloud - Azure
Database for
PostgreSQL
M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY
Azure Database for PostgreSQL is available in
two deployment options
Single Server
Fully-managed, single-nodePostgreSQL
Example use cases
• Apps with JSON, geospatial support, or full-text search
• Transactional and operational analytics workloads
• Cloud-native apps built with modern frameworks
Hyperscale (Citus)
High-performancePostgres for scale out
Example use cases
• Scaling PostgreSQL multi-tenant, SaaS apps
• Real-time operational analytics
• Building high throughput transactional apps
Enterprise-ready,fully
managed community
PostgreSQL with built-in HA
and multi-layered security
We’re talking about
Hyperscale (Citus)
today
Shardyour Postgres database across multiple nodes
to give your application more memory, compute,
and disk storage
Easilyadd worker nodes to achieve horizontalscale
Scale up to 100s of nodes
Scale horizontally across hundreds of cores with Hyperscale (Citus)
Select from table Coordinator
Table metadata
Select from table_1001
Select from table_1003
Select from table_1002
Select from table_1004
Data node N
Data node 2
Data node 1
Table_1001
Table_1003
Table_1002
Table_1004
Each node PostgreSQL with Citus installed
1 shard = 1 PostgreSQL table
Sharding data across multiple nodes
Terminology
Coordinator – Stores Metadata. Node which application connects to.
Worker / Data nodes – Nodes which store data in form of shards.
Sharding – Process of dividing data among nodes.
Shards – A partition of the data containing a subset of rows.
Co-location
Co-location based on data-type of the distribution column. Not the name of the
column.
Co-location handles
Joins
Foreign keys/ Primary keys
Rollups
Others in future slides…
Co-located join
• APPLICATION
SELECT
FROM
WHERE
AND
count(*)
ads JOIN campaigns ON
ads.company_id = campaigns.company_id
ads.designer_name = ‘Isaac’
campaigns.company_id = ‘Elly Co’ ;
METADATA
COORDINATOR NODE
WORKER NODES
W1
W2
W3 … Wn
SELECT…
FROM
ads_1001,
campaigns_2001
…
It’s logical to place shards containing related rows of related tables together on the same nodes
Join queries between related rows can reduce the amount of data sent over the network
Effectively manage
data scale out
Shard rebalancer redistributes shards across
old and new worker nodes for balanced data
scale out without any downtime.
Shard rebalancer will recommend rebalance
when shards can be placed more evenly
For more control, use tenant isolation to easily
allocate dedicated to specific tenants with
greater needs
APPLICATION
BEGIN;
UPDATE
SET
WHERE
UPDATE
SET
WHERE
COMMIT;
campaigns
feedback = ‘relevance’
company_type = ‘platinum’ ;
ads
feedback = ‘relevance’
company_type = ‘platinum’ ;
METADATA
COORDINATOR NODE
W1
W2
W3 … Wn
BEGIN …
assign_Scaled-out_
transaction_id…
UPDATE campaigns_2009
…
COMMITPREPARED …
BEGIN …
assign_Scaled-out_
transaction_id…
UPDATE campaigns_2001
…
COMMITPREPARED …
BEGIN …
assign_Scaled-out_
transaction_id…
UPDATE campaigns_2017
…
COMMITPREPARED …
Scaled-out transaction
Hyperscale (Citus) leverages built-in 2PC protocol to prepare transactions via a
coordinator node
Once worker nodes commit to transactions, release their locks, and send
acknowledgements, the coordinator node completes the scaled-out transaction
WORKER NODES
Table Classification
3 Table Types
• Distributed Tables
• Reference Tables
• Local Tables
Distributed Tables
Definition:
• Tables that are sharded.
Classification:
• Large tables (>10GB) – shard on same key (may require addition of shard key)
• All tables are be co-located
• Enables localized and fast joins on workers
• Ex: transactions, events etc
SELECT create_distributed_table(table_name, column_name);
Definition:
• Replicated to all the nodes (extra latency)
Classification:
• Small tables < 10GB
• Efficient joins with distributed tables
• Cannot have sharding dimension
• Ex: countries, categories
SELECT create_reference_table(table_name);
Reference Tables
• Plain Postgres tables on the coordinator node.
• Admin Tables that don’t interact with main tables
• Separate micro-service that doesn’t need sharding
Local Tables
M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY
Hyperscale (Citus): Customer view
Application
PostgreSQL
client
Coordinator
w/ public IP
Worker node 0,
no public IP
Worker node 1,
no public IP
Server group
M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY
Hyperscale (Citus): High availability
Application
PostgreSQL
client
Coordinator
w/ public IP
Worker node 0,
no public IP
Worker node 1,
no public IP
AZ[0]
Coordinator’s
standby
Worker node 0’s
standby
Worker node 1’s
standby
AZ[1]
Postgres sync replication
Postgres sync replication
Postgres sync replication
M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY
Features: High availability (HA)
 Standby nodes for each primary node in Hyperscale (Citus)
 Standby nodes are created in another AZ selected by service
 Synchronous Postgres replication
 Transparent for apps: Same connection string after failover
 Detection, failover, new standby creation
 Detection: Up to 150 seconds (five 30 sec probes)
 Failover: Up to 90 seconds
 Total downtime: Up to 240 seconds
 New standby creation: Up to 1 hour
M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY
Backup and restore
 Fully automated backup
 Enabled on each node
 Stored for 35 days
 Deleted server
 Backup is taken as a part of dropping the server and only this last backup is preserved
 Restore
 Can restore to a date stamp with 5-minute increment
 Need to open a support ticket to request PITR
Want to learn more?
http://tiny.cc/80lljz - Hyperscale
http://tiny.cc/n2lljz - ora2pg
Warsaw
Prague
Stuttgart
Geneva
Munich
Cologne
Paris
London
Amsterdam
Madrid
Oslo
Milan
Rome
Istanbul
Thank you!

More Related Content

What's hot

Data Factoryの勘所・大事なところ
Data Factoryの勘所・大事なところData Factoryの勘所・大事なところ
Data Factoryの勘所・大事なところ
Tsubasa Yoshino
 
[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정
[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정
[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정
PgDay.Seoul
 
MySQL Multi-Source Replication for PL2016
MySQL Multi-Source Replication for PL2016MySQL Multi-Source Replication for PL2016
MySQL Multi-Source Replication for PL2016
Wagner Bianchi
 
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
StreamNative
 
MariaDB MaxScale
MariaDB MaxScaleMariaDB MaxScale
MariaDB MaxScale
MariaDB plc
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication Tutorial
Jean-François Gagné
 
From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...
From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...
From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...
confluent
 
Planning for Disaster Recovery (DR) with Galera Cluster
Planning for Disaster Recovery (DR) with Galera ClusterPlanning for Disaster Recovery (DR) with Galera Cluster
Planning for Disaster Recovery (DR) with Galera Cluster
Codership Oy - Creators of Galera Cluster
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
Chandler Huang
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
Mike Dirolf
 
PostGreSQL Performance Tuning
PostGreSQL Performance TuningPostGreSQL Performance Tuning
PostGreSQL Performance Tuning
Maven Logix
 
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Databricks
 
Postgres Performance for Humans
Postgres Performance for HumansPostgres Performance for Humans
Postgres Performance for Humans
Citus Data
 
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Keisuke Takahashi
 
Databricksを初めて使う人に向けて.pptx
Databricksを初めて使う人に向けて.pptxDatabricksを初めて使う人に向けて.pptx
Databricksを初めて使う人に向けて.pptx
otato
 
Elasticsearch
ElasticsearchElasticsearch
Elasticsearch
Shagun Rathore
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
Altinity Ltd
 
MariaDB 마이그레이션 - 네오클로바
MariaDB 마이그레이션 - 네오클로바MariaDB 마이그레이션 - 네오클로바
MariaDB 마이그레이션 - 네오클로바
NeoClova
 
PostgreSQL: Advanced indexing
PostgreSQL: Advanced indexingPostgreSQL: Advanced indexing
PostgreSQL: Advanced indexing
Hans-Jürgen Schönig
 
Introduction à ElasticSearch
Introduction à ElasticSearchIntroduction à ElasticSearch
Introduction à ElasticSearchFadel Chafai
 

What's hot (20)

Data Factoryの勘所・大事なところ
Data Factoryの勘所・大事なところData Factoryの勘所・大事なところ
Data Factoryの勘所・大事なところ
 
[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정
[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정
[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정
 
MySQL Multi-Source Replication for PL2016
MySQL Multi-Source Replication for PL2016MySQL Multi-Source Replication for PL2016
MySQL Multi-Source Replication for PL2016
 
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
 
MariaDB MaxScale
MariaDB MaxScaleMariaDB MaxScale
MariaDB MaxScale
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication Tutorial
 
From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...
From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...
From Postgres to Event-Driven: using docker-compose to build CDC pipelines in...
 
Planning for Disaster Recovery (DR) with Galera Cluster
Planning for Disaster Recovery (DR) with Galera ClusterPlanning for Disaster Recovery (DR) with Galera Cluster
Planning for Disaster Recovery (DR) with Galera Cluster
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
PostGreSQL Performance Tuning
PostGreSQL Performance TuningPostGreSQL Performance Tuning
PostGreSQL Performance Tuning
 
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...
 
Postgres Performance for Humans
Postgres Performance for HumansPostgres Performance for Humans
Postgres Performance for Humans
 
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
Azure Database for PostgreSQL 入門 (PostgreSQL Conference Japan 2021)
 
Databricksを初めて使う人に向けて.pptx
Databricksを初めて使う人に向けて.pptxDatabricksを初めて使う人に向けて.pptx
Databricksを初めて使う人に向けて.pptx
 
Elasticsearch
ElasticsearchElasticsearch
Elasticsearch
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
 
MariaDB 마이그레이션 - 네오클로바
MariaDB 마이그레이션 - 네오클로바MariaDB 마이그레이션 - 네오클로바
MariaDB 마이그레이션 - 네오클로바
 
PostgreSQL: Advanced indexing
PostgreSQL: Advanced indexingPostgreSQL: Advanced indexing
PostgreSQL: Advanced indexing
 
Introduction à ElasticSearch
Introduction à ElasticSearchIntroduction à ElasticSearch
Introduction à ElasticSearch
 

Similar to Architecting peta-byte-scale analytics by scaling out Postgres on Azure with CItus | Devops Meetup Zurich | Alicja Kucharczyk

Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020
Riccardo Zamana
 
DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...
DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...
DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...
Docker, Inc.
 
Introduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web Services Introduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web Services
Amazon Web Services
 
AWS Cloud for HPC and Big Data
AWS Cloud for HPC and Big DataAWS Cloud for HPC and Big Data
AWS Cloud for HPC and Big Data
inside-BigData.com
 
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
confluent
 
MongoDB 4.0 새로운 기능 소개
MongoDB 4.0 새로운 기능 소개MongoDB 4.0 새로운 기능 소개
MongoDB 4.0 새로운 기능 소개
Ha-Yang(White) Moon
 
inmation Presentation
inmation Presentationinmation Presentation
inmation Presentation
inmation Software GmbH
 
HIPAS UCP HSP Openstack Sascha Oehl
HIPAS UCP HSP Openstack Sascha OehlHIPAS UCP HSP Openstack Sascha Oehl
HIPAS UCP HSP Openstack Sascha Oehl
Sascha Oehl
 
1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release
Jen Stirrup
 
Azure Databases for PostgreSQL, MySQL and MariaDB
Azure Databases for PostgreSQL, MySQL and MariaDBAzure Databases for PostgreSQL, MySQL and MariaDB
Azure Databases for PostgreSQL, MySQL and MariaDB
rockplace
 
Microsoft Azure
Microsoft AzureMicrosoft Azure
Microsoft Azure
David Chou
 
Keynote sp summit 2014 final
Keynote sp summit 2014  finalKeynote sp summit 2014  final
Keynote sp summit 2014 final
Amazon Web Services LATAM
 
Azure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layerAzure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layer
Microsoft Tech Community
 
MongoDB and the Internet of Things
MongoDB and the Internet of ThingsMongoDB and the Internet of Things
MongoDB and the Internet of Things
MongoDB
 
AWS Summit Atlanta Keynote
AWS Summit Atlanta KeynoteAWS Summit Atlanta Keynote
AWS Summit Atlanta Keynote
Kristana Kane
 
Red hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategyRed hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategy
Orgad Kimchi
 
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Saptak Sen
 
AWS Summit Berlin 2013 - Big Data Analytics
AWS Summit Berlin 2013 - Big Data AnalyticsAWS Summit Berlin 2013 - Big Data Analytics
AWS Summit Berlin 2013 - Big Data Analytics
AWS Germany
 
DS_2016_StreamAnalytix_real_time_streaming_analytics_platform
DS_2016_StreamAnalytix_real_time_streaming_analytics_platformDS_2016_StreamAnalytix_real_time_streaming_analytics_platform
DS_2016_StreamAnalytix_real_time_streaming_analytics_platform
Aditya Singh
 
Peek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapPeek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and Roadmap
Neo4j
 

Similar to Architecting peta-byte-scale analytics by scaling out Postgres on Azure with CItus | Devops Meetup Zurich | Alicja Kucharczyk (20)

Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020
 
DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...
DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...
DCEU 18: From Legacy Mainframe to the Cloud: The Finnish Railways Evolution w...
 
Introduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web Services Introduction to Cloud Computing with Amazon Web Services
Introduction to Cloud Computing with Amazon Web Services
 
AWS Cloud for HPC and Big Data
AWS Cloud for HPC and Big DataAWS Cloud for HPC and Big Data
AWS Cloud for HPC and Big Data
 
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
 
MongoDB 4.0 새로운 기능 소개
MongoDB 4.0 새로운 기능 소개MongoDB 4.0 새로운 기능 소개
MongoDB 4.0 새로운 기능 소개
 
inmation Presentation
inmation Presentationinmation Presentation
inmation Presentation
 
HIPAS UCP HSP Openstack Sascha Oehl
HIPAS UCP HSP Openstack Sascha OehlHIPAS UCP HSP Openstack Sascha Oehl
HIPAS UCP HSP Openstack Sascha Oehl
 
1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release
 
Azure Databases for PostgreSQL, MySQL and MariaDB
Azure Databases for PostgreSQL, MySQL and MariaDBAzure Databases for PostgreSQL, MySQL and MariaDB
Azure Databases for PostgreSQL, MySQL and MariaDB
 
Microsoft Azure
Microsoft AzureMicrosoft Azure
Microsoft Azure
 
Keynote sp summit 2014 final
Keynote sp summit 2014  finalKeynote sp summit 2014  final
Keynote sp summit 2014 final
 
Azure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layerAzure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layer
 
MongoDB and the Internet of Things
MongoDB and the Internet of ThingsMongoDB and the Internet of Things
MongoDB and the Internet of Things
 
AWS Summit Atlanta Keynote
AWS Summit Atlanta KeynoteAWS Summit Atlanta Keynote
AWS Summit Atlanta Keynote
 
Red hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategyRed hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategy
 
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
 
AWS Summit Berlin 2013 - Big Data Analytics
AWS Summit Berlin 2013 - Big Data AnalyticsAWS Summit Berlin 2013 - Big Data Analytics
AWS Summit Berlin 2013 - Big Data Analytics
 
DS_2016_StreamAnalytix_real_time_streaming_analytics_platform
DS_2016_StreamAnalytix_real_time_streaming_analytics_platformDS_2016_StreamAnalytix_real_time_streaming_analytics_platform
DS_2016_StreamAnalytix_real_time_streaming_analytics_platform
 
Peek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapPeek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and Roadmap
 

More from Citus Data

Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Citus Data
 
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
Citus Data
 
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Citus Data
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensWhats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Citus Data
 
When it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberWhen it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will Leinweber
Citus Data
 
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncAmazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Citus Data
 
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
Citus Data
 
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisDeep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Citus Data
 
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Citus Data
 
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncA story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
Citus Data
 
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Citus Data
 
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
Citus Data
 
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...
Citus Data
 
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberWhen it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
Citus Data
 
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
Citus Data
 
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Citus Data
 
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineHow to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
Citus Data
 
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberWhen it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
Citus Data
 
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoWhy PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Citus Data
 
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Citus Data
 

More from Citus Data (20)

Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
 
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
 
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensWhats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
 
When it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberWhen it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will Leinweber
 
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncAmazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
 
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
 
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisDeep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
 
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
 
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncA story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
 
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
 
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
 
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...
 
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberWhen it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
 
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
 
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
 
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineHow to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
 
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberWhen it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
 
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoWhy PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
 
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
 

Recently uploaded

みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 

Recently uploaded (20)

みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 

Architecting peta-byte-scale analytics by scaling out Postgres on Azure with CItus | Devops Meetup Zurich | Alicja Kucharczyk

  • 1. Architecting petabyte-scale analytics by scaling out Postgres on Azure with Citus Alicja Kucharczyk EMEA Global Blackbelt OSS Data Tech Specialist DevOps Meetup Zürich, 2020-03-04
  • 4. Hyperscale (Citus) • Open source extension • Pure Postgres, not a fork • Turns Postgres into distributed, sharded database • All the benefits of Postgres, without worry about scale
  • 5. Hyperscale (Citus) • Open source extension • Pure Postgres, not a fork • Turns Postgres into distributed, sharded database • All the benefits of Postgres, without worry about scale
  • 7. How do you know if the next update to your software is ready for hundreds of millions of customers?
  • 9. RQV analytics dashboard is a critical tool for Windows engineers, program managers, and execs.
  • 10. The short story Min Wei, Principal Engineer at Microsoft discovered the open source Citus extension to Postgres by listening to a recorded conference talk on his drive home Impressed with the early results, he transitioned the project from a proof of concept into an official project. A few months later Microsoft had acquired Citus Data.
  • 11. Measuring the quality of Windows “Release Quality View” (RQV) dashboard tracks 20,000 diagnostic and quality metrics over 800M unique devices monthly supports over 6 million queries per day hundreds of concurrent users 1000s of monthly active users 100s of dashboard pages
  • 12. Production database cluster 2816 Cores, 18TB DRAM, 1PB Azure Premium Storage, Multi-PB Azure Blob Storage - for the staging queue and raw Windows event data • 2 Physical clusters behind a query router (Azure Web Service and Azure Redis Service) • Ingest and delete ~5TB data per day • P75 query latency ~90ms/200ms (response times for 75 percent of queries are less than 200 milliseconds) • Support long running queries up to 4 mins. • Support batch scheduled jobs that can run up for 2hours
  • 13. Run Anywhere On-Premises In the Cloud - Azure Database for PostgreSQL
  • 14. M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY Azure Database for PostgreSQL is available in two deployment options Single Server Fully-managed, single-nodePostgreSQL Example use cases • Apps with JSON, geospatial support, or full-text search • Transactional and operational analytics workloads • Cloud-native apps built with modern frameworks Hyperscale (Citus) High-performancePostgres for scale out Example use cases • Scaling PostgreSQL multi-tenant, SaaS apps • Real-time operational analytics • Building high throughput transactional apps Enterprise-ready,fully managed community PostgreSQL with built-in HA and multi-layered security We’re talking about Hyperscale (Citus) today
  • 15. Shardyour Postgres database across multiple nodes to give your application more memory, compute, and disk storage Easilyadd worker nodes to achieve horizontalscale Scale up to 100s of nodes Scale horizontally across hundreds of cores with Hyperscale (Citus) Select from table Coordinator Table metadata Select from table_1001 Select from table_1003 Select from table_1002 Select from table_1004 Data node N Data node 2 Data node 1 Table_1001 Table_1003 Table_1002 Table_1004 Each node PostgreSQL with Citus installed 1 shard = 1 PostgreSQL table Sharding data across multiple nodes
  • 16. Terminology Coordinator – Stores Metadata. Node which application connects to. Worker / Data nodes – Nodes which store data in form of shards. Sharding – Process of dividing data among nodes. Shards – A partition of the data containing a subset of rows.
  • 17. Co-location Co-location based on data-type of the distribution column. Not the name of the column.
  • 18. Co-location handles Joins Foreign keys/ Primary keys Rollups Others in future slides…
  • 19. Co-located join • APPLICATION SELECT FROM WHERE AND count(*) ads JOIN campaigns ON ads.company_id = campaigns.company_id ads.designer_name = ‘Isaac’ campaigns.company_id = ‘Elly Co’ ; METADATA COORDINATOR NODE WORKER NODES W1 W2 W3 … Wn SELECT… FROM ads_1001, campaigns_2001 … It’s logical to place shards containing related rows of related tables together on the same nodes Join queries between related rows can reduce the amount of data sent over the network
  • 20. Effectively manage data scale out Shard rebalancer redistributes shards across old and new worker nodes for balanced data scale out without any downtime. Shard rebalancer will recommend rebalance when shards can be placed more evenly For more control, use tenant isolation to easily allocate dedicated to specific tenants with greater needs
  • 21. APPLICATION BEGIN; UPDATE SET WHERE UPDATE SET WHERE COMMIT; campaigns feedback = ‘relevance’ company_type = ‘platinum’ ; ads feedback = ‘relevance’ company_type = ‘platinum’ ; METADATA COORDINATOR NODE W1 W2 W3 … Wn BEGIN … assign_Scaled-out_ transaction_id… UPDATE campaigns_2009 … COMMITPREPARED … BEGIN … assign_Scaled-out_ transaction_id… UPDATE campaigns_2001 … COMMITPREPARED … BEGIN … assign_Scaled-out_ transaction_id… UPDATE campaigns_2017 … COMMITPREPARED … Scaled-out transaction Hyperscale (Citus) leverages built-in 2PC protocol to prepare transactions via a coordinator node Once worker nodes commit to transactions, release their locks, and send acknowledgements, the coordinator node completes the scaled-out transaction WORKER NODES
  • 23. 3 Table Types • Distributed Tables • Reference Tables • Local Tables
  • 24. Distributed Tables Definition: • Tables that are sharded. Classification: • Large tables (>10GB) – shard on same key (may require addition of shard key) • All tables are be co-located • Enables localized and fast joins on workers • Ex: transactions, events etc SELECT create_distributed_table(table_name, column_name);
  • 25. Definition: • Replicated to all the nodes (extra latency) Classification: • Small tables < 10GB • Efficient joins with distributed tables • Cannot have sharding dimension • Ex: countries, categories SELECT create_reference_table(table_name); Reference Tables
  • 26. • Plain Postgres tables on the coordinator node. • Admin Tables that don’t interact with main tables • Separate micro-service that doesn’t need sharding Local Tables
  • 27. M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY Hyperscale (Citus): Customer view Application PostgreSQL client Coordinator w/ public IP Worker node 0, no public IP Worker node 1, no public IP Server group
  • 28. M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY Hyperscale (Citus): High availability Application PostgreSQL client Coordinator w/ public IP Worker node 0, no public IP Worker node 1, no public IP AZ[0] Coordinator’s standby Worker node 0’s standby Worker node 1’s standby AZ[1] Postgres sync replication Postgres sync replication Postgres sync replication
  • 29. M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY Features: High availability (HA)  Standby nodes for each primary node in Hyperscale (Citus)  Standby nodes are created in another AZ selected by service  Synchronous Postgres replication  Transparent for apps: Same connection string after failover  Detection, failover, new standby creation  Detection: Up to 150 seconds (five 30 sec probes)  Failover: Up to 90 seconds  Total downtime: Up to 240 seconds  New standby creation: Up to 1 hour
  • 30. M ICR O S O FT CO N FIDE N T IAL – IN T E R N AL O N LY Backup and restore  Fully automated backup  Enabled on each node  Stored for 35 days  Deleted server  Backup is taken as a part of dropping the server and only this last backup is preserved  Restore  Can restore to a date stamp with 5-minute increment  Need to open a support ticket to request PITR
  • 31. Want to learn more? http://tiny.cc/80lljz - Hyperscale http://tiny.cc/n2lljz - ora2pg Warsaw Prague Stuttgart Geneva Munich Cologne Paris London Amsterdam Madrid Oslo Milan Rome Istanbul