SlideShare a Scribd company logo
Scalar DL Technical Overview
1 Jan, 2021
1
© 2021Scalar, inc.
Confidential
Scalar DL: Tamper-Evident and Scalable Database System
2
© 2021Scalar, inc.
Confidential
Key Characteristics in Scalar DL
3
Tamper-evident Scalable Correct
Database-agnostic Cloud-agnostic
Byzantine faults can be detected
as long as N > f
Performance and availability
can be linearly scaled
No intermediate states are visible,
and data is always up-to-date
No dependency on
particular database products
No dependency on
particular public clouds
N: # of administrative domains in a
database system, f: # of faulty domains
Cassandra, Amazon DynamoDB, Azure
Cosmos DB, and JDBC-supported major
RDBMSs are supported.
AWS, Azure, EKS, and AKS are
supported.
ACID-compliant and strong consistent
database system.
DVi = DV / N・RF
(RF: Replication Factor << N)
© 2021Scalar, inc.
Confidential
Scalar DL System Architecture
4
Pattern 1
Ledger
Pattern 2
Pattern 3
Client SDK
Ledger
Ordering
Client SDK
Proofs
Ledger
Client SDK
Smart
Contract
(signed)
Private keys Public keys
Auditor
© 2021Scalar, inc.
Confidential
Scalar DL: Ledger
5
User (Client)
Java
Contract
Signature
Private Key
Scalar DL network
function invoke() {
if (accounts[0].data.balance < args.val) {
throw new Error(“not enough balance”);
}
accounts[0].data.balance -= args.val;
accounts[1].data.balance += args.val;
results = { … };
}
Payment Contract
Request :
(contract, args, sig)
Asset ID Age Data (before) Data (after) Sigs Func (ref) Args Hash
A 1 { } { balance = 100,
…}
charge { val = 100 }
B 1 { }
{ balance = 200,
…} charge { val = 200 }
A 2 { A: {balance = 100},
B: {balance = 200} …}
{ balance = 90,
…}
payment { val = 10 }
SN = Func (SN-1 , Args)
Deterministic & TE TE
If S0 is TE ⇒ SN is TE
Public Key
B 2 { A: {balance = 100},
B: {balance = 200} …}
{ balance = 210,
…} payment { val = 10 }
H(A1)
H(B1)
State
hash
chain
Tamper Evidence (TE) of ledger entries:
Arguments
{ accounts = [“A”, “B”], val = 10, …}
* includes other accounts data
Argument
© 2021Scalar, inc.
Confidential
Scalar DL: Auditor
6
Auditor
managed by
a different organization
Client
Execute
Asset Proof
Request Proof
Asset Proof
T
Ledger
Compare to
detect
tampering
Auditor manages the proofs of execution to recompute assets without trusting
the integrity of data given from Ledger (1 patented, 1 patent-pending)
© 2021Scalar, inc.
Confidential
Scalar DL: Ordering
7
7
Client SDK
Scalar DL
Ordering
Private key
Smart
Contract
(signed)
Extracts parallelism
by utilizing partial order property
of contracts execution
without violating determinism
(patented) Execute contracts independently
…
Compare to
detect
tampering
© 2021Scalar, inc.
Confidential
Key Differences between Public Blockchains and Scalar DL
8
Public Blockchains
(Ethereum …)
Scalar DL
• History diverges in normal cases
– It diverges even if no malicious
activities ⇒ Impossible to
guarantee finality
• History diverges only in abnormal cases
– If it diverges, it is caused by
malicious activities ⇒ Possible to
guarantee finality
IEEE ICDCS Workshop’16
© 2021Scalar, inc.
Confidential
Key Differences between Private Blockchains and Scalar DL
9
Private/Consortium Blockchains
(HL fabric, Tendermint…)
Scalar DL
• Execution and validation are coupled
⇒ Hard to Scale
• Execution and validation are decoupled
⇒ Scalable and Flexible
• Data is totally ordered
⇒ Hard to parallelize
• Data is partially ordered
⇒ Easy to parallelize
Sequential processing is required
Parallelizable
Should be
homogeneous
performance
© 2021Scalar, inc.
Confidential
Unique Features 1 : ACID Nested Contract Execution
• Multiple contracts can be executed atomically
10
Contract 1
Transaction
Tamper-evident ledger
Contract 2
Contract 3
Atomic
execution
© 2021Scalar, inc.
Confidential
Unique Features 2 : User-defined Function (patent-pending)
• Function is a business logic for creating deletable and easy to search
records
• Remedy the downsides of ledger structure and tamper-evidence property
11
Mutable database
Function
Contract
Transaction
Tamper-evident ledger
Atomic
execution
Business logic
Evidence that business
logic is executed
© 2021Scalar, inc.
Confidential
Benefits of User-defined Function
12
App
Not deletable
Not searchable flexibly
App
Not tamper-evident
Blockchain
DB
App DB Blockchain
Inconsistent
App
Scalar DL
log
log
Mutable records
Tamper-evident records
Tamper-evident
Deletable
Searchable flexibly
Consistent
© 2021Scalar, inc.
Confidential
Benchmark Results
• Each node: i3.4xlarge (16 vCPUs, 122 GB DRAM, 1900 GB NVMe SSD * 2
• Workload: Writing evidence of operations
13
Achieved 90 % scalability in
100-node cluster
(Compared to the Ideal TPS based on
the performance of 3-node cluster)
© 2021Scalar, inc.
Confidential
Preliminary Benchmark Results with Caliper
• Caliper for Scalar DL is supported (unofficially)
– https://github.com/scalar-labs/caliper (scalardl branch)
• Smallbank
– Fabric 1.4.1 (3 peers + 3 orderers), Scalar DL (3 C* + 3 Scalar DL nodes)
14
0
2
4
6
8
10
12
14
16
18
20
0 50 100 150 200 250 300 350
Average
Latency
(s)
Throughput (tps)
Fabric (3org1peer)
Fabric (1org3peer)
Scalar DLT
Fabric 1.4.1, EP=ALL
Fabric 1.4.1, EP=1/3
© 2021Scalar, inc.
Confidential
Verification Results
• Scalar DL has been heavily tested with Jepsen and our
destructive tools
– Note that Jepsen tests are created and conducted by Scalar
– It has passed both tests for a long time
– See https://github.com/scalar-labs/scalar-jepsen for more detail
• TLA+ formal verification is also passing
15
Jepsen
Passed
© 2021Scalar, inc.
Confidential
Use Cases : Where to use Scalar DL
16
• Data integrity is required
– Data can not be altered or deleted maliciously
• Scalability is required
– Many requests from many end-users or devices
• There is a main organization
– And there are a few auditors
• On-demand tamper detection is acceptable
Digital evidence
One of our focuses:

More Related Content

What's hot

ゲームエンジニアのためのデータベース設計
ゲームエンジニアのためのデータベース設計ゲームエンジニアのためのデータベース設計
ゲームエンジニアのためのデータベース設計sairoutine
 
次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208
次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208 次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208
次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208 Kazuhiro Mitsuhashi
 
Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DB
Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DBDistributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DB
Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DBYugabyteDB
 
Snowflake Architecture and Performance
Snowflake Architecture and PerformanceSnowflake Architecture and Performance
Snowflake Architecture and PerformanceMineaki Motohashi
 
DataProcessingInBuffettCode-20190213
DataProcessingInBuffettCode-20190213DataProcessingInBuffettCode-20190213
DataProcessingInBuffettCode-20190213Shu (shoe116)
 
How to be Successful with Scylla
How to be Successful with ScyllaHow to be Successful with Scylla
How to be Successful with ScyllaScyllaDB
 
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...NoSQLmatters
 
Cassandra vs. ScyllaDB: Evolutionary Differences
Cassandra vs. ScyllaDB: Evolutionary DifferencesCassandra vs. ScyllaDB: Evolutionary Differences
Cassandra vs. ScyllaDB: Evolutionary DifferencesScyllaDB
 
Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)Rick Branson
 
Real-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache PinotReal-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache PinotXiang Fu
 
Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -
Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -
Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -Yahoo!デベロッパーネットワーク
 
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...HostedbyConfluent
 
Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...
Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...
Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...オラクルエンジニア通信
 
ビッグデータ処理データベースの全体像と使い分け
2018年version
ビッグデータ処理データベースの全体像と使い分け
2018年versionビッグデータ処理データベースの全体像と使い分け
2018年version
ビッグデータ処理データベースの全体像と使い分け
2018年versionTetsutaro Watanabe
 
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion RecordsScylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion RecordsScyllaDB
 
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayDatadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayC4Media
 
Advanced Apache Spark Meetup Project Tungsten Nov 12 2015
Advanced Apache Spark Meetup Project Tungsten Nov 12 2015Advanced Apache Spark Meetup Project Tungsten Nov 12 2015
Advanced Apache Spark Meetup Project Tungsten Nov 12 2015Chris Fregly
 
【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...
【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...
【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...SolarisJP
 
Snowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleSnowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleAdam Doyle
 

What's hot (20)

ゲームエンジニアのためのデータベース設計
ゲームエンジニアのためのデータベース設計ゲームエンジニアのためのデータベース設計
ゲームエンジニアのためのデータベース設計
 
次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208
次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208 次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208
次世代データ基盤としてのSnowflakeの可能性 SnowDay 20211208
 
Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DB
Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DBDistributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DB
Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DB
 
Snowflake Architecture and Performance
Snowflake Architecture and PerformanceSnowflake Architecture and Performance
Snowflake Architecture and Performance
 
DataProcessingInBuffettCode-20190213
DataProcessingInBuffettCode-20190213DataProcessingInBuffettCode-20190213
DataProcessingInBuffettCode-20190213
 
How to be Successful with Scylla
How to be Successful with ScyllaHow to be Successful with Scylla
How to be Successful with Scylla
 
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
 
Cassandra vs. ScyllaDB: Evolutionary Differences
Cassandra vs. ScyllaDB: Evolutionary DifferencesCassandra vs. ScyllaDB: Evolutionary Differences
Cassandra vs. ScyllaDB: Evolutionary Differences
 
Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)
 
Real-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache PinotReal-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache Pinot
 
Real time data quality on Flink
Real time data quality on FlinkReal time data quality on Flink
Real time data quality on Flink
 
Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -
Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -
Apache Kafkaによるログ転送とパフォーマンスチューニング - Bonfire Backend #2 -
 
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
 
Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...
Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...
Oracle Database 11g,12cからのアップグレード対策とクラウド移行 (Oracle Cloudウェビナーシリーズ: 2021年7...
 
ビッグデータ処理データベースの全体像と使い分け
2018年version
ビッグデータ処理データベースの全体像と使い分け
2018年versionビッグデータ処理データベースの全体像と使い分け
2018年version
ビッグデータ処理データベースの全体像と使い分け
2018年version
 
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion RecordsScylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
 
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayDatadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
 
Advanced Apache Spark Meetup Project Tungsten Nov 12 2015
Advanced Apache Spark Meetup Project Tungsten Nov 12 2015Advanced Apache Spark Meetup Project Tungsten Nov 12 2015
Advanced Apache Spark Meetup Project Tungsten Nov 12 2015
 
【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...
【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...
【第二回 ゼロからはじめる Oracle Solaris 11】02 Solaris 11 を支える最強のファイルシステム ZFS ~ ZFS ファイルシ...
 
Snowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleSnowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at Scale
 

Similar to Scalar DL Technical Overview

Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012CLOUDIAN KK
 
Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk Eran Gampel
 
Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...
Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...
Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...IDERA Software
 
Day Of Cloud - Windows Azure Platform
Day Of Cloud - Windows Azure PlatformDay Of Cloud - Windows Azure Platform
Day Of Cloud - Windows Azure PlatformWade Wegner
 
Kerberos presentation
Kerberos presentationKerberos presentation
Kerberos presentationChris Geier
 
Technical Deck Delta Live Tables.pdf
Technical Deck Delta Live Tables.pdfTechnical Deck Delta Live Tables.pdf
Technical Deck Delta Live Tables.pdfIlham31574
 
Enterprise Architecture
Enterprise ArchitectureEnterprise Architecture
Enterprise ArchitectureRaman Kannan
 
Streaming Data Into Your Lakehouse With Frank Munz | Current 2022
Streaming Data Into Your Lakehouse With Frank Munz | Current 2022Streaming Data Into Your Lakehouse With Frank Munz | Current 2022
Streaming Data Into Your Lakehouse With Frank Munz | Current 2022HostedbyConfluent
 
SQL Server SQL Server
SQL Server SQL ServerSQL Server SQL Server
SQL Server SQL Serverwebhostingguy
 
SQL Server SQL Server
SQL Server SQL ServerSQL Server SQL Server
SQL Server SQL Serverwebhostingguy
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
Hoverboards, Jetpacks, Clusters and Flux Capacitors
Hoverboards, Jetpacks,  Clusters and Flux CapacitorsHoverboards, Jetpacks,  Clusters and Flux Capacitors
Hoverboards, Jetpacks, Clusters and Flux CapacitorsKubeAcademy
 
Running Enterprise Workloads in the Cloud
Running Enterprise Workloads in the CloudRunning Enterprise Workloads in the Cloud
Running Enterprise Workloads in the CloudDataWorks Summit
 
Azure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layerAzure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layerMicrosoft Tech Community
 
Building applications using sql azure
Building applications using sql azureBuilding applications using sql azure
Building applications using sql azurepedrojcj
 
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023Matt Houghton
 
Clustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmarkClustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmarkClustrix
 
Ralph Kemperdick – IT-Tage 2015 – Microsoft Azure als Datenplattform
Ralph Kemperdick – IT-Tage 2015 – Microsoft Azure als DatenplattformRalph Kemperdick – IT-Tage 2015 – Microsoft Azure als Datenplattform
Ralph Kemperdick – IT-Tage 2015 – Microsoft Azure als DatenplattformInformatik Aktuell
 

Similar to Scalar DL Technical Overview (20)

Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012Cloudian_Cassandra Summit 2012
Cloudian_Cassandra Summit 2012
 
Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk
 
Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...
Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...
Geek Sync | Linux, Containers, and SQL Server—Get Ready for Big Data Clusters...
 
Day Of Cloud - Windows Azure Platform
Day Of Cloud - Windows Azure PlatformDay Of Cloud - Windows Azure Platform
Day Of Cloud - Windows Azure Platform
 
Kerberos presentation
Kerberos presentationKerberos presentation
Kerberos presentation
 
Technical Deck Delta Live Tables.pdf
Technical Deck Delta Live Tables.pdfTechnical Deck Delta Live Tables.pdf
Technical Deck Delta Live Tables.pdf
 
Sql Server - Apresentação
Sql Server - ApresentaçãoSql Server - Apresentação
Sql Server - Apresentação
 
Enterprise Architecture
Enterprise ArchitectureEnterprise Architecture
Enterprise Architecture
 
Streaming Data Into Your Lakehouse With Frank Munz | Current 2022
Streaming Data Into Your Lakehouse With Frank Munz | Current 2022Streaming Data Into Your Lakehouse With Frank Munz | Current 2022
Streaming Data Into Your Lakehouse With Frank Munz | Current 2022
 
SQL Server SQL Server
SQL Server SQL ServerSQL Server SQL Server
SQL Server SQL Server
 
SQL Server SQL Server
SQL Server SQL ServerSQL Server SQL Server
SQL Server SQL Server
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Hoverboards, Jetpacks, Clusters and Flux Capacitors
Hoverboards, Jetpacks,  Clusters and Flux CapacitorsHoverboards, Jetpacks,  Clusters and Flux Capacitors
Hoverboards, Jetpacks, Clusters and Flux Capacitors
 
Running Enterprise Workloads in the Cloud
Running Enterprise Workloads in the CloudRunning Enterprise Workloads in the Cloud
Running Enterprise Workloads in the Cloud
 
Azure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layerAzure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layer
 
Building applications using sql azure
Building applications using sql azureBuilding applications using sql azure
Building applications using sql azure
 
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
 
Clustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmarkClustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmark
 
DBMS Practical File
DBMS Practical FileDBMS Practical File
DBMS Practical File
 
Ralph Kemperdick – IT-Tage 2015 – Microsoft Azure als Datenplattform
Ralph Kemperdick – IT-Tage 2015 – Microsoft Azure als DatenplattformRalph Kemperdick – IT-Tage 2015 – Microsoft Azure als Datenplattform
Ralph Kemperdick – IT-Tage 2015 – Microsoft Azure als Datenplattform
 

Recently uploaded

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2
 
Advanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowAdvanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowPeter Caitens
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems ApproachNeo4j
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandIES VE
 
GraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisGraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisNeo4j
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1KnowledgeSeed
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownloadvrstrong314
 
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfImplementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfVictor Lopez
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Krakówbim.edu.pl
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationHelp Desk Migration
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)Max Lee
 
Studiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareStudiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareinfo611746
 
iGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockiGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockSkilrock Technologies
 
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfMastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfmbmh111980
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with StrimziStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzisteffenkarlsson2
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAlluxio, Inc.
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfkalichargn70th171
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAlluxio, Inc.
 

Recently uploaded (20)

Corporate Management | Session 3 of 3 | Tendenci AMS
Corporate Management | Session 3 of 3 | Tendenci AMSCorporate Management | Session 3 of 3 | Tendenci AMS
Corporate Management | Session 3 of 3 | Tendenci AMS
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
Advanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowAdvanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should Know
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
GraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisGraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysis
 
Top Mobile App Development Companies 2024
Top Mobile App Development Companies 2024Top Mobile App Development Companies 2024
Top Mobile App Development Companies 2024
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
 
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfImplementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Kraków
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
 
Studiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareStudiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting software
 
iGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockiGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by Skilrock
 
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfMastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with StrimziStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in Michelangelo
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 

Scalar DL Technical Overview

  • 1. Scalar DL Technical Overview 1 Jan, 2021 1
  • 2. © 2021Scalar, inc. Confidential Scalar DL: Tamper-Evident and Scalable Database System 2
  • 3. © 2021Scalar, inc. Confidential Key Characteristics in Scalar DL 3 Tamper-evident Scalable Correct Database-agnostic Cloud-agnostic Byzantine faults can be detected as long as N > f Performance and availability can be linearly scaled No intermediate states are visible, and data is always up-to-date No dependency on particular database products No dependency on particular public clouds N: # of administrative domains in a database system, f: # of faulty domains Cassandra, Amazon DynamoDB, Azure Cosmos DB, and JDBC-supported major RDBMSs are supported. AWS, Azure, EKS, and AKS are supported. ACID-compliant and strong consistent database system. DVi = DV / N・RF (RF: Replication Factor << N)
  • 4. © 2021Scalar, inc. Confidential Scalar DL System Architecture 4 Pattern 1 Ledger Pattern 2 Pattern 3 Client SDK Ledger Ordering Client SDK Proofs Ledger Client SDK Smart Contract (signed) Private keys Public keys Auditor
  • 5. © 2021Scalar, inc. Confidential Scalar DL: Ledger 5 User (Client) Java Contract Signature Private Key Scalar DL network function invoke() { if (accounts[0].data.balance < args.val) { throw new Error(“not enough balance”); } accounts[0].data.balance -= args.val; accounts[1].data.balance += args.val; results = { … }; } Payment Contract Request : (contract, args, sig) Asset ID Age Data (before) Data (after) Sigs Func (ref) Args Hash A 1 { } { balance = 100, …} charge { val = 100 } B 1 { } { balance = 200, …} charge { val = 200 } A 2 { A: {balance = 100}, B: {balance = 200} …} { balance = 90, …} payment { val = 10 } SN = Func (SN-1 , Args) Deterministic & TE TE If S0 is TE ⇒ SN is TE Public Key B 2 { A: {balance = 100}, B: {balance = 200} …} { balance = 210, …} payment { val = 10 } H(A1) H(B1) State hash chain Tamper Evidence (TE) of ledger entries: Arguments { accounts = [“A”, “B”], val = 10, …} * includes other accounts data Argument
  • 6. © 2021Scalar, inc. Confidential Scalar DL: Auditor 6 Auditor managed by a different organization Client Execute Asset Proof Request Proof Asset Proof T Ledger Compare to detect tampering Auditor manages the proofs of execution to recompute assets without trusting the integrity of data given from Ledger (1 patented, 1 patent-pending)
  • 7. © 2021Scalar, inc. Confidential Scalar DL: Ordering 7 7 Client SDK Scalar DL Ordering Private key Smart Contract (signed) Extracts parallelism by utilizing partial order property of contracts execution without violating determinism (patented) Execute contracts independently … Compare to detect tampering
  • 8. © 2021Scalar, inc. Confidential Key Differences between Public Blockchains and Scalar DL 8 Public Blockchains (Ethereum …) Scalar DL • History diverges in normal cases – It diverges even if no malicious activities ⇒ Impossible to guarantee finality • History diverges only in abnormal cases – If it diverges, it is caused by malicious activities ⇒ Possible to guarantee finality IEEE ICDCS Workshop’16
  • 9. © 2021Scalar, inc. Confidential Key Differences between Private Blockchains and Scalar DL 9 Private/Consortium Blockchains (HL fabric, Tendermint…) Scalar DL • Execution and validation are coupled ⇒ Hard to Scale • Execution and validation are decoupled ⇒ Scalable and Flexible • Data is totally ordered ⇒ Hard to parallelize • Data is partially ordered ⇒ Easy to parallelize Sequential processing is required Parallelizable Should be homogeneous performance
  • 10. © 2021Scalar, inc. Confidential Unique Features 1 : ACID Nested Contract Execution • Multiple contracts can be executed atomically 10 Contract 1 Transaction Tamper-evident ledger Contract 2 Contract 3 Atomic execution
  • 11. © 2021Scalar, inc. Confidential Unique Features 2 : User-defined Function (patent-pending) • Function is a business logic for creating deletable and easy to search records • Remedy the downsides of ledger structure and tamper-evidence property 11 Mutable database Function Contract Transaction Tamper-evident ledger Atomic execution Business logic Evidence that business logic is executed
  • 12. © 2021Scalar, inc. Confidential Benefits of User-defined Function 12 App Not deletable Not searchable flexibly App Not tamper-evident Blockchain DB App DB Blockchain Inconsistent App Scalar DL log log Mutable records Tamper-evident records Tamper-evident Deletable Searchable flexibly Consistent
  • 13. © 2021Scalar, inc. Confidential Benchmark Results • Each node: i3.4xlarge (16 vCPUs, 122 GB DRAM, 1900 GB NVMe SSD * 2 • Workload: Writing evidence of operations 13 Achieved 90 % scalability in 100-node cluster (Compared to the Ideal TPS based on the performance of 3-node cluster)
  • 14. © 2021Scalar, inc. Confidential Preliminary Benchmark Results with Caliper • Caliper for Scalar DL is supported (unofficially) – https://github.com/scalar-labs/caliper (scalardl branch) • Smallbank – Fabric 1.4.1 (3 peers + 3 orderers), Scalar DL (3 C* + 3 Scalar DL nodes) 14 0 2 4 6 8 10 12 14 16 18 20 0 50 100 150 200 250 300 350 Average Latency (s) Throughput (tps) Fabric (3org1peer) Fabric (1org3peer) Scalar DLT Fabric 1.4.1, EP=ALL Fabric 1.4.1, EP=1/3
  • 15. © 2021Scalar, inc. Confidential Verification Results • Scalar DL has been heavily tested with Jepsen and our destructive tools – Note that Jepsen tests are created and conducted by Scalar – It has passed both tests for a long time – See https://github.com/scalar-labs/scalar-jepsen for more detail • TLA+ formal verification is also passing 15 Jepsen Passed
  • 16. © 2021Scalar, inc. Confidential Use Cases : Where to use Scalar DL 16 • Data integrity is required – Data can not be altered or deleted maliciously • Scalability is required – Many requests from many end-users or devices • There is a main organization – And there are a few auditors • On-demand tamper detection is acceptable Digital evidence One of our focuses: