SlideShare a Scribd company logo
1 of 18
Download to read offline
Strongly Consistent Global Indexes for
Apache Phoenix
Kadir Ozdemir
September 2019
Why Phoenix at Salesforce?
Massive Data Scale w/
Familiar Interface
Trusted storage Consistent
Multi-cloud
Salesforce
Multi-tenancy
HDFS
HBase Server
(Da
Application Server HBase Region Servers
Phoenix
Server
Phoenix
Application
Phoenix Client
HBase Client
SQL
Table Scans/
Mutations
Table
Region
RPC
Secondary Indexing
ID Name City
1234 Ashley Seattle
2345 Kadir San Francisco
Primary Key Secondary Key
Secondary Indexing
ID Name City
1234 Ashley Seattle
2345 Kadir San Francisco
Primary KeyPrimary Key
ID Name City
1234 Ashley Seattle
2345 Kadir San Francisco
Primary Key
City ID Name
San Francisco 2345 Kadir
Seattle 12345 Ashley
Secondary Key
Data Table Index Table
Secondary Indexing - Update
ID Name City
1234 Ashley Seattle
Primary KeyPrimary Key
City ID Name
San Francisco 2345 Kadir
ID Name City
2345 Kadir San Francisco
City ID Name
Seattle 12345 Ashley
Data Table Index Table
Secondary Indexing - Update
ID Name City
1234 Ashley Seattle
Primary KeyPrimary Key
City ID Name
ID Name City
2345 Kadir San Francisco
City ID Name
Seattle 12345 Ashley
Data Table Index Table
Global Secondary Indexing - Update
ID Name City
1234 Ashley Seattle
Primary KeyPrimary Key
City ID Name
ID Name City
2345 Kadir Seattle
City ID Name
Seattle 1234 Ashley
Seattle 2345 Kadir
Data Table Index Table
Current Design Challenges
● Tries to make tables consistent at the write time by relying on client retries
○ May not handle correlated failures and may leave data table inconsistent with its indexes
● Needs external tools to detect inconsistencies and repair them
Design Objectives
● Secondary indexes should be always in sync with their data tables
● Strong consistency should not result in significant performance impact
● Strong consistency should not impact scalability significantly
Observations
● Data must be consistent at read time
○ An index table row can be repaired from the corresponding data table row at read time
● In HBase writes are fast
○ We can add extra write phase without severely impacting write performance
Strongly Consistent Design
Operation Strongly Consistent Design
Read
1. Read the index rows and check their status
2. The unverified rows repaired from the data table
Strongly Consistent Design
Operation Strongly Consistent Design
Read
1. Read the index rows and check their status
2. The unverified rows repaired from the data table
Write
1. Set the status of existing index rows unverified and write the new index
rows with the unverified status
2. Write the data table rows
3. Delete the existing index rows and set the status of new rows to verified
Strongly Consistent Design
Operation Strongly Consistent Design
Read
1. Read the index rows and check their status
2. The unverified rows repaired from the data table
Write
1. Set the status of existing index rows unverified and write the new index
rows with the unverified status
2. Write the data table rows
3. Delete the existing index rows and set the status of new rows to verified
Delete
1. Set the index table rows with the unverified status
2. Delete the data table rows
3. Delete index table rows
Correctness Without Concurrent Row Updates
● Missing index row is not possible
○ An index row is updated first before its data row
■ If the index update is failed then the data row update will not be attempted
○ An index row is deleted only after its data table row is deleted
● Verified index row implies existence of the corresponding data row
○ The status for an index row is set to verified only after the corresponding data row is written
○ The status for an index row is set to unverified before the corresponding data row is deleted
● Unverified index rows are not used for serving user queries
○ An unverified index row is repaired from its data row during scans
Correctness With Concurrent Row Updates
● The third phase is skipped for concurrent updates
○ Detect concurrent updates and leave them in the unverified state
● Use two phase row locking to detect concurrent updates on a data row
read the data
table
(phase 1) index
table update
(phase 2) update
the data table
phase 3 index
table update
Pending Rows
add remove
Performance Impact of Strong Consistency
● Setup: A data table with two indexes on a 10 node cluster
○ 1 billion large rows with random primary key
○ Top N queries on indexes where N is 50
● Less than 25% increase in write latency
○ Due to setting row status in phase 3
● No noticeable increase in read latency
○ The number of unverified rows due to pending updates on a given table region is limited by the
number of RPC threads and mutation batch size
Questions?

More Related Content

What's hot

Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
Chandler Huang
 
Apache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL DatabaseApache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL Database
DataWorks Summit
 

What's hot (20)

Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
 
File Format Benchmark - Avro, JSON, ORC & Parquet
File Format Benchmark - Avro, JSON, ORC & ParquetFile Format Benchmark - Avro, JSON, ORC & Parquet
File Format Benchmark - Avro, JSON, ORC & Parquet
 
Iceberg: a fast table format for S3
Iceberg: a fast table format for S3Iceberg: a fast table format for S3
Iceberg: a fast table format for S3
 
Iceberg: A modern table format for big data (Strata NY 2018)
Iceberg: A modern table format for big data (Strata NY 2018)Iceberg: A modern table format for big data (Strata NY 2018)
Iceberg: A modern table format for big data (Strata NY 2018)
 
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic DatasetsApache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
 
Optimizing Hive Queries
Optimizing Hive QueriesOptimizing Hive Queries
Optimizing Hive Queries
 
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBaseApache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
 
HBase Application Performance Improvement
HBase Application Performance ImprovementHBase Application Performance Improvement
HBase Application Performance Improvement
 
Introduction to Apache Kudu
Introduction to Apache KuduIntroduction to Apache Kudu
Introduction to Apache Kudu
 
Intro to HBase
Intro to HBaseIntro to HBase
Intro to HBase
 
Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsa...
Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsa...Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsa...
Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsa...
 
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
 
Apache Iceberg Presentation for the St. Louis Big Data IDEA
Apache Iceberg Presentation for the St. Louis Big Data IDEAApache Iceberg Presentation for the St. Louis Big Data IDEA
Apache Iceberg Presentation for the St. Louis Big Data IDEA
 
Making Apache Spark Better with Delta Lake
Making Apache Spark Better with Delta LakeMaking Apache Spark Better with Delta Lake
Making Apache Spark Better with Delta Lake
 
Parquet performance tuning: the missing guide
Parquet performance tuning: the missing guideParquet performance tuning: the missing guide
Parquet performance tuning: the missing guide
 
Hadoop World 2011: Advanced HBase Schema Design - Lars George, Cloudera
Hadoop World 2011: Advanced HBase Schema Design - Lars George, ClouderaHadoop World 2011: Advanced HBase Schema Design - Lars George, Cloudera
Hadoop World 2011: Advanced HBase Schema Design - Lars George, Cloudera
 
Dataflow with Apache NiFi
Dataflow with Apache NiFiDataflow with Apache NiFi
Dataflow with Apache NiFi
 
Processing Large Data with Apache Spark -- HasGeek
Processing Large Data with Apache Spark -- HasGeekProcessing Large Data with Apache Spark -- HasGeek
Processing Large Data with Apache Spark -- HasGeek
 
HDFS on Kubernetes—Lessons Learned with Kimoon Kim
HDFS on Kubernetes—Lessons Learned with Kimoon KimHDFS on Kubernetes—Lessons Learned with Kimoon Kim
HDFS on Kubernetes—Lessons Learned with Kimoon Kim
 
Apache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL DatabaseApache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL Database
 

Similar to Strongly Consistent Global Indexes for Apache Phoenix

Similar to Strongly Consistent Global Indexes for Apache Phoenix (20)

SQL
SQLSQL
SQL
 
12c Database new features
12c Database new features12c Database new features
12c Database new features
 
Galene - LinkedIn's Search Architecture: Presented by Diego Buthay & Sriram S...
Galene - LinkedIn's Search Architecture: Presented by Diego Buthay & Sriram S...Galene - LinkedIn's Search Architecture: Presented by Diego Buthay & Sriram S...
Galene - LinkedIn's Search Architecture: Presented by Diego Buthay & Sriram S...
 
Deploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWSDeploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWS
 
Query parameterization
Query parameterizationQuery parameterization
Query parameterization
 
Lsmw ppt in SAP ABAP
Lsmw ppt in SAP ABAPLsmw ppt in SAP ABAP
Lsmw ppt in SAP ABAP
 
Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...
Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...
Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...
 
Flink Forward SF 2017: Timo Walther - Table & SQL API – unified APIs for bat...
Flink Forward SF 2017: Timo Walther -  Table & SQL API – unified APIs for bat...Flink Forward SF 2017: Timo Walther -  Table & SQL API – unified APIs for bat...
Flink Forward SF 2017: Timo Walther - Table & SQL API – unified APIs for bat...
 
What's new in MariaDB TX 3.0
What's new in MariaDB TX 3.0What's new in MariaDB TX 3.0
What's new in MariaDB TX 3.0
 
Sql Server Query Parameterization
Sql Server Query ParameterizationSql Server Query Parameterization
Sql Server Query Parameterization
 
Database Performance
Database PerformanceDatabase Performance
Database Performance
 
Apache HAWQ Architecture
Apache HAWQ ArchitectureApache HAWQ Architecture
Apache HAWQ Architecture
 
PostgreSQL - масштабирование в моде, Valentine Gogichashvili (Zalando SE)
PostgreSQL - масштабирование в моде, Valentine Gogichashvili (Zalando SE)PostgreSQL - масштабирование в моде, Valentine Gogichashvili (Zalando SE)
PostgreSQL - масштабирование в моде, Valentine Gogichashvili (Zalando SE)
 
2017 AWS DB Day | Amazon Redshift 소개 및 실습
2017 AWS DB Day | Amazon Redshift  소개 및 실습2017 AWS DB Day | Amazon Redshift  소개 및 실습
2017 AWS DB Day | Amazon Redshift 소개 및 실습
 
Amazon Redshift For Data Analysts
Amazon Redshift For Data AnalystsAmazon Redshift For Data Analysts
Amazon Redshift For Data Analysts
 
10 sql tips
10 sql tips10 sql tips
10 sql tips
 
SE2016 Java Roman Ugolnikov "Migration and source control for your DB"
SE2016 Java Roman Ugolnikov "Migration and source control for your DB"SE2016 Java Roman Ugolnikov "Migration and source control for your DB"
SE2016 Java Roman Ugolnikov "Migration and source control for your DB"
 
Roman Ugolnikov Migrationа and sourcecontrol for your db
Roman Ugolnikov Migrationа and sourcecontrol for your dbRoman Ugolnikov Migrationа and sourcecontrol for your db
Roman Ugolnikov Migrationа and sourcecontrol for your db
 
Structured streaming in Spark
Structured streaming in SparkStructured streaming in Spark
Structured streaming in Spark
 
Hpverticacertificationguide 150322232921-conversion-gate01
Hpverticacertificationguide 150322232921-conversion-gate01Hpverticacertificationguide 150322232921-conversion-gate01
Hpverticacertificationguide 150322232921-conversion-gate01
 

Recently uploaded

如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一
hwhqz6r1y
 
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一
w7jl3eyno
 
如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一
如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一
如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一
0uyfyq0q4
 
如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一
如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一
如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一
hwhqz6r1y
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertainty
RafigAliyev2
 
如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一
如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一
如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一
fztigerwe
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
Amil baba
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
pyhepag
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
great91
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
ju0dztxtn
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Valters Lauzums
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
ppy8zfkfm
 

Recently uploaded (20)

2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call
 
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证成绩单原版一比一
 
basics of data science with application areas.pdf
basics of data science with application areas.pdfbasics of data science with application areas.pdf
basics of data science with application areas.pdf
 
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一
如何办理澳洲悉尼大学毕业证(USYD毕业证书)学位证书成绩单原版一比一
 
如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一
如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一
如何办理滑铁卢大学毕业证(Waterloo毕业证)成绩单本科学位证原版一比一
 
如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一
如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一
如何办理新加坡国立大学毕业证(NUS毕业证)学位证成绩单原版一比一
 
123.docx. .
123.docx.                                 .123.docx.                                 .
123.docx. .
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertainty
 
Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)
 
Generative AI for Trailblazers_ Unlock the Future of AI.pdf
Generative AI for Trailblazers_ Unlock the Future of AI.pdfGenerative AI for Trailblazers_ Unlock the Future of AI.pdf
Generative AI for Trailblazers_ Unlock the Future of AI.pdf
 
2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting
 
如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一
如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一
如何办理哥伦比亚大学毕业证(Columbia毕业证)成绩单原版一比一
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
 
社内勉強会資料  Mamba - A new era or ephemeral
社内勉強会資料   Mamba - A new era or ephemeral社内勉強会資料   Mamba - A new era or ephemeral
社内勉強会資料  Mamba - A new era or ephemeral
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
 
Machine Learning for Accident Severity Prediction
Machine Learning for Accident Severity PredictionMachine Learning for Accident Severity Prediction
Machine Learning for Accident Severity Prediction
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
 

Strongly Consistent Global Indexes for Apache Phoenix

  • 1. Strongly Consistent Global Indexes for Apache Phoenix Kadir Ozdemir September 2019
  • 2. Why Phoenix at Salesforce? Massive Data Scale w/ Familiar Interface Trusted storage Consistent Multi-cloud Salesforce Multi-tenancy
  • 3. HDFS HBase Server (Da Application Server HBase Region Servers Phoenix Server Phoenix Application Phoenix Client HBase Client SQL Table Scans/ Mutations Table Region RPC
  • 4. Secondary Indexing ID Name City 1234 Ashley Seattle 2345 Kadir San Francisco Primary Key Secondary Key
  • 5. Secondary Indexing ID Name City 1234 Ashley Seattle 2345 Kadir San Francisco Primary KeyPrimary Key ID Name City 1234 Ashley Seattle 2345 Kadir San Francisco Primary Key City ID Name San Francisco 2345 Kadir Seattle 12345 Ashley Secondary Key Data Table Index Table
  • 6. Secondary Indexing - Update ID Name City 1234 Ashley Seattle Primary KeyPrimary Key City ID Name San Francisco 2345 Kadir ID Name City 2345 Kadir San Francisco City ID Name Seattle 12345 Ashley Data Table Index Table
  • 7. Secondary Indexing - Update ID Name City 1234 Ashley Seattle Primary KeyPrimary Key City ID Name ID Name City 2345 Kadir San Francisco City ID Name Seattle 12345 Ashley Data Table Index Table
  • 8. Global Secondary Indexing - Update ID Name City 1234 Ashley Seattle Primary KeyPrimary Key City ID Name ID Name City 2345 Kadir Seattle City ID Name Seattle 1234 Ashley Seattle 2345 Kadir Data Table Index Table
  • 9. Current Design Challenges ● Tries to make tables consistent at the write time by relying on client retries ○ May not handle correlated failures and may leave data table inconsistent with its indexes ● Needs external tools to detect inconsistencies and repair them
  • 10. Design Objectives ● Secondary indexes should be always in sync with their data tables ● Strong consistency should not result in significant performance impact ● Strong consistency should not impact scalability significantly
  • 11. Observations ● Data must be consistent at read time ○ An index table row can be repaired from the corresponding data table row at read time ● In HBase writes are fast ○ We can add extra write phase without severely impacting write performance
  • 12. Strongly Consistent Design Operation Strongly Consistent Design Read 1. Read the index rows and check their status 2. The unverified rows repaired from the data table
  • 13. Strongly Consistent Design Operation Strongly Consistent Design Read 1. Read the index rows and check their status 2. The unverified rows repaired from the data table Write 1. Set the status of existing index rows unverified and write the new index rows with the unverified status 2. Write the data table rows 3. Delete the existing index rows and set the status of new rows to verified
  • 14. Strongly Consistent Design Operation Strongly Consistent Design Read 1. Read the index rows and check their status 2. The unverified rows repaired from the data table Write 1. Set the status of existing index rows unverified and write the new index rows with the unverified status 2. Write the data table rows 3. Delete the existing index rows and set the status of new rows to verified Delete 1. Set the index table rows with the unverified status 2. Delete the data table rows 3. Delete index table rows
  • 15. Correctness Without Concurrent Row Updates ● Missing index row is not possible ○ An index row is updated first before its data row ■ If the index update is failed then the data row update will not be attempted ○ An index row is deleted only after its data table row is deleted ● Verified index row implies existence of the corresponding data row ○ The status for an index row is set to verified only after the corresponding data row is written ○ The status for an index row is set to unverified before the corresponding data row is deleted ● Unverified index rows are not used for serving user queries ○ An unverified index row is repaired from its data row during scans
  • 16. Correctness With Concurrent Row Updates ● The third phase is skipped for concurrent updates ○ Detect concurrent updates and leave them in the unverified state ● Use two phase row locking to detect concurrent updates on a data row read the data table (phase 1) index table update (phase 2) update the data table phase 3 index table update Pending Rows add remove
  • 17. Performance Impact of Strong Consistency ● Setup: A data table with two indexes on a 10 node cluster ○ 1 billion large rows with random primary key ○ Top N queries on indexes where N is 50 ● Less than 25% increase in write latency ○ Due to setting row status in phase 3 ● No noticeable increase in read latency ○ The number of unverified rows due to pending updates on a given table region is limited by the number of RPC threads and mutation batch size