This document summarizes Salesforce's use of HBase and Phoenix for storing and querying large volumes of structured and unstructured data at scale. Some key details: 1) Salesforce heavily uses HBase and Phoenix for both customer-facing and internal use cases, including storing login data, user activity, thread dumps, and more. 2) Salesforce operates over 100 HBase clusters of varying sizes to support over 4 billion write requests and 600 million read requests per day, totaling over 80 terabytes of data written and 500 gigabytes read daily. 3) An example use case is a central metrics database collecting data from over 80,000 machines, storing 11.4 trillion metrics and growing, with

1. HBase/PH ENIX @ Scale
A study of Salesforce’s use of HBase and Phoenix
Lars Hofhansl
Vice President and Principal Architect at Salesforce
Apache HBase, Apache Phoenix committer and PMC member

2. Two Years Ago, I showed you this

4. Zookeeper?

5. Zookeeper?
HBase?

6. Zookeeper?
HBase?
HDFS?

7. Zookeeper?
Commodity
Hardware?
HBase?
HDFS?

8. Zookeeper?
Commodity
Hardware?
HBase?
HDFS?Unstructured
Data?

9. HBase/Phoenix are BIG* at Salesforce
* Numbers are from some time in the past and do not reflect the current scale

10. Heavy users of relational and semi-structure data

11. Mix of customer* and internal data
* Through the Salesforce Platform Offering

12. Typical Use Cases
Samples of Customer Data:
Login data to track anomalies in real-time
Archiving, historical data moved from operational, relational storage to HBase
Denormalized feed views for Salesforce Chatter
Chatter @mention low-latency relevancy queries
Storage of user activity on marketing campaigns for reporting and AI/ML
Samples of Internal usage:
Periodic thread dumps from all AppServers

13. Customer Facing

14. > 100 clusters of varying size

15. ~4bn write requests / day

16. ~80TB written / day

17. That’s about ~8 gbit/s, sustained

18. ~600m read requests / day

19. ~500GB read / day

20. Central Metrics Database

21. Central Metrics Database

22. Collecting data from > 80.000 machines

23. 11.4 trillion metrics stored and growing

24. 2.8 tn metrics in 6 months and growing

25. 210 bn reads in 6 months