The document discusses Neo4j's causal cluster configuration on AWS. A 3 node causal cluster supports 120 million nodes and 3 billion relationships, with high performance. The configuration uses Terraform to deploy the cluster on AWS, along with Ignition for initialization. Backups are scheduled daily to ensure data integrity, and restores and consistency checks are automated. Future plans include upgrading to the latest Neo4j version and improving availability.

1. © 2022 Neo4j, Inc. All rights reserv ed.
What is Behance?
2
Leading online social platform to
showcase and discover creative
work
More than 25 million creatives
share their work daily
Creatives from over 160 countries
use Behance to get hired, sell their
work & discover other creatives for
collaborations
Tutorials and Tips from the experts
via interactive Livestreams

2. © 2022 Neo4j, Inc. All rights reserv ed.
3
The For You Feed

3. © 2022 Neo4j, Inc. All rights reserv ed.
4
Social as a Service

4. © 2022 Neo4j, Inc. All rights reserv ed.
5
Our 3 Node Neo4j Causal Cluster Supports
120,000,000 total nodes in graph
3,000,000,000 total relationships in graph
225 feed fetch queries per second
60 update ops per second

5. © 2022 Neo4j, Inc. All rights reserv ed.
© 2022 Neo4j, Inc. All rights reserv ed.
6
SLA level of 99.9999 % uptime results in the following
periods of allowed downtime:
• Daily: 0.1s
• Weekly: 0.6s
• Monthly: 2.6s
• Yearly: 31.6s
Uptime and downtime with 99.9999 %

6. © 2022 Neo4j, Inc. All rights reserv ed.
7
Causal Cluster Configuration and
Operations
The design…

7. © 2022 Neo4j, Inc. All rights reserv ed.
8
Our Causal Cluster Configuration

8. © 2022 Neo4j, Inc. All rights reserv ed.
© 2022 Neo4j, Inc. All rights reserv ed.
9
Terraform
The Workflow:
• Terraform
• AWS
• Ignition
• Systemd units created in S3
• ENI created beforehand

9. © 2022 Neo4j, Inc. All rights reserv ed.
© 2022 Neo4j, Inc. All rights reserv ed.
10
AWS
The Workflow:
• Terraform
• AWS
• Ignition

10. © 2022 Neo4j, Inc. All rights reserv ed.
© 2022 Neo4j, Inc. All rights reserv ed.
11
Ignition +
The Workflow:
• Terraform
• AWS
• Ignition
• Systemd units created in S3
◦ Circumvent userdata size
limits

11. © 2022 Neo4j, Inc. All rights reserv ed.
© 2022 Neo4j, Inc. All rights reserv ed.
12
- Git
- Terraform
- Ignition
- AWS
So… How do we make changes?

12. © 2022 Neo4j, Inc. All rights reserv ed.
13
Our Causal Cluster Configuration

13. © 2022 Neo4j, Inc. All rights reserv ed.
14
How’s The Data?
Have confidence in your answer

14. © 2022 Neo4j, Inc. All rights reserv ed.
15
Backups
Scheduled to run on every node using systemd
Leverages an etcd cluster to ensure that it will not
run on more than 1 node in the cluster at a time
Monitors configured to
alert:
A backup failed
A backup has not occurred
in a certain time frame

15. © 2022 Neo4j, Inc. All rights reserv ed.
16
Automated Restores and Consistency Checks
Restores a backup
(defaults to the most
recent).
Executes a consistency
check to confirm the
validity of the data
Monitors configured to
alert on any issues
reported

16. © 2022 Neo4j, Inc. All rights reserv ed.
17
What We Didn't Know that We
Didn’t Know
Could it be that easy? What am I missing?

17. © 2022 Neo4j, Inc. All rights reserv ed.
Uncovered and Discovered
RAFT leader election with ephemeral ports
ENI CoreOS routing
Database Unique Constraints
18

18. © 2022 Neo4j, Inc. All rights reserv ed.
19
3.5.8 is cool, but what about 4.4.5
A short overview of the 4.4.5 upgrade process and
what is changing....

19. © 2022 Neo4j, Inc. All rights reserv ed.
Upgrade and Changes
3.5.8 to 4.x is not a rolling upgrade
Server Side routing
No impact backups
Vault Integration for TLS Keys
20

20. © 2022 Neo4j, Inc. All rights reserv ed.
21
What's Next

21. © 2022 Neo4j, Inc. All rights reserv ed.
Call To Action
Do you want confidence that all nodes in the cluster will be
running – Create a single instanceASG per node.
Do you want to ensure the quickest time to recovery –
Use an ENI on each node
Do you want confidence in the integrity of your data - run
at least once daily backups and daily restores
22

22. © 2022 Neo4j, Inc. All rights reserv ed.
© 2022 Neo4j, Inc. All rights reserv ed.
23
Thank you!
Contacts:
Gabe Tucker: Data Architect
gtucker@adobe.com
Daniel Kang: DevOps Engineer
dakang@adobe.com
Daniel Vilajeti: DevOps Engineer
tek68325@adobe.com
David Fox: Senior Software Engineer
dafox@adobe.com