Ansible is a flexible orchestration tool for ScyllaDB clusters. Learn tried and tested patterns with Ansible to maximize the uptime of your ScyllaDB clusters when making changes. This talk will go over tangible code snippets to help operators and developers safely make changes to their ScyllaDB clusters. These are tips learned the hard way in production so you don't have to. Come along and learn how to orchestrate your ScyllaDB clusters with confidence and not have to take a maintenance window in the middle of the night.

1. Maximum Uptime
Cluster Orchestration
with Ansible
Ryan Ross, Senior Site Reliability Engineer

2. Ryan Ross
■ Senior Site Reliability Engineer at dbt Labs
■ Senior DevOps Engineer at Martin DSP/Pubmatic
■ DevOps Engineer at ConstructConnect
■ Senior Operations Engineer at Alveo Health
Your photo
goes here,
smile :)

3. Talk Objective:
Provide actionable tips and
patterns to build confidence
in orchestrating ScyllaDB in
Production with Ansible.

4. ■ ScyllaDB Use Cases at a DSP
■ Define “Cluster Uptime”
■ Tips for Using Ansible with ScyllaDB
■ Demo
Presentation Agenda

5. ScyllaDB Use Cases

6. What is a Demand-Side Platform (DSP)?
A Demand-Side Platform is a system that allows buyers of
digital advertising inventory to manage multiple ad exchange
and data exchange accounts through one interface - Wikipedia

7. What is a Demand-Side Platform (DSP)?
A Demand-Side Platform is a system that allows buyers of
digital advertising inventory to manage multiple ad exchange
and data exchange accounts through one interface - Wikipedia

8. What is a Demand-Side Platform (DSP)?
A Demand-Side Platform is a system that allows buyers of
digital advertising inventory to manage multiple ad exchange
and data exchange accounts through one interface - Wikipedia

9. What is a Demand-Side Platform (DSP)?
A Demand-Side Platform is a system that allows buyers of
digital advertising inventory to manage multiple ad exchange
and data exchange accounts through one interface - Wikipedia

10. What is a Demand-Side Platform (DSP)?
A Demand-Side Platform is a system that allows buyers of
digital advertising inventory to manage multiple ad exchange
and data exchange accounts through one interface - Wikipedia
The DSP connected advertisers and agencies with audiences to
allow real-time bidding for and displaying of online-advertising.

11. Use Cases for ScyllaDB at a DSP

12. Use Cases for ScyllaDB at a DSP
Billions of requests per day

13. Use Cases for ScyllaDB at a DSP
Billions of requests per day
Equal read and write access patterns

14. Use Cases for ScyllaDB at a DSP
Billions of requests per day
Equal read and write access patterns
24 x 7 x 365 Uptime

15. Use Cases for ScyllaDB at a DSP
Billions of requests per day
Equal read and write access patterns
24 x 7 x 365 Uptime
Millisecond response times

16. Use Cases for ScyllaDB at a DSP
Billions of requests per day
Equal read and write access patterns
24 x 7 x 365 Uptime
Millisecond response times
Huge datasets

17. Use Cases for ScyllaDB at a DSP
Billions of requests per day
Equal read and write access patterns
24 x 7 x 365 Uptime
Millisecond response times
Huge datasets

18. Defining “Uptime”

19. Defining “Uptime”
Provide a cluster that is available for dependent applications.

20. Whole Cluster Uptime

21. Whole Cluster Uptime

22. Whole Cluster Uptime

23. Whole Cluster Uptime

24. Whole Cluster Uptime

25. Whole Cluster Uptime

26. Tips for Using Ansible
to Orchestrate
ScyllaDB with Whole
Cluster Maximum Uptime
* Use at your own risk. Always test in a non-customer facing environment first!

27. Inventory to Cluster Architecture
Structuring your Inventory will go a long
way in helping manage multiple clusters
and data centers. I saw success with:

28. Inventory to Cluster Architecture
Structuring your Inventory will go a long
way in helping manage multiple clusters
and data centers. I saw success with:
■ Inventory -> Cluster

29. Inventory to Cluster Architecture
Structuring your Inventory will go a long
way in helping manage multiple clusters
and data centers. I saw success with:
■ Inventory -> Cluster
■ Inventory Group -> Data Center

30. Inventory to Cluster Architecture
Ansible can have multiple Inventories

31. Dynamic Inventory Scripts
An underused superpower of your Cloud Provider is using them as your
asset management database. Ansible can tie into provider’s API to get
metadata information on instances with Dynamic Inventory Scripts.

32. Dynamic Inventory Scripts

33. Dynamic Inventory Scripts

34. Dynamic Inventory Scripts

35. Dynamic Inventory Scripts

36. Use Lineinfile Module Instead of Template
The ansible.builtin.lineinfile module should be
used instead of others such as copy or template.

37. Use Lineinfile Module Instead of Template
The ansible.builtin.lineinfile module should be
used instead of others such as copy or template.
■ Copy and Template assume Ansible has a valid
copy of the file

38. Use Lineinfile Module Instead of Template
The ansible.builtin.lineinfile module should be
used instead of others such as copy or template.
■ Copy and Template assume Ansible has a valid
copy of the file

39. Use Lineinfile Module Instead of Template
The ansible.builtin.lineinfile module should be
used instead of others such as copy or template.
■ Copy and Template assume Ansible has a valid
copy of the file

40. Use Lineinfile Module Instead of Template
The ansible.builtin.lineinfile module should be
used instead of others such as copy or template.
■ Copy and Template assume Ansible has a valid
copy of the file
■ This becomes problematic with files that are
changed by the project, such as config files
installed with the distribution’s installer

41. Use Lineinfile Module Instead of Template
The ansible.builtin.lineinfile module should be used instead
of others such as copy or template.
■ Copy and Template assume Ansible has a valid copy of
the file
■ This becomes problematic with files that are changed by
the project, such as config files installed with the
distribution’s installer
■ Use Lineinfile to ensure the most recent and valid file
settings are preserved and only change what’s needed.

42. Lineinfile Continued
Preserve project defaults and change only what
you’re sure of.

43. Lineinfile Continued
Preserve project defaults and change only what
you’re sure of.

44. Run Playbooks Serially When Needed

45. Run Playbooks Serially When Needed
By default, Ansible will try to run each
Task one at a time on all hosts in the
Inventory before moving onto the next
Task, like so:

46. Run Playbooks Serially When Needed
By default, Ansible will try to run each
Task one at a time on all hosts in the
Inventory before moving onto the next
Task, like so:

47. Run Playbooks Serially When Needed
Use the Serial argument at the Play to
run all Tasks on one host before
moving onto the next host in an
Inventory

48. Run Playbooks Serially When Needed
Use the Serial argument at the Play to
run all Tasks on one host before
moving onto the next host in an
Inventory

49. Run Playbooks Serially When Needed
Use the Serial argument at the Play to
run all Tasks on one host before
moving onto the next host in an
Inventory

50. Check Cluster Membership Before Making Changes
Cluster changes require all nodes to be Up and Normal (UN). I wrote a Role that was
prerequisite for all Plays that made cluster.

51. Check Cluster Membership Before Making Changes
Cluster changes require all nodes to be Up and Normal (UN). I wrote a Role that was
prerequisite for all Plays that made cluster.

52. Use Automation to Run Ansible
Ansible is an open source cli tool, but it should be
wrapped in another automation tool for Production
use. This has many benefits:
■ Run Playbooks on a schedule
■ Auditable output and outcomes
■ Improved security
■ Send alerts to teams

53. Use Automation to Run Ansible
There are lots of options when it comes to writing
automation around Ansible:
■ Your current C.I. tool
■ Jenkins
■ Github Actions
■ Ansible Automation Platform from RedHat (paid)
■ AWX (upstream F.O.S.S./Community version of Ansible
Automation Platform)

54. Ansible Ecosystem
Ansible has a thriving ecosystem of official and unofficial Collections to use.

55. Ansible Ecosystem
Ansible has a thriving ecosystem of official and unofficial Collections to use.
■ Collection Index
■ Galaxy

56. Ansible Ecosystem
Ansible has a thriving ecosystem of official and unofficial Collections to use.

57. Ansible Ecosystem
Ansible has a thriving ecosystem of official and unofficial Collections to use. To show
a few:

58. Demo Time
* Use at your own risk. Always test in a non-customer facing environment first!

59. Thank You
Stay in Touch
Ryan Ross
nazufel@protonmail.com
@nazufel
www.linkedin.com/in/rossryan1/
github.com/nazufel/scylla-summit-2023