Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

High Availability - Jobs

195 views

Published on

Learn how to configure Rundeck Enterprise to automatically failover to another cluster member if an instance goes down during a job execution or is not available when a job is scheduled to run.

In this webinar you will learn how to configure:
Automatic Job Schedule takeovers
Remote execution policies
Automatic job resume for executions running during an outage
Manually move job schedule ownership to another Rundeck instance

See the full video here: https://rundeck.wistia.com/medias/tuznbnw8eg

Links:
List Running Executions: https://docs.rundeck.com/docs/api/rundeck-api.html#listing-running-executions
Cluster mode, including takeover and listing schedules: https://docs.rundeck.com/docs/api/rundeck-api.html#cluster-mode
List incomplete executions: https://docs.rundeck.com/docs/api/rundeck-api.html#list-executions-with-incomplete-log-storage
Metrics Data: https://docs.rundeck.com/docs/api/rundeck-api.html#metrics-data
Cluster Overview: https://docs.rundeck.com/docs/enterprise/#overview
Heartbeat setting: https://docs.rundeck.com/docs/administration/cluster/autotakeover/#configure-autotakeover
Remote Execution: https://docs.rundeck.com/docs/administration/configuration/remote-job-execution.html#cluster-remote-execution
Autotaker of job schedule: https://docs.rundeck.com/docs/administration/cluster/autotakeover/#configure-autotakeover

Published in: Technology
  • Be the first to comment

  • Be the first to like this

High Availability - Jobs

  1. 1. High Availability - Jobs Automatic schedule and execution retry
  2. 2. Agenda Cluster overview Job execution Heartbeat Recovery/retry Schedule auto takeover
  3. 3. Rundeck cluster
  4. 4. Remote execution options # Policy type None, Random, RoundRobin, Load, Preset rundeck.clusterMode.remoteExecution.policy = RoundRobin # Identifies rundeck servers rundeck.clusterMode.remoteExecution.config.allowed = other,self # Send to a member marked inactive rundeck.clusterMode.remoteExecution.profile.profile1.config.activeOnly = true
  5. 5. Heartbeat settings # Frequency rundeck.clusterMode.heartbeat.interval=30 # Startup delay rundeck.clusterMode.heartbeat.delay=10 # Mark another member inactive rundeck.clusterMode.heartbeat.considerInactive=60 # Mark another member dead rundeck.clusterMode.heartbeat.considerDead=120 # remote execute/abort message processing rundeck.clusterMode.remoteExec.process.interval=10
  6. 6. Job execution recover/auto retry # enable auto cleanup of stale jobs on member death rundeck.clusterMode.recoverExecutions.enabled=true # policy for members to accept as targets of auto cleanup. Can be 'Any' or 'Static' rundeck.clusterMode.recoverExecutions.policy=Any # delay in seconds before proceeding with autoretry proposal rundeck.clusterMode.recoverExecutions.delay=30 # delay in seconds before doing another autoretry of the same member rundeck.clusterMode.recoverExecutions.sleep=60
  7. 7. Scheduled job autotakeover # Enable for members detected as "dead" rundeck.clusterMode.autotakeover.enabled=true # "Any": all dead nodes. "Static": only allowed uuids rundeck.clusterMode.autotakeover.policy=any # Wait after sending autotakeover proposal rundeck.clusterMode.autotakeover.delay = 60 # Sleep min between autotakeover attempts rundeck.clusterMode.autotakeover.sleep = 300
  8. 8. Combining policies for HA
  9. 9. Questions pwonson@rundeck.com
  10. 10. Resource/References ● API ○ List running executions ○ Cluster mode, including takeover and listing schedules ○ List incomplete executions ○ Metrics data ● Cluster Overview ● Heartbeat setting ● Remote Execution ● Autotaker of job schedule

×