HashiCorp Nomad is a flexible and straightforward scheduler and orchestrator to deploy and manage containers and non-containerized applications across on-prem and clouds at scale. Nomad can be seen as: - an alternative to Kubernetes to deploy and scale containers without complexity - a supplement to Kubernetes to implement a multi-orchestrator platform On the other hand, this session will present how to ease the Cloud Native Transformation using Nomad.

1. Easy Cloud Native Transformation
with HashiCorp Nomad
Bram Vogelaar
@attachmentgenie

2. • Used to be a Molecular Biologist
• Then became a Dev, now an Ops
• Currently Cloud Engineer @ The Factory
• Amsterdam HUG organizer
• Almost author => Workload Orchestration Made Easy With Nomad
$ whoami

3. Moving it all to the cloud

4. Vertical scaling

5. And than stuff got complicated….

6. l Open-source tool for dynamic workload scheduling
l Batch, containerized, and non-containerized applications.
l Has native Consul and Vault integrations.
l Has token based access setup.
l Jobs written in (H)ashiCorp (C)onfiguration (L)anguage
https://www.nomadproject.io/
Nomad

7. Deploy the blog
job "blog" {
group ”hugo" {
network {
port "http" { to = ”80” }
}
service {
name = ”attachmentgeniedotcom"
port. = ”http"
}
task ”nginx" {
driver = "docker"
config {
image = "${PRIVATE}.dkr.ecr.us-east-1.amazonaws.com/blog:19”
ports = ["http"]
}
}
}

8. 1 == None
job "blog" {
datacenters = ["aws"]
type = "service"
group "hugo" {
count = 2

9. Force onto different hardware
job "blog" {
datacenters = ["aws"]
type = "service"
group "hugo" {
count = 2
constraint {
operator = "distinct_hosts"
value = "true"
}

10. Suggest onto different hardware
job "blog" {
datacenters = ["aws"]
type = "service"
group "hugo" {
count = 2
Spread {
attribute = "${node.datacenter}"
}

11. Based on custom meta-data
/etc/nomad.d/config.hcl
Client {
Enabled = true
Meta {
"rack" = "his"
}
}
blog.nomad
job "blog" {
group "hugo" {
Spread {
attribute = "${meta.rack}"
target "his" {
percent = 50
}
target "her" {
percent = 50
}
}

12. service {
name = "grafana"
provider = "nomad"
port = "grafana_ui"
check {
type = "http"
path = "/api/health"
interval = "10s"
timeout. = "2s"
}
tags = [
"traefik.enable=true",
]
}
Native Service Discovery ( > 1.3)

13. template {
data = <<EOH
server {
listen *:80;
server_name grafana.teambla.com;
location / {
{{ range nomadService ”grafana" }}
proxy_pass http://{{ .Address }}:{{ .Port }};
{{ end }}
….
}
EOH
destination = "local/nginx.conf"
}
Native Service Discovery ( > 1.3)

14. $ nomad alloc checks f6683eb8
Status of 1 Nomad Service Checks
ID = e47351cef1c81ca56bf70be6b48ae44a
Name = service: "grafana" check
Group = grafana.grafana[0]
Task = grafana
Service = grafana
Status = success
StatusCode = 200
Mode = healthiness
Timestamp = 2022-10-03T12:43:48Z
Output = nomad: http ok
Service Checks

15. Service Overview

16. $ cat grafana.nomad
service {
name = "grafana"
provider = "nomad"
port = "grafana_ui"
tags = [
"traefik.enable=true",
]
}
Ingress / Traefik
$ cat traefik.yaml
providers:
nomad:
address: http://${attr.unique.network.ip-address}:4646”
exposedByDefault: false
https://traefik.io <- 2.8.0

17. Metrics / Prometheus
https://prometheus.io <- 2.37.0
$ cat grafana.nomad
service {
name = "grafana"
provider = "nomad"
port = "grafana_ui"
tags = [
"metrics",
]
}
$ cat prometheus.yaml
scrape_configs:
- job_name: 'nomad_sd'
nomad_sd_configs:
- server: 'http://{{ env "attr.unique.network.ip-address" }}:4646'
relabel_configs:
- source_labels: ['__meta_nomad_tags']
regex: '(.*),metrics,(.*)'
action: keep
- source_labels: [__meta_nomad_service]
target_label: job

18. l Open-Source Service Discovery Tool
l Build-in KV store
l Service Mesh tool
l Uses watchers to have near instant feedback loops
https://www.consul.io/
Consul

19. • Levant
• Templating and packaging tool
• Easily deploy popular applications to Nomad
• Re-use common patterns across internal applications
• Find and share job definitions with the Nomad community
• Jobs written in (H)ashiCorp (C)onfiguration (L)anguage
• Templates are written using Go Template Syntax.
• Nightlies only right now!
Nomad Pack
https://github.com/hashicorp/nomad-pack

20. variable "datacenters" {
description = "A list of datacenters in the region which are eligible for task placement"
type = list(string)
default = [“dc1”]
}
Variable “resources” {
description = “The resource to assign to the blog service task”
type = object({
cpu = number
memory = number
})
default = {
cpu = 200,
memory = 256
}
}
variables.hcl

21. $ cat packs/blog/templates/blog.nomad.tpl
….
datacenters = [[ .my.datacenters | toStringList ]]
…
resources {
cpu = [[ .my.blog_resources.cpu ]]
memory = [[ .my.blog_resources.memory ]]
}
…
Pack Templates

22. • nomad-pack registry list
• nomad-pack registry add https://github.com/attachmentgenie/hashitalks-registry
• nomad-pack run blog --var namespace=dashboard
• nomad-pack run packs/blog -f vars/blog.hcl
Pack Registries
https://github.com/hashicorp/nomad-pack-community-registry

23. Nomad UI (> 1.4.3)

24. (secure) Variables (> 1.4)

25. Secrets
task "grafana" {
driver = "docker"
config {
image = "grafana/grafana:9.2.3"
ports = ["grafana_ui"]
}
env {
{{- with nomadVar "nomad/jobs/grafana" -}}
GF_SECURITY_ADMIN_USER = {{. GF_SECURITY_ADMIN_USER}}
GF_SECURITY_ADMIN_PASSWORD = {{. GF_SECURITY_ADMIN_PASSWORD}}
{{- end -}}
}

26. l Open-Source tool to do secrets management
l Secure, store and tightly control access to tokens, passwords,
encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.
l Certificate management
l Password rotation (e.g. SSH, MySQL)
https://www.vaultproject.io/
Vault

27. Nomad autoscaler
Introduced in/with Nomad 0.11
(Currently) independently release cycle
Gaining new functionality every release
Build in Functionality for horizontal and vertical scaling
But extendable by your own (community) plugins

28. ● Makes decisions based on a checks
● Checks are a combination of
• Data queried from an APM
• Defined STRATEGY
• Attempt to approach TARGET value
● Multiple Checks can be combined
• Answer with the most resources will win!
• ScaleOut and ScaleIn => ScaleOut
• ScaleOut and ScaleNone => ScaleOut
• ScaleIn and ScaleNone => ScaleNone
• ScaleOut(10) and ScaleOut(9) => ScaleOut(10)
• ScaleIn(3) and ScaleIn(4) => ScaleIn(4)
Auto-scaling TLDR

29. /etc/nomad.d/config.hcl
nomad {
address = "http://{{env "attr.unique.network.ip-address" }}:4646"
}
apm "prometheus" {
driver = "prometheus"
config = {
address = "http://prometheus.service.consul:9090"
}
}
strategy "target-value" {
driver = "target-value"
}
Config for the autoscaler

30. job "blog" {
datacenters = ["aws"]
type = "service"
group "hugo" {
count = 3
scaling {
enabled = true
min = 1
max = 20
policy {
cooldown = “20s”
check “avg_instance_sessions” {
source = “prometheus”
query = “scalar(avg(traefik_service_open_connections{service=”blog@consulcatalog”}))”
strategy “target-value” {
target = 5
}
Enable autoscaling for the blog

31. Observe Load

32. Correlate events with metrics

33. Moving it all to the cloud - QED

34. Questions ?
The Floor is yours…
bram@attachmentgenie.com
@attachmentgenie
https://www.slideshare.net/attachmentgenie