A beginners guide to the unsung Nomad feature, Parameterized Jobs: a great way to represent batch workloads that require additional context. We'll look at Nomad as a whole, how parameterized jobs fit into the bigger picture, and then put the feature (as well as the related nomad job dispatch command) into practice with some example jobs. You will leave knowing how to create a parameterized job, how to dispatch a parameterized job, and a general understanding of all the behind the scenes machinery that makes your workload run smoothly.

1. ⁄
Ground Control to
Nomad Job Dispatch
Michael Lange
@DingoEatingFuzz

4. https://github.com/ashleymcnamara/gophers/blob/master/Kubernetes_Gophers.png

6. “What is Nomad?”
–like, a lot of people

8. Yes No I don’t know

9. What is a parameterized job?
Welcome to the talk.

10. What is a job?
Welcome to the preface.

11. Standard Job Flow

12. Periodic Job Flow

13. Parameterized Job Flow
Meta Params: Small configuration strings/ids
Payload: Larger configuration blob.

14. Parameterized Job Flow
Meta Params: Query Params/GET
Payload: Request Body/POST

15. The Simplest Imaginable
Parameterized Job
DEMO

16. Our job file
Batch Job
Parameterized options
Simple bash command
CODE EDITOR
job "hi" {
datacenters = ["dc1"]
type = "batch"
parameterized {
payload = "required"
}
task "hi" {
driver = "exec"
dispatch_payload {
file = "message.txt"
}
config {
command = "bash"
args = [
"-c",
"echo "Hello, $(cat ${NOMAD_TASK_DIR}/message.txt)"; sleep 120"
]
}
resources {
cpu = 100
memory = 64
}
}
}

17. Our job file
Batch Job
Parameterized options
Simple bash command
CODE EDITOR
job "hi" {
datacenters = ["dc1"]
type = "batch"
parameterized {
payload = "required"
}
task "hi" {
driver = "exec"
dispatch_payload {
file = "message.txt"
}
config {
command = "bash"
args = [
"-c",
"echo "Hello, $(cat ${NOMAD_TASK_DIR}/message.txt)"; sleep 120"
]
}
resources {
cpu = 100
memory = 64
}
}
}

18. Our job file
Batch Job
Parameterized options
Simple bash command
CODE EDITOR
job "hi" {
datacenters = ["dc1"]
type = "batch"
parameterized {
payload = "required"
}
task "hi" {
driver = "exec"
dispatch_payload {
file = "message.txt"
}
config {
command = "bash"
args = [
"-c",
"echo "Hello, $(cat ${NOMAD_TASK_DIR}/message.txt)"; sleep 120"
]
}
resources {
cpu = 100
memory = 64
}
}
}

19. Our job file
Batch Job
Parameterized options
Simple bash command
CODE EDITOR
job "hi" {
datacenters = ["dc1"]
type = "batch"
parameterized {
payload = "required"
}
task "hi" {
driver = "exec"
dispatch_payload {
file = "message.txt"
}
config {
command = "bash"
args = [
"-c",
"echo "Hello, $(cat ${NOMAD_TASK_DIR}/message.txt)"; sleep 120"
]
}
resources {
cpu = 100
memory = 64
}
}
}

20. “Cool story, bro”
–you, probably

21. What if…
1. I have various datasets about the world
2. I want to make a choropleth*
3. I don’t know what map projection I want
4. I’d like to quickly generate a variety of options
*A map where regions are colored proportionally according to some value

22. The tools we have available
▪ Meta params
▪ Payload

23. The tools we have available
▪ Meta params Good for specifying a projection
▪ Payload Good for attaching the dataset

24. 16KiB
The maximum payload size for
a dispatched job.

25. Artifact
Stanza
A way of telling Nomad to
download files before
starting a task.
CODE EDITOR
task "map" {
# ...
artifact {
source = “https://the.inter.net“
destination = "data.json"
mode = "file"
}
# ...
}

26. Combine
Concepts
Parameterized meta
params can be
interpolated and will also
exist in the task env.
CODE EDITOR
parameterized {
payload = "forbidden"
meta_required = [ "DATA_FILE", "PROJECTION" ],
}
task "map" {
# ...
artifact {
source = "${NOMAD_META_DATA_FILE}"
destination = "data.json"
mode = "file"
}
# ...
}

29. Let’s Make Some Maps!
DEMO

30. A final note about
that demo…

31. ¯_(ツ)_/¯

32. “Workflows, not technologies”
–Tao of HashiCorp

33. Exec Task
Driver
Get an isolated context for
your task. Run whatever
you want.
CODE EDITOR
task "map" {
driver = "exec"
artifact {
source = “https://.../make-svg-linux”
}
config {
command = "make-svg-linux"
args = [
"${NOMAD_META_PROJECTION}",
"data.json",
]
}
}

34. Recap!
1. Nomad! It’s HashiCorp’s workload orchestrator
2. Parameterized jobs! A way of modeling batch jobs as a template
3. Nomad Job Dispatch! A way of launching a job based on a parameterized template
4. Payload and Meta Params! Two different ways of providing context to a dispatched job
5. Artifacts! Another Nomad concept for downloading files. Compose utilities for great affect
6. Exec Driver! Maybe you don’t need to containerize everything.
7. Map Projections! They’re so neat

36. LEARN FORUMS
learn.hashicorp.com/nomad discuss.hashicorp.com/c/nomad

37. Thank you!
Source Code: https://github.com/DingoEatingFuzz/hashitalks-2020-map-dispatch