Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Operational Simplicity with Nomad and OpenFaaS

47 views

Published on

Presentation from Docker Cambridge meetup introducing the Nomad provider for OpenFaaS

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Operational Simplicity with Nomad and OpenFaaS

  1. 1. DEVOPS DELIVERED @sheriffjackson OPERATIONAL SIMPLICITY WITH Nomad AND OpenFaaS +
  2. 2.  Nic Jackson Developer Advocate, HashiCorp I once accidentally got drunk before giving a presentation. @sheriffjackson
  3. 3. @sheriffjackson
  4. 4. X NOMAD Nomad
  5. 5. NOMAD OVERVIEW
  6. 6. X NOMAD DESIGN PRINCIPLES Integrated scheduler and cluster manager Distributed, shared state, optimistically concurrent Agent-based, client/server No dependencies
  7. 7. X NOMAD CHARACTERISTICS Multi-datacenter and multi-region Highly performant and highly available Hybrid workloads with multiple schedulers and drivers Seamlessly integrates with HashiCorp ecosystem
  8. 8. NOMAD FUNDAMENTALS
  9. 9. X SINGLE REGION DEPLOYMENT SERVER SERVER SERVER CLIENT CLIENT CLIENTDC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC
  10. 10. X MULTI REGION DEPLOYMENT SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER
  11. 11. X SERVER ARCHITECTURE Omega Class Scheduler Pluggable Logic Internal Coordination and State Multi-Region / Multi-Datacenter
  12. 12. X CLIENT ARCHITECTURE Broad OS Support Host Fingerprinting Pluggable Drivers Job restarts and lifecycle management
  13. 13. X CLIENT DRIVERS Containerized Docker rkt Windows Server Containers Virtualized Qemu / KVM Hyper-V Xen Standalone Java Jar C# Static Binaries
  14. 14. X CLIENT FINGERPRINTING Type Examples Operating System Kernel, OS, Version Hardware CPU, Memory, Disk Apps (Capabilities) Docker, Java, Consul Environment AWS, GCE
  15. 15. X AUTO BOOTSTRAPPING
  16. 16. NOMAD JOB CONFIGURATION
  17. 17. X JOB FILE Declarative Scheduler, driver, and resource needs Lifecycle behavior Constraints Versioned
  18. 18. 18 redis.nomad JOB FILE job "redis" { datacenters = ["us-east-1"] task "redis" { driver = "docker" config { image = "redis:v13" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 dynamic_ports = ["redis"] } } } }
  19. 19. 19 redis.nomad JOB FILE: TASK GROUPS job "app" { group "app" { task "redis" { # ... } task "app" { # ... } } }
  20. 20. 20 redis.nomad JOB FILE: CONSTRAINTS job "redis" { constraint { attribute = "${attr.kernel.version}" operator = "version" value = "> 3.19" } constraint { attribute = "${attr.platform.aws.instance-type}" value = "p2.16xlarge" } task "redis" { # ... } }
  21. 21. 21 redis.nomad JOB FILE: CONSUL SERVICE DISCOVERY job "redis" { task "redis" { # ... service { port = “redis” check { type = “tcp” interval = “10s” } } } }
  22. 22. 22 redis.nomad JOB FILE: CONSUL CONFIGURATION job "redis" { task "redis" { # ... template { data = <<EOH bind_port: {{ env "NOMAD_PORT_db" }} scratch_dir: {{ env "NOMAD_TASK_DIR" }} service_key: {{ key "service/my-key" }} EOH destination = "local/file.yml" } } }
  23. 23. 23 redis.nomad JOB FILE: VAULT INTEGRATION job "redis" { task "redis" { # ... template { data = <<EOH {{ with secret "secret/credentials" }} username: {{ .Data.username }} password: {{ .Data.password }} {{ end }} EOH destination = "local/file.yml" } } }
  24. 24. 24 redis.nomad JOB FILE: PARAMETERIZED job "encode" { type = "batch" parameterized { payload = "required" meta_required = ["s3-input", "s3-output", ...] } # ... task "ffmpeg" { driver = "exec" config { command = "ffmpeg" # When dispatched, the payload is written to a file that is then # read by the created task upon startup args = ["-config=${NOMAD_TASK_DIR}/config.json"] # ... }
  25. 25. 25 $ nomad job dispatch encode video-config.json $ $ cat video-config.json { "s3-input": "https://s3-us-west-1.com/video-bucket/cb31dabb1", "s3-output": "https://s3-us-west-1.com/video-bucket/a149adbe3", "input-codec": "mp4", "output-codec": "webm", "quality": "1080p" } Text JOB FILE: PARAMETERIZED
  26. 26. NOMAD - OPENFAAS ARCHITECTURE
  27. 27. @sheriffjackson
  28. 28. DEMO
  29. 29. THANKS FOR LISTENING • https://www.nomadproject.io • https://github.com/hashicorp/faas-nomad • https://www.openfaas.com • https://registry.terraform.io/modules/nicholasjackson/open-faas- nomad/aws/0.5.0 FURTHER INFO: @sheriffjackson, nic@hashicorp.com DEVOPS DELIVERED

×