Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
01
02
Yamlware Basically, any (configuration|infrastructure|deployment)-as-code software nowadays. “ 03
04
05
YAML: Good or bad? 06
YAML is great! 07
For computers YAML is a human friendly data serialization standard for all programming languages. “ 08
For humans YAML is a human friendly data serialization standard for all programming languages. “ 09
10
11
YAML is awful! 12
13
14
#NoYaml 15
Did you know? !!python/object/apply:os.system args: ['ls /'] 01. 02. 16
Duplicate keys - name: A name: B 01. 02. 17
Duplicate keys 18
Indentation - name: nano apt: name: nano 01. 02. 03. 19
Base 60 Using ":" allows expressing integers in base 60, which is convenient for time and angle values. “ 20
Port mappings 25:25 = 25*60 + 25*3600 = 91500 53:53 = ... = 193980 • • 21
Anchors and references - &sometask name: "Some Task" some_module: with a lot of parameters with_items: list_A - <<: *somet...
Boolean y|Y|yes|Yes|YES| n|N|no|No|NO| true|True|TRUE| false|False|FALSE| on|On|ON| off|Off|OFF 01. 02. 03. 04. 05. 06. 23
24
Content is trunca 25
Yaml is not Turing- complete!26
Ok, that's unfair! 27
Of course, it's not! It's not a computational language, it's a data language List of maps of lists of maps of maps of list...
Is JSON better? 29
No comments 30
XML? Anyone? 31
32
Stop whining 33
Yaml is a tool! 34
Industry needs a data language We need a simple language, to declare our intent, describe configuration, manage state, sha...
Industry needs it, industry creates it36
JSONNET 37
JSON5 38
TOML 39
DHALL 40
YTT 41
HCL/HCL2 42
Standards, standards,... 43
How about a real language? 44
Declarations, not actions Yaml is good for defining serialized data and declared state But it does not support constructs ...
46
The machine 47
48
The machine Yaml (or any similar language) is just the declaration of WHAT Heavy lifting of HOW is done by the machinery •...
50
51
New version They just released v1.16 this month 52
U.S.E. $ use version 1.16 (Orbit) 01. 02. 53
CLI $ use apply solar_system.yml01. 54
55
56
Let's play! 57
58
Pluto's status kind: Planet metadata: type: planet name: Pluto 01. 02. 03. 04. 59
Pluto's status kind: Planet metadata: type: dwarf_planet name: Pluto 01. 02. 03. 04. 60
Pluto's status $ use apply solar_system.yml use: Modifying Pluto... use: Pluto's metadata modified in 10ns fume: libraries...
62
Comet kind: Comet metadata: name: MyComet replicas: 1 speed: 1000km/s 01. 02. 03. 04. 05. 63
Let's check it $ use apply solar_system.yml use: Creating "MyComet"... use: MyCommet is created fume: WARNING: Unknown obj...
Let's remove the code $ use apply solar_system.yml use: all is up-to-date. no changes have been applied. fume: WARNING: Un...
Compensating actions $ use delete comet/MyComet use: Comet "MyComet" has been destroyed 01. 02. 66
67
That's no moon 68
That's no moon kind: Planet metadata: type: planet name: Earth moons: - name: Moon - name: Moon2 01. 02. 03. 04. 05. 06. 0...
Go! Go! Go! $ use apply solar_system.yml use: moon "Moon" already exists. Skipped. use: Creating moon "Moon2"... use: Done...
Wait! What? fume: WARNING: Earth: unexpected floods in Sri-Lanka due to irregular tides patterns fume: WARNING: Earth: rot...
SOL-101: Bug report Earth is overpopulated. We need to scale it up to house more biological forms. 72
Bug fix kind: Planet metadata: type: planet name: Earth dimensions: radius: 8000km 01. 02. 03. 04. 05. 06. 73
Push it $ git add solar_system.yml $ git commit -m "SOL-101: increased Earth radius " 1 file changed, 1 insertions(+), 0 d...
No way! jobs: Starting job for commit b64049e jobs: Running script jobs: use apply solar_system.yml use: Destroying planet...
Backup kind: PlanetBackup target: name: Earth schedule: cron: * * * * * 01. 02. 03. 04. 05. 76
10 days later... fume: There is no space left in space.01. 77
Real life state machines Kubernetes Terraform CloudFormation Serverless.com Ansible Puppet etc. • • • • • • • 78
Idempotance 79
Idempotance 80
Idempotent #!/bin/bash apt-get -y install nano 01. 02. 81
Not idempotent #!/bin/bash apt-get -y install nano useradd andrey useradd: user 'andrey' already exists 01. 02. 03. 04.05....
More code #!/bin/bash apt-get -y install nano if ! id -u andrey > /dev/null 2>&1; then useradd andrey fi 01. 02. 03. 04. 0...
Declarative catalogue - user: name: andrey shell: /bin/zsh - package: name: nano version: 1.23.1-1 01. 02. 03. 04. 05. 06....
Readability! 85
Pipeline- integratable 86
Abstraction 87
Resource graph 88
Parallel execution 89
Automatic ordering 90
Modularity 91
Data query - package: name: nano-${os.version} version: 1.23.1-1 01. 02. 03. 92
Resource deletion - user: name: andrey shell: /bin/zsh # - package: # name: nano # version: 1.23.1-1 01. 02. 03. 04. 05. 0...
Attribute management Write-once Updatable, applied right-away Updatable, requires extra action to take effect Updatable, f...
Resource recreation 95
Resource protection 96
Dry-run 97
Compensating Compensating actions may sometimes be required Not every state change is allowed Any software has bugs! • • •...
State verification Deployment-time Run-time • • 99
Module-level dependencies 100
Resource- level dependencies101
SM-index idempotence maturity declarativeness maturity (readiblity) data query richness execution engine maturity (paralle...
103
104
Ansible machine 105
106
Ansible machine Signs of state management, idempotance is not always achievable. Can be parallel only on node level, not r...
Kubernetes machine 108
109
Kubernetes machine api-server will accept any valid object definition. Will it be created and running? Eventually, it shou...
kubectl wait 111
Helm 112
Terraform machine 113
114
Terraform machine Terraform remembers previous state. Deployment time validation, no run-time validation. Not YAML, rich e...
Final words 116
Side effects are real Know the tool Know which resources you are managing Know the side effects • • • 117
Tools There are tools for tools: linters, IDE plugins, schema validators. 118
Everything is a spectrum Avoid dichotomy (bad/good, persistent/volatlie etc.), use a spectrum to define your relationship ...
Keep it sane If you need a complex logic, use a %&?# programming language or a better declarative language with functions ...
Keep it simple 121
Thank you! 122
123
124
References 125
Whining https://arp242.net/yaml-config.html https://noyaml.com https://boyter.org/posts/an-informal-survey-of-10-million-g...
Languages https://jsonnet.org/ https://json5.org/ https://dhall-lang.org/ https://get-ytt.io/ https://github.com/bazelbuil...
Tips https://www.redhat.com/sysadmin/yaml-tips https://networkgenomics.com/ansible/ • • 128
Kubernetes https://www.infoq.com/articles/kubernetes-workloads-serverless-era/• 129
Upcoming SlideShare
Loading in …5
×

Yamlware and the state machine

107 views

Published on

Yaml FTW! Infrastructure as code is eating the world (well, part of it). YAML, JSON, TOML, HCL and several other languages tend to describe the desired state of your infrastructure controlled by the STATE MACHINE. How does it work? Let’s dive into an illustrated journey behind the machinery of various tools like Terraform, Ansible, Kubernetes.

Published in: Technology
no profile picture user

  • Be the first to comment

  • Be the first to like this

Yamlware and the state machine

  1. 1. 01
  2. 2. 02
  3. 3. Yamlware Basically, any (configuration|infrastructure|deployment)-as-code software nowadays. “ 03
  4. 4. 04
  5. 5. 05
  6. 6. YAML: Good or bad? 06
  7. 7. YAML is great! 07
  8. 8. For computers YAML is a human friendly data serialization standard for all programming languages. “ 08
  9. 9. For humans YAML is a human friendly data serialization standard for all programming languages. “ 09
  10. 10. 10
  11. 11. 11
  12. 12. YAML is awful! 12
  13. 13. 13
  14. 14. 14
  15. 15. #NoYaml 15
  16. 16. Did you know? !!python/object/apply:os.system args: ['ls /'] 01. 02. 16
  17. 17. Duplicate keys - name: A name: B 01. 02. 17
  18. 18. Duplicate keys 18
  19. 19. Indentation - name: nano apt: name: nano 01. 02. 03. 19
  20. 20. Base 60 Using ":" allows expressing integers in base 60, which is convenient for time and angle values. “ 20
  21. 21. Port mappings 25:25 = 25*60 + 25*3600 = 91500 53:53 = ... = 193980 • • 21
  22. 22. Anchors and references - &sometask name: "Some Task" some_module: with a lot of parameters with_items: list_A - <<: *sometask name: "Some OTHER Task" with_items: list_B 01. 02. 03. 04. 05.06. 07. 08. 22
  23. 23. Boolean y|Y|yes|Yes|YES| n|N|no|No|NO| true|True|TRUE| false|False|FALSE| on|On|ON| off|Off|OFF 01. 02. 03. 04. 05. 06. 23
  24. 24. 24
  25. 25. Content is trunca 25
  26. 26. Yaml is not Turing- complete!26
  27. 27. Ok, that's unfair! 27
  28. 28. Of course, it's not! It's not a computational language, it's a data language List of maps of lists of maps of maps of lists... • • 28
  29. 29. Is JSON better? 29
  30. 30. No comments 30
  31. 31. XML? Anyone? 31
  32. 32. 32
  33. 33. Stop whining 33
  34. 34. Yaml is a tool! 34
  35. 35. Industry needs a data language We need a simple language, to declare our intent, describe configuration, manage state, share serialized data. 35
  36. 36. Industry needs it, industry creates it36
  37. 37. JSONNET 37
  38. 38. JSON5 38
  39. 39. TOML 39
  40. 40. DHALL 40
  41. 41. YTT 41
  42. 42. HCL/HCL2 42
  43. 43. Standards, standards,... 43
  44. 44. How about a real language? 44
  45. 45. Declarations, not actions Yaml is good for defining serialized data and declared state But it does not support constructs for variables, functions, deep references And it is not so good for imperative/sequential scripts (I'm looking at you Ansible) ? • • • 45
  46. 46. 46
  47. 47. The machine 47
  48. 48. 48
  49. 49. The machine Yaml (or any similar language) is just the declaration of WHAT Heavy lifting of HOW is done by the machinery • • 49
  50. 50. 50
  51. 51. 51
  52. 52. New version They just released v1.16 this month 52
  53. 53. U.S.E. $ use version 1.16 (Orbit) 01. 02. 53
  54. 54. CLI $ use apply solar_system.yml01. 54
  55. 55. 55
  56. 56. 56
  57. 57. Let's play! 57
  58. 58. 58
  59. 59. Pluto's status kind: Planet metadata: type: planet name: Pluto 01. 02. 03. 04. 59
  60. 60. Pluto's status kind: Planet metadata: type: dwarf_planet name: Pluto 01. 02. 03. 04. 60
  61. 61. Pluto's status $ use apply solar_system.yml use: Modifying Pluto... use: Pluto's metadata modified in 10ns fume: libraries on Earth have been updated to reflect changes in Pluto's status 01. 02. 03. 04. 05. 61
  62. 62. 62
  63. 63. Comet kind: Comet metadata: name: MyComet replicas: 1 speed: 1000km/s 01. 02. 03. 04. 05. 63
  64. 64. Let's check it $ use apply solar_system.yml use: Creating "MyComet"... use: MyCommet is created fume: WARNING: Unknown object is aproaching planet "Earth" 01. 02. 03. 04. 64
  65. 65. Let's remove the code $ use apply solar_system.yml use: all is up-to-date. no changes have been applied. fume: WARNING: Unknown object is aproaching planet "Earth" 01. 02. 03. 65
  66. 66. Compensating actions $ use delete comet/MyComet use: Comet "MyComet" has been destroyed 01. 02. 66
  67. 67. 67
  68. 68. That's no moon 68
  69. 69. That's no moon kind: Planet metadata: type: planet name: Earth moons: - name: Moon - name: Moon2 01. 02. 03. 04. 05. 06. 07. 69
  70. 70. Go! Go! Go! $ use apply solar_system.yml use: moon "Moon" already exists. Skipped. use: Creating moon "Moon2"... use: Done in 2 years 23 days. use: Moon2 is up and running. 01. 02. 03. 04. 05. 70
  71. 71. Wait! What? fume: WARNING: Earth: unexpected floods in Sri-Lanka due to irregular tides patterns fume: WARNING: Earth: rotation slowed down by 0.3%. Risk of seasonal shift! 01. 02. 03. 04. 71
  72. 72. SOL-101: Bug report Earth is overpopulated. We need to scale it up to house more biological forms. 72
  73. 73. Bug fix kind: Planet metadata: type: planet name: Earth dimensions: radius: 8000km 01. 02. 03. 04. 05. 06. 73
  74. 74. Push it $ git add solar_system.yml $ git commit -m "SOL-101: increased Earth radius " 1 file changed, 1 insertions(+), 0 deletions(-) $ git push b64049e..ed99f47 master -> master 01. 02. 03. 04. 05. 74
  75. 75. No way! jobs: Starting job for commit b64049e jobs: Running script jobs: use apply solar_system.yml use: Destroying planet Earth... use: Still destroying... fume: FATAL: Earth is being destroyed... 01. 02. 03. 04. 05. 06. 75
  76. 76. Backup kind: PlanetBackup target: name: Earth schedule: cron: * * * * * 01. 02. 03. 04. 05. 76
  77. 77. 10 days later... fume: There is no space left in space.01. 77
  78. 78. Real life state machines Kubernetes Terraform CloudFormation Serverless.com Ansible Puppet etc. • • • • • • • 78
  79. 79. Idempotance 79
  80. 80. Idempotance 80
  81. 81. Idempotent #!/bin/bash apt-get -y install nano 01. 02. 81
  82. 82. Not idempotent #!/bin/bash apt-get -y install nano useradd andrey useradd: user 'andrey' already exists 01. 02. 03. 04.05. 82
  83. 83. More code #!/bin/bash apt-get -y install nano if ! id -u andrey > /dev/null 2>&1; then useradd andrey fi 01. 02. 03. 04. 05. 83
  84. 84. Declarative catalogue - user: name: andrey shell: /bin/zsh - package: name: nano version: 1.23.1-1 01. 02. 03. 04. 05. 06. 84
  85. 85. Readability! 85
  86. 86. Pipeline- integratable 86
  87. 87. Abstraction 87
  88. 88. Resource graph 88
  89. 89. Parallel execution 89
  90. 90. Automatic ordering 90
  91. 91. Modularity 91
  92. 92. Data query - package: name: nano-${os.version} version: 1.23.1-1 01. 02. 03. 92
  93. 93. Resource deletion - user: name: andrey shell: /bin/zsh # - package: # name: nano # version: 1.23.1-1 01. 02. 03. 04. 05. 06. 93
  94. 94. Attribute management Write-once Updatable, applied right-away Updatable, requires extra action to take effect Updatable, forces resource recreation • • • • 94
  95. 95. Resource recreation 95
  96. 96. Resource protection 96
  97. 97. Dry-run 97
  98. 98. Compensating Compensating actions may sometimes be required Not every state change is allowed Any software has bugs! • • • 98
  99. 99. State verification Deployment-time Run-time • • 99
  100. 100. Module-level dependencies 100
  101. 101. Resource- level dependencies101
  102. 102. SM-index idempotence maturity declarativeness maturity (readiblity) data query richness execution engine maturity (parallel execution, auto-ordering of dependencies, deletion of obsolete objects) modularity maturity documentation maturity • • • • • • 102
  103. 103. 103
  104. 104. 104
  105. 105. Ansible machine 105
  106. 106. 106
  107. 107. Ansible machine Signs of state management, idempotance is not always achievable. Can be parallel only on node level, not resource (task) level. No model for resource dependencies. Imperative logic elements implemented in a declarative, but hardly readable way ( register , set_fact , when ). • • • • 107
  108. 108. Kubernetes machine 108
  109. 109. 109
  110. 110. Kubernetes machine api-server will accept any valid object definition. Will it be created and running? Eventually, it should. Run-time state validation. Not all attributes are changeable. Custom resource definitions (+ operators). Queries and dependencies only through labels. • • • • • • 110
  111. 111. kubectl wait 111
  112. 112. Helm 112
  113. 113. Terraform machine 113
  114. 114. 114
  115. 115. Terraform machine Terraform remembers previous state. Deployment time validation, no run-time validation. Not YAML, rich expression language. Powerful query capabilities. • • • • 115
  116. 116. Final words 116
  117. 117. Side effects are real Know the tool Know which resources you are managing Know the side effects • • • 117
  118. 118. Tools There are tools for tools: linters, IDE plugins, schema validators. 118
  119. 119. Everything is a spectrum Avoid dichotomy (bad/good, persistent/volatlie etc.), use a spectrum to define your relationship with a tool/language. 119
  120. 120. Keep it sane If you need a complex logic, use a %&?# programming language or a better declarative language with functions and variables. Infrastructure-as-code does not mean you should use the same data language all over the place. Do not use YAML for imperative logic. Do not let the history repeating. • • • 120
  121. 121. Keep it simple 121
  122. 122. Thank you! 122
  123. 123. 123
  124. 124. 124
  125. 125. References 125
  126. 126. Whining https://arp242.net/yaml-config.html https://noyaml.com https://boyter.org/posts/an-informal-survey-of-10-million-github- bitbucket-gitlab-projects/#yaml-or-yml https://www.darkcoding.net/software/a-developer-goes-to-a-devops- conference/ • • • • 126
  127. 127. Languages https://jsonnet.org/ https://json5.org/ https://dhall-lang.org/ https://get-ytt.io/ https://github.com/bazelbuild/starlark https://github.com/hashicorp/hcl2/blob/master/hcl/hclsyntax/spec.md https://github.com/toml-lang/toml#toml https://www.openpolicyagent.org/docs/latest/policy-language/ https://en.wikipedia.org/wiki/Datalog • • • • • • • • • 127
  128. 128. Tips https://www.redhat.com/sysadmin/yaml-tips https://networkgenomics.com/ansible/ • • 128
  129. 129. Kubernetes https://www.infoq.com/articles/kubernetes-workloads-serverless-era/• 129

×