Spinnaker and SpEL

Evolving an Expression Language for
Continuous Delivery Pipelines
|Tomas Lin
@tomaslin
BakeTrigger Deploy Smoke
Test
Approval
BakeTrigger Deploy Smoke
Test
Approval
Branch
name
BakeTrigger Deploy Smoke
Test
Approval
Branch
name
myapp-feature-
Branch
name
BakeTrigger Deploy Smoke
Test
Approval
run smoke tests?
BakeTrigger Deploy Smoke
Test
Approval
run smoke tests?
Spring Expression Language
( SpEL )
“The Spring Expression Language (SpEL for
short) is a powerful expression language that
supports querying and manipulating...
Also in:
Spring 







Thymeleaf





a. Cluster-${ 7+2 } Cluster-9
b. Cluster-${ 7+2 }-${ 9 + 1 }-${ ‘string’ } Cluster-9-10-string
c. ${‘ some stringaroony ’....
a. ${ trigger.user ?: ‘britney’ }
b. ${ deploymentDetails?.[ region == ‘us-west-1’ ] }
c. ${ #toJson( deploymentDetails ) ...
Yay
We were already heavily invested in Spring Ecosystem - Spring Boot, Spring Batch.
No external library dependency.
Seem...
Nay
Couldn’t find a lot of examples of usage outside of Spring and Thymeleaf.
Could it work outside of managing bean depen...
Spinnaker Pipelines
Open Source successor to Asgard
Multi-region, multi-account, multi-cloud
Used to deploy around 98% of microservices
at Net...
Resize, Disable, Enable, Destroy, Deploy, Clone
Shrink, Scale Down, Disable
Wiring up SpEL
Note: Everything we’re talking about is available
in Github.

Easiest way to check it out is search for
‘ContextParameterP...
Trigger
( builds a ami )
Smoke Test Merge PR
Enable
Server
Group
Deploy
Server
Group
Customizing the parser
task context
Evaluating SpEL
evaluationContext
${ SpEL expression }
parserContext
result
${ job }
faily-godmother
Adding Properties
Execution - the current pipeline Execution
Trigger - the pipeline trigger
Parameters - pipeline parameters
trigger - pipel...
${ deployedServerGroups?.[region==‘us-east-1’][‘serverGroup’] }
myapp-v001
Adding Methods
Execution - the current pipeline Execution
Trigger - the pipeline trigger
Parameters - pipeline parameters
trigger - pipel...
Execution - the current pipeline Execution
Trigger - the pipeline trigger
Parameters - pipeline parameters
trigger - pipel...
${ #alphanumerical(‘jenny from the block’) }
jennyfromtheblock
Getting data from Jenkins
Trigger
Smoke Test Merge PR
ami=i-xxxyyyy
region=us-west-1
Property Files
${ trigger.properties[‘ami’] }
Trigger
Smoke Test Merge PR
retries=true
number of retries=2
${ retries }
${ #stage(‘Smoke Test’)[‘context’][‘retries’] }
Example: Jenkins parameter
passing
${ #alphanumerical( trigger.properties[‘us-west-1’] ).toUpperCase() }
Global SpEL Evaluation
stage
stage
task
task
task
context
definition
stage
stage
task
task
task
context
definitiondefinition
stage
Not everything is a String…
}
Pipeline parameters
Using SpEL to control
execution of stages
}
Using SpEL in Spinnaker UI
Common mistakes around
Expressions
Culture of Freedom and Responsibility
You build it, you run it, you support it.
#spinnaker slack channel
Office Hours
Anyt...
${ parameters.ami ?: ${ ‘ami’+ ‘x’} }
${ parameters.use ami in us-west-1 }
${ parameters[‘use ami in us-west-1’] }
${ #deploymentDetails?.[ region ==
parameters.region ] }
${ #deploymentDetails?.[ region ==
#root.parameters.region ] }
${ ‘a’+ ‘b’} ${ derp } ${ 1 + 1 }
“${ my escaped expression }”.toString()
Simplifying Stage Executions
read from url
#jsonFromUrl( url )
${ #jsonFromUrl( ‘http://archeius/rest/v1/
props/myapp’)[‘propertiesList’][0][‘value’]}
${ deploymentDetails[0].jenkins.nu...
Expressions can be a nice
temporary solution
${ trigger.user ?: ‘myteam@netflix.com’}
What’s next?
Better support for on-the-fly expression
testing and construction.
Helper methods such as #pipeline( ‘xxx’) and
#stageInPi...
Results
Over 1100 pipelines running in production ( of
5100 + ) use at least one SpEL expression.
Thank you.
spinnaker.io jobs.netflix.com
Try Spinnaker Come work with us!
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix
Upcoming SlideShare
Loading in …5
×

Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix

306 views

Published on

SpringOne Platform 2016
Speaker: Tomas Lin; Senior Software Engineer, Netflix.

Spinnaker is an open source, multi-cloud continuous delivery platform. It is used by over 90% of cloud deployments at Netflix. The Spring Expression Language (SpEL) is heavily leveraged to fully articulate and extend Spinnaker’s automated deployment pipelines. SpEL is used heavily to interact with pipeline stages and triggers for variable substitution, information retrieval and conditional evaluation. This session will walk through the evolution of Spinnaker’s use of SpEL over two years of active development. We will share the code, features and wisdom gained from helping the Netflix and open source community adopt SpEL. You should attend if you are interested in the Spinnaker platform and the continuous delivery pipelines feature, or wish to learn how to fully harness SpEL for your own tools.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
306
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
39
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Putting a SpEL on Spinnaker: Evolving an Expression Language for Continuous Delivery at Netflix

  1. 1. Spinnaker and SpEL
 Evolving an Expression Language for Continuous Delivery Pipelines |Tomas Lin @tomaslin
  2. 2. BakeTrigger Deploy Smoke Test Approval
  3. 3. BakeTrigger Deploy Smoke Test Approval Branch name
  4. 4. BakeTrigger Deploy Smoke Test Approval Branch name myapp-feature- Branch name
  5. 5. BakeTrigger Deploy Smoke Test Approval run smoke tests?
  6. 6. BakeTrigger Deploy Smoke Test Approval run smoke tests?
  7. 7. Spring Expression Language ( SpEL )
  8. 8. “The Spring Expression Language (SpEL for short) is a powerful expression language that supports querying and manipulating an object graph at runtime. The language syntax is similar to Unified EL but offers additional features, most notably method invocation and basic string templating functionality.” - Spring documentation Chapter 10
  9. 9. Also in: Spring 
 
 
 
 Thymeleaf
 
 

  10. 10. a. Cluster-${ 7+2 } Cluster-9 b. Cluster-${ 7+2 }-${ 9 + 1 }-${ ‘string’ } Cluster-9-10-string c. ${‘ some stringaroony ’.trim() } some stringaroony d. ${ deploymentDetails[0] } e. ${ trigger[‘parameters’][‘ideal temperature’] }
  11. 11. a. ${ trigger.user ?: ‘britney’ } b. ${ deploymentDetails?.[ region == ‘us-west-1’ ] } c. ${ #toJson( deploymentDetails ) } d. ${ new java.text.SimpleDateFormat('mm-dd-yyyy').format(new java.util.Date()) }
  12. 12. Yay We were already heavily invested in Spring Ecosystem - Spring Boot, Spring Batch. No external library dependency. Seem pretty robust and powerful. Looked to have great documentation.

  13. 13. Nay Couldn’t find a lot of examples of usage outside of Spring and Thymeleaf. Could it work outside of managing bean dependencies? Unsure of how easy it was to extend and customize. Did it have a heavy dependency on Java?
  14. 14. Spinnaker Pipelines
  15. 15. Open Source successor to Asgard Multi-region, multi-account, multi-cloud Used to deploy around 98% of microservices at Netflix. http://spinnaker.io
  16. 16. Resize, Disable, Enable, Destroy, Deploy, Clone
  17. 17. Shrink, Scale Down, Disable
  18. 18. Wiring up SpEL
  19. 19. Note: Everything we’re talking about is available in Github.
 Easiest way to check it out is search for ‘ContextParameterProcessor’ in orca, https://github.com/spinnaker/orca/search? q=contextparameterprocessor
  20. 20. Trigger ( builds a ami ) Smoke Test Merge PR Enable Server Group Deploy Server Group
  21. 21. Customizing the parser
  22. 22. task context
  23. 23. Evaluating SpEL evaluationContext ${ SpEL expression } parserContext result
  24. 24. ${ job } faily-godmother
  25. 25. Adding Properties
  26. 26. Execution - the current pipeline Execution Trigger - the pipeline trigger Parameters - pipeline parameters trigger - pipeline trigger parameters - pipeline parameters scmInfo - git details in trigger deployedServerGroups : details from last cluster deployed execution the current pipeline execution parameters pipeline parameters trigger pipeline trigger scmInfo git details deploymentDetails results of the findImage or Bake stage deployedServerGroups last created server group
  27. 27. ${ deployedServerGroups?.[region==‘us-east-1’][‘serverGroup’] } myapp-v001
  28. 28. Adding Methods
  29. 29. Execution - the current pipeline Execution Trigger - the pipeline trigger Parameters - pipeline parameters trigger - pipeline trigger parameters - pipeline parameters scmInfo - git details in trigger deployedServerGroups : details from last cluster deployed #alphanumerical trims all non-alphanumerical characters #fromJson converts JSON to a map #fromUrl contents from a url #jsonFromUrl JSON content from an url #judgment choice from manual judgment stage
  30. 30. Execution - the current pipeline Execution Trigger - the pipeline trigger Parameters - pipeline parameters trigger - pipeline trigger parameters - pipeline parameters scmInfo - git details in trigger deployedServerGroups : details from last cluster deployed #stage shortcut to stage contents by name #toBoolean conversion to boolean #toFloat conversion to float #toInt conversion to int #toJson conversion from Map to Json
  31. 31. ${ #alphanumerical(‘jenny from the block’) } jennyfromtheblock
  32. 32. Getting data from Jenkins
  33. 33. Trigger Smoke Test Merge PR ami=i-xxxyyyy region=us-west-1 Property Files
  34. 34. ${ trigger.properties[‘ami’] }
  35. 35. Trigger Smoke Test Merge PR retries=true number of retries=2
  36. 36. ${ retries } ${ #stage(‘Smoke Test’)[‘context’][‘retries’] }
  37. 37. Example: Jenkins parameter passing
  38. 38. ${ #alphanumerical( trigger.properties[‘us-west-1’] ).toUpperCase() }
  39. 39. Global SpEL Evaluation
  40. 40. stage stage task task task context definition
  41. 41. stage stage task task task context definitiondefinition stage
  42. 42. Not everything is a String…
  43. 43. }
  44. 44. Pipeline parameters
  45. 45. Using SpEL to control execution of stages
  46. 46. }
  47. 47. Using SpEL in Spinnaker UI
  48. 48. Common mistakes around Expressions
  49. 49. Culture of Freedom and Responsibility You build it, you run it, you support it. #spinnaker slack channel Office Hours Anytime walk-in 1-1 help with first responder
  50. 50. ${ parameters.ami ?: ${ ‘ami’+ ‘x’} }
  51. 51. ${ parameters.use ami in us-west-1 } ${ parameters[‘use ami in us-west-1’] }
  52. 52. ${ #deploymentDetails?.[ region == parameters.region ] } ${ #deploymentDetails?.[ region == #root.parameters.region ] }
  53. 53. ${ ‘a’+ ‘b’} ${ derp } ${ 1 + 1 }
  54. 54. “${ my escaped expression }”.toString()
  55. 55. Simplifying Stage Executions
  56. 56. read from url
  57. 57. #jsonFromUrl( url )
  58. 58. ${ #jsonFromUrl( ‘http://archeius/rest/v1/ props/myapp’)[‘propertiesList’][0][‘value’]} ${ deploymentDetails[0].jenkins.number != #jsonFromUrl( ‘https://spinnakerapi/ applications/orca/clusters/mgmt/orca-main/ aws/us-west-2/serverGroups/target/NEWEST? onlyEnabled=true' )['buildInfo']['jenkins'] ['number'] }
  59. 59. Expressions can be a nice temporary solution
  60. 60. ${ trigger.user ?: ‘myteam@netflix.com’}
  61. 61. What’s next?
  62. 62. Better support for on-the-fly expression testing and construction. Helper methods such as #pipeline( ‘xxx’) and #stageInPipeline( ‘Wait’, ‘xxx’). Better UI support.
  63. 63. Results
  64. 64. Over 1100 pipelines running in production ( of 5100 + ) use at least one SpEL expression.
  65. 65. Thank you. spinnaker.io jobs.netflix.com Try Spinnaker Come work with us!

×