Join this session to understand how Cloud Workflows resolves challenges in connecting services, HTTP based service orchestration and automation. We are going to dive deep how serverless HTTP service automation works to automate step engines. Based on practical examples we will demonstrate the built-in decision and conditional executions, subworkflows, support for external built-in API calls, and integration with any Google Cloud product without worrying about authentication. We are going to cover Marketing, Retail, Industrial and Developer possibilities, such as event driven marketing workflow execution, or inventory chain operations, generating and automatic state machines, or orchestrate DevOps workflows and automating the Cloud.

1. Serverless orchestration and automation
with GCPWorkflows
December 2020 talk for GDG Cloud Portland
Márton Kodok / @martonkodok
Google Developer Expert at REEA.net

2. ● Among the Top 3 romanians on Stackoverflow 185k reputation
● Google Developer Expert on Cloud technologies
● Crafting Web/Mobile backends at REEA.net
● BigQuery + Redis database engine expert
Slideshare: martonkodok
Twitter: @martonkodok
StackOverflow: pentium10
GitHub: pentium10
Serverless orchestration and automation with GCP Workflows @martonkodok
About me

3. 1. Challenges in connecting services
2. What is Workflows? - HTTP based service orchestration and automation
3. Introduction to Workflows - automate complex processes
4. Practical use cases
5. Automate, orchestrate and provide reliable line-of-business automation
6. Conclusions
Agenda
Serverless orchestration and automation with GCP Workflows @martonkodok

4. Connectivity - should be easy, but in reality you need to figure out
● Common connection format
● Make the connections
● Parse the results
● Decisions and conditional step executions
● Error handling, logging
● Retries
● Scaling up and down to zero
● Authentication
Challenges in connecting services
Serverless orchestration and automation with GCP Workflows @martonkodok

5. Meet Workflows
https://cloud.google.com/workflows
Serverless orchestration and automation with GCP Workflows @martonkodok

6. Workflows in Google Cloud portfolio
Introduction
Orchestrate any Google Cloud
API, SaaS API or private APIs.
Serverless
Compute
External
API’s
Google
API’s
etc...
Workflows - orchestrate & integrate
SaaS
API’s
Private
API’s
Other
Clouds

7. Step-Automation-as-a-Service - Serverless HTTP service automation
Declarative workflow language (YAML, JSON)
Decent pricing (internal: $1/100K steps, external: $2.5/100K) *Dec 2020
Built-in decision and conditional executions expression formulas, operation on var
Subworkflows similar to routine in a programming language with input/return var
Support for external API calls out of the box support outside of Google Cloud
Integrates with any Google Cloud product without worrying about authentication
What is GCP Workflows?
Serverless orchestration and automation with GCP Workflows @martonkodok

8. OAuth, OIDC, Secret Manager integration
Enterprise
Security
Keep your workflows secure
X
Authenticated
Invocations
Authenticated
calls to Google Cloud
services
Integration with
Secret Manager
Encryption at rest and
in transit
External API 1
External API 2 External API

9. Code example
@martonkodok
- callMyFunction:
call: http.get
args:
url: https://us-central1-project123.cloudfunctions.net/...
query:
metric: NoOfTrs
result: metricResult
- saveResult:
switch:
- condition: ${ metricResult.body.TrNo > 100 }
call: http.post
args:
url: https://mydeployment.notify....
body:
Metric: ${ metricResult.body.TrNo }
getMetric
TrNo>100?
End
YesNo
{“metric”:”NoOfTrs”}
Notify

10. hello.yaml
Anatomy of a Cloud Workflow!
Serverless orchestration and automation with GCP Workflows @martonkodok
workflowRevisionId: 000001-de1
argument: 'null'
startTime: '2020-10-16T20:19:34.448323739Z'
name:
projects/985596417983/locations/us-central1/workflows/hello/executions/95c99744-c73c-
4065-a696-940bc7658c33
result: '"Hello, Cloud Workflows!"'
endTime: '2020-10-16T20:19:34.845536315Z'
state: SUCCEEDED
Deploying gcloud beta workflows deploy hello --source=hello.yaml
Executing gcloud beta workflows execute hello --data={“var”:”value”}
Describing gcloud beta workflows executions describe-last

11. Sample Workflows
Serverless orchestration and automation with GCP Workflows @martonkodok

12. E-commerce invoice generation with Workflows
Steps orchestration
Reliable execution, with error
handling and retries
Orchestration microservices
or other API’s
Create an invoice
Generate PDF
Send PDF via email

13. Sending reminders to accounts with overdue payments
Process array elements
Execute steps
conditionally
For each customer:
Get a list of customers
Send reminder
overdue
?
Yes

14. IT management automation
Combine automation with
scheduler
Wait for service checks
Orchestrate work across
Compute Engine, PubSub,
Stackdriver and other Google
Cloud Products
9 AM trigger
Start a Compute
Engine VM
Log the event
App
Started?
No
Wait 60 seconds
Notify the team

15. Code Examples
Serverless orchestration and automation with GCP Workflows @martonkodok

16. HTTP Post
Sequence two steps
HTTP Post combined with Secret Manager credentials
Switch block
Working with subworkflows
Code Examples
Serverless orchestration and automation with GCP Workflows @martonkodok

17. http_post.yaml
Making an external HTTP POST request
Serverless orchestration and automation with GCP Workflows @martonkodok
{
"archived":false,
"created_at":"2020-10-16T17:40:17+0000",
"id":"bit.ly/35452TM",
"link":"https://bit.ly/35452TM",
"long_url":"<truncated>",
}

18. wikipedia.yaml
Sequence two steps to get data from Wikipedia
Serverless orchestration and automation with GCP Workflows @martonkodok

19. switch.yaml
Switch block
Serverless orchestration and automation with GCP Workflows @martonkodok
9 13 14
9
13
14

20. subworkflow.yaml
Subworkflow
Serverless orchestration and automation with GCP Workflows @martonkodok
call
call
def

21. retries.yaml
Retries
Serverless orchestration and automation with GCP Workflows @martonkodok
Retries HTTP status codes [429, 502, 503, 504], connection error, or timeout

22. Orchestrate and automate the Cloud
Serverless orchestration and automation with GCP Workflows @martonkodok

23. Using Workflows to load Cloud Storage files into BigQuery
- Cloud Workflows
- Cloud Storage
- BigQuery
Full guide article on:
martonkodok.medium.com
Cloud
Storage
BigQueryCloud
Workflows
Authenticated
Invocations
foreach

24. Run shell commands and orchestrate Compute Engine VMs
- Cloud Workflows
- Cloud Build
- Compute Engine
- Identity-Aware Proxy
Full guide article on:
martonkodok.medium.com
Cloud Build
Shell command
Serverless Secure Connect Exec command
Firewall
Compute Engine
Cloud IAP Tunnel
Cloud
Workflows

25. Run shell commands and orchestrate Compute Engine VMs
- Cloud Workflows
- Cloud Build
- Compute Engine
- Identity-Aware Proxy
Full guide article on:
martonkodok.medium.com
Cloud Workflow Steps
Cloud
Workflows
Execution
Authorize
Cloud Build
Roles and
Permissions
Cloud IAM
Shell
Service Account
Start VM
Compute
Engine
Submit Stop VM
Compute
Engine
Firewall
Compute Engine
Cloud
IAP
Tunnel
wait wait

26. Conclusions
Serverless orchestration and automation with GCP Workflows @martonkodok

27. Reliable workflow execution - execute workflows for enterprise business apps
Low latency of execution - no cold starts
Built-in error handling out of the box error handling with configurable retry policies
Passing variable built-in JSON parsing and expression-based variable manipulation
Rich runtime iterating through an array, embedded steps for readability
Secret Manager integration out of the box
Cloud Logging out of the box integration with Cloud Logging
Reading from Firestore read/write an entry using Yaml syntax
Benefits of Cloud Workflows
Serverless orchestration and automation with GCP Workflows @martonkodok

28. “Automate, orchestrate and provide reliable
line-of-business automation.
Serverless orchestration and automation with GCP Workflows @martonkodok
Google Cloud Workflows

29. “Enables kids to build their first serverless
product by using only YAML language.
Serverless orchestration and automation with GCP Workflows @martonkodok
Google Cloud Workflows

30. Easy to build/operate
Scales out
Does not lose state
Handles errors/timeouts
Out-of-the-box support of Cloud APIs
Auditable
Developer friendly
Serverless orchestration and automation with GCP Workflows @martonkodok

31. The possibilities are endless
Marketing Retail IndustrialandIoT Developer
Event driven marketing
workflow execution
Relay conversions to
customer profiles in external
services
Workflow based emails,
discounts, promotions
Order management
Inventory chain operations
Data gathering and
processing
Synchronize systems
Generate state machines
Verify equipment lifecycle
Workflow based
maintenance needs
Digitalization of internal
policies
Automate the Cloud
Shell-script replacement
Orchestrate devops
workflows
@martonkodok

32. Thank you. Q&A.
Slides available on:
slideshare.net/martonkodok
Reea.net - Integrated web solutions driven by creativity
to deliver projects.