Download as PDF, PPTX
![retries.yaml
Retries
Serverless orchestration and automation with GCP Workflows @martonkodok
Retries HTTP status codes [429, 502, 503, 504], connection error, or timeout](https://image.slidesharecdn.com/serverlessorchestrationandautomationwithgcpworkflows1-201215211428/85/Serverless-orchestration-and-automation-with-Cloud-Workflows-21-320.jpg)
Uploaded byMárton Kodok
Serverless orchestration and automation with Cloud Workflows
The document presents a talk on serverless orchestration and automation using GCP Workflows, highlighting challenges in connecting services and introducing workflows as a solution for automating complex processes. It covers practical use cases, including integration with APIs and microservices, as well as features like error handling and authentication. The presentation emphasizes the ease, scalability, and flexibility of GCP Workflows for building and managing serverless applications.
More Related Content
Similar to Serverless orchestration and automation with Cloud Workflows
Serverless orchestration and automation with Cloud Workflows
- 1. Serverless orchestration andautomation with GCPWorkflows December 2020 talk for GDG Cloud Portland Márton Kodok / @martonkodok Google Developer Expert at REEA.net
- 2. ● Among theTop 3 romanians on Stackoverflow 185k reputation ● Google Developer Expert on Cloud technologies ● Crafting Web/Mobile backends at REEA.net ● BigQuery + Redis database engine expert Slideshare: martonkodok Twitter: @martonkodok StackOverflow: pentium10 GitHub: pentium10 Serverless orchestration and automation with GCP Workflows @martonkodok About me
- 3. 1. Challenges inconnecting services 2. What is Workflows? - HTTP based service orchestration and automation 3. Introduction to Workflows - automate complex processes 4. Practical use cases 5. Automate, orchestrate and provide reliable line-of-business automation 6. Conclusions Agenda Serverless orchestration and automation with GCP Workflows @martonkodok
- 4. Connectivity - shouldbe easy, but in reality you need to figure out ● Common connection format ● Make the connections ● Parse the results ● Decisions and conditional step executions ● Error handling, logging ● Retries ● Scaling up and down to zero ● Authentication Challenges in connecting services Serverless orchestration and automation with GCP Workflows @martonkodok
- 5. Meet Workflows https://cloud.google.com/workflows Serverless orchestrationand automation with GCP Workflows @martonkodok
- 6. Workflows in GoogleCloud portfolio Introduction Orchestrate any Google Cloud API, SaaS API or private APIs. Serverless Compute External API’s Google API’s etc... Workflows - orchestrate & integrate SaaS API’s Private API’s Other Clouds
- 7. Step-Automation-as-a-Service - ServerlessHTTP service automation Declarative workflow language (YAML, JSON) Decent pricing (internal: $1/100K steps, external: $2.5/100K) *Dec 2020 Built-in decision and conditional executions expression formulas, operation on var Subworkflows similar to routine in a programming language with input/return var Support for external API calls out of the box support outside of Google Cloud Integrates with any Google Cloud product without worrying about authentication What is GCP Workflows? Serverless orchestration and automation with GCP Workflows @martonkodok
- 8. OAuth, OIDC, SecretManager integration Enterprise Security Keep your workflows secure X Authenticated Invocations Authenticated calls to Google Cloud services Integration with Secret Manager Encryption at rest and in transit External API 1 External API 2 External API
- 9. Code example @martonkodok - callMyFunction: call:http.get args: url: https://us-central1-project123.cloudfunctions.net/... query: metric: NoOfTrs result: metricResult - saveResult: switch: - condition: ${ metricResult.body.TrNo > 100 } call: http.post args: url: https://mydeployment.notify.... body: Metric: ${ metricResult.body.TrNo } getMetric TrNo>100? End YesNo {“metric”:”NoOfTrs”} Notify
- 10. hello.yaml Anatomy of aCloud Workflow! Serverless orchestration and automation with GCP Workflows @martonkodok workflowRevisionId: 000001-de1 argument: 'null' startTime: '2020-10-16T20:19:34.448323739Z' name: projects/985596417983/locations/us-central1/workflows/hello/executions/95c99744-c73c- 4065-a696-940bc7658c33 result: '"Hello, Cloud Workflows!"' endTime: '2020-10-16T20:19:34.845536315Z' state: SUCCEEDED Deploying gcloud beta workflows deploy hello --source=hello.yaml Executing gcloud beta workflows execute hello --data={“var”:”value”} Describing gcloud beta workflows executions describe-last
- 11. Sample Workflows Serverless orchestrationand automation with GCP Workflows @martonkodok
- 12. E-commerce invoice generationwith Workflows Steps orchestration Reliable execution, with error handling and retries Orchestration microservices or other API’s Create an invoice Generate PDF Send PDF via email
- 13. Sending reminders toaccounts with overdue payments Process array elements Execute steps conditionally For each customer: Get a list of customers Send reminder overdue ? Yes
- 14. IT management automation Combineautomation with scheduler Wait for service checks Orchestrate work across Compute Engine, PubSub, Stackdriver and other Google Cloud Products 9 AM trigger Start a Compute Engine VM Log the event App Started? No Wait 60 seconds Notify the team
- 15. Code Examples Serverless orchestrationand automation with GCP Workflows @martonkodok
- 16. HTTP Post Sequence twosteps HTTP Post combined with Secret Manager credentials Switch block Working with subworkflows Code Examples Serverless orchestration and automation with GCP Workflows @martonkodok
- 17. http_post.yaml Making an externalHTTP POST request Serverless orchestration and automation with GCP Workflows @martonkodok { "archived":false, "created_at":"2020-10-16T17:40:17+0000", "id":"bit.ly/35452TM", "link":"https://bit.ly/35452TM", "long_url":"<truncated>", }
- 18. wikipedia.yaml Sequence two stepsto get data from Wikipedia Serverless orchestration and automation with GCP Workflows @martonkodok
- 19. switch.yaml Switch block Serverless orchestrationand automation with GCP Workflows @martonkodok 9 13 14 9 13 14
- 20. subworkflow.yaml Subworkflow Serverless orchestration andautomation with GCP Workflows @martonkodok call call def
- 21. retries.yaml Retries Serverless orchestration andautomation with GCP Workflows @martonkodok Retries HTTP status codes [429, 502, 503, 504], connection error, or timeout
- 22. Orchestrate and automatethe Cloud Serverless orchestration and automation with GCP Workflows @martonkodok
- 23. Using Workflows toload Cloud Storage files into BigQuery - Cloud Workflows - Cloud Storage - BigQuery Full guide article on: martonkodok.medium.com Cloud Storage BigQueryCloud Workflows Authenticated Invocations foreach
- 24. Run shell commandsand orchestrate Compute Engine VMs - Cloud Workflows - Cloud Build - Compute Engine - Identity-Aware Proxy Full guide article on: martonkodok.medium.com Cloud Build Shell command Serverless Secure Connect Exec command Firewall Compute Engine Cloud IAP Tunnel Cloud Workflows
- 25. Run shell commandsand orchestrate Compute Engine VMs - Cloud Workflows - Cloud Build - Compute Engine - Identity-Aware Proxy Full guide article on: martonkodok.medium.com Cloud Workflow Steps Cloud Workflows Execution Authorize Cloud Build Roles and Permissions Cloud IAM Shell Service Account Start VM Compute Engine Submit Stop VM Compute Engine Firewall Compute Engine Cloud IAP Tunnel wait wait
- 26. Conclusions Serverless orchestration andautomation with GCP Workflows @martonkodok
- 27. Reliable workflow execution- execute workflows for enterprise business apps Low latency of execution - no cold starts Built-in error handling out of the box error handling with configurable retry policies Passing variable built-in JSON parsing and expression-based variable manipulation Rich runtime iterating through an array, embedded steps for readability Secret Manager integration out of the box Cloud Logging out of the box integration with Cloud Logging Reading from Firestore read/write an entry using Yaml syntax Benefits of Cloud Workflows Serverless orchestration and automation with GCP Workflows @martonkodok
- 28. “Automate, orchestrate andprovide reliable line-of-business automation. Serverless orchestration and automation with GCP Workflows @martonkodok Google Cloud Workflows
- 29. “Enables kids tobuild their first serverless product by using only YAML language. Serverless orchestration and automation with GCP Workflows @martonkodok Google Cloud Workflows
- 30. Easy to build/operate Scalesout Does not lose state Handles errors/timeouts Out-of-the-box support of Cloud APIs Auditable Developer friendly Serverless orchestration and automation with GCP Workflows @martonkodok
- 31. The possibilities areendless Marketing Retail IndustrialandIoT Developer Event driven marketing workflow execution Relay conversions to customer profiles in external services Workflow based emails, discounts, promotions Order management Inventory chain operations Data gathering and processing Synchronize systems Generate state machines Verify equipment lifecycle Workflow based maintenance needs Digitalization of internal policies Automate the Cloud Shell-script replacement Orchestrate devops workflows @martonkodok
- 32. Thank you. Q&A. Slidesavailable on: slideshare.net/martonkodok Reea.net - Integrated web solutions driven by creativity to deliver projects.