This document discusses AWS Step Functions, a serverless workflow service for coordinating Lambda functions and other services. It provides examples of how Step Functions allows sequencing functions, selecting functions based on data, retrying functions, handling errors with try/catch/finally, running long-running code over hours, and running functions in parallel. Quotes from customers discuss how Step Functions allows building automated workflows for food delivery operations and product data updates.
AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions.
Building Distributed Applications with AWS Step FunctionsAmazon Web Services
This talk covers how you can use the new AWS Step Functions service to coordinate different components of your application, maintain state, and build sophisticated serverless solutions.
AWS' philosophy and recommended best practices for building microservices applications, how AWS services like Lambda and API gateway benefit developers building microservices apps, and how customers are using these two and other AWS services to deliver their microservices apps
In this session we’ll take a high-level overview of AWS Lambda, a serverless compute platform that has changed the way that developers around the world build applications. We’ll explore how Lambda works under the hood, the capabilities it has, and how it is used. By the end of this talk you’ll know how to create Lambda based applications and deploy and manage them easily.
Speaker: Chris Munns - Principal Developer Advocate, AWS Serverless Applications, AWS
Training for AWS Solutions Architect at http://zekelabs.com/courses/amazon-web-services-training-bangalore/.This slide describes about cloud trail key concepts, workflow and event history
___________________________________________________
zekeLabs is a Technology training platform. We provide instructor led corporate training and classroom training on Industry relevant Cutting Edge Technologies like Big Data, Machine Learning, Natural Language Processing, Artificial Intelligence, Data Science, Amazon Web Services, DevOps, Cloud Computing and Frameworks like Django,Spring, Ruby on Rails, Angular 2 and many more to Professionals.
Reach out to us at www.zekelabs.com or call us at +91 8095465880 or drop a mail at info@zekelabs.com
AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions.
Building Distributed Applications with AWS Step FunctionsAmazon Web Services
This talk covers how you can use the new AWS Step Functions service to coordinate different components of your application, maintain state, and build sophisticated serverless solutions.
AWS' philosophy and recommended best practices for building microservices applications, how AWS services like Lambda and API gateway benefit developers building microservices apps, and how customers are using these two and other AWS services to deliver their microservices apps
In this session we’ll take a high-level overview of AWS Lambda, a serverless compute platform that has changed the way that developers around the world build applications. We’ll explore how Lambda works under the hood, the capabilities it has, and how it is used. By the end of this talk you’ll know how to create Lambda based applications and deploy and manage them easily.
Speaker: Chris Munns - Principal Developer Advocate, AWS Serverless Applications, AWS
Training for AWS Solutions Architect at http://zekelabs.com/courses/amazon-web-services-training-bangalore/.This slide describes about cloud trail key concepts, workflow and event history
___________________________________________________
zekeLabs is a Technology training platform. We provide instructor led corporate training and classroom training on Industry relevant Cutting Edge Technologies like Big Data, Machine Learning, Natural Language Processing, Artificial Intelligence, Data Science, Amazon Web Services, DevOps, Cloud Computing and Frameworks like Django,Spring, Ruby on Rails, Angular 2 and many more to Professionals.
Reach out to us at www.zekelabs.com or call us at +91 8095465880 or drop a mail at info@zekelabs.com
Slides for a short presentation I gave on AWS Lambda, which "lets you run code without provisioning or managing servers". Lambda is to running code as Amazon S3 is to storing objects.
Adapting the capacity of your compute infrastructure to the demands of your applications is the domain of Auto Scaling. Adding and removing Amazon EC2 instances is only part of the story, though – there is more to it than first meets the eye. This session introduces the basics of how to use Auto Scaling before moving on to more advanced topics such as mixing Spot and On-Demand instances to optimize cost or strategies for blue/green deployments. If you have used Auto Scaling before, you can learn about useful new features like lifecycle hooks and step scaling policies that make Auto Scaling even more widely applicable.
AWS CloudFormation is a comprehensive templating language that enables you to create managed 'stacks' of AWS resources, with a growing library of templates available for you to use. But how do you create one from scratch? This presentation will take you through building an AWS CloudFormation template from the ground up, so you can see all the essential template constructs in action.
Watch a recording of the webinar based on this presentation on YouTube here: http://youtu.be/6R44BADNJA8
Check out other upcoming webinars in the Masterclass Series here: http://aws.amazon.com/campaigns/emea/masterclass/
Amazon Athena is a new serverless query service that makes it easy to analyze data in Amazon S3, using standard SQL. With Athena, there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3.
AWS Lambda enables developers to build scalable application components with minimal effort. With Step Functions, we can solve the challenge of building large distributed applications using visual workflows. In this session, learn how to get started with Step Functions, and understand how to use them to take your Lambda-based applications to the next level. We start with a few granular functions and stitch them up using Step Functions. As we build the application, we'll add monitoring to ensure that the changes we make result in improvements.
by Joyjeet Banerjee, Solutions Architect, AWS
Amazon Athena is a new serverless query service that makes it easy to analyze data in Amazon S3, using standard SQL. With Athena, there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3. Level 200
In this session, we will show you how easy it is to start querying your data stored in Amazon S3, with Amazon Athena. First we will use Athena to create the schema for data already in S3. Then, we will demonstrate how you can run interactive queries through the built-in query editor. We will provide best practices and use cases for Athena. Then, we will talk about supported queries, data formats, and strategies to save costs when querying data with Athena.
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatchAmazon Web Services
You may already know that you can use Amazon CloudWatch to view graphs of your AWS resources like Amazon Elastic Compute Cloud instances or Amazon Simple Storage Service. But, did you know that you can monitor your on-premises servers with Amazon CloudWatch Logs? Or, that you can integrate CloudWatch Logs with Elasticsearch for powerful visualization and analysis? This session will offer a tour of the latest monitoring and automation capabilities that we’ve added, how you can get even more done with Amazon CloudWatch.
Come learn about new and existing Amazon S3 features that can help you better protect your data, save on cost, and improve usability, security, and performance. We will cover a wide variety of Amazon S3 features and go into depth on several newer features with configuration and code snippets, so you can apply the learnings on your object storage workloads.
Learn how the Blue/Green Deployment methodology combined with AWS tools and services can help reduce the risks associated with software deployment. We will illustrate common patterns and highlight ways deployment risks are mitigated by each pattern. Topics will include how services like AWS CloudFormation, AWS Elastic Beanstalk, Amazon EC2 Container Service, Amazon Route53, Auto Scaling and Elastic Load Balancing can help automate deployment. We will also address how to effectively manage deployments in the context of data model and schema changes. Learn how you can adopt blue/green for your software release processes in a cost-effective and low-risk way.
Deploy, Manage, and Scale your Apps with AWS Elastic BeanstalkAmazon Web Services
AWS Elastic Beanstalk is the fastest and simplest way to deploy your application on AWS. It is ideal for developers that are new to the platform but is also used by large organizations that want to manage and scale production workloads with minimum operational overhead. This session shows you how to deploy your code to AWS Elastic Beanstalk, easily manage multiple environments (e.g. Test & Production) and perform zero-downtime deployments through interactive demos and code samples.
Microservices architecture is a method of developing software applications as a suite of independently deployable, small, modular services. Learn how to leverage the security and automation of the Amazon Web Services platform, to build, maintain and operate a microservices environment.
Speaker: Adam Lynch, Sr. Technical Account Manager, Amazon Web Services
Do you want to run your code without the cost and effort of provisioning and managing servers? Find out how in this deep dive session on AWS Lambda, which allows you to run code for virtually any type of application or back end service – all with zero administration. During the session, we’ll look at a number of key AWS Lambda features and benefits, including automated application scaling with high availability; pay-as-you-consume billing; and the ability to automatically trigger your code from other AWS services or from any web or mobile app.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any Web application. Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management.
Presented by: Danilo Poccia, Technical Evangelist, Amazon Web Services
Amazon EC2 Container Service is a new AWS service that makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. Amazon EC2 Container Service lets you define, schedule, and stop sets of containers. You have access to the state of your resources, making it easy to confirm that tasks are running or view the utilization of Amazon EC2 instances in your cluster. This session will describe the benefits of containers, introduce the Amazon EC2 Container Service, and demonstrate how to use Amazon EC2 Container Service for your applications.
Speakers:
Ian Massingham, AWS Technical Evangelist and
Boyan Dimitrov, Platform Automation Lead, Hailo Cabs
Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key features, and the concept of instance generations.
AWS Control Tower is a new AWS service for cloud administrators to set up and govern their secure, compliant, multi-account environments on AWS.
In this session, University of York will discuss their implementation of AWS Landing Zone. We’ll also explain how AWS Control Tower automates AWS Landing Zone creation with best-practice blueprints.
AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions. We share how customers are using AWS Step Functions to reliably scale multi-step applications such as order processing, report generation, and data transformation–all without managing any infrastructure.
AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions, so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions. Learn how easy it is to create Step Functions state machines and activities using CloudFormation Templates, and then start them with Amazon API Gateway. We share how customers are using AWS Step Functions to reliably orchestrate and scale multi-step applications such as order processing, report generation, and data transformation–all without managing any infrastructure.
Slides for a short presentation I gave on AWS Lambda, which "lets you run code without provisioning or managing servers". Lambda is to running code as Amazon S3 is to storing objects.
Adapting the capacity of your compute infrastructure to the demands of your applications is the domain of Auto Scaling. Adding and removing Amazon EC2 instances is only part of the story, though – there is more to it than first meets the eye. This session introduces the basics of how to use Auto Scaling before moving on to more advanced topics such as mixing Spot and On-Demand instances to optimize cost or strategies for blue/green deployments. If you have used Auto Scaling before, you can learn about useful new features like lifecycle hooks and step scaling policies that make Auto Scaling even more widely applicable.
AWS CloudFormation is a comprehensive templating language that enables you to create managed 'stacks' of AWS resources, with a growing library of templates available for you to use. But how do you create one from scratch? This presentation will take you through building an AWS CloudFormation template from the ground up, so you can see all the essential template constructs in action.
Watch a recording of the webinar based on this presentation on YouTube here: http://youtu.be/6R44BADNJA8
Check out other upcoming webinars in the Masterclass Series here: http://aws.amazon.com/campaigns/emea/masterclass/
Amazon Athena is a new serverless query service that makes it easy to analyze data in Amazon S3, using standard SQL. With Athena, there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3.
AWS Lambda enables developers to build scalable application components with minimal effort. With Step Functions, we can solve the challenge of building large distributed applications using visual workflows. In this session, learn how to get started with Step Functions, and understand how to use them to take your Lambda-based applications to the next level. We start with a few granular functions and stitch them up using Step Functions. As we build the application, we'll add monitoring to ensure that the changes we make result in improvements.
by Joyjeet Banerjee, Solutions Architect, AWS
Amazon Athena is a new serverless query service that makes it easy to analyze data in Amazon S3, using standard SQL. With Athena, there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3. Level 200
In this session, we will show you how easy it is to start querying your data stored in Amazon S3, with Amazon Athena. First we will use Athena to create the schema for data already in S3. Then, we will demonstrate how you can run interactive queries through the built-in query editor. We will provide best practices and use cases for Athena. Then, we will talk about supported queries, data formats, and strategies to save costs when querying data with Athena.
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatchAmazon Web Services
You may already know that you can use Amazon CloudWatch to view graphs of your AWS resources like Amazon Elastic Compute Cloud instances or Amazon Simple Storage Service. But, did you know that you can monitor your on-premises servers with Amazon CloudWatch Logs? Or, that you can integrate CloudWatch Logs with Elasticsearch for powerful visualization and analysis? This session will offer a tour of the latest monitoring and automation capabilities that we’ve added, how you can get even more done with Amazon CloudWatch.
Come learn about new and existing Amazon S3 features that can help you better protect your data, save on cost, and improve usability, security, and performance. We will cover a wide variety of Amazon S3 features and go into depth on several newer features with configuration and code snippets, so you can apply the learnings on your object storage workloads.
Learn how the Blue/Green Deployment methodology combined with AWS tools and services can help reduce the risks associated with software deployment. We will illustrate common patterns and highlight ways deployment risks are mitigated by each pattern. Topics will include how services like AWS CloudFormation, AWS Elastic Beanstalk, Amazon EC2 Container Service, Amazon Route53, Auto Scaling and Elastic Load Balancing can help automate deployment. We will also address how to effectively manage deployments in the context of data model and schema changes. Learn how you can adopt blue/green for your software release processes in a cost-effective and low-risk way.
Deploy, Manage, and Scale your Apps with AWS Elastic BeanstalkAmazon Web Services
AWS Elastic Beanstalk is the fastest and simplest way to deploy your application on AWS. It is ideal for developers that are new to the platform but is also used by large organizations that want to manage and scale production workloads with minimum operational overhead. This session shows you how to deploy your code to AWS Elastic Beanstalk, easily manage multiple environments (e.g. Test & Production) and perform zero-downtime deployments through interactive demos and code samples.
Microservices architecture is a method of developing software applications as a suite of independently deployable, small, modular services. Learn how to leverage the security and automation of the Amazon Web Services platform, to build, maintain and operate a microservices environment.
Speaker: Adam Lynch, Sr. Technical Account Manager, Amazon Web Services
Do you want to run your code without the cost and effort of provisioning and managing servers? Find out how in this deep dive session on AWS Lambda, which allows you to run code for virtually any type of application or back end service – all with zero administration. During the session, we’ll look at a number of key AWS Lambda features and benefits, including automated application scaling with high availability; pay-as-you-consume billing; and the ability to automatically trigger your code from other AWS services or from any web or mobile app.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any Web application. Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management.
Presented by: Danilo Poccia, Technical Evangelist, Amazon Web Services
Amazon EC2 Container Service is a new AWS service that makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. Amazon EC2 Container Service lets you define, schedule, and stop sets of containers. You have access to the state of your resources, making it easy to confirm that tasks are running or view the utilization of Amazon EC2 instances in your cluster. This session will describe the benefits of containers, introduce the Amazon EC2 Container Service, and demonstrate how to use Amazon EC2 Container Service for your applications.
Speakers:
Ian Massingham, AWS Technical Evangelist and
Boyan Dimitrov, Platform Automation Lead, Hailo Cabs
Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key features, and the concept of instance generations.
AWS Control Tower is a new AWS service for cloud administrators to set up and govern their secure, compliant, multi-account environments on AWS.
In this session, University of York will discuss their implementation of AWS Landing Zone. We’ll also explain how AWS Control Tower automates AWS Landing Zone creation with best-practice blueprints.
AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions. We share how customers are using AWS Step Functions to reliably scale multi-step applications such as order processing, report generation, and data transformation–all without managing any infrastructure.
AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions, so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions. Learn how easy it is to create Step Functions state machines and activities using CloudFormation Templates, and then start them with Amazon API Gateway. We share how customers are using AWS Step Functions to reliably orchestrate and scale multi-step applications such as order processing, report generation, and data transformation–all without managing any infrastructure.
Announcing AWS Step Functions - December 2016 Monthly Webinar SeriesAmazon Web Services
AWS Step Functions is a new, fully managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to coordinate components and step through the functions of your application at scale. This session shows how to use Step Functions to create, run, and debug multi-service applications in a matter of minutes. We also share how customers are using Step Functions to build and operate complex applications, such as order processing, report generation, and data transformation – and to innovate faster.
Learning Objectives:
• Learn about the capabilities and features of AWS Step Functions
• Learn about the benefits of AWS Step Functions
• Learn about the different use cases
• Learn how to get started using AWS Step Functions
• Build and coordinate the components of your application as a series of steps
• Link AWS Lambda functions or Amazon EC2 instances together to create multi-step apps
• Learn to automatically catch and recover from errors
• Operate and scale your application in a reliable fashion
NEW LAUNCH! Building Distributed Applications with AWS Step FunctionsAmazon Web Services
AWS Step Functions is a new, fully managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to coordinate components and step through the functions of your application. A graphical console helps you arrange and visualize the components of your application as a series of steps. Step Functions automatically triggers and tracks each step and retries when there are errors so that your application executes in order―and as expected―every time. This session shows how to use Step Functions to create, run, and debug multi-service applications in a matter of minutes. We also share how customers are using Step Functions to reliably build and scale multi-step applications such as order processing, report generation, and data transformation―and to innovate faster.
Serverless Orchestration with AWS Step Functions - May 2017 AWS Online Tech T...Amazon Web Services
Learning Objectives:
- Learn how to build and operate serverless applications using Step Functions state machines
- See reference architectures, blueprints, and example use cases to get started quickly
- Know how to integrate Step Functions with other AWS services to develop and deploy applications faster
- Understand how to gain visibility into the operation of your application
- Learn how to automate and customize error handling
Are you building a serverless application with two or more Lambda functions? AWS Step Functions makes it easy to coordinate multiple functions and microservices as a series of steps using visual workflows. You create Step Functions state machines to specify and reliably step through the functions of your application at scale. In this deep-dive session, we will show how to use AWS CloudFormation and the AWS Serverless Application Model to deploy Step Functions state machines, AWS Lambda functions, and IAM roles and policies. We will demonstrate how AWS Step Functions state machines orchestrate state transitions and error handling, and how state input/output works.
Serverless Orchestration of AWS Step Functions - July 2017 AWS Online Tech TalksAmazon Web Services
Learning Objectives:
- Learn how to build and operate serverless applications using Step Functions state machines
- See reference architectures, blueprints, and example use cases to get started quickly
- Know how to integrate Step Functions with other AWS services to develop and deploy applications faster
Are you building a serverless application with two or more Lambda functions? AWS Step Functions makes it easy to coordinate multiple functions and microservices as a series of steps using visual workflows. You create Step Functions state machines to specify and reliably step through the functions of your application at scale. In this deep-dive session, we will show how to use AWS CloudFormation and the AWS Serverless Application Model to deploy Step Functions state machines, AWS Lambda functions, and IAM roles and policies. We will demonstrate how Step Functions state machines orchestrate state transitions and error handling, and how state input/output works.
Designing and Implementing a Serverless Media Processing Workflow Using AWS S...Amazon Web Services
This workshop demonstrates how to use AWS Step Functions to coordinate multiple AWS Lambda functions using visual workflows. You learn how to build a Step Functions state machine to orchestrate a multi-step serverless application. You work in teams to design and implement an image recognition and processing workflow using AWS Step Functions, AWS Lambda, Amazon S3, Amazon DynamoDB, and Amazon Rekognition. The workflow process photos uploaded to Amazon S3 and extract metadata from the image, such as geolocation, size, format, and time. It then uses image recognition to tag objects in the photo and produce a thumbnail. Prerequisites: Experience using AWS, an AWS account, AWS CLI. We provide AWS credits for use in the hands-on lab. Bring a laptop.
Create Agile, Automated and Predictable IT Infrastructure in the CloudRightScale
RightScale Webinar: February 9, 2010 – In this Webinar, RightScale founder Thorsten von Eicken walks you through the differences between RightScale's ServerTemplate technology and machine images. During the live demonstration, see how ServerTemplates allow you to deploy agile, automated and predictable IT infrastructure in the cloud using RightScale. We also present research on how much time our customers have saved by utilizing this technology.
Similar to Introduction to AWS Step Functions (20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
Mastering the Concepts Tested in the Databricks Certified Data Engineer Assoc...SkillCertProExams
• For a full set of 760+ questions. Go to
https://skillcertpro.com/product/databricks-certified-data-engineer-associate-exam-questions/
• SkillCertPro offers detailed explanations to each question which helps to understand the concepts better.
• It is recommended to score above 85% in SkillCertPro exams before attempting a real exam.
• SkillCertPro updates exam questions every 2 weeks.
• You will get life time access and life time free updates
• SkillCertPro assures 100% pass guarantee in first attempt.
Collapsing Narratives: Exploring Non-Linearity • a micro report by Rosie WellsRosie Wells
Insight: In a landscape where traditional narrative structures are giving way to fragmented and non-linear forms of storytelling, there lies immense potential for creativity and exploration.
'Collapsing Narratives: Exploring Non-Linearity' is a micro report from Rosie Wells.
Rosie Wells is an Arts & Cultural Strategist uniquely positioned at the intersection of grassroots and mainstream storytelling.
Their work is focused on developing meaningful and lasting connections that can drive social change.
Please download this presentation to enjoy the hyperlinks!
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
4. “I want to sequence functions”
“I want to select functions based on data”
“I want to retry functions”
“I want try/catch/finally”
Functions into apps
“I have code that runs for hours”
“I want to run functions in parallel”
12. “State Machine”(noun)
1. A concept used by CompSci
profs for torturing
undergrads, full of arcane
math.
2. A practical way to build and
manage modern Serverless
Cloud apps.
Dictionary
13. “I want to sequence functions”
AWS Step Functions, we can easily change and iterate on
the application workflow of our food delivery service in
order to optimize operations and continually improve
delivery times. AWS Step Functions lets us dynamically
scale the steps in our food delivery algorithm so we can
manage spikes in customer orders and meet demand.
Mathias Nitzsche, CTO, foodpanda
“
14.
15.
16. var Twit = require('twit');
var T = new Twit(require('botfiles/config.js'));
exports.handler = function myBot(event, context, callback) {
var list = event.inputList;
var textToTweet = list.shift();
var output = { inputList: list }
T.post('statuses/update',
{ status: textToTweet }, function(err, reply) {
if (err) {
console.log('error:', err); context.fail();
} else {
console.log('tweet:', reply); callback(null, output);
}
});
};
17. How it works – API-level
Create State Machine – input is a
machine spec in a JSON DSL
Run Machine – input JSON blob,
returns Execution ID
List Executions
Describe Execution
Stop Execution
21. “I want to select functions based on data”
“With AWS Step Functions, it was easy to build a multi-step product
updating system to ensure our database and website always have the
latest price and availability information.
AWS Step Functions let us replace a manual updating process with an
automated series of steps, including built-in retry conditions and error
handling, so we can reliably scale before a big show, and keep pace
with rapidly changing fashions.”
Jared Browarnik, CTO, TheTake
“
24. “I want to retry functions”
We get transient errors from a RESTful
service we depend on, once every four
or five times we call it. But if we keep
retrying, it eventually works.
“
26. “I want to run functions in parallel”
We want to send the captured image to
three OCR providers and take the result
with the highest confidence value.“
31. Input processing
Q: InputPath not provided?
A: State gets raw input as-is.
Q: InputPath is null?
A: State gets an empty JSON object: {}
Q: InputPath produces plural output?
A: State gets it wrapped in a JSON array.
32. Result placement
{
"title": "Numbers to add",
"numbers": [ 3, 4 ]
}
{
"Type": "Task",
"InputPath": "$.numbers",
"ResultPath": "$.sum”,
…
Raw input:
State spec:
Output: {
"title": "Numbers to add",
"numbers": [ 3, 4 ],
”sum": 7
}
33. Result placement
Q: ResultPath not provided?
A: Input discarded, raw output used.
Q: ResultPath is null?
A: State input is state output.
Q: ResultPath produces plural output?
A: Not allowed, validator won’t accept.
34. “I want try/catch/finally”
AWS Step Functions makes it simple to coordinate information
from many different infrastructure systems using easy to design
workflows and create a more intelligent monitoring system for our
Platform as a Service (PaaS).
With AWS Step Functions, we can reliably automate monitoring
decisions and actions in order to reduce human intervention by
over 60%, which improves infrastructure operation productivity and
customer application availability on our platform.
Pedro Pimenta, VP R&D, OutSystems
“
35. 13 AWS Lambda Task States
6 Choice States
1 Fail State
“I want try/catch/finally”
41. “I want to sequence functions”
“I want to select functions based on data”
“I want to retry functions”
“I want try/catch/finally”
Is this you?
“I have code that runs for hours”
“I want to run functions in parallel”
Hello there, and thanks everyone for coming out to ServerlessPalooza!
I’m here to tell you about AWS Step Functions, a new AWS service announced in Werner’s keynote this morning. Preparing this speech was interesting – for my first draft, I made a very few points about who might use it and the bare minimum of “what it does”. That came in at 27 minutes. Now, the current version I’m going to give you covers essentially all of the features and APIs, and has customer stories, and will still run a little bit short. Thus, I’m going to claim that what I’m talking about this morning is a simple thing.
I normally run a little slower on stage than I do in rehearsals, so possibly I’ll use all the time and there won’t be any for questions. If you’re interested, I promise to go stand outside in the hallway and talk about Step Functions to anyone who wants to until they get bored or fall over, whichever comes first.
So, let’s dive in and look at the issues that Step Functions is here to address. Warning: This is a 300-level session, and There. Will. Be. Code. That OK?
In 2014, the launch of AWS Lambda expanded our minds, with its core idea of a stateless function in the cloud, reacting to an event.
But there aren’t that many apps with only one function, one entry point, one module, one component. So there’ll be more than one function. NEXT
In fact, I suspect that it’ll be common to have LOTS of functions, lots of them talking to each other. NEXT
And in fact applications, serverless or not, tend to have databases. NEXT
And in the cloud, I notice that a lot of them have queues of one kind or another – that gold thing is the iconfor Amazon Simple Queue Service, SQS, which I sometimes work on, and which I’m seeing a lot of in Cloud-Native apps. NEXT
And, now I’m going to do something that’s maybe in bad taste. This is the serverless track, but the world still does have servers. And this is a cloud conference, but some of those servers aren’t in the cloud. NEXT
Now we’re starting to see a picture of what an actual modern serverless app might really look like.
You’ll notice that those arrows are in different colors – that’s because there are lots of ways for functions and other serverless components to talk with each other and control each other and orchestrate each other. Before we dive into the “how”, let’s talk about WHAT we want to do with functions when we have more than one.
These are the things that people are trying to accomplish with those different-colored arrows, and I’m not gonna spend too much time diving in, because each one will get a little love of its own as we go through this story. But I suspect anyone who’s tried to build a nontrivial serverless app around functions in the cloud has dealt with a few of these problems. I know I have. So what are the techniques that people are using today to make these things happen?
As a senior engineer inside AWS I don’t treat the 12-factor app as religious scripture. You may not always have to follow all the rules, but you really have to do think about them. One of the nice things about the serverless approach is that it snaps beautifully onto the notion of statelessness.
I’d like to call your attention, in particular to that last paragraph – in a well-architected application, state should stored in a “stateful backing service”. That’s what this is.
OK, we’ve agreed that it’s a good thing to link together multiple functions to get our job done. People are doing this today, and getting good results. Let’s look at how:
You could just link your functions together, and that’s not a terrible idea – for example, the Zappa project implements the whole Django Web app framework – or actually any WSGI-based Python app - as a Lambda function, and that’s a very promising direction. But you know, I want more than that – I want a substantial number of modular, independent, functions that each do one thing and do it really well. That feels more like the serverless future to me.
If you invoke one Lambda from another, and you do it synchronously, well, they might as well be linked together. So if you’re going to do this, you should chain them with asynchronous calls, and stick with the Lambda paradigm of a function that does its job then bloody well exits. But then error handling gets hard. Even finding out about errors requires extra assembly. People are doing this and it’s not terrible, but it seems to be a lot more work than it should be.
Also, you could keep track of your state by writing it into a database – for example Amazon DynamoDB or another postrelational database – or you could even use SQL, particularly if you need transactions. People are doing this and they’re getting OK results, but once again, it feels like they’re doing too much work to get there. On top of which, there’s an impedance mismatch. The nice thing about serverless is that you expect immense elasticity, scale out a thousand-fold without any pre-arrangement. I don’t know about you, but I often find that databases sort of don’t; you ask them to scale suddenly by 10x or 100x and, well, sometimes it’s not a happy story.
Another approach is to pass your state and control around through queues, or streams, or whatever. I actually have a lot of time for this approach, it feels more Cloud-native. But once again, it seems to involve a lot of book-keeping. If the function that reads the queue gets a failure, you need to catch that and rewrite the state of the world back into the queue, and if you want time-out and retries, you’re going to have to arrange that for yourself.
So what I’m saying is that function co-ordination in the serverless world is possible, but it’s too ad-hoc and too difficult. So if we were going to automate all that, what are the must-haves. I suspect most fo these are pretty non-controversial or rather crushingly obvious, but I think it’s the last two that are really important. The things we’re trying to accomplish are easy to describe in words, so they should be easy for a developer to accomplish. And one of the things that *none* of the ad-hoc alternatives have is a good audit trail or log file.
So, anyhow, that list is exactly what this announcement and this talk are all about.
We launched Step Functions today, generally available in 5 regions, which offers.. Multi-Colored Boxes and Arrows As A Service! Just kidding; Step Functions is a fully managed service that tracks and co-ordinates and manages application state and does all those things we were just talking about, and is really easy to use, and has a centralized audit trail, and scales down to little one-off shell-script equivalents, and up to billions of complex multi-phase tasks. When I say “billions” that’s not a figure of speech – a predecessor product is in heavy use inside Amazon, both AWS and Retail, and is currently running about a billion state machines a week.
For the people in the crowd who know about the AWS “simple workflow” product, it turns out that Step Functions uses some of the SWF back-end technology. But it’s orders of magnitude easier to use, which I hope to prove to you.
So what is it at the end of the day? Let’s go back and quote from the 12-facto app text: Any data that needs to persist must be stored in a stateful backing service, typically a database. That’s this, a “stateful backing service”, specialized for running distributed applications in the cloud.
So those boxes and arrows on the screen are actually grabbed from the Step Functions console. So, in this talk so far there have been an awful lot of thingies on the screen connected by arrows,
Now when I see see a lot of little nodes with arrows connecting them, with transitions along those arrows, you know what it makes me think of? It makes me think of State Machines.
So, if you had a really bad time in 3rd year Computer Science with state machines, you might find this talk kind of tough, because they’re pretty well the whole rest of the talk.
Personally, I like state machines and have got good results using them over the years. and the notion of making it easy for anyone to use them to organize their serverless apps makes me happy. Having said that, I’m totally not gonna spend any time going into the theory of what they are and how they work, because this isn’t a computer science class. However, I am going to show you a few of them at work on building serverless apps.
So, since this is an Amazon Web Service, of course it has an API and an architecture and all that stuff, and I will show you that, but let’s lead with some use cases and examples, to make it real
FoodPanda is a take-out food service that is focusing on the developing world, and are really making excellent use of Cloud Infrastructure. They have been in the Step Functions beta, and their problem is deceptively simple. They get food orders and have delivery people and get the dinners taken to the hungry. Of course, optimizing this which requires solving the NP-complete traveling salesman problem, but they’re you know, smart. Anyhow, they want to do this with Lambda functions.
So here’s the actual state machine they built. Most of the states are executed by Lambda tasks. But I want to zero in on one small part of the problem, in the middle there where they run their assignment code and then dispatch a vehicle. Which is an obvious basic thing that many people want to do all the time; run this Lambda function, then run another. So let’s zero on in a small state machine that does just that. Also, if I only look at those two states, I’ll have something small enough to fit all of on one slide.
I don’t feel the slightest bit guilty about investing time in this moronically simple sequencing problem, because I think it’s actually a strong real-world use case. Often, the first step you take after you deploy a cool Lambda function, is you realize that wait, what I want to do is run THIS function then run THAT function. This sounds easy and it should be easy.
So there’s a visual rendition of the machine on the top half, and a JSON expression of it on the bottom. This is an actual screen grab from the Step Functions console. The picture is self-explanatory, so let’s look at the JSON. A state machine’s top level, NEXT along with a comment, has a structure called “States” that contains, well, all the States. And a string field StartAt that says which state to do first. In this case we start with “RunAssignmentAlgorithm”, NEXT which is a Task state that runs a Lambda, then has a Next field saying go to “Compute destinations”, which runs another Lambda, and has an End = true field, so that’s the whole machine.
It’s easy enough. Step Functions goes through the states, executes each, and moves along to the next. Obviously it can get fancier,. But let’s use this one to dive into how these things work. I didn’t include foodpanda’s actual Lambda function, I whipped them out and substituted demo-ware. And in fact it’s the same function twice. So let’s look at it.
Hope most of you can read NodeJS code. Basically, this function does a Tweet to prove it really ran. But I want to focus on the input and output.
So, ignore the first couple of lines for now and let’s start here. NEXT
The input is an arbitraryJSON blob, and we grab its inputList field, which in this machine is a list of strings. NEXT We assign the first string to the the variable “textToTweet”, and pop that off the front of the list. Then we tweet that text, and assuming everything went OK, NEXT we output whatever was left of the list after we popped off the first string.
It turns out this whole notion of input and output is important, so leave a bookmark in your mind, and let’s watch this thing work. Now, everybody knows that demos fail, so I made a screencast of me running the demo, which I could show you, but I noticed that the Internet in here wasn’t too bad, especially if you use the secret speakers’ WiFi. So let’s take a chance and try the live demo before I give up and run the screencast.
Now, what I just showed you was a state machine running in the console, which is a reasonable thing to do. But just as with any other AWS service, behind the console there’s an API and CLI, and at the end of the day they are the truth. I suspect the people in this room would like to hear about the API. This is not a complete drill-down with the exact method names and arguments, just a hit-the-highlights tour..
The first call is the most interesting – you create a state machine. Now let’s take a side trip to the console
and, just as with Lambda, you don’t have to start from scratch – there are a bunch of pre-cooked blueprints provided, and lots more on the way.
This is called the States language, and there’s a full specification for it on the Web here at this address.
Why is it in JSON? Could have been YAML, XML, Ion, a new syntax we invented for the purpose? I just don’t really want to have that argument; JSON is so widely implemented these days that it’s become the path of least resistance. For example, can show you examples and basically 100% of the people in the room will be able to read them. In this particular case, I found there was a reasonably good match between what the States Language wants to do and what JSON offers. Having said that, I hate hand-editing JSON, so I look forward to people inventing lots of smoother syntaxes that we can compile down to the States Language.
Sigh, I notice that a whole lot of you are now looking at your laptops and are at risk of spending the rest of this session looking at the spec, not listening to me. Be warned: I’m going to show off some more code that isn’t in the spec.
It turns out that when you upload a state machine, Step Functions runs a syntax checker and kicks it back at you if you’ve made a goof. It turns out that that can be a little annoying, especially when you’re trying to get started.
So, we also made a Ruby gem called “statelint”, promoted to rubygems.org this morning, so if you want to fool around with state machines on your desktop, it will give them a pretty thorough shakedown for you and let you know if you’ve got any problems. Just say “gem install statelint” (or maybe “sudo gem install statelint” depending on your setup). Sigh; now I now that the remaining few of you who aren’t reading the spec are installing the gem to fool around with it.
The construction of the validator is actually a little interesting, and does not rely on JSON schema – I wrote a blog on it, my blog’s easy to find if you know my name.
OK, let’s go back to some of the things we want to do with State Machines to build serverless applications.
Our customer here is TheTake, who are in the business of movie-based marketing – if you see Matt Damon looking cool in a movie, you can find out what he’s wearing.
Anyhow, they work with Amazon, which is nice, but they also go to a few other places, and if all else fails, they resort to screen-scraping. And, they want to do it all with Lambdas. So let’s see how they do this.
There’s a picture of their state machine. Some of those boxes are Lambda functions. It’s all pretty straightforward, and of course the interesting part is that box at the top labeled “ChoiceState”, where they decide which way they’re going to branch. Let’s dive in and look at the JSON form of that part of their machine, which doesn’t actually involve Lambdas.
I hope it’s pretty self-explanatory. <pause> So the Choice has four branches. Maybe the most interesting thing about it is that field labeled “Variable”. NEXT Remember how in that last machine, the input to the state was a list of strings? Well, it turns out that the data we pass from state to state is an arbitrary JSON text. So in this case, for their Choice, in the first branch, it looks at the input JSON’s Variable field, and the value has to be a JsonPath that selects one of the fields out of the input data. Then it takes that value and does the String Equals, and if it gets true, takes the “Next” branch. If that doesn’t work it goes to the next choice branch and tries that. It turns out that along with StringEquals there’s a full assortment of logical comparators: Integer, boolean, timestamp, and so on. And of course, there are And/Or/Not connectors so you can build hideous deeply-nested booleans that I would have no chance of fitting on a slide here at re:Invent
So I think it’s pretty obvious how the Choice state works. Once again, I want to emphasize this notion passing data along from one state to another in the machine. Generally, each state gets some input, which it can update or replace, and pass along to the next state.
Now, speaking for myself, my software always works first time and never ever fails. But in the cloud, we have Dependencies. I’ve noticed that when you’re trying to dodge the blame for something, dependencies are very helpful.
Seriously, in the real world, functions actually fail. And I’ve noticed that in distributed, asynchronous, cloud-native applications, from time to time, well, uh, “stuff” happens. I got this story from a customer who relies on an external RESTful service that sometimes just blows up. Now that’s a DEPENDENCY.
So in this case, let’s just jump straight to the JSON.
So, to illustrate, this is just a one-state machine. This Task state runs a Lambda called RestCallout, but then NEXT it’s got a “Retry”, that say “hey, if this blows up, keep trying, as many as ten times. I’m not going to dive into the deep details – Notice that the Retry value is actually an array, so you can have a bunch of error handlers for different values that do different things.
Let’s see this thing run.
This customer has an app where one of their customers takes a picture of a reward code with their phone and enters the contest by sending it in. What they wants to do is OCR it, but OCR is a bit of a chancy business, so what they do is send the picture off to multiple OCR services and take the result that’s got the highest confidence value. And obviously, the customer’s waiting, they don’t want to do do these in sequence.
So, here’s a picture of their state machine, mostlybuilt out of Lambdas. There are three branches, each with a couple of states, and then you get the three OCR results back and the Pick Result state decides which OCR result to go with. Easy enough to understand.
Once again, most of these states are Lambdas, but not all. It turns out that the ”Go” states, Go1, Go2, and Go3, all actually use the same Lambda function, which uses a standardized callout interface for the OCR conversion. So the Prep states, Prep1, Prep2, and Prep3, are there to set them up to point at the right OCR service. Let’s dive into the JSON and have a closer look.
Now, I can’t fit the whole machine on this slide, so let’s zero in on on where it does that parallel branching. So there’s a Parallel state, which has a bunch of Branches, and each one is a mini-state machine, with a bunch of states and a StartAt Pointer. I’ve managed to squeeze Branch 1 onto this slide, so let’s go look into that. This branch has two states. The first one is called Prep1, and it’s not a Task state, it’s a “Pass” state, which basically doesn’t do anything. But, it has this “Result” field, and that becomes the output of the Pass state. So in this case, it passes a JSON structure containing the string to the Go1 state, which actually invokes the Lambda that calls out to the OCR function.
People who look closely can spot that the Lambda function I’m actually using is the one that sends tweets, so if I wanted to run a demo of this, you’d get three tweets posted at almost exactly the same time, showing that the OCR Provider 1, Provider 2, and Provider 3, branches got called.
We think Parallel is a really big deal for state machines, and that customers will probably test the limits of our system by sending Parallel states that have thousands of branches, nested within other parallel states that have thousands of branches.
So I’m assuming you’ve noticed a pattern here. Step Functions is all about managing application state in the cloud. Now, a large part of the application state lives in your app’s databases and queues and so on. But Step Functions’ big value-add is that it takes care of all the transient state, and encodes it in that JSON that’s being logically passed from state to state. Now if you look under the covers, it turns out that what we’re doing is stashing that in a database, just exactly the way that people often co-ordinate functions these days by stashing state in databases. Except for, we’re doing it for you and letting you make declarative state machines.
So the basic idea is that a state machine execution has an inpu in the form of JSON. Since I am the editor of the IETF’s latest JSON RFC you will allow me to be a bit pedantic. To be exactly correct, the input to a state machine, and also the input to and output from any state, has to be a “JSON Text”. If you are a real JSON weenie, you’ll already know that this means it doesn’t have to be an object or an array – ordinary text containing a number or a quotation-mark delimited string or whatever is just fine.
Anyhow, the state machine has an input text, and which becomes the input to the first state, and then by default that state’s output becomes the input to the next state, and the last state’s output from the machine. Only sometimes you don’t want the default, you want to interfere with the JSON coming into or going out of a state. In fact, I wouldn’t be surprised, in complicated state machines, if there were the occasional state containing a Lambda that just rearranges the JSON to prepare it for later processing. But let’s look at some of the built-in stuff we’re providing.
OK, so let’s imagine a state that adds numbers. And in this particular machine, the data being passed along is that JSON object in the top box. But suppose that state doesn’t know about JSON objects, it just wants ot get an array of numbers to add up. Which is probably good design. NEXT So, we can use the InputPath field, which is a JsonPath, to select the piece of the input that we want to feed to the state. In this case, give it just the array of numbers that it needs.
Some frequently-asked questions about this stuff. Sometimes you don’t want to fool with the input. In which case, you just don’t use InputPath and nothing happens. Sometimes a state just doesn’t care about its input, it’s all set up with its task in a database or a queue or whatever, so say InputPath: null, and there just won’t be any useful input. Now, if you’re a JsonPath expert, it turns out you can write one that takes a slice of an array, or all the fields in an object – so it returns more than one JSON text. Since our rules say that the input always has to be a JSON text, we just wrap it in square brackets and turn it into an array.
This is just bookkeeping but it turns out to hit a pretty good 80/20 point.
Now, what about the stuff coming OUT of a state?
OK, let’s revisit that lambda function that adds numbers. Let’s suppose that it gets an array of numbers as input, and as output it just gives you a sequence of numbers representing the result – in this case, the single digit 7. Remember, this is a valid JSON text. So, if we don’t do anything the output of the state would just be that single digit. But in a lot of cases, we don’t want to discard the input, we want to enrich it. So NEXT we provide a ResultPath field here, that says “keep the input, but put the result of the task here in the input”. So you can see what happens. This turns out to be super-useful – I saw one app where they had JSON that represented a customer and they had a state to normalize the phone number, and another to look up the email address and so on, so the JSON got richer and richer as it worked its way through the machine.
Now, some FAQ’s for Result processing
By default the output of the stae is just whatever its task spits out. So if you have no resultPath, that’s what you’ll get. Sometimes you don’t actually care about the output from the state, it was only there for its side-effects. Just say ResultPath: null, and that’ll pass the input through to the output.
Here’s one tricky case – the JsonPath that says where in the input to put the task output? It can’t have array slices or wild-cards, it has to point at one particular place in the input to put the output.
So that’s enough input/output bookkeeping help. Oh I should mention that there is also an OutputPath field, which can apply a JsonPath to the JSON on the way out of the state, but it’s most just there for completeness, doesn’t seem to be that widely used.
Let’s go back to some customer situations and how they used stae machines to address them.
OutSystems automates their infrastructure using AWS Step Functions to integrate alarm information from many sources.
I mentioned already that we have a retry capability, but sometimes, things just break. In most programming languages, we have a try/catch/finally idiom, and I’m pretty convinced that serverless apps are going to need that as well. So let’s dive in and see how to do that.
So, what we have here is a serious state machine that does some heavy lifting. But it does a lot of try/catch/finally. To spare your eyes, and to make it easy for me to fit things into a re:Invent slide, let’s look at a much smaller macine that shows this off.
This is a screen grab of a running state machine, an app that accesses a media file over the network and in an ideal world, just makes a graceful exit. But if something goes wrong, it drops into Clean Up mode, and then exits. But sometimes even the Cleanup breaks, and at least we can let the humans know when that happens. So once again, let’s look at the JSON.
So, here’s your Try/Catch/Finally. It’s a task state that runs a lambda, but it’s got a new thing, a TimeoutSeconds field, that limits how long the state is allowed to run. After two seconds it’ll throw a States.Timeout error.
You’ve already seen a Retry state. In this one we catch that States.Timeout error, and will retry twice, once after 2 seconds, and once after 3 seconds. You noticed that in that earlier Retry example, I didn’t have all the backoff parameters. You can leave them out and Step Functions will use reasonable defaults.
Anyhow, if we retry twice, and the FetchMedia lambda still didn’t get the job done, eventually the error falls through and hits the Catch clause, which redirects to another state called Clean UP that I’m not showing here; it runs a cleanup function and if *THAT* fails, then it redirects to last gap ”finally” sort of state.
OK, let’s look at the last thing we hear about from our customers.
I mentioned already that we have a retry capability, but sometimes, things just break. In most programming languages, we have a try/catch/finally idiom, and I’m pretty convinced that serverless apps are going to need that as well. So let’s dive in and see how to do that.
It turns out that while Lambdas are great for lots of things, there are a few things where they aren’t so hot. Like, for example, monitoring machinery for hours at a time. For that, there’s a thing called an “Activity”. The way it works, you register tasks that an Activity can do, and Step Functions gives you an ARN for them. Then, you can use that ARN in your state machine’s Task States. Instead of Step Functions invoking a Lambda, Step Functions will wait for your activity worker to poll for tasks, hand the task off to it, and wait until your activity either times out or calls back.
Basically an Activity can be anything that can request work, receive it, complete it, and send it back.
The poll is a standard HTTPS long-poll, and you can poll from anywhere; an AWS instance or out there on your own machines on your own premises. Your code can be written in anything the AWS SDK supports. If you have a lot of state machines running and they have lots of activity tasks to hand out, you have to decide how many activity workers to poll for them, which determines how much you pay and how fast they get done.
OK, I acknowledge that this is not exactly serverless. But I think we in the Serverless future need to co-exist at least for a while with the server-full past. So here’s a glance at what an activity might look like in a state machine.
So here’s an example of a Task state, just like all those others we’ve been looking at that had Lambda functions. But if you look closely at that ARN, you’ll notice that it’s different, it refers to an activity. Also, you can see that aside from what’s inside the state definition, there’s nothing but the ARN details that says what kind of a Task this is. So in the future you can imagine other kinds of tasks, aside from lambda functions and poll-for-work activities, that Step Functions might be able to manage.
Anyhow, the other interesting thing here are the time values. In this case, if the activity runs for more than a little over eight hours, we decide it’s broken and time it out. We did that before in the try/catch/finally example. But there’s also HeartBeatSeconds. The idea is that the activity task needs to convince us it’s still alive and working, so we require it to hearbeat, using that API I mentioned on the last slide, regularly, and if it doesn’t for two minutes, we time the task out. This stuff turns out to be super-useful in practice.
Raises the question “How long can a State Machine run?” In theory, forever. In practice, one year.
From what customers tell us, I bet that there are actually a lot of you.
Now, this hasn’t been a complete tour of Step Functions, but I think I’ve hit most of the high points.
I’ll be honest the Product Managers have visions of people running huge machines that grind away for days and choreograph hundreds of Lambda invocations. What would actually make me happy is to see Step Functions replace lots and lots of ad-hoc shell scripts that people are using for configuration and control and cleanup and other miscellaneous tasks.
Anyhow, that’s more or less all, folks. It’s here today in five regions, feel free to dive in and use it. We think it’s easy.
Oh, I should mention, it’s not free. But we tried hard to make the pricing easy to understand and surprise-free.
I want to highlight that readout there at the bottom of the screen. As our applications get more and more heterogeneous and have more and more moving parts, something like Step Functions becomes more and more useful. One reason is that it also provides a global log of everything that went on; when each step started, what its input was, whether it succeeded or not, what its output was – and with everything timestamped. Thoroughly greppable, too.
See? State machines are your friends!
So, we’ve talked about running machines in sequence, and branching based on data, and now dealing with errors. What else? Well, Lambdas are stateless functions in the cloud, and the moment I hear the word “stateless” I think “I want to run them in parallel!”