Welcome to my talk about how to Automate & Autoscale AWS elastic container service
My name is Enri Peters a Mission Critical Engineer working at Schuberg Philis.
I am 30 years old and I am living in Zutphen together with my girlfriend, 2 daughters and a cute french bulldog.
I have a passion for automating IT and cloud environments and I also have an addiction for learning new things.
At the moment I am helping a customer managing their enterprise messaging platform, which is fully running on AWS elastic container service.
Now.. imagine having to manage over thousands of applications running on AWS ECS.
And also having to Secure, Monitor and autoscale this huge platform.
Each application stack that gets deployed, consists around 500 lines of CloudFormation code.
The configuration and needs per application are very different.
So copying and pasting an already existing stack was not that easy, and also not very efficient.
Let me give you a simple overview of what this messaging platform looks like.
Producers are able to sent their data via an API URI.
Via Route53 these requests are routed to a network loadbalancer.
Which forwards the requests to a HAProxy server that handles the client certificate authentication.
When allowed by the access control list, the request is again forwarded to an application loadbalancer which has listener rules setup so it can forward the traffic to the correct target group, which contain the registered ECS Services.
Via this way, each application that needs to be externally accessible, can be securely accessed trough the API of the messaging platform.
When the message has been processed by the platform it then get’s delivered to one or more consumers.
To succesfully run this platfrom we are using GitLab for holding our source code and to deploy stacks that support this messaging platform, such as lambda’s.
We are also using HAProxy, which does the client certificate authentication.
For CI/CD we are using Concourse to deploy over thousand of ECS services/tasks.
And the rest is powered by using these AWS services.
We are using CloudFormation to deploy everything to AWS.
And to give another example, the systems manager parameter store is helping us to manage our configuration data, whether in plain-text or secret data.
S3 is sending s3 events to SQS when messages are uploaded. Applications running on ECS are then receiving the messages from SQS.
And all the other services listed are also helping us with managing this messaging platform. From persisting messages to service discovery.
Well.. with all these amazing technologies in place, we now can truelly start automating.
A great way to start automating is by making use of the AWS Software Development Kit.
The AWS SDK helps to take the complexity out of coding by providing classes in the programming language of your choice for many AWS services.
This way you can more easily develop applications on AWS.
For example by using Python or Ruby and even more programming languages.
We have used the Ruby SDK as most of our team members were already quite familiar with writing code in Ruby.
With this Ruby program we now compile our own cloudformation templates. The only thing developers need to do is providing it with what we call a Deployment file. More about this deployment file later.
Combining the AWS SDK with CloudFormation Custom Resources is a very powerful option, and will offer you endless possibilities.
Custom resources enable you to write custom provisioning logic that CloudFormation runs anytime you create, update, or delete stacks. For example, you might want to include some resources that are not available as CloudFormation resource types. By creating your own custom resources you can still manage all your related resources in a single stack.
Custom resources require one property: A service token, which specifies where CloudFormation sends the requests to.
When the shown template gets deployed. The Lambda specified in the servicetoken, will receive an event including all the shown properties. The lambda will then execute the code to put this metadata inside a DynamoDB table. We are using this data to update our status page during the deployment. This way our status page always contains real-time data, which is really valuable, as the business can do reporting, cost calculation and many more things with this data.
Now please let me explain how we are making use of all these amazing technologies in our deployment strategy.
We are using GitLab for storing our application source code and deployment files.
For continues integration and delivery we are using Concourse CI. In Concourse, jobs can depend on other jobs by configuring passed constraints. The result is a pipeline that continuously pushes our projects forward, from source code to production. Dynamically setting up a pipeline as shown, needs quite some configuration. This is why we created a pipeline manager for this. Now developers can create these pipelines for their application in no time, just by pushing some application specific YAML config files to the pipeline manager repository.
At the top you can see that we are hosting the Deployment file inside gitlab repositories.
This pipeline manager can also generate single pipelines which are getting input from multiple deployment files. Because sometimes we have to deploy multiple applications which are working together.
Now you may wonder what is written inside a Deployment file.
Well to be able to deploy an application to ECS you first need an ECS task definition, which is like a blueprint for the application. A task definition is required to run containers in Amazon ECS. With this task definition we can create the Amazon ECS service which launches and maintains a specified number of copies of the task definition in a cluster. On the service we can configure auto scaling and optionally we can enable load balancing to allow the application to be accessed outside the ECS cluster.
To provide all information to compile this into a single CloudFormation template for a specific environment, a huge amount of configuration settings per environment are necessary, like the ECS cluster name, AWS default region, load balancer name, security groups, subnets, discovery namespace and so on.
As these settings are common per environment for all the applications to be deployed, we are providing concourse jobs with sets of global variables per environment, so they do not have to be specified in the deployment file.
The rest of the configuration will be taken out of the deployment file, from which you can see an example on the screen. Now the only thing a developer has to do is maintaining these deployments files. Variables with APP_ in front of it will also be available as environment variables inside the container. In this example we are making use of the Cloudwatch metric math function to make a SUM of multiple custom metrics. More about this later.
The Ruby program takes all variables and will compile a large cloudformation template, containing the custom resources, ECS task definition, ECS service an much more.
6Let me now show you how it all comes together.
The first job in Concourse get’s triggered by a webhook from Gitlab whenever commits have been pushed to the repository. We are using webhooks, to lower the amount of checks Concourse does by default. This results in running way less containers on the Concourse workers.
The next few jobs will run a dockerlint and build and push the image to the AWS elastic container registry. Which in turn will automatically run the image scanner service to help us identifying software vulnerabilities in our container images.
After these jobs a new job gets triggered in which the deployment file gets checked out. This file is then automatically validated.
Next up the almighty Ruby program reads the deployment file, which consist of the variables for that application to be deployed. The ruby program will then run tests, like checking if provided systems manager parameters are existing and then it will compile the Cloudformation file. This job is using a prebuild container image which contains all the code that is needed for the deployment. In this stage our cloudformation custom resources are also being triggered by the deployment, and thus our status page gets automatically updated.
If this job succeeds, the application is successfully deployed to the test environment and the next job will test the application functionally by running integration tests specific for this application.
When the deployment succeeded, the pipeline will trigger it’s next job which is creating a merge request in a special gitlab project, that we name the acceptance gate. This is where the first manually intervention comes to approve the merge request and continue.
After the merge request has been approved, Concourse get’s again triggered by a webhook and the same application get’s deployed to the acceptance environment. How cool is that?
After this, well I think you can guess… the same will happen for production.
So far we have deployed everything automically to AWS. At this moment the custom metric lambda is already fetching the metrics for this application.
The application will use these metrics to autoscale. Which brings me to the latest subject of this talk.
I think for most of the companies, autoscaling is no longer a nice to have.
For this customer we already had CPU and Memory auto scaling in place for quite a while.
But CPU and Memory metrics were not always giving us the behaviour we expected. And next to this the covid 19 pandamic, caused this company services to be used way more by its customers. Peaks were getting higher and higher.
We needed something better and we also wanted to be able to scale to zero. Because that is what cool people do.
Scaling to zero would allow us to only run containers when there is demand.
To achieve this, we needed custom metrics, these are metrics that you can generate yourself.
We are running a lambda which fetches the data we need for each application and then creates these metrics for us.
We also added a feature to our Ruby program so it is able to generate the correct autoscaling cloudformation resources. Based on the variables we saw in the deployment file, including the metric math function.
Now let me show you parts of the compiled cloudformation code, so you will have an idea of how to also start implementing this yourself.
It starts with deploying an IAM role which is able to assume the application-autoscaling aws service.
Next up a scalable target is needed. The ScalableTarget specifies a resource that the Application Auto Scaling service can scale, such as an AWS::ECS::Service resource. You can specify this resource at the ScalableDimension property.
Then we need 2 scaling policies, one for scaling up and one for scaling down. For the ScalingTargetId we refer to the previously created ScalableTarget by using the !Ref function. The adjustment type is set to Change in capacity as we want to upper or lower the number of running tasks.
Lastly we need 2 CloudWatch alarms that can trigger the scaling policies we just created. 1 alarm which will trigger upscaling. And another alarm which will trigger downscaling. In these alarms you can see the “e1” metric which is a sum of metrics m1 and m2. This is the metric math function, With metric math you can use Basic arithmetic functions and Comparison and logical operators, which is really powerfull. I have provided the url to the documentation on the screen.
With this amazing setup the customer is now also able to scale to zero, which is resulting in saving a lot money.
I think There is almost no better way then ending a talk with mentioning an achieved costs saving.
I want to thank you all for your time and I hope you enjoyed it.
If you are interested in more details of this setup, please feel free to contact me at my email adres shown in the presentation.
I wish you all a great AWS Community Day!
See ya!
