Chatops - Modern workflow

No notes for slide

Hi, good evening.

Welcome at These Days, I hope you all enjoy this evening.

Today we are going to talk about Chatops and how we implemented this in our workflow at These Days.

Are there people who don’t speak or understand Dutch?
Yess: ok, then I will give the talk in english
No: ok dan ga ik verder gaan in het nederlands.

But first let me introduce myself.

I am Joren Van Hocht, 23 years old. Building website’s started at a very young age and with the years coding became my passion. Last year I graduated at Multimedia Technolgy Antwerp. After my internship at These Days I had some meetings and conversations with the awesome people of These Days and in August I started on my first full time job here.

We needed a way to get our dev projects on our dev and staging servers.
We needed a way to improve and speed up our workflow so that we don’t lose lots of time to get our projects available for other colleagues or clients.

What tools did we used to achieve this?

Well off course we used Slack, more specific we used their incoming web hooks and slash commands to build all this awesome stuff.

At the moment we are running these applications on Laravel 5.1 but we will soon upgrade to Laravel 5.2. It was also the first time we worked with queues and I have to say that worked out pretty good.

The deploy and the composer command are both pushed on the queue, for the simple reason that Slack expects a response with in 3 seconds.

Our preferred VCS.

You may be thinking what did they do before? Well we had a in some way similar tool. This tool was a web application and was build a long time ago at the beginning of the SVN days within These Days.

Yii

But since last summer we made the switch to git, but now what the other application was out of date, merging it to git would be a hell of a job.

So we had to start all over again. But how did want to do this. At first we where thinking of a new web application.

(people still working with SVN instead of GIT)?

But little while before we made the switch from SVN to GIT we also started using Slack in our production team. And so it goes, we started brainstorming and Slack came to the rescue with their “Slash commands”.

So we started hacking our first version together. And in the mean time we are at version 2 witch has clean code and lots of more features then originally intended.

Enough talking, let me give you a small demo of how our deploy tool works.

I created a small demo project with a clean Laravel installation. The project is created and now we want to make it available on our dev and staging servers.

So first steps first we have to deploy this project.

Let’s start with our dev server.

We start typing /deploy and slack will autocomplete this, because we told Slack to this for us. Next we have to provide some parameters.

The first parameter is our environment, we have to options in here dev/staging.
The second parameter is the branch we would like to deploy. In this case we want to deploy the develop branch.

Next we know it is a Laravel project so we have to do a little bit more:

Setting the env file, running composer install, migrating and seeding the database and making the right folder writable.

We have a custom command that takes care of it all. It has an obvious name /laravel.
Here we also have to define our env first. And then we can specify their version of Laravel we want to use. 4/5. We did this beceause Laravel 4 and 5 have some different steps that we have to take care of. Laravel 4 has no env file that has to been set for example.

So this was a short a demo of what our tool can do.

The advantage of Slack is that it has a really powerful api.
We made use of their custom ”slash commands” to send data to our api and used incoming web hooks to send data back to the slack channel.

We made use of the Laravel framework to hack our first version together. And by the time that we saw that this was a good solution that we wanted to use for a longer time and that we wanted to extend. We got started on version 2.

We now knew all the pitfalls and how everything should be working so now we could start building a more scalable tool for the future.

Their where three main reason why the code had to scale more:

We want to be able to create a new feature quickly
What if we had to leave slack for some reason? We will only have to rewrite our controllers.
What if we had to leave Laravel for some reason? We will only have to rewrite our controllers.

After all that talking about features you are probably wondering what features is he talking about. Let me start with the first one.

The info command.

This command displays all the available data for a project like:

The url to the git repo
The url to the dev and staging server
Ands ome additional information about what is currenlty deployed on those servers.
Database credentials for the dev and staging env

The main command where it all started with.

This command is used to deploy our projects to the dev or staging server, maybe in the short future we will also use this to deploy to our live server.

Well most of our projects use composer, the composer cammand allows us to run composer install/update.

We use Laravel for almost every project, so yes we also have to migrate our databases.

And don’t forget we also need to seed them.

Dump, is used to run composer dump

Want to fetch all new branches? No problem we have a command for that to.

Note: if you deploy a branch that is not known to the server yet, we run git fetch once in the background and call the deploy function again. If our minion still sees a path spec did not match error we abort the process and throw the error to slack.

Obviously some of our folders have to be writable so that chmod makes sure you can do this an a limited set of folder names.

Env files are introduced for security reasons and are ignored by git. But hey we still need them. So for every project we make a .env.dev and .env.staging file.

When we run the setenvfile command we take the right file based on the the server we want to set the file to and copy it’s contents to the .env file.

When it all comes together, we have just one command to run everything we need for a Laravel installation on our servers.

Just deploy your project, run /laravel tell the command witch server you want to run it on, specify the Laravel version and everything will be runned.

composer install
Set env file
Php artisan migrate
Php artisan db:seed
Chmod –R 777 storage

And we are good to go.

Nothing just works, this was all new for us so we had a lot of challenges. I will cover a few of them in the next slides.

We needed a flow to startup a new project, this tool or whatever had to initialize a project. This is a separate web application that is used by our IT team. It is simple Laravel app where they can set all the project settings.

Once they submit the tool will create the required folders on the server, create a vhost file, creates a new git repo (or uses an existing one) creates a slack channel and a database if necessary.

After that we are good to go and we can start building awesome stuff.

It took some time to find out how to do all this stuff with Slack. We had to find out how we where going to pass our parameters correctly and still keep the code scalable. Who says we are going to stick with Slack forever?

Pushing all information to Slack would be nuts and would make our channels durty. So instead of pushing everything to Slack we only push our errors and a basic success message to Slack. All the rest is written to a log file for each project.

Nothing just works, this was all new for us so we had a lot of challenges. I will cover a few of them in the next slides.

If there are any questions please fire them off!

Do you have ideas? Did we inspire you our do you have awesome suggestions for features we didn’t think about, please fire them off or discuss them later on the evening with a drink.

Thanks for you attention!