Creating and maintaining different environments can be difficult and a real time-sink. We'll see in this talk how you can automate this tasks by building and updating your development and production environments on demand using Chef, Vagrant, Docker and Amazon Web Services.

1. Let’s Code our Infrastructure

2. Who I am
● Oswald De Riemaecker
● Continuous S.A. CEO & Co-Founder
o Agile Delivery specialist
o Editor of the continuousphp PaaS
● « Agilist », Cloud Architect

3. Agile Culture
● Improvement in the development team
o Better collaboration
o Continuous improvement
o All skills / No silos
o Testing Framework and continuous integration
● Improvement in Agile product management
o Iterative development
o Features driven
o Maximizing Value
o MVP/Lean Startup

4. DevOps Culture
● Agile + Operation
● 3 main pillars:
o Infrastructure as Code (IaC)
o Continuous Delivery / Deployment
o Culture of cooperation

5. Infrastructure as Code
● Coding machines resources of the infrastructure
● Coding and versioning the infrastructure
● Testing both resources and infrastructure
● On demand rebuilding machines and infrastructure

6. Organization
My Infrastructure Your Infrastructure Their Infrastructure

7. Local
Environments
Development Testing Staging Production

8. Roles
Load Balancer
Application Servers
Indexes Servers
Databases Servers

9. Resources
● A resource is a part of the system and its configuration
Apache MySQL Solr

10. Complexity
● Maintaining infrastructure is complex
● It lives and evolves, so you have to
o Add new resources
o Update resources
o Apply Security Patches
o Change/Update OS
o ...

11. Tools
● The answer to this:

12. Chef
● Chef is an open-source tool
● Management of automation systems and infrastructure
● Chef is based on Ruby

13. Chef
● Nodes represent the servers within your infrastructure
● Nodes can be:
o physical servers
o virtual servers
o instances in the cloud
● Each node belongs to:
o an organization
o an environment
o has one or more role(s)
Node
NodeNodeNode
Node Node

14. Chef Client / Server
Node
NodeNodeNode
Node Node
Chef
Server
chef-client
Resources

15. Chef Cookbooks
● The cookbooks contain artifacts of our resources
Recipes
Templates
Files
Attributes State of the resources

16. Chef Cookbooks
● The cookbooks contain artifacts of our resources
Recipes
Templates
Files
Attributes State of the resources
Installation / configuration of resources

17. Chef Cookbooks
● The cookbooks contain artifacts of our resources
Recipes
Templates
Files
Attributes State of the resources
Installation / configuration of resources
Configuration templates

18. Chef Cookbooks
● The cookbooks contain artifacts of our resources
Recipes
Templates
Files
Attributes State of the resources
Installation / configuration of resources
Configuration templates
Configuration/script files

19. Provisioning

20. Role Provisioning
Node
Role:
Database
Chef
Server
Hey, I am role “Database”
and my environment is “Development”
Chef clientRecipes
Templates
Attributes
Files
Cookbook Recipes
Templates
Attributes
Files
Cookbook

21. Infrastructure Example
● Web Server - Apache 2.4/PHP-5.5
● Database Server - MySQL
● Index Server - Solr Web
Server
Solr
MySQL

22. The Chef Community

23. Apache Cookbook Analysis
● Support many platforms
● Support different Apache versions
● Attributes:
o dir, docroot_dir
o log_dir, error_log, access_log
o default_modules
o version
o ...

24. Cookbook support and dependencies
Recipes
Templates
Attributs
Files
Metadata
Dependancies

25. The attributes
Recipes
Templates
Attributes
Files
Metadata

26. The recipes
Recipes
Templates
Attributes
Files
Metadata
cd recipes
ls -l
default.rb
iptables.rb
logrotate.rb
mpm_prefork.rb
mpm_worker.rb
mod_php5.rb
mod_*.rb

27. The Templates
Recipes
Templates
Attributes
Files
Metadata

28. The Files
Recipes
Templates
Attributes
Files
Metadata
cd files/default/
ls -l
apache2_module_conf_generate.pl

29. Let’s code our cookbooks

30. Role: Web Server / Env: Production
Apache / php-5.5
Recipes
Templates
Attributes
Files
Metadata
Apache2
Recipes
Templates
Attributes
Files
Metadata
iptables
Recipes
Templates
Attributes
Files
Metadata
logrotate
PRODUCTION: Replacement Attributes
Node Role: WebServer
R
O
L
E

31. Chef Development Kit
● Installation of chef-dk
o Berkself
o Knife
o ChefSpec
o Test-Kitchen
o Serverspec
o Foodcritic

32. ● knife cookbook create mon_org-webserver
Creating our web server cookbook

33. Adding our dependencies
Dépendance(s)

34. ● berks init
Initialization with Berkshelf

35. ● berks install
Downloading dependencies

36. ● vi attributes/defaults.rb
The Attributes

37. The Recipes

38. Testing our cookbook
● kitchen list

39. Testing our cookbook
● kitchen verify default-ubuntu-1404

40. ● knife cookbook upload mon_org-webserver
Versioning and uploading our cookbook

41. Repeating the exercise for database
and indexes roles

42. Provisioning our Sandbox
Development

43. Role: Sandbox / Environment: Dev
mon_org-webserver
mon_org-index
mon_org-database
Composer NodeJs
Recipes
Templates
Attributs
Files
Metadata
Recipes
Templates
Attributs
Files
Metadata
Recipes
Templates
Attributs
Files
Metadata
Recipes
Templates
Attributs
Files
Metadata
Recipes
Templates
Attributs
Files
Metadata
mon_org-webserver mon_org-Index mon_org-database composer nodejs
DEVELOPMENT: Replacement of Attributs
Node Role: Sandbox
R
O
L
E

44. Creating our sandbox cookbook
● knife cookbook create mon_org-sanbox

45. Adding dependencies
Dependancies

46. Downloading dependencies

47. The Recipes
● vi recipes/default.rb

48. ● vi attributes/default.rb
The attributes
Attributs

49. Testing our cookbook

50. ● Cookbooks by role of our infrastructure
● Specific Cookbook for our development environment
o uses the cookbooks of our infrastructure roles
● We can reliably reproduce our infrastructure roles, it is:
o Coded
o Versioned
o Tested
o Deployed
Provisioning Summary

51. Let’s create our environments

52. Local
Creating our development environment
Development

53. Vagrant
● Create and set up development environments
● Providers (Drivers)
o Virtualbox/VMWare
o Docker
o Instance EC2
● Provisioning with:
o Chef Client/Solo
o Docker
o Puppet Agent
o …

54. Vagrantfile

55. Vagrant up

56. Immutable Infrastructure

57. Immutable Infrastructure
● Benefits:
o Saving time when supplying
o No more maintenance of the servers => Replacement
o Static machine Image pre-built and tested
o Easy rollback to a previous version
V1
V2
V3
Data

58. Packer
● Packer
o Builders (Amazon AMI, Microsoft Azure, DigitalOcean, Docker,
Google Compute Engine, …)
o Provisioners (Shell script, Chef client, Ansible, Puppet, Salt, …)
o Post-processor (Docker-push, Vagrant, Atlas, …)

59. Packer Config
Builders

60. Packer Config
Provisioners

61. ● packer build packer.json
Creating our image

62. Creating our image

63. Repeating the exercise for
the index role

64. Creating our Production/Staging environment

65. AWS CloudFormation
● Easy way to create and manage an AWS resource
● CloudFormation takes care of dependencies
● Once the resources are deployed, you can edit and update in a predictable and
controlled manner
● AWS CloudFormation supports the following resources:
o Auto Scaling/Elastic Load Balancing
o VPC
o EC2
o RDS
o IAM
o …

66. VPC
Private Subnet
Public Subnet
JSON
Autoscale
Web Web Web
Autoscale
Index Index Index

67. Cloud Formation Structure

68. Cloud Formation Structure

69. Cloud Formation Structure

70. Cloud Formation Structure

71. Cloud Formation Structure

72. Cloud Formation Structure

73. Creating our web server stack

74. Creating our web server stack

75. Creating our web server stack

76. Creating our web server stack

77. ● Machines Provisioning
● Immutable Infrastructure
o Static Machine Images
● Using Machine Images in AWS Cloud Formation
● Cloud Formation environment parameters
● We can now use our artifacts to build our different environments
Infrastructure Summary

78. Orchestrating our infrastructure
Testing
Development
Vagrant
Feature branch
develop branch
revision branch
Staging
Production
CI/CD
prepare test package
prepare test package
prepare test package

79. Questions?

80. Thank you!
oswald_odr
oswald@continuousphp.com