Workshop presented at Summer camp 2016

1. Visuel à insérer ici
WEB SUMMER CAMP 2016
Rovinj
eZ Publish 5: from zero to automated deployment
(and no regressions!) in one afternoon

2. WHO AM I
The short version
• « the guy who has written a lot of answers
on the eZPublish forums »
• Working at Kaliop in London since 2014
• Principal consultant at eZSystems for 7 years
• Love coffee and bicycles ?
• @gggeek
• Little known fact: today it’s my birthday!
VOTRE SCHEMA

3. WHAT THIS WORKSHOP IS ABOUT
The short version, too

4. TALK IS CHEAP
Show me the code
https://github.com/kaliop-uk/websummercamp2016

5. HOUSE RULES
Marek and Jakub are the Lethal Enforcers Minions Gentle Assistants
• Interrupting is welcome at any time
• fall behind: the assistants are authorized
to stop me for all the time needed for you
to catch up with the rest
• fall asleep: the assistants are authorized to
kill you empty your pockets pinch your ear

6. WHO ARE YOU?
The short version, please
• Any experience with eZPublish ?
• Any experience with eZPublish Legacy?
• Did anybody attend the Docker workshop
from Marek on Wednesday ?
• Any Docker experience at all ?
• Who has been up all night partying?
VOTRE SCHEMA

7. SUMMARY
1. INTRO TO DOCKER
2. (MOAR) DOCKER!
3. MANAGING ENVIRONMENTS
4. MANAGING DB CHANGES
5. A DEPLOYMENT SCRIPT
6. TESTING THE SITE

8. If you have never heard of used
Docker
A. I am living under a rock
B. I am living on the moon
01

9. What is it, in one phrase
Definition according to Me™
"just like a VM, but lighter"

10. Virtual Machines vs. Containers
VM on the left

11. Getting started is easy
Getting off the hook on the other hand…
"just like a VM, but lighter"
all it takes to get started:
docker run -ti ubuntu:15.10 bash

12. Getting started is easy
what has just happened:
• the Docker daemon connected to a public repository to download an
image (https://hub.docker.com/)
• a container was started using that image
• the bash program was started in the container, and its I/O attached to
the shell in execution

13. Getting started is easy
• ideal for testing a command / script in a different environment
• makes it very easy to deploy (some types of) applications as self-
contained packages
• not created for multi-process applications, nor for Continuous
Integration scenarios
• but it is getting there quickly

14. Terminology
• container: a container is a runtime instance of a Docker image
• image: Docker images are the basis of containers (recursion?)
• an image is an ordered collection of root filesystem changes and the
corresponding execution parameters for use within a container runtime
• dockerfile: a Dockerfile is a text document that contains all the
commands you would normally execute manually in order to build a
Docker image
• volume: a data volume is a specially-designated directory within one or
more containers that bypasses the Union File System
• in other words: a volume is a directory on the host which gets mounted into the
container

15. Sample Dockerfile
Reference at: https://docs.docker.com/engine/reference/builder/
FROM UBUNTU:15.10
RUN apt-get update && apt-get install -y php5 php5-curl php5-gd
ADD src/ /var/www/html/
ADD scripts/start.sh /start.sh
EXPOSE 443 80
CMD ["/start.sh"]

16. Docker philosophy
Definition according to Me™, again
• one process per container
• containers should be immutable (or close to it)

17. Useful one-liners
• docker images lists all images present on the host computer
• docker ps lists all running containers
• docker ps -a lists all containers, including stopped ones
• docker rm <cont.> removes a container
• docker rmi <image> removes an image

18. Where the magic stops
• how to share application data with the container?
• how to access the apps in the container (map container ports to the
host)?
• how to start multiple containers together and connect them?
• how to access the log files from the application?

19. Then things get tricky
• mapping of user ids to properly handle filesystem permissions
• linking containers when there are circular dependencies (solved with
recent Docker versions)
• applying security updates to existing containers
• and more…
• for some more of the fine details, see f.e. this blog post:
http://blog.kaliop.com/en/blog/2015/05/27/docker-in-real-life-the-tricky-
parts/

20. Docker Compose to the rescue!
• a single file describes multiple containers
• a single command to start/stop them all together
• permits further automation (via eg. introduction of variables)

21. Sample docker-compose.yml
Reference docs: https://docs.docker.com/compose/compose-file/
version: '2'
services:
web:
build: images/web
hostname: web
ports:
- "80:80"
volumes:
- ./site/:/var/www/site
env_file:
- docker-compose.env
cli:
build: images/cli
hostname: cli
environment:
- SSH_AUTH_SOCK=/ssh-agent
cap_add:
- SYS_PTRACE

22. A Docker stack for eZPublish
A. I need a break
B. Bring it on!
02

23. Why use Docker for eZPublish website development?
Because you get to go to the Summer Camp, duh!
• easier to share with a distributed dev team than a whole VM
just a bunch of scripts, not multi-MB image
easy to fix/improve and redistribute
• easy to convert to the same version of the OS/Apps used in production
• forces separation of applications to separate environments
closer to a production environment (for non-trivial sites)
good to avoid surprises when deploying

24. Introducing the stack
• apache
• mysql
• memcached
• solr
• php cli
• varnish
• nginx
plus
• control panels for all of the above
• phpmyadmin
• maildev (an smtp server for development purposes)
• dockerui (a container monitoring the whole container stack [Inception!])

25. In 1000 words
Ascii art rocks!

26. How it works
All the things someone else has figured out for you
• configuration of the applications is mounted as volume, not copied into
the image
a simple service restart is sufficient to take changes into account
• application source code is kept on the host computer
no risk of losing changes when rebuilding containers
no need to have containers running GUI or IDEs
• all log files accessible from the host, in one place
• use ssh-agent to avoid developers storing their own private keys inside
the containers

27. How it works, II
All the things someone else has figured out for you
• allow developers to change some variables, while providing good
defaults
a configuration file is provided, which has all the variables that developers
can customize
a 2nd configuration file has to be created with local changes (empty is fine)
ex: id of the user account on the host computer, or github credentials
the config file with local changes is not committed to git
• not tied to a specific project
can be quickly converted for usage of Drupal instead of eZPublish
two of them can be run in parallel

28. How it works, III
All the things someone else has figured out for you
• the containers have to replicate as much as possible a production
environment
the base images used are Debian or Ubuntu (the platforms we generally
deploy on)
the applications are started using service start and service stop
commands
cronjob tasks definitions are kept as part of the sources of the stack

29. On Disk layout
• config configuration for services running in containers
• data data persisted by the apps in containers (eg: mysql db)
• images Dockerfiles and files used to build the images
• logs log files from services running in containers
• site the website installation folder
• src source code for tools which are part of the stack

30. On Disk layout and volume mounts
( incomplete example )
• config/…
• data/…
• images X
• logs/…
• site
• src X
Mysql container
/etc/mysql/conf.d/
/var/lib/mysql/
/var/log/mysql/
Web container
/etc/apache2/sites-available
/etc/php5/apache2/
/var/log/apache/
/var/www/site/
Cli container
/tmp/cron.d
/etc/php5/cli
/var/log/php/
/var/www/site/

31. Exercise time
At last!

32. Managing environments
A. Not my job
B. DevOps is the new black
03

33. Where to store per-environment configuration files
• Symfony: excellent native support
• eZPublish legacy: somewhat, hackish support (eg. in extensions?)
• Webserver config etc… : no standard solution

34. All-in-the-stack solution
• Store eZPublish legacy settings and app. settings in a specific dir
in the source code
• Use symlinks to deploy them
• But not manually!
• kaliop/ezpublish5universalinstaller
• Advantage: only one git repo to take care of
• Advantage: supports an ‘override’ logic

35. On-Disk layout

36. Exercise time
Again!

37. Managing database changes
A. I connect to the Admin Interface
B. A solved problem
04

38. Sharing the Database
Many possibilities:
A. All developers connect to the same db
B. The database is committed to git
C. Managing changes via scripts
D. Live DB replication (really ???)
E. More ?

39. Connecting to a shared database
• good for teams which are *not* geographically distributed
• works when developers do not blow up the database with junk
content: needs some developer discipline
• works only up to the 1st deployment to UAT env;
then 2 dbs have to be managed anyway

40. Committing the database to git
• good for when the development database does not contain a
huge number of contents and assets
• good for when the development database does not change too
frequently
• developers might be working on different versions
of the db: needs some developer discipline
• works only up to the 1st deployment to UAT env;
then 2 dbs have to be managed anyway

41. Managing database changes via script
• Safest option
• Good for when the development database does not change too
frequently
• Perfect after deployment to UAT/PROD
• kaliop/ezmigrationbundle

42. Exercise time
Again!

43. Automating deployment
A. FTP is all I need!
B. I put a Jenkins in my Jenkins
05

44. Automating deployments
• do I even have to explain why it is a good idea ?
• use the same deployment script during development as in
production
so that it gets well tested
• good for automated testing too

45. eZPublish deployment tasks
1. alter webserver config to disallow access
2. get the latest version of the source code
3. composer install
4. apply changes to database
5. clear Opcache cache (just in case)
6. custom scripts?
7. reindex content (optional)
8. clear shared caches
always: inner to outer
Memcached
Varnish
9. alter webserver config to allow access

46. eZPublish deployment tasks
extras:
1. run security checks (Symfony has a built-in script)
2. notify monitoring systems of deployment (eg: NewRelic)

47. Exercise time
Again!

48. You do write tests for CMS-
powered websites, don’t you?
A. HAHAHA!
B. Are you joking?
C.No, really?
06

49. Conventional wisdom on testing
Is it really the good choice for sites built on CMS ?

50. Going against the grain
How much non-project-specific code have you used on this project ?
UI
Unit
???

51. Going against the grain
Functional testing rocks the world?
• Use Behat if you *really* have customers / PMs / Analysts willing to read
the scenarios
• Functional tests can be written in PHPUnit too
• Functional testing is slow
Run it on a CI platform
On every commit is perfect, but before deployment is good enough
Use Selenium-as-a-service if you can afford it (even if you can’t)

52. Exercise time
Again!

53. THANK YOU
@gggeek
https://github.com/kaliop-uk
https://github.com/kaliop
http://www.kaliop.co.uk/info/ez.html