DevOPS training - Day 1/2

DevOPS – Day 1
Thursday, May 20th 2016

About me
Vincent Mercier
vincemercier
System Architect
Integrate voice applications to your professional tools. Our
features include: managing calls in real time, Call Tracking,
Call Controlling, Push SMS, Click-To-Call, real time
statistics, audio conferences, …
Smart Voice Services

Basics of DevOPS
What you should know…

Infrastructure domains
Network
Database
System
Virtualization
Security
Anna
Creative Director

Several roles in the team
Engineer
Technical expert
Architect
Global vision of the infrastructure
Administrator
Deploy services defined by engineers
Execute daily tasks
CTO
Define technical roadmap based on business objectives
Technician
Customer support / Helpdesk
Execute daily tasks

DevOPS
DEV
Develops applications based
on business objectives
OPS
Deploy services to run
applications with
production constraints
Production
Company business

IT revolutions
World Wide Web
1989
Cloud services
2006
What’s next?
~2020
Virtualization
1999
Containers
2013

Monolithic architecture
Server
Application

3 tiers architecture
Server
Presentation
Application
Data

Modern web architecture
Frontend
Client side
Backend API
CDN
Application Hub
CI
Deployment tools
Queue system
DNS
Database Cache

Server
HTTP(macro)
Body
Client
HTTP/1.1 200 OK
Date: Thu, 19 May 2016 08:25:10 GMT
Content-Type: text/html; charset=UTF-8
Content-Encoding:UTF-8
Content-Length:89
Last-Modified: Sun, 29 May 2016 08:25:10 GMT
Server:Apache/2.2.15 (CentOS)
Accept-Ranges:bytes
Connection:close
<html>
<head>
<title>Blog</title>
</head>
<body>
Welcome on MyCorp!
</body>
</html>
GET /contact HTTP/1.1
Host: blog.mycorp.local
User-Agent:cli/1.0
Accept: text/html
Header

Virtual hosting
• Method to host multiple websites on the same server
• 2 methods:
• Per IP
• 1 IP address per website
• Works with all protocols
• Difficult in production
• Per domain names
• HTTP/1.1 required
• Server Name Indication(SNI) for HTTPS
Headers fields are key-value
in clear-text.
HTTP headers are in request
and response.
Popular headers: Host, Content-
Type, Etag, X-Forwarded-For
HTTP headers

Virtual hosting
Server
Blog
Extranet
Always define a default
website.
Best practice
User-Agent:cli/1.0
Accept: */*
GET /calendar HTTP/1.1
Host: extranet.mycorp.local
User-Agent:cli/1.0
Accept: */*

HTTPS
• Create a secure channel over network
• Not a protocol, just HTTP with a secure layer (TLS or SSL)
• Certificates must be signed by a certificate authority

HTTPS
16
Application
Presentation
Session
Transport
Network
Data link
Physical
OSI model
HTTP
HTTP request
TCP
IPv4 / IPv6
HTTP
HTTPS request
TCP
IPv4 / IPv6
SSL / TLS

HTTPS
Browser
Server
Root
certificate
authorities
Public
certificate
CSR
SSL Key
Root
certificate
public
keys
Root
certificate
public
keys
Pre-installed
root public
certificates

HTTP2
Second major version of HTTP since
HTTP 1.1 in 1999!
Revolution
Server Push
Server push cache content to browser
Compression
Compress HTTP body and HTTP
headers. Differential encoding
(Headers are stored)
Multiplexed
Allowing multiple requests and
responses to be sent at the same time.
Binary
HTTP/2 is a binary protocol
SPDY
Based on SPDY protocol created at
Google to reduce web page load
latency and improve web security

Request 2
HTTP1
Referer: http://blog.mycorp.local/
Date: Tue, 17 May 2016 20:16:21 GMT
Via: 1.1 varnish
Cache-Control: max-age=600
Expires: Mon, 16 May 2016 08:57:14 GMT
X-Served-By: cache-fra1238-FRA
X-Cache: HIT
X-Cache-Hits: 1
Server: Apache/2.2.15 (CentOS)
Vary: Accept-Encoding,Country-Code
X-Fastly-Request-ID:
2cfe9ab75a31cf355f59f0ab2491932398432831
X-Powered-By: PHP/5.3.3
[Body]
HTTP1 vs HTTP2
02
Request 1
HTTP1
GET /home HTTP/1.1
Referer: http://blog.mycorp.local/
Date: Tue, 17 May 2016 20:16:06 GMT
Via: 1.1 varnish
X-Cache: HIT
X-Cache-Hits: 1
X-Fastly-Request-ID:
2cfe9ab75a31cf355f59f0ab2491932398432831
[Body]
Request 2
HTTP2
:method: GET
:scheme: http
:host: blog.mycorp.local
:path: /home
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT
X-Cache-Hits: 1
X-Fastly-Request-ID: 2cfe9ab75a31cf355f59f0ab2491932398432831
[Body]
Request 2
HTTP2
:path: /contact
referer: http://blog.mycorp.local/home
X-Fastly-Request-ID: 2cfe9ab75a31cf355f59f0ab2491932398434234
[Body]

HTTP1 vs HTTP2HTTP1
HTTP2
Domainsharding
Asset contatenation
Keep-alive
Domain sharding
Asset contatenation
TCP connexion

Let’s go for HTTP2?
• Clients and servers libraries are quiet new…
• HTTP2 is implemented only with TLS in most of browsers
• http://caniuse.com/#feat=http2
• https://github.com/http2/http2-spec/wiki/Implementations
• Next step?
• QUIC!
• Built on top of UDP
• https://www.chromium.org/quic

Resources for DevOPS
• MAN…
• Helpful websites
• https://www.howtoforge.com
• http://highscalability.com
• French mailing list
• http://www.frsag.com
• Meetup

Workshop
Webhosting for a Symphony application
24

Goals
• Install a webserver
• Linux
• MariaDB
• PHP
• Nginx
• Running a Symphony application
• Backup and restore the website

Technical details
• Server
• Virtual machine (Virtualbox + Vagrant)
• IP: 192.168.42.42
• Linux / Nginx / PHP / MariaDB
• Website
• Domain: blog.mycorp.local
• SSL
• Symphony application

Layers
PHP interpreter
/var/run/php-fpm.socket
Database
TCP 3306
Virtualization system
Managed by Vagrant
Web server
TCP 80 and TCP 443
Web application
/var/www/blog.mycorp.local
Linux Debian
Nginx
Symphony
Linux
PHP
MariaDB
Virtualbox
01
03
05
02
04
06

Linux Debian
• One of the most popular Linux distribution
• Focused on collaborative software development (Social contract / DFSG)
• Used by Ubuntu
• Release names come from Toy Story movies
• Current stable version : 8.4 (Linux kernel 3.16.0)

Linux Debian – Installation
• Initialize Vagrant environment(create Vagrantfile)
• Edit vagrant configuration file
• Add private network interface
• Start virtual machine
• Connect to the server
vagrant init debian/jessie64
vim Vagrantfile
config.vm.network"private_network",ip: "192.168.42.42"
vagrant up
vagrant ssh

Sudo?
• Switch to super user
• Edit vagrant filesudo -i
$ is a user
# is a super user
The prompt

Installation
• Install a text editor
• Install GIT
• Install troubleshooting tools
• Install system administration tools
apt-get install vim
apt-get install git
apt-get install curl tcpdump
apt-get install htop

MariaDB
• MySQL fork
• https://mariadb.com/kb/en/mariadb/mariadb-vs-mysql-features/
• Current stable version : 10.1

MariaDB – Installation
• Install MariaDB
• Connect to the database
• Create database
• Create user
apt-get install mariadb-server
mysql -p
CREATE DATABASE mycorp_blog;
GRANT ALL PRIVILEGES ON mycorp_blog.*TO mycorp_blog@localhost IDENTIFIED by 'hackme';

PHP
• Current stable version : 5.6.21 / 7.0.6
• ⚠️ 5.6.20 in Debian

PHP – Installation
• Install PHP
• Install PHP additional modules
• Edit PHP configuration files
• Set timezone to UTC
• Restart PHP FPM
apt-get install php5-fpm
apt-get install php5-mysql php5-intl
vim /etc/php5/cli/php.ini/etc/php5/fpm/php.ini
date.timezone = UTC
service php5-fpm restart

PHP composer – Installation
• Install PHP composer
• Check PHP composer version
curl -sS https://getcomposer.org/installer | sudo php -- --install-dir=/usr/local/bin--filename=composer
composer -V

Nginx
• Current stable version : 1.9.15
• ⚠️ 1.6.2 in Debian

Nginx – Installation
• Install Nginx
• Check default page
apt-get install nginx
curl -v http://192.168.42.42

SSL certificate – Installation
• Go to private SSL certificates directory
• Define filename
• Generate SSL private key and CSR
• Generate self-sign SSL certificate
cd /etc/ssl/private/
FQDN=blog.mycorp.local
openssl req -nodes -newkey rsa:2048-sha256 -keyout $FQDN.key -out $FQDN.csr
openssl x509 -req -days 365 -in $FQDN.csr -signkey $FQDN.key -out $FQDN.crt

SSL certificate – Installation
• Restrict access to the SSL key file
• Allow root to modify the key and www-data to read it
chmod 0640 blog.mycorp.local.key
chown root:www-data blog.mycorp.local.key

Download website source code
• Go to websites directory
• Clone Git repository
• Copy default configuration file
• Edit database parameters
cd /var/www/
git clone https://github.com/vmercierfr/training-ops-1.gitblog.mycorp.local
cd blog.mycorp.local/app/config/
cp parameters.yml.distparameters.yml
vim parameters.yml

Nginx – Configuration
• Go to nginx configuration directory
• Create blog.mycorp.local
cd /etc/nginx/sites-available/
vim blog.mycorp.local

server {
listen 80;
server_name blog.mycorp.local;
return 301 https://blog.mycorp.local/$1;
}
server {
listen 443 ssl;
server_name blog.mycorp.local;
ssl on;
ssl_certificate /etc/ssl/private/blog.mycorp.local.crt;
ssl_certificate_key /etc/ssl/private/blog.mycorp.local.key;
root /var/www/blog.mycorp.local;
access_log /var/log/nginx/blog.mycorp.local.log;
error_log /var/log/nginx/blog.mycorp.local.error.log info;
}
Force HTTPS
SSL

• Go to Nginx configuration directory
• Link configuration file
• Check Nginx configuration
• Reload Nginx
cd /etc/nginx/sites-enabled/
ln -s ../sites-available/blog.mycorp.local
nginx -t
service nginx reload

DNS configuration
• Edit your local hosts file
• Add following line
• Connect to the website
sudo vim /etc/hosts
192.168.42.42 blog.mycorp.local
curl -v https://blog.mycorp.local
%SystemRoot%System32driversetchosts
Windows host file

Nginx + PHP – Configuration
• Edit website configuration file
• Add the following lines
• Check and reload Nginx configuration
vim /etc/nginx/sites-available/blog.mycorp.local
root /var/www/blog.mycorp.local/web;
location / {
try_files $uri /app.php$is_args$args;
}
location ~ ^/app.php(/|$) {
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_split_path_info ^(.+.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT$realpath_root;
internal;
}
nginx -t && service nginx reload

PHP errors
• Check the errors log file
tail -f /var/log/nginx/blog.mycorp.local.error.log
2016/05/18 23:02:12 [error] 8885#0: *119 FastCGI sent in stderr: "PHP message: PHP Warning: require(/var/www/blog.mycorp.local/app/../vendor/autoload.php): failed to open stream: No
such file or directory in /var/www/blog.mycorp.local/app/autoload.php on line 11
PHP message: PHP Fatal error: require(): Failed opening required '/var/www/blog.mycorp.local/app/../vendor/autoload.php' (include_path='.:/usr/share/php:/usr/share/pear') in
/var/www/blog.mycorp.local/app/autoload.php on line 11" while reading response header from upstream, client: 192.168.42.1, server: blog.mycorp.local, request: "GET /en/blog/
HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "blog.mycorp.local", referrer: "https://blog.mycorp.local/"
2016/05/18 23:03:07 [error] 8885#0: *119 FastCGI sent in stderr: "PHP message: PHP Warning: require(/var/www/blog.mycorp.local/app/../vendor/autoload.php): failed to open stream: No
such file or directory in /var/www/blog.mycorp.local/app/autoload.php on line 11
PHP message: PHP Fatal error: require(): Failed opening required '/var/www/blog.mycorp.local/app/../vendor/autoload.php' (include_path='.:/usr/share/php:/usr/share/pear') in
/var/www/blog.mycorp.local/app/autoload.php on line 11" while reading response header from upstream, client: 192.168.42.1, server: blog.mycorp.local, request: "GET /en/blog/
HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "blog.mycorp.local"

Initiate application
• Go to website directory
• Install application’s requirements with composer
• Set rights on app directory
• Initialize the database
• Load fixtures
cd /var/www/blog.mycorp.local/
composer install
chown -R www-data app
php app/console doctrine:schema:create
php app/console doctrine:fixtures:load

Configure backup
• Install backup ninja
• Launch ninjabackup wizard
• Create a backup job
• Run a backup job
apt-get install backupninja
ninjahelper

Restore database backup (Quick and very dirtymethod)
• Connect to MySQL
• Delete database
• Create database
• Select database
• Import backup
mysql -p
DROP DATABASE mycorp_blog;
CREATE DATABASE mycorp_blog;
use mycorp_blog;
source /var/backups/mysql/sqldump/mycorp_blog.sql;
Don’t delete database in production, rename it!
(Need to move each tables)
Best pratice

DevOPS training - Day 1/2

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to DevOPS training - Day 1/2

Similar to DevOPS training - Day 1/2 (20)

Recently uploaded

Recently uploaded (20)

DevOPS training - Day 1/2