• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Cloud party 2014 - Deploy your infrastructure with Saltstack - Salt Cloud with OpenStack

Cloud party 2014 - Deploy your infrastructure with Saltstack - Salt Cloud with OpenStack



Deploy in minutes your web infrastructure on OpenStack with Saltstack and Salt Cloud extension!

Deploy in minutes your web infrastructure on OpenStack with Saltstack and Salt Cloud extension!



Total Views
Views on SlideShare
Embed Views



4 Embeds 108

http://walterdalmut.com 97
https://twitter.com 6
http://www.slideee.com 3
http://plus.url.google.com 2



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Cloud party 2014 - Deploy your infrastructure with Saltstack - Salt Cloud with OpenStack Cloud party 2014 - Deploy your infrastructure with Saltstack - Salt Cloud with OpenStack Presentation Transcript

    • CloudParty 2014 Application and Infrastructure deployment on Cloud Providers by / - www.corley.it Walter Dal Mut @walterdalmut Corley S.r.l.
    • The Problem Your web application never scale
    • It is really scalable?
    • Our daily goal?
    • A simple scalable web app
    • Why so many layers? Manage every layer by it-self Optimize every layer by it-self Scale every layer by it-self Monitor every layer by it-self Securize every layer by it-self ...
    • The Cloud Add more resources when we need in seconds Remove resources when we don't need them anymore Reduce time to market Turn a fixed cost into a variable costs pay only for what you use
    • Deploy & Orchestation Automate your infrastructure and deployment
    • Distributed Application
    • Split out your Infrastructure Networks Subnetworks
    • Security Groups We will assign a different security-group for every group of VMs. In that way we can apply our security policy in a simple and powerful way.
    • Security Map Database layer (MySQL) PORT 3306 <- from web layer Cache layer (Memcached) PORT 11211 <- from web layer Web layer (Apache2) PORT 80 <- from proxy layer Proxy layer (Nginx) PORT 80 <- from everywhere
    • Security Groups
    • We will use Salt-Cloud It means that we also need to allow SSH connections from the "master" Every "minion" has also another security-group "salt-minion" that allows SSH connections from the "salt-master" instance
    • Salt TOP.SLS base:     '*':         ‐ base dev:     ... prod:     'proxy.milan.enter.*.prod':         ‐ nginx     'web.milan.enter.*.prod':         ‐ webserver         ‐ webapp     'cache.milan.enter.*.prod':         ‐ memcached     'rdb.milan.enter.*':         ‐ mysql         ‐ mysql.master     'srdb.milan.enter.*':         ‐ mysql
    • What about "salt-cloud" Salt cloud is made to integrate Salt into cloud providers in a clean way so that minions on public cloud systems can be quickly and easily modeled and provisioned. http://salt-cloud.readthedocs.org/en/latest/
    • salt-cloud for OpenStack We need a Provider definition enter‐openstack‐config:   minion:     master:   identity_url: https://api‐legacy.entercloudsuite.com:5000/v2.0/tokens   compute_name: nova   protocol: ipv4   compute_region: ItalyMilano1   user: name@user.tld   password: YourPassword   tenant: name@user.tld   provider: openstack
    • salt-cloud for OpenStack We need VMs profiles rdb:     provider: enter‐openstack‐config     size: e1standard.x4     image: GNU/Linux Ubuntu Server 12.04 LTS Precise Pangolin x64     ssh_username: ubuntu     ssh_key_file: /root/private‐key.pem     ssh_key_name: 'private‐key‐name'     ssh_interface: public_ips     security_groups: salt‐minion,mysql     networks:         ‐ fixed:             ‐ xxxxxxxx‐xxxx‐xxxx‐xxxx‐xxxxxxxxxxxx web:     ...
    • Automatic IP management Proxies need Web instaces IPs and so on...
    • Enable Peer communication allow Salt minions to pass commands to each other peer:   .*:     ‐ .* We will use this features to share IP addresses You can use "grains" or "mines" instead
    • Let's go
    • Database layer Create a new resource with salt-cloud salt‐cloud ‐p PROFILE VM‐NAME salt‐cloud ‐p rdb rdb.milan.enter.1.prod
    • Deploy the Master RDB salt 'rdb.*' state.highstate
    • Create a group of slaves Can we paralelize all VM creation? salt‐cloud ‐Pp PROFILE VM‐NAME VM‐NAME ... "-P" option means "parallel" salt‐cloud ‐Pp srdb      srdb.milan.enter.1.prod      srdb.milan.enter.2.prod      srdb.milan.enter.3.prod
    • Deploy Slave RDB salt 'srdb.*' state.highstate
    • Prepare all databases Now we have a Master instance and 3 slaves We have to prepare Read-Replicas CHANGE MASTER TO     MASTER_HOST='xxx.xxx.xxx.xxx',     MASTER_USER='repl‐user',     MASTER_PASSWORD='repl‐pass',     MASTER_LOG_FILE='mysql‐bin‐xxxxx',     MASTER_LOG_POS=xxx
    • Execute MySQL commands salt 'rdb.milan.enter.1.prod' mysql.query mysql 'show master status' salt 'srdb.*' mysql.query mysql '     CHANGE MASTER TO     MASTER_HOST="xxx.xxx.xxx.xxx",     MASTER_USER="repl‐user",     MASTER_PASSWORD="repl‐pass",     MASTER_LOG_FILE="mysql‐bin‐xxxxx",     MASTER_LOG_POS=xxx     '
    • Now we have our databases
    • Now the cache layer
    • Add cache resources salt‐cloud ‐Pp cache    cache.milan.enter.1.prod    cache.milan.enter.2.prod    cache.milan.enter.3.prod    cache.milan.enter.4.prod salt 'cache.*' state.highstate
    • Memcached will help us with caching and session management
    • When we distribute the load across a group of VMs all information should be available to the group otherwise we have connectivity problems Cache warm up, disconnected users, and more...
    • Distribute the load
    • The Web Tier All Web VMs need to know DB and Cache nodes addresses Master DB address Slaves DB addresses Session VMs addresses Cache VMs addresses
    • Distribute the load
    • Memcached Session handler # php.ini session.save_handler = memcached session.save_path = ",,," {% set memcached_servers = [] %} {% for server,ip in salt['publish.publish']('cache.*', 'network.interfaces').items() %} {% set m = memcached_servers.append(ip.eth0.inet[0].address) %} {% endfor %} session.save_handler = memcached session.save_path = "{{ memcached_servers|join(", ") }}
    • Database The problem: no default multiple connections MySQLi($host, $username, $password); //PDO, ... How to handle multiple connections? Write-Read and Read only? Master/Slave Async Replication
    • MySQL_ND Replace libmysql driver Default connector in PHP 5.4 MySQL_ND Master/Slave (plugin)
    • MySQLND_MS Configuration {     "myapp": {         "master": {             "master_0": {                 "host": "localhost",             }         },         "slave": {             "slave_0": {                 "host": "",             }         }     } }
    • Configuration in Salt {     "cwitter.db": {         "master": {             {% for server,ip in salt['publish.publish']('rdb.*', 'network.interfaces').items() %}             "master_{{ server[0] }}": {                 "host": "{{ ip.eth0.inet[0].address }}"             }             {% endfor %}         },         "slave": {             {% for server,ip in salt['publish.publish']('srdb.*', 'network.interfaces').items() %}                 "slave_{{ server[0] }}": {                 "host": "{{ ip.eth0.inet[0].address }}"             }{% if not loop.last %},{% endif %}             {% endfor %}         },         "trx_stickiness": "master"     } }
    • Transaction Aware By default MySQLND_MS is not transaction aware trx_stickiness: master BEGIN TRANSACTION INSERT INTO ... DELETE FROM SELECT u1, u2, ... FROM ... UPDATE FROM COMMIT
    • Now the application distribute user sessions and DB queries
    • Distribute also HTTP requests!
    • Proxy HTTP/s requests
    • Proxy configuration needs Pubic IPs proxy:     provider: enter‐openstack‐config     size: e1standard.x1     image: GNU/Linux Ubuntu Server 12.04 LTS Precise Pangolin x64     ssh_username: ubuntu     ssh_key_file: /root/test.pem     ssh_key_name: 'my key name'     ssh_interface: public_ips     security_groups: salt‐minion,proxy     networks:         ‐ fixed:             ‐ xxxxxxxx‐xxxx‐xxxx‐xxxx‐xxxxxxxxxxxx         ‐ floating:             ‐ yyyyyyyy‐yyyy‐yyyy‐yyyy‐yyyyyyyyyyyy
    • NGINX as a proxy upstream app {     server;     server;     server;     # web server list } server {     listen 80;     location / {         proxy_pass http://app;     } }
    • NGINX proxy with Salt upstream app {     {% for server,ip in salt['publish.publish']('web.*', 'network.interfaces').items() %}     server {{ ip.eth0.inet[0].address }}:80;     {% endfor %} } server {     listen 80;     location / {          proxy_pass http://app;     } }
    • Use DNS round-robin feature in order to resolve proxies's IP
    • The app is ready!
    • Thanks for listening Walter Dal Mut Github: Twitter: Linkedin: wdalmut @walterdalmut Walter Dal Mut