Austin Linux Meetup
Linux Du Jour
A tour of Texas Linux Fest
in 5 little talks
Texas Linux Fest
June 13 - 14
Austin
texaslinuxfest.org
I’m Matthew
DevOps Engineer at RetailMeNot
We’re a Linux shop
systemd
Docker
Configuration Management
Distributed Log Collection
OpenStack
Topics
I’m not an expert in these technologie...
systemd
system and service manager
(first program started by the kernel)
systemd
• Replaces init
• Dependency-based
• Also replaces inetd
starts programs at boot
programs are started only when ne...
systemd
• units, not scripts
!
versus traditional init
Lines 77 - 162
/etc/init.d/ssh
Debian Squeeze
case "$1" in
start)
[...]
	
 ;;
!
stop)
[...]
	
 ;;
!
reload|force-reload)
[...
[Unit]
Description=OpenSSH server daemon
After=syslog.target network.target auditd.service
!
[Service]
EnvironmentFile=/et...
systemd
• units, not scripts
• dependencies, not ordering
versus traditional init
# Provides: sshd
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Defau...
[Unit]
Description=OpenSSH server daemon
After=syslog.target network.target auditd.service
!
[Service]
EnvironmentFile=/et...
systemd
• units, not scripts
• dependencies, not ordering
• systemctl, not service
versus traditional init
root@box-sol-aus-eop-pup-aio-000-i-simulate:~# service --status-all
[ + ] acpid
[ + ] apache2
[ ? ] apt-cacher-ng
[ + ] at...
UNIT LOAD ACTIVE SUB DESCRIPTION
[...]
chronyd.service loaded active running NTP client/server
crond.service loaded active...
Unit Commands:
list-units List loaded units
list-sockets List loaded sockets ordered by address
start [NAME...] Start (act...
learning systemd
http://0pointer.de/blog/projects/inetd.html
(includes links to 10 predecessors)
Configuration
Management
tracking system changes
i.e. ansible, cfengine, chef, puppet, saltstack
Configuration Management
• Declarative, not imperative
• Extensible
• Master-client or standalone
Some are less declarativ...
Architecture
master
client
config
repo
disc.
agent
ansible chef puppet salt
repo
module/
playbook
recipe/
cookbook
module/
...
Ansible
• Python-based (2.X)
• Push-based (no server)
• Works over SSH
• Config format: YAML
• Modules: Any language
Empha...
Chef
• Ruby-based
• Config format: Ruby
• Modules: Ruby
Emphasis on developer-friendly, agile experience
Puppet
• Ruby-based
• Config format: Puppet DSL
• Modules: Puppet DSL, Ruby (for extensions)
Emphasis on declaration, not ...
SaltStack
• Python-based
• Config format: YAML
• Modules: Python/Cython
Emphasis on speed, scalability
Installing Apache
… with ansible
/etc/ansible/hosts
localhost ansible_connection=local
local-apache.yml (playbook)
---
- h...
# mkdir cache
# chef-solo --override-runlist 
"recipe[apache::install]" -c config.rb
Installing Apache
… with chef
config.r...
# puppet apply apache-install.pp
Installing Apache
… with puppet
apache-install.pp
package {'apache2':
ensure => 'installe...
Installing Apache
… with saltstack
/etc/salt/minion
file_client: local
/srv/salt/top.sls
base:
'*':
- apache
# salt-call -...
learn more
ansible
http://docs.ansible.com/
chef
http://docs.opscode.com/
puppet
http://docs.puppetlabs.com/
saltstack
htt...
docker
container deployment and management
Containers
• lightweight virtual machine
• chroot on steroids
Lighter than a VM
• Same kernel
• Little or no boot time
• A...
Docker
• layered filesystem templates
• container build and run automation
• expose network services
Docker
• installation
• first docker container
• dockerfiles
Demo: Install
• yum install -y docker-io
• systemctl start do...
learning docker
http://docs.docker.io/introduction/working-with-docker/
https://www.stgraber.org (LXC itself)
Distributed Log Collection
Architecture
client
client
client
aggregator
files
database
search
index
Aggregators
• Accept
• Transform
• Filter
• Forward
Accept
• native formats like system logs, message queues, snmp
• via c...
What’s available?
logstash

fluentd

flume

http://logstash.net

http://fluentd.org

https://flume.apache.org/

Logstash -...
ElasticSearch + Kibana
• Auto-balancing, auto-scaling search index, REST API
• Pretty GUI for searching logs
Works with an...
learn more
http://jasonwilder.com/blog/2012/01/03/centralized-logging/

http://jasonwilder.com/blog/2013/11/19/fluentd-vs-...
OpenStack
open source virtualization platform
OpenStack Capabilities
• Virtual Machines
• Storage (Object and Block)
• Database (MySQL-compatible)
• Networking
• REST A...
AWS Equivalents
OpenStack AWS
Compute Nova EC2
Networking Neutron Classic + VPC
Database Trove RDS
Storage (Block) Cinder ...
Public OpenStack Clouds
The Competition
• Apache CloudStack
• VMware vCloud
• Eucalyptus
Setting Up Your Own
• DevStack (http://devstack.org/)
• OpenStack Guides
• http://docs.openstack.org/icehouse/
• Ubuntu Cl...
learn more
http://docs.openstack.org/
puppet
chef
saltstackansible openstack
fluentd
flume
logstash
docker
systemd
configuration management
distributed logging
lin...
Upcoming SlideShare
Loading in …5
×

Linux Du Jour

567 views

Published on

A short overview of current technologies plucked from the Texas Linux Fest schedule for 2014. Includes overviews of systemd, popular configuration management tools, docker, distributed log collection, and openstack.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
567
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • We’re a Linux shop
  • I’m not an expert in these technologies
    Mostly learned about them in the last few days
  • starts programs at boot
    programs are started only when needed by other programs
    parallel startup
    can start programs in response to TCP connections, but also via DBUS
  • Hard to see, important thing is the case statement that responds to start, stop, etc.
    86 lines, less than half the init script for ssh
  • Contrast
    15 lines!
  • Metadata at the top of init scripts
    Helps maintains run level link madness
  • Requirements
    Target: runlevel equivalent
  • Tracks state automatically
  • Standard set of commands for all services
  • Some are less declarative than others
    All are extensible
    All can run master-client or standalone
  • Emphasis on virtual infrastructure integration
  • Emphasis on developer-friendly, agile experience
  • Emphasis on declaration, not process
  • Emphasis on speed, scalability
  • Lighter than a VM
    Same kernel
    Little or no boot time
    As little as one process
    Mightier than chroot
    resource constraints (memory, CPU)
    separate pids, users, groups, networking
  • Demo: Install
    yum install -y docker-io
    systemctl start docker
    systemctl enable docker
    docker pull ubuntu:14.04
    docker images
    Demo: Run
    docker run -i -t —name demo ubuntu:14.04 /bin/bash
    dpkg -l
    lsb_release -d
    exit
    docker ps -a
    docker start demo
    docker attach demo
    ps awfux
  • Accept
    native formats like system logs, message queues, snmp
    via client agents
    Transform
    Add structure (json, etc)
    Extract data into fields (hostname, message, time, etc)
    Filter
    Remove or combine entries
    combine multi-line
    criteria matching
    Forward
    Send logs to their destination
    storage, notifications/alerts, message queues
  • Logstash - Aimed at flexibility
    includes tons of inputs, filters, and outputs
    FluentD - Aimed at robustness
    built-in support for high availability, delivery assurance
    Flume - Part of the Hadoop ecosystem
    stores data in HDFS
  • Works with any of these aggregators
    Great replacement for Splunk
    Missing the alerting component, but easy to work around
  • Some are similar, but not direct equivalents (e.g. Keystone)

    OpenStack-based providers usually provide several other services like DNS, CDN, or message queues (Amazon does too)
  • DevStack is installable on a VM, even Vagrant
    Guides for CentOS, Debian Wheezy, Fedora, RedHat, SuSE, Ubuntu
    Cloud Installer requires six hosts (can be VMs)
  • Linux Du Jour

    1. 1. Austin Linux Meetup
    2. 2. Linux Du Jour A tour of Texas Linux Fest in 5 little talks
    3. 3. Texas Linux Fest June 13 - 14 Austin texaslinuxfest.org
    4. 4. I’m Matthew DevOps Engineer at RetailMeNot We’re a Linux shop
    5. 5. systemd Docker Configuration Management Distributed Log Collection OpenStack Topics I’m not an expert in these technologies Mostly learned about them in the last few days
    6. 6. systemd system and service manager (first program started by the kernel)
    7. 7. systemd • Replaces init • Dependency-based • Also replaces inetd starts programs at boot programs are started only when needed by other programs • parallel startup can start programs in response to TCP connections, but also via DBUS
    8. 8. systemd • units, not scripts ! versus traditional init
    9. 9. Lines 77 - 162 /etc/init.d/ssh Debian Squeeze case "$1" in start) [...] ;; ! stop) [...] ;; ! reload|force-reload) [...] ;; ! restart) [...] ;; ! try-restart) [...] ;; ! status) [...] ;; ! *) [...] esac Hard to see, important thing is the case statement that responds to start, stop, etc. 86 lines, less than half the init script for ssh
    10. 10. [Unit] Description=OpenSSH server daemon After=syslog.target network.target auditd.service ! [Service] EnvironmentFile=/etc/sysconfig/sshd ExecStartPre=/usr/sbin/sshd-keygen ExecStart=/usr/sbin/sshd -D $OPTIONS ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure RestartSec=42s ! [Install] WantedBy=multi-user.target /usr/lib/systemd/system/sshd.service Fedora 20 15 Lines Contrast 15 lines!
    11. 11. systemd • units, not scripts • dependencies, not ordering versus traditional init
    12. 12. # Provides: sshd # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: # Short-Description: OpenBSD Secure Shell server lrwxrwxrwx 1 root root 17 Mar 26 02:05 S01vboxadd -> ../init.d/vboxadd lrwxrwxrwx 1 root root 25 Mar 26 02:05 S02vboxadd-service -> ../init.d/vboxadd-service lrwxrwxrwx 1 root root 17 Mar 26 02:06 S16portmap -> ../init.d/portmap lrwxrwxrwx 1 root root 20 Mar 26 02:06 S17nfs-common -> ../init.d/nfs-common lrwxrwxrwx 1 root root 23 Mar 27 04:28 S19apt-cacher-ng -> ../init.d/apt-cacher-ng lrwxrwxrwx 1 root root 20 Mar 27 04:33 S19fancontrol -> ../init.d/fancontrol lrwxrwxrwx 1 root root 17 Mar 26 02:06 S19rsyslog -> ../init.d/rsyslog lrwxrwxrwx 1 root root 14 Mar 26 02:06 S19sudo -> ../init.d/sudo lrwxrwxrwx 1 root root 17 Mar 27 04:25 S20apache2 -> ../init.d/apache2 lrwxrwxrwx 1 root root 15 Mar 27 04:25 S21acpid -> ../init.d/acpid lrwxrwxrwx 1 root root 13 Mar 27 04:25 S21atd -> ../init.d/atd lrwxrwxrwx 1 root root 14 Mar 27 04:25 S21cron -> ../init.d/cron lrwxrwxrwx 1 root root 13 Mar 27 04:25 S21ntp -> ../init.d/ntp lrwxrwxrwx 1 root root 20 Mar 27 04:33 S21postgresql -> ../init.d/postgresql lrwxrwxrwx 1 root root 16 Mar 27 04:25 S21puppet -> ../init.d/puppet lrwxrwxrwx 1 root root 18 Mar 27 04:34 S21puppetdb -> ../init.d/puppetdb lrwxrwxrwx 1 root root 22 Mar 27 04:28 S21puppetmaster -> ../init.d/puppetmaster lrwxrwxrwx 1 root root 18 Mar 27 04:28 S21puppetqd -> ../init.d/puppetqd lrwxrwxrwx 1 root root 15 Mar 27 04:25 S21rsync -> ../init.d/rsync lrwxrwxrwx 1 root root 15 Mar 27 04:33 S21snmpd -> ../init.d/snmpd lrwxrwxrwx 1 root root 13 Mar 27 04:25 S21ssh -> ../init.d/ssh lrwxrwxrwx 1 root root 17 Mar 27 04:34 S21sysstat -> ../init.d/sysstat lrwxrwxrwx 1 root root 15 Mar 27 04:33 S22exim4 -> ../init.d/exim4 lrwxrwxrwx 1 root root 18 Mar 27 04:25 S23bootlogs -> ../init.d/bootlogs lrwxrwxrwx 1 root root 18 Mar 27 04:25 S24rc.local -> ../init.d/rc.local lrwxrwxrwx 1 root root 19 Mar 27 04:25 S24rmnologin -> ../init.d/rmnologin lrwxrwxrwx 1 root root 23 Mar 27 04:25 S24stop-bootlogd -> ../init.d/stop-bootlogd Metadata at the top of init scripts Helps maintains run level link madness
    13. 13. [Unit] Description=OpenSSH server daemon After=syslog.target network.target auditd.service ! [Service] EnvironmentFile=/etc/sysconfig/sshd ExecStartPre=/usr/sbin/sshd-keygen ExecStart=/usr/sbin/sshd -D $OPTIONS ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure RestartSec=42s ! [Install] WantedBy=multi-user.target Requirements Target: runlevel equivalent
    14. 14. systemd • units, not scripts • dependencies, not ordering • systemctl, not service versus traditional init
    15. 15. root@box-sol-aus-eop-pup-aio-000-i-simulate:~# service --status-all [ + ] acpid [ + ] apache2 [ ? ] apt-cacher-ng [ + ] atd [ - ] bootlogd [ - ] bootlogs [ ? ] bootmisc.sh [ ? ] checkfs.sh [ - ] checkroot.sh [ ? ] console-setup [ ? ] cron [ - ] exim4 [ - ] fancontrol [ - ] hostname.sh ... service(8) System Manager's Manual service(8) ! [...] ! DESCRIPTION [...] The SCRIPT parameter specifies a System V init script, located in /etc/init.d/SCRIPT. The supported values of COMMAND depend on the invoked script, service passes COMMAND and OPTIONS it to the init script unmodified. All scripts should support at least the start
    16. 16. UNIT LOAD ACTIVE SUB DESCRIPTION [...] chronyd.service loaded active running NTP client/server crond.service loaded active running Command Scheduler cryptsetup.target loaded active active Encrypted Volumes dbus.service loaded active running D-Bus System Message Bus dbus.socket loaded active running D-Bus System Message Bus So dev-dmx2d0.swap loaded active active /dev/dm-0 dev-hugepages.mount loaded active mounted Huge Pages File System dev-mqueue.mount loaded active mounted POSIX Message Queue File Sy dm-event.socket loaded active listening Device-mapper event daemon docker.service loaded active running Docker Application Containe fedora-readonly.service loaded active exited Configure read-only root su firewalld.service loaded active running firewalld - dynamic firewal getty.target loaded active active Login Prompts getty@tty1.service loaded active running Getty on tty1 kmod-static-nodes.service loaded active exited Create list of required sta local-fs-pre.target loaded active active Local File Systems (Pre) local-fs.target loaded active active Local File Systems lvm2-lvmetad.service loaded active running LVM2 metadata daemon lvm2-lvmetad.socket loaded active running LVM2 metadata daemon socket lvm2-monitor.service loaded active exited Monitoring of LVM2 mirrors, lvm2-pvscan@8:2.service loaded active exited LVM2 PV scan on device 8:2 multi-user.target loaded active active Multi-User System network.target loaded active active Network paths.target loaded active active Paths polkit.service loaded active running Authorization Manager proc-sys...t_misc.automount loaded active waiting Arbitrary Executable File F remote-fs.target loaded active active Remote File Systems session-3.scope loaded active running Session 3 of user vagrant slices.target loaded active active Slices sockets.target loaded active active Sockets sound.target loaded active active Sound Card sshd.service loaded active running OpenSSH server daemon [...] Tracks state automatically
    17. 17. Unit Commands: list-units List loaded units list-sockets List loaded sockets ordered by address start [NAME...] Start (activate) one or more units stop [NAME...] Stop (deactivate) one or more units reload [NAME...] Reload one or more units restart [NAME...] Start or restart one or more units try-restart [NAME...] Restart one or more units if active reload-or-restart [NAME...] Reload one or more units if possible, otherwise start or restart reload-or-try-restart [NAME...] Reload one or more units if possible, otherwise restart if active isolate [NAME] Start one unit and stop all others kill [NAME...] Send signal to processes of a unit is-active [NAME...] Check whether units are active is-failed [NAME...] Check whether units are failed status [NAME...|PID...] Show runtime status of one or more units show [NAME...|JOB...] Show properties of one or more units/jobs or the manager set-property [NAME] [ASSIGNMENT...] Sets one or more properties of a unit help [NAME...|PID...] Show manual for one or more units reset-failed [NAME...] Reset failed state for all, one, or more units list-dependencies [NAME] Recursively show units which are required or wanted by this unit or by which this unit is required or wanted Standard set of commands for all services
    18. 18. learning systemd http://0pointer.de/blog/projects/inetd.html (includes links to 10 predecessors)
    19. 19. Configuration Management tracking system changes i.e. ansible, cfengine, chef, puppet, saltstack
    20. 20. Configuration Management • Declarative, not imperative • Extensible • Master-client or standalone Some are less declarative than others All are extensible All can run master-client or standalone
    21. 21. Architecture master client config repo disc. agent ansible chef puppet salt repo module/ playbook recipe/ cookbook module/ repo state/pillar server ansible chef master master client N/A client agent minion discovery agent ansible facts ohai facter salt grains remote commands (built in) knife mcollective (built in) GUI Tower Chef Manage Puppet Enterprise Halite (alpha)
    22. 22. Ansible • Python-based (2.X) • Push-based (no server) • Works over SSH • Config format: YAML • Modules: Any language Emphasis on virtual infrastructure integration
    23. 23. Chef • Ruby-based • Config format: Ruby • Modules: Ruby Emphasis on developer-friendly, agile experience
    24. 24. Puppet • Ruby-based • Config format: Puppet DSL • Modules: Puppet DSL, Ruby (for extensions) Emphasis on declaration, not process
    25. 25. SaltStack • Python-based • Config format: YAML • Modules: Python/Cython Emphasis on speed, scalability
    26. 26. Installing Apache … with ansible /etc/ansible/hosts localhost ansible_connection=local local-apache.yml (playbook) --- - hosts: localhost tasks: - name: install apache apt: name=apache2 - name: ensure apache is running service: name=apache2 state=started # ansible-playbook local-apache.yml
    27. 27. # mkdir cache # chef-solo --override-runlist "recipe[apache::install]" -c config.rb Installing Apache … with chef config.rb base_dir Dir.pwd + "/" file_cache_path base_dir + "cache/" cookbook_path base_dir + "cookbooks/" cookbooks/apache/recipes/install.rb package 'apache2' do action :install end service 'apache2' do action [ :enable, :start ] end
    28. 28. # puppet apply apache-install.pp Installing Apache … with puppet apache-install.pp package {'apache2': ensure => 'installed', } ! service {'apache2': ensure => 'running', enable => true, require => Package['apache2'], }
    29. 29. Installing Apache … with saltstack /etc/salt/minion file_client: local /srv/salt/top.sls base: '*': - apache # salt-call --local state.highstate /srv/salt/top.sls apache2: pkg: - installed service: - running - require: - pkg: apache2
    30. 30. learn more ansible http://docs.ansible.com/ chef http://docs.opscode.com/ puppet http://docs.puppetlabs.com/ saltstack http://docs.saltstack.com/
    31. 31. docker container deployment and management
    32. 32. Containers • lightweight virtual machine • chroot on steroids Lighter than a VM • Same kernel • Little or no boot time • As little as one process Mightier than chroot • resource constraints (memory, CPU) • separate pids, users, groups, networking
    33. 33. Docker • layered filesystem templates • container build and run automation • expose network services
    34. 34. Docker • installation • first docker container • dockerfiles Demo: Install • yum install -y docker-io • systemctl start docker • systemctl enable docker • docker pull ubuntu:14.04 • docker images Demo: Run • docker run -i -t —name demo ubuntu:14.04 /bin/bash • dpkg -l • lsb_release -d • exit • docker ps -a • docker start demo • docker attach demo • ps awfux
    35. 35. learning docker http://docs.docker.io/introduction/working-with-docker/ https://www.stgraber.org (LXC itself)
    36. 36. Distributed Log Collection
    37. 37. Architecture client client client aggregator files database search index
    38. 38. Aggregators • Accept • Transform • Filter • Forward Accept • native formats like system logs, message queues, snmp • via client agents Transform • Add structure (json, etc) • Extract data into fields (hostname, message, time, etc) Filter • Remove or combine entries • combine multi-line • criteria matching Forward • Send logs to their destination • storage, notifications/alerts, message queues
    39. 39. What’s available? logstash
 fluentd
 flume
 http://logstash.net
 http://fluentd.org
 https://flume.apache.org/
 Logstash - Aimed at flexibility • includes tons of inputs, filters, and outputs FluentD - Aimed at robustness • built-in support for high availability, delivery assurance Flume - Part of the Hadoop ecosystem • stores data in HDFS
    40. 40. ElasticSearch + Kibana • Auto-balancing, auto-scaling search index, REST API • Pretty GUI for searching logs Works with any of these aggregators Great replacement for Splunk Missing the alerting component, but easy to work around
    41. 41. learn more http://jasonwilder.com/blog/2012/01/03/centralized-logging/
 http://jasonwilder.com/blog/2013/11/19/fluentd-vs-logstash/
    42. 42. OpenStack open source virtualization platform
    43. 43. OpenStack Capabilities • Virtual Machines • Storage (Object and Block) • Database (MySQL-compatible) • Networking • REST API
    44. 44. AWS Equivalents OpenStack AWS Compute Nova EC2 Networking Neutron Classic + VPC Database Trove RDS Storage (Block) Cinder EBS Storage (Object) Swift S3 Identity Keystone IAM Monitoring Celiometer CloudWatch Orchestration Heat CloudFormation Dashboard Horizon AWS Console Some are similar, but not direct equivalents (e.g. Keystone) ! OpenStack-based providers usually provide several other services like DNS, CDN, or message queues (Amazon does too)
    45. 45. Public OpenStack Clouds
    46. 46. The Competition • Apache CloudStack • VMware vCloud • Eucalyptus
    47. 47. Setting Up Your Own • DevStack (http://devstack.org/) • OpenStack Guides • http://docs.openstack.org/icehouse/ • Ubuntu Cloud Installer
 http://www.ubuntu.com/download/cloud/install- ubuntu-cloud DevStack is installable on a VM, even Vagrant Guides for CentOS, Debian Wheezy, Fedora, RedHat, SuSE, Ubuntu Cloud Installer requires six hosts (can be VMs)
    48. 48. learn more http://docs.openstack.org/
    49. 49. puppet chef saltstackansible openstack fluentd flume logstash docker systemd configuration management distributed logging linux containers elasticsearch kibana

    ×