“Debugging is on the table”
Dr. House pergunta a um Sysadmin
Leonardo Martins Abril/2016
POR ONDE COMEÇAR?
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
Leonardo Martins
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
Leonardo Martins
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
http://i3.cpcache.com/product/228415899/rtfm_mug.jpg
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
Hipótese
confirmada?
Problema
resolvido!!
SIM
NÃO
Coletar
informações
Definir
problema
Construir
hipótese
Testar
hipótese
Construir
nova hipótese
Guardar os
dados coletados
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
Hipótese
confirmada?
Problema
resolvido!!
Coletar
informações
Definir
problema
Construir
hipótese
Testar
hipótese
Construir
nova hipótese
Guardar os
dados coletados
SIM
NÃO
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
DEFINIÇÃO DE PROBLEMA
O que te faz pensar que existe um problema?
O que causou esse problema?
O que foi mudado recentemente?
Isso já funcionou alguma vez ?
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÕES
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÃO
# apt-get install htop
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÕES
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÕES
# apt-get install sysstat
# apt-get install dstat
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
- tcpdump -i eth0 -T snmp "(src port 161 or 162)"
- tcpdump -i eth0 'udp port 53'
- tcpdump "tcp[tcpflags] & (tcp-syn|tcp-ack) != 0"
- tcpdump -i eth0 'tcp port 22' -w saida.pcap
- tshark -r saida.pcap
COLETA DE INFORMAÇÕES
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
- tcpdump -i eth0 -T snmp "(src port 161 or 162)"
- tcpdump -i eth0 'udp port 53'
- tcpdump "tcp[tcpflags] & (tcp-syn|tcp-ack) != 0"
- tcpdump -i eth0 'tcp port 22' -w saida.pcap
- tshark -r saida.pcap
COLETA DE INFORMAÇÕES
# apt-get install tshark
# apt-get install tcpdump
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÕES
- strace ./get_http.py
- strace -e trace=open,close ./get_http.py
- strace -f -e trace=network ./test_tcp.sh
# apt-get install strace
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÕES
- lsof /var/log/nginx/access.log
- lsof -u $USER
- lsof -p PID
- lsof -i |grep LISTEN
- lsof -i :80
# apt-get install lsof
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
COLETA DE INFORMAÇÕES
- sysdig -c topprocs_cpu
- sysdig -c topscalls_time
- sysdig -c topfiles_time proc.name=nginx
# apt-get install sysdig
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
ANÁLISE DAS INFORMAÇÕES
http://www.zastavki.com/pictures/1600x1200/2008/Movies_Movies_H_House_M.D._010043_.jpg
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
Hipótese
confirmada?
Problema
resolvido!!
SIM
NÃO
Coletar
informações
Definir
problema
Construir
hipótese
Testar
hipótese
Construir
nova hipótese
Guardar os
dados coletados
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
TESTAR HIPÓTESES
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
TOP DOWN
http://computernetworkhomeworkhelp.com/wp-content/uploads/2014/01/comparision.png
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
BOTTOM UP
http://computernetworkhomeworkhelp.com/wp-content/uploads/2014/01/comparision.png
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
http://computernetworkhomeworkhelp.com/wp-content/uploads/2014/01/comparision.png
DIVIDIR E CONQUISTAR
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
CONSTRUIR NOVAS HIPÓTESES
?
?
?
?
?
?
?
?
http://watchesinmovies.info/img/f/House-Pathfinder.jpg
?
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
TOP DOWN
https://d0.awsstatic.com/architecture-diagrams/customers/parse-arch-diagram.png
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
BOTTOM UP
https://d0.awsstatic.com/architecture-diagrams/customers/parse-arch-diagram.png
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
FOLLOW THE REQUEST
https://d0.awsstatic.com/architecture-diagrams/customers/parse-arch-diagram.png
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
DRILL DOWN
https://d0.awsstatic.com/architecture-diagrams/customers/parse-arch-diagram.png
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
LOGS
https://www.graylog.org/overview
- Graylog
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
LOGS
https://www.graylog.org/overview
- Graylog
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
MONITORAÇÃO
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
MONITORAÇÃO
- CollectD + InfluxDB + Grafana
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
AUTOMAÇÃO
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
“Everybody lies” by Dr. House
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
http://www.brendangregg.com/Perf/linux_observability_tools.png
REFERÊNCIAS
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
REFERÊNCIAS
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
- http://www.zabbix.com
- https://www.graylog.org
- https://collectd.org
- https://influxdata.com
- http://grafana.org
- https://www.chef.io
- https://puppet.com
- https://www.ansible.com
- http://saltstack.com
https://br.linkedin.com/in/leonardoml Leonardo Martins - leonardoml@gmail.com
REFERÊNCIAS
OBRIGADO!!
leonardoml@gmail.com
https://br.linkedin.com/in/leonardoml
http://images.fanpop.com/images/image_uploads/House-MD-house-md-630240_1360_768.jpg

“Debugging is on the table” Dr. House pergunta a um Sysadmin