"Ops Tools with Perl" 2012/05/12 Hokkaido.pm
Upcoming SlideShare
Loading in...5

"Ops Tools with Perl" 2012/05/12 Hokkaido.pm






Total Views
Views on SlideShare
Embed Views



6 Embeds 1,187

http://hokkaido.pm.org 870
http://blog.riywo.com 309
http://www.perl-hokkaido.org 3
http://a0.twimg.com 2
http://webcache.googleusercontent.com 2
https://si0.twimg.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

"Ops Tools with Perl" 2012/05/12 Hokkaido.pm "Ops Tools with Perl" 2012/05/12 Hokkaido.pm Presentation Transcript

  • OpsToolswith Perl Hokkaido.pm #7 2012/05/12 riywo
  • Self IntroductionRyosuke IWANAGAa.k.a. riywo(りーお)DeNA(2009∼) Mobage Server-side Ops-Engi / DBA / Manager
  • AcknowledgementThanks for JPA and Hokkaido.pm :) 地域PM向け講師派遣支援プログラム http://japan.perlassociation.org/ services/send-a-monger
  • But...Am I a Perl Monger?I’m not a Programer Mobage uses Perl heavily but I have never written the business logic...But, I read Perl heavily Because I should cope with all system trouble including application bugs ;PI wrote some operation tools with Perl And I also use some Perl tools
  • But...Am I a Perl Monger?So...I’ll try to talk about my knowledge, thought,and tools as much as possibleしょぼくてごめんなさい><
  • Perl Ops-Tools Introduction
  • kazeburo-ware
  • http://blog.nomadscafe.jp/ @kazeburo
  • st ca re Fo d uloC http://www.slideshare.net/kazeburo/introduction-to-cloudforecast-yapcasia-2010-tokyo
  • CloudForecastServer Resource Collecting/MonitoringEasy install git clone & cpanm --installdeps . SNMP / RRDs is neededLightWeight - about 5,000 lines (*.pm)WAF: Shirahata.pmSee also リソースモニタリングツール「CloudForecast」入門 - As a Futurist... ( http://blog.riywo.com/2011/02/27/043646 )
  • s t ca re Fo th wroG http://blog.nomadscafe.jp/2011/12/growthforecast.html http://blog.nomadscafe.jp/2012/02/growthforecast1json.html http://d.hatena.ne.jp/tagomoris/20120218/1329558305
  • GrowthForecastAPI and Graph Render of RRDToolEasy install git clone & cpanm --installdeps . RRDs is neededLightWeight - about 1,800 lines (*.pm)WAF: Kossy.pmSee also GrowthForecastというグラフ表示ツールで る話 - blog.nomadscafe.jp ( http://blog.nomadscafe.jp/2011/12/growthforecast.html )
  • **Alien::RRDtool**RRDs is not CPAN module...Alien::RRDtool Install rrdtool as CPAN module extlib/lib/perl5/{archname}/RRDs.pm extlib/lib/perl5/auto/share/dist/Alien-RRDtool/Especially useful for perlbrew* Developed by gfxSee also Alien::RRDtool - RRDtoolをCPANから入れる - Perl Advent Calendar Japan 2011 Hacker Track( http://perl-users.jp/articles/advent- calendar/2011/hacker/2 )
  • chase-tail
  • chase-tail$ tail -f error_log | chase-tail -l 10 -t various_errorOne sheet perl script Highlight some keywords Detect fast flooding of logSee also @hirose31 垂れ流されるログのおしりを追いかける - (ひ)メモ( http:// d.hatena.ne.jp/hirose31/20120120/1327025989 )
  • App::Ikachan
  • App::Ikachan$ curl -F channel=#yappo -Fmessage=oppai http://hoge.com/noticeHTTP API of IRC posting use it from any languagesEasy install (git clone & cpanm) @YappoSee also YappoLogs: App::Ikachan - 様々なサーバのバッチ処理の結果等を IRC のチャンネルに通知するサーバ( http://blog.yappo.jp/yappo/ archives/000760.html )
  • Percona Toolkit
  • Percona ToolkitMixed tools - Maartkit and Aspersa Almost all tools are written in PerlSystem utility / MySQL Ops @Percona “pt-query-digest” is quite efficient for performance tuning of MySQL serverSee also Percona Toolkit - Percona Software( http://www.percona.com/ software/percona-toolkit/ )
  • Touryo
  • “Touryo” is configuration management tool I needed a Configuration Management Tool Chef is good but difficult to customize So, I started to think a plan of Touryo Simple and Lightweight Testable able to write DSL in Perl freely
  • How to write DSL in Perl?I didn’t know it X(I asked gfx(@__gfx__) who is a great PerlMonger and works at DeNA
  • use strict; [dsl.pl]use warnings; test {Hoge->do_test; my $hoge = shift; print "Hello $hoge!n"; };#---------------------package Hoge;my $func;sub Hoge::test(&) { my $block = shift; $func = $block;}sub do_test { package Hoge; do "dsl.pl"; $func->("DSL");}
  • What can Touryo do? $ touryo test web01 web01 crontab apache FAIL? $ touryo run web01 sysctl fastcgi admin: touryo rpms appgame1:web - “touryo test host” - check the state of the host db01 db02 - correctly installed? crontab crontab - correctly configured? sysctl sysctl rpms rpms - “touryo run host” mysql - setup to the “correct” state mysql game1:db:master backup -> Then, what is the “correct” state? game1: db:backup
  • How to define the “correct” state? (1) Touryo config web01 role: game1 blueprint: crontab apache - crontab sysctl fastcgi - sysctl - rpms rpms app role: game1:web role: game1:dbgame1:web blueprint: blueprint: - apache - mysql db01 - fastcgi role: game1:db:backup db02 - app blueprint: crontab crontab - backup sysctl sysctl admin: rpms rpms touryo - Touryo use “blueprint” config mysql mysql - means what should be installed game1: - The config defined by “role” backupdb:master game1: - should be defined the role of hosts db:backup - inherited searching - separated by comma
  • What is the “blueprint”? crontab blueprint blueprint/crontab/root.tx web01 0 <: $cron_hour :> * * * /path/to/command crontab apache blueprint/crontab/blueprint.pl admin: touryo sysctl fastcgi bootstrap { my $c = shift; rpms app $c->add_attribute({ cron_hour => 5, game1:web cron => root.tx, });- blueprint locate the directory }; db01blueprint name - named the db02 crontab test {- contain “blueprint.pl”crontab my $c = shift; sysctl - Perl DSL for Touryo sysctl is_diff $c->sudo_ssh->crontab, - able to use templaterpms engine $c->template(cron)->content; rpms done_testing; - Text::Xslate::Syntax::Kolon }; mysql mysql - using “attribute” valuables game1: -db:master backup have other useful functions run { game1: my $c = shift; - sudo_ssh, template, crontab db:backup $c->sudo_ssh->crontab($c->template(cron)); - etc... };
  • What is the “attribute”? blueprint/crontab/root.tx web01 0 <: $cron_hour :> * * * /path/to/command admin: crontab apache blueprint/crontab/blueprint.pl touryo bootstrap { sysctl fastcgi my $c = shift; $c->add_attribute({ rpms app cron_hour => 5, game1:web cron => root.tx, }); };- “attribute” can be used to db01 db02change the behavior of test/run default rendering crontab crontab 0 5 * * * /path/to/command - changing logic, template, etc.. sysctl sysctl- overwriting of “attribute” If defined attribute on - bootstrap rpms rpms the Touryo Config ... - role config Touryo Config mysql mysql role: game1:web - defined for the role game1: backup attribute: -db:masterconfig node cron_hour: 9 game1: - defined for the node(host) db:backup 0 9 * * * /path/to/command
  • How to define the “correct” state? (2) Touryo config web01 role: game1 crontab apache blueprint: attribute: sysctl fastcgi - crontab somaxconn: 1024 - sysctl rpms app - rpmsgame1:web cron_hour: 9 somaxconn: 10240 role: game1:web fcgi_proc: 100 blueprint: attribute: - apache cron_hour: 9 - fastcgi somaxconn: 10240 web02 - app fcgi_proc: 100 crontab apache sysctl fastcgi node: web02 rpms app blueprint: attribute: admin: - git fcgi_proc: 10game1:web git touryo cron_hour: 9 - You can vary correct “blueprint” and somaxconn: 10240 “attribute” by Touryo config fcgi_proc: 10 - role config and node config
  • Lightweight/Testable/Freedomfind lib/ -name "*.pm" | xargs cat | wc -l about 1500 (including html templates)Like test scripts of Perl Software, you cantest the servers’ configurationyou can write anything on blueprint.pl Conversely you have to write anything
  • Other Featuresin terms of Programming use Amon2 easy to develop Web Interface and CLI use TAP::Parser to get the test result test function uses Test::More use IPC::Cmd and “ssh” / “sudo ssh” easy to configure with .ssh/config or others blueprint dependance resolving easy to cascade the config
  • What languageshould I use?
  • There are many languagesscript / compile Perl,Ruby,Python,PHP / C,C++,Javaprocedural / OO / functional Perl,C / Java,Ruby / Scala,Haskellpopular / minority ???(religious problem...)
  • My ThoughtYou can use any languages you like Engineer have to treat multi languagesDevelopment speed / Maintenance cost They are usually trade-off Popular LLs are often suited for both Simple / Light / Thin / Useful Modules Everyone can understand it easily Perl/Ruby/Python are similar
  • I ♡ PerlFastest to develop for me I am used to Perl because DeNA use Perl There are many Perl Mongers around :) colleague and friendsCPAN has many useful modules long history and super hackers
  • Finally...
  • DeNA ♡ EngineersWe need more powerful engineersWe use Perl and other languages Ruby/JavaScript/C/C++/Java/Objective-C SocialGame/Platform/SDK/Data miningOr let’s make very useful tools with me :) Server Ops/Database Admin/NetworkIf you are interested in us, pleeeeeeeeeasecontact me or DeNA staffs! hirose31, gfx, hidek, zigorou, nekokak...
  • Thanks! http://blog.riywo.com http://twitter.com/riywoThanks for JPA, @kazeburo, @hirose31, @Yappo, @Percona, and All Perl Mongers!