Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Ops Tools with Perl" 2012/05/12


Published on

Published in: Technology
  • Be the first to comment

"Ops Tools with Perl" 2012/05/12

  1. 1. OpsToolswith Perl #7 2012/05/12 riywo
  2. 2. Self IntroductionRyosuke IWANAGAa.k.a. riywo(りーお)DeNA(2009∼) Mobage Server-side Ops-Engi / DBA / Manager
  3. 3. AcknowledgementThanks for JPA and :) 地域PM向け講師派遣支援プログラム services/send-a-monger
  4. 4. But...Am I a Perl Monger?I’m not a Programer Mobage uses Perl heavily but I have never written the business logic...But, I read Perl heavily Because I should cope with all system trouble including application bugs ;PI wrote some operation tools with Perl And I also use some Perl tools
  5. 5. But...Am I a Perl Monger?So...I’ll try to talk about my knowledge, thought,and tools as much as possibleしょぼくてごめんなさい><
  6. 6. Perl Ops-Tools Introduction
  7. 7. kazeburo-ware
  8. 8. @kazeburo
  9. 9. st ca re Fo d uloC
  10. 10. CloudForecastServer Resource Collecting/MonitoringEasy install git clone & cpanm --installdeps . SNMP / RRDs is neededLightWeight - about 5,000 lines (*.pm)WAF: Shirahata.pmSee also リソースモニタリングツール「CloudForecast」入門 - As a Futurist... ( )
  11. 11. s t ca re Fo th wroG
  12. 12. GrowthForecastAPI and Graph Render of RRDToolEasy install git clone & cpanm --installdeps . RRDs is neededLightWeight - about 1,800 lines (*.pm)WAF: Kossy.pmSee also GrowthForecastというグラフ表示ツールで る話 - ( )
  13. 13. **Alien::RRDtool**RRDs is not CPAN module...Alien::RRDtool Install rrdtool as CPAN module extlib/lib/perl5/{archname}/ extlib/lib/perl5/auto/share/dist/Alien-RRDtool/Especially useful for perlbrew* Developed by gfxSee also Alien::RRDtool - RRDtoolをCPANから入れる - Perl Advent Calendar Japan 2011 Hacker Track( calendar/2011/hacker/2 )
  14. 14. chase-tail
  15. 15. chase-tail$ tail -f error_log | chase-tail -l 10 -t various_errorOne sheet perl script Highlight some keywords Detect fast flooding of logSee also @hirose31 垂れ流されるログのおしりを追いかける - (ひ)メモ( http:// )
  16. 16. App::Ikachan
  17. 17. App::Ikachan$ curl -F channel=#yappo -Fmessage=oppai API of IRC posting use it from any languagesEasy install (git clone & cpanm) @YappoSee also YappoLogs: App::Ikachan - 様々なサーバのバッチ処理の結果等を IRC のチャンネルに通知するサーバ( archives/000760.html )
  18. 18. Percona Toolkit
  19. 19. Percona ToolkitMixed tools - Maartkit and Aspersa Almost all tools are written in PerlSystem utility / MySQL Ops @Percona “pt-query-digest” is quite efficient for performance tuning of MySQL serverSee also Percona Toolkit - Percona Software( software/percona-toolkit/ )
  20. 20. Touryo
  21. 21. “Touryo” is configuration management tool I needed a Configuration Management Tool Chef is good but difficult to customize So, I started to think a plan of Touryo Simple and Lightweight Testable able to write DSL in Perl freely
  22. 22. How to write DSL in Perl?I didn’t know it X(I asked gfx(@__gfx__) who is a great PerlMonger and works at DeNA
  23. 23. use strict; []use warnings; test {Hoge->do_test; my $hoge = shift; print "Hello $hoge!n"; };#---------------------package Hoge;my $func;sub Hoge::test(&) { my $block = shift; $func = $block;}sub do_test { package Hoge; do ""; $func->("DSL");}
  24. 24. What can Touryo do? $ touryo test web01 web01 crontab apache FAIL? $ touryo run web01 sysctl fastcgi admin: touryo rpms appgame1:web - “touryo test host” - check the state of the host db01 db02 - correctly installed? crontab crontab - correctly configured? sysctl sysctl rpms rpms - “touryo run host” mysql - setup to the “correct” state mysql game1:db:master backup -> Then, what is the “correct” state? game1: db:backup
  25. 25. How to define the “correct” state? (1) Touryo config web01 role: game1 blueprint: crontab apache - crontab sysctl fastcgi - sysctl - rpms rpms app role: game1:web role: game1:dbgame1:web blueprint: blueprint: - apache - mysql db01 - fastcgi role: game1:db:backup db02 - app blueprint: crontab crontab - backup sysctl sysctl admin: rpms rpms touryo - Touryo use “blueprint” config mysql mysql - means what should be installed game1: - The config defined by “role” backupdb:master game1: - should be defined the role of hosts db:backup - inherited searching - separated by comma
  26. 26. What is the “blueprint”? crontab blueprint blueprint/crontab/root.tx web01 0 <: $cron_hour :> * * * /path/to/command crontab apache blueprint/crontab/ admin: touryo sysctl fastcgi bootstrap { my $c = shift; rpms app $c->add_attribute({ cron_hour => 5, game1:web cron => root.tx, });- blueprint locate the directory }; db01blueprint name - named the db02 crontab test {- contain “”crontab my $c = shift; sysctl - Perl DSL for Touryo sysctl is_diff $c->sudo_ssh->crontab, - able to use templaterpms engine $c->template(cron)->content; rpms done_testing; - Text::Xslate::Syntax::Kolon }; mysql mysql - using “attribute” valuables game1: -db:master backup have other useful functions run { game1: my $c = shift; - sudo_ssh, template, crontab db:backup $c->sudo_ssh->crontab($c->template(cron)); - etc... };
  27. 27. What is the “attribute”? blueprint/crontab/root.tx web01 0 <: $cron_hour :> * * * /path/to/command admin: crontab apache blueprint/crontab/ touryo bootstrap { sysctl fastcgi my $c = shift; $c->add_attribute({ rpms app cron_hour => 5, game1:web cron => root.tx, }); };- “attribute” can be used to db01 db02change the behavior of test/run default rendering crontab crontab 0 5 * * * /path/to/command - changing logic, template, etc.. sysctl sysctl- overwriting of “attribute” If defined attribute on - bootstrap rpms rpms the Touryo Config ... - role config Touryo Config mysql mysql role: game1:web - defined for the role game1: backup attribute: -db:masterconfig node cron_hour: 9 game1: - defined for the node(host) db:backup 0 9 * * * /path/to/command
  28. 28. How to define the “correct” state? (2) Touryo config web01 role: game1 crontab apache blueprint: attribute: sysctl fastcgi - crontab somaxconn: 1024 - sysctl rpms app - rpmsgame1:web cron_hour: 9 somaxconn: 10240 role: game1:web fcgi_proc: 100 blueprint: attribute: - apache cron_hour: 9 - fastcgi somaxconn: 10240 web02 - app fcgi_proc: 100 crontab apache sysctl fastcgi node: web02 rpms app blueprint: attribute: admin: - git fcgi_proc: 10game1:web git touryo cron_hour: 9 - You can vary correct “blueprint” and somaxconn: 10240 “attribute” by Touryo config fcgi_proc: 10 - role config and node config
  29. 29. Lightweight/Testable/Freedomfind lib/ -name "*.pm" | xargs cat | wc -l about 1500 (including html templates)Like test scripts of Perl Software, you cantest the servers’ configurationyou can write anything on Conversely you have to write anything
  30. 30. Other Featuresin terms of Programming use Amon2 easy to develop Web Interface and CLI use TAP::Parser to get the test result test function uses Test::More use IPC::Cmd and “ssh” / “sudo ssh” easy to configure with .ssh/config or others blueprint dependance resolving easy to cascade the config
  31. 31. What languageshould I use?
  32. 32. There are many languagesscript / compile Perl,Ruby,Python,PHP / C,C++,Javaprocedural / OO / functional Perl,C / Java,Ruby / Scala,Haskellpopular / minority ???(religious problem...)
  33. 33. My ThoughtYou can use any languages you like Engineer have to treat multi languagesDevelopment speed / Maintenance cost They are usually trade-off Popular LLs are often suited for both Simple / Light / Thin / Useful Modules Everyone can understand it easily Perl/Ruby/Python are similar
  34. 34. I ♡ PerlFastest to develop for me I am used to Perl because DeNA use Perl There are many Perl Mongers around :) colleague and friendsCPAN has many useful modules long history and super hackers
  35. 35. Finally...
  36. 36. DeNA ♡ EngineersWe need more powerful engineersWe use Perl and other languages Ruby/JavaScript/C/C++/Java/Objective-C SocialGame/Platform/SDK/Data miningOr let’s make very useful tools with me :) Server Ops/Database Admin/NetworkIf you are interested in us, pleeeeeeeeeasecontact me or DeNA staffs! hirose31, gfx, hidek, zigorou, nekokak...
  37. 37. Thanks! for JPA, @kazeburo, @hirose31, @Yappo, @Percona, and All Perl Mongers!