"Ops Tools with Perl" 2012/05/12 Hokkaido.pm

  • 2,927 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,927
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
8
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. OpsToolswith Perl Hokkaido.pm #7 2012/05/12 riywo
  • 2. Self IntroductionRyosuke IWANAGAa.k.a. riywo(りーお)DeNA(2009∼) Mobage Server-side Ops-Engi / DBA / Manager
  • 3. AcknowledgementThanks for JPA and Hokkaido.pm :) 地域PM向け講師派遣支援プログラム http://japan.perlassociation.org/ services/send-a-monger
  • 4. But...Am I a Perl Monger?I’m not a Programer Mobage uses Perl heavily but I have never written the business logic...But, I read Perl heavily Because I should cope with all system trouble including application bugs ;PI wrote some operation tools with Perl And I also use some Perl tools
  • 5. But...Am I a Perl Monger?So...I’ll try to talk about my knowledge, thought,and tools as much as possibleしょぼくてごめんなさい><
  • 6. Perl Ops-Tools Introduction
  • 7. kazeburo-ware
  • 8. http://blog.nomadscafe.jp/ @kazeburo
  • 9. st ca re Fo d uloC http://www.slideshare.net/kazeburo/introduction-to-cloudforecast-yapcasia-2010-tokyo
  • 10. CloudForecastServer Resource Collecting/MonitoringEasy install git clone & cpanm --installdeps . SNMP / RRDs is neededLightWeight - about 5,000 lines (*.pm)WAF: Shirahata.pmSee also リソースモニタリングツール「CloudForecast」入門 - As a Futurist... ( http://blog.riywo.com/2011/02/27/043646 )
  • 11. s t ca re Fo th wroG http://blog.nomadscafe.jp/2011/12/growthforecast.html http://blog.nomadscafe.jp/2012/02/growthforecast1json.html http://d.hatena.ne.jp/tagomoris/20120218/1329558305
  • 12. GrowthForecastAPI and Graph Render of RRDToolEasy install git clone & cpanm --installdeps . RRDs is neededLightWeight - about 1,800 lines (*.pm)WAF: Kossy.pmSee also GrowthForecastというグラフ表示ツールで る話 - blog.nomadscafe.jp ( http://blog.nomadscafe.jp/2011/12/growthforecast.html )
  • 13. **Alien::RRDtool**RRDs is not CPAN module...Alien::RRDtool Install rrdtool as CPAN module extlib/lib/perl5/{archname}/RRDs.pm extlib/lib/perl5/auto/share/dist/Alien-RRDtool/Especially useful for perlbrew* Developed by gfxSee also Alien::RRDtool - RRDtoolをCPANから入れる - Perl Advent Calendar Japan 2011 Hacker Track( http://perl-users.jp/articles/advent- calendar/2011/hacker/2 )
  • 14. chase-tail
  • 15. chase-tail$ tail -f error_log | chase-tail -l 10 -t various_errorOne sheet perl script Highlight some keywords Detect fast flooding of logSee also @hirose31 垂れ流されるログのおしりを追いかける - (ひ)メモ( http:// d.hatena.ne.jp/hirose31/20120120/1327025989 )
  • 16. App::Ikachan
  • 17. App::Ikachan$ curl -F channel=#yappo -Fmessage=oppai http://hoge.com/noticeHTTP API of IRC posting use it from any languagesEasy install (git clone & cpanm) @YappoSee also YappoLogs: App::Ikachan - 様々なサーバのバッチ処理の結果等を IRC のチャンネルに通知するサーバ( http://blog.yappo.jp/yappo/ archives/000760.html )
  • 18. Percona Toolkit
  • 19. Percona ToolkitMixed tools - Maartkit and Aspersa Almost all tools are written in PerlSystem utility / MySQL Ops @Percona “pt-query-digest” is quite efficient for performance tuning of MySQL serverSee also Percona Toolkit - Percona Software( http://www.percona.com/ software/percona-toolkit/ )
  • 20. Touryo
  • 21. “Touryo” is configuration management tool I needed a Configuration Management Tool Chef is good but difficult to customize So, I started to think a plan of Touryo Simple and Lightweight Testable able to write DSL in Perl freely
  • 22. How to write DSL in Perl?I didn’t know it X(I asked gfx(@__gfx__) who is a great PerlMonger and works at DeNA
  • 23. use strict; [dsl.pl]use warnings; test {Hoge->do_test; my $hoge = shift; print "Hello $hoge!n"; };#---------------------package Hoge;my $func;sub Hoge::test(&) { my $block = shift; $func = $block;}sub do_test { package Hoge; do "dsl.pl"; $func->("DSL");}
  • 24. What can Touryo do? $ touryo test web01 web01 crontab apache FAIL? $ touryo run web01 sysctl fastcgi admin: touryo rpms appgame1:web - “touryo test host” - check the state of the host db01 db02 - correctly installed? crontab crontab - correctly configured? sysctl sysctl rpms rpms - “touryo run host” mysql - setup to the “correct” state mysql game1:db:master backup -> Then, what is the “correct” state? game1: db:backup
  • 25. How to define the “correct” state? (1) Touryo config web01 role: game1 blueprint: crontab apache - crontab sysctl fastcgi - sysctl - rpms rpms app role: game1:web role: game1:dbgame1:web blueprint: blueprint: - apache - mysql db01 - fastcgi role: game1:db:backup db02 - app blueprint: crontab crontab - backup sysctl sysctl admin: rpms rpms touryo - Touryo use “blueprint” config mysql mysql - means what should be installed game1: - The config defined by “role” backupdb:master game1: - should be defined the role of hosts db:backup - inherited searching - separated by comma
  • 26. What is the “blueprint”? crontab blueprint blueprint/crontab/root.tx web01 0 <: $cron_hour :> * * * /path/to/command crontab apache blueprint/crontab/blueprint.pl admin: touryo sysctl fastcgi bootstrap { my $c = shift; rpms app $c->add_attribute({ cron_hour => 5, game1:web cron => root.tx, });- blueprint locate the directory }; db01blueprint name - named the db02 crontab test {- contain “blueprint.pl”crontab my $c = shift; sysctl - Perl DSL for Touryo sysctl is_diff $c->sudo_ssh->crontab, - able to use templaterpms engine $c->template(cron)->content; rpms done_testing; - Text::Xslate::Syntax::Kolon }; mysql mysql - using “attribute” valuables game1: -db:master backup have other useful functions run { game1: my $c = shift; - sudo_ssh, template, crontab db:backup $c->sudo_ssh->crontab($c->template(cron)); - etc... };
  • 27. What is the “attribute”? blueprint/crontab/root.tx web01 0 <: $cron_hour :> * * * /path/to/command admin: crontab apache blueprint/crontab/blueprint.pl touryo bootstrap { sysctl fastcgi my $c = shift; $c->add_attribute({ rpms app cron_hour => 5, game1:web cron => root.tx, }); };- “attribute” can be used to db01 db02change the behavior of test/run default rendering crontab crontab 0 5 * * * /path/to/command - changing logic, template, etc.. sysctl sysctl- overwriting of “attribute” If defined attribute on - bootstrap rpms rpms the Touryo Config ... - role config Touryo Config mysql mysql role: game1:web - defined for the role game1: backup attribute: -db:masterconfig node cron_hour: 9 game1: - defined for the node(host) db:backup 0 9 * * * /path/to/command
  • 28. How to define the “correct” state? (2) Touryo config web01 role: game1 crontab apache blueprint: attribute: sysctl fastcgi - crontab somaxconn: 1024 - sysctl rpms app - rpmsgame1:web cron_hour: 9 somaxconn: 10240 role: game1:web fcgi_proc: 100 blueprint: attribute: - apache cron_hour: 9 - fastcgi somaxconn: 10240 web02 - app fcgi_proc: 100 crontab apache sysctl fastcgi node: web02 rpms app blueprint: attribute: admin: - git fcgi_proc: 10game1:web git touryo cron_hour: 9 - You can vary correct “blueprint” and somaxconn: 10240 “attribute” by Touryo config fcgi_proc: 10 - role config and node config
  • 29. Lightweight/Testable/Freedomfind lib/ -name "*.pm" | xargs cat | wc -l about 1500 (including html templates)Like test scripts of Perl Software, you cantest the servers’ configurationyou can write anything on blueprint.pl Conversely you have to write anything
  • 30. Other Featuresin terms of Programming use Amon2 easy to develop Web Interface and CLI use TAP::Parser to get the test result test function uses Test::More use IPC::Cmd and “ssh” / “sudo ssh” easy to configure with .ssh/config or others blueprint dependance resolving easy to cascade the config
  • 31. What languageshould I use?
  • 32. There are many languagesscript / compile Perl,Ruby,Python,PHP / C,C++,Javaprocedural / OO / functional Perl,C / Java,Ruby / Scala,Haskellpopular / minority ???(religious problem...)
  • 33. My ThoughtYou can use any languages you like Engineer have to treat multi languagesDevelopment speed / Maintenance cost They are usually trade-off Popular LLs are often suited for both Simple / Light / Thin / Useful Modules Everyone can understand it easily Perl/Ruby/Python are similar
  • 34. I ♡ PerlFastest to develop for me I am used to Perl because DeNA use Perl There are many Perl Mongers around :) colleague and friendsCPAN has many useful modules long history and super hackers
  • 35. Finally...
  • 36. DeNA ♡ EngineersWe need more powerful engineersWe use Perl and other languages Ruby/JavaScript/C/C++/Java/Objective-C SocialGame/Platform/SDK/Data miningOr let’s make very useful tools with me :) Server Ops/Database Admin/NetworkIf you are interested in us, pleeeeeeeeeasecontact me or DeNA staffs! hirose31, gfx, hidek, zigorou, nekokak...
  • 37. Thanks! http://blog.riywo.com http://twitter.com/riywoThanks for JPA, @kazeburo, @hirose31, @Yappo, @Percona, and All Perl Mongers!