Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to cloudforecast

3,416 views

Published on

Introduction to cloudforecast

Published in: Technology
  • Be the first to comment

Introduction to cloudforecast

  1. 1. Introduction to CloudForecast Yokohama.pm #6.x @kazeburo
  2. 2. • • @kazeburo • 2010 5 • 2010 6 • Web
  3. 3. CloudForecast • • • perl rrdtool sysadmin (by tokuhirom)
  4. 4. • •
  5. 5. (1)
  6. 6. (1) • ping
  7. 7. (1) • ping • HTTP,SSH
  8. 8. (1) • ping • HTTP,SSH • Disk
  9. 9. (2) • • • • Nagios • monit
  10. 10. (1)
  11. 11. (1) •
  12. 12. (1) • • CPU
  13. 13. (2) • • • • Cacti • munin • Ganglia • CloudForecast
  14. 14. CloudForecast • • YAML • Web • Perl!
  15. 15. CPAN • Plack + Starlet • Gearman • Parallel::Prefork • DBI DBD::SQLite • YAML::Syck • Text::Xslate • Data::Section::Simple • local::lib
  16. 16. • RRDTool RRDs.pm • SNMP.pm • Shirahata(WAF)
  17. 17. CloudForecast 1 RRD Radar Web SQLite
  18. 18. CloudForecast 1 (1) RRD Radar Web SQLite
  19. 19. CloudForecast 1 (1) RRD Radar Web SQLite (2)
  20. 20. CloudForecast 1 (1) RRD Radar Web SQLite (2) (3)
  21. 21. CloudForecast 1 (1) RRD Radar Web SQLite (2) (3)
  22. 22. CloudForecast 2 fetcher worker RRD update Radar Web worker SQLite
  23. 23. CloudForecast 2 Gearman Worker fetcher worker RRD update Radar Web worker SQLite
  24. 24. CloudForecast 2 Gearman Worker RRD update Radar Web worker SQLite
  25. 25. 1 # ubuntu $ sudo apt-get install librrds-perl libsnmp-perl # CentOS rrdtool EPEL $ sudo yum install net-snmp-perl $ sudo yum install rrdtool-perl
  26. 26. 2 $ git clone git://github.com/kazeburo/cloudforecast.git $ cd cloudforecast $ cpanm -l extlib --installdeps . $ ls Changes cloudforecast_web Makefile.PL docs README host_config cf_devstarter htdocs cf_fetcher_worker lib cf_updater_worker server_list_sample.yaml cloudforecast_radar t cloudforecast_sample.yaml
  27. 27. # $ cp cloudforecast_sample.yaml cloudforecast.yaml # $ cp server_list_sample.yaml server_list.yaml
  28. 28. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers servers: - config: basic.yaml hosts: ....
  29. 29. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers servers: - config: basic.yaml hosts: ....
  30. 30. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers servers: - config: basic.yaml hosts: ....
  31. 31. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers IP servers: - config: basic.yaml hosts: ....
  32. 32. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers IP servers: - config: basic.yaml hosts: ....
  33. 33. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers IP servers: - config: basic.yaml hosts: ....
  34. 34. (basic.yaml) --- component_config: resources: - basic - traffic:eth0 basic = eth0 CPU TCP
  35. 35. (basic.yaml) --- component_config: resources: - basic - traffic:eth0 basic = eth0 CPU TCP
  36. 36. (basic.yaml) --- component_config: resources: - basic - traffic:eth0 basic = eth0 CPU TCP
  37. 37. • Basic(CPU TCP ) • Traffic • Disk Usage • DiskIO Count • Apache Status • Nginx Status • Squid • Mysql • Innodb
  38. 38. $ perldoc CloudForecast::Data::
  39. 39. $ perldoc CloudForecast::Data::
  40. 40. CloudForecast http://blog.nomadscafe.jp/2010/07/cloudforecast-1.html
  41. 41. # $ ./cloudforecast_radar -c cloudforecast.yaml -l server_list.yaml # Web $ ./cloudforecast_web -p 5000 -c cloudforecast.yaml -l server_list.yaml
  42. 42. Web ( )
  43. 43. Web ( )
  44. 44. http://blog.nomadscafe.jp/2010/07/cloudforecast-2.html
  45. 45.
  46. 46. • •
  47. 47. • • •
  48. 48. # site-lib $ mkdir -p site-lib/CloudForecast/Data
  49. 49. Gearman Worker gearman worker process http://blog.nomadscafe.jp/2010/07/gearman-worker-process.html
  50. 50. % ./gearman-starter.pl --max-prcess 8 -s 127.0.0.1:7004 --scoreboard-dir /var/run/worker --port 7005 MyWorker % telnet localhost 7005 Trying 127.0.0.1... System: gearman_servers: 127.0.0.1:7004 class: MyWorker BusyWorkers: 0 IdleWorkers: 8 -- pid Status Counter Comment 1630 . 0 1631 . 0 1632 . 0 1633 . 0 1634 . 0 1635 . 0 1636 . 0 1637 . 0
  51. 51. % ./gearman-starter.pl --max-prcess 8 -s 127.0.0.1:7004 --scoreboard-dir /var/run/worker --port 7005 MyWorker % telnet localhost 7005 Trying 127.0.0.1... System: gearman_servers: 127.0.0.1:7004 class: MyWorker BusyWorkers: 0 IdleWorkers: 8 -- pid Status Counter Comment 1630 . 0 1631 . 0 1632 . 0 1633 . 0 1634 . 0 1635 . 0 1636 . 0 1637 . 0
  52. 52. • •
  53. 53. package CloudForecast::Data::Gearmanstarter; use CloudForecast::Data -base; use IO::Socket::INET; =pod host_config) resources: - gearmanstarter[:port] eg) - gearmanstarter # 9000 - gearmanstarter:9005 # port =cut
  54. 54. rrds map { [$_,'GAUGE'] } qw/busy idle/; # rrds ‘key’, ‘TYPE’ or rrds [‘key’,‘TYPE’], []..
  55. 55. • COUNTER • • • ) Traffic ( octet ) • GAUGE • • • )
  56. 56. (1) graphs 'status' => 'worker status'; # graphs ‘key1’ => ‘label’; # graphs ‘key2’ => ‘the great graph’;
  57. 57. (2) graphs 'status' => 'worker status'; .. __DATA__ @@ status DEF:my1=<%RRD%>:busy:AVERAGE DEF:my2=<%RRD%>:idle:AVERAGE AREA:my1#00C000:Busy GPRINT:my1:LAST:Cur: %4.1lf GPRINT:my1:AVERAGE:Ave: %4.1lf GPRINT:my1:MAX:Max: %4.1lf GPRINT:my1:MIN:Min: %4.1lfc STACK:my2#0000C0:Idle GPRINT:my2:LAST:Cur: %4.1lf GPRINT:my2:AVERAGE:Ave: %4.1lf GPRINT:my2:MAX:Max: %4.1lf GPRINT:my2:MIN:Min: %4.1lfc
  58. 58. (2) graphs 'status' => 'worker status'; .. key __DATA__ @@ status DEF:my1=<%RRD%>:busy:AVERAGE DEF:my2=<%RRD%>:idle:AVERAGE AREA:my1#00C000:Busy GPRINT:my1:LAST:Cur: %4.1lf GPRINT:my1:AVERAGE:Ave: %4.1lf GPRINT:my1:MAX:Max: %4.1lf GPRINT:my1:MIN:Min: %4.1lfc STACK:my2#0000C0:Idle GPRINT:my2:LAST:Cur: %4.1lf GPRINT:my2:AVERAGE:Ave: %4.1lf GPRINT:my2:MAX:Max: %4.1lf GPRINT:my2:MIN:Min: %4.1lfc
  59. 59. fetcher { my $c = shift; my $host = $c->address; my $port = $c->args->[0] || 9000; my $sock = IO::Socket::INET->new( PeerAddr => $host, PeerPort => $port, Proto => 'tcp', ); my $raw_stats; $sock->sysread( $raw_stats, 8192 ); my ($busy, $idle); foreach my $line ( split /[rn]+/, $raw_stats ) { if ( $line =~ /^Busy.+: (d+)/ ) { $busy = $1; } if ( $line =~ /^Idle.+: (d+)/ ) { $idle = $1; } } return [$busy,$idle]; };
  60. 60. fetcher { my $c = shift; my $host = $c->address; my $port = $c->args->[0] || 9000; my $sock = IO::Socket::INET->new( PeerAddr => $host, PeerPort => $port, Proto => 'tcp', ); my $raw_stats; $sock->sysread( $raw_stats, 8192 ); my ($busy, $idle); foreach my $line ( split /[rn]+/, $raw_stats ) { if ( $line =~ /^Busy.+: (d+)/ ) { $busy = $1; } if ( $line =~ /^Idle.+: (d+)/ ) { $idle = $1; } } return [$busy,$idle]; };
  61. 61.
  62. 62. IDLE BUSY
  63. 63. TODO/Plan • Web • RRD SQLite •
  64. 64. • CloudForecast Perl • •

×