Introduction to cloudforecast

3,354 views

Published on

Introduction to cloudforecast

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,354
On SlideShare
0
From Embeds
0
Number of Embeds
317
Actions
Shares
0
Downloads
17
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide





































































  • Introduction to cloudforecast

    1. 1. Introduction to CloudForecast Yokohama.pm #6.x @kazeburo
    2. 2. • • @kazeburo • 2010 5 • 2010 6 • Web
    3. 3. CloudForecast • • • perl rrdtool sysadmin (by tokuhirom)
    4. 4. • •
    5. 5. (1)
    6. 6. (1) • ping
    7. 7. (1) • ping • HTTP,SSH
    8. 8. (1) • ping • HTTP,SSH • Disk
    9. 9. (2) • • • • Nagios • monit
    10. 10. (1)
    11. 11. (1) •
    12. 12. (1) • • CPU
    13. 13. (2) • • • • Cacti • munin • Ganglia • CloudForecast
    14. 14. CloudForecast • • YAML • Web • Perl!
    15. 15. CPAN • Plack + Starlet • Gearman • Parallel::Prefork • DBI DBD::SQLite • YAML::Syck • Text::Xslate • Data::Section::Simple • local::lib
    16. 16. • RRDTool RRDs.pm • SNMP.pm • Shirahata(WAF)
    17. 17. CloudForecast 1 RRD Radar Web SQLite
    18. 18. CloudForecast 1 (1) RRD Radar Web SQLite
    19. 19. CloudForecast 1 (1) RRD Radar Web SQLite (2)
    20. 20. CloudForecast 1 (1) RRD Radar Web SQLite (2) (3)
    21. 21. CloudForecast 1 (1) RRD Radar Web SQLite (2) (3)
    22. 22. CloudForecast 2 fetcher worker RRD update Radar Web worker SQLite
    23. 23. CloudForecast 2 Gearman Worker fetcher worker RRD update Radar Web worker SQLite
    24. 24. CloudForecast 2 Gearman Worker RRD update Radar Web worker SQLite
    25. 25. 1 # ubuntu $ sudo apt-get install librrds-perl libsnmp-perl # CentOS rrdtool EPEL $ sudo yum install net-snmp-perl $ sudo yum install rrdtool-perl
    26. 26. 2 $ git clone git://github.com/kazeburo/cloudforecast.git $ cd cloudforecast $ cpanm -l extlib --installdeps . $ ls Changes cloudforecast_web Makefile.PL docs README host_config cf_devstarter htdocs cf_fetcher_worker lib cf_updater_worker server_list_sample.yaml cloudforecast_radar t cloudforecast_sample.yaml
    27. 27. # $ cp cloudforecast_sample.yaml cloudforecast.yaml # $ cp server_list_sample.yaml server_list.yaml
    28. 28. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers servers: - config: basic.yaml hosts: ....
    29. 29. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers servers: - config: basic.yaml hosts: ....
    30. 30. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers servers: - config: basic.yaml hosts: ....
    31. 31. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers IP servers: - config: basic.yaml hosts: ....
    32. 32. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers IP servers: - config: basic.yaml hosts: ....
    33. 33. (server_list.yaml) --- #LOCALHOST servers: - config: basic.yaml hosts: - 127.0.0.1 server1 my great localhost --- #My Servers IP servers: - config: basic.yaml hosts: ....
    34. 34. (basic.yaml) --- component_config: resources: - basic - traffic:eth0 basic = eth0 CPU TCP
    35. 35. (basic.yaml) --- component_config: resources: - basic - traffic:eth0 basic = eth0 CPU TCP
    36. 36. (basic.yaml) --- component_config: resources: - basic - traffic:eth0 basic = eth0 CPU TCP
    37. 37. • Basic(CPU TCP ) • Traffic • Disk Usage • DiskIO Count • Apache Status • Nginx Status • Squid • Mysql • Innodb
    38. 38. $ perldoc CloudForecast::Data::
    39. 39. $ perldoc CloudForecast::Data::
    40. 40. CloudForecast http://blog.nomadscafe.jp/2010/07/cloudforecast-1.html
    41. 41. # $ ./cloudforecast_radar -c cloudforecast.yaml -l server_list.yaml # Web $ ./cloudforecast_web -p 5000 -c cloudforecast.yaml -l server_list.yaml
    42. 42. Web ( )
    43. 43. Web ( )
    44. 44. http://blog.nomadscafe.jp/2010/07/cloudforecast-2.html
    45. 45.
    46. 46. • •
    47. 47. • • •
    48. 48. # site-lib $ mkdir -p site-lib/CloudForecast/Data
    49. 49. Gearman Worker gearman worker process http://blog.nomadscafe.jp/2010/07/gearman-worker-process.html
    50. 50. % ./gearman-starter.pl --max-prcess 8 -s 127.0.0.1:7004 --scoreboard-dir /var/run/worker --port 7005 MyWorker % telnet localhost 7005 Trying 127.0.0.1... System: gearman_servers: 127.0.0.1:7004 class: MyWorker BusyWorkers: 0 IdleWorkers: 8 -- pid Status Counter Comment 1630 . 0 1631 . 0 1632 . 0 1633 . 0 1634 . 0 1635 . 0 1636 . 0 1637 . 0
    51. 51. % ./gearman-starter.pl --max-prcess 8 -s 127.0.0.1:7004 --scoreboard-dir /var/run/worker --port 7005 MyWorker % telnet localhost 7005 Trying 127.0.0.1... System: gearman_servers: 127.0.0.1:7004 class: MyWorker BusyWorkers: 0 IdleWorkers: 8 -- pid Status Counter Comment 1630 . 0 1631 . 0 1632 . 0 1633 . 0 1634 . 0 1635 . 0 1636 . 0 1637 . 0
    52. 52. • •
    53. 53. package CloudForecast::Data::Gearmanstarter; use CloudForecast::Data -base; use IO::Socket::INET; =pod host_config) resources: - gearmanstarter[:port] eg) - gearmanstarter # 9000 - gearmanstarter:9005 # port =cut
    54. 54. rrds map { [$_,'GAUGE'] } qw/busy idle/; # rrds ‘key’, ‘TYPE’ or rrds [‘key’,‘TYPE’], []..
    55. 55. • COUNTER • • • ) Traffic ( octet ) • GAUGE • • • )
    56. 56. (1) graphs 'status' => 'worker status'; # graphs ‘key1’ => ‘label’; # graphs ‘key2’ => ‘the great graph’;
    57. 57. (2) graphs 'status' => 'worker status'; .. __DATA__ @@ status DEF:my1=<%RRD%>:busy:AVERAGE DEF:my2=<%RRD%>:idle:AVERAGE AREA:my1#00C000:Busy GPRINT:my1:LAST:Cur: %4.1lf GPRINT:my1:AVERAGE:Ave: %4.1lf GPRINT:my1:MAX:Max: %4.1lf GPRINT:my1:MIN:Min: %4.1lfc STACK:my2#0000C0:Idle GPRINT:my2:LAST:Cur: %4.1lf GPRINT:my2:AVERAGE:Ave: %4.1lf GPRINT:my2:MAX:Max: %4.1lf GPRINT:my2:MIN:Min: %4.1lfc
    58. 58. (2) graphs 'status' => 'worker status'; .. key __DATA__ @@ status DEF:my1=<%RRD%>:busy:AVERAGE DEF:my2=<%RRD%>:idle:AVERAGE AREA:my1#00C000:Busy GPRINT:my1:LAST:Cur: %4.1lf GPRINT:my1:AVERAGE:Ave: %4.1lf GPRINT:my1:MAX:Max: %4.1lf GPRINT:my1:MIN:Min: %4.1lfc STACK:my2#0000C0:Idle GPRINT:my2:LAST:Cur: %4.1lf GPRINT:my2:AVERAGE:Ave: %4.1lf GPRINT:my2:MAX:Max: %4.1lf GPRINT:my2:MIN:Min: %4.1lfc
    59. 59. fetcher { my $c = shift; my $host = $c->address; my $port = $c->args->[0] || 9000; my $sock = IO::Socket::INET->new( PeerAddr => $host, PeerPort => $port, Proto => 'tcp', ); my $raw_stats; $sock->sysread( $raw_stats, 8192 ); my ($busy, $idle); foreach my $line ( split /[rn]+/, $raw_stats ) { if ( $line =~ /^Busy.+: (d+)/ ) { $busy = $1; } if ( $line =~ /^Idle.+: (d+)/ ) { $idle = $1; } } return [$busy,$idle]; };
    60. 60. fetcher { my $c = shift; my $host = $c->address; my $port = $c->args->[0] || 9000; my $sock = IO::Socket::INET->new( PeerAddr => $host, PeerPort => $port, Proto => 'tcp', ); my $raw_stats; $sock->sysread( $raw_stats, 8192 ); my ($busy, $idle); foreach my $line ( split /[rn]+/, $raw_stats ) { if ( $line =~ /^Busy.+: (d+)/ ) { $busy = $1; } if ( $line =~ /^Idle.+: (d+)/ ) { $idle = $1; } } return [$busy,$idle]; };
    61. 61.
    62. 62. IDLE BUSY
    63. 63. TODO/Plan • Web • RRD SQLite •
    64. 64. • CloudForecast Perl • •

    ×