Your SlideShare is downloading. ×
Metrics that talk on cloud using ganglia
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Metrics that talk on cloud using ganglia

909

Published on

Perform metering and monitoring on your cloud applications

Perform metering and monitoring on your cloud applications

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
909
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
13
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Metrics that “talk” on Cloud using Ganglia How do we monitor performance on Megam Cloud
  • 2. We'll Cover ● Our Experience Using Ganglia ● How does it work in our Platform ● Chef cookbooks for metering and setup. ● Dashboard integration © 2012-2013 Megam Systems
  • 3. What is Ganglia Scalable distributed monitoring system for high-performance computing systems. Sends information about your cloud instance. Can be used as a live cloud monitor. Can be extended using python plugins © 2012-2013 Megam Systems
  • 4. What have we accomplished ? Oh Yeah - <flip to next page> © 2012-2013 Megam Systems
  • 5. Dash #2 (AngularJs Client) Metrics API gmon gmon gmetad gmetad gmetad gmon gmon © 2012-2013 Megam Systems
  • 6. Would you like to setup one ? Yes you can (or) http://www.megam.co © 2012-2013 Megam Systems
  • 7. Few facts on gmetad/gmond gmetad can run standalone or along with gmond. gmetad can be configured to collect metrics of gmond servers of same cluster or different cluster. gmetad stores data at ➔ /var/lib/ganglia/rrds/CLUSTER_NAME/GMOND_SERVER_NAME GMOND_SERVER_NAME can be changed in gmond.conf © 2012-2013 Megam Systems
  • 8. gmetad is the metrics collector gmond is the metrics sender Ok. Got it. © 2012-2013 Megam Systems
  • 9. What is our setup #1 gmetad : monitor1.megam.co #2 gmetad : montior2.megam.co Several gmonds(Cloud Apps) pumping data to gmetad © 2012-2013 Megam Systems
  • 10. What are Cloud Apps Any app for lang := range ProgLanguages { Java Scala Go …. meteor } && DB, Queue © 2012-2013 Megam Systems
  • 11. Do you need Graphite No Why ? Needs rrds formatted metric files It copies rrds files from gmetad Twice storage Is this the only soln ? Eager to hear feedback. © 2012-2013 Megam Systems
  • 12. Let us setup gmetad 3.3.8-1 Ubuntu(raring) : package is gmetad Ubuntu(saucy) has 3.6.0 sudo apt-get install gmetad sudo apt-get install ganglia-webfrontend (*optional) We used Opscode cookbook to setup => Link © 2012-2013 Megam Systems
  • 13. Configure gmetad nano /etc/ganglia/gmetad.conf data_source “megcluster” <gmond1>.megam.co:8649 <gmond2>.megam.co:8649 ➔ The above says “megcluster” collects metrics from <gmond1>.megam.co and <gmond2>.megam.co ➔ Which is like monitoring a Java App in <gmond1>.megam.co (or) ➔ Your favorite App in <gmond2>.megam.co © 2012-2013 Megam Systems
  • 14. gmetad - start/stop. Start : sudo gmetad Stop : good old kill ps -ef | grep gmetad sudo kill -9 <pid> © 2012-2013 Megam Systems
  • 15. Cool gmetad - monitor1.megam.co is running © 2012-2013 Megam Systems
  • 16. gmond Install ganglia-monitor-python package will be installed in a server which is to be monitored. ➔ Package has methods to collect basic metrics(cpu...) using python scripts in /usr/lib/ganglia/ ➔ Extended by enabling additional python scripts at /usr/lib/ganglia/python_modules. ➔ For an exhaustive list : https://github.com/ganglia/gmond_python_modules ➔ © 2012-2013 Megam Systems
  • 17. Let us setup gmond 3.3.8-1 Ubuntu(raring) : package is ganglia-monitor-python Ubuntu(saucy) has 3.6.0 sudo apt-get install ganglia-monitor-python We used Opscode cookbook to setup => Link © 2012-2013 Megam Systems
  • 18. Configure gmond nano /etc/ganglia/gmond.conf globals { override_hostname = <gmond1>.megam.co override_ip = 127.0.0.1 } udp_send_channel“megcluster” collects metrics from <gmond1>.megam.co and ➔ The above says { host = monitor1.megam.co <gmond2>.megam.co port = 8649 ttl ➔ 1 = Which is like monitoring a Java App in <gmond1>.megam.co } (or) cluster { nameYour favorite App in <gmond2>.megam.co = "megcluster" ➔ owner = "unspecified" } © 2012-2013 Megam Systems
  • 19. What did we configure ? ➔ In the globals we say our monitoring Java App's host name < gmond1>.megam.co ➔ We provide the UDP channel of the gmetad (monitor1.megam.co) ➔ We need to specify the gmetad cluster (megcluster) ➔ cluster attribute groups all gmond to a gmetd <CLUSTER> in our case megcluster. © 2012-2013 Megam Systems
  • 20. gmond - start/stop. Start : sudo gmond Stop : good old kill ps -ef | grep gmond sudo kill -9 <pid> © 2012-2013 Megam Systems
  • 21. Cool gmond - <gmond1>.megam.co is running & pumping to monitor1.megam.co © 2012-2013 Megam Systems
  • 22. We customized chef - cookbooks © 2012-2013 Megam Systems
  • 23. How do we use the chef - cookbooks © 2012-2013 Megam Systems
  • 24. Tweak cookbook for gmetad https://github.com/indykish/chef-repo/tree/master/cookbo ➔ Attributes default[:ganglia][:cluster_name] = "megcluster" default[:ganglia][:unicast] = true default[:ganglia][:hostname] = “monitoring1.megam.co” © 2012-2013 Megam Systems
  • 25. Chef Run : gmetad Run chef : runlist 'recipe[megam_ganglia::gmetad]' © 2012-2013 Megam Systems
  • 26. Tweak recipes for gmond For Any App ➔ Default : installs and configures ganglia-monitor-python. It collects the basic meterings like cpu_usage, memory_usage etc. ➔ Nginx : collects nginx status details. ➔ Rabbit : collects rabbbitmq metrics. ➔ Redis : collects redis metrics. ➔ Riak : collects riak metrics. © 2012-2013 Megam Systems
  • 27. Chef Run : gmond To monitor an app include_recipe “megam_ganglia” Nginx frontended apps include_recipe “megam_ganglia::nginx” Rabbitmq apps include_recipe “megam_ganglia::rabbit” Riak apps include_recipe “megam_ganglia::riak” Redis apps include_recipe “megam_ganglia::redis” © 2012-2013 Megam Systems
  • 28. Configure gmond nano /etc/ganglia/gmond.conf globals { daemonize = yes setuid = yes user = nobody debug_level = 0 max_udp_msg_len = 1472 mute = no deaf = no host_dmax = 86400 /*secs */ cleanup_threshold = 300 /*secs */ gexec = no send_metadata_interval = 30 override_hostname = "<gmond1>.megam.co" override_ip = 127.0.0.1 } cluster { name = "megcluster" owner = "unspecified" latlong = "unspecified" url = "unspecified" } /* The host section describes attributes of the host, like the location */ host { location = "unspecified" } /* Feel free to specify as many udp_send_channels as you like. Gmond used to only support having a single channel */ udp_send_channel { host = monitoring1.megam.co port = 8649 ttl = 1 } © 2012-2013 Megam Systems
  • 29. Verifying gmond data Open up your browser http://gmond1.megam.co:8649 ➔ It will list the metrics of the gmond instance. ➔ Not recommended for prod. © 2012-2013 Megam Systems
  • 30. Sample gmond data (for redis server) © 2012-2013 Megam Systems
  • 31. Checking gmeta data Data files in the below dir increases – Get metrics data at /var/lib/ganglia/rrds/megamcluster/gmond1.megam.co © 2012-2013 Megam Systems
  • 32. Sample gmetad data (for thomas.work.local) © 2012-2013 Megam Systems
  • 33. Dash Integration in rails Built on – – We'll cover it detail in a separate slideshare. If you are hungry “Code is the design” :) For questions on this area:rajthilak@megam.co.in © 2012-2013 Megam Systems
  • 34. References Ganglia Wiki megam chef-repo © 2012-2013 Megam Systems
  • 35. Our Organization(Megam Systems) Beta Launch of Megam Cloud (Polygot PaaS) Our PaaS design => Link Register http://www.megam.co for an invite Twitter : @indykish © 2012-2013 Megam Systems
  • 36. Screencast illustrating the Cloud API Servers working live © 2012-2013 Megam Systems
  • 37. Thank you for watching © 2012-2013 Megam Systems

×