Real time system_performance_mon
Upcoming SlideShare
Loading in...5
×
 

Real time system_performance_mon

on

  • 5,696 views

 

Statistics

Views

Total Views
5,696
Views on SlideShare
5,695
Embed Views
1

Actions

Likes
4
Downloads
61
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Real time system_performance_mon Real time system_performance_mon Presentation Transcript

  • Real time system performance monitoring AMQP and CatalystX::JobServer Tomas (t0m) Doran São Paulo.pm perl workshop 2010
  • Aka the “my code doesn’t work yet” talk • I wrote half the slides for this at 6am Friday (São Paulo time) • The title is a bit misleading, as I wrote the talk abstract in March, then wrote entirely different software. • Sorry about that.
  • Message queueing • Lets not talk about the (not working) code just yet... • This talk is about Message Queueing. • What is Message Queueing? • Why would I want to use it?
  • What is message queueing. • In it’s simplest form, it’s just a list. • 1 (or more) ‘producers’ (writers) • 1 (or more) ‘consumers’ (readers) • Queue if rate of production > rate of consumption
  • Why do I want it? • Decouples producers and consumers (probably across the network). • Lets you manage load, and spikes in load (only n consumers). • In a web environment - lets you serve more pages, quicker.
  • Other cases • View as a list a little simplistic. • Depending on your application, you may want: • One to one • One to many • One to many (broadcast) • Many to all of the above • More complex topology?
  • AMQP • Queuing protocol • Fast • Interoperable (maybe?)
  • AMQP AMQP enables complete interoperability for messaging middleware, both the networking protocol and the semantics of broker services are defined in AMQP.
  • AMQP Say what?
  • AMQP • Full abstraction of all the mechanics of messaging across the network. • Serialization and framing of messages. • Wiring of message routing is part of the protocol.
  • AMQP • So all your clients know (at least half) of the wiring. • Different topologies depending on routing configuration. • Can specify other options such as durability • Nice when your server dies - no ‘current config’
  • AMQP Concepts RabbitMQ vhost Publisher Exchange Queue Consumer
  • Concepts - Exchanges • Named • Messages are published (sent) to one exchange • Can be durable (lasts till deleted), or auto- deleted
  • Queues • Queues can be named or (or named by the server - private queues). • Queues are bound to one (or more) exchanges. • Queues can have 0 or more clients • Queues may persist with 0 clients or not. • FIFO (if you have 1 consumer) • Message never delivered to > 1 client
  • Bindings • Binding is what joins a queue and an exchange.
  • Traditional queue • Named exchange • Bound to 1 Named queue • 0 or more listeners get round-robin messages • Messages queue when nobody listens / if consumers are slow
  • Broadcast • Named exchange • Each client creates an anonymous ephermeral queue • Client binds the queue to that exchange • All clients get all messages • Messages go to /dev/null if no clients
  • Others • Different exchange types - direct, topic, fanout • You can do a lot using just these and a mix of named and anonymous queues • More complex topologies possible
  • I accidentally wrote software • I try to avoid doing this • I’m not very good at it ;)
  • I blame Net::RabbitFoot • It’s written using AnyEvent and Coro • Which I hadn’t used before, but looked good for this type of thing. • I felt my way around writing simple things standalone. • Got all the web servers logging into Rabbit
  • Logging into message queuing • Good example of broadcast • Want to aggregate logs to files • And be able to ‘tail’ them • Logging directly from the application • Also tailing (normal) log files to message queue
  • Getting the logs back
  • Ok, there is some setup
  • Ok, there is some setup
  • Dumping them to a file • That’s pretty simple after that.. • Except: • Log rotation • Not flushing to disk once per message • etc...
  • Viewing them live • Someone wrote an AMQP client in flash • AMQP security model not useful publicly • Cute prototype • (Sorry, no demo - refuses to work locally)
  • Queueing Jobs • Next application, simple job queue. • 1 worker, serialized.. (Crap, good enough) • Log JSON into RabbitMQ (again), inflate, call the ->run method. • Works!
  • Job Server • I now have several scripts, all doing bits with queuing. All duplicating code. • I want to aggregate messages • Send messages of aggregates at regular intervals • Need something more generic
  • Job server • Wrote a simple abstraction on getting channels, making exchanges and queues, etc • Was still going to end up with a load of scripts/jobs that needed running and managing.Yuk. • Inspecting the state of each job ‘service’ hard / boring..
  • But wait • Each ‘thing’ (timer, listener, logger, emitter, worker etc) can be an instance of a class with attributes for the state. • Construct a set from some config file. • Lets me aggregate jobs together / share code (Can use AnyEvent/Coro for threads). • Less processes to manage. Working out the state just got harder still.
  • But wait • Make all the classes use MooseX::Storage • Catalyst makes instances of things from config... • So, I could adapt all these instances into Catalyst models, write a controller, done! • Erm, but Catalyst, and message queueing, and what?!?!?
  • I blame Miyagawa • For Plack, and Catalyst::Engine::Plack, and Corona (the Coro PSGI server). • I had this INSANE idea one night. • I tried it. • IT WORKED. • WOAH.
  • I blame clkao • Next step - browser kicks off a job. • Status via long poll? • He did it already! • With long hair!
  • Hippie • Async pipe to the browser. • Abstracts all the nasty ajax details (also does long poll, or websockets) • Applications: • Automatically updated monitor of app components • Start a job and then get an AJAX update screen for the job progress.
  • Useable? • The basics work. • Running it production at work • (Since WedneThursday....) • If you need a high volume, simple, production ready job scheduler right now, use Gearman.
  • Useable? • Needs my github versions of Catalyst::Engine::PSGI and Web::Hippie • Running jobs from the web app doesn’t work/exist yet. • All the pieces are there - next week?
  • Demo?
  • Next steps? • Production battle scars ;) • Hippie for watching jobs run, not just component status updates. • Docs would be good... • Hippe::Pipe for bidirectional • More traits / plugins / components
  • What to go look at • http://www.rabbitmq.com • Docs, setup info, FAQ • Slideshare - lots of good and more detailed AMQP introductions.
  • What to go look at (CPAN) • Net::RabbitFoot • App::RabbitTail • Plack • Web::Hippie • CatalystX::JobServer (github)
  • Bonus slide! • I forgot to mention this originally... • DO NOT FORK AFTER USING Net::RabbitFoot (and then try to use it in the child process) • This will go horribly horribly wrong.
  • Other solutions • Gearman • STOMP • ActiveMQ (or Rabbit has a STOMP adaptor) • Catalyst::Engine::STOMP
  • Thanks! • Questions? • Anyone interested in playing, grab me on irc • Happy to hand-hold as I need more people using this. • I’ll love you forever if you write docs.