Advertisement
Low Latency Logging with RabbitMQ (Brno PHP, CZ - 20th Sep 2014)
Sep. 20, 2014•0 likes•2,024 views
0 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Technology
Logging is an absolute must for any API or web application, but when starting out, questions such as "how can we do it without disrupting everything else" and "what is the easiest way to log" often come up. We’re going to examine a tried and tested method to carry out high-performance, low-latency logging using the power of RabbitMQ to ensure minimal impact to the performance of your runtime application. The talk will show you that a really great logging architecture is a low-cost investment in your application that will definitely pay off in the long run.
James TitcumbFollow
Freelance at Roave, LLCAdvertisement
Low Latency Logging with RabbitMQ (Brno PHP, CZ - 20th Sep 2014)
- Low Latency Logging with RabbitMQ BrnoPHP Conference 2014
- Who is this guy? James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim
- Let’s go back to basics...
- Errors
- source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
- error_reporting(0);
- They look rubbish!
- @
- Log to file only
- Ways Around // Will raise E_WARNING fopen($somefile, 'r');
- Ways Around // No error! :) if (file_exists($somefile)) { fopen($somefile, 'r'); } else { // nice handling... // maybe throw exception... }
- Exceptions
- Jargon Buster ● throw Triggering ● try Try to run ● catch Handle it ● finally Run code after try/catch
- source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
- C♯/C++
- Catchable
- Turn into fatal errors if not caught
- Classy
- Descriptive exceptions
- Example (exception class) class DivisionByZeroException extends LogicException { }
- Example (throw) class Mathematics { public function divide($a, $b) { if ($b == 0) { $msg = sprintf(“Divide %s by zero”, $a); throw new DivisionByZeroException($msg); } return ($a / $b); } }
- Example (catch) $maths = new Mathematics(); try { $result = $maths->divide(5, 0); } catch (DivisionByZeroException $exception) { $logger->warning($exception->getMessage()); }
- Throw Descriptively
- Logging
- What is “logging”?
- What is “logging”? Keeping a record of all events...
- What is “logging”? Keeping a record of all events... exceptions, errors, warnings, info, debug
- Paper Trail
- Log like you just don’t care
- Log like you just don’t care
- Log wherever you like
- What about Apache’s error_log? source: http://up-ship.com/blog/?p=20903
- Doin’ it rightwrong… // Magic file that makes your entire project work perfectly @ini_set('display_errors', 0); @error_reporting(0); function __globalErrorHandler() { return true; } @set_error_handler('__globalErrorHandler'); @set_exception_handler('__globalErrorHandler'); @register_shutdown_function(function() { if(error_get_last()) echo "Script executed successfully!"; }); https://github.com/webarto/boostrap.php
- Fire & Forget
- Low Latency
- Highly Available
- PSR-3
- Logging Options ● monolog (PSR-3) ● Drupal - PSR-3 Watchdog ● phpconsole ● log4php ● RavenPHP + Sentry ● FirePHP (dev environment) ● Roll your own
- source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html
- Capturing Messages
- Capturing Messages set_error_handler()
- Capturing Messages set_exception_handler()
- Capturing Messages register_shutdown_function()
- Capturing Messages Using the Logger Directly
- Adapter / Handler
- Typical PSR-3 Compatible Design Capture Method Logger (PSR-3) Handler / Adapter Data Storage
- MonologErrorHandler ->handleException() MonologLogger ->log() MonologHandler ->handle() Monolog
- Low Latency Logging
- Make it easy
- Fire & Forget
- Minimum Impact
- Slow Logging Browser Application Log Server HTTP request Send log message to log server Error! Acknowledge message HTTP response to client
- Zero Latency Logging (ideal) Browser Application Log Server HTTP request Send log message to log server Error! HTTP response to client
- UDP?
- We need a balance.
- Low Latency Logging (balance) Browser Application Log Server HTTP request Send log message to log server Error! HTTP response to client
- …so how?
- Disclaimer...
- Say hello to RabbitMQ
- RabbitMQ - Basic test_queue 1 2 3 4 5 Producer Consumer source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html
- RabbitMQ - Exchanges Exchange Consumer test_queue 1 2 3 4 5 source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html Consumer Producer Producer Producer
- Exchanges are flexible
- RabbitMQ === Fast!
- Low Latency (using AMQP) EdLogHandlerErrorHandler ->handleException() EdLogLogger ->log() EdLogPublisherAmqpPublisher ->publish() RabbitMQ JSON payload Logging Server
- Low Latency Logging (with AMQP) Browser Application Log Server Fetch message HTTP request JSON via AMQP Error! HTTP response RabbitMQ
- Why bother? ● Scalability RabbitMQ Application A Application B Application C Log Worker Log Worker Log Worker Log Worker
- Single Point of Failure... RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 6 RabbitMQ Node 5
- To recap...
- Scalable.
- Easy.
- Fast.
- Future proof.
- Questions?
- Thanks for watching! James Titcumb @asgrim
Advertisement