Successfully reported this slideshow.

Low Latency Logging with RabbitMQ (Brno PHP, CZ - 20th Sep 2014)

0

Share

1 of 81
1 of 81

Low Latency Logging with RabbitMQ (Brno PHP, CZ - 20th Sep 2014)

0

Share

Download to read offline

Logging is an absolute must for any API or web application, but when starting out, questions such as "how can we do it without disrupting everything else" and "what is the easiest way to log" often come up. We’re going to examine a tried and tested method to carry out high-performance, low-latency logging using the power of RabbitMQ to ensure minimal impact to the performance of your runtime application. The talk will show you that a really great logging architecture is a low-cost investment in your application that will definitely pay off in the long run.

Logging is an absolute must for any API or web application, but when starting out, questions such as "how can we do it without disrupting everything else" and "what is the easiest way to log" often come up. We’re going to examine a tried and tested method to carry out high-performance, low-latency logging using the power of RabbitMQ to ensure minimal impact to the performance of your runtime application. The talk will show you that a really great logging architecture is a low-cost investment in your application that will definitely pay off in the long run.

More Related Content

More from James Titcumb

Related Books

Free with a 14 day trial from Scribd

See all

Low Latency Logging with RabbitMQ (Brno PHP, CZ - 20th Sep 2014)

  1. 1. Low Latency Logging with RabbitMQ BrnoPHP Conference 2014
  2. 2. Who is this guy? James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim
  3. 3. Let’s go back to basics...
  4. 4. Errors
  5. 5. source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
  6. 6. error_reporting(0);
  7. 7. They look rubbish!
  8. 8. @
  9. 9. Log to file only
  10. 10. Ways Around // Will raise E_WARNING fopen($somefile, 'r');
  11. 11. Ways Around // No error! :) if (file_exists($somefile)) { fopen($somefile, 'r'); } else { // nice handling... // maybe throw exception... }
  12. 12. Exceptions
  13. 13. Jargon Buster ● throw Triggering ● try Try to run ● catch Handle it ● finally Run code after try/catch
  14. 14. source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
  15. 15. C♯/C++
  16. 16. Catchable
  17. 17. Turn into fatal errors if not caught
  18. 18. Classy
  19. 19. Descriptive exceptions
  20. 20. Example (exception class) class DivisionByZeroException extends LogicException { }
  21. 21. Example (throw) class Mathematics { public function divide($a, $b) { if ($b == 0) { $msg = sprintf(“Divide %s by zero”, $a); throw new DivisionByZeroException($msg); } return ($a / $b); } }
  22. 22. Example (catch) $maths = new Mathematics(); try { $result = $maths->divide(5, 0); } catch (DivisionByZeroException $exception) { $logger->warning($exception->getMessage()); }
  23. 23. Throw Descriptively
  24. 24. Logging
  25. 25. What is “logging”?
  26. 26. What is “logging”? Keeping a record of all events...
  27. 27. What is “logging”? Keeping a record of all events... exceptions, errors, warnings, info, debug
  28. 28. Paper Trail
  29. 29. Log like you just don’t care
  30. 30. Log like you just don’t care
  31. 31. Log wherever you like
  32. 32. What about Apache’s error_log? source: http://up-ship.com/blog/?p=20903
  33. 33. Doin’ it rightwrong… // Magic file that makes your entire project work perfectly @ini_set('display_errors', 0); @error_reporting(0); function __globalErrorHandler() { return true; } @set_error_handler('__globalErrorHandler'); @set_exception_handler('__globalErrorHandler'); @register_shutdown_function(function() { if(error_get_last()) echo "Script executed successfully!"; }); https://github.com/webarto/boostrap.php
  34. 34. Fire & Forget
  35. 35. Low Latency
  36. 36. Highly Available
  37. 37. PSR-3
  38. 38. Logging Options ● monolog (PSR-3) ● Drupal - PSR-3 Watchdog ● phpconsole ● log4php ● RavenPHP + Sentry ● FirePHP (dev environment) ● Roll your own
  39. 39. source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html
  40. 40. Capturing Messages
  41. 41. Capturing Messages set_error_handler()
  42. 42. Capturing Messages set_exception_handler()
  43. 43. Capturing Messages register_shutdown_function()
  44. 44. Capturing Messages Using the Logger Directly
  45. 45. Adapter / Handler
  46. 46. Typical PSR-3 Compatible Design Capture Method Logger (PSR-3) Handler / Adapter Data Storage
  47. 47. MonologErrorHandler ->handleException() MonologLogger ->log() MonologHandler ->handle() Monolog
  48. 48. Low Latency Logging
  49. 49. Make it easy
  50. 50. Fire & Forget
  51. 51. Minimum Impact
  52. 52. Slow Logging Browser Application Log Server HTTP request Send log message to log server Error! Acknowledge message HTTP response to client
  53. 53. Zero Latency Logging (ideal) Browser Application Log Server HTTP request Send log message to log server Error! HTTP response to client
  54. 54. UDP?
  55. 55. We need a balance.
  56. 56. Low Latency Logging (balance) Browser Application Log Server HTTP request Send log message to log server Error! HTTP response to client
  57. 57. …so how?
  58. 58. Disclaimer...
  59. 59. Say hello to RabbitMQ
  60. 60. RabbitMQ - Basic test_queue 1 2 3 4 5 Producer Consumer source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html
  61. 61. RabbitMQ - Exchanges Exchange Consumer test_queue 1 2 3 4 5 source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html Consumer Producer Producer Producer
  62. 62. Exchanges are flexible
  63. 63. RabbitMQ === Fast!
  64. 64. Low Latency (using AMQP) EdLogHandlerErrorHandler ->handleException() EdLogLogger ->log() EdLogPublisherAmqpPublisher ->publish() RabbitMQ JSON payload Logging Server
  65. 65. Low Latency Logging (with AMQP) Browser Application Log Server Fetch message HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  66. 66. Why bother? ● Scalability RabbitMQ Application A Application B Application C Log Worker Log Worker Log Worker Log Worker
  67. 67. Single Point of Failure... RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 6 RabbitMQ Node 5
  68. 68. To recap...
  69. 69. Scalable.
  70. 70. Easy.
  71. 71. Fast.
  72. 72. Future proof.
  73. 73. Questions?
  74. 74. Thanks for watching! James Titcumb @asgrim

×