Logging is an absolute must for any API or web application, but when starting out, questions such as "how can we do it without disrupting everything else" and "what is the easiest way to log" often come up. We’re going to examine a tried and tested method to carry out high-performance, low-latency logging using the power of RabbitMQ to ensure minimal impact to the performance of your runtime application. The talk will show you that a really great logging architecture is a low-cost investment in your application that will definitely pay off in the long run.

1. Low Latency Logging
with RabbitMQ
BrnoPHP Conference 2014

2. Who is this guy?
James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim

3. Let’s go back to basics...

4. Errors

5. source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/

10. error_reporting(0);

11. They look rubbish!

12. @

13. Log to file only

14. Ways Around
// Will raise E_WARNING
fopen($somefile, 'r');

15. Ways Around
// No error! :)
if (file_exists($somefile)) {
fopen($somefile, 'r');
} else {
// nice handling...
// maybe throw exception...
}

16. Exceptions

17. Jargon Buster
● throw
Triggering
● try
Try to run
● catch
Handle it
● finally
Run code after try/catch

18. source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/

19. C♯/C++

20. Catchable

21. Turn into fatal errors
if not caught

22. Classy

23. Descriptive exceptions

24. Example (exception class)
class DivisionByZeroException
extends LogicException
{
}

25. Example (throw)
class Mathematics {
public function divide($a, $b) {
if ($b == 0) {
$msg = sprintf(“Divide %s by zero”, $a);
throw new DivisionByZeroException($msg);
}
return ($a / $b);
}
}

26. Example (catch)
$maths = new Mathematics();
try
{
$result = $maths->divide(5, 0);
}
catch (DivisionByZeroException $exception)
{
$logger->warning($exception->getMessage());
}

27. Throw Descriptively

28. Logging

30. What is “logging”?

31. What is “logging”?
Keeping a record of all events...

32. What is “logging”?
Keeping a record of all events...
exceptions, errors, warnings, info, debug

33. Paper Trail

34. Log like you just don’t care

35. Log like you just don’t care

37. Log wherever you like

38. What about Apache’s error_log?
source: http://up-ship.com/blog/?p=20903

39. Doin’ it rightwrong…
// Magic file that makes your entire project work perfectly
@ini_set('display_errors', 0);
@error_reporting(0);
function __globalErrorHandler() {
return true;
}
@set_error_handler('__globalErrorHandler');
@set_exception_handler('__globalErrorHandler');
@register_shutdown_function(function() {
if(error_get_last())
echo "Script executed successfully!";
}); https://github.com/webarto/boostrap.php

40. Fire & Forget

41. Low Latency

42. Highly Available

44. PSR-3

45. Logging Options
● monolog (PSR-3)
● Drupal - PSR-3 Watchdog
● phpconsole
● log4php
● RavenPHP + Sentry
● FirePHP (dev environment)
● Roll your own

46. source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html

47. Capturing Messages

48. Capturing Messages
set_error_handler()

49. Capturing Messages
set_exception_handler()

50. Capturing Messages
register_shutdown_function()

51. Capturing Messages
Using the Logger Directly

52. Adapter / Handler

53. Typical PSR-3 Compatible Design
Capture Method
Logger (PSR-3)
Handler / Adapter
Data Storage

54. MonologErrorHandler
->handleException()
MonologLogger
->log()
MonologHandler
->handle()
Monolog

55. Low Latency Logging

56. Make it easy

57. Fire & Forget

58. Minimum Impact

59. Slow Logging
Browser Application Log Server
HTTP request
Send log message to log server
Error!
Acknowledge message
HTTP response to client

60. Zero Latency Logging (ideal)
Browser Application Log Server
HTTP request
Send log message to log server
Error!
HTTP response to client

61. UDP?

62. We need a balance.

63. Low Latency Logging (balance)
Browser Application Log Server
HTTP request
Send log message to log server
Error!
HTTP response to client

64. …so how?

65. Disclaimer...

66. Say hello to RabbitMQ

67. RabbitMQ - Basic
test_queue
1 2 3 4 5
Producer Consumer
source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html

68. RabbitMQ - Exchanges
Exchange
Consumer test_queue
1 2 3 4 5
source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html
Consumer
Producer
Producer
Producer

69. Exchanges are flexible

70. RabbitMQ === Fast!

71. Low Latency (using AMQP)
EdLogHandlerErrorHandler
->handleException()
EdLogLogger
->log()
EdLogPublisherAmqpPublisher
->publish()
RabbitMQ JSON payload
Logging Server

72. Low Latency Logging (with AMQP)
Browser Application Log Server
Fetch message
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

73. Why bother?
● Scalability
RabbitMQ
Application A
Application B
Application C
Log Worker
Log Worker
Log Worker
Log Worker

74. Single Point of Failure...
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 6
RabbitMQ
Node 5

75. To recap...

76. Scalable.

77. Easy.

78. Fast.

79. Future proof.

80. Questions?

81. Thanks for watching!
James Titcumb
@asgrim