Logging is an absolute must for any API or web application, but when starting out, questions such as "how can we do it without disrupting everything else" and "what is the easiest way to log" often come out. I'm going to explore a couple of infrastructure ideas to carry out what I call "high-performance, low-latency" logging to ensure minimal impact to the performance of the runtime application. The talk will show you that a really great logging architecture is a low-cost investment in your application that will definitely pay off in the long run.

1. Low Latency Logging
at BrightonPHP

2. James Titcumb
@asgrim
github.com/asgrim

3. ● Started on Amiga 500
● PHP for 11 years
● ZCE PHP 5.3
●
●
●
●
●
Run PHP Hampshire user group
Lead dev on browscap.ini
First open source GoDeploy
Development Manager at Protected.co.uk
Mostly a ZF2 developer
@asgrim
github.com/asgrim

4. Hello! :)
@asgrim
github.com/asgrim

5. Errors

6. What are errors?
● Something broke… :)
● e.g.
○ Can’t connect to MySQL (mysqli_connect)
○ No such file or directory (fopen)
● Usually PHP core
● Sometimes “fatal”

7. Problems?
source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/

8. Problems.
●
●
●
●
●
error_reporting setting
Errors look ugly
“@”
Limited options
Not very “OO”

9. Ways Around
// Will raise E_NOTICE
fopen($somefile, 'r');

10. Ways Around
// No error! :)
if (file_exists($somefile)) {
fopen($somefile, 'r');
} else {
// nice handling...
// maybe throw exception...
}

11. Exceptions

12. Jargon Buster
● throw
Triggering
● try
Try to run
● catch
Handle it
● finally
Run code after try/catch

13. What are exceptions?
●
●
●
●
●
Something still broke
OO
Catchable
Fatal errors
They are classes

14. SPL Exceptions
● Built in to PHP
● More descriptive than just “Exception”, e.g.:
○
○
○
○
○
InvalidArgumentException
LogicException
OutOfBoundsException
RuntimeException
see PHP manual for more

15. Example (exception class)
class DivisionByZeroException
extends LogicException
{
}

16. Example (throw)
class Mathematics
{
public function divide($a, $b)
{
if ($b == 0) {
$msg = sprintf(‘Tried to divide %s by zero”, $a);
throw new DivisionByZeroException($msg);
}
return ($a / $b);
}
}

17. Example (catch)
$maths = new Mathematics();
try
{
$result = $maths->divide(5, 0);
}
catch (DivisionByZeroException $exception)
{
$logger->warning($exception>getMessage());
}

18. Logging

20. What is “logging”?
Keeping a record of all events...
exceptions, errors, warnings, info, debug

21. Why use logging?
●
●
●
●
Easier to find problems
More detail
“paper trail” for code
Log where you want

22. What about Apache’s error_log?
source: http://up-ship.com/blog/?p=20903

23. Why?
● error_log is too basic
● Reading / parsing
● error_reporting (again)

24. Doin’ it right wrong…
/************************************************************
*
Magic file that makes your entire project work perfectly
*
************************************************************/
@ini_set('display_errors', 0);
@error_reporting(0);
function __globalErrorHandler()
{
return true;
}
@set_error_handler('__globalErrorHandler');
@set_exception_handler('__globalErrorHandler');
@register_shutdown_function(function() {
if(error_get_last())
{
echo "Script executed successfully!";
}
});
https://github.com/webarto/boostrap.php

25. Requirements (for everyone)
●
●
●
●
Fire & forget
Minimum or zero latency
Highly available
Log everything:
○
○
○
○
Exceptions
Errors
Fatal Errors
Debug & info
● PSR-3 compatible

26. PSR-3
● Common logging interface
→ LoggerInterface
● RFC-5424 Levels
(debug, info, notice, warning, error, critical, alert, emergency)
● Interoperability
● Reusability

27. Logging Options
●
●
●
●
●
●
●
monolog (PSR-3)
Drupal - PSR-3 Watchdog
phpconsole
log4php
RavenPHP + Sentry
FirePHP (dev environment)
Roll your own

28. How they work...
source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html

29. Capturing Logging
Use “capture methods”, send to $logger
● set_exception_handler()
○ Handles all uncaught exceptions
● set_error_handler()
○ Handles most errors
● register_shutdown_function()
○ Handles fatal errors

30. Sending Log Messages
● Handler/Adapter translates
● However you want…
● Monolog has loads:
○
○
○
○
syslog-compatible / error_log
Email, HipChat
AMQP, Sentry, Zend Monitor, Graylog2
Redis, MongoDB, CouchDB

31. Typical PSR-3 Compatible Design
Capture Method
Logger (PSR-3)
Handler / Adapter
Data Storage

32. Monolog
MonologErrorHandler
->handleException()
MonologLogger
->log()
MonologHandler
->handle()

33. Low Latency
High Performance

34. What do I mean?
● Easy to add
● Fire & Forget
● Minimum impact to request

35. Slow Logging
Browser
Application
Log Server
HTTP request
Error!
Send log message to database
Acknowledge message
HTTP response to client

36. Zero Latency Logging (ideal)
Browser
Application
Log Server
HTTP request
Error!
HTTP response to client
Send log message to database

37. What about UDP?
● Yes… Zero latency, but….
● Do you even care about your logs?
(UDP means log messages may get lost...)
● TCP means guaranteed network delivery
● Any non-blocking fails

38. Low Latency Logging (balance)
Browser
Application
Log Server
HTTP request
Error!
HTTP response to client
Send log message to database

39. So how?

40. Say hello to RabbitMQ

41. What is RabbitMQ?
●
●
●
●
Robust messaging for applications
Easy to use
Runs on all major operating systems
Supports a huge number of developer
platforms
● Open source and commercially supported
www.rabbitmq.com

42. RabbitMQ - Basic
queue
“Consumer”
“Publisher”
source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html

43. RabbitMQ - Exchanges
“Publisher”
“Publisher”
queue 1
“Consumer 1”
“Publisher”
“Exchange”
queue 2
“Consumer 2”
“Publisher”
“Publisher”
source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html

44. Using Queues === Fast!
Add RabbitMQ to logging architecture...

45. Low Latency (using AMQP)
EdLogHandlerErrorHandler
->handleException()
EdLogLogger
->log()
EdLogPublisherAmqpPublisher
->publish()
RabbitMQ
Logging Server
JSON payload

46. Low Latency Logging (with AMQP)
Browser
Application
RabbitMQ
Log Server
HTTP request
Error!
JSON via AMQP
HTTP response
Fetch message

47. Why bother?
● Scalability
Log Worker
Application A
Log Worker
Application B
RabbitMQ
Log Worker
Application C
Log Worker

48. Single Point of Failure...
● RabbitMQ can do HA
RabbitMQ
Node 6
RabbitMQ
Node 1
RabbitMQ
Node 2
RabbitMQ
Node 5
RabbitMQ
Node 3
RabbitMQ
Node 4

49. Live demo
(pre-recorded… meh)

50. Questions?

51. Feedback please...
https://joind.in/9928

52. Thanks!
https://joind.in/9928
@asgrim
github.com/asgrim