Low Latency Logging
at BrightonPHP

James Titcumb
@asgrim
github.com/asgrim

● Started on Amiga 500
● PHP for 11 years
● ZCE PHP 5.3
●
●
●
●
●
Run PHP Hampshire user group
Lead dev on browscap.ini
First open source GoDeploy
Development Manager at Protected.co.uk
Mostly a ZF2 developer
@asgrim
github.com/asgrim

Hello! :)
@asgrim
github.com/asgrim

Errors

What are errors?
● Something broke… :)
● e.g.
○ Can’t connect to MySQL (mysqli_connect)
○ No such file or directory (fopen)
● Usually PHP core
● Sometimes “fatal”

Problems?
source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/

Problems.
●
●
●
●
●
error_reporting setting
Errors look ugly
“@”
Limited options
Not very “OO”

Ways Around
// Will raise E_NOTICE
fopen($somefile, 'r');

Ways Around
// No error! :)
if (file_exists($somefile)) {
fopen($somefile, 'r');
} else {
// nice handling...
// maybe throw exception...
}

Exceptions

Jargon Buster
● throw
Triggering
● try
Try to run
● catch
Handle it
● finally
Run code after try/catch

What are exceptions?
●
●
●
●
●
Something still broke
OO
Catchable
Fatal errors
They are classes

SPL Exceptions
● Built in to PHP
● More descriptive than just “Exception”, e.g.:
○
○
○
○
○
InvalidArgumentException
LogicException
OutOfBoundsException
RuntimeException
see PHP manual for more

Example (exception class)
class DivisionByZeroException
extends LogicException
{
}

Example (throw)
class Mathematics
{
public function divide($a, $b)
{
if ($b == 0) {
$msg = sprintf(‘Tried to divide %s by zero”, $a);
throw new DivisionByZeroException($msg);
}
return ($a / $b);
}
}

Example (catch)
$maths = new Mathematics();
try
{
$result = $maths->divide(5, 0);
}
catch (DivisionByZeroException $exception)
{
$logger->warning($exception>getMessage());
}

Logging

What is “logging”?
Keeping a record of all events...
exceptions, errors, warnings, info, debug

Why use logging?
●
●
●
●
Easier to find problems
More detail
“paper trail” for code
Log where you want

What about Apache’s error_log?
source: http://up-ship.com/blog/?p=20903

Why?
● error_log is too basic
● Reading / parsing
● error_reporting (again)

Doin’ it right wrong…
/************************************************************
*
Magic file that makes your entire project work perfectly
*
************************************************************/
@ini_set('display_errors', 0);
@error_reporting(0);
function __globalErrorHandler()
{
return true;
}
@set_error_handler('__globalErrorHandler');
@set_exception_handler('__globalErrorHandler');
@register_shutdown_function(function() {
if(error_get_last())
{
echo "Script executed successfully!";
}
});
https://github.com/webarto/boostrap.php

Requirements (for everyone)
●
●
●
●
Fire & forget
Minimum or zero latency
Highly available
Log everything:
○
○
○
○
Exceptions
Errors
Fatal Errors
Debug & info
● PSR-3 compatible

PSR-3
● Common logging interface
→ LoggerInterface
● RFC-5424 Levels
(debug, info, notice, warning, error, critical, alert, emergency)
● Interoperability
● Reusability

Logging Options
●
●
●
●
●
●
●
monolog (PSR-3)
Drupal - PSR-3 Watchdog
phpconsole
log4php
RavenPHP + Sentry
FirePHP (dev environment)
Roll your own

How they work...
source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html

Capturing Logging
Use “capture methods”, send to $logger
● set_exception_handler()
○ Handles all uncaught exceptions
● set_error_handler()
○ Handles most errors
● register_shutdown_function()
○ Handles fatal errors

Sending Log Messages
● Handler/Adapter translates
● However you want…
● Monolog has loads:
○
○
○
○
syslog-compatible / error_log
Email, HipChat
AMQP, Sentry, Zend Monitor, Graylog2
Redis, MongoDB, CouchDB

Typical PSR-3 Compatible Design
Capture Method
Logger (PSR-3)
Handler / Adapter
Data Storage

Monolog
MonologErrorHandler
->handleException()
MonologLogger
->log()
MonologHandler
->handle()

Low Latency
High Performance

What do I mean?
● Easy to add
● Fire & Forget
● Minimum impact to request

Slow Logging
Browser
Application
Log Server
HTTP request
Error!
Send log message to database
Acknowledge message
HTTP response to client

Zero Latency Logging (ideal)
Browser
Application
Log Server
HTTP request
Error!
HTTP response to client
Send log message to database

What about UDP?
● Yes… Zero latency, but….
● Do you even care about your logs?
(UDP means log messages may get lost...)
● TCP means guaranteed network delivery
● Any non-blocking fails

Low Latency Logging (balance)
Browser
Application
Log Server
HTTP request
Error!
HTTP response to client
Send log message to database

So how?

Say hello to RabbitMQ

What is RabbitMQ?
●
●
●
●
Robust messaging for applications
Easy to use
Runs on all major operating systems
Supports a huge number of developer
platforms
● Open source and commercially supported
www.rabbitmq.com

RabbitMQ - Basic
queue
“Consumer”
“Publisher”
source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html

RabbitMQ - Exchanges
“Publisher”
“Publisher”
queue 1
“Consumer 1”
“Publisher”
“Exchange”
queue 2
“Consumer 2”
“Publisher”
“Publisher”
source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html

Using Queues === Fast!
Add RabbitMQ to logging architecture...

Low Latency (using AMQP)
EdLogHandlerErrorHandler
->handleException()
EdLogLogger
->log()
EdLogPublisherAmqpPublisher
->publish()
RabbitMQ
Logging Server
JSON payload

Low Latency Logging (with AMQP)
Browser
Application
RabbitMQ
Log Server
HTTP request
Error!
JSON via AMQP
HTTP response
Fetch message

Why bother?
● Scalability
Log Worker
Application A
Log Worker
Application B
RabbitMQ
Log Worker
Application C
Log Worker

Single Point of Failure...
● RabbitMQ can do HA
RabbitMQ
Node 6
RabbitMQ
Node 1
RabbitMQ
Node 2
RabbitMQ
Node 5
RabbitMQ
Node 3
RabbitMQ
Node 4

Live demo
(pre-recorded… meh)

Questions?

Feedback please...
https://joind.in/9928

Thanks!
https://joind.in/9928
@asgrim
github.com/asgrim