Application Diagnosis with Zend Server TracingKevin SchroederZend Technologies

About meKevin SchroederConsultant for ZendProgrammerSys AdminAuthorEtc.Me!

"As soon as we started programming, we found to our surprise that it wasn't as easy to get programs right as we had thought. Debugging had to be discovered. I can remember the exact instant when I realized that a large part of my life from then on was going to spent in finding mistakes in my own programs" - Maurice Wilkes, 1949

Why do we need diagnostic tools?

Let’s see that again in slow motion

The pointNOT to highlight an individual companyWhy not?How many web-based applications look like this when an error occurs?AnswerAll of themWeb requests are intended to be short and quickIf an error occurs, recover, quit and go on to the next request

Debugging the old wayvar_dump($var)(I still use this occasionally)Some obvious problemsWhat if you forget to remove it?Are you doing this in a production environment? If you are …Very limited scope

Debugging the better way – ZS DebuggingYou can see what the variable values are at run time and modify them tooGood for developmentNot good for production

MonitoringMUCH better for production systemsYou get a lot of really good information to help replay the issue

MonitoringYou are able to debug or profile the event on a remote machine

Events that can be monitored

Instrumentation on the next levelZend Code Tracing

Code Tracing: Rapid Problem ResolutionSimilar in concept to a black box flight recorder - Eliminates the need to reproduce problems in order to find root causeActivated automatically (through rules) or manuallyRecords live application execution Function callsArguments and return valuesErrorsDurationUseful in all parts of the software development lifecycle

Practical Code Tracing FunctionalityBy capturing PHP applications code in real time:Code Tracing enables to capture problems when they occur.Code Tracing recorded data eliminate the need to reproduce the steps led up to a failure.Code Tracing enables to analyze PHP applications execution both in production and in test lab environments, by that, it enables to quickly find the root cause of problems.14Insert->Header & Footer

Supported WorkflowsManual workflow: Used while performing unit tests during development or when running functional tests in QA. Calling URLs as part of the testing processMonitoring events workflowUsed when running automated tests and load tests and when running your application in staging or production environments. Code tracing extends the monitoring rules mechanism by enabling to save trace data when a monitoring event is generated. Suitable for production in Standby mode

Detailed Runtime InstrumentationPinpoint root cause by viewing the sequence of events that led up to an error

Detailed Runtime InstrumentationOptimize your code by analyzing application performance at the function level

Code Tracing – Monitoring Level ModesAlways onCollects trace data when monitoring event is kicked offCollects trace data for manual requests.InactiveCollects data only for manual requests.StandbyOff (turn code tracing extension off)

Code Tracing – Operating ModesStandby modeA monitoring event activates code tracingTrace mode:OnStandbyTracing switches back for “Standby” modeTracing is activated for X minutesTimeA monitoring event with an “Activate Trace” action is generatedCollect trace data according to the monitoring set of rules

The basicsTrace a Request

Interpreting the resultsScreenshots are boring. Let’s look at live data

SetupAttaching traces to events Tracing can be attached to any PHP-specific events

Tracing Events – Standby ModeSet zend_monitor.event_tracing_modeActive, Inactive or StandbyDetermines integration with tracer mechanism in case of events that require a trace data dumpCreate an Event Action – “Enable Tracing” will change

Create an Event Action – “Enable Tracing” will change

Tracing Process SignalsTracing can also be initiated upon a fatal signal

Tracing LogStep 1 – Make sure that you have it set to logStep 2 – Set the verbosity between 1 and 5Restart

Example log file[Tracing Log 08.10.2009 14:14:56 DBG1] 5387 Tracer request startup [Tracing Log 08.10.2009 14:14:56 DBG3] 5387 Setting dump reason to 0 [Tracing Log 08.10.2009 14:14:56 DBG3] 5387 Dump reason is now 0 [Tracing Log 08.10.2009 14:14:56 DBG1] 5387 Remote address is 127.0.0.1 [Tracing Log 08.10.2009 14:14:56 DBG2] 5387 Allow hostmask is 127.0.0.1/32,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 [Tracing Log 08.10.2009 14:14:56 DBG2] 5387 Deny hostmask is [Tracing Log 08.10.2009 14:14:56 DBG1] 5387 Remote access allowed [Tracing Log 08.10.2009 14:14:56 DBG1] 5387 Request parameter requires dump [Tracing Log 08.10.2009 14:14:56 DBG3] 5387 Setting dump reason to 2 [Tracing Log 08.10.2009 14:14:56 NOTICE] 5387 Installing signal handler [Tracing Log 08.10.2009 14:14:56 DBG2] 5387 Staring trace generation [Tracing Log 08.10.2009 14:14:56 DBG2] 5387 Tracer request startup done [Tracing Log 08.10.2009 14:14:56 DBG3] 5387 Options updated to 7FFF [Tracing Log 08.10.2009 14:14:57 DBG1] 5387 Tracer request shutdown [Tracing Log 08.10.2009 14:14:57 WARNING] 5387 Code tracing buffer full [Tracing Log 08.10.2009 14:14:57 DBG1] Using socket /usr/local/zend/tmp/monitor_events.sock rules /usr/local/zend/etc/events_rules.xml [Tracing Log 08.10.2009 14:14:58 NOTICE] Event type 1 (0) reported [Tracing Log 08.10.2009 14:14:58 NOTICE] 5387 Generating trace dump, format 2 [Tracing Log 08.10.2009 14:14:58 DBG2] 5387 Dump name is /usr/local/zend/var/codetracing/dump.5387.1 Monitoring Event was triggered

Functional ExamplesUncaught exceptionsDestructor being calledPrepared Statements

Where did all my memory go?Memory usageLarge variable going out of scope

Impact

The all-important question…“Where should I use this?”Development – Possibly, though profiling and debugging may be more pertinentTesting – AbsolutelyStaging – AbsolutelyProduction - AbsolutelyIn Standby modeActivated on a subset of servers

Questions?Want to join in the public beta? Go to http://www.zend.com/en/products/server/zend-server-5-newfor more information kevin@zend.com

Additional Information

What happens when you have too much dataNeed to up the zend_codetracing.buffer_size setting

Increasing the buffer size to 5MBThat’s better

Settings

Settings

Settings

Settings