DashProfiler 200807

DashProﬁler Lightweight Code Instrumentation Tim.Bunce@pobox.com - July 2008

A Problem A web application ~100K lines of code Using many external services If response time goes up... what’s causing it?

A Problem A web application ~100K lines of code Using many external services If response time goes up... what’s causing it? Continuous monitoring in production Must have very low CPU and I/O cost Minimal code changes

A Typical Approach package MyNetIO; sub send_request { my ($hostname, $request) = @_ ...send to $hostname... } How much time was spent sending the request?

A Typical Approach package MyNetIO; use Time::Hires qw(time); sub send_request { my ($hostname, $request) = @_ my $start = time(); ...send to $hostname... $durations->{MyNetIO}{$hostname} = time() - $start; }

A Typical Approach package MyNetIO; use Time::Hires qw(time); sub send_request { my ($hostname, $request) = @_ my $start = time(); ...send to $hostname... $durations->{MyNetIO}{$hostname} = time() - $start; } • Doesn’t record count so can’t produce averages.

A Typical Approach package MyNetIO; use Time::Hires qw(time); sub send_request { my ($hostname, $request) = @_ my $start = time(); ...send to $hostname... $durations->{MyNetIO}{$hostname} = time() - $start; } • Doesn’t record count so can’t produce averages. • Two lines of code. Worse if multiple return statements.

A Typical Approach package MyNetIO; use Time::Hires qw(time); sub send_request { my ($hostname, $request) = @_ my $start = time(); ...send to $hostname... $durations->{MyNetIO}{$hostname} = time() - $start; } • Doesn’t record count so can’t produce averages. • Two lines of code. Worse if multiple return statements. • Doesn’t record time if function exits via an exception.

A Solution: DashProﬁler Simple Flexible Lightweight

DashProﬁler • Can group samples into granular time units • Can measure exclusive time in a period • Can ﬂush to disk at intervals • Just needs one line of code per sample

DashProﬁler Internals Built on DBI::Proﬁle, part of the DBI Aggregates measurements into a data tree Two-level tree by default: $root->{ $key1 }->{ $key2 }->[ ...leaf node... ] $root->{ ‘MyNetIO’ }->{ $hostname }->[ ...leaf node... ]

DashProﬁler Data Each leaf node in the tree is a reference to an array: $root->{ $key1 }->{ $key2 } = [ 106, # 0: count of samples at this node 0.0312958955764771, # 1: total duration 0.000490069389343262, # 2: first duration 0.000176072120666504, # 3: shortest duration 0.00140702724456787, # 4: longest duration 1023115819.83019, # 5: time of first sample 1023115819.86576, # 6: time of last sample ]

DashProﬁler By-Time Optional extra time level in the data tree $time = int(time() / $granularity) * $granularity; $root->{ $time }->{ ‘MyNetIO’ }->{ $hostname }->[ ... ] So a new sub-tree is grown each granularity seconds

DashProﬁler Conﬁg use DashProfiler; DashProfiler->add_profile( foo => { } ); DashProfiler->add_profile( foo => { granularity => 10, flush_interval => 600, flush_hook => sub { ... }, sample_class => ‘DashProfiler::Sample’, dbi_profile_class => ‘DBI::Profile’, period_exclusive => ..., period_summary => ..., ... });

Without DashProﬁler package MyNetIO; use Time::Hires qw(time); sub send_request { my ($hostname, $request) = @_ my $start = time(); ...send to $hostname... $durations->{MyNetIO}{$hostname} = time() - $start; }

Without DashProﬁler package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ); ...send to $hostname... }

With DashProﬁler package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ); ...send to $hostname... } Duration is measured when $sample goes out of scope

With DashProﬁler Name of proﬁle created with add_profile() package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ); ...send to $hostname... } Duration is measured when $sample goes out of scope

With DashProﬁler Name of proﬁle created with add_profile() Value to use for ‘key1’ package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ); ...send to $hostname... } Duration is measured when $sample goes out of scope

With DashProﬁler Name of proﬁle created with add_profile() Value to use for ‘key1’ package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { Value to use for ‘key2’ my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ); ...send to $hostname... } Duration is measured when $sample goes out of scope

With DashProﬁler package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ) if foo_profiler_enabled(); ...send to $hostname... }

With DashProﬁler package MyNetIO; use DashProfiler::Import foo_profiler => [ ‘MyNetIO’ ]; sub send_request { my ($hostname, $request) = @_ my $sample = foo_profiler( $hostname ) if foo_profiler_enabled(); ...send to $hostname... Automatically imported compile-time constant } reduces cost to zero if proﬁle is disabled

DashProfiler Flush Data is written to STDERR on exit, by default Regular flushing is enabled by specifying a flush_interval The dbi_profile_class handles the flush. Choices include: DBI::Profile DBI::ProfileData DBI::ProfileData::Apache DashProfiler->add_profile( foo => { ..., flush_interval => 600, dbi_profile_class => ‘DBI::ProfileData’, flush_hook => sub { ... }, ... });

DashProﬁler Periods • Group samples into periods - e.g. http request to response - start_sample_period() and end_sample_period() - counted, to enable averages and totals per period - can output period counts instead of sample counts • Measure ‘exclusive’ time - time from period start to end that’s not been accounted for by other samples - enabled via period_exclusive option

Example Data Average response times over 24 hours DashProﬁler doesn’t generate graphs itself, but the data can be used to create graphs like these

Example Data Worst case response times over 24 hours

DashProfiler Perspectives • Each DashProfiler can have multiple DBI Profile objects attached • Samples accumulate in all attached profiles • Each profile can have a different Path • giving different ‘perspectives’ or level of detail - key1 + key2 - key1 + country + browser type - key2 + browser type - ... etc.

DashProﬁler Per-Period • Optional extra ‘per-period’ DBI proﬁle • Enabled via period_summary option • Automatically attached and reset by start_sample_period() • Gives current totals for this period • Great for ‘debug footers’ on web page showing how much time was spent generating this page

DashProﬁler Cost

DashProﬁler Cost Time cost of taking a sample: 0.000022s

Questions?

DashProfiler 200807

by Tim Bunce on Sep 10, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

DashProfiler 200807 — Presentation Transcript