1H.com                         1H.com                                       Benchmarking the         Efficien...
             AGENDA                         <ul>                                          <li>No benchmarking  </li>      ...
             $  $$  $$$  $$  $                                If you rate my survey, I'll hook you up with $20 cPCache $$$...
             Tools of trade                         <ul>                                          <li>time </li>          ...
             Before benchmarking                         <ul>                 <li>Perl interpreter and its environment </l...
             Before benchmarking                         @INC:        /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-mul...
             Before benchmarking                         <ul>                 <li>Perl interpreter and its environment </l...
             Before benchmarking                         <ul>                                          <li>The effect of p...
             After benchmarking                         <ul>                 <li>The five major factors: </li>         <ul...
             Understanding the data                         <ul>                 <li>before optimization </li>            ...
             file-bench-module.pl                         #!/usr/bin/perl        use strict;        use warnings;        u...
             file-bench-plain.pl                         #!/usr/bin/perl        use warnings;        use strict;          ...
             file-bench-sub.pl                         #!/usr/bin/perl        use warnings; use strict;        my $dir = '...
             Using a module or our own function                         <ul>                 <li>File::Util </li>         ...
             Reading the /proc                         <ul>                 <li>File::Util </li>               </ul>      ...
             Reading of data from files                         <ul>                                                   <li...
             A module load times                         <ul>                 <li>File::Util </li>         <li>real 0.084 ...
             A module load times                         $ perl -d:Dependencies -MFile::Util -e 0        Devel::Dependenci...
                                     Profile your apps                                    Devel::NYTProf
             <ul>                 Common misconceptions                </ul>                        <ul>                  ...
             Conclusions                         <ul>                                          <li>Premature optimizations...
             Questions                           ?  ?  ?  ?        ?  ?  ?  ?    ?  ?        ?  ? ?  ? ?  ?  ?        ?   ...
                    Thank you               Please visit us at Booth 23                         Marian Marinov - mm@1h.com...
Upcoming SlideShare
Loading in...5
×

Benchmarking the Efficiency of Your Tools

300

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
300
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • \n \n \n \n \n \n
  • \n \n \n \n This talk was inspired by a talk from Tim Bunce who wrote the NYTProf \n \n I want to share with you, the things I haven&apos;t seen in other presentations on the subject. \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n For me the most important tools are time, Devel::NYTProf and Devel::Dependencies \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n \n The interpreter has different compile options which makes it load slower or faster. Linking to more or less libraries. \n \n So this talk is mainly about how to identify what is important \n \n \n \n \n
  • \n \n \n \n \n Shared resources: \n \n \n \n \n shared memory \n \n \n locks \n \n \n files \n \n \n databases \n \n \n \n \n \n \n
  • \n \n \n \n \n I/O usage is one of the most misleading factor because on live environments the I/O profile is completely different then on the development and testing systems \n Deployment environment – \n how often the app is executed \n what data it handles \n what is the average load of the machines without running our app \n \n \n \n
  • \n \n \n \n \n What most ppl forget about this is that the number of executions is not the same and most time they don&apos;t take this into account. \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n Most of the ppl will fail to understand the meaning of REALTIME. It is the SUM of USERTIME, SYSTIME and the time it had to wait for other processes to be executed or wait for resources. \n \n Note the difference between the results on beast and on the vm which is running on beast. \n \n \n
  • \n \n \n \n Note here how there is almost no difference between the beast and its VM here \n \n Different filesystems affect the performance in different ways. \n \n \n
  • \n \n \n \n \n \n
  • \n \n \n \n \n \n
  • \n \n \n \n 2005 - Jean-Louis Leroy – A Timely Start \n \n \n
  • \n \n \n \n NYTProf shows you the number of calls and the time \n \n inclusive time – includes the time executing the actual sub \n exclusive time – only the time executing the statement \n \n \n
  • \n \n \n \n \n \n \n \n \n OOP \n \n \n Good for code maintainability \n \n \n Its actually killing performance \n \n \n \n \n Functional \n \n \n Better for performance if not overused \n \n \n Easily lures you in performance traps \n \n \n \n \n Better load time may mean \n \n \n worse CPU or Memory usage \n \n \n \n \n \n \n \n \n
  • \n \n \n \n Premature optimizations are infecting all of us \n \n Wrongly implemented algorithms or usage of things like for/foreach when you can use while are clear performance hogs. \n \n \n
  • \n \n \n \n \n
  • \n \n \n \n \n
  • Benchmarking the Efficiency of Your Tools

    1. 1. 1H.com 1H.com Benchmarking the Efficiency of Your Tools Marian Marinov - mm@1h.com Co-founder & CEO of 1H Ltd.
    2. 2. AGENDA <ul> <li>No benchmarking </li> <ul> <ul> <li>you already know Benchmark.pm </li> </ul> </ul> <li>What we should consider </li> <ul> <ul> <li>before and after benchmarks </li> </ul> </ul> <li>Some common misconceptions </li> <li>Identify places where we need to optimize </li> </ul>
    3. 3. $ $$ $$$ $$ $ If you rate my survey, I'll hook you up with $20 cPCache $$$. Go to this address to take the survey: http://go.cpanel.net/b11 and come up to the podium once you've completed it.
    4. 4. Tools of trade <ul> <li>time </li> <li>Benchmark.pm </li> <li>Devel::SmallProf </li> <li>Devel::FastProf </li> <li>Devel::NYTProf </li> <li>DBI::Profiler </li> <li>Devel::Dependencies </li> </ul>
    5. 5. Before benchmarking <ul> <li>Perl interpreter and its environment </li> <li>Application load time </li> <ul> <li>Interpreter </li> <li>Modules </li> </ul> <li>Perl does not release memory </li> <li>Try to test only the suspected code </li> <li>How to profile a big web application </li> <ul> <li>HTTP::Server::Simple::* </li> </ul> <li>How to identify what is important </li> </ul>
    6. 6. Before benchmarking @INC: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 ................
    7. 7. Before benchmarking <ul> <li>Perl interpreter and its environment </li> <li>Application load time </li> <ul> <li>Interpreter </li> <li>Modules </li> </ul> <li>Perl does not release memory </li> <li>Try to test only the suspected code </li> <li>How to profile a big web application </li> <ul> <li>HTTP::Server::Simple::* </li> </ul> <li>How to identify what is important </li> </ul>
    8. 8. Before benchmarking <ul> <li>The effect of profiling/benchmarking </li> <ul> <li>the code will run slower </li> <li>it will use more memory </li> </ul> <li>Parallel or single execution </li> <li>Competing for shared resources </li> <li>Using external programms </li> </ul>
    9. 9. After benchmarking <ul> <li>The five major factors: </li> <ul> <li>CPU usage </li> <li>Memory usage </li> <li>I/O usage </li> <li>Time spent waiting </li> <li>Deployment environment </li> </ul> <li>Never look only at a single factor! </li> <li>!!! Always consider them together !!! </li> </ul>
    10. 10. Understanding the data <ul> <li>before optimization </li> </ul> 332340 1.82s analyze_proc::CORE:match <ul> <li>after optimization </li> </ul> 1. 332337 1.52s analyze_proc::CORE:match 2. 312320 1.52s analyze_proc::CORE:match <ul> <li>actual performance increase </li> <li>1. 0.30s gain </li> <li>2. 0.11s gain </li> </ul>
    11. 11. file-bench-module.pl #!/usr/bin/perl use strict; use warnings; use File::Util; my $dir = '/home/hackman/bench-tests'; my $f = File::Util->new(); foreach my $obj ($f->list_dir($dir)) { print &quot;$objn&quot;; }
    12. 12. file-bench-plain.pl #!/usr/bin/perl use warnings; use strict; my $dir = '/home/hackman/bench-tests'; opendir B, $dir; while (my $obj = readdir(B)) { print &quot;$objn&quot;; } closedir B;
    13. 13. file-bench-sub.pl #!/usr/bin/perl use warnings; use strict; my $dir = '/home/hackman/bench-tests'; sub check_dir { my $dir = shift; opendir B, $dir; while (my $obj = readdir(B)) { print &quot;$objn&quot;; } closedir B; } check_dir($dir);
    14. 14. Using a module or our own function <ul> <li>File::Util </li> </ul> real 0.067 user 0.059 sys 0.008 - hotstare real 0.030 user 0.017 sys 0.013 - beast real 0.013 user 0.007 sys 0.002 - vm on beast real 0.031 user 0.020 sys 0.005 - remote to beast <ul> <li>Using a sub </li> </ul> real 0.019 user 0.015 sys 0.004 - hotstare real 0.006 user 0.003 sys 0.003 - beast real 0.004 user 0.002 sys 0.001 - vm on beast real 0.012 user 0.004 sys 0.003 - remote to beast <ul> <li>Plain opendir </li> </ul> real 0.019 user 0.015 sys 0.004 - hotstare real 0.005 user 0.003 sys 0.002 - beast real 0.004 user 0.002 sys 0.001 - vm on beast real 0.011 user 0.004 sys 0.003 - remote to beast
    15. 15. Reading the /proc <ul> <li>File::Util </li> </ul> real 0.066 user 0.058 sys 0.008 - hotstare real 0.013 user 0.010 sys 0.003 - beast real 0.013 user 0.008 sys 0.002 - vm on beast <ul> <li>Using a sub </li> </ul> real 0.019 user 0.015 sys 0.004 - hotstare real 0.006 user 0.003 sys 0.003 - beast real 0.006 user 0.002 sys 0.001 - vm on beast <ul> <li>Plain opendir </li> </ul> real 0.018 user 0.014 sys 0.004 - hotstare real 0.005 user 0.003 sys 0.002 - beast real 0.004 user 0.002 sys 0.001 - vm on beast <ul> </ul>
    16. 16. Reading of data from files <ul> <li>Hot and cold caches </li> <li>Common files and directories </li> <li>the /proc file system </li> <li>files on NFS or other shared storage </li> </ul>
    17. 17. A module load times <ul> <li>File::Util </li> <li>real 0.084 user 0.071 sys 0.013 </li> <li>Load times </li> <li>real 0.060 user 0.054 sys 0.006 </li> <li>Time spent in the code </li> <li>real 0.024 user 0.017 sys 0.007 </li> </ul> $ for i in {1..1000}; do time perl -MFile::Util -e 0; done > results 2>&1 $ awk 'simple counters here' results Average real: 0.060 user: 0.054 sys: 0.006
    18. 18. A module load times $ perl -d:Dependencies -MFile::Util -e 0 Devel::Dependencies finds 10 dependencies: /usr/lib/perl5/site_perl/5.8.8/auto/File/Util/autosplit.ix AutoLoader.pm Class/OOorNO.pm Exporter/Heavy.pm Fcntl.pm File/Util.pm XSLoader.pm constant.pm vars.pm warnings/register.pm No DB::DB routine defined at -e line 1.
    19. 19. Profile your apps Devel::NYTProf
    20. 20. <ul> Common misconceptions </ul> <ul> <li>Object-oriented programming </li> <li>Functional programming </li> <li>Better load time is always good </li> <ul> </ul> </ul>
    21. 21. Conclusions <ul> <li>Premature optimizations are a spreading disease </li> <li>Understanding the numbers and the surroundings is a must </li> <li>Using Perl the wrong way will cost you </li> <li>You always trade flexibility and maintainability for performance </li> </ul>
    22. 22. Questions ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
    23. 23. Thank you Please visit us at Booth 23 Marian Marinov - mm@1h.com Co-founder & CIO at 1H Ltd. 1H.com 1H.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×