A Challenging Craft
That‘s me
 My day to day job is to help customers
 fixing software performance

                                                  Fabian
 People really struggle with performance
 But most of them struggle with coding as well



                   If creating good software is a craft,
                    performance should be one as well

  Customer
 Some say:
 „creating software is art“

 Art and software development
 need creativity
                                 Art
 Fixing performance is
 often considered magic

 Magic is something
   only you understand
   you do in a hard to
                          Magic
    follow fashion
 Correctness in detail


 Research and verification




                              Craft
 All types share
   Passion
   Learning
   Experience



                    Which way?
 Value == Money


 We need a business case
 for performance

                                   Money
 What is the impact of     (in parts of the world)
 bad performance?
 Amazon measured the
 impact of 100ms delay

 Sales dropped by 1%


 In a year that would be   Typical (?) retailer
 245 Million USD
 Instable or slow software
 delays time to market

 Slow software is no longer
 accepted by customers
                               New?
 Insurances like to send
 paper via mail

 Not meeting deadlines can
 cause
   Legal issues
                              Snail Mail
   Canceled contracts
   Loss of money
Is A Craft
 Calculate Execution Time
   Code x = 5ms
   Code y = 2*x = 10ms


 Know code in advance
 Waterfall approach                    Performance
                                        Engineering


 Proving the performance of software
 is more difficult than proving the correct function
 We prove functional
 correctness with automated
 tests

 High coverage

                                          Look closely


 Run examples and see if they are fast
 Done late in project


 If done at all
                               How much load
                              breaks the system?

 No chance to fix anything
 Avoid human errors


 Require machine decidable
 fail / pass check

 What is the measure?        42cm are fast?
 Functionality is independent
 of the environment

 Performance characteristics
 can vary
   Unusable slow
                                 Our Environment
   Lightning fast
 Underpowered hardware


 Loaded with tools
 and stuff

                                Developers
 Luckily not the production   driving fast
 environment
 More power


 But also more load


 How much faster is
 production than development?    Crawling Production


 Any estimation on how much better or worse the
 environments are is incorrect
 Real performance tests
 need real systems

 Test in production


 Clone production         Stop playing
 infrastructure
 Amount of data is
 unpredictable

 Application usage is
 unpredictable
                                     Tweets per second


 How thought of using Twitter for build notification?
Dev             Prod
 Test Test1     Fabian Lange
 Test Test2     Uwe Friedrichsen
 Test Test3     Mirko Novakovic
 Showing 3/3    Showing 3/6,434,867
 Syntethic load tests are
  unrealistic

 No application has hundreds
  of users doing the same
  procedure again and again
                                         Load Baselines


 Understanding real load is difficult
 Real usage cannot
 be generated

 Real usage can be
 captured & replayed
                       Live Systems are live
 Be careful 
 Continous performance tests


 Close to real setup
                                App Monitor

 Observe production behavior


 Fix issues fast
 Conflicting interests
   Development: Change
   Operations: Stability

                            Another Movement
 Need to work together
Is A Challenge
 Can‘t we do anything before
 production?

 We want to deliver something,   Let users test?
 which works perfectly!
 Optimizations might have
 no impact

 Micro-Optimizations are    Missed Target
 dangerous
 Soft Measure


 Works good for code quality
                                          Sonar



 Are there performance best practices?
 Yahoo Best Practices


 Google Best Practices


 Plenty of tools
                          Good Waterfalls

 Work well
 Naive implementation looks fine




 But is not multithreaded
 This is threadsafe




 But slow
 This is correct




 Correct synchronisation is hard
 Check Log Level (Ugly)




 Check Log Level (Nicer)
 Static SimpleDateFormat is wrong




 Working with Dates
 and Calendars is
 very expensive!
 Some people misuse it as loop




 Results are unexpected behavior or slow execution
 Analyzes Java Bytecode


 Knows 58 Performance Bugs


 Most are rather trivial
                              Indeed finds bugs
 Detecting deadlocks
 is difficult

 Many thesis's on deadlock
 detection
                              Verifies Java
 Few code
 Hidden Gem


 Tries to cause Deadlocks



                                   IBM
                             Support Assistant
 We need more and reliable
   Code Performance Metrics
   Best Practices
   Tools
Is A Challenging Craft
Q&A
   Art                                                  YSlow
    flickr.com/photos/37202375@N03/3490107873             developer.yahoo.com/yslow
   Magic                                                Pagespeed
    flickr.com/photos/princes_milady/4238912548           code.google.com/speed/page-speed
   Crossroads                                           Java Locking
    flickr.com/photos/40481200@N06/4857965280             en.wikipedia.org/wiki/Double-checked_locking
   Envelopes                                            PHP Ternary Operator
    flickr.com/photos/creatingmotherhood/3663914542       fabien.potencier.org/article/48/the-php-ternary-
                                                          operator-fast-or-not
   Test
    flickr.com/photos/cayusa/2159980025                  JavaScript for .. In
                                                          stackoverflow.com/questions/500504/javascript-for-
   Scales                                                in-with-arrays
    flickr.com/photos/reinvented/309512320
                                                         Findbugs
   Tape Measure                                          findbugs.sourceforge.net
    flickr.com/photos/hippie/2466583497
                                                         Jlint
   World                                                 jlint.sourceforge.net
    flickr.com/photos/heimdahlrecounts/4346082743
                                                         IBM Multicore SDK
   Toys                                                  www-
    flickr.com/photos/sbluerock/364123380                 01.ibm.com/support/docview.wss?uid=swg21456945
   TPS
    flickr.com/photos/twitteroffice/5681263084
   Bananas
    flickr.com/photos/design-dog/1249337589
   Dart
    flickr.com/photos/malavoda/2369876508

Performance - a challenging craft

  • 1.
  • 2.
    That‘s me  Myday to day job is to help customers fixing software performance Fabian  People really struggle with performance  But most of them struggle with coding as well  If creating good software is a craft, performance should be one as well Customer
  • 4.
     Some say: „creating software is art“  Art and software development need creativity Art
  • 5.
     Fixing performanceis often considered magic  Magic is something  only you understand  you do in a hard to Magic follow fashion
  • 6.
     Correctness indetail  Research and verification Craft
  • 7.
     All typesshare  Passion  Learning  Experience Which way?
  • 11.
     Value ==Money  We need a business case for performance Money  What is the impact of (in parts of the world) bad performance?
  • 12.
     Amazon measuredthe impact of 100ms delay  Sales dropped by 1%  In a year that would be Typical (?) retailer 245 Million USD
  • 13.
     Instable orslow software delays time to market  Slow software is no longer accepted by customers New?
  • 14.
     Insurances liketo send paper via mail  Not meeting deadlines can cause  Legal issues Snail Mail  Canceled contracts  Loss of money
  • 15.
  • 16.
     Calculate ExecutionTime  Code x = 5ms  Code y = 2*x = 10ms  Know code in advance  Waterfall approach Performance Engineering  Proving the performance of software is more difficult than proving the correct function
  • 17.
     We provefunctional correctness with automated tests  High coverage Look closely  Run examples and see if they are fast
  • 18.
     Done latein project  If done at all How much load breaks the system?  No chance to fix anything
  • 19.
     Avoid humanerrors  Require machine decidable fail / pass check  What is the measure? 42cm are fast?
  • 20.
     Functionality isindependent of the environment  Performance characteristics can vary  Unusable slow Our Environment  Lightning fast
  • 21.
     Underpowered hardware Loaded with tools and stuff Developers  Luckily not the production driving fast environment
  • 22.
     More power But also more load  How much faster is production than development? Crawling Production  Any estimation on how much better or worse the environments are is incorrect
  • 23.
     Real performancetests need real systems  Test in production  Clone production Stop playing infrastructure
  • 24.
     Amount ofdata is unpredictable  Application usage is unpredictable Tweets per second  How thought of using Twitter for build notification?
  • 25.
    Dev Prod  Test Test1  Fabian Lange  Test Test2  Uwe Friedrichsen  Test Test3  Mirko Novakovic  Showing 3/3  Showing 3/6,434,867
  • 26.
     Syntethic loadtests are unrealistic  No application has hundreds of users doing the same procedure again and again Load Baselines  Understanding real load is difficult
  • 27.
     Real usagecannot be generated  Real usage can be captured & replayed Live Systems are live  Be careful 
  • 28.
     Continous performancetests  Close to real setup App Monitor  Observe production behavior  Fix issues fast
  • 29.
     Conflicting interests  Development: Change  Operations: Stability Another Movement  Need to work together
  • 30.
  • 31.
     Can‘t wedo anything before production?  We want to deliver something, Let users test? which works perfectly!
  • 32.
     Optimizations mighthave no impact  Micro-Optimizations are Missed Target dangerous
  • 33.
     Soft Measure Works good for code quality Sonar  Are there performance best practices?
  • 34.
     Yahoo BestPractices  Google Best Practices  Plenty of tools Good Waterfalls  Work well
  • 35.
     Naive implementationlooks fine  But is not multithreaded
  • 36.
     This isthreadsafe  But slow
  • 37.
     This iscorrect  Correct synchronisation is hard
  • 38.
     Check LogLevel (Ugly)  Check Log Level (Nicer)
  • 39.
     Static SimpleDateFormatis wrong  Working with Dates and Calendars is very expensive!
  • 41.
     Some peoplemisuse it as loop  Results are unexpected behavior or slow execution
  • 42.
     Analyzes JavaBytecode  Knows 58 Performance Bugs  Most are rather trivial Indeed finds bugs
  • 43.
     Detecting deadlocks is difficult  Many thesis's on deadlock detection Verifies Java  Few code
  • 44.
     Hidden Gem Tries to cause Deadlocks IBM Support Assistant
  • 45.
     We needmore and reliable  Code Performance Metrics  Best Practices  Tools
  • 46.
  • 47.
  • 48.
    Art  YSlow flickr.com/photos/37202375@N03/3490107873 developer.yahoo.com/yslow  Magic  Pagespeed flickr.com/photos/princes_milady/4238912548 code.google.com/speed/page-speed  Crossroads  Java Locking flickr.com/photos/40481200@N06/4857965280 en.wikipedia.org/wiki/Double-checked_locking  Envelopes  PHP Ternary Operator flickr.com/photos/creatingmotherhood/3663914542 fabien.potencier.org/article/48/the-php-ternary- operator-fast-or-not  Test flickr.com/photos/cayusa/2159980025  JavaScript for .. In stackoverflow.com/questions/500504/javascript-for-  Scales in-with-arrays flickr.com/photos/reinvented/309512320  Findbugs  Tape Measure findbugs.sourceforge.net flickr.com/photos/hippie/2466583497  Jlint  World jlint.sourceforge.net flickr.com/photos/heimdahlrecounts/4346082743  IBM Multicore SDK  Toys www- flickr.com/photos/sbluerock/364123380 01.ibm.com/support/docview.wss?uid=swg21456945  TPS flickr.com/photos/twitteroffice/5681263084  Bananas flickr.com/photos/design-dog/1249337589  Dart flickr.com/photos/malavoda/2369876508