Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • scale_perf_best_practices

    1. 1. Scalability and Performance Best Practices Laura Thomson OmniTI [email_address]
    2. 2. Standing in for George
    3. 3. Scalability vs. Performance <ul><li>Scalability: Ability to gracefully handle additional traffic while maintaining service quality. </li></ul><ul><li>Performance: Ability to execute a single task quickly. </li></ul><ul><li>Often linked, not the same. </li></ul>
    4. 4. Why are Scalability and Performance Important? <ul><li>No hope of growth otherwise. </li></ul><ul><li>Scalability means you can handle service commitments of the future. </li></ul><ul><li>Performance means you can handle the service commitments of today. </li></ul><ul><li>Both act symbiotically to mean cost-efficient growth. </li></ul>
    5. 5. Why PHP? <ul><li>PHP is a completely runtime language </li></ul><ul><li>Compiled, statically typed languages are faster. </li></ul><ul><li>BUT: </li></ul><ul><ul><li>Scalability is (almost) never a factor of the language you use </li></ul></ul><ul><ul><li>Most bottlenecks are not in user code </li></ul></ul><ul><ul><li>PHP’s heavy lifting is done in C </li></ul></ul><ul><ul><li>PHP is fast to learn </li></ul></ul><ul><ul><li>PHP is fast to write </li></ul></ul><ul><ul><li>PHP is easy to extend </li></ul></ul>
    6. 6. When to Start <ul><li>Premature optimization is the root of all evil – Donald Knuth </li></ul><ul><li>Without direction and goals, your code will only get more obtuse with little hope of actual improvement in scalability or speed. </li></ul><ul><li>Design for refactoring, so that when you need to make changes, you can. </li></ul>
    7. 7. Knowing When to Stop <ul><li>Optimizations get exponentially more expensive as they are accrued. </li></ul><ul><li>Strike a balance between performance, scalability and features. </li></ul><ul><li>Unless you ship, all the speed in the world is meaningless. </li></ul>
    8. 8. No Fast = True <ul><li>Optimization takes effort. </li></ul><ul><li>Some are easier than others, but no silver bullet. </li></ul><ul><li>Be prepared to get your hands dirty. </li></ul>
    9. 9. General Best Practices <ul><li>Profile early, profile often. </li></ul><ul><li>Dev-ops cooperation is essential. </li></ul><ul><li>Test on production data. </li></ul><ul><li>Track and trend. </li></ul><ul><li>Assumptions will burn you. </li></ul>
    10. 10. Scalability Best Practices <ul><li>Decouple. </li></ul><ul><li>Cache. </li></ul><ul><li>Federate. </li></ul><ul><li>Replicate. </li></ul><ul><li>Avoid straining hard-to-scale resources. </li></ul>
    11. 11. Performance Best Practices <ul><li>Use a compiler cache. </li></ul><ul><li>Be mindful of using external data sources. </li></ul><ul><li>Avoid recursive or heavy looping code. </li></ul><ul><li>Don’t try to outsmart PHP. </li></ul><ul><li>Build with caching in mind. </li></ul>
    12. 12. 1. Profiling <ul><li>Pick a profiling tool and learn it in and out. </li></ul><ul><ul><li>APD, XDebug, Zend Platform </li></ul></ul><ul><li>Learn your system profiling tools </li></ul><ul><ul><li>strace, dtrace, ltrace </li></ul></ul><ul><li>Effective debugging profiling is about spotting deviations from the norm. </li></ul><ul><li>Effective habitual profiling is about making the norm better. </li></ul><ul><li>Practice, practice, practice. </li></ul>
    13. 13. 2. Dev-Ops Cooperation <ul><li>The most critical difference in organizations that handles crises well. </li></ul><ul><li>Production problems are time-critical and usually hard to diagnose. </li></ul><ul><li>Build team unity before emergencies happen. </li></ul><ul><li>Operations staff should provide feedback on behavior changes when code is pushed live. </li></ul><ul><li>Development staff must heed warnings from operations staff. </li></ul><ul><li>Established code launch windows, developer escalation procedures, and fallback plans are very helpful. </li></ul>
    14. 14. 3. Test on Production(-ish) Data <ul><li>Code behavior (especially performance) is often data driven. </li></ul><ul><li>Using data that looks like production data will minimize surprises. </li></ul><ul><li>Having a QA environment that simulates production load on all components will highlight problems before they occur. </li></ul>
    15. 15. 4. Track and Trend <ul><li>Understanding your historical performance characteristics is essential for spotting emerging problems. </li></ul><ul><ul><li>Access logs (with hi-res timings) </li></ul></ul><ul><ul><li>System metrics </li></ul></ul><ul><ul><li>Application and query profiling data </li></ul></ul>
    16. 16. Access log timings <ul><li>Apache 2 natively supports hi-res timings </li></ul><ul><li>For Apache 1.3 you’ll need to patch it (timings in seconds = not very useful) </li></ul>
    17. 17. 5. When you assume… <ul><li>Systems are complex and often break in unexpected ways. </li></ul><ul><li>If you knew how your system was broken, you probably would have designed it better in the first place. </li></ul><ul><li>Confirming your suspicions is almost always cheaper than acting on them. </li></ul><ul><li>Time is your most precious commodity. </li></ul>
    18. 18. 6. Decouple <ul><li>Isolate performance failures. </li></ul><ul><li>Put refactoring time only where needed. </li></ul><ul><li>Put hardware only where needed. </li></ul><ul><li>Impairs your ability to efficiently join two decoupled application data sets. </li></ul>
    19. 19. Example: Static versus dynamic content <ul><li>Apache + PHP is fast for dynamic content </li></ul><ul><li>Waste of resources to serve static content from here: images, CSS, JavaScript </li></ul><ul><li>Move static content to a separate faster solution for static content e.g. lighttpd on a separate box -> on a geographically distributed CDN </li></ul>
    20. 20. Example: Session data <ul><li>Using the default session store limits scale out </li></ul><ul><li>Decouple session data by putting it elsewhere: </li></ul><ul><ul><li>In a database </li></ul></ul><ul><ul><li>In a distributed cache </li></ul></ul><ul><ul><li>In cookies </li></ul></ul>
    21. 21. 7. Cache <ul><li>Caching is the core of most optimizations. </li></ul><ul><li>Fundamental question is: how dynamic does this bit have to be. </li></ul><ul><li>Many levels of caching </li></ul><ul><ul><li>Algorithmic </li></ul></ul><ul><ul><li>Data </li></ul></ul><ul><ul><li>Page/Component </li></ul></ul><ul><li>Good technologies out there: </li></ul><ul><ul><li>APC (local data) </li></ul></ul><ul><ul><li>Memcache (distributed data) </li></ul></ul><ul><ul><li>Squid (distributed page/component/data) </li></ul></ul><ul><ul><li>Bespoke </li></ul></ul>
    22. 22. Caching examples <ul><li>Compiler cache (APC or Zend) </li></ul><ul><li>MySQL query cache (tune and use where possible) </li></ul><ul><li>Cache generated pages or iframes (disk or memcache) </li></ul><ul><li>Cache calculated data, datasets, page fragments (memcache) </li></ul><ul><li>Cache static content (squid) </li></ul>
    23. 23. 8. Federate <ul><li>Data federation is taking a single data set and spreading it across multiple database/application servers. </li></ul><ul><li>Great technique for scaling data. </li></ul><ul><li>Does not inherently promote data reliability. </li></ul><ul><li>Reduces your ability to join within the data set. </li></ul><ul><li>Increases overall internal connection establishment rate. </li></ul>
    24. 24. 9. Replicate <ul><li>Replication is making synchronized copies of data available in more than one place. </li></ul><ul><li>Useful scaling technique, very popular in ‘modern’ PHP architectures. </li></ul><ul><li>Mostly usable for read-only data. </li></ul><ul><li>High write rates can make it difficult to keep slaves in sync. </li></ul>
    25. 25. Problems <ul><li>On the slave, you should see two threads running: an I/O thread, that reads data from the master, and an SQL thread, that updates the replicated tables. </li></ul><ul><li>(You can see these with SHOW PROCESSLIST) </li></ul><ul><li>Since updates on the master occur in *multiple* threads, and on the slave in a *single* thread, the updates on the slave take longer. </li></ul><ul><li>Slaves have to use a single SQL thread to make sure queries are executed in the same order as on the master </li></ul>
    26. 26. <ul><li>The more writes you do, the more likely the slaves are to get behind, and the further behind they will get. </li></ul><ul><li>At a certain point the only solution is to stop the slave and re-image from the master. </li></ul><ul><li>Or use a different solution: multi master, federation, split architectures between replication and federation, etc </li></ul>
    27. 27. Other uses of replication <ul><li>Remember replication has other uses than scale out </li></ul><ul><li>Failover </li></ul><ul><li>Backups </li></ul>
    28. 28. 10. Avoid Straining Hard-to-Scale Resources <ul><li>Some resources are inherently hard to scale </li></ul><ul><ul><li>‘ Uncacheable’ data </li></ul></ul><ul><ul><li>Data with a very high read+write rate </li></ul></ul><ul><ul><li>Non-federatable data </li></ul></ul><ul><ul><li>Data in a black-box </li></ul></ul><ul><li>Be aware of these limitations and be extra careful with these resources. </li></ul><ul><li>Try and poke holes in the assumptions about why the data is hard to manage. </li></ul>
    29. 29. 11. Compiler Cache <ul><li>PHP natively reparses a script and its includes whenever it executes it. </li></ul><ul><li>This is wasteful and a huge overhead. </li></ul><ul><li>A compiler cache sits inside the engine and caches the parsed optrees. </li></ul><ul><li>The closest thing to ‘fast = true’ </li></ul><ul><li>In PHP5 the real alternatives are APC and Zend Platform. </li></ul>
    30. 30. 12. Xenodataphobia <ul><li>External data (RDBMS, App Server, 3 rd Party data feeds) are the number one cause of application bottlenecks. </li></ul><ul><li>Minimize and optimize your queries. </li></ul><ul><li>3 rd Party data feeds/transfers are unmanageable. Do what you can to take them out of the critical path. </li></ul>
    31. 31. Managing external data and services <ul><li>Cache it (beware of AUPs for APIs) </li></ul><ul><li>Load it dynamically (iframes/XMLHttpRequest) </li></ul><ul><li>Batch writes </li></ul><ul><li>Ask how critical the data is to your app. </li></ul>
    32. 32. Query tuning <ul><li>Query tuning is like PHP tuning: what you think is slow may not be slow. </li></ul><ul><li>Benchmarking is the only way to truly test this. </li></ul><ul><li>When tuning, change one thing at a time </li></ul><ul><li>Your toolkit: </li></ul><ul><ul><li>EXPLAIN </li></ul></ul><ul><ul><li>Slow Query Log </li></ul></ul><ul><ul><li>mytop </li></ul></ul><ul><ul><li>Innotop </li></ul></ul><ul><ul><li>Query profilers </li></ul></ul>
    33. 33. Indexing problems <ul><li>Lack of appropriate indexing </li></ul><ul><li>Create relevant indexes. Make sure your queries use them. (EXPLAIN is your friend here.) </li></ul><ul><li>The order of multi-column indexes is important </li></ul><ul><li>Remove unused indexes to speed writes </li></ul>
    34. 34. Schema design (MySQL) <ul><li>Use the smallest data type possible </li></ul><ul><li>Use fixed width rows where possible (prefer char over varchar: disk is cheap) </li></ul><ul><li>Denormalize where necessary </li></ul><ul><li>Take static data out of the database or use MEMORY tables </li></ul><ul><li>Use the appropriate storage engine for each table </li></ul>
    35. 35. Queries <ul><li>Minimizing the number of queries is always a good start. Web pages that need to make 70-80 queries to be rendered need a different strategy: </li></ul><ul><ul><li>Cache the output </li></ul></ul><ul><ul><li>Cache part of the output </li></ul></ul><ul><ul><li>Redesign your schema so you can reduce the number of queries </li></ul></ul><ul><ul><li>Decide if you can live without some of these queries. </li></ul></ul><ul><li>Confirm that your queries are using the indexes you think that they are </li></ul><ul><li>Avoid correlated subqueries where possible </li></ul><ul><li>Stored procedures are notably faster </li></ul>
    36. 36. 13. Be Lazy <ul><li>Deeply recursive code is expensive in PHP. </li></ul><ul><li>Heavy manual looping usually indicates that you are doing something wrong. </li></ul><ul><li>Learn PHP’s idioms for dealing with large data sets or parsing/packing data. </li></ul>
    37. 37. 14. Don’t Outsmart Yourself <ul><li>Don’t try to work around perceived inefficiencies in PHP (at least not in userspace code!) </li></ul><ul><li>Common bad examples here include: </li></ul><ul><ul><li>Writing parsers in PHP that could be done with a simple regex. </li></ul></ul><ul><ul><li>Trying to circumvent connection management in networking/database libraries. </li></ul></ul><ul><ul><li>Performing complex serializations that could be done with internal extensions. </li></ul></ul><ul><ul><li>Calling out to external executables when a PHP extension can give you the same information. </li></ul></ul><ul><ul><li>Reimplementing something that already exists in PHP </li></ul></ul>
    38. 38. 15. Caching <ul><li>Mentioned before, but deserves a second slide: caching is the most important tool in your tool box. </li></ul><ul><li>For frequently accessed information, even a short cache lifespan can be productive. </li></ul><ul><li>Watch your cache hit rates. A non-effective cache is worse than no cache. </li></ul>
    39. 39. Thanks! <ul><li>There are longer versions of this talk at </li></ul><ul><li>There are good books on these topics as well: </li></ul><ul><ul><li>Advanced PHP Programming, G. Schlossnagle </li></ul></ul><ul><ul><li>Building Scalable Web Sites, C. Henderson </li></ul></ul><ul><ul><li>Scalable Internet Architectures, T. Schlossnagle </li></ul></ul><ul><li>Compulsory plug: OmniTI is hiring for a number of positions (PHP, Perl, C, UI design) </li></ul><ul><ul><li> </li></ul></ul>