Performance Tuning 101

Problem Approach Nature of ﬁxes War Stories Performance Tuning 101 An introduction to performance tuning an application in managed environment Pavan KS1 Janmejay Singh2 1 mail: itspanzi@gmail.com blog: http://itspanzi.blogspot.com 2 mail: singh.janmejay@gmail.com blog: http://codehunk.wordpress.com Rootconf Bangalore, May 2012

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact. Requires mechanical approach, attack worst problem ﬁrst.

Problem Approach Nature of ﬁxes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact. Requires mechanical approach, attack worst problem ﬁrst. Prefer correctness over performance.

Problem Approach Nature of fixes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact. Requires mechanical approach, attack worst problem first. Prefer correctness over performance. Reasoning every fix (of paramount importance in multithreaded app).

Problem Approach Nature of fixes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact. Requires mechanical approach, attack worst problem first. Prefer correctness over performance. Reasoning every fix (of paramount importance in multithreaded app). Do one fix at a time, allows easy reasoning and verification, can be reverted.

Problem Approach Nature of fixes War StoriesGist Don’t be out there with a gun looking for problems. Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact. Requires mechanical approach, attack worst problem first. Prefer correctness over performance. Reasoning every fix (of paramount importance in multithreaded app). Do one fix at a time, allows easy reasoning and verification, can be reverted. Don’t jump in the lake without knowing whats waiting inside.

Problem Approach Nature of ﬁxes War StoriesContext App in reference: ThoughtWorks Studios - Go (CI/CD suite)

Problem Approach Nature of ﬁxes War StoriesContext App in reference: ThoughtWorks Studios - Go (CI/CD suite) Java (backend) and JRuby on Rails (frontend) webapp.

Problem Approach Nature of ﬁxes War StoriesContext App in reference: ThoughtWorks Studios - Go (CI/CD suite) Java (backend) and JRuby on Rails (frontend) webapp. Managed environment.

Problem Approach Nature of ﬁxes War StoriesContext App in reference: ThoughtWorks Studios - Go (CI/CD suite) Java (backend) and JRuby on Rails (frontend) webapp. Managed environment. GNU/Linux

Problem Approach Nature of ﬁxes War StoriesContext App in reference: ThoughtWorks Studios - Go (CI/CD suite) Java (backend) and JRuby on Rails (frontend) webapp. Managed environment. GNU/Linux Approach and learning is independent of tech-stack.

Problem Approach Nature of ﬁxes War StoriesOutline 1 Problem Identify scope Acceptance criteria 2 Approach How to analyze it Measure Fix 3 Nature of ﬁxes Synchronization GC DB Thrashing Pitfalls 4 War Stories GC Locking Caching

Problem Approach Nature of ﬁxes War StoriesDon’t go looking for performance issues Doing it proactively is a wild goose chase! Varying parameters and running into a real problem is improbable, if not impossible.

Problem Approach Nature of ﬁxes War StoriesRemember... Don’t be out there with a gun looking for problems. Its easier to reproduce and ﬁx a performance issue when you have seen it happen in a real setup. That way, you also know for sure, it is real.

Problem Approach Nature of ﬁxes War StoriesMy app is slow, it doesn’t respond... In case of Go, this could mean, The dashboard is slow

Problem Approach Nature of ﬁxes War StoriesMy app is slow, it doesn’t respond... In case of Go, this could mean, The dashboard is slow Javascript is slow

Problem Approach Nature of ﬁxes War StoriesMy app is slow, it doesn’t respond... In case of Go, this could mean, The dashboard is slow Javascript is slow Builds taking too long to schedule

Problem Approach Nature of ﬁxes War StoriesMy app is slow, it doesn’t respond... In case of Go, this could mean, The dashboard is slow Javascript is slow Builds taking too long to schedule Worker processes picking up work too late

Problem Approach Nature of ﬁxes War StoriesMy app is slow, it doesn’t respond... In case of Go, this could mean, The dashboard is slow Javascript is slow Builds taking too long to schedule Worker processes picking up work too late SCM changes reﬂect after a long time

Problem Approach Nature of ﬁxes War StoriesMy app is slow, it doesn’t respond... In case of Go, this could mean, The dashboard is slow Javascript is slow Builds taking too long to schedule Worker processes picking up work too late SCM changes reﬂect after a long time Process is hogging resources(requires 16 cores, 8 gig ram for a basic setup)

Problem Approach Nature of ﬁxes War StoriesBenchmark How slow is slow?

Problem Approach Nature of ﬁxes War StoriesBenchmark How slow is slow? How fast is fast?

Problem Approach Nature of ﬁxes War StoriesBenchmark How slow is slow? How fast is fast? Benchmark using realistic setup.

Problem Approach Nature of ﬁxes War StoriesBenchmark How slow is slow? How fast is fast? Benchmark using realistic setup. Use a repeatable and automated setup.

Problem Approach Nature of ﬁxes War StoriesRemember... Do not go overboard. Know your end goal.

Problem Approach Nature of ﬁxes War StoriesTrust in testing What if I start at ﬁx?

Problem Approach Nature of fixes War StoriesTrust in testing What if I start at fix? What if I stop at fix?

Problem Approach Nature of fixes War StoriesTrust in testing What if I start at fix? What if I stop at fix? Why do I need to iterate?

Problem Approach Nature of fixes War StoriesTrust in testing What if I start at fix? What if I stop at fix? Why do I need to iterate? How long should I iterate?

Problem Approach Nature of ﬁxes War StoriesRemember... Intution tends to work very poorly. Belive proof, not gut feel.

Problem Approach Nature of ﬁxes War StoriesWhat to measure Easy and obvious: Page response time

Problem Approach Nature of ﬁxes War StoriesWhat to measure Easy and obvious: Page response time Not so easy: App throughput Enqueue/Dequeue time for message queues CPU/IO churn Database performance Lock contention Managed environment speciﬁc parameters(Memory usage, GC churn etc). Environmental issues(too many processes, too low memory, very slow IO).

Problem Approach Nature of ﬁxes War StoriesRemember... Understand your environment. Know what to measure.

Problem Approach Nature of ﬁxes War StoriesTools to measure User load ab htperf

Problem Approach Nature of ﬁxes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts

Problem Approach Nature of ﬁxes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts Enqueue/Dequeue time: Use broker’s management console/logs/statistics.

Problem Approach Nature of ﬁxes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts Enqueue/Dequeue time: Use broker’s management console/logs/statistics. CPU/IO churn: Load Avg, iostat, klogd, inotify.

Problem Approach Nature of ﬁxes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts Enqueue/Dequeue time: Use broker’s management console/logs/statistics. CPU/IO churn: Load Avg, iostat, klogd, inotify. Database performance: Proﬁler.

Problem Approach Nature of fixes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts Enqueue/Dequeue time: Use broker’s management console/logs/statistics. CPU/IO churn: Load Avg, iostat, klogd, inotify. Database performance: Profiler. Lock contention: Profiler.

Problem Approach Nature of fixes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts Enqueue/Dequeue time: Use broker’s management console/logs/statistics. CPU/IO churn: Load Avg, iostat, klogd, inotify. Database performance: Profiler. Lock contention: Profiler. Managed env specific params: GC logs, GC profiling.

Problem Approach Nature of fixes War StoriesTools to measure User load ab htperf App throughput: measure app-level churn(creation/processing/deletion/delivery) counts Enqueue/Dequeue time: Use broker’s management console/logs/statistics. CPU/IO churn: Load Avg, iostat, klogd, inotify. Database performance: Profiler. Lock contention: Profiler. Managed env specific params: GC logs, GC profiling. Environmental issues: ps, dmesg, /proc/<pid>/status, /proc/net/dev, syslog, messages.

Problem Approach Nature of ﬁxes War StoriesRemember... Round pegs for round holes. Use appropriate tools. Conﬁgure them correctly with least possible overhead.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical. Use exhaggerated load to maginify the problem.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical. Use exhaggerated load to maginify the problem. Use the same load profile to test your fix.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical. Use exhaggerated load to maginify the problem. Use the same load profile to test your fix. Be rigorous.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical. Use exhaggerated load to maginify the problem. Use the same load profile to test your fix. Be rigorous. Mob pair on complicated fixes, especially fixes involving data mutation across threads.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical. Use exhaggerated load to maginify the problem. Use the same load profile to test your fix. Be rigorous. Mob pair on complicated fixes, especially fixes involving data mutation across threads. Attack one and only one problem.

Problem Approach Nature of fixes War StoriesHow to carry out fixes Understand libraries: read fine print in documentation, or better yet, read code. Be mechanical. Use exhaggerated load to maginify the problem. Use the same load profile to test your fix. Be rigorous. Mob pair on complicated fixes, especially fixes involving data mutation across threads. Attack one and only one problem. Having said all that, correctness always comes first.

Problem Approach Nature of fixes War StoriesRemember... Requires mechanical approach Attack worst problem first Reason every fix Reasoning is of paramount importance in multithreaded app Do one fix at a time Allows easy reasoning and verification, can be reverted Prefer correctness over performance

Problem Approach Nature of ﬁxes War StoriesLock granularity and lockless When you ﬁx lock contention somewhere, ensure it hasn’t moved elsewhere, making overall performance worse.

Problem Approach Nature of ﬁxes War StoriesLock granularity and lockless When you ﬁx lock contention somewhere, ensure it hasn’t moved elsewhere, making overall performance worse. More than one shared variable mandates synchronization usually, watch out!

Problem Approach Nature of ﬁxes War StoriesLock granularity and lockless When you ﬁx lock contention somewhere, ensure it hasn’t moved elsewhere, making overall performance worse. More than one shared variable mandates synchronization usually, watch out! Atomic operations and globaly unique objects(like interned symbols) are your friends.

Problem Approach Nature of ﬁxes War StoriesRemember... Jumps places, unlike functional bugs Look beyond local impact of a ﬁx

Problem Approach Nature of ﬁxes War StoriesGarbage Collection When you hear unresponsiveness is intermittent, be alarmed! It probably is GC It is likely not easy to ﬁx Root cause may be you or a library

Problem Approach Nature of ﬁxes War StoriesGarbage Collection When you hear unresponsiveness is intermittent, be alarmed! It probably is GC It is likely not easy to ﬁx Root cause may be you or a library Scan GC logs

Problem Approach Nature of fixes War StoriesGarbage Collection When you hear unresponsiveness is intermittent, be alarmed! It probably is GC It is likely not easy to fix Root cause may be you or a library Scan GC logs Profile to see if majority of allocations are from just a handful of calls If its several calls with each contributing very little to the pie, don’t even try fixing it.

Problem Approach Nature of fixes War StoriesGarbage Collection When you hear unresponsiveness is intermittent, be alarmed! It probably is GC It is likely not easy to fix Root cause may be you or a library Scan GC logs Profile to see if majority of allocations are from just a handful of calls If its several calls with each contributing very little to the pie, don’t even try fixing it. Understand GC ergonomics and each algorithm(if you have alternatives) in detail. Tweek your max/min permitted allocation judiciously, malloc/free are expensive system calls.

Problem Approach Nature of ﬁxes War StoriesSQL Database Optimize connection pool size. Expand is very expensive. Is dispatched on caller thread(unless your connection pool is super-human).

Problem Approach Nature of ﬁxes War StoriesSQL Database Optimize connection pool size. Expand is very expensive. Is dispatched on caller thread(unless your connection pool is super-human). Watch out for slow queries, it can snowball.

Problem Approach Nature of ﬁxes War StoriesSQL Database Optimize connection pool size. Expand is very expensive. Is dispatched on caller thread(unless your connection pool is super-human). Watch out for slow queries, it can snowball. Understand your DB settings, MVCC, table-locking scenarios, row-locking scenarios, transaction isolation tradeoffs.

Problem Approach Nature of ﬁxes War StoriesSQL Database Optimize connection pool size. Expand is very expensive. Is dispatched on caller thread(unless your connection pool is super-human). Watch out for slow queries, it can snowball. Understand your DB settings, MVCC, table-locking scenarios, row-locking scenarios, transaction isolation tradeoffs. Contd...

Problem Approach Nature of ﬁxes War StoriesSQL Database contd... Explain and Analyze are your friends.

Problem Approach Nature of fixes War StoriesSQL Database contd... Explain and Analyze are your friends. Understanding implications of file-system IO helps realize table scans can be horrible DB runs with fixed page cache. Table scan can wipe it all cold. Use indexes to avoid table scan. Indexes are order sensitive. Tweek inner queries to reduce selectivity.

Problem Approach Nature of fixes War StoriesSQL Database contd... Explain and Analyze are your friends. Understanding implications of file-system IO helps realize table scans can be horrible DB runs with fixed page cache. Table scan can wipe it all cold. Use indexes to avoid table scan. Indexes are order sensitive. Tweek inner queries to reduce selectivity. Every DB may be subtly different(joins can be more expensive than materialzed views(if you have em))

Problem Approach Nature of fixes War StoriesSQL Database contd... Explain and Analyze are your friends. Understanding implications of file-system IO helps realize table scans can be horrible DB runs with fixed page cache. Table scan can wipe it all cold. Use indexes to avoid table scan. Indexes are order sensitive. Tweek inner queries to reduce selectivity. Every DB may be subtly different(joins can be more expensive than materialzed views(if you have em)) Denormalization can help crack tough nuts.

Problem Approach Nature of ﬁxes War StoriesContext Switch Thrashing Too few cores, too many processes will obviously hurt.

Problem Approach Nature of ﬁxes War StoriesContext Switch Thrashing Too few cores, too many processes will obviously hurt. Too many locks will cause too many sleeps, hence too much ctx switching.

Problem Approach Nature of ﬁxes War StoriesContext Switch Thrashing Too few cores, too many processes will obviously hurt. Too many locks will cause too many sleeps, hence too much ctx switching. Ctx switch may schedule a different process, causing CPU caches and TLB to get cold.

Problem Approach Nature of ﬁxes War StoriesContext Switch Thrashing Too few cores, too many processes will obviously hurt. Too many locks will cause too many sleeps, hence too much ctx switching. Ctx switch may schedule a different process, causing CPU caches and TLB to get cold. Too frequent IO will cause io-wait, hence switch. Use of swap can completely kill performance. Swap is for emergency, not for regular usage. Over logging can hurt, be careful! Concept of buffered IO exists for a reason, use it! Use memory mapped IO if it makes sense.

Problem Approach Nature of ﬁxes War StoriesContext Switch Thrashing Too few cores, too many processes will obviously hurt. Too many locks will cause too many sleeps, hence too much ctx switching. Ctx switch may schedule a different process, causing CPU caches and TLB to get cold. Too frequent IO will cause io-wait, hence switch. Use of swap can completely kill performance. Swap is for emergency, not for regular usage. Over logging can hurt, be careful! Concept of buffered IO exists for a reason, use it! Use memory mapped IO if it makes sense. Assigning very low max memory limit can cause frequent GCs, allocate enough memory so its your functional thread that works and not GC threads.

Problem Approach Nature of ﬁxes War StoriesCommon Pitfalls Prejudice SSL is bad Using asynchronicity to avoid ﬁxing root cause

Problem Approach Nature of ﬁxes War StoriesCommon Pitfalls Prejudice SSL is bad Using asynchronicity to avoid ﬁxing root cause Lack of testing: In Go context, myth of 150 agent limit.

Problem Approach Nature of ﬁxes War StoriesCommon Pitfalls Prejudice SSL is bad Using asynchronicity to avoid ﬁxing root cause Lack of testing: In Go context, myth of 150 agent limit. Lack of setup understanding: Misconception that 64 bit JVM is always bad.

Problem Approach Nature of ﬁxes War StoriesCommon Pitfalls Prejudice SSL is bad Using asynchronicity to avoid ﬁxing root cause Lack of testing: In Go context, myth of 150 agent limit. Lack of setup understanding: Misconception that 64 bit JVM is always bad. Atomic operations and globaly unique objects(like interned symbols) are your friends.

Problem Approach Nature of fixes War StoriesCommon Pitfalls Prejudice SSL is bad Using asynchronicity to avoid fixing root cause Lack of testing: In Go context, myth of 150 agent limit. Lack of setup understanding: Misconception that 64 bit JVM is always bad. Atomic operations and globaly unique objects(like interned symbols) are your friends. Unchartred waters ERB garbage churn Strange JVM allocations Single threaded database JRuby class hirarchy modification(runtime level) locks.

Problem Approach Nature of ﬁxes War StoriesKnow what you are getting into... Know what you are getting into...

Problem Approach Nature of fixes War StoriesGist, to reiterate... Do not go overboard, know your end goal. Intution tends to work very poorly. Belive proof, not gut feel. Understand your environment. Know what to measure. Round pegs for round holes. Use appropriate tools. Jumps places, unlike functional bugs, look beyond local impact. Requires mechanical approach, attack worst problem first. Prefer correctness over performance. Reasoning every fix (of paramount importance in multithreaded app). Do one fix at a time, allows easy reasoning and verification, can be reverted. Don’t jump in the lake without knowing whats waiting inside.

Problem Approach Nature of ﬁxes War StoriesGarbage churn issues Debug log statements

Problem Approach Nature of ﬁxes War StoriesGarbage churn issues Debug log statements ERB rendering

Problem Approach Nature of ﬁxes War StoriesGarbage churn issues Debug log statements ERB rendering Pooling objects with signiﬁcant memory footprint XML parser factory Buffered reader

Problem Approach Nature of ﬁxes War StoriesGarbage churn issues Debug log statements ERB rendering Pooling objects with signiﬁcant memory footprint XML parser factory Buffered reader Stream being dereferenced into a string

Problem Approach Nature of ﬁxes War StoriesGarbage churn issues Debug log statements ERB rendering Pooling objects with signiﬁcant memory footprint XML parser factory Buffered reader Stream being dereferenced into a string Thread locals to the rescue(used for caching big reusable objects)

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods Copy on to stack(using lock), mutate locklessly, copy back to shared memory(using lock)

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods Copy on to stack(using lock), mutate locklessly, copy back to shared memory(using lock) Use read/write lock when appropriate

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods Copy on to stack(using lock), mutate locklessly, copy back to shared memory(using lock) Use read/write lock when appropriate Use atomic operations for lockess concurrency

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods Copy on to stack(using lock), mutate locklessly, copy back to shared memory(using lock) Use read/write lock when appropriate Use atomic operations for lockess concurrency Concurrent data-structure != thread-safety

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods Copy on to stack(using lock), mutate locklessly, copy back to shared memory(using lock) Use read/write lock when appropriate Use atomic operations for lockess concurrency Concurrent data-structure != thread-safety Take difﬁcult design tradeoffs to keep code requiring a class of lock in one place

Problem Approach Nature of ﬁxes War StoriesContention and reasoning issues Locking on interned strings generated for context sensitive locking Death of synchronized methods Copy on to stack(using lock), mutate locklessly, copy back to shared memory(using lock) Use read/write lock when appropriate Use atomic operations for lockess concurrency Concurrent data-structure != thread-safety Take difﬁcult design tradeoffs to keep code requiring a class of lock in one place Double check locking can be very helpful in frequently called code

Problem Approach Nature of ﬁxes War StoriesCaching Eager caching is dangerous(using miss => load => prime => hit works better in practice)

Problem Approach Nature of fixes War StoriesCaching Eager caching is dangerous(using miss => load => prime => hit works better in practice) Testing cache with boundry conditions Test to ensure cache-hit Ensure boundry conditions are well understood, overflow is judiciously defined/used

Problem Approach Nature of fixes War StoriesCaching Eager caching is dangerous(using miss => load => prime => hit works better in practice) Testing cache with boundry conditions Test to ensure cache-hit Ensure boundry conditions are well understood, overflow is judiciously defined/used LRU eviction is the right(only?) solution sometimes

Problem Approach Nature of fixes War StoriesCaching Eager caching is dangerous(using miss => load => prime => hit works better in practice) Testing cache with boundry conditions Test to ensure cache-hit Ensure boundry conditions are well understood, overflow is judiciously defined/used LRU eviction is the right(only?) solution sometimes Double check cache invalidation flows, its hard, sometimes bleeds across classes.

Problem Approach Nature of fixes War StoriesCaching Eager caching is dangerous(using miss => load => prime => hit works better in practice) Testing cache with boundry conditions Test to ensure cache-hit Ensure boundry conditions are well understood, overflow is judiciously defined/used LRU eviction is the right(only?) solution sometimes Double check cache invalidation flows, its hard, sometimes bleeds across classes. Be vigilant when using hirarchical caches(no associative-arrays or maps should be cached), its easy to run memory out with this.

Problem Approach Nature of ﬁxes War StoriesCaching contd... Do not cache uncommited data or invalidate cache before commit(transactional caches may come in handy for this).

Problem Approach Nature of ﬁxes War StoriesCaching contd... Do not cache uncommited data or invalidate cache before commit(transactional caches may come in handy for this). Ensure transactions use cached data iff no other transaction modiﬁes it or ensure transactionally consistent invalidation.

Problem Approach Nature of fixes War StoriesCaching contd... Do not cache uncommited data or invalidate cache before commit(transactional caches may come in handy for this). Ensure transactions use cached data iff no other transaction modifies it or ensure transactionally consistent invalidation. Understand your caching library, too low overflow limit may cause IO issues.

Problem Approach Nature of fixes War StoriesCaching contd... Do not cache uncommited data or invalidate cache before commit(transactional caches may come in handy for this). Ensure transactions use cached data iff no other transaction modifies it or ensure transactionally consistent invalidation. Understand your caching library, too low overflow limit may cause IO issues. Use one and only one cache instance(its easy to end up in a memory runout with this). We had view fragments, urls, domain objects and view models cached in the same instance.

Problem Approach Nature of ﬁxes War StoriesJavascript JS dom frequent offset/size calls hurt

Problem Approach Nature of ﬁxes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops.

Problem Approach Nature of fixes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops. Using beautiful framework provided methods like filter, find and select can have detrimental consequences.

Problem Approach Nature of fixes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops. Using beautiful framework provided methods like filter, find and select can have detrimental consequences. Avoid multiple lookups for the same element, use a loaded copy as far as possible.

Problem Approach Nature of fixes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops. Using beautiful framework provided methods like filter, find and select can have detrimental consequences. Avoid multiple lookups for the same element, use a loaded copy as far as possible. Avoid CSS selectors in frequently called code.

Problem Approach Nature of fixes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops. Using beautiful framework provided methods like filter, find and select can have detrimental consequences. Avoid multiple lookups for the same element, use a loaded copy as far as possible. Avoid CSS selectors in frequently called code. Avoid too much DOM churn.

Problem Approach Nature of fixes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops. Using beautiful framework provided methods like filter, find and select can have detrimental consequences. Avoid multiple lookups for the same element, use a loaded copy as far as possible. Avoid CSS selectors in frequently called code. Avoid too much DOM churn. Use tools like Chromium dev toolset to nail leaks and profile methods.

Problem Approach Nature of fixes War StoriesJavascript JS dom frequent offset/size calls hurt Framework provided foreach loops are bad, use native for loops. Using beautiful framework provided methods like filter, find and select can have detrimental consequences. Avoid multiple lookups for the same element, use a loaded copy as far as possible. Avoid CSS selectors in frequently called code. Avoid too much DOM churn. Use tools like Chromium dev toolset to nail leaks and profile methods. Auto-refreshing dashboards should refresh only visible and changed area.

Problem Approach Nature of ﬁxes War StoriesQuestions? Questions?

http://funnel.hasgeek.com	173
http://a0.twimg.com	8
http://us-w1.rockmelt.com	3

Performance Tuning 101

by Janmejay Singh

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 184

Statistics

Performance Tuning 101 Presentation Transcript