Downloaded 10 times
![Primitive Arrays
// Sequential Access = Predictable
for (int i=0; i<someArray.length; i++)
someArray[i]++;](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-49-320.jpg)
![Primitive Arrays - Skipping Elements
// Holes Hurt
for (int i=0; i<someArray.length; i += SKIP)
someArray[i]++;](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-50-320.jpg)
![Multidimensional Arrays
Multidimensional Arrays are really Arrays of
Arrays in Java. (Unlike C)
Some people realign their accesses:
for (int col=0; col<COLS; col++) {
for (int row=0; row<ROWS; row++) {
array[ROWS * col + row]++;
}
}](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-52-320.jpg)
![Bad Access Alignment
Strides the wrong way, bad
locality.
array[COLS * row + col]++;
Strides the right way, good
locality.
array[ROWS * col + row]++;](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-53-320.jpg)
More Related Content
Similar to Performance and predictability
![[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...](https://cdn.slidesharecdn.com/ss_thumbnails/paperreadinginterleavingwithcoroutinesapracticalapproachforrobustindexjoin-211011055753-thumbnail.jpg?width=640&height=640&fit=bounds)
Performance and predictability
- 1.
- 3. Why care aboutlow level rubbish? Branch Prediction Memory Access Storage Conclusions
- 5. Performance Discussion Product Solutions “Justuse our library/tool/framework, and everything is web-scale!” Architecture Advocacy “Always design your software like this.” Methodology & Fundamentals “Here are some principles and knowledge, use your brain“
- 6. Do you care? DON'Tlook at access patterns first Many problems not Pattern Related Networking Database or External Service Minimising I/O Garbage Collection Insufficient Parallelism Harvest the low hanging fruit first
- 7. Low Hanging isa cost-benefit analysis
- 8. So when doesit matter?
- 9. Informed Design &Architecture * * this is not a call for premature optimisation
- 11. That 10% thatactually matters
- 12. Performance of PredictableAccesses
- 13. Why care aboutlow level rubbish? Branch Prediction Memory Access Storage Conclusions
- 14. What 4 thingsdo CPUs actually do?
- 15. Fetch, Decode, Execute,Writeback
- 16.
- 17.
- 18. What about branches? publicstatic int simple(int x, int y, int z) { int ret; if (x > 5) { ret = y + z; } else { ret = y; } return ret; }
- 19. Branches cause stalls,stalls kill performance
- 20.
- 21. Can we eliminatebranches?
- 22. Naive Compilation if (x> 5) { ret = z; } else { ret = y; } cmp x, 5 jmp L1 mov z, ret br L2 L1: mov y, ret L2: ...
- 23. Conditional Branches if (x> 5) { ret = z; } else { ret = y; } cmp x, 5 mov z, ret cmovle y, ret
- 24. Strategy: predict branchesand speculatively execute
- 25. Static Prediction A forwardbranch defaults to not taken A backward branch defaults to taken
- 27. Static Hints (Pentium4 or later) __emit 0x3E defaults to taken __emit 0x2E defaults to not taken don’t use them, flip the branch
- 28. Dynamic prediction: recordhistory and predict future
- 29. Branch Target Buffer(BTB) a log of the history of each branch also stores the address its finite!
- 30. Local record per conditionalbranch histories Global record shared history of conditional jumps
- 31. Loop specialised predictor whenthere’s a loop (jumping in a cycle n times) Function specialised buffer for predicted nearby function returns Two level Adaptive Predictor accounts for up patterns of up to 3 if statements
- 32. Measurement and Monitoring UsePerformance Event Counters (Model Specific Registers) Can be configured to store branch prediction information Profilers & Tooling: perf (linux), VTune, AMD Code Analyst, Visual Studio, Oracle Performance Studio
- 33. Approach: Loop Unrolling for(int x = 0; x < 100; x+=5) { foo(x); foo(x+1); foo(x+2); foo(x+3); foo(x+4); }
- 34. Approach: minimise branchesin loops
- 35. Approach: Separation ofConcerns Little branching on a latency critical path or thread Heavily branchy administrative code on a different path Helps adjust for the inflight branch prediction not tracking many targets Only applicable with low context switching rates Also easier to understand the code
- 36.
- 37. Division/Modulus index = (index+ 1) % SIZE; vs index = index + 1; if (index == SIZE) { index = 0; }
- 38. / or %are 92 Cycles on Core i*
- 39. Summary CPUs are Super-pipelinedand Superscalar Branches cause stalls Branches can be removed, minimised or simplified Frequently get easier to read code as well
- 40. Why care aboutlow level rubbish? Branch Prediction Memory Access Storage Conclusions
- 41. The Problem VeryFast Relatively Slow
- 42. The Solution: CPUCache Core Demands Data, looks at its cache If present (a "hit") then data returned to register If absent (a "miss") then data looked up from memory and stored in the cache Fast memory is expensive, a small amount is affordable
- 43. Multilevel Cache: IntelSandybridge Shared Level 3 Cache Level 2 Cache Level 1 Data Cache Level 1 Instruction Cache Physical Core 0 HT: 2 Logical Cores .... Level 2 Cache Level 1 Data Cache Level 1 Instruction Cache Physical Core N HT: 2 Logical Cores
- 44. How bad isa miss? Location Latency in Clockcycles Register 0 L1 Cache 3 L2 Cache 9 L3 Cache 21 Main Memory 150-400
- 45. Cache Lines Data transferredin cache lines Fixed size block of memory Usually 64 bytes in current x86 CPUs Between 32 and 256 bytes Purely hardware consideration
- 46. Prefetch = Eagerlyload data Adjacent Cache Line Prefetch Data Cache Unit (Streaming) Prefetch Problem: CPU Prediction isn't perfect Solution: Arrange Data so accesses are predictable Prefetching
- 47. Sequential Locality Referring todata that is arranged linearly in memory Spatial Locality Referring to data that is close together in memory Temporal Locality Repeatedly referring to same data in a short time span
- 48. General Principles Use smallerdata types (-XX:+UseCompressedOops) Avoid 'big holes' in your data Make accesses as linear as possible
- 49. Primitive Arrays // SequentialAccess = Predictable for (int i=0; i<someArray.length; i++) someArray[i]++;
- 50. Primitive Arrays -Skipping Elements // Holes Hurt for (int i=0; i<someArray.length; i += SKIP) someArray[i]++;
- 51. Primitive Arrays -Skipping Elements
- 52. Multidimensional Arrays Multidimensional Arraysare really Arrays of Arrays in Java. (Unlike C) Some people realign their accesses: for (int col=0; col<COLS; col++) { for (int row=0; row<ROWS; row++) { array[ROWS * col + row]++; } }
- 53. Bad Access Alignment Stridesthe wrong way, bad locality. array[COLS * row + col]++; Strides the right way, good locality. array[ROWS * col + row]++;
- 54. Full Random Access L1D- 5 clocks L2 - 37 clocks Memory - 280 clocks Sequential Access L1D - 5 clocks L2 - 14 clocks Memory - 28 clocks
- 55. Primitive Collections (GNUTrove, GS-Coll) Arrays > Linked Lists Hashtable > Search Tree Avoid Code bloating (Loop Unrolling) Custom Data Structures Judy Arrays an associative array/map kD-Trees generalised Binary Space Partitioning Z-Order Curve multidimensional data in one dimension Data Layout Principles
- 56. Data Locality vsJava Heap Layout 0 1 2 class Foo { Integer count; Bar bar; Baz baz; } // No alignment guarantees for (Foo foo : foos) { foo.count = 5; foo.bar.visit(); } 3 ... Foo bar baz count
- 57. Data Locality vsJava Heap Layout Serious Java Weakness Location of objects in memory hard to guarantee. GC also interferes Copying Compaction
- 58. Summary Cache misses causestalls, which kill performance Measurable via Performance Event Counters Common Techniques for optimizing code
- 59. Why care aboutlow level rubbish? Branch Prediction Memory Access Storage Conclusions
- 60. Hard Disks Commonly usedpersistent storage Spinning Rust, with a head to read/write Constant Angular Velocity - rotations per minute stays constant
- 61. A simple model ZoneConstant Angular Velocity (ZCAV) / Zoned Bit Recording (ZBR) Operation Time = Time to process the command Time to seek Rotational speed latency Sequential Transfer TIme
- 62. ZBR implies fastertransfer at limits than centre
- 63. Seeking vs Sequentialreads Seek and Rotation times dominate on small values of data Random writes of 4kb can be 300 times slower than theoretical max data transfer Consider the impact of context switching between applications or threads
- 64.
- 65. Sector alignment isdisk-level irrelevant
- 68. Summary Simple, sequential accesspatterns win Fragmentation is your enemy Alignment can be relevant in RAID scenarios SSDs are a different game
- 69. Why care aboutlow level rubbish? Branch Prediction Memory Access Storage Conclusions
- 70. Software runs onHardware, play nice
- 71. Predictability is arunning theme
- 72.
- 73. YMMV: measured incrementalimprovements >>> going nuts
- 74.
- 76.