PHPConPl 2013 - Allowed memory size of X bytes exhausted

3,087 views
2,960 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,087
On SlideShare
0
From Embeds
0
Number of Embeds
1,211
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

PHPConPl 2013 - Allowed memory size of X bytes exhausted

  1. 1. Allowed memory size of X bytes exhausted Piotr Pasich piotr.pasich @xsolve.pl @piotrpasich
  2. 2. „A good understanding of how variables are stored and manipulated is essential to becoming a Hacker.” php.net
  3. 3. „A good understanding of how variables are stored and manipulated is essential to becoming a Hacker.” php.net
  4. 4. Have you ever thought why don't you do this?
  5. 5. public function foo() { $a = new stdClass; $b = array(); //(...) unset($a); unset($b); return; }
  6. 6. It might save memory
  7. 7. But it doesn't
  8. 8. It...
  9. 9. S...KS
  10. 10. STACKS
  11. 11. First in, last out
  12. 12. It's mostly used to store variables in functions
  13. 13. So, variables live only in functions
  14. 14. Not PHP functions, but C functions
  15. 15. One PHP function != One C function
  16. 16. They are allocated and freed automatically
  17. 17. At function return the stack is popped
  18. 18. public function foo($file1, $file2){ $obj = new Obj(); $data = array(); $data[] = $obj->importAFile($file1); $data[] = $obj->importAFile($file2); return $data; }
  19. 19. Does the memory allocated for $obj get freed after the return?
  20. 20. Yes, it is
  21. 21. And when it isn't?
  22. 22. public function foo($file1, $file2){ $obj = new Obj(); $data = array(); $data[] = $obj->importAFile($file1); $data[] = $obj->importAFile($file2); return $data; }
  23. 23. public function importAFile($file) { //(...) return $this; }
  24. 24. It frees all variables to which there are no references left
  25. 25. Size limited by OS
  26. 26. HEAPS
  27. 27. Fragmented
  28. 28. No limit on memory size*
  29. 29. You have to manage it
  30. 30. Variables can be accessed globally
  31. 31. Extensions Parsed code Variables
  32. 32. EXTENSIONS
  33. 33. Consume memory
  34. 34. memory_get_usage() profiles the heap
  35. 35. echo memory_get_usage(true);
  36. 36. PHP 5.3 786 432
  37. 37. PHP 5.4 262 144
  38. 38. Upgrade!
  39. 39. Also extensions
  40. 40. PARSED CODE
  41. 41. more code == more memory
  42. 42. more code == more time
  43. 43. but...
  44. 44. OPcache
  45. 45. sudo pecl install zendopcache-7.0.2
  46. 46. Substitute for APC
  47. 47. Stores precompiled script bytecode in shared memory
  48. 48. Difference ab -n 10000 -c 20
  49. 49. Without cache 1743.994 seconds
  50. 50. OPcache 658.408 seconds
  51. 51. VARIABLES
  52. 52. $startMemory = memory_get_usage(); $array = range(1, 100000); echo memory_get_usage() - $startMemory, ' bytes';
  53. 53. C 100 000 * 8 bytes = 800 000 bytes
  54. 54. PHP 14 649 016 bytes
  55. 55. !?
  56. 56. All variables are represented by one structure
  57. 57. The zval
  58. 58. typedef struct _zval_struct { zvalue_value value; /* variable value */ zend_uint refcount__gc; /* reference counter */ zend_uchar type; /* value type */ zend_uchar is_ref__gc; /* reference flag */ } zval;
  59. 59. typedef union _zvalue_value { long lval; /* long value */ double dval; /* double value */ struct { char *val; int len; /* this will always be set for strings */ } str; /* string (always has length) */ HashTable *ht; /* an array */ zend_object_value obj; /* stores an object store handle, and handlers */ } zvalue_value;
  60. 60. Each variable needs much more memory than in C
  61. 61. | 64 bit | 32 bit --------------------------------------------------zval | 24 bytes | 16 bytes + cyclic GC info | 8 bytes | 4 bytes + allocation header | 16 bytes | 8 bytes =================================================== zval (value) total | 48 bytes | 28 bytes =================================================== bucket | 72 bytes | 36 bytes + allocation header | 16 bytes | 8 bytes + pointer | 8 bytes | 4 bytes =================================================== bucket (array element) total | 96 bytes | 48 bytes =================================================== total total | 144 bytes | 76 bytes
  62. 62. 144 bytes * 100 000 = 1 440 0000
  63. 63. What happened to the rest 249 016 bytes?
  64. 64. Unfilled buckets in C
  65. 65. 2^17 = 131 072 131 072 – 100 000 = 31 072 pointers
  66. 66. 31 072 pointers * 8 bytes = 248 576 missing bytes
  67. 67. PHP ain’t C
  68. 68. We can use SplFixedArray
  69. 69. That’s 56 bytes per element
  70. 70. SplFixedArray is writting 33 % faster reading 10% faster
  71. 71. Uff, now you know a lot
  72. 72. „Am I a hacker now?”
  73. 73. Not yet, wait a second
  74. 74. Be like a ninja!
  75. 75. Looking for leaks
  76. 76. PHP Fatal Error: Allowed memory size of 8388608 bytes exhausted
  77. 77. ini_set("memory_limit","12M");
  78. 78. ini_set("memory_limit",”512M");
  79. 79. ini_set("memory_limit",”64G");
  80. 80. Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2798048 bytes) in /***/app/http.php on line 6331
  81. 81. That’s the one way
  82. 82. Let’s move to another
  83. 83. Xdebug will help you
  84. 84. Memory needs time
  85. 85. You can debug it step by step
  86. 86. or by xdebug profiler
  87. 87. xdebug.profiler_enable=1 xdebug.profiler_output_dir=/var/logs/profiler
  88. 88. xdebug profiler
  89. 89. Holy moly!
  90. 90. xdebug.trace_output_name
  91. 91. xdebug.collect_params
  92. 92. Default 0.0004 114272 -> str_split() ../trace.php:8 0.0153 117424 -> ret_ord() ../trace.php:10 1 0.0004 0.0007 114272 117424 -> str_split(string(6)) ../trace.php:8 -> ret_ord(string(1)) ../trace.php:10 3 0.0004 0.0007 114272 117424 -> str_split('Xdebug') ../trace.php:8 -> ret_ord('X') ../trace.php:10 4 0.0004 0.0007 114272 117424 -> str_split('Xdebug') ../trace.php:8 -> ret_ord($c = 'X') ../trace.php:10
  93. 93. What's inside?
  94. 94. What's inside?
  95. 95. You can use Webgrind
  96. 96. You can use Webgrind
  97. 97. Analyzer in PHPStorm
  98. 98. After all
  99. 99. Everything consumes memory
  100. 100. Thank you!

×