This document provides an overview of PHP memory usage and management. It introduces key concepts like the Zend Memory Manager (ZendMM), which handles memory allocation and freeing for each PHP request. The document demonstrates how to monitor memory usage from PHP using functions like memory_get_usage() and from the OS perspective using /proc. It also discusses potential memory issues like references and circular references, and how to track reference counts. The goal is to help understand and optimize PHP memory consumption.

1. PHP 5 memory
Understand and master

2. Hello everybody
 Julien PAULI
 Programming with PHP since early 2000s
 Programming in C
 PHP Internals programmer/reviewer
 PHP 5.5 & 5.6 Release Manager
 @julienpauli
 Tech blog at jpauli.github.io
 jpauli@php.net

3. What about you ?
 Got some Unix/Linux understandings ?
 Have already experienced C programming ?
 Have already experienced PHP programming ?

4. What we'll cover together
 Memory , what's that ?
 bytes, stack, heap, etc.
 Measuring a process memory consumption
 memory image map analysis
 Understanding PHP memory consumption
 Zend Memory Manager coming
 Measuring PHP memory consumption
 from PHP land or from system land

5. Memory from hardware

6. Memory from software

7. Memory in a user process
 The Virtual Memory image is divided in segments
 text
 data
 heap
 stack

8. Memory usage can grow
 Stack will grow as functions will get called
 And will narrow as the calls stop and return
 Heap will grow as the programmer will decide
 Using dynamic allocation functions (malloc, mmap)
 Programmer has to free memory by hand
 If not : memory leak

9. Monitoring memory

10. Linux memory monitoring
 'top' or /proc FS
> cat /proc/28754/status
VmPeak: 20452 kB
VmSize: 20324 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 316 kB
VmRSS: 316 kB
VmData: 16440 kB
VmStk: 136 kB
VmExe: 4 kB
VmLib: 1664 kB
VmPTE: 28 kB
VmSwap: 0 kB
Size of the VM map (out of total mem)
Resident Set Size : Size actually in PM
Size of the data segment in VM
Size of the stack segment in VM
Size of the text segment in VM
pid

11. Going even deeper
 Let's show the detailed process memory map :
> cat /proc/28754/smaps
shared segment
private mem
shared mem

12. PHP !

13. PHP, just a process
 PHP is just a process like any other
 You then can monitor its memory usage by asking
the OS
<?php
passthru(sprintf('cat /proc/%d/status', getmypid()));
<?php
function heap() {
return shell_exec(sprintf('grep "VmRSS:" /proc/%d/status', getmypid()));
}

14. The PHP model
Startup
Shutdown
Request #1
Request #2
Request #3
Startup
Shutdown
Request #1
Request #2
Request #3
php-fpm worker#1 php-fpm worker#2

15. The PHP model
 One same process may treat many requests
 If the process leaks memory, you'll suffer from that
 Request n+1 must know nothing about request n
 Need to "flush" the request-allocated memory
 Need to track request-bound memory claims
 ZendMM is the layer that does the job
 Share-nothing architecture : by design.

16. Inside PHP
 PHP internal request-bound heap allocator :
Zend Memory Manager

17. Why a custom layer ?
 ZendMM
 Allows monitoring of request-bound heap usage by
basically counting malloc/free calls
 Allows the PHP user to limit the heap mem usage
 Allows caching of alloced blocks to prevent memory
fragmentation and syscalls
 Allows preallocating blocks of well-known-sizes for
PHP internal structures to fit-in in an aligned way
 Ease memory leaks debugging in core and exts

18. ZendMM guards and leak tracker
 Only works in debug mode PHP
 report_memleaks=1 in php.ini
 Does NOT replace valgrind

19. Zend MM test script
<?php
ini_set('memory_limit', -1); /* unlimited ZendMM heap */
function heap() {
return shell_exec(sprintf('grep "VmRSS:" /proc/%d/status', getmypid()));
}
echo heap();
$a = range(1, 1024*1024); /* Stress memory by allocating */
echo heap();
unset($a); /* Stress memory by freeing */
echo heap();

20. Zend MM launch test
> time USE_ZEND_ALLOC=1 php zendmm.php
VmRSS: 9640 kB
VmRSS: 159296 kB
VmRSS: 10068 kB
real 0m0.237s
user 0m0.148s
sys 0m0.080s
> time USE_ZEND_ALLOC=0 php zendmm.php
VmRSS: 9608 kB
VmRSS: 148988 kB
VmRSS: 140804 kB
real 0m0.288s
user 0m0.176s
sys 0m0.108s

21. Valgrind tests for Symfony2 command
 app/console runs lots of PHP code under SF2
>USE_ZEND_ALLOC=1 valgrind php app/console debug:container
total heap usage: 84,000 allocs, 84,000 frees, 25,966,154 bytes allocated
>USE_ZEND_ALLOC=0 valgrind php app/console debug:container
total heap usage: 813,570 allocs, 813,570 frees, 74,579,732 bytes allocated

22. ZendMM internals

23. ZendMM benefits
 Better memory management
 More memory efficient
 Far less malloc/free calls
 Less context switches, less Kernel stress
 Less CPU usage
 Less heap fragmentation / compaction
 A PHP ~10% faster with ZendMM enabled
 Really depends on use-case

24. Memory manager internals
 Layer on top of the heap
 Will allocate memory from the heap by chunks of
customizable size (segments)
 Will use a customizable low level heap (malloc /
mmap)

25. A quick word on ZendMM internals
 ZEND_MM_SEG_SIZE env variable to customize
segment size
 Must be power-of-two aligned
 Default value is 256Kb

26. ZendMM in PHP user land
 memory_limit (INI setting)
 memory_get_usage(true)
 Returns the size of all the allocated segments
 memory_get_usage()
 Returns the occupied size in all the allocated
segments
 memory_get_[peak]_usage([real])
 Returns the max memory that has been
allocated/used. Could have been freed meantime

27. ZendMM memory monitoring

28. ini_set('memory_limit', -1); // unlimited memory
function get_mem_stats() {
printf("Memory usage %.2f Kbn", memory_get_usage() / 1024);
if ($segSize = getenv('ZEND_MM_SEG_SIZE')) {
printf("Heap segmentation : %d segments of %d bytes (%d Kb used)n",
memory_get_usage(1)/$segSize, $segSize, memory_get_usage(1)/1024);
}
}
get_mem_stats();
$a = str_repeat('a', 1024*1024*10); // 10 Mb
get_mem_stats();
Adjusting heap segment size

29. Adjusting heap segment size
 Segment size = 256Kb (default value)
 Segment size = 4Mb
$> ZEND_MM_SEG_SIZE=262144 php /tmp/mem.php
Memory usage 221.26 Kb
Heap segmentation : 1 segments of 262144 bytes (256 Kb used)
Memory usage 10461.47 Kb
Heap segmentation : 42 segments of 262144 bytes (10752 Kb used)
get_mem_stats();
$a = str_repeat('a', 1024*1024*10); // 10 Mb
get_mem_stats();
$> ZEND_MM_SEG_SIZE=4194304 php /tmp/mem.php
Memory usage 221.26 Kb
Heap segmentation : 1 segments of 4194304 bytes (4096 Kb used)
Memory usage 10461.47 Kb
Heap segmentation : 4 segments of 4194304 bytes (16384 Kb used)

30. memory_get_usage() ?
 memory_get_usage() tells you how much your
allocated blocs consume
 They usually don't fill their segment entirely
 Thus memory_get_usage(true) shows more
 This doesn't count stack
 This does only count request-bound memory
 This doesn't count linked libraries present in the
process memory map
 This doesn't show non-request-bound memory

31. Real memory image
 valgrind massif

32. Massif memory monitoring
--------------------------------------------------------------------------------
n time(ms) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
0 0 0 0 0 0
1 309 56 37 19 0
2 341 384 357 27 0
3 376 1,343,992 1,340,371 3,621 0
4 408 1,360,480 1,355,289 5,191 0
5 443 1,376,512 1,367,265 9,247 0
6 471 1,753,976 1,735,671 18,305 0
7 505 2,345,152 2,275,927 69,225 0
8 535 2,405,224 2,329,798 75,426 0
9 570 2,433,408 2,348,849 84,559 0
10 613 2,592,912 2,461,650 131,262 0
11 657 2,681,240 2,534,247 146,993 0

33. Massif details
94.52% (2,534,247B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->39.11% (1,048,576B) 0x87A7BB: zend_interned_strings_init (zend_string.c:48)
|
->17.70% (474,464B) 0x866449: _zend_hash_quick_add_or_update (zend_alloc.h:95)
| ->16.48% (441,936B) 0x85F66D: zend_register_functions (zend_API.c:2138)
|
->11.93% (320,000B) 0x878625: gc_init (zend_gc.c:124)
| ->11.93% (320,000B) 0x8585DF: OnUpdateGCEnabled (zend.c:82)
| ->11.93% (320,000B) 0x86E353: zend_register_ini_entries (zend_ini.c:208)

34. Recommendations / statements
 Understand memory_get_usage()
 It only shows request-bound allocations, not
persistent allocations (that reside through requests)
 PHP extensions may allocate persistent memory
 Do NOT activate extensions you will not use
 Libraries used by PHP may also allocate persistent
memory
 Use your OS to monitor your process memory
accurately

35. Playing with memory in PHP user land

36. Master your PHP mem usage
 In PHP land ...
 all variable types consume memory
 every script asked for compilation will eat memory
 This memory will be allocated using ZendMM
 The memory for parsed script is freed when the
request ends
 The memory for user variable is freed when the
data is not used any more
 And here comes the challenge
 When isn't the data needed any more ??

37. Compilation eats memory
 Compiling a script eats request-bound memory
 If you compile a class, that eats lots of memory
 You'd better use that class at runtime
 Use an autoloader to be sure
<?php
$mem = memory_get_usage();
require __DIR__ . '/../vendor/autoload.php';
require __DIR__ . '/../src/Symfony/Component/DependencyInjection/ContainerBuilder.php'
echo memory_get_usage() - $mem . "n";
php app/bar.php
246,692

38. PHP Variables
 Variables are internally zval structs

39. PHP Variables
 What eats memory is what is stored into the zval,
not really the zval itself :
 A huge string
 Tip : a file_get_contents('huge-file') is a huge string
 A complex array or object
 Resources don't really consume mem in zval
<?php
/* This consumes sizeof(zval) + 1024 bytes */
$a = str_repeat('a', 1024);

40. PHP Variables
 What you want to avoid is have PHP duplicate the
zval
 But PHP is kind about that
 What you want to happen in PHP freeing the
memory ASAP
 Should you know when PHP duplicates or frees zval
, that's the most important !

41. zvals and refcount
 PHP simply counts how many symbols (PHP
vars) point to a zval
 This is called the refcount
<?php
$a = "foo";
$b = $a;

42. zvals and refcount
 PHP uses a CopyOnWrite (COW) system for
zvals
 Memory is saved
 Memory gets allocated only on changes
<?php
$a = "foo";
$b = $a;
$a = 17;

43. zvals and refcount
 PHP frees memory for a zval
when its refcount reaches 0
 Yes, unset() just refcount-- ,
that's all
<?php
$a = "foo";
$b = $a;
$c = $b;
$b = "bar";
unset($a);

44. No references needed
 You see how smart PHP is with memory ?
 It's been designed with that in mind
 No references needed to hint PHP !
 Don't try to hint PHP with references
 References can lead to adverse effects
 Force PHP to copy a zval
 Prevents PHP from freeing memory of a zval
&

45. Tracking refcount
 xdebug_debug_zval()
 symfony_zval_info()
namespace Foo;
class A
{
public $var = 'varA';
}
$a = new A();
xdebug_debug_zval('a');
a: (refcount=1, is_ref=0)=class FooA { public $var =
(refcount=2, is_ref=0)='varA'; }

46. Tracking refcount
namespace Foo;
class C
{
public $b;
public function __construct(B $b) {
$this->b = $b;
}
}
$c = new C($b = new B);
xdebug_debug_zval('c');
c: (refcount=1, is_ref=0)=class FooC { public $b = (refcount=2, is_ref=0)=class FooB { } }
namespace Foo;
class B
{
}
unset($b);
xdebug_debug_zval('c');
c: (refcount=1, is_ref=0)=class FooC { }
c: (refcount=1, is_ref=0)=class FooC { public $b = (refcount=1, is_ref=0)=class FooB { } }
unset($c->b);
xdebug_debug_zval('c');

47. Garbage collector ?
 As of PHP5.3 , a garbage collector exists
 Used to free circular references
 And that's all !
 PHP already frees itself your vars as their
refcount reaches 0
 And it's always been like that

48. Circular references
$a = new A;
$b = new B
(object) 'A'
refcount = 1
$a (object) 'B'
refcount = 1
$b

49. Circular references
$a->b = $b;
$b->a = $a;
(object) 'A'
refcount = 2
$a (object) 'B'
refcount = 2
$b
$b->a $a->b

50. Circular references
 Objects are still in memory but no more PHP var
point to them
 We can call that a "PHP Userland memory leak"
unset($a, $b);
(object) 'A'
refcount = 1
$b->a (object) 'B'
refcount = 1
$a->b

51. Garbage collector
$a = new A;
$b = new B;
$a->b = $b;
$b->a = $a;
unset($a, $b);
echo gc_collect_cycles(); // 2

52. PHP references

53. PHP references main line
 Using references (&) in PHP can really fool you
 They usually force the engine to duplicate memory
containers
 Which is bad for performance
 Especially when the var container is huge

54. References mismatch
<?php
function foo($data)
{
echo "in function : " . memory_get_usage() . "n";
}
echo "Initial memory : " . memory_get_usage() . "n";
$r = range(1, 1024);
echo "Array created : " . memory_get_usage() . "n";
foo($r);
echo "End of function " . memory_get_usage() . "n";
Initial memory : 227.136
Array created : 374.912
in function : 374.944
End of function 374.944

55. References mismatch
<?php
function foo($data)
{
echo "in function : " . memory_get_usage() . "n";
}
echo "Initial memory : " . memory_get_usage() . "n";
$r = range(1, 1024);
$r2 = &$r;
echo "Array created : " . memory_get_usage() . "n";
foo($r);
echo "End of function " . memory_get_usage() . "n";
Initial memory : 227.208
Array created : 375.096
in function : 473.584
End of function 375.128

56. When does the engine separate ?
 In any mismatch case :
zval passed to function arginfo decl. zval received in function separated by engine?
is_ref=0
refcount = 1
pass_by_ref=0 is_ref=0
refcount = 2
NO
is_ref=1
refcount > 1
pass_by_ref=0 is_ref=1
refcount =1
YES
is_ref=0
refcount > 1
pass_by_ref=0 is_ref=0
refcount > 1 ++
NO
is_ref=0
refcount = 1
pass_by_ref=1 is_ref=1
refcount = 2
YES
is_ref=1
refcount > 1
pass_by_ref=1 is_ref=1
refcount > 1 ++
NO
is_ref=0
refcount > 1
pass_by_ref=1 is_ref=1
refcount = 2
YES

57. Symfony_debug to notice mismatches
 https://github.com/symfony/debug
function bar($var) { }
$a = "foo";
$b = &$a;
bar($a);
Notice: Separating zval for call to function 'bar' in /tmp/memory.php on line 20

58. foreach behavior

59. Foreach separation behavior
 It happens sometimes foreach() duplicates your
variable for iteration
 This will eat performances in case of big or
complex arrays being iterated
 There is nothing special to say about objects
implementing Traversable
 The behavior is then just yours

60. foreach iterating array #1
 If the variable has a refcount of 1, no duplication is
performed by foreach()
$a = range(1,1024);
echo memory_get_usage() . "n" ;
foreach ($a as $v) {
if ($v == 1) {
echo memory_get_usage() . "n" ;
}
}
echo memory_get_usage() . "n" ;
373936
374056
374056

61. foreach iterating array #2
 If the variable has a refcount >1, foreach() will
duplicate it fully for iteration
$a = range(1,1024);
$b = $a;
echo memory_get_usage() . "n" ;
foreach ($a as $v) {
if ($v == 1) {
echo memory_get_usage() . "n" ;
}
}
echo memory_get_usage() . "n" ;
373936
472512
374056

62. foreach iterating array #3
 If the variable is a reference, foreach() will work
onto that array and won't perform duplication
$a = range(1,1024);
$b = &$a;
echo memory_get_usage() . "n" ;
foreach ($a as $v) {
if ($v == 1) {
echo memory_get_usage() . "n" ;
}
}
echo memory_get_usage() . "n" ;
373936
374056
374056

63. Monitoring memory usage
 Not much tools exist (for PHP)
 memory_get_usage()
 OS' help (/proc , pmap , etc...)
 Valgrind with massif tool
 PHP Extensions
 Xdebug
 memprof
 memtrack

64. Quick intro to memprof
 https://github.com/arnaud-lb/php-memory-profiler
$b = range(1, 1024 * 1024); /* a lot of memory */
$b[] = foo();
loader('/Zend/Date.php'); /* a lot of PHP source code */
memprof_dump_callgrind(fopen('/tmp/trace.out', 'w'));

65. Thank you for listening