Scaling API-first – The story of a global engineering organization
TLPI - 7 Memory Allocation
1. TLPI - Chapter 7 MEMORY ALLOCATION
Shu-Yu Fu (shuyufu@gmail.com)
July 15, 2012
2. TLPI - Chapter 7 MEMORY ALLOCATION
This chapter describes the functions that are used to allocate
memory on the heap or the stack.
3. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
We begin the a description of brk() (program break, the current
limit of the heap) and sbrk(), upon which the malloc functions
are based.
4. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Adjusting the Program Break: brk() and sbrk()
Resizing the heap is actually as simple as telling the kernel to
adjust its idea of where the process’s program break is. After
the program break is increased, the program may access any
address in the newly allocated area, but no physical memory
pages are allocated yet.
1 #include <u n i s t d . h>
2 i n t brk ( void ∗ e n d d a t a s e g m e n t ) ;
3 R e t u r n s 0 on s u c c e s s , o r −1 on e r r o r
The brk() system call sets the program break to the location
specified by end data segment (page-aligned).
1 #include <u n i s t d . h>
2 void ∗ sbrk ( i n t p t r t i n c r e m e n t ) ;
3 R e t u r n s p r e v i o u s program break on s u c c e s s , o r ( void ∗ )−1 on e r r o r
A call to sbrk() adjusts the program break by adding increment
to it. On success, sbrk() returns the previous address of the
program break.
5. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Allocating Memory on the Heap: malloc() and f ree()
1 #include < s t d l i b . h>
2 void ∗ m a l l o c ( s i z e t s i z e ) ;
3 R e t u r n s p o i n t e r t o a l l o c a t e d memory on s u c c e s s , o r NULL on e r r o r
The malloc() function allocate size bytes from the heap and
returns a pointer to the start of the newly allocated block of
memory.
1 #include < s t d l i b . h>
2 void f r e e ( void ∗ p t r ) ;
In general, f ree() doesn’t lower the program break, but instead
adds the block of memory to a list of free blocks.
Making any use of ptr after the call to f ree() is an error that
can lead to unpredictable results.
The glibc f ree() function calls sbrk() to lower the
program break only when the free block at the top end
is ”sufficiently” large, where ”sufficient” is determined
by parameters controlling the operation of the malloc
package (128 KB is a typical value).
6. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Implementation of malloc() and f ree()
Scan the list of memory blocks previously released by
f ree() in order to find one whose size is larger than or
equal to its requirements.
If the block is larger, the it is split, so that a block of the
correct size is returned to the caller and a smaller free
block is left on the free list.
If no block on the free list is large enough, malloc()
increases the program break in larger units, putting the
excess memory onto the free list.
7. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Implementation of malloc() and f ree()
Q: When f ree() places a block of memory onto the free list,
how does it know what size that block is?
A: When malloc() allocates the block, it allocates extra bytes
to hold an integer containing the size of the block.
When a block is placed on the free list, f ree() uses the bytes of
the block itself in order to add the block to the list.
8. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Tools and libraries for malloc debugging
Among the malloc debugging tools provided by glibc are the
following:
The mtrace() and muntrace() functions allow a program
to turn tracing of memory allocation calls on and off.
The mcheck() and mprobe() functions allow a program to
perform consistency check on blocks of allocated memory.
Programs that employ these functions must be linked with
the mcheck library using the −lcheck option.
The MALLOC CHECK environment variable serves a
similar purpose to mcheck() and mprobe() (One notable
difference between the two techniques is that using
MALLOC CHECK doesn’t require modification and
recompilation of the program.). For security reasons, the
setting of MALLOC CHECK is ignored by set-user-ID
and set-group-ID programs.
Further information about all of the above features can be
found in the glibc manual.
9. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Controlling and monitoring the malloc package
The glibc manual describes a range of nonstandard functions
(not portable) that can be used to monitor and control the
allocation of memory by functions in the malloc package.
The mallopt() function modifies various parameters that
control the algorithm used by malloc().
The mallinf o() function returns a structure containing
various statistics about the memory allocated by malloc().
10. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Other Methods of Allocating Memory on the Heap
The C library provides a range of other functions for allocating
memory on the heap.
11. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Allocating memory with calloc() and realloc()
1 #include < s t d l i b . h>
2 void ∗ c a l l o c ( s i z e t numitems , s i z e t s i z e )
3 R e t u r n s p o i n t e r t o a l l o c a t e d memory on s u c c e s s , o r NULL on e r r o r
The calloc() function allocates memory for an array of identical
items. Unlike malloc(), calloc() initializes the allocated memory
to 0.
1 #include < s t d l i b . h>
2 void ∗ r e a l l o c ( void ∗ p t r , s i z e t s i z e )
3 R e t u r n s p o i n t e r t o a l l o c a t e d memory on s u c c e s s , o r NULL on e r r o r
The realloc() function is used to resize (usually enlarge) a block
of memory previously allocated by one of the functions in the
malloc package.
12. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Allocating memory with calloc() and realloc()
For the usual case, where we are increasing the size of the block
of memory,
realloc() attempts to coalesce the block with an
immediately following block of memory on the free list, if
one exists and is large enough.
If the block lies at the end of the heap, then realloc()
expands the heap.
If the block of memory lies in the middle of the heap, and
there is insufficient free space immediately following it,
realloc() allocates a new block of memory and copies all
existing data from the old block to the new block.
13. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Allocating memory with calloc() and realloc()
Since realloc() may relocate the block of memory, we must use
the returned pointer from realloc() for future references to the
memory block.
1 nptr = r e a l l o c ( ptr , newsize ) ;
2 i f ( n p t r == NULL) {
3 /∗ H a n d l e e r r o r ∗/
4 } e l s e { /∗ r e a l l o c ( ) s u c c e e d e d ∗/
5 ptr = nptr ;
6 }
Memory allocated using calloc() or realloc() should be
deallocated with f ree().
14. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Allocating aligned memory: memalign() and posix memalign()
1 #include <m a l l o c . h>
2 void ∗ memalign ( s i z e t boundary , s i z e t s i z e ) ;
3 R e t u r n s p o i n t e r t o a l l o c a t e d memory on s u c c e s s , o r NULL on e r r o r
The memalign() function allocates size bytes starting at an
address aligned to a multiple of boundary, which must be a
power of two.
1 #include < s t d l i b . h>
2 i n t p o s i x m e m a l i g n ( void ∗∗memptr , s i z e t a l i g n m e n t , s i z e t size ) ;
3 R e t u r n s 0 on s u c c e s s , o r a p o s i t i v e e r r o r number on e r r o r
The memory is aligned to a multiple of alignment, which must
be a power-of-two multiple of sizeof (void∗) (4 or 8 bytes on
most hardware architectures). Note also the unusual return
value of this function.
Blocks of memory allocated using memalign() or
posix memalign() should be deallocated with f ree().
15. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Heap
Allocating aligned memory: memalign() and posix memalign()
On some UNIX implementations, it is not possible to
call f ree() on a block of memory allocated via
memalign(), because the memalign() implementation
uses malloc() to allocate a block of memory, and then
returns a pointer to an address with a suitable
alignment in that block. The glibc implementation of
memalign() doesn’t suffer this limitation.
16. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Stack: alloca()
Instead of obtaining memory from the heap, alloca() obtains
memory from the stack by increasing the size of the stack frame.
1 #include <a l l o c a >
2 void ∗ a l l o c a ( s i z e t s i z e ) ;
3 Returns p o i n t e r to a l l o c a t e d block o f memory
We need no call f ree() to deallocate memory allocated with
alloca(). Likewise, it is not possible to use realloc() to resize a
block of memory allocated by alloca().
Older versions of glibc, and some other UNIX
implementations (mainly BSD derivatives), require the
inclusion of <stdlib.h> instead of <alloca.h> to obtain
the declaration of alloca().
17. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Stack: alloca()
If the stack overflow as a consequence of calling alloca(), the
program behavior is unpredictable. In particular, we don’t get a
NULL return to inform us of the error. (In fact, in this
circumstance, we may receive a SIGSEGV signal.)
We can’t use alloca() within a function argument list, as in this
example:
1 func (x , a l l o c a ( s i z e ) , z ) ; /∗ WRONG! ∗/
Instead, we must use code such as this:
1 void ∗y ;
2 y = alloca ( size ) ;
3 func (x , y , z ) ;
18. TLPI - Chapter 7 MEMORY ALLOCATION
Allocating Memory on the Stack: alloca()
Using alloca() to allocate memory has a few advantages over
malloc().
alloca() is faster than malloc().
The memory that alloca() allocates is automatically freed
when the stack frame is removed.
Using alloca() can be especially useful if we employ longjmp()
or siglongjmp() to perform a nonlocal goto from a signal
handler.
19. TLPI - Chapter 7 MEMORY ALLOCATION
Summary
Using the malloc family of functions, a process can
dynamically allocate and release memory on the heap.
The alloca() function allocates memory on the stack.