In this talk CPython reference counting and PyPy incremental mini mark algorithms are explained.

1. Knowing your garbage collector
Francisco Fernandez Castano
Rushmore.fm
francisco.fernandez.castano@gmail.com @fcofdezc
September 27, 2014
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 1 / 30

2. Overview
1 Introduction
Motivation
Concepts
2 Algorithms
CPython RC
PyPy
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 2 / 30

3. Motivation
Managing memory manually is hard.
Who owns the memory?
Should I free these resources?
What happens with double frees?
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 3 / 30

4. Dangling pointers
int * func ( void )
{
int num = 1234;
/* ... */
return &num ;
}
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 4 / 30

5. John Maccarthy
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 5 / 30

6. Basic concepts
Heap
A data structure in which objects may be allocated or deallocated in any
order.
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 6 / 30

7. Basic concepts
Heap
A data structure in which objects may be allocated or deallocated in any
order.
Mutator
The part of a running program which executes application code.
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 7 / 30

8. Basic concepts
Heap
A data structure in which objects may be allocated or deallocated in any
order.
Mutator
The part of a running program which executes application code.
Collector
The part of a running program responsible of garbage collection.
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 8 / 30

9. Garbage collection
De

10. nition
Garbage collection is automatic memory management. While the
mutator runs , it routinely allocates memory from the heap. If more
memory than available is needed, the collector reclaims unused memory
and returns it to the heap.
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 9 / 30

11. CPython GC
CPython implementation has garbage collection.
CPython GC algorithm is Reference counting with cycle detector
It also has a generational GC.
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 10 / 30

12. Young objects
[ elem * 2 for elem in elements ]
balance = (a / b / c) * 4
'asdadsasd - xxx '. replace ('x', 'y'). replace ('a', 'b')
foo.bar ()
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 11 / 30

13. PyObject
typedef struct _object {
_PyObject_HEAD_EXTRA
Py_ssize_t ob_refcnt ;
struct _typeobject * ob_type ;
} PyObject ;
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 12 / 30

14. PyTypeObject
typedef struct _typeobject {
PyObject_VAR_HEAD
const char * tp_name ;
Py_ssize_t tp_basicsize , tp_itemsize ;
destructor tp_dealloc ;
printfunc tp_print ;
getattrfunc tp_getattr ;
setattrfunc tp_setattr ;
void * tp_reserved ;
.
.
} PyTypeObject ;
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 13 / 30

15. Reference Counting Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 14 / 30

16. Reference Counting Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 15 / 30

17. Reference Counting Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 16 / 30

18. Reference Counting Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 17 / 30

19. Reference Counting Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 18 / 30

20. Cycles
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 19 / 30

21. Cycles
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 20 / 30

22. PyObject
typedef struct _object {
_PyObject_HEAD_EXTRA
Py_ssize_t ob_refcnt ;
struct _typeobject * ob_type ;
} PyObject ;
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 21 / 30

23. PyTypeObject
typedef struct _typeobject {
PyObject_VAR_HEAD
const char * tp_name ;
Py_ssize_t tp_basicsize , tp_itemsize ;
destructor tp_dealloc ;
printfunc tp_print ;
getattrfunc tp_getattr ;
setattrfunc tp_setattr ;
void * tp_reserved ;
.
.
} PyTypeObject ;
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 22 / 30

24. Demo
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 23 / 30

25. Reference counting
Pros: Is incremental, as it works, it frees memory.
Cons: Detecting Cycles could be hard.
Cons: Size overhead on objects.
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 24 / 30

26. PyPy
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 25 / 30

27. Mark and Sweep Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 26 / 30

28. Mark and Sweep Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 27 / 30

29. Mark and Sweep Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 28 / 30

30. Mark and Sweep Algorithm
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 29 / 30

31. Mark and sweep
Pros: Can collect cycles.
Cons: Basic implementation stops the world
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 30 / 30

32. Questions?
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 31 / 30

33. The End
Francisco Fernandez Castano (@fcofdezc) Python GC September 27, 2014 32 / 30