Uploaded on

Objects in memory, Objects at python level, Objects in python virtual machine

Objects in memory, Objects at python level, Objects in python virtual machine

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,971
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
38
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.
    • Python Objects
      • Mykola Kharechko, Developer
      • © Quintagroup, 2008
  • 2. Table of contents
      • Objects in memory
      • Objects at python level
      • Objects in python virtual machine
  • 3. Objects in memory
      • Object structure
      • Python memory management
      • Garbage collection
  • 4. Object structure
  • 5. Objects with same class
  • 6. Python Memory Management
  • 7. Allocation Strategy For small requests, the allocator sub-allocates <Big> blocks of memory. Requests greater than 256 bytes are routed to the system's allocator.
  • 8. Pymalloc structure Arena 256kb Pool 4kb Fixed size block
  • 9. Fixed size block * Request in bytes Size of allocated block Size class idx * ------------------------------------------------------------------------------ * 1-8 8 0 * 9-16 16 1 * 17-24 24 2 * 25-32 32 3 * 33-40 40 4 * 41-48 48 5 * 49-56 56 6 * 57-64 64 7 * 65-72 72 8 * ... ... ... * 241-248 248 30 * 249-256 256 31
  • 10. Garbage Collection Python performs garbage collection via reference counting and a cyclic garbage collector that is able to detect and break reference cycles.
  • 11. Example >>> import gc >>> gc.disable() # 2.9mb >>> a = range(1000000) # 18mb >>> del a # 14mb >>> gc.enable() # 14mb >>> gc.collect() # 14mb 0 >>> gc.disable() ‏ >>> a = range(1000000) # 18mb >>> a.append(a) # 18mb >>> del a # 18mb >>> gc.enable() # 18mb >>> gc.collect() # 14 mb 1 >>>
  • 12. Weak references
      • weakref module
      • weakref.ref class
      • weakref.proxy class
      • weakref.WeakKeyDictionary
  • 13. weakref example >>> from weakref import ref, proxy >>> class A(object):pass ... >>> a = A() ‏ >>> ref_a = ref(a) ‏ >>> a.a = 16 >>> ref_a().a 16 >>> proxy_a = proxy(a) ‏ >>> proxy_a.a 16 >>> proxy_a <weakproxy at 0x94e25f4 to A at 0x94e0fcc> >>> del a >>> ref_a() ‏ >>> ref_a().a Traceback (most recent call last): File &quot;<stdin>&quot;, line 1, in <module> AttributeError: 'NoneType' object has no attribute 'a' >>> proxy_a <weakproxy at 0x94e23c4 to NoneType at 0x81479b8>
  • 14. List object typedef struct { PyObject_VAR_HEAD // ob_size /* Vector of pointers to list elements. list[0] is ob_item[0], etc. */ PyObject **ob_item; // 0 <= ob_size <= allocated // len(list) == ob_size Py_ssize_t allocated; } PyListObject;
  • 15. List creation PyObject * PyList_New(Py_ssize_t size) ‏ { PyListObject *op; size_t nbytes; .... nbytes = size * sizeof(PyObject *); ..... op->ob_item = (PyObject **) PyMem_MALLOC(nbytes); memset(op->ob_item, 0, nbytes); ... op->ob_size = size; op->allocated = size; ... }
  • 16. Integer objects typedef struct { PyObject_HEAD long ob_ival; } PyIntObject;
  • 17. Integer optimization. Problem >>> a = -2 >>> b = -2 >>> id(a) == id(b) ‏ True >>> a = 300 >>> b = 300 >>> id(a) == id(b) ‏ False
  • 18. Integer optimization. Solution #define NSMALLPOSINTS 257 #define NSMALLNEGINTS 5 /* References to small integers are saved in this array so that they can be shared. The integers that are saved are those in the range -NSMALLNEGINTS (inclusive) to NSMALLPOSINTS (not inclusive). */ static PyIntObject * small_ints[NSMALLNEGINTS + NSMALLPOSINTS];
  • 19. Objects at python level
      • Objects creation
      • Object attributes
      • Specific attributes
  • 20. Objects creation Constructor: obj = type->tp_new(type, args, kwds); type = Py_TYPE(obj); type->tp_init(obj, args, kwds); Classes created by metaclasses
  • 21. Object attributes Attributes stored in __dict__ or in object when __slots__ used Attribute can be accessed via descriptor protocol The most famous descriptor - property
  • 22. Specific attributes
      • __getattribute__ -access to all attrs
      • __setattr__ -opposite to __getattribute__
      • Descriptor protocol: __get__, __set__, __delete__
      • Pickler protocol: __getstate__, __setstate__, __reduce__
  • 23. Objects in PVM typedef struct { PyObject_HEAD int co_argcount; /* #arguments, except *args */ int co_nlocals; /* #local variables */ int co_stacksize; /* #entries needed for evaluation stack */ int co_flags;/* CO_..., see below */ PyObject *co_code; /* instruction opcodes */ PyObject *co_consts; /* list (constants used) */ PyObject *co_names;/* list of strings (names used) */ PyObject *co_varnames;/* tuple of strings (local variable names) */ PyObject *co_freevars;/* tuple of strings (free variable names) */ PyObject *co_cellvars; /* tuple of strings (cell variable names) */ /* The rest doesn't count for hash/cmp */ PyObject *co_filename;/* string (where it was loaded from) */ PyObject *co_name;/* string (name, for reference) */ int co_firstlineno;/* first source line number */ PyObject *co_lnotab;/* string (encoding addr<->lineno mapping) */ void *co_zombieframe; /* for optimization only (see frameobject.c) */ } PyCodeObject;
  • 24. CodeObject example File test.py a = 17 b = a + 3 print b
  • 25. CodeObject Example >>> import test 34 >>> pyc = open('test.pyc') ‏ >>> pyc.read(8) ‏ 'xb3xf2 >Cx8eH' >>> from marshal import load >>> co = load(pyc) ‏ >>> dir(co) ‏ [... 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames'] >>> co.co_consts (17, None) ‏ >>> co.co_names ('a', 'b') ‏
  • 26. CodeObject example >>> from dis import dis >>> dis(co) ‏ 1 0 LOAD_CONST 0 (17) ‏ 3 STORE_NAME 0 (a) ‏ 2 6 LOAD_NAME 0 (a) ‏ 9 LOAD_CONST 0 (17) ‏ 12 BINARY_ADD 13 STORE_NAME 1 (b) ‏ 3 16 LOAD_NAME 1 (b) ‏ 19 PRINT_ITEM 20 PRINT_NEWLINE 21 LOAD_CONST 1 (None) ‏ 24 RETURN_VALUE