Python Evolution

Python Evolution 2.4, 2.5, 2.6, 3.0 by Myroslav Opyr, CTO © Quintagroup 2008

Contents

Python 2.x features

2.4

2.5

2.6

Why Python 3k?

Distribution as competitive benefit

References

Python 2.4 features

built-in sets ( PEP 218 )

generator expressions ( PEP 289 )

new subprocess module (PEP 324)

function/method decorators ( PEP 318 )

Python 2.5 features

abstract syntax tree (AST)

'with' statement ( PEP 343 )

absolute/relative imports ( PEP 328 )

distutils gained dependency info ( PEP 314 )

wsgiref package ( PEP 333 )

coroutines ( PEP 342 )

Non-features

PyPI accepts uploads

Continuous Integration with buildbot

Python 2.6 features

no more string exceptions

The "bytes" Object (PEP 358)

warnings for features to be removed in P3k

multiprocessing (PEP 371)

ast module

setuptools module

2to3

Non-features

Python bugtracker moved to roundup

Questions?

pause...

Python 3k/3000/3.0

Incompatible changes

What Python 3.0 won't be

What Python 3.0 will be

Release Plan

Migration process

2to3

Python 3.0. Evolution philosophy

Joel on Software: Things You Should Never Do, Part I:

Netscape was waiting so long between releases by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

http://www.joelonsoftware.com/articles/fog0000000069.html

What Python 3.0 won't be

PEP-3099 highlights:

Python 3000 will not be case-insensitive

Python 3000 will not be a rewrite from scratch

self will not become implicit

lambda will not be renamed

No braces (from __future__ import braces)

Iterating over a dictionary will continue to yield the keys

The interpreter prompt (>>>) will not change. It gives Guido warm fuzzy feelings

What Python 3.0 will be

PEP-3000 Python 3.0 development guidelines:

PEP Numbering

Timeline

Compatibility and Transition

Implementation Language

PEP-3100 Python 3.0 implementation

"reduce feature duplication by removing old ways of doing things. A general principle of the design will be that one obvious way of doing something is enough."

Python 3.0. Core language

True division becomes default behavior

exec as a statement is not worth it -- make it a function

Add optional declarations for static typing (annotations)

Support only new-style classes; classic classes will be gone

Replace print by a function

The softspace attribute of files goes away.

Use except E1, E2, E3 as err: if you want the error variable.

None becomes a keyword; also True and False

as becomes a keyword

Comparisons other than == and != between disparate types will raise an exception unless explicitly supported by the type

Python 3.0. Core language (cont.)

floats will not be acceptable as arguments in place of ints for operations where floats are inadvertantly accepted

Remove from ... import * at function scope.

Imports

Imports will be absolute by default.

Relative imports must be explicitly specified.

Some expressions will require parentheses that didn't in 2.x:

List comprehensions will require parentheses around the iterables. This will make list comprehensions more similar to generator comprehensions. [x for x in 1, 2] will need to be: [x for x in (1, 2)]

Set literals and comprehensions. {x} means set([x]); {x, y} means set([x, y]).

Python 3.0. Core language (cont.)

To be removed:

String exceptions: use instances of an Exception class

raise Exception, "message": use raise Exception("message")

`x`: use repr(x)

The <> operator: use != instead

Drop unbound methods

__getslice__, __setslice__, __delslice__; remove slice opcodes and use slice objects.

__oct__, __hex__: use __index__ in oct() and hex() instead.

__methods__ and __members__

Python 3.0. Atomic Types

Remove distinction between int and long types; 'long' built-in type and literals with 'L' or 'l' suffix disappear

Make all strings be Unicode, and have a separate bytes() type. The new string type will be called 'str'. See PEP 3137 .

Return iterable views instead of lists where appropriate for atomic type methods (e.g. dict.keys(), dict.values(), dict.items(), etc.); iter* methods will be removed.

To be removed:

file.xreadlines() method

dict.has_key() method; use in operator

list.sort() and builtin.sorted() methods: eliminate cmp parameter

Python 3.0. Built-in Namespace

Make built-ins return an iterator where appropriate (e.g. range(), zip(), map(), filter(), etc.)

Remove input() and rename raw_input() to input(). If you need the old input(), use eval(input()).

Introduce trunc(), which would call the __trunc__() method on its argument; suggested use is for objects like float where calling __int__() has data loss, but an integral representation is still desired

Exception hierarchy changes

Add a bin() function for a binary representation of integers

Python 3.0. Built-in Namespace (cont.)

To be removed:

apply(): use f(*args, **kw) instead

buffer(): must die (use a bytes() type instead)

callable(): just use hasattr(x, '__call__')

compile(): put in sys

coerce(): no longer needed

execfile(), reload(): use exec()

intern(): put in sys

reduce(): put in functools, a loop is more readable most of the times

xrange(): use range()

StandardError: this is a relic from the original exception hierarchy; subclass Exception instead.

Python 3.0. Release Plan

PEP-361: Python 2.6 and 3.0 are lock-stepped (the same procedure was used for 1.6 and 2.0):

Feb 29 2008: Python 2.6a1 and 3.0a3 are released

Apr 02 2008: Python 2.6a2 and 3.0a4 are released

May 08 2008: Python 2.6a3 and 3.0a5 are released

Jun 18 2008: Python 2.6b1 and 3.0b1 are released

Jul 02 2008: Python 2.6b2 and 3.0b2 planned

Aug 06 2008: Python 2.6rc1 and 3.0rc1 planned

Aug 20 2008: Python 2.6rc2 and 3.0rc2 planned

Sep 03 2008: Python 2.6 and 3.0 final

Python 3.0. Migration

Python 3.0 will break backwards compatibility with Python 2.x.

There is no requirement that Python 2.6 code will run unmodified on Python 3.0. Not even a subset.

Python 2.6 will support forward compatibility in the following two ways:

It will support a "Py3k warnings mode" which will warn dynamically (i.e. at runtime) about features that will stop working in Python 3.0, e.g. assuming that range() returns a list.

It will contain backported versions of many Py3k features, either enabled through __future__ statements or simply by allowing old and new syntax to be used side-by-side (if the new syntax would be a syntax error in 2.x).

Python 3.0. Migration (cont.)

The recommended development model for a project that needs to support Python 2.6 and 3.0 simultaneously is as follows:

You should have excellent unit tests with close to full coverage.

Port your project to Python 2.6.

Turn on the Py3k warnings mode.

Test and edit until no warnings remain.

Use the 2to3 tool to convert this source code to 3.0 syntax. Do not manually edit the output!

Test the converted source code under 3.0.

If problems are found, make corrections to the 2.6 version of the source code and go back to step 3.

When it's time to release, release separate 2.6 and 3.0 tarballs (or whatever archive form you use for releases).

It is recommended not to edit the 3.0 source code until you are ready to reduce 2.6 support to pure maintenance (i.e. the moment when you would normally move the 2.6 code to a maintenance branch anyway).

Python 3.0. Questions?

pause...

Distributing Python code

Why distribution is important?

distutils

setuptools

easy_install

virtualenv

Private PyPI

Why distribution is important?

Simpler installation makes Python software reusable andmore accessible

More users

More pressure to package developers

Better distributions

more meta-data

more documentation

more tests

more platforms

"CheeseCake" rating system (better package kwalitee)

distutils

For end-users:

python setup.py install

For developers:

python setup.py sdist

For packagers:

python setup.py bdist

Since Python-2.5, you can host your packages directly on PyPI

python setup.py upload

setuptools

Automatically find/download/install/upgrade dependencies at build time using the EasyInstall tool , which supports downloading via HTTP, FTP, Subversion, and SourceForge, and automatically scans web pages linked from PyPI to find download links. (It's the closest thing to CPAN currently available for Python.)

easy_install grockproject

Create Python Eggs - a single-file importable distribution format

python setup.py bdist_egg

setuptools (cont.)

Command aliases - create project-specific, per-user, or site-wide shortcut names for commonly used commands and options

PyPI upload support - upload your source distributions and eggs to PyPI

Deploy your project in "development mode", such that it's available on sys.path, yet can still be edited directly from its source checkout.

Easily extend the distutils with new commands or setup() arguments, and distribute/reuse your extensions for multiple projects, without copying code.

Create extensible applications and frameworks that automatically discover extensions, using simple "entry points" declared in a project's setup script.

easy_install

easy_install SQLObject easy_install -f http://pythonpaste.org/package_index.html SQLObject easy_install http://example.com/path/to/MyPackage-1.2.3.tgz easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg easy_install --upgrade PyProtocols easy_install . easy_install --editable --build-directory ~/projects SQLObject

virtualenv

virtualenv is a tool to create isolated Python environments. The basic usage is:

$ python virtualenv.py ENV

This creates ENV/lib/python2.4/site-packages , where any libraries you install will go. It also creates ENV/bin/python, which is a Python interpreter that uses this environment. Allows bootstrap scripts that prepare environment. Alternatives:

workingenv

virtual-python

zc.buildout

Private PyPI

Folder

...with eggs exposed (and password-protected) via Apache. Works with easy_install, requires "manual" uploading to folder, can be as simple as copy to shared folder

haufe.eggserver grok-powered solution. Recently upload support added. No dedicated security model (everyone is trusted and allowed to upload packages).