Python 3000 (PyCon, 24-Feb-02007)‏ Guido van Rossum [email_address] [email_address]

What Is Python 3000? The next major Python release To be released as Python 3.0 The first one in a long time to be incompatible But not completely different or unusual Concept first formed around 2000 Py3k nickname was a play on Windows 2000 Goal: to correct my early design mistakes Those that would require incompatibility to fix Reduce cognitive load for first-time learners Work and thinking started for real last year

The next major Python release

To be released as Python 3.0

The first one in a long time to be incompatible

But not completely different or unusual

Concept first formed around 2000

Py3k nickname was a play on Windows 2000

Goal: to correct my early design mistakes

Those that would require incompatibility to fix

Reduce cognitive load for first-time learners

Work and thinking started for real last year

Activity Since Last Year Lots of design discussions (too many, if you ask me :-)‏ Some PEPs were written (but not enough…)‏ Lots of code was written (just the right amount!)‏ (but we're not done yet!!)‏

Lots of design discussions

(too many, if you ask me :-)‏

Some PEPs were written

(but not enough…)‏

Lots of code was written

(just the right amount!)‏

(but we're not done yet!!)‏

Python 3.0 Timeline PEPs to be completed: April 2007 3.0a1: June 2007 3.0 final: June 2008 For comparison, the 2.6 timeline: 2.6a1: December 2007 2.6 final: April 2008 There will also be a 2.7 timeline

PEPs to be completed: April 2007

3.0a1: June 2007

3.0 final: June 2008

For comparison, the 2.6 timeline:

2.6a1: December 2007

2.6 final: April 2008

There will also be a 2.7 timeline

Rest of the Talk Highlight some of the most visible changes print function, dict views, comparisons, unicode, … How to convert 2.x to 3.0 code Notational convention: * = incompletely implemented ** = not yet implemented

Highlight some of the most visible changes

print function, dict views, comparisons, unicode, …

How to convert 2.x to 3.0 code

Notational convention:

* = incompletely implemented

** = not yet implemented

No More Classic Classes In 2.2 … 2.9: class C: # classic class (0.1 … 2.1)‏ class C(object): # new-style class (old now :-)‏ In 3.0: both are new-style classes (just say "classes")‏ Differences are subtle, few of you will notice

In 2.2 … 2.9:

class C: # classic class (0.1 … 2.1)‏

class C(object): # new-style class (old now :-)‏

In 3.0:

both are new-style classes (just say "classes")‏

Differences are subtle, few of you will notice

Print is a Function print x, y -> print(x, y)‏ print x, -> print(x, end=" ")‏ print >>f, x -> print(x, file=f)‏ Automatic translation is 98% correct Fails for cases involving softspace cleverness: print "x
", "y" doesn 't insert a space before y print("x
", "y") does ditto for print "x ", "y"

print x, y -> print(x, y)‏

print x, -> print(x, end=" ")‏

print >>f, x -> print(x, file=f)‏

Automatic translation is 98% correct

Fails for cases involving softspace cleverness:

print "x
", "y" doesn 't insert a space before y

print("x
", "y") does

ditto for print "x ", "y"

Dictionary Views Inspired by Java Collections Framework Remove .iterkeys(), .iteritems(), .itervalues()‏ Change .keys(), .items(), .values()‏ These return a dict view Not an iterator A lightweight object that can be iterated repeatedly .keys(), .items() have set semantics .values() has "collection" semantics supports iteration and not much else

Inspired by Java Collections Framework

Remove .iterkeys(), .iteritems(), .itervalues()‏

Change .keys(), .items(), .values()‏

These return a dict view

Not an iterator

A lightweight object that can be iterated repeatedly

.keys(), .items() have set semantics

.values() has "collection" semantics

supports iteration and not much else

Default Comparison Changed Default ==, != compare object identity (this is unchanged)‏ Default <, <=, >, >= raise TypeError Example: [1, 2, &quot;&quot;].sort() raises TypeError Rationale: 2.x default ordering is bogus depends on type names depends on addresses

Default ==, != compare object identity

(this is unchanged)‏

Default <, <=, >, >= raise TypeError

Example: [1, 2, &quot;&quot;].sort() raises TypeError

Rationale: 2.x default ordering is bogus

depends on type names

depends on addresses

**Unicode Strings Java-like model: strings (the str type) are always Unicode separate bytes type must explicitly specify encoding to go between these Open issues: implementation fixed-width characters for O(1) indexing maybe 3 internal widths: 1, 2, 4 byte characters C API issues (many C APIs use C char* pointers)‏ optimize slicing and concatenation??? lots of issues, supporters, detractors

Java-like model:

strings (the str type) are always Unicode

separate bytes type

must explicitly specify encoding to go between these

Open issues:

implementation

fixed-width characters for O(1) indexing

maybe 3 internal widths: 1, 2, 4 byte characters

C API issues (many C APIs use C char* pointers)‏

optimize slicing and concatenation???

lots of issues, supporters, detractors

The Bytes Type A mutable sequence of small ints (0…255)‏ b[0] is an int; b[:1] is a new bytes object Implemented efficiently as unsigned char[] Has some list-like methods, e.g. .extend()‏ Has some string-like methods, e.g. .find()‏ But none that depend on locale bytes literals: b&quot;ascii or xDD or 12&quot; bytes has .decode() method returning a string str has a .encode() method returning bytes

A mutable sequence of small ints (0…255)‏

b[0] is an int; b[:1] is a new bytes object

Implemented efficiently as unsigned char[]

Has some list-like methods, e.g. .extend()‏

Has some string-like methods, e.g. .find()‏

But none that depend on locale

bytes literals: b&quot;ascii or xDD or 12&quot;

bytes has .decode() method returning a string

str has a .encode() method returning bytes

**New I/O Library Stackable components (inspired by Java, Perl)‏ Lowest level: unbuffered byte I/O platform-specific; don't use C stdio Add buffering Add unicode encoding/decoding encoding explicitly specified or somehow guessed Add CRLF/LF mapping Compatible API open(filename) returns a buffered text file read() and readline() return strings open(filename, &quot;b&quot;) returns a buffered binary file read() returns bytes; can't use readline()‏

Stackable components (inspired by Java, Perl)‏

Lowest level: unbuffered byte I/O

platform-specific; don't use C stdio

Add buffering

Add unicode encoding/decoding

encoding explicitly specified or somehow guessed

Add CRLF/LF mapping

Compatible API

open(filename) returns a buffered text file

read() and readline() return strings

open(filename, &quot;b&quot;) returns a buffered binary file

read() returns bytes; can't use readline()‏

Int/Long Unification There is only one built-in integer type Its name is int Its implementation is like long in Python 2.x C API is a bit murky Performance could use a boost

There is only one built-in integer type

Its name is int

Its implementation is like long in Python 2.x

C API is a bit murky

Performance could use a boost

Int Division Returns a Float Always! Same effect in 2.x with from __future__ import division Use // for int division Use -Q option to Python 2.x to find old usage

Always!

Same effect in 2.x with

from __future__ import division

Use // for int division

Use -Q option to Python 2.x to find old usage

**Raise and Except Changes All exceptions must derive from BaseException Exceptions have __traceback__ attribute Must use raise E(arg) instead of raise E, arg Can still use raise E and raise without args Use raise E(arg).with_traceback(tb)‏ instead of raise E, arg, tb Use &quot;except E as v:&quot; instead of &quot;except E, v:&quot; Variable v is deleted at end of except block!!!

All exceptions must derive from BaseException

Exceptions have __traceback__ attribute

Must use raise E(arg) instead of raise E, arg

Can still use raise E and raise without args

Use raise E(arg).with_traceback(tb)‏

instead of raise E, arg, tb

Use &quot;except E as v:&quot; instead of &quot;except E, v:&quot;

Variable v is deleted at end of except block!!!

Signature Annotations NOT type declarations! Example: def foo(x: &quot;whatever&quot;, y: list(range(3))) -> 42*2: … Argument syntax is (roughly): NAME [':' expr] ['=' expr] Both expressions are evaluated at 'def' time foo.func_annotations is: {'a': &quot;whatever&quot;, 'b': [0, 1, 2], &quot;return&quot;: 84} NO other use is made of these annotations

NOT type declarations!

Example:

def foo(x: &quot;whatever&quot;, y: list(range(3))) -> 42*2: …

Argument syntax is (roughly):

NAME [':' expr] ['=' expr]

Both expressions are evaluated at 'def' time

foo.func_annotations is:

{'a': &quot;whatever&quot;, 'b': [0, 1, 2], &quot;return&quot;: 84}

NO other use is made of these annotations

Keyword-Only Parameters Example def: def foo(a, b=1, *, c=42, d): … Example call: foo(1, 2, d=3)‏ Cannot use: foo(1, 2, 3) # raises TypeError

Example def:

def foo(a, b=1, *, c=42, d): …

Example call:

foo(1, 2, d=3)‏

Cannot use:

foo(1, 2, 3) # raises TypeError

Set Literals {1, 2, 3} is the same as set([1, 2, 3])‏ No empty set literal; use set()‏ No frozenset literal; use frozenset({…})‏ **Set comprehensions: { f ( x ) for x in S if P ( x )} same as set( f ( x ) for x in S if P ( x ))‏

{1, 2, 3} is the same as set([1, 2, 3])‏

No empty set literal; use set()‏

No frozenset literal; use frozenset({…})‏

**Set comprehensions:

{ f ( x ) for x in S if P ( x )}

same as set( f ( x ) for x in S if P ( x ))‏

Absolute Import Same effect in 2.5 with from __future__ import absolute_import Within a package &quot;import foo&quot; does NOT search the package path, only sys.path Use &quot;from . import foo&quot; for relative import Or use from <full-package-name> import foo

Same effect in 2.5 with

from __future__ import absolute_import

Within a package &quot;import foo&quot; does NOT search the package path, only sys.path

Use &quot;from . import foo&quot; for relative import

Or use from <full-package-name> import foo

**String Formatting Examples (see PEP 3101 for more): &quot;See {0}, {1} and {foo}&quot;.format(&quot;A&quot;, &quot;B&quot;, foo=&quot;C&quot;)‏ &quot;See A, B and C&quot; &quot;my name is {0} :-{{}}&quot;.format(&quot;Fred&quot;)‏ &quot;my name is Fred :-{}&quot; &quot;File name {0.foo}&quot;.format(open(&quot;foo.txt&quot;))‏ File name foo.txt &quot;Name is {0[name]}&quot;.format({&quot;name&quot;: &quot;Fred&quot;})‏ &quot;Name is Fred&quot; Shoe size {0:8}&quot;.format(42)‏ &quot;Shoe size 42&quot;

Examples (see PEP 3101 for more):

&quot;See {0}, {1} and {foo}&quot;.format(&quot;A&quot;, &quot;B&quot;, foo=&quot;C&quot;)‏

&quot;See A, B and C&quot;

&quot;my name is {0} :-{{}}&quot;.format(&quot;Fred&quot;)‏

&quot;my name is Fred :-{}&quot;

&quot;File name {0.foo}&quot;.format(open(&quot;foo.txt&quot;))‏

File name foo.txt

&quot;Name is {0[name]}&quot;.format({&quot;name&quot;: &quot;Fred&quot;})‏

&quot;Name is Fred&quot;

Shoe size {0:8}&quot;.format(42)‏

&quot;Shoe size 42&quot;

**Nonlocal Statement def outer(): x = 42 def inner(): nonlocal x # <---- new print(x) x += 1 return inner Doesn't work today; x becomes a local in inner Different keywords proposed: nonlocal, global, outer, … (see PEP 3104)‏

def outer(): x = 42 def inner(): nonlocal x # <---- new print(x) x += 1 return inner

Doesn't work today; x becomes a local in inner

Different keywords proposed:

nonlocal, global, outer, … (see PEP 3104)‏

**Abstract Base Classes? Still highly speculative (no PEP yet)‏ wiki.python.org/moin/AbstractBaseClasses Introduce a standard abstract class hierarchy for type categories like file, container, sequence, iterable etc. Standard types to use these as base classes User-defined types may use these When used, can help distinguishing e.g. sequence from mapping, or file-like behavior, or &quot;stringiness&quot;, or &quot;numericity&quot;, etc.

Still highly speculative (no PEP yet)‏

wiki.python.org/moin/AbstractBaseClasses

Introduce a standard abstract class hierarchy for type categories like file, container, sequence, iterable etc.

Standard types to use these as base classes

User-defined types may use these

When used, can help distinguishing e.g. sequence from mapping, or file-like behavior, or &quot;stringiness&quot;, or &quot;numericity&quot;, etc.

**Switch/Case Statement??? Highly speculative; see PEP 3103 switch EXPR: case EXPR: SUITE case EXPR: # or case in EXPRLIST: SUITE … [else: SUITE] Problem: when to compile EXPR? Would prefer precompilation for faster execution But this would introduce unusual semantics

Highly speculative; see PEP 3103

switch EXPR: case EXPR: SUITE case EXPR: # or case in EXPRLIST: SUITE … [else: SUITE]

Problem: when to compile EXPR?

Would prefer precompilation for faster execution

But this would introduce unusual semantics

Miscellaneous Changes exec becomes a function again range() becomes xrange()‏ input() becomes raw_input()‏ zip() returns an iterator Moved intern() into sys module Renamed __nonzero__ to __bool__ 'as' and 'with' are keywords And more, planned and implemented

exec becomes a function again

range() becomes xrange()‏

input() becomes raw_input()‏

zip() returns an iterator

Moved intern() into sys module

Renamed __nonzero__ to __bool__

'as' and 'with' are keywords

And more, planned and implemented

Miscellaneous Removals classic classes: new-style classes default backticks: use repr()‏ Removed <>: use != apply(): use func(*args)‏ coerce(), __coerce__: not needed dict.has_key(): use key in dict 'softspace' attribute on file objects

classic classes: new-style classes default

backticks: use repr()‏

Removed <>: use !=

apply(): use func(*args)‏

coerce(), __coerce__: not needed

dict.has_key(): use key in dict

'softspace' attribute on file objects

**Library Reform Not my priority Others are interested, but effort seems stalled Need help! May happen after 3.0a1 is released

Not my priority

Others are interested, but effort seems stalled

Need help!

May happen after 3.0a1 is released

*C API Changes Too early to tell what will happen 3rd party extension authors want to know For now, these simple rules: Adding APIs is okay (of course)‏ Deleting APIs is okay Changing APIs incompatibly is NOT OKAY

Too early to tell what will happen

3rd party extension authors want to know

For now, these simple rules:

Adding APIs is okay (of course)‏

Deleting APIs is okay

Changing APIs incompatibly is NOT OKAY

Converting 2.x Code to 3.0 Generic conversion tool exists sandbox/2to3 accurate source-to-source transformation parse tree decorated with whitespace & comments New conversions are easily added create a class from boilerplate add a class variable PATTERN to match nodes add a method transform() to transform one node Separately, Python 2.6 will help can warn about out-of-date usages can provide forward-compatible alternatives

Generic conversion tool exists

sandbox/2to3

accurate source-to-source transformation

parse tree decorated with whitespace & comments

New conversions are easily added

create a class from boilerplate

add a class variable PATTERN to match nodes

add a method transform() to transform one node

Separately, Python 2.6 will help

can warn about out-of-date usages

can provide forward-compatible alternatives

Examples of What It Can Do apply(fun, args, kwds) -> fun(*args, **kwds)‏ d.iterkeys() -> d.keys()‏ exec a in b, c -> exec(a, b, c)‏ print >>sys.stderr, x, -> print(x, end=&quot; &quot;, file=sys.stderr)‏ except E, v: -> except E as v: d.has_key(k) -> k in d intern(s) -> sys.intern(s)‏ a <> b -> a != b; `x` -> repr(x); int -> long automatically adds parentheses where needed

apply(fun, args, kwds) -> fun(*args, **kwds)‏

d.iterkeys() -> d.keys()‏

exec a in b, c -> exec(a, b, c)‏

print >>sys.stderr, x, -> print(x, end=&quot; &quot;, file=sys.stderr)‏

except E, v: -> except E as v:

d.has_key(k) -> k in d

intern(s) -> sys.intern(s)‏

a <> b -> a != b; `x` -> repr(x); int -> long

automatically adds parentheses where needed

Examples of What It Can't Do detect whether d is a dict (in d.iterkeys())‏ detect whether you use d.keys() as a list later turn int()/int() into int()//int()‏ fix code that depends on int() < str()‏ remove redundant code fix custom classes emulating dictionaries fix string exceptions, non-Exception exceptions in general: limited to syntactic conversions can't follow control flow, doesn't do type inference

detect whether d is a dict (in d.iterkeys())‏

detect whether you use d.keys() as a list later

turn int()/int() into int()//int()‏

fix code that depends on int() < str()‏

remove redundant code

fix custom classes emulating dictionaries

fix string exceptions, non-Exception exceptions

in general: limited to syntactic conversions

can't follow control flow, doesn't do type inference

What You Can Do Today Don't worry about stuff that can be automated Don't try to write source-level compatible code Use Python 2.6 when it comes out Write unit tests with maximal coverage Use keys = sorted(d.iterkeys())‏ Use list(d.iterkeys()) when you really need a list Derive all exceptions from Exception Derive all classes from object Don't rely on subtle print/softspace semantics use print line.rstrip(&quot;
&quot;) instead of print line, Use // for int division

Don't worry about stuff that can be automated

Don't try to write source-level compatible code

Use Python 2.6 when it comes out

Write unit tests with maximal coverage

Use keys = sorted(d.iterkeys())‏

Use list(d.iterkeys()) when you really need a list

Derive all exceptions from Exception

Derive all classes from object

Don't rely on subtle print/softspace semantics

use print line.rstrip(&quot;
&quot;) instead of print line,

Use // for int division

Questions