Python3 @ Linuxtage 2010

1,934 views

Published on

Getting started with Python 3 coming from Python 2. How to port your libraries.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,934
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
31
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Python3 @ Linuxtage 2010

  1. 1. 3 Armin Ronacher http://lucumr.pocoo.org/ // armin.ronacher@active-4.com // http://twitter.com/mitsuhiko
  2. 2. About Me •using Python since version 2.2 •not-a-committer-with-commit-rights!? •Part of the Pocoo Team: Jinja, Werkzeug, Sphinx, Zine, Flask
  3. 3. A little bit of History
  4. 4. 1991 Python Appears ‣ Exceptions ‣ Multiple Inheritance ‣ C inspired IO system
  5. 5. 1995 Python 1.5 ‣ Regular Expressions ‣ Exceptions are Classes ‣ Built in Package Support ‣ It can be embedded!
  6. 6. 2000 Python 2.0! Spoiler: this is where it gets interesting ‣ Unicode Support ‣ Augmented assignments (+=, -= etc.) ‣ List Comprehensions ‣ Garbage Collector ‣ Python Enhancement Proposals! Footnote: some of this was in 1.6 already
  7. 7. 2004 Python 2.4! Spoiler: unicode becomes interesting ‣ Source Encoding ‣ Boolean Type (True/False) ‣ sets ‣ reverse iteration ‣ generator expressions Footnote: some of this was in 2.3 already
  8. 8. Not all is Fine
  9. 9. String Coercion It’s bytes — it’s charpoints !!!!"!#!$%&'&($ !!!!)!#!*$+,()&-$ !!!!$."--/!$!0!" 1."--/!%&'&(1 !!!!$."--/!$!0!) *1."--/!+23&4()&-1 !!!!5(67'!8 ."--/!+,()&-
  10. 10. String Coercion [fail] It’s an exception !!!!"!#!*$%&'&($ !!!!)!#!$+,()&-$ "#$%&'$%()9:/;'!(&<&7'!<"--!-";'=* ))>>> +,-%./&0&%./&1##.#*)?6))&(6;@
  11. 11. Print is a Statement why is that? !!!!5(67'!$.&--/$A!4B .&--/!4B !!!!5(67'9$.&--/$A!4B= 91.&--/1A!4B= !!!!3!#!5(67' "#$%&'$%()9:/;'!(&<&7'!<"--!-";'=* ))>>> 23,4$51##.#*)67C"-6D!;E7'"3
  12. 12. O HAI — IM TEH ITERATOR … and I want my place !!!!"!#!F1G//1H!IA!1)"(1H!BJ !!!!">K&E;9= L1G//1A!1)"(1M !!!!">6'&(K&E;9= ND6<'6/7"(EOK&E6'&("'/(!/)P&<'!>>>Q
  13. 13. Exceptions let’s do some guessing #$-6&!" #$-6&!"A!)A!< #$-6&!99"A!)=A!<=A!D &5%&74!RH &5%&74!RA!+H &5%&74!9RA!+=A!SH
  14. 14. Exceptions improved that should clear it up: #$-6&!" #$-6&!)>T6'@8'("<&)"<K9<= #$-6&!99"A!)=A!<=A!D &5%&74!RH &5%&74!R!$6!+H &5%&74!9RA!+=!$6!SH
  15. 15. New Stuff in 3.x
  16. 16. non-local names Re-bind variables from outer scopes: /&8!:"K&8</*7'&(9676'6"-#U=H !!!!C"-!#!676'6"- !!!!/&8!67<9=H !!!!!!!!,.,9.%$9!C"- !!!!!!!!(C!#!C"- !!!!!!!!C"-!0#!I !!!!!!!!#&4:#,!(C !!!!#&4:#,!67<
  17. 17. New Literals Set Literal !!!!"!#!FIA!BA!VJ !!!!)!#!;&'9= Footnote: there is no literal for an empty set
  18. 18. Extended Unpacking Unpack in variables !!!!G6(;'A!;&</7DA!W(&;'!#!*75"<K9= Footnote: The rest is a list with all the unassigned items
  19. 19. Abstract Base Classes Duck Typing Improved !!!!%9$66!XEY'&("'/(9/)P&<'=H ;;;!!!!/&8!887&3'889;&-G=H ;;;!!!!!!!!#$-6&!Z'/5Y'&("'6/79= ;;;!!!!/&8!886'&(889;&-G=H ;;;!!!!!!!!#&4:#,!;&-G ;;; !!!!8#.<!</--&<'6/7;!-<7.#4!Y'&("'/( !!!!6;67;'"7<&9XEY'&("'/(9=A!Y'&("'/(= [(*& Footnote: Implicit subclasses, awesome!
  20. 20. Annotations! Add type information to your signatures 8#.<!7*:)&(;!-<7.#4!*:)&( /&8!"DD9"H!*:)&(A!)H!*:)&(=!OQ!*:)&( !!!!#&4:#,!"!0!) Footnote: These are unchecked, Sphinx can use them
  21. 21. Goodies print a function, super with magic !!!!"!#!5(67' !!!!"94B= 4B !!!!%9$66!["?Z&'9;&'=H ;;;!!!!!/&8!"DD9;&-GA!'"?=H ;;;!!!!!!!!!;*5&(9=>"DD9'"?>-/T&(9== ;;; !!!!'E5&9675*'9== .&--/!]/(-D^ N'E5&!_;'(`Q Brrr: How does super() work? You don’t wanna know
  22. 22. New Land
  23. 23. Backwards Incompatible be courageous ‣ Break up with the Past ‣ Fix the issues ‣ Help the transition with automation
  24. 24. Side By Side how can we do that? ‣ Python 3.x and Python 2.x are developed side by side ‣ Python 2.x slowly adopts features that come from 3.x or make the automated conversion easier
  25. 25. Automation 2to3 for the rescue ‣ A tool to convert from Python 2.x sources to 3.x ‣ Operates on the parsing-time information only. ‣ Not 100% correct, but close
  26. 26. There Be Dragons
  27. 27. Step by Step #1 Library or Application? ‣ 2to3 caters for both ‣ But the experience differs ‣ generally: application easier
  28. 28. Step by Step #2 Unicode or Bytes? ‣ did your API use bytes? ‣ did it use unicode? ‣ what should it use now?
  29. 29. Step by Step #3 Are you sure? ‣ Beware of the stdlib ‣ things unicodified along the way ‣ explicit encoding!
  30. 30. Step by Step #4 Adapt to new idioms ‣ str.format instead of % ‣ keys() and not iterkeys(), beware on custom classes ‣ abstract base classes ‣ the new IO system
  31. 31. Step by Step #5 Port your Tests ‣ Make sure the tests run on 3.x ‣ Get rid of your doctests. Now! ‣ make sure 2to3 only has to correct syntax, not semantics. Runtime checks.
  32. 32. 2to3 will byte you
  33. 33. Unicode Breakage Affects many 2.x libraries ‣ class implements ==:,-%./&== ‣ and ==64#== calls into ==:,-%./&== ‣ on 3.x: RuntimeError ‣ Can be fixed with a custom fixer
  34. 34. Preprocessing If everything else fails … ‣ write a preprocessor ‣ monkey-patch yourself in -8!;E;>C&(;6/7867G/!Q#!9VA!U=H !!!!8#.<!5(&5(/<&;;/(!-<7.#4!(&G"<'/(8;'(67? !!!!8#.<!-6)B'/V>(&G"<'/(!-<7.#4!a&G"<'/(67?[//- !!!!a&G"<'/(67?[//->(&G"<'/(8;'(67?!#!(&G"<'/(8;'(67? Footnote: that’s what SQLAlchemy does
  35. 35. Changing Protocols What if you depend on something? ‣ WSGI changes and is broken right now ‣ web applications are an unrealistic target for the time being. Rejoice: WSGI 1.1 is coming around
  36. 36. Fixing with Fixers
  37. 37. What are Fixers? 2to3 does not work at runtime ‣ 2to3 is guessing ‣ has certain assumptions of your code ‣ fixers operate on a parse tree ‣ fixers can refactor and modify code
  38. 38. Anatomy of a Fixer A fixer to rename functions 8#.<!-6)B'/V!-<7.#4!G63&(8)";& 8#.<!-6)B'/V>G63&(8*'6-!-<7.#4!":&A!+-"7Kb67& %9$66!c63R-'d76</D&9!"#$%&'()$*+()$,"#=H !!!!%R[[ea!#!$$$ !!!!G*7<#G*7<D&GN!1D&G1!7":&#188*76</D&881 !!!!!!!!!!!!!!!!!!5"(":&'&(;N!191!RXe!1=1!Q!"7E0!Q !!!!$$$ !!!!/&8!'("7;G/(:9)$-!A!7/D&A!(&;*-';=H !!!!!!!!7":&!#!(&;*-';L17":&1M !!!!!!!!7":&>(&5-"<&9":&9188;'(881A!5(&G63#7":&>5(&G63==
  39. 39. Running custom Fixers use distribute! ‣ distribute replaces setuptools ‣ has builtin 2to3 support ‣ the distribution system for 3.x distribute: http://pypi.python.org/pypi/distribute
  40. 40. Hook into distribute add this to your setup.py &3'("!#!FJ -8!;E;>C&(;6/7867G/!Q#!9VA!U=H !!!!&3'(">*5D"'&9 !!!!!!!!*;&8B'/V#[(*&A !!!!!!!!*;&8B'/V8G63&(;#L1<*;'/:8G63&(;1M !!!!= ;&'*59>>>A!WW&3'("= distribute: http://pypi.python.org/pypi/distribute
  41. 41. Porting Jinja2 What is it? ‣ Jinja2 — a template engine ‣ Codebase: 10KLOC ‣ In 2.x unicode based
  42. 42. Porting Jinja2 Porting Experience? ‣ # custom fixers: 3 ‣ preprocessing? nope ‣ passing tests? all! (except for doctests in the documentation)
  43. 43. Porting Jinja2 An Example Fixer 8#.<!-6)B'/V!-<7.#4!G63&(8)";&A!5E'(&& 8#.<!-6)B'/V>G63&(8*'6-!-<7.#4!":&A!+-"7Kb67&A!":&A!R''(A!R(?b6;' %9$66!c63+(/K&7a&("6;67?9!"#$%&'()$*+()$,"#=H !!!!%R[[ea!#!$$$ !!!!("6;&8;':'N!1("6;&1!"7E!1A1!C"-#"7E!1A1!')#"7E!Q !!!!$$$ !!!!f!(*7!)&G/(&!'@&!)(/K&7!B'/V!<@&<K&(!T6'@!'@&!;":&!?/"- !!!!f!'(6&;!'/!(&T(6'&!6'!T6'@!"!(*-&!'@"'!D/&;!7/'!T/(K!/*'!G/(!P67P" !!!!(*78/(D&(!#!I !!!!/&8!'("7;G/(:9)$-!A!7/D&A!(&;*-';=H !!!!!!!!')!#!(&;*-';L1')1M><-/7&9= !!!!!!!!')>5(&G63!#!11 !!!!!!!!T6'@8')!#!R''(9(&;*-';L1C"-1M><-/7&9=A!":&91T6'@8'("<&)"<K1==!0!2 !!!!!!!!!!!!!!!!!!LR(?b6;'9L')M=M !!!!!!!!7&T!#!5E'(&&>/D&9;&-G>;E:;>;6:5-&8;':'A!L":&91("6;&1=M!0!T6'@8')= !!!!!!!!7&T>5(&G63!#!7/D&>5(&G63 !!!!!!!!#&4:#,!7&T
  44. 44. Recipe follow these steps for enlightenment 1) port doctests to unittest or your test suite 2) decide on your API for Python 3 3) check if 2to3 works on the tests 4) make the tests work 5) run 2to3 on your own code 6) add runtime fixes or custom fixers
  45. 45. Unladen Swallow
  46. 46. Unladen Swallow Making Python Fast 1) LLVM JIT based 2) a development branch of Python 3) currently targeting 2.x, but major changes will land in 3.x
  47. 47. Numbers Current Status Pystones Plurk Profile 70 52.5 35 17.5 0 Python 2.5 Python 2.6 US Q1 US Q2
  48. 48. Current Status Not very impressive 1) It’s not there yet 2) There will be a development branch 3) Python is hard to optimize, will take some time
  49. 49. Ported Libraries
  50. 50. Ported Libraries Web Related Libraries ‣ Jinja2 ‣ Mako ‣ lxml ‣ SQLAlchemy ‣ httplib2
  51. 51. Ported Libraries General Purpose ‣ blinker ‣ Pygments ‣ PyYAML ‣ tc [Tokio Cabinet]
  52. 52. Ported Libraries Database / GUI ‣ sqlite3 [part of stdlib] ‣ py-postgresql / pg8000-py3 ‣ PyQt
  53. 53. In The Pipeline WSGI ‣ Werkzeug ‣ CherryPy ‣ mod_wsgi Moving Target: WSGI 1.1 spec probably based on mod_wsgi for Python
  54. 54. Any Questions?
  55. 55. Legal licensed under the creative commons attribution-noncommercial- share alike 3.0 austria license © Copyright 2010 by Armin Ronacher images in this presentation used under compatible creative commons licenses. sources: http://www.flickr.com/photos/42311564@N00/2355590508/ http://www.flickr.com/photos/special/1597251/ http://www.flickr.com/photos/reillyandrew/2943521972/ http://www.flickr.com/photos/etobicokesouth/566912940/ http:// www.flickr.com/photos/azrasta/4528604334/ http://www.flickr.com/photos/oneeighteen/481873844/ http://www.flickr.com/photos/ mar00ned/260473163/ http://www.flickr.com/photos/revilla/1347710195/ http://www.flickr.com/photos/tommyhj/346075714/ http:// www.flickr.com/photos/asoka/1443066116/ http://www.flickr.com/photos/thetruthabout/4285919377/ http://www.flickr.com/photos/ nasedojp/2317015728/ http://www.flickr.com/photos/dullhunk/202872717/ http://www.flickr.com/photos/carbonnyc/3149965963/ http://www.flickr.com/photos/beezly/213880808/ http://www.flickr.com/photos/juniorvelo/4490511204/ http://www.flickr.com/photos/ niwota-studios/3586953497/ http://www.flickr.com/photos/the-lees/134610871/ http://www.flickr.com/photos/kubina/131673530/ http://www.flickr.com/photos/bootbearwdc/37621686/ http://www.flickr.com/photos/spursfan_ace/2328879637/ http://www.flickr.com/ photos/9619972@N08/1182211122/ http://www.flickr.com/photos/alanvernon/3659772103/ http://www.flickr.com/photos/jurvetson/ 17095584/ http://www.flickr.com/photos/lemsipmatt/4291448020/ http://www.flickr.com/photos/eurodrifter/182713903/ http:// www.flickr.com/photos/xfp/3286570553/

×