Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Type Annotations in Python: Whats, Whys and Wows!

3,180 views

Published on

My EuroPython 2017 talk on type annotations in Python.

Published in: Engineering
  • Be the first to comment

Type Annotations in Python: Whats, Whys and Wows!

  1. 1. Type Annotations in Python: Whats, Whys & Wows Andreas Dewes (@japh44) Europython 2017 - Rimini
  2. 2. Outline Explain why type annotations are interesting and how they came to be. Show you how you can use them in your own code base. Analyze how people actually use them and show you what else you can do with them.
  3. 3. Motivation: When We Discover Bugs Formal Proof of Correctness Design Review Unit / Integration / … Testing Compiler Errors via Type Checker Code Reviews Static Analysis We Don‘t, Our Customer Does Position in Software Life Cycle Enter: Type Hinting
  4. 4. The Python Way: Gradual Typing Unannotated code Annotated code external code our code external type information
  5. 5. History of Type Hints in Python https://www.python.org/dev/peps/**** PEP 482 – Literature Overview PEP 483 – Theory of Type Hints PEP 484 – Type Hints 2014 3.5 PEP 3107 – Function Annotations 2006 3.0 DraftInformal Accepted / Final PEP 544 – Protocols: Structural Subtyping PEP 526 – Variable Annotations 2016 3.6 2017
  6. 6. Annotation Syntax in Python Return type annotation Argument annotation Variable annotation (Python 3.6 only)
  7. 7. Architecture of Type Hinting in Python Python Interpreter Just stores type annotations in a special __annotations__ variable. No runtime effects otherwise! The „typing“ module Allows us to specify the types that we want to annotate our code with (required for non-standard types and advanced use cases) External tools (e.g. mypy) Uses annotations to perform type checking (or other functionality)
  8. 8. Getting Started With Type Hints: MyPy • Originally Written by Jukka Lehtosalo, now also strongly pushed by Dropbox and Guido van Rossum Easy to install & use > pip install mypy > mypy [file to check] … http://mypy-lang.org/
  9. 9. Example Code Base: Flor
  10. 10. Our initial code • Small but functional codebase: Less than 200 lines of code • No external dependencies: Good as an example • Not many „exotic“ types: Easy to annotate • Compatible with Python 2+3: Good for testing different approaches
  11. 11. A Test Script
  12. 12. Adding Type Hints • Go through the code function by function, adding hints to all arguments and return types • Possibly add hints to ambiguous variable initializations (if needed) • Import and use necessary types from the typing module • Try to make mypy happy
  13. 13. I broke mypy! Quick fix: bytes → _bytes
  14. 14. Running MyPy: It (Finally) Works! Argument 1 to „BloomFilter“ has incompatible type „float“; expected „int“ (…) Argument 1 to „add“ of „BloomFilter“ has incompatible type „str“; expected „bytes“ Unsupported operand types for + (List[int] and „str“)
  15. 15. But … now we lost Python 2 compatibility
  16. 16. Second Approach: Type Comments • Instead of writing hints as code, we add them as comments • A special syntax tells mypy to treat them as type hints • We still need to import the „typing“ module (there is a workaround though)
  17. 17. Nice! But what about code we can‘t change?
  18. 18. Stub files (.pyi) Mypy will look for Stub files in several places (search path, current directory, typeshed, …). If it finds a .pyi file and a .py file, it will only load the .pyi! Third Approach: Stub Files
  19. 19. Building a Stub File • As before, start with the code • Remove all actual values and function bodies, just leaving the signatures • Use ellipsis (…) to indicate missing parts • Add type hints • Think „header files“ for Python
  20. 20. Comparing Our Approaches https://travis-ci.org/DCSO/flor commentedinline https://stackoverflow.com/questions/43516780/adding-type-information- without-dependency-on-typing-module
  21. 21. Pros & Cons Inline + The „canonical“ way of using type hints + Easy to read + Code and hints are kept in one place - Only compatible with Python ≥ 3.3 (or ≥ 3.6 if using variable annotations) Type Comments + Keeps code compatible with Python ≥ 2.7 + Allows variable annotations regardless of Python version - Ugly (?) - Still requires importing the „typing“ module (but there is a workaround) Stub Files + Does not modify original source at all + Allows use of latest features regardless of Python version - Duplicates maintenance effort - Does not (yet) allow checking of the actual code against the stubs
  22. 22. There‘s (Much) More To Type Hints! • Generics • Type variables • Classes • Generators • …
  23. 23. Are People Actually Using This? Let‘s Check! We download code from the top 1000 Python repositories on Github and check it for any kind of type hinting: Inline annotations, type comments and stubs. https://github.com/adewes/type-annotations-in-the-wild
  24. 24. 103 projects with at least 1 type annotation 53 projects with at least 10 24 projects with at least 100 inline comments pyi files
  25. 25. Top Python Repositories with Type Hints
  26. 26. Function & Variable Annotations: Not Only For Type Hinting > foo.__annotations__ {‘x‘ : int, ‘return‘ : bool}
  27. 27. Example: Contracts in Python (not saying this is a good idea, just that it works…) https://gist.github.com/adewes/b87c8adc95e768ebf6366130ad6d85a7
  28. 28. Summary Type hinting works & makes your code more robust (for you and for others) You can use it now already (regardless if you use Python 2 or 3) Annotations can do more than type hinting (but think about it twice)
  29. 29. Thanks! Slides: https://slideshare.com/japh44 Me: @japh44 andreas@7scientists.com License: CC BY-NC 3.0: https://creativecommons.org/licenses/by-nc/3.0/en/

×