How to write a well-behaved Python command line application

14,933 views

Published on

Tutorial #1 from PyCon AU 2012

Python is a fantastic scripting language. It is easy to hack up quick scripts for all sorts of problems - without a lot more effort, that hack can become a robust, easily maintained command line application that your users love.
This tutorial covers how to write useful, well-behaved command line applications that are a joy to use:

* Easily process command line options
* Write a script that can be used interactively or as a filter
* Display help to the user
* Gracefully handle and report errors, to the user and the shell
* Trap and process signals in a robust manner
* Create an easily configured application
* Use a range of the Python standard library modules for easier command line scripting
* Test your application
* Set up the supporting files that any well-behaved application should have, eg. a man page
* How to package your application for other people to use

This tutorial will assume a very basic knowledge of Python and some familiarity with the command line environment of your choice.

Published in: Technology
1 Comment
12 Likes
Statistics
Notes
  • to save others the time I spent finding them, the notes and example code can be found at http://www.curiousvenn.com/2012/08/pycon-australia-2012-tutorial-1-notes/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
14,933
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
84
Comments
1
Likes
12
Embeds 0
No embeds

No notes for slide

How to write a well-behaved Python command line application

  1. 1. How to write a well-behavedPython command lineapplication PyCon AU 2012 Tutorial Graeme Cross
  2. 2. This is an introduction to…• Writing robust, maintainable command line applications• Easily processing command line options• Filters and files for input/output• Handling errors and signals• Testing your app• Documenting and packaging your app
  3. 3. I am assuming…• You know how to program (at least a bit!)• Some familiarity with Python basics• Python 2.6 or 2.7 (but not Python 3)• Know the difference between: – GUI – Command prompt (C:> or ~$ )
  4. 4. The examples and these notes• We will use a number of demo scripts – This is an interactive tutorial – If you have a laptop, join in! – We will use ipython for the demos• Code is on USB sticks being passed around• Code & these notes are also at: http://www.curiousvenn.com/
  5. 5. ipython• A very flexible Python shell• Works on all platforms• Lots of really nice features: – Command line editing with history – Coloured syntax – Edit and run within the shell – Web notebooks, Visual Studio, Qt, … – %lsmagic• http://ipython.org/ http://ipython.org/presentation.html
  6. 6. Prelude
  7. 7. The command line• One liners → shell scripts → applications• Lots of interfaces: – Files – Pipes – User input/output – Processes – Networking• The Unix model: lots of small tools that can be combined in lots of useful ways
  8. 8. What is a “well-behaved” app?• Does one thing well• Flexible – eg. handles input from files or pipes• Robustly handles bad input data• Gracefully handles errors• Well-documented for new users
  9. 9. Why Python for the command line?• Available on a wide range of platforms• Readable, consistent syntax – Easy to write & easy to maintain• Scales well for large apps & libraries• Lots of modules = excellent support for: – Operating system functions (eg POSIX) – Networking – File systems
  10. 10. Why not Python?• Simple one-liners often easier in bash• eg. Neatly list all users in an LDAP group: smbldap-groupshow $1 | tail -1 | tr [:lower:] [:upper:] | sed s/,/ /g | sed s/MEMBERUID: //• Some operating systems are rumoured to not ship with Python• Any other reasons??? Ummmm.....
  11. 11. Be platform aware• Lots of standard library support• No excuse to not support other platforms!• Recommended modules for portability: – os – os.path – shutil – fileinput – tempfile• Lots of other modules in PyPI
  12. 12. Here we go!
  13. 13. if __name__ == __main__• For any Python script, break it up into: – Functions – A “main” function, called from command line• Makes it easy to: – Test functionality – Reuse functions• Example: main1.py
  14. 14. Anatomy of the command line
  15. 15. Files• Reading, writing & appending to files• Text or binary formats• This is a tutorial on its own!• Example: file1.py• Example: file2.py
  16. 16. Pipes• Instead of a filename, pipe input/output• Create chains of tools• Standard pipes: – Input: stdin – Output: stdout & stderr• The sys module has support for these• The fileinput module supports reading from stdin and files• Example: stdout.py
  17. 17. Argument parsing• Allow the user to specify arguments – Edit the script? – Modify a configuration file? – Specify arguments on the command line• Need to handle: – Flags: -h or --help – Strings: “Run Forrest, Run” – Pipes – Invalid number of commands – Ideally: type checking, range checking, etc.
  18. 18. Argument parsing options• Standard library: 3 different modules!• Recommended module: argparse• A series of examples: – uniq1.py → uniq4.py – uniqsort.py• Lots more in PyPI!• Recommended modules from PyPI: – clint – docopt
  19. 19. Argument parsing thoughts• Always provide help at the command line• Be consistent – Short and/or long flags? – Intuitive? – Ensure dangerous flags are obvious – Sensible mnemonics for abbreviated flags
  20. 20. Configuration files• Useful for arguments that: – Could change – Dont change very often – Are probably machine- or user-specific• Number of standard library modules: – ConfigParser (INI file format) – json (human & machine readable) – xml.* (if you must) – As well as csv, plistlib (for Mac .plist)• Dont roll your own config file format!
  21. 21. Calling commands• Python can execute other applications• The subprocess module – The best of the standard library modules – Spawn a process – Read/write the input/output/error pipes – Get return code for error checking – Does not scale well – Examples: subprocess1.py & subprocess2.py
  22. 22. Calling commands, the easy way• The envoy module (from PyPI) – A whole lot easier – More Pythonic – Recommended alternative to the subprocess module – https://github.com/kennethreitz/envoy/ – Example: envoy1.py
  23. 23. Error handling• Robust apps gracefully handle errors – Catch (all reasonable) errors – Report errors to the user• Silently failing is rarely acceptable• Blowing up is not much better!
  24. 24. Error handling: catching errors• Exceptions – Recommended way to handle errors in Python – Also used for non-error notification – Example: exception1.py• Error codes – Some functions return an error code (instead of raising an exception) – Common with C/C++ code interfaced to Python – Best to wrap these and then raise an exception
  25. 25. Error handling: reporting errors• For command line apps: – Print to stderr – Dont just print errors (to stdout)• For daemons/services: – Dedicated log file for the application – Write to the operating system event log(s)• Use the logger module – Dont roll your own! – http://docs.python.org/library/logging.html
  26. 26. Signal handling• Support is provided via the signal module – Can raise signals – Can handle incoming signals• Useful to catch keyboard interrupts – eg. interrupt a long running process• Good form to not ignore system exit events• Example: signal1.py• Example: signal2.py
  27. 27. Lets take a breather...
  28. 28. Testing• Well-tested = happy users and maintainers – Design your app for unit testing – doctest and unittest are two good approaches – nose (from PyPI) builds on unittest – mock for mock testing – pylint and pychecker: good “lint” tools• Python Testing Tools Taxonomy: – Links to testing libraries and tools – http://wiki.python.org/moin/PythonTestingToolsTaxonomy
  29. 29. Documenting your application• Essential: – README.txt (overview) – LICENSE.txt (essential) – CHANGES.txt (application changelog) – User documentation/manual• Formats – Text file – HTML (for online or offline use) – man page – Example: rsync man page
  30. 30. Packaging your application• Another whole tutorial topic!• Use the standard Python distribution tools: – setup.py – PyPI (for public distribution) – http://guide.python-distribute.org/• Other approaches for specific platforms: – Debian package (.deb) – RedHat/SuSE/CentOS (.rpm) – MSI (Windows) – etc.
  31. 31. This is the end...
  32. 32. For more information...• The Python tutorial – http://python.org/• Python Module of the Week – http://www.doughellmann.com/PyMOTW/
  33. 33. Some good books…• “Learning Python”, Mark Lutz• “Hello Python”, Anthony Briggs
  34. 34. In the beginning• http://www.cryptonomicon.com/beginning.html

×