Python & Perl                           Lecture 19                      Vladimir Kulyukin               Department of Comp...
Outline   ●       Project Proposal   ●       Modules & Packages   ●       File and Directory Processingwww.youtube.com/vke...
Project Proposalwww.youtube.com/vkedco              www.vkedco.blogspot.com
Project Ideas   ●       The best idea is one you find interesting   ●       Sample projects from the past:          – A Ga...
Project Proposal   ●       Basic Description: what your application will do; give a sample       use case (2 - 3 paragraph...
Moduleswww.youtube.com/vkedco         www.vkedco.blogspot.com
Modules   ●       A Python module is a .py file   ●       A .py file may contain any syntactically valid       Python code...
Syntax Patterns for Importing Modules   ●       import module   ●       import module1, module2, …, moduleN   ●       impo...
Telling Python Where to Look: Method 01   ●       Put your modules into one of the paths in       sys.path   ●       To fi...
Telling Python Where to Look: Method 03   ●       There are standard directories specifically created for third-       par...
Telling Python Where to Look: Method 02   ●       This method should be used when you have to put your modules       into ...
Telling Python Where to Look: Method 03   ●       This method should be used when you have to put your       modules into ...
What Python Loads   ●       When Python needs a modules byte code, it generates it       automatically   ●       Python lo...
Packageswww.youtube.com/vkedco         www.vkedco.blogspot.com
Package   ●     A package is a directory that contains a set     of modules   ●     What makes a directory a package is th...
Example   ●       Suppose I want to implement a package       my_math (this is a toy package)   ●       Suppose the packag...
Example: Package Structure       __init__.py       mconstants.py       mclasses.py       my_math_loader.py       add/     ...
Example: mconstants.py         version         = 1.0         app_name        = "My Math"         company_name = "XYZ, LLC"...
Example: mclasses.py       __metaclass__ = type       import mconstants       class Add:         def add(self): pass      ...
Example: add/madd2.py       __metaclass__ = type       import mconstants       import mclasses       def add2(x, y): retur...
Example: add/madd3.py       __metaclass__ = type       import mconstants       import mclasses       def add3(x, y, z): re...
Example: add/__init__.py        ## add package has two modules: madd2        ## and madd3        __all__ = [              ...
Example: mult/mmult2.py       __metaclass__ = type       import mconstants       import mclasses       def mult2(x, y): re...
Example: mult/mmult3.py       __metaclass__ = type       import mconstants       import mclasses       def mult3(x, y, z):...
Example: mult/mmult4.py       __metaclass__ = type       import mconstants       import mclasses       def mult4(x, y, z, ...
Example: mult/__init__.py        ## mult package has three modules:        ## mmult2, mmult3, mmult4        __all__ = [   ...
Example: subt/msubt2.py       __metaclass__ = type       import mconstants       import mclasses       def subt2(x, y): re...
Example: subt/msubt2.py        ## subt package has one module -        ## msubt2        __all__ = [           msubt2,     ...
Example: my_math/__init__.py        ## my_math package has two modules -        ## mconstants and mclasses        __all__ ...
Example: my_math/my_math_loader.py       import sys       my_math_path = /home/vladimir/programming/python/code/my_math   ...
File & Directory Processingwww.youtube.com/vkedco         www.vkedco.blogspot.com
File Objects   ●       A file object is an object that supports a set of methods       such as read, readline, write, writ...
File Objects   ●       Standard streams: stdin, stdout, stderr   ●       Python file objects are iterables: you can iterat...
Example: Processing File Line By Line        def process_file(fn, process_fn=id):           with open(fn, r) as filehandle...
Example: Processing File Character By Character        def process_chars(fn, process_fn=id):          with open(fn, r) as ...
Example: Copying Files        def copy_file(infn, outfn):          with open(infn, r) as filein:            with open(outf...
Exploring Local Directory System   ●       A common scenario: find all files whose name matches a       pattern that may b...
OS.WALK()   ●       os.walk() is the workhorse of processing local file systems   ●       This function uses generators to...
Feedback   Bugs, comments to vladimir dot kulyukin at usu dot eduwww.youtube.com/vkedco            www.vkedco.blogspot.com
Source Code    http://vkedco.blogspot.com/2012/03/python-perl-lecture-19.htmlwww.youtube.com/vkedco                  www.v...
Reading & References  ●      http://docs.python.org/py3k/tutorial/modules.html  ●      www.dabeaz.com/generators  ●      M...
Upcoming SlideShare
Loading in...5
×

Python & Perl: Lecture 19

1,006

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,006
On Slideshare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Python & Perl: Lecture 19

  1. 1. Python & Perl Lecture 19 Vladimir Kulyukin Department of Computer Science Utah State Universitywww.youtube.com/vkedco www.vkedco.blogspot.com
  2. 2. Outline ● Project Proposal ● Modules & Packages ● File and Directory Processingwww.youtube.com/vkedco www.vkedco.blogspot.com
  3. 3. Project Proposalwww.youtube.com/vkedco www.vkedco.blogspot.com
  4. 4. Project Ideas ● The best idea is one you find interesting ● Sample projects from the past: – A Game (this one is really up to you) – Extension of an existing application, e.g. a Perl or Python extension of GIMP – AI module of a robot – XML book database manager – Web-based scheduler for a valuable resourcewww.youtube.com/vkedco www.vkedco.blogspot.com
  5. 5. Project Proposal ● Basic Description: what your application will do; give a sample use case (2 - 3 paragraphs) ● Target User: who will use your application (1 paragraph) ● Target Operating System: list of specific target platforms ● Third-Party Libraries: list and description (with URLs) of third- party libraries, APIs, data sources, etc ● Risks: analysis of the parts of the project you have concerns about; any project carries risks (2-3 paragraphs) ● Skills: describe your skills relevant to your project (2-3 paragraphs)www.youtube.com/vkedco www.vkedco.blogspot.com
  6. 6. Moduleswww.youtube.com/vkedco www.vkedco.blogspot.com
  7. 7. Modules ● A Python module is a .py file ● A .py file may contain any syntactically valid Python code ● Some Python modules are not associated with .py files: – The sys module is not associated with any .py file – Modules written in other languageswww.youtube.com/vkedco www.vkedco.blogspot.com
  8. 8. Syntax Patterns for Importing Modules ● import module ● import module1, module2, …, moduleN ● import module as another_name ● from module import name ● from module import name1, name2, …, nameN ● from module import name as another name ● from module import *www.youtube.com/vkedco www.vkedco.blogspot.com
  9. 9. Telling Python Where to Look: Method 01 ● Put your modules into one of the paths in sys.path ● To find out the current value of sys.path: >>> import sys >>> print sys.path >>> [/home/vladimir, /usr/bin, /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2, /usr/lib/python2.7/lib-tk, /usr/lib/python2.7/lib-old, /usr/lib/python2.7/lib- dynload, /usr/local/lib/python2.7/dist-packages, /usr/lib/python2.7/dist-packages]www.youtube.com/vkedco www.vkedco.blogspot.com
  10. 10. Telling Python Where to Look: Method 03 ● There are standard directories specifically created for third- party modules: – C:Pyhon2.7site-packages (Windows) – /usr/local/lib/python2.7/dist-packages (Linux) ● Problems with Method 01: – You may not have permission to the Python directories – You may not want to clutter the Python directories – You may want to put your modules in a different directorywww.youtube.com/vkedco www.vkedco.blogspot.com
  11. 11. Telling Python Where to Look: Method 02 ● This method should be used when you have to put your modules into a directory, not available in sys.path ● Set the name of the environment variable PYTHONPATH – Windows: Right click on Computer, Properties, Advanced System Settings, Environment Variables – Linux/Unix: Add this line to /etc/bash.bashrc (or similar file); on some Linux flavors, you may have to do this as the superuser export PYTHONPATH=$PYTHONPATH:~/my_python_dirwww.youtube.com/vkedco www.vkedco.blogspot.com
  12. 12. Telling Python Where to Look: Method 03 ● This method should be used when you have to put your modules into a directory, not available in sys.path ● Create a custom loader file that adds a specific directory to sys.path import sys my_math_path = /home/vladimir/programming/python/code/my_math if __name__ == __main__: if not my_math_path in sys.path: sys.path.append(my_math_path)www.youtube.com/vkedco www.vkedco.blogspot.com
  13. 13. What Python Loads ● When Python needs a modules byte code, it generates it automatically ● Python looks for a file with the same name as the modules .py file that has the extension .pyo ● If there is no .pyo file or if it is older than the .py file, Python looks for a file with the extension .pyc ● If an up-to-date .pyc file is found, Python loads it ● If there is no up-to-date .pyc file, Python loads and compiles the .py filewww.youtube.com/vkedco www.vkedco.blogspot.com
  14. 14. Packageswww.youtube.com/vkedco www.vkedco.blogspot.com
  15. 15. Package ● A package is a directory that contains a set of modules ● What makes a directory a package is the presence of the file __init__.py ● __init__.py defines the structure of a package ● __init__.py can be emptywww.youtube.com/vkedco www.vkedco.blogspot.com
  16. 16. Example ● Suppose I want to implement a package my_math (this is a toy package) ● Suppose the package will be implemented in /home/vladimir/programming/python/code/my_math ● The package has three sub-packages: add, mult, and subt ● For clarity, all module names start with mwww.youtube.com/vkedco www.vkedco.blogspot.com
  17. 17. Example: Package Structure __init__.py mconstants.py mclasses.py my_math_loader.py add/ __init__.py madd2.py madd3.py mult/ __init__.py mmult2.py mmult3.py mmult4.py subt/ __init__.py msubt2.pywww.youtube.com/vkedco www.vkedco.blogspot.com
  18. 18. Example: mconstants.py version = 1.0 app_name = "My Math" company_name = "XYZ, LLC"www.youtube.com/vkedco www.vkedco.blogspot.com
  19. 19. Example: mclasses.py __metaclass__ = type import mconstants class Add: def add(self): pass class Mult: def mult(self): pass class Subt: def subt(self): passwww.youtube.com/vkedco www.vkedco.blogspot.com
  20. 20. Example: add/madd2.py __metaclass__ = type import mconstants import mclasses def add2(x, y): return x + y class Add2(mclasses.Add): def __init__(self, x=0, y=0): self.x, self.y = x, y def add(self): return self.x + self.ywww.youtube.com/vkedco www.vkedco.blogspot.com
  21. 21. Example: add/madd3.py __metaclass__ = type import mconstants import mclasses def add3(x, y, z): return x + y + z class Add3(mclasses.Add): def __init__(self, x=0, y=0, z=0): self.x, self.y, self.z = x, y, z def add(self): return self.x + self.y + self.zwww.youtube.com/vkedco www.vkedco.blogspot.com
  22. 22. Example: add/__init__.py ## add package has two modules: madd2 ## and madd3 __all__ = [ madd2, madd3 ]www.youtube.com/vkedco www.vkedco.blogspot.com
  23. 23. Example: mult/mmult2.py __metaclass__ = type import mconstants import mclasses def mult2(x, y): return x * y class Mult2(mclasses.Mult): def __init__(self, x=1,y=1): self.x, self.y = x, y def mult(self): return self.x * self.ywww.youtube.com/vkedco www.vkedco.blogspot.com
  24. 24. Example: mult/mmult3.py __metaclass__ = type import mconstants import mclasses def mult3(x, y, z): return x * y * z class Mult3(mclasses.Mult): def __init__(self, x=1, y=1, z=1): self.x, self.y, self.z = x, y, z def mult(self): return self.x * self.y * self.zwww.youtube.com/vkedco www.vkedco.blogspot.com
  25. 25. Example: mult/mmult4.py __metaclass__ = type import mconstants import mclasses def mult4(x, y, z, w): return x * y * z * w class Mult4(mclasses.Mult): def __init__(self, x=1,y=1,z=1,w=1): self.x, self.y, self.z, self.w = x, y, z, w def mult(self): return self.x * self.y * self.z * self.wwww.youtube.com/vkedco www.vkedco.blogspot.com
  26. 26. Example: mult/__init__.py ## mult package has three modules: ## mmult2, mmult3, mmult4 __all__ = [ mmult2, mmult3, mmult4 ]www.youtube.com/vkedco www.vkedco.blogspot.com
  27. 27. Example: subt/msubt2.py __metaclass__ = type import mconstants import mclasses def subt2(x, y): return x - y class Subt2(mclasses.Subt): def __init__(self, x=1,y=1): self.x, self.y = x, y def subt(self): return self.x - self.ywww.youtube.com/vkedco www.vkedco.blogspot.com
  28. 28. Example: subt/msubt2.py ## subt package has one module - ## msubt2 __all__ = [ msubt2, ]www.youtube.com/vkedco www.vkedco.blogspot.com
  29. 29. Example: my_math/__init__.py ## my_math package has two modules - ## mconstants and mclasses __all__ = [ mconstants, mclasses ]www.youtube.com/vkedco www.vkedco.blogspot.com
  30. 30. Example: my_math/my_math_loader.py import sys my_math_path = /home/vladimir/programming/python/code/my_math if __name__ == __main__: if not my_math_path in sys.path: sys.path.append(my_math_path) import mconstants import mclasses from add import * from mult import * from subt import *www.youtube.com/vkedco www.vkedco.blogspot.com
  31. 31. File & Directory Processingwww.youtube.com/vkedco www.vkedco.blogspot.com
  32. 32. File Objects ● A file object is an object that supports a set of methods such as read, readline, write, writeline, etc ● There are two basic methods for opening and closing files ● open(name[, mode[, buffering]]) – open file object in a specific object; standard modes are r (read), w (write), b (binary) ● close(file_object) – close opened file objectwww.youtube.com/vkedco www.vkedco.blogspot.com
  33. 33. File Objects ● Standard streams: stdin, stdout, stderr ● Python file objects are iterables: you can iterate over file contents line by line or character by character ● Consider using the with statement that lets you allow you to open a file and safely close it even if an exception is thrownwww.youtube.com/vkedco www.vkedco.blogspot.com
  34. 34. Example: Processing File Line By Line def process_file(fn, process_fn=id): with open(fn, r) as filehandle: for line in filehandle: process_fn(line)www.youtube.com/vkedco www.vkedco.blogspot.com
  35. 35. Example: Processing File Character By Character def process_chars(fn, process_fn=id): with open(fn, r) as filein: c = filein.read(1) while c: process_fn(c) c = filein.read(1)www.youtube.com/vkedco www.vkedco.blogspot.com
  36. 36. Example: Copying Files def copy_file(infn, outfn): with open(infn, r) as filein: with open(outfn, w) as fileout: c = filein.read(1) while c: fileout.write(c) c = filein.read(1) fileout.flush()www.youtube.com/vkedco www.vkedco.blogspot.com
  37. 37. Exploring Local Directory System ● A common scenario: find all files whose name matches a pattern that may be scattered all over a local directory system and process them ● Example 01: find all .gz archives and back them up on an external drive ● Example 02: find all .txt files and index them to be searchable ● Example 03: find all .jpg/.bmp/.png files and create an catalogue mapping a file name to its exact location on the local drivewww.youtube.com/vkedco www.vkedco.blogspot.com
  38. 38. OS.WALK() ● os.walk() is the workhorse of processing local file systems ● This function uses generators to walk through a local file system import os ## given some directory, startdir, os.walk ## returns a generator object that yields triples ## of the form (path, dirlist, filelist) for path, dirlist, filelist in os.walk(startdir): ## path is current directory ## dirlist is list of subdirectories of path ## filelist is list of files in pathwww.youtube.com/vkedco www.vkedco.blogspot.com
  39. 39. Feedback Bugs, comments to vladimir dot kulyukin at usu dot eduwww.youtube.com/vkedco www.vkedco.blogspot.com
  40. 40. Source Code http://vkedco.blogspot.com/2012/03/python-perl-lecture-19.htmlwww.youtube.com/vkedco www.vkedco.blogspot.com
  41. 41. Reading & References ● http://docs.python.org/py3k/tutorial/modules.html ● www.dabeaz.com/generators ● M. Hetland, Chrs 09, 11, “Beginning Python: From Novice to Professional,” APRESS nd ● M. Summerfield, Ch. 05, “Programming in Python 3,” 2 Edition, Addison-Wesleywww.youtube.com/vkedco www.vkedco.blogspot.com

×