Your SlideShare is downloading. ×
Python & Perl: Lecture 19
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Python & Perl: Lecture 19

946
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
946
On Slideshare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Python & Perl Lecture 19 Vladimir Kulyukin Department of Computer Science Utah State Universitywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 2. Outline ● Project Proposal ● Modules & Packages ● File and Directory Processingwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 3. Project Proposalwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 4. Project Ideas ● The best idea is one you find interesting ● Sample projects from the past: – A Game (this one is really up to you) – Extension of an existing application, e.g. a Perl or Python extension of GIMP – AI module of a robot – XML book database manager – Web-based scheduler for a valuable resourcewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 5. Project Proposal ● Basic Description: what your application will do; give a sample use case (2 - 3 paragraphs) ● Target User: who will use your application (1 paragraph) ● Target Operating System: list of specific target platforms ● Third-Party Libraries: list and description (with URLs) of third- party libraries, APIs, data sources, etc ● Risks: analysis of the parts of the project you have concerns about; any project carries risks (2-3 paragraphs) ● Skills: describe your skills relevant to your project (2-3 paragraphs)www.youtube.com/vkedco www.vkedco.blogspot.com
  • 6. Moduleswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 7. Modules ● A Python module is a .py file ● A .py file may contain any syntactically valid Python code ● Some Python modules are not associated with .py files: – The sys module is not associated with any .py file – Modules written in other languageswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 8. Syntax Patterns for Importing Modules ● import module ● import module1, module2, …, moduleN ● import module as another_name ● from module import name ● from module import name1, name2, …, nameN ● from module import name as another name ● from module import *www.youtube.com/vkedco www.vkedco.blogspot.com
  • 9. Telling Python Where to Look: Method 01 ● Put your modules into one of the paths in sys.path ● To find out the current value of sys.path: >>> import sys >>> print sys.path >>> [/home/vladimir, /usr/bin, /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2, /usr/lib/python2.7/lib-tk, /usr/lib/python2.7/lib-old, /usr/lib/python2.7/lib- dynload, /usr/local/lib/python2.7/dist-packages, /usr/lib/python2.7/dist-packages]www.youtube.com/vkedco www.vkedco.blogspot.com
  • 10. Telling Python Where to Look: Method 03 ● There are standard directories specifically created for third- party modules: – C:Pyhon2.7site-packages (Windows) – /usr/local/lib/python2.7/dist-packages (Linux) ● Problems with Method 01: – You may not have permission to the Python directories – You may not want to clutter the Python directories – You may want to put your modules in a different directorywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 11. Telling Python Where to Look: Method 02 ● This method should be used when you have to put your modules into a directory, not available in sys.path ● Set the name of the environment variable PYTHONPATH – Windows: Right click on Computer, Properties, Advanced System Settings, Environment Variables – Linux/Unix: Add this line to /etc/bash.bashrc (or similar file); on some Linux flavors, you may have to do this as the superuser export PYTHONPATH=$PYTHONPATH:~/my_python_dirwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 12. Telling Python Where to Look: Method 03 ● This method should be used when you have to put your modules into a directory, not available in sys.path ● Create a custom loader file that adds a specific directory to sys.path import sys my_math_path = /home/vladimir/programming/python/code/my_math if __name__ == __main__: if not my_math_path in sys.path: sys.path.append(my_math_path)www.youtube.com/vkedco www.vkedco.blogspot.com
  • 13. What Python Loads ● When Python needs a modules byte code, it generates it automatically ● Python looks for a file with the same name as the modules .py file that has the extension .pyo ● If there is no .pyo file or if it is older than the .py file, Python looks for a file with the extension .pyc ● If an up-to-date .pyc file is found, Python loads it ● If there is no up-to-date .pyc file, Python loads and compiles the .py filewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 14. Packageswww.youtube.com/vkedco www.vkedco.blogspot.com
  • 15. Package ● A package is a directory that contains a set of modules ● What makes a directory a package is the presence of the file __init__.py ● __init__.py defines the structure of a package ● __init__.py can be emptywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 16. Example ● Suppose I want to implement a package my_math (this is a toy package) ● Suppose the package will be implemented in /home/vladimir/programming/python/code/my_math ● The package has three sub-packages: add, mult, and subt ● For clarity, all module names start with mwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 17. Example: Package Structure __init__.py mconstants.py mclasses.py my_math_loader.py add/ __init__.py madd2.py madd3.py mult/ __init__.py mmult2.py mmult3.py mmult4.py subt/ __init__.py msubt2.pywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 18. Example: mconstants.py version = 1.0 app_name = "My Math" company_name = "XYZ, LLC"www.youtube.com/vkedco www.vkedco.blogspot.com
  • 19. Example: mclasses.py __metaclass__ = type import mconstants class Add: def add(self): pass class Mult: def mult(self): pass class Subt: def subt(self): passwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 20. Example: add/madd2.py __metaclass__ = type import mconstants import mclasses def add2(x, y): return x + y class Add2(mclasses.Add): def __init__(self, x=0, y=0): self.x, self.y = x, y def add(self): return self.x + self.ywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 21. Example: add/madd3.py __metaclass__ = type import mconstants import mclasses def add3(x, y, z): return x + y + z class Add3(mclasses.Add): def __init__(self, x=0, y=0, z=0): self.x, self.y, self.z = x, y, z def add(self): return self.x + self.y + self.zwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 22. Example: add/__init__.py ## add package has two modules: madd2 ## and madd3 __all__ = [ madd2, madd3 ]www.youtube.com/vkedco www.vkedco.blogspot.com
  • 23. Example: mult/mmult2.py __metaclass__ = type import mconstants import mclasses def mult2(x, y): return x * y class Mult2(mclasses.Mult): def __init__(self, x=1,y=1): self.x, self.y = x, y def mult(self): return self.x * self.ywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 24. Example: mult/mmult3.py __metaclass__ = type import mconstants import mclasses def mult3(x, y, z): return x * y * z class Mult3(mclasses.Mult): def __init__(self, x=1, y=1, z=1): self.x, self.y, self.z = x, y, z def mult(self): return self.x * self.y * self.zwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 25. Example: mult/mmult4.py __metaclass__ = type import mconstants import mclasses def mult4(x, y, z, w): return x * y * z * w class Mult4(mclasses.Mult): def __init__(self, x=1,y=1,z=1,w=1): self.x, self.y, self.z, self.w = x, y, z, w def mult(self): return self.x * self.y * self.z * self.wwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 26. Example: mult/__init__.py ## mult package has three modules: ## mmult2, mmult3, mmult4 __all__ = [ mmult2, mmult3, mmult4 ]www.youtube.com/vkedco www.vkedco.blogspot.com
  • 27. Example: subt/msubt2.py __metaclass__ = type import mconstants import mclasses def subt2(x, y): return x - y class Subt2(mclasses.Subt): def __init__(self, x=1,y=1): self.x, self.y = x, y def subt(self): return self.x - self.ywww.youtube.com/vkedco www.vkedco.blogspot.com
  • 28. Example: subt/msubt2.py ## subt package has one module - ## msubt2 __all__ = [ msubt2, ]www.youtube.com/vkedco www.vkedco.blogspot.com
  • 29. Example: my_math/__init__.py ## my_math package has two modules - ## mconstants and mclasses __all__ = [ mconstants, mclasses ]www.youtube.com/vkedco www.vkedco.blogspot.com
  • 30. Example: my_math/my_math_loader.py import sys my_math_path = /home/vladimir/programming/python/code/my_math if __name__ == __main__: if not my_math_path in sys.path: sys.path.append(my_math_path) import mconstants import mclasses from add import * from mult import * from subt import *www.youtube.com/vkedco www.vkedco.blogspot.com
  • 31. File & Directory Processingwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 32. File Objects ● A file object is an object that supports a set of methods such as read, readline, write, writeline, etc ● There are two basic methods for opening and closing files ● open(name[, mode[, buffering]]) – open file object in a specific object; standard modes are r (read), w (write), b (binary) ● close(file_object) – close opened file objectwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 33. File Objects ● Standard streams: stdin, stdout, stderr ● Python file objects are iterables: you can iterate over file contents line by line or character by character ● Consider using the with statement that lets you allow you to open a file and safely close it even if an exception is thrownwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 34. Example: Processing File Line By Line def process_file(fn, process_fn=id): with open(fn, r) as filehandle: for line in filehandle: process_fn(line)www.youtube.com/vkedco www.vkedco.blogspot.com
  • 35. Example: Processing File Character By Character def process_chars(fn, process_fn=id): with open(fn, r) as filein: c = filein.read(1) while c: process_fn(c) c = filein.read(1)www.youtube.com/vkedco www.vkedco.blogspot.com
  • 36. Example: Copying Files def copy_file(infn, outfn): with open(infn, r) as filein: with open(outfn, w) as fileout: c = filein.read(1) while c: fileout.write(c) c = filein.read(1) fileout.flush()www.youtube.com/vkedco www.vkedco.blogspot.com
  • 37. Exploring Local Directory System ● A common scenario: find all files whose name matches a pattern that may be scattered all over a local directory system and process them ● Example 01: find all .gz archives and back them up on an external drive ● Example 02: find all .txt files and index them to be searchable ● Example 03: find all .jpg/.bmp/.png files and create an catalogue mapping a file name to its exact location on the local drivewww.youtube.com/vkedco www.vkedco.blogspot.com
  • 38. OS.WALK() ● os.walk() is the workhorse of processing local file systems ● This function uses generators to walk through a local file system import os ## given some directory, startdir, os.walk ## returns a generator object that yields triples ## of the form (path, dirlist, filelist) for path, dirlist, filelist in os.walk(startdir): ## path is current directory ## dirlist is list of subdirectories of path ## filelist is list of files in pathwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 39. Feedback Bugs, comments to vladimir dot kulyukin at usu dot eduwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 40. Source Code http://vkedco.blogspot.com/2012/03/python-perl-lecture-19.htmlwww.youtube.com/vkedco www.vkedco.blogspot.com
  • 41. Reading & References ● http://docs.python.org/py3k/tutorial/modules.html ● www.dabeaz.com/generators ● M. Hetland, Chrs 09, 11, “Beginning Python: From Novice to Professional,” APRESS nd ● M. Summerfield, Ch. 05, “Programming in Python 3,” 2 Edition, Addison-Wesleywww.youtube.com/vkedco www.vkedco.blogspot.com