• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Python & Perl: Lecture 18

Python & Perl: Lecture 18






Total Views
Views on SlideShare
Embed Views



25 Embeds 426

http://vkedco.blogspot.com 258
http://www.vkedco.blogspot.com 64
http://vkedco.blogspot.in 20
http://vkedco.blogspot.de 16
http://www.vkedco.blogspot.in 8
http://vkedco.blogspot.co.uk 7
http://vkedco.blogspot.com.es 6
http://vkedco.blogspot.pt 6
http://vkedco.blogspot.fr 6
http://vkedco.blogspot.com.br 5
http://vkedco.blogspot.ca 4
http://vkedco.blogspot.mx 3
http://vkedco.blogspot.jp 3
http://vkedco.blogspot.com.au 3
http://vkedco.blogspot.co.il 2
http://vkedco.blogspot.com.ar 2
http://vkedco.blogspot.hu 2
http://vkedco.blogspot.nl 2
http://vkedco.blogspot.ch 2
http://vkedco.blogspot.kr 2
http://vkedco.blogspot.it 1
http://vkedco.blogspot.dk 1
http://vkedco.blogspot.ru 1
http://www.vkedco.blogspot.fr 1
http://www.vkedco.blogspot.com.br 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Python & Perl: Lecture 18 Python & Perl: Lecture 18 Presentation Transcript

    • Python & Perl Lecture 18 Vladimir Kulyukin Department of Computer Science Utah State Universitywww.youtube.com/vkedco www.vkedco.blogspot.com
    • Outline ● OOP Background: Factory Design Pattern ● Generators ● Generator Factories ● Generator Objects ● Generator Comprehensionwww.youtube.com/vkedco www.vkedco.blogspot.com
    • OOP Background: Factory Design Patternwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Design Patterns ● In OOP, design patterns are referred to standardized ways of capturing specify types of behavior ● Design patterns typically evolve bottom-up: after repeating the same task for a few times, a software engineer or a team of software engineers may notice a sequence of actions that may be worth reusing – and a design pattern is bornwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Factory Design Pattern ● Factory is an OOP design pattern that generalizes the constructor concept ● In some circumstances, it is conceptually easier or more efficient to create a class whose objects produce other types of objects ● A factory class has a method for constructing each type of object it is designed to constructwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generatorswww.youtube.com/vkedco www.vkedco.blogspot.com
    • Motivation ● Python generators are beautiful (I know, I know – beauty is in the eye of the beholder) ● They provide for very succinct computational patterns They are useful both in research and systems programming ● They are still relatively under-used: probably because we as programmers feel safer and more confident with for loopswww.youtube.com/vkedco www.vkedco.blogspot.com
    • Definition ● Let us use this definition: a generator is an object that iterates over some data in a lazy fashion ● This definition has several implications: – Generators are iterators – Generators are associated with specific data – Generators are lazy: they yield data items one at a time and without necessarily storing all data items in memorywww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generators as Memory Conservation Devices ● You can think of generators as memory conservation devices ● They allow you to iterate over a set (or several sets) of objects without having to store each and every one of those objects in memory ● You cannot do the same with iterables, such as lists, tuples, or dictionarieswww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Construction ● In Python, there are two ways to construct generators: generator factories and generator comprehensions (generator expressions) ● Generator factories are used when more complex generators are required ● Generator comprehensions are used for simpler generatorswww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Factorieswww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Factories ● Every Python function that has the keyword yield in its body defines a generator factory ● For example: def gen_factory_1234(): yield 1 yield 2 yield 3 yield 4www.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Factories ● Why is gen_factory_1234 a generator factory? ● Because on each invocation it creates a new generator object >>> g1 = gen_factory_123() >>> g1.next() 1 >>> g1.next() 2 >>> g1.next() 3 >>> g1.next() 4 >>> g1.next() StopIteration Exceptionwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Objects ● Unlike functions that return values, generators yield values and remember the point at which the last value is yielded ● On the next invocation, a generator pick up from the point at which the previous value was yielded ● When there are no more values to yield, StopIteration is raisedwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Objects ● Generator objects are one-time only: once a generator object goes over its data (if the number of data items is finite), it cannot be restarted ● You can create as many generator objects coupled to the same data as you want ● Generator objects can be consumed by standard Python sequence constructors or sequence processorswww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Factories ● Here is a more generic way of defining a generator factory that yields each number in a range def gen_factory_range(lower, upper): for i in xrange(lower, upper+1): yield iwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Examples ## two generators defined over the same data >>> gfr1 = gen_factory_range(10, 15) >>> gfr2 = gen_factory_range(10, 15) ## two generator objects consumed by zip constructor >>> zip(gfr1, gfr2) [(10, 10), (11, 11), (12, 12), (13, 13), (14, 14), (15, 15)] >>> gfr3 = gen_factory_range(10, 15) ## generator object consumed by list constructor >>> list(gfr3) [10, 11, 12, 13, 14, 15] ## generator object consumed by sum >>> sum(gen_factory_range(1, 5)) 15www.youtube.com/vkedco www.vkedco.blogspot.com
    • Examples ## generator object consumed by max >>> max(gen_factory_range(1, 5)) 5 ## generator object consumed by set constructor >>> set(gen_factory_range(1, 5)) set([1, 2, 3, 4, 5]) ## generator object used by the boolean in operator >>> 5 in gen_factory_range(1, 5) Truewww.youtube.com/vkedco www.vkedco.blogspot.com
    • Examples ## generator factory of Fibonacci number generators def gen_fib(): prev, curr = 0, 1 yield prev yield curr while True: nxt = prev + curr yield nxt prev, curr = curr, nxtwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Comprehensionwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Generator Comprehension vs. List Comprehension ● Both generator comprehension and list comprehension apply some operation to each item in some data if some condition is true ● List comprehension constructs the list of operation applications ● Generator comprehension yields each operation application without constructing any list ● List returned by list comprehension can be reused ● Generator returned by generator comprehension is used only oncewww.youtube.com/vkedco www.vkedco.blogspot.com
    • Example Suppose there is a data file (e.g., a spread sheet saved as a tabbed file, a bank transactions log, an Apache server log, etc.). Write a function to process data from that file.www.youtube.com/vkedco www.vkedco.blogspot.com
    • Example: DATA.TXT A1 10 14.5 15 18.78 20.1 A2 890 10 997 109 13.45 A3 98 11 791 901 45.8 A4 56 61 94 105 4.1 A5 0.3 0.56 91.2 15 543www.youtube.com/vkedco www.vkedco.blogspot.com
    • Example: Logical Steps of Data File Processing ● Open a file ● Iterate through a file one line at a time ● Extract required data from each line ● Accumulate extracted data in required statistic being computed for the entire file ● When the end of the file is reached, return the statisticwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Examples of Statistics ● Which key has the maximum value for COL0? ● Which key has the minimum sum across all columns, i.e., COL0, COL1, COL2, COL3, and COL4? ● Which key has the maximum sum across all columns?www.youtube.com/vkedco www.vkedco.blogspot.com
    • Example: Data Extractors & Data Collectors ## map data file line to tuple (key, COL0s value) def data_extractor_01(line): split_line = line.split(t) return (split_line[0], float(split_line[1])) ## map data extracted from line to tuple with ## maxium second value def data_collector_01(processed_data): return max(processed_data, key=lambda x: x[1])www.youtube.com/vkedco www.vkedco.blogspot.com
    • Example: Data Extractors & Data Collectors ## map data file line to tuple (key, sum of column values) def data_extractor_02(line): split_line = line.split(t) return (split_line[0], sum(float(x) for x in split_line[1:])) ## map data extracted from line to tuple with ## min second value def data_collector_02(processed_data): return min(processed_data, key=lambda x: x[1])www.youtube.com/vkedco www.vkedco.blogspot.com
    • Example: Data File Analyzer Function ## data_extractor and data_collector are function object references ## both default to the built-in id function def analyze_data_file(data_file, data_extractor=id, data_collector=id): instream = open(data_file) line_data = (data_extractor(line) for line in instream if line != and line != n) rslt = data_collector(line_data) instream.close() return rsltwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Example: Sample Calls >>> analyze_data_file(data.txt, data_extractor=data_extractor_01, data_collector=data_collector_01) (A2, 890.0) >>> analyze_data_file(data.txt, data_extractor=data_extractor_02, data_collector=data_collector_02) (A1, 78.38)www.youtube.com/vkedco www.vkedco.blogspot.com
    • Source Code Availability http://vkedco.blogspot.com/2012/03/python-perl-lecture-18.htmlwww.youtube.com/vkedco www.vkedco.blogspot.com
    • Reading & References ● http://en.wikipedia.org/wiki/Factory_method_pattern ● http://docs.python.org/tutorial/classes.html#generators ● http://www.dabeaz.com/generators/index.html- excellent tutorial on Python generators by David M. Beazleywww.youtube.com/vkedco www.vkedco.blogspot.com
    • Feedback Bugs, comments to vladimir dot kulyukin at usu dot eduwww.youtube.com/vkedco www.vkedco.blogspot.com