This document discusses Python iteration, comprehensions, generators, functional programming idioms, and provides examples of each. It covers that iterators and iterables are defined by implementing certain methods, comprehensions can create lists, sets, dicts or generators from iterables, generators are lazily evaluated and both iterable and iterators, and functional programming idioms like map and filter can often provide more readable solutions than comprehensions. Examples show common idioms for enumerating lists, finding the first occurrence, and collating data from multiple lists.

1. Python
Iteration
Iterators,
Comprehensions,
Generators,
Func4onal
Programming
and
Idioms

2. Iterators
• An
object
is
iterable
if
it
defines
the
__iter__
method
or
implements
the
sequence
protocol
(defines
__ge4tem__
with
integers
star4ng
at
0)
• An
object
is
an
iterator
if
it
implements
the
iterator
protocol:
• Defines
the
__iter__
method
which
returns
itself
• Defines
a
next
method
which
returns
the
next
value
with
each
subsequent
call
• for
loops
implicitly
create
an
iterator
• iter
explicitly
creates
an
iterator
• Generators
encapsulate
logic
in
an
iterator
• All
Sequence
Types
are
iterable
• Dicts,
sets,
and
other
collec4on
containers
are
iterable
• File
objects
are
iterable
• Many
DB
cursors
are
iterable

3. Comprehensions
• Comprehensions
are
iterables
iterable = [3,6,4]
list_comprehension = [i for i in iterable] # [3, 6, 4]
set_comprehension = {i for i in iterable} # {3, 6, 4}
dict_comprehension = {str(i):i for i in iterable} # { ‘3’: 3, ‘6’: 6, ‘4’: 4 }
generator_comprehension = (i for i in iterable) # <generator object …>
• Comprehensions
can
have
1
or
more
for
expressions
• Comprehensions
can
0
or
more
if
expressions
• Comprehensions
can
also
nested
[[j for j in i if j % 2] for i in [iterable] * 3] # [[3], [3], [3]]
[i for i in iterable if not i%2 if i>5] # [6]
[(i,j) for i in iterable for j in iterable if i < 6 and j > 4] # [(3, 6), (4, 6)]

4. Generators
• Generators
can
be
created
as
a
comprehension
or
by
calling
a
func4on
that
uses
yield
instead
of
return
• Generators
are
lazily
evaluated
• Generators
are
both
iterable
and
iterators
def mk_generator():
for i in [1, 2, 3]:
yield i
generator_a = mk_generator()
generator_b = (i for i in [1, 2, 3])
# generator_a and generator_b are functionally equivalent

5. Functional
Programming
• Solu4ons
using
comprehensions
can
oOen
be
rewriPen
using
Func4onal
Programming
(FP)
idioms.
• Always
choose
the
implementa4on
that
provides
the
least
amount
of
code
that
is
readable
and
maintainable
• Always
use
the
iterator
versions
where
possible:
imap, ifilter,
izip, reduce,
etc.
Refer
to
itertools.*
and
buil4ns
Differences:
• Comprehensions
do
not
create
a
new
scope
• FP
idioms
oOen
require
new
func4ons
to
be
created
• FP
idioms
can
be
easily
generalized
to
support
distributed
programming

6. Idiom:
Enumerating
Lists
Some4mes
you
need
to
get
the
index
of
each
element
in
an
ordered
list.
# Non Idiomatic
d = {}
for i, value in enumerate(values):
d[value] = i
# Idiomatic
d = dict(value, i for i, value in enumerate(values))

7. Idiom:
First
Occurrence
Finding
the
first
occurrence
in
a
collec4on
of
data
is
a
common
problem.
# Non Idiomatic
found_line = None
for line in logfile:
if regex.match(line):
found_line = line
break
return found_line
# Idiomatic
return next((line for line in logfile if regex.match(line)), None)

8. Idiom:
Collating
Lists
You
oOen
have
to
collate
data
from
different
lists
together.
This
oOen
done
to
create
a
data
structure
that
represents
the
rela4onship
between
the
data.
The
can
also
be
used
to
create
dicts.
# Non Idiomatic
collated_list = []
for i in xrange(max(len(first_list), len(second_list)):
collated_list.append((first_list[i], second_list[i]))
# Idiomatic
from itertools import izip
collated_list_iterator = izip(first_list, second_list)