Mnh csv python

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Building flexible tools to store
sums and report on CSV data
Presented by
Margery Harrison
Audience level: Novice
09:45 AM...

Python Flexibility
● Basic, Fortran, C, Pascal, Javascript,...
● At some point, there's a tendency to think
the same way, ...

Using Python data structures to
report on CSV data
● Lists
● Sets
● Tuples
● Dictionaries
● CSV Reader
● DictReader
● Coun...

Also,
● Using tuples as dictionary keys
● Using enumerate() to count how many
times you've looped
– See “Loop like a Nativ...

Code Development Method
● Start with simplest possible version
● Test and validate
● Iterative improvements
– Make it pret...

This is a CSV file
color,size,shape,number
red,big,square,3
blue,big,triangle,5
green,small,square,2
blue,small,triangle,1...

https://c1.staticflickr.com/3/2201/2469586703_cfdaf88195.jpg

http://i239.photobucket.com/albums/ff263/peacelovebones/two-
pandas-rolling-1.jpg

CSV DictReader
>>> import csv
>>> import os
>>> with open("simpleCSV.txt") as f:
... r=csv.DictReader(f)
... for row in r:...

Running DictReader

DictReader is sequential

Tabulate All Possible Values

How many of each?
● It's nice to have a listing that shows the
variety of objects that can appear in each
column.
● Next, ...

collections.Counter

Playing with Counters

Index into Counters

Counter + DictReader
Let's use counters to tell us how many of each
value was in each column.

Print number of each value

Output
color
blue : 3
green : 1
red : 2
shape
square : 3
triangle: 3
number
1 : 1
3 : 2
2 : 1
5 : 1
7 : 1
size
small : 3
b...

You might ask, why not this?
for row in r:
for head in r.fieldnames:
field_value = row[head]
possible_values[head].add(fie...

Because, Counter likes to count
Counter({'e': 13, 'l': 12, 'a': 9, 'r': 9, 'g': 7, 'b': 6, 'i': 6, 's':
6, 'u': 6, 'n': 4,...

Output
color
blue : 3
green : 1
red : 2
shape
square : 3
triangle: 3
number
1 : 1
3 : 2
2 : 1
5 : 1
7 : 1
size
small : 3
b...

How many red squares?
● We can use tuples as an index into the
counter
– (red,square)
– (big,red,square)
– (small,blue,tri...

Let's use a simpler CSV
color,size,shape
red,big,square
blue,big,triangle
green,small,square
blue,small,triangle
red,big,s...

Counting Tuples
trying to use magic update()
>>> c=collections.Counter([('a,b'),('c,d,e')])
>>> c
Counter({'a,b': 1, 'c,d,...

Oh well
>>> c.update([(('a','b'),)])
>>> c
Counter({'a': 2, 'b': 2, (('a', 'b'),): 1, 'c,d,e': 1, 'a,b': 1,
('a', 'b'): 1}...

Combo Count Part 1: Initialize

Combo Count 2: Counting

Combo Count 3: Printing

Combo Count Output
color
blue : 3
3 blue in 1 combinations:
('blue', 'big'): 1
('blue', 'small'): 2
3 blue in 2 combinatio...

Well, that's ugly
● We need to make it prettier
● We need to write out to a file
● We need to break things up into Classes

Printing Combination Levels
Number of Squares
Number of Red Squares
Number of Blue Squares
Number of Triangles
Number of R...

Indentation per level
● If we're indexing by tuple, then the
indentation level could correspond to the
number of items in ...

A class write_indent() method
If part of class with counter and msgs dict,
just pass in the tuple:
def write_indent(self, ...

class-less indent_message()
def indent_message(level, msg, sum,
space_per_indent=2, space=' '):
num_spaces = self.space_pe...

Adjustable field widths
Depending on data, we'll want different
field widths
red squares 5
Blue squares 21
Large Red Squar...

Using format to format a format
string
>>> f='{{0:{0:d}s}}'.format(3)
>>> f
'{0:3s}'
>>> f='{{0:{0:d}s}}{{1:{1:d}d}}'.form...

$Format 3 values ● Our formatting string will print 3 values: – String of space chars: {0:s} – Message: {1:[msg_width]s} – ...$

Class For Flexible Indentation

Flexible Indent Class Variables

Flexible Indent Method

Testing IndentMessages class

SimpleCSVReporter
● Open a CSV File
● Create
– Set of possible values
– Set of possible tuples
– Counter indexed by each v...

SimpleCSVReporter class vars

readCSV() begins
initialize sets..

readCSV() continued:
Loop to collect & sum

Write to Report File

Using recursion for limitless
indentation

Recursive print sub-levels

Word transform stubs

General method to test

Test with simpler CSV

Output for simpler CSV

A bigger CSV file
"CCN","REPORTDATETIME","SHIFT","OFFENSE","METHOD","BLOCKSIT
EADDRESS","WARD","ANC","DISTRICT","PSA","NEI...

Deleted all but 4 columns
"SHIFT","OFFENSE","METHOD","DISTRICT"
"MIDNIGHT","HOMICIDE","KNIFE","FIRST"
"MIDNIGHT","SEX ABUS...

Method to run crime report

Output - top

Output - bottom

Improvements
● Allow user-specified order for values, e.g.
FIRST, SECOND, THIRD
● Other means of tabulating
● Keeping trac...

https://c1.staticflickr.com/3/2201/2469586703_cfdaf88195.jpg

Links
This talk: http://www.slideshare.net/pargery/mnh-csv-python
● https://github.com/pargery/csv_utils2
● Also some note...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

Python - Lecture 12 by Ravi Kiran Khareedi 739 views
Xml by Jose Rojas 2246 views
Csv file read and write by comsats universit... 5396 views
The CSV File Strikes Back by Sascha Wenninger 1056 views
CSV - 2016 by Vigyan Ashram 1595 views
Lección 3. Leer un archivo CSV en R by Carlos Pérez Lara 45305 views

View on Slideshare

1 of 61 Ad

View on Slideshare

1 of 61 Ad