Creative Data Analysis with Python

1,238 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,238
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Creative Data Analysis with Python

  1. 1. Creative Data Analysis with Python Grant Paton-Simpson Senior Data & Implementation Specialist Optima Corporation Creator of SOFA Statistics
  2. 2. Great Python Tools Available ● Matplotlib (see Creating Interactive Applications in Matplotlib by Jake Vanderplas http://vimeo.com/63260224) ● Numpy ● Python sets, ordered dicts, named tuples ● PANDAS ● SQL Alchemy, adodbapi, dbapi ● Easy text processing (e.g. HTML) ● CSV ● Python!
  3. 3. Get Inspired!
  4. 4. Flexibility
  5. 5. Use Freedom Responsibly! See http://blog.revolutionanalytics.com/2010/04/when-infographics-go-bad.html etc and http://www.netmagazine.com/features/seven-dirty-secrets-data-visualisation
  6. 6. The point is in there somewhere – honest!
  7. 7. Simple can be best
  8. 8. Make a Simple Point Make complex things simple ● Extract small information from large data ● Present truth, do not deceive ● http://www.dataists.com/2010/10/... … what-data-visualization-should-do-simple-small-truth/
  9. 9. Make it easy for the audience
  10. 10. Flexible analysis needs flexible tools
  11. 11. Matplotlib can do it
  12. 12. is your friend ● How to shift a legend outside the plot ● How to have a major and minor axis ● How to shift x axis labels to the middle of a bar ● ● How to position a triangle a certain percentage along the x axis How to apply a heat map to circles etc etc
  13. 13. Annotations, layers, shape placement and much more!
  14. 14. Example with Percentile Lines
  15. 15. Iterate
  16. 16. Colour adds meaning
  17. 17. The power of ... ● Planned non-obsolescence SQL ● Nothing you can't do ● Scales ● Can decouple ● SQL Alchemy, dbapi, adodbapi etc ● ● In my current role, I use SQL with safe data where there is no significant potential for dangerous input. In this case, the most readable and maintainable way of building SQL strings is to use dicts and string interpolation: “SELECT %(fld1)s, %(fld2)s FROM ...” % {“fld1”: dest_arrive_time, “fld2”: dest_depart_time}. But this is not a good habit otherwise – search on “SQL injection” if you don't know why! Read data using dicts: row[“dest_x”]
  18. 18. dbapi ● con = db.connect(host=...) ● cur = con.cursor() ● sql = “SELECT fname FROM data WHERE age > 40” ● cur.execute(sql) ● print(“, ”.join(x[“fname”] for x in cur.fetchall()))
  19. 19. The power of ... ● ● ● ● ● Text Nothing you can't do HTML Easy to display tabular data, hyperlinks, subreports Clean HTML can be opened as documents and spreadsheets Conditional highlighting e.g. class_str = “class = 'highlight' if age > 10 else ”” html.append(“<td %(class_str)s>%(age_val)</td>”)
  20. 20. Imagine, create, iterate ...

×