Python and sysadmin I

3,991 views

Published on

introduce python for system administrator.

Published in: Technology
1 Comment
13 Likes
Statistics
Notes
No Downloads
Views
Total views
3,991
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
165
Comments
1
Likes
13
Embeds 0
No embeds

No notes for slide

Python and sysadmin I

  1. 1. Python & SA Guixing<khsing.cn@gmail.com>
  2. 2. do what?
  3. 3. install software or hardware
  4. 4. backup
  5. 5. repair
  6. 6. monitor
  7. 7. analysis
  8. 8. adjust
  9. 9. even more
  10. 10. find program bugs
  11. 11. fix it
  12. 12. meeting
  13. 13. HOPE FOR
  14. 14. Automatic
  15. 15. try
  16. 16. before try
  17. 17. No God
  18. 18. Not Perfect
  19. 19. More light
  20. 20. Continuable
  21. 21. introduce Generator
  22. 22. iteration
  23. 23. for
  24. 24. [list] & (tuple) >>>for i in [1,2,3,4] ... print i 1 2 3 4 >>>for i in (1,2,3,4): ...
  25. 25. {dict} >>>d = {‘a’:1,’b’:2} >>>for i in d: ... print i,d[i] a1 b2
  26. 26. “string” >>>for i in “python”: ... print i p y t h o n
  27. 27. also file
  28. 28. file >>>for i in open(‘example.txt’): ... print i 1st line 2nd line
  29. 29. iterable
  30. 30. .__iter__(self) .next(self)
  31. 31. class countdown(object): def __init__(self,start): self.count = start def __iter__(self): return self def next(self): if self.count <= 0: raise StopIteration r = self.count self.count -= 1 return self
  32. 32. >>>a = countdown(5) >>>for i in a: ... print i ... 54321 >>>
  33. 33. yield
  34. 34. def d1(t): newt=[] for i in t: newt.append(i+1) return newt def d2(t): for i in t: yield i+1
  35. 35. yield def countdown(n): while n > 0: yield n n -= 1 >>>a = countdown(5) >>>for i in a: print i 54321
  36. 36. this is a Generator
  37. 37. Generator Expressions
  38. 38. generator & yield (expr for i in s if condition) #same as for i in s: if condition: yield expr
  39. 39. 1st Problem
  40. 40. analyze access log
  41. 41. 127.0.0.1 - - [31/Mar/2009:21:53:25 +0800] quot;GET /apache_pb.gif HTTP/1.1quot; 200 2326 host referrer user [datetime] quot;requestquot; status bytes re.compile(r'(S+) (S+) (S+) [(.*?)] quot;(S+) (S+) (S+)quot; (S+) (S+)') match().groups() line = ('127.0.0.1','-','-','31/Mar/ 2009:21:53:25 +0800','GET','/ apache_pb.gif','HTTP/1.1','200','2326')
  42. 42. cols=('host','referrer','user','datetime', 'method','request','proto','status','bytes ') dict(zip(cols,line)) {'bytes': '2326', 'datetime': '31/Mar/2009:21:53:25 +0800', 'host': '127.0.0.1', 'method': 'quot;GET', 'proto': 'HTTP/1.1', 'referrer': '-', 'request': '/apache_pb.gif', 'status': '200', 'user': '-'}
  43. 43. RealOne
  44. 44. lines=(dict(zip(cols,logpats.match(i).grou ps())) for i in open('./access_log')) for i in lines: print i {'status': '200', 'proto': 'HTTP/1.1', 'referrer': '-', 'request': '/', 'datetime': '31/Mar/2009:21:53:25 +0800', 'bytes': '1456', 'host': '::1', 'user': '-', 'method': 'GET' }
  45. 45. def fmap(dictseq,col,func): for d in dictseq: d[col] = func(d[col]) yield d log = fmap(lines,’status’,int) log = fmap(log,’bytes’,lambda s:int(s) if s != ‘-’ else 0) {'status': 200, ... 'bytes': 1456, ... }
  46. 46. Show Time
  47. 47. >>>print sum(r[‘bytes’] for r in log) 2456 >>>print “%d %s” % max(r[‘bytes’],r[‘request’] for r in log) 233 /aaa.jpg >>>print set(r[‘host’] for r in log) >>>print set(r[‘request’] for r in log if r[‘status’] == 404)
  48. 48. 2nd Problem
  49. 49. find specific files
  50. 50. os.walk
  51. 51. Is a Generator
  52. 52. for p,dl,fl in os.walk(t) #t: a top directory #p: current path #dl: directory list in current #fl: file list in current
  53. 53. pyfind import os import fnmatch def pyfind(top,pats): for p,dl,fl in os.walk(top): for f in fnmatch.filter(fl,pats): yield os.path.join(p,f)
  54. 54. pyfind big file if os.path.getsize(ffp) > s*1024*1024: yield ffp
  55. 55. os.path
  56. 56. get stat os.path.getctime os.path.getmtime os.path.getatime os.path.getsize os.stat
  57. 57. do a test os.path.exists os.path.isabs os.path.isdir os.path.isfile os.path.islink os.path.ismount
  58. 58. deal with path os.path.abspath os.path.join os.path.basename os.path.dirname os.path.commonprefix os.path.split os.path.splitext
  59. 59. next
  60. 60. process thread queue
  61. 61. Reference
  62. 62. • Generator Tricks for Systems Programmers http://www.dabeaz.com/generators-uk/index.html • Python Module of the Week http://www.doughellmann.com/projects/ PyMOTW/ • Python documentation http://docs.python.org/ • Book: Python for Unix and Linux System Administration http://www.amazon.com/Python-Unix-Linux- System-Administration/dp/0596515820
  63. 63. Q&A
  64. 64. Thanks

×