Python on FreeBSD


Published on

by 許立文 (lwhsu)

Published in: Technology

Python on FreeBSD

  1. 1. Python on FreeBSDLi-Wen Hsu <lwhsu@{lwhsu,FreeBSD}.org> PyCon Taiwan 2012
  2. 2. Who is this ↘
  3. 3. lwhsu: 2006 開始接觸 Python 2007 得到 FreeBSD ports commit bit2008 Sysadm & Instructor at CS.NCTU 2009 國軍 online 2010~ 菜鳥碼農+修電腦的另外也是管 {ftp,www,cvsup,anoncvs, portsnap,update} 的
  4. 4. </開場白>
  5. 5. Python onFreeBSD
  6. 6. What isFreeBSD?
  7. 7. The FreeBSD Project• FreeBSD is an advanced operating system derived from BSD, the version of UNIX® developed at the University of California, Berkeley. o
  8. 8. Who’s using FreeBSD?• FreeBSD is used in the devices from the world’s big companies: o Apple o Blue Coat o Cisco o F5 Networks o Juniper o NetApp o Sony’s PlayStation 3
  9. 9. Who’s also using FreeBSD?• FreeBSD is also used to power some of the busiest sites on Internet: o Apache o Netflix o Pixnet o WhatsApp o Yahoo! o Yandex
  10. 10. More?• Projects based on FreeBSD o PC-BSD o m0n0wall o psSense o FreeNAS o Debian GNU/kFreeBSD o Gentoo/FreeBSD
  11. 11. FreeBSD: The Good Parts• BSD License • TrustedBSD MAC• GEOM Framework• Netgraph • TrustedBSD Audit• Jail • LLVM & Clang• ZFS • Ports system• DTrace • And more…
  12. 12. </FreeBSD>
  13. 13. <FreeBSD+Python>
  14. 14. Origin• /usr/ports/lang/python/Makefile:# New ports collection makefile for: python# Date created: 08 August 1995# Whom: jkh "Director of Engineering of Unix Technologies" at Apple
  15. 15.• Mailing list o FreeBSD-specific Python issues o Founded in January 2004 o o Maintaining python and python related ports, except py-qt and py-gtk • •
  16. 16.• Wiki o o Who has the “Hat” o Action items o Contact information• IRC o #freebsd-python on FreeNode
  17. 17. Friends of• o Zope for FreeBSD -- porting and maintaining o o Founded in April 2012• M_FBSD.python at
  18. 18.• People o alexbl (alumni) o perky (alumni) o clsung (Taiwan) o rm o lwhsu (Taiwan) o sunpoet (Taiwan) o miwi o wen o nivit o yzlin (Taiwan)
  19. 19. Install Python on FreeBSD• Via ports system o cd /usr/ports/lang/python o make install clean
  20. 20. Install Python on FreeBSD• Ports options
  21. 21. Compiling...
  22. 22. import antigravity
  23. 23. Install Python on FreeBSD• Via package system o pkg_add -r python
  24. 24. Install Python on FreeBSD• Via package system (advanced) o setenv PACKAGEROOT o setenv PACKAGESITE`un ame -p`/packages-`uname -r | cut -d - -f 1,2 | tr "[:upper:]" "[:lower:]"`/Latest/ o pkg_add -r python
  25. 25. Install Python on FreeBSD• In the (on so far) future:• Pkgng o pkg install python
  26. 26. Wait, I have a question!• Q: Why compile Python myself?• A: o Optimize for your system o Customize build options o Easier to hack into Python in the system
  27. 27. Wait, I have a question!• Q: Do I really need to compile myself on *every* machine I want to deploy?• A: Build once, install everywhere. o cd /usr/ports/lang/python o make package • /usr/ports/packages/lang/python-2.7,2.tbz o install • pkg_add python-2.7,2.tbz
  28. 28. Run Python on FreeBSD• In /usr/ports • Also in /usr/ports o lang/python o lang/jython • (meta package) o lang/pypy o lang/python24 o lang/ironpython o lang/python25 o lang/python26 o devel/ipython o lang/python27 o lang/python31 o lang/python32
  29. 29. Run many Python versions on FreeBSD• /usr/local/bin/python o A symbolic link to default version o PYTHON_DEFAULT_VERSION • Can be manually defined in /etc/make.conf • Possible values: python{2.4,2.5,2.6,2.7,3.1,3.2}• You can install as many different of Python versions you want o just install via /usr/ports/lang/python* o First installed Python version will be the default
  30. 30. Run many Python versions on FreeBSD• Install modules for different python versions o cd /usr/ports/foo/py-bar o make PYTHON_VERSION=python2.6 install clean o This is still work in progress, there might be some minor errors. • For modules install scripts in bin/
  31. 31. Wait, I have a question!• Q: there is XXX, why do I need ports to install python modules? o (XXX=distutils, setuptools, distribute, distutils2, pip)• A: o For better integration with system o Do more complex deploy o Ex: install web applications like reviewboard, trac • A system only consists with python and python module • Also apache/nginx, mod_wsgi, etc.
  32. 32. Develop with Python on FreeBSD• virtualenv + Jenkins o devel/py-virtualenv o devel/jenkins• Use virtualenv in jenkins, make sure clean dependency.• Have a job to build a virtualenv that is usable by other jobs.
  33. 33. Develop with Python on FreeBSD• ZFS + jail• Jail o Light weight virtual machine o /etc/rc.conf • jail_enable="YES”• ZFS o zpool create tank ad1 o zfs create tank/jail1
  34. 34. Develop with Python on FreeBSD• Create a new jail o bsdinstall jail /tank/jail1 o freebsd-update -b /tank/jail1 fetch install o Modify /etc/rc.conf • jail_jail1_rootdir="/tank/jail1” • jail_jail1_hostname="jail1” • jail_jail1_ip="” • jail_jail1_interface="lo0" o /etc/rc.d/jail start jail1
  35. 35. Develop with Python on FreeBSD• Clone a jail o zfs snapshot tank/jail1@milestone1 o zfs clone tank/jail1@milestone1 tank/jail2 o Modify /etc/rc.conf o /etc/rc.d/jail start jail2 o Another development environment is created!
  36. 36. Run your Python Apps on FreeBSD• Nothing special or different with other Unix-like platform
  37. 37. Run your Python Web Apps• GIL problem<VirtualHost *:80> DocumentRoot "/home/www/webroot" ServerName WSGIProcessGroup pythonexample WSGIDaemonProcess pythonexample processes=16 threads=128 maximum-requests=1024 display-name=%{GROUP}</VirtualHost>
  38. 38. Make Pythonmodule port
  39. 39. FreeBSD Ports• Each ``port’’ contains any patches necessary to make the original application source code compile and run on FreeBSD. Installing an application is as simple as typing make in the port directory. o
  40. 40. Inside a port• ls -1 /usr/ports/devel/py-coverage Makefile distinfo pkg-descr pkg-plist
  41. 41. Create a new Python port• Use porttools to help: o cd /usr/ports/ports-mgmt/porttools o make install clean• Create a new port o cd /your/workspace o port create py-newport
  42. 42. Makefile# New ports collection makefile for: py-coverage# Date created: Jun. 22, 2009# Whom: Li-Wen Hsu <>## $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $#PORTNAME= coveragePORTVERSION= 3.5.2CATEGORIES= devel pythonMASTER_SITES= CHEESESHOPPKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}MAINTAINER= lwhsu@FreeBSD.orgCOMMENT= Code coverage measurement for PythonLICENSE= BSDUSE_PYTHON= yesUSE_PYDISTUTILS= easy_installPYEASYINSTALL_ARCHDEP= yes.include <>
  43. 43. Makefile# New ports collection makefile for: py-coverage# Date created: Jun. 22, 2009# Whom: Li-Wen Hsu <>## $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $#
  45. 45. MakefileLICENSE= BSDUSE_PYTHON= yes # 2.6 2.6+ -2.7 3.1+USE_PYDISTUTILS= easy_installPYEASYINSTALL_ARCHDEP= yes.include <>
  46. 46. distinfoSHA256 (coverage-3.5.2.tar.gz) = b9e0dc65e42236a3c9b8978701bc9e7298ed7e24d39f99 ea2ba4b945d537fe55SIZE (coverage-3.5.2.tar.gz) = 115497
  47. 47. pkg-descrCoverage measures code coverage, typically during test execution.It uses the code analysis tools and tracing hooks provided in thePython standard library to determine which lines are executable,and which have been executed.WWW:
  49. 49. Before submitting• Check o Use portlint (ports-mgmt/portlint) o Command:portint –abct• Test o Use porttools (ports-mgmt/porttools) o Command: port test
  50. 50. Before submitting• More test o Use Tinderbox (ports-mgmt/tinderbox) o A comprehensive way to test ports • And builds packages!
  51. 51. Wait, I have a question!• Q: Setting up a tinderbox is too complex• A: Use RedPorts
  52. 52. Submit!• In your port directory• Command: o port submit• Quit editor: o s)end, e)dit or a)bort? s
  53. 53. After submitting• A PR (problem report) is created• A committer will handle it• A new port is in the ports tree! o See on• To update your ports tree: o portsnap fetch update• Install the new port: o cd /usr/ports/foo/bar; make install clean
  54. 54. Future Work of• Remove 2.4 and 2.5 in the ports tree o 2012-06-30• Handle packages with pip/distribute• Debug versions of lang/python*• Install only unzipped eggs• Truly modules for multiple python versions co-exist
  55. 55. Q&A