Python on FreeBSD
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Python on FreeBSD

  • 10,319 views
Uploaded on

by 許立文 (lwhsu)

by 許立文 (lwhsu)

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
10,319
On Slideshare
10,313
From Embeds
6
Number of Embeds
1

Actions

Shares
Downloads
45
Comments
0
Likes
4

Embeds 6

http://www.techgig.com 6

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Python on FreeBSDLi-Wen Hsu <lwhsu@{lwhsu,FreeBSD}.org> PyCon Taiwan 2012
  • 2. Who is this ↘
  • 3. lwhsu: 2006 開始接觸 Python 2007 得到 FreeBSD ports commit bit2008 Sysadm & Instructor at CS.NCTU 2009 國軍 online 2010~ 菜鳥碼農+修電腦的另外也是管 {ftp,www,cvsup,anoncvs, portsnap,update}.tw.freebsd.org 的
  • 4. </開場白>
  • 5. Python onFreeBSD
  • 6. What isFreeBSD?
  • 7. The FreeBSD Project• FreeBSD is an advanced operating system derived from BSD, the version of UNIX® developed at the University of California, Berkeley. o http://www.freebsd.org/about.html
  • 8. Who’s using FreeBSD?• FreeBSD is used in the devices from the world’s big companies: o Apple o Blue Coat o Cisco o F5 Networks o Juniper o NetApp o Sony’s PlayStation 3
  • 9. Who’s also using FreeBSD?• FreeBSD is also used to power some of the busiest sites on Internet: o Apache o Netflix o Pixnet o WhatsApp o Yahoo! o Yandex
  • 10. More?• Projects based on FreeBSD o PC-BSD o m0n0wall o psSense o FreeNAS o Debian GNU/kFreeBSD o Gentoo/FreeBSD
  • 11. FreeBSD: The Good Parts• BSD License • TrustedBSD MAC• GEOM Framework• Netgraph • TrustedBSD Audit• Jail • LLVM & Clang• ZFS • Ports system• DTrace • And more…http://www.freebsd.org/features.html
  • 12. </FreeBSD>http://wiki.freebsd.org/Myths
  • 13. <FreeBSD+Python>
  • 14. Origin• /usr/ports/lang/python/Makefile:# New ports collection makefile for: python# Date created: 08 August 1995# Whom: jkh "Director of Engineering of Unix Technologies" at Apple
  • 15. python@FreeBSD.org• Mailing list freebsd-python@FreeBSD.org o FreeBSD-specific Python issues o Founded in January 2004 o http://lists.freebsd.org/mailman/listinfo/freebsd-python o Maintaining python and python related ports, except py-qt and py-gtk • gnome@FreeBSD.org • kde@FreeBSD.org
  • 16. python@FreeBSD.org• Wiki o http://wiki.freebsd.org/Python o Who has the “Hat” o Action items o Contact information• IRC o #freebsd-python on FreeNode
  • 17. Friends of python@FreeBSD.org• zope@FreeBSD.org o Zope for FreeBSD -- porting and maintaining o http://lists.freebsd.org/mailman/listinfo/freebsd-zope o Founded in April 2012• M_FBSD.python at bbs.cs.nctu.edu.tw
  • 18. python@FreeBSD.org• People o alexbl (alumni) o perky (alumni) o clsung (Taiwan) o rm o lwhsu (Taiwan) o sunpoet (Taiwan) o miwi o wen o nivit o yzlin (Taiwan)
  • 19. Install Python on FreeBSD• Via ports system o cd /usr/ports/lang/python o make install clean
  • 20. Install Python on FreeBSD• Ports options
  • 21. Compiling...http://xkcd.com/303/
  • 22. import antigravityhttp://xkcd.com/353/
  • 23. Install Python on FreeBSD• Via package system o pkg_add -r python
  • 24. Install Python on FreeBSD• Via package system (advanced) o setenv PACKAGEROOT http://ftp.tw.freebsd.org o setenv PACKAGESITE http://ftp.tw.freebsd.org/pub/FreeBSD/ports/`un ame -p`/packages-`uname -r | cut -d - -f 1,2 | tr "[:upper:]" "[:lower:]"`/Latest/ o pkg_add -r python
  • 25. Install Python on FreeBSD• In the (on so far) future:• Pkgng o pkg install python
  • 26. Wait, I have a question!• Q: Why compile Python myself?• A: o Optimize for your system o Customize build options o Easier to hack into Python in the system
  • 27. Wait, I have a question!• Q: Do I really need to compile myself on *every* machine I want to deploy?• A: Build once, install everywhere. o cd /usr/ports/lang/python o make package • /usr/ports/packages/lang/python-2.7,2.tbz o install • pkg_add python-2.7,2.tbz
  • 28. Run Python on FreeBSD• In /usr/ports • Also in /usr/ports o lang/python o lang/jython • (meta package) o lang/pypy o lang/python24 o lang/ironpython o lang/python25 o lang/python26 o devel/ipython o lang/python27 o lang/python31 o lang/python32
  • 29. Run many Python versions on FreeBSD• /usr/local/bin/python o A symbolic link to default version o PYTHON_DEFAULT_VERSION • Can be manually defined in /etc/make.conf • Possible values: python{2.4,2.5,2.6,2.7,3.1,3.2}• You can install as many different of Python versions you want o just install via /usr/ports/lang/python* o First installed Python version will be the default
  • 30. Run many Python versions on FreeBSD• Install modules for different python versions o cd /usr/ports/foo/py-bar o make PYTHON_VERSION=python2.6 install clean o This is still work in progress, there might be some minor errors. • For modules install scripts in bin/
  • 31. Wait, I have a question!• Q: there is XXX, why do I need ports to install python modules? o (XXX=distutils, setuptools, distribute, distutils2, pip)• A: o For better integration with system o Do more complex deploy o Ex: install web applications like reviewboard, trac • A system only consists with python and python module • Also apache/nginx, mod_wsgi, etc.
  • 32. Develop with Python on FreeBSD• virtualenv + Jenkins o devel/py-virtualenv o devel/jenkins• Use virtualenv in jenkins, make sure clean dependency.• Have a job to build a virtualenv that is usable by other jobs.
  • 33. Develop with Python on FreeBSD• ZFS + jail• Jail o Light weight virtual machine o /etc/rc.conf • jail_enable="YES”• ZFS o zpool create tank ad1 o zfs create tank/jail1
  • 34. Develop with Python on FreeBSD• Create a new jail o bsdinstall jail /tank/jail1 o freebsd-update -b /tank/jail1 fetch install o Modify /etc/rc.conf • jail_jail1_rootdir="/tank/jail1” • jail_jail1_hostname="jail1” • jail_jail1_ip="192.168.xxx.xxx” • jail_jail1_interface="lo0" o /etc/rc.d/jail start jail1
  • 35. Develop with Python on FreeBSD• Clone a jail o zfs snapshot tank/jail1@milestone1 o zfs clone tank/jail1@milestone1 tank/jail2 o Modify /etc/rc.conf o /etc/rc.d/jail start jail2 o Another development environment is created!
  • 36. Run your Python Apps on FreeBSD• Nothing special or different with other Unix-like platform
  • 37. Run your Python Web Apps• GIL problem<VirtualHost *:80> DocumentRoot "/home/www/webroot" ServerName python.example.com WSGIProcessGroup pythonexample WSGIDaemonProcess pythonexample processes=16 threads=128 maximum-requests=1024 display-name=%{GROUP}</VirtualHost>
  • 38. Make Pythonmodule port
  • 39. FreeBSD Ports• Each ``port’’ contains any patches necessary to make the original application source code compile and run on FreeBSD. Installing an application is as simple as typing make in the port directory. o http://www.freebsd.org/ports/
  • 40. Inside a port• ls -1 /usr/ports/devel/py-coverage Makefile distinfo pkg-descr pkg-plist
  • 41. Create a new Python port• Use porttools to help: o cd /usr/ports/ports-mgmt/porttools o make install clean• Create a new port o cd /your/workspace o port create py-newport
  • 42. Makefile# New ports collection makefile for: py-coverage# Date created: Jun. 22, 2009# Whom: Li-Wen Hsu <lwhsu@FreeBSD.org>## $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $#PORTNAME= coveragePORTVERSION= 3.5.2CATEGORIES= devel pythonMASTER_SITES= CHEESESHOPPKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}MAINTAINER= lwhsu@FreeBSD.orgCOMMENT= Code coverage measurement for PythonLICENSE= BSDUSE_PYTHON= yesUSE_PYDISTUTILS= easy_installPYEASYINSTALL_ARCHDEP= yes.include <bsd.port.mk>
  • 43. Makefile# New ports collection makefile for: py-coverage# Date created: Jun. 22, 2009# Whom: Li-Wen Hsu <lwhsu@FreeBSD.org>## $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $#
  • 44. MakefilePORTNAME= coveragePORTVERSION= 3.5.2CATEGORIES= devel pythonMASTER_SITES= CHEESESHOPPKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
  • 45. MakefileLICENSE= BSDUSE_PYTHON= yes # 2.6 2.6+ -2.7 3.1+USE_PYDISTUTILS= easy_installPYEASYINSTALL_ARCHDEP= yes.include <bsd.port.mk>
  • 46. distinfoSHA256 (coverage-3.5.2.tar.gz) = b9e0dc65e42236a3c9b8978701bc9e7298ed7e24d39f99 ea2ba4b945d537fe55SIZE (coverage-3.5.2.tar.gz) = 115497
  • 47. pkg-descrCoverage measures code coverage, typically during test execution.It uses the code analysis tools and tracing hooks provided in thePython standard library to determine which lines are executable,and which have been executed.WWW: http://nedbatchelder.com/code/coverage
  • 48. pkg-plistbin/coverage%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/PKG-INFO%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/SOURCES.txt%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/dependency_links.txt%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/entry_points.txt%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/native_libs.txt%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/not-zip-safe%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/top_level.txt%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.py%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyc%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyo[...]%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.py%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyc%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyo@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/htmlfiles@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%
  • 49. Before submitting• Check o Use portlint (ports-mgmt/portlint) o Command:portint –abct• Test o Use porttools (ports-mgmt/porttools) o Command: port test
  • 50. Before submitting• More test o Use Tinderbox (ports-mgmt/tinderbox) o A comprehensive way to test ports • And builds packages!
  • 51. Wait, I have a question!• Q: Setting up a tinderbox is too complex• A: Use RedPorts http://redports.org/
  • 52. Submit!• In your port directory• Command: o port submit• Quit editor: o s)end, e)dit or a)bort? s
  • 53. After submitting• A PR (problem report) is created• A committer will handle it• A new port is in the ports tree! o See on http://freshports.org/foo/bar• To update your ports tree: o portsnap fetch update• Install the new port: o cd /usr/ports/foo/bar; make install clean
  • 54. Future Work of python@FreeBSD.org• Remove 2.4 and 2.5 in the ports tree o 2012-06-30• Handle packages with pip/distribute• Debug versions of lang/python*• Install only unzipped eggs• Truly modules for multiple python versions co-exist
  • 55. Q&A