Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pysmbc Python C Modules are Easy

1,668 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Pysmbc Python C Modules are Easy

  1. 1. PySmbC:C Modules are EasyEuroPython 2012, 6th July - FirenzeBabel Srl P.zza S. Benedetto da Norcia, 33 0040, Pomezia (RM) – www.babel.it Roberto Polli - roberto.polli@babel.it
  2. 2. What? Who? Why?A story about how easy is to contribute to a Python project usingGitHub and Nose testing framework.Roberto Polli - Community Manager @ Babel.it. Loves writing in C,Java and Python. Red Hat Certified Engineer.Fabio Isgrò – System Engineer @ Babel.it. Linux and Samba expert.Babel – Proud sponsor of this talk ;) Delivers large mail infrastructurebased on Open Source software for Italian ISP and PA. Contributesto various FLOSS. Roberto Polli - roberto.polli@babel.it
  3. 3. Agenda - 1 Contributing to a Python C Extension is easier than thought+ GitHub allows a fast way to fork and merge patches to opensource projects+ Write and run Unit and Functional tests is easy with NoseTest = Dont have to be a Guru to support FLOSS! Roberto Polli - roberto.polli@babel.it
  4. 4. Agenda - 2SMB is the protocol used by Windows for sharing folders. Samba is itsFLOSS implementation.smbclient library supports almost all SMB features. PySmbC wrapssome functions provided by this library.To support extended permissions (ACL) we wrapped two morefunctions: get_xattr and set_xattr. You dont need to know SMB to extend PySmbC Roberto Polli - roberto.polli@babel.it
  5. 5. GitHub – social coding GitHub is a social coding platform.● Clone my repository and fork the project https://github.com/ioggstream/pysmbc● Patch and commit to your repository● Push your changes to my repo● Discuss for approval● Merge my changes Did you say...fork? Roberto Polli - roberto.polli@babel.it
  6. 6. GitHub – social codingForking is the nightmare of everyfloss maintainer.Sparse writes, increasing mergeefforts, lost changes.GitHub, like the I/O Scheduler,helps to merge writes! – Tracks forks; – Push changes to master. Roberto Polli - roberto.polli@babel.it
  7. 7. GitHub – social codingPatch and commit to your repository● dont lose your changes;● track your own history.Push your changes to my repo● no more for .patch;● I get the change history.Discuss for approval● Github portal supports code annotations.Merge my changes● use git to merge from a remote repository! Roberto Polli - roberto.polli@babel.it
  8. 8. Enter PySmbC - ExtensionsPython wrapper around libsmbclient: run C code from python – enjoy a stable C library; – just manage input, output and errors: – usually faster than a pure python implementation. rpolli$ ipython ln [1]: import smbc smbc.so ln [2]: print smbc.XATTR_ACL system.nt_sec_desc.acl libsmbclient.so.0 ln [3]: Roberto Polli - roberto.polli@babel.it
  9. 9. Example - Wrapping factorial() - 1The wrapping function my_factorial(): // Python C Extension // uses factorial from fact.c● Parses and validates the input; #include <fact.h> wrapperfy.c● Calls the wrapped function(); // returns a python object! PyObject *my_factorial(...) { ...● Returns a python object. ret = factorial(n); ... return PyLong_asLong(ret); }A given structure maps python methods toC functions. // Maps _wrapper.factorial # python script // to my_factorial from _wrapper import factorialNow we can invoke a PyMethodDef BabelMethods[] = { {"factorial", my_factorial, ... }, print _wrapper.factorial(4)wrapped function! }; {NULL, NULL, 0, NULL} /*Sentinel*/ Roberto Polli - roberto.polli@babel.it
  10. 10. Example - Wrapping factorial() - 2Parsing and validating Input and Output // returns a python object! PyObject *my_factorial(..., *args) {is fundamental. We dont want python   // NULL indicates an error   if (!PyArg_ParseTuple(args, "i", &n))to SEGFAULT!     return NULL;   // n! needs more than 8byte   if (n>21) {     ...     PyErr_SetString(FactError, Create new exceptions in the       “Bad value”);   }initialization function.   ...    return PyLong_asLong(ret); } PyObject *FactError;Throw exceptions in the function: // in extension initialization... ...● setting PyErr; init_wrapper(void) {   ...   // define an exception● returning NULL.   FactError =      PyErr_NewException("_wrapper.error",        NULL, NULL);   ... } Roberto Polli - roberto.polli@babel.it
  11. 11. Example - Wrapping factorial() - 3 // Python C ExtensionC Extension components: #include <Python.h>● wrapping functions; // exceptions PyObject *FactError; PyObject *FiboError;● method/function map; // functions● exceptions; PyObject *my_factorial(...); PyObject *my_fibonacci(...);● initialization function. // Function/Method Maps  PyMethodDef BabelMethods[] = {   {"factorial",my_factorial,... },Functions and Exception should be static   {"fibonacci",my_fibonacci,... },   {NULL, NULL, 0, NULL} /*Sentinel*/ }; PyMODINIT_FUNC You have to track memory usage! init_wrapper(void) { PyObject *m; m = Py_InitModule("_wrapper", BabelMethods); // … Allocate Exceptions FactError = PyErr_NewException(...) FiboError = PyErr_NewException(...) } Roberto Polli - roberto.polli@babel.it
  12. 12. Enters PySmbC - ModulesPython C extensionsmay enjoy both C andPython code.Wrap the C extension ina Python module.Extend the module withpython classes.$PYTHONPATH/ wrapper/ __init__.py In [1]: import wrapper In [2]: assert wrapper.helpers _wrapper.so In [3]: wrapper.helpers.is_integer(10) helpers.py Roberto Polli - roberto.polli@babel.it
  13. 13. Nose – lets contribute - 1Before adding features to PySmbC we checked the project status # git clone https://github.com/ioggstream/pysmbc . # vim tests/settings.py # set samba credential # nosetests test/NoseTest - a python script that auto-discovers and run test cases.Wraps python-unittest.Add new features only after successful tests. Verify your environment(eg. Samba credentials, directory acls ) Roberto Polli - roberto.polli@babel.it
  14. 14. Nose – lets contribute - 2On successful tests, we can start developingFollow the Git way: create a separate branch. Well merge it onsuccess # git checkout -b ioggstream_setxattrWrite the tests before writing the code. Youll be more focused onyour targets With nosetest its simpler than ever! Roberto Polli - roberto.polli@babel.it
  15. 15. Nose – is like UnitTestUnitTest Nosefrom unittest import TestCase, main import noseclass MyTest(UnitTest): class MyTest: def setUp(self): def setup(self): print”setup every” print ”setup” def tearDown(self): def teardown(self): print “teardown every” print “teardown” def test_do(self): def test_do(self): print “do 1” print “do 1”if __name__== “__main__”: # nose script will auto-discover main() # this script named test_script.py Roberto Polli - roberto.polli@babel.it
  16. 16. Nose – is simpler than UnitTestNose: simple test Nose: annotations# dont need to import nose from nose import SkipTest,with_setup# or define a class def pre(): print “setup”def setup(): def post(): print “teardown” print”setup once for all tests”def teardown(): @with_setup(pre,post) print “teardown once for all test” def test_do(): print “do 1”def test_do(): print “do 1” @SkipTestdef test_fail(): def test_dont(): assert False Print “not done yet” Roberto Polli - roberto.polli@babel.it
  17. 17. Nose – InvocationYou can run your all tests in a given directory # nosetests ./path/Or just one file # nosetests ./path/test_sample.pyOr even a single test method # nosetests ./path/test_sample.py:test_do1Or suite, eventually setting the working directory ex1# nosetests ./path/test_class.py:TestOne ex2# nosetests -w ./path test_class:TestOneFor a verbose output just use: #nosetests -sv [args] Roberto Polli - roberto.polli@babel.it
  18. 18. PySmbC – add getxattrNose ensures that were not going to break # from test_context.py def test_xattr_constants():anything. reuse variable defined in smbclient.h assert smbc.XATTR_ACL assert smbc.XATTR_OWNERStart writing tests, not coding functionalities. assert smbc.XATTR_GROUPYou can @SkipTest until new functions are def test_xattr_get():ready. test xattr with all possible values . . . for xa in valid_xatts:Play with the wrapped functions. assert ctx.getxattr(url, xa) def test_xattr_get_error():Start with the simpler one: getxattr() xattr_get should recognize bad values● embed C constants into python; . . . for xa in invalid_xatts:● test good values; try: ctx.getxattr(url, xa)● check behavior with bad values. assert False except RuntimeError as e: . . . #get errnoCode until tests are successful. assert errno == EINVAL Roberto Polli - roberto.polli@babel.it
  19. 19. PySmbC – add setxattr and futuresHelper methods for parsing and creating # from test_context.pyACL def test_xattr_set():attrs_new = uREVISION:1 . . . ctx.setxattr(url, a_name, + ,OWNER:RPOLLIbabel attrs_new, REPLACE) + ,GROUP:Unix Groupbabel attrs_1 = ctx.getxattr(url, a_name) + ,ACL:RPOLLIbabel:0/0/0x001e01ff assert attrs_1 == attrs_new + ,ACL:Unix Groupbabel:0/0/0x00120089 + ,ACL:Unix Groupgames:0/0/0x001e01ff def test_xattr_set_error(): setxattr should + ,ACL:Everyone:0/0/0x00120089 recognize bad values . . . for xa in invalid_xatts:Shift from smbc.so to smbc module: try: ctx.setxattr(url, a_name,● smbc/_smbc.so xa, REPLACE) assert False● smbc/__init__.py except RuntimeError as e: . . . #get errno● smbc/helper.py assert errno == EINVAL except TypeError pass Roberto Polli - roberto.polli@babel.it
  20. 20. PySmbChttps://github.com/ioggstream/pysmbchttp://pypi.python.org/pypi/pysmbc/http://www.samba.orgBabelhttp://www.babel.ithttp://vaunaspada.babel.it/blog Thank You! roberto.polli@babel.it Roberto Polli - roberto.polli@babel.it

×