• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Ottimizzare il ciclo di sviluppo e test mediante Python, Nose e GitHub

Ottimizzare il ciclo di sviluppo e test mediante Python, Nose e GitHub



Il TechAdvisor Roberto Polli condivide l'esperienza maturata su PySmbC, un modulo python che permette di accedere ad un Server SMB utilizzando le funzioni della libreria C fornita dal team Samba. ...

Il TechAdvisor Roberto Polli condivide l'esperienza maturata su PySmbC, un modulo python che permette di accedere ad un Server SMB utilizzando le funzioni della libreria C fornita dal team Samba. Attraverso degli esempi pratici dimostra che mediante il Test Driven Development, la libreria Nose per i test e GitHub, scrivere dei Python bindings in C può essere abbastanza facile.

Durante la presentazione mostra inoltre come:
- GitHub ha velocizzato il ciclo di sviluppo e la revisione delle patch;
- nosetests ha permesso di scrivere più test con meno codice;
- modificare facilmente un modulo Python scritto in C, rendendo disponibili nuove feature.





Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Ottimizzare il ciclo di sviluppo e test mediante Python, Nose e GitHub Ottimizzare il ciclo di sviluppo e test mediante Python, Nose e GitHub Presentation Transcript

    • PySmbC:C Modules are EasyEuroPython 2012, 6th July - Firenze Babel Srl P.zza S. Benedetto da Norcia, 33 0040, Pomezia (RM) – www.babel.it Roberto Polli - roberto.polli@babel.it
    • What? Who? Why?A success story about using GitHub and Nose testing framework.Roberto Polli - Community Manager @ Babel.it. Loves writing in C,Java and Python. Red Hat Certified Engineer.Fabio Isgrò – System Engineer @ Babel.it. Linux and Sambaexpert.Babel – Proud sponsor of this talk ;) Delivers large mailinfrastructures based on Open Source software for Italian ISP andPA. Contributes to various FLOSS. Roberto Polli - roberto.polli@babel.it
    • Agenda – 1 - prerequisites People enjoy FLOSS but...why is afraid of contributing?+ GitHub: track forks, fast merges+ Python: C Extension are easy+ Nose Test: simplify testing process = Dont have to be a Guru to support FLOSS! Roberto Polli - roberto.polli@babel.it
    • Agenda – 2 - contributingSamba and samba-client library.Clone a project, get in touch with the maintainer.Write test first.Wrapping get_xattr and set_xattr, manage ACL. Dont need to know SMB to extend PySmbC Roberto Polli - roberto.polli@babel.it
    • GitHub – social codingFLOSS Coding is a relational stuff.Coding 1.0: send .patch to a mailing listCoding 2.0: fork me on GitHubhttps://github.com/ioggstream/pysmbc Did you say...fork? Roberto Polli - roberto.polli@babel.it
    • GitHub – social codingForking is the nightmare of everyfloss maintainer.Sparse writes, increasing mergeefforts, lost changes.GitHub, like the I/O Scheduler,helps to merge writes!● Tracks forks;● Push changes to master. Roberto Polli - roberto.polli@babel.it
    • GitHub – social codingPatch and commit to your repository● dont lose your changes;● track your own history.Push your changes to my repo● no more for .patch;● I get the change history.Discuss for approval● Github portal supports code annotations.Merge my changes● use git to merge from a remote repository! Roberto Polli - roberto.polli@babel.it
    • Enter PySmbC - ExtensionsPython wrapper around libsmbclient: run C code from python– enjoy a stable C library;– just manage input, output and errors:– usually faster than a pure python implementation. rpolli$ ipython ln [1]: import smbc smbc.so ln [2]: print smbc.XATTR_ACL system.nt_sec_desc.acl libsmbclient.so.0 ln [3]: Roberto Polli - roberto.polli@babel.it
    • Example - Wrapping factorial() - 1The wrapping function my_factorial(): // Python C Extension wrapperfy.c● Parses and validates the input; // uses factorial from fact.c● Calls the wrapped function(); #include <fact.h> // returns a python object!● Returns a python object. PyObject *my_factorial(...) { ... ret = factorial(n); ... return PyLong_asLong(ret);A given structure maps python methods to }C functions.gcc wrapperfy.c -o _wrapper.so -shared // Maps _wrapper.factorial # python script // to my_factorial from _wrapper import factorial PyMethodDef BabelMethods[] = {Now we can invoke a {"factorial", my_factorial, ... }, print factorial(4) {NULL, NULL, 0, NULL} /*Sentinel*/wrapped function! }; Roberto Polli - roberto.polli@babel.it
    • Example - Wrapping factorial() - 2Parsing and validating Input and // returns a python object! PyObject *my_factorial(..., *args) {Output is crucial. We dont want python   // NULL indicates an error   if (!PyArg_ParseTuple(args, "i", &n))to SEGFAULT!     return NULL;   // n! > max(long long int)   if (n>21) {     ...     PyErr_SetString(FactError, Create new exceptions in the       “Bad value”);     return NULL;initialization function.   }   ...    return PyLong_asLong(ret); }Throw exceptions in the function: PyObject *FactError;● setting PyErr; // in extension initialization... ... init_wrapper(void) {● returning NULL.   ...   // define an exception   FactError =      PyErr_NewException("_wrapper.error",        NULL, NULL);   ... } Roberto Polli - roberto.polli@babel.it
    • Example - Wrapping factorial() - 3 // Python C ExtensionC Extension components: #include <Python.h>● wrapping functions; // exceptions PyObject *FactError; PyObject *FiboError;● method/function map; // functions● exceptions; PyObject *my_factorial(...); PyObject *my_fibonacci(...);● initialization function. // Function/Method Maps  PyMethodDef BabelMethods[] = {   {"factorial",my_factorial,... },Functions and Exception should be static   {"fibonacci",my_fibonacci,... },   {NULL, NULL, 0, NULL} /*Sentinel*/ }; PyMODINIT_FUNC You have to track memory usage! init_wrapper(void) { PyObject *m; m = Py_InitModule("_wrapper", BabelMethods); // … Allocate Exceptions FactError = PyErr_NewException(...) FiboError = PyErr_NewException(...) } Roberto Polli - roberto.polli@babel.it
    • Enters PySmbC - ModulesPython C extensionsmay enjoy both C andPython code.Wrap the C extension ina Python module.Extend the module withpython classes.$PYTHONPATH/ wrapper/ __init__.py In [1]: import wrapper In [2]: assert wrapper.helpers _wrapper.so In [3]: wrapper.helpers.is_integer(10) helpers.py Roberto Polli - roberto.polli@babel.it
    • Nose – lets contribute - 1Before adding features to PySmbC we checked the projectstatus # git clone https://github.com/ioggstream/pysmbc . # vim tests/settings.py # set samba credential # nosetests test/NoseTest - a python script that auto-discovers and run testcases. Wraps python-unittest.Add new features only after successful tests. Verify yourenvironment (eg. Samba credentials, directory acls ) Roberto Polli - roberto.polli@babel.it
    • Nose – lets contribute - 2On successful tests, we can start developingFollow the Git way: create a separate branch. Well mergeit on success # git checkout -b ioggstream_setxattrWrite the tests before writing the code. Youll be more focused onyour targets With nosetest its simpler than ever! Roberto Polli - roberto.polli@babel.it
    • Nose – is like UnitTestUnitTest Nosefrom unittest import TestCase, main import noseclass MyTest(UnitTest): class MyTest: def setUp(self): def setup(self): print”setup every” print ”setup” def tearDown(self): def teardown(self): print “teardown every” print “teardown” def test_do(self): def test_do(self): print “do 1” print “do 1”if __name__== “__main__”: # nose script will auto-discover main() # this script named test_script.py Roberto Polli - roberto.polli@babel.it
    • Nose – is simpler than UnitTestNose: simple test Nose: annotations# dont need to import nose from nose import SkipTest,with_setup# or define a class def pre(): print “setup”def setup(): def post(): print “teardown” print”setup once for all tests”def teardown(): @with_setup(pre,post) print “teardown once for all test” def test_do(): print “do 1”def test_do(): @SkipTest print “do 1” def test_dont():def test_fail(): Print “not done yet” assert False Roberto Polli - roberto.polli@babel.it
    • Nose – InvocationYou can run your all tests in a given directory # nosetests ./path/Or just one file # nosetests ./path/test_sample.pyOr even a single test method # nosetests ./path/test_sample.py:test_do1Or suite, eventually setting the working directory ex1# nosetests ./path/test_class.py:TestOne ex2# nosetests -w ./path test_class:TestOneFor a verbose output just use: #nosetests -sv [args] Roberto Polli - roberto.polli@babel.it
    • PySmbC – add getxattr # from test_context.pyNose ensures that were not going to def test_xattr_constants():break anything. reuse variable defined in smbclient.h assert smbc.XATTR_ACLStart writing tests, not coding assert smbc.XATTR_OWNER assert smbc.XATTR_GROUPfunctionalities. def test_xattr_get():You can @SkipTest until new functions are ready. test xattr with all possible values . . .Play with the wrapped functions. for xa in valid_xatts: assert ctx.getxattr(url, xa) def test_xattr_get_error():Start with the simpler one: getxattr() xattr_get should●embed C constants into python; recognize bad values . . .●test good values; for xa in invalid_xatts:●check behavior with bad values. try: ctx.getxattr(url, xa) assert False except RuntimeError as e:Code until tests are successful. . . . #get errno assert errno == EINVAL Roberto Polli - roberto.polli@babel.it
    • PySmbC – add setxattr and futuresHelper methods for parsing and # from test_context.pycreating ACL def test_xattr_set():attrs_new = uREVISION:1 . . . ctx.setxattr(url, a_name, + ,OWNER:RPOLLIbabel attrs_new, REPLACE) + ,GROUP:Unix Groupbabel attrs_1 = ctx.getxattr(url, + ,ACL:RPOLLIbabel:0/0/0x001e01ff a_name) + ,ACL:Unix Groupbabel:0/0/0x00120089 assert attrs_1 == attrs_new + ,ACL:Unix Groupgames:0/0/0x001e01ff def test_xattr_set_error(): + ,ACL:Everyone:0/0/0x00120089 setxattr should recognize bad values . . .Shift from smbc.so to smbc module: for xa in invalid_xatts: try:● smbc/_smbc.so ctx.setxattr(url, a_name, xa, REPLACE)● smbc/__init__.py assert False● smbc/helper.py except RuntimeError as e: . . . #get errno assert errno == EINVAL except TypeError pass Roberto Polli - roberto.polli@babel.it
    • PySmbChttps://github.com/ioggstream/pysmbchttp://pypi.python.org/pypi/pysmbc/http://www.samba.orgBabelhttp://www.babel.ithttp://vaunaspada.babel.it/blog Thank You! roberto.polli@babel.it Roberto Polli - roberto.polli@babel.it