Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20101026 ASAP Seminar

766 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

20101026 ASAP Seminar

  1. 1. Development and deployment of a cross-platform open-source software package: the Infobiotics Workbench Jonathan BlakesJonathan Blakes 26/10/201026/10/2010 Supervised by Professor Natalio KrasnogorSupervised by Professor Natalio Krasnogor
  2. 2. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 2 Outline  Introduction to Infobiotics  Organisation of software  Supporting technologies  Tour of the Infobiotics Dashboard UI  Interface considerations  Deployment on Windows, Mac and Linux  Finishing touches and availability  Conclusions
  3. 3. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 3 Infobiotics Workbench
  4. 4. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 4 Infobiotics Models (syntax)
  5. 5. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 5 Infobiotics Models (semantics)
  6. 6. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 6 Infobiotics Experiments  Command line tools with a common parameter file format:  simulation (mcss)  model checking (pmodelchecker)  model structure and parameter optimisation (poptimizer)  Desktop applications (not cloud-based)  Written in C++ for speed
  7. 7. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 7 Infobiotics Dashboard  User friendly graphical user interface for perform Infobiotics experiments  setting, loading and saving of parameters  running experiments and reporting progress  plotting output  Present options in a clear and understandable manner, setting limits on parameter values  Give feedback when things go wrong!  Written in Python
  8. 8. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 8 Why Python?  Dynamic language: rapid prototyping  Large stdlib: tempfile, XML parsers, itertools  3rd party Libraries:  NumPy – n-dimensional arrays and ufuncs  pytables – HDF5 (simulator output)  PyQt – Qt GUI bindings (GPL), QScintilla text editor  Matplotlib – 2D plotting  Mayavi – 3D VTK-based visualisation (TVTK)  Traits...
  9. 9. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 9 Traits = Reactive Programming
  10. 10. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 10 Simulation Parameters (file)
  11. 11. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 11 Simulation Parameters (GUI)
  12. 12. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 12 Model checker parameters (1) MC2
  13. 13. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 13 Model checker parameters (2)
  14. 14. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 14 Simulator Results  Simulation output – amounts of each species in each compartment at each timepoint for each run - quickly become very large (gigabytes), but only some data is of use to the modeller.  selecting important information  scaling amounts (molecules) to biologist-friendly units (e.g. nanomolar)  calculating statistics across runs (and other dimensions)  plotting timeseries, exporting data, etc.
  15. 15. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 15 SimulatorResultsDialog
  16. 16. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 16 Selecting data (filtering)  Filter compartments by name using wildcards  Using set class difference method  Custom ListWidget is a factory for filter LineEdit
  17. 17. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 17 Selecting data (sorting)  By providing custom QListWidget with regular expression and function map  Sort species by name  Sort compartments by name and/or positions
  18. 18. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 18 Selecting data (sampling)  Reducing data size by sampling reduces memory requirements, speeds up calculations and unclutters plots  100 of 10,000 runs  Every 100th recorded timepoint
  19. 19. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 19 Transforming data (scaling)  Converts time, substance quantities and volumes from simulator data units into user- friendly display units, using python-quantities  For calculating concentrations, use compartment volumes (at each timepoint) if recorded, otherwise use one volume for all compartments at all timepoints
  20. 20. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 20 Exporting data  Selected data can be exported in several common formats for use with other tools:  text (.csv)  Excel (.xls)  NumPy (.npz)
  21. 21. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 21 Plotting timeseries (old)
  22. 22. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 22 Plotting timeseries (improved)
  23. 23. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 23 Exporting figures  Added ability to save figures resized to specific screen dimensions (pixels), as surprisingly, matplotlib only supports physical dimensions (inches)
  24. 24. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 24 Model checker results
  25. 25. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 25 Visualising simulations (Mayavi)
  26. 26. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 26 Cross-platform issues  A lot of communication in Workbench uses temporary files – these behave differently on Windows to Linux  must close temp files before use, which can delete them!  Pseudo-terminal emulation library pexpect used to gather progress information not for Windows  winpexpect port not functioning at time of release  fall back to 'working' progress indicator not good for long experiments
  27. 27. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 27 Deploying Python apps  A lot of dependencies inc. Python  Python comes with most Linux distributions, but not with Windows (unlike JVM) and Mac system Python is crippled  Therefore we needed to bundle Python interpreter and all libraries for Windows/Mac  Possible solution: 'freezing' Python applications, creates a single executable (~100 megabytes!)  Need a solid base to start from and add missing libraries also python(x,y) for Windows
  28. 28. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 28 Freezing Python applications  Freezing works well for small scripts, not so well for applications with many libraries  uses static code analysis for determine libraries for bundling – TraitsUI/Matplotlib use dynamic imports  Freezers: cx_freeze, bbfreeze, PyInstaller,  Wrap python setup.py py2exe/py2app with bash scripts  copy missed files, unzip zipped Python classes  patch Matplotlib  fix Mac library paths using install_name_tool
  29. 29. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 29 Windows and Mac non-Python  mcss, pmodelchecker and poptimizer need cross- compiling for 32/64-bit Linux (chroot), Windows (32- bit) and Mac (OSX Snow Leopard only)  pmodelchecker requires PRISM and MC2 which are both Java applications  InstallJammer builds cross-platform installer:
  30. 30. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 30 Linux (Debian)  Not trivial either – no authorative packaging guide  Make packages for mcss, ..., PRISM, MC2, libhdf5- 1.8  Used python-support to install Dashboard for system Python(s)  (1) add apt repository to sources.list  (2) sudo apt-get install infobiotics-workbench  Automatic updates!  Fedora/Suse packages made using Alien
  31. 31. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 31 Documentation (User Guide)
  32. 32. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 32 Documentation (User Guide)  Produced using Sphinx:  input is reStructuredText  output is high-quality formatted HTML, PDF, …  Used Enthought reST editor for writing
  33. 33. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 33 Documentation (API)  HTML API generated from docstrings using Endo  Alternatives:  Epydoc  Doxygen
  34. 34. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 34 Code availability
  35. 35. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 35 Soliciting user feedback  getsatisfaction.com Javascript added to Sphinx template
  36. 36. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 36 Conclusions  Cross-platform application development in Python is easy, deployment is hard  Need to become an expert in knowing the differences between platforms  50 / 50 development vs. the rest  Good to be part of a team :)
  37. 37. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 37 Publications
  38. 38. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 38 Acknowledgements  Jamie Twycross (mcss, packaging)  Fran Romero Campero (IML, pmodelchecker, docs)  Claudio Lima (poptimizer, docs)  Pawel (for turning me on to Python)  Natalio Krasnogor
  39. 39. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 39 Questions?
  40. 40. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 40 Slides available on web
  41. 41. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 41 Infobiotics

×