20101026 ASAP Seminar

628
-1

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
628
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

20101026 ASAP Seminar

  1. 1. Development and deployment of a cross-platform open-source software package: the Infobiotics Workbench Jonathan BlakesJonathan Blakes 26/10/201026/10/2010 Supervised by Professor Natalio KrasnogorSupervised by Professor Natalio Krasnogor
  2. 2. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 2 Outline  Introduction to Infobiotics  Organisation of software  Supporting technologies  Tour of the Infobiotics Dashboard UI  Interface considerations  Deployment on Windows, Mac and Linux  Finishing touches and availability  Conclusions
  3. 3. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 3 Infobiotics Workbench
  4. 4. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 4 Infobiotics Models (syntax)
  5. 5. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 5 Infobiotics Models (semantics)
  6. 6. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 6 Infobiotics Experiments  Command line tools with a common parameter file format:  simulation (mcss)  model checking (pmodelchecker)  model structure and parameter optimisation (poptimizer)  Desktop applications (not cloud-based)  Written in C++ for speed
  7. 7. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 7 Infobiotics Dashboard  User friendly graphical user interface for perform Infobiotics experiments  setting, loading and saving of parameters  running experiments and reporting progress  plotting output  Present options in a clear and understandable manner, setting limits on parameter values  Give feedback when things go wrong!  Written in Python
  8. 8. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 8 Why Python?  Dynamic language: rapid prototyping  Large stdlib: tempfile, XML parsers, itertools  3rd party Libraries:  NumPy – n-dimensional arrays and ufuncs  pytables – HDF5 (simulator output)  PyQt – Qt GUI bindings (GPL), QScintilla text editor  Matplotlib – 2D plotting  Mayavi – 3D VTK-based visualisation (TVTK)  Traits...
  9. 9. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 9 Traits = Reactive Programming
  10. 10. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 10 Simulation Parameters (file)
  11. 11. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 11 Simulation Parameters (GUI)
  12. 12. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 12 Model checker parameters (1) MC2
  13. 13. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 13 Model checker parameters (2)
  14. 14. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 14 Simulator Results  Simulation output – amounts of each species in each compartment at each timepoint for each run - quickly become very large (gigabytes), but only some data is of use to the modeller.  selecting important information  scaling amounts (molecules) to biologist-friendly units (e.g. nanomolar)  calculating statistics across runs (and other dimensions)  plotting timeseries, exporting data, etc.
  15. 15. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 15 SimulatorResultsDialog
  16. 16. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 16 Selecting data (filtering)  Filter compartments by name using wildcards  Using set class difference method  Custom ListWidget is a factory for filter LineEdit
  17. 17. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 17 Selecting data (sorting)  By providing custom QListWidget with regular expression and function map  Sort species by name  Sort compartments by name and/or positions
  18. 18. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 18 Selecting data (sampling)  Reducing data size by sampling reduces memory requirements, speeds up calculations and unclutters plots  100 of 10,000 runs  Every 100th recorded timepoint
  19. 19. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 19 Transforming data (scaling)  Converts time, substance quantities and volumes from simulator data units into user- friendly display units, using python-quantities  For calculating concentrations, use compartment volumes (at each timepoint) if recorded, otherwise use one volume for all compartments at all timepoints
  20. 20. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 20 Exporting data  Selected data can be exported in several common formats for use with other tools:  text (.csv)  Excel (.xls)  NumPy (.npz)
  21. 21. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 21 Plotting timeseries (old)
  22. 22. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 22 Plotting timeseries (improved)
  23. 23. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 23 Exporting figures  Added ability to save figures resized to specific screen dimensions (pixels), as surprisingly, matplotlib only supports physical dimensions (inches)
  24. 24. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 24 Model checker results
  25. 25. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 25 Visualising simulations (Mayavi)
  26. 26. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 26 Cross-platform issues  A lot of communication in Workbench uses temporary files – these behave differently on Windows to Linux  must close temp files before use, which can delete them!  Pseudo-terminal emulation library pexpect used to gather progress information not for Windows  winpexpect port not functioning at time of release  fall back to 'working' progress indicator not good for long experiments
  27. 27. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 27 Deploying Python apps  A lot of dependencies inc. Python  Python comes with most Linux distributions, but not with Windows (unlike JVM) and Mac system Python is crippled  Therefore we needed to bundle Python interpreter and all libraries for Windows/Mac  Possible solution: 'freezing' Python applications, creates a single executable (~100 megabytes!)  Need a solid base to start from and add missing libraries also python(x,y) for Windows
  28. 28. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 28 Freezing Python applications  Freezing works well for small scripts, not so well for applications with many libraries  uses static code analysis for determine libraries for bundling – TraitsUI/Matplotlib use dynamic imports  Freezers: cx_freeze, bbfreeze, PyInstaller,  Wrap python setup.py py2exe/py2app with bash scripts  copy missed files, unzip zipped Python classes  patch Matplotlib  fix Mac library paths using install_name_tool
  29. 29. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 29 Windows and Mac non-Python  mcss, pmodelchecker and poptimizer need cross- compiling for 32/64-bit Linux (chroot), Windows (32- bit) and Mac (OSX Snow Leopard only)  pmodelchecker requires PRISM and MC2 which are both Java applications  InstallJammer builds cross-platform installer:
  30. 30. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 30 Linux (Debian)  Not trivial either – no authorative packaging guide  Make packages for mcss, ..., PRISM, MC2, libhdf5- 1.8  Used python-support to install Dashboard for system Python(s)  (1) add apt repository to sources.list  (2) sudo apt-get install infobiotics-workbench  Automatic updates!  Fedora/Suse packages made using Alien
  31. 31. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 31 Documentation (User Guide)
  32. 32. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 32 Documentation (User Guide)  Produced using Sphinx:  input is reStructuredText  output is high-quality formatted HTML, PDF, …  Used Enthought reST editor for writing
  33. 33. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 33 Documentation (API)  HTML API generated from docstrings using Endo  Alternatives:  Epydoc  Doxygen
  34. 34. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 34 Code availability
  35. 35. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 35 Soliciting user feedback  getsatisfaction.com Javascript added to Sphinx template
  36. 36. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 36 Conclusions  Cross-platform application development in Python is easy, deployment is hard  Need to become an expert in knowing the differences between platforms  50 / 50 development vs. the rest  Good to be part of a team :)
  37. 37. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 37 Publications
  38. 38. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 38 Acknowledgements  Jamie Twycross (mcss, packaging)  Fran Romero Campero (IML, pmodelchecker, docs)  Claudio Lima (poptimizer, docs)  Pawel (for turning me on to Python)  Natalio Krasnogor
  39. 39. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 39 Questions?
  40. 40. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 40 Slides available on web
  41. 41. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 41 Infobiotics
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×