Agile Computer Control of a Complex Experiment by  Gaël Varoquaux presented by  Jonathan Blakes on 18 th  September 2009 f...
Outline of this talk <ul><li>Why I chose this paper </li></ul><ul><li>Outline of the paper </li></ul><ul><li>Section by se...
Why this paper? <ul><li>Highly practical </li></ul><ul><li>Mirrors common experimental workflow </li></ul><ul><li>Nice exa...
Shameless plug for the Infobiotics Workbench
Outline of the paper <ul><li>Introduction </li></ul><ul><li>Interfacing with specialist hardware </li></ul><ul><li>Unit te...
Introduction <ul><li>Experimentalists use (and build) specialist equipment </li></ul><ul><li>They are are computer literat...
Interfacing with hardware <ul><li>Critical code often written in C (controller API / performance) </li></ul><ul><ul><li>Di...
Unit testing the experiment <ul><li>As complexity of task grows the elementary operations on which relies must be robust <...
Parallel operation <ul><li>Often have to wait for devices or data-processing – but how long? </li></ul><ul><li>Should run ...
Threading <ul><li>Actually very easy in Python: </li></ul><ul><li>Prevents blocking UI </li></ul>
Event-driven programs <ul><li>Problem: Events occuring out of sequence can bring down experiment </li></ul><ul><li>Solutio...
Super-simple event queue <ul><li>Inherits deque, a fast stack/queue: O(1)  pop  &  insert(0, x) </li></ul><ul><li>dispatch...
Building GUIs <ul><li>Want to setup and run experiments interactively </li></ul><ul><ul><li>display and (sensible) editing...
What are Traits? <ul><li>Static type definitions for Python variables (otherwise dynamically typed) </li></ul><ul><li>Exam...
TraitsUI example
Other benefits of TraitsUI <ul><li>Multiple views per object </li></ul><ul><ul><li>Reuse objects in different contexts </l...
Data-driven programs <ul><li>Responding to data changes from </li></ul><ul><ul><li>Devices </li></ul></ul><ul><ul><li>User...
Putting it all together
Acknowledgements (plug) <ul><li>Traits are an innovation of Enthought Inc. </li></ul><ul><ul><li>sponsor SciPy (includes N...
References <ul><li>Varoquaux G. Agile  Computer Control of a Complex Experiment.   Computing in Science & Engineering  (20...
Other resources <ul><li>Software Carpentry course </li></ul><ul><ul><li>http://software-carpentry.org/ </li></ul></ul><ul>...
finally: <ul><li>'Passing that knowledge along' </li></ul><ul><li>Check out   .com </li></ul><ul><li>a free programming Q&...
Upcoming SlideShare
Loading in …5
×

20090918 Agile Computer Control of a Complex Experiment

668 views

Published on

2009 Journal Club

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
668
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

20090918 Agile Computer Control of a Complex Experiment

  1. 1. Agile Computer Control of a Complex Experiment by Gaël Varoquaux presented by Jonathan Blakes on 18 th September 2009 for Inter-disciplinary Optimisation Laboratory Journal Club
  2. 2. Outline of this talk <ul><li>Why I chose this paper </li></ul><ul><li>Outline of the paper </li></ul><ul><li>Section by section discussion of underlying technologies and practical implementation </li></ul><ul><li>Some useful references </li></ul>
  3. 3. Why this paper? <ul><li>Highly practical </li></ul><ul><li>Mirrors common experimental workflow </li></ul><ul><li>Nice example of an appropriate degree of programming language evangelism </li></ul><ul><li>Introduces TraitsUI for building GUIs </li></ul><ul><ul><li>I use Mayavi2’s mlab module for 3D plotting in our application, maintained by Gael </li></ul></ul>
  4. 4. Shameless plug for the Infobiotics Workbench
  5. 5. Outline of the paper <ul><li>Introduction </li></ul><ul><li>Interfacing with specialist hardware </li></ul><ul><li>Unit testing the experiment </li></ul><ul><li>Performing operations in parallel </li></ul><ul><li>Event-driven programming </li></ul><ul><li>Building GUIs </li></ul><ul><li>Data-driven programming </li></ul><ul><li>Putting it all together </li></ul>
  6. 6. Introduction <ul><li>Experimentalists use (and build) specialist equipment </li></ul><ul><li>They are are computer literate and mathematically competent but not computer scientists </li></ul><ul><li>Bad software design choices can hold up research, even though it is not the 'hard' part </li></ul><ul><li>Need software that evolves with the physical experiment </li></ul><ul><li>A 'flexible and reliable code base' can help the lab reorientate with goal and resource changes </li></ul>
  7. 7. Interfacing with hardware <ul><li>Critical code often written in C (controller API / performance) </li></ul><ul><ul><li>Difficult for new people to contribute with steep learning curve </li></ul></ul><ul><ul><li>New code can introduce memory-management problems </li></ul></ul><ul><li>Advocates Python as high-level language for connecting to devices </li></ul><ul><ul><li>Python has modules for controlling VISA instruments </li></ul></ul><ul><ul><li>Wrapping API calls (using 'ctypes' module) to a generic interface enables new devices to be incorporated without changing data-processing/visualisation routines </li></ul></ul><ul><ul><li>'Pushes' memory-management problems out of C. </li></ul></ul>
  8. 8. Unit testing the experiment <ul><li>As complexity of task grows the elementary operations on which relies must be robust </li></ul><ul><li>Automated test harness ensures bugs are picked up early </li></ul><ul><li>Can replace temperamental hardware with 'mock objects' </li></ul><ul><ul><li>test software without them </li></ul></ul><ul><ul><li>forces you to understand instruments more fully </li></ul></ul>
  9. 9. Parallel operation <ul><li>Often have to wait for devices or data-processing – but how long? </li></ul><ul><li>Should run aspects concurrently using threads: </li></ul><ul><ul><li>One thread for user interface </li></ul></ul><ul><ul><li>another for hard/long computations </li></ul></ul><ul><ul><li>and one for each device </li></ul></ul><ul><li>Rule of thumb: no two threads to modify the same object </li></ul>
  10. 10. Threading <ul><li>Actually very easy in Python: </li></ul><ul><li>Prevents blocking UI </li></ul>
  11. 11. Event-driven programs <ul><li>Problem: Events occuring out of sequence can bring down experiment </li></ul><ul><li>Solution: respond to events in the order they happen rather than in the order we program them to </li></ul><ul><li>How? </li></ul><ul><ul><li>Listen for events (with listener thread), accumulate callbacks in queue, execute in order. </li></ul></ul>
  12. 12. Super-simple event queue <ul><li>Inherits deque, a fast stack/queue: O(1) pop & insert(0, x) </li></ul><ul><li>dispatch time-consuming function calls to queue </li></ul><ul><li>queue executes them in order on a separate thread </li></ul>
  13. 13. Building GUIs <ul><li>Want to setup and run experiments interactively </li></ul><ul><ul><li>display and (sensible) editing of parameters </li></ul></ul><ul><li>But designing and building GUIs is time-consuming and difficult </li></ul><ul><li>Code becomes messy as model, view and controller get merged </li></ul><ul><li>Python library TraitsUI can help by automatically generating GUIs based on underlying datamodel (Traits) </li></ul><ul><ul><li>Removes GUI implementation from code </li></ul></ul>
  14. 14. What are Traits? <ul><li>Static type definitions for Python variables (otherwise dynamically typed) </li></ul><ul><li>Example traits: </li></ul><ul><ul><li>aperture = Range(0,10,1) </li></ul></ul><ul><ul><li>plot = Instance(MLabSceneModel) </li></ul></ul><ul><li>Traits facilitate: </li></ul><ul><ul><li>Variable initialization – simple, complex or deferred </li></ul></ul><ul><ul><li>Validation – type checking and value checking </li></ul></ul><ul><ul><li>Notification – trigger events when values change (built-in, transparent Observer pattern) </li></ul></ul><ul><ul><li>Visualisation – knowing types enables appropriate widget choice </li></ul></ul><ul><li>Classes inherit from HasTraits </li></ul>
  15. 15. TraitsUI example
  16. 16. Other benefits of TraitsUI <ul><li>Multiple views per object </li></ul><ul><ul><li>Reuse objects in different contexts </li></ul></ul><ul><li>Compound views </li></ul><ul><ul><li>Reuse views in larger applications </li></ul></ul><ul><li>Don't have to embed in GUI: </li></ul><ul><ul><li>configure_traits(view=object.view) </li></ul></ul><ul><li> runs in its own event loop </li></ul><ul><li>Serialize state easily: </li></ul><ul><ul><li>configure_traits(filename=~/app/ini) </li></ul></ul>
  17. 17. Data-driven programs <ul><li>Responding to data changes from </li></ul><ul><ul><li>Devices </li></ul></ul><ul><ul><li>User input </li></ul></ul><ul><li>Traits fire events when their values change </li></ul><ul><li>Capture events with methods: </li></ul><ul><li>@aperture_changed </li></ul><ul><li>@focus_changed </li></ul><ul><li>def update_display(self): </li></ul><ul><li>self.camera.take_photo() </li></ul>
  18. 18. Putting it all together
  19. 19. Acknowledgements (plug) <ul><li>Traits are an innovation of Enthought Inc. </li></ul><ul><ul><li>sponsor SciPy (includes NumPy) </li></ul></ul><ul><ul><li>run free training webinars (US time) </li></ul></ul><ul><li>Part of Enthought Tool Suite (ETS), which also includes: </li></ul><ul><ul><li>Mayavi2 (TVTK) – 3D data visualisation </li></ul></ul><ul><ul><li>Chaco – 2D plotting, complements Matplotlib </li></ul></ul><ul><ul><li>Envisage – Eclipse-like application framework </li></ul></ul><ul><li>All open source, BSD License (very liberal </li></ul>
  20. 20. References <ul><li>Varoquaux G. Agile Computer Control of a Complex Experiment. Computing in Science & Engineering (2008) 20-25 </li></ul><ul><li>Ramachandran P. Mayavi: Making 3D Data Visualization Reusable. Proc. SciPy 2008 , 51-57 </li></ul><ul><li>Varoquax G. Writing a graphical application for scientific programming using TraitsUI. http://gael-varoquaux.info/computers/ </li></ul><ul><li>traits_tutorial/traits_tutorial.pdf </li></ul><ul><li>Enthought Inc. http://code.enthought.com/ </li></ul>
  21. 21. Other resources <ul><li>Software Carpentry course </li></ul><ul><ul><li>http://software-carpentry.org/ </li></ul></ul><ul><ul><li>For non computer scientists </li></ul></ul><ul><ul><li>Uses Python </li></ul></ul><ul><ul><li>Explains unit-testing, version control </li></ul></ul><ul><ul><li>Mentioned in Nature: </li></ul></ul><ul><ul><ul><ul><li>Wilson G. Open-source offers solutions for science software education. Nature 436: 600 (27 July 2005) doi:10.1038/nj7050-600b </li></ul></ul></ul></ul>
  22. 22. finally: <ul><li>'Passing that knowledge along' </li></ul><ul><li>Check out .com </li></ul><ul><li>a free programming Q&A wiki </li></ul>

×