Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Vips 4mar09e


Published on

draft e of talk on VIPS for lgm09

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

Vips 4mar09e

  1. 1. VIPS: An Image Processing Library for Large, and Not So Large, Images <ul><ul><li>John Cupitt </li></ul></ul><ul><ul><li>Imperial College </li></ul></ul><ul><ul><li>London, UK </li></ul></ul><ul><ul><li>Kirk Martinez </li></ul></ul><ul><ul><li>University of Southampton </li></ul></ul><ul><ul><li>Southampton, UK </li></ul></ul>Presented by Nicolas Robidoux LGM Montreal, 2009
  2. 2. Introduction <ul><li>VIPS is a 2D scientific image processing system. It needs little memory and runs quickly, especially on multi-CPU machines. It has good support for large images and for colour. </li></ul><ul><li>The library is mostly C with some C++. There are C, C++, Python, GUI and command-line interfaces. It runs on any Unix, with convenient packages in the major distributions, and on Windows. It is licensed under the LGPL. </li></ul><ul><li>The GUI is an unsual blend of a spreadsheet and a paint program. It is licensed under the GPL. </li></ul>
  3. 3. Speed and memory use <ul><li>Load, crop, shrink, sharpen and save a 5k x 5k pixel RGB tiled TIFF image. Fastest real time of three runs on a quiet system. Tests run on a 2 CPU Opteron server. </li></ul><ul><li>Image and Graphicsmagick both compiled with Q16. Freeimage does not have a sharpening operation, so that part of that test was skipped. Nip2 seems slow because it has a long start-up time: once it starts, it processes at the same speed as the Python and C++ versions. Octave (Matlab) does not aim to be quick, we include the time for interest. The source-code for the various implementations is on the VIPS website. </li></ul>
  4. 4. SMP scaling <ul><li>Load, crop, shrink, colour-correct, sharpen and save a 10k x 10k pixel CIELAB image in VIPS format. Number of CPUs on horizontal axis, speed-up on vertical. Run on a 64-CPU Itanium2 supercomputer (SGI Origin 2000). </li></ul><ul><li>Our first run on the 64-CPU machine was linear to about 6 CPUs then tailed off to a maximum of about 10x speedup. After the changes described in the SMP scaling slide previously we had the result above. We're not sure why scaling breaks above 32 CPUs, we ran out of computer time. </li></ul>
  5. 5. History <ul><li>VIPS was started in 1990 as the image processing system for the VASARI project (multispectral imaging of old-master paintings to detect long-term colour change). </li></ul><ul><li>10,000 x 10,000 pixels, seven colour bands, 16 bits per band, up to 1.6 GB for the final image. </li></ul><ul><ul><li>Our Sun4 had 64 MB of RAM and a 25 Mhz processor. Ouch! </li></ul></ul>
  6. 6. History <ul><li>VIPS therefore looks something like this: </li></ul><ul><li>Several operations run at once, images are pulled through the pipeline by demand from the sink. No intermediate images. </li></ul><ul><li>We have a very lightweight system for combining operations. </li></ul>
  7. 7. History <ul><li>We added SMP support in 1993: </li></ul><ul><li>Run a pipeline on each thread, sources and sinks arrange synchronisation. </li></ul>
  8. 8. History <ul><li>We added SMP support in 1993: </li></ul><ul><li>Run a pipeline on each thread, sinks arrange synchronisation. </li></ul>
  9. 9. History <ul><li>We added large file (>2GB) support in 2002: </li></ul><ul><li>We map a small window into each input file. These windows are shared between threads when possible. Window positions are calculated using 64-bit arithmetic. </li></ul>
  10. 10. History <ul><li>We improved SMP scaling in 2006: </li></ul><ul><li>A buffer manager decouples workers from the image write library, so they never have to wait. We also added a system for quickly sharing and recycling pixel buffers. </li></ul>
  11. 11. General principles <ul><li>2D colour images only: no video, no volumes </li></ul><ul><li>Any number of bands </li></ul><ul><li>All band elements in an image have the same pixel format (eg. 32-bit signed int) </li></ul><ul><li>All operations are non-destructive </li></ul><ul><li>Images are uninterpreted arrays: alpha channels, CMYK, layers, etc. are up to applications to implement on top of VIPS </li></ul><ul><li>No changing of operation parameters. Apps have to destroy and rebuild to make a change. </li></ul>
  12. 12. API <ul><li>The C++ and Python APIs are very like PIL, but without the destructive operations. The C API is somewhat like HIPS, if anyone remembers that. </li></ul><ul><ul><li>import sys </li></ul></ul><ul><ul><li>from vipsCC import * </li></ul></ul><ul><ul><li>im = VImage.VImage (sys.argv[1]) </li></ul></ul><ul><ul><li>im = im.extract_area (100, 100, im.Xsize () - 200, im.Ysize () - 200) </li></ul></ul><ul><ul><li>im = im.similarity (0.9, 0, 0, 0) </li></ul></ul><ul><ul><li>mask = VMask.VIMask (3, 3, 8, 0, </li></ul></ul><ul><ul><li>[-1, -1, -1, </li></ul></ul><ul><ul><li>-1, 16, -1, </li></ul></ul><ul><ul><li>-1, -1, -1]) </li></ul></ul><ul><ul><li>im = im.conv (mask) </li></ul></ul><ul><ul><li>im.write (sys.argv[2]) </li></ul></ul><ul><li>Load, crop, shrink, sharpen, save. The Python binding is generated automatically by SWIG. </li></ul>
  13. 13. Other features <ul><li>Pixels can be 8/16/32-bit integer, signed and unsigned, 32/64-bit float, 64/128-bit complex </li></ul><ul><li>XYZ, Lab, Yuv, Yxy, Lch, RGB colour spaces; ICC colour management with lcms </li></ul><ul><li>Operation database, plugins, metadata, many file formats, memory, disc and screen sinks </li></ul><ul><li>~350 operations, mostly simple filters: rank, Fourier, morphological operators, convolutions, histogram operations, colour, arithmetic, affine </li></ul><ul><li>Simple: ~10k lines of C for the core, 50k in operations </li></ul>
  14. 14. GUI <ul><li>nip2, the VIPS GUI, is a spreadsheet where cells can be complex objects: images, plots, widgets, matrices. </li></ul><ul><li>nip2 has its own lazy, higher-order, pure functional language with classes, somewhat like dynamically typed Haskell. Spreadsheet cells hold class instances. Cells are joined with snippets of this language. </li></ul><ul><li>As the spreadsheet recalculates, it builds optimised VIPS pipelines behind the scenes. Image generation is then pure VIPS. </li></ul><ul><li>So: fast, low memory use, huge images. </li></ul>
  15. 15. GUI
  16. 16. GUI <ul><li>You can use nip2 for quite large, complex applications. We have a set of linked workspaces which analyze four-dimensional images (volumes over time) from PET scanners to calculate tissue inflammation indicies. </li></ul><ul><li>The workspaces read about 200MB of image data, have about 1,200 complex cells, 20 GB of intermediate images and take about 2 minutes to completely recalculate. They need 350 MB of RSS to run. </li></ul><ul><li>A useful tool for a technical user, not aimed at a general audience. </li></ul>
  17. 17. GUI Paste shot of PET worksapces here
  18. 18. Bad things about VIPS <ul><li>Rather old-fashioned, slightly clunky C API </li></ul><ul><li>Still uses manpages. We get a lot of complaints about hard-to-navigate docs. </li></ul><ul><li>Limited range of operators. It would be nice to have segmentation, for example. </li></ul><ul><li>Awkward to extend, despite a plugin system. Operators have a fixed set of arguments and it's difficult to add functionality. </li></ul>
  19. 19. TODO <ul><li>We've started moving VIPS on top of GObject. </li></ul><ul><li>The current stable version has several GObject-based systems. We plan to move most of the VIPS types to GObject in the next version, then start rewriting operations in the version after. We will switch to doxygen for API docs. </li></ul><ul><li>This should give us a sane, extensible, well-documented, easy to bind API with hopefully similar performance to the current version. </li></ul><ul><li>We'll aim to have a vips7 compatibility layer </li></ul>
  20. 20. Other stuff we could talk about <ul><li>Contrast Gegl vs vips </li></ul><ul><li>More on nip2? Show the PET analysis workspaces (>1,000 cells in total?) and tot up how large the volume of images is </li></ul><ul><li>Supported file formats: not sure where this could go </li></ul><ul><li>Users ... museums, medical, university, ...? </li></ul><ul><li>Other sinks: file, memory, screen </li></ul>