Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010

Generic Image
Library
Lubomir Bourdev
Adobe Systems and U.C. Berkeley

Hailin Jin
Adobe Systems.

Christian Henning
Independent consultant

1
2010 Adobe Systems Incorporated. All Rights Reserved.

Agenda

 What is GIL?
 Basic concepts and navigation
 Generic code
 Image view transformations
 1D iterators; pixel algorithms
 GIL extensions
 Is GIL for you?

2

Agenda

 What is GIL?
 Generic code
 GIL extensions
 Is GIL for you?

3

Image Represenations

 4x3 image in which the second pixel is hilighted
color space (RGB, CMYK…)
 In interleaved (chunky) form:
channel order (RGB vs. BGR)
row padding policy
channel depth 8-bit, 16-bit…
planar vs. interleaved
 In planar form:

 It is challenging to write code that is reusable and efficient

4

Generic Image Library (GIL)

 GIL abstracts image representations and algorithms
 Allows you to write an algorithm once and have it work on any image format
 GIL is one of the boost C++ libraries (see www.boost.org)
 GIL is all header files – requires no linking
 GIL is open-source and free, including for commercial use

5

Agenda

 What is GIL?
 Generic code
 GIL extensions
 Is GIL for you?

6

Example – Computing the Gradient Image

Simplifying assumptions
- Only horizontal gradient
- Only 8-bit grayscale image
- Central difference:

( - )/2 →

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D

7

Example – Computing the Gradient Image

Simplifying assumptions
- Only horizontal gradient
- Only 8-bit grayscale image
- Central difference
- Ignore boundary cases

( - )/2 →

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D

8

Interfacing with code outside GIL

void ComputeXGradientGray8(int width, int height,
const unsigned char* src_pix, int src_rowbytes,
signed char* dst_pix, int dst_rowbytes)
{
gray8c_view_t src = interleaved_view(width, height,
(const gray8_pixel_t*)src_pix, src_rowbytes);

}

9


{

GIL image view

} - Provides uniform access to the pixels of an image
- Lightweight (16 bytes: pointer + dimensions + rowbytes)

10


{
gray8s_view_t dst = interleaved_view(width, height,
(gray8s_pixel_t*)dst_pix, dst_rowbytes);
x_gradient(src,dst);
}

11


grayscale
{
}

12


8-bit (unsigned char)
{
}

13


8-bit (signed char)
{
}

14


const (read-only)
{
}

15


image view
{
}

16

GIL Concrete Types

rgb32fc_planar_step_view_t

rgb 8 _ _ _ _ pixel
bgr 16 f c planar step ref
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

17

GIL Concrete Types


rgb 8 _ _ _ _ pixel
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

18

GIL Concrete Types


rgb 8 _ _ _ _ pixel
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

19

GIL Concrete Types


rgb 8 _ _ _ _ pixel
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

20

GIL Concrete Types


rgb 8 _ _ _ _ pixel
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

21

GIL Concrete Types


rgb 8 _ _ _ _ pixel
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

22

GIL Concrete Types


rgb 8 _ _ _ _ pixel
gray 32 s ptr
lab 64 loc
cmyk … view
rgba image
… …

23

Simplest Implementation

#include <boost/gil/gil_all.hpp>
using namespace boost::gil;

void x_gradient(const gray8c_view_t& src,
const gray8s_view_t& dst)
{
for (int y=0; y<src.height(); ++y)
for (int x=1; x<src.width()-1; ++x)
dst(x,y) = (src(x+1,y) - src(x-1,y)) / 2;
}

24

The simplest version is too slow

 Better to remember the beginning of each row and offset from it

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D

25

Faster Version

{
{
gray8c_view_t::x_iterator src_it = src.row_begin(y);
gray8s_view_t::x_iterator dst_it = dst.row_begin(y);

dst_it[x] = (src_it[x+1] - src_it[x-1]) / 2;
}
}

26

Faster Version

{
pixel iterator
{

}
}

27

GIL constructs are very lightweight

{
a raw C pointer
{

}
} raw C pointer indexing operator

28

Compute gradient in the vertical dimension

I I I I I
I I I I I D D D D D
I I I I I D D D D D
I I I I I D D D D D
I I I I I

29

Computing the X-Gradient

{
{

}
}

30

Computing the Y-Gradient

void y_gradient(const gray8c_view_t& src,
{
for (int x=0; x<src.width(); ++x)
{
gray8c_view_t::y_iterator src_it = src.col_begin(x);
gray8s_view_t::y_iterator dst_it = dst.col_begin(x);

for (int y=1; y<src.height()-1; ++y)
dst_it[y] = (src_it[y+1] - src_it[y-1]) / 2;
}
}

31

Computing the Y-Gradient

void y_gradient(const gray8c_view_t& src,
{
pixel step iterator (8 bytes)
for (int x=0; x<src.width(); ++x)
{
gray8c_view_t::y_iterator src_it = src.col_begin(x);
gray8s_view_t::y_iterator dst_it = dst.col_begin(x);

for (int y=1; y<src.height()-1; ++y)
dst_it[y] = (src_it[y+1] - src_it[y-1]) / 2;
}
} multiplies index by the step

32

Using Locators

 Pixel locators are the 2D equivalents of pixel iterators
 Lightweight and fast:
I I I I I
I I I I I D D D D D
I I I I I D D D D D
I I I I I D D D D D
I I I I I

gray8c_view_t::xy_locator loc = src.xy_at(0,1);
loc.x()++;
(“below” – “above”) / 2
loc.y()+=5;
(*dst) = (loc(0,1) - loc(0,-1)) / 2;

33

Agenda

 What is GIL?
 Generic code
 GIL extensions
 Is GIL for you?

34

Making the Code Generic

template <typename SrcView, typename DstView>
void x_gradient(const SrcView& src, const DstView& dst) {

 GIL constructs and algorithms operate on Concepts
 You can use your own images, image views, locators, iterators, pixels
channels, color spaces, etc.

35


gil_function_requires<ImageViewConcept<SrcView> >();

 Using boost::concept_check to enforce concept syntactic requirements
 Zero run-time overhead (gets compiled out)
 Sometimes significant compile-time overhead

36



 SrcView must be satisfy all requirements of an image view

37


gil_function_requires<MutableImageViewConcept<DstView> >();

 DstView must be an image view that is mutable (i.e. writable)

38


gil_function_requires<MutableImageViewConcept<DstView> >();
gil_function_requires<ColorSpacesCompatibleConcept<
typename SrcView::color_space_t,
typename DstView::color_space_t> >();

 Both views must have the same number and interpretation of channels.

39

The Full Version

void x_gradient(const SrcView& src, const DstView& dst)
{
int max_x = src.width()-1;
{
typename SrcView::x_iterator src_it = src.row_begin(y);
typename DstView::x_iterator dst_it = dst.row_begin(y);

for (int x=1; x<max_x; ++x)
for (int c=0; c<src.num_channels(); ++c)
dst_it[x][c] = (src_it[x+1][c]-src_it[x-1][c])/2;
}
} explicit loop over
the channels

40

And if you are a generic programming enthusiast…

{
{

static_transform(src_it[x+1], src_it[x-1], dst_it[x],
(_1 - _2)/constant(2));
}
}

41


{
{

(_1 - _2)/constant(2));
} GIL channel-level
algorithm
}

42


{
{

(_1 - _2)/constant(2));
}
} boost::lambda
expression

43

GIL Channel Level Algorithms

STL GIL
std::for_each gil::static_for_each
std::transform gil::static_transform
std::fill gil::static_fill
std::max_element gil::static_max
std::min_element gil::static_min

 Channel algorithms are compile-time recursive (no explicit loop)
 Channels are paired semantically (consider RGB and BGR)

44

GIL Type Metafunctions

 Generators
image_type<Channel, Layout, IsPlanar>::type
view_type<Channel, Layout, IsPlanar, …>::type
derived_image_type<Img, Channel, Layout, …>::type
derived_view_type<View, Channel, Layout, …>::type

45

GIL Type Metafunctions

 Generators
image_type<Channel, Layout, IsPlanar>::type
view_type<Channel, Layout, IsPlanar, …>::type
derived_image_type<Img, Channel, Layout, …>::type
derived_view_type<View, use_default, …>::type

 Type analysis metafunctions:
[image/view/locator/iterator] + _is_ + [basic/mutable/planar/step]
if (view_is_planar<View>::value) {
…
}

46

But how generic is it?

47

Interfacing with Your Code

void XGradientRGB8_BGR16(int w, int h,
const unsigned char* src_pixels, int src_rowbytes,
signed short* dst_pixels, int dst_rowbytes)
{
rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pix,src_rowbytes);
rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pix,dst_rowbytes);
}

void XGradientPlanarLAB8_LAB32(int w, int h,
const uint16* src_l, const uint16* src_a, const uint16* src_b, int src_rowbytes,
signed int* dst_pixels, int dst_rowbytes)
{
lab16c_planar_view_t src=planar_lab_view (w,h, src_l,src_a,src_b, src_rowbytes);
lab32s_view_t dst=interleaved_view(w,h,(lab32s_pixel_t*)dst_pixels,dst_rowbytes);
}

48


{
any color space
}

{
}

49


{
} any channel depth

{
}

50


any channel ordering
{
}

{
}

51


{
}

{
}

any pixel organization

52

Reading between the bytes

 Channels may not be byte aligned

R R G G G B B B R R G G G B B B R R G G G B B B
Byte 1 Byte 2 Byte 3
typedef packed_image3_type<uint8_t,2,3,3,rgb_layout_t>::type img_t;

 Pixels may not be byte aligned:
Pixel 1 Pixel 2 Pixel 3

R R R G G G B B B R R R G G G B B B R R R G G G
Byte 1 Byte 2 Byte 3
typedef bit_aligned_image3_type<3,3,3,rgb_layout_t>::type img_t;

 You may need to define operator- and operator/ for new channels

53

Synthetic Image Views

 Define a view of an arbitrary function
 Apply an arbitrary view transformation on it
 Run an arbitrary algorithm on it
 Details in the GIL Tutorial

54

Extending GIL

 Every component of GIL can be replaced
 Rachel can make a channel with a range -1..5
 Monica can construct his own XYZab color space
 Chandler can create a Pixel iterator that has variable step
 Matt can make view of an image that is inside a jpeg file

 … and you can use your x_gradient on Matt’s view with
Monica’s color space and Rachel’s channel
 … and no developers need to coordinate with each other

55

GIL is generic.
But how fast is it?

 Test a complex case (both src and dst are planar, say 8-bit rgb)
 Use a very generic version (with function objects and boost::lambda)

56

Contestant 1: The “generic enthusiasts” version

static_transform(src_it[x+1], src_it[x-1],
dst_it[x], (_1 - _2)/constant(2));

57



Planar pointer:
struct {
char *r, *g, *b;
};

58



operator[] returns by
value a reference proxy:
struct {
char &r, &g, &b;
};

59



- Uses compile-time recursion
- Uses specialization to pair
channels semantically

60



boost::lambda
anonymous function object

61

Contestant 2: Good Ol’ C version

char *sr, *sg, *sb;
unsigned char *dr, *dg, *db;
int max_x;

for (int x=1; x<max_x; ++x) {
dr[x] = (sr[x+1] - sr[x-1]) / 2;
dg[x] = (sg[x+1] - sg[x-1]) / 2;
db[x] = (sb[x+1] - sb[x-1]) / 2;
}

62

Assembly code of the inner loop (MSVC 8)
Good Ol’ C
00410C12 movzx edx,byte ptr [ecx]
00410C15 movzx eax,byte ptr [ecx-2]
00410C19 sub eax,edx
00410C1B cdq
00410C1C sub eax,edx
00410C1E mov edx,dword ptr [esp+14h]
00410C22 sar eax,1
00410C24 mov byte ptr [edx+esi],al
00410C27 movzx edx,byte ptr [ebx+ecx]
00410C2B mov eax,dword ptr [esp+18h]
00410C2F movzx eax,byte ptr [eax+edi]
00410C35 cdq
00410C38 sar eax,1
00410C3A mov byte ptr [esi],al
00410C3C movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi]
00410C45 cdq
00410C48 mov edx,dword ptr [esp+1Ch]
00410C4C sar eax,1
00410C4E mov byte ptr [edx+esi],al
00410C51 add ecx,1
00410C54 add esi,1
00410C57 add edi,1
00410C5A sub dword ptr [esp+38h],1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h)

63

Good Ol’ C Generic Enthusiasts
00414FD0 mov edx,dword ptr [esp+14h]
00410C12 movzx edx,byte ptr [ecx] 00414FD4 movzx edx,byte ptr [edx+ecx]
00410C15 movzx eax,byte ptr [ecx-2] 00414FD8 mov eax,dword ptr [esp+18h]
00410C19 sub eax,edx 00414FDC movzx eax,byte ptr [eax+edi]
00410C1B cdq 00414FE0 sub eax,edx
00410C1C sub eax,edx 00414FE2 cdq
00410C1E mov edx,dword ptr [esp+14h] 00414FE3 sub eax,edx
00410C22 sar eax,1 00414FE5 mov edx,dword ptr [esp+1Ch]
00410C24 mov byte ptr [edx+esi],al 00414FE9 sar eax,1
00410C27 movzx edx,byte ptr [ebx+ecx] 00414FEB mov byte ptr [edx+esi],al
00410C2B mov eax,dword ptr [esp+18h] 00414FEE movzx edx,byte ptr [ecx]
00410C2F movzx eax,byte ptr [eax+edi] 00414FF1 movzx eax,byte ptr [ecx-2]
00410C33 sub eax,edx 00414FF5 sub eax,edx
00410C35 cdq 00414FF7 cdq
00410C38 sar eax,1 00414FFA sar eax,1
00410C3A mov byte ptr [esi],al 00414FFC mov byte ptr [esi],al
00410C3C movzx edx,byte ptr [ecx+ebp] 00414FFE movzx edx,byte ptr [ecx+ebp]
00410C40 movzx eax,byte ptr [edi] 00415002 movzx eax,byte ptr [edi]
00410C43 sub eax,edx 00415005 sub eax,edx
00410C45 cdq 00415007 cdq
00410C48 mov edx,dword ptr [esp+1Ch] 0041500A mov edx,dword ptr [esp+20h]
00410C4C sar eax,1 0041500E sar eax,1
00410C4E mov byte ptr [edx+esi],al 00415010 mov byte ptr [edx+esi],al
00410C51 add ecx,1 00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1
00410C5A sub dword ptr [esp+38h],1 0041501C sub ebx,1
00410C5F jne gil::x_gradient_no_gil+72h (410C12h) 0041501F jne gil::x_gradient1...+0B0h (414FD0h)

64

00410C35 cdq 00414FF7 cdq
00410C45 cdq 00415007 cdq
00410C51 add ecx,1 00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1

 Some code shifted around

65

00410C35 cdq 00414FF7 cdq
one extra
mov
00410C45 cdq 00415007 cdq
00410C51 add ecx,1 00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1

 Some very minor changes

66

00410C35 cdq 00414FF7 cdq
one extra
mov
00410C45 cdq 00415007 cdq
indexed
00415008 sub eax,edx
00410C4C sar eax,1
mem.
0041500E sar eax,1
00410C51 add ecx,1
addressing
00415013 add esi,1
00410C54 add esi,1 00415016 add ecx,1
00410C57 add edi,1 00415019 add edi,1

 Some very minor changes
 No measurable performance difference

67

GIL is generic
GIL is fast
But how readable is it?

68


 Generic:
static_transform(src_it[x-1], src_it[x+1],

 Non-generic:
dr[x] = (sr[x+1] - sr[x-1]) / 2;
dg[x] = (sg[x+1] - sg[x-1]) / 2;
db[x] = (sb[x+1] - sb[x-1]) / 2;
}

69


 Generic: same assembly code
for (int x=1; x<max_x; ++x) (MSVC8 unrolls the loop)
for (int c=0; c<SrcView::num_channels; ++c)
dst_it[x][c] = (src_it[x+1][c]-src_it[x-1][c])/2;

 Non-generic:
for (int x=1; x<max_x; ++x) { for (int x=1; x<max_x; ++x) = (s1[x-4] – s1[x+4]) / 2;
d[x]
d1[x] {
= (s1[x-1] d2[x] = (s2[x-3] – s2[x+5]) / 2;
– s1[x+1]) / 2;
d1[x] = (s1[x-1] – s1[x+1]) / 2; (s2[x-1] d3[x] = (s3[x-2] – s3[x+6]) / 2;
d[x+1] = – s2[x+1]) / 2;
d2[x] = (s2[x-1] – s2[x+1]) / 2; (s3[x-1] d4[x] = (s4[x-1] – s4[x+7]) / 2;
d[x+2] = – s3[x+1]) / 2;
dr[x] = (sr[x+1] - sr[x-1]) / 2;
d3[x] = (s3[x-1] –
d4[x] = (s4[x-1] –
s3[x+1]) /
}
=
s4[x+1]) / 2;
}
d[x+3]2; (s4[x-1] – s4[x+1]) / 2;
d[x] = (s[x-3] – s[x+3]) / 2;
} d[x+1] = (s[x-2] – s[x+4]) / 2;
d[x+2] = (s[x-1] – s[x+5]) / 2;

dg[x] = (sg[x+1] - sg[x-1]) / 2; }
for (int x=1; x<max_x; ++x) { d1[x] = (s1[x-1] – s1[x+1]) / 2;
d1[x] = (s[x-3] – s[x+3]) / 2; }
d1[x] = (s1[x-1] – s1[x+1]) / 2;
d2[x] = (s[x-2] – s[x+4]) / 2;
db[x] = (sb[x+1] - sb[x-1]) / 2;
}
d3[x] = (s[x-1] – s[x+5]) / 2;
d2[x] = (s2[x-1] – s2[x+1]) / 2;
d3[x] = (s3[x-1] – s3[x+1]) / 2;
d4[x] = (s4[x-1] – s4[x+1]) / 2;
}

}

70

Agenda

 What is GIL?
 Generic code
 GIL extensions
 Is GIL for you?

71

Using Image View Transformations

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D

void y_gradient(const SrcView& src, const DstView& dst)
{
x_gradient(rotated90cw_view(src),
rotated90cw_view(dst));
}

72

Using Image View Transformations

I
I
I
I
I

D D D
D D D
D D D
D D D
D D D
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I

template <typename SrcView, typename DstView> I

void y_gradient(const SrcView& src, const DstView& dst)
{
x_gradient(rotated90cw_view(src),
rotated90cw_view(dst));
} returns a new view
Fast: no copying of data!

73

Compute gradient just over the N-th channel

R G B R G B R G B R G B R G B

x_gradient(nth_channel_view(src, n), dst);

returns a step view

74

Compute gradient just over the N-th channel

R R R R R G G G G G B B B B B

x_gradient(nth_channel_view(src, n), dst);

returns a simple, non-
step view

75

Compute gradient just over the even pixels


x_gradient(subsampled_view(src,2,2), dst);

76

Compute luminosity gradient of the image

RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G
RGRAYB
G RGRAYB
G RGRAYB
G GRAY
R G B RGRAYB
G

x_gradient(color_converted_view<gray8_pixel_t>(src),dst);

77

GIL View Transformation Functions

template <View> View flipped_up_down_view(const View& src);
template <View> View1 flipped_left_right_view(const View& src);
template <View> View1 transposed_view(const View& src);
template <View> View1 rotated180_view(const View& src);
template <View> View1 rotated90cw_view(const View& src);
template <View> View1 rotated90ccw_view(const View& src);

template <View> View subimage_view(const View& src,
const View::point_t& top_left, const View::point_t& dims);

template <View> View1 subsampled_view(const View& src,
const View::point_t& step);

template <View,P> View2 color_converted_view(const View& src);
template <View> View3 nth_channel_view(const View& view, int n);

78

Sample Code – Using Image View Transformations

jpeg_read_image(in_dir + "monkey.jpg“, img);
step1 = subimage_view(view(img), 200,300,150,150);
step2 = color_converted_view<gray8_pixel_t>(step1);
step3 = rotated180_view(step2);
step4 = subsampled_view(step3, 2,1);
x_gradient(step4, grad_view);

 No extra memory used
 No copying of pixels (working on original data)
 All the work is done inside x_gradient

79

Agenda

 What is GIL?
 Generic code
 GIL extensions
 Is GIL for you?

80

Subimage View

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D

81

Subimage View

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D

82

Subimage View

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
const gray8s_view_t& dst) {
if (src.width()<=2) return;
x_gradient_unsafe(
subimage_view(src,1,0,src.width()-2,src.height()),
subimage_view(dst,1,0,src.width()-2,src.height())
);
}

83

Subimage View

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
x_gradient_unsafe(
);
}

84

Subimage View

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
x_gradient_unsafe(
);
}

85

Subimage View

I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
I I I I I D D D
if (src.width()<=2) return; canonical algorithm
x_gradient_unsafe(
);
}

86

1-D pixel iterators

void x_gradient_unsafe(const gray8c_view_t& src,
{
gray8c_view_t::iterator src_it = src.begin();
gray8s_view_t::iterator dst_it = dst.begin();
while (dst_it!=dst.end()) {
*dst_it = (src_it.x()[1] - src_it.x()[-1]) / 2;
++src_it;
++dst_it;
}
}

87

1-D pixel iterators

{
++src_it;
++dst_it;
} P P P P
} 1-D iterator over all pixels:
P P P P

P P P P

88

1-D pixel iterators

{
++src_it;
++dst_it;
}
} STL-like interface

89

Image View as 1-D array

{
for (size_t i=0; i<src.size(); ++i)
dst[i] = (src.at(i).x()[1] - src.at(i).x()[-1])/2;
}

90

Image View as 1-D array

{
for (size_t i=0; i<src.size(); ++i)
dst[i] = (src.at(i).x()[1] - src.at(i).x()[-1])/2;
}

Can use view like
std::vector

91

STL-like pixel algorithms

struct half_x_difference {
int operator()(const gray8c_loc_t& src_loc) const {
return (src_loc.x()[1] - src_loc.x()[-1]) / 2;
}
};

transform_pixel_positions(src, dst,
half_x_difference());
}
GIL pixel-level algorithm

92

GIL Pixel-Level Algorithms

gil::fill_pixels std::fill
gil::for_each_pixel std::for_each
gil::transform_pixels std::transform
gil::equal_pixels std::equal
gil::copy_pixels std::copy

gil::for_each_pixel_position adobe::for_each_position
gil::transform_pixel_positions

gil::copy_and_convert_pixels

93

Agenda

 What is GIL?
 Generic code
 GIL extensions
 Is GIL for you?

94

GIL Extensions

 I/O (Christian Henning) Support for reading/writing common
 read/write jpeg/tiff/png/bmp image formats
 Dynamic Image (Lubomir Bourdev, Hailin Jin)
 Support for images with runtime instantiated types
Support for images whose type
is specified at run time
 Numeric (Hailin Jin, Lubomir Bourdev)
bilinear resampling, 1D convolution (resize, blur, etc)

Basic image processing
 FreetypeGIL (Tom Brinkman)
 Freetype wrapper to GIL

 GIL2.Bindings (Hirotaka Niitsuma)
GIL wrappers for popular
image libraries
 Interface to VIGRA, OpenCV, uBLAS, TEO

 OpenCV (Christian Henning)
Improved Debugging
 Support for OpenCV algorithms

 SDL (Christian Henning)
 Improves debugging – displays current image in a window Multithreading
 GIL-Threaded (Victor Bogado)
multithreading and multi-core support for GIL algorithms

Rendering with antialiasing
 WulineGIL (Tom Brinkman)
 Draw antialiased lines using the Wu algorithm

 Toolbox (Christian Henning)
Additional color spaces
 Support for HSL and HSV color spaces and some new RGB algorithms

95

GIL Extensions Wanted

 Comprehensive image processing
 FFT, even 2D convolution, high-quality efficient resampling

 Computer vision algorithms
 Canny edge detection, SIFT, GB, segmentation, HOG…

 Support for video and multi-dimensional data
 Support for Intel’s IPP

Can you help?

96

Agenda

 What is GIL?
 Generic code
 GIL Extensions
 Is GIL for you?

97

“I would use GIL but…”

1. “… I can’t hand-tune and SSE-optimize generic code”
2. “… I am planning to adopt / have adopted another imaging library”
3. “… I already have imaging code. I can’t afford to throw it away”
4. “… I don’t want to have any extra library dependencies”
5. “… who is going to support me?”
6. “… I am concerned about portability. How portable is GIL?”
7. “… compile errors in generic code are a nightmare”
8. “… GIL has a steep learning curve”
9. “… generic code can result in code bloat”
10. “… generic code is opaque. I can’t easily tell if the generated code is
efficient”

98

Can generic code be as efficient as hand-written code?

GENERIC copy_pixels while (first!=last)
*dst++ = *first++;

copy copy copy copy copy
chunky rgb chunky rgb planar rgb planar rgb planar cmyk
CONCRETE chunky rgb planar rgb chunky rgb planar rgb planar cmyk

99

Concrete versions can be made much faster

*dst++ = *first++;

an order of magnitude faster
than the generic version
memmove(…)
memmove(…)
memmove(…)


100

GIL provides overloads at appropriate level of genericity

*dst++ = *first++;

memmove each
single color plane
memmove
SEMI- copy copy
GENERIC chunky to planar to
chunky planar


101

Advantage: New image types can be handled efficiently

*dst++ = *first++;

memmove each
single color plane
memmove
SEMI- copy copy
GENERIC chunky to planar to
chunky planar

copy copy copy copy copy copy
chunky rgb chunky rgb planar rgb planar xylab planar rgb planar cmyk
CONCRETE chunky rgb planar rgb chunky rgb planar xylab planar rgb planar cmyk

102


efficient”

103


efficient”

104


efficient”

105


efficient”

106


efficient”

107

Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010

Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010

Recommended

Recommended

More Related Content

Similar to Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010

Similar to Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010 (20)

More from zukun

More from zukun (20)

Cvpr2010 open source vision software, intro and training part iv generic image library - bourdev, henning - agenda - 2010