SlideShare a Scribd company logo
1 of 72
Download to read offline
FUNDAMENTALS OF TEXTURE PROCESSING 

FOR BIOMEDICAL IMAGE ANALYSIS
Adrien Depeursinge, PhD
MICCAI 2015 Tutorial on Biomedical Texture Analysis (BTA), Munich, Oct 5 2015
nts for textures de-
nates. Solid textures
s that a number of
Euclidean space is
Foncubierta-Rodrí-
textured surface in
004), or 2.5-dimen-
et al. (2008), where
jects and can be in-
also used in Kajiya
aindl (2009), where
bjects to create real-
s in videos can also
analysis problem
‘‘dynamic texture’’
rowley (1999), and
the literature pub-
dical solid textures
is text is on the fea-
hniques, since only
re analysis.
undamentals of 3-D
on 2. Section 3 de-
atically retrieve pa-
n and retrieval. The
e literature are re-
he various expecta-
sulting application-
d grouped together
s of the various ap-
given in Section 7,
establish a general
Julesz, 1981), it is
atical model of tex-
is problem (Mallat,
s approaches based
defined by (x,y,z) 2 [iDx,(i + 1)Dx]; [jDy,(j + 1)Dy]; [kDz,(k +
1)Dz]) (Toriwaki and Yoshida, 2009). This cuboid is called a voxel.
The three spherical coordinates (r,h,/) are unevenly sampled to
(R,H,U) as shown in Fig. 3.
2.2. Texture primitives
The notion of texture primitive has been widely used in 2-D tex-
ture analysis and defines the elementary building block of a given
texture class (Haralick, 1979; Jain et al., 1995; Lin et al., 1999). All
texture processing approaches aim at modeling a given texture
using sets of prototype primitives. The concept of texture primitive
is naturally extended in 3-D as the geometry of the voxel sequence
used by a given texture analysis method. We consider a primitive
C(i,j,k) centered at a point (i,j,k) that lives on a neighborhood of
this point. The primitive is constituted by a set of voxels with gray
tone values that forms a 3-D structure. Typical C neighborhoods
are voxel pairs, linear, planar, spherical or unconstrained. Signal
assignment to the primitive can be either binary, categorical or
continuous. Two example texture primitives are shown in Fig. 4.
Texture primitives refer to local processing of 3-D images and local
patterns (see Toriwaki and Yoshida, 2009).
Fig. 2. 3-D digitized images and sampling in Cartesian coordinates.
Fig. 3. 3-D digitized images and sampling in spherical coordinates.
ized images and sampling
an coordinates, a generic 3-D continuous image is de-
nction of three variables f(x,y,z), where f represents a
oint ðx; y; zÞ 2 R3
. A 3-D digital image F(i,j,k) of dimen-
 O is obtained from sampling f at points ði; j; kÞ 2 Z3
red array (see Fig. 2). Increments in (i,j,k), correspond
displacements in R3
parametrized by the respective
,Dy,Dz). For every cell of the digitized array, the value
ypically obtained by averaging f in the cuboid domain
(x,y,z) 2 [iDx,(i + 1)Dx]; [jDy,(j + 1)Dy]; [kDz,(k +
waki and Yoshida, 2009). This cuboid is called a voxel.
herical coordinates (r,h,/) are unevenly sampled to
hown in Fig. 3.
primitives
n of texture primitive has been widely used in 2-D tex-
and defines the elementary building block of a given
(Haralick, 1979; Jain et al., 1995; Lin et al., 1999). All
essing approaches aim at modeling a given texture
prototype primitives. The concept of texture primitive
xtended in 3-D as the geometry of the voxel sequence
ven texture analysis method. We consider a primitive
ered at a point (i,j,k) that lives on a neighborhood of
he primitive is constituted by a set of voxels with gray
that forms a 3-D structure. Typical C neighborhoods
irs, linear, planar, spherical or unconstrained. Signal
to the primitive can be either binary, categorical or
Two example texture primitives are shown in Fig. 4.
itives refer to local processing of 3-D images and local
Toriwaki and Yoshida, 2009).
-D digitized images and sampling in Cartesian coordinates.
lattices)isnotstraightforwardandraisesseveralchal-
atedtotranslation,scalingandrotationinvariancesand
esthatarebecomingmorecomplexin3-D.
dworkandscopeofthissurvey
dingonresearchcommunities,varioustaxonomiesare
ferto3-Dtextureinformation.Aclarificationofthetax-
proposedinthissectiontoaccuratelydefinethescopeof
y.ItispartlybasedonToriwakiandYoshida,2009.Three-
altextureandvolumetrictexturearebothgeneraland
termsdesigningatexturedefinedinR3
andinclude:
umetrictexturesexistingin‘‘filled’’objects
:x;y;z2Vx;y;z&R3
gthataregeneratedbyavolumetric
aacquisitiondevice(e.g.,tomography,confocalimaging).
Dtexturesexistingonsurfacesof‘‘hollow’’objectsas
:x;y;z2Cu;v&R3
g,
namictexturesintwo–dimensionaltimesequencesas
x;y;t2Sx;y;t&R3
g,
exturereferstocategory(1)andaccountsfortexturesde-
volumeVx,y,zindexedbythreecoordinates.Solidtextures
ntrinsicdimensionof3,whichmeansthatanumberof
equaltothedimensionalityoftheEuclideanspaceis
representthesignal(Bennett,1965;Foncubierta-Rodrí-
.,2013a).Category(2)isdesignedastexturedsurfacein
Nayar(1999)andCulaandDana(2004),or2.5-dimen-
turesinLuetal.(2006)andAguetetal.(2008),where
Careexistingonthesurfaceof3-Dobjectsandcanbein-
quelybytwocoordinates(u,v).(2)isalsousedinKajiya
1989),Neyret(1995),andFilipandHaindl(2009),where
etriesareaddedontothesurfaceofobjectstocreatereal-
eringofvirtualscenes.Motionanalysisinvideoscanalso
deredamulti-dimensionaltextureanalysisproblem
tocategory(3)andisdesignedby‘‘dynamictexture’’
myandFablet(1998),ChomatandCrowley(1999),and
al.(2000).
survey,acomprehensivereviewoftheliteraturepub-
classificationandretrievalofbiomedicalsolidtextures
ory(1))iscarriedout.Thefocusofthistextisonthefea-
actionandnotmachinelearningtechniques,sinceonly
xtractionisspecificto3-Dsolidtextureanalysis.
ureofthisarticle
urveyisstructuredasfollows:Thefundamentalsof3-D
xtureprocessingaredefinedinSection2.Section3de-
ereviewmethodologyusedtosystematicallyretrievepa-
ngwith3-Dsolidtextureclassificationandretrieval.The
modalitiesandorgansstudiedintheliteraturearere-
Sections4and5,respectivelytolistthevariousexpecta-
needsof3-Dimageprocessing.Theresultingapplication-
chniquesaredescribed,organizedandgroupedtogether
6.Asynthesisofthetrendsandgapsofthevariousap-
conclusionsandopportunitiesaregiveninSection7,
ely.
mentalsofsolidtextureprocessing
ghseveralresearchersattemptedtoestablishageneral
onthe3-Dgeometricalpropertiesoftheprimitivesused,i.e.,the
elementarybuildingblockconsidered.Thesetofprimitivesused
andtheirassumedinteractionsdefinethepropertiesofthetexture
analysisapproaches,fromstatisticaltostructuralmethods.
InSection2.1,wedefinethemathematicalframeworkand
notationsconsideredtodescribethecontentof3-Ddigitalimages.
Thenotionoftextureprimitivesaswellastheirscalesanddirec-
tionsaredefinedinSection2.2.
2.1.3-Ddigitizedimagesandsampling
InCartesiancoordinates,ageneric3-Dcontinuousimageisde-
finedbyafunctionofthreevariablesf(x,y,z),wherefrepresentsa
scalaratapointðx;y;zÞ2R3
.A3-DdigitalimageF(i,j,k)ofdimen-
sionsMÂNÂOisobtainedfromsamplingfatpointsði;j;kÞ2Z3
ofa3-Dorderedarray(seeFig.2).Incrementsin(i,j,k),correspond
tophysicaldisplacementsinR3
parametrizedbytherespective
spacings(Dx,Dy,Dz).Foreverycellofthedigitizedarray,thevalue
ofF(i,j,k)istypicallyobtainedbyaveragingfinthecuboiddomain
definedby(x,y,z)2[iDx,(i+1)Dx];[jDy,(j+1)Dy];[kDz,(k+
1)Dz])(ToriwakiandYoshida,2009).Thiscuboidiscalledavoxel.
Thethreesphericalcoordinates(r,h,/)areunevenlysampledto
(R,H,U)asshowninFig.3.
2.2.Textureprimitives
Thenotionoftextureprimitivehasbeenwidelyusedin2-Dtex-
tureanalysisanddefinestheelementarybuildingblockofagiven
textureclass(Haralick,1979;Jainetal.,1995;Linetal.,1999).All
textureprocessingapproachesaimatmodelingagiventexture
usingsetsofprototypeprimitives.Theconceptoftextureprimitive
isnaturallyextendedin3-Dasthegeometryofthevoxelsequence
usedbyagiventextureanalysismethod.Weconsideraprimitive
C(i,j,k)centeredatapoint(i,j,k)thatlivesonaneighborhoodof
thispoint.Theprimitiveisconstitutedbyasetofvoxelswithgray
tonevaluesthatformsa3-Dstructure.TypicalCneighborhoods
arevoxelpairs,linear,planar,sphericalorunconstrained.Signal
assignmenttotheprimitivecanbeeitherbinary,categoricalor
continuous.TwoexampletextureprimitivesareshowninFig.4.
Textureprimitivesrefertolocalprocessingof3-Dimagesandlocal
patterns(seeToriwakiandYoshida,2009).
Fig.2.3-DdigitizedimagesandsamplinginCartesiancoordinates.
A.Depeursingeetal./MedicalImageAnalysis18(2014)176–196
OUTLINE
• Biomedical texture analysis: background
• Defining texture processes
• Notations, sampling and texture functions
• Texture operators, primitives and invariances
• Multiscale analysis
• Operator scale and uncertainty principle
• Region of interest and operator aggregation
• Multidirectional analysis
• Isotropic versus directional operators
• Importance of the local organization of image directions
• Conclusions
• References
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f (x
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gra
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in t
representation corresponds to one realization (i.e., full image)
· m
BACKGROUND – RADIOMICS - HISTOPATHOLOMICS
• Personalized medicine aims at enhancing the patient’s 

quality of life and prognosis
• Tailored treatment and medical decisions based 

on the molecular composition of diseased tissue
• Current limitations [Gerlinger2012]
• Molecular analysis of tissue composition 

is invasive (biopsy), slow and costly
• Cannot capture molecular heterogeneity
3
Intratumor Heterogeneity Reveale
B Regional Distribution of Mutations
A Biopsy Sites
SOX9
CENPN
PSMD7
RIMBP2
GALNT11
ABHD11
UGT2A1
MTOR
PPP6R2
ZNF780A
WSCD2
CDKN1B
PPFIA1
TH
SSNA1
CASP2
PLRG1
SETD2
CCBL2
SESN2
MAGEB16
NLRP7
IGLON5
KLK4
WDR62
KIAA0355
CYP4F3
AKAP8
ZNF519
DDX52
ZC3H18
TCF12
NUSAP1
X4
KDM2B
MRPL51
C11orf68
ANO5
EIF4G2
MSRB2
RALGDS
EXT1
ZC3HC1
PTPRZ1
INTS1
CCR6
DOPEY1
ATXN1
WHSC1
CLCN2
SSR3
KLHL18
SGOL1
VHL
C2orf21
ALS2CR12
PLB1
FCAMR
IFI16
BCAS2
IL12RB2
Ubiquitous Shared prima
10 cm
R7 (G4)
R5 (G4)
R9
R3 (G4)
R1 (G3) R2 (G3)
R4 (G1)
R6 (G1)
Hilum
R8 (G4)
ution of Mutations
ationships of Tumor Regions D Ploidy Profiling
PreP
PreM
R1
R2
R3
R5
R8
R9
R4
M1
M2a
M2b
C2orf85
WDR7
SUPT6H
CDH19
LAMA3
DIXDC1
HPS5
NRAP
KIAA1524
SETD2
PLCL1
BCL11A
IFNAR1
DAMTS10
C3
KIAA1267
RT4
CD44
ANKRD26
TM7SF4
SLC2A1
DACH2
MMAB
ZNF521
HMG20A
DNMT3A
RLF
MAMLD1
MAP3K6
HDAC6
PHF21B
FAM129B
RPS8
CIB2
RAB27A
SLC2A12
DUSP12
ADAMTSL4
NAP1L3
USP51
KDM5C
SBF1
TOM1
MYH8
WDR24
ITIH5
AKAP9
FBXO1
LIAS
TNIK
SETD2
C3orf20
MR1
PIAS3
DIO1
ERCC5
KL
ALKBH8
DAPK1
DDX58
SPATA21
ZNF493
NGEF
DIRAS3
LATS2
ITGB3
FLNA
SATL1
KDM5C
KDM5C
RBFOX2
NPHS1
SOX9
CENPN
PSMD7
RIMBP2
GALNT11
ABHD11
UGT2A1
MTOR
PPP6R2
ZNF780A
WSCD2
CDKN1B
PPFIA1
TH
SSNA1
CASP2
PLRG1
SETD2
CCBL2
SESN2
MAGEB16
NLRP7
IGLON5
KLK4
WDR62
KIAA0355
CYP4F3
AKAP8
ZNF519
DDX52
ZC3H18
TCF12
NUSAP1
X4
KDM2B
MRPL51
C11orf68
ANO5
EIF4G2
MSRB2
RALGDS
EXT1
ZC3HC1
PTPRZ1
Privatebiquitous Shared primary Shared metastasis
Lung
metastases
Chest-wall
metastasis
Perinephric
metastasis
M1
10 cm
R2 (G3)
R4 (G1)
R6 (G1)
HilumR8 (G4)
Primary
tumor
M2b
M2a
ution of Mutations
PreP
PreM
R1
R2
R3
R5
R8
R9
R4
M1
M2a
M2b
C2orf85
WDR7
SUPT6H
CDH19
LAMA3
DIXDC1
HPS5
NRAP
KIAA1524
SETD2
PLCL1
BCL11A
IFNAR1
DAMTS10
C3
KIAA1267
RT4
CD44
ANKRD26
TM7SF4
SLC2A1
DACH2
MMAB
ZNF521
HMG20A
DNMT3A
RLF
MAMLD1
MAP3K6
HDAC6
PHF21B
FAM129B
RPS8
CIB2
RAB27A
SLC2A12
DUSP12
ADAMTSL4
NAP1L3
USP51
KDM5C
SBF1
TOM1
MYH8
WDR24
ITIH5
AKAP9
FBXO1
LIAS
TNIK
SETD2
C3orf20
MR1
PIAS3
DIO1
ERCC5
KL
ALKBH8
DAPK1
DDX58
SPATA21
ZNF493
NGEF
DIRAS3
LATS2
ITGB3
FLNA
SATL1
KDM5C
KDM5C
RBFOX2
NPHS1
SOX9
CENPN
PSMD7
RIMBP2
GALNT11
ABHD11
UGT2A1
MTOR
PPP6R2
ZNF780A
WSCD2
CDKN1B
PPFIA1
TH
SSNA1
CASP2
PLRG1
SETD2
CCBL2
SESN2
MAGEB16
NLRP7
IGLON5
KLK4
WDR62
KIAA0355
CYP4F3
AKAP8
ZNF519
DDX52
ZC3H18
TCF12
NUSAP1
X4
KDM2B
MRPL51
C11orf68
ANO5
EIF4G2
MSRB2
RALGDS
EXT1
ZC3HC1
PTPRZ1
Privatebiquitous Shared primary Shared metastasis
Lung
metastases
Chest-wall
metastasis
Perinephric
metastasis
M1
10 cm
R2 (G3)
R4 (G1)
R6 (G1)
Hilum
R8 (G4)
Primary
tumor
M2b
M2a
BACKGROUND – RADIOMICS - HISTOPATHOLOMICS
• Huge potential for computerized medical image analysis
• Explore and reveal tissue structures related to tissue composition,
function, ….
• Local quantitative image feature extraction
• Supervised and unsupervised machine learning
4
malignant, nonresponder
malignant, responder
benign
pre-malignant
undefined
quant. feat. #1
quant.feat.#2
Supervised learning, 

big data
BACKGROUND – RADIOMICS - HISTOPATHOLOMICS
• Huge potential for computerized medical image analysis
• Create imaging biomarkers to predict diagnosis, prognosis, 

treatment response [Aerts2014]
5
Radiomics [Kumar2012] “Histopatholomics” [Gurcan2009]
Reuse existing 

diagnostic images ✓ radiology data1
✓ digital pathology
Capture tissue 

heterogeneity
✓ 3D neighborhoods

(e.g., 512x512x512)
✓ large 2D regions

(e.g., 15,000x15,000)
Analytic power beyond 

naked eyes
✓ complex 3D tissue
morphology
✓exhaustive characterization
of 2D tissue structures
Non-invasive ✓ x
1e.g., X-ray, Ultrasound, CT, MRI, PET, OCT, …
BACKGROUND – RADIOMICS - HISTOPATHOLOMICS
• Huge potential for computerized medical image analysis
• Explore and reveal tissue structures related to tissue composition,
function, ….
• Local quantitative image feature extraction
• Supervised and unsupervised machine learning
6
malignant, nonresponder
malignant, responder
benign
pre-malignant
undefined
quant. feat. #1
quant.feat.#2
Supervised learning, 

big data
Specific to texture!
OUTLINE
• Biomedical texture analysis: background
• Defining texture processes
• Notations, sampling and texture functions
• Texture operators, primitives and invariances
• Multiscale analysis
• Operator scale and uncertainty principle
• Region of interest and operator aggregation
• Multidirectional analysis
• Isotropic versus directional operators
• Importance of the local organization of image directions
• Conclusions
• References
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f (x
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gra
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in t
representation corresponds to one realization (i.e., full image)
· m
• Definition of texture
• Everybody agrees that nobody agrees on the definition of “texture” 

(context-dependent)
• “coarse”, “edgy”, “directional”, “repetitive”, “random”, …
• Oxford dictionary: “the feel, appearance, or consistency of a surface or a
substance”
• [Haidekker2011]: “Texture is a systematic local variation of the image values”
• [Petrou2011]: “The most important characteristic of texture is that it is scale
dependent. Different types of texture are visible at different scales”
COMPUTERIZED TEXTURE ANALYSIS
8
A. Depeursinge et al. / Medical Image Analysis 18 (2014) 176–196 177
[Depeursinge2014a]
COMPUTERIZED TEXTURE ANALYSIS
directions
9
scales
• Spatial scales and directions in images are fundamental

for visual texture discrimination [Blakemore1969, Romeny2011]
• Relating to directional frequencies (shown in Fourier)
COMPUTERIZED TEXTURE ANALYSIS
10
directionsscales
• Spatial scales and directions in images are fundamental

for visual texture discrimination [Blakemore1969, Romeny2011]
• Most approaches are leveraging these two properties
• Explicitly: Gray-level co-occurrence matrices (GLCM), run-length matrices
(RLE), directional filterbanks and wavelets, Fourier, histograms of gradients
(HOG), local binary patterns (LBP)
• Implicitly: Convolutional neural networks (CNN), scattering transform,
topographic independant component analysis (TICA)
NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS
11
• 2-D continuous texture functions in space and Fourier
• Cartesian coordinates:
• Polar coordinates:
• 2-D digital texture functions
• Cartesian coordinates:
• Polar coordinates:
• Sampling (Cartesian)
• Increments in corresponds to physical displacements in as
f(k), k =
✓
k1
k2
◆
2 Z2
f(R, ⇥), R 2 Z+
, ⇥ 2 [0, 2⇡)
(k1, k2) R2
✓
x1
x2
◆
=
✓
x1 · k1
x2 · k2
◆
x1
x2 k2
k1
)
x1
x2
R2
Z2
·
f(x) f(k)
f(x), x =
✓
x1
x2
◆
2 R2
, f(x)
F
! ˆf(!) =
Z
R2
f(x)e jh!,xi
dx, ! 2 R2
f(r, ✓), r 2 R+
, ✓ 2 [0, 2⇡), f(r, ✓)
F
! ˆf(⇢, ), ⇢ 2 R+
, 2 [0, 2⇡)
NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS
12
• 3-D continuous texture functions in space and Fourier
• Cartesian coordinates:
• Polar coordinates:
• 3-D digital texture functions
• Cartesian coordinates:
• Polar coordinates:
• Sampling (Cartesian)
• Increments in corresponds to physical displacements in as
x1
x2
k2
k1
)
x1x2
·
f(x) f(k)
f(k), k =
0
@
k1
k2
k3
1
A 2 Z3
f(r, ✓, ), r 2 R+
, ✓ 2 [0, 2⇡), 2 [0, 2⇡)
f(R, ⇥, ), R 2 Z+
, ⇥ 2 [0, 2⇡), 2 [0, 2⇡)
(k1, k2, k3) R3
0
@
x1
x2
x3
1
A =
0
@
x1 · k1
x2 · k2
x3 · k3
1
A
x3
x3 k3
R3 Z3
f(x), x =
0
@
x1
x2
x3
1
A 2 R3
, f(x)
F
! ˆf(!) =
Z
R3
f(x)e jh!,xi
dx, ! 2 R3
• We consider a texture function as a realization of a
spatial stochastic process of 





where is the value at the spatial position indexed by
• The values of follow one or several probability density functions
• Examples
NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS
13
f(x)
Rd
{Xm, m 2 RM1⇥···⇥Md
}
Xm m
moving average Gaussian pointwise Poisson biomedical: lung fibrosis in CT
m 2 R128⇥128
m 2 R32⇥32
m 2 R84⇥84
Xm fXm
(q)
NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS
14
• Stationarity of spatial stochastic processes
• A spatial process is stationary if the 

probability density functions are equivalent for all
• Example: heteroscedastic moving average Gaussian process
{Xm, m 2 RM1⇥···⇥Md
}
mfXm
(q)
stationary
non-stationary
(strict sense)
fb,Xm (q) =
1
3
p
2⇡
e
(q 0)2
2 32
fa fb
fa,Xm (q) =
1
1
p
2⇡
e
(q 0)2
2 12
NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS
15
• Stationarity of textures and human perception / tissue biology
• Strict/weak process stationarity and texture class definition is not equivalent
• Image analysis tasks when textures are considered as
• Stationary (wide sense): texture classification
• Non-stationary: texture segmentation
Outex “canvas039”: stationary? brain glioblastoma in T1-MRI: stationary?
)
[Storath2014]
OUTLINE
• Biomedical texture analysis: background
• Defining texture processes
• Notations, sampling and texture functions
• Texture operators, primitives and invariances
• Multiscale analysis
• Operator scale and uncertainty principle
• Region of interest and operator aggregation
• Multidirectional analysis
• Isotropic versus directional operators
• Importance of the local organization of image directions
• Conclusions
• References
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f (x
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gra
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in t
representation corresponds to one realization (i.e., full image)
· m
TEXTURE OPERATORS AND PRIMITIVES
17
• Texture operators
• A -dimensional texture analysis approach is characterized by a set of 

local operators centered at the position
• is local in the sense that each element only depends
on a subregion of
• The subregion is the support of
• can be linear (e.g., wavelets) or non-linear (e.g., median, GLCMs, LBPs)
• For each position , maps the texture function into a 

-dimensional space
N
d
f(x)
xL1 ⇥ · · · ⇥ Ld
L1
L2
M1
M2
·
m
gn(x, m) : RM1⇥···⇥Md
7! RP
, n = 1, . . . , N
gn(x, m) p=1,...,P
gn
m
gn
m gn
P
gn(f(x), m) : RM1⇥···⇥Md
7! RP
L1 ⇥ · · · ⇥ Ld gn
TEXTURE OPERATORS AND PRIMITIVES
18
• From texture operators to texture measurements (i.e., features)
• The operator is typically applied to all positions of the image
by “sliding” its window over the image
• Regional texture measurements can be obtained from the
aggregation of over a region of interest
• For instance, integration can be used to aggregate over
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m) m
µ 2 RP
gn(f(x), m) M
M
m
gn(f(x), m) M
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
TEXTURE OPERATORS AND PRIMITIVES
• Texture primitives
• A “texture primitive” (also called “texel”) is a fundamental elementary unit
(i.e., a building block) of a texture class [Haralick1979, Petrou2006]
• Intuitively, given a collection of texture functions , an appropriate set of
texture operators must be able to:
(i) Detect and quantify the presence of all distinct primitives in
(ii) Characterize the spatial relationships between the primitives (e.g., geometric 

transformations, density) when aggregated
texture primitives
,
19
texture primitive
,
fj
f1 f2 primitive
,
primitive
,
fj=1,...,J
gn
TEXTURE OPERATORS AND PRIMITIVES
• General-purpose texture operators
• In general, the texture primitives are neither well-defined, nor known in
advance (e.g., biomedical tissue)
• General-purpose operator sets are useful to estimate the primitives
• How to build such operator sets?
20
patterns in the whole HRCT volume.
Methods
ataset used is part of an internal multimedia database of ILD cases con-
HRCT images with annotated ROIs created in the Talisman project1
.
OIs from healthy and five pathologic lung tissue patterns are selected for
g and testing the classifiers selecting classes with sufficiently high repre-
on (see Table 1).
e wavelet frame decompositions with dyadic and quincunx subsampling
plemented in Java [11, 16] as well as optimization of SVMs. The basic
mentation of the SVMs is taken from the open source Java library Weka2
.
1. Visual aspect and distribution of the ROIs per class of lung tissue pattern.
al
ect
ss healthy emphysema ground glass fibrosis micronodules macronodules
ROIs 113 93 148 312 155 22
tients 11 6 14 28 5 5
esults
sotropic Polyharmonic B–Spline Wavelets
ntioned in Section 1.1, isotropic analysis is preferable for lung texture
terization. The Laplacian operator plays an important role in image pro-
and is clearly isotropic. Indeed, ∆ = ∂2
∂x2
1
+ ∂2
∂x2
2
, is rotationally invariant.
olyharmonic B–spline wavelets implement a multiscale smoothed version
Laplacian [16]. This wavelet, at the first decomposition level, can be char-
ed as
2-D lung tissue in CT images
[Depeursinge2012a]
3-D normal and osteoporotic bone in CT [Dumas2009]µ
• General-purpose texture operators
• The exhaustive analysis of spatial scales and directions is computationally
expensive when the support of the operators is large:
choices are required
• Directions (e.g., GLCMs, RLE, HOG)
• Scales
the classification accuracy of RLE versus GLCMs for categorizing
lung tissue patterns associated with diffuse lung disease in HRCT.
Using identical choices of the directions for RLE and GLCMs, they
found found no statistical differences between the classification
performance. Gao et al. (2010) and Qian et al. (2011) compared
the performance of three-dimensional GLCM, LBP, Gabor filters
and WT in retrieving similar MR images of the brain. They ob-
served a small increase in retrieval performance for LBP and GLCM
when compared to Gabor filters and WT. However, the database
used is rather small and the results might not be statistically
significant.
Several papers compared the performance of texture analysis
algorithms in their 2-D versus 3-D forms. As expected, 2-D texture
analysis is most often less discriminative than 3-D, which was ob-
served for various applications and techniques, such as:
 GLCMs, RLE and fractal dimension for the classification of lung
tissue types in HRCT in Xu et al. (2005, 2006b).
 GLCMs for the classification of brain tumors in MRI in Mah-
moud-Ghoneim et al. (2003) and Allin Christe et al. (2012).
 GLCMs for the classification of breast in contrast–enhanced
MRI, where statistical significance was assessed in Chen et al.
(2007).
 GMRF for the segmentation of gray matter in MRI in Ranguelova
and Quinn (1999).
 LBP for synthetic texture classification in Paulhac et al. (2008).
This demonstrates that 2-D slice-based discrimination of 3-D
native texture does not allow fully exploiting the information
available in 3-D datasets. An exception was observed with 2-D ver-
sus 3-D WTs in Jafari-Khouzani et al. (2004), where the 2-D ap-
proach showed a small increase in classification performance of
abnormal regions responsible for temporal lobe epilepsy. A separa-
ble 3-D WT was used, which did not allow to adequately exploit
the 3-D texture information available and may explain the ob-
served results.
7. Discussion
In the preceding sections, we have reviewed the current state-
of-the-art in 3-D biomedical texture analysis. The papers were cat-
egorized in terms of imaging modality used, organ studied and im-
age processing techniques. The increasing number of papers over
the past 32 years clearly shows a growing interest in computerized
characterization of three-dimensional texture information (see
Fig. 5). This is a consequence of increasingly available 3-D data
acquisition devices that are reaching high spatial resolutions
allowing to capture tissue properties in its natural space.
The analysis of the medical applications in 100 papers in Sec-
tion 5 shows the diversity of 3-D biomedical textures. The various
geometrical properties of the textures are summarized in Table 2,
which defines the multiple challenges of 3-D texture analysis.
The need for methods able to characterize structural scales and
Fig. 15. Amounts of pixels/voxels covered by the main 13 directions. The number N of discarded pixels/voxels (gray regions in (a) and (b)) according to the neighborhood size
R is shown in (c).
A. Depeursinge et al. / Medical Image Analysis 18 (2014) 176–196 191
 GMRF for the segmentation of gray matter in MRI in Ranguelova
and Quinn (1999).
 LBP for synthetic texture classification in Paulhac et al. (2008).
geometrical properties of the textures are summarized
which defines the multiple challenges of 3-D textu
The need for methods able to characterize structural
Fig. 15. Amounts of pixels/voxels covered by the main 13 directions. The number N of discarded pixels/voxels (gray regions in (a) and (b)) according to the neig
R is shown in (c).
TEXTURE OPERATORS AND PRIMITIVES
21
2-D 3-D (one quadrant) such that
numberofdiscarded
pixels/voxelsof order −1/2 (an isotropic smoothing operator) of f: Rf =
−∇∆−1/2
f. Let’s indeed recall the Fourier-domain definition of
these operators: ∇
F
←→ jω and ∆−1/2 F
←→ ||ω||−1
. Unlike the
usual gradient ∇, the Riesz transform is self-reversible
R⋆
Rf(ω) =
(jω)∗
(jω)
||ω||2
ˆf(ω) = ˆf(ω).
This allows us to define a self-invertible wavelet frame of L2(R3
)
(tight frame). We however see that there exists a singularity for the
frequency (0, 0, 0). This issue will be fixed later, thanks to the van-
ishing moments of the primary wavelet transform.
2-D GLCMs with various spatial distances [Haralick1979]
2-D isotropic dyadic wavelets
in Fourier [Chenouard2011]
d = 1 ( k1 = 1, k2 = 0) d = 2 ( k1 = 2, k2 = 0) d = 2
p
2 ( k1 = 2, k2 = 2)
r
L1 ⇥ · · · ⇥ Ld
L1 = L2 = L3 = 2r + 1
d
g(f(x), m) = g(f(R✓0 x x0), m), 8x0 2 R2
, ✓0 2 [0, 2⇡)
• Invariances of the texture operators to geometric
transformations can be desirable
• E.g., scaling, rotations and translations
• Invariances of texture operators can be enforced
• Example with 2-D Euclidean transforms (i.e., rotation and translation)







with the rotation matrix
INVARIANCE OF TEXTURE OPERATORS
22
R✓0
=
✓
cos ✓0 sin ✓0
sin ✓0 cos ✓0
◆
INVARIANCE OF TEXTURE OPERATORS
23
• Computer vision versus biomedical imaging
Computer vision Biomedical image analysis
translation translation-invariant translation-invariant
rotation rotation-invariant rotation-invariant
scale scale-invariant multi-scale
160
Fig. 10. (a) A digitized histopathology image of Grade 4 CaP and different graph-based r
Diagram, and Minimum Spanning tree.
Fig. 11. Digitized histological image at successively higher scales (magnifica-
tions) yields incrementally more discriminatory information in order to detect
suspicious regions.
or resolution. For instance at low or coarse scales color or tex-
ture cues are commonly used and at medium scales architec-
tural arrangement of individual histological structures (glands
and nuclei) start to become resolvable. It is only at higher res-
olutions that morphology of specific histological structures can
be discerned.
In [93], [94], a multiresolution approach has been used for the
classification of high-resolution whole-slide histopathology im-
ages. The proposed multiresolution approach mimics the eval-
uation of a pathologist such that image analysis starts from the
lowest resolution, which corresponds to the lower magnification
levels in a microscope and uses the higher resolution represen-
Fig. 12
image
1, (c) r
as susp
show
three
scale
(scal
the n
dition
highe
tumo
At
is com
COMPUTERIZED TEXTURE ANALYSIS
7
• Invariances: computer vision versus biomedical imaging
Computer vision Biomedical image analysis
scale scale-invariant multi-scale
rotation rotation-invariant rotation-invariant
[4] Histopathological image analysis: a review, Gurcan et al., IEEE Reviews in Biomed Eng, 2:147-71, 2009

160 IE
Fig. 10. (a) A digitized histopathology image of Grade 4 CaP and different graph-based representation
Diagram, and Minimum Spanning tree.
Fig. 11. Digitized histological image at successively higher scales (magnifica-
tions) yields incrementally more discriminatory information in order to detect
suspicious regions.
or resolution. For instance at low or coarse scales color or tex-
ture cues are commonly used and at medium scales architec-
tural arrangement of individual histological structures (glands
and nuclei) start to become resolvable. It is only at higher res-
olutions that morphology of specific histological structures can
be discerned.
In [93], [94], a multiresolution approach has been used for the
classification of high-resolution whole-slide histopathology im-
ages. The proposed multiresolution approach mimics the eval-
uation of a pathologist such that image analysis starts from the
lowest resolution, which corresponds to the lower magnification
levels in a microscope and uses the higher resolution represen-
tations for the regions requiring more detailed information for
a classification decision. To achieve this, images were decom-
posed into multiresolution representations using the Gaussian
pyramid approach [95]. This is followed by color space con-
version and feature construction followed by feature extraction
and feature selection at each resolution level. Once the classifier
is confident enough at a particular resolution level, the system
assigns a classification label (e.g., stroma-rich, stroma-poor or
Fig. 12. Results fr
image with the tum
1, (c) results at scale
as suspicious at low
shows the origin
three columns s
scales. Pixels cl
(scale) are disc
the number of p
ditionally, the p
higher scales al
tumor and nont
At lower reso
is commonly us
pattern of gland
tized histologic
scenes can be
to every pixel i
dient, and Gab
the scale, orien
gion of interest
features within
[Gurcan2009][Lazebnik2005]
OUTLINE
• Biomedical texture analysis: background
• Defining texture processes
• Notations, sampling and texture functions
• Texture operators, primitives and invariances
• Multiscale analysis
• Operator scale and uncertainty principle
• Region of interest and operator aggregation
• Multidirectional analysis
• Isotropic versus directional operators
• Importance of the local organization of image directions
• Conclusions
• References
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f (x
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gra
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in t
representation corresponds to one realization (i.e., full image)
· m
k2
k1
x1x2f(k)
x3
k3
MULTISCALE TEXTURE OPERATORS
25
• Inter-patient and inter-dimension scale normalization
• Most medical imaging protocols yield images with 

various sampling steps
• Inter-patient scale normalization is required to 

ensure the correspondance of spatial frequencies
• Inter-dimension scale normalization is required to ensure isotropic 

scale/directions definition
( x1, x2, x3)
x1 = x2 = 0.4mm x1 = x2 = 1.6mm
spatial Fourier Fourierspatial
0 0
0 0
⇡ ⇡⇡ ⇡
⇡ ⇡
x1x2
x3 )
x0
1x0
2
x0
3
d = 1 ( k1 = 1, k2 = 0)
GLCMs
0.4mm
1.6mm
MULTISCALE TEXTURE OPERATORS
26
• Which scales for texture measurements?
• Two aspects:
A. How to define the size(s) of the operator(s) ?
B. How to define the size of the region of interest ?
TEXTURE OPERATORS AND PRIMITIVES
• From texture operators to texture measurements
• The operator is typically applied to all positions
by “sliding” its window over the image
• Regional texture measurements can be obtained from
aggregation of over a region
• For instance, integration can be used to aggregate
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m) m
µ 2 RP
gn(f(x), m) M
M
m
gn(f(x), m
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
M
original
image d
increasingly
small
operator size
L1
 L2
 · · ·  LN
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
µ1
1
...
µ1
P
µ2
1
...
µ2
P
...
µN
1
...
µN
P
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
= µM: concatenated measurements from

multiscale operators

aggregated over
Ln
1 ⇥ · · · ⇥ Ln
d
gn=1,...,N (f(x), m)
M
f(x)
M
[Depeursinge2012b]
MULTISCALE TEXTURE OPERATORS
27
• Which scales for texture measurements?
• Two aspects:
A. How to define the size(s) of the operator(s) ?
B. How to define the size of the region of interest ?
TEXTURE OPERATORS AND PRIMITIVES
• From texture operators to texture measurements
• The operator is typically applied to all positions
by “sliding” its window over the image
• Regional texture measurements can be obtained from
aggregation of over a region
• For instance, integration can be used to aggregate
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m) m
µ 2 RP
gn(f(x), m) M
M
m
gn(f(x), m
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
M
original
image d
increasingly
small
operator size
L1
 L2
 · · ·  LN
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
µ1
1
...
µ1
P
µ2
1
...
µ2
P
...
µN
1
...
µN
P
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
= µM: concatenated measurements from

multiscale operators

aggregated over
Ln
1 ⇥ · · · ⇥ Ln
d
gn=1,...,N (f(x), m)
M
f(x)
M
[Depeursinge2012b]
MULTISCALE TEXTURE OPERATORS
28
• Size and spectral coverage of operators
• Uncertainty principle: operators cannot be well located both in 

space and Fourier [Petrou2006]
• In 2D, the trade-off between the spatial support (i.e., ) and
frequency support (i.e., ) of the operators is given by
L1 ⇥ L2
⌦1 ⇥ ⌦2
F
!
ˆf(!)f(x)
L2
1 ⌦2
1 L2
2 ⌦2
2
1
16
MULTISCALE TEXTURE OPERATORS
29
• Size and spectral coverage of operators
• Becomes a problem in the case of non-stationary texture:
,accurate spatial
localization
poor spectrum
characterization
Gaussian
window: 

= 3.2mm
0 ⇡|!1|
F
!
MULTISCALE TEXTURE OPERATORS
30
• Size and spectral coverage of operators
• Becomes a problem in the case of non-stationary texture:
,accurate spatial
localization
poor spectrum
characterization
Gaussian
window: 

= 3.2mm
Gaussian
window: 

= 38.4mm
0 ⇡|!1|
F
!
MULTISCALE TEXTURE OPERATORS
31
• Size and spectral coverage of operators
• Becomes a problem in the case of non-stationary texture:
,accurate spatial
localization
poor spectrum
characterization
Gaussian
window: 

= 3.2mm
Gaussian
window: 

= 38.4mm
0 ⇡|!1|
F
!
The spatial support should have the minimum size that allows 

rich enough texture-specific spectral characterization
MULTISCALE TEXTURE OPERATORS
32
• Other consequence:
• Large influence of proximal objects 

when the support of operators is 

larger than the region of interest
• Example with band-limited operators (2D isotropic wavelets) 

and lung boundary [Ward2015, Depeursinge2015a]
• Tuning the shape/bandwidth was found to have a strong influence on lung tissue
classification accuracy
M
0
0.05
0.1
0.15
0.2
0.25
0.3
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
0
5
10
15
20
25
30
35
0
0.05
0.1
0.15
0.2
0
0.5
1
1.5
2
0 ⇡|!1|0 0 ⇡|!1||!1| ⇡x1 x1 x1
ˆf(!)f(x) f(x) f(x)ˆf(!) ˆf(!)
COMPUTERIZED TEXTURE ANALYSIS
31
• Other consequence:
• Large influence of proximal objects 

when the support of operators is 

larger than the region of interest:
• Example with band-limited operators (2D isotropic wavelets) 

and lung boundary [DPC2015,WPU2015]
• Tuning the shape/bandwidth was found to have a strong influence on lung tissue
classification accuracy
L1
L2
M1
M2
·
M
m
M
0
0.05
0.1
0.15
0.2
0.25
0.3
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
0
5
10
15
20
25
30
35
0
0.05
0.1
0.15
0.2
0
0.5
1
1.5
2
0 ⇡|!1|0 0 ⇡|!1||!1| ⇡x1 x1 x1
ˆf(!)f(x) f(x) f(x)ˆf(!) ˆf(!)
better spatial localization
worse spectral localization
TEXTURE OPERATORS AND PRIMITIVES
• From texture operators to texture measurements
• The operator is typically applied to all positions
by “sliding” its window over the image
• Regional texture measurements can be obtained from
aggregation of over a region of interest
• For instance, integration can be used to aggregate
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m) m
µ 2 RP
gn(f(x), m) M
m
gn(f(x), m
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
MULTISCALE TEXTURE OPERATORS
33
• Which scales for texture measurements?
• Two aspects:
A. How to define the size(s) of the operator(s) ?
B. How to define the size of the region of interest ?M
original
image d
increasingly
small
operator size
L1
 L2
 · · ·  LN
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
µ1
1
...
µ1
P
µ2
1
...
µ2
P
...
µN
1
...
µN
P
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
= µM: concatenated measurements from

multiscale operators

aggregated over
Ln
1 ⇥ · · · ⇥ Ln
d
gn=1,...,N (f(x), m)
M
f(x)
M
M
[Depeursinge2012b]
MULTISCALE TEXTURE OPERATORS
• How large must be the region of interest ?
• No more than enough to evaluate texture stationarity 

in terms of human perception / tissue biology
• Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets
[Portilla2000] applied to all image positions
• Operators’ responses are averaged over
M
• The operator is typically applied to all position
by “sliding” its window over the image
• Regional texture measurements can be obtained
aggregation of over a region of interest
• For instance, integration can be used to aggregate
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m)
µ 2 RP
gn(f(x), m) M
m
gn(f(
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
M
f(x) g1(f(x), m) g2(f(x), m)
original image with
regions I
1
|M|
Z
M
|g1(f(x), m)|dm
M
feature space
1
|M|
Z
M
|g2(f(x),m)|dm
f(x)
Ma, Mb, Mc
The averaged responses
over the entire image
does not correspond 

to anything visually!
ˆg1(⇢) =
⇢
cos ⇡
2 log2
2⇢
⇡ , ⇡
4  ⇢  ⇡
0, otherwise.
ˆg2(⇢) =
⇢
cos ⇡
2 log2
4⇢
⇡ , ⇡
8  ⇢  ⇡
2
0, otherwise.
g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, )
m 2 RM1⇥M2
MULTISCALE TEXTURE OPERATORS
• How large must be the region of interest ?
• No more than enough to evaluate texture stationarity 

in terms of human perception / tissue biology
• Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets
[Portilla2000] applied to all image positions
• Operators’ responses are averaged over
M
• The operator is typically applied to all position
by “sliding” its window over the image
• Regional texture measurements can be obtained
aggregation of over a region of interest
• For instance, integration can be used to aggregate
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m)
µ 2 RP
gn(f(x), m) M
m
gn(f(
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
M
f(x) g1(f(x), m)
m 2 RM1⇥M2
g2(f(x), m)
original image with
regions I
1
|M|
Z
M
|g1(f(x), m)|dm
M
feature space
1
|M|
Z
M
|g2(f(x),m)|dm
f(x)
Ma, Mb, Mc
The averaged responses
over the entire image
does not correspond 

to anything visually!
ˆg1(⇢) =
⇢
cos ⇡
2 log2
2⇢
⇡ , ⇡
4  ⇢  ⇡
0, otherwise.
ˆg2(⇢) =
⇢
cos ⇡
2 log2
4⇢
⇡ , ⇡
8  ⇢  ⇡
2
0, otherwise.
g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, )
Nor biologically!
MULTISCALE TEXTURE OPERATORS
• How large must be the region of interest ?
• No more than enough to evaluate texture stationarity 

in terms of human perception / tissue biology
• Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets
[Portilla2000] applied to all image positions
• Operators’ responses are averaged over
M
• The operator is typically applied to all position
by “sliding” its window over the image
• Regional texture measurements can be obtained
aggregation of over a region of interest
• For instance, integration can be used to aggregate
• e.g., average:
L1
L2
M1
M2
L1 ⇥ · · · ⇥ Ld
·
gn(x, m)
µ 2 RP
gn(f(x), m) M
m
gn(f(
µ =
0
B
@
µ1
...
µP
1
C
A =
1
|M|
Z
M
gn(f(x), m) p=1,...,P
dm
M
f(x) g1(f(x), m)
m 2 RM1⇥M2
g2(f(x), m)
original image with
regions I
1
|M|
Z
M
|g1(f(x), m)|dm
M
feature space
1
|M|
Z
M
|g2(f(x),m)|dm
f(x)
Ma, Mb, Mc
The averaged responses
over the entire image
does not correspond 

to anything visually!
ˆg1(⇢) =
⇢
cos ⇡
2 log2
2⇢
⇡ , ⇡
4  ⇢  ⇡
0, otherwise.
ˆg2(⇢) =
⇢
cos ⇡
2 log2
4⇢
⇡ , ⇡
8  ⇢  ⇡
2
0, otherwise.
g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, )
Nor biologically!
Define regions that are homogeneous 

in terms of operators’ responses


(e.g., pixelwise clustering, graph cuts [Malik2001], 

Pott’s model [Storath2014])
MULTISCALE TEXTURE OPERATORS
37
• How large must be the region of interest ?
• Example: supervised texture segmentation with undecimated isotropic
Simoncelli’s wavelets and linear support vector machines (SVM)
M
feature space (training)
predicted labels
1
|M|
Z
M
|g1(f(x), m)|dm
1
|M|
Z
M
|g2(f(x),m)|dm
decision values
-10
-8
-6
-4
-2
0
2
4
6
8
train class 1 (128x128) test image (256x256)train class 2 (128x128)
segmentation error=0.05127
MULTISCALE TEXTURE OPERATORS
38
• How large must be the region of interest ?
• Example: supervised texture segmentation with undecimated isotropic
Simoncelli’s wavelets and linear support vector machines (SVM)
M
predicted labels
decision values
-10
-8
-6
-4
-2
0
2
4
6
8
train class 1 (128x128) test image (256x256)train class 2 (128x128)
segmentation error=0.05127
patch radius
0 20 40 60 80 100 120
segmentationerror
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
: circular patch with 
M
r = 30
: circular patch with 

MULTISCALE TEXTURE OPERATORS
39
• How large must be the region of interest ?
• Example: supervised texture segmentation with undecimated isotropic
Simoncelli’s wavelets and linear support vector machines (SVM)
M
patch radius
0 20 40 60 80 100 120
segmentationerror
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
train class 1 (128x128) train class 2 (128x128) test image (256x256)
r = 8
decision values
-2
-1
0
1
2
3
predicted labels
segmentation error=0.23853
M
MULTISCALE TEXTURE OPERATORS
40
• How large must be the region of interest ?
• Example: supervised texture segmentation with undecimated isotropic
Simoncelli’s wavelets and linear support vector machines (SVM)
M
patch radius
0 20 40 60 80 100 120
segmentationerror
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
: circular patch with 

r = 128
test image (256x256)train class 1 (128x128) train class 2 (128x128) predicted labels
decision values
-1.5
-1
-0.5
0
0.5
1
segmentation error=0.22839
M
MULTISCALE TEXTURE OPERATORS
41
• How large must be the region of interest ?
• Tissue properties are not homogeneous (i.e., non-stationary) over 

the entire organ
• Importance of building tissue atlases and digital phenotypes [Depeursinge2015b]
M
FIGURE 3. The 36 subregions of the lungs localized the prototype regional distributions of the texture properties. Figure 3 can be viewed online in color
Depeursinge et al Investigative Radiology • Volume 00, Number 00, Month 2015
M1
M2
= µM1,...,S
OUTLINE
• Biomedical texture analysis: background
• Defining texture processes
• Notations, sampling and texture functions
• Texture operators, primitives and invariances
• Multiscale analysis
• Operator scale and uncertainty principle
• Region of interest and operator aggregation
• Multidirectional analysis
• Isotropic versus directional operators
• Importance of the local organization of image directions
• Conclusions
• References
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f (x
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gra
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in t
representation corresponds to one realization (i.e., full image)
· m
43
• Which directions for texture measurements?
• Isotropic operators: insensitive to image directions
• Linear:
• Non-linear: e.g., median filter
• Directional operators
• Linear:
Other: e.g., Fourier, circular and spherical harmonics [Unser2013, Ward2014], MR8 [Varma2005], HOG [Dalal2005], 

Simoncelli’s pyramid [Simoncelli1995], curvelets [Candes2000]
• Non-linear:
Other: e.g., RLE [Galloway1975]
2D Gaussian filter 2D isotropic wavelets [Portilla2000]
MULTIDIRECTIONAL TEXTURE OPERATORS
2D Gaussian derivatives (e.g., Riesz wavelets [Unser2011])
@
@x1
@
@x2
@2
@x2
2
@2
@x2
1
@2
@x1@x2
d = 2 ( k1 = 2, k2 = 0)
2D GLCMs
d = 2
p
2 ( k1 = 2, k2 = 2)
2D LBPs [Ojala2002]
spaced pixels on a circle of radius R …R  0† that form a
circularly symmetric neighbor set.
T % t…s…g0 À gc†; s…g1 À gc†; . . . ; s…gPÀ1 À gc††; …5†
where
OJALA ET AL.: MULTIRESOLUTION GRAY-SCALE AND ROTATION INVARIANT TEXTURE CLASSIFICATION WITH LOCAL BINARY PATTERNS 973
Fig. 1. Circularly symmetric neighbor sets for different (P; R).
spaced pixels on a circle of radius R …R  0† that form a
circularly symmetric neighbor set.
T % t…s…g0 À gc†; s…g1 À gc†; . . . ; s…gPÀ1 À gc††; …5†
where
OJALA ET AL.: MULTIRESOLUTION GRAY-SCALE AND ROTATION INVARIANT TEXTURE CLASSIFICATION WITH LOCAL BINARY PATTERNS 973
Fig. 1. Circularly symmetric neighbor sets for different (P; R).
[?] Texture in Biomedical Images, Petrou M.,
L1
L2
M1
M2
Mass-
in a
able
om-
ures.
tion
rom
use
ping
iven
cific
nt of
the
tally
for
of1(x) : of2(x) :
R
X
Ø
Ø @f (x)
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image grad
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x)
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in th
representation corresponds to one realization (i.e., full image)
· m
gn(r, ✓, m) 7! gn(r, m)x
MULTIDIRECTIONAL TEXTURE OPERATORS
44
• Which directions for texture measurements?
• Is directional information important for texture discrimination?
F
!
ˆf(!)f(x)
F
!
ˆf(!)f(x)
MULTIDIRECTIONAL TEXTURE OPERATORS
45
• Which directions for texture measurements?
• Importance of the local organization of image directions (LOID)
• i.e., how directional structures intersect
MULTIDIRECTIONAL TEXTURE OPERATORS
46
• Which directions for texture measurements?
• Isotropic and unidirectional operators can hardly characterize the LOIDs,
especially when aggregated over a region [Sifre2014, Depeursinge2014b]
• Example of feature representation when integrated over entire image
M
isotropic Simoncelli wavelets
scale 1
scale2
o o
GLCM contrast
GLCMcontrast
d = 1 ( k1 = 1, k2 = 0)
d=1(k1=0,k2=1)
GLCMsgradients along and
1
|M|
Z
M
✓
@f(x)
@x1
◆2
dx
1
|M|
Z
M
✓
@f(x)
@x2
◆2
dx
x1 x2
M ⌘
L1
L2
M1
M2
·
M
m L1
L2
M1
M2
·
M
m
MULTIDIRECTIONAL TEXTURE OPERATORS
47
• Which directions for texture measurements?
• Isotropic and unidirectional operators can hardly characterize the LOIDs,
especially when aggregated over a region [Sifre2014, Depeursinge2014b]
• Example of feature representation when integrated over entire image
M
isotropic Simoncelli wavelets
scale 1
scale2
o o
GLCM contrast
GLCMcontrast
d = 1 ( k1 = 1, k2 = 0)
d=1(k1=0,k2=1)
GLCMsgradients along and
1
|M|
Z
M
✓
@f(x)
@x1
◆2
dx
1
|M|
Z
M
✓
@f(x)
@x2
◆2
dx
x1 x2
M ⌘
L1
L2
M1
M2
·
M
m L1
L2
M1
M2
·
M
m
Very poor discrimination! 

Solutions proposed in a few slides…
MULTIDIRECTIONAL TEXTURE OPERATORS
48
• Locally rotation-invariant operators over
• Isotropic operators:
• By definition
• Directional:
• Averaging operators’ responses

over all directions:
2D GLCMs
¯µ1 (e.g., GLCM contrast)
No characterization of
image directions!
µ
⇡/2
1µ
⇡/4
1 µ
3⇡/4
1
µ0
1
L1 ⇥ L2
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
REFERENCES
[?] Texture in Biomedical Images, Petrou M.,
L1
L2
M1
M2
M
MULTIDIRECTIONAL TEXTURE OPERATORS
49
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• MR8 filterbank [Varma2005]
• Rotation-invariant LBP [Ojala2002, Ahonen2009]
• Steerable Riesz wavelets [Unser2013, Depeursinge2014b]
L1 ⇥ L2
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
50
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Maximum response 8 (MR8) filterbank [Varma2005]
• Filter responses are obtained for each pixel from the convolution of the filter and the image
• For each position , only the maximum responses among 

gradient and Laplacian filters are kept
isotropic mutliscale oriented gradients multiscale oriented Laplacians
m
L1 ⇥ L2
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
51
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Maximum response 8 (MR8) filterbank [Varma2005]
• Filter responses are obtained for each pixel from the convolution of the filter and the image
• For each position , only the maximum responses among 

gradient and Laplacian filters are kept
isotropic mutliscale oriented gradients multiscale oriented Laplacians
m
L1 ⇥ L2
Yields approximate local rotation invariance
Poor characterization of the LOIDs
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
52
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Local binary patterns (LBP) [Ojala2002]
• Rotation-invariant LBP [Ahonen2009]
L1 ⇥ L2
1) define a circular neighborhood 2) Binarize and build a number
that encode the LOIDs
)
3) Aggregate over the entire image
and count code occurrences
0 170
⌘
4) make codes invariant to circular shifts
U8(1, 0) = 10101010 = 170
U8(1, 0) = 10101010
U8(1, 1) = 01010101
m
rotation r
discrete Fourier transform
The new measures
are independent of the rotation
) r
H8(1, u) =
7X
r=0
hI(U8(1, r))e j2⇡ur/8
µp = |H8(1, u)|
µ0,p = hI (U8(1, 0))
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
53
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Local binary patterns (LBP) [Ojala2002]
• Rotation-invariant LBP [Ahonen2009]
L1 ⇥ L2
1) define a circular neighborhood 2) Binarize and build a number
that encode the LOIDs
)
3) Aggregate over the entire image
and count code occurrences
0 170
⌘
4) make codes invariant to circular shifts
U8(1, 0) = 10101010 = 170
U8(1, 0) = 10101010
U8(1, 1) = 01010101
m
rotation r
discrete Fourier transform
The new measures
are independent of the rotation
) r
H8(1, u) =
7X
r=0
hI(U8(1, r))e j2⇡ur/8
µp = |H8(1, u)|
µ0,p = hI (U8(1, 0))
Encodes the LOIDs independently 

from their local orientations!
Requires binarization…
Spherical sequences are undefined in 3D…
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
54
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Steerable Riesz wavelets [Unser2013, Depeursinge2014b]
• Operators: th-order multi-scale image derivatives
L1 ⇥ L2
input imageN4 A. Depeursinge et al.
N = 1 N = 2
N = 3
Fig. 1. Templates corresponding to the Riesz kernels convolved with a Gaussian
smoother for N=1,2,3.
N = 1
g(1,0)(x, m)
f(x)
g(1,0)(f(x), m)
g(0,1)(x, m)
g(0,1)(f(x), m)
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
55
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Steerable Riesz wavelets [Unser2013, Depeursinge2014b]
• Operators: th-order multi-scale image derivatives
L1 ⇥ L2
N4 A. Depeursinge et al.
N = 1 N = 2
N = 3
Fig. 1. Templates corresponding to the Riesz kernels convolved with a Gaussian
smoother for N=1,2,3.
N = 1 N = 2
N = 3
g(1,0)(x, m) g(2,0)(x, m)g(0,1)(x, m) g(0,2)(x, m)
g(0,3)(x, m)
g(1,1)(x, m)
g(3,0)(x, m) g(2,1)(x, m) g(1,2)(x, m)
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
56
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Steerable Riesz wavelets [Unser2013, Depeursinge2014b]
• Steerability:
L1 ⇥ L2
g(1,0)(R✓0 x, 0) = cos ✓0 g(1,0)(x, 0) + sin ✓0 g(0,1)(x, 0)
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
57
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Steerable Riesz wavelets [Unser2013, Depeursinge2014b]
• Local rotation-invariance:
L1 ⇥ L2
✓max(m) := arg max
✓02[0,2⇡)
✓
cos ✓0 g(1,0)(f(x), m) + sin ✓0 g(0,1)(f(x), m)
◆
µM =
1
|M|
Z
M
✓
g(1,0)(f(R✓max(m) x), m)
◆2
dm)
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
MULTIDIRECTIONAL TEXTURE OPERATORS
58
• Locally rotation-invariant operators over
• Locally “aligning” directional operators
• Steerable Riesz wavelets [Depeursinge2014b, Unser2013]
• Local rotation-invariance:
L1 ⇥ L2
✓max(m) := arg max
✓02[0,2⇡)
✓
cos ✓0 g(1,0)(f(x), m) + sin ✓0 g(0,1)(f(x), m)
◆
µM =
1
|M|
Z
M
✓
g(1,0)(f(R✓max(m) x), m)
◆2
dm)
Encodes the LOIDs independently 

from their local orientations!
No binarization required!
Available in 3D [Chenouard2012, Depeursinge2015a],
and combined with feature learning [Depeursinge2014b].
g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
• Operators characterizing the LOIDs
MULTIDIRECTIONAL TEXTURE OPERATORS
GLCMs Riesz wavelets ( )
59
GLCM contrast
GLCMcontrast
d = 1 ( k1 = 1, k2 = 0)
d=1(k1=0,k2=1)
N = 2
1
|M|
Z
M
✓
g(2,0)(f(x),m)
◆2
dm
1
|M|
Z
M
✓
g(0,2)(f(x), m)
◆2
dm
aligned Riesz
wavelets ( )N = 2
1
|M|
Z
M
✓
g(0,2)(f(R✓max(m) x), m)
◆2
dm
1
|M|
Z
M
✓
g(2,0)(f(R✓max(m)x),m)
◆2
dm
• Operators characterizing the LOIDs
MULTIDIRECTIONAL TEXTURE OPERATORS
60
GLCMs
GLCM contrast
GLCMcontrast
d = 1 ( k1 = 1, k2 = 0)
d=1(k1=0,k2=1)
Riesz wavelets ( )
1
|M|
Z
M
✓
g(2,0)(f(x),m)
◆2
dm
1
|M|
Z
M
✓
g(0,2)(f(x), m)
◆2
dm
N = 2
aligned Riesz
wavelets ( )N = 2
1
|M|
Z
M
✓
g(0,2)(f(R✓max(m) x), m)
◆2
dm
1
|M|
Z
M
✓
g(2,0)(f(R✓max(m)x),m)
◆2
dm
MULTIDIRECTIONAL TEXTURE OPERATORS
61
• Isotropic or directional analysis? [Depeursinge2014b]
• Outex [Ojala2002]: 24 classes, 180 images/class, 9 rotation angles in
• Texture classification: linear SVMs trained with unrotated images only
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5
1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021
9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035
17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009
Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database.
1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin
9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool
Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite.
180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 ,
135 and 150 of the other seven Brodatz images for each
class. The total number of images in the test set is 672.
G. Experimental setup
OVA SVM models using Gaussian kernels as K(xi, xj) =
Inc., 2012. The computational complexity is dominated by the
local orientation of N
c in Eq. 11, which consists of finding the
roots of the polynomials defined by the steering matrix A✓
.
It is therefore NP–hard (Non–deterministic Polynomial–time
hard), where the order of the polynomials is controlled by the
order of the Riesz transform N.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1 2 3 4 5 6 7 8 9 10
ini/alcoeffs
aligned1sttemplate
order of the Riesz transformN
classificationaccuracy
Riesz wavelets
aligned Riesz wavelets
MULTIDIRECTIONAL TEXTURE OPERATORS
62
• Isotropic or directional analysis? [Depeursinge2014b]
• Outex [Ojala2002]: 24 classes, 180 images/class, 9 rotation angles in
• Texture classification: linear SVMs trained with unrotated images only
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5
1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021
9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035
17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009
Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database.
1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin
9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool
Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite.
180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 ,
135 and 150 of the other seven Brodatz images for each
class. The total number of images in the test set is 672.
G. Experimental setup
OVA SVM models using Gaussian kernels as K(xi, xj) =
Inc., 2012. The computational complexity is dominated by the
local orientation of N
c in Eq. 11, which consists of finding the
roots of the polynomials defined by the steering matrix A✓
.
It is therefore NP–hard (Non–deterministic Polynomial–time
hard), where the order of the polynomials is controlled by the
order of the Riesz transform N.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1 2 3 4 5 6 7 8 9 10
ini/alcoeffs
aligned1sttemplate
order of the Riesz transformN
classificationaccuracy
Riesz wavelets
aligned Riesz wavelets
Isotropic operators (i.e., ) perform best 

when not aligned!
N = 0
OUTLINE
• Biomedical texture analysis: background
• Defining texture processes
• Notations, sampling and texture functions
• Texture operators, primitives and invariances
• Multiscale analysis
• Operator scale and uncertainty principle
• Region of interest and operator aggregation
• Multidirectional analysis
• Isotropic versus directional operators
• Importance of the local organization of image directions
• Conclusions
• References
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f (x
@x1
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gra
Fig. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f (x
@x2
able to discriminate between the two textures classes f1(x)
when integrated over the image domain X . One circle in t
representation corresponds to one realization (i.e., full image)
· m
CONCLUSIONS
• We presented a general framework to describe 

and analyse texture information in 2D and 3D
• Tissue structures in 2D/3D medical images contain
extremely rich and valuable information to optimize
personalized medicine in a non-invasive way
• Invisible to the naked eye!
64
L1
L2
M1
M2
M
of1(x) : of2(x) :
R
X
Ø
Ø @f
@x
R
X
Ø
Ø@f(x)
@x2
Ø
Ødx
image gr
. 1. The joint responses of image gradients
≥
|
@f (x)
@x1
|,|
@f
@x
e to discriminate between the two textures classes f1(x
en integrated over the image domain X . One circle in
· m
malignant, nonresponder
malignant, responder
benign
pre-malignant
undefined
quant. feat. #1
quant.feat.#2
MULTISCALE TEXTURE OPERATORS
• How large must be the region of interest ?
• Enough to evaluate texture stationarity in terms 

of human perception / tissue biology
• Example with operator: undecimated isotropic Simoncelli’s dyadi
[PoS2000] applied to all image positions
• Operators’ responses are averaged over
M
TEXTURE OPERATORS AND P
• From texture operators to
• The operator is typ
by “sliding” its window
• Regional texture measuremen
aggregation of
• For instance, integration can b
• e.g., average:
M1
L1 ⇥ ·
gn(x, m)
gn(f(x), m)
µ =
0
B
@
µ1
...
µP
1
C
A =
|Mf(x) g1(f(x), m)
m 2 RM1⇥M2
g2
original image with
regions I
1
|M|
Z
M
|g1(f(x), m)|dm
M
feature space
1
|M|
Z
M
|g2(f(x),m)|dm
f(x)
Ma, Mb, Mc
The average
over the en
does not c
to anythin
ˆg1(⇢) =
⇢
cos ⇡
2 log2
2⇢
⇡ , ⇡
4  ⇢  ⇡
0, otherwise.
ˆg2(⇢) =
⇢
cos ⇡
2 log2
4⇢
⇡ , ⇡
8  ⇢  ⇡
2
0, otherwise.
g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, )
Nor biolo
BoVW can be used to first reveal the
intra-class visual diversity
texture operators
region of interest

and aggregation
scales
uncertainty principle averaging operators’ responses
directions
isotropic versus directional importance of LOIDs
CONCLUSIONS
• Biomedical textures are realizations of complex 

non-stationary spatial stochastic processes
• General-purpose image operators are necessary to
identify data-specific discriminative scales and directions
65
MULTISCALE TEXTURE OPERATORS
• How large must be the region of interest ?
• Enough to evaluate texture stationarity in terms 

of human perception / tissue biology
• Example with operator: undecimated isotropic Simoncelli’s dyad
[PoS2000] applied to all image positions
• Operators’ responses are averaged over
M
TEXTURE OPERATORS AND P
• From texture operators to
• The operator is ty
by “sliding” its window
• Regional texture measureme
aggregation of
• For instance, integration can
• e.g., average:
M1
L1 ⇥
gn(x, m)
gn(f(x), m)
µ =
0
B
@
µ1
...
µP
1
C
A =
f(x) g1(f(x), m)
m 2 RM1⇥M2
g
original image with
regions I
1
|M|
Z
M
|g1(f(x), m)|dm
M
feature space
1
|M|
Z
M
|g2(f(x),m)|dm
f(x)
Ma, Mb, Mc
The average
over the e
does not c
to anythin
ˆg1(⇢) =
⇢
cos ⇡
2 log2
2⇢
⇡ , ⇡
4  ⇢  ⇡
0, otherwise.
ˆg2(⇢) =
⇢
cos ⇡
2 log2
4⇢
⇡ , ⇡
8  ⇢  ⇡
2
0, otherwise.
g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, )
Nor bio
BoVW can be used to first reveal the
intra-class visual diversity
F
!
ˆf(!)f(x)
)TextureQbased'biomarkers:'current'limitaGons'
x  Assume'homogeneous'texture'properGes'over'the'
enGre'lesion'[5]'
'
x  NonQspecific'features'
x  Global'vs'local'characterizaGon'of'image'direcGons'[6]'
REVIEW: Quantitative Imaging in Cancer Evolution and Ecology Gatenby et al
with the mean signal value. By using just
two sequences, a contrast-enhanced T1
sequence and a fluid-attenuated inver-
sion-recovery sequence, we can define
four habitats: high or low postgadolini-
um T1 divided into high or low fluid-at-
tenuated inversion recovery. When these
voxel habitats are projected into the tu-
mor volume, we find they cluster into
spatially distinct regions. These habitats
can be evaluated both in terms of their
relative contributions to the total tumor
volume and in terms of their interactions
with each other, based on the imaging
characteristics at the interfaces between
regions. Similar spatially explicit analysis
can be performed with CT scans (Fig 5).
Analysis of spatial patterns in
cross-sectional images will ultimately re-
quire methods that bridge spatial scales
from microns to millimeters. One possi-
ble method is a general class of numeric
tools that is already widely used in ter-
restrial and marine ecology research to
link species occurrence or abundance
with environmental parameters. Species
distribution models (48–51) are used to
gain ecologic and evolutionary insights
and to predict distributions of species or
morphs across landscapes, sometimes
extrapolating in space and time. They
can easily be used to link the environ-
mental selection forces in MR imaging-
defined habitats to the evolutionary dy-
namics of cancer cells.
Summary
Imaging can have an enormous role in
the development and implementation of
rise to local-regional phenotypic adap-
tations. Phenotypic alterations can re-
sult from epigenetic, genetic, or chro-
mosomal rearrangements, and these in
turn will affect prognosis and response
to therapy. Changes in habitats or the
relative abundance of specific ecologic
communities over time and in response
to therapy may be a valuable metric with
which to measure treatment efficacy and
emergence of resistant populations.
microenvironment can be rewarded by
increased proliferation. This evolution-
ary dynamic may contribute to distinct
differences between the tumor edges
and the tumor cores, which frequently
can be seen at analysis of cross-sec-
tional images (Fig 5).
Interpretation of the subsegmenta-
tion of tumors will require computa-
tional models to understand and predict
the complex nonlinear dynamics that
lead to heterogeneous combinations
of radiographic features. We have ex-
Figure 4
Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome
Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial
distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low
fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with
low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions.
REVIEW: Quantitative Imaging in Cancer Evolution and Ecology Gatenby et a
with the mean signal value. By using jus
two sequences, a contrast-enhanced T1
sequence and a fluid-attenuated inver
sion-recovery sequence, we can define
four habitats: high or low postgadolini
um T1 divided into high or low fluid-at
tenuated inversion recovery. When these
voxel habitats are projected into the tu
mor volume, we find they cluster into
spatially distinct regions. These habitat
can be evaluated both in terms of thei
relative contributions to the total tumo
volume and in terms of their interaction
with each other, based on the imaging
characteristics at the interfaces between
regions. Similar spatially explicit analysi
can be performed with CT scans (Fig 5)
Analysis of spatial patterns in
cross-sectional images will ultimately re
quire methods that bridge spatial scale
from microns to millimeters. One possi
ble method is a general class of numeric
tools that is already widely used in ter
restrial and marine ecology research to
link species occurrence or abundance
with environmental parameters. Specie
distribution models (48–51) are used to
gain ecologic and evolutionary insight
and to predict distributions of species o
morphs across landscapes, sometime
extrapolating in space and time. They
can easily be used to link the environ
mental selection forces in MR imaging
defined habitats to the evolutionary dy
namics of cancer cells.
Summary
Imaging can have an enormous role in
the development and implementation o
patient-specific therapies in cancer. The
rise to local-regional phenotypic adap-
tations. Phenotypic alterations can re-
sult from epigenetic, genetic, or chro-
mosomal rearrangements, and these in
turn will affect prognosis and response
to therapy. Changes in habitats or the
relative abundance of specific ecologic
communities over time and in response
to therapy may be a valuable metric with
which to measure treatment efficacy and
emergence of resistant populations.
Emerging Strategies for Tumor Habitat
microenvironment can be rewarded by
increased proliferation. This evolution-
ary dynamic may contribute to distinct
differences between the tumor edges
and the tumor cores, which frequently
can be seen at analysis of cross-sec-
tional images (Fig 5).
Interpretation of the subsegmenta-
tion of tumors will require computa-
tional models to understand and predict
the complex nonlinear dynamics that
lead to heterogeneous combinations
of radiographic features. We have ex-
ploited ecologic methods and models to
Figure 4
Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome
Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial
distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low
fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with
low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions.
[5]'QuanGtaGve'imaging'in'cancer'evoluGon'and'ecology,'Gatenby'et'al.,'Radiology,'269(1):8Q15,'2013'
5'
global'direcGonal'operators:' local'grouped'steering:'
[6]'RotaGonQcovariant'texture'learning'using'steerable'Riesz'wavelets,'Depeursinge'et'al.,'IEEE'Trans'Imag'Proc.,'23(2):898Q908,'2014.'
TextureQbased'biomarkers:'current'limita
x  Assume'homogeneous'texture'properGes'over't
enGre'lesion'[5]'
'
x  NonQspecific'features'
x  Global'vs'local'characterizaGon'of'image'direcGo
REVIEW: Quantitative Imaging in Cancer Evolution and Ecology
with th
two se
sequen
sion-re
four h
um T1
tenuat
voxel
mor v
spatia
can be
relativ
volum
with e
charac
region
can be
An
cross-
quire
from m
ble me
tools t
restria
link s
with e
distrib
gain e
and to
morph
extrap
can ea
menta
define
namic
Summ
Imagin
the de
patien
achiev
metho
place
assess
The n
been c
Cance
mation
work.
consor
ducibl
extrac
rise to local-regional phenotypic adap-
tations. Phenotypic alterations can re-
sult from epigenetic, genetic, or chro-
mosomal rearrangements, and these in
turn will affect prognosis and response
to therapy. Changes in habitats or the
relative abundance of specific ecologic
communities over time and in response
to therapy may be a valuable metric with
which to measure treatment efficacy and
emergence of resistant populations.
Emerging Strategies for Tumor Habitat
Characterization
A method for converting images to spa-
tially explicit tumor habitats is shown in
Figure 4. Here, three-dimensional MR
imaging data sets from a glioblastoma
are segmented. Each voxel in the tumor
is defined by a scale that includes its
image intensity in different sequences.
In this case, the imaging sets are from
(a) a contrast-enhanced T1 sequence,
(b) a fast spin-echo T2 sequence, and
(c) a fluid-attenuated inversion-recov-
microenvironment can be rewarded by
increased proliferation. This evolution-
ary dynamic may contribute to distinct
differences between the tumor edges
and the tumor cores, which frequently
can be seen at analysis of cross-sec-
tional images (Fig 5).
Interpretation of the subsegmenta-
tion of tumors will require computa-
tional models to understand and predict
the complex nonlinear dynamics that
lead to heterogeneous combinations
of radiographic features. We have ex-
ploited ecologic methods and models to
investigate regional variations in cancer
environmental and cellular properties
that lead to specific imaging character-
istics. Conceptually, this approach as-
sumes that regional variations in tumors
can be viewed as a coalition of distinct
ecologic communities or habitats of cells
in which the environment is governed,
at least to first order, by variations in
vascular density and blood flow. The
environmental conditions that result
from alterations in blood flow, such as
Figure 4
Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome
Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial
distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low
fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with
low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions.
REVIEW: Quantitative Imaging in Cancer Evolution and Ecology
rise to local-re
tations. Pheno
sult from epig
mosomal rearr
turn will affect
to therapy. Ch
relative abunda
communities ov
to therapy may
which to measu
emergence of r
Emerging Stra
Characterizati
A method for c
tially explicit tu
Figure 4. Here
imaging data s
are segmented.
is defined by
image intensity
In this case, th
microenvironment can be rewarded by
increased proliferation. This evolution-
ary dynamic may contribute to distinct
differences between the tumor edges
and the tumor cores, which frequently
can be seen at analysis of cross-sec-
tional images (Fig 5).
Interpretation of the subsegmenta-
tion of tumors will require computa-
tional models to understand and predict
the complex nonlinear dynamics that
lead to heterogeneous combinations
of radiographic features. We have ex-
ploited ecologic methods and models to
investigate regional variations in cancer
environmental and cellular properties
that lead to specific imaging character-
istics. Conceptually, this approach as-
sumes that regional variations in tumors
can be viewed as a coalition of distinct
ecologic communities or habitats of cells
in which the environment is governed,
at least to first order, by variations in
Figure 4
Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034
Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma mu
distribution of MR imaging–defined habitats within the tumor. The blue regio
fluid-attenuated inversion recovery) is particularly notable because it presum
low blood flow but high cell density, indicating a population presumably adap
[5]'QuanGtaGve'imaging'in'cancer'evoluGon'and'ecology,'Gatenby'et'al.,'Radiology,'269(1):8Q15,'2013'
5'
global'direcGonal'operators:' local'groupe
[6]'RotaGonQcovariant'texture'learning'using'steerable'Riesz'wavelets,'Depeursinge'et'al.,'IEEE'Trans'Imag'Proc.,'23(2):898
MULTIDIRECTIONAL TEXTURE OPERATORS
58
• Isotropic or directional analysis? [DFV2014]
• Outex [OPM2002]: 24 classes, 180 images/class, 9 rotation angles in
• Texture classification: linear SVMs trained with unrotated images only
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5
1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021
9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035
17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009
Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database.
1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin
9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool
Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite.
180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 ,
135 and 150 of the other seven Brodatz images for each
class. The total number of images in the test set is 672.
G. Experimental setup
OVA SVM models using Gaussian kernels as K(xi, xj) =
||xi xj ||2
Inc., 2012. The computational complexity is dominated by the
local orientation of N
c in Eq. 11, which consists of finding the
roots of the polynomials defined by the steering matrix A✓
.
It is therefore NP–hard (Non–deterministic Polynomial–time
hard), where the order of the polynomials is controlled by the
order of the Riesz transform N.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1 2 3 4 5 6 7 8 9 10
ini/alcoeffs
aligned1sttemplate
order of the Riesz transformN
classificationaccuracy
Riesz wavelets
aligned Riesz wavelets
Isotropic operators (i.e., ) perform best 

when not aligned!
N = 0
ULTIDIRECTIONAL TEXTURE OPERATORS
58
• Isotropic or directional analysis? [DFV2014]
• Outex [OPM2002]: 24 classes, 180 images/class, 9 rotation angles in
• Texture classification: linear SVMs trained with unrotated images only
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5
1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021
9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035
17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009
Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database.
1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin
9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool
Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite.
180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 ,
135 and 150 of the other seven Brodatz images for each
class. The total number of images in the test set is 672.
G. Experimental setup
OVA SVM models using Gaussian kernels as K(xi, xj) =
exp(
||xi xj ||2
2 2
k
) are used both to learn texture signatures and
to classify the texture instances in the final feature space
obtained after k iterations. A number of scales J = 6
Inc., 2012. The computational complexity is dominated by the
local orientation of N
c in Eq. 11, which consists of finding the
roots of the polynomials defined by the steering matrix A✓
.
It is therefore NP–hard (Non–deterministic Polynomial–time
hard), where the order of the polynomials is controlled by the
order of the Riesz transform N.
III. RESULTS
The performance of our approach is demonstrated with
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1 2 3 4 5 6 7 8 9 10
ini/alcoeffs
aligned1sttemplate
order of the Riesz transformN
classificationaccuracy
Riesz wavelets
aligned Riesz wavelets
Isotropic operators (i.e., ) perform best 

when not aligned!
N = 0
THANKS !
BIG @ EPFL

Michael Unser

Julien Fageot

Arash Amini

and all members
MedGIFT @ HES-SO

Henning Müller

Yashin Dicente

Roger Schaer

Ranveer Joyseree

Oscar Jimenez

Manfredo Atzori
66
Source code and data available!
https://sites.google.com/site/btamiccai2015/
adrien.depeursinge@epfl.ch
PROCESSING FOR BIOMEDICAL IMAGE AN
Adrien Depeursinge, PhD
MICCAI 2015 Tutorial on Biomedical Texture Analysis (BTA), Munich, O
09), where
create real-
os can also
s problem
ic texture’’
1999), and
ature pub-
id textures
on the fea-
since only
sis.
tals of 3-D
ction 3 de-
etrieve pa-
trieval. The
ure are re-
us expecta-
pplication-
d together
various ap-
Section 7,
h a general
1981), it is
del of tex-
m (Mallat,
ches based
using sets of prototype primitives. The concept of texture primitive
is naturally extended in 3-D as the geometry of the voxel sequence
used by a given texture analysis method. We consider a primitive
C(i,j,k) centered at a point (i,j,k) that lives on a neighborhood of
this point. The primitive is constituted by a set of voxels with gray
tone values that forms a 3-D structure. Typical C neighborhoods
are voxel pairs, linear, planar, spherical or unconstrained. Signal
assignment to the primitive can be either binary, categorical or
continuous. Two example texture primitives are shown in Fig. 4.
Texture primitives refer to local processing of 3-D images and local
patterns (see Toriwaki and Yoshida, 2009).
Fig. 2. 3-D digitized images and sampling in Cartesian coordinates.
Fig. 3. 3-D digitized images and sampling in spherical coordinates.
2 [iDx,(i + 1)Dx]; [jDy,(j + 1)Dy]; [kDz,(k +
Yoshida, 2009). This cuboid is called a voxel.
coordinates (r,h,/) are unevenly sampled to
Fig. 3.
ure primitive has been widely used in 2-D tex-
nes the elementary building block of a given
k, 1979; Jain et al., 1995; Lin et al., 1999). All
pproaches aim at modeling a given texture
e primitives. The concept of texture primitive
in 3-D as the geometry of the voxel sequence
ure analysis method. We consider a primitive
point (i,j,k) that lives on a neighborhood of
ive is constituted by a set of voxels with gray
ms a 3-D structure. Typical C neighborhoods
r, planar, spherical or unconstrained. Signal
rimitive can be either binary, categorical or
mple texture primitives are shown in Fig. 4.
fer to local processing of 3-D images and local
ki and Yoshida, 2009).
d images and sampling in Cartesian coordinates.
plexin3-D.
varioustaxonomiesare
Aclarificationofthetax-
ratelydefinethescopeof
andYoshida,2009.Three-
rearebothgeneraland
edinR3
andinclude:
in‘‘filled’’objects
neratedbyavolumetric
raphy,confocalimaging).
sof‘‘hollow’’objectsas
ionaltimesequencesas
accountsfortexturesde-
oordinates.Solidtextures
meansthatanumberof
ftheEuclideanspaceis
1965;Foncubierta-Rodrí-
nedastexturedsurfacein
na(2004),or2.5-dimen-
guetetal.(2008),where
-Dobjectsandcanbein-
(2)isalsousedinKajiya
andHaindl(2009),where
eofobjectstocreatereal-
nalysisinvideoscanalso
xtureanalysisproblem
edby‘‘dynamictexture’’
andCrowley(1999),and
ewoftheliteraturepub-
biomedicalsolidtextures
ofthistextisonthefea-
gtechniques,sinceonly
textureanalysis.
Thefundamentalsof3-D
Section2.Section3de-
ystematicallyretrievepa-
ficationandretrieval.The
intheliteraturearere-
olistthevariousexpecta-
Theresultingapplication-
edandgroupedtogether
dgapsofthevariousap-
aregiveninSection7,
andtheirassumedinteractionsdefinethepropertiesofthetexture
analysisapproaches,fromstatisticaltostructuralmethods.
InSection2.1,wedefinethemathematicalframeworkand
notationsconsideredtodescribethecontentof3-Ddigitalimages.
Thenotionoftextureprimitivesaswellastheirscalesanddirec-
tionsaredefinedinSection2.2.
2.1.3-Ddigitizedimagesandsampling
InCartesiancoordinates,ageneric3-Dcontinuousimageisde-
finedbyafunctionofthreevariablesf(x,y,z),wherefrepresentsa
scalaratapointðx;y;zÞ2R3
.A3-DdigitalimageF(i,j,k)ofdimen-
sionsMÂNÂOisobtainedfromsamplingfatpointsði;j;kÞ2Z3
ofa3-Dorderedarray(seeFig.2).Incrementsin(i,j,k),correspond
tophysicaldisplacementsinR3
parametrizedbytherespective
spacings(Dx,Dy,Dz).Foreverycellofthedigitizedarray,thevalue
ofF(i,j,k)istypicallyobtainedbyaveragingfinthecuboiddomain
definedby(x,y,z)2[iDx,(i+1)Dx];[jDy,(j+1)Dy];[kDz,(k+
1)Dz])(ToriwakiandYoshida,2009).Thiscuboidiscalledavoxel.
Thethreesphericalcoordinates(r,h,/)areunevenlysampledto
(R,H,U)asshowninFig.3.
2.2.Textureprimitives
Thenotionoftextureprimitivehasbeenwidelyusedin2-Dtex-
tureanalysisanddefinestheelementarybuildingblockofagiven
textureclass(Haralick,1979;Jainetal.,1995;Linetal.,1999).All
textureprocessingapproachesaimatmodelingagiventexture
usingsetsofprototypeprimitives.Theconceptoftextureprimitive
isnaturallyextendedin3-Dasthegeometryofthevoxelsequence
usedbyagiventextureanalysismethod.Weconsideraprimitive
C(i,j,k)centeredatapoint(i,j,k)thatlivesonaneighborhoodof
thispoint.Theprimitiveisconstitutedbyasetofvoxelswithgray
tonevaluesthatformsa3-Dstructure.TypicalCneighborhoods
arevoxelpairs,linear,planar,sphericalorunconstrained.Signal
assignmenttotheprimitivecanbeeitherbinary,categoricalor
continuous.TwoexampletextureprimitivesareshowninFig.4.
Textureprimitivesrefertolocalprocessingof3-Dimagesandlocal
patterns(seeToriwakiandYoshida,2009).
Fig.2.3-DdigitizedimagesandsamplinginCartesiancoordinates.
Stanford University

Daniel Rubin

Olivier Gevaert

Ann Leung
Dimitri Van de Ville, UNIGE

Camille Kurtz, Paris Descartes

Pierre-Alexandre Poletti, HUG

John-Paul Ward, UCF
REFERENCES (SORTING IN ALPHABETICAL ORDER)
67
[Ahonen2009] Ahonen, T.; Matas, J.; He, C.  Pietikäinen, M.

Salberg, A.-B.; Hardeberg, J.  Jenssen, R. (Eds.)

Rotation Invariant Image Description with Local Binary Pattern Histogram Fourier Features

Image Analysis, Springer Berlin Heidelberg, 2009, 5575, 61-70
[Aerts2014] Aerts, H. J. W. L.; Velazquez, E. R.; Leijenaar, R. T. H.; Parmar, C.; Grossmann, P.; Carvalho, S.; Bussink, J.;
Monshouwer, R.; Haibe-Kains, B.; Rietveld, D.; Hoebers, F.; Rietbergen, M. M.; Leemans, C. R.; Dekker, A.;
Quackenbush, J.; Gillies, R. J.  Lambin, P.

Decoding Tumour Phenotype by Noninvasive Imaging Using a Quantitative Radiomics Approach

Nature Communications, Nature Publishing Group, a division of Macmillan Publishers Limited. All Rights
Reserved, 2014, 5
[Blakemore1969] Blakemore, C.  Campbell, F. W.

On the Existence of Neurones in the Human Visual System Selectively Sensitive to the Orientation and Size
of Retinal Images

The Journal of Physiology, 1969, 203, 237-260
[Candes2000] Candès, E. J.  Donoho, D. L.

Curvelets - A Surprisingly Effective Nonadaptive Representation For Objects with Edges

Curves and Surface Fitting, Vanderbilt University Press, 2000, 105-120
[Chenouard2011] Chenouard, N.  Unser, M.

3D Steerable Wavelets and Monogenic Analysis for Bioimaging

2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2011, 2132-2135
[Chenouard2012] Chenouard, N.  Unser, M.

3D Steerable Wavelets in Practice

IEEE Transactions on Image Processing, 2012, 21, 4522-4533
REFERENCES (SORTING IN ALPHABETICAL ORDER)
68
[Dalal2005] Dalal, N.  Triggs, B.

Histograms of Oriented Gradients for Human Detection

Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(CVPR'05), IEEE Computer Society, 2005, 1, 886-893
[Depeursinge2012a] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D.  Müller, H.

Multiscale Lung Texture Signature Learning Using The Riesz Transform

Medical Image Computing and Computer-Assisted Intervention MICCAI 2012, Springer Berlin /
Heidelberg, 2012, 7512, 517-524
[Depeursinge2012b] Depeursinge, A.; Van De Ville, D.; Platon, A.; Geissbuhler, A.; Poletti, P.-A.  Müller, H.

Near-Affine-Invariant Texture Learning for Lung Tissue Analysis Using Isotropic Wavelet Frames

IEEE Transactions on Information Technology in BioMedicine, 2012, 16, 665-675
[Depeursinge2014a] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D.  Müller, H.

Three-Dimensional Solid Texture Analysis and Retrieval in Biomedical Imaging: Review and Opportunities

Medical Image Analysis, 2014, 18, 176-196
[Depeursinge2014b] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D.  Müller, H.

Rotation-Covariant Texture Learning Using Steerable Riesz Wavelets

IEEE Transactions on Image Processing, 2014, 23, 898-908
[Depeursinge2015a] Depeursinge, A.; Pad, P.; Chin, A. C.; Leung, A. N.; Rubin, D. L.; Müller, H.  Unser, M.

Optimized Steerable Wavelets for Texture Analysis of Lung Tissue in 3-D CT: Classification of Usual
Interstitial Pneumonia

IEEE 12th International Symposium on Biomedical Imaging, IEEE, 2015, 403-406
[Depeursinge2015b] Depeursinge, A.; Chin, A. C.; Leung, A. N.; Terrone, D.; Bristow, M.; Rosen, G.  Rubin, D. L.

Automated Classification of Usual Interstitial Pneumonia Using Regional Volumetric Texture Analysis in
High-Resolution CT

Investigative Radiology, 2015, 50, 261-267
REFERENCES (SORTING IN ALPHABETICAL ORDER)
69
[Dumas2009] Dumas, A.; Brigitte, M.; Moreau, M.; Chrétien, F.; Baslé, M.  Chappard, D.

Bone Mass and Microarchitecture of Irradiated and Bone Marrow-Transplanted Mice: Influences of the
Donor Strain

Osteoporosis International, Springer-Verlag, 2009, 20, 435-443
[Galloway1975] Galloway, M. M.

Texture Analysis Using Gray Level Run Lengths

Computer Graphics and Image Processing, 1975, 4, 172-179
(Gal1975)
[Gerlinger2012] Gerlinger, M.; Rowan, A. J.; Horswell, S.; Larkin, J.; Endesfelder, D.; Gronroos, E.; Martinez, P.; Matthews,
N.; Stewart, A.; Tarpey, P.; Varela, I.; Phillimore, B.; Begum, S.; McDonald, N. Q.; Butler, A.; Jones, D.;
Raine, K.; Latimer, C.; Santos, C. R.; Nohadani, M.; Eklund, A. C.; Spencer-Dene, B.; Clark, G.; Pickering,
L.; Stamp, G.; Gore, M.; Szallasi, Z.; Downward, J.; Futreal, P. A.  Swanton, C.

Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing

New England Journal of Medicine, 2012, 366, 883-892
[Gurcan2009] Gurcan, M. N.; Boucheron, L. E.; Can, A.; Madabhushi, A.; Rajpoot, N. M.  Yener, B.

Histopathological Image Analysis: A Review

IEEE Reviews in Biomedical Engineering, 2009, 2, 147-171
[Haidekker2011] Haidekker, M. A.

Texture Analysis

Advanced Biomedical Image Analysis, John Wiley  Sons, Inc., 2010, 236-275
[Haralick1979] Haralick, R. M.

Statistical and Structural Approaches to Texture

Proceedings of the IEEE, 1979, 67, 786-804
REFERENCES (SORTING IN ALPHABETICAL ORDER)
70
[Khan2013] Khan, A. M.; El-Daly, H.; Simmons, E.  Rajpoot, N. M.

HyMaP: A Hybrid Magnitude-Phase Approach to Unsupervised Segmentation of Tumor Areas in Breast Cancer
Histology Images

Journal of pathology informatics, Medknow Publications, 2013, 4
[Kumar2012] Kumar, V.; Gu, Y.; Basu, S.; Berglund, A.; Eschrich, S. A.; Schabath, M. B.; Forster, K.; Aerts, H. J. W. L.; Dekker,
A.; Fenstermacher, D.; Goldgof, D. B.; Hall, L. O.; Lambin, P.; Balagurunathan, Y.; Gatenby, R. A.  Gillies, R. J.

Radiomics: The Process and the Challenges

Magnetic Resonance Imaging, 2012, 30, 1234-1248
[Lazebnik2005] Lazebnik, S.; Schmid, C.  Ponce, J.

A Sparse Texture Representation Using Local Affine Regions

IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, 2005, 27, 1265-1278
[Malik2001] Malik, J.; Belongie, S.; Leung, T.  Shi, J.

Contour and Texture Analysis for Image Segmentation

International Journal of Computer Vision, Kluwer Academic Publishers, 2001, 43, 7-27
[Mosquera2015] Mosquera-Lopez, C.; Agaian, S.; Velez-Hoyos, A.  Thompson, I.

Computer-Aided Prostate Cancer Diagnosis From Digitized Histopathology: A Review on Texture-Based
Systems

IEEE Reviews in Biomedical Engineering, 2015, 8, 98-113
[Ojala2002] Ojala, T.; Pietikäinen, M.  Mäenpää, T.

Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24, 971-987
REFERENCES (SORTING IN ALPHABETICAL ORDER)
71
[Petrou2006] Petrou, M.  Garcia Sevilla, P.

Image Processing: Dealing with Texture

Wiley, 2006
[Petrou2011] Petrou, M.

Deserno, T. M. (Ed.)

Texture in Biomedical Images

Biomedical Image Processing, Springer-Verlag Berlin Heidelberg, 2011, 157-176
[Portilla2000] Portilla, J.  Simoncelli, E. P.

A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients

International Journal of Computer Vision, Kluwer Academic Publishers, 2000, 40, 49-70
[Romeny2011] ter Haar Romeny, B. M.

Deserno, T. M. (Ed.)

Multi-Scale and Multi-Orientation Medical Image Analysis

Biomedical Image Processing, Springer-Verlag Berlin Heidelberg, 2011, 177-196
[Sifre2014] Sifre, L.  Mallat, S.

Rigid-Motion Scattering for Texture Classification

Submitted to International Journal of Computer Vision, 2014, abs/1403.1687, 1-19
[Simoncelli1995] Simoncelli, E. P.  Freeman, W. T.

The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation

Proceedings of International Conference on Image Processing, 1995, 3, 444-447
[Storath2014] Storath, M.; Weinmann, A.  Unser, M.

Unsupervised Texture Segmentation Using Monogenic Curvelets and the Potts Model

IEEE International Conference on Image Processing (ICIP), 2014, 4348-4352
REFERENCES (SORTING IN ALPHABETICAL ORDER)
72
[Unser2011] Unser, M.; Chenouard, N.  Van De Ville, D.

Steerable Pyramids and Tight Wavelet Frames in 

IEEE Transactions on Image Processing, 2011, 20, 2705-2721
[Unser2013] Unser, M.  Chenouard, N.

A Unifying Parametric Framework for 2D Steerable Wavelet Transforms

SIAM Journal on Imaging Sciences, 2013, 6, 102-135
[Varma2005] Varma, M.  Zisserman, A.

A Statistical Approach to Texture Classification from Single Images

International Journal of Computer Vision, Kluwer Academic Publishers, 2005, 62, 61-81
[Ward2014] Ward, J.  Unser, M.

Harmonic Singular Integrals and Steerable Wavelets in

Applied and Computational Harmonic Analysis, 2014, 36, 183-197
[Ward2015] Ward, J.; Pad, P.  Unser, M.

Optimal Isotropic Wavelets for Localized Tight Frame Representations

IEEE Signal Processing Letters, 2015, 22, 1918-1921
[Xu2012] Xu, J.; Napel, S.; Greenspan, H.; Beaulieu, C. F.; Agrawal, N.  Rubin, D.

Quantifying the Margin Sharpness of Lesions on Radiological Images for Content-Based Image Retrieval

Medical Physics, 2012, 39, 5405-5418
L2(Rd
)
L2(Rd
)

More Related Content

What's hot

Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...
Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...
Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...
Arif S
 
magnetic resonance angiography
magnetic resonance angiographymagnetic resonance angiography
magnetic resonance angiography
qavi786
 

What's hot (20)

Diffusion Tensor Imaging (2011-10-04 이정원)
Diffusion Tensor Imaging (2011-10-04 이정원)Diffusion Tensor Imaging (2011-10-04 이정원)
Diffusion Tensor Imaging (2011-10-04 이정원)
 
Diagnostic Imaging of Orbital Lesions
Diagnostic Imaging of Orbital LesionsDiagnostic Imaging of Orbital Lesions
Diagnostic Imaging of Orbital Lesions
 
Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...
Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...
Diffusion-weighted and Perfusion MR Imaging for Brain Tumor Characterization ...
 
magnetic resonance angiography
magnetic resonance angiographymagnetic resonance angiography
magnetic resonance angiography
 
CT perfusion physics and its application in Neuroimaging
CT perfusion physics and its application in NeuroimagingCT perfusion physics and its application in Neuroimaging
CT perfusion physics and its application in Neuroimaging
 
Mra
MraMra
Mra
 
Post Processing of CT Thorax
Post Processing of CT ThoraxPost Processing of CT Thorax
Post Processing of CT Thorax
 
Tissue harmonic imaging
Tissue harmonic imaging Tissue harmonic imaging
Tissue harmonic imaging
 
TARGET DELINEATION OF PARAMETRIUM
TARGET DELINEATION OF PARAMETRIUMTARGET DELINEATION OF PARAMETRIUM
TARGET DELINEATION OF PARAMETRIUM
 
Mammography
MammographyMammography
Mammography
 
Ultrasound Elastography
Ultrasound Elastography Ultrasound Elastography
Ultrasound Elastography
 
Breast elastography
Breast elastographyBreast elastography
Breast elastography
 
Fat suppression techniques in MRI
Fat suppression techniques in MRIFat suppression techniques in MRI
Fat suppression techniques in MRI
 
MRI Artifact Radiology
MRI Artifact RadiologyMRI Artifact Radiology
MRI Artifact Radiology
 
Anatomy and imaging of wrist joint (MRI AND XRAY)
Anatomy and imaging of wrist joint (MRI AND XRAY)Anatomy and imaging of wrist joint (MRI AND XRAY)
Anatomy and imaging of wrist joint (MRI AND XRAY)
 
Elastography
ElastographyElastography
Elastography
 
MR ANGIOGRAPHY PHYSICS
MR ANGIOGRAPHY PHYSICSMR ANGIOGRAPHY PHYSICS
MR ANGIOGRAPHY PHYSICS
 
Imaging in breast cancer
Imaging in breast cancerImaging in breast cancer
Imaging in breast cancer
 
Usg artefacts
Usg artefacts Usg artefacts
Usg artefacts
 
CT Angiography Lower Limb
CT Angiography Lower LimbCT Angiography Lower Limb
CT Angiography Lower Limb
 

Viewers also liked

"Development Methods for Intelligent Systems"
"Development Methods for Intelligent Systems""Development Methods for Intelligent Systems"
"Development Methods for Intelligent Systems"
butest
 
lec21.ppt
lec21.pptlec21.ppt
lec21.ppt
butest
 
Introduction to Biomedical Imaging
Introduction to Biomedical ImagingIntroduction to Biomedical Imaging
Introduction to Biomedical Imaging
u.surgery
 
Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...
Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...
Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...
Cirdan
 
Radioterapi of lung cancer
Radioterapi of lung cancerRadioterapi of lung cancer
Radioterapi of lung cancer
Mulkan Fadhli
 
Introduction to Multi Sensory Taste Perception
Introduction to Multi Sensory Taste PerceptionIntroduction to Multi Sensory Taste Perception
Introduction to Multi Sensory Taste Perception
Jozef_a
 

Viewers also liked (20)

A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
A 3-D Riesz-Covariance Texture Model for the Prediction of Nodule Recurrence ...
 
"Development Methods for Intelligent Systems"
"Development Methods for Intelligent Systems""Development Methods for Intelligent Systems"
"Development Methods for Intelligent Systems"
 
lec21.ppt
lec21.pptlec21.ppt
lec21.ppt
 
Credit seminar 1
Credit seminar 1Credit seminar 1
Credit seminar 1
 
Biomedical Imaging In Vivo 2009
Biomedical Imaging In Vivo 2009Biomedical Imaging In Vivo 2009
Biomedical Imaging In Vivo 2009
 
Medical image analysis, retrieval and evaluation infrastructures
Medical image analysis, retrieval and evaluation infrastructuresMedical image analysis, retrieval and evaluation infrastructures
Medical image analysis, retrieval and evaluation infrastructures
 
Computer aided sensory evaluation
Computer aided sensory evaluationComputer aided sensory evaluation
Computer aided sensory evaluation
 
Predicting NSCLC prognosis by automated pathology
Predicting NSCLC prognosis by automated pathologyPredicting NSCLC prognosis by automated pathology
Predicting NSCLC prognosis by automated pathology
 
Biomedical Imaging Informatics
Biomedical Imaging InformaticsBiomedical Imaging Informatics
Biomedical Imaging Informatics
 
Introduction to Biomedical Imaging
Introduction to Biomedical ImagingIntroduction to Biomedical Imaging
Introduction to Biomedical Imaging
 
Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...
Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...
Peter Hamilton on Next generation Imaging and Computer Vision in Pathology: p...
 
Computer aided detection of pulmonary nodules using genetic programming
Computer aided detection of pulmonary nodules using genetic programmingComputer aided detection of pulmonary nodules using genetic programming
Computer aided detection of pulmonary nodules using genetic programming
 
Radioterapi of lung cancer
Radioterapi of lung cancerRadioterapi of lung cancer
Radioterapi of lung cancer
 
L.T.D second seminar
L.T.D second seminarL.T.D second seminar
L.T.D second seminar
 
폐 CT영상에서 voxel classification을 이용한 폐 결절 검출
폐 CT영상에서 voxel classification을 이용한 폐 결절 검출폐 CT영상에서 voxel classification을 이용한 폐 결절 검출
폐 CT영상에서 voxel classification을 이용한 폐 결절 검출
 
Introduction to Multi Sensory Taste Perception
Introduction to Multi Sensory Taste PerceptionIntroduction to Multi Sensory Taste Perception
Introduction to Multi Sensory Taste Perception
 
Surface Texture: The Perception
Surface Texture: The PerceptionSurface Texture: The Perception
Surface Texture: The Perception
 
Introduction of image processing
Introduction of image processingIntroduction of image processing
Introduction of image processing
 
computer aided detection of pulmonary nodules in ct scans
computer aided detection of pulmonary nodules in ct scanscomputer aided detection of pulmonary nodules in ct scans
computer aided detection of pulmonary nodules in ct scans
 
Automatic Detection of Non-Proliferative Diabetic Retinopathy Using Fundus Im...
Automatic Detection of Non-Proliferative Diabetic Retinopathy Using Fundus Im...Automatic Detection of Non-Proliferative Diabetic Retinopathy Using Fundus Im...
Automatic Detection of Non-Proliferative Diabetic Retinopathy Using Fundus Im...
 

Similar to FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS

4 image segmentation through clustering
4 image segmentation through clustering4 image segmentation through clustering
4 image segmentation through clustering
IAEME Publication
 
4 image segmentation through clustering
4 image segmentation through clustering4 image segmentation through clustering
4 image segmentation through clustering
prjpublications
 

Similar to FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS (20)

DOMAIN SPECIFIC CBIR FOR HIGHLY TEXTURED IMAGES
DOMAIN SPECIFIC CBIR FOR HIGHLY TEXTURED IMAGESDOMAIN SPECIFIC CBIR FOR HIGHLY TEXTURED IMAGES
DOMAIN SPECIFIC CBIR FOR HIGHLY TEXTURED IMAGES
 
GREY LEVEL CO-OCCURRENCE MATRICES: GENERALISATION AND SOME NEW FEATURES
GREY LEVEL CO-OCCURRENCE MATRICES: GENERALISATION AND SOME NEW FEATURESGREY LEVEL CO-OCCURRENCE MATRICES: GENERALISATION AND SOME NEW FEATURES
GREY LEVEL CO-OCCURRENCE MATRICES: GENERALISATION AND SOME NEW FEATURES
 
A comparative study on content based image retrieval methods
A comparative study on content based image retrieval methodsA comparative study on content based image retrieval methods
A comparative study on content based image retrieval methods
 
Computed Tomography Image Reconstruction in 3D VoxelSpace
Computed Tomography Image Reconstruction in 3D VoxelSpaceComputed Tomography Image Reconstruction in 3D VoxelSpace
Computed Tomography Image Reconstruction in 3D VoxelSpace
 
A New Approach for Segmentation of Fused Images using Cluster based Thresholding
A New Approach for Segmentation of Fused Images using Cluster based ThresholdingA New Approach for Segmentation of Fused Images using Cluster based Thresholding
A New Approach for Segmentation of Fused Images using Cluster based Thresholding
 
4 image segmentation through clustering
4 image segmentation through clustering4 image segmentation through clustering
4 image segmentation through clustering
 
4 image segmentation through clustering
4 image segmentation through clustering4 image segmentation through clustering
4 image segmentation through clustering
 
IJET-V2I6P17
IJET-V2I6P17IJET-V2I6P17
IJET-V2I6P17
 
Textural Feature Extraction of Natural Objects for Image Classification
Textural Feature Extraction of Natural Objects for Image ClassificationTextural Feature Extraction of Natural Objects for Image Classification
Textural Feature Extraction of Natural Objects for Image Classification
 
IMAGE RETRIEVAL USING QUADRATIC DISTANCE BASED ON COLOR FEATURE AND PYRAMID S...
IMAGE RETRIEVAL USING QUADRATIC DISTANCE BASED ON COLOR FEATURE AND PYRAMID S...IMAGE RETRIEVAL USING QUADRATIC DISTANCE BASED ON COLOR FEATURE AND PYRAMID S...
IMAGE RETRIEVAL USING QUADRATIC DISTANCE BASED ON COLOR FEATURE AND PYRAMID S...
 
E018212935
E018212935E018212935
E018212935
 
Poster(3)-1
Poster(3)-1Poster(3)-1
Poster(3)-1
 
Texture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
Texture Unit based Approach to Discriminate Manmade Scenes from Natural ScenesTexture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
Texture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
 
Person re-identification, PhD Day 2011
Person re-identification, PhD Day 2011Person re-identification, PhD Day 2011
Person re-identification, PhD Day 2011
 
Texture features from Chaos Game Representation Images of Genomes
Texture features from Chaos Game Representation Images of GenomesTexture features from Chaos Game Representation Images of Genomes
Texture features from Chaos Game Representation Images of Genomes
 
DIGITAL TOPOLOGY OPERATING IN MEDICAL IMAGING WITH MRI TECHNOLOGY.pptx
DIGITAL TOPOLOGY OPERATING IN MEDICAL IMAGING WITH MRI TECHNOLOGY.pptxDIGITAL TOPOLOGY OPERATING IN MEDICAL IMAGING WITH MRI TECHNOLOGY.pptx
DIGITAL TOPOLOGY OPERATING IN MEDICAL IMAGING WITH MRI TECHNOLOGY.pptx
 
Component-Based Ethnicity Identification from Facial Images.pdf
Component-Based Ethnicity Identification from Facial Images.pdfComponent-Based Ethnicity Identification from Facial Images.pdf
Component-Based Ethnicity Identification from Facial Images.pdf
 
Bloom Graph
Bloom GraphBloom Graph
Bloom Graph
 
A combined method of fractal and glcm features for mri and ct scan images cla...
A combined method of fractal and glcm features for mri and ct scan images cla...A combined method of fractal and glcm features for mri and ct scan images cla...
A combined method of fractal and glcm features for mri and ct scan images cla...
 
Prac ex'cises 3[1].5
Prac ex'cises 3[1].5Prac ex'cises 3[1].5
Prac ex'cises 3[1].5
 

More from Institute of Information Systems (HES-SO)

Solar production prediction based on non linear meteo source adaptation
Solar production prediction based on non linear meteo source adaptationSolar production prediction based on non linear meteo source adaptation
Solar production prediction based on non linear meteo source adaptation
Institute of Information Systems (HES-SO)
 

More from Institute of Information Systems (HES-SO) (20)

MIE20232.pptx
MIE20232.pptxMIE20232.pptx
MIE20232.pptx
 
Classification of noisy free-text prostate cancer pathology reports using nat...
Classification of noisy free-text prostate cancer pathology reports using nat...Classification of noisy free-text prostate cancer pathology reports using nat...
Classification of noisy free-text prostate cancer pathology reports using nat...
 
Machine learning assisted citation screening for Systematic Reviews - Anjani ...
Machine learning assisted citation screening for Systematic Reviews - Anjani ...Machine learning assisted citation screening for Systematic Reviews - Anjani ...
Machine learning assisted citation screening for Systematic Reviews - Anjani ...
 
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
 
L'IoT dans les usines. Quels avantages ?
L'IoT dans les usines. Quels avantages ?L'IoT dans les usines. Quels avantages ?
L'IoT dans les usines. Quels avantages ?
 
Studying Public Medical Images from Open Access Literature and Social Network...
Studying Public Medical Images from Open Access Literature and Social Network...Studying Public Medical Images from Open Access Literature and Social Network...
Studying Public Medical Images from Open Access Literature and Social Network...
 
Risques opérationnels et le système de contrôle interne : les limites d’un te...
Risques opérationnels et le système de contrôle interne : les limites d’un te...Risques opérationnels et le système de contrôle interne : les limites d’un te...
Risques opérationnels et le système de contrôle interne : les limites d’un te...
 
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
Le contrôle interne dans les administrations publiques tient-il toutes ses pr...
 
Le système de contrôle interne : Présentation générale, enjeux et méthodes
Le système de contrôle interne : Présentation générale, enjeux et méthodesLe système de contrôle interne : Présentation générale, enjeux et méthodes
Le système de contrôle interne : Présentation générale, enjeux et méthodes
 
Crowdsourcing-based Mobile Application for Wheelchair Accessibility
Crowdsourcing-based Mobile Application for Wheelchair AccessibilityCrowdsourcing-based Mobile Application for Wheelchair Accessibility
Crowdsourcing-based Mobile Application for Wheelchair Accessibility
 
Quelle(s) valeur(s) pour le leadership stratégique ?
Quelle(s) valeur(s) pour le leadership stratégique ?Quelle(s) valeur(s) pour le leadership stratégique ?
Quelle(s) valeur(s) pour le leadership stratégique ?
 
Challenges in medical imaging and the VISCERAL model
Challenges in medical imaging and the VISCERAL modelChallenges in medical imaging and the VISCERAL model
Challenges in medical imaging and the VISCERAL model
 
NOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
NOSE: une approche Smart-City pour les zones périphériques et extra-urbainesNOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
NOSE: une approche Smart-City pour les zones périphériques et extra-urbaines
 
Medical image analysis and big data evaluation infrastructures
Medical image analysis and big data evaluation infrastructuresMedical image analysis and big data evaluation infrastructures
Medical image analysis and big data evaluation infrastructures
 
How to detect soft falls on devices
How to detect soft falls on devicesHow to detect soft falls on devices
How to detect soft falls on devices
 
MOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLS
MOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLSMOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLS
MOBILE COLLECTION AND DISSEMINATION OF SENIORS’ SKILLS
 
Enhanced Students Laboratory The GET project
Enhanced Students Laboratory The GET projectEnhanced Students Laboratory The GET project
Enhanced Students Laboratory The GET project
 
Solar production prediction based on non linear meteo source adaptation
Solar production prediction based on non linear meteo source adaptationSolar production prediction based on non linear meteo source adaptation
Solar production prediction based on non linear meteo source adaptation
 
Exploring the New Trends of Chinese Tourists in Switzerland
Exploring the New Trends of Chinese Tourists in SwitzerlandExploring the New Trends of Chinese Tourists in Switzerland
Exploring the New Trends of Chinese Tourists in Switzerland
 
Social Media Data analyzis and Semantics for Tourism Understanding
Social Media Data analyzis and Semantics for Tourism UnderstandingSocial Media Data analyzis and Semantics for Tourism Understanding
Social Media Data analyzis and Semantics for Tourism Understanding
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Choreo: Empowering the Future of Enterprise Software Engineering
Choreo: Empowering the Future of Enterprise Software EngineeringChoreo: Empowering the Future of Enterprise Software Engineering
Choreo: Empowering the Future of Enterprise Software Engineering
 
Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern Enterprise
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational Performance
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 

FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS

  • 1. FUNDAMENTALS OF TEXTURE PROCESSING 
 FOR BIOMEDICAL IMAGE ANALYSIS Adrien Depeursinge, PhD MICCAI 2015 Tutorial on Biomedical Texture Analysis (BTA), Munich, Oct 5 2015 nts for textures de- nates. Solid textures s that a number of Euclidean space is Foncubierta-Rodrí- textured surface in 004), or 2.5-dimen- et al. (2008), where jects and can be in- also used in Kajiya aindl (2009), where bjects to create real- s in videos can also analysis problem ‘‘dynamic texture’’ rowley (1999), and the literature pub- dical solid textures is text is on the fea- hniques, since only re analysis. undamentals of 3-D on 2. Section 3 de- atically retrieve pa- n and retrieval. The e literature are re- he various expecta- sulting application- d grouped together s of the various ap- given in Section 7, establish a general Julesz, 1981), it is atical model of tex- is problem (Mallat, s approaches based defined by (x,y,z) 2 [iDx,(i + 1)Dx]; [jDy,(j + 1)Dy]; [kDz,(k + 1)Dz]) (Toriwaki and Yoshida, 2009). This cuboid is called a voxel. The three spherical coordinates (r,h,/) are unevenly sampled to (R,H,U) as shown in Fig. 3. 2.2. Texture primitives The notion of texture primitive has been widely used in 2-D tex- ture analysis and defines the elementary building block of a given texture class (Haralick, 1979; Jain et al., 1995; Lin et al., 1999). All texture processing approaches aim at modeling a given texture using sets of prototype primitives. The concept of texture primitive is naturally extended in 3-D as the geometry of the voxel sequence used by a given texture analysis method. We consider a primitive C(i,j,k) centered at a point (i,j,k) that lives on a neighborhood of this point. The primitive is constituted by a set of voxels with gray tone values that forms a 3-D structure. Typical C neighborhoods are voxel pairs, linear, planar, spherical or unconstrained. Signal assignment to the primitive can be either binary, categorical or continuous. Two example texture primitives are shown in Fig. 4. Texture primitives refer to local processing of 3-D images and local patterns (see Toriwaki and Yoshida, 2009). Fig. 2. 3-D digitized images and sampling in Cartesian coordinates. Fig. 3. 3-D digitized images and sampling in spherical coordinates. ized images and sampling an coordinates, a generic 3-D continuous image is de- nction of three variables f(x,y,z), where f represents a oint ðx; y; zÞ 2 R3 . A 3-D digital image F(i,j,k) of dimen- Â O is obtained from sampling f at points ði; j; kÞ 2 Z3 red array (see Fig. 2). Increments in (i,j,k), correspond displacements in R3 parametrized by the respective ,Dy,Dz). For every cell of the digitized array, the value ypically obtained by averaging f in the cuboid domain (x,y,z) 2 [iDx,(i + 1)Dx]; [jDy,(j + 1)Dy]; [kDz,(k + waki and Yoshida, 2009). This cuboid is called a voxel. herical coordinates (r,h,/) are unevenly sampled to hown in Fig. 3. primitives n of texture primitive has been widely used in 2-D tex- and defines the elementary building block of a given (Haralick, 1979; Jain et al., 1995; Lin et al., 1999). All essing approaches aim at modeling a given texture prototype primitives. The concept of texture primitive xtended in 3-D as the geometry of the voxel sequence ven texture analysis method. We consider a primitive ered at a point (i,j,k) that lives on a neighborhood of he primitive is constituted by a set of voxels with gray that forms a 3-D structure. Typical C neighborhoods irs, linear, planar, spherical or unconstrained. Signal to the primitive can be either binary, categorical or Two example texture primitives are shown in Fig. 4. itives refer to local processing of 3-D images and local Toriwaki and Yoshida, 2009). -D digitized images and sampling in Cartesian coordinates. lattices)isnotstraightforwardandraisesseveralchal- atedtotranslation,scalingandrotationinvariancesand esthatarebecomingmorecomplexin3-D. dworkandscopeofthissurvey dingonresearchcommunities,varioustaxonomiesare ferto3-Dtextureinformation.Aclarificationofthetax- proposedinthissectiontoaccuratelydefinethescopeof y.ItispartlybasedonToriwakiandYoshida,2009.Three- altextureandvolumetrictexturearebothgeneraland termsdesigningatexturedefinedinR3 andinclude: umetrictexturesexistingin‘‘filled’’objects :x;y;z2Vx;y;z&R3 gthataregeneratedbyavolumetric aacquisitiondevice(e.g.,tomography,confocalimaging). Dtexturesexistingonsurfacesof‘‘hollow’’objectsas :x;y;z2Cu;v&R3 g, namictexturesintwo–dimensionaltimesequencesas x;y;t2Sx;y;t&R3 g, exturereferstocategory(1)andaccountsfortexturesde- volumeVx,y,zindexedbythreecoordinates.Solidtextures ntrinsicdimensionof3,whichmeansthatanumberof equaltothedimensionalityoftheEuclideanspaceis representthesignal(Bennett,1965;Foncubierta-Rodrí- .,2013a).Category(2)isdesignedastexturedsurfacein Nayar(1999)andCulaandDana(2004),or2.5-dimen- turesinLuetal.(2006)andAguetetal.(2008),where Careexistingonthesurfaceof3-Dobjectsandcanbein- quelybytwocoordinates(u,v).(2)isalsousedinKajiya 1989),Neyret(1995),andFilipandHaindl(2009),where etriesareaddedontothesurfaceofobjectstocreatereal- eringofvirtualscenes.Motionanalysisinvideoscanalso deredamulti-dimensionaltextureanalysisproblem tocategory(3)andisdesignedby‘‘dynamictexture’’ myandFablet(1998),ChomatandCrowley(1999),and al.(2000). survey,acomprehensivereviewoftheliteraturepub- classificationandretrievalofbiomedicalsolidtextures ory(1))iscarriedout.Thefocusofthistextisonthefea- actionandnotmachinelearningtechniques,sinceonly xtractionisspecificto3-Dsolidtextureanalysis. ureofthisarticle urveyisstructuredasfollows:Thefundamentalsof3-D xtureprocessingaredefinedinSection2.Section3de- ereviewmethodologyusedtosystematicallyretrievepa- ngwith3-Dsolidtextureclassificationandretrieval.The modalitiesandorgansstudiedintheliteraturearere- Sections4and5,respectivelytolistthevariousexpecta- needsof3-Dimageprocessing.Theresultingapplication- chniquesaredescribed,organizedandgroupedtogether 6.Asynthesisofthetrendsandgapsofthevariousap- conclusionsandopportunitiesaregiveninSection7, ely. mentalsofsolidtextureprocessing ghseveralresearchersattemptedtoestablishageneral onthe3-Dgeometricalpropertiesoftheprimitivesused,i.e.,the elementarybuildingblockconsidered.Thesetofprimitivesused andtheirassumedinteractionsdefinethepropertiesofthetexture analysisapproaches,fromstatisticaltostructuralmethods. InSection2.1,wedefinethemathematicalframeworkand notationsconsideredtodescribethecontentof3-Ddigitalimages. Thenotionoftextureprimitivesaswellastheirscalesanddirec- tionsaredefinedinSection2.2. 2.1.3-Ddigitizedimagesandsampling InCartesiancoordinates,ageneric3-Dcontinuousimageisde- finedbyafunctionofthreevariablesf(x,y,z),wherefrepresentsa scalaratapointðx;y;zÞ2R3 .A3-DdigitalimageF(i,j,k)ofdimen- sionsMÂNÂOisobtainedfromsamplingfatpointsði;j;kÞ2Z3 ofa3-Dorderedarray(seeFig.2).Incrementsin(i,j,k),correspond tophysicaldisplacementsinR3 parametrizedbytherespective spacings(Dx,Dy,Dz).Foreverycellofthedigitizedarray,thevalue ofF(i,j,k)istypicallyobtainedbyaveragingfinthecuboiddomain definedby(x,y,z)2[iDx,(i+1)Dx];[jDy,(j+1)Dy];[kDz,(k+ 1)Dz])(ToriwakiandYoshida,2009).Thiscuboidiscalledavoxel. Thethreesphericalcoordinates(r,h,/)areunevenlysampledto (R,H,U)asshowninFig.3. 2.2.Textureprimitives Thenotionoftextureprimitivehasbeenwidelyusedin2-Dtex- tureanalysisanddefinestheelementarybuildingblockofagiven textureclass(Haralick,1979;Jainetal.,1995;Linetal.,1999).All textureprocessingapproachesaimatmodelingagiventexture usingsetsofprototypeprimitives.Theconceptoftextureprimitive isnaturallyextendedin3-Dasthegeometryofthevoxelsequence usedbyagiventextureanalysismethod.Weconsideraprimitive C(i,j,k)centeredatapoint(i,j,k)thatlivesonaneighborhoodof thispoint.Theprimitiveisconstitutedbyasetofvoxelswithgray tonevaluesthatformsa3-Dstructure.TypicalCneighborhoods arevoxelpairs,linear,planar,sphericalorunconstrained.Signal assignmenttotheprimitivecanbeeitherbinary,categoricalor continuous.TwoexampletextureprimitivesareshowninFig.4. Textureprimitivesrefertolocalprocessingof3-Dimagesandlocal patterns(seeToriwakiandYoshida,2009). Fig.2.3-DdigitizedimagesandsamplinginCartesiancoordinates. A.Depeursingeetal./MedicalImageAnalysis18(2014)176–196
  • 2. OUTLINE • Biomedical texture analysis: background • Defining texture processes • Notations, sampling and texture functions • Texture operators, primitives and invariances • Multiscale analysis • Operator scale and uncertainty principle • Region of interest and operator aggregation • Multidirectional analysis • Isotropic versus directional operators • Importance of the local organization of image directions • Conclusions • References L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f (x @x1 R X Ø Ø@f(x) @x2 Ø Ødx image gra Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in t representation corresponds to one realization (i.e., full image) · m
  • 3. BACKGROUND – RADIOMICS - HISTOPATHOLOMICS • Personalized medicine aims at enhancing the patient’s 
 quality of life and prognosis • Tailored treatment and medical decisions based 
 on the molecular composition of diseased tissue • Current limitations [Gerlinger2012] • Molecular analysis of tissue composition 
 is invasive (biopsy), slow and costly • Cannot capture molecular heterogeneity 3 Intratumor Heterogeneity Reveale B Regional Distribution of Mutations A Biopsy Sites SOX9 CENPN PSMD7 RIMBP2 GALNT11 ABHD11 UGT2A1 MTOR PPP6R2 ZNF780A WSCD2 CDKN1B PPFIA1 TH SSNA1 CASP2 PLRG1 SETD2 CCBL2 SESN2 MAGEB16 NLRP7 IGLON5 KLK4 WDR62 KIAA0355 CYP4F3 AKAP8 ZNF519 DDX52 ZC3H18 TCF12 NUSAP1 X4 KDM2B MRPL51 C11orf68 ANO5 EIF4G2 MSRB2 RALGDS EXT1 ZC3HC1 PTPRZ1 INTS1 CCR6 DOPEY1 ATXN1 WHSC1 CLCN2 SSR3 KLHL18 SGOL1 VHL C2orf21 ALS2CR12 PLB1 FCAMR IFI16 BCAS2 IL12RB2 Ubiquitous Shared prima 10 cm R7 (G4) R5 (G4) R9 R3 (G4) R1 (G3) R2 (G3) R4 (G1) R6 (G1) Hilum R8 (G4) ution of Mutations ationships of Tumor Regions D Ploidy Profiling PreP PreM R1 R2 R3 R5 R8 R9 R4 M1 M2a M2b C2orf85 WDR7 SUPT6H CDH19 LAMA3 DIXDC1 HPS5 NRAP KIAA1524 SETD2 PLCL1 BCL11A IFNAR1 DAMTS10 C3 KIAA1267 RT4 CD44 ANKRD26 TM7SF4 SLC2A1 DACH2 MMAB ZNF521 HMG20A DNMT3A RLF MAMLD1 MAP3K6 HDAC6 PHF21B FAM129B RPS8 CIB2 RAB27A SLC2A12 DUSP12 ADAMTSL4 NAP1L3 USP51 KDM5C SBF1 TOM1 MYH8 WDR24 ITIH5 AKAP9 FBXO1 LIAS TNIK SETD2 C3orf20 MR1 PIAS3 DIO1 ERCC5 KL ALKBH8 DAPK1 DDX58 SPATA21 ZNF493 NGEF DIRAS3 LATS2 ITGB3 FLNA SATL1 KDM5C KDM5C RBFOX2 NPHS1 SOX9 CENPN PSMD7 RIMBP2 GALNT11 ABHD11 UGT2A1 MTOR PPP6R2 ZNF780A WSCD2 CDKN1B PPFIA1 TH SSNA1 CASP2 PLRG1 SETD2 CCBL2 SESN2 MAGEB16 NLRP7 IGLON5 KLK4 WDR62 KIAA0355 CYP4F3 AKAP8 ZNF519 DDX52 ZC3H18 TCF12 NUSAP1 X4 KDM2B MRPL51 C11orf68 ANO5 EIF4G2 MSRB2 RALGDS EXT1 ZC3HC1 PTPRZ1 Privatebiquitous Shared primary Shared metastasis Lung metastases Chest-wall metastasis Perinephric metastasis M1 10 cm R2 (G3) R4 (G1) R6 (G1) HilumR8 (G4) Primary tumor M2b M2a ution of Mutations PreP PreM R1 R2 R3 R5 R8 R9 R4 M1 M2a M2b C2orf85 WDR7 SUPT6H CDH19 LAMA3 DIXDC1 HPS5 NRAP KIAA1524 SETD2 PLCL1 BCL11A IFNAR1 DAMTS10 C3 KIAA1267 RT4 CD44 ANKRD26 TM7SF4 SLC2A1 DACH2 MMAB ZNF521 HMG20A DNMT3A RLF MAMLD1 MAP3K6 HDAC6 PHF21B FAM129B RPS8 CIB2 RAB27A SLC2A12 DUSP12 ADAMTSL4 NAP1L3 USP51 KDM5C SBF1 TOM1 MYH8 WDR24 ITIH5 AKAP9 FBXO1 LIAS TNIK SETD2 C3orf20 MR1 PIAS3 DIO1 ERCC5 KL ALKBH8 DAPK1 DDX58 SPATA21 ZNF493 NGEF DIRAS3 LATS2 ITGB3 FLNA SATL1 KDM5C KDM5C RBFOX2 NPHS1 SOX9 CENPN PSMD7 RIMBP2 GALNT11 ABHD11 UGT2A1 MTOR PPP6R2 ZNF780A WSCD2 CDKN1B PPFIA1 TH SSNA1 CASP2 PLRG1 SETD2 CCBL2 SESN2 MAGEB16 NLRP7 IGLON5 KLK4 WDR62 KIAA0355 CYP4F3 AKAP8 ZNF519 DDX52 ZC3H18 TCF12 NUSAP1 X4 KDM2B MRPL51 C11orf68 ANO5 EIF4G2 MSRB2 RALGDS EXT1 ZC3HC1 PTPRZ1 Privatebiquitous Shared primary Shared metastasis Lung metastases Chest-wall metastasis Perinephric metastasis M1 10 cm R2 (G3) R4 (G1) R6 (G1) Hilum R8 (G4) Primary tumor M2b M2a
  • 4. BACKGROUND – RADIOMICS - HISTOPATHOLOMICS • Huge potential for computerized medical image analysis • Explore and reveal tissue structures related to tissue composition, function, …. • Local quantitative image feature extraction • Supervised and unsupervised machine learning 4 malignant, nonresponder malignant, responder benign pre-malignant undefined quant. feat. #1 quant.feat.#2 Supervised learning, 
 big data
  • 5. BACKGROUND – RADIOMICS - HISTOPATHOLOMICS • Huge potential for computerized medical image analysis • Create imaging biomarkers to predict diagnosis, prognosis, 
 treatment response [Aerts2014] 5 Radiomics [Kumar2012] “Histopatholomics” [Gurcan2009] Reuse existing 
 diagnostic images ✓ radiology data1 ✓ digital pathology Capture tissue 
 heterogeneity ✓ 3D neighborhoods
 (e.g., 512x512x512) ✓ large 2D regions
 (e.g., 15,000x15,000) Analytic power beyond 
 naked eyes ✓ complex 3D tissue morphology ✓exhaustive characterization of 2D tissue structures Non-invasive ✓ x 1e.g., X-ray, Ultrasound, CT, MRI, PET, OCT, …
  • 6. BACKGROUND – RADIOMICS - HISTOPATHOLOMICS • Huge potential for computerized medical image analysis • Explore and reveal tissue structures related to tissue composition, function, …. • Local quantitative image feature extraction • Supervised and unsupervised machine learning 6 malignant, nonresponder malignant, responder benign pre-malignant undefined quant. feat. #1 quant.feat.#2 Supervised learning, 
 big data Specific to texture!
  • 7. OUTLINE • Biomedical texture analysis: background • Defining texture processes • Notations, sampling and texture functions • Texture operators, primitives and invariances • Multiscale analysis • Operator scale and uncertainty principle • Region of interest and operator aggregation • Multidirectional analysis • Isotropic versus directional operators • Importance of the local organization of image directions • Conclusions • References L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f (x @x1 R X Ø Ø@f(x) @x2 Ø Ødx image gra Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in t representation corresponds to one realization (i.e., full image) · m
  • 8. • Definition of texture • Everybody agrees that nobody agrees on the definition of “texture” 
 (context-dependent) • “coarse”, “edgy”, “directional”, “repetitive”, “random”, … • Oxford dictionary: “the feel, appearance, or consistency of a surface or a substance” • [Haidekker2011]: “Texture is a systematic local variation of the image values” • [Petrou2011]: “The most important characteristic of texture is that it is scale dependent. Different types of texture are visible at different scales” COMPUTERIZED TEXTURE ANALYSIS 8 A. Depeursinge et al. / Medical Image Analysis 18 (2014) 176–196 177 [Depeursinge2014a]
  • 9. COMPUTERIZED TEXTURE ANALYSIS directions 9 scales • Spatial scales and directions in images are fundamental
 for visual texture discrimination [Blakemore1969, Romeny2011] • Relating to directional frequencies (shown in Fourier)
  • 10. COMPUTERIZED TEXTURE ANALYSIS 10 directionsscales • Spatial scales and directions in images are fundamental
 for visual texture discrimination [Blakemore1969, Romeny2011] • Most approaches are leveraging these two properties • Explicitly: Gray-level co-occurrence matrices (GLCM), run-length matrices (RLE), directional filterbanks and wavelets, Fourier, histograms of gradients (HOG), local binary patterns (LBP) • Implicitly: Convolutional neural networks (CNN), scattering transform, topographic independant component analysis (TICA)
  • 11. NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS 11 • 2-D continuous texture functions in space and Fourier • Cartesian coordinates: • Polar coordinates: • 2-D digital texture functions • Cartesian coordinates: • Polar coordinates: • Sampling (Cartesian) • Increments in corresponds to physical displacements in as f(k), k = ✓ k1 k2 ◆ 2 Z2 f(R, ⇥), R 2 Z+ , ⇥ 2 [0, 2⇡) (k1, k2) R2 ✓ x1 x2 ◆ = ✓ x1 · k1 x2 · k2 ◆ x1 x2 k2 k1 ) x1 x2 R2 Z2 · f(x) f(k) f(x), x = ✓ x1 x2 ◆ 2 R2 , f(x) F ! ˆf(!) = Z R2 f(x)e jh!,xi dx, ! 2 R2 f(r, ✓), r 2 R+ , ✓ 2 [0, 2⇡), f(r, ✓) F ! ˆf(⇢, ), ⇢ 2 R+ , 2 [0, 2⇡)
  • 12. NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS 12 • 3-D continuous texture functions in space and Fourier • Cartesian coordinates: • Polar coordinates: • 3-D digital texture functions • Cartesian coordinates: • Polar coordinates: • Sampling (Cartesian) • Increments in corresponds to physical displacements in as x1 x2 k2 k1 ) x1x2 · f(x) f(k) f(k), k = 0 @ k1 k2 k3 1 A 2 Z3 f(r, ✓, ), r 2 R+ , ✓ 2 [0, 2⇡), 2 [0, 2⇡) f(R, ⇥, ), R 2 Z+ , ⇥ 2 [0, 2⇡), 2 [0, 2⇡) (k1, k2, k3) R3 0 @ x1 x2 x3 1 A = 0 @ x1 · k1 x2 · k2 x3 · k3 1 A x3 x3 k3 R3 Z3 f(x), x = 0 @ x1 x2 x3 1 A 2 R3 , f(x) F ! ˆf(!) = Z R3 f(x)e jh!,xi dx, ! 2 R3
  • 13. • We consider a texture function as a realization of a spatial stochastic process of 
 
 
 where is the value at the spatial position indexed by • The values of follow one or several probability density functions • Examples NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS 13 f(x) Rd {Xm, m 2 RM1⇥···⇥Md } Xm m moving average Gaussian pointwise Poisson biomedical: lung fibrosis in CT m 2 R128⇥128 m 2 R32⇥32 m 2 R84⇥84 Xm fXm (q)
  • 14. NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS 14 • Stationarity of spatial stochastic processes • A spatial process is stationary if the 
 probability density functions are equivalent for all • Example: heteroscedastic moving average Gaussian process {Xm, m 2 RM1⇥···⇥Md } mfXm (q) stationary non-stationary (strict sense) fb,Xm (q) = 1 3 p 2⇡ e (q 0)2 2 32 fa fb fa,Xm (q) = 1 1 p 2⇡ e (q 0)2 2 12
  • 15. NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS 15 • Stationarity of textures and human perception / tissue biology • Strict/weak process stationarity and texture class definition is not equivalent • Image analysis tasks when textures are considered as • Stationary (wide sense): texture classification • Non-stationary: texture segmentation Outex “canvas039”: stationary? brain glioblastoma in T1-MRI: stationary? ) [Storath2014]
  • 16. OUTLINE • Biomedical texture analysis: background • Defining texture processes • Notations, sampling and texture functions • Texture operators, primitives and invariances • Multiscale analysis • Operator scale and uncertainty principle • Region of interest and operator aggregation • Multidirectional analysis • Isotropic versus directional operators • Importance of the local organization of image directions • Conclusions • References L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f (x @x1 R X Ø Ø@f(x) @x2 Ø Ødx image gra Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in t representation corresponds to one realization (i.e., full image) · m
  • 17. TEXTURE OPERATORS AND PRIMITIVES 17 • Texture operators • A -dimensional texture analysis approach is characterized by a set of 
 local operators centered at the position • is local in the sense that each element only depends on a subregion of • The subregion is the support of • can be linear (e.g., wavelets) or non-linear (e.g., median, GLCMs, LBPs) • For each position , maps the texture function into a 
 -dimensional space N d f(x) xL1 ⇥ · · · ⇥ Ld L1 L2 M1 M2 · m gn(x, m) : RM1⇥···⇥Md 7! RP , n = 1, . . . , N gn(x, m) p=1,...,P gn m gn m gn P gn(f(x), m) : RM1⇥···⇥Md 7! RP L1 ⇥ · · · ⇥ Ld gn
  • 18. TEXTURE OPERATORS AND PRIMITIVES 18 • From texture operators to texture measurements (i.e., features) • The operator is typically applied to all positions of the image by “sliding” its window over the image • Regional texture measurements can be obtained from the aggregation of over a region of interest • For instance, integration can be used to aggregate over • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) m µ 2 RP gn(f(x), m) M M m gn(f(x), m) M µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm
  • 19. TEXTURE OPERATORS AND PRIMITIVES • Texture primitives • A “texture primitive” (also called “texel”) is a fundamental elementary unit (i.e., a building block) of a texture class [Haralick1979, Petrou2006] • Intuitively, given a collection of texture functions , an appropriate set of texture operators must be able to: (i) Detect and quantify the presence of all distinct primitives in (ii) Characterize the spatial relationships between the primitives (e.g., geometric 
 transformations, density) when aggregated texture primitives , 19 texture primitive , fj f1 f2 primitive , primitive , fj=1,...,J gn
  • 20. TEXTURE OPERATORS AND PRIMITIVES • General-purpose texture operators • In general, the texture primitives are neither well-defined, nor known in advance (e.g., biomedical tissue) • General-purpose operator sets are useful to estimate the primitives • How to build such operator sets? 20 patterns in the whole HRCT volume. Methods ataset used is part of an internal multimedia database of ILD cases con- HRCT images with annotated ROIs created in the Talisman project1 . OIs from healthy and five pathologic lung tissue patterns are selected for g and testing the classifiers selecting classes with sufficiently high repre- on (see Table 1). e wavelet frame decompositions with dyadic and quincunx subsampling plemented in Java [11, 16] as well as optimization of SVMs. The basic mentation of the SVMs is taken from the open source Java library Weka2 . 1. Visual aspect and distribution of the ROIs per class of lung tissue pattern. al ect ss healthy emphysema ground glass fibrosis micronodules macronodules ROIs 113 93 148 312 155 22 tients 11 6 14 28 5 5 esults sotropic Polyharmonic B–Spline Wavelets ntioned in Section 1.1, isotropic analysis is preferable for lung texture terization. The Laplacian operator plays an important role in image pro- and is clearly isotropic. Indeed, ∆ = ∂2 ∂x2 1 + ∂2 ∂x2 2 , is rotationally invariant. olyharmonic B–spline wavelets implement a multiscale smoothed version Laplacian [16]. This wavelet, at the first decomposition level, can be char- ed as 2-D lung tissue in CT images [Depeursinge2012a] 3-D normal and osteoporotic bone in CT [Dumas2009]µ
  • 21. • General-purpose texture operators • The exhaustive analysis of spatial scales and directions is computationally expensive when the support of the operators is large: choices are required • Directions (e.g., GLCMs, RLE, HOG) • Scales the classification accuracy of RLE versus GLCMs for categorizing lung tissue patterns associated with diffuse lung disease in HRCT. Using identical choices of the directions for RLE and GLCMs, they found found no statistical differences between the classification performance. Gao et al. (2010) and Qian et al. (2011) compared the performance of three-dimensional GLCM, LBP, Gabor filters and WT in retrieving similar MR images of the brain. They ob- served a small increase in retrieval performance for LBP and GLCM when compared to Gabor filters and WT. However, the database used is rather small and the results might not be statistically significant. Several papers compared the performance of texture analysis algorithms in their 2-D versus 3-D forms. As expected, 2-D texture analysis is most often less discriminative than 3-D, which was ob- served for various applications and techniques, such as: GLCMs, RLE and fractal dimension for the classification of lung tissue types in HRCT in Xu et al. (2005, 2006b). GLCMs for the classification of brain tumors in MRI in Mah- moud-Ghoneim et al. (2003) and Allin Christe et al. (2012). GLCMs for the classification of breast in contrast–enhanced MRI, where statistical significance was assessed in Chen et al. (2007). GMRF for the segmentation of gray matter in MRI in Ranguelova and Quinn (1999). LBP for synthetic texture classification in Paulhac et al. (2008). This demonstrates that 2-D slice-based discrimination of 3-D native texture does not allow fully exploiting the information available in 3-D datasets. An exception was observed with 2-D ver- sus 3-D WTs in Jafari-Khouzani et al. (2004), where the 2-D ap- proach showed a small increase in classification performance of abnormal regions responsible for temporal lobe epilepsy. A separa- ble 3-D WT was used, which did not allow to adequately exploit the 3-D texture information available and may explain the ob- served results. 7. Discussion In the preceding sections, we have reviewed the current state- of-the-art in 3-D biomedical texture analysis. The papers were cat- egorized in terms of imaging modality used, organ studied and im- age processing techniques. The increasing number of papers over the past 32 years clearly shows a growing interest in computerized characterization of three-dimensional texture information (see Fig. 5). This is a consequence of increasingly available 3-D data acquisition devices that are reaching high spatial resolutions allowing to capture tissue properties in its natural space. The analysis of the medical applications in 100 papers in Sec- tion 5 shows the diversity of 3-D biomedical textures. The various geometrical properties of the textures are summarized in Table 2, which defines the multiple challenges of 3-D texture analysis. The need for methods able to characterize structural scales and Fig. 15. Amounts of pixels/voxels covered by the main 13 directions. The number N of discarded pixels/voxels (gray regions in (a) and (b)) according to the neighborhood size R is shown in (c). A. Depeursinge et al. / Medical Image Analysis 18 (2014) 176–196 191 GMRF for the segmentation of gray matter in MRI in Ranguelova and Quinn (1999). LBP for synthetic texture classification in Paulhac et al. (2008). geometrical properties of the textures are summarized which defines the multiple challenges of 3-D textu The need for methods able to characterize structural Fig. 15. Amounts of pixels/voxels covered by the main 13 directions. The number N of discarded pixels/voxels (gray regions in (a) and (b)) according to the neig R is shown in (c). TEXTURE OPERATORS AND PRIMITIVES 21 2-D 3-D (one quadrant) such that numberofdiscarded pixels/voxelsof order −1/2 (an isotropic smoothing operator) of f: Rf = −∇∆−1/2 f. Let’s indeed recall the Fourier-domain definition of these operators: ∇ F ←→ jω and ∆−1/2 F ←→ ||ω||−1 . Unlike the usual gradient ∇, the Riesz transform is self-reversible R⋆ Rf(ω) = (jω)∗ (jω) ||ω||2 ˆf(ω) = ˆf(ω). This allows us to define a self-invertible wavelet frame of L2(R3 ) (tight frame). We however see that there exists a singularity for the frequency (0, 0, 0). This issue will be fixed later, thanks to the van- ishing moments of the primary wavelet transform. 2-D GLCMs with various spatial distances [Haralick1979] 2-D isotropic dyadic wavelets in Fourier [Chenouard2011] d = 1 ( k1 = 1, k2 = 0) d = 2 ( k1 = 2, k2 = 0) d = 2 p 2 ( k1 = 2, k2 = 2) r L1 ⇥ · · · ⇥ Ld L1 = L2 = L3 = 2r + 1 d
  • 22. g(f(x), m) = g(f(R✓0 x x0), m), 8x0 2 R2 , ✓0 2 [0, 2⇡) • Invariances of the texture operators to geometric transformations can be desirable • E.g., scaling, rotations and translations • Invariances of texture operators can be enforced • Example with 2-D Euclidean transforms (i.e., rotation and translation)
 
 
 
 with the rotation matrix INVARIANCE OF TEXTURE OPERATORS 22 R✓0 = ✓ cos ✓0 sin ✓0 sin ✓0 cos ✓0 ◆
  • 23. INVARIANCE OF TEXTURE OPERATORS 23 • Computer vision versus biomedical imaging Computer vision Biomedical image analysis translation translation-invariant translation-invariant rotation rotation-invariant rotation-invariant scale scale-invariant multi-scale 160 Fig. 10. (a) A digitized histopathology image of Grade 4 CaP and different graph-based r Diagram, and Minimum Spanning tree. Fig. 11. Digitized histological image at successively higher scales (magnifica- tions) yields incrementally more discriminatory information in order to detect suspicious regions. or resolution. For instance at low or coarse scales color or tex- ture cues are commonly used and at medium scales architec- tural arrangement of individual histological structures (glands and nuclei) start to become resolvable. It is only at higher res- olutions that morphology of specific histological structures can be discerned. In [93], [94], a multiresolution approach has been used for the classification of high-resolution whole-slide histopathology im- ages. The proposed multiresolution approach mimics the eval- uation of a pathologist such that image analysis starts from the lowest resolution, which corresponds to the lower magnification levels in a microscope and uses the higher resolution represen- Fig. 12 image 1, (c) r as susp show three scale (scal the n dition highe tumo At is com COMPUTERIZED TEXTURE ANALYSIS 7 • Invariances: computer vision versus biomedical imaging Computer vision Biomedical image analysis scale scale-invariant multi-scale rotation rotation-invariant rotation-invariant [4] Histopathological image analysis: a review, Gurcan et al., IEEE Reviews in Biomed Eng, 2:147-71, 2009
 160 IE Fig. 10. (a) A digitized histopathology image of Grade 4 CaP and different graph-based representation Diagram, and Minimum Spanning tree. Fig. 11. Digitized histological image at successively higher scales (magnifica- tions) yields incrementally more discriminatory information in order to detect suspicious regions. or resolution. For instance at low or coarse scales color or tex- ture cues are commonly used and at medium scales architec- tural arrangement of individual histological structures (glands and nuclei) start to become resolvable. It is only at higher res- olutions that morphology of specific histological structures can be discerned. In [93], [94], a multiresolution approach has been used for the classification of high-resolution whole-slide histopathology im- ages. The proposed multiresolution approach mimics the eval- uation of a pathologist such that image analysis starts from the lowest resolution, which corresponds to the lower magnification levels in a microscope and uses the higher resolution represen- tations for the regions requiring more detailed information for a classification decision. To achieve this, images were decom- posed into multiresolution representations using the Gaussian pyramid approach [95]. This is followed by color space con- version and feature construction followed by feature extraction and feature selection at each resolution level. Once the classifier is confident enough at a particular resolution level, the system assigns a classification label (e.g., stroma-rich, stroma-poor or Fig. 12. Results fr image with the tum 1, (c) results at scale as suspicious at low shows the origin three columns s scales. Pixels cl (scale) are disc the number of p ditionally, the p higher scales al tumor and nont At lower reso is commonly us pattern of gland tized histologic scenes can be to every pixel i dient, and Gab the scale, orien gion of interest features within [Gurcan2009][Lazebnik2005]
  • 24. OUTLINE • Biomedical texture analysis: background • Defining texture processes • Notations, sampling and texture functions • Texture operators, primitives and invariances • Multiscale analysis • Operator scale and uncertainty principle • Region of interest and operator aggregation • Multidirectional analysis • Isotropic versus directional operators • Importance of the local organization of image directions • Conclusions • References L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f (x @x1 R X Ø Ø@f(x) @x2 Ø Ødx image gra Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in t representation corresponds to one realization (i.e., full image) · m
  • 25. k2 k1 x1x2f(k) x3 k3 MULTISCALE TEXTURE OPERATORS 25 • Inter-patient and inter-dimension scale normalization • Most medical imaging protocols yield images with 
 various sampling steps • Inter-patient scale normalization is required to 
 ensure the correspondance of spatial frequencies • Inter-dimension scale normalization is required to ensure isotropic 
 scale/directions definition ( x1, x2, x3) x1 = x2 = 0.4mm x1 = x2 = 1.6mm spatial Fourier Fourierspatial 0 0 0 0 ⇡ ⇡⇡ ⇡ ⇡ ⇡ x1x2 x3 ) x0 1x0 2 x0 3 d = 1 ( k1 = 1, k2 = 0) GLCMs 0.4mm 1.6mm
  • 26. MULTISCALE TEXTURE OPERATORS 26 • Which scales for texture measurements? • Two aspects: A. How to define the size(s) of the operator(s) ? B. How to define the size of the region of interest ? TEXTURE OPERATORS AND PRIMITIVES • From texture operators to texture measurements • The operator is typically applied to all positions by “sliding” its window over the image • Regional texture measurements can be obtained from aggregation of over a region • For instance, integration can be used to aggregate • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) m µ 2 RP gn(f(x), m) M M m gn(f(x), m µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm M original image d increasingly small operator size L1 L2 · · · LN 0 B B B B B B B B B B B B B B B B B B B B B B B B @ µ1 1 ... µ1 P µ2 1 ... µ2 P ... µN 1 ... µN P 1 C C C C C C C C C C C C C C C C C C C C C C C C A = µM: concatenated measurements from
 multiscale operators
 aggregated over Ln 1 ⇥ · · · ⇥ Ln d gn=1,...,N (f(x), m) M f(x) M [Depeursinge2012b]
  • 27. MULTISCALE TEXTURE OPERATORS 27 • Which scales for texture measurements? • Two aspects: A. How to define the size(s) of the operator(s) ? B. How to define the size of the region of interest ? TEXTURE OPERATORS AND PRIMITIVES • From texture operators to texture measurements • The operator is typically applied to all positions by “sliding” its window over the image • Regional texture measurements can be obtained from aggregation of over a region • For instance, integration can be used to aggregate • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) m µ 2 RP gn(f(x), m) M M m gn(f(x), m µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm M original image d increasingly small operator size L1 L2 · · · LN 0 B B B B B B B B B B B B B B B B B B B B B B B B @ µ1 1 ... µ1 P µ2 1 ... µ2 P ... µN 1 ... µN P 1 C C C C C C C C C C C C C C C C C C C C C C C C A = µM: concatenated measurements from
 multiscale operators
 aggregated over Ln 1 ⇥ · · · ⇥ Ln d gn=1,...,N (f(x), m) M f(x) M [Depeursinge2012b]
  • 28. MULTISCALE TEXTURE OPERATORS 28 • Size and spectral coverage of operators • Uncertainty principle: operators cannot be well located both in 
 space and Fourier [Petrou2006] • In 2D, the trade-off between the spatial support (i.e., ) and frequency support (i.e., ) of the operators is given by L1 ⇥ L2 ⌦1 ⇥ ⌦2 F ! ˆf(!)f(x) L2 1 ⌦2 1 L2 2 ⌦2 2 1 16
  • 29. MULTISCALE TEXTURE OPERATORS 29 • Size and spectral coverage of operators • Becomes a problem in the case of non-stationary texture: ,accurate spatial localization poor spectrum characterization Gaussian window: 
 = 3.2mm 0 ⇡|!1| F !
  • 30. MULTISCALE TEXTURE OPERATORS 30 • Size and spectral coverage of operators • Becomes a problem in the case of non-stationary texture: ,accurate spatial localization poor spectrum characterization Gaussian window: 
 = 3.2mm Gaussian window: 
 = 38.4mm 0 ⇡|!1| F !
  • 31. MULTISCALE TEXTURE OPERATORS 31 • Size and spectral coverage of operators • Becomes a problem in the case of non-stationary texture: ,accurate spatial localization poor spectrum characterization Gaussian window: 
 = 3.2mm Gaussian window: 
 = 38.4mm 0 ⇡|!1| F ! The spatial support should have the minimum size that allows 
 rich enough texture-specific spectral characterization
  • 32. MULTISCALE TEXTURE OPERATORS 32 • Other consequence: • Large influence of proximal objects 
 when the support of operators is 
 larger than the region of interest • Example with band-limited operators (2D isotropic wavelets) 
 and lung boundary [Ward2015, Depeursinge2015a] • Tuning the shape/bandwidth was found to have a strong influence on lung tissue classification accuracy M 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 30 35 0 0.05 0.1 0.15 0.2 0 0.5 1 1.5 2 0 ⇡|!1|0 0 ⇡|!1||!1| ⇡x1 x1 x1 ˆf(!)f(x) f(x) f(x)ˆf(!) ˆf(!) COMPUTERIZED TEXTURE ANALYSIS 31 • Other consequence: • Large influence of proximal objects 
 when the support of operators is 
 larger than the region of interest: • Example with band-limited operators (2D isotropic wavelets) 
 and lung boundary [DPC2015,WPU2015] • Tuning the shape/bandwidth was found to have a strong influence on lung tissue classification accuracy L1 L2 M1 M2 · M m M 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 30 35 0 0.05 0.1 0.15 0.2 0 0.5 1 1.5 2 0 ⇡|!1|0 0 ⇡|!1||!1| ⇡x1 x1 x1 ˆf(!)f(x) f(x) f(x)ˆf(!) ˆf(!) better spatial localization worse spectral localization
  • 33. TEXTURE OPERATORS AND PRIMITIVES • From texture operators to texture measurements • The operator is typically applied to all positions by “sliding” its window over the image • Regional texture measurements can be obtained from aggregation of over a region of interest • For instance, integration can be used to aggregate • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) m µ 2 RP gn(f(x), m) M m gn(f(x), m µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm MULTISCALE TEXTURE OPERATORS 33 • Which scales for texture measurements? • Two aspects: A. How to define the size(s) of the operator(s) ? B. How to define the size of the region of interest ?M original image d increasingly small operator size L1 L2 · · · LN 0 B B B B B B B B B B B B B B B B B B B B B B B B @ µ1 1 ... µ1 P µ2 1 ... µ2 P ... µN 1 ... µN P 1 C C C C C C C C C C C C C C C C C C C C C C C C A = µM: concatenated measurements from
 multiscale operators
 aggregated over Ln 1 ⇥ · · · ⇥ Ln d gn=1,...,N (f(x), m) M f(x) M M [Depeursinge2012b]
  • 34. MULTISCALE TEXTURE OPERATORS • How large must be the region of interest ? • No more than enough to evaluate texture stationarity 
 in terms of human perception / tissue biology • Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets [Portilla2000] applied to all image positions • Operators’ responses are averaged over M • The operator is typically applied to all position by “sliding” its window over the image • Regional texture measurements can be obtained aggregation of over a region of interest • For instance, integration can be used to aggregate • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) µ 2 RP gn(f(x), m) M m gn(f( µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm M f(x) g1(f(x), m) g2(f(x), m) original image with regions I 1 |M| Z M |g1(f(x), m)|dm M feature space 1 |M| Z M |g2(f(x),m)|dm f(x) Ma, Mb, Mc The averaged responses over the entire image does not correspond 
 to anything visually! ˆg1(⇢) = ⇢ cos ⇡ 2 log2 2⇢ ⇡ , ⇡ 4 ⇢  ⇡ 0, otherwise. ˆg2(⇢) = ⇢ cos ⇡ 2 log2 4⇢ ⇡ , ⇡ 8 ⇢  ⇡ 2 0, otherwise. g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, ) m 2 RM1⇥M2
  • 35. MULTISCALE TEXTURE OPERATORS • How large must be the region of interest ? • No more than enough to evaluate texture stationarity 
 in terms of human perception / tissue biology • Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets [Portilla2000] applied to all image positions • Operators’ responses are averaged over M • The operator is typically applied to all position by “sliding” its window over the image • Regional texture measurements can be obtained aggregation of over a region of interest • For instance, integration can be used to aggregate • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) µ 2 RP gn(f(x), m) M m gn(f( µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm M f(x) g1(f(x), m) m 2 RM1⇥M2 g2(f(x), m) original image with regions I 1 |M| Z M |g1(f(x), m)|dm M feature space 1 |M| Z M |g2(f(x),m)|dm f(x) Ma, Mb, Mc The averaged responses over the entire image does not correspond 
 to anything visually! ˆg1(⇢) = ⇢ cos ⇡ 2 log2 2⇢ ⇡ , ⇡ 4 ⇢  ⇡ 0, otherwise. ˆg2(⇢) = ⇢ cos ⇡ 2 log2 4⇢ ⇡ , ⇡ 8 ⇢  ⇡ 2 0, otherwise. g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, ) Nor biologically!
  • 36. MULTISCALE TEXTURE OPERATORS • How large must be the region of interest ? • No more than enough to evaluate texture stationarity 
 in terms of human perception / tissue biology • Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets [Portilla2000] applied to all image positions • Operators’ responses are averaged over M • The operator is typically applied to all position by “sliding” its window over the image • Regional texture measurements can be obtained aggregation of over a region of interest • For instance, integration can be used to aggregate • e.g., average: L1 L2 M1 M2 L1 ⇥ · · · ⇥ Ld · gn(x, m) µ 2 RP gn(f(x), m) M m gn(f( µ = 0 B @ µ1 ... µP 1 C A = 1 |M| Z M gn(f(x), m) p=1,...,P dm M f(x) g1(f(x), m) m 2 RM1⇥M2 g2(f(x), m) original image with regions I 1 |M| Z M |g1(f(x), m)|dm M feature space 1 |M| Z M |g2(f(x),m)|dm f(x) Ma, Mb, Mc The averaged responses over the entire image does not correspond 
 to anything visually! ˆg1(⇢) = ⇢ cos ⇡ 2 log2 2⇢ ⇡ , ⇡ 4 ⇢  ⇡ 0, otherwise. ˆg2(⇢) = ⇢ cos ⇡ 2 log2 4⇢ ⇡ , ⇡ 8 ⇢  ⇡ 2 0, otherwise. g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, ) Nor biologically! Define regions that are homogeneous 
 in terms of operators’ responses 
 (e.g., pixelwise clustering, graph cuts [Malik2001], 
 Pott’s model [Storath2014])
  • 37. MULTISCALE TEXTURE OPERATORS 37 • How large must be the region of interest ? • Example: supervised texture segmentation with undecimated isotropic Simoncelli’s wavelets and linear support vector machines (SVM) M feature space (training) predicted labels 1 |M| Z M |g1(f(x), m)|dm 1 |M| Z M |g2(f(x),m)|dm decision values -10 -8 -6 -4 -2 0 2 4 6 8 train class 1 (128x128) test image (256x256)train class 2 (128x128) segmentation error=0.05127
  • 38. MULTISCALE TEXTURE OPERATORS 38 • How large must be the region of interest ? • Example: supervised texture segmentation with undecimated isotropic Simoncelli’s wavelets and linear support vector machines (SVM) M predicted labels decision values -10 -8 -6 -4 -2 0 2 4 6 8 train class 1 (128x128) test image (256x256)train class 2 (128x128) segmentation error=0.05127 patch radius 0 20 40 60 80 100 120 segmentationerror 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 : circular patch with 
M r = 30
  • 39. : circular patch with 
 MULTISCALE TEXTURE OPERATORS 39 • How large must be the region of interest ? • Example: supervised texture segmentation with undecimated isotropic Simoncelli’s wavelets and linear support vector machines (SVM) M patch radius 0 20 40 60 80 100 120 segmentationerror 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 train class 1 (128x128) train class 2 (128x128) test image (256x256) r = 8 decision values -2 -1 0 1 2 3 predicted labels segmentation error=0.23853 M
  • 40. MULTISCALE TEXTURE OPERATORS 40 • How large must be the region of interest ? • Example: supervised texture segmentation with undecimated isotropic Simoncelli’s wavelets and linear support vector machines (SVM) M patch radius 0 20 40 60 80 100 120 segmentationerror 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 : circular patch with 
 r = 128 test image (256x256)train class 1 (128x128) train class 2 (128x128) predicted labels decision values -1.5 -1 -0.5 0 0.5 1 segmentation error=0.22839 M
  • 41. MULTISCALE TEXTURE OPERATORS 41 • How large must be the region of interest ? • Tissue properties are not homogeneous (i.e., non-stationary) over 
 the entire organ • Importance of building tissue atlases and digital phenotypes [Depeursinge2015b] M FIGURE 3. The 36 subregions of the lungs localized the prototype regional distributions of the texture properties. Figure 3 can be viewed online in color Depeursinge et al Investigative Radiology • Volume 00, Number 00, Month 2015 M1 M2 = µM1,...,S
  • 42. OUTLINE • Biomedical texture analysis: background • Defining texture processes • Notations, sampling and texture functions • Texture operators, primitives and invariances • Multiscale analysis • Operator scale and uncertainty principle • Region of interest and operator aggregation • Multidirectional analysis • Isotropic versus directional operators • Importance of the local organization of image directions • Conclusions • References L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f (x @x1 R X Ø Ø@f(x) @x2 Ø Ødx image gra Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in t representation corresponds to one realization (i.e., full image) · m
  • 43. 43 • Which directions for texture measurements? • Isotropic operators: insensitive to image directions • Linear: • Non-linear: e.g., median filter • Directional operators • Linear: Other: e.g., Fourier, circular and spherical harmonics [Unser2013, Ward2014], MR8 [Varma2005], HOG [Dalal2005], 
 Simoncelli’s pyramid [Simoncelli1995], curvelets [Candes2000] • Non-linear: Other: e.g., RLE [Galloway1975] 2D Gaussian filter 2D isotropic wavelets [Portilla2000] MULTIDIRECTIONAL TEXTURE OPERATORS 2D Gaussian derivatives (e.g., Riesz wavelets [Unser2011]) @ @x1 @ @x2 @2 @x2 2 @2 @x2 1 @2 @x1@x2 d = 2 ( k1 = 2, k2 = 0) 2D GLCMs d = 2 p 2 ( k1 = 2, k2 = 2) 2D LBPs [Ojala2002] spaced pixels on a circle of radius R …R 0† that form a circularly symmetric neighbor set. T % t…s…g0 À gc†; s…g1 À gc†; . . . ; s…gPÀ1 À gc††; …5† where OJALA ET AL.: MULTIRESOLUTION GRAY-SCALE AND ROTATION INVARIANT TEXTURE CLASSIFICATION WITH LOCAL BINARY PATTERNS 973 Fig. 1. Circularly symmetric neighbor sets for different (P; R). spaced pixels on a circle of radius R …R 0† that form a circularly symmetric neighbor set. T % t…s…g0 À gc†; s…g1 À gc†; . . . ; s…gPÀ1 À gc††; …5† where OJALA ET AL.: MULTIRESOLUTION GRAY-SCALE AND ROTATION INVARIANT TEXTURE CLASSIFICATION WITH LOCAL BINARY PATTERNS 973 Fig. 1. Circularly symmetric neighbor sets for different (P; R). [?] Texture in Biomedical Images, Petrou M., L1 L2 M1 M2 Mass- in a able om- ures. tion rom use ping iven cific nt of the tally for of1(x) : of2(x) : R X Ø Ø @f (x) @x1 R X Ø Ø@f(x) @x2 Ø Ødx image grad Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x) @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in th representation corresponds to one realization (i.e., full image) · m gn(r, ✓, m) 7! gn(r, m)x
  • 44. MULTIDIRECTIONAL TEXTURE OPERATORS 44 • Which directions for texture measurements? • Is directional information important for texture discrimination? F ! ˆf(!)f(x) F ! ˆf(!)f(x)
  • 45. MULTIDIRECTIONAL TEXTURE OPERATORS 45 • Which directions for texture measurements? • Importance of the local organization of image directions (LOID) • i.e., how directional structures intersect
  • 46. MULTIDIRECTIONAL TEXTURE OPERATORS 46 • Which directions for texture measurements? • Isotropic and unidirectional operators can hardly characterize the LOIDs, especially when aggregated over a region [Sifre2014, Depeursinge2014b] • Example of feature representation when integrated over entire image M isotropic Simoncelli wavelets scale 1 scale2 o o GLCM contrast GLCMcontrast d = 1 ( k1 = 1, k2 = 0) d=1(k1=0,k2=1) GLCMsgradients along and 1 |M| Z M ✓ @f(x) @x1 ◆2 dx 1 |M| Z M ✓ @f(x) @x2 ◆2 dx x1 x2 M ⌘ L1 L2 M1 M2 · M m L1 L2 M1 M2 · M m
  • 47. MULTIDIRECTIONAL TEXTURE OPERATORS 47 • Which directions for texture measurements? • Isotropic and unidirectional operators can hardly characterize the LOIDs, especially when aggregated over a region [Sifre2014, Depeursinge2014b] • Example of feature representation when integrated over entire image M isotropic Simoncelli wavelets scale 1 scale2 o o GLCM contrast GLCMcontrast d = 1 ( k1 = 1, k2 = 0) d=1(k1=0,k2=1) GLCMsgradients along and 1 |M| Z M ✓ @f(x) @x1 ◆2 dx 1 |M| Z M ✓ @f(x) @x2 ◆2 dx x1 x2 M ⌘ L1 L2 M1 M2 · M m L1 L2 M1 M2 · M m Very poor discrimination! 
 Solutions proposed in a few slides…
  • 48. MULTIDIRECTIONAL TEXTURE OPERATORS 48 • Locally rotation-invariant operators over • Isotropic operators: • By definition • Directional: • Averaging operators’ responses
 over all directions: 2D GLCMs ¯µ1 (e.g., GLCM contrast) No characterization of image directions! µ ⇡/2 1µ ⇡/4 1 µ 3⇡/4 1 µ0 1 L1 ⇥ L2 g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 49. REFERENCES [?] Texture in Biomedical Images, Petrou M., L1 L2 M1 M2 M MULTIDIRECTIONAL TEXTURE OPERATORS 49 • Locally rotation-invariant operators over • Locally “aligning” directional operators • MR8 filterbank [Varma2005] • Rotation-invariant LBP [Ojala2002, Ahonen2009] • Steerable Riesz wavelets [Unser2013, Depeursinge2014b] L1 ⇥ L2 g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 50. MULTIDIRECTIONAL TEXTURE OPERATORS 50 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Maximum response 8 (MR8) filterbank [Varma2005] • Filter responses are obtained for each pixel from the convolution of the filter and the image • For each position , only the maximum responses among 
 gradient and Laplacian filters are kept isotropic mutliscale oriented gradients multiscale oriented Laplacians m L1 ⇥ L2 g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 51. MULTIDIRECTIONAL TEXTURE OPERATORS 51 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Maximum response 8 (MR8) filterbank [Varma2005] • Filter responses are obtained for each pixel from the convolution of the filter and the image • For each position , only the maximum responses among 
 gradient and Laplacian filters are kept isotropic mutliscale oriented gradients multiscale oriented Laplacians m L1 ⇥ L2 Yields approximate local rotation invariance Poor characterization of the LOIDs g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 52. MULTIDIRECTIONAL TEXTURE OPERATORS 52 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Local binary patterns (LBP) [Ojala2002] • Rotation-invariant LBP [Ahonen2009] L1 ⇥ L2 1) define a circular neighborhood 2) Binarize and build a number that encode the LOIDs ) 3) Aggregate over the entire image and count code occurrences 0 170 ⌘ 4) make codes invariant to circular shifts U8(1, 0) = 10101010 = 170 U8(1, 0) = 10101010 U8(1, 1) = 01010101 m rotation r discrete Fourier transform The new measures are independent of the rotation ) r H8(1, u) = 7X r=0 hI(U8(1, r))e j2⇡ur/8 µp = |H8(1, u)| µ0,p = hI (U8(1, 0)) g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 53. MULTIDIRECTIONAL TEXTURE OPERATORS 53 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Local binary patterns (LBP) [Ojala2002] • Rotation-invariant LBP [Ahonen2009] L1 ⇥ L2 1) define a circular neighborhood 2) Binarize and build a number that encode the LOIDs ) 3) Aggregate over the entire image and count code occurrences 0 170 ⌘ 4) make codes invariant to circular shifts U8(1, 0) = 10101010 = 170 U8(1, 0) = 10101010 U8(1, 1) = 01010101 m rotation r discrete Fourier transform The new measures are independent of the rotation ) r H8(1, u) = 7X r=0 hI(U8(1, r))e j2⇡ur/8 µp = |H8(1, u)| µ0,p = hI (U8(1, 0)) Encodes the LOIDs independently 
 from their local orientations! Requires binarization… Spherical sequences are undefined in 3D… g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 54. MULTIDIRECTIONAL TEXTURE OPERATORS 54 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Steerable Riesz wavelets [Unser2013, Depeursinge2014b] • Operators: th-order multi-scale image derivatives L1 ⇥ L2 input imageN4 A. Depeursinge et al. N = 1 N = 2 N = 3 Fig. 1. Templates corresponding to the Riesz kernels convolved with a Gaussian smoother for N=1,2,3. N = 1 g(1,0)(x, m) f(x) g(1,0)(f(x), m) g(0,1)(x, m) g(0,1)(f(x), m) g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 55. MULTIDIRECTIONAL TEXTURE OPERATORS 55 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Steerable Riesz wavelets [Unser2013, Depeursinge2014b] • Operators: th-order multi-scale image derivatives L1 ⇥ L2 N4 A. Depeursinge et al. N = 1 N = 2 N = 3 Fig. 1. Templates corresponding to the Riesz kernels convolved with a Gaussian smoother for N=1,2,3. N = 1 N = 2 N = 3 g(1,0)(x, m) g(2,0)(x, m)g(0,1)(x, m) g(0,2)(x, m) g(0,3)(x, m) g(1,1)(x, m) g(3,0)(x, m) g(2,1)(x, m) g(1,2)(x, m) g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 56. MULTIDIRECTIONAL TEXTURE OPERATORS 56 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Steerable Riesz wavelets [Unser2013, Depeursinge2014b] • Steerability: L1 ⇥ L2 g(1,0)(R✓0 x, 0) = cos ✓0 g(1,0)(x, 0) + sin ✓0 g(0,1)(x, 0) g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 57. MULTIDIRECTIONAL TEXTURE OPERATORS 57 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Steerable Riesz wavelets [Unser2013, Depeursinge2014b] • Local rotation-invariance: L1 ⇥ L2 ✓max(m) := arg max ✓02[0,2⇡) ✓ cos ✓0 g(1,0)(f(x), m) + sin ✓0 g(0,1)(f(x), m) ◆ µM = 1 |M| Z M ✓ g(1,0)(f(R✓max(m) x), m) ◆2 dm) g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 58. MULTIDIRECTIONAL TEXTURE OPERATORS 58 • Locally rotation-invariant operators over • Locally “aligning” directional operators • Steerable Riesz wavelets [Depeursinge2014b, Unser2013] • Local rotation-invariance: L1 ⇥ L2 ✓max(m) := arg max ✓02[0,2⇡) ✓ cos ✓0 g(1,0)(f(x), m) + sin ✓0 g(0,1)(f(x), m) ◆ µM = 1 |M| Z M ✓ g(1,0)(f(R✓max(m) x), m) ◆2 dm) Encodes the LOIDs independently 
 from their local orientations! No binarization required! Available in 3D [Chenouard2012, Depeursinge2015a], and combined with feature learning [Depeursinge2014b]. g(f(x), m) = g(f(R✓0 x), m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2
  • 59. • Operators characterizing the LOIDs MULTIDIRECTIONAL TEXTURE OPERATORS GLCMs Riesz wavelets ( ) 59 GLCM contrast GLCMcontrast d = 1 ( k1 = 1, k2 = 0) d=1(k1=0,k2=1) N = 2 1 |M| Z M ✓ g(2,0)(f(x),m) ◆2 dm 1 |M| Z M ✓ g(0,2)(f(x), m) ◆2 dm aligned Riesz wavelets ( )N = 2 1 |M| Z M ✓ g(0,2)(f(R✓max(m) x), m) ◆2 dm 1 |M| Z M ✓ g(2,0)(f(R✓max(m)x),m) ◆2 dm
  • 60. • Operators characterizing the LOIDs MULTIDIRECTIONAL TEXTURE OPERATORS 60 GLCMs GLCM contrast GLCMcontrast d = 1 ( k1 = 1, k2 = 0) d=1(k1=0,k2=1) Riesz wavelets ( ) 1 |M| Z M ✓ g(2,0)(f(x),m) ◆2 dm 1 |M| Z M ✓ g(0,2)(f(x), m) ◆2 dm N = 2 aligned Riesz wavelets ( )N = 2 1 |M| Z M ✓ g(0,2)(f(R✓max(m) x), m) ◆2 dm 1 |M| Z M ✓ g(2,0)(f(R✓max(m)x),m) ◆2 dm
  • 61. MULTIDIRECTIONAL TEXTURE OPERATORS 61 • Isotropic or directional analysis? [Depeursinge2014b] • Outex [Ojala2002]: 24 classes, 180 images/class, 9 rotation angles in • Texture classification: linear SVMs trained with unrotated images only IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5 1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021 9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035 17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009 Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database. 1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin 9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite. 180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 , 135 and 150 of the other seven Brodatz images for each class. The total number of images in the test set is 672. G. Experimental setup OVA SVM models using Gaussian kernels as K(xi, xj) = Inc., 2012. The computational complexity is dominated by the local orientation of N c in Eq. 11, which consists of finding the roots of the polynomials defined by the steering matrix A✓ . It is therefore NP–hard (Non–deterministic Polynomial–time hard), where the order of the polynomials is controlled by the order of the Riesz transform N. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 9 10 ini/alcoeffs aligned1sttemplate order of the Riesz transformN classificationaccuracy Riesz wavelets aligned Riesz wavelets
  • 62. MULTIDIRECTIONAL TEXTURE OPERATORS 62 • Isotropic or directional analysis? [Depeursinge2014b] • Outex [Ojala2002]: 24 classes, 180 images/class, 9 rotation angles in • Texture classification: linear SVMs trained with unrotated images only IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5 1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021 9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035 17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009 Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database. 1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin 9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite. 180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 , 135 and 150 of the other seven Brodatz images for each class. The total number of images in the test set is 672. G. Experimental setup OVA SVM models using Gaussian kernels as K(xi, xj) = Inc., 2012. The computational complexity is dominated by the local orientation of N c in Eq. 11, which consists of finding the roots of the polynomials defined by the steering matrix A✓ . It is therefore NP–hard (Non–deterministic Polynomial–time hard), where the order of the polynomials is controlled by the order of the Riesz transform N. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 9 10 ini/alcoeffs aligned1sttemplate order of the Riesz transformN classificationaccuracy Riesz wavelets aligned Riesz wavelets Isotropic operators (i.e., ) perform best 
 when not aligned! N = 0
  • 63. OUTLINE • Biomedical texture analysis: background • Defining texture processes • Notations, sampling and texture functions • Texture operators, primitives and invariances • Multiscale analysis • Operator scale and uncertainty principle • Region of interest and operator aggregation • Multidirectional analysis • Isotropic versus directional operators • Importance of the local organization of image directions • Conclusions • References L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f (x @x1 R X Ø Ø@f(x) @x2 Ø Ødx image gra Fig. 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f (x @x2 able to discriminate between the two textures classes f1(x) when integrated over the image domain X . One circle in t representation corresponds to one realization (i.e., full image) · m
  • 64. CONCLUSIONS • We presented a general framework to describe 
 and analyse texture information in 2D and 3D • Tissue structures in 2D/3D medical images contain extremely rich and valuable information to optimize personalized medicine in a non-invasive way • Invisible to the naked eye! 64 L1 L2 M1 M2 M of1(x) : of2(x) : R X Ø Ø @f @x R X Ø Ø@f(x) @x2 Ø Ødx image gr . 1. The joint responses of image gradients ≥ | @f (x) @x1 |,| @f @x e to discriminate between the two textures classes f1(x en integrated over the image domain X . One circle in · m malignant, nonresponder malignant, responder benign pre-malignant undefined quant. feat. #1 quant.feat.#2
  • 65. MULTISCALE TEXTURE OPERATORS • How large must be the region of interest ? • Enough to evaluate texture stationarity in terms 
 of human perception / tissue biology • Example with operator: undecimated isotropic Simoncelli’s dyadi [PoS2000] applied to all image positions • Operators’ responses are averaged over M TEXTURE OPERATORS AND P • From texture operators to • The operator is typ by “sliding” its window • Regional texture measuremen aggregation of • For instance, integration can b • e.g., average: M1 L1 ⇥ · gn(x, m) gn(f(x), m) µ = 0 B @ µ1 ... µP 1 C A = |Mf(x) g1(f(x), m) m 2 RM1⇥M2 g2 original image with regions I 1 |M| Z M |g1(f(x), m)|dm M feature space 1 |M| Z M |g2(f(x),m)|dm f(x) Ma, Mb, Mc The average over the en does not c to anythin ˆg1(⇢) = ⇢ cos ⇡ 2 log2 2⇢ ⇡ , ⇡ 4 ⇢  ⇡ 0, otherwise. ˆg2(⇢) = ⇢ cos ⇡ 2 log2 4⇢ ⇡ , ⇡ 8 ⇢  ⇡ 2 0, otherwise. g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, ) Nor biolo BoVW can be used to first reveal the intra-class visual diversity texture operators region of interest
 and aggregation scales uncertainty principle averaging operators’ responses directions isotropic versus directional importance of LOIDs CONCLUSIONS • Biomedical textures are realizations of complex 
 non-stationary spatial stochastic processes • General-purpose image operators are necessary to identify data-specific discriminative scales and directions 65 MULTISCALE TEXTURE OPERATORS • How large must be the region of interest ? • Enough to evaluate texture stationarity in terms 
 of human perception / tissue biology • Example with operator: undecimated isotropic Simoncelli’s dyad [PoS2000] applied to all image positions • Operators’ responses are averaged over M TEXTURE OPERATORS AND P • From texture operators to • The operator is ty by “sliding” its window • Regional texture measureme aggregation of • For instance, integration can • e.g., average: M1 L1 ⇥ gn(x, m) gn(f(x), m) µ = 0 B @ µ1 ... µP 1 C A = f(x) g1(f(x), m) m 2 RM1⇥M2 g original image with regions I 1 |M| Z M |g1(f(x), m)|dm M feature space 1 |M| Z M |g2(f(x),m)|dm f(x) Ma, Mb, Mc The average over the e does not c to anythin ˆg1(⇢) = ⇢ cos ⇡ 2 log2 2⇢ ⇡ , ⇡ 4 ⇢  ⇡ 0, otherwise. ˆg2(⇢) = ⇢ cos ⇡ 2 log2 4⇢ ⇡ , ⇡ 8 ⇢  ⇡ 2 0, otherwise. g1,2 f(⇢, ) = ˆg1,2(⇢, ) · ˆf(⇢, ) Nor bio BoVW can be used to first reveal the intra-class visual diversity F ! ˆf(!)f(x) )TextureQbased'biomarkers:'current'limitaGons' x  Assume'homogeneous'texture'properGes'over'the' enGre'lesion'[5]' ' x  NonQspecific'features' x  Global'vs'local'characterizaGon'of'image'direcGons'[6]' REVIEW: Quantitative Imaging in Cancer Evolution and Ecology Gatenby et al with the mean signal value. By using just two sequences, a contrast-enhanced T1 sequence and a fluid-attenuated inver- sion-recovery sequence, we can define four habitats: high or low postgadolini- um T1 divided into high or low fluid-at- tenuated inversion recovery. When these voxel habitats are projected into the tu- mor volume, we find they cluster into spatially distinct regions. These habitats can be evaluated both in terms of their relative contributions to the total tumor volume and in terms of their interactions with each other, based on the imaging characteristics at the interfaces between regions. Similar spatially explicit analysis can be performed with CT scans (Fig 5). Analysis of spatial patterns in cross-sectional images will ultimately re- quire methods that bridge spatial scales from microns to millimeters. One possi- ble method is a general class of numeric tools that is already widely used in ter- restrial and marine ecology research to link species occurrence or abundance with environmental parameters. Species distribution models (48–51) are used to gain ecologic and evolutionary insights and to predict distributions of species or morphs across landscapes, sometimes extrapolating in space and time. They can easily be used to link the environ- mental selection forces in MR imaging- defined habitats to the evolutionary dy- namics of cancer cells. Summary Imaging can have an enormous role in the development and implementation of rise to local-regional phenotypic adap- tations. Phenotypic alterations can re- sult from epigenetic, genetic, or chro- mosomal rearrangements, and these in turn will affect prognosis and response to therapy. Changes in habitats or the relative abundance of specific ecologic communities over time and in response to therapy may be a valuable metric with which to measure treatment efficacy and emergence of resistant populations. microenvironment can be rewarded by increased proliferation. This evolution- ary dynamic may contribute to distinct differences between the tumor edges and the tumor cores, which frequently can be seen at analysis of cross-sec- tional images (Fig 5). Interpretation of the subsegmenta- tion of tumors will require computa- tional models to understand and predict the complex nonlinear dynamics that lead to heterogeneous combinations of radiographic features. We have ex- Figure 4 Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions. REVIEW: Quantitative Imaging in Cancer Evolution and Ecology Gatenby et a with the mean signal value. By using jus two sequences, a contrast-enhanced T1 sequence and a fluid-attenuated inver sion-recovery sequence, we can define four habitats: high or low postgadolini um T1 divided into high or low fluid-at tenuated inversion recovery. When these voxel habitats are projected into the tu mor volume, we find they cluster into spatially distinct regions. These habitat can be evaluated both in terms of thei relative contributions to the total tumo volume and in terms of their interaction with each other, based on the imaging characteristics at the interfaces between regions. Similar spatially explicit analysi can be performed with CT scans (Fig 5) Analysis of spatial patterns in cross-sectional images will ultimately re quire methods that bridge spatial scale from microns to millimeters. One possi ble method is a general class of numeric tools that is already widely used in ter restrial and marine ecology research to link species occurrence or abundance with environmental parameters. Specie distribution models (48–51) are used to gain ecologic and evolutionary insight and to predict distributions of species o morphs across landscapes, sometime extrapolating in space and time. They can easily be used to link the environ mental selection forces in MR imaging defined habitats to the evolutionary dy namics of cancer cells. Summary Imaging can have an enormous role in the development and implementation o patient-specific therapies in cancer. The rise to local-regional phenotypic adap- tations. Phenotypic alterations can re- sult from epigenetic, genetic, or chro- mosomal rearrangements, and these in turn will affect prognosis and response to therapy. Changes in habitats or the relative abundance of specific ecologic communities over time and in response to therapy may be a valuable metric with which to measure treatment efficacy and emergence of resistant populations. Emerging Strategies for Tumor Habitat microenvironment can be rewarded by increased proliferation. This evolution- ary dynamic may contribute to distinct differences between the tumor edges and the tumor cores, which frequently can be seen at analysis of cross-sec- tional images (Fig 5). Interpretation of the subsegmenta- tion of tumors will require computa- tional models to understand and predict the complex nonlinear dynamics that lead to heterogeneous combinations of radiographic features. We have ex- ploited ecologic methods and models to Figure 4 Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions. [5]'QuanGtaGve'imaging'in'cancer'evoluGon'and'ecology,'Gatenby'et'al.,'Radiology,'269(1):8Q15,'2013' 5' global'direcGonal'operators:' local'grouped'steering:' [6]'RotaGonQcovariant'texture'learning'using'steerable'Riesz'wavelets,'Depeursinge'et'al.,'IEEE'Trans'Imag'Proc.,'23(2):898Q908,'2014.' TextureQbased'biomarkers:'current'limita x  Assume'homogeneous'texture'properGes'over't enGre'lesion'[5]' ' x  NonQspecific'features' x  Global'vs'local'characterizaGon'of'image'direcGo REVIEW: Quantitative Imaging in Cancer Evolution and Ecology with th two se sequen sion-re four h um T1 tenuat voxel mor v spatia can be relativ volum with e charac region can be An cross- quire from m ble me tools t restria link s with e distrib gain e and to morph extrap can ea menta define namic Summ Imagin the de patien achiev metho place assess The n been c Cance mation work. consor ducibl extrac rise to local-regional phenotypic adap- tations. Phenotypic alterations can re- sult from epigenetic, genetic, or chro- mosomal rearrangements, and these in turn will affect prognosis and response to therapy. Changes in habitats or the relative abundance of specific ecologic communities over time and in response to therapy may be a valuable metric with which to measure treatment efficacy and emergence of resistant populations. Emerging Strategies for Tumor Habitat Characterization A method for converting images to spa- tially explicit tumor habitats is shown in Figure 4. Here, three-dimensional MR imaging data sets from a glioblastoma are segmented. Each voxel in the tumor is defined by a scale that includes its image intensity in different sequences. In this case, the imaging sets are from (a) a contrast-enhanced T1 sequence, (b) a fast spin-echo T2 sequence, and (c) a fluid-attenuated inversion-recov- microenvironment can be rewarded by increased proliferation. This evolution- ary dynamic may contribute to distinct differences between the tumor edges and the tumor cores, which frequently can be seen at analysis of cross-sec- tional images (Fig 5). Interpretation of the subsegmenta- tion of tumors will require computa- tional models to understand and predict the complex nonlinear dynamics that lead to heterogeneous combinations of radiographic features. We have ex- ploited ecologic methods and models to investigate regional variations in cancer environmental and cellular properties that lead to specific imaging character- istics. Conceptually, this approach as- sumes that regional variations in tumors can be viewed as a coalition of distinct ecologic communities or habitats of cells in which the environment is governed, at least to first order, by variations in vascular density and blood flow. The environmental conditions that result from alterations in blood flow, such as Figure 4 Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions. REVIEW: Quantitative Imaging in Cancer Evolution and Ecology rise to local-re tations. Pheno sult from epig mosomal rearr turn will affect to therapy. Ch relative abunda communities ov to therapy may which to measu emergence of r Emerging Stra Characterizati A method for c tially explicit tu Figure 4. Here imaging data s are segmented. is defined by image intensity In this case, th microenvironment can be rewarded by increased proliferation. This evolution- ary dynamic may contribute to distinct differences between the tumor edges and the tumor cores, which frequently can be seen at analysis of cross-sec- tional images (Fig 5). Interpretation of the subsegmenta- tion of tumors will require computa- tional models to understand and predict the complex nonlinear dynamics that lead to heterogeneous combinations of radiographic features. We have ex- ploited ecologic methods and models to investigate regional variations in cancer environmental and cellular properties that lead to specific imaging character- istics. Conceptually, this approach as- sumes that regional variations in tumors can be viewed as a coalition of distinct ecologic communities or habitats of cells in which the environment is governed, at least to first order, by variations in Figure 4 Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma mu distribution of MR imaging–defined habitats within the tumor. The blue regio fluid-attenuated inversion recovery) is particularly notable because it presum low blood flow but high cell density, indicating a population presumably adap [5]'QuanGtaGve'imaging'in'cancer'evoluGon'and'ecology,'Gatenby'et'al.,'Radiology,'269(1):8Q15,'2013' 5' global'direcGonal'operators:' local'groupe [6]'RotaGonQcovariant'texture'learning'using'steerable'Riesz'wavelets,'Depeursinge'et'al.,'IEEE'Trans'Imag'Proc.,'23(2):898 MULTIDIRECTIONAL TEXTURE OPERATORS 58 • Isotropic or directional analysis? [DFV2014] • Outex [OPM2002]: 24 classes, 180 images/class, 9 rotation angles in • Texture classification: linear SVMs trained with unrotated images only IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5 1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021 9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035 17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009 Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database. 1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin 9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite. 180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 , 135 and 150 of the other seven Brodatz images for each class. The total number of images in the test set is 672. G. Experimental setup OVA SVM models using Gaussian kernels as K(xi, xj) = ||xi xj ||2 Inc., 2012. The computational complexity is dominated by the local orientation of N c in Eq. 11, which consists of finding the roots of the polynomials defined by the steering matrix A✓ . It is therefore NP–hard (Non–deterministic Polynomial–time hard), where the order of the polynomials is controlled by the order of the Riesz transform N. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 9 10 ini/alcoeffs aligned1sttemplate order of the Riesz transformN classificationaccuracy Riesz wavelets aligned Riesz wavelets Isotropic operators (i.e., ) perform best 
 when not aligned! N = 0 ULTIDIRECTIONAL TEXTURE OPERATORS 58 • Isotropic or directional analysis? [DFV2014] • Outex [OPM2002]: 24 classes, 180 images/class, 9 rotation angles in • Texture classification: linear SVMs trained with unrotated images only IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2013 5 1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021 9) canvas022 10) canvas023 11) canvas025 12) canvas026 13) canvas031 14) canvas032 15) canvas033 16) canvas035 17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009 Fig. 5. 128 ⇥ 128 blocks from the 24 texture classes of the Outex database. 1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin 9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite. 180 ⇥ 180 images from rotation angles 20 , 70 , 90 , 120 , 135 and 150 of the other seven Brodatz images for each class. The total number of images in the test set is 672. G. Experimental setup OVA SVM models using Gaussian kernels as K(xi, xj) = exp( ||xi xj ||2 2 2 k ) are used both to learn texture signatures and to classify the texture instances in the final feature space obtained after k iterations. A number of scales J = 6 Inc., 2012. The computational complexity is dominated by the local orientation of N c in Eq. 11, which consists of finding the roots of the polynomials defined by the steering matrix A✓ . It is therefore NP–hard (Non–deterministic Polynomial–time hard), where the order of the polynomials is controlled by the order of the Riesz transform N. III. RESULTS The performance of our approach is demonstrated with 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 9 10 ini/alcoeffs aligned1sttemplate order of the Riesz transformN classificationaccuracy Riesz wavelets aligned Riesz wavelets Isotropic operators (i.e., ) perform best 
 when not aligned! N = 0
  • 66. THANKS ! BIG @ EPFL
 Michael Unser
 Julien Fageot
 Arash Amini
 and all members MedGIFT @ HES-SO
 Henning Müller
 Yashin Dicente
 Roger Schaer
 Ranveer Joyseree
 Oscar Jimenez
 Manfredo Atzori 66 Source code and data available! https://sites.google.com/site/btamiccai2015/ adrien.depeursinge@epfl.ch PROCESSING FOR BIOMEDICAL IMAGE AN Adrien Depeursinge, PhD MICCAI 2015 Tutorial on Biomedical Texture Analysis (BTA), Munich, O 09), where create real- os can also s problem ic texture’’ 1999), and ature pub- id textures on the fea- since only sis. tals of 3-D ction 3 de- etrieve pa- trieval. The ure are re- us expecta- pplication- d together various ap- Section 7, h a general 1981), it is del of tex- m (Mallat, ches based using sets of prototype primitives. The concept of texture primitive is naturally extended in 3-D as the geometry of the voxel sequence used by a given texture analysis method. We consider a primitive C(i,j,k) centered at a point (i,j,k) that lives on a neighborhood of this point. The primitive is constituted by a set of voxels with gray tone values that forms a 3-D structure. Typical C neighborhoods are voxel pairs, linear, planar, spherical or unconstrained. Signal assignment to the primitive can be either binary, categorical or continuous. Two example texture primitives are shown in Fig. 4. Texture primitives refer to local processing of 3-D images and local patterns (see Toriwaki and Yoshida, 2009). Fig. 2. 3-D digitized images and sampling in Cartesian coordinates. Fig. 3. 3-D digitized images and sampling in spherical coordinates. 2 [iDx,(i + 1)Dx]; [jDy,(j + 1)Dy]; [kDz,(k + Yoshida, 2009). This cuboid is called a voxel. coordinates (r,h,/) are unevenly sampled to Fig. 3. ure primitive has been widely used in 2-D tex- nes the elementary building block of a given k, 1979; Jain et al., 1995; Lin et al., 1999). All pproaches aim at modeling a given texture e primitives. The concept of texture primitive in 3-D as the geometry of the voxel sequence ure analysis method. We consider a primitive point (i,j,k) that lives on a neighborhood of ive is constituted by a set of voxels with gray ms a 3-D structure. Typical C neighborhoods r, planar, spherical or unconstrained. Signal rimitive can be either binary, categorical or mple texture primitives are shown in Fig. 4. fer to local processing of 3-D images and local ki and Yoshida, 2009). d images and sampling in Cartesian coordinates. plexin3-D. varioustaxonomiesare Aclarificationofthetax- ratelydefinethescopeof andYoshida,2009.Three- rearebothgeneraland edinR3 andinclude: in‘‘filled’’objects neratedbyavolumetric raphy,confocalimaging). sof‘‘hollow’’objectsas ionaltimesequencesas accountsfortexturesde- oordinates.Solidtextures meansthatanumberof ftheEuclideanspaceis 1965;Foncubierta-Rodrí- nedastexturedsurfacein na(2004),or2.5-dimen- guetetal.(2008),where -Dobjectsandcanbein- (2)isalsousedinKajiya andHaindl(2009),where eofobjectstocreatereal- nalysisinvideoscanalso xtureanalysisproblem edby‘‘dynamictexture’’ andCrowley(1999),and ewoftheliteraturepub- biomedicalsolidtextures ofthistextisonthefea- gtechniques,sinceonly textureanalysis. Thefundamentalsof3-D Section2.Section3de- ystematicallyretrievepa- ficationandretrieval.The intheliteraturearere- olistthevariousexpecta- Theresultingapplication- edandgroupedtogether dgapsofthevariousap- aregiveninSection7, andtheirassumedinteractionsdefinethepropertiesofthetexture analysisapproaches,fromstatisticaltostructuralmethods. InSection2.1,wedefinethemathematicalframeworkand notationsconsideredtodescribethecontentof3-Ddigitalimages. Thenotionoftextureprimitivesaswellastheirscalesanddirec- tionsaredefinedinSection2.2. 2.1.3-Ddigitizedimagesandsampling InCartesiancoordinates,ageneric3-Dcontinuousimageisde- finedbyafunctionofthreevariablesf(x,y,z),wherefrepresentsa scalaratapointðx;y;zÞ2R3 .A3-DdigitalimageF(i,j,k)ofdimen- sionsMÂNÂOisobtainedfromsamplingfatpointsði;j;kÞ2Z3 ofa3-Dorderedarray(seeFig.2).Incrementsin(i,j,k),correspond tophysicaldisplacementsinR3 parametrizedbytherespective spacings(Dx,Dy,Dz).Foreverycellofthedigitizedarray,thevalue ofF(i,j,k)istypicallyobtainedbyaveragingfinthecuboiddomain definedby(x,y,z)2[iDx,(i+1)Dx];[jDy,(j+1)Dy];[kDz,(k+ 1)Dz])(ToriwakiandYoshida,2009).Thiscuboidiscalledavoxel. Thethreesphericalcoordinates(r,h,/)areunevenlysampledto (R,H,U)asshowninFig.3. 2.2.Textureprimitives Thenotionoftextureprimitivehasbeenwidelyusedin2-Dtex- tureanalysisanddefinestheelementarybuildingblockofagiven textureclass(Haralick,1979;Jainetal.,1995;Linetal.,1999).All textureprocessingapproachesaimatmodelingagiventexture usingsetsofprototypeprimitives.Theconceptoftextureprimitive isnaturallyextendedin3-Dasthegeometryofthevoxelsequence usedbyagiventextureanalysismethod.Weconsideraprimitive C(i,j,k)centeredatapoint(i,j,k)thatlivesonaneighborhoodof thispoint.Theprimitiveisconstitutedbyasetofvoxelswithgray tonevaluesthatformsa3-Dstructure.TypicalCneighborhoods arevoxelpairs,linear,planar,sphericalorunconstrained.Signal assignmenttotheprimitivecanbeeitherbinary,categoricalor continuous.TwoexampletextureprimitivesareshowninFig.4. Textureprimitivesrefertolocalprocessingof3-Dimagesandlocal patterns(seeToriwakiandYoshida,2009). Fig.2.3-DdigitizedimagesandsamplinginCartesiancoordinates. Stanford University
 Daniel Rubin
 Olivier Gevaert
 Ann Leung Dimitri Van de Ville, UNIGE
 Camille Kurtz, Paris Descartes
 Pierre-Alexandre Poletti, HUG
 John-Paul Ward, UCF
  • 67. REFERENCES (SORTING IN ALPHABETICAL ORDER) 67 [Ahonen2009] Ahonen, T.; Matas, J.; He, C. Pietikäinen, M.
 Salberg, A.-B.; Hardeberg, J. Jenssen, R. (Eds.)
 Rotation Invariant Image Description with Local Binary Pattern Histogram Fourier Features
 Image Analysis, Springer Berlin Heidelberg, 2009, 5575, 61-70 [Aerts2014] Aerts, H. J. W. L.; Velazquez, E. R.; Leijenaar, R. T. H.; Parmar, C.; Grossmann, P.; Carvalho, S.; Bussink, J.; Monshouwer, R.; Haibe-Kains, B.; Rietveld, D.; Hoebers, F.; Rietbergen, M. M.; Leemans, C. R.; Dekker, A.; Quackenbush, J.; Gillies, R. J. Lambin, P.
 Decoding Tumour Phenotype by Noninvasive Imaging Using a Quantitative Radiomics Approach
 Nature Communications, Nature Publishing Group, a division of Macmillan Publishers Limited. All Rights Reserved, 2014, 5 [Blakemore1969] Blakemore, C. Campbell, F. W.
 On the Existence of Neurones in the Human Visual System Selectively Sensitive to the Orientation and Size of Retinal Images
 The Journal of Physiology, 1969, 203, 237-260 [Candes2000] Candès, E. J. Donoho, D. L.
 Curvelets - A Surprisingly Effective Nonadaptive Representation For Objects with Edges
 Curves and Surface Fitting, Vanderbilt University Press, 2000, 105-120 [Chenouard2011] Chenouard, N. Unser, M.
 3D Steerable Wavelets and Monogenic Analysis for Bioimaging
 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2011, 2132-2135 [Chenouard2012] Chenouard, N. Unser, M.
 3D Steerable Wavelets in Practice
 IEEE Transactions on Image Processing, 2012, 21, 4522-4533
  • 68. REFERENCES (SORTING IN ALPHABETICAL ORDER) 68 [Dalal2005] Dalal, N. Triggs, B.
 Histograms of Oriented Gradients for Human Detection
 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), IEEE Computer Society, 2005, 1, 886-893 [Depeursinge2012a] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D. Müller, H.
 Multiscale Lung Texture Signature Learning Using The Riesz Transform
 Medical Image Computing and Computer-Assisted Intervention MICCAI 2012, Springer Berlin / Heidelberg, 2012, 7512, 517-524 [Depeursinge2012b] Depeursinge, A.; Van De Ville, D.; Platon, A.; Geissbuhler, A.; Poletti, P.-A. Müller, H.
 Near-Affine-Invariant Texture Learning for Lung Tissue Analysis Using Isotropic Wavelet Frames
 IEEE Transactions on Information Technology in BioMedicine, 2012, 16, 665-675 [Depeursinge2014a] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D. Müller, H.
 Three-Dimensional Solid Texture Analysis and Retrieval in Biomedical Imaging: Review and Opportunities
 Medical Image Analysis, 2014, 18, 176-196 [Depeursinge2014b] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D. Müller, H.
 Rotation-Covariant Texture Learning Using Steerable Riesz Wavelets
 IEEE Transactions on Image Processing, 2014, 23, 898-908 [Depeursinge2015a] Depeursinge, A.; Pad, P.; Chin, A. C.; Leung, A. N.; Rubin, D. L.; Müller, H. Unser, M.
 Optimized Steerable Wavelets for Texture Analysis of Lung Tissue in 3-D CT: Classification of Usual Interstitial Pneumonia
 IEEE 12th International Symposium on Biomedical Imaging, IEEE, 2015, 403-406 [Depeursinge2015b] Depeursinge, A.; Chin, A. C.; Leung, A. N.; Terrone, D.; Bristow, M.; Rosen, G. Rubin, D. L.
 Automated Classification of Usual Interstitial Pneumonia Using Regional Volumetric Texture Analysis in High-Resolution CT
 Investigative Radiology, 2015, 50, 261-267
  • 69. REFERENCES (SORTING IN ALPHABETICAL ORDER) 69 [Dumas2009] Dumas, A.; Brigitte, M.; Moreau, M.; Chrétien, F.; Baslé, M. Chappard, D.
 Bone Mass and Microarchitecture of Irradiated and Bone Marrow-Transplanted Mice: Influences of the Donor Strain
 Osteoporosis International, Springer-Verlag, 2009, 20, 435-443 [Galloway1975] Galloway, M. M.
 Texture Analysis Using Gray Level Run Lengths
 Computer Graphics and Image Processing, 1975, 4, 172-179 (Gal1975) [Gerlinger2012] Gerlinger, M.; Rowan, A. J.; Horswell, S.; Larkin, J.; Endesfelder, D.; Gronroos, E.; Martinez, P.; Matthews, N.; Stewart, A.; Tarpey, P.; Varela, I.; Phillimore, B.; Begum, S.; McDonald, N. Q.; Butler, A.; Jones, D.; Raine, K.; Latimer, C.; Santos, C. R.; Nohadani, M.; Eklund, A. C.; Spencer-Dene, B.; Clark, G.; Pickering, L.; Stamp, G.; Gore, M.; Szallasi, Z.; Downward, J.; Futreal, P. A. Swanton, C.
 Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing
 New England Journal of Medicine, 2012, 366, 883-892 [Gurcan2009] Gurcan, M. N.; Boucheron, L. E.; Can, A.; Madabhushi, A.; Rajpoot, N. M. Yener, B.
 Histopathological Image Analysis: A Review
 IEEE Reviews in Biomedical Engineering, 2009, 2, 147-171 [Haidekker2011] Haidekker, M. A.
 Texture Analysis
 Advanced Biomedical Image Analysis, John Wiley Sons, Inc., 2010, 236-275 [Haralick1979] Haralick, R. M.
 Statistical and Structural Approaches to Texture
 Proceedings of the IEEE, 1979, 67, 786-804
  • 70. REFERENCES (SORTING IN ALPHABETICAL ORDER) 70 [Khan2013] Khan, A. M.; El-Daly, H.; Simmons, E. Rajpoot, N. M.
 HyMaP: A Hybrid Magnitude-Phase Approach to Unsupervised Segmentation of Tumor Areas in Breast Cancer Histology Images
 Journal of pathology informatics, Medknow Publications, 2013, 4 [Kumar2012] Kumar, V.; Gu, Y.; Basu, S.; Berglund, A.; Eschrich, S. A.; Schabath, M. B.; Forster, K.; Aerts, H. J. W. L.; Dekker, A.; Fenstermacher, D.; Goldgof, D. B.; Hall, L. O.; Lambin, P.; Balagurunathan, Y.; Gatenby, R. A. Gillies, R. J.
 Radiomics: The Process and the Challenges
 Magnetic Resonance Imaging, 2012, 30, 1234-1248 [Lazebnik2005] Lazebnik, S.; Schmid, C. Ponce, J.
 A Sparse Texture Representation Using Local Affine Regions
 IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, 2005, 27, 1265-1278 [Malik2001] Malik, J.; Belongie, S.; Leung, T. Shi, J.
 Contour and Texture Analysis for Image Segmentation
 International Journal of Computer Vision, Kluwer Academic Publishers, 2001, 43, 7-27 [Mosquera2015] Mosquera-Lopez, C.; Agaian, S.; Velez-Hoyos, A. Thompson, I.
 Computer-Aided Prostate Cancer Diagnosis From Digitized Histopathology: A Review on Texture-Based Systems
 IEEE Reviews in Biomedical Engineering, 2015, 8, 98-113 [Ojala2002] Ojala, T.; Pietikäinen, M. Mäenpää, T.
 Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns
 IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24, 971-987
  • 71. REFERENCES (SORTING IN ALPHABETICAL ORDER) 71 [Petrou2006] Petrou, M. Garcia Sevilla, P.
 Image Processing: Dealing with Texture
 Wiley, 2006 [Petrou2011] Petrou, M.
 Deserno, T. M. (Ed.)
 Texture in Biomedical Images
 Biomedical Image Processing, Springer-Verlag Berlin Heidelberg, 2011, 157-176 [Portilla2000] Portilla, J. Simoncelli, E. P.
 A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients
 International Journal of Computer Vision, Kluwer Academic Publishers, 2000, 40, 49-70 [Romeny2011] ter Haar Romeny, B. M.
 Deserno, T. M. (Ed.)
 Multi-Scale and Multi-Orientation Medical Image Analysis
 Biomedical Image Processing, Springer-Verlag Berlin Heidelberg, 2011, 177-196 [Sifre2014] Sifre, L. Mallat, S.
 Rigid-Motion Scattering for Texture Classification
 Submitted to International Journal of Computer Vision, 2014, abs/1403.1687, 1-19 [Simoncelli1995] Simoncelli, E. P. Freeman, W. T.
 The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation
 Proceedings of International Conference on Image Processing, 1995, 3, 444-447 [Storath2014] Storath, M.; Weinmann, A. Unser, M.
 Unsupervised Texture Segmentation Using Monogenic Curvelets and the Potts Model
 IEEE International Conference on Image Processing (ICIP), 2014, 4348-4352
  • 72. REFERENCES (SORTING IN ALPHABETICAL ORDER) 72 [Unser2011] Unser, M.; Chenouard, N. Van De Ville, D.
 Steerable Pyramids and Tight Wavelet Frames in 
 IEEE Transactions on Image Processing, 2011, 20, 2705-2721 [Unser2013] Unser, M. Chenouard, N.
 A Unifying Parametric Framework for 2D Steerable Wavelet Transforms
 SIAM Journal on Imaging Sciences, 2013, 6, 102-135 [Varma2005] Varma, M. Zisserman, A.
 A Statistical Approach to Texture Classification from Single Images
 International Journal of Computer Vision, Kluwer Academic Publishers, 2005, 62, 61-81 [Ward2014] Ward, J. Unser, M.
 Harmonic Singular Integrals and Steerable Wavelets in
 Applied and Computational Harmonic Analysis, 2014, 36, 183-197 [Ward2015] Ward, J.; Pad, P. Unser, M.
 Optimal Isotropic Wavelets for Localized Tight Frame Representations
 IEEE Signal Processing Letters, 2015, 22, 1918-1921 [Xu2012] Xu, J.; Napel, S.; Greenspan, H.; Beaulieu, C. F.; Agrawal, N. Rubin, D.
 Quantifying the Margin Sharpness of Lesions on Radiological Images for Content-Based Image Retrieval
 Medical Physics, 2012, 39, 5405-5418 L2(Rd ) L2(Rd )