Spectra from Correlation

pectra
S
from Correlation
Peter Morovič, Ján Morovič, Juan Manuel García–Reyero 
Hewlett–Packard Española S. L., Barcelona, Catalonia, Spain
Presented on 8th November 2013 at 21st IS&T/SID Color and Imaging Conference, Albuquerque, NM
© Copyright 2013 Hewlett-Packard Development Company, L.P.

utline
O
Background

Conclusions

Spectral Correlation

Acknowledgements

Correlation Proﬁle
Generating Spectra based on
Correlation Proﬁle
Relation to Multivariate Analysis


ackground
B
Spectral Reflectance Studies
•

Multivariate Analysis (MVA)

•

Data Collection to feed MVA

•

MVA to synthesise linear model bases for Data

•

“The set of all reflectances” question

General Assumptions
•

MVA relies on redundancy in data

•

If Reflectances were perfectly/uniformly random variables at each
wavelength, we could not find bases that reduces the dimensionality

•

Linear model bases represent the axes of variation well, but loose
the boundedness of the data - arbitrary linear model weights can yield
reflectances that are outside the domain of the data used to derive them


tHe human demOsaicing Agorithm
David Brainard – CIC 2011 Keynote
•

Bayesian model used to reverse engineer the
human visual system (HVS)

•

Based on data from the birthplace of the HVS
•

Digital Camera (RGB) capture

•

Spectral reflectance measurements

Table 2. Album

cd03b

cd05b

sausage

cd08b

elephant

cd09b

old figs

cd10b

fresh fig

cd11b

old jacke

woods, g

fresh bu

cd14b

fresh jac

cd15b

Figure 3. Pairwise correlations in natural scenes. We analyzed 23
images of the same grass scrub scene, taken from different distances
(black – smallest distance, red – largest distance). For every image, we

marula n

cd13b


sticky gr

cd07b

Spectra/Reflectances low-dimensional based on
MVA analysis

scrub, gr

cd12b

•

salt depo

cd04b

RGBs spatially correlated

sand, gro

cd06b

•

dirt, grou

cd02b

Data found to have different kinds of correlation

Keywor

cd01b

•

Album

semiold

cd16b

old palm

cd17b

fresh pa

d
f
,
d
s

d
e
r
s
l

o
f
l
s
h
o
e
e
o

ing correlation and characterizing its specific behavior.
i
Spectral correlation is understood to be the relationship of R(λ )
i+1
i
i+1
against R(λ ), where R() denotes reflectance and λ and λ are
the wavelengths of neighboring intervals in nanometers. Such relationships are Spectral correlation is Fig. relationshipthem for against R(λi+1), where R() denotes
easily visualized, with the 1 showing of R(λi) the
Definition:
SOCS dataset of 53489 measured samples, with pseudo-colored
i
i+1
reflectance and λ and λ are wavelengths of neighbouring intervals in nanometers.
dots indicating their respective wavelengths.

sPectral Correlation

SOCS data set containing
53489 reflectances of different
surface kinds, represented at 16
sample spectral points: 400nm
to 700nm at 20nm steps.

The data seems to be clearly
highly correlated but 
it’s not a trivial relation… 
Is there more to it?
i+1

Figure 1. Correlation plot of the SOCS reflectance data set plotting R(λ )


reconstruction error is to be below 0.5 ∆E*ab (Kohonen, 2006).
It is also apparent from Fig. 1 that there are biases and outliers and
that not all wavelengths have an equal spread along the diagonal
axis. A wavelength-by-wavelength view (Fig. 2) shows the differences between individual correlations in more detail.

neighBouring waveleNgths
Not all wavelengths are
equal… why?
•

E.g. [440 - 460]nm vs [500 520]nm?

•

Prime Wavelengths/Crossover
wavelengths?

•

Measurement (multiple device)
artefacts?

•

Lower sensitivity at extremes of
visible range?

•

Some bias to increasing
reﬂectance: more points above
identity than below
i+1


Figure 2. Correlation plot of the SOCS reflectance dataset plotting R(λ )
i
against R(λ ) for each wavelength from 400nm (showing the relationship be-

summarY correl@ion profile
Distributions
characteristics
•

Median constant ~0

•

Per-wavelength correlation
has narrow peak

•

90% of the distribution
occupies a small range

•

Top/bottom 5% could be
noise? Different
measurement instrument
artifacts?


λ max] values per wavelength as shown above in Eq. (1). For a simple case where both min and max are fixed and constant at 0.1
along the wavelength range Fig. 7 shows an example of reflectances that satisfy both the constraint of correlation and physical realisability.

syNthesizing reflec|nces
Simple example:
•

Every neighbouring
wavelength is related
to the the previous
one +/- 0.1

•

What does its
correlation proﬁle look
like?

•

So…how do we
generate
relfectances given a
correlation proﬁle?


Figure 7. Synthetic reflectances with constant, wavelength independent corre-


The per-wavelength correlation profile of the above data set is then
shown in Fig. 8, and as expected shows a synthetic and regular
distribution (compare against that of the SOCS data set in Fig. 2).

Simple example:
•

Every neighbouring
wavelength is related
to the the previous
one +/- 0.1

•

What does its
correlation proﬁle look
like?

•

So…how do we
generate
relfectances given a
correlation proﬁle?


Figure 8. Per-wavelength correlation plot of synthetic reflectances with con-

under half of all generated samples and the time to generate this
entire set is ~140 ms on a 2.66 GHz Intel Core i7 with 8GB RAM.
For a real-world example instead, the correlation profile of the
SOCS data set is used below to generate reflectances as outlined
above. The filtered correlation profile here is that shown in Fig. 3
above and Fig. 9 shows the ‘forward’ direction (Formula (2)) and
‘reverse’ direction (Formula (3)) of the synthesized reflectances.


A simple algorithm:
•

A spectral correlation profile is defined as a
[N-1 x 2] matrix S such that: 
min
max
Sλi = [Sλi Sλi ]

Synthetic reflectances with constant, wavelength independent corre• Given any (random or not) value of reflectance
.1.

Rj(λi), the next value of Rj(λi+1) should be in the
range of: 
min
max
wavelengthj(λi+1) ∈ [Rj(λi) - Sλiof the jabove Sλi set is then
R correlation profile
, R (λi) + data ]
n Fig. 8, and as expected shows a synthetic and regular
• To envelope the values, for any reflectance Rj
ion (compare against that of the SOCS data set in Fig. 2).
at wavelength λi we generate two reflectances
R’j and R’’j at λi+1: 
min 
R’j(λi+1) = Rj(λi) - Sλi
max
R’’j(λi+1) = Rj(λi) + Sλi
•

Start with a regular grid of seed values at
400nm, e.g.: [0, 0.2, 0.4, … , 1] and build our
way to 700nm and do the same in reverse,
start at 700nm and work back to 400nm


ion (compare against that of the SOCS data set in Fig. 2).


A simple algorithm:
•

A spectral correlation profile is defined as a
[N-1 x 2] matrix S such that: 
min
max
Sλi = [Sλi Sλi ]

•

Given any (random or not) value of reflectance
Rj(λi), the next value of Rj(λi+1) should be in the
range of: 
min
max
Rj(λi+1) ∈ [Rj(λi) - Sλi , Rj(λi) + Sλi ]

Per-wavelength correlationthe values, for reflectances with con• To envelope plot of synthetic any reflectance Rj
elength independent correlation at 0.1.

at wavelength λi we generate two reflectances
R’j and R’’j at λi+1: 
min 
R’j(λi+1) = Rj(λi) - Sλi
the initialR’’ (λ for= R (λ ) + Sthe reflectances were values
seed ) generating max
j i+1
j i
λi
2, 0.4, 0.6, 0.8, 1] at 400nm and each subsequent waveStart with follows:
as then•generated as a regular grid of seed values at
! 400nm, e.g.: ![0, 0.2, 0.4, … , 1] and build our
!
! !
! ! + !!"#
=
(2)
!
! way to 700nm and do the same in reverse,
!
! !
! ! − !!"#
start at 700nm and work back to 400nm
i
i
i
(λ ) is the set of all partial reflectances up until λ (i.e. λ

i

Figure 9. Forward (top) and reverse (bottom) direction of synthesized reflectances based on the SOCS correlation profile.

The above procedure results in an exhaustive, fully descriptive set
of reflectances that envelopes the original data set defined by the
i
i
[λ min, λ max] ranges. Fig. 10 shows the first two such data sets starting at 400nm and 700nm using the SOCS correlation profile.


here.

Alternative (more complete) strategy: start with [0 1] at every wavelength and generate
reﬂectances in both directions – results in full spectral convex hull at minimal number of samples.

inations of perre sufficient to
on of convexity
ince colorimetry
mples, any samlinear combinad in terms of the
his way a linear
inear model baper-wavelength
color and specbe thought of as
relation method
A maximizes de-


Figure 10. Synthesized reflectancesstart with [0 1] at every wavelength and generate
based on the SOCS correlation profile for
Alternative (more complete) strategy:
the reﬂectancesinitial seed values of– results in full (top) and 700nm (bottom).
intrain both directions [0 1] at 400nm spectral convex hull at minimal number of samples.

nores
s related to the


•

Sampling

appLcations

•

•

Can respect per-wavelength distribution or gaussian fit
to per-wavelength correlation statistics, not just range

•

•

Given a (small) set of representative measurements,
compute the correlation profile and generate ‘random’
reflectances that follow the profile

More efficient than sampling in PCA basis space where
vast majority of random linear model weights (samples)
are out of convex hull of original data

Analysis
•

•

Given a correlation profile from previous data, see how
new measured data fits with the correlation profile?

Priors
•

A natural way to design reflectance/spectral priors


i
λ max]

values per wavelength as shown above in Eq. (1). For a simple case where both min and max are fixed and constant at 0.1
along the wavelength range Fig. 7 shows an example of reflectances that satisfy both the constraint of correlation and physical realisability.

relatioNship to mVa

and in memory req
ed, the same proc
initial seed values
400nm with the ran
!
! !
!!!
! !
=
!
! !
In this synthetic ex
correlation differen
total number of ref
under half of all g
entire set is ~140 m
For a real-world e
SOCS data set is u
above. The filtered
above and Fig. 9 s
‘reverse’ direction

Figure 7. Synthetic reflectances with constant, wavelength independent correSynthetic (constant +/- 0.1 neighboring wavelentgh difference) example
lation of 0.1.

synthetized for a ‘flat’ correlation profile, like the one shown in
Figs. 7 and 8 where the correlation bounds are a constant ±0.1. Fig.
14 therefore shows the first five bases of that set of correlationsynthetizes spectra, which account for 99.1% of their variance.

relatioNship to mVa

top) and per wavelength
nerated reflectances,
0

h to the original data set
reflectance values are
] of neighboring waveof absolute reflectance
e needed. Results using
400
450
500
550
600
650
700
er.
SyntheticFigure 14. PCA bases of spectra synthesizes using a constant ±0.1 correla(constant +/- 0.1 neighboring wavelentgh difference) example
measured and synthetic
tion profile.
e analysis and to com© Copyright 2013 Hewlett-Packard Development Company, L.P.

pute their principal component bases. Fig. 12 therefore shows the
first five SOCS bases both for the measured (accounting for 99.7%
variance) and the synthetic data (accounting for 99.8% variance).

relatioNship to mVa
0

0

400

450

500

550

600

650

700

400

450

500

550

600

650

700

Figure 12. PCA bases of PCA of SOCS(left) and synthetic (right) SOCS spectra.
measured reflectances (left) vs 
PCA of synthesised reflectances from SOCS correlation profile (right).

Spectral correlation

cOnclusions

•

A new way to analyse reflectance data that preserves the
spectral correlation profile

•

A way to extract a correlation profile and use it to
generate reflectances that maintain it

•

Ability to synthetically define correlation profile and
generate reflectances accordingly

•

Elegant way to sample reflectance domain

•

Initial thoughts on a relationship to traditional MVA

Next steps
•

Study the relationship of spectral correlation and PCA bases
in more detail

•

Use specific spectral correlation profile in bayesian methods
as reflectance priors


Carlos Amselem
Jordi Arnabat
David Brainard
David Gaston
Rafael Giménez

tHank You

Spectra from Correlation

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Similar to Spectra from Correlation

Similar to Spectra from Correlation (20)

More from Peter Morovic

More from Peter Morovic (7)

Recently uploaded

Recently uploaded (20)

Spectra from Correlation