Hyperspectral imaging can be used to detect early signs of cobweb disease on mushroom caps that are invisible to the naked eye. A study found that infected areas lost water from the first day of infection, unlike areas with mechanical injuries. Linear discriminant analysis could classify infection types and support vector machines identified untreated samples and those treated with different antifungal methods, including biological and synthetic pre-treatments. However, the models rely on water absorption bands, so drying from other causes could interfere with detection. Comparisons across sample sets from different sources may also be less reliable. Hyperspectral imaging shows promise for early detection but real-world applications face challenges around sample variability.

1. Hyperspectral imaging on Food
data acquisition and processing tools
some beneficial application field
Ferenc Firtha – Corvinus University of Budapest
Faculty of Food Science,
Physics and Control Department

2. Introduction: Zeutec, 2003  Tulln, Austria + Gödöllő, Hungary
We had to
• control
• get signal
• calibrate
• process
• apply

3. 1. Colour: what looks like?
quick, but contact :
-> average RGB/Lab/Lch
remote sensing + data reduction:
-> position: colour, shape, pattern
2. Image processing: where?
4. Spectral imaging: where and what?
remote + stat. analysis + image processing
-> position: distribution of compounds
contact + statistical analysis:
NIR -> water, fat, oil, protein,…
3. Spectroscopy: what?
What is spectral imaging:

4. lipids:
triacilglicerin (fat)
alcohol:
metil,etil,propil…
protein
water free / bound:
HDW (hydrofil) / LDW (hydrofob)
amid, amin
secondary amid
aromatic alcohol:
e.g. benzyl
What to measure in
NIR 900 – 1700 nm?
OH: 970, 1450, 1980
Fiber: 1100, 1300, 1350, 1403, 1483, 1500, 1534
Cellulose: 1490
Lignin (wood): 1170, 1410, 1417, 1420, 1440
aromatic hydrocarbon:
e.g. benzene

5. Hyperspectral Imaging vs NIR spectrometer
Advantages:
1. remote sensing
no need for preparation
2. inspecting non‐homogeneous surface
areas or pattern might hold information
See invisible: Water fingerwriting
becomes visible on 1450nm image
Conclusion: algorithms are needed to
1. help calibration and control measurements
2. correct 3D effect, preprocess and reduce data
3. analyse data for building final multispectral application
Disadvantages:
• non‐izolated, noisy circumstances
• setup dependant calibration (distance, illumination)
• indefinite geometry (illumination/observation angle)
• huge amount of data (50MB/frame) must be processed

6. Hyperspectral software tools
for controlling measurement and preprocessing
Some beneficial application
for food quality and safety assessment
Statistical methods and
industrial application

7. 1. Hardware: SWIR (900-1700nm) push-broom hyperspectral setup
HeadWall: Xenincs camera, Specim spectrograph
halogen, 45/0 geometry
3. lens zooming
2. altitude of lens
1. plane of focus
camera: 320*167, A/D 14bit
spectrograph, 25µm slit: 5nm
NIR lens: F/2 (fast)
field of view is
determined by
Y-table moved by stepping motor and gear

8. 2. Software for controlling measurement: Argus (100‐eyed giant)

9. Opening, AD parameters, tuning focus: ARGUS left panel
open NIR sensor
and calibration file containing basic parameters
integration time and gain to get optimal signal level
AD properties to get optimal range (mean and width)
sensor cooling
actual R(x,b) frame as a grayscale image (14bit->8bit)
- spectral and spatial region of interest (ROI)
is shown by red rectangle
- spectral (gray) and spatial (yellow) cross sections
at selected point
histogram of signal (14bit:16’383 level)
to check proper range of AD conversion
spatial cross section of frame
to check homogenity of illumination
to tune focus plane of lens by contrast

10. Calibration, data acquisition: ARGUS middle panel
spectral crosssection of frame:
reflectance spectum (yellow)
between bright and dark signal
2-point spectral calibration
spectral and spatial ROI
spatial calib  px size
saving bright/low surfaces
 reflectance factor
 absolute reflectance
controlling Y-table
go to anywhere and back
set Y length
and start measurement
Let’s see the result of
• proper AD properties
• spectral calibration
• optimal signal level
2 bands can be identified
by their wavelengthes

11. Absorbance of REMO (rare earth metal oxides) reflection standard
Absorbance measured by HSI system (non-isolated circumstances)
Wavelength
Rare-earth
oxide
1064,92 Dy2O3
1132,21 Ho2O3
1261,87 Dy2O3
1321,33 Dy2O3
1478,07 Er2O3
1535,92 Er2O3
1643,34 Dy2O3
1682,7 Dy2O3

12. Display measured hypercube: ARGUS right panel
e.g. parrot food 1.) single band cross‐section:
3.) 3‐channels pseudo RGB: 4.) linear combination (R1600‐R1000nm):2.) scrolling bands:

13. 3. Display hypercube and preprocess data: CuBrowser (Matlab) algorithm
• for browsing hypercube
• manual selection of ROIs
• display and save average spectra and derivatives

14. browsing hypercube: to show cross‐section, select ROIs and display average spectra
1. remove extreme values (salt&pepper)
2. display cross-section
3. manual selection of ROI
4. red: selection of even a pixel
5. Mean+-Variance and Min and Max
6. Get info on one data of spectrum
reflection spectra

15. 3D correction: how to handle indefinite geometry
spectra along radius are similar
r
shaded image
on 1329nm
Normalization on pixels:
• subtraction of average
• division by average
homogeneous image
shifted or supressed

16. Derivatives, saving: smoothing, transform reflectance spectra to absorbance and save
We are ready to assign
significant wavelengths

17. Hyperspectral software tools
for controlling measurement and preprocessing
Some beneficial application
for food quality and safety assessment
Statistical methods and
industrial application

18. 4. Statistical analysis of spectral data
a.) Principal Component Analysis (PCA): Dimension reduction (not supervised)
Finds the main axes (eigenvalues) of data space, those separate best data points.
These PCs come as the linear combination of n dimensional source space.
PCA:
b.) Fisher’s Discriminant Analysis (FDA): Dimensionality reduction and classification
Finds a linear combination of features, which separates two or more classes.
Steps: finds linear/quadratic classifier -> dimensionality reduction -> classification
• Analysis of Variance (ANOVA): categorical independent and continuous dependent variables
• Fisher’s Discriminant Analysis (FDA): continuos independent and categorical dependent variables
• Discriminant Correspondence Analysis: categorical independent and categorical dependent variables
• Partial Least Squares (PLS) continuous independent and continuous dependent variables
LDA: QDA:
[loadings,scores] = princomp(X); % coeff of linear combinations
[Z,W] = FDA(X, Y, 2); % dimensionality reduction by FDA script
cqs = fitcdiscr(X,Y,'DiscrimType','quadratic'); % create classifier

19. c.) Partial Least Squares (PLS) regression builds a linear modell between
• X source space (independent variables) and absorbance on different bands
• Y target space (dependent, predicted variables) like moisture, fat, protein content
Inside, it makes a PCA on X space, a PCA on Y space, then builds a linear regression
between the first p dim (latent variables, factors) of two PCA spaces.
The optimal number of latent variables are
determined by cross validation (building
model on calibration data set, then checking
prediction on validation set) on the base of
minimal Root Mean Squared Error of Prediction
(RMSEP):
n
oy
RMSEP ii 

2
)(
number of latent variables
[XL,YL, XS,YS, beta, PCTvar, mse] = …
plsregress(X,Y, LVno, 'cv',20, 'mcreps',10000);

20. The coefficient of determination (r2) characterizes the efficiency of PLS model.
The significant wavelengths can be assigned by the loading values of regression.
Loading values of enzym and fat content in cheese

21. d.) Partial Least Squares Discriminant Analys (PLS DA): variant for classification
PLS-DA consists in a classical PLS regression,
where the response variable is a categorical one (replaced by the set of dummy
variables describing the categories) expressing the class membership.
PCA space is rotated such that a maximum separation among classes is obtained,
and to understand which variables carry the class separating information. (Camo)
3D score plot of a two-class PLS-DA model of
GREEN versus RED/BLUE:
e.) Orthogonal PLS DA (OPLS-DA)
Class-orthogonal variation is combined
with traditional PLS-DA.
It gives better performance if such
within-class variation exists.
(J.of Chemometrics)
pls_model = pls(x,y,vl,'da');
Matlab toolboxes, like Eigenvector
other chemometric tools: SIMCA-P, Unscrambler, R (gnu), …

22. 5. Industrial application: does not use expensive HSI
Multispectral sensors:
logistic function:
HIDDEN
Artificial neural networks (ANN):
used to connect some input cells (sensors) with some output cells (actuators).
• like statistical models they are teached on calibration set, then tested on validation set
• contrary to statistical models they use non-linear relations, with much more efficiency
ANN is a black box. We don’t exactly know, how it works, but it works well.
They are used therefore mostly not in scientific work, but for industrial applications.
Multilayer back-propagation neural network (MBPN):

23. Hyperspectral software tools
for controlling measurement and preprocessing
Some beneficial application
for food quality and safety assessment
Statistical methods and
industrial application

24. seeing invisible ‐> by comparing areas inspecting changes through packaging
handling 3D can also be advantage spatial distribution to fructose content

25. 1. Invisible: Early detection of cobweb disease on champignon caps
(Viktória Parrag – Felföldi ‐ Firtha)
Reflection of dactylium infected and control areas
a. infected spots loss water from the first day
(contrary to mechanical injuries)
Linear discriminant analysis (LDA) plot of dactylium
or trichoderma infected and control samples
infection types can be classified by LDA
(true within one sample set)
Method: Follow healthy, injured and infected
areas back to the first day

26. b. Even antifungal pre‐treatments can be identified
1. untreated
2. Natamycin : biological
3. Prochoraz‐manganese: synthetic
4. Bacillus Subtilis : biological
untreated Natamycin Prochoraz‐Mn B. Subtilis
Dactylium 88.46% 75.16% 83.21% 61.11%
control 89.76% 53.88% 14.29% 71.61%
Classification by Support Vector Machine (SVM):
biological
synthetic
Optimistic: Infection and antifungal pre‐treatment can be identified
Realistic:
1. If the modell uses water band, drying might occur as the result of several other causes
2. Different sample sets are less comparable: different genotype, breed, age, region, storage, humidity
control
Spots show the infection, not the average spectra
1. significant wavelenght should be assigned first
2. areas segmented by image processing method
3. spot shape and spectral differences together
will identify the infection dactilium: 1000-1450nm

27. 2. Tracking enzymes in cheese along aging (through packaging)
(Lívia Darnay ‐ Firtha)
8 groups (enzyme/control, 4 fat content) during 4‐10 weeks of aging.
Development of structure was also checked via texture properties.
LDA for enzyme contentPLS for fat content
PLS for aging time
choosing significant wavelegths
Quadratic DA for enzyme
using only 2 wavelengths:
1387nm and 1134nm

28. Classification of different cheeses. Checking the effect of storage temperature
(Flóra Králik ‐ Firtha)
9 types (3 group) of norvegian cheeses
were stored on 4 temperature.
• Spectra were measured along
cross‐sections during storage.
• Structural properties were
measured finally by texture analyser LDA for 3 groups LDA for 9 types
Optimal storage temperature can be specified on the base of
• spectral and rheological changes
• PLS models
PLS for days stored
camember: R2=0.97

29. 3. Handling indefinite geometry means advantage
Normalization methods of spectra:
• subtraction of average (shifting)
• division by average (streching)
Which method is optimal?
Depends on the object type
shaded image on 1329nm normalized image

30. How to measure moisture content of tea leaves
(Firtha)
Right half of tea leaves were in humid air for 1 hour
(only 1 drop water in the chamber)
type1: dry
type2
type3
wet
high variance
no clear singal
Normalized reflectance at 1459nm
low variance
clear singal
Using pixel normalization results higher signal, then a spectrometer

31. 4. From spatial distribution to fructose content in marzipan
(Szabina Németh – Katalin Kerti ‐ Firtha)
In marzipan invertase converts sucrose into glucose and fructose.
The mixture of sucrose, glucose and fructose has a lower viscosity
and shows less tendency to crystallize than sucrose alone.
So the product stays softer.
Unfortunately the produced fructose
cannot be detected by its NIR spectra.
Polarimetric measurement could make difference
between components, but only in solution form,
because reflection also disperses the polar plane.
sacharose 66.56
glükóz (38% alfa + 62% béta) 52.61
fructose ‐93.72
invert sugar (syrup) ‐39.7
How to measure fructose content?
Fructose is hygroscopic: it attracts and binds water.

32. Normal material: cooling model
Moisture distribution: Cosine‐like
  
)cos(),(
2
XeAX n
t
n
n
 
Dimensionless numbers:





TT
TtxT
0
),(
:
L
x
X :
Solution:
In case of hygroscopic material, like fructose:
constant in the middle and changing on the edges only
destructive:
only edges drying
contact:
only edges drying
optical:
only edges drying

33. 5. Bonus example: Meat marbling: classifying lean, IMCT and fat
(Ferenc Firtha)
Boston butt (shoulder, blade roast):
intermuscular fat tissue
between muscles
intramuscular connective tissue
between fibres
+ fat deposite
absorbance at 1600nm
zooming
spectra of lean - IMCT - fat

34. Thank you very much for your attention
Hyperspectral imaging, on Food
hardware, software tools, some applications
dactilium: 1000-1450nm
Distribution -> Fructose in marzipan
Spots -> Early detection of fungal infection Enzym, fat, type, optimal storage
3D correction -> Moisture in tea leaves