The document presents a multi-frame marked point process model for extracting targets from ISAR (Inverse Synthetic Aperture Radar) image sequences. The model integrates information across frames using priors on target shape persistency and smooth motion. Experiments show the model achieves better target line and center extraction compared to frame-by-frame detection. Future work involves generalizing the model to identify other objects like airplanes and using extracted features for target classification.

1. ISAR Image Sequence based Automatic Target
Recognition by using a Multi-frame Marked Point
Process Model
Csaba Benedek1 Marco Martorella2
1 Distributed Events Analysis Research Group
Computer and Automation Research Institute, Hungary
2 University of Pisa, Department of Information Engineering
Work partially funded by the APIS Project of EDA
IGARSS 2011, Vancouver, Canada

2. Content
1 Introduction
2 Multiframe Marked Point Process Model
Model elements and configuration energy
Optimization
3 Experiments
4 Future steps and conclusions
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 2 / 19

3. Introduction
Content
1 Introduction
2 Multiframe Marked Point Process Model
Model elements and configuration energy
Optimization
3 Experiments
4 Future steps and conclusions
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 3 / 19

4. Introduction
Introduction
Extracting ship scattering centers in airborne Inverse Synthetic
Aperture Radar (ISAR) image sequences
Framework: “Array Passive ISAR Adaptive Processing” (APIS)
Project of EDA
ISAR images in Automatic Target Recognition (ATR) systems
applicable where other imaging techniques (e.g. SAR) fail
post processing step after detection & imaging
frames have different quality parameters (e.g. image focus)
Goals:
Measuring relevant features for target identification and behaviour
analysis
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 4 / 19

5. Introduction
Proposed approach
Robust multi-frame technique, integrating the noisy image
information with prior constraints of target shape persistency and
smooth motion.
Multiframe Marked Point Process model
Input:
ISAR image sequence of the detected target
Output:
center line segment parameters of the target in each frame
length and orientation
positions of permanent characteristic feature points
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 5 / 19

6. Multiframe Marked Point Process Model
Content
1 Introduction
2 Multiframe Marked Point Process Model
Model elements and configuration energy
Optimization
3 Experiments
4 Future steps and conclusions
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 6 / 19

7. Multiframe Marked Point Process Model Model elements and configuration energy
Configuration model: notations
Observation: n-frame-long ISAR
image sequence
S: joint pixel lattice of the ISAR
frames, s ∈ S: a single pixel
Bt : binarized input image observed
at time frame t ∈ {1, 2, . . . , n}
Bt (s) ∈ {0, 1}: value of pixel s in Bt
ut ∈ H: a target candidate in frame t
Goal: extract a sequence of objects:
ω = {u1 , u2 , . . . , un } ∈ H n
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 7 / 19

8. Multiframe Marked Point Process Model Model elements and configuration energy
Target modeling in a single ISAR frame
Parameters describing a target u:
c(u) = [x (u), y (u)] center pixel, l(u) length and θ(u) orientation
Misalignment problem
periodicity of ISAR images both in horizontal and vertical directions
target may break into two/four pieces
using a duplicated mosaic image
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 8 / 19

9. Multiframe Marked Point Process Model Model elements and configuration energy
Fm MPP energy function
Object sequence or configuration:
ω = {u1 , u2 , . . . , un }
Configuration energy:
n n−1
ΦD (ω) = AD (ut ) +γ · I (ut , ut+1 )
t=1 t=1
AD (ut ): D-data dependent unary object potential
I (ut , ut+1 ) prior interaction potential function between objects of
consecutive frames
Maximum Likelihood (ML) configuration estimate:
ω = argmin ΦD (ω)
ω∈H n
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 9 / 19

10. Multiframe Marked Point Process Model Model elements and configuration energy
Data term
Unary potentials:
evaluation of proposed ship
candidates in independent frames
Calculation:
 
1
AD (ut ) = Q  Bt (s) + (1 − Bt (s))
Area{Rut ∪ Tut }
s∈Rut s∈Tut
Q(ζ) : R → [−1, 1]: a non-linear monotonously decreasing function
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 10 / 19

11. Multiframe Marked Point Process Model Model elements and configuration energy
Interaction potentials
Key role: enforcing prior geometrical constraints.
persistent frame rate → small object displacements between two
consecutive frames
Feature: length and angle difference (center is not relevant)
I (ut , ut+1 ) = δθ · |θ (ut ) − θ (ut+1 )| + δl · |l (ut ) − l (ut+1 ) |
Penalized configuration ×
√
Favored configuration
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 11 / 19

12. Multiframe Marked Point Process Model Optimization
Optimization: iterative stochastic algorithms
The algorithm
1 Start with a frame-by-frame initialization process
Hough transform based line estimation in each binarized frame Bt ,
t = 1...n
2 Iterate object perturbation and acceptance steps till convergence is
obtained in the extracted object sequence
Object perturbation: for each t we propose an object u ∗ which is the
random perturbation of ut−1 OR ut OR ut+1
Acceptance: we accept or reject a move replacing ut width u ∗
Important properties:
Acceptance: inverse approach considering simultaneously data
and prior features
Stochastic process both for object perturbation and acceptance
(unlike in conventional hypothesis generation-acceptance
techniques)
Simulated annealing framework to ensure convergence
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 12 / 19

13. Multiframe Marked Point Process Model Optimization
Optimization: iterative stochastic algorithms
The algorithm
1 Start with a frame-by-frame initialization process
Hough transform based line estimation in each binarized frame Bt ,
t = 1...n
2 Iterate object perturbation and acceptance steps till convergence is
obtained in the extracted object sequence
Object perturbation: for each t we propose an object u ∗ which is the
random perturbation of ut−1 OR ut OR ut+1
Acceptance: we accept or reject a move replacing ut width u ∗
Important properties:
Acceptance: inverse approach considering simultaneously data
and prior features
Stochastic process both for object perturbation and acceptance
(unlike in conventional hypothesis generation-acceptance
techniques)
Simulated annealing framework to ensure convergence
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 12 / 19

14. Multiframe Marked Point Process Model Optimization
Target identification
Permanent scatterer extraction and counting
Permanent scatterer
responses: characteristic
target features
high false/missing alarm rate
in the individual frames
(>50%)
histograming technique for
extracting the permanent
scatters
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 13 / 19

15. Experiments
Content
1 Introduction
2 Multiframe Marked Point Process Model
Model elements and configuration energy
Optimization
3 Experiments
4 Future steps and conclusions
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 14 / 19

16. Experiments
Experiments - qualitative results
Center alignment and target line extraction results
Top: input sequence. Center: frame-by-frame detection. Bottom: detection by the
proposed Fm MPP model
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 15 / 19

17. Experiments
Experiments - quantitative results
Test sequences:
four airborne ISAR image sequences (each has 15-40 frames)
different ship targets.
Error measure:
n
E ({ut }, {utgt }) = |x(ut ) − x(utgt )| + |y(ut ) − y(utgt )|+
t=1
+ |l(ut ) − l(utgt )| + |θ(ut ) − θ(utgt )|
Sequence Frames Init Err. Fm MPP Err.
Ship 1 13 52.0 7.5
Ship 2 13 67.1 37.8
Ship 3 13 17.2 12.8
Ship 4 54 43.7 12.6
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 16 / 19

18. Future steps and conclusions
Content
1 Introduction
2 Multiframe Marked Point Process Model
Model elements and configuration energy
Optimization
3 Experiments
4 Future steps and conclusions
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 17 / 19

19. Future steps and conclusions
Generalization for various objects
Identifying Airplanes in ISAR sequences
Cross shaped model
Shadowed wing
Result by the two step process
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 18 / 19

20. Future steps and conclusions
Conclusions
Detecting and featuring ship/airplane targets in ISAR image
sequences through energy minimization
Proposed Multi-frame Marked Point Process schema
advantages versus a frame-by-frame direct detection technique
Towards target classification
permanent scatterer detection algorithm based on histograming
Benedek & Martorella (SZTAKI, CNIT) Target Extraction in ISAR Image Sequences 28 July 2011 19 / 19