This document discusses different approaches for extracting gradient features from images to be used for image segmentation. It begins by discussing traditional gradient approaches like the first order derivative of Gaussian (DOG) and issues with those approaches. It then proposes using a fractional order derivative of Gaussian (FDOG) filter that can better approximate higher order derivatives with fewer parameters. It discusses evaluating such filters using metrics like peak signal-to-noise ratio and variance variation to select an optimal fractional order. The document concludes by discussing building a filter bank using learned FDOG filters to create features for image segmentation.

2. BENCHMARK
BERKLEY SEGMENTATION
DATASET
HUMAN- Segmented Benchmark
D. Martin and C. Fowlkes and D. Tal and J. Malik;
A Database of Human Segmented Natural Images
and its Application to Evaluating Segmentation
Algorithms and Measuring Ecological Statistics;
ICCV 2001

3. SO FAR....
Gray Scale Image-
Contour Detection and Hierarchical Image
Segmentation. P. Arbelaez, M. Maire, C. Fowlkes and J.
Malik; IEEE TPAMI, Vol. 33, No. 5, pp. 898-916, May
2011. SCORE – 0.68
RGB Image-
Xiaofeng Ren and Liefeng Bo, "Discriminatively
Trained Sparse Code Gradients for Contour Detection.",
NIPS 2012. SCORE – 0.71

6. D. Martin, C. Fowlkes, J. Malik. "Learning to Detect Natural Image
Boundaries Using Local Brightness, Color and Texture Cues", TPAMI 2004

7. DISCRIMINATIVE POWER OF
FEATURES
“Not for too smart people who loves
complex features and complex algorithms”
-
‘Pattern Recognition and Machine
Learning’,
Christopher M. Bishop, ISBN-10:
0387310738,Springer, 2006.

8. STEP - I
Gradient Features Proved to be significant.
Gradients are Important !!
* Let’s Talk !

9. Gradients
 Edges are significant local changes of
intensity in an image.
 Goal-
Produce a line drawing of a scene from an
image of that scene
Extract as much Structural Information as
possible

10. What causes intensity changes?
 Geometric events-
object boundary (discontinuity
in depth and/or surface
color and texture)
surface boundary (discontinuity
in surface orientation and/or surface color
and texture)
Sounds Great-
this is what we were hoping for!!

11. Issues
Non-geometric events-
specularity (direct reflection Of light,
such as a mirror)
shadows (from other objects
or from the same object)
Inter- reflections
Fooled by Noisy Peaks – We will
Concentrate on this as of now ..

12. Criteria for optimal edge detection
 Good Detection-
Minimize Misdetection, False Alarm
 Good localization-
edges detected must be as close as
possible to the true edges
 Sparsity -
Non- Maxima Suppression

13. FINDING OPTIMAL DETECTOR BY
NUMERICAL OPTIMIZATION
 Very Hard to find optimal Detector in closed form
 Numerical Approximation
 First order Derivative of Gaussian is good
Approximation. – around 20% worse than the optimal.
JOHN CANNY; A Computational Approach to Edge
Detection; IEEE TRANSACTIONS ON PATTERN
ANALYSIS AND MACHINE INTELLIGENCE,
VOL. PAMI-8, NO. 6, NOVEMBER 1986

14. Canny Detector
 Gradient Computation
 Non- Maxima Suppression
 Hysteresis
Works Fine !! But, we are not looking for
detection results here. We want to get the
Feature only.

15. Gradient
 Should be Discriminative Enough
 We will stop before Thresholding
 Gradient Image should reveal the structural
information – High values at boundaries –
Should be high contrast between non-edge
and edge.

16. 1st Order DOG
1 parameter to tune - σ

17. 1st Order DOG
 Does it really represent the High Level
structural Information?
 Is it really leading us near to Human Drawn
Outline Benchmark?
 Can we Do better?
 Tuning σ Doesn’t really Help !!

18. Anisotropic Diffusion
 Diffuse along Non-Edges
 Keep Structural information
 Remove Noise
*Pietro Perona, Jitendra Malik; Scale-space
and edge detection using anisotropic
Diffusion; ITPAMI 1990

19. Issues
 Computationally Expensive
 Can afford when the Goal is solely Image
Enhancement and we are working on a
single Image.
 Finding Optimal Kappa for the Training
Data – Too much Computation.

20. Another Option..
 5 –tap or 7-tap Differentiation
 Derived from Canny’s
Formulation
 Much Better !!
*Hany Farid Eero,P. Simoncelli; Differentiation of
Discrete Multi-Dimensional Signals; IEEE
TRANSACTIONS ON IMAGE PROCESSING,2004

21. 7th Order Kernel
 Kernel Size increased to 7
 Computational Expense more.
 Can we do equally good or better with a
smaller Convolution Kernel ?

22. Proposed Kernel (FDOG)
 Any random order derivative Can be approximated by
Infinite Sequence
 Well Established in Control System Design
-CRONE Solvers.
*B. Mathieu, P. Melchior, A. Oustaloup, Ch.
Ceyral; Fractional differentiation for edge detection;
Signal Processing, Elsevier 2002

23. 3rd Order Approximation
 Additional Parameter- Order of the Filter v
along with σ of Gaussian.
 V can take any non-integer value
 We will concentrate on{0,2}
 V=1 and V=2 are traditional DOG and LOG,
special cases of Fractional Gradient

24. Some Results

25. Observation
 Edge vs. Non edge contrast decrease as we
increase V from 0.1 to 0.9.
 Lower V means higher PSNR score.
 Lower V gives lower weights to weak edges.
Loose finer details.

26. Optimal Order?
Two Options-
 Learn V using the Training images.
 Choose something around 0.5-0.6
 Choosing V in {0,1} we can only do Better
than DOG !!

27. Some Results after Non- Max
Suppression
Orders Top to Bottom Left to Right , 0.3,0.5,0.7,1

28. Evaluating The Results

29. How do you know when the
detection algorithm is good?
 Looks great on these four images.
*Not Acceptable
Jianbo Shi, Charless Fowlkes, David Martin, Eitan
Sharon ;Graph Based Image Segmentation Tutorial;
CVPR 2004

30. Commonly Used Popular
Quantitative Measures
 False Alarm (Type- II error)
 Misdetection (Type- I error)
 Signal to Noise Ratio*
*Canny’s Thesis evolves around maximizing
this SNR.

31. Which one is the Best Measure
for our purpose?
 None !!!!
Let’s think about this.

32. Our Goal is Segmentation
 This Gradient Filter design is one of the steps of
Image Segmentation.
 Later Stages are already Computationally
Complex.
 Do not need to fine tune the Gradient parameter
too much as V= 0.7 and V= 0.7763 won’t affect
the segmentation result significantly.

33. Evaluation
 Should not train rigorously to fine tune V.
 Finding False Alarm, Misdetection etc using
the Human Benchmark is unnecessary
 Anyways we will eventually do this at the
end.

34. SNR
 Signal to Noise Ratio requires a Noise
Model.
 Noise Estimation is itself a different
problem.
 Too Much work to calculate SNR for the
Dataset and also we do not have Camera
Model etc. Information.

35. So How to evaluate?
 We Propose two simple methods.
Peak Signal Noise Ratio (PSNR):
Widely Used in Image Compression.
Measures how much information retained.
PSNR= - 10 log (MSE)+20 log Max(I)
Higher the Better

36. Variance Variation
Var(noisy Image)= Var(trueImage)+Var(noise)
After applying Smoothing
Noise Removed Equally for all cases
Variation in Variance Now accounts for Structural Loss
Lower The Better
 Henstock,, Peter V. and Chelberg, David M., "Automatic Gradient Threshold
Determination for Edge Detection Using a Statistical Model A Description of the
Model and Comparison of Algorithms" (1996).ECE Technical Reports. Paper 95

37. Score Function
 J=PSNR*(1/Variation in Variance)
 Find v s.t. Max(J(v))
 We suggest simply finding J for different
settings of V and find a suitable value from
the plot. (Run over the entire Training Data
and take average responses over all the
training examples for each V )
 Exact Value of V does not give you much
some close approximation is Good Enough.

38. V for Berkley Dataset

39. Can even find optimal σ
 Again Trying to Maximize over the Score
Function for different σ.

40. Extension to Frequency Domain
 Certainly if Denoising is our goal we can
spend as much time as we can trying to find
better filters and can be very precisely done
in Frequency Domain.

41. Justification of Fractional Poles
 Say we have a filter of order 0.5.
 So, we have a 0.5 pole??
 What is a 0.5 pole?

42. Generalized Frequency Domain
 Riemann Sheets-
Shared Poles
 Stability-
Matignon’s Theorem

43. W Plane Filter Design
 Mapping from Fractional Plane to a Plane which is
a linear combination of all the Riemann Sheets
projected together.
 Design filter in W plane, Figure out the Mapping
Functions, Map Back to frequency plane
*A.Acharya, S. Das, I.Pan, Sh.Das ; Extending the concept of analog
Butterworth filter for fractional order Systems; Signal Processing,
Elsevier, Volume 94, January 2014, Pages 409–420

44. Future Plans of this Project
 Build the Filter Bank with these learned Filters
considering different orientation and other cues.
 Form the Feature Space
 Run AP Clustering/ N-Cut with refined
affinities.
 Hope to go beyond the bar

45. Thank You