Presentation held by Sieow Yeek Tan, Dickson Lukose at the Agricultural Ontology Service (AOS) Workshop 2012 in Kutching, Sarawak, Malaysia from September 3 - 4, 2012

1. Semantic Hybridized Image
Features in Visual
Diagnostic of Plant Health
Sieow Yeek Tan, Dickson Lukose
Image Understanding
Artificial Intelligence Lab / MIMOS Berhad

2. Background of paper
• A picture is worth more than a thousand words
• Objective is to use image to tell the Plant State
• Plant State growing process
• Plant nutrient deficiency type and level
Leaf spot
Lack of phosphorus
Lack of nitrogen
Lack of potassium
2

3. Agenda
• Image Understanding – Introduction
• Research Motivation
– Current Technology
– Future Technology
– Research Gap
• Visual Feature Descriptors
– CEDD: Color and Edge Directivity Descriptor
– FCTH: Fuzzy Color and Texture Histrogram
• Plant Health State Conceptualization
– Supervised Learning of Plant Health State
– Construction of Plant Health State Ontology
• Plant Health Visual Diagnostic Model
– Feature Similarity Calculation
• Visual Plant Diagnostic Laboratory System
• Experiment & Results
– Experiment & Training dataset
– Experiment Results
3

4. Image Understanding - Introduction
4

5. Research Motivation – Current Technology
User Request: Return
Get me similar image
of this
5

6. Research Motivation – Future Technology
Computer Improved Capability :
•Detected building KLCC!!!
•Inference result:
•Located in Kuala Lumpur.
•KL located in Malaysia.
•KLCC hasHeight 532meter
•KLCC hasGeoLocation xxx
6

7. Research Motivation – Research Gap
Color : Shape : Texture :
•Red •Circle •...
•Green •Square
Image Feature
•Black •Rectangle Conceptualization
•Brown •...
•...
KLCC Object
Conceptualization
Knowledge from the
INTERNET
7

8. Visual Feature Descriptors
• General visual descriptors:
• An international standard of descriptions to represent a
visual content
• Encode visual content in a form of vector
• Goal: To search, identify, filter and browse visual content
• Visual content can be color, texture, shape
Three color images and
their MPEG-7 histogram
color distribution, depicted
using a simplified color
histogram
Based on the color
distribution, the two left
images would be recognized
as more similar compared to
the one on the right.
8

9. Visual Feature Descriptors (Cont.)
2 Visual descriptors are used in this paper
CEDD FCTH
Encode Encode
Color Texture
Direction of edge
Savvas A. C., Yiannis S. B., CEDD: color and edge directivity descriptor: a compact descriptor for image indexing and
retrieval, Proceedings of the 6th international conference on Computer vision systems, Santorini, Greece (2008)
Savvas A. C., Yiannis S. B., FCTH: Fuzzy Color and Texture Histogram - A Low Level Feature for Accurate Image Retrieval,
Proceedings of the Ninth International Workshop on Image Analysis for Multimedia Interactive Services, 191-196
(2008) 9

10. Plant Health State Conceptualization:
Supervised Learning of Plant Health State
10

11. Plant Health State Conceptualization:
Construction of Plant Health State Ontology
𝐀𝐆𝐑𝐎𝐕𝐎𝐂
The world’s largest agriculture thesaurus. It has been built over the last 10 years by
the Office of Knowledge Exchange, Research and Extension at the Food and
Agriculture Organization of the United Nation (FAO)
ALOD (2011). AGROVOC Linked Open Data, Food and Agriculture Organization of the United Nations (FAO), Rome,
Italy. URL: http://aims.fao.org/website/Linked-Open-Data/sub (last visited April 2011)
11

12. Plant Health State Conceptualization:
Construction of Plant Health State Ontology (Cont.)
12

13. Plant Health State Conceptualization:
Construction of Plant Health State Ontology (Cont.)
13

14. Plant Health State Conceptualization
14

15. Multiple sub-Windows for Image Blocks
15

16. Features Similarity Calculation & Measurement
Visual descriptors
Each of the image blocks to
CEDD and FCTH
extract the CEDD and FCTH
extracted from training Images
visual descriptors
Similarity
Matching
16

17. Plant Health State Conceptualization
- Feature Similarity Calculation
Based on
𝑐 (𝑐)
COSINE SIMILARITY 𝑖,𝑗
(𝑓𝑡,𝑖 . 𝑓𝑒,𝑗 )
fundamental S𝑐 = (𝑐) (𝑐)
||𝑓𝑡,𝑖 | × | 𝑓𝑒,𝑗 ||
𝑓 𝑐
𝑓𝑡,𝑖 , 𝑓𝑡,𝑖 : Feature vector 𝑓 (𝑓)
𝑖,𝑗
(𝑓𝑡,𝑖 . 𝑓𝑒,𝑗 )
extracted from training images S𝑓 = (𝑓) (𝑓)
||𝑓𝑡,𝑖 | × | 𝑓𝑒,𝑗 ||
(𝑓) (𝑐)
𝑓𝑒,𝑗 , 𝑓𝑒,𝑗 : Feature vector extracted
from target image blocks
𝑖,𝑗 𝑖,𝑗
S 𝑐 , S 𝑓 : Ranged from 0 to 1
Threshold configuration for analysis
17

18. Visual Plant Diagnostic Laboratory System
Multiple sub-Windows Detected
for Image Blocks Plant Health State
System Operational Panel. For exporting Visual Feature Descriptors
experiment results and perform configuration analysis details
18

19. Experiment & Training dataset
Unhealthy Sample of training images 5 𝑡𝑦𝑝𝑒𝑠 𝑜𝑓
Plant States “Cropped” category “Original” category 𝑢𝑛ℎ𝑒𝑎𝑙𝑡ℎ 𝑝𝑙𝑎𝑛𝑡
Iris leaf
spot 50 𝑖𝑚𝑎𝑔𝑒𝑠 𝑜𝑓
𝑒𝑥𝑝𝑒𝑟𝑖𝑚𝑒𝑛𝑡 𝑑𝑎𝑡𝑎𝑠𝑒𝑡
Nitrogen 𝑓𝑜𝑟 𝑒𝑎𝑐ℎ 𝑡𝑦𝑝𝑒 𝑜𝑓
deficiency 𝑝𝑙𝑎𝑛𝑡 𝑠𝑡𝑎𝑡𝑒
Oedema 30 𝑖𝑚𝑎𝑔𝑒𝑠
𝑡𝑟𝑎𝑖𝑛𝑖𝑛𝑔 𝑑𝑎𝑡𝑎𝑠𝑒𝑡
Phosphorous “Cropped”
deficiency
and
“Original”
Potassium
deficiency 1 𝑠𝑒𝑡 𝑜𝑓 ℎ𝑒𝑎𝑙𝑡ℎ𝑦
𝑝𝑙𝑎𝑛𝑡 𝑠𝑡𝑎𝑡𝑒
19

20. Experiment Results
Result showing the “Cropped” category
potentiality and feasibility Seem to show
of using the proposed model in better result than
performing visual plant diagnosis “Original” category
20

21. 21