One of the popular knowledge elicitation technique

1. Concept Sorting
Shyam Krishna Khadka
MSc. in Computer Systems and Knowledge
Engineering
Institute of Engineering(IoE)
Pulchowk Campus, Nepal

2. Concept sorting – What ?
• A form of contrived knowledge elicitation technique
• Technique:
– present expert with shuffled set of cards with concept
names
– expert is asked to sort cards in piles
• Helps to find relations among a set of concepts
2
• Helps to find relations among a set of concepts
• Useful in case of subtle dependencies
• Simple to apply
• Also called “Card Sorting”

3. Procedure
• Decide what classes of concepts you want to explore (in
particular their properties – attributes and values)
• Write the name of each concept on a separate card
• Ask the expert to sort the cards into either a fixed number
of piles or any number of piles expert finds appropriate
• Collect the cards and ask the expert to sort again
•
3
• Repeat until the expert can’t sort anymore
• Ask the expert to name the piles in each sort
• Often referred to as dimension along which concepts are
sorted
• It typically identifies a particular property or attribute
associated with a class of objects

4. Example : Geology domain
• The concepts printed on a set of cards are the names of
igneous rocks drawn from a structured interview with the
expert
• He had previously described 18 rock types
1 adamellite 10 granite
2 andesite 11 lherzolite
4
2 andesite 11 lherzolite
3 basalt 12 microgranite
4 dacite 13 peridotite
5 diorite 14 picrite basalt
6 dolerite 15 rhyodacite
7 dunite 16 rhyolite
8 gabbro 17 syenite
9 granodiorite 18 trachyte
Table 1: The names
of 18 types
of igneous rock
elicited from a
geologist as part of
a structured
interview

5. Example : Geology domain contd..
• The dimensions/piles which the expert used for the
individual card sorts are presented in Table 2.
Sort # Dimension Piles
1grain size 1=coarse, 2=medium, 3=fine
2colour
1=melanocratic, 2=mesocratic,
3=leucocratic
5
3emplacement 1=intrusive, 2=extrusive
4presence of olivine 1=always, 2=possibly, 3=never
5presence of quartz 1=always, 2=possibly, 3=never
6percentage of silica 1= >68%, 2= <68%, 3= about 68%
7density
1=very light, 2=light, 3=medium, 4=dense,
5=very dense
Table 2: The results of seven card sorts undertaken as part of a
concept sorting knowledge elicitation session with a geologist.

6. Example : Geology domain contd..
6
Table 3: The positioning of cards representing different types of
igneous rock (see Table 1) in the piles resulting from seven card sorts
with a geologist (see Table 2)

7. Example : Geology domain contd..
• Using the results of the card sorts, we can attempt to
extract decision rules directly. An example of a rule
extracted from the card sorting data is:
IF the grain size is fine (sort 1/pile 3)
AND the color is mesocratic (sort 2/pile 2)
AND its emplacement is extrusive (sort 3/pile 2)
AND it does NOT contain olivine (sort 4/pile 3)
7
AND it does NOT contain olivine (sort 4/pile 3)
AND may possibly contain quartz (sort 5/pile 2)
AND it contains less than 68% silica (sort 6/pile
2)
AND its density is medium (sort 7/pile 3)
THEN the rock is andesite (outcome 2)

8. Applications
– Tourism sector
– Education sector etc.
8

9. Pros and Cons
• Prons
– Fast to apply and easy to analyze
– Often instructive to the expert in a sense that it may
lead expert to see structure that he himself has not
consciously articulated before
– Time saving by not having to transcribe and analyse
lengthy verbal reports
9
lengthy verbal reports
– Can be used for images and objects as well

10. Pros and Cons contd..
• Cons
– Experts can often confound dimensions by not
consistently applying the same semantic distinctions
throughout an elicitation session
– May face problem in categorization of elements in
meaningful way
10

11. References
• Nigel R. Shadbolt and Paul R. Smart, Knowledge Elicitation:
Methods, Tools and Techniques, Electronics and Computer Science,
University of Southampton, Southampton, SO17 1BJ, UK
• Yimin Wang, York Sure, Robert Stevens and Alan Rector,
Knowledge Elicitation Plug-in for Protégé: Card Sorting and
Laddering. Available from:
http://www.aifb.kit.edu/images/e/e7/2006_1243_Wang_Knowledge_Elici
_1.pdf
11
_1.pdf
• https://www.cse.ust.hk/~dekai/523/notes/KM_Slides_Ch10.pdf
• Milton, Springer-Verlag, Knowledge Acquisition in Practice: A step by
step guide

12. Questions ??
Thank you
12
Thank you