This document discusses knowledge acquisition and concept sorting. It defines knowledge acquisition as the process of acquiring knowledge for an unknown domain, which is an important first step in knowledge engineering. Concept sorting is presented as a knowledge elicitation technique where domain experts organize concepts into groups based on common attributes. The document provides details on how concept sorting is performed and analyzed, noting its benefits in articulating an expert's domain knowledge, as well as its limitations for large numbers of concepts.

1. “To know what we know and what we
do not know, that is true
knowledge”
- CONFUCIUS

2. Knowledge
Acquisition
CONCEPT SORTING
Adarsha Dhakal
Jaydeep Shah
Prakash Upadhyaya
Ritu Ratnam

3. Knowledge Acquisition
Basically it is first step on Knowledge Engineering
The process of acquiring knowledge for an unknown domain
Most important and crucial task for development of Expert System
Referred as bottleneck in the development of Expert System and Knowledge-Based
System
◦ As it is difficult and time-consuming activities
Sometimes process gets complicated because
◦ most of the information is inside the heads of the domain’s Experts
◦ domain’s experts’ unwillingness of disclosing the information

4. Epistemology (theory of
Knowledge)
Cognitive psychology
Cognitive neuroscience
Logic and inference
Machine learning
Knowledge discovery
Linguistics
Information Technology
Field of Application
Knowledge Elicitation overcomes the requirements of long lists and documents in
order to extract meaning.

5. Knowledge Engineer’s Roles in Interactive Knowledge Acquisition

6. Knowledge Acquisition (contd.)
Gathering and collecting knowledge from various sources
Adding and refining new knowledge for the previous knowledge base
The process of expanding the capabilities of a system or improving its performance
Acquired knowledge may consist:
◦ facts, rules, concepts, procedures, heuristics, formulas, relationships, statistics, or any other
useful information.
Source of these knowledges may be:
◦ experts in the domain of interest,
◦ textbooks, technical papers, database reports,
◦ journals and the environments
The process is continuous and lasts for an entire lifetime

7. Knowledge Acquisition Techniques
1. Interview
2. Protocol Analysis
3. Laddering
4. Concept Sorting (T)
5. Repertory Grids
6. Structural Assessment

8. Concept Sorting
Concept sorting is a psychological technique
Useful in taping organizational knowledge
A form of simple and easy knowledge elicitation techniques
Contrived technique of knowledge elicitation
Way of finding out how the experts compare and order concepts
Can reveal knowledge about classes, properties, and relations
Works best in small group
Used to capture concept knowledge and tacit knowledge

9. Concept Sorting
Used to generate information about the associations and grouping of
specific data items’ concepts
Present experts with a shuffled set of cards with concept names
Experts are asked to organize individual, unsorted concepts into
groups and may, depending on the technique, also provide labels for
these groups.
Concept sorting is usually conducted as a specific activity in the early
design phase of defining a project architecture
But can be used during a product evaluation to determine if usability
issues are due to problems with grouping or group labels

10. Concept Sorting
Simplest form is Card Sorting
 collection on concepts (or other knowledge concepts) are
written in separate cards
 cards sorted into piles by an expert in to piles – cards in
the piles must have some thing in common
 each time the cards are sorted it will be based on an
attribute and each pile will represent a value
Can also sort objects or pictures instead of
cards
Enable significant elicitation of properties
and dimensions

11. Concept Sorting (how to?)
1) Decide classes of concepts that s/he wants to explore
2) Consult textbook, experts, and manual to identify top-level
concepts represented in the domain
3) Write the concept of each concept on each card
4) Asks expert to sort these cards placing them in groups that
belong together
5) Ask domain expert to determine why they are placed together
6) Repeat 4 and 5 until the expert cannot sort the cards anymore
Knowledge Engineers in action

12. Concept Sorting
Knowledge Engineers in action

13. Concept Sorting
Experts in action (on basis of Habitat)
Terrestrial Animals Amphibian Animals Aquatic Animals

14. Concept Sorting
Experts in action (on basis of Eating Habits)
Carnivores Animals Omnivores Animals Herbivores Animals

15. Concept Sorting
Experts
in
action
Card Size Backbone Mammalia
Horse 3 1 1
Bear 3 1 1
Deer 2 1 1
Crab 1 2 2
Cow 3 1 1
Cat 1 1 1
Octopus 2 2 2
Frog 1 1 2
Shark 3 1 2
Star Fish 1 2 2
Crocodile 3 1 2
Panda 2 1 1
Turtle 2 1 2
Seal 2 1 1
Toad 1 1 2
Tiger 3 1 1
Goat 2 1 1
Size:
1. Small
2. Medium
3. Big
Backbone
1. Vertebrate
2. Non-vertebrate
Mammalia:
1. Mammals
2. Non-mammals

16. Concept Sorting (Pros)
Fast to apply, simple and easy to analyze
Can be helpful to articulate about domain for experts too
Time-saving by not having to transcribe and analyze lengthy verbal
reports
Can be done by using images and objects as well

17. Concept Sorting (Cons)
Creation of cards can be tedious depending upon the size of the
concepts numbers
Categorization of elements in meaningful way could be tricky
Applying the same semantic distinctions can be monotonous for
large numbers of card
Experts can confound concepts dimensions by not applying the
appropriate semantics thoroughly

18. “To know that we know what we
know and to know that we do not
know what we do not know, that is
true knowledge”
- NICOLAS COPERNICUS

19. References:
1. Shadbolt, N. R., & Smart, P. R. (2015) Knowledge Elicitation. In J. R. Wilson & S. Sharples (Eds.),
Evaluation of Human Work (4th ed.). CRC Press, Boca Raton, Florida, USA.
2. http://academic.cankaya.edu.tr/~agorur/AI/lect/knowledge.html
3. http://syllabus.cs.manchester.ac.uk/ugt/2021/COMP34512/slides/day2.pdf
4. http://www.cse.aucegypt.edu/~rafea/csce485es/slides/ch.10.pdf
5. https://www.brainkart.com/article/Knowledge-Acquisition-Techniques_8888/
6. https://www.slideshare.net/neerav_adhikari/knowledge-elicitation-techiniques-concept-sorting
7. https://www.brainyquote.com/topics/knowledge-quotes