Upcoming SlideShare
×

# Formal Concept Analysis

1,662 views
1,460 views

Published on

AACIMP 2011 Summer School. Operational Research Stream. Lecture by Erik Kropat.

Published in: Education, Technology
3 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
1,662
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
71
0
Likes
3
Embeds 0
No embeds

No notes for slide

### Formal Concept Analysis

1. 1. Summer School“Achievements and Applications of Contemporary Informatics, Mathematics and Physics” (AACIMP 2011) August 8-20, 2011, Kiev, Ukraine ̶ Formal Concept Analysis ̶ Erik Kropat University of the Bundeswehr Munich Institute for Theoretical Computer Science, Mathematics and Operations Research Neubiberg, Germany
2. 2. Formal Concept AnalysisFormal Concept Analysis studies, how objects can be hierarchically grouped togetheraccording to their common attributes. Tree of Life Source: Tree of Life Web Project http://tolweb.org/tree/
3. 3. Formal Concept Analysiswww.arthursclipart.org
4. 4. What is a “concept” ?A concept is a cognitive unit of meaning or a unit of knowledge. Concept Bird properties − feathered − warm-blooded − winged − egg-laying − bipedal − vertebrate objects blackbird, sparrow, raven,…
5. 5. Formal Concept Analysis• . . . is a powerful tool for data analysis, information retrieval, and knowledge discovery in large databases.• . . . is a conceptual clustering method, which clusters simultaneously objects and their properties.• . . . can mathematically represent, identify and analyze green yellow conceptual structures. red 2-dim cylinder disk 3-dim triangle cube
6. 6. yellow triangle cube greenExample disk cylinder red 3-dim 2-dim 3-dim 2-dim yellow green red
7. 7. Formal Concept Analysis• . . . models concepts as units of thought, consisting of two parts: − extension = objects belonging to the concept − intension = attributes common to all those objects.• . . . is an exploratory data analysis technique for discovering new knowledge.• . . . can be used for efficiently computing association rules applied in decision support systems.• . . . can extract and visualize hierarchies !!!
8. 8. Formal Concept AnalysisGoal: Derive automatically an ontology from a – very large – collection of objects and their properties or features. Target Marketing Set of objects ⇒ clusters of objects customers correspond ⇔ one-for-one Set of attributes age, sex, income level, ⇒ clusters of attributes spending habits, … predict customer purchase decisions / ⇒ recommend products to customers
9. 9. Sensitive advertisement clusters of objects correspond one-for-one clusters of attributes
10. 10. Formal Contexts
11. 11. Example: Classification of plants and animals Animal Dog Cat Plant lives on land Reed Water lily Oak lives in water Carp Potato Objects Attributes
12. 12. Formal Concept AnalysisExample: Classification of plants and animals AttributesQuestion: Lives in water Lives on landHas object g the attribute m ( Yes / No ) ? Animal Plant Dog x x Cat x x Oak x xBinary Relation Objects Potato x xA formal context can be represented Carp x x Water lily x xby a cross table (bit-matrix). Reed x x x
13. 13. Formal ContextA formal context describes the relation betweenobjects and attributes. A formal context (G, M, I) consists of a set G of objects, a set M of attributes and a binary relation I ⊂ G x M. Has object g the attribute m ( yes / no ) ?
14. 14. Notation• g I m means: “object g has attribute m”.Example: (a) dog I animal (b) carp I lives in water
15. 15. Derivation Operators
16. 16. The Derivation Operators (Type I)A ⊂ G selection of objects.Question: Which attributes from M are common to all these objects? Lives in waterSet of common attributes of the objects in A Lives on landA’ := A↑:= { m ∈ M | g I m for all g ∈ A } Animal Plant Dog x x Cat x x A⊂G A′ ⊂ M Oak x x Potato x x{Dog, Cat} Carp x x{Oak, Potato} Water lily x x Reed x x x
17. 17. The Derivation Operators (Type I)A ⊂ G selection of objects.Question: Which attributes from M are common to all these objects? Lives in waterSet of common attributes of the objects in A Lives on landA’ := A↑:= { m ∈ M | g I m for all g ∈ A } Animal Plant Dog x x Cat x x A⊂G A′ ⊂ M Oak x x Potato x x{Dog, Cat} {Animal, lives on land} Carp x x{Oak, Potato} Water lily x x Reed x x x
18. 18. The Derivation Operators (Type I)A ⊂ G selection of objects.Question: Which attributes from M are common to all these objects? Lives in waterSet of common attributes of the objects in A Lives on landA’ := A↑:= { m ∈ M | g I m for all g ∈ A } Animal Plant Dog x x Cat x x A⊂G A′ ⊂ M Oak x x Potato x x{Dog, Cat} {Animal, lives on land} Carp x x{Oak, Potato} Water lily x x Reed x x x
19. 19. The Derivation Operators (Type I)A ⊂ G selection of objects.Question: Which attributes from M are common to all these objects? Lives in waterSet of common attributes of the objects in A Lives on landA’ := A↑:= { m ∈ M | g I m for all g ∈ A } Animal Plant Dog x x Cat x x A⊂G A′ ⊂ M Oak x x Potato x x{Dog, Cat} {Animal, lives on land} Carp x x{Oak, Potato} {Plant, lives on land} Water lily x x Reed x x x
20. 20. The Derivation Operators (Type II)B ⊂ M a set of attributes.Question: Which objects have all the attributes from B? Lives in waterSet of objects that have all the attributes from B Lives on landB’ := B↓:= { g ∈ G | g I m for all m ∈ B } Animal Plant Dog x x Cat x x B⊂M B′ ⊂ G Oak x x Potato x x{Plant, lives on land} Carp x x{Animal, lives in water} Water lily x x Reed x x x
21. 21. The Derivation Operators (Type II)B ⊂ M a set of attributes.Question: Which objects have all the attributes from B? Lives in waterSet of objects that have all the attributes from B Lives on landB’ := B↓:= { g ∈ G | g I m for all m ∈ B } Animal Plant Dog x x Cat x x B⊂M B′ ⊂ G Oak x x Potato x x{Plant, lives on land} {Oak, Potato, Reed} Carp x x{Animal, lives in water} Water lily x x Reed x x x
22. 22. The Derivation Operators (Type II)B ⊂ M a set of attributes.Question: Which objects have all the attributes from B? Lives in waterSet of objects that have all the attributes from B Lives on landB’ := B↓:= { g ∈ G | g I m for all m ∈ B } Animal Plant Dog x x Cat x x B⊂M B′ ⊂ G Oak x x Potato x x{Plant, lives on land} {Oak, Potato, Reed} Carp x x{Animal, lives in water} Water lily x x Reed x x x
23. 23. The Derivation Operators (Type II)B ⊂ M a set of attributes.Question: Which objects have all the attributes from B? Lives in waterSet of objects that have all the attributes from B Lives on landB’ := B↓:= { g ∈ G | g I m for all m ∈ B } Animal Plant Dog x x Cat x x B⊂M B′ ⊂ G Oak x x Potato x x{Plant, lives on land} {Oak, Potato, Reed} Carp x x{Animal, lives in water} {Carp} Water lily x x Reed x x x
24. 24. 1) If a selection of objects is enlarged,Derivation Operators - Facts then the attributes which are commonLet (G, M, I) be a formal context. to all objects of the larger selection are amongA, A1, A2 ⊂ G sets of objects. the common attributes of the smaller selection.B, B1, B2 ⊂ G sets of attributes. 1) A1 ⊂ A2 ⇒ A′2 ⊂ A′1 1′) B1 ⊂ B2 ⇒ B′2 ⊂ B′1 2) A ⊂ A′′ 2′) B ⊂ B′′ 3) A′ = A′′′ 3′) B′ = B′′′ 4) A ⊂ B′ ⇔ B ⊂ A′ ⇔ A x B ⊂ I The derivation operators constitute a Galois connection between the power sets P(G) and P (M).
25. 25. Formal Concepts
26. 26. Formal ConceptsFormal Context: Defines a relation between objects and attributes.Real World: Objects are characterized by particular attributes. Object Attributes
27. 27. Formal Concepts Let (G, M, I) be a formal context, where A ⊂ G and B ⊂ M. (A, B) is a formal concept of (G, M, I), iff A′ = B and B′ = A. The set A is called the extent and the set B is called the intent of the formal concept (A, B).
28. 28. Formal Concepts• Extent A and intent B of a formal concept (A,B) correspond to each other by the binary relation I of the underlying formal context.• The description of a formal concept is redundant, because each of the two parts determines the other Extent Intent (objects) (attributes) Duality
29. 29. How can we find “formal concepts”? Lives in water Lives on landA formal concept (A, B) corresponds to a Animal Plantfilled rectangular subtablewith row set A and column set B. Dog x x Cat x x Oak x x Potato x x( {Dog, Cat}, {Animal, lives on land} ) Carp x x Water lily x x Reed x x x
30. 30. How can we find “formal concepts”? Lives in water Lives on landA formal concept (A, B) corresponds to a Animal Plantfilled rectangular subtablewith row set A and column set B. Dog x x Cat x x Oak x x Potato x x( {Dog, Cat}, {Animal, lives on land} ) Carp x x Water lily x x Reed x x x Each of the two parts determines the other!
31. 31. ExerciseDetermine the sets of objects A and the set of attributes Bsuch that the pair (A, B) represents a formal concept.(a) A = {oak, potato, reed}, B = ?(b) A = ?, B = {animal, lives in water}
32. 32. How can we find “formal concepts”? Lives in water Lives on landA formal concept (A, B) corresponds to a Animal Plantfilled rectangular subtablewith row set A and column set B. Dog x x Cat x x Oak x x Potato x x( {Dog, Cat}, {Animal, lives on land} ) Carp x x Water lily x x Reed x x( {Oak, Potato, Reed}, {Plant, lives on land} ) x
33. 33. How can we find “formal concepts”? Lives in water Lives on landA formal concept (A, B) corresponds to a Animal Plantfilled rectangular subtablewith row set A and column set B. Dog x x Cat x x Oak x x Potato x x( {Dog, Cat}, {Animal, lives on land} ) Carp x x Water lily x x Reed x x( {Oak, Potato, Reed}, {Plant, lives on land} ) x( {Carp}, {Animal, lives in water} )
34. 34. How can we find “formal concepts”? Lives in water Lives on land A formal concept (A, B) corresponds to a Animal Plant filled rectangular subtable with row set A and column set B. Dog x x Cat x x Oak x x Potato x xQuestion: Is the following pair a formal concept? Carp x x Water lily x x Reed x x x( {Oak, Potato}, {Plant, lives on land} )
35. 35. How can we find “formal concepts”? Lives in water Lives on land A formal concept (A, B) corresponds to a Animal Plant filled rectangular subtable with row set A and column set B. Dog x x Cat x x Oak x x Potato x xQuestion: Is the following pair a formal concept? Carp x x Water lily x x Reed x x x( {Oak, Potato}, {Plant, lives on land} )There exist filled rectangular subtables that do not determine formal concepts
36. 36. Computing all Formal ConceptsLemmaEach formal concept (A, B) of a formal context (G,M,I) has the form (A′′, A′) for some subset A⊂G and the form (B′, B′′) for some subset B ⊂ M.Conversely, all such pairs are formal concepts. Compute all formal concepts
37. 37. Observations• (A′′, A′) ist a formal concept.• A ⊂ G extent ⇔ A = A′′. B ⊂ M intent ⇔ B = B′′.• The intersection of arbitrary many extents is an extent. The intersection of arbitrary many intents is an intent.
38. 38. Algorithm for Computing all Formal ConceptsA) Determine all Concept Extents 1. Initialize a list of concept extents. Write for each attribute m ∈ M the extent {m}’ to the list. 2. For any two sets in the list, compute their intersection. If the result is set that is not yet in the list, then extend the list by this set. With the extended list, continue to build all pairwise intersections. Extend the list by the set G. ⇒ The list contains all concept extents.B) Determine all Concept Intents 3. Compute intents For every concept extent A in the list compute the corresponding intent A′ to obtain a list of all formal concepts (A, A′).
39. 39. ExerciseCompute the formal concepts of the following formal context.
40. 40. Exercise1. Initialize a list of concept extents. Write for each attribute m ∈ M the extent {m}’ to the list. Item Extent {m} Attribute m∈M e1 {Dog, Cat, Carp} {Animal} e2 {Oak, Potato, Water lily, Reed} {Plant} e3 {Dog, Cat, Oak, Potato, Reed} {Lives on land} e4 {Carp, Water lily, Reed} {Lives in water}
41. 41. Exercise2. For any two sets in the list, compute their intersection. - If the result is a set that is not yet in the list, then extend the list by this set. - With the extended list, continue to build all pairwise intersections. - Extend the list by the set G. Item Extent Defined by e1 {Dog, Cat, Carp} {Animal} e2 {Oak, Potato, Water lily, Reed} {Plant} e3 {Dog, Cat, Oak, Potato, Reed} {Lives on land} e4 {Carp, Water lily, Reed} {Lives in water} e5 ∅ e1 ∩ e2 e6 {Dog, Cat} e1 ∩ e3 e7 {Carp} e1 ∩ e4 e8 {Oak, Potato, Reed} e2 ∩ e3 e9 {Water lily, Reed} e2 ∩ e4 e10 {Reed} e3 ∩ e4 e11 {Dog, Cat, Oak, Potato, Carp, Water lily, Reed} G
42. 42. Exercise2. For any two sets in the list, compute their intersection. - If the result is a set that is not yet in the list, then extend the list by this set. - With the extended list, continue to build all pairwise intersections. - Extend the list by the set G. Item Extent Defined by e1 {Dog, Cat, Carp} {Animal} e2 {Oak, Potato, Water lily, Reed} {Plant} e3 {Dog, Cat, Oak, Potato, Reed} {Lives on land} e4 {Carp, Water lily, Reed} {Lives in water} e5 ∅ e1 ∩ e2 e6 {Dog, Cat} e1 ∩ e3 e7 {Carp} e1 ∩ e4 e8 {Oak, Potato, Reed} e2 ∩ e3 e9 {Water lily, Reed} e2 ∩ e4 e10 {Reed} e3 ∩ e4 e11 {Dog, Cat, Oak, Potato, Carp, Water lily, Reed} G
43. 43. Exercise3. Determine intents For every concept extent A in the list compute the corresponding intent A′ to obtain a list of all formal concepts (A, A′). Item Extent A Intent A′ e1 {Dog, Cat, Carp} {Animal} e2 {Oak, Potato, Water lily, Reed} {Plant} e3 {Dog, Cat, Oak, Potato, Reed} {Lives on land} e4 {Carp, Water lily, Reed} {Lives in water} e5 ∅ M e6 {Dog, Cat} {Animal, lives on land} e7 {Carp} {Animal, lives in water} e8 {Oak, Potato, Reed} {Plant, lives on land} e9 {Water lily, Reed} {Plant, lives in water} e10 {Reed} {Plant, lives on land, lives in water} e11 {Dog, Cat, Oak, Potato, Carp, Water lily, Reed} ∅
44. 44. Conceptual Hierarchies and Concept Lattices
45. 45. Is there a relation between the formal concepts? Animal super-concept Dog, Cat, Carp ≤ Animal, lives on land Animal, lives in water sub-concept Dog, Cat CarpIdea: Order concepts in a sub-concept ̶ super-concept hierarchy
46. 46. Is there a relation between the formal concepts? Animal super-concept Dog, Cat, Carp ≤ Animal, lives on land Animal, lives in water sub-concept Dog, Cat CarpThe extent of the sub-concept is a subset of the extent of the super-conceptThe intent of the super-concept is a subset of the intent of the sub-concept
47. 47. Conceptual HierarchyLet (A1, B1) and (A2, B2) be formal concepts of (G,M,I). (A1, B1) sub-concept of (A2, B2) :⇔ A1 ⊂ A2 [⇔ B2 ⊂ B1 ]. Animal Dog, Cat, Carp• (A2, B2) is a super-concept of (A1, B1).• Notation: (A1, B1) ≤ (A2, B2) Animal, lives on land Dog, Cat
48. 48. Conceptual Hierarchy• The set of all formal concepts of (G, M, I) is called the concept lattice of the formal context (G, M, I) and is denoted by B (G,M,I) .
49. 49. Conceptual HierarchyTheoremThe concept lattice of a formal context is a partially ordered set. We need a notion of neighborhood⇒ We can draw figures that indicate intricate relationships!!
50. 50. Conceptual HierarchyLet P be a set and ≤ is a binary relation on P.A partially ordered set is a pair (P, ≤), iff 1) x≤x (reflexive) 2) x ≤ y and x ≠ y ⇒ ¬ y ≤ x (antisymmetric) 3) x ≤ y and y ≤ z ⇒ x ≤ z (transitive)for all x, y, z ∈ P.
51. 51. Conceptual HierarchyLet (A1, B1) and (A2, B2) be formal concepts of the context (G,M,I). (A1, B1) proper sub-concept of (A2, B2) [ (A1, B1) < (A2, B2)] :⇔ (A1, B1) ≤ (A2, B2) and (A1, B1) ≠ (A2, B2) . (A2 , B2) (A1 , B1)
52. 52. Conceptual HierarchyExamples: In the following examples (A1, B1) is a proper sub-concept of (A2, B2) (a) (A2 , B2) (b) (A2 , B2) (A1 , B1) (A , B ) (A1 , B1)Question: What is the difference between (a) and (b)?Answer: In (a) the concept (A1, B1) is the lower neighbor of (A2, B2). In (b) the concept (A1, B1) is not the lower neighbor of (A2, B2).
53. 53. Conceptual HierarchyProper sub-concepts can be used to define a notion of neighborhood.Let (A1, B1) and (A2, B2) be formal concepts of the context (G,M,I) (A2 , B2)and (A1, B1) is a proper sub-concept of (A2, B2).(A1, B1) is a lower neighbor of (A2, B2) [(A1, B1)  (A2, B2)], (A , B )if no formal concept (A, B) exists with (A1 , B1) (A1, B1) < (A, B) < (A2, B2).
54. 54. Drawing Concept Lattices• Draw formal concepts Draw a small circle for every formal concept. A circle for a concept is always positioned higher than the circles of its proper sub-concepts.• Draw lines Connect each formal concept (circle) with the circles of its lower neighbors.• Label with attribute names Attach the attribute m to the circle representing the concept ( {m}′, {m}′′ ).• Label with object names Attach each object g to the circle representing the ({g}′′ , {g}′).
55. 55. ExerciseCompute the concept lattice of the following formal concept.
56. 56. Drawing Concept Lattices G e11 plant e2 e4 aquatic e1 animal e3 terrestrial water water plant e9 e7 animal e6 land animal terrestrial e8 plants water lily carp dog, cat oak, potato plants, on land e10 & in water reed e5 ∅
57. 57. ExerciseCompute the formal concepts of the following formal context: Attributes Habital zone Terrestrial Gas giant Moon Earth x x x Jupiter x x Objects Mercury x Mars x x
58. 58. Exercise1. Initialize a list of concept extents. Write for each attribute m ∈ M the extent {m}’ to the list. Item Extent {m} Attribute m∈M e1 {jupiter} {gas giant} e2 {earth, mercury, mars} {terrestrial} e3 {earth, jupiter, mars} {moon} e4 {earth} {habital zone}
59. 59. Exercise 2. For any two sets in the list, compute their intersection. If the result is a set that is not yet in the list, then extend the list by this set. With the extended list, continue to build all pairwise intersections. Extend the list by the set G. Item Extent Defined by e1 {jupiter} {gas giant} e2 {earth, mercury, mars} {terrestrial} e3 {earth, jupiter, mars} {moon} e4 {earth} {habital zone} e5 ∅ e1 ∩ e2 e6 {earth, mars} e2 ∩ e3 e7 {earth, jupiter, mercury, mars} G
60. 60. Exercise3. Determine intents For every concept extent A in the list compute the corresponding intent A′ to obtain a list of all formal concepts (A, A′). Item Extent Intent e1 {jupiter} {gas giant, moon} e2 {earth, mercury, mars} {terrestrial} e3 {earth, jupiter, mars} {moon} e4 {earth} {terrestrial, moon, habital zone} e5 ∅ M e6 {earth, mars} {terrestrial, moon} e7 {earth, jupiter, mercury, mars} ∅
61. 61. Exercise GConcept Lattice e7 terrestrial moon earth, mercury, e2 e3 earth, jupiter, mars mars terrestrial, e6 moon earth, mars gas giant, terrestrial, e4 e1 moon moon, habitual jupiter earth e5 ∅
62. 62. Applications
63. 63. Applications• Web information retrieval → How can web search results retrieved by search engines be conceptualized and represented in a human-oriented form.• Partner selection for interfirm collaborations → Identification of structural similarities between potential partners according to the characteristics of the prospective partner firms.• Information systems for IT security management → Identification of security-sensitive operations performed by a server.• Data warehousing and database analysis → Controlling the trade of stocks and shares.
64. 64. Bioinformatics Verducci J S et al. Physiol. Genomics 2006;25:355-363©2006 by American Physiological Society
65. 65. Bioinformatics Biclustering / co-clustering Simultaneous clustering of the rows and columns of a matrix. Verducci J S et al. Physiol. Genomics 2006;25:355-363©2006 by American Physiological Society
66. 66. Summary• Formal concept analysis provides methods for an automatic derivation of ontologies from very large collections of objects and their attributes.• Reveal unknown, hidden and meaningful connections between groups of objects and groups of attributes.• The methods are supported by algebra, lattice theory and order theory.• Visualization techniques are available.• Strong connections to co-clustering (bi-clustering) methods (important tools in DNA-microarray analysis).
67. 67. Literature• Bernhard Ganter, Gerd Stumme, Rudolf Wille (ed.) Formal Concept Analysis. Foundations and Applications. Springer, 2005.• Claudio Carpineto, Giovanni Romano Concept Data Analysis: Theory and Applications. Wiley, 2004.Software www.fcahome.org.uk/fcasoftware.html
68. 68. Thank you very much!