[Slides3]

824 views
751 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
824
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

[Slides3]

  1. 1. BISC Fuzzy Set: 1965 … Fuzzy Logic: 1973 … Soft Decision: 1981 … BISC: 1990 … Human-Machine Perception: 2000 - … Theory and the Applications of Natural Language Computing: Computation and Reasoning with Information Presented in Natural Languages Masoud Nikravesh BISC Program, EECS-UCB & Informatics and Imaging- Life Sciences Lawrence Berkeley National Laboratory (LBNL) http://www-bisc.cs.berkeley.edu/ Email: Nikravesh@cs.berkeley.edu Tel: (510) 643-4522; Fax: (510) 642-5775 Acknowledgements: James S. Albus Senior NIST Fellow Intelligent Systems Division Manufacturing Engineering Laboratory National Institute of Standards and Technology Acknowledgements: Prof. Lotfi A. Zadeh BISC Program EECS-UCB ICMLA'05 The Fourth International Conference on Machine Learning and Applications 15-17 December 2005, Sheraton Gateway Hotel, Los Angeles, CA, USA
  2. 2. Outline <ul><li>BISC Program </li></ul><ul><li>Introduction </li></ul><ul><li>Natural Language Computing </li></ul><ul><li>CONCEPTS </li></ul><ul><li>ORIGIN </li></ul><ul><li>APPLICATIONS </li></ul><ul><ul><li>Neu-Search* </li></ul></ul><ul><ul><li>BISC-DSS </li></ul></ul>
  3. 3. MACHINE INTELLIGENCE
  4. 4. TURING’s TEST Turing: A computer can be said to be intelligent if its answers are indistinguishable from the answers of a human being ? ? Computer
  5. 5. Artificial Neural Network vs. Human Brain <ul><li>Largest neural computer: </li></ul><ul><ul><li>20,000 neurons </li></ul></ul><ul><li>Worm’s brain: </li></ul><ul><ul><li>1,000 neurons </li></ul></ul><ul><li>But the worm’s brain outperforms neural computers </li></ul><ul><li>It’s the connections, not the neurons! </li></ul><ul><li>Human brain: </li></ul><ul><ul><li>100,000,000,000 neurons </li></ul></ul><ul><ul><li>200,000,000,000,000 connections </li></ul></ul>
  6. 6. <ul><li>• Processing Speed: Milliseconds VS Nanoseconds. </li></ul><ul><li>• Processing Order: Massively parallel.VS serially. </li></ul><ul><li>• Abundance and Complexity: 10 11 and 10 14 of neurons operate in </li></ul><ul><ul><li>parallel in the brain at any given moment, each with between 10 3 and 10 4 abutting connections per neuron. </li></ul></ul><ul><li>• Knowledge Storage: Adaptable VS New information destroys old information. </li></ul><ul><li>• Fault Tolerance: Knowledge is retained through the redundant, distributed encoding information VS the corruption of a conventional computer's memory is irretrievevable and leads to failure as well. </li></ul>Brain vs. Computer Processing Cesare Pianese
  7. 7. Machine Intelligence – Human Intelligence Year is 2020 <ul><li>Computing Power == > Quadrillion/sec/$100 </li></ul><ul><ul><li>5-15 Quadrillion/sec (IBM’s Fastest computer = 100 Trillion) </li></ul></ul><ul><li>High Resolution Imaging (Brian and Neuroscience) </li></ul><ul><ul><li>Human Brain, Reverse Engineering </li></ul></ul><ul><ul><li>Dynamic Neuron Level Imaging/Scanning and Visualization </li></ul></ul><ul><li>Searching, Logical Analysis Reasoning </li></ul><ul><ul><li>Searching for Next Google; Internet Protocol TV (IPTV) </li></ul></ul><ul><ul><ul><li>Every viewer could potentially receive different advertisement based on its profile, search, and shows the viewer has been watched Family will not skip the ads, because it is targeted advertising </li></ul></ul></ul><ul><li>Technology goes Nano and Molecular Level </li></ul><ul><ul><li>Nanotechnology </li></ul></ul><ul><ul><li>Nano Wireless Devices and OS </li></ul></ul><ul><ul><ul><li>Tiny- blood-cell-size robots </li></ul></ul></ul><ul><ul><ul><li>Virtual Reality through controlling Brain Cell Signals </li></ul></ul></ul><ul><li>Who should work and who should get paid? </li></ul>Human or Robots/Machines?
  8. 8. Reasoning ? Deduction ?
  9. 9. Structured Un-Structured Semi-Structured Reasoning ? Deduction ?
  10. 10. Human-Machine-Perception-Based Reasoning Phantoms - Humanoid Machine Agent Intelligent Agent Phantoms Humanoid Human Computer Mind Machine based on Human Brain and Neuroscience
  11. 11. THE HUMAN MIND
  12. 12. The Human Mind Arguably the most important for humankind. The next unexplored frontier of science Mind is what distinguishes humans from the rest of creation Your mind is who you are
  13. 13. Recent Breakthroughs Neurosciences – Focused on understanding the brain - chemistry, synaptic transmission, axonal connectivity, functional MRI Cognitive Modeling – Focused on representation and use of knowledge in performing cognitive tasks - mathematics, logic, language Intelligent Control – Focused on making machines behave appropriately (i.e., achieve goals) in an uncertain environment - manufacturing, autonomous vehicles, agriculture, mining Depth Imaging – Enables geometrical modeling of 3-D world. Facilitates grouping and segmentation. Provides solution to symbol-grounding problem. Computational Power – Enables processes that rival the brain in operations per second. At 10 10 ops, heading for 10 15 ops.
  14. 14. COMPUTATION
  15. 15. EVOLUTION OF COMPUTATION natural language arithmetic algebra algebra differential equations calculus differential equations numerical analysis symbolic computation computing with words precisiated natural language symbolic computation + + + + + + +
  16. 16. COMMON SENSE
  17. 17. <ul><li>Classical Logic is inadequate for ordinary life </li></ul><ul><li>Intuitionism </li></ul><ul><li>Non- Monotonic Logic </li></ul><ul><ul><li>Second thoughts </li></ul></ul><ul><li>Plausible reasoning </li></ul><ul><ul><li>Quick, efficient response to problems when an exact solution is not necessary </li></ul></ul>COMMON SENSE The World Of Objects The Measure Space Qualitative Reasoning Heuristics Rules of thumbs George Polya: “Heuretics&quot;
  18. 18. FUZZY LOGIC COMMON SENSE LOGIC
  19. 19. EVOLUTION OF LOGIC <ul><li>two-valued (Aristotelian): nothing is a matter of degree </li></ul><ul><li>multi-valued: truth is a matter of degree </li></ul><ul><li>fuzzy: everything is a matter of degree </li></ul>
  20. 20. <ul><li>In bivalent logic, BL, truth is bivalent, implying that every proposition, p, is either true or false, with no degrees of truth allowed </li></ul><ul><li>In multivalent logic, ML, truth is a matter of degree </li></ul><ul><li>In fuzzy logic, FL: </li></ul><ul><ul><li>everything is, or is allowed to be, to be partial, i.e., a matter of degree </li></ul></ul><ul><ul><li>everything is, or is allowed to be, imprecise (approximate) </li></ul></ul><ul><ul><li>everything is, or is allowed to be, granular (linguistic) </li></ul></ul><ul><ul><li>everything is, or is allowed to be, perception based </li></ul></ul>
  21. 21. EVOLUTION OF FUZZY LOGIC A PERSONAL PERSPECTIVE (L.A. Zadeh) generality time 1965 1973 1999 1965: crisp sets fuzzy sets 1973: fuzzy sets granulated fuzzy sets (linguistic variable) 1999: measurements perceptions nl-generalization f.g-generalization f-generalization classical bivalent computing with words and perceptions (CWP)
  22. 22. Natural Language Computing Concepts
  23. 23. <ul><li>it is 35 C° </li></ul><ul><li>Eva is 28 </li></ul><ul><li>probability is 0.8 </li></ul><ul><li>It is very warm </li></ul><ul><li>Eva is young </li></ul><ul><li>probability is high </li></ul><ul><li>it is cloudy </li></ul><ul><li>traffic is heavy </li></ul><ul><li>it is hard to find parking near the campus </li></ul>INFORMATION measurement-based numerical perception-based linguistic MEASUREMENT-BASED VS. PERCEPTION-BASED INFORMATION <ul><li>measurement-based information may be viewed as special case of perception-based information </li></ul>
  24. 24. MEASUREMENT-BASED <ul><li>a box contains 20 black and white balls </li></ul><ul><li>over seventy percent are black </li></ul><ul><li>there are three times as many black balls as white balls </li></ul><ul><li>what is the number of white balls? </li></ul><ul><li>what is the probability that a ball picked at random is white? </li></ul><ul><li>a box contains about 20 black and white balls </li></ul><ul><li>most are black </li></ul><ul><li>there are several times as many black balls as white balls </li></ul><ul><li>what is the number of white balls </li></ul><ul><li>what is the probability that a ball drawn at random is white? </li></ul>PERCEPTION-BASED version 2 version 1
  25. 25. COMPUTATION (version 2) <ul><li>measurement-based </li></ul><ul><li>X = number of black balls </li></ul><ul><li>Y 2 number of white balls </li></ul><ul><li>X  0.7 • 20 = 14 </li></ul><ul><li>X + Y = 20 </li></ul><ul><li>X = 3Y </li></ul><ul><li>X = 15 ; Y = 5 </li></ul><ul><li>p =5/20 = .25 </li></ul><ul><li>perception-based </li></ul><ul><li>X = number of black balls </li></ul><ul><li>Y = number of white balls </li></ul><ul><li>X = most × 20* </li></ul><ul><li>X = several *Y </li></ul><ul><li>X + Y = 20* </li></ul><ul><li>P = Y/N </li></ul>
  26. 26. BASIC POINT <ul><li>Conventional methods of systems analysis are oriented toward numerical attributes and measurement-based information. They lack the capability to deal with linguistic attributes and perception-based information </li></ul><ul><li>Computing with words is aimed at adding to methods of systems analysis and decision analysis an important high-level capability—the capability to deal computationally and logically with linguistic attributes and perception-based information </li></ul>
  27. 27. Natural Language Computing Origin
  28. 28. Computation? <ul><li>Traditional Sense: Manipulation of Numbers </li></ul><ul><li>Human: Uses Word for Computation and Reasoning </li></ul><ul><li>Words are less precise than numbers! </li></ul><ul><li>Computing Word <== Natural Language </li></ul>
  29. 29. Natural Language Computing? <ul><li>Human: Uses Natural Languages </li></ul><ul><li>Human: Uses Word for Computation </li></ul><ul><li>Human: Uses Reasoning </li></ul><ul><ul><ul><ul><li>Logic </li></ul></ul></ul></ul><ul><li>Words are less precise than numbers! </li></ul><ul><li>Reality vs. Being Certain </li></ul><ul><li>Fuzzy Set and Fuzzy Logic as basis for Natural Language Computing </li></ul>
  30. 30. Inspired by human’s remarkable capability to perform a wide variety of physical and mental tasks without any measurements and computations and dissatisfied with classical logic as a tool for modeling human reasoning in an imprecise environment, Lotfi A. Zadeh developed the theory and foundation of fuzzy logic with his 1965 paper “Fuzzy Sets” [1] and extended his work with his 2005 paper “Toward a Generalized Theory of Uncertainty (GTU)—An Outline” Theory of Natural Language Computing ORIGIN, CONCEPTS, AND TRENDS Fuzzy Set and Fuzzy Logic as basis for Natural Language Computing
  31. 31. WHAT IS FUZZY LOGIC? fuzzy logic (FL) is aimed at a formalization of modes of reasoning which are approximate rather than exact examples: exact all men are mortal Socrates is a man Socrates is mortal approximate most Swedes are tall Magnus is a Swede it is likely that Magnus is tall
  32. 32. <ul><li>“ Fuzzy logic” is not fuzzy logic </li></ul><ul><li>Fuzzy logic is a precise logic of approximate reasoning and approximate computation </li></ul><ul><li>The principal distinguishing features of fuzzy logic are: </li></ul><ul><li>In fuzzy logic everything is, or is allowed to be graduated, that is, be a matter of degree </li></ul><ul><li>In fuzzy logic everything is allowed to be granulated </li></ul>FUZZY LOGIC—KEY POINTS
  33. 33. When do we use Fuzzy Logic? <ul><li>To exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness, low solution cost and better rapport with reality </li></ul><ul><li>Crisp, fine grained information is not available </li></ul><ul><ul><li>Economic systems, everyday decision-making </li></ul></ul><ul><li>Precise information is costly </li></ul><ul><ul><li>Diagnosis systems, quality control, decision analysis </li></ul></ul><ul><li>Fine-grained information is not necessary </li></ul><ul><ul><li>Cooking, balancing, parking a car </li></ul></ul><ul><li>Coarse-grained information reduces cost </li></ul><ul><ul><li>Camera, consumer products </li></ul></ul>
  34. 34. Natural Language Computing? <ul><li>Fuzzy Set and Fuzzy Logic </li></ul><ul><li>as basis for </li></ul><ul><li>Natural Language Computing </li></ul><ul><li>+ </li></ul><ul><li>NEW TOOLS </li></ul><ul><li>Computing with words and perceptions (CWP) & Precisiated Natural Language (PNL) </li></ul>
  35. 35. Fuzzy Sets (Zadeh 1965) (Info. And Control, 8, 338-353 (1965) <ul><li>A fuzzy set is a class of objects with a continuum of grades of membership </li></ul><ul><li>Each Set is characterized by membership function which assigns to each object a grade of membership </li></ul><ul><li>The notion of a fuzzy set is completely non statistical </li></ul>
  36. 36. PRECISIATION OF “approximately a,” *a x x x a a a 0 1 0 0 1 p  fuzzy graph probability distribution interval x 0 a possibility distribution  x a 0 1   s-precisiation singleton g-precisiation cg-precisiation
  37. 37. CONTINUED <ul><li>GCL-based (maximal generality) </li></ul>x p 0 bimodal distribution g-precisiation X isr R GC-form *a g-precisiation
  38. 38. Fuzzy Concept (Zadeh, 1971) <ul><li>If x is a term, then its meaning, M(x), is a concept </li></ul><ul><li>Level 1 concept : </li></ul><ul><ul><li>K : a set of objects </li></ul></ul><ul><ul><li>Concepts (labels for concepts); “white”, “yellow”, “green”, … </li></ul></ul><ul><ul><li>“ redder than”, “darker than” are level 1 concept, since </li></ul></ul><ul><ul><ul><ul><li>If y1 and y2 are objects (y1,y2), we can calculate µ (y1,y2) such as “darker than” </li></ul></ul></ul></ul><ul><li>Level 2 Concept </li></ul><ul><ul><li>Example : “color” </li></ul></ul><ul><ul><ul><li>This concept is a collection of the concepts M(white), M(green), …, M(black), … </li></ul></ul></ul><ul><li>Level 3 Concept </li></ul><ul><ul><li>Example : “visual attribute” </li></ul></ul><ul><ul><ul><li>This concept is a collection of the concepts such as “color”, “shape”, “size”, … </li></ul></ul></ul><ul><li>Concept at higher level than 1 is much harder to define by exemplification than concepts at level 1. </li></ul>
  39. 39. Fuzzy Concept <ul><ul><li>Natural language for teaching a kid or machine </li></ul></ul><ul><ul><li>Exemplification a set of primitive concepts at level 1 (vocabulary) </li></ul></ul><ul><ul><li>Build up on this vocabulary by defining other level 1 concepts in term of the already defined </li></ul></ul><ul><ul><li>Build up on this vocabulary by defining other higher level concepts in term of the already defined level 1 (or lower level concepts) </li></ul></ul>
  40. 40. Language as fuzzy relation is closer to the Linguistic than formal languages <ul><ul><li>Each Word x in a natural language L may be viewed as a summarized description of a fuzzy subset M(x) of universe of discourse U, with M(x) representing the meaning of x </li></ul></ul><ul><ul><li>Language is a fuzzy correspondence between the element of T and U, where T is a set of terms </li></ul></ul><ul><ul><li>Let x be a term in T. Then the meaning of x, denoted by M(x) is a fuzzy subset of U characterized by a membership function µ (x l y). </li></ul></ul><ul><ul><li>T: white, gray, green, blue, yellow, red, black, … </li></ul></ul><ul><ul><li>T: young, old, middle-aged, not old, not young, …. </li></ul></ul>Computing with Words and Perception
  41. 41. Computation of Meaning by the use of Quantitative Semantics <ul><ul><li>Meaning: </li></ul></ul><ul><ul><ul><li>Simple terms (young, old, very, not, and, or) </li></ul></ul></ul><ul><ul><ul><li>Composite terms (not very young and not very old) </li></ul></ul></ul><ul><ul><li>Quantitative Semantics is a procedure for computing the meaning, M(x), of a composite x in T from the knowledge of the meanings of the simple terms x 1 x 2 … x n </li></ul></ul><ul><ul><li>Computing with Words and Perception </li></ul></ul><ul><ul><li>Precisation of Variables </li></ul></ul>
  42. 42. Fuzzy Grammar for Computation of Meaning for composite terms <ul><ul><li>not very young </li></ul></ul><ul><ul><li>not very young and not very old </li></ul></ul><ul><ul><li>young and not old </li></ul></ul><ul><ul><li>old or not old </li></ul></ul><ul><ul><li>old or not very very young </li></ul></ul><ul><ul><li>young and (old or not young) </li></ul></ul><ul><ul><li>S  A C  O </li></ul></ul><ul><ul><li>S  S or A C  Y </li></ul></ul><ul><ul><li>A  B O  very O </li></ul></ul><ul><ul><li>A  A and B Y  very Y </li></ul></ul><ul><ul><li>B  not C O  old </li></ul></ul><ul><ul><li>C  (S) Y  young </li></ul></ul><ul><ul><li>B  C </li></ul></ul>
  43. 43. Fuzzy Grammar for Computation of Meaning for composite terms
  44. 44. Fuzzy Grammar for Computation of Meaning for composite terms
  45. 45. Fuzzy Grammar for Computation of Meaning for composite terms
  46. 46. Fuzzy Logic (Outline of New Approach to the Analysis of Complex Systems and Decision Process, IEEE Trans. On system, man and cybernetics, Vol. SMC-3, No. 1, Jan 1973, 28-44) <ul><li>The use of Linguistic variables </li></ul><ul><li>Simple relations between variables by fuzzy conditional statement </li></ul><ul><li>Complex relations by fuzzy Algorithms </li></ul><ul><li>IF x is small and x is not large THEN y is very large </li></ul><ul><li>IF x is not very small THEN y is very large </li></ul><ul><li>IF x is not small and not large THEN y is not very large </li></ul>
  47. 47. The use of Linguistic variables Meaning Information Summarization Humans: Ability to summarize information finds its most pronounce manifestation in the use of Natural Languages Linguistic Variables: Each word x in a natural language L may be viewed as a summarized description of a fuzzy subset M(x) of universe of discourse U, with M(x) representing the meaning of x.
  48. 48. The use of Linguistic variables Object: red Meaning: M(Red) Object: flower Meaning: M(flower) Object: red flower Meaning: M (red) ∩ M (flower) ∩ : Intersection or Min. Operator Object: Variable: Color of the object Values: red, blue, yellow, green Values for the Object: Labels of fuzzy sets Attribute: Color  Fuzzy Variable Values: red, blue, …, Labels of the fuzzy sets Attributes: Height Values: tall, not tall, somewhat tall, very tall, … Sentences: Label (tall, red,….), Negation (not), Connective (and, but, or), and hedges (very, somewhat, quite, more or less, …)
  49. 49. Simple relations between variables by fuzzy conditional statement <ul><li>IF x is small THEN y is very large </li></ul><ul><li>IF x is not very small THEN y is very large </li></ul><ul><li>IF x is not small and not large THEN y is not very large </li></ul>
  50. 50. Complex relations by fuzzy Algorithms <ul><li>Fuzzy Algorithm is an ordered sequences of instructions </li></ul><ul><li>Reduce x slightly if y is large </li></ul><ul><li>Increase x very slightly if y is not very large and not very small </li></ul><ul><li>If x is small then stop; otherwise increase x by 2. </li></ul>
  51. 51. VARIABLES AND LINGUISTIC VARIABLES <ul><li>one of the most basic concepts in science is that of a variable </li></ul><ul><li>variable -numerical (X=5; X=(3, 2); …) </li></ul><ul><li>-linguistic (X is small; (X, Y) is much larger) </li></ul><ul><li>a linguistic variable is a variable whose values are words or sentences in a natural or synthetic language (Zadeh 1973) </li></ul><ul><li>the concept of a linguistic variable plays a central role in fuzzy logic and underlies most of its applications </li></ul>
  52. 52. Fuzzy sets, logics and reasoning Examples (Zadeh 1973) <ul><li>Set: U </li></ul><ul><li>U = 1 + 2 + 3 + 4 + 5 </li></ul><ul><li>small = 1/1 + .8/2 + .6/3 + .4/4 + .2/5 </li></ul><ul><li>(degree/set) </li></ul><ul><li>very x := x 2 </li></ul><ul><li>very small = 1/1 + 0.64/2 + 0.36/3 + 0.16/4 + 0.04 /5 </li></ul><ul><li>very very small = very (very x) = very (x 2 )= x 4 </li></ul><ul><li>very very small = 1/1 + 0.4/2 + 0.1/3 </li></ul>
  53. 53. Computation of the Meaning of Values of Linguistic Variables <ul><li>not very small = not (very small)= ¬ (very small) = ¬ (small 2 ) </li></ul><ul><li>very small = 1/1 + 0.64/2 + 0.36/3 + 0.16/4 + 0.04 /5 </li></ul><ul><li>not very small = 0/1 + 0.36/2 + 0.64/3 + 0.84/4 + 0.96/5 </li></ul>
  54. 54. Computation of the Meaning of Values of Linguistic Variables <ul><li>very very large = very (very large)= very (large 2 ) large 4 </li></ul><ul><li>not very very large = ¬ large 4 </li></ul><ul><li>X= not very small and not very very large: </li></ul><ul><li>¬ (small 2 ) ∩ ¬ ( large 4 ) = Min. ( ¬ (small 2 ), ¬ ( large 4 ) </li></ul>
  55. 55. Fuzzy Conditional Statement and Compositional Rule of Inference <ul><li>In general : IF A THEN B ; A x B ; A  B </li></ul><ul><li>Min (A,B) ; x : Intersection or ∩ </li></ul><ul><li>A = 1/1 + 0.8/2 </li></ul><ul><li>B = 0.6/1 + 0.9/2 + 1/3 </li></ul>IF ( x is) large THEN (y is) small IF (the road) slippery THEN (driving is) dangerous R : Fuzzy Relation
  56. 56. Fuzzy Conditional Statement and Compositional Rule of Inference <ul><li>IF A 1 THEN B 1 ELSE IF A 2 THEN B 2 ELSE IF A n THEN B n ELSE </li></ul><ul><li>= A 1 x B 1 + A 1 x B 1 + . . .+ A n x B n </li></ul><ul><li>IF A THEN (IF B THEN C ELSE D) ELSE E </li></ul><ul><li>= A x B x C + A x ¬ B x D + ¬ A x E </li></ul>IF A THEN B ELSE C : A x B + ( ¬ A x C) + : union or U
  57. 57. Fuzzy Conditional Statement and Compositional Rule of Inference IF ( x is) very small THEN (y is) large ELSE (y is) not very large  A x B + (¬A x C) A: small B: large C: not very large x : Min. or ∩ or intersection + : Max. or U or union Operation to calculate: Min Max operator small = 1/1 + 0.8/2 + 0.6/3 + 0.4/4 + 0.2/5 large = 0.2/1 + 0.4/2 + 0.3/3 + 0.8/4 + 1/5 R : Fuzzy Relation = A x B + (¬A x C)
  58. 58. Fuzzy Conditional Statement and Compositional Rule of Inference R : Fuzzy Relation IF ( x is) very small THEN (y is) large ELSE (y is) not very large x = very small (x 2 ) x o R = y =[0.36 0.4 0.6 0.8 1]
  59. 59. Fuzzy Conditional Statement and Compositional Rule of Inference R : Fuzzy Relation IF ( x is) very small THEN (y is) large ELSE (y is) not very large x : very small = (small 2 ) = [ 1 0.64 0.36 0.16 0.04] x o R = y =[0.36 0.4 0.6 0.8 1] ≈ not very small
  60. 60. Fuzzy Conditional Statement and Compositional Rule of Inference small or medium not very large very very large very very small very small large very very large very large medium medium not very large not small large small x y A B Inferred Given
  61. 61. Joint Probability <ul><li>P: </li></ul><ul><li>if X is small then p is small </li></ul><ul><li>If X is medium then p is large </li></ul><ul><li>If X is large then p is small </li></ul><ul><li>Q: </li></ul><ul><li>if Y is small then q is large </li></ul><ul><li>If Y is medium then q is small </li></ul><ul><li>If Y is large then q is large </li></ul>
  62. 62. Joint Probability P: small x small + medium x large + large x small Q: small x large + medium x small + large x small (P,Q)= small x small (small* large) + + small x medium x (small *small) + … Large x large x (small * large) * : the arithmetic product in fuzzy arithmetic small large large medium small small X p P small large small medium large small Y q Q
  63. 63. Possibilistic Relational Universal Fuzzy PRUF
  64. 64. PRUF – A meaning Representation Language for Natural Languages (Zadeh, 1977) <ul><li>Possibilistic Relational Universal Fuzzy </li></ul><ul><li>Assumption: imprecision is possibilistic rather than probabilistic in nature </li></ul><ul><li>The Logic: Fuzzy logic, rather than two-valued or multivalued- logic </li></ul><ul><li>The quantifiers in PRUF are allowed to be linguistic, “most”, “many”, “some”, “few” </li></ul>
  65. 65. PRUF – A meaning Representation Language for Natural Languages (Zadeh, 1977) <ul><li>The concept of Semantic Equivalence and Semantic Entailment in PRUF provide a basis for Question-Answering (Q&A) and Inference from fuzzy premises </li></ul><ul><li>Foundation for Approximate Reasoning </li></ul><ul><li>Language for representation of imprecise knowledge and as a means of precisiation of fuzzy propositions expressed in a natural language . </li></ul><ul><li>Precisiated Natural Language </li></ul><ul><li>Precisation of Meaning </li></ul>
  66. 66. PRUF – A meaning Representation Language for Natural Languages (Zadeh, 1977) <ul><li>Translation rules in PRUF: </li></ul><ul><li>Type I: pertaining to modification </li></ul><ul><li>Type II: pertaining to composition </li></ul><ul><li>Type III: pertaining to quantification </li></ul><ul><li>Type IV: pertaining to qualification </li></ul>
  67. 67. PRUF Type I: pertaining to modification X is very small X is much larger than Y Eleanor was very upset The Man with the blond hair is very tall PRUF Type II: pertaining to composition X is small and Y is large (conjunctive composition) X is small or Y is large (disjunctive composition) If X is small then Y is large (conditional and conjunctive composition) If X is small the Y is large else Y is very large (conditional and conjunctiv composition)
  68. 68. PRUF – Type III: pertaining to quantification Most Swedes are tall Many men are much taller than most men Most tall men are very intelligent PRUF – Type IV: pertaining to qualification <ul><li>Abe is young is not very true (truth qualification) </li></ul><ul><li>Abe is young is quite probable (probability qualification </li></ul><ul><li>Abe is young is almost impossible (possibility qualification) </li></ul>
  69. 69. Proposition p p : N is F Modified proposition p+ p+ = N is mF m: not, very, more or less, quite, extremely, etc. PRUF Type I: pertaining to modification Rules of Type I: Basis is the Modifier
  70. 70. PRUF Type II: pertaining to composition Rules of Type II: operation of composition
  71. 71. PRUF Type II: pertaining to composition Rules of Type II: operation of composition
  72. 72. PRUF Type II: pertaining to composition Rules of Type II: operation of composition
  73. 73. PRUF Type II: pertaining to composition Rules of Type II: operation of composition F mn … … F m1 … … … … F 1n … F 12 F 11 X 1n … X 2 X 1 R
  74. 74. Compactification Algorithm Interpretation A Simple Algorithm for Qualitative Analysis Rule Extraction and Building Decision Tree Dr. Nikravesh and Prof. Zadeh (2005) (Zadeh, 1976)
  75. 75. Compactification Algorithm Interpretation Test Attribute Set a 1 a 2 o a m F 1 a 1 n a 2 n o a m n O O O a 1 2 a 2 2 o a m 2 a 1 1 a 2 1 o a m 1 A n o A 2 A 1 ?b a n o a 2 a 1
  76. 76. Table 1 (intermediate results) Group 1(initial) Pass (1) Pass (2) Pass (3) a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 a 1 F 1 * * * * a 2 2 a 2 2 a 2 2 a 2 2 a 1 1 a 2 1 a 3 1 * a 2 3 * a 3 1 a 1 3 a 2 3 a 2 2 a 2 2 * * a 1 3 a 1 3 a 1 3 a 1 3 a 2 3 a 2 3 a 2 3 a 2 3 a 1 2 a 2 2 a 2 2 a 2 2 a 1 2 a 2 2 a 2 2 a 2 2 a 1 1 a 1 1 a 2 1 a 3 1 a 3 1 a 1 1 a 2 1 a 3 1 A 3 A 2 A 1
  77. 77. MAXIMALLY COMPACT REPRESENTATION
  78. 79. PRUF Type II: pertaining to composition Rules of Type II: operation of composition small  SMALL large  LARGE very small  SMALL 2 not small  SMALL’ not very large  (LARGE 2 )’ R  SMALL x LARGE + (SMALL 2 ) x (LARGE 2 )’ + SMALL’ x SMALL 2 Large not very large very small small very small not small Y X
  79. 80. Type III: pertaining to quantification Rules of Type III: p: Q N are F
  80. 81. Type III: pertaining to quantification Rules of Type III: p: Q N are F
  81. 82. PRUF – Type IV: pertaining to qualification Rules of Type IV: q: p is ү
  82. 83. PRUF – Type IV: pertaining to qualification Rules of Type IV: q: p is ү
  83. 84. PRUF – Type IV: pertaining to qualification Rules of Type IV: q: p is ү
  84. 85. NEW TOOLS
  85. 86. PT BL FL + bivalent logic probability theory Theory of Generalized-Constraint-Based Reasoning CW PT: standard bivalent-logic-based probability theory CTPM : Computational Theory of Precisiation of Meaning PNL: Precisiated Natural Language CW: Computing with Words GTU: Generalized Theory of Uncertainty GCR: Theory of Generalized-Constraint-Based Reasoning CTPM GTU PNL GC Tools in current use New Tools GCR Generalized Constraint fuzzy logic NEED FOR NEW TOOLS
  86. 87. PRECISIATED NATURAL LANGUAGE PNL
  87. 88. WHAT IS PRECISIATED NATURAL LANGUAGE (PNL)? PRELIMINARIES <ul><li>a proposition, p, in a natural language, NL, is precisiable if it translatable into a precisiation language </li></ul><ul><li>in the case of PNL, the precisiation language is the Generalized Constraint Language, GCL </li></ul><ul><li>precisiation of p, p*, is an element of GCL (GC-form) </li></ul>
  88. 89. WHAT IS PNL? <ul><li>PNL is a sublanguage of precisiable propositions in NL which is equipped with two dictionaries: (A) NL to GCL; (B) GCL to PFL (Protoform Language); and (C) a collection of rules of deduction (rules of generalized constrained propagation) expressed in PFL. </li></ul>
  89. 90. PRECISIATED NATURAL LANGUAGE (PNL) NL GCL generalized constraint form of type r p X isr R translation generalized constraint form of type r (GC(p)) p translation precisiation language (GCL) precisiation explicitation GC-form CSNL precisiable propositions in NL p*
  90. 91. PNL AND THE COMPUTATIONAL THEORY OF PERCEPTIONS in the computational theory of perceptions (CTP), perceptions are dealt with through their descriptions in a natural language perception = descriptor(s) of perception <ul><li>a proposition, p, in NL qualifies to be an object of </li></ul><ul><li>computation in CTP if p is in PNL </li></ul>
  91. 92. DEFINITION OF p: ABOUT 20-25 MINUTES time time time time 20 25 20 25 20 25 A P B 6 0 1 0 0 1 1 c-definition: f-definition: f.g-definition: PNL-definition: Prob (Time is A) is B   
  92. 93. PRECISIATION OF “approximately a,” *a x x x a a 20 25 0 1 0 0 1 p  fuzzy graph probability distribution interval x 0 a possibility distribution  x a 0 1   s-precisiation singleton g-precisiation cg-precisiation
  93. 94. CONTINUED <ul><li>GCL-based (maximal generality) </li></ul>x p 0 bimodal distribution g-precisiation X isr R GC-form *a g-precisiation
  94. 95. GC
  95. 96. THE CENTERPIECE OF PNL IS THE CONCEPT OF A GENERALIZED CONSTRAINT (ZADEH 1986)
  96. 97. THE BASICS OF PNL <ul><li>The point of departure in PNL is the key idea: </li></ul><ul><ul><li>A proposition, p, drawn from a natural language, NL, is precisiated by expressing its meaning as a generalized constraint </li></ul></ul><ul><ul><li>In general, X, R, r are implicit in p </li></ul></ul>p X isr R constraining relation Identifier of modality (type of constraint) constrained (focal) variable The concept of a generalized constraint serves as a bridge from natural languages to mathematics
  97. 98. GENERALIZED CONSTRAINT (Zadeh 1986) <ul><li>Bivalent constraint (hard, inelastic, categorical:) </li></ul>X  C constraining bivalent relation X isr R constraining non-bivalent (fuzzy) relation index of modality (defines semantics) constrained variable <ul><li>Generalized constraint: </li></ul>r:  | = |  |  |  | … | blank | p | v | u | rs | fg | ps |… bivalent non-bivalent (fuzzy)
  98. 99. CONTINUED <ul><li>constrained variable </li></ul><ul><ul><li>X is an n-ary variable, X= (X 1 , …, X n ) </li></ul></ul><ul><ul><li>X is a proposition, e.g., Leslie is tall </li></ul></ul><ul><ul><li>X is a function of another variable: X=f(Y) </li></ul></ul><ul><ul><li>X is conditioned on another variable, X/Y </li></ul></ul><ul><ul><li>X has a structure, e.g., X= Location (Residence(Carol)) </li></ul></ul><ul><ul><li>X is a generalized constraint, X: Y isr R </li></ul></ul><ul><ul><li>X is a group variable. In this case, there is a group, G[A]: (Name 1 , …, Name n ), with each member of the group, Name i , i =1, …, n, associated with an attribute-value, A i . A i may be vector-valued. Symbolically </li></ul></ul><ul><ul><li>G[A]: (Name 1 /A 1 +…+Name n /A n ) </li></ul></ul><ul><ul><li>Basically, X is a relation </li></ul></ul>
  99. 100. SIMPLE EXAMPLES <ul><li>“ Check-out time is 1 pm,” is an instance of a generalized constraint on check-out time </li></ul><ul><li>“ Speed limit is 100km/h” is an instance of a generalized constraint on speed </li></ul><ul><li>“ Vera is a divorcee with two young children,” is an instance of a generalized constraint on Vera’s age </li></ul>
  100. 101. GENERALIZED CONSTRAINT—MODALITY r X isr R r: = equality constraint: X=R is abbreviation of X is=R r: ≤ inequality constraint: X ≤ R r:  subsethood constraint: X  R r: blank possibilistic constraint; X is R; R is the possibility distribution of X r: v veristic constraint; X isv R; R is the verity distribution of X r: p probabilistic constraint; X isp R; R is the probability distribution of X
  101. 102. CONTINUED <ul><li>r: rs random set constraint; X isrs R; R is the set- </li></ul><ul><li>valued probability distribution of X </li></ul><ul><li>r: fg fuzzy graph constraint; X isfg R; X is a function and R is its fuzzy graph </li></ul><ul><li>r: u usuality constraint; X isu R means usually (X is R) </li></ul><ul><li>r: g group constraint; X isg R means that R constrains the attribute-values of the group </li></ul><ul><li>Primary constraints: possibilistic, probabilisitic and veristic </li></ul><ul><li>Standard constraints: bivalent possibilistic, probabilistic and bivalent veristic </li></ul>
  102. 103. Natural Language Computing NeuFCSearch Conceptual-Based Text Search Based on PNL and Neuroscience
  103. 104. BASIC PROBLEM <ul><li>identification of query-relevant information </li></ul><ul><li>relevance-ranking of query-relevant information </li></ul>question-answering system search engine <ul><li>deduction from query-relevant information </li></ul>meta-deduction +
  104. 106. Concept-Based Intelligent Decision Analysis BISC-DSS Deductive Web Engine Beyond the Semantic Web NeuFCSearch
  105. 107. NeuSearch: Neuroscience Approach Search Engine Based on Conceptual Semantic Indexing Neuro-Fuzzy Conceptual Search (NeuFCS) Imprecise Search Lycos, etc. GA-GP Context-Based tf-idf; Ranked tf-idf Topic, Title, Summarization Concept-Based Indexing Keyword search; classical techniques; Google, Teoma, etc. Use Graph Theory and Semantic Net. NLP with GA-GP Based NLP; Possibly AskJeeves. NeuFCS RBF PRBF GRRBF ANFIS RBFNN (BP, GA-GP, SVM) Probability Bayesian Fuzzy NNnet (BP, GA-GP, SVM) LSI FCS Based on Neuroscience Approach Classical Search
  106. 108. NeuFCS Neuro-Fuzzy Conceptual Search based on Neuro-Science
  107. 109. Documents Space or Concept and Context Space Based on SOM or PCA Word Space Concept-Context Dependent Word Space W(i, j) is calculated based on Fuzzy-LSI or Probabilistic LSI (In general form, it can be Calculated based on PNL) i: neuron in word layer j: neuron in document or Concept-Context layer j: neuron in document or Concept-Context layer k: neuron in word layer Document (Corpus) Neu-FCS
  108. 110. PNL-Based Neuro-Fuzzy Conceptual Search Using Neuroscience
  109. 111. ORGANIZATION OF WORLD KNOWLEDGE EPISTEMIC (KNOWLEDGE-DIRECTED) LEXICON (EL) (ONTOLOGY-RELATED) <ul><li>i (lexine): object, construct, concept (e.g., car, Ph.D. degree) </li></ul><ul><li>K(i): world knowledge about i (mostly perception-based) </li></ul><ul><li>K(i) is organized into n(i) relations R ii , …, R in </li></ul><ul><li>entries in R ij are bimodal-distribution-valued attributes of i </li></ul><ul><li>values of attributes are, in general, granular and context-dependent </li></ul>network of nodes and links w ij = granular strength of association between i and j i j r ij K(i) lexine w ij
  110. 112. EPISTEMIC LEXICON lexine i lexine j r ij : i is an instance of j (is or isu) i is a subset of j (is or isu) i is a superset of j (is or isu) j is an attribute of i i causes j (or usually) i and j are related r ij
  111. 113. GENERALIZED CONSTRAINT <ul><li>standard constraint: X  C </li></ul><ul><li>generalized constraint: X isr R </li></ul>X isr R copula GC-form (generalized constraint form of type r) type identifier constraining relation constrained variable <ul><li>X= (X 1 , …, X n ) </li></ul><ul><li>X may have a structure: X=Location (Residence(Carol)) </li></ul><ul><li>X may be a function of another variable: X=f(Y) </li></ul><ul><li>X may be conditioned: (X/Y) </li></ul>
  112. 114. CONTINUED r : rs random set constraint; X isrs R; R is the set- valued probability distribution of X r: fg fuzzy graph constraint; X isfg R; X is a function and R is its fuzzy graph r: u usuality constraint; X isu R means usually (X is R) r: ps Pawlak set constraint: X isps ( X , X) means that X is a set and X and X are the lower and upper approximations to X
  113. 115. Neuroscience Approach for Concept and Context Dependency
  114. 116. Based on PNL approach, w(i,j) is defined based on ri,j as follows:
  115. 117. NeuSearch Model
  116. 118. Radial Basis Function is used to extract Concept ... x1 x2 x3 N x X ... ) ( 1 X  ) ( 3 X  ) ( 1 X m  ) ( 2 X  Increased dimension: N->m1 function linear non : ) ( X i  3 w 1 m w 1 w 2 w  Output y More likely to be linearly separated    i m i i i X w y 1 ) ( 
  117. 119. SOM are used to make 2-D plot of Concepts
  118. 120. Concept 1 Concept 2 Concept n N Concepts are Extracted based on SOM and RBFs SOM/LVQ are used to make 2-D plot of Concepts Supervised and Unsupervised
  119. 121. PNL-Based Conceptual Fuzzy Sets Using Neuroscience Interconnection based on Mutual Information i: neuron in document layer j: neuron in word layer r ij : i is an instance of j (is or isu) i is a subset of j (is or isu) i is a superset of j (is or isu) j is an attribute of i i causes j (or usually) i and j are related Word Space Concept-Context Dependent Word Space i: neuron in word layer j: neuron in document or Concept-Context layer j: neuron in document or Concept-Context layer k: neuron in word layer Document (Corpus)
  120. 122. Word Space Input: Word Neu-FCS Activated Document or Concept-Context Output: Concept-Context Dependent Word
  121. 123. Activated Document or Concept-Context Word Space Input: Word Neu-FCS Output: Concept-Context Dependent Word Document (Corpus)
  122. 124. FC-DNA as a basis for Common Sense Knowledge, Human Reasoning and Deduction Chinese-DNA ==> M(x) Fuzzy Logic ==> Human Reasoning Fuzzy Logic ==> Reasoning with C-DNA Z-Compact ==> Fine Tune C-DNA Tree Z-Compact ==> C-DNA & M(x) C-DNA & Z-Compact ==> Chinese NLC Rules PRUF & C-DNA ==> Foundation for Chinese NLP PRUF & C-DNA ==> Human Reasoning and Deduction Capability
  123. 125. FC-DNA as a basis for Next Generation of Concept-Based Search Engine Chinese-DNA ==> M(x) Fuzzy Logic ==> Human Reasoning Fuzzy Logic ==> Reasoning with C-DNA Z-Compact ==> Fine Tune C-DNA Tree Z-Compact ==> C-DNA & M(x) C-DNA & Z-Compact ==> Chinese NLC Rules PRUF & C-DNA ==> Foundation for Chinese NLP PRUF & C-DNA ==> Human Reasoning and Deduction Capability FC-DNA ==> NLP Concept-Based Search Engine
  124. 126. Field Theory and Computing with Words New Computing Machine for Computation and Reasoning with Information Presented in Natural Language and Words Masoud Nikravesh and Germano Resconi

×