Natural Language Generation from First-Order Expressions 1               Natural Language Generation from First-Order Expr...
Natural Language Generation from First-Order Expressions 2Sentence (1) can be represented in a first-order form as LOVE(JO...
Natural Language Generation from First-Order Expressions 3A descriptive predicate describes a property of its only argumen...
Natural Language Generation from First-Order Expressions 4An expression may have multiple instance of the same predicate. ...
Natural Language Generation from First-Order Expressions 5Quantifiers: These include ∃ and ∀. Quantifiers can be associate...
Natural Language Generation from First-Order Expressions 6All the sentences can be associated with a first-order represent...
Natural Language Generation from First-Order Expressions 7∪a∈A Ra where Ra∈RSaExtract the set of all quantifier instances ...
Natural Language Generation from First-Order Expressions 8Figure 3: Overgeneration process
Natural Language Generation from First-Order Expressions 9Each path identified represents a candidate natural language sen...
Natural Language Generation from First-Order Expressions 10This method shall become computationally intensive if there are...
Natural Language Generation from First-Order Expressions 11Determine a map between each instance of p and an element r∈Rpr...
Natural Language Generation from First-Order Expressions 12            a    woman      love(s)    John            r3     r...
Natural Language Generation from First-Order Expressions 13The classes used by this model are:         Class              ...
Natural Language Generation from First-Order Expressions 14required to process limited expressions. Possible future enhanc...
Upcoming SlideShare
Loading in …5
×

Natural Language Generation from First-Order Expressions

1,073 views

Published on

In this paper I discuss an approach for generating natural language from a first-order logic representation. The approach shows that a grammar definition for the natural language and a
lambda calculus based semantic rule collection can be applied for bi-directional translation using an overgenerate-and-prune mechanism.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,073
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
52
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Natural Language Generation from First-Order Expressions

  1. 1. Natural Language Generation from First-Order Expressions 1 Natural Language Generation from First-Order Expressions Thomas Mathew tam52@georgetown.edu Department of Linguistics Georgetown University, Washington DC 20057-1232 May 09 2008 Abstract: In this paper I discuss an approach for generating natural language from a first-order logic representation. The approach shows that a grammar definition for the natural language and a lambda calculus based semantic rule collection can be applied for bi-directional translation using an overgenerate-and-prune mechanism. Keywords: First-Order Logic, Natural Language Generation, Reversible Grammar/Semantic Rules1. IntroductionFirst-order logic has been applied in natural language for question-and-answering systems which utilize the benefitof a mathematical representation of a domain against which independent expressions can be validated or discreditedusing mathematical operations. A second functional area, which is of greater interest from the perspective of theproblem I am trying to solve, is to utilize a first-order representation as an intermediate abstract representation in alanguage ‘transducer’ which converts one natural language representation to another. Application areas includeMachine Translation, Text Summarization and Text Simplification. Such a transducer would depend on bi-directional translations between natural language and first-order logic. The next section in this paper glosses over thetranslation of language to first-order logic with the intent of introducing the reader to the grammar and semanticrules that can be utilized for such a translation. The rest of the paper discusses how the same grammar and rules canbe applied to solve the reverse problem of translating first-order logic to natural language.The concept of using bi-directional grammars is not a new one - Strzalkowski et al 1994 discuss in depth aboutgrammars that can perform bi-directional translation between natural language and a semantic abstraction. Oneaspect discussed is the ease of maintainability when comparing the effort of maintaining one grammar and rulesetversus that of maintaining two independent grammars and rulesets. Strzalkowski classifies three approaches on howa bi-directional grammar can be utilized – (a) a parser and generator using a different evaluation strategy but sharinga grammar (b) a parser and generator sharing an evaluation strategy but are developed as independent programs (c) aprogram that can perform as both a parser and a generator. This paper suggests an approach that falls into category(a).With regards to similar work – (a) Whitney 1988 discusses a method using an independent grammar for translatinghigher-order semantic representations into language using first-order predicate calculus (b) Dymetman et al 1988discuss using a bi-directional grammar for machine translation via an intermediate higher-order semanticrepresentation.2. First-Order LogicFirst-order logic is a representation system utilizing an unambiguous formal language based on mathematicalstructures. The language used allows constructions covered by prepositional logic extended with predicates andquantification. Such a representation can be used to symbolize the semantic content in a natural languageconstruction. Consider: (1) John loves Mary
  2. 2. Natural Language Generation from First-Order Expressions 2Sentence (1) can be represented in a first-order form as LOVE(JOHN, MARY) where LOVE is a predicate whichdefines a relation between its arguments JOHN and MARY. Methods for deriving a first-order form from Englishhave been explored in depth. Blackburn & Bos 2003 describe the use of lambda calculus as one such method whichinvolves annotating the grammatical rules that model the language with an appropriate lambda expression, This isshown below for sentence (1): Grammar Rule λ-Expression Predicate Argument Generated Generated S→ NP VP NP@VP - - NP → Nproper Nproper - - Nproper → John λp.p@JOHN - JOHN Nproper → Mary λp.p@MARY - MARY VP → Vtransitive NP Vtransitive@NP - - Vtransitive → love(s) λpλq.(p@λr.LOVE(q, r)) LOVE - Table 1: Sample rules to generate first-order representation from EnglishBy parsing the sentence using the grammar rules and appropriately evaluating and β-reducing the correspondinglambda expressions (from Blackburn & Bos 2003), the equivalent first-order logic form can be derived. The β-reduction steps are shown in Figure 1. Figure 1: Deriving a first-order logic representationThe discussion above is an over-simplification of the process to generate first-order logic – there are many specificissues with natural language relating to ambiguity and actual language specification that would make this a hardtask. For the task at hand, however I am more concerned about the reverse translation ability and for this reason Iassume that grammatical rules defining a natural language which are annotated with lambda expressions wouldsuffice in generating a sufficiently descriptive first-order translation.Predicate Types: A predicate in a first-order logic expression can be any of three types – relational, descriptive orfunctional. A relational predicate describes the relationship between 2 or more entities. An example of a relationalpredicate is LOVE(JOHN, MARY). A relational predicate would generally be derived from either a transitive/di-transitive verb, a dependent clause or a prepositional phrase.
  3. 3. Natural Language Generation from First-Order Expressions 3A descriptive predicate describes a property of its only argument and would generally be derived from either a noun,adjective or an intransitive verb. This is shown in the examples below: (2) BOXER(MARY) ≡ Mary is a boxer (3) BOXER(MARY) ∧ WOMAN(MARY) ≡ Mary is a boxer and a womanA functional predicate is used to describe an underspecified entity using a fully-described entity e.g.MOTHER(JOHN) which can be derived from genitives such as John’s mother or NP constructions such asMother of John. A functional predicate can be used as an argument to a relational or descriptive predicate e.g.LOVE(JOHN, MOTHER(JOHN)). Unlike the other predicate types, a functional predicate does not establish atruth condition. Figure 2: Predicate TypesArgument Types: Arguments to a predicate can be constants, variables or a functional predicate. A constantargument specifies a named entity such as JOHN or MARY. If a variable is used as an argument to a predicate, itwould be quantified in some form to represent either an existential or universal condition as shown in (4). (4) ∃x (BOXER(x) ∧ WOMAN(x))3. Natural Language GenerationGiven that the first-order representation is unambiguous, if a natural language sentence S can be transformed into alogical expression E without loss of meaning, then it should be possible to transform E back to a set of sentences ofwhich S is a member. For machine translation problems, it may suffice to select the first valid language sentencethat can be mapped to the expression. For other problems such as text simplification, the language generator mayneed to generate multiple sentences and apply a ranking method to determine the simplest construction.Each component of a first-order representation creates different complexities to address in the language generationprocess. The following is a limited discussion of some issues that make the reverse translation a hard problem. Thediscussion is based on the notion that a component of an expression can be considered to be associated to a token inthe language representation which has a specific part-of-speech.Predicates: A specific predicate can be associated to tokens with different part-of-speech tags e.g. the descriptivepredicate WOMAN() can be realized from translation from either a noun (a woman) or an adjective (a lady ..). Inlanguage generation, selection of one specific token associated with a particular part-of-speech would influenceselection of tokens associated with other descriptive predicates that control the same argument e.g. two nounsdescribing the same entity in a transitive construction is unlikely.
  4. 4. Natural Language Generation from First-Order Expressions 4An expression may have multiple instance of the same predicate. In a corresponding language representation, it ispossible that all such predicate instances with the same underlying predicate can be associated with a singlelanguage token as shown below. This would generally involve language conjunction. (5) LAUGH(JOHN) ∧ LAUGH(MARY) ≡ John and Mary laugh ≡ John laughs and Mary laughsFunctional predicates can be associated to either genitives or prepositions and may need to be treated independentlyof non-functional predicates.Arguments: An argument is likely to be referenced by more than one predicate in a sentence. It is also likely that allsuch instances of the same argument would all be associated back to a single token. This is not always the casethough as conjunction and pronoun instances could result in multiple tokens referencing the same entity. (6) LOVE(JOHN, MARY) ∧ LOVE(JOHN, DOG(JOHN)) ≡ John loves Mary and John loves his dogFor the purpose of this paper, resolution of arguments to pronouns is considered out of scope and it is consideredsufficient to render the translation of the above expression as John loves Mary and John loves John’s dog. Thisis because the anaphor resolution of the pronoun his cannot be handled by the approach described in the previoussection. As an alternative, anaphor resolution can be considered as a pre-processing step in first-order expressiongeneration and the generation of pronouns as a post-processing step in language generation.Operators: The first-order language supports the set of operators {∧, ∨, →, =, ¬}. An operator can be directlyrelated back to a token in the language representation or it could also be implied by the ordering of tokens. Specificoperators are described below:The conjunction operator ∧ can be directly associated to the conjunction token and. It could also represent therelationship between a verb and a noun managed by a determiner such as a or some (7) or an adjective+noun (8). (7) ∃x (BARK(x) ∧ DOG(x)) ≡ a dog barks (8) ∃x (BARK(x) ∧ GREEN(x) ∧ DOG(x)) ≡ a green dog barksThe disjunction operator ∨ can be directly associated to the token or.The implication operator → can be directly associated to tokens such as if, whenever. It could also represent therelationship between a verb and a noun managed by a determiner such as every (9) or an adjective+noun (10) or acopular construction. (9) ∀x (DOG(x) → BARK(x)) ≡ every dog barks (10) ∀x (EGG(x) ∧ GREEN(x) → LIKE(SAM, x)) ≡ Sam likes green eggsThe identity operator can also be represented as a predicate using the form EQUAL(x, y). Identity can result out ofan adjective.The negation ¬ operator can be directly associated to the adverb token not. It could also represent the relationshipbetween a verb and a noun managed by an adjective such as two: (11) ∃x (DOG(x) ∧ BARK(y) ∧ ∃x (DOG(y) ∧ BARK(y) ∧ ¬(x = y)) ≡ two dogs barkThe negation operator can influence the association between other operators to tokens. In the example below theoperator ∧ is associated to the token or because of the negation operation. (12) (¬LIKE(SAM, EGG) ∧ ¬LIKE(SAM, HAM)) ≡ Sam does not like eggs or ham
  5. 5. Natural Language Generation from First-Order Expressions 5Quantifiers: These include ∃ and ∀. Quantifiers can be associated to both determiners and adjectives. These areshown in the above examples.Parenthesis: This could impact the scoping of a predicate or operator.The complexity in deciphering the generation issues listed earlier is quite considerable. Given the structure of thefirst-order representation, the approach described in the previous section of using lambda calculus to translateEnglish to first-order logic is not feasible for language generation. It would be still be ideal to utilize the rules thatare used to derive first-order logic to guide the reverse translation.One possible solution is to apply an overgenerate-and-prune strategy. This approach has been successfully applied tonatural language problems including language generation. This involves weakening the restrictions to allow forovergeneration of language. Selectional restrictions can them be imposed to weed out bad translations. Thisapproach is described in the following sections.4. Overgeneration approachConsider the simple grammar below annotated with lambda expressions:Non-terminal rules: Grammar Rule λ-Expression Predicate Argument Generated Generated S → NP VP NP@VP - - VP → Vtransitive NP Vtransitive@NP - - VP → Vintransitive Vintransitive - - NP → Nproper Nproper - - NP → D N D@N - - NP → D N C (D@N)@C - - NP → D AP D@AP - - AP → A N λp.(A@p ∧ N@p) - - C → RP Vintransitive Vintransitive - -Terminal rules: Grammar Rule λ-Expression Predicate Argument Generated Generated Vtransitive → love(s) λpλq.(p@λr.LOVE(q, r)) LOVE - Vintransitive → boxes λp.BOXER(p) BOXER - Nproper → John λp.p@JOHN - JOHN A → lady λp.WOMAN(p) WOMAN - N → woman λp.WOMAN(p) WOMAN - A → boxing λp.BOXER(p) BOXER - N → boxer λp.BOXER(p) BOXER - RP → who − - - D→a λpλq.∃x (p@r ∧ q@r) - - D → some λpλq.∃x (p@r ∧ q@r) - -Consider the sentences below, all of which can be derived from the rules above: (13) John loves a lady boxer (14) ?John loves a boxing woman (15) John loves some woman who boxes
  6. 6. Natural Language Generation from First-Order Expressions 6All the sentences can be associated with a first-order representation of the form: (16) ∃x (LOVE(JOHN, x) ∧ BOXER(x) ∧ WOMAN(x))Our goal is to determine a set of rules which would allow a direct derivation of (13), (14) and (15) from (16) usingthe annotated grammar as a reference. It is straightforward to parse the expression and determine all predicates,quantifiers, constant arguments and operators. Each such component must be directly derived from a terminal rule(note that a single terminal rule can produce more than one first-order component). A set of candidate terminal rulescan be determined by looking up rules whose lambda expressions contain components found in the first orderexpression. Starting from such a collection of candidate rules, all possible paths using the set of available non-terminal grammar rules and terminal rules without an associated lambda expression can be navigated bottom-up tilla rule is reached whose LHS cannot be produced by any other rule. If any of the selected candidate rules have notbeen used, the specific navigation path shall be marked as a failure. The mapping process between the first-ordercomponents and the terminal rules is what determines the domain of sentences that get generated. This mapping isnot necessarily one-to-one – many first-order components may be associated with a single terminal rule - considerexample (5) above. This approach overgenerates since there is no selectional restriction enforced between realizationof predicates and their arguments.The approach is explained in more detail below. A bold symbol X indicates a set of elements. #X indicates thenumber of elements within the set X. R is the set of rules in the grammar. It is assumed that a lambda-expression fora rule can contain at most one predicate or one argument. Extract the set of all predicate instances P and distinct predicate functions P from E. E.g. for the expression E = Z(x)∧Z(y), P = {Z, Z} and P = {Z} For every p∈P, if there are n instances of p in P, determine the set of all possible rulesets RSp where for each possible ruleset Rp∈RSp if r∈Rp then: - r ∈ {x∈R | λ-expression(x) contains p} - #Rp ≤ n E.g. if P = {Z, Z} and P = {Z} and if there are 2 rules r1 and r2 whose lambda-expression contains Z then RSp = {{r1}, {r2}, {r1, r1}, {r1, r2}, {r2, r2}} Determine the set of all possible rulesets RSpred where for each possible ruleset Rpred∈RSpred, Rpred = ∪p∈P Rp where Rp∈RSp. E.g. if P = {Y, Z} and RSp=Y = {{r1}, {r1, r1}} and RSp=Z = {{r2}, {r3}} then RSpred = {{r1, r2}, {r1, r3}, {r1, r1, r2}, {r1, r1, r3}} Extract the set of all constant argument instances A and distinct predicate functions A from E. E.g. for the expression Z(x)∧Y(x), A = {x, x} and A = {x} For every a∈A, if there are n instances of a in A, determine the set of all possible rulesets RSa where for each possible ruleset Ra∈RSa if r∈Ra then: - r ∈ {x∈R | λ-expression(x) contains a} - #Ra ≤ n Determine the set of all possible rulesets RSarg where for each possible ruleset Rarg∈RSarg, Rarg =
  7. 7. Natural Language Generation from First-Order Expressions 7∪a∈A Ra where Ra∈RSaExtract the set of all quantifier instances Q from E E.g. for the expression ∃x (Z(x)∧Y(x)), Q = {∃}For every q∈Q, determine the set of all possible rulesets RSquant where for each possible rulesetRquant∈RSquant if r∈Rquant then: - r ∈ {x∈R | λ-expression(x) contains q ∧ ¬λ-expression(x) contains a predicate} - #Rquant ≤ #Q E.g. if Q = {∃, ∀, ∀} and if lambda-expression of r1 contains ∃ and that of r2 and r3 contains ∀ then RSquant = {{r1, r2, r2}, {r1, r2, r3}, {r1, r3, r3}}Extract the set of all quantifier instances O from E E.g. for the expression ∃x (Z(x)∧Y(x)), O = {∧}For every o∈O, determine the set of all possible rulesets RSoper where for each possible rulesetRoper∈RSoper if r∈Roper then: - r ∈ {x∈R | λ-expression(x) contains o ∧ ¬λ-expression(x) contains a predicate/ quantifier} - #Roper ≤ #O E.g. if O = {∧, ∧} and if lambda-expression of r1 and r2 contain ∧ then RSquant = {{r1, r1,}, {r1, r2}, {r2, r2}}Determine the set of all possible rulesets RS where Rcandidate=Rpred∪Rarg∪Rquant∪Roper whereRcandidate∈RS, Rpred∈RSpred, Rarg∈RSarg, Rquant∈RSquant, Roper∈∅∪RSoper E.g. if E = ∃x (LOVE(JOHN, x)∧WOMAN(x)) and r1 = [Vtransitive → love(s)], r2 = [Nproper → John], r3 = [D → a], r4 = [D → some], r5 = [N → woman], r6 = [CONJ → and], then RSpred = {{r1, r5}}, RSarg = {{r2}}, RSquant = {{r3}, {r4}}, RSoper = {{r6}} and RS = {{r1, r5, r2, r3}, {r1, r5, r2, r4}, {r1, r5, r2, r3, r6}, {r1, r5, r2, r4, r6}}For each Rcandidate∈RS, find all paths to a non-terminal rule with an un-referenced LHS (i.e. S)such that - The path has to use every element of Rcandidate one time - The path can use any non-terminal rule any number of times - The path can use any terminal rule without a λ−expression any number of times E.g. using the previous example one possible Rcandidate = {r1, r5, r2, r3} would generate John love(s) a woman and a woman love(s) John
  8. 8. Natural Language Generation from First-Order Expressions 8Figure 3: Overgeneration process
  9. 9. Natural Language Generation from First-Order Expressions 9Each path identified represents a candidate natural language sentence. This approach will produce the sentences(13), (14) and (15), amongst others, when applied to the expression (16). It will satisfy the requirement that if S canbe translated to E using a set of rules, then E can be translated back to S.The results for the first-order expression in (16) are: Input First-order Overgenerated Valid Expression Translations John love(s) some lady boxer • some lady boxer love(s) John John love(s) a lady boxer • a lady boxer love(s) John John love(s) some boxing woman • ∃x (LOVE(JOHN, x) ∧ some boxing woman love(s) John BOXER(x) ∧ WOMAN(x)) John love(s) a boxing woman • a boxing woman love(s) John some woman who boxes love(s) John John love(s) some woman who boxes • a woman who boxes love(s) John John love(s) a woman who boxes •As mentioned earlier there is no context applied while processing the grammar. As a result the output will includeunwanted results such as: Incorrect NP assignment: both John loves a lady boxer and A lady boxer loves John are outputs Incorrect quantifier assignment: The expression (∃x (MAN(x) ∧ ∀y (WOMAN(y) → LOVE(x, y))) will generate both a man loves every woman and every man loves a woman).The annotated grammar rules used shall need to include all orphan tokens. An orphan token is one that has noassociation with an element in the first-order representation. This might require some duplication of terminal rules.An example is a copular construction. Consider the example below: (17) BOXER(JOHN) ≡ John is a boxerIn order for this approach to handle bi-directional translations that can handle such a case, there shall need to be 2logical representations for the rule D → a. T his may result in ambiguity in generating first-order representations.5. Pruning processThe overgeneration step will create well-formed language sentences – however some of the output will not be agood translation for the first-order expression. The pruning process applies a filter to drop out such bad translations.Option 1One solution to weed out bad results is by regenerating the first-order representation back and comparing with theoriginal expression E to verify if the translation was valid. This does not require development of any new translationrule as the methodology described in section 2 using lambda calculus would suffice.The complexity in this approach arises from determining if two first-order representations are equivalent. This is dueto the nature of mathematical representation as shown below. (18) ∃x (BOXER(x) ∧ WOMAN(x)) ≡ ∃x (WOMAN(x) ∧ BOXER(x))This means that a literal equivalency test cannot be applied – instead mathematical equivalence would need to beestablished. One method of doing this is using the system Otter.
  10. 10. Natural Language Generation from First-Order Expressions 10This method shall become computationally intensive if there are a lot of candidate sentences to evaluate. On the flip-side it is guaranteed to produce perfect pruning results provided a fool-proof system for equating logical expressionscan be developed.Option 2So far the discussions have stayed away from unraveling a first-order representation. A solution that does this isdescribed below. This approach focuses on multi-argument predicates to validate the generated language. If asentence does not have at least one multi-argument predicates (such as copular constructions), the algorithm doesnot prune.In English, a multi-argument predicate can generally be mapped to a verb. The first argument of the predicate isgenerally realized to the left of the element that realizes the predicate. The exception to this is a passive construction.The ordering of the tokens that realize the arguments and predicate can be determined by the order of the lambdavariables with respect to the predicate arguments. This is shown in the expressions below: Active form ⇒ λpλq.(p@λr.LOVE(q, r)) Passive form ⇒ λpλq.(p@λr.LOVE(r, q))In a di-transitive case, the active construction check is the same as that for the transitive case. In the passive form, itneeds to be verified that the first argument to the predicate is realized to the right of the element that realizes thepredicate. This is shown in the expressions below: Active form ⇒ λpλqλr.(p@q@λsλt.GIVE(r, s, t)) Passive form ⇒ λpλqλr.(p@q@λsλt.GIVE(s, r, t))A simple rule can be based on tracking the orientation of all argument realizations for a predicate with respect to therealization of the predicate. Note that this rule is very specific to a SVO language – but can conceivable be adaptedto other structures. It is to be determined if the lambda expression can be processed so as to have a more genericrule.To enforce such an ordering rule, there has to be traceability between (a) the arguments and their realization (b) thepredicates and their realization. This is generally straightforward to do (the sentence generation rules can bemodified to keep a trace) for sentences where there is a 1:1 association between predicate+argument componentsand rules. If there is no direct alignment, one possible scenario is multiple predicate instances all mapped to one rule(see (5)) – in which case a trace can also be maintained with effort. For any greater combination, such as threepredicate instances mapped to two rule instances, creating a trace is more difficult as multiple combinations mayneed to be evaluated – such combinations may be unlikely.Two special cases for argument realization are: (a) Variable Arguments: The sentence generation rules must besubstantially be modified to allow for tracing of variables to its realization. If the variable is quantified, therealization of the quantifier can be used to assess the order. The case of unquantified variables has not beenevaluated. (b) Functional Predicates: In this case, the argument realization can be mapped to the realization of thefunctional predicate.The generation rules will now include the following modified and new steps in order to maintain a trace:
  11. 11. Natural Language Generation from First-Order Expressions 11Determine a map between each instance of p and an element r∈Rpred E.g. if P = {Z, Z, Y} and P = {Z, Y} and if there are 2 rules r1, r2 whose λ-expression contains Z and 1 rule r3 whose λ-expression contains Z then RSpred = {{r1, r3}, {r2, r3}, {r1, r1, r3}, {r1, r2, r3}, {r2, r2, r3}} map = {{{1,2}, {3}}, {{1,2}, {3}}, {{1}, {2}, {3}}, {{1}, {2}, {3}}, {{1}, {2}, {3}}} P = {Z, Z, Y} 1, 2, 3 refer to the index of the elements in PFor each predicate rule p∈P, identify the arguments to the predicate: - if argument is a constant argument, link p to a rule rarg∈Rarg∈RSarg where rarg contains an argument for p - if argument is a variable argument, link p to a rule rquant∈Rquant∈RSquant where rquant contains the quantifier scope for the argument for r E.g. if E = ∃x (LOVE(JOHN, x)∧WOMAN(x)) r1 = [Vtransitive → love(s)], r2 = [Nproper → John], r3 = [D → a], r4 = [D → some], r5 = [N → woman], then RSpred = {{r1, r5}}, RSarg = {{r2}}, RSquant = {{r3}, {r4}} and RS = {{r1, r5, r2, r3}, {r1, r5, r2, r4}}For each Rcandidate∈RS, find all paths to a non-terminal rule with an un-referenced LHS (i.e. S) suchthat - The path has to use every element of Rcandidate one time - The path can use any non-terminal rule any number of times - The path can use any terminal rule without a λ−expression any number of times - For every p∈P determine the ordering of arguments with respect to the predicate in the path using the map to resolve how the predicate was realized in the generated sentence and using the links to determine how the arguments were realized E.g. using the previous example for one possible Rcandidate = {r1, r5, r2, r3} then P = {LOVE, WOMAN}, RSpred = {{r1, r5}}, map = {{{1}, {2}}} The trace for generated sentences is shown below: John love(s) a woman r2 r1 r3 r5
  12. 12. Natural Language Generation from First-Order Expressions 12 a woman love(s) John r3 r5 r1 r2The rules to handle pruning are as follows: Extract the set of all predicate instances with more than one argument Pmultiargs from E E.g. for the expression E = ∃x (LOVE(JOHN, x)∧WOMAN(x)), Pmultiargs = {LOVE} For each p∈Pmultiargs determine whether the first argument should be realized to the left or right of the token realizing the predicate based on the order of the variables that describe the order of the arguments in the lambda expression for the rule associated with p E.g. for the example above, if r1 = [Vtransitive → love(s)] : λpλq.(p@λr.LOVE(q, r)) then q which is the first argument is defined by λq which precedes λr which is the second argument. For the predicate LOVE, the first argument is expected to be to the left of the token that realizes the predicate. The sentence order is expected to be rarg1 rpred rarg2. For each path discard the path if the argument ordering of any predicate is incorrect E.g. for the example E = ∃x (LOVE(JOHN, x)∧WOMAN(x)), for the specific predicate LOVE, for the candidate rule {r1, r5, r2, r3}, the sentence order should be r2 r1 r3. This marks the sentence a woman love(s) John as invalidA second issue is that the rule does not cover restrictions based on agreement. The output will include John love alady boxer. This limitation cannot be addressed with the current state of the rules. To solve this issue the rules shallneed to be annotated with agreement restrictions which can be evaluated as the paths are navigated.6. ImplementationI implemented this system using the language C# using the .NET framework. The object model used is below:
  13. 13. Natural Language Generation from First-Order Expressions 13The classes used by this model are: Class Description Rule This represents a grammar rule annotated with a lambda expression. The class provides methods for parsing a rule into RHS and LHS components. This class inherits from the generic Expression class. FOExpression This represents a first-order expression. This class inherits from the generic Expression class. The method ProcessExpression()accepts a grammar in the form of RuleCollection and determines candidate terminal rules that can be associated with the expression. For each set of terminal rules, the method calls RuleCollection .GetRules() to determine if the set can generate a language sentence. This method internally uses recursion to determine all possible sets of terminal rules. Expression Provides a representation for a mathematical expression with quantifiers, constant arguments, predicates and operators. RuleCollection This represents a collection of rules i.e. represents the full grammar of the natural language. The method GetRules()provides a mechanism for determining all valid paths through the grammar if a list of terminal rules are provided. This method internally uses recursion to navigate the grammar rules.7. Cost and OptimizationA valid concern against this approach is the question of cost. The cost implications are driven from two angles – oneis the complexity and scale of the grammar and the second is the complexity of the expression being evaluated.The first issue is of lesser concern since it can be assumed that in an almost full-fledged grammar of English, alarger percentage of the rules shall specify terminal rules as compared to non-terminal rules – this is because of thelarge vocabulary. The algorithm discussed selects early on for terminal rules and hence avoids a performance hit ofhaving to use a huge grammar in generating a sentence.The second issue has a larger impact. This limitation is because the algorithm searches for every valid candidate –consider for example that there are n! permutations for the task to fit n noun phrases into a sentence with n noun-phrase place-holders. That said, every application may not require every valid language translation for the processedexpression – in which case it may suffice to do an immediate pruning check for every successful sentence generatedand a successful pass through the pruning rules for one sentence could stop the entire algorithm from processing anyfurther.8. ConclusionI have so far defined an approach in which a grammar and ruleset for the compositional construction of a first-orderrepresentation from English can be utilized successfully for the reverse translation problem as well. The approach isrobust in that there is very limited additional information that is required for the system to function. However thisrobustness comes at a computation cost which is reasonable for simple first-order expressions but which can becomeconsiderable for lengthy discourse. This overhead may not be that detrimental for application in the space of TextSummarization and Text Simplification problems where it can be anticipated that an ideal system would be only
  14. 14. Natural Language Generation from First-Order Expressions 14required to process limited expressions. Possible future enhancements that can be taken into account to make thisapproach more efficient are listed below: Integrate pruning steps into the generator process wherever possible Apply a mix of a Top-down and Bottom-Up evaluation strategy in determining if the selected ruleset is a good candidate for a well-formed sentence. This can avoid the overhead of unnecessary evaluation of a frequently used rule such as S → NP VP for every pass. Another alternative to navigating the hierarchy is to cache navigation paths found using the LHS of all the candidate rules involved as a key to the cache. For every new candidate ruleset, the cache shall be searched first – if no entry is found then the system can navigate the rules. Each time the grammar changes though the cache shall need to be droppedThe current pruning system which translates the ordering in a lambda expression into a natural language ordering formulti-argument predicates does have a dependency on non-terminal grammar rules which has been ignored. A morerobust scenario would be to process the non-terminal rules to infer the ordering. Language generation issues relatingto adverbs, adjuncts and tense has not been covered by this paper and is another area for future review.As for future steps, the author would like to develop a text simplification system which performs the following: 1. Translate a complex sentence into first-order logic 2. Develop a processing layer that can mathematically simplify a first-order expression 3. Develop a processing layer that can break a first-order expression into independent expressions 4. Apply the approach described above to generate a natural language translation from the simplified expressionI would like to thank Prof. Graham Katz for providing the opportunity to write this paper and for his feedback.9. ReferencesBlackburn, P & Bos, J. (2003). Computational Semantics for Natural Language. NASSLLI.Strzalkowski, T. et al. (1994). Reversible Grammar in Natural Language Processing. Kluwer Academic Publishers.Whitney, R. (1988). Semantic Transformations for Natural Language Production. ISI Research Report.Dymetman, M. & Isabelle, P. (1988). Reversible Logic Grammars for Machine Translation. Second International Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages.Smullyan, R. (1994). First-Order Logic. Dover Publications.

×