Upcoming SlideShare
×

# Large sample property of the bayes factor in a spline semiparametric regression model

155 views

Published on

International peer-reviewed academic journals call for papers, http://www.iiste.org

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
155
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
2
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Large sample property of the bayes factor in a spline semiparametric regression model

1. 1. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 www.iiste.org Large Sample Property of The Bayes Factor in a Spline Semiparametric Regression Model Ameera Jaber Mohaisen and Ammar Muslim Abdulhussain Mathematics Department College of Education for Pure Science AL-Basrah University-Iraq E-mail: ammar.muslim1@yahoo.com Abstract In this paper, we consider semiparametric regression model where the mean function of this model has two part, the first is the parametric part is assumed to be linear function of p-dimensional covariates and nonparametric ( second part ) is assumed to be a smooth penalized spline. By using a convenient connection between penalized splines and mixed models, we can representation semiparametric regression model as mixed model. In this model, we investigate the large sample property of the Bayes factor for testing the polynomial component of spline model against the fully spline semiparametric alternative model. Under some conditions on the prior and design matrix, we identify the analytic form of the Bayes factor and show that the Bayes factor is consistent. Keywords: Mixed Models, Semiparametric Regression Model, Penalized Spline, Bayesian Model, , Marginal Distribution, Prior Distribution, Posterior Distribution, Bayes Factor, Consistent. 1. Introduction In many applications in different fields, we need to use one of a collection of models for correlated data structures, for example, multivariate observations, clustered data, repeated measurements, longitudinal data and spatially correlated data. Often random effects are used to describe the correlation structure in clustered data, repeated measurements and longitudinal data. Models with both fixed and random effects are called mixed models. The general form of a linear mixed model for the ith subject (i = 1,…, n) is given as follows (see [10,13,15]), = +∑ + , ~ 0, , ~ (0, ) (1) where the vector has length , and are, respectively, a × design matrix and a × design matrix of fixed and random effects. β is a p-vector of fixed effects and u ! are the q -vectors of random effects. The variance matrix is a × matrix and is a × matrix. We assume that the random effects { ; % = 1, … , ( ; ) = 1, … , *} and the set of error terms { , … , , } are independent. In matrix notation (see [13,15]), = + + (2) . Here = ( , … . , , ). has length = ∑, , = ( . , … , , ). is a × design matrix of fixed effects, Z is a × block diagonal design matrix of random effects, q = ∑/ q! , u = (u0 , … , u0 )0 is a q-vector of / ! random effects, = 1%23( , … , , ) is a × matrix and = 1%23( , … , ) is a × block diagonal matrix. In this paper, we consider semiparametric regression model (see [1,6,8,9,10,13,15]), for which the mean function has two part, the parametric ( first part ) is assumed to be linear function of p-dimensional covariates and nonparametric ( second part) is assumed to be a smooth penalized spline. By using a convenient connection between penalized splines and mixed models, we can representation semiparametric regression model as mixed model. In this model we investigate large sample properties of the Bayes factor for testing the pure polynomial component of spline null model whose mean function consists of only the polynomial component against the fully spline semiparametric alternative model whose mean function comprises both the pure polynomial and the component spline basis functions. The asymptotic properties of the Bayes factor in nonparametric or semiparametric models have been studied mainly in nonparametric density estimation problems related to goodness of fit testing. These theoretical results include (see [4,7,12,14]). Compared to the previous approaches to density estimations problems for goodness of fit testing and model selection, little work has been done on nonparametric regression problems. Choi and et al 2009 studied the semiparametric additive regression models as the encompassing model with algebraic smoothing and obtained the closed form of the Bayes factor and studied the asymptotic behavior of the Bayes factor based on the closed form ( see [3] ). In this paper, we obtain the closed form and studied the asymptotic behavior of the Bayes factor in spline semiparametric regression model and we proved that the Bayes factor converges to infinity under the pure polynomial model and the Bayes factor converges to zero almost surely under the spline semiparametric regression alternative and we show that the Bayes factor is consistent. 91
2. 2. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 www.iiste.org 2. Semiparametric and Penalized Spline Consider the model: 6 4 = ∑ 8 5 + (567 , ) + , % = 1,2, … , ( (3) Where 4 , … , 4, response variables and the unobserved errors are , … , , are known to be i.i.d. normal with < < mean 0 and covariance :; = with :; known. The mean function of the regression model in (3) has two parts. The parametric ( first part ) is assumed to be linear function of p-dimensional covariates 5 and nonparametric (second part) (567 , ) is function defined on some index set > ⊂ . The model (3) can be expressed as a smooth penalized spline with q degree, then it's become as: 6 @ @ D (4) 4 = ∑ 8 5 +∑ 67 567 , + ∑A { A (567 , − CA )7 + Where C , … , CD are inner knots 2 < C < , , , < CD < F. By using a convenient connection between penalized splines and mixed models. Model (4) is rewritten as follows (see [13,15]) = + + (5) where 4 = G⋮I , 4, L ⋮ O K N 6 N , = K K 67 N K ⋮ N J 67@ M 8 = G ⋮ I, = P D (567 (567 − C )7 ⋮ @ ,, − C )7 , @ ⋯ ⋱ ⋯ (567 (567 − CD )7 S ⋮ @ ,, − CD )7 @ , 567 , … 567 , 1 5 … 56 L O @ K 1 5 < … 56< 567 ,< … 567 ,< N = K N ⋮ ⋱ ⋮ ⋮ K ⋮ ⋮ ⋱ N @ J1 5 , … 56, 567 ,, … 567 ,, M We assume ( < C + + + 1, ( + T = C + + + 1, (T is integer number and greater than 1 ), and we assume that the design matrix is orthogonal in the following way: . U, U, = (=, ∀( ≥ 1 (6) where: be the ( − ( + + 1) × ( − ( + + 1) matrix, and let = + X U, = ( , ) and Based on the above setup for regression model (5) and the assumption of the orthogonal design matrix (6), we consider a Bayesian model selection problem in a spline semiparametric regression problem. Specifically, we would like to choose between a Bayesian spline semiparametric model and its pure polynomial counterpart by the criterion of the Bayes factor for two hypotheses, Y8 : = + versus Y: = + + (7) As for the prior of , 2(1 under Y , we assume and u are independent and < ~ (0, ∑8 ) , ∑8 = :[ =67@7 (8) < ~ (0, ∑ ) , ∑ = : =D (9) @ 3. Bayes factor and marginal distribution 6 < The response 4 in (4) follows the normal distribution with mean ] and variance :; , where ] = ∑ 8 5 + )=1 )5 +1,%)+C=1^{ C(5 +1,%−^C)+ , for, %=1,2,…,(. Thus, given covariates and ](=(]1, …, ]()>, the n-dimensional response vector , = (4 , … , 4, ). follows the n-dimensional normal distribution with mean ], < and covariance matrix :; =, where =, is the ( × ( identity matrix. Also from the prior distributions specified in (8) and (9), we can deduce the joint distribution of ], is the multivariate normal distribution (U_ ) with mean zero and ( × ( covariance matrix ∑8 . + ∑ . . In summary, we have the following:, |], ~ U_ < , (], , :; =, ) , ], ~ U_ , (0, , ∑8 . + ∑ . ) (10) Result 1: Suppose the distribution of ], and , are given by (10). Then the posterior distribution of P(], | , )is the multivariate normal with the following mean and variance. < < a(], | , , :; ) = ∑< (∑< + :; =, )b , 92
3. 3. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 www.iiste.org < < < < c2*(], | , , :; ) = (∑b + (:; =, )b )b = ∑< (∑< + :; =, )b :; =, < . . Where ∑< = ∑8 + ∑ . Furthermore, marginally , follows < (11) , ~ U_ (0, :; =, + ∑< ) Note the (%, ))th element of ∑< , 67@7 < < ∑< (%, )) = ∑A :[ 5 A 5A + ∑D : d A dA , %, ) = 1, … , ( (12) A @ where d A = (567 , − CA )7 Applications of the previous result yield that, the marginal distribution of , under Y8 and Y are respectively , (0, _8 ) and , (0, _ ), where < _8 = :; =, + ∑8 . < _ = :; =, + ∑8 . + ∑ . Hence, the Bayes factor for testing problem (7) is given by: 1 1 . b exp { − , _8 , } 2 ( |Y8 ) f2gdet (_8 ) e8 = = 1 1 . ( |Y ) exp { − , _ b , } 2 f2gdet (_ ) = fmno (pq ) fmno (pr ) exp{− . ( < , b _8 −_ b ) , } (13) 4. Asymptotic behavior of Bayes factor The Bayes factor is said to be consistent if ( see[3,14]) u s% ,→u e8 = ∞ , 2. T. w8 under the Y8 , (14) s% ,→u e8 = 0 , 2. T. wu under the Y , (15) We investigate the consistency of the Bayes factor, by establishing (14) and (15), under the spline semiparametric regression model described in the section 2. By truncating the regression up to the first ( terms. ,b(67@7 ) 6 @ @ We get 4 = ∑ 8 5 + ∑ { A (567 , − CA )7 } + 67 567 , + ∑A Now define _ ,, as the covariance matrix of the marginal distribution of , under Y as:< _ ,, = :; =, + ∑8 . + ∑ , . < . = :; =, + U, x, U, (16) < Where ∑ , is diagonal matrix with diagonal elements : from size ( − ( + + 1) × ( − ( + + 1) and ∑ 0 z x, = y 8 0 ∑, where 0 , 0 are the matrices for zeros element with size ( + + 1) × ( − ( + + 1) and ( − ( + + 1) × ( + + 1), respectively. Then, we can rewrite (16) by using x, as: |~ . _ ,, = U, { } =, + x, • U, (17) , Let _8 be the covariance matrix of the marginal distribution of , under Y8 , then _8 can be represented as: < _8 = :; =, + ∑8 . + 0 . < . = :; =, + U, €, U, where: ∑ 0 €, = y 8 z 0 0 where 0 is the matrix for zeros element with size ( − ( + + 1) × ( − ( + + 1), then we can rewrite _8 as the following: _8 = U, { ~ |} , . =, + €, • U, (18) Then we can use the two matrices above (_8 , _ ,, ) for matrix inversions and calculating determinants as shown in following lemma. Lemma1: Let _ ,, and _8 be defined as in (17) and (18) respectively, and suppose U, satisfied (6). Then, the following hold. . . i) U, U, = U, U, = (=, ii) det(_8 ) = (67@7 ( ,~ U, { , ~ |} iii) det _ ,, = (, ( b iv) _8 = ~ |} , ~ |} , < < + :[ )67@7 :; < + :[ )67@7 ( =, + €, • b ~ |} , . U, (,b(67@7 )) < + : ),b(67@7 93 )
4. 4. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 www.iiste.org b |~ . v) _ b = ~ U, { } =, + x, • U, ,, , , Proof:. i) By multiplying U, and U, to equation (6) from right and left side, we have:. . . U, U, U, U, = U, (=, U, . = U, (U, . = (U, U, . Multiplying (U, U, )b to the above equation from left side, we get:. . . . . U, U, U, U, (U, U, )b = (U, U, (U, U, )b . U, U, = (=, ~ |} ii) _8 = U, { = • , ~ |} Then L(( , K K = K K K J , . =, + €, • U, , and by using (6), we get |~ } L( , K K ‚K K K K J < + :[ ) 0 ⋮ 0 0 < + :[ ) 0 ⋮ 0 0 det(_8 ) = (67@7 ( ~ |} , L(( , K K = K K K K J |~ < + :[ ) 0 , ⋮ 0 0 0 < + :[ ) ⋮ 0 0 ⋮ 0 0 < + :[ ) ⋮ 0 0 (( ~ |} , 0 + ⋮ 0 0 ~ |} ⋯ , ⋯ ⋯ 0 0 0O N 0N ⋮N N 0N < :; M (,b(67@7 )) ( , 0 < + :[ ) ⋮ 0 0 < :[ ) ⋱ ⋯ ⋯ ⋯ = = = = .b U, …q ƒ„ , { , ~ |} ⋱ ⋯ ⋯ , . (=, U, U U. ,~ , , ,~ U, { b =, + €, • b { .b U, ~ |} , ~ |} , { U, b =, + €, • ~ |} , b =, + €, • b =, + €, • . U, ⋯ 0 ⋯ ~ |} , 0 0 ⋮ + 0 ‚. 0 < : O N 0 N ⋮ N• , N 0 N ~ N |} < + : M , 0 O N ⋯ 0 0 N N ⋮ ⋮ ~ N |} < (( + : ) 0 N , ~ N |} < 0 (( + : )M , < < :; :; < < det _ ,, = (, ( + :[ )67@7 ( + : ),b(67@7 ( ( |~ b . iv) _8 = {U, { } =, + €, • U, }b Then 0O N 0N ⋮N • , N 0N ~ |} N ,M ⋯ 0 ⋱ ⋮ ⋯ 0 ⋱ ⋮ < ⋯ :; ⋯ 0 ~ |} 0 ⋯ 0 < :[ ) . =, + x, • U, } L( , K K ‚K K K K J = • , ~ |} (( + , < < + :[ )67@7 :; , ~ |} iii) _ ,, = U, { ~ |} ( 0 ~ |} ) U, b (=, b U, b …q ƒ„ , . U, U, by lemma 1.i 94 ‚.
5. 5. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 v) _ b = {U, { . = U, = = = …q ƒ„ b { ~ |} = , , ~ |} = , , , . (=, U, ,~ U, { U U. ,~ , , b . + x, • U, }b + x, • { .b U, ~ |} , www.iiste.org ~ |} , { b U, b =, + x, • ~ |} , =, + x, • b =, + x, • Now let b . U, † e8 = U, b (=, b U, b , . U, U, fmno (pq,„) fmno (pr ) …q ƒ„ exp{− . ( < , b _8 −_ b ) , } ,, Which an approximation of e8 with _ replaced by _ ,, in (13) Thus, 1 det (_ ,, ) 1 . b † s‡3 e8 = s‡3 − ( _ −_ b ) , ,, det (_8 ) 2 , 8 2 det (_ ,, ) . b † − , ( _8 −_ b ) , 2 log e8 = log ,, det (_8 ) By results of lemma1 we have::< :< < < (, ( ; + :[ )67@7 ( ; + : ),b(67@7 ) ( ( † 2 log e8 = log (,b(67@7 )) :< < < (67@7 ( ; + :[ )67@7 :; ( b b < < 1 . :; :; . − < ‹ , U, ‹Œ =, + €, • − Œ =, + x, • Ž U, , Ž ( ( ( = log where ”, = = log( ~ ’ ~ ,„…(••‘•q) ( } 7|“ )„…(••‘•q) ~ ~ |} 7,|“ ~ |} „ ~ („…(••‘•q)) |} ),b(67@7 ) − 1 0(67@7 )×(67@7 U y < , 0 ( and ” = x%23 ( ~ ,~ |“ ~ 7,| ~ )| ~ (|} “ } ) ) − ,~ . , ”, , } { . , U, 0(67@7 y )×(67@7 ) 0 0 . z U, ” (19) ,} 0 . z U, ” † To establish the consistency of the Bayes factor we focus on e8 , first and the remaining terms will be < considered later. Without loss of generality we assume :; = 1 in the remainder of this paper. Now let •,, = ∑, •,,< = ∑ , ~ ( 7,|“ ) {∑ ~ ,~ |“ ~ 7,|“ ) = ( ( ~ 7,|“ ) + (20) < (: ) (21) (22) = (4 , … , 4, ). where 4 ′T are independent normal random variable with mean ∑ +)5 +1,%) and variance : 2=1, then there exist a positive constant €1 such that Lemma2: let ˜„,q = ~ ( 7,|“ ) < log(1 + (: ) = ( log(1 ~ ~ ,|“ < |“ – < •,,– = ∑, ( )=1 ~ ,|“ , ,b(67@7 ) < log(1 + (: ) − . , ”, , • This implies that, under Y8 . ,→u † ›œ log e8 š › ∞, in probability. > € , with probability tending to 1. Proof Let , = ( , < , … , , ) are independent standard normal random variables, Note that . ). ”, ( , + ) , ”, , = ( , + . . = , ”, , + ( ). ”, ( ) + , ”, + ( ). ”, , . By the orthogonality of the design matrix 95 , • , + 6 8 and 5 +
6. 6. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 ( ). ”, = ( ). www.iiste.org 0(67@7 )×(67@7 ) U y ,~ , 0 . , ”, , . 0 . z U, = 0. ” . Then , ”, , = Using the expectation formula of the quadratic form given in (12), we obtain , < (: . 0 < a( , ”, , ) = ž*(”, ) = Ÿ = •,, < (1 + (: ) Note that 1 0(67@7 )×(67@7 ) 0 0(67@7 )×(67@7 < . ”, = U, y z U, U, y 0 ” 0 ( 0(67@7 )×(67@7 ) 0 . zU = ¡ U, y , 0 ”< , and < ž*(”, ) = ∑, ( ,~ ~ ,~ |“ ~ 7,|“ ) < = ∑, ( ~ ,|“ ~ 7,|“ ) 0 . z U, ” )< = •,,– . Similarly, using the variance formula of the quadratic form of the multivariate normal variables, we have . , ”, , ) c2*( Let ¢, = w* { ˜„,q { < = 2ž*(”, ) = 2 ∑, ( ˜„,~ b˜„,q {∑ <˜„,q ,b(67@7 ) . Then, . , ”, , } ~ ,|“ ~ 7,|“ < log(1 + (: )< − )< = 2•,,– . . , ”, , • ≤ ¢, } •,,< ≥ − ¢, ‚ •,, •,, . . . { , ”, , } a( , ”, , ) •,,< a( , ”, , ) = w* • − ≥ − ¢, − ‚ •,, •,, •,, •,, . 1 •,,< − a( , ”, , ) . . = w* • {( , ”, , ) − a( , ”, , )} ≥ − ¢, ‚ •,, •,, = w* • ≤ ¤¥ § ¦„ ¨„ ¦„ ~ ˜„,q (©„,~ …©„,q )~ ~©„,q = ª˜„,¡ (˜„,~ b˜„,q )~ ,→u š › 0, ›œ where the last statement holds from the Chebyshev inequality. Then •,,< − •,, ≥ ¢ •,, for some ¢ > 0 and let ¢, ≥ ¢ /2. Hence, we conclude that there exists a positive constant € = ¢ /2 such that ,b(67@7 ) < . {∑ log(1 + (: )< − , ”, , • > € , with probability tending to 1. ˜„,q Consequently, it follows that ,b(67@7 ) < † 2 log e8 = ∑ log(1 + (: )< − ,→u † ›œ log e8 š › ∞ . , ”, , ,→u š › ∞ in probability under Y8 ›œ Lemma3 Suppose we have the penalized spline smoothing, then ,→u 1 ž* _ b − _ b š› 0 ›œ ,, •,, Proof: Since _ − _ ,, is the covariance matrix of ] ∗ (5) = ∑,7X A ,b(67@)7 dA A = X it is a positive definite matrix. Thus _ b − _ b is also a positive definite matrix. Thus, ž* _ b ≥ ž*( _ b ), also from the matrix analogue ,, ,, of the Cauchy-Schwarz inequality, we have ž* . X where ¯_ b ¯ ,, _b ,, = { ž* Note that ≤ -ž*( _ b< ,, }< . X X )ž* _ b< = ® X ® ¯_ b ¯ ,, ,, 67@7 < 1 :; < = • < { Ÿ ° + :[ ± ( ( b< + ,b(67@7 ) Ÿ 96 < :; < ° + : ± ( b< }‚ <
7. 7. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 ∑67@7 ² ~ |} , And ,b(67@7 ) Ÿ ° < + :[ ³ b< = < :; < + : ± ( 67@7 ~ ’~ ~ ´ } 7|µ ¶ „ b< = = + 1) < :; < + : ¶ ( < ⁄< ,~ (67@7 ) = 67@7 ~ ’~ •„ ’~ } µ ¸ · „ (−( + ´ www.iiste.org ≤ ´ ≤ ~ ~ (|} 7, |µ )~ ( < :; < + : ¶ ( < = 67@7 ¹ |µ < (:; = º(1) (– (– ≤ = º(() < )< < ((: )< + (: Thus, it follows that ¯_ b ¯ = »º ² ³¼ = º(()b ⁄< ,, , Since sup¿∈•8, ‚ ® X ® < 1 from the distances between truncated power functions, it follows that ® X ®< = < < ∑, XA (5 ) ≤ ( , where XA (5 ) = (567 , − CA )@ is element in X 7 Thus, ˜„,q = ≤ ž* _ b − _ b ,, ˜„,q ~ |“ ž* _ b ( _ − _ ,, ) _ b ,, ˜„,q . < ∑,7X : ( A A ,b(67@7 ) ˜„,q ~ ,|“ ≤ = ¯_ b ¯ ∑,7X ,, A ,b(67@7 , ˜„,q < ∑,7X A ,b(67@7 Thus 1 ž* _ b − _ b ,, •,, ) 1≤ _b ( ,, ) ® ~ D |“ ˜„,q . A A® = _ b ). ) < ~ D|“ „~ ’~ “ (q•„’~ ) “ < ^(1 + (: ) ,→u š› 0 ›œ < ( ≤ ~ D( 7,|“ ) = ,~ Theorem(1) , Consider the testing problem in (7), suppose that the true model is Y8 ( pure polynomial model ) and let w8 . denote the true distribution of the whole data, with p.d.f w8 (4| 8 ) = ∅{( 4 − 8 )/ :; }, where ∅(∙) is the standard normal density, then, the Bayes factor is consistent under the null hypothesis Y8 : , lim,→u e8 = ∞ in w8 probability. 1 det (_ ) 1 . b log − ( _ −_ b ) 2 det (_8 ) 2 , 8 mno (pq,„ ) mno (pq ) = log + log − Proof log e8 = < . b ( _8 < mno (pr ) < , mno (pq ) . log − , (_ b −_ b ,, < mno (pq,„) < mno (pq,„ ) † = loge8 + † = loge8 + Å + Å< where Å = log < mno (pq ) mno (pq,„ ) , Å< = − , . (_ b −_ b ,, < , ) −_ b ) ,, ) , , − . (_ b −_ b ,, < , ) , , Since _ − _ ,, is a positive definite matrix, 1Æž (_ ) ≥ 1Æž (_ ,, ), thus Å ≥ 0 . < a( , _ b −_ b ) , = :; ž* _ b − _ b + ( ). _ b − _ b , ,, ,, ,, b b b since _ ,, − _ , _ ,, and _ b are all nonnegative definite matrices, it follows that 0 ≤ ( ). (_ b − _ b ) ≤ ( ). _ b . ,, ,, . By orthogonality of the design matrix (6), we have . U, = U, = ( =67@7 , 0,b(67@7 Thus similarly to the proof of lemma (3), we get: ( ). _ b ,, = = = ,~ ,~ ( . . ). U, { ~ |} , b =, + x, • ( =67@7 , 0,b(67@7 =67@7 , 0,b(67@7 ) b ) { ~ |} , . U, { ~ |} , =, + x, • =, + x, • b b ( =67@7 , 0,b(67@7 =67@7 , 0,b(67@7 67@7 < < = ∑ { + :[ • = º(1). , Let Ç > 0, by the Markov inequality when r = 1 and lemma (3), we have: . 1 a( , _ b −_ b ) , 1 . b ,, b { } *{ , _ ,, −_ ) , > Ç È ≤ •,, Ç •,, ~ |} 97 ) ) ) .
8. 8. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 _b ,, ≤ ˜„,q www.iiste.org …q …q pq,„ bpq É + Ê ˜„,q …q (Ë[)§ pq,„ Ë[ ,→u Ê . (_ b −_ b , ,, š › 0. ›œ Note that − _ is nonnegative definite and thus, ) Therefore, Å< = ‡ •,, . The above results with lemma (2) imply there exists a constant € such that 1 1 † log e8 ≥ log e8 + ‡(1) •,, •,, > € + ‡(1). That b ,→u , is nonnegative random variable. , w8 e8 ≥ exp( •,, { € + ‡(1)}) š › ∞ in ›œ probability. This completes the proof. Now we consider asymptotic behavior of the Bayes factor under Y and assuming that the true model for in Y . Lemma(4): 6 Let , = (4 , … , 4, ) where 4 ′T are independent normal random variables with mean ∑ )=1 +)5 +1,%)+C=1(−( + +1)d%C C and variance : 2=1 and let Ì(= a ( > ”( a ( . Then, lim,→u . , ”, , Í„ = 1, in wu probability. 8 is, , is 5 + Proof: From the moment formula of the quadratic form of normal random variables, the expectation and variance of . quadratic form , ”, , are given by: . a( , ”, , ) = ž* ( ”, ) + a( , ). ”, a( , ) c2*( Note Ì, = a( =∑ = = . = = . since T , ). ”, a( , . ,b(67@7 ) , , ~ ,|“ + a( ~ ( 7,|“ ) = •,, + Ì, ) = 2ž* ”, < + 4a( . , ”, , . ,b(67@7 ) , . ,b(67@7 ) . Ab(67@7 ) ” ) ,~ Ì, ≤ T ( ∑, ”, a( < ). , , ) ~ ,|“ ~ 7,|“ ,b(67@7 = ∑ = º ( •,, ~ ~ ) ,~ |“ Ï ~ 7,|“ ~ 7,|“ Ì, = ≥ . For such ( and < ,b(67@7 ) Ÿ . ≥ 1 such that . < < (< : < (< : ≥ < < 1 + (: 1 + (: =( ~ ,|“ ~ 7,|“ < Ð < Ð ≥( ~ Ñ < , . , ) ”, a( , ) . ,b(67@7 ) . . < Ð > 0 and a sufficiently large ( with . Further, similarly to the previous calculation, we have 0(67@7 )×(67@7 1 . < . a( , ). ”, a( , ) = . , ,b(67@7 ) U, – U, y ( 0 1 . = ”< ) ( ,b(67@7 ) ,,b(67@7 ) ,b(67@7 ¹ ~ ,¹ |“ Ï ,b(67@7 ) = ∑ ~ ~ . As in case of a( ) and , On the other hand, consider a fixed positive integer ~ ,|“ , U, y ,b(67@7 ) < ∞ , we have < ). ”, a( 0(67@7 )×(67@7 ) 0 . z U, U, . , 0 ” 0(67@7 )×(67@7 ) 0 . . (. ~ y z( , ,b(67@7 , 0 ” 0(67@7 )×(67@7 ) 0 . . . y z , ,b(67@7 ) 0 ” . U, . ,b(67@7 ) , ). , ( 7,|“ ) 98 ) 0 . z U, U, ”< . , . . ,b(67@7 )
9. 9. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 a( , < ). ”, a( ) ≤ ∑, , ¹ ~ ,¡ |“ Ï ~ ~ 7,|“ Consider a fixed positive integer such n and N, a( , < ). ”, a( , ) = ∑,b(67@7 www.iiste.org ≤( T < ≥ 1 such that ¹ ~ ) ,¡ |“ Ï ~ ~ 7,|“ ≥( ∑, < Ð < Ð ² ¹ ,~ |“ ~ 7,|“ > 0 and a sufficiently large n with ~ ,|“ ~ 7,|“ < Í„ . , ”, , Since lim,→u lim,→u Í„ − § Ó ¦„ ¨„ ¦„ § Ó ¦„ ¨„ ¦„ Í„ . , ”, , Í„ Ò> Ç¼≤ = lim,→u ² ¤¥ ˜„,q Í„ = 1 , in probability + § ¦„ ¨„ ¦„ ~ Í„ Ê ~ Í„ Í„ ~ Ñ ³ ≥ By combining value of •,, and the previous result, we have . c2*( , ”, , ) ≤ º ( •,, . Furthermore, note that a( , ). ”, a( , ) ≥ a( , ). ”, < a( Let Ç > 0, by the Chebshev inequality. *»Ò = º( (•,,– ). ~ , (. ~ ,|“ ~ 7,|“ ≥ < . Again, for ). ,→u = º( (b< ) š › 0. ›œ ³=1 This completes the proof. Lemma(5): Suppose we have the penalized spline smoothing, then 1 1Æž(_ ) lim log =0 ,→u Ì, 1Æž(_ ,, ) Proof: From a property of the determinant of the positive definite matrix, we have 67@7 < < < 1Æž(_ ) ≤ ∏, :; + ∑A :[ 5 < + ∑,7X : d < A A A = (, ∏, ´ ~ |} , 67@7 + ∑A ~ ~ |µ ¿ÏÕ , + ∑,7X A ~ ~ |“ ÖÏÕ , where 5 A , d A is the k th element of 5 , d respectively. From lemma (1), we have: |~ |~ < < det _ ,, = (, ( } + : ),b(67@7 ) ( } + :[ )67@7 . , , Thus 1Æž(_ ) log = log(1Æž(_ )) − log(1Æž _ ,, ) 1Æž(_ ,, ) ≤ ( log ( + ∑, log ´ ,→u ¶, ~ |} , 67@7 + ∑A ~ ~ |µ ¿ÏÕ , + ∑, A ~ ~ |“ ÖÏÕ : 2+(− + +1: 2(+: 2=º1−º1=º(1). , ¶ − { ( log ( + ( + + 1) log ² ~ |} , + Therefore, it follows that Í„ log •×É(pq ) •×É(pq,„ ) š› 0 ›œ Theorem(2): Consider the testing problem in (7), suppose that the true model is in Y ( the penalized spline semiparametric model ) and let w, denote the true distribution of the whole data, with p.d.f. (4| , ) = ∅{(4 − ( . + ))/:; }, where ∅(∙) is the standard normal density. Then the Bayes factor is consistent under the alternative hypothesis Y : lim,→u e8 = 0 in w, probability. Proof: 1 det (_ ) 1 . b log − ( _ −_ b ) 2 det (_8 ) 2 , 8 mno (pq,„ ) mno (pq ) = log + log − log e8 = < mno (pq,„ ) < mno (pr ) b _8 −_ b ) , ,, † = loge8 − † = loge8 + Å + Å< , . ( < , , . b ( _8 −_ b ) , ,, < , . b b − , (_ ,, −_ ) , < 99 − . (_ b −_ b ,, < , ) ,
10. 10. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 where 1 det (_ ,, ) 1 . b log − , ( _8 −_ b ) , ,, det (_8 ) 2 2 mno (pq ) . log , Å< = − , (_ b −_ b ) ,, † loge8 = Å = < www.iiste.org mno (pq,„ ) First, by lemma (1), mno (pq,„ ) mno (pr ) = ~ ~ < ’ ’ ~ ~ ,„ ( } 7|µ )••‘•q ( } 7|“ )„…(••‘•q) „ „ ’~ ~ ~ („…(••‘•q)) ,••‘•q ( } 7|µ )••‘•q |} „ ~ ,b(67@7 ) ,|“ ~ |} =²1 + ³ = , ’~ } ~ ,„…(••‘•q) ´ „ 7|“ ¶ = º(() ⟹ log ²1 + And from lemma (4) Ì, ≤ º((< ). Then ,→u mno (pq,„) log ≤ º((b ) š › 0. ›œ Í„ . „…(••‘•q) ~ („…(••‘•q)) |} ~ ,|“ ~ |} ³ ,b(67@7 ) =² ~ ~ |} 7,|“ = º((). ~ |} ³ ,b(67@7 ) mno (pr ) Therefore, 1 1 1 det _ ,, 1 . b † loge8 = ° log − ( _ −_ b ) , ± ,, det(_8 ) Ì, 2 Ì Ì , 8 b b Since _8 − _ ,, = ”, , then b b . . , ( _8 −_ ,, ) , = , ”, , , from lemma (4), Í„ lim,→u Í„ . , ”, , Í„ = 1. Then ,→u 1 1 det _ ,, 1 1 † ›œ loge8 š › = ° log − Ì, 2 Ì det(_8 ) Ì ,→u b † ∴ loge8 š › › œ , in the w, probability. Í„ < ,→u . ,( ,→u b _8 −_ b ) , ± š › = ›œ ,, 1 −1 (0 − 1) = 2 2 And from lemma (5), Í„ Å š › 0, finally, note that Å< is nonpositive random variable since _ b − _ b is a nonnegative definite ›œ ,, matrix. Therefore, by combining the above results, there exist €< > 0 such that bÚ lim,→u sup log e8 ≤ ~ , with probability tending to 1. Í„ ,→u < Hence, e8 š › 0, in w, probability. ›œ This completes the proof. 5.Conclusion 1- The Bayes factor for testing problem Y8 : 2- = + versus Y : = + + b . e8 = exp{− , ( _8 −_ b ) , }. < fmno (pr ) If , = (4 , … , 4, ). where 4 ′T are independent normal random variable < ∑6 8 5 + ∑@ 67 567 , and variance :; = 1, then there exist a positive constant ,b(67@7 ) < . {∑ log(1 + (: ) − , ”, , • > € , with probability tending to 1. ˜„,q ,→u † ›œ This implies that, under Y8 . log e8 š › ∞, in probability. fmno (pq ) is given by: with mean € such that 3- If the true model is Y8 ( the pure polynomial model ), then, the Bayes factor is consistent under the null , hypothesis Y8 . This implies that, lim,→u e8 = ∞ in w8 probability. 6 4- If , = (4 , … , 4, ) where 4 ′T are independent normal random variables with mean ∑ 8 5 + )=1 +)5 +1,%)+C=1(−( + +1)d%C C and variance : 2=1 and if Ì(=a (>”(a( (). Then, . u lim,→u , ”, , = 1, in w probability. Í„ 5- If the true model is in Y ( the penalized spline semiparametric model ) ,then the Bayes factor is consistent under the alternative hypothesis Y . This implies that, lim,→u e8 = 0 in w, probability. 6.References [1] Angers, J.-F. and Delampady, M. (2001). Bayesian nonparametric regression using wavelets. Sankhy2 Ser. B Û 63, 287–308. [2] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer, New York. 100
11. 11. Mathematical Theory and Modeling ISSN 2224-5804 (Paper) ISSN 2225-0522 (Online) Vol.3, No.11, 2013 www.iiste.org [3] Choi, T., Lee, J. and Roy, A. (2008). A note on the Bayes factor in a semiparametric regression model, Journal of Multivariate Analysis. 1316-1327. [4] G.Gosal, J. Lember, A. Van der Vaart, Nonparametric Bayesian model selection and averaging, Electron.J. Stat. 2 (2008) 63-89. [5] Ghosh, J. K., Delampady, M. and Samanta, T. (2006). Introduction to Bayesian Analysis: Theory and Methods. Springer, New York. [6] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametric. Springer, New York. [7] I. Verdinelli, L. Wasserman, Bayesian goodness –of-fit testing using infinite- dimentional exponential families, Ann. Stat.26 (4) (1998)1215-1241. [8] L. H. Zhao, Bayesian aspects of some nonparametric problems., Ann.Statist. 28 (2000) 532-552. [9] P. J. Lenk, Bayesian inference for semiparametric regression using a Fourier representation, J. R, Stat. Soc. Ser. B Stat. Methodol. 61 (4) 1999) 863-879. [10] Natio, k. (2002) " Semiparametric regression with multiplicative adjustment '' Communications in statistics, Theory and methods 31, 2289-2309. [11] Pelenis, J (2012) Bayesian Semiparametric Regression. Institute for Advanced Studies, Vienna 1-39. [12] R.McVinish, J. Rousseau,K. Mengerson, Bayesian goodness –of-fit testing with mixtures of triangular distributions, Scand. J, statist. (2008), in press (doi:10.1111/j.1467-9469.2008.00620.x). [13] Ruppert, D, Wand, M. P. and Carroll, R. J.(2003) Semiparametric Regression. Cambridge University Press. [14] S.C. Dass, lee, A note on the consistency of Bayes factors for testing point null versus non-parametric alternative, J.Statist. 26 (4) (1998) 1215-1241. [15] Tarmaratram, K. (2011) Robust Estimation and Model Selection in Semiparametric Regression Models, proefschrift voorgedragen tot het behalen van de grad van Doctor. 101