Your SlideShare is downloading. ×
Ethier s.n., kurtz t.g.   markov processes characterization and convergence
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Ethier s.n., kurtz t.g. markov processes characterization and convergence


Published on

Published in: Technology, Education

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Markov Processes
  • 3. Copyright 8 1986,2005by John Wiley ti Sons, Inc. All rights reserved.Publishedby John Wiley & Sons, Inc., Hoboken. New Jersey.Publishedsimultaneouslyin Canada.No part of this publication may be rcproduccd, stored in a retrieval system or transmittcdin any form or by any means, electronic, mechanical, photocopying, recording, scanningor otherwise, except as pcrmittcd under Sections 107 or 108 of the 1976 United StatesCopyright Act, without either thc prior written permission of the Publisher, orauthorization through paymen1of the appropriate per-copy fee to the CopyrightClearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax(978) 750-4470. Requests to the Publisher for permission should he addressed to thePermissions Deparlment, John Wiley B Sons, Inc., 111 River Strcet, Hoboken, NJ 07030,(201) 748-601I , fax (201) 748-6008.Limit of Liability/Disclaimero f Warranty: While the publisherand author have usedtheir bcstefforts in preparingthis book, they make no representationsor warranties with respect to theaccuracy or completenessof the contentsof this book and specifically disclaim any impliedwarranties of merchantability or fitness for a particular purpose. No warranty may be createdorextended by sales representativesor written salesmaterials. The adviceand strategies containedherein may not be suitablc for your situation. You should consult with a professional whereappropriate.Neither the publishernor author shall be liable for any loss of profit or any othercommercial damages, including but not limited to special, incidental, consequential. or otherdamages.For general information on our other products and servicesor for technical support, please contactour Customer Care Department within the US. at (800) 762-2974,outside the U.S. at (317)572-3993 or fax (317)572-4002.Wiley also publishes its books in a variety of electronic formats. Somecontent that appearsin printmay not be availablein electronic format. For informationabout Wiley products, visit our web site of CongressCataloginpin-Publicationis awilable.ISBN- I3 978-0-471-76986-6ISBN-I0 0-471-76986-XPrintedin the United Statesof America1 0 9 8 7 6 5 4 3 2 1
  • 4. The original aim of this book was a discussion of weak approximation resultsfor Markov processes. The scope has widened with the recognition that eachtechnique for verifying weak convergenceis closely tied to a method of charac-terizing the limiting process. The result is a book with perhaps more pagesdevoted to characterization than to convergence.The lntroduction illustrates the three main techniques for proving con-vergence theorems applied to a single problem. The first technique is based onoperator semigroup convergence theorems. Convergence of generators (in anappropriate sense) implies convergence of the corresponding sernigroups,which in turn implies convergence of the Markov processes. Trotter’s originalwork in this area was motivated in part by diffusion approximations. Thesecond technique, which is more probabilistic in nature, is based on the mar-tingale characterization of Markov processes as developed by Stroock andVaradhan. Here again one must verify convergence of generators, but weakcompactness arguments and the martingale characterization of the limit areused to complete the proof. The third technique depends on the representationof the processes as solutions of stochastic equations, and is more in the spiritof classical analysis. If the equations “converge,” then (one hopes) the solu-tions converge.Although the book is intended primarily as a reference, problems areincluded in the hope that it will also be useful as a text in a graduate course onstochastic processes. Such a course might include basic material on stochasticprocesses and martingales (Chapter 2, Sections 1-6). an introduction to weakconvergence (Chapter 3, Sections 1-9, omitting some of the more technicalresults and proofs), a development of Markov processes and martingale prob-lems (Chapter 4, Sections 1-4 and 8). and the martingale central limit theorem(Chapter 7, Section I). A selection of applications to particular processes couldcomplete the course.V
  • 5. Vi PREFACEAs an aid to the instructor of such a course, we include a flowchart for allproofs in the book. Thus, if ones goal is to cover a particular section, the chartindicates which of the earlier results can be skipped with impunity. (It alsoreveals that the courseoutline suggestedabove is not entirelyself-contained.)Results contained in standard probability texts such as Billingsley (1979) orBreiman (1968) are assumed and used without reference, as are results frommeasure theory and elementary functional analysis. Our standard referencehere is Rudin (1974). Beyond this, our intent has been to make the bookself-contained (an exception being Chapter 8). At points where this has notseemed feasible, we have included complete references, frequently discussingthe needed material in appendixes.Many people contributed toward the completion of this project. CristinaCostantini, Eimear Goggin, S.J. Sheu, and Richard Stockbridge read largeportions of the manuscript and helped to eliminate a number of errors.Carolyn Birr, Dee Frana, Diane Reppert, and Marci Kurtz typed the manu-script. The National Science Foundation and the University of Wisconsin,through a Romnes Fellowship, provided support for much of the research inthe book.We are particularly grateful to our editor, Beatrice Shube, for her patienceand constant encouragement. Finally, we must acknowledge our teachers,colleagues,and friends at Wisconsin and Michigan State, who have providedthe stimulatingenvironment in which ideas germinateand flourish. They con-tributed to this work in many uncredited ways. We hope they approve of theresult.STEWARTN. ETHIERTHOMASG. KURTZSalt Lake City, UtahMadison, WisconsinAugust 198s
  • 6. Introduction1 Operator SemigroupsDefinitions and Basic Properties, 6The Hille-Yosida Theorem, 10Cores, 16Multivalued Operators, 20Semigroups on Function Spaces, 22Approximation Theorems, 28Perturbation Theorems, 37Problems, 42Notes, 472 Stochastic Processesand Martingales12345678910Stochastic Processes, 49Martingales, 55Local Martingales, 64The Projection Theorem, 71The Doob-Meyer Decomposition, 74Square Integrable Martingales, 78Semigroups of Conditioned Shifts, 80Martingales Indexed by Directed Sets,Problems, 89Notes, 938449vii
  • 7. viii CONTENTS3 Convergence of Probability Measures1 The Prohorov Metric, 962 Prohorov’sTheorem, 1033 Weak Convergence, 1074 Separatingand ConvergenceDeterminingSets, 1115 The Space D,[O, GO), 1166 The Compact Setsof DEIO,a), 1227 Convergencein Distribution in &[O, m), 1278 Criteria for RelativeCompactnessin DKIO,a), 1329 Further Criteria for Relative Compactnessin D,[O, oo), 14110 Convergenceto a Processin C,[O, a), 14711 Problems, 15012 Notes, 1544 Generators and Markov Processes1 Markov Processes and Transition Functions, 1562 Markov Jump Processes and Feller Processes, 1623 The MartingaleProblem: Generalitiesand SamplePath Properties, 1734 The Martingale Problem: Uniqueness, the MarkovProperty,and Duality, 1825 The MartingaleProblem: Existence, 1966 The Martingale Problem: Localization, 2167 The MartingaleProblem:Generalizations, 22I8 ConvergenceTheorems, 2259 Stationary Distributions, 23810 Perturbation Results, 253I 1 Problems, 26112 Notes, 2735 Stochastic Integral Equations1 Brownian Motion, 2752 StochasticIntegrals, 2793 StochasticIntegral Equations, 2904 Problems, 3025 Notes, 3056 Random Time Changes1 One-Parameter Random Time Changes, 3062 Multiparameter Random Time Changes, 3113 convergence, 32195155275306
  • 8. 4 Markov Processesin Zd,3295 Diffusion Processes, 3286 Problems, 3327 Notes, 3357 InvariancePrinciplesand DiffusionApproximations1 The Martingale Central Limit Theorem, 3382 Measures of Mixing, 3453 Central Limit Theorems for Stationary Sequences, 3504 Diffusion Approximations, 3545 Strong Approximation Theorems, 3566 Problems, 3607 Notes, 3648 Examplesof Generators1 NondegenerateDiffusions, 3662 Degenerate Diffusions, 3713 Other Processes, 3764 Problems, 3825 Notes, 3859 BranchingProcesses1 Galton-Watson Processes, 3862 Two-Type Markov Branching Processes, 3923 Branching Processes in Random Environments, 3964 Branching Markov Processes, 4005 Problems, 4076 Notes, 40910 Genetic ModelsI The Wright-Fisher Model, 4112 Applications of the Diffusion Approximation, 4153 Genotypic-FrequencyModels, 4264 Infinitely-Many-AlleleModels, 4355 Problems, 4486 Notes, 45111 Density DependentPopulationProcesses1 Examples, 4522 Law of Large Numbers and Central Limit Theorem, 455337365386410452
  • 9. 3 Diffusion Approximations, 4594 Hitting Distributions, 4645 Problems, 4666 Notes, 46712 RandomEvolutions1 Introduction, 4682 Driving Process in a Compact StateSpace, 4723 Driving Process in a Noncompact State Space, 4794 Non-Markovian Driving Process, 4835 Problems, 4916 Notes, 491Appendixes1 Convergenceof Expectations, 4922 Uniform Integrability, 4933 Bounded PointwiseConvergence, 4954 MonotoneClass Theorems, 4965 Gronwall’sInequality, 4986 The Whitney Extension Theorem, 4997 Approximation by Polynomials, 5008 Bimeasuresand Transition Functions, 5029 Tulcea’sTheorem, 50410 MeasurableSelectionsand Measurabilityof Inverses, 50611 AnalyticSets, 506ReferencesIndexFlowchart168492508521529
  • 10. The development of any stochastic model involves !he identification of proper-ties and parameters that, one hopes, uniquely characterize a stochastic process.Questions concerning continuous dependence on parameters and robustnessunder perturbation arise naturally out of any such characterization. In fact themodel may well be derived by some sort of limiting or approximation argu-ment. The interplay between characterization and approximation or con-vergence problems for Markov processes is the central theme of this book.Operator semigroups, martingale problems, and stochastic equations provideapproaches to the characterization of Markov processes, and to each of theseapproaches correspond methods for proving convergenceresulls.The processes of interest to us here always have values in a complete,separable metric space E, and almost always have sample paths in DE(O,m),the space of right continuous E-valued functions on [O, 00) having left limits.We give DEIO, 00) the Skorohod topology (Chapter 3), under which it alsobecomes a complete, separable metric space. The type of convergence weare usually concerned with is convergence in distribution; that is, for asequence of processes { X J we are interested in conditions under whichlimn.+mE[f(X.)J = &ff(X)] for everyfg C(D,[O, 00)). (For a metric space S,C(S)denotes the space of bounded continuous functions on S. Convergence indistribution is denoted by X,=. X . ) As an introduction to the methods pre-sented in this book we consider a simple but (we hope) illuminatingexample.For each n 2 1, defineU x ) = 1 + 3x x - - , y,(x) = 3x + +-t>(.- r>.( 1 ) ( :> 1Markov Processes Characterizationand ConvergenceEdited by STEWARTN. ETHIER and THOMASG.KURTZCopyright 01986,2005 by John Wiley & Sons,Inc
  • 11. 2 INTRODUCTIONand let U, be a birth-and-death process in b, with transition probabilitiessatisfying(2) P{K(r +h) =j + I I ~ ( t )a j } = n~,,(:)h +~ ( h )and(3)as Ado+. In this process, known as the sChlo8l model, x(r)represents thenumber of molecules at time t of a substanceR in a volume n undergoing thechemical reactions(4)with the indicated rates. (See Chapter 11, Section 1.)(5) x,,(t)= n’/*(n- yn(n1/2r)- 1). r 2 0.The problem is to show that X,convergesin distribution to a Markov processX to be characterized below.The first method we consider is based on a semigroup characterization ofX. Let En= {n‘/*(n-‘y - I) :y E Z+},and note that1 33 1Ro R, R2 +2R S 3R,We rescale and renormalize letting(6) ~ w m=Erm.(t)) I x m = XJdefinesa semigroup {T,(I)}on B(E,) with generator of the form(7) G,/(x) =: n3’2L,(1 +n -‘/‘x){f(x +n -’I4)-/(x)}+n3/2pn(1 +n -l/*x){/(x - -3/41 - ~ ~ x ~ ~ .(SeeChapter I.) Letting A(x) = 1 +3x2,p(x) =3x +x3, and(8) G~’(x)= 4/”(x) -x ~ ’ ( x ) ,a Taylor expansionshows that(9) G,f ( x )=Gf(x)+t1”~{,4,,(I +n.-‘/*x) -A( 1 +n -‘l4x)}{f(x +n -’I*) -/(x)}+n3/3{p,(1 +n-‘l4x)-I(1 +~t-I/*x)}{J(X- n-3/4) -f(x)}+ A(1 +n-l/*x) I’(1 -u){f”(x +un-”*) -r(x)} du
  • 12. for all/€ C2(R)withf‘ E Cc(R)and all x E Em.Consequently.for such/;lim sup I G,f(x)- Gf(x)1 = 0.n-m x c E .Now by Theorem 1.1 of Chapter 8,(1 1) A E ((AGf):f€C[-00, 001n C’(R), G/E C[-aO, 001)is the generator of a Feller semigroup {T(t)}on C[-00, 001. By Theorem 2.7of Chapter 4 and Theorem I. I of Chapter 8, there exists a diffusion process Xcorresponding to (T(t)),that is, a strong Markov process X with continuoussample paths such that(12) ECJ(X(t))I *.*I = - S)S(X(d)for allfe C[-00, a03 and t 2 s 2 0.(4c: = a(X(w):u 5 s).)To prove that X, 3X (assuming convergence of initial distributions), itsuffices by Corollary 8.7 of Chapter 4 to show that (10) holds for all/in zt coreD for the generator A, that is, for allf in a subspace D of 9 ( A ) such that A isthe closure of the restriction of A to D.We claim that(13) D -= (/+ g:/I Q E C’(R),/’ E: Cc(W),(x’g)‘ E Cc(W)}is a core, and that (10) holds for all/€ D. To see that D is a core, first checkthat(14) ~ ( A ) = ( J E C [ - C Q , ~ ]nC2(R):f”~~(W),x3f’~C[-oo,oo]}.Then let h E C;(R) satisfy xI- 5 h sf E 9 ( A ) ,choose g E: D with (x’g)’ E Cc(W)and x 3 ( f - g)’ E e(R) and define(15)Thenj,, +g E D for each m,f, +g -+f, and G(fm +Q)-+ C/.a martingale problem. Observe thatand put h,(x) = h(x/m).GivenSdX) =S(0) - do) + (j-gY(Y)hm( Y1d ~ .s:The second method is based on the characterization of X as the solution ofis an {.Ffn)-martingalefor each /E B(E,) with compact support. Conse-quently, if some subsequence {A’,,,) converges in distribution to X , then, by thecontinuous mapping theorem (Corollary 1.9 of Chapter 3) and Problem 7 ofChapter 7,
  • 13. 4 ~ O W c r I o Nis an {Pf)-martingale for eachfe C,(R), or in other words, X is a solution ofthe martingale problem for {(AG f ) : f cC,(W)}. But by Theorem 2.3 ofChapter 8, this property characterizes the distribution on Dn[O, 00) of X .Therefore,Corollary 8.16 of Chapter 4 gives X,=X (assumingconvergenceofinitial distributions),provided we can show thatLet (p(x) I ex +e-x, and check that there exist constants C , , a O such.thatC,,a<G,cp I;C,,,rp on [-a, u] foreach n 2 I and ct > 0, and Ka+-00. Letting = inf ( f 2 0: IX,,(t) I2 a},we haveIinf C P ( Y ) ~SUP Ixn(t)lka{ostsre-G.4 TIrl L a(19)ELeXP -Cn,a(?n, 8 A 73)cp(Xn(Tn, a A VJ5 QdXn(O))lby Lemma 3.2 of Chapter 4 and the optional sampling theorem. An additional(mild)assumption on the initial distributionsthereforeguarantees(18).Actually we can avoid having to verify (18) by observing that the uniformconvergence of G,f to Gf forfe C:(R) and the uniqueness for the limitingmartingale problem imply (again by Corollary 8.16 of Chapter 4) that X, =. Xin Dad[O, 00) where WA denotes the one-point compactification of R. Con-vergence in &LO, 00) then follows from the fact that X, and X have samplepaths in DRIO,00).Both of the approachesconsidered so far have involvedcharacterizationsinterms of generators. We now consider methods based on stochasticequations.First, by Theorems 3.7 and 3.10 of Chapter 5, we can characterize X as theunique solutionof the stochasticintegralequationwhere W is a standard one-dimensional, Brownian motion. (In the presentexample, the term 2JW(t) corresponds to the stochastic integral term.) Aconvergencetheory can be developed using this characterization of X,but wedo not do so here. The interested reader is referred to Kushner(1974).The final approach we discuss is based on a characterizationof X involvingrandom time changes. We observe first that U,satisfies
  • 14. where N, and N- are independent, standard (parameter I), Poisson processes.ConsequentIy,X,satisfiesX,(r) = X,(O) +n- 3/4R+(n3/2 A,( I+n- /*X,(s))ds(22)- n-"R.(nl" 6p,(l + n-/4X,(s)) ds)+ n3l4[(A, - p&I + n - 4X,(s)) ds,where R+(u)= N+(u)- u and R_(u)= N-(u)-u are independent, centered,standard, Poisson processes. Now i t is easy to see that(23) (n /*R+(n3/2* 1, n l4R -(n32.))=.(W+,W-1,where W+ and W- are independent, standard, one-dimensional Brownianmotions. Consequently, if some subsequence {A".) converges in distribution toX, one might expect thatX(t)= X(0) + W+(4t)+ W ( 4 t )- X ( S ) ~ds.(24) s.(In this simple example, (20) and (24) are equivalent, but they will not be so ingeneral.) Clearly, (24) characterizes X,and using the estimate (18) we concludeX,-X (assuming convergence of initial distributions) from Theorem 5.4 ofChapter 6.For a further discussion of the Schlogl model and related models seeSchlogl (1972) and Malek-Mansour et al. (1981). The martingale proof ofconvergence is from Costantini and Nappo (1982), and the time change proofis from Kurtz(1981c).Chapters 4-7 contain the main characterization and convergence results(with the emphasis in Chapters 5 and 7 on diffusion processes). Chapters 1-3contain preliminary material on operator semigroups, martingales, and weakconvergence, and Chapters 8- I 2 are concerned with applications.
  • 15. 1Operator semigroups provide a primary tool in the study of Msrkov pro-cesses. In this chapter we develop the basic background for their study and theexistence and approximation results that are used later as the basis for exis-tence and approximation theorems for Markov processes. Section 1 gives thebasic definitions,and Section 2 the Hille-Yosida theorem, which characterizesthe operators that are generators of semigroups. Section 3 concerns theproblem of verifying the hypotheses of this theorem, and Sections4 and 5 aredevoted to generalizations of the concept of the generator. Sections 6 and 7present the approximationand perturbation resuJts.Throughout the chapter, L denotesa real Banach space with norm 11 * 11.OPERATOR SEMICROUPS1. DEFINITIONS AND BASIC PROPERRESA one-parameter family { T(t):t 2 0) of bounded linear operators on aBanach space L is called a semigroupif T(0)= I and T(s+t ) = T(s)T(c)for alls, t 2 0. A semigroup(T(t))on L is said to be strongly continuousif lim,,o T(r)/=/for everyfe L;it is said to be a contraction semigroupif 11T(t)II5 1 for allt 2 0.Given a bounded linear operator B on L,defineMarkov Processes Characterizationand ConvergenceEdited by STEWARTN. ETHIER and THOMASG.KURTZCopyright 01986,2005 by John Wiley & Sons,Inc
  • 16. 1. DmNmoNz AND EASIC ?ROPERTIES 7A simple calculation gives e") = e""e for all s,t 2 0, and hence {e"} is asemigroup, which can easily be seen to be strongly continuous. Furthermorewe haveAn inequality of this type holds in general for strongly continuous serni-groups.1.1 Propositionthere exist constants M 2 1 and o 2 0 such that(1-3) II T(t)lI 5 Me", t 2 0.Let (T(t))be a strongly continuous semigroup on L. ThenProof. Note first that there exist constants M 2 I and ro > 0 such that11 T(t)11 5 M for 0 I t s t o . For if not, we could find a sequence (t,} of positivenumbers tending to zero such that 11 T(t,)((-+ 00, but then the uniformboundedness principle would imply that sup,((T(rJfI1 = 00 for some f E L,contradicting the assumption of strong continuity. Now let o = t i log M.Given t 2 0, write t = kt, +s, where k is a nonnegative integer and 0 s s <t,; then(1.4) 0I(T(t)I(= II f(~)T(t,,)~Ils MM r; MM/O = Me".1.2 Corollaryeach$€ L, t -+ T(t)/is a continuousfunction from [0, 00) into L.Let {T(r))be a stronglycontinuoussemigroupon L.Then, for1.3 Remark Let { T(r)}be a strongly continuous semigroup on L such that(1.3) holds, and put S(t) = e-"T(r) for each t 2 0. Then {S(t)) is a stronglycontinuoussemigroupon L such that(1.7) IIW II s M, t 2 0.
  • 17. 8 OraATORS€MIGROWSIn particular, if M = 1, then {S(t)} is a strongly continuous contraction semi-group on L.Let {S(t)} be a strongly continuous semigroup on L such that (1.7) holds,and define the norm 111 111 on L byThen 11f11 5; IIIJIII 5; Mllfll for eachfE L, so the new norm is equivalent tothe original norm; also, with respect to 111 * 111, {S(t)) is a strongly continuouscontractionsemigroupon L.Most of the results in the subsequent sections of this chapter are stated interms of strongly continuous contraction semigroups. Using these reductions,however, many of them can be reformulated in terms of noncontraction semi-groups. 0A (possibly unbounded) linear operator A on L is a linear mapping whosedomain 9 ( A ) is a subspaceof L and whose range a ( A ) lies in L. The graph ofA is given byNote that L x L is itself a Banach space with componentwise addition andscalar multiplication and norm [l(J @)[I= llfll + IIg 11. A is said to be closed if9 ( A )is a closed subspaceof L x L.The (injinitesimal) generator of a semigroup {T(c))on L is the linear oper-ator A defined by(1.10)1A , = lim ;{T(t)f-J}.1-0The domain 9 ( A ) of A is the subspaceof allJE L for which this limit exists.Before indicating some of the propertiesof generators,we briefly discuss thecalculus of Banach space-valued functions.Let A be a closed interval in (- 00, a),and denote by CJA) the space ofcontinuous functions u: A+ L. Let Cl(A)be the space of continuouslydiffer-entiablefunctionsu: A +L.If A is the finite interval [a, b], u : A + L is said to be (Rietnann)integrableover A if limd,, u(sk)(fk - t,,-I) exists, where a = to S s, 5 Il I .. 5;t,- ,s s, s f n = b and S = max (rr - f k - l); the limit is denoted by jb,u(t)dt oru(t)dt. If A = [a, a),u: A + L is said to be integrable over A if u I , ~ , ~ ,isintegrable over [a, b] for each b 2 a and limg,, Jtu(t)dt exists; again, thelimit is denoted by {A ~ ( t )dt or {; u(r)dt.We leave the proof of the followinglemma to the reader (Problem3).
  • 18. 1. MflMTlONS AND 8ASlC PROPERTIES 91.4 Lemma (a) If u E C,jA) and JAll u(t)I1 dt < 00, then u is integrable overA and(1.1 I )In particular, if A is the finite interval [a, 61, then every function in C,(A) isintegrable over A.Let B be a closed linear operator on L. Suppose that u E CJA),u(t) E 9 ( E )for all t E A, Bu E CJA), and both u and Bu are integrable overA. Then JA U(t) dt E 9 ( B )and(1.12)(c) If u E Ci,[a, b], then(b)B u(t) dt Bu(t) dr.I =I(1.13) I$u(t)dt = u(b)- u(a).1.5 Proposition Let (T(t)}be a strongly continuous semigroup on L withgenerator A.(a) Iff€ L and t 2 0, then So T(s)fdsE 9 ( A )and(1.14)(b)(1.15)(c) Iff€ 9 ( A )and r 2 0, thenIff€ 9 ( A )and t 2 0. then T(t)/EB(A)andd--r(t)j=A T ( t ) / = T(r)AJdt(1.16) T(t)J-j= A T(.s)jds = T(s)Afds.Proof. (a) Observe thatfor all h > 0, and as h -,0 the right side of(I.17)converges to T(t)/-f:
  • 19. 10 OPERATOR SEMlGROUPS(b) Since(1.18)for all h > 0, where A, = h-[T(h)-I], it follows that T(t)fe9 ( A )and (d/dt)+T(t)f= A T(r)/= T(t)A$ Thus, it sufices to check that(d/df)-T(r)f -- T(r)Af(assumingt > 0).But this followsfrom the identity(1.19)1-- h- h ) f - W)SI- T(t)A/= T(t -h)[A, -A]f+ [T(I -h) - T(t)]Af,valid for 0 < h 5 t.(c) This is a consequenceof(b)and Lemma 1.4(c). 01.6 Corollary If A is the generator of a strongly continuous semigroup{ T(t)}on L, then 9 ( A )is densein L and A is closed.Proof. Since Iim,,o + t - fo T(s)f ds =f for every fc L, Proposition that 9 ( A ) is dense in L. To show that A is closed, let {f,} c 9 ( A )satisfy$,4f and AS,- g. Then T(r)f,-Jn = roT(s)AJnds for each t > 0, so,letting n-+ a,we find that T(r)f-f= 6 T(s)gds. Dividing by t and letting0I-+ 0, we concludethatje 9 ( A )and Af= g.2. THE HILL€-YOSIDA THEORfMLet A be a closed linear operator on L. If, for some real 2, A - A (K A1 - A) isone-to-one, W ( l -A) = L, and (1-A)- is a bounded linear operator on L,then 1 is said to belong to the resoluent set p(A) of A, and RA = (A -A)- iscalled the resoluenr (at A) of A.2.1 Proposition Let {T(I))be a strongly continuous contraction semigroupon L with generator A. Then (0,00) c p(A) and(2.1) (A-A)-g = e-AT(tb drfor all g E L and d > 0.Proof. Let 1 > 0 be arbitrary. Define U, on L by UAg = J$e-"T(t)g df.Since(2.2)0)It U ~ g l l Lrne-"l/ T(r)sll df 9~-llgll
  • 20. 2. THE HILLL-YOSIDA THEOREM 11foreach g E L, U Ais a bounded linear operator on L. Now given g E L,for every h > 0,so, letting h-, 0,.we find that UAg E g ( A ) and AUAg=AU,g - g, that is,(2.4) (1- A)UAg 9, 9 E L.In addition,if g E $@(A),then (using Lemma 1.4(b))(2.5) UAAg = e- "T(t)Ag dt = [A(e-"T(t)g) dt= A lme-"(t)g dt = AuAg,so(2.6) uA(A- A)g = 99 g E %A).By (2.6),A - A is one-to-one, and by (2.4),9 ( A - A) = L.Also, (A - A)- =U Aby (2.4) and (2.6), so A E p(A). Since rl > 0 was arbitrary, the proof iscomplete. 0Let A be a closed linear operator on L. Since (A - A)(p - A ) =(p - AHA - A ) for all A, p E p(A), we have (p - A)-(A - A).. = (A - A)--(p - A ) I , and a simple calculationgives the resolvent identity(2.7) RA R, = R, RA = (A - p)-(R, - RA), A, p E p(A).IfI.Ep(A)andJA-pI< I)R,II-,then(2.8)definesa bounded linear operator that is in fact (p - A ) - . In particular, thisimplies that p(A) is open in R.A linear operator A on L is said to be dissipative if IIJ j - AjII 2 Allfll forevery/€ B(A)and I > 0.2.2 lemma Let A be a dissipative linear operator on L and let 1 > 0. ThenA is closed if and only if #(A - A) is closed.Proof. Suppose A is closed. If (1;)c 9 ( A ) and (A - A)jw-+ h, then the dissi-pativity of A implies that {J.} is Cauchy. Thus, there exists/€ L such that
  • 21. 12 OPERATORSEMICRourSL.+J and hence Al,,--+ Af - h. Since A is closed,fe 9 ( A ) and h = (A - A)J Itfollowsthat @(I - A ) is closed.Suppose*(A -A) isclosed.If {L}c 9(A),S,-J and A h 3g, then (A -A)fn-+?/- g, which equals (A - A)J, for somefo E 9(A). By the dissipativity of A,0f n d f o ,and hence/=fO E 9 ( A )and As= g. Thus, A is closed.2.3 lemma Let A be a dissipative closed linear operator on L, and putp+(A)= p(A) n (0, 00). If p+(A)is nonempty,then p+(A)= (0, a).froof. It suffices to show that p+(A)is both open and closed in (0, a).Since&A) is necessarily open in R,p+(A) is open in (0, 00). Suppose that {i"}cp+(A)and A,-+ A > 0. Given g E L,let g,, = (A - AKA, - A)-g for each ti, andnote that, because A is dissipative,(2.9) lim IIg,, -g11 = lim 11(I- Am)& - A)-g 11 5 lim 1.1-1.111g11 = 0.Hence @(A -A) is dense in L, but because A is closed and dissipative,9 ( A -A) is closed by Lemma 2.2, and therefore @(A - A) = L. Using thedissipativity of A once again, we conclude that I -A is one-to-one andII(A -A)-(I s I - . It follows that 1 B p+(A),so p+(A)is closed in (0, a),asI - a l *-.OD n-al 4required. 02.4 lemma Let A be a dissipativeclosed linear operator on L, and supposethat 9 ( A ) is dense in L and (0, 03) c p(A). Then the Yosida approximation A,of A, defined for each A > 0 by A, = RA(A -A)-, has the following proper-ties:la) For each A >0, Al is a bounded linear operator on L and {PJ}is a(b) A, A, = A, A, for all A, p > 0.(c) lim,-m A, f= Affor everyfe 9(A).strongly continuouscontraction semigroupon L.Proof.(I - A)R, = I on L and R,(A - A ) = I on $+I),it followsthat(2.10) A,=AR,-Al on L, A > O ,andFor each R > 0. let R, = (A-A)- and note that 11R, 11 5 A - I . Since
  • 22. 2. T M HILL€-YOSIDA THEOREM 13for all t 2: 0, proving (a).Conclusion (b) is a consequenceof (2.10)and (2.7). Asfor (c),we claim first that(2.13) lim I R , f = f , SE L.d-+mNoting that llLRaf-lll = IIRAAfll s A-I(A/II 4 0 as A+ a, for eachf e 9 ( A ) , (2.13) follows from the facts that 9 ( A ) is dense in L andlll.Ra - Ill S 2 for all 1 > 0. Finally, (c) is a consequence of (2.1 I) and(2.I 3). 02.5 lemma If B and C are bounded linear operators on L such thatBC = CB and 11elB(II; I and 11efc11 5 I for all t 1 0, then(2.14) IIe"!f - elC/ It It It Bf - C/I1for everyfe L and t 2 0.Proof. The result follows from the identity= [e"e- B - C)fds.(Notethat the last equality uses the commutivity of B and C.) 0We are now ready to prove the Hille-Yosida theorem.2.6 Theorem A linear operator A on L is the generator of a strongly contin-uous contraction semigroup on L if and only if:(a) 9 ( A ) is dense in L.(b) A is dissipative.(c) a(1- A) = L for some R > 0.Proof. The necessity of the conditions (a)+) follows from Corollary 1.6 andProposition 2.1. We therefore turn to the proof of sulliciency.By (b),(c),and Lemma 2.2, A is closed and p(A) n (0, m) is nonempty, soby Lemma 2.3, (0, m) c p(A). Using the notation of Lemma 2.4, we define foreach L > 0 the strongly continuous contraction semigroup {T(c)} on L byK(t)= erAA.By Lemmas 2.4b) and 2.5,(2.16) IInw- q(t)/ll 111AJ- AJll
  • 23. 14 OrUATOROMCROUISfor all f~ L, t 2 0, and A, p > 0. Thus, by Lemma 2.4(c), limA*mT,(t)/existsfor-all t 2 0, uniformly on bounded intervals, for allfe 9(A), hence for everyf~ B(A)= L.Denoting the limit by T(t)fand using the identity(2.17) T(s+t ) j - T(s)T(t)f=[T(s+r) - T,(s +t)Jf+ T,(s)CT,(t) - 7(01S+ CT,(s) - WJWJ;we concludethat { T(t)}is a stronglycontinuouscontractionsemigroupon L.I.5(c),It remains only to show that A is the generator of {T(t)}.By Proposition(2.18)foraltfE L, t 2 0,and R > 0. For eachfE 9 ( A )and r 2 0, the identity(2.19)together with Lemma 244, implies that G(s)AJ-r T(s)Afas A+ bc), uni-formlyin 0 5 s s t. Consequently,(2.18) yieldsT,(s)As- T(s)Af= T*(sXAJ-Af)+ cTAW - 7wl A/;(2.20)for all/€ 9 ( A ) and t 2 0. From this we find that the generator B of { T(r)}isan extension of A. But, for each 1 >0,A -B is one-to-one by the necessity of(b),and #(A -A) = L since rl E p(A). We conclude that B = A, completing theproof. 0The above proof and Proposition 2.9 below yield the followingresult as aby-product.2.7 Proposition Let {T(t)}be a strongly continuous contraction semigroupon L with generator A, and let Ad be the Yosida approximation of A (definedin Lemma 2.4). Then(2.21)so, for each fE L, liniA-,me"1/= T(r)ffor all I 2 0, uniformly on boundedintervals.1Ie"Y- T(t)fII 5 tit As-AfII, f s %4), t & 0,rt > 0,2 8 Corollary Let {T(r)}be a strongly continuouscontraction semigrouponL with generator A. For M c L,let(2.22) Ay i= { A > 0: A(A -A)- : M 4M}.If either (a)M is a closed convex subset ofL and AM is unbounded,or (b)M isa closed subspaceof L and AM is nonempty, then(2.23) T(t):M-+M, t 2 0.
  • 24. 1. TH€ HNLE-VOSIDA THEOREM 1sProof. If A, j~> 0 and I1 -p/lI < I, then (cf.(2.8))(2.24) p ( p - A ) - = n = Of ;(*-$[A(I-A)-1]"?Consequently, if M is a closed convex subset of L, then I E AM implies(0, A] c AM, and if M is a closed subspaceof L, then A. E AM implies(0, 2 4 tA,,, .Therefore, under either (a)or (b),we have AM = (0, 00). Finally, by (2.10).(2.25) exp {IA,} = exp { - t I ) exp {tA[l(lt - A ) - ] )forall I 2 0 and I > 0, so the conclusion follows from Proposition 2.7. 02.9 Proposition Let { T(t)} and {S(t)} be strongly continuous contractionsemigroups on L with generators A and B, respectively. If A = B, thenT(t)= S(t) for all r 2 0.Proof. This result is a consequenceof the next proposition. 02.10 Proposition Let A be a dissipative linear operator on L. Suppose thatu : [0, a)-+L is continuous, ~ ( t )E Q(A) for all r > 0, Au: (0, a)-+L is contin-uous, and(2.26) u(t) = U(E) + Au(s) ds,for all t > E > 0. Then IIu(r)II 5 II40)It for all t 2 0.
  • 25. 16 OPERATOR SEMlCROUrSwhere the first inequality is due to the dissipativity of A. The result followsfrom the continuity of Au and u by first letting max (t, -ti- ,)+ 0 and thenlettingc+ 0. 0In many applications, an alternative form of the Hille-Yosida theorem ismore useful. To state it, we need two definitionsand a lemma.A linear operator A on L is said to be closable if it has a closed linearextension. If A is closable, then the closure A of A is the minimal closed linearextension of A; more specifically, it is the closed linear operator 6 whosegraph is the closure(in L x L)of the graph of A.2.11 lemma Let A be a dissipativelinear operator on L with 9 ( AL.Then A is ciosableand L@(A -A) =9?(A -A^)forevery I > 0.dense inProof. For the first assertion, it suffices to show that if {A}c 9 ( A ) , 0,and Af,-+g E L,&heng = 0. Choose {g,} c $(A) such that g,,,--tg. By thedissipativity of A,(2.28) IIV - - 4It = lim II(A- A h , + &)I1a-m2 lim AIlgm + KII AIIgmIIn- mfor every 1 >0 and each m. Dividing by I and letting A+ 00, we find thatIIg, -g II 2 IIg, II foreach m. Letting m--, 00,we conclude that g = 0.Let 1 > 0. The inclusion @(A - A) =)@(A - A) is obvious, so ro proveequality, we need only show that 5?(I -A) is closed. But this is an immediateconsequenceof Lemma 2.2. 02.12 Theorem A linear operator A on L is closable and its closure A is thegenerator of a strongly continuouscontractionsemigroupon L if and only i f(a) 9 ( A )is dense in L.(b) A is dissipative.(c) B(1- A) is dense in L for some A > 0.Proof. By Lemma 2.11, A satisfies(a)-+) above if and only if A is closable andA’ satisfies(a)+) of Theorem 2.6. a3. CORESIn this section we introduce a concept that is of considerable importance inSections6 and 7.
  • 26. Let A be a closed linear operator on L. A subspace D of 9 ( A ) is said to be acore for A if the closure of the restriction of A to D is equal to A (i.e., ifAJ, = A).-3.1 Proposition Let A be the generator of a strongly continuous contractionsemigroup on L. Then a subspace D of 9 ( A )is a core for A if and only if D isdense in L and w(1. - AID)is dense in L for some 1> 0.3.2 Remark A subspace of L is dense in L if and only if it is weakly dense(Rudin (l973), Theorem 3.12). 0Proof. The sufficiency follows from Theorem 2.12 and from the observationthat, if A and B generate strongly continuous contraction semigroups on Land if A is an extension of 8, then A = B. The necessity depends on Lemma2.1 1. 03.3 Proposition Let A be the generator of a strongly continuous contractionsemigroup IT([)}on L. Let Do and D be dense subspaces of L with Do c D c9 ( A ) .(Usually,Do = D.)If T(r):Do-+ D for all t 2 0, then D is a core for A.Proof. Givenf E Doand L > 0,(3.1)for n = I, 2,. ...By the strongcontinuity of { T(t)}and Proposition 2.1,(3.2)Ilim (i.- A)S, = lim - e ak/n7(:)(,l - A)/n-m n-(u k = O= lme -"T(t)(d - A)$&= (1- A ) - ( L - A)!=/:so a(>.- A ID) 3 Do.This sufices by Proposition 3.I since Dois dense in L. 0Given a dissipative linear operator A with 9 ( A )dense in L, one often wantsto show that A generates a strongly continuous contraction semigroup on L.By Theorem 2.12, a necessary and sufficient condition is that .%(A - A ) bedense in L for some A > 0. We can view this problem as one of characterizinga core (namely, g ( A ) )for the generator of a strongly continuous contractionsemigroup, except that, unlike the situation in Propositions 3.1 and 3.3, thegenerator is not provided in advance. Thus, the remainder of this section isprimarily concerned with verifying the range condition (condition (c)) ofTheorem 2.12.Observe that the followingresult generalizes Proposition 3.3.
  • 27. 18 OrUATOR YMIGROUK3.4 Propositlon Let A be a dissipative linear operator on L,and Do a sub-space of B(A)that is dense in L. Suppose that, for eachJE Do, there exists acontinuous function u,: [O, 00)" L such that u,(O) =1; u,(t) E .@(A) for allr > 0, Au,: (0, a)-+L is continuous,and(3.3)for all t > E > 0. Then A is closable, the closure of A generates a stronglycontinuouscontraction semigroup {T(f)}on L,and T(t)J= u,(t) for allfE Doand r 2 0.Proof. By Lemma 2.11, A is closable. Fix f~ Do and denote uf by u. Letto > E > 0, and note that I:"e-u(t) dt E 9(A)and(3.4) 2loe-u(t) dt = e-Au(t) At.Consequently,(3.5)I0I"e-u(r) dt = (e-a -e-O)u(c) + loe- [Au(s) ds dt= (e-- e-O)u(c) += A I"e 3 ( t ) dt +e-u(c) -e-Ou(t,).(e-# - e-O)Au(s) dsI"Since IIu(t)(l5 llfll for all t 2 0 by Proposition 2.10, we can let 6-0 andto-+ Q) in (3.5)to obtain $; e-u(t) dr E B(2)and(3.6) (I - 2)ime-u(t)dr =J:We conclude that @(l -2)3 Do, which by Theorem 2.6 proves that 2gener-ates a strongly continuous contraction semigroup {T(r)}on L. Now for eachfE Do.(3.7) W f- W f=Im4mfor all t > E > 0. Subtracting (3.3) from this and applying Proposition 2.100once again,we obtain the second conclusion of the proposition.The next result shows that a suficient condition for A to generate is that Abe triangulizable. Of course, this is a very restrictive assumption, but it isoccasionallysatisfied.
  • 28. 3. CORES 193.5 Proposition Let A be a dissipative linear operator on L, and supposethat L,, L,, L 3 , a sequence of finite-dimensionalsubspaces of 9 ( A )suchthat u."-,L, is dense in L. If A : L , 4 L, for n = I, 2, . ..,then A is closableand the closure of A generates a strongly continuous contraction semigroupon L.Proof. For n = 1, 2, .. .,(A - AWL,) L, for all 1 not belonging to the set ofeigenvalues of AIL., hence for all but at most finitely many L > 0. Conse-quently,(A - AWU,", ,L,) = u:=,L,for all but at most countably many L > 0and in particular for some A > 0. Thus, the conditions of Theorem 2.12 aresatisfied. C3We turn next to a generalization of Proposition 3.3 in a different direction.The idea is to try to approximate A sufficiently well by a sequence of gener-ators for which the conditions of Proposition 3.3 are satisfied. Before statingthe result we record the followingsimple but frequently useful lemma.3.6 Lemma Let A,, A 2 , .. I and A be linear operators on L, Do a subspaceof L, and A > 0. Suppose that, for each g E Do, there existsJ, E g(A,)nd(A)for n = 1.2,. . .such that g, = ( A - A,)f,+gasn-+ 60 andlim [[(A,- A)Ll[= 0.n-.m(3.8)Then *(A - A) 3 Do.Proof. Given g E Do, choose {f,} and {g,} as in the statement of thelemma, and observe that limn-mII(A - A)J, -g,II -- 0 by (3.8). It follows that0limn+mI(( A - A)f, - g 11 = 0, giving the desired result.3.7 Proposition Let A be a linear operator on L and Do and D, densesubspaces of L satisfying Do c 9 ( A ) c D, c L. Let 111 . 111 be a norm on D,.For n = 1,2, . ..,suppose that A, generates a strongly continuous contractionsemigroup IT&)) on L and d ( A )c O(A,). Suppose further that there existw 2 0 and a sequence {&,} c (0, 60) tending to zero such that, for n = 1.2, ...,and(3.11) T,(t):Do+ 9(A), r 2 0.Then A is closable and the closure of A generates a strongly continuouscontraction semigroupon L.
  • 29. 20 OPERATOISMCROUPSProof. Observe first that O(A)is dense in L and, by (3.9) and the dissipativityof each A,, A is dissipative. It therefore sufices to verify condition (c) ofTheorem 2.12.Fix 1 > o.Given g E Do, let(3.12)for each m, n 2 1 (cf. (3.1)). Then, for n = 1, 2, ..., (A - An)fm,,-+e-T(f)(A - An)g dt = g as m-r 00, so there exists a sequence {m,f ofpositiveintegerssuch that (A -A,,)S,,,-+ gas n--, 03. Moreover,(3.13) It(An -.Alfm., n II 111fm. n 111M 2k = OIllg1115 enm,-1 C e- Wa&h- 0 as n+mby (3.9) and (3.10), so Lemma 3.6 gives the desired conclusion. 03.8 Corollary Let A be a linear operator on L with B(A) dense in L, and letIll * 111 be a norm on 9 ( A ) with respect to which 9 ( A ) is a Banach space.For n = 1, 2, ..., let T. be a linear 11 ))-contraction on L such thatT,: 9(A)-+ 9 ( A ) , and define A, = n(T, - I). Suppose there exist w 2 0 and asequence {t,} c (0, a)tending to zero such that, for n = 1, 2, ...,(3.9) holdsand(3.14)Then A is closable and the closure of A generates a strongly continuouscontraction semigroupon L.Proof. We apply Proposition 3.7 with Do = D, = 9(A). For n = I, 2,. , .,exp (t.4,):9 ( A )+9 ( A )and(3.15) 111~ X P(tAn) I m A ) 111 S ~ X P{ -nil exp {nt111T.(@(A)111f s ~ X P{allfor all t 2 0, so the hypothesesof the proposition are satisfied. 04. MULTlVAlUED OPERATORSRecall that if A is a linear operator on L,then the graph g(A) of A is asubspace of L x L such that (0,g) E g(A) implies g = 0. More generally, weregard an arbitrary subset A of L x L as a multiualued operator on L withdomain 9 ( A ) = {/: (J g) E A for some g } and range *(A) = (g: (JI g ) e A forsome/}. A c L x L is said to be linear if A is a subspace of L x L. If A islinear, then A is said to be sinyfe-uaiuedif (0, g) E A impliesg = 0; in chis case,
  • 30. 4. MULTIVALUED OPERATORS 21A is a graph of a linear operator on L, also denoted by A, so we write Af = g if(Jg) E A. If A c L x L is linear, then A is said to be dissipariue if(I lf- g II 2 R (I.fII for all (5g) E A and R > 0 ; the closure A’ of A is of coursejust the closure in L x L of the subspace A. Finally, we define1 - A = ((JAf- g): (Jg) E A } for each 1> 0.Observe that a (single-valued)linear operator A is closable if and only if theclosure of A (in the above sense) is single-valued. Consequently. the term“closable” is no longer needed.We begin by noting that the generator of a strongly continuous contractionsemigroup is a maximal dissipative (multivalued)linear operator.4.1 Proposition Let A be the generator of a strongly continuous contractionsemigroup on L. Let B c L x L be linear and dissipative, and suppose thatA c 8.Then A = B.Proof. Let U;g) E B and 1 > 0. Then ( f . 1.- g) E I - B. Since A E p(A),there exists h E 9 ( A ) such that Ah - Ah = AJ- g. Hence (h, If--g) E1 - A c A - B. By linearity, (1-h, 0)E I - B, so by dissipativity, J = h.0Hence g = Ah, so (J; g) E A.We turn next to an extension of Lemma 2.1 1.4.2 Lemma Let A t L x L be linear and dissipative.Then-(4.1) A0 = {(SI 8) E A’: 9 E @A)}is single-valued and cR(A - A ) = 9(1 - A)for every 1 > 0.Proof. Given (0,g) E A,, we must show that g = 0. By the definition of A,,there exists a sequence {(g., h,)] c A such that g,-+g. For each n,(g,, h, + l,g) E A by the linearity of A, so II Ag, - h,, - Ag I1 2 dIIg, II for every1. > 0 by the dissipativity of A’. Dividing by 1 and letting A- a,we find thatIlg,, - gll 2 )lg. I1for each n. Letting n-, a,we conclude that g = 0.The proof of the second assertion is similar to that of the second assertionof Lemma 2.I I. 0The main result of this section is the following version of the Hille-Yosidatheorem.4.3 Theorem Let A c L x L be linear and dissipative, and define A. by(4.1). Then A. is the generator of a strongly continuous contraction semigroupon 9 ( A )if and only if 9?(R - A) 2 9 ( A )for some A > 0.-Proof. A, is single-valued by Lemma 4.2 and is clearly dissipative, so by theHille-Yosida theorem (Theorem- 2.6), A, generates a strongly continuous-contraction semigroup on 9 ( A ) if and only if 9 ( A , ) is dense in 9 ( A ) and@(I. - A,) = 9 ( A ) for some A > 0. The latter condition is clearly equivalent to
  • 31. 22 OPERATOR SEMIGROUPS9 ( L - A)=3 a(A)for some A >0. which by Lemma 4.2 is equivalent to41(1 - A) 3 d(A)for some 1->0. Thus, to complete the proof,-it suffices toshow that 9 ( A o ) is dense in 9 ( A ) assuming that 5?(A - A,) = B(A)for some1 > 0.By Lemma 2.3, Se(1- A,)= 9 ( A ) for every A >O, so 9(1- A ) =9 ( R - A)3 9 ( A ) for every R > 0. By the dissipativity of A, we may regard(A - A)- as a (single-valued)bounded linear operator on .@(A - A) of normat most L- for each 1> 0. Given cf;g) E A and R > 0, Af -g e @R - A)and/E 9 ( X )c 9 ( A ) c W(A-A),so g E g(A- X),and therefore IIA(d - A)-f--/Il= II(A - A)-gll 5 1-IIgII. Since 9(A) is dense in O(A),itfollowsthat(4.2)----lim A(L - A)-y=S, fE 9 ( ~ ) .I - m(Note that this does not follow from-(2.13).) But clearly, (A- A)-:&(A - A0)+ 9(Ao), that is, (A - A)-:9(A)-+ 9(Ao), for all L > 0. In view0of (4.2), this completesthe proof.Milltivalued operators arise naturally in several ways. For example, thefollowingconcept is crucial in Sections6 and 7.For n = 1, 2, ..., let L,, in addition to L, be a Banach space with normalso denoted by 11 * 11, and let n,: L-. L, be a bounded linear transformation.Assume that sup, IIn,,II < 00. If A, c L, x L, is linear for each n 2 I, theextended limit of the sequence {A,} is defined by(4.3) ex-lim A, = {U;g) c L x L:there exists u,,8,) E A, for eachn-mn 2 1 such that IIf, -rrJll+ 0 and 11g, - n,g 113 O}.We leave it to the reader to show that cx-lim,,, A, is necessarily closed inL x L (Problem 11).To see that ex-lim,,,A, need not be single-valued even if each A, is, letL, = L, a, = I, and A, = B +nC for each n 2 1, where B and C are boundedlinear operators on L.If/ belongs to N(C),the null space of C, and h E L,then A,,(f+ (I/n)h)+ Bf+ Ch,so{(ABf+ Ch):JeN(C),h E L}c ex-lim A,.(4.4)n-mAnother situation in which multivalued operators arise is described in thenext section.5. SEMIGROUPS ON FUNCTION SPACESIn this section we want to extend the notion of the generator of a semigroup,but to do so we need to be able to integrate functions u: [O, a)+L that are
  • 32. 5. SEMICROUIS ON FUNCllON SPACES 23not continuous and to which the Riemann integral of Section 1 does notapply. For our purposes, the most efficient way to get around this difficulty isto restrict the class of Banach spaces L under consideration. We thereforeassume in this section that L is a “function space” that arises in the followingway.Let (M,a)be a measurable space, let r be a collection of positive mea-sures on A, and let 2‘ be the vector space of .,#-measurable functionsf suchthat(5.1)Note that 11. [I is a seminorm on Y but need not be a norm. LetN = { f ~9’:llfll = 0) and let L be the quotient space 9 / N ,that is, L is thespace of equivalence classes of functions in 9,wheref- g if I[/- gll = 0. Asis typically the case in discussions of Lp-spaces, we do not distinguishbetweena function in Y and its equivalenceclass in L unless necessary.L is a Banach space, the completenessfollowing as for E-spaces. In fact, if vis a o-finite measureon A’, 1 s q 5 ao,p-’ +q-’ = 1, andIlSIl --= SUP If1dP < m.r c r I(5.2)where (1 . 11, is the norm on U(v), then L = E(v). Of course, if r is the set ofprobability measures on A, then L = B(M, A),the space of bounded 4-measurable functionson M with the sup norm.Let (S, 9,v) be a a-finite measure space, let f:S x M -+R be 9’x A-measurable, and let g: S+ 10, 00) be 9’-measurable. If Ilf(s, .)[I5 g(s) for alls E S and g(s)v(ds) < m, then(5.3)and we can define j f ( s , .)v(ds) E L to be the equivalence class of functions in2’equivalent to h, where(5.4)With the above in mind, we say that u : S-+ L is measurable if there existsan Y x A-measurable function u such that u(s, .) E u(s) for each s E S.Wedefine a semigroup (T(t)}on t to be measurable if T( * )J is measurable as afunction on ([O, m), a[O,00)) for each/€ L. We define thefull generaror A’ ofa measurablecontraction semigroup (T(r)}on L by
  • 33. We note that A is not, in general,single-valued.For example, if L = B(R)withthe sup norm and T(t)f(x)s f ( x +t), then (0, g) E A^ for each y E B(R) that iszero almost everywherewith respect to Lebesguemeasure.5.1 Proposition Let L be as above, and let {T(r)}be a measurable contrac-tion semigroup on L.Then the full generator A^ of {T(t))is linear and dissi-pative and satisfiesfor all h E W(A-A)and A > 0. IfT(s) e-"T(t)h dt = I"e-"T(s +t)h dt0(5.7)for all h E L, 1 > 0, and s 2 0, then 5#(1 - 2)= L for every 1 > 0.Proof. Let V; g) E A,A=- 0, and h = y- g. Then(5.8) lme-"T(r)hdr = A dpe-"T(r)fdt - e-"T(t)g dr= 1 r e-"T(t)fdt -1 e-" T(s)gds dt=JConsequently, IlflI s A- 11 h 11, proving dissipativity,and (5.6)holds.g = 4.j- h. Then(5.9) T(s)gds = 1Assuming (5.7), let h E L and A >0, and define f- e-"T(t)hdt andlme-"T(s +u)h du ds - T(s)h ds= I en*ime-"T(u)hdu ds - T(s)hds= elSIle-"T(u)h du -1."e-AuT(u)hdu+ T(s)hds - T(s)hds= Wf-ffor all t 2 0,soU;g) E Aand h = Af-g E SI(A - A). 0
  • 34. 5. SEMKROUrJONFUNCllONWACES 25The following proposition,which is analogousto Proposition I.s(a), gives auseful description of someelementsof 2.5.2 Proposition Let L and (T(t))be as in Theorem 5.1, let h B t and u 2 0,and supposethat(5.10)forall I z 0.Then(5.1 1)T(t)lT(s)hds = 1T(t +s)h ds(lT(s)hds, T(u)h-h E A’.)p d . Put1=Zt;T(s)hds. Then= I”‘T(s)hds -1T(SPds=6‘T(s)(T(u)h-h)dsfor all r 2 0. 0In the present context,given a dissipative closed linear operator A c L x L,it may be possible to find measurable functions u: KO, a)-+L andu: [O, oo)+ tsuch that (u(t), u(t)) E A for every t >0 and(5.13) u(t) = u(0)+ 4s)ds, t ;I0.lOne would expect u to be continuous, and since A is closed and linear, it isreasonableto expect thatfor all t > 0. With these considerations in mind, we have the following multi-valued extension of Proposition 2.10. Note that this result is in fact valid forarbitraryL.
  • 35. 26 OIflAlOISEMKiROUrS5.3 Proposition Let A c L x L be a dissipative closed linear operator.Suppose u: [O, a)-,L is continuous and (sou(s) ds, u(t) -u(0))E A for eacht > 0.Then(5.15)for all t 2 0. Given I > 0, defineIIu(4II s II 40)II(5.16) l= e-&u(t) dt, g = 1 e-*"(u(t) -40))dr.Then cf,g) E A and y- g = u(0).Proof. Fix r 2 0,and for each E > 0, put u,(t) = ti-(5.17)Since (u,(r), & - I ( & +e) -~ ( 1 ) ) )E A, it follows as in Proposition 2.10 thatIIu,(t)II S llu8(0)ll.Letting&-+0, we obtain (5.15).(5.18) j = e-*qt) dt = 1 e-*l$ u(s) ds dt,so U;8) E A by the continuity of u and the fact that A is closed and linear.Theequation 1f-g = u(0)follows immediately from the definitionoffand g. 0u(s) ds. Thenu,(t) = ~(0)+ E-(u(s +E ) -u(s))ds.Integrating by parts,Heuristically,if {S(r)}has generator 8 and {T(t)}has generator A +B, then(cf. Lemma 6.2)(5.19)for all t 2 0. Consequently,a weak form of the equation u, = (A +B)uis(5.20)We extend Proposition 5.3 to this setting.T(t)f=S(t)f+ r S ( r -s)AT(s)/ds0u(t) = S(t)u(O) +5S(t -s)Au(s)ds.05.4 Proposition Let L be as in Proposition 5.1, let A c L x L be a dissi-pative closed linear operator, and let {S(t)}be a strongly continuous, measur-able, contraction semigroup on L. Suppose u: [O, 00)- L is continuous,u: LO, 00)- L is bounded and measurable,and
  • 36. 5. SEMICROWS ON FUNCnON SPAACES 27(5.21)for all r z 0. If(5.22)for every t > 0, and(5.23)for all q. r, r 2 0, then (5.15)holds for all I z 0.S(q +r)D(s) ds = S(q) S(r)o(s)dsc5.5 Remark The above result holds in an arbitrary Banach space under theassumption that u is strongly measurable, that is, u can be uniformly approx-0imated by measurable simple functions.Proof. Assume first that u: [O, m)-+ L is continuously differentiable,u: [O, a)--+L is continuous, and (u(t),41))E A for all t z 0. Let 0 = to < t , <(5.24)< t, = t. Then, as in the proof of Proposition 2.10,nIIu(t)I1= II 40)II + 1cI14tO I1 - II44- I ) Ill
  • 37. 28 O?ERATORJEMK;ROU‘Swhere s’= t,- I and s” = t, for r,- I ss < r,. Since the integrand on the right isbounded and tends to zero as max (t, -ti, 0, we obtain (5.15) in this case.In the generalcase, fix t 2 0, and for each E >0, putu(s) ds, u,(t) = e-I“’lsb+‘= & - I 1S(r +S)U(O) ds +& - I(5.25)Then(5.26) u,(t) = u(r +s) dsU#) = e-IS(t +s -r)dr)dr ds= ~ - l S ( t )(dS(s)u(O) ds +6- I s’5’S(t +s -r)u(r)dr ds0 0+ I’r S ( t -r)u(r+s) dr ds0 01= S(t)[.s-I S(s)u(O)ds +6 - l 5’I’S(s - r)u(r)dr ds0 0+ S(t -r)ua(r)dr.By the specialcase already treated,(5.27) II u,(t)I1S )Ie -and lettingE--, 0, we obtain (5.15) in general. 06. APPROXIMATION THEOREMSIn this section, we adopt the following conventions. For n = 1, 2, ...,L,,inaddition to L,is a Banach space (with norm also denoted by I[6 11) and n,:L+ L,,is a bounded linear transformation. We assume that sup,,IIn, II < 00.We writef.-+fiff. E t,,foreach n 2 1,Je L, and lirn,-= [If, - a, Ill = 0.6.1 Theorem For n = I, 2,. ,.,let (T,(t)) and { T(r))be strongly continuouscontraction semigroups on L, and L with generators A, and A. Let D be acore for A. Then the following are equivalent:(3intervals.For each1E L,T,(t)n,f-+ T(r)ffor all t 2 0, uniformly on bounded
  • 38. 6. APWOXIMATION THEOREMS 29(b)(c)For eachf E L, T,(l)n,J+ T(t)ffor all t 2 0.For each f~ D, there exists 1,E Q(A,) for each n 2 I such thatj,,-.Jand A,f,--+ Af(i.e., {(J AS):/€ D ) c e~-Iim,,+~A,,).The proof of this result depends on the following two lemmas, the first ofwhich generalizesLemma Lemma Fix a positive integer n. Let {S,(r)} and {S(t)} be stronglycontin-uous contraction semigroups on L, and L with generators B,, and B. Let/E 9(B)and assume that n,,S(s)j~g(B,,) for all s 2 0 and that B,n,S( * )j:[O, 00) -+L,, is continuous. Then, for each t 2 0,(6.1)and therefore(6.2)S,(t)n, f - n,,S(f)j= S,,(C- sWB, n,,- n, B)S(s)fds,LIISn(t)n,f - n, S(tV It 5 II(B,n n - n, B)s(s)/II ds.Plod. It suffices to note that the integrand in (6.1) is -(d/ds)S,(t - s)n,S(s)/for 0 s s ,< t. 06.3 Lemma Suppose that the hypotheses of Theorem 6.1 are satisfiedtogether with condition (c) of that theorem. For n = 1, 2,. ..and R > 0, let Atand A be the Yosida approximations of A, and A (cf. Lemma 2.4). ThenA: n, f-+Ayfor everyfe L and R > 0.Proof. Fix R > 0. Let /E D and g =(A - A)f By assumption, there exists1;E B(A,)for each n 2 I such that /;--+fand Ad,-+AJ and therefore (A - A,)S,-+g. Now observethat(6.3) I1A:nng- nnA"gl1= II[AZ(R - AJ-1 - Rf]n,g -n,[RZ(R - A)- - Af-JgII= A2(1(R - An)- ring - nn(A - A)-eIts R211(R - A n F 1ring -Lit + RItSn - nn(R - A)-gII5 LIInng - ( A - An)/nII + nZII/n - nSIIfor every n 2 I. Consequently, 11A: n,g - R, Ag II -+ 0 for all g E - At,,).But &(A - AID) is dense in L and the linear transformations Ain,, - n,AL,n = I, 2,. ..,are uniformly bounded,so the conclusion of the lemma follows.0Proof of Theorem 6.1. (a *b) Immediate.
  • 39. 30 OPERATOR SEMICROWS(b =5 c) Let 1 > 0.fE 4W), and g = (A - A)A so that f= e-"T(t)gdt. For each n 2 1, put fn = jz e-"X(t)n,,g dr E B(A,). By (b) and thedominated convergence theorem,S,-.l; so since (A - An)f, = n,g-+ g = (A-A)J we also have A,,&-, A/:(c =.a) For n = 1, 2,. ..and A >0, let {Ti(t)}and {T(r))be the strong-ly continuouscontraction semigroupson t,and L generated by the YosidaapproximationsA: and A. Given/€ D, choose {jJas in (c).Then(6.4) T,(l)nn f- nm T(tlf= UtKnn f-L) + CUt)f,- T$l)LI+ Ti(tMS,.-n, n+"CWnf - n, T A W ]+ nnCT?t).f- T(l)fJforevery n 2 I and t 2 0. Fix to 2 0. By Proposition 2.7 and Lemma 6.3,lim SUP 11 X(t).t, - T,"(t)LII5 lim to 11An S,- Aijn 11n- w 0 sI sfo n-m(6.5)lim to{ IIAn S. - nn MII + IInn(AS- AWIIn - m+ IInnAY- AfnnfII + I I A ~ ~ ~ . ~ - L ) I I Is K~oIlAf- AYII,where A= sup,((It,((.Using Lemmas 6.2, 6.3, and the dominated con-vergence theorem,we obtain(6.6) lim sup 11 T;(t)n,f -n, Ta(r)fIIn-m OLILIOs lim II(R."n. - n,A")T"s)Jl/ ds = 0.n-mApplying(6.5), (6.6). and Proposition 2.7 to (6.4), we find that(6.7) SUP I1T,(t)nnf -n, T(t)fll S 2Kr011A!f- AfII.I - C O O s r s t oSince I was arbitrary, Lemma 2.4(c) shows that the left side of (6.7) is zero.But this is valid for allfe D,and sinceD is dense in L, it holds for allJe L.0There is a discrete-parameter analogue of Theorem 6.1, the proof of whichdependson the followinglemma.6.4 lemma Let B be a linear contraction on L.Then(6.8) IIBY- en(8-Yll 5 J;;IIBJ-JIIfor allfs L and n = 0, 1,. ...
  • 40. 6. APFUOXIMATION THEOREMS 31Proof. Fix/€ L and n 2 0. Fork = 0, I,. ..,(6.9)Therefore(6.10)(Note that the last equality follows from the fact that a Poisson random0variable with parameter n has mean n and variance n.)6.5 Theorem For n = I, 2,. ..,let T,, be a linear contraction on L,, let E, bea positive number, and put A, = E;(T,, - I). Assume that Iim,,,&, = 0. Let{ T(t)}be a strongly continuous contraction semigroup on L with generator A,and let D be a core for A. Then the following are equivalent:(a)intervals.(b)(c)For each/€ L, T!,!Cnln,/-tT(t)ffor all t 2 0, uniformly on boundedFor each/€ L, T!,!%,, f- T(t)/for all t 2 0.For each / E D,there exists S. E L, for each n 2 I such that h4/and Anf,-+ AJ(i.e., ((JA ~ ) : / ED}c ex-limn.,, A,).Proof. (a b) Immediate.(b 3C ) Let A > 0,/ E B(A),and g = (A - AM; so that f = jg e-"f(t)edt. For each n 2 I, put(6.1I )
  • 41. 32 OPERATORSMCROUISBy (b) and the dominated convergence theorem,L-+J and a simple calcu-lation shows that(6.12) (1-AalL = nag -trlE,naga3+ - 1 +e-Aca) e-A*cnT~+n,gk = Ofor every n 2 1, so (A -A,).& -,g =(A -A ) j It followsthat A,,S,-+Af:(6.13) T!IbJn,J- n, T(r)f(c*a) Givenfe 0,choose {fa} as in (c). Thenand by Theorem6.1,(6.15)Consequently,(6.16) lim sup 11 T~laln,J-n, T(r)f11= 0.But this is valid for allfE D, and sinceD is dense in L, it holds for allfE L.lim sup I(exp {&a[ i ] ~ a } n a1-na VIUII =0.a-m OSCSIOn-m 05151006.6 Corollary Let {V(t):f 2 0) be a family of linear contractions onL with V(0)= I, and let {T(r)} be a strongly continuous contractionsemigroup on L with generator A. Let D be a core for A. If lims40~ - * [ V ( & ) f - f j= Affor every/€ D, then, for eachfe L, V(r/n)y-+ T(t)ffor allr r:0, uniformly on bounded intervals.Proof. It sunices to show that if {tn) is a sequence of positive numbers suchthat in-* r 2 0, then V(t,,/n)"+ T(t)ffor everyfe t.But this is an immediateconsequenceof Theorem 6.5 with T.= V(tJn) and E, = tJn for each n 2 I. 0
  • 42. 6. APPROXlMATltM THEOREMS 336.7 Codary Let {T(t)), (S(t)}, and (V(r)} be strongly continuous contrac-tion semigroups on L with generators A, B, and C,respectively. Let D be acore for A, and assume that D c 9(B) n 9(C)and that A = B + C on D.Then, for each/ E L.(6.I 7)for all r 2 0, uniformly on bounded intervals. Alternatively, if (E,} is asequenceof positive numbers tending to zero, then, foreach/€ L,(6.18)for all t 2 0, uniformly on bounded intervals.Proof. The first result follows easily from Corollary 6.6 with V(t)IS(c)U(t)0for all t 2 0. The second followsdirectly from Theorem Corollary Let (T(t)}be a stronglycontinuouscontraction semigrouponL with generator A. Then, for each / E L,(I -(r/n)A)-"J- T(t)ffor all I 2 0,uniformly on bounded intervals. Alternatively,if {en} is a sequence of positivenumbers tending to zero, then, for each f e t,(I -E,,A)-~"~Y--+T(t)Jfor allt ;r 0, uniformly on bounded intervals.Proof. The first result is a consequence of Corollary 6.6. Simply takeV(i)= (I - tA)- for each f 2 0, and note that if E > 0 and 1 = E - , thenwhere AI is the Yosida approximation of A (cf. Lemma 2.4). The second result0follows from (6.19) and Theorem 6.5.We would now like to generalizeTheorem 6.1 in two ways. First, we wouldlike to be able to use some extension A, of the generator A, in verifying theconditions for convergence. That is, given U;g) E A, it may be possible to findu,,g,) E A, for each n 2 1 such that /.-/ and g,+ g when it is not possible(or at least more diflicult) to find u,,g,) E A, for each n 2 1. Second, wewould like to consider notions of convergence other than norm convergence.For example, convergence of bounded sequences of functions pointwise oruniformly on compact sets may be more appropriate than uniform con-vergencefor some applications.An analogous generalization of Theorem 6.5 isalso given.
  • 43. 34 N TORS EM CROUPSLet LIM denote a notion of convergence of certain sequences f,E L,,n = 1,2,...,to elementsf€ L satisfying the followingconditions:(6.20) LIMf, =f and LIM g, =g implyLIM (aJ;+Pg,) = cf+ /?g for all a, /3 E R.(6.21) LIMf:) = f k ) for each k 2 1 andlim sup ll/!hJ -J, 11 V llj4kJ-/[I = 0 imply LIMA, =/:There exists K >0 such that for eachfe L,there is asequenceA, E L, with Ilf.11 s KIIfII, n = 1, 2,.. .,satisfyingLIML =f.h-m r Z 1 ,(6.22)If A, c L, x L, is linear for each n 2 1, then, by analogy with (4.3).we define(6.23) ex-LIM A, = (U;g) E L x L:there exists ( f . ,8,) E A,for each n 2 1 such that LIMA, =/and LIM g, = g}.6.9 Theorem For n = 1, 2,. .., let A, c L, x L, and A c L x L be linearand dissipative with 9 ( A - A,) = L, and 9 ( A -A) = L for some (hence all)A > 0, and let {T,(r)} and {T(t)) be the-corresponding strongly continuouscontraction semigroups on 9(A,) and 9(A). Let LIM satisfy (6.20H6.22)together with(6.24) LIMf, = 0implies LIM (A -A,)-% = 0 for all 1>0.(1) If A c ex-LIM A,, then, for each U;g) E A, there exists u,,9,) E A,for each n z 1 such that sup, /If. 11 < 00, sup, IIg, II< 00, LIM J, =f,LIM 8,= g, and LIM T,(t)J, = T(r)ffor all t 2 0.(b) If in addition {x(r)}extends to a contraction semigroup (alsodenoted by {x(t)})on L, for each n 2 1, and if(6.25) LIMA = 0implies LIM T,(r)f. = 0 for all t 2 0,then, for eachfe B(A),LIMJ;=/implies LIM x(t)f. = T(t)/for all t 2 0.-6.10 Remark Under the hypotheses of the theorem, ex-LIM A, is closedin L x L (Problem 16). Consequently, the conclusion of (a) is valid for allUI Q)E A’. 0Proof. By renorming L,, n = 1, 2,...,if necessary, we can assume K = 1 in(6.22).Let 2’denote the Banach spa& (naLILJx L with norm given byI I ( { L J s f)II= SUPnz1111; IIV IIf II, and let(6.26)
  • 44. 6. APFROXlMATlON THKMFMS 35Conditions (6.20)and (6.21) imply that Yois a closed subspacc of 9,andCondition (6.22) (with K = 1) implies that, for each/€ L, there is an element( { f n } , / ) 6 9 0 with II({fn}*AII= IIJll.Let(6.27) d = {[({fn}*jh ({gn}. 911 E 9 X An for eachn 2 1 and U;g)E A}.Then Iis linear and dissipative, and @(A - .d)= Y for all 1 > 0. The corre-sponding strongly continuous semigroup {.T(f)} on 9(d)is given by-(6.28)We would like to show that(6.29)To do so, we need the following observation. If V; g) E A, 1 > 0, h = AJ- g,((hn), h) E Y o . and(6.30) (f"* 9,) = ((A - A n ) *k9 - h n )for each n z I, thenTo prove this, since A c ex-LIM A,,, choose c/"., 8,) E A, for each n 2 1 suchthat LIM3, =f and LIM 3, = g. Then LIM (h, -(ly",- 8,))= 0, so by (6.24),LIM (1- A,)-h, -f, = 0. It follows that LIMf, = LIM (A - A,,)-*h, =LIMA =f and LIM g,, = LIM (@, -h,) = V-h = g. Also, sup, IIj, II s1-I SUP, IIh, 11 < 00 and SUP. IIgn II 5 2 SUP, IIh n 11 -= 00. Consequently, [({h),n,((9,). g)] belongs to 9,x Y o ,and it clearly also belongs to d .Given ({h,},h) E Y oand rl > 0, there exists c(,g) E A such that ly- g = h.Define u,,g,) E A, for each n z 1 by (6.30). Then (A - d)-({h,,},h) =( { f n } , J ) E 90by (6.31)v SO(6.32) (1- d ) - :9 0 3 Y o , L > 0.By Corollary 2.8, this proves (6.29).To prove (a), let (1g) E A, A > 0, and h = Af- g. By (6.22). there exists({h,}, h) E Y owith ll({h,,}, h)II = IIh 11. Define (h,g,) E A, for each n 2 1 by(6.30). By (6.31). (6.29), and (6.28), ({T,,(t)f,,}, T(t)f) E Y ofor all t 2 0, so theconclusion of (a)is satisfied.As for (b), observe that, by (a) together withI_(6.25), LIML =fB B(A)implies LIM T(t)/,-- T(t)ffor all t 2 0. Letfs d(A)and choose {$&I} c B(A)such that II/I -/[I s 2-& for each k 2 1. Put Po = 0, and by (6.22), choose
  • 45. for each k 2 1. Since(6.34)andfor each n 2 1 and k 2 1, (6.21) implies that(6.36)Q, mLIM 1u!~==A LZM T,(t)Cut)= T(t)J;I Iso the conclusionof (b)followsfrom (6.25). 06.11 Theorem For n = 1, 2,..., let T, be a linear contraction on L,, letE, > 0, and put A, = &;(T, -I). Assume that limn-mc,, = 0. Let A c L x Lbc linear and dissipative with 9?(1 -A) = L for some (henceall) 1 > 0, and letIT(t)} be the corresponding strongly continuous contraction semigroup on9(A).Let LIM satisfy(6.20)-(6.22),(6.24),and(6.37) lim JjhII =0 implies LIM 2= 0.W If A c ex-LIM A,, then, for each U;g) E A, there exists f,,E L,for each n 2 1 such that sup,Ilf,jl < 00, sup,)IA,J,(I < 00, LIMA -I;LIM AJ, =g, and LIM ch!&= T(r)/for all r z 0.(6.38) LIMJ, = 0 implies LIM T!/-y, = 0 for all t 2 0,then for eachftz 9(A),LIMA Efimplies LIM c/"!f,,= T(t)ffor all r 2 0.(bJ If in addition-Proof. Let U;g) E A. By Theorem 6.9, there cxistsI; E L, for each n L 1 suchthat SUp,!lfn I1 < a, sup,II Af,Il < 00, LIMf, -S, LIM AS, = g, andLJM e"X = T(r)Jfor all t 2 0.Since(6.39)
  • 46. 7. NRTUROATION THEOREMS 37for all t 2 0, we deduce from (6.37) that(6.40)The conclusion of(a)therefore follows from (6.14)and (6.37).The proof of (b)is completelyanalogousto that of Theorem 6.9(b). 07. PERTURBATION THEOREMSOne of the main results of this section concerns the approximation of semi-groups with generators of the form A + B,where A and B themselves generatesemigroups.(By definition, O(A+ B)= O(A)n 9(B).)First, however, we givesome suflicient conditions for A + B to generate a semigroup.7.1 Theorem Let A be a linear operator on L such that A’ is single-valuedand generates a strongly continuous contraction semigroup on L.Let B be adissipative linear operator on L such that 9 ( B )3 9(A). (In particular, 6 issingle-valuedby Lemma 4.2.) Ifwhere 0 5 a c I and /I2 0, then A + B is single-valued and generates astrongly continuouscontraction semigroup on L. Moreover, A + B = A + 8.Proof. Let y 2 0 be arbitrary. Clearly, 9 ( A +yB)= 9 ( A ) is dense in L. Inaddition, A + yB is dissipative. To see this, let A be the Yosida approx-imation of A’ for each p > 0, so that A, = p[p(p - .$)-I -11. If/€ d ( A )andA > 0. thenby Lemma 24c)and the dissipativity of yB.
  • 47. I f j e 9(A),then there exists (f.} c 9 ( A ) such thatf.+/and AS,-+ 26 BY(7.1), {Bf;)is Cauchy, s o f ~9(B)and BS,+ BJ Hence 9(J)t 9(B)and (7.1)extends to(7.3)In addition,if/€ 9(A)and if (I,)is as above, then(7.4)implying that A -t- yB is a dissipativeextension of A +ys.(7.5) T = { y 2 0: 4?(6 - A -yb)= L for some (henceall) 6 > 0).To complete the proof, it sufficesby Theorem 2.6 and Proposition 4.1 to showthat 1 E r.Noting that 0 E r by assumption,it is enough to show that(A+yg)f= lim A& +y lim Bf. = lim ( A +yB)/,= (A +yE)Ja a aLet1 - ayy E r n Lo, 1) implies [y, y -+ 7 )c rTo prove (7.6), let y E r n [O, I), 0 5 E < (2a)-(l -ay), and L > 0. Ifg E B(A),then/= (I- A -y@- g satisfies(7.7)by (7.3), that is,(7.8)and consequently,(7.9) IIB(L-A-;.B)-gli ~ [ 2 a ( l-q)-+/?(~- a y ) - ~ - ] l l g l l .Thus, for I suficiently large, IIE&(A -A -B)-II < 1, which implies ,thatI -11lgsrr 5: all 4.31+811f11 dl(A+rb)fll +aril mr +PlifliIlj3Jll 5 -aY)-JJ(A +ytr>/n +P(1-aY)-llJIl,- A - yb)- is invertible.We concludethat(7.10) B(6-A -(y -k e)B) 3 .@((A - A -(y 4- 6)&1 - A - yB)-)=@(I - &&I -A- yB)-)= Lforsuch 6, so y +E E r,implying(7.6) and completingthe proof. 07.2 Corollary If A generates a strongly continuous contraction semigroupon L and E is a bounded linear operator on L,then A + B generates astronglycontinuoussemigroup {T(t))on L such that(7.1 I) 11 T(r)i)5 e"", r 2 0.Proof. Apply Theorem 7.1 with B - [IB 11I in place of B. El
  • 48. Before turning to limit theorems, we state the following lemma, the proof ofwhich is left to the reader (Problem 18). For an operator A, letM ( A ) 5 {fe.$@(A):Af = 0) denote the null space of A.7.3 Lemma Let B generate a strongly continuous contraction semigroup{S(t))on L, and assume that(7.12) tim A e-"S(r)(dr = Pf exists for all (e L.Then the following conclusions hold :a-o+(a) P is a linear contraction on L and P2= P.(b) S(r)P = PS(r)= P for all t 2 0.(c) @P) = XCB).-(d) N(P) = W(E).7.4 Remark If in the lemma(7.13) B = y - ( Q - I),where Q is a linear contraction on L and y > 0, then a simple calculationshows that (7.12) is equivalent tom(7.14) lim (I - p) 1 pkQL/=Pf exists for all /E L. 0p - l - k = O7.5 Remarkholds andIf in the lemma lim,+mS(r)( exists for every /E L, then (7.12)(7.15) Pf = lim S(i)J / E L.t-mIf E is as in Remark 7.4 and if limk-mQYexists for every (E L. then (7.14)holds (in fact,so does (7.15)) and(7.16) Pf= lim Q? (E L.k-mThe proofs of these assertionsare elementary. 0For the following result, recall the notation introduced in the first para-graph of Section 6,as well as the notion of the extended limit of a sequenceofoperators (Section4).7.6 Theonm Let A c L x L be linear, and let B generate a strongly contin-uous contraction semigroup {S(t)}on L satisfying(7.12). Let D be a subspace
  • 49. 40 OPERATORS€MIGROWSof 9 ( A )and D a corefor B. For n = 1,2,. ..,let A, be a linear operator on L,and let a, > 0. Supposethat limn,man= 00 and that(7.17) {U;g) E A:fE D}c ex-lim A,,n - e(7.18) ((h,Bh): h E D) t ex-Jim a;A,.Define C = (U;fg):U;g) E A, f~ D} and assume that {(Ag) E c:g E 0)issingle-valued and generates a strongly continuous contraction semigroup{~ ( c ) } on 6.n-oD(a) If A, is the generator of a strongly continuous contraction semi-group {F(t)}on Lafor each n 2 1, then, for eachfe 6,x(t)nJ--r T(t)fforall 2 0, uniformly on bounded intervals.(b) If A, = E,-I(T, -I) for each n 2 1, where T. is a linear contractionon L, and E, >0, and if lim,,,~, =0, then, for eachfE D, T!%, f-. T(f)ffor all f 2 0, uniformly on bounded intervals.Proof. Theorems6.1 and 6.5 are applicable,provided we can show that(U;g) E C:g E 6)c ex-Jim A, n (b x 6).(7.19) ( n - r n )Since ex-lim,,, A, is closed, it sufficesto show that C c ex-limn,, A,. GivenU;g) B A with ftz D, choosef . E 9(An) for each n 2 1 such that fa- f andA,f,-, g. Given h E D, choose h, E B(A,) for each n 2 I such that h,+ h anda,- A, h, +Bh. Then f . +a, h, -+f and A,cf, +a; h,)3 g +Bh. Conse-quently,(7.20) {U;g +Bh):U;g) E A, f E D, h E D} c ex-lim A,.But sinceex-limn,, A, is closed and since, by Lemma 7.3(d),(7.21)for all g E L, we conclude that(7.22)1-4)7 -Pg - g E M ( P )= 9 ( B )= 9t(B(n*){U;Pg):V;g) E A, f e D) c ex-lim A,,n-mcompletingthe proof. 0We conclude this section with two corollaries. The first one extends theconclusions of Theorem 7.6, and the other describesan important special caseof the theorem.7.7 Corollary Assume the hypotheses of Theorem and suppose that(7.15) holds. If h E M(P) and if {t,} c 10, GO) satisfies tima,, t.u, = 00,
  • 50. 7. PERTUIIATION THEOREMS 41then T,,(r,)n,h-+ 0. Consequently, for each f E P-(6) and 6 E (0, I),%(r)n,f-+ T(r)P/;uniformly in b s t g 6-.Assume the hypotheses of Theorem 7.6(b), and suppose that either(i) lim,,,a,q, = 0 and (7.15) holds, or (ii) lim,,-.,a,,c, = y > 0 and (7.16)holds (where Q is as in (7.13)).If h E N(P)and if {&,) c (0, 1,. ..} satisfiesk,a, E, = m, then TFn,h -+ 0. Consequently, for eachf E P - (6)and6 E (0, I), T!"%J-, T(~)PJuniformly in b s r 5 6 - .Proof. We give the proof assuming the hypotheses of Theorem 7.6(a), theother case being similar. Let b E J(r(P),let (t,} be as above, and let E > 0.Choose s 2 0 such that II S(s)hII 5 c/2K, where K = supnrI 11n, 11, and let s, =sAr,a, for each n 2 I. Thenfor all n suficiently large by (7.18) and Theorem 6.1. If J E L, thenf - Pf E .N(P), so 7Jrn)n,(J- Pf)+ 0 whenever {t,} c LO, 00) satisfiest, = r # 0. If f e P-(d), this, together with the conclusion of the theorem0applied to PJ completesthe proof.7.8 Corollary Let ll,A, and B be linear operators on L such that B generatesa strongly continuous contraction semigroup {S(r))on L satisfying (7.12).Assume that 9(n)n 9 ( A ) n 9 ( E )is a core for B. For each a sufkiently large,suppose that an extension of ll +aA +aE generates a strongly continuouscontraction semigroup { T,(r)} on L. Let D be a subspaceof(7.24) (/E 9(n)n 9 ( A ) n .N(B):there exists h E Q(n)n 9 ( A ) n 9(B) with Bh = - A / } ,and defineThen C is dissipative, and if ((J8) E c:g E 0).which is therefore single-valued, generates a strongly continuous contraction semigroup (T(r))on 6,then, for eachJE D, lima+,., x(r)/=T(r)/for all t 2 0, uniformly on boundedintervals.Proof.limn+ma, = GO, and apply Theorem with L, = L, n, = I, A replaced by(7.26) (U;n/+ A h ) : / € D, h E 9(n)n 9 ( A )n 9(B), Bh = -A!},A, equal to the generator of {T*(r)},a, replaced by af. andD = WJ)n 9 ( A ) n 9(B).Since A,,cf+ a;h) = nf+Ah +a i l l h when-ever/€ D, h E 9(n)n 9 ( A ) n 9(B),Bh = -AS, and n 2 1, and since limn--Let {a,} be a sequence of (sufficientlylarge)positive numbers such that
  • 51. 42 OIEIATORSMGROUISa,-2A,h = Bh for all h E D, we find that (7.17) and (7.18) hold, so the theoremis applicable. The dissipativity of C followsfrom the dissipativity of ex-lim,,,A". 07.9 Remark (a) Observe that in Corollary 7.8 it is necessary that PAf= 0(b) Let /E 9 ( A ) satisfy PAf= 0. To actually solve the equationfor allfE D by Lemma 7.3(d).Bh = -Affor h, supposethat(7.27) II(s(t)- p)g11 dt < 00, g E L.Then h -" limA-o+(A - B)-Af= j; (S(t) - P)A/dt belongs to 9(B)(sinceB is closed) and satisfies Bh = -A$ Of course, the requirement that hbelong to 9(n)A 9 ( A ) must also be satisfied.(c) When applying Corollary 7.8, it is not necessary to determine Cexplicitly. instead, suppose a linear operator Co on b can be found suchthat Cogenerates a strongly continuous contraction semigroup on b andCoc C.Then {V; g) E (f:g E b} = Coby Proposition 4.1.(d) See Problem 20 for a generalization and Problem 22 for a closelyrelated result. {T(r)}on &R) by T(t)J(x)=/(x +I). Show that {T(t)}is a strong-ly continuouscontractionsemigroupon t,and determine its generator A.(In particular,this requires that 9 ( A )be characterized.)Define {T(r)}on c(R) byfor each r >0 and T(0)= I. Show that {T(t)}is a strongly continuouscontraction semigroupon L,and determineits generator A.Prove Lemma 1.4.Let (T(r)}be a strongly continuous contraction semigroup on L withgenerator A, and let/€ 9(A2).(a) Prove thatJo
  • 52. a m o m 43(b) Show that IIASII 5 411AJII 11/11.Let A generate a strongly continuous semigroup on L. Show that fl.i I9 ( A " )is dense in L.Show directly that the linear operator A = fd2/dxzon L satisfies condi-tions (a)-@)of Theorem 2.6 when 9 ( A )and L are as follows:(a) g ( ~ )= { f ~C2[0,11:a,f"(i)-(- l)&f(i) = 0, i= 0, I}.L = CCO, 11. ao.Po. a I ,PI 2 0, a. I-Po 7 0, al i-PI > 0.(b) L@(A)-= {fe CCO, 00):ao/"(0)- Bof(O) = 0)L = CCO, 001, ao,Po 2 0, a. +Do > 0.(c)Hint: Look .for solutions of A ,-4/"= g of the form f ( x )=exp { -a x } & ) .Show that CF(R) is a core for the generators of the semigroups of Prob-lems 1 and 2.In this problem, every statement involving k, I, or n is assumed to holdfor all k,I, n 2 a sequence of closed subspaces of L. Let0,. M,,and MP be bounded linear operators on L. Assume that u, andMp)map L,into L, ,and that for some fl, >0, IIMPII <fi, and9 ( A ) -- C,(Pa), L = Qua).Let L, c L, c L, c * I(Mf""- M,1) = 0.r))" mSuppose that the restriction of A, Ithat there exist nonnegativeconstants dlk( (= a(,), & I , and y such that(8.4)Mf"Uj to L, is dissipative andf E t,II u h U J - UI UJll s ad11 UJll + IIUJII),(8.7)Define A =(8.8)If 9 ( A ) is dense in L, show that A is single-valued and generates astronglycontinuouscontractionsemigroupon L.I Mj[I,on1OD W~ ( A I= {I. u Ln: 1 fijllujflI < 00 .n = l j = J
  • 53. 4 (wMToII6McROWSHint: Fix A > 3y and apply Lemma 3.6.Show first that for g E 9 ( A ) andf n * (a - AA-’g,n1-1(8.9) (a-Y)IlUd~ll IIuhgll + (fikJ+r,akj)lluj/;ll.Denoting by p the positive measure on the set of positive integers thatgives mass P h to k,observe that the formula(8.10)definesa positive bounded linear operatoron L’(p)of norm at most 27.9. As an application of Corollary 3.8, prove the following result, whichyields the conclusionof Theorem 7.1under a different set of hypotheses.Let A and E generate strongly continuous contraction semigroups{T(r))and {S(t)}on L. Let D be a dense subspace of L and 111 * 111 a normon D with respect to which D is a Banach space. Assume that 111fIII 2 11/11for allfc D. Supposethere existsp 2 0 such that(8.11) D =W2);IIA’Ill S rlllflll, fQ D;(8.12)(8.13) T(t):D-, 0, S(t): D-, D, t 2 0;(8.14) 111W )111 s e’, 111S(0 111 s e”’, 2 0.Then the closure of the restriction of A +B to D is single-valued andgeneratesa strongly continuouscontractionsemigroupon L.We remark that only one of the two conditions (8.11) and (8.12) isreally needed.See Ethier (1976).10. Define the bounded linear operator E on L =C([O, 13 x [O, 11) byBf(x,y) =(8.15)f(x, z) dz, and defineA c L x L byA = {Ut/,=+W:SEC2(C0,13 x CO, 11)n W ? AfA0, Y) =f3, y) =0 for all y E LO, 11,h E Jlr(B)).Show that A satisfiesthe conditionsof Theorem 4.3.11. Show that ex-lim,,, A,, defined by (4.3X is closedin L x L.12. Does the dissipativity of A, for each n 2 1 imply the dissipativity ofex-lim,,, A,?13. In Theorem 6.1 (and Theorem 6.5).show that (a)-+) are equivalent to thefollowing:
  • 54. a. raocmts 45(d) There exists 1 > 0 such that (A - A,)-’n,,g+(1 - A ) - ’ g for allg E L.14. Let L, {L,,},and In,) be as in Section 6. For each n 2 1, let {T,(t)) be acontraction semigroup on L,, or, for each n 1 I, let (T,(r)} be defined interms of a linear contraction T, on L, and a number E, > 0 by 7Jr) =E, = 0. Let { T(t)}be a contraction semigroup on L, let J g E L, and suppose that lim,4mT(t)j= 8 andfor all t 2 0; in the latter case assume that(8.16) lim sup I[7Jr)nJ- n, T(r)jII = 0for every ro > 0. Show that(8.17)if and only if(8.18)IS. Using the results of Problem 2 and Theorem 6.5, prove the central limittheorem. That is, if X,,X,,... are independent, identically distributed,real-valued random variables with mean 0 and variance I, show thatn- c;=I X , converges in distribution to a standard normal randomvariable as n-+ 00. (Define TJ(x) = E u ( x +n-’”X,)] and c, = n-’,)Under the hypotheses of Theorem 6.9, show that ex-LIM A,, is closed inL x L.17. Show that (6.21) implies(6.37) under the following(very reasonable)addi-tional assumption.(8.19) If j,E L, for each n 2 1 and if, for some no 2 1,j,= 0for all n 2 no, then LIMS, = 0.Prove Lemma 7.3 and the remarks followingit.Under the assumptionsof Corollary 6.7, prove (6.18) using Theorem 7.6.Hinr: For each n 2 I, define the contraction operator T,on L x L byn-. w 0 SI 610lim sup 11 T,(t)nJ- nnT(t)fll= 0n-w 120lim sup IIT,(r)n,g - n, T(r)g)I = 0.n-m t a O16.18.19.(8.20)20. Corollary 7.8 has been called a second-order limit theorem. Prove thefollowingkth-order limit theorem as an application of Theorem 7.6.Let A,,, A , , ...,A, be linear operators on L such that A, generates astrongly continuous contraction semigroup {S(c)} on L satisfying (7.12).Assume that 5% = n $ - 0 9 ( A , ) is a core for A,. For each a suficiently
  • 55. 16 OPEUTORS€MICIOUPSlarge, suppose that an extension of Cf=oajAjgeneratesa strongly contin-uous contraction semigroup { 7Jf)) on L.Let D be a subspaceof(8.21) {fo E 9:there exist fl,fz,.. ., f , - l E .9 withIm110AL-m+j/;=O for m = O , . .. ,k- 1 ,and defineIk - Il = O(8.22) C = {(fo, PAj&): fo E D,f,,...,&-, as above .Then C is dissipative and if {U;g) E c:g E 61,which is therefore single-valued, generates a strongly continuouscontraction semigroup {T(r)}on6, then, for eachfE 6,lima-,,,, lf&)f=T(t)f for all t 2 0, uniformly onbounded intervals.21. Prove the followinggeneralization of Theorem 7.6.Let M be a closed subspace of L,let A t L x L be linear, and let B,and B, generate strongly continuous contraction semigroups (S,(t)} and{S,(r)} on M and L,respectively,satisfying(8.23) lim R 1e-A"S,(t)fdr = P,f exists for all ffs M,(8.24) lim R e-"S,(f)fdt-= P,f exists for all f E L.Assume that @P,) c M.Let D be a subspace of 9(A), D,a core for B,,and D,a core for B,. For n = 1. 2,. ..,let A, be a linear operator on L,and let a,, /In> 0. Suppose that lim,-ma, = 00,(8.25) (U;g) E A : ~ ED}c ex-lim A,,(8.26) {(h, B,h): h c D,}c ex-lim a;A,,(8.27) {(k,B, k): k E D2}c ex-lim A,.Define C = {U;P I P ,9): (Jg) E A,fe D} and assume that {Ug) e c:g Eb} generates a strongly continuous contraction semigroup {~ ( t ) fon D.Then conclusions(a)and (b)of Theorem 7.6 hold.22. Prove the followingmodificationof Corollary 7.8.Let n, A. and B be linear operators 0 1 1 L such that 8 generates astrongiy continuous contraction semigroup {S(C)) on L satisfying(7.12).Assume that 9(n)n D(A) n B(B) is a core for B. For each a sufkientlylarge, suppose that an extension of ll +aA +a2B generates a stronglyA-O+A-O+ c/I, = 00, andn-mn-mn-m
  • 56. 9. NOTES 47continuous contraction semigroup { T#)) on t.Let D be a subspace of9(n)n 9 ( A ) n N(B)with m P ) c 6, and define C = {(JPA/):/E D}.Then C is dissipative. Suppose that c generates a strongly continuouscontraction semigroup{ V(r)} on D,and thatm(8.28) lim L [ e-"U(r)fdt = P,f exists for every f e 6.A - O + JOLet Do be a subspace of {/E D:there exists h E 9(n)n 9 ( A ) n 9 ( B )with Bh = - A t } , and define(8.29) Co = {(J P o P n f + P , P A h ) : / € Do,h E 9(n)n 9 ( A ) n 9(B), Bh = -AS).Then C, is dissipative, and if {U;8) E co:g E a,} generates a stronglycontinuous contraction semigroup { T(r))on 6,. then, for each /E Do,Iirnadm T&)f= T(r)/for all t 2 0, uniformly on bounded intervals.23. Let A generate a strongly continuous semigroup {T(t)} on L, letB(t):L-4 L, t 2 0, be bounded linear operators such that (B(t)} isstronglycontinuousin t L 0 (i.e., t-+ B(r)fiscontinuousfor eachJE L).(a) Show that for each f~ L there exists a unique u: [O, o o ) ~Lsatisfying(8.30) ~ ( t )= T(t)f+ T(t - s)B(s)u(s)ds.(b) Show that if B(t)g is continuously differentiable in c for each g E L,andf E 9 ( A ) ,then the solution of (8.30)satisfies(8.31)a-u(t) = Au(r) + B(t)u(t).at9. NOTESAmong the best general references on operator semigroups are Hille andPhillips (1957),Dynkin (1965),Davies (1980),Yosida (1980).and Pazy (1983).Theorem 2.6 is due to Hille (1948)and Yosida (1948).To the best of our knowledge, Proposition 3.3 first appeared in a paper ofWatanabe (1968).Theorem 4.3 is the linear version of a theorem of Crandall and Liggett(1971). The concept of the extended limit is due to Sova (1967) and Kurtz(1969).Sufficient conditions for the convergence of semigroups in terms of con-vergence of their generators were first obtained by Neveu (1958). Skorohod(l958), and Trotter (1958).The necessary and suflicient conditionsof Theorems
  • 57. 48 OrUATORSMCIOUPS6.1 and 6.5 were found by Sova (1967) and Kurtz (1969). The proof given herefollows Goldstein (1976). Hasegawa (1964) and Kato (1966) found necessaryand sufficient conditions of a different sort. Lemma 6.4 and Corollary 6.6 aredue to Chernoff (1968). Corollary 6.7 is known as the Trotter (1959) productformula. Corollary 6.8 can be found in Hille (1948). Theorems 6.9 and 6.11were proved by Kurtz (1970a).Theorem 7.1 was obtained by Kato (1966) assuminga < and in general byGustafson (1966). Lemma 7.3 appears in Hille (1948). Theorem 7.6 is due toEthier and Nagylaki (1980) and Corollary 7.7 to Kurtz (1977). Corollary 7.8was proved by Kurtz (1973) and Kertz (1974); related results are given inDavies (1980).Problem 4(b) is due to Kallman and Rota (1970), Problem 8 to Liggett(1972), Problem 9 to Kurtz (see Ethier (1976)), Problem 13 to Kato (1966), andProblem 14 to Norman (1977). Problem 20 is closely related to a theorem ofKertz(1978).
  • 58. 2This chapter consists primarily of background material that is needed later.Section I defines various concepts in the theory of stochastic processes, inparticular the notion of a stopping time. Section 2 gives a basic introductionto martingale theory including the optional sampling theorem, and local mar-tingales are discussed in Section 3, in particular the existence of the quadraticvariation or square bracket process. Section 4 contains additional technicalmaterial on processes and conditional expectations, including a Fubinitheorem. The DoobMeyer decomposition theorem for submartingales isgiven in Section 5, and some of the special properties of square integrablemartingalesare noted in Section 6. The semigroupof conditioned shifts on thespace of progressiveprocesses is discussed in Section 7. The optional samplingtheorem for martingalesindexed by a metric lattice is given in Section 8.STOCHASTIC PROCESSESAND MARTINGALES1. STOCHASTIC PROCESSESA stochastic process X (or simply a process) with index set 1 and state space(E, a)(a measurable space) defined on a probability space (Cl, 9,P) is afunction defined on 1 x Q with values in E such that for each r E 1,X(t, .): R-+ E is an E-valued random variable, that is, {UJ: X(f, UJ) E r}E .Ffor every E a.We assume throughout that E is a metric space with metric r49Markov Processes Characterizationand ConvergenceEdited by STEWARTN. ETHIER and THOMASG.KURTZCopyright 01986,2005 by John Wiley & Sons,Inc
  • 59. 50 STOCHfiTIC PROCESS AND MARTINGALESand that 1is the Bore1 a-algebra B(E).As is usually done, we write X(t) andX(t,* ) interchangeably.In this chapter, with the exception of Section 8, we take N = [O, 00). We areprimarily interested in viewing X as a “random” function of time. Conse-quently, it is natural to put further restrictions on X. We say that X ismeasurable if X:[O, 00) x f2-t E is g[O, 00) x $-measurable. We say that Xis (almost surely) continuous (right continuous, lefz continuous) if for (almost)every o E R, X(., w) is continuous (right continuous, left continuous). Notethat the statements “ X is measurable” and “X is continuous” are not parallelin that “X is measurable” is stronger than the statement that X( .,w) ismeasurable for each o E R. The function X(-,a)is called the sample path ofthe process at w.A collection (S,}E {F,,t E LO, 00)) of 0-algebras of sets in F is a fir-tration if 9,c $,+, for t, s E [O, m). Intuitively 9,corresponds to the infor-mation known to an observer at time t. In particular, for a process X wedefine (4:) by 9;= a(X(s):s 5 c); that is, 9: is the information obtainedby observingX up to time t.We occasionally need additional structure on {9J.We say {S,}is rightcontinuous if for each t L 0, SI=,sit,. = r)a,04tlt,. Note the filtration{F,+}is always right continuous (Problem 7). We say (9,)is complete if(a,9,P)is completeand { A E 9: P(A) = 0)c So,A process X is adapted to a filtration {S,)(or simply {F,}-adapted)if X(r)is 6,-measurable for each t L 0. Since6,is increasing in I, X is {$,}-adaptedif and only if 9; c S,for each t 2 0.A process X is {.F,}-progressive (or simply progressive if (9,)= (9:))iffor each t 2 0 the restriction of X to [O,t] x R is &[O,t] x 9,-measurable.Note that if X is {4F,}-progressive,then X is (FJ-adapted and measurable,but the converse is not necessarily the case (see Section 4 however). However,every righf (left) continuous (9J-adapted process is {.F,}-progressive(Problem 1).There are a variety of notions of equivalence between two stochastic pro-cesses. For 0 s f , < t2 < * - * < f,, let p,,,....,-be the probability measure ong ( E ) x - .* x 9 ( E ) induced by the mapping (X(t,),...,X(c,))- Em,that is,p I , * . ..,, ~ r )= P{(X(t,),...,X(t,)) E r}, r E a ( E ) x - - x @(E). The prob-ability measures {p,,,.., , m 2 1, 0 5 t , < * e . < t,} are called the Jinite-dimensional distributions of X. If X and Y are stochastic processes with thesame finite-dimensionaldistributions, then we say Y is a version of X (and X isa version of Y).Note that X and Y need not be defined on the same probabil-ity space. If X and Y are defined on the same probability space and for eachc 2 0, P(X(t)= Y(t)}= 1, then we say Y is a modijication of X. (We areimplicitly assuming that (X(t), Y(t)) is an E x E-valued random variable,which is always the case if E is separable.) If Y is a modification of X,thenclearly Y is a version of A’. Finally if there exists N E 9 such that ON)= 0and X(-,w ) = Y(a , w ) for all w $ N, then we say X and Y are indistinguish-able. If X and Y are indistinguishable,then clearly Y is a modification of X.
  • 60. 1. STOCHASTIC m o m m SIA random variable T with values in [O, GO] is an {9,}-stopping time if{I s t } E 9,for every t 2 0. (Note that we allow I = 00.) If I < 00 as., we sayI isfinite as. If T s 7 < 00 for some constant T, we say T is bounded. In somesense a stopping time is a random time that is recognizable by an observerwhose informationat time t is 9,.If r is an {PI)-stoppingtime, then for s < r, {T s s} E 9,c 9,,{T < t } =U,(z I; I - l/n} E 9,and (I = t } = {I5 t } - (z< t } e 9,.If T is discrete(i.e.,if there exists a countableset D c [O, 003 such that {IE D)= a),then I isan (9,)-stoppingtime if and only if {I = t } E S,for each t E D n [O, m).1.1 Lemma A [O, 001-valued random variable T is an {Pl+)-stoppingtime ifand only if {I < t} E 9,for every t 2 0.Proof. If { t < t } e 9,for every t z 0, then {I < t +n - I } E St+,-,for n 2 mand { 7 <11 = on{?< t + n u ) E flm91+m-,= .(PI+. The necessity wasobserved above. 01.2 PropositionThen the following hold.Let t l rT ~ ,... be {SF,}-stopping times and let c E [O,oo).(a) rl +c and A c are {9,}-stoppingtimes.(b) sup, I, is an {.F,}-stoppingtime.(c) minks,. rkis an {9,}-stoppingtime for each n 2 1.(d) If (9,)is right continuous, then inf,r,, and I,-are {F,}-stoppingtimes.Proof. We prove (b) and (d) and leave (a) and (c) to the reader. Note that{sup,,I" s t } = on{z, s t } E: PI so (b) follows. Similarly {inf,,?, e t ) =U,{I, < I} E P I ,so if (9,)is right continuous, then inf,?, is a stopping timeby Lemma 1.1. Sinceiimn4rnT,, = ~up,,,inf,,~,,,~,and limn-* z, = inf,sup,,,r,,(d)follows. 0-By Proposition 1.2(a) every stopping time I can be approximated by asequenceof bounded stopping times, that is, limn-mT A n = I. This fact is veryuseful in proving theorems about stopping times. A second equally usefulapproximation is the approximation of arbitrary stopping times by a nonin-creasing sequenceof discrete stoppingtimes.1.3 Propositionand suppose thatand defineFor n = 1, 2,..., let 0 = r: < tl < * - * and limk-rntl: = 00,sup&+ - I;) = 0. Let I be an {F,+}-stoppingtime
  • 61. 52 STOCHASTIC PRCK€SSES AND MAWlNCALEsThen t, is an {S,}-stopping time and limndm7, = 7. If in addition {I:} t{t;"), then t, 2 tn+l.Recall the intuitive description of 9,as the information known to anobserver at time t. For an (9,)-stopping time 7, the a-algebra9,should havethe sameintuitivemeaning. For technical reasons S,is defined by(1.3)Similarly, PC+is defined by replacing 9,by 9,+.See Problem 6 for somemotivation as to why the definition is reasonable. Given an E-valued processX,define X(a0) c xo for some fixed xo E E.9,= { A E 9:A n ( 7 s t } E 9,for all t 2 0).1.4 Proposition Let t and u be {9,}-stoppingtimes, let y be a nonnegative9,-measurable random variable, and let X be an ($r,}-progressive E-valuedprocess. Define X and Y by Xr(r)= X(7At) and Y(t)= X(7 +r), and define9,= F I h ,and MI = f,,,,t 2 0. (Recall that r h t and .c +r are stoppingtimes.)Then the followinghold:(4 .Fris a u-algebra.(b) T and 7 A u are SP,-measurable.(c) If t 5 usthen F,c F..(d) X(t)is fr-measurablc.(e) {Y,} is a filtration and X is both {gJ-progressive and(f) {Ju;)is a filtration and Y is {J1PIj-progressive.0 7 +- y is an {fJ-stopping time.{#,}-progressive.Proof. (3 Clearly 0and 0 are in PI,since9,is a u-algebra and {r 5 I } EF,.If A A (7 S c} EP,,then A n {t s, t } = (t 5 t ) - A n (7 s t ) E .F,,and hence A E implies A B 9,.Similarly Ak A {s 5; t } E s,,k = I,&. .., implies (UrA,) n (7 s t } = U&(Akn {T I; t } )E S,, andhence f,is closed under countableunions.(1.4) {TAU s c } n {T s r } = { T A U5 c A t } n {T s r }(b) For each c 2 0 and t 2 0,= ( { T 5 c A t } u {a I;cArj) n (t 5 t ) E F,.Hence { f A u 5 c] E .Frand r A d is S,-measurable, as is 7 (takeu = 1).
  • 62. 1. STOCHASTIC moassEs 53(c) If A E .Ft,then A n {a S t } = A n { t < t } n {IT s t } E 9,for allr 2 0. Hence A E 9#.(d) Fix t 2 0. By (b), T A t is .F,-measurable. Consequently the mappingo - r (t(o)Ar, o)is a measurable mapping of (a,9,)into ([O, r ] x Q,a[O,t ] x 9,)and since X is IF,}-progressive, (s, a)-+X(s, w) is a measur-able mapping of ([0, t ] x R, a[O,t3 x 9,)into (E, 1(E)).Since X ( t A t ) isthe composition of these two mappings, it is .F,-measurable. Finally, forP E @E), {X(r)E r}n { 7 s t } = {X(TA I ) E T} n {T s t } E .F,and henceBy (a) and (c), (Y,} is a filtration, and since 9,c 9,by (c), X is(9,)-progressive if it is {Y,}-progressive.To see that X is (Y,}-progressive,we begin by showing that if s 5 t and H E a[O,t ] x .Fs,then(1.5) H n (10,t ] x { T A t 2 S } ) E taco, t ] x Flh,= a[O,13 x 9,.To verify this, note that the collection X,,, of H E: a[O,t ] x 9,satisfying (1.5) is a a-algebra. Since A E 9,implies A n { T A t 2 s} E F,,,,it followsthat if B E a[O,13 and A E 9,.then(1.6) (B x A) n ([0,t ] x { T A C2 s)){x(t)E rjE 9,.(el= B x ( A n { T A Cz s}) E a[O,C] x Y,,so B x A Ea[O,r] x YS.(1.7) {(s, W ) E LO, t3 x R: x ( T ( ~ ) A ~ ,0))E r}But the collection of B x A of this form generatesFinally, for r E d ( E )and t 2 0,= {(S,W):X(T(W)AS,W)E~,~ ( w ) A r5;sst)= ({(s,w):T(w)At 5 s 5; I } n([O, r ] x {X(TA I)E r}))u {(s, 0):x(s, E r-, s < t(w)A C)since(1.8) {(s, 0)): ~ ( w ) A tI,s t }and since the last set on the right in (1.7) is in a[O,I ] x Y, by (1.5).(0 Again {HI}is a filtration by (a) and (c). Fix r 2 0. By part (e) themapping (s, u)-+X((t(w)+t)As, w) from ([O, 003 x Q, a [ O , 003 x F,,,)
  • 63. 54 STOCHASTIC CIIOCESSLS AND MARTINGALESinto (E, @E)) is measurable, as is the mapping (u, a)-+(r(w)+ u, 0) from([O, t] x fi, a[O,t] x gFt+J into ([0, 003 x Q, S[O, 003 x gr+J.Themapping (u, a)+ X(T(O)+u, o)from ([O, t ] x Q, a[O,r ] x Yr+Jinto(E, A?(&) is a compositionof the first two mappings so it too is measurable.SinceZ1= F,+,,Y is {X1j-progressive,C@ Let y. = [ny]/n. Note that ( 7 +y. s t } n {y, = k/n} ={ T 5 t -k/n} n (7. = k/n} E 91-r,m,since (7. = k/n} E 9,.Consequently,{ T +y. S t } E 9,.Since 7 +y = SUPAT +7,). part (g) follows by Proposi-tion 1.2(b). 0Let X be an E-valued process and let rE S(E).Thefirst entruncetime intor is defined by(1.9) Te(I‘) = inf (t: X(t)E r}(where inf 0 = m), and for a [O, m]-valued random variable a, the firstentrance time into I‘after u is defined by(1.10) Te(r,0 ) = inf {t 2 u:X(r)e r}.For each w r s n and O S S 5 t, let Fx(s, t, w ) c E be the closure of{X(u, a):s ,< u I; t}. Thejrst contact time with ris defined by(1.1 1) Tc(r)= inf { t : F,(O, t) n r # 0)and thejrst contact time with I’after a by(1.12) q(r,a)= inf { t 2 a: Fx(a,I ) n r it a}.Thefirst exit time from r (after a) is the first entrance time of Iy (after cr).Although intuitively the above times are “recognizable”to our observer, theyare not in general stopping times (or even random variables).We do, however,have the followingresult, which is sufficientfor our purposes.1.5 Proposition Suppose that X is a right continuous, {.F,}-adapted, E-valued process and that d is an {@,}-stoppingtime.(a) If r is closed and X has left limits at each t > 0 or if r is compact,(b) If ris open, then re(r,0 )is an (b,+)-stoppingtime.then Tc(r,a)is an {4tl}-stoppingtime.Proof. Using the right continuityof X,if ris open,(1.13) {t,(r,U ) < t ) = u {x(s)E r)n {U c S} E F,,a 6 0 n l O . 0implying part (b). For n = 1. 2,. ..let r, = {x:r(x, r)< l/n}. Then, under theconditionsofpart (a),zc(I‘, Q) = limn-mre(r,,,a),and(1.14) {rc(r.4s-r}~ P ( ( ~ ~ t } n { x ( t ) ~ r } ) u n . { ~ ~ ( r . , a ) < t }o
  • 64. 2 mnwAus 55Under slightly more restrictive hypotheses on {F,},a much more generalresult than Proposition 1.5 holds. We do not need this generality, so we simplystate the result without proof.1.6 Theorem Let IS,}be complete and right continuous, and let X be anE-valued {St,)-progressive process. Then for each r E @E), r,(Q is an(9,)-stoppingtime.Proof. See,for example, Elliott (1982). page 50. 02. MRTINGALESA real-valued process X with E[IX(t)lJ e 00 for all r 2 0 and adapted to afiltration (S,}is an {.%,}-martingale if(2.1) ECX(t +s)IS,]= X(r), t, s 2 an {SF,}-submarringaleif(2.2) ECWt +s)l.!FJ 2 XO), r, s 2 0,and is an {SFIP,)-supermartingakif the inequality in (2.2) is reversed. Note thatX is a supermartingaleif -X is a submartingale,and that X is a martingale ifboth X and - X are submartingales. Consequently, results proved for sub-martingales immediately give analogous results for martingales and super-martingales. If {@,} = {Sf}we simply say X is a martingale (submartingale,supermartingale).Jensens inequality gives the following.2.1 Proposition (a) Suppose X is an {.!F,}-martingale, cp is convex, and&[lcp(X(t))(]< 00 for all t 2 0,Then cp 0 X is an (9,}-submartingale.(b) Suppose X is an (9,)-submartingale, cp is convex and nonde-creasing, and &[lcp(X(t))l]< 00 for all t 2 0. Then cp 0 X is an(9,)-submartingale.Note that for part (a) the last inequality is in fact equality, and in part (b) the0last inequality followsfrom the assumption that cp is nondecreasing.2.2 Lemma Let r , and r2 be {.!F,}-stopping times assuming values inItl,t 2 , ...,tm} c [O, 00). If X is an {9,}-submartingale,then(2.4) ECX(r2)lFI,12 x(flA rd.
  • 65. 56 STOCHASTIC PROCESSES AND MARTINGALESProof. Assume t, K tz < .* * < t,. We must show that for every A E 9,,(2.5)Since A = uysl(A n ( t l = ti}), it is sufficientto show thatnThe following is a simple application of Lemma 2.2. Let x t = x VO,2.3 Lemma Let X be a submartingale, T > 0. and F c [O, TJ be finite.Then for each x > 0,(2.9) I;x - E [ X + ( T ) ]andProof.ThenLet T = min {r E F: X(t)2 x) and set T , = T AT and T , = T in (2.4).(2.1 1) E[X(T)]2 E [ X ( t A711 = ECX(T)Zlr<mJ + ECX(T)Xir=a)I*and hencewhich implies (2.9).The proof of (2.10)is similar. 0
  • 66. 2. MARTINGALES 572.4 Corollary Let X be a submartingale and let F c [0, 00) be countable.Then for each x > 0 and 7 > 0,(2.13) .( sup x ( t ) 2 s x ~ E C X + ( T ) It a F n ( 0 . T Iand(2.14)Proot Let F,c F, c . . . be finite and F = UF,. Then, for 0 < y < x,(2.15)P{ inf X(r)S - x s; .Y (E[Xt(7)1 - E[X(O)]).p{ sup x ( t ) 2x s y - l ~ ~ ~ + ( r ) ~r c F n ( 0 . TI0Letting y- x we obtain (2.13), and (2.14) follows similarly.Let X be a real-valued process, and let F c LO, 00) be finite. For a < hdefine rl = min { t E F:X(t) I a}, and for k = 1,2,. ..define ak= min { t > t k :I E F, X(r)2 h} and r k t , = min { t > a*:t E F,X(r)< u}. Define(2.16) V(a,h, F) = max {k:ak< ao}.The quantity V(a,b, F) is called the number of upcrossings of the interval (a, h)by X restricted to F.I E F A 10. 7 12.5 lemmathenLet X be a submartingale. If T > 0 and F c [0, 71 is finite,(2.17)Proof. Since ut A T I rk + I A 7 ,Lemma 2.2 implicswhich gives (2.17).
  • 67. 58 STOCHASTIC PROCESSES AND MARTINGALES2.6 Corollary Let X be a submartingale. Let T > 0. let F be a countablesubset of [O, TI, and let F, c F, c - - be finite subsets with F = UF,,.Define V(a, b, F) = lim,,-.m V(a, 6, FJ Then V(a, b, F)depends only on F (notthe particular sequence (F,,})and(2.19)Proof. The existence of the limit defining V(a,6, F) as well as the indepen-dence of V(a,b, F) from the choice of {F,,} follows from the fact that G c Himplies V(a, 6,G) 5 U(a, b, H). Consequently (2.19) follows from (2.17) and themonotone convergence theorem. 0One implication of the upcrossing inequality (2.19) is that submartingaleshave modifications with “nice” sample paths. To see this we need the follow-ing lemmas.2.7 Lemma Let (E, r) be a metric space and let x: [O,oo)--t €. Supposex(t +) =lim,,,, x(s) exists for all r L 0 and ~ ( t - )=lim,-l- x(s) exists for allt > 0. Then there exists a countable set r such that for f E (0,oo)-I-,x(t -) = x(t) = x(t +).Let r,,= { t : r(x(t-)l x(i))Vr(x(i-), x(t+))Vr(x(t), x(f+)) > n-I}. Thenr,,n [0, T ] is finite for each T > 0.Proof. Since we may take r = U,,r,,it is enough to verify the last statement.If r,,n [0, 7 3 had a limit point r then either x(t -) or x(r+) would fail to0exist. Consequently rmn LO, T ] must be finite.2.8 Lemma Let (€, r) be a metric space, let F be a dense subset of [0, a),and let x: F - r E. If for each t 2 0(2.20)exists, then y is right continuous. If for each I > 0(2.21) y - ( t ) = lim x(s)s - I -S S Fexists, then y - is left continuous (on (0, 00)). If for each r > 0 both (2.20) and(2.21) exist, then y-(f) = f i t - ) for all I r 0.
  • 68. 2. MARTINGALES 59Proof. Suppose (2.20)exists for all f 2 0. Given to > 0 and E > 0, there existsa 6 > 0 such that r(y(to),x(s)) 5 E for all s E F n ( t o , to + a), and hence(2.22)for all s E ( t o , to +6) and the right continuity of y follows. The proof of theother parts is similar. 0Let F be a countable dense subset of [O, 00). For a submartingale X,Corollary 2.4 implies P { ~ u p ~ ~ ~ ~ ~ ~ , ~ ~ X ( t )< cn) = 1 and P{infleFnlo,rlX(f)>-OD} = I for each 7> 0, and Corollary 2.6 givesP( V(a,h, F n [O, TI)< a)) = I for all ci < h and T > 0. Let(2.23) Ro = fi ({ sup X(t) < 00} n { inf X(r) > - m1n = l l c F n l O , n l IE F n lo. nln n (W,h, F n [O. t i ] ) < 0 0 )a r ha. b e 8Then P(Qo) = 1. For w E Ro,(2.24)exists for all t 2 0, and(2.25)Y(r,w ) = lim X(s, o)S - I t.s E Fexists for all I > 0; furthermore, Y( ., o)is right continuous and has left limitswith Y(i -, o)= Y - ( I , o)for all I > 0 (Problem 9). Define Y(t,o)= 0 for allw 4 R, and t 2 0.2.9 Proposition Let X be a submartingale, and let Y be defined by (2.24).Then r = ( 1 : P( Y ( i )# Y(i -)} > 0) is countable, P(X(r)= Y(r))= 1 for r 4 r,and(2.26)defines a modification of X almost all of whose sample paths have right andleft limits at all 2 0 and are right continuous at all t $ r.Proof.(2.27)For real-valued random variables q and <(defined on (R, 9,P))definey(q, t)= inf ( E > 0: P ( ) q- tl > E } < E } .
  • 69. 60 STOCHASTIC PROCESSES AND MARTINGALESThen y is a metric corresponding to convergence in probability (Problem 8).Since Y has right and left limits in this metric at all f 2 0, Lemma 2.7 impliesr is countable.Let a E R. Then X V a is a submartingale by Proposition 2.1 so for anyT > 0,(2.28)and since ( E [ X ( T ) V a ( f f ] :0 5 t 5 T) is uniformly integrable (Problem lo),it follows that { X ( f ) V a :0 I;t 5 T}is uniformly integrable.Therefore(2.29) X(r)Va I lim E [ X ( s ) V a 1 9 f ] i= E[Y(r)Va)9;], r 2 0.a s X ( t ) v as E[X(T)V~~.%;K], O s t s T,S + l +S C QFurthermore if t f r,then(2.30) E[E[Y(t)VaIF(P,Y]- X(r)Va) I; lirn E[Y(r)Va - X(s)Va] = 0,s-I -s c Qand hence, since Y(t)= Y(t-) as. and V(t-) is 9/-measurable,(2.31) X(r)V a = ELY(t)V aJ = Y(r)V a as.Since a is arbitrary, P{X(r)= Y(r))= 1 for t 6 r.To see that almost all sample paths of 8 have right and left limits at allt 2 0 and are right continuous at all r 4 r,replace F in the construction of Yby F u r. Note that this replaces noby 0,c no,but that for o E no,Y(a , w) and 8(a , w) do not change. Since for w E 0,(2.32) Y(t. w) = lim Y(s,w) = lim X(s, w), I 2 0,s-1+ s-I+s a F u Tit follows that(2.33)which gives both the existence of right limits and the right continuity of0Y(t,o)= Iim &s, w), r 2 0,s-I +8(-,w) at t $ r.The existence of left limits followssimilarly.2.10 Corollary Let 2 be a random variable with EL I Z 11 < 00. Then for anyfiltration (9,)and t L 0,E[ZIsF,J-, E[Z19,+]in L ass-+ I + .Proof. Let X(r) = EIZJ.FIJ,I z 0. Then X is a martingale and by Proposi-tion 2.9 we may assume X has right limits as. at each r 2 0. Since (X(l)}isuniformly integrable, X ( t + ) s lirn,,,, X(s) exists a.s. and in L! for all f 2 0.
  • 70. 2. MARTINGALES 61We need only check that X(t + ) = E [ Z ( . 4 t l + ] . Clearly X(r + 1 is.Fl+ -measurable and for A E 9,+ ,(2.34)hence X(t +) = E [ Z I.Fl+I.X ( t + ) d P = lim02.11 Corollary If {,Fl}is a right continuous filtration and X is an(9,)-martingale,then X has a right continuous modification.2.12 Remarksample paths ofa right continuous submartingale have left limits at all r > 0.Proof. With reference to (2.24)and Corollary 2.10, for t < T,(2.35)It follows from the construction of Y in (2.24) that almost all0Y(r)= lim X(s) = lirn E[X(’I’)I.f,]s -*I t s - I +s c F r e F= E / X ( T ) I . F , + J= E [ X ( T ) I . f , ]= X ( i ) as.,so Y is the desired modification. 0Essentially, Proposition 2.9 says that we may assume every submartingalehas well-behaved sample paths, that is, if all that is prescribed about a sub-martingale is its finite-dimensional distributions, then we may as well assumethat the sample paths have the properties given in the proposition. In fact, invirtually all cases of interest, I- = a,so we can assume right continuity at allt 2 0. We do just that in the remainder of this section. Extension of the resultsto the somewhat more general case is usually straightforward.Our next result is the optional sampling theorem.2.13 Theoremand f 2 be (f,}-stoppingtimes. Then for each T > 0,(2.36)If, in addition, r2 is finite as., E[lX(r,)(] < 00, andLet X be a right continuous {.f,}-submartingale,and let T ,E[X(T, A T)14tr,32 X(r, A r2A T).(2.37)then2.14 Remark Note that if X is a martingale (Xand -X are submartingales),then equality holds in (2.36) and (2.38). Note also that any right continuous{.f,}-submartingale is an {.fl+ f-submartingale, and hence correspondingUinequalities hold for (9,+ )-stopping times.
  • 71. 62 STOCHASTIC PRocfsSES AND MAITINGALESProof. = (k f IM2" if k/2" rjT~ < (k + 1)/2". Then by Proposition 1.3, $1 is an {$,}-stopping time, and byLemma 2.2, for each a E: R and T > 0,(2.39)Since T,,t(2.40)Since Lemma 2.2 implies(2.41) a 5 X ( r y A T ) V a S E [ X ( T ) V a l f , p I ,{X(r!j"AT ) V a j is uniformly integrable as is ( X ( ~ Y A T Y AT ) V a } (Problem10).Letting n-+ 00, the right continuity of X and the uniform integrability ofthe sequencesgivesFor i = 1, 2, let T?) = a, if T~ = GO and letE[X(ty T)V a I9,?,]2 X(r,"A T? A T ) Proposition I .4(c),(2.39) impliesE[X(ry A 7)V a IT,,]2 E[X(r:"A 7$"A T )VaISr,].(2.42) E [ X ( r , A T ) V o l I S , , ] z E [ X ( r , At2AT)Val.Frll= X ( T I A T ~ A T ) V U .Lerting a+ -00 gives (2.36), and under the additional hypotheses, letting0T 3 00 gives (2.38).The followingis an application of the optional sampling theorem.2.75 Proposition Let X be a right continuous nonnegative{~f}-superinartingale,and let r,(O) be the first contact time with 0. ThenX(r)= 0 for all I 2 r,(O) with probability one.proof. For 11 = 1.2,..., let T, = ~ ~ ( [ 0 , n - ~ ) ) ,the first entrance time intoC0,n-I). (By Proposition 1.5, 7, is an {9f+}-stoppingtime.) Then TJO) =limm+mT". If T, < 00, then X(T,)5 n - . Consequently,for every r 2 0,(2.43) ECX(l)I$1. +3 5 X ( t A TJrand hence(2.44) ECX(f)I~,"+lX,r,sf,5 0 - l .(2.45) ~ r ~ ( ~ ) X , I < , o , r r , l= 0.Taking expectationsand letting n-+ 00, we haveThe proposition follows by the nonnegativity and right continuity. 0Ncxt we extend Lemma 2.3.
  • 72. 1. MARTINGALES 632.16 Proposition (a) Let X be a right continuous submartingale. Then foreach x > 0 and T > 0,(2.46) X(t) 2 x Ix-E[X+(T)]1andP inf X(r)5 - x Ix (E[X+(T)] - E[X(O)]).I S T(2.47) {(b) Let X be a nonnegative right continuous submartingale. Then fora > I and T > 0.(2.48)Proof. Corollary 2.4 implies (2.46) and (2.47), but we need to extend (2.46) inorder to obtain (2.48). Under the assumptions of part (b) let x > 0, and defineT = inf { I : X ( t ) > x). Then T is an {4tl.)-stopping time by Proposition l.S(b),and the right continuity of X implies A(?)2 .Y if T c 00. Consequently forT > 0,(2.49)and the three events have equal probability for all but countably many x > 0.By Theorem 2.13,{;;! x(t)> x} c 5 T J c sup X(r)2 x ,{i 6 J 1(2.50)and henceE[X(r A T)] 5 E[X(T)],(2.5I ) x P { T5 E[A(T)X{tsTI] 5 ECX(T)X(~.llnLet cp be absolutely continuous on bounded intervals of [0,m) with cp 2 0and cp(0)= 0. Define Z = supls X ( l ) . Then for p > 0,(2.52) E[p(ZA/?)]= [p(x)P(Z> x) d xI cp(x)x ~ ECX(77xlzr,,1d.u= E [ X ( T)&%A p)]where $(z) = f: v(.u)x- dx.
  • 73. 64 STOCHASrlC PROCESSES AND MMTfNGMESIf cp(x) =xu for somea > 1, thenaa - 1-E[X(T)"]"aE[(ZAfl~]u-"a,and henceE[X(T)]".aE[(ZA p)7"" 5 --a - 1(2.54)Letting fi-, og gives (2.48). 02.17 Corollary Let X be a right continuous martingale. Then for x > 0 andT > 0,(2.55)and for a > 1 and T > 0,P sup IX(t)l 2 x s X-~E[IX(T)I],Lsr(2.56)Proof. Since 1x1 is a submartingale by Proposition 2.1, (2.5s) and (2.56)0followdirectly from (2.46) and (2.48).3. LOCAL MARTINGALESA real-valued process X is an {9,}-local martingale if there exist{f,)-stopping times T S r3 s * with zn+ 00 as. such that Xrn=X(* A t,,)is an {9,}-rnartingale. Local submartingales and local supermartingalesaredefined similarly. Of course a martingale is a local martingale. In studyingstochastic integrals (Chapter 5) and random time changes (Chapter a), one isled naturally to local martingalesthat are not martingales.3.1 Proposition If X is a right continuous {9,}-local martingale and t is an{f,}-stoppingtime, then X =X( A t ) is an {f,}-local martingale.Proof. There exist {S,f-stopping times s s - * . such that t,-+ 00 as.and X" is an (.F,}-martingale. But then X(.AT,,) = X"(.A t ) is an0(S,}-martingale,and hence X is an {4r,}-localmartingale.
  • 74. 3. LOCALminN<ims 65In the next result the stochastic integral is just a Stieltjes integral andconsequently needs no special definition. As before, when we say a process Vis continuous and locally of bounded variation, we mean that for all w E fl,V( .,w )is continuous and of bounded variation on bounded intervals.3.2 Proposition Suppose X is a right continuous (.F,}-local martingale, andV is real-valued, continuous,locally of bounded variation, and {S,}-adapted.Then(3.1) M(t)=[V(s)dX(s)= V(t)X(t)- V(O)X(O)- X(S)dV(s) an {.F,)-localmartingale.Proof. The last equality in (3.1) is just integration by parts. There exist(9,j-sropping times T , s r2 -< - such that T,,+ 00 as. and Xrnis an{Sc,}-martingale. Without loss of generality we may assume r,, sT,(( -00, -n] u [n,a)),the first contact time of (- a,-n] u [n,m) by X. (Ifnot, replace T,, by the minimum of the two stopping times.) Let R be the totalvariation processwhere the supremum is over partitions of [O, t], 0 = so < s, < * . < s, = t.For n = 1,2,. ..let y,, = inf { t : R(t) 2 n}. Since R is continuous, y,, is the firstcontact time of [n,a))and is an {SF,}-stoppingtime by Proposition 1.5. Thecontinuity of R also implies y,, -+ 00 as.Let 6,= ?,,AT,,. Then u,,+ 00 as. and we claim M(* At?,,) is an{.F,}-martingale.To verify this we must showfor all t, s 2 0.Let t = uo < uI < . . * < u, = t +s.Then(3.4)since Xu" is an (9,}-martingale and Vu"fu,) is Sm,-measurable. Lettingmax,lu,, I - uk1+ 0, the sum in (3.4) converges to the second integral in (3.3)
  • 75. 66 STOCHASTIC PROCESSES AND MARTINGALESas. However, to obtain (3.3). we must show that the convergence is in f!Observe(3.5) I IV""(uk)(X8I(uk+,) - Xea(uk))k = O-- vyr +s)Xqf +s)- Vuqo)xuqo)m - 1I- c X U " ( U k + 1 W V U ( I ( ~ k + , )- VYU,))(k = O5 Ivyt +S)X"(t +s) - VI(0)XuyO)Im- Ik = O+ c IX%k +I ) I I V Y h +I ) - V"(u*)Is I vyt +S)Xqt +s) - v"~(0)XyO)I +(nv IXYl +s)I)R(a,).The right side is in L, so the desired convergence follows by the dominatedconvergencetheorem. 03.3 Corollary Let X and Y be real-valued, right continuous, {.Fl}-adaptedprocesses. Suppose that for each I, infss, X(s) > 0.Then(3.6) M,(r)=X(r)-6Y(s)dsis an (9,}-local martingale if and only if(3.7)is an (9,)-local martingale.Proof. Suppose M,is an (.Fl)-localmartingale.Then by Proposition 3.2,
  • 76. 3. LOCAL MARTlNGALES 67is an {9,}-local martingale. Conversely, if M, is an (9,}-local martingale,then(3.9)= X ( i ) -X(0)- Y(s)dsis an {.f,}-localmartingale. 0We close this section with a result concerning the quadratic variation of thesample paths of a local martingale. By an "increasing" process, we mean aprocess whose sample paths are nondecreasing.3.4 Proposition Let X be a right continuous {.F,}-local martingale. Thenthere exists a right continuous increasing process, denoted by [XI,suchthat for each f 2 0 and each sequence of partitions {u:"} of [0, r] withmax,(up! I - up)-b 0,(3.10)as n--. m. If, in addition, X is a martingale and & [ X ( C ) ~ ]< n3 for all r 2 0,then the convergencein (3.10)is in I!!.Proof. Convergencein probability is metrizable (Problem 8); consequently wewant to show that {xk(X(u:"!,) - X(up))} is a Cauchy sequence. If this werenot the case, then there would exist E > 0 and (n,) and {mi) such that n,-r 00,mi--+m, and(3.1 I)for all i.Since any pair of partitions of [0, t3 has a common refinement, that is, thereexists (uk} such that {up)}c ( u k } and {u:")) c {u,}, the followinglemma con-tradicts (3.11) and hence proves that the left side of (3.10) converges in prob-ability.3.5 Lemma Let X be a right continuous {F,)-localmartingale. Fix T > 0.For n = 1.2,. ..,let {@) and {or)} be partitions of [0, T ] with {or} c {ur)and maxk(ur! - up)+ 0. Then
  • 77. 68 STOCHASTJC HOCLsSES AND MARTINGALESProof. Without loss of generality we can assume X is a martingale(otherwiseconsider the stopped process P),and X(0) = 0. Fix M > 0, and lett = inf (s: (X(s)(2 M or (X(s-)l r:M}.Note that P{r s I } s E[lX(r)l]/MbyCorollary2.17.Let {uk)and {uk} be partitions of [0, t], and suppose { u k } t {uk}.Let wk =max {v,: vl 5 uk},and defineqk s Xr(uk)-x(uk- ,) and(3.13) z= ~ ( x ( ~ , )- X(ul-1)) -c(x(uk) - X(uk-1))= c t k ,where Ck = 2(xr(uk)-x(uk- I))(X(Uk- -Xr(wk-or ltkls 4MlXr(uk)-X(uk,l)( and that E[{k+l14t,,J = 0. Consequently,Note that either tk=0(3.14)mis a discrete-parametermartingale.Let(3.15)1x1s 4M2,- 16M4, 1x1>4M.where the last inequality follows from the convexity of rp and the fact that fork 5 K,IX(u,)l 5 M.Using the fact that {Z,} is a discrete-parametermartingale,
  • 78. 3. LOCAL MARTINGAUES 69Fix E > 0. Let a1= min {uk:IX(uk)- X(ul)( 2 c} u {ul+ ,} and /?,= u,+Note that if vI = W k - l 5 U k - l < ul+, and a, > u l t l r then (X(u,. ,)- X(w,. ,)) 5; cz. Consequently, by (3.16)and (3.17),+ E C X ~ ~ ~ < ~ ~ +,) 16M(cp(Xr(Dl))- v(X(al)))l.IFix N 2 I, and let t = min {I: ~ ~ = O ~ f a i z , , i + , )= N } . Let y = aL if L < 00 andy = T otherwise. Then y is an {$,}-stopping time, and hence by (3.18)and theconvexity of cp,Given E, E > 0, let D = {s E [O, TI :IX(s) - X ( s - ) ) > 4 2 ) . Then thereexists a positive random variable 8 such that s E D and s s t 5 s +6 implyIX(t) - X(s)I 5 E, and 0 5 s < t s T, t - s s 8, and I W t ) - X(s)I 2 E imply(s,13 n D # 0.Let ID1 denote the cardinality of D. On {max(u,+I - ui) 5; S),(3.20)5 (101A N)8M2d.Let S(T)be the collection of {F,)-stoppingtimes a with a < T. Sincefor all a E S(T),{cp(X(a)):a E S(T)}is uniformly integrable. Consequently, theright side of (3.19)can be made arbitrarily small by taking E small, N large (sothat PIN < IDI} is small), E small, and max (u,+, - 0,) small. Note that ifN > [Dl and max (u,+I - of) < S,then y = T.
  • 79. 70 STOCMNC PROC€SSES AND MMNNCALESThus, if Z() is defined for {up} and {up)}as in (3.13), the estimate in (3.19)implies(3.22)which, since M is arbitrary,implies(3.12). 0Proof of Proposition 3.4-conrinued. Assume X is a martingale andE[X(T)] < 00. Let {Uk) be a partition of 10, TI,and let X be as in the proofof Lemma 3.5. Then(3.23) EIIc(x(uh+l)- x(uk))2 -c(xr(uk+,)- xr(uk))211S EC(X(T)-X(TAr))21+ ECl(X(ug+1) -W)Mx(t)- X(UK))IXI~<TJ,where K = max {k: uk < r). Since for M 2 1, cp defined by (3.15) satisfies1x1 sE +&)/E for every E >0, the estimatesin the proof of Lemma 3.5 imply{ ~ ( X ( u : " : , )-Xr(up)))l} is a Cauchy sequence in Ll (note that we need&[X(T)] < 00 in order that this sequence be in L).Consequently, since theright side of (3.23) can be made small by taking M large and max (uk + I -uk)small, it follows that {c(X(iifiConvergence of the left side of (3.10) determines [X](t) a.s. for each 1 2 0.We must show that [ X ] has a right continuous modification.Since {Z,,,}givenby (3.14) is a discrete-parametermartingale, Proposition 2.16 gives-X(ut))} is a Cauchy sequence in L.(3.24)Consequently,for 1 L n-, co and T > 0,k S 2nTand it follows that we can define [XI on the dyadic rationals sonondecreasingand satisfies-30,that it isThe right continuity of [XI on the dyadic rationals follows from the rightcontinuity of X.For arbitrary t 2 0, define(3.27)Clearly this definition makes [XJright continuous. We must verify that (3.10)is satisfied.
  • 80. 4. THE FROIECTION THEOREM 71Let (uf}= {i/2":0 5 i s [2"t]} u { t } . Then-% 0,and (3.10)follows. 03.6 Proposition Let X be a continuous (.Ff}-localmartingale. Then [ X I canbe taken to be continuous.Proof. Let [ X I be as in Proposition 3.4. Almost sure continuity of [ X Irestricted to the dyadic rationals follows from (3.26) and the continuity of X.Since [ X I is nondecreasing,it must therefore be almost surely continuous. (-J4. THE PROJECTION THEOREMRecall that an E-valued process X is (P,}-progressiveif the restriction of X to[O, t ] x R is W[O, t ] x gf-measurablefor each t 2 0, that is, if(4.1) ((s, w): x(s, 0)E r-1 n "0, ti x 0)E a[o,CIx 9,for each t 2 0 and r E ~ ( E ) .Alternatively, we define the a-algebra of{9,)-progressiue sets W by(4.2) W = { A E a[O,ao) x 9:A n ([0,t ] x Q) E a[O.t] x 9,for all t 2 0).(The proof that W is a a-algebra is the same as for 9,in Proposition 1.4.)Then (4.1) is just the requirement that X is a W-measurable function on[O, 00)x Q.The a-algebra of {.F,}-oprional sets 0 is the a-algebra of subsets of[O,oo) x Q generated by the real-valued, right continuous {9,)-adapted pro-cesses. An E-valued process X is (9,}-optional if it is an @measurable func-tion on LO,00) x Q. Since every right continuous (9,)-adapted processis (*,}-progressive, 0 c W , and every (9rf}-optional process is{9,)-progressive.
  • 81. M STOCHASTIC fRocEuLE AND MARTINGALESThroughout the remainder of this section we fix {SJand simply sayadapted,optional, progressive, and so on to mean {$,}-adapted, and so on. Inaddition we assumethat (9,)is complete.4.1 lemma Every martingalehas an optional modification.Proof. By Proposition 2.9, every martingale has a modification X whosesample paths have right and left limits at every c E [0, co)and are right contin-uous at every t except possibly for f in a countable, deterministic set r.Weshow that X is optional. Since we are assuming (6,)is complete, X isadapted. First define(4.3)k + ln n(set X(-t/n) = X(0)). and note thatadapted and right continuous, Y is optional.(4.4)x(t) E Y(r)= X(t -). Since Y. isFix E > 0. Define I,, = Oand,for n L= 0, 1,2,.. .,I , + ~= inf {s> I,: IX(s)-X(s-)I >6 or JX(s+) -X(s-)l> Eor JX(s+)-X(s)l >6).Since X(s+) = X(s)except for s E r,(4.5) { I , c t } = un u ii(Ix(r,) -WSJI > + 6 p,1 n ( ~ i .11) I = 1where {s,, t,) ranges over all sets of the form 0 5; sl <f , < sz < t2 < * * * <s, < ti < t, It, -s,I < l/m, and r,, sIE l- u Q. Definema= I(4.6) U t ) = C xit., r. +l,rn)(t)~[~XCr.) -x(r.-)I >er(X(tn) -X ( T ~-1)+ ~ ( l ~ ( I ~ - ~ ( I - } l > ~ ~ ~ ~ ( f )-W-)).Since X has right and left limits at each t E [O, a), I, = 00, and hence2: is right continuous and has left limits. By (4.5). {I, < s} E 6,for s s t, andan examination of the right side of (4.6) shows that ZL(f)is 4t,-measurable.Therefore 2: is optional. Finally observe that IY(r)+1irnm+,, Z:(c) -X(t)l S0E, and sinceE is arbitrary,X is optional.4.2 Theorem Let X be a nonnegative real-valued,measurable process. Thenthere existsa [O, a]-valued optional process Y such that(4.7) E[X(t)I6 r l = Y(T)
  • 82. 4. THE FROjtcnON THEOREM 73for all stopping times T with P{T< a}= 1. (Note that we allow both sides of(4.7) to be infinite.)4.3 Remark Y is called the optional projection of X . This theorem implies apartial converse to the observation that an optional process is progressive.Every real-valued, progressive process has an optional modification. Theoptional process Y is unique in the sense that, if Yl and Y, are optionalprocesses satisfying(4.7), then Y,and Y,are indistinguishable.(See Dellacherieand Meyer (1982). page 103.) 0Proof. Let A e 9 and B E O[O, a),and let 2 be an optional processsatisfying E [ x , , ) ~ , ]= Z(t). 2 exists, since E[x,,)S,] is a martingale. Theoptional sampling theorem implies E[XAIgFr] = Z(t). Consequently, xe(r)Z(t)is optional, and(4.8) ECXI(T)XA i9tI = xe(r)Z(r).Therefore the collection M of bounded nonnegative measurable processes Xfor which there exists an optional Y satisfying(4.7) contains processes of theform xS x,,, B E a[O,a)),A E 9.Since M is closed under nondecreasing limits,and X , , X z E M,XI 2 X 2 implies XI -X 2E M,the Dynkin class theoremimplies M contains all indicators of sets in O[O, a))x 9,and hence allbounded nonnegative measurable processes. The general case is proved by0approximating X by X An, n = I, 2,. ..,4.4 Corollary Let X be a nonnegative real-valued,measurableprocess. Thenthere exists Y: [0, 00) x 10, oo)x Q 4[O, 00], measurable with respect toW[O,00) x 0,such that(4.9) E[X(T +.$)I.(p1] = Y(s,t)for all a.s.finite stopping times T and all s 2 0.Proof. ReplaceX&) by Xa(t +s)in the proof above. 04.5 Corollary Let X :E x [O,m) x Q - b [0,m) be 9 ( E ) x 9[0,00) x b-measurable. Then there exists Y: E x [0,oo) x Q-, [0, oo], measurable withrespect to g(E)x 0,such that(4.10) ECX(x, ~ ) ls r l = Y(x, T)for all 8.s. finitestopping times T and all x E E.Proof. Replace Xs(t) by xdx,I), B E g ( E ) x W[O,a)), in the proof ofTheorem 4.2. 0The argument used in the proof of Theorem 4.2 also gives us a Fubinitheorem for conditional expectations.
  • 83. 74 STOCHASTIC mocwEs AND t.wnNcALEs4.6 Proposltion Let X: E x il-+R be B(E)x *-measurable, and let p be aa-finite measure on a(E). Suppose IE[IX(x)(]p(dx)< 00. Then for every 6-algebra 9 c I,there exists Y:E x 0- R such that Y is @E) x 9-measurable, Y(x)= E[X(x)l9]for all x E E, Y(x)(lr(dx)-ca3 a.s., and(4.11)4.7 Remark With this proposition in mind, we do not hesitate to write(4.12) 0Proof. First assume p is finite, verify the result for X = xPxA. B E a(€),A E 9,and then apply the Dynkin class theorem. The a-finitecase follows by0writing p as a sum of finitemeasures.5. THE DOOB-MEYER DECOMPOSITIONLet S denote the collection of all (Il}-stopping times. A right continuous(.Fl}-submartingale is of class DL if for each T >0, {X(t.AT):t E S} is uni-formly integrable. If X is an (9,)-martingale or if X is bounded below,then Xis of class DL (Problem 10).A process A is increasing if A( *, a)is nondecreasing for all UJ E z1. Everyright continuous nondecreasing function a on [O, 00) with 40)= 0 determinesa Bore1 measurep,,on [O, a)by p,,[O, t ] -- a(t).We definewhen the integral on the right exists. Note that this is not a Stieltjesintegral iffand a have common discontinuities.5.1 Theorem Let {9,}be complete and right continuous, and let X be aright continuous (Sl}-submartingale of class DL.Then there exists a unique(up to indistinguishability) right continuous {.FI}-adaptedincreasing processA with A(0)= 0 and the followingproperties:(a) M = X -A is an {S,}-martingale.
  • 84. 5. THE DOOI&MEVIER MCOkYOSITION 75(b) For every nonnegative right continuous {F,)-martingale Y andevery I 2 0 and t E S,Y ( s - ) dA(s)] = E[sb^ Y(s)(5.2) E[lAr = E[ Y(tA T ) A ( ~A T)].5.2 Remark (a) We allow the possibility that all three terms in (5.2) areinfinite. If (5.2) holds for all bounded nonnegative right continuous{S,}-martingales Y, then it holds for all nonnegative (9,)-martingales,since on every bounded interval [O, T ] a nonnegative martingale Y is thelimit of an increasing sequence { V,) of bounded nonnegative martingales(e.g., take U, to be a right continuous modification of Y:(t) = E [ Y ( T ) A(b) If A is continuous, then the first equality in (5.2) is immediate. Thesecond equality always holds, since (assuming Y is bounded) by the rightcontinuity of Y(5.3)n I~ , 3 .+n-1(A(k;-) - A ( ? - ) ) ]= E [ y ( t A ~ ) A ( l h ? ) ] .The third equality in (5.3) follows from the fact that Y is a martingale.Property (b) is usually replaced by the requirement that A be pre-dictable, but we do not need this concept elsewhere and hence do notintroduce it here. See Dellacherie and Meyer (1982), page 194, or Elliot(1 982),Theorem 8.15. 0(c)Proof. For each E > 0, let X8be the optional projection of E - yoX(* +s)ds,(5.4)Then X,is a submartingaleand(5.5) lim E[IX,(t) - X(t)(J= 0, t 2 0.8 - 0
  • 85. 76 STOCHASTIC PRocEssEs AND MARTINGALESLet V, be the optional projection of E-(X(* +E) -X( )), and defineSince X is a submartingale,(5.7) YJr) = E -E[X(t +E) -X(t)IS,]2 0,and hence A, is an increasing process. Furthermore(5.8) M, = X, -A,is a martingale, sincefor t, u 2 0 and 5 E .IF,,(5.9) (MAC+4 - M,(t))dP-j, E-(X(S+e) -X(s))ds dP = 0.)We next observe that {A,(t):0 < e S 1) is uniformly integrable for eacht 2 0. To see this, let T: = inf (s: A,(s) 2 A}. Then(5.10) E[Aa(t)-1A A&)] = E[A,(t) -A,(tfrAt)]= E[X,(t) -x,(t:A t)]= EC&; < ,) (X,(t)-XXTf A t))ISince P(T:c t ) 5; A-E[A,(t)] 5 I-E[X(t +e) -X(O)], the uniform intcgra-bility of (X(rA(t+ 1)): t E S} implies the right side of (5.10) goes to zero asA+ 00 uniformly in 0 < E s 1. Consequently {A,(t):0 <E s 1) is uniformlyintegrable (Appendix 2). For each t L 0, this uniform integrability implies theexistenceof a sequence {em} with E. -+ 0, and a random variable A(t) on (a,9)such that(5.1 1)for every E E f(Appendix 2). By a diagonalization argument we may assumethe same sequence {e,) works for all t E Q n [O, 00).Let 0 s s < t, s,c E Q, and B = {A(t)< A(s)). Then(5.12) E[(A(t)-A(S))X,I = lim EC(A,(t) - A,(~)lXBI 2 4a-m
  • 86. 5. THE DOOU-MEYER OECOMMSlTlON 77so A(s) s A(t) as. For s, t L 0,s,t E Q, and B E F,,(5.13) E[(X(f +s) - A(f +s) - x(f)+ A(t))XB]= lim E[(M,"(t +s) - M,m(t)),ysJ= 0,n - mand defining M(c)= X(t)- A(t) for t E Q n [O, m), we have€[M(t +s ) ( 9 , ] = M(t) for all s, t E Q n LO, 00). By the right continuity of(9,;and Corollary 2.11, M extends to a right continuous (.F,)-martingale,and it follows that A has a right continuous increasingmodification.To see that (5.2) holds, let Y be a bounded right continuous{SF,)-martingale.Then for t L 0,(5.14) E[Y(t)A(t)]-- lim E[ Y(t)A,l(t)Ja-+m= lirn E[l Y(s-) dAJs)]n-mand the same argument works with t replaced by t A r.Finally, to obtain the uniqueness, suppose Al and A, are processes with thedesired properties. Then A, - A, is a martingale, and by Problem 15, if Y is abounded, right continuous martingale,(5.15)= Y(t)A,(Ol.Let B = { A , ( t ) > A2(t)} and Y(s)= E [ X ~ J S , ](by Corollary 2.11, Y can betaken to be right continuous). Then (5.15) implies(5.16) EC(Al(t) - AZ(t))XB] = 0.Similarlytake B = { A # ) > A&)} and it follows that A#) = A,(t) as. for eacht L 0. The fact that A , and A, are indistinguishable follows from the rightcontinuity. 0
  • 87. 78 STOCHASTlC ?ROCEESES AND MARTINGALES5.3 Corollary If, in addition to the assumptionsof Theorem 5.1, X is contin-uous, then A can be taken to be continuous.Proof. Let A be as in Theorem 5.1. Let a > 0 and 7 = inf { t : X(t)-A(r)[-a, a]}, and define Y = A( - A r) -X( AT) +a. Since X is continuous,Y 2 0, and hence by (5.2),(5.17) .[LAY(s-) dA(s)] = Y(s)dA(r)], t 2 0.For 0 5 s 5 I, Y(s-) s a, and hence (5.17) is finite, and(5.18) .[IAr(Y(s)- Y(s-)) dA(s)]0= .[IAt(A@)- A@-)) dA(s)] = 0, t 2 0.Sincea is arbitrary, it follows that A is almost surelycontinuous. 05.4 Corollary Let X be a right continuous, {.W,}-localsubmartingale.Thenthere exists a right continuous, {9,}-adapted, increasing process A satisfyingProperty (b) of Theorem 5.1 such that At s X -A is an {.F,}-local martin-gale.Proof. Let tls T~ S * * * be stopping times such that 7,,+ oo and Xrnis asubmartingale,and let yn = inf { t : X(t)s -a}. Then XraAvnis a submartingaleof class DL, since for any {Ft}-stoppingtime T,(5.19) X"" "(7)A( -n) S X" A "(TAT) < E[XrnAY"(T)ISr].Let A, be the increasing process for XraAyagiven by Theorem 5.1. Then A =limn+mA,,. 06. SQUARE INTEGRABLE MARTINGALESFix a filtration (9,).and assume {S,}is complete and right continuous. Inthis section all martingales, local martingales, and so on are (f,}-martingales,{PJ-Iwal martingales,and soon.A martingale M is square integrable if E[1M(t)I2]< 00 for all t 2 0. A rightcontinuous process M is a local square integrable martingale if there existstopping times T, 4 f l s - * such that 7,,+ 00 as. and for each n 2 I, Mh iM(.hr,,)is a square integrable martingale. Let A denote the collection ofright continuous,squareintegrablemartingales,and let .Alocdenote the collec-tion of right continuous local square integrable martingales. We also need to
  • 88. 6. SQUARE INTEGRABLE MARTINGALES 79define At,the collection of continuous square integrable martingales, andAt,lot, the collection of continuouslocal martingales. (Note that a continuouslocal martingale is necessarily a local square integrablemartingale.)Each of these collections is a linear space. Let r be a stopping time. IfM E A(Alo,,A,, A,, then clearly M =M( A r) E A (Aloc,A,,A,.IOC ).6.1 Propositionmartingale).If M E A(AlOc),then M 2- [MI is a martingale (localProof. Let M E A.Since for c, s 2 0 and t = uo c u, c . . < u, = t +s,(6.1) E[ M (t +S) - M(t)I.F1]= E[(M(t +s) - M(t)) IPI]the result follows by Proposition 3.4. The extension to local martingales isimmediate. 0If M E A(Aloc),then M 2satisfies the conditions of Theorem 5.1 (Corollary5.4). Let ( M )be the increasing process given by the theorem (corollary)withX = M. Then M 2 - ( M ) is a martingale (local martingale). If M EA~(.Mc,loc),then by Proposition 3.6, [MI is continuous, and Proposition 6.1implies [MI has the properties required for A in Theorem 5.1 (Corollary 5.4).Consequently,by uniqueness,[MI = ( M ) (up to indistinguishability).For M,N E Alocwe define(6.2) [ M , N ] = f ( [ M + N , M + N ] - [ M , M ] - [ N , N ] )and(6.3) ( M , N ) = f ( ( M + N , M +N ) - ( M , M ) - ( N , N)).Of course, [ M , N ) is the cross uuriation of M and N , that is (cf.(3.10)),(6.4) [ M , N#t) = lim 1(M(uPi ,) - M(uP)))(N(u:"l,)- N(uf))s-rm kin probability. Note that [ M , M ] = [MI and (M,M ) = ( M ) . The followingproposition indicatesthe interest in these quantities.6.2 Proposition If M , N E A (.,NloC),then MN - [ M , N ) andM N - ( M , N ) are martingales (local martingales).
  • 89. 80 STOCHASTIC PROCESSES AND MUTINGALESProof. Observe that(6.5) M N - [M,N] =&(M +N)’ - [M +N,M + IV]-(M’ - CMI)-(N2-CNI)),and similarlyfor MN - (M,N). 0If (M, N) = 0, then M and N are said to be orthogonal. Note that(M, N) = 0 implies MN and [M,N]are martingales(local martingales).7. SEMICROUPS OF CONDITIONED SHIFTSLet {sf}be a complete filtration. Again all martingales, stopping times,and so on are {.F,)-martingales, {f,}-stopping rimes, and so on. LetY be the space of progressive (i.e., {9,}-progressive)processes Y such thatsup,ELI Y(r)I]c a.Defining(7.1)and JV = { Y E 9:11 Y 11 = 01,then 9/Jv (the quotient space) is a Banachspace with norm 11 * 11 satisfying the conditions of Chapter I, Section 5, that is,(7.1) is of the form (5.1) of Chapter 1 (r= (6,x P : r E [O, a)}).Since there islittlechance of confusion, we do not distinguish between 14 and Y / N .We definea semigroupof operators (.T(s)} on 9by(7.2) .F(s)Y(t)= E[Y(t +s)lS,JBy Corollary 4.4, we can assume (s, t, a)+ f(s)Y(t, o)is a[O,a)x 8-measurable.The semigroupproperty followsby(7.3) w)aS)Y(t)= UECY(t + +4 If ,+“1I$,I= E[Y(t +u +S ) I 9 , ]= Y ( u +S)Y(t).Since(7.4) SUP ECIf‘(s)Y(r)lls sup ECIY(t)lI,f I{Y(s)}is a measurablecontraction semigroupon 9.surablefwith jt ls(~)Idu < 00 and Z E 9’by (5.4) of Chapter l,Integrals of the form W = Ef(u)Y(u)Z du are well defined for Bore1 mea-(7.5)
  • 90. 7. SEMICROUPS OF CONDITIONED SHlRs 81and(7.6)DefineSince ( Y , 2)E .G? if and only if.T(s)Y = Y + Y(u)Zdu, s 2 0,(7.8)s? is the full generator for ( f ( s ) }as defined in Chapter I, Section 5. Note thatthe "harmonic functions", that is, the solutions of J Y = 0, are the martin-gales in 9.7.1 Theorem The operator 2 defined in (7.7) is a dissipative linear operatorwith 910.- d )= Y for all A > 0 and resolvent(7.9) (A -d)-w= 5. e-".Y(s)W ds.The largest closed subspace Y oof 9on which (Y(s)}is strongly continuousis the closure of B(d),and for each Y E Y oand s 2 0,Lm(7.10)Proof. (Cf. the proof of Proposition 5.1 of Chapter I .) Suppose ( Y, Z) E 2.Then(7.11) e-"Y(sHLY - Z)(r)ds= lme-"E[AY(r + s) - Z(r +s)J@,] ds
  • 91. 82 STOCHASTIC PROCESSES AND MARTINGALESThe last equality follows by interchanging the order of integration in thesecond term. This identity implies (7.9), which since $(s) is a contraction,implies d is dissipative.To see that @(A -d )= 9,let W E 9,Y 5: e-"S(s)W ds, and2 = LY - W. An interchangein the order of integration gives the identity(7.12) Z ( r ) Y = [Ae-AJ S ( r +u)W du ds= I"Ae-AJ[ + f ( u ) W du ds,and we have(7.13) l . F ( u ) Z du = Ae-As .T(s +u)W du ds - S(u)W dul= [Ae-As[+ .T(u)W du ds - S(u)W du.Subtracting(7.13) from(7.12) gives(7.14) Y ( r ) Y- f ( u ) Z du = laAe-A*1J(u)W du ds-l 2 e - L y ( u ) ~du t/s +l . ~ t u ) Wdu= Irne-Au.T(u)Wdu - (I - e-")S(u)W du+ T(u)W du = Y,which verifies(7.8) and implies(Y,2)E d.e-"F(s)WdsE B(d)and lirn,..,Il," e-"F(s)W ds= W (the limit being the strong limit in the Banach space 9).If (Y,2)E d,thenIf W E 9,,,then 1and hence 9(.2)c Y o .Therefore 4po is the closure of 9(&.Corollary 6.8 ofChapter 1 gives(7.10). 0
  • 92. 7. SEMlCROUPS OF CONMTIONEO SHIFTS 83The followinglemma may be useful in showing that a process is in S(d).7.2 lemma Let Y, Z , , Z2 E Y and suppose that Y is right continuous andthat Z,(t)s Z,(r) as. for all t. If Y(r)-fo Z,(s)ds is a submartingale, andY(t)- yoZ,(s) ds is a supermartingale, then there exists 2 E 9 satisfyingZ,(t)s Z(t)I, Z,(t) a.s. for all c L 0, such that Y(t)-fo Z(s)ds is a martingale.7.3 Remark The assumption that Y is right continuous is for convenienceonly. There always exists a modification of Y that has right limits (sinceY(t)- PoZ,(s) ds is a supermartingale).The lemma as stated then implies theexistence of Z (adapted to {9c,,}) for which Y(r+) -fo Z(s)ds is an{.%,,)-martingale. Since E[Y(t+)(4F,] = YO) and E[j:+Z(s)dsIS,] =E[j:+E[Z(s)JFJJdsISe,], Y(r)- fo E[Z(s)ISe,Jdsis an (Sc,}-martingale. 0Proof. Without loss of generality we may assume Z , = 0. Then Y is a sub-martingale, and since Y V 0 and (roZ,(s) ds - Y(r))V 0 are submartingales ofclass DL, Y and roZ,(s)ds- Y(t) are also (note that IY(r)Js Y(r)VO+(yoZ2(s)ds - Y(r))VO). Consequently, by Theorem 5.1, there exist right con-tinuous increasing processes A I and A, with Property (b)of Theorem 5.1 suchthat Y - A, and Y(t)- PoZ,(s)ds + A,(t) are martingales. Since Y + A, is asubmartingale of class DL, and Y + A, - (A, + A,) and Y(t)+ A,(f)-PoZ,(s)ds are martingales, the uniqueness in Theorem 5.I implies that withprobability one,(7.16) A&) + A,(f)= Z,(s) ds, t 2 0.Since A, is increasing,A ,(f+ u) - A s Z,(s) ds, t, u 2 0,0ltM(7.17)so A , is absolutely continuous with derivative Z, where 0 s Z s Z,.7.4 Corollary If Y E 9,Y is right continuous, and there exists a constant Msuch that(7.18) I E[ Y(t + s) - Y(t)IS,]I 5 Ms, t. s 2 0,then there exists Z E 9 with lZl s M as. such that Y(r)- SoZ(s)ds is amartingale.Proof. Take Z,(d = -M and Z2(t)= M in Lemma 7.2. 07.5 Proposition Let Y E 9 and let t be the optional projection ofY( +s)ds and q the optional projection of Y(.+ 6) - Y, that is, t(r)=E[lt Y(t +s) ds IS,]and q(t) = E[ Y(t +6)- Y(t)IS,].Then (C, q) E d.
  • 93. 84 STOCHASTIC FROCESSES AND MARTINGALESProof. This isjust Proposition 5.2 of Chapter 1. 07.6 Proposition Let <E 49. If {s- E[t((r+s) - {(t)l4F,]: s > 0, f 2 0)is uni-formly integrableand(7.19) ~ - E [ < ( c+s) - WIS-,]A&) as s-o+, a.e. t,then (C,q) E 2.Proof. Let <,(r) = E - E[ t(t +s) ds IS,] and q,(t) = E - E[t(t +E )-((t) I F,].Then (t,,q,) E d and as E--, 0, C,(t)-+ t(f)and(7.20)in L! for each.t L 0. 0We close this section with some observations about the relationshipbetween the semigroup of conditioned shifts and the semigroup associatedwith a Markov process.For an adapted process X with values in a metric space (E, r), let YU be thesubspace of Y of processes of the form {f(X(t),t)}, whereS EE(E x [0, OD)),and let YUo be the subspace of processes of the form {j(X(t))},f E B(E). ThenX is a Markov process if and only if S(s): .&+YU for all s 2 0, and it isnatural to call X temporally homogeneousif Y(s):A,--,dofor all s 2 0.Suppose X is a Markov process corresponding to a transition functionf(s, x, r),define the semigroup {T(t)]on B(E)by(7.21)and let b denoteits full generator.Then for Y Z ~ OX E A0,(7.22) S(s)Y = T(s)j0 x, s 2 0,and for U;h) E A,(Jo X , h 0 X ) E 2.8. MARTINGALES INDEXED BY DIRECTED SITSIn Chapter 6, we need a generalization of the optional sampling theorem tomartingales indexed by directed sets. We give this generalization here becauseof its close relationshipto the other materialin this chapter.
  • 94. 8. MARTINGALES INDEXED BY DIRECTED SETS 85A set I is partially ordered if some pairs (u, u) E f x 9 are ordered by arelation denoted u < u (or u 2 u) that has the following properties:(8.1) For all u E I, u Iu.(8.2) If u s, u and v 5 u, then u = u.(8.3) If u I u and us w, then II s w.A partially ordered set J (together with a metric p on .P)is a metric lattice if(1,p) is a metric space, if for u, u E 9 there exist unique elements u A u E 9and u V L: E 9 such that(8.4) { w ~ f : w s u }n { w ~ f :wiu} = { W E # : w s u A u ]and(8.5) ( w E .f:w 2 u ) n { w E .f:w 2 u } = { w E 9:w 2 u v o } ,and if (u, u)-+ u A u and (u, u)-+ LC V v are continuous mappings of .f x f onto3. We write min { u , ,...,urn) for u, A * . . A u,, and max { u , ,...,u,} foru , V . .V urn.We assume throughout this section that f is a metric lattice.For u, u E f with u 5 u, the set [u, u] zs { w E f :u 5 w g u} is calledan interval. Note that [u, u] is a closed subset of f.A subset F c f isseparable from above if there exists a sequence {a,) t F such that w =limndmmin {ai: w Ia,, i s n} for all w E F. We call the sequence {a,,} aseparating sequence. Note that F can be separable without being separablefrom above. Define f,,= {u E 9:u 5 u}.Let (0,.F,P)be a probability space. As in the case .f = [O, a),a collection(9,)= {F,,,u E f}of sub-a-algebras of 9 is a jiltration if u 1; u implies9c,,c 9,.and an 3-valued random variable r is a stopping time if ( r s u } E9,for all u E .f.For a stopping time t,(8.6){ A E 9 :P(A)= 01 for all u E .f.9,= { A E 9:A n { r S u ) E 9, for all u E 1).A filtration {F,,}is complete if (a,9,P) is complete and 9,,2Let rz = { v : infw,,p(u, w ) < n-). We say that {P,,]is right continuous ifSee Problem 20 for an alternative definition of right continuity.8.1 Propositionhold :Let r , , r2,. .. be (F,}-stopping times. Then the following(a) maxks,,rkis an IS,}-stopping time.(b) Suppose {9,}is right continuous and complete. If t is an .#-valuedrandom variable and T = r, a.s., then r is an {SE,)-stoppingtime.
  • 95. 86 STOCHASTIC PROCESSES AND MARTINGALESProof. (a) As in the case .# = [O, a),{maxkj,rks u ) = n k i r { ? k 5 u } E SY.(b) By the right continuity,and hence(8.9)8.2 Proposition Suppose r is an {.F,,}-stoppingtime and a E f.Define(8.10)on {T ~ a }Then T* is an (F,)-stopping time.8.3 Remark Note that rais not in generalequal to r A a, which need not be astopping time. aProof. If u =a, 5 u ) = fl E SF,. If Y sa, but u # a , then {r"S u) =0{r s u} c 9,.In general, {r" 5 u } = {r"S u A a ) E 9,,,c 9,.8.4 Proposition Suppose r is an {F,)-stopping time, a E f with T 5 a, andfais separable from above. Let {a,} be a separating sequence for 3, witha, = a, and define(8.11) r, = min {a,: r s a,, i s n}, n 2 1.Then T , is a stopping time for each n 2 1, a = r , 2 T~ 2 ..., andT, = T .Proof. Let F, be the finitecollection of possible values of r,. For u e F, ,(8.12) {r, = u ) = { T 5 u } n n { r su } ~E P,,and in generalusF,nJ,u # u(8.13) {r, s u ) = (J {r, = u } E 9,.The rest followsfrom the definitionofa separating sequence.V P F,nJ,0Let X be an €-valued process indexed by 1.Then X is {91t.)-adaptedifX(u) is 9,-measurable for each u E 9,and X is {F,}-progressive if for eachu E 3, the restriction of X to f,x fl is O(3,)x 9,-measurable. As in the
  • 96. 8. MARTINGALES INDEXED BY OllECTED S n S 87case .f = [0, a),if J, is separable from above for each u E 3,and X is rightcontinuous (i,e., lim,,,X(uVu, w) = X(u, w) for all u E f and w E Q) and(9,}-adapted, then X is {9,}-progressive.8.5 Proposition Let t and d be {f,}-stopping times with r In, and let X be{9,)-progressive. Then the following hold :(a) 9,is a a-algebra.(b) 9,c P,,.(c) If .fu is separable from above for each u E f,then t and X(T)are.F,-measura ble.Proof. The proofs for parts (a) and (b) are the same as for the correspondingresults in Proposition 1.4. Fix a E 1.We first want to show that T* is9,-measurable. Let (an}be a separating sequence for J, with a, = a, anddefine t; = min (a,: T s a,, i g n). Then (8.12) implies r: is SiF,-measurable.Since limn-mt: = t", ro is 9,-measurable, and X(T")is 9,-measurable by theargument in the proof of Proposition 1.4(d). Finally, (t E P) n (T s a} ={T E r)n {T s a) E 9,for all a E f and r z a(f),so {T E P}E 9,and T isF,-measurable. The same argument implies that X(r) is F,-measurable. 08.6 Proposition Suppose {S,}is a right continuous filtration indexed by J.For each t 2 0, let T(C) be an {b,}-stopping time such that s s t impliesr(s)I;t(t)and ~ ( t )is a right continuous function of 1. Let H,= ft,l,,and let qbe an (N,}-stopping time. Then r(q) is an {.%,}-stoppingtime.Proof.(8.14) {W S; u) = u({lr = I,} n {W5 u)).Since {q = t,} E f,,,,,,{q = 1,) n {t(tI)5 u } E 9,.For general q approximateq by a decreasing sequence of discrete stopping times (cf. Proposition 1.3) andapply Proposition 8.I(b). 0First assume q is discrete.ThenIA real-valued process X indexed by J is an {f,}-martinga/e ifE[IX(u)I] < m for all u E J,X is {*,}-adapted, and(8.15)lor all u. u E .fwith u Iu.ECN o )IP U I = X ( U )8.7 Theorem Let f be a metric lattice and suppose each interval in f isseparable from above. Let X be a right continuous (.F,}-martingale. and let r,
  • 97. STOCHASTIC moasss AND MAITINCALESand t 2be (SU}-stoppingtimes with rI S T ~ .Suppose there exist {u,,}, {u,} cJ such that(8.16)and(8.17)lim P{u, s tl 5 t 2s urn) = 1n - mProof. Fix m 2 1 and for i = I, 2 define(8.19)Let {a,) be separating for [u,, urn] with a1 = u,, and define T:, = min (all:rIm<c f k , k n). Note T;, assumes finitely many values, and T;, s rl,,,. Fix n,and let r be the set of values assumed by rl;, and T:,. For a E r with a # urn,{r;,,,= a) =I { r , 5 a} n {rl;,,,= a}, and hence A n {r;,,,= a) = A n {rl s a}n {r;,,,= a} E 9,.Consequently for a E r with a # urn,(8.20) 1 X(r1,) dPA n It;. -4c ECX(UaJ)I9#IXIt;, I 8) dP-I~ n [ t ; ~ - a lp a r= z J X(um) dPp c A n It;. -a) n It;, -#)=I X(u,)dPA n It;, -a)= I X(a)dP.A n It;,, -a;Since slm= u, implies r;, = urn,(8.20) is immediate for a = urn,and summingover a E r,(8.20) implies(8.21)Letting n+ 00 and then m-+ a, gives(8.22)which implies (8.18). 0
  • 98. 9. PROBLEMSI. Show that if X is right (left) continuous and {*,}-adapted, then X is{9,}-progressive.Hint: Fix r > 0 and approximate X on [O,t] x R by X, given byX&s, w) = X(t A ((Cnsl + l)/n). 4.2. (a) Suppose X is E-valued and (9,)-progressive, and / E HE). Showthat f o X is (9,}-progressive and Y(t)E yof(X(s))ds is{9,)-adapted.(b) Suppose X is E-valued, measurable, and {PI}-adapted,and/€ B(E).Show that/. X is {9,}-adapted and that Y(t)z fo/(X(s)) ds has an($,)-adapted modification.Let Y be a version of X.Suppose X is right continuous. Show that thereis a modification of Y that is right continuous.4. Let (E, r) be a complete, separable metric space.(a) Let <,,t., ,... be E-valued random variables defined on (0,9,P).Let A = {a:lim t,,(w)exists}.Show that A E 9,and that for x E E, a random variable (i.e.,is f-measurable).(b) Let X be an E-valued process that is right continuous in probability,that is, for each E > 0 and t 2 0,(9.2)Show that X has a modification that is progressive.Hint: Show that for each n there exists a countable collection ofdisjoint intervals [t;, s:) such that [O, ao) = u[t:, s:) and(9.3) P{r(X(lZ),X(s))> 2-"} < 2-",lim P{r(X(t),X(s))> E } = 0.$-+I +f." s s < s:.5. Suppose X is a modification of Y, and X and Y are right continuous.Show that X and Y are indistinguishable.6. Let X be a stochastic process, and let T be a discrete {9:}-stopping time.Show that(9.4)Let (9,)be a filtration. Show that {9,+)is right continuous.Let (Q, 9,P ) be a probability space. Let S be the collection of equiva-lence classes of real-valued random variables where two random vari-S,"= a(X(tA r):f2 0).7.8.
  • 99. 90 STOCHASTIC PROCESSES AND MARTINCWSables are equivalent if they are almost surely equal. Let y be definedby (2.27). Show that y is a metric on S corresponding to convergence inprobability.9. (a) Let {x,} satisfy supnxn< OD and inf,x, > -00, and assume that foreach a < b either {n: x, 2 b} or {n: x, 5 a} is finite. Show thatlimn.+ x, exists.(b) Verify the existence of the limits in (2.24) and (2.25) and show thatY-(t, a)= Y(r-, a)for all t >Oand w E Q,.10. (a) Suppose X is a real-valued integrablerandom variable on (Q 9,P).Let r be the collection of sub-a-algebras of 9.Show that{ECX 191:43E r}is uniformly integrable.(b) Let X be a right continuous {S,}-martingaleand let S be the collec-tion of {9,}-stopping times. Show that for each T > Oy{ X ( TA T): T E S } is uniformly integrable.(c) Let X be a right continuous, nonnegative {4F,}-submartingale.Showthat for each T > Oy(X(TA7.r):T E Sjis uniformly integrable.11. (a) Let X be a right continuous {~l}-submartingalcand T a finite{IP,}-stopping time. Suppose that for each c > 0, E[sup,,, [X(T+s)-X(r)l] < 00. Show that Y(t)=X(T + t ) -X(T) is an(9,+,1-submattingale.(b) Let X be a right continuous (9,)-submartingale and T , and befinite {.F,}-stopping times. Suppose 5 , 5 I ) and E[sup,(X((r, +s)A T ~ )- X(T,)(]< 00. Show that E[X(r,) -X(T,)I.FJ 2 0.12. Let X be a submartingale. Show that sup,E[lX(r)(] < OD if and only ifSUP, E[X (t)] c= 00.13. Let 4 and < be independent random variables with P{q = I } =P{q = -1) =f.and E[JCJ]= a.Define(9.5)andShow that X is an (..@,}-localmartingale, but that X is not an {sC,X}-localmartingale.14. Let E be a separable Banach space with norm II * 11, and let X be anE-valued random variable defined on (a,9,P).
  • 100. 9. mouws 91(a) Show that for every E > 0 there exist {x,) c E and (6;)c 9 withB; n Bj = 0 for i #j,such that(9.7) xe = C X r X e :and show thatso that one can define(9.10)Extend Theorem 4.2 and Corollary 4.5 to bounded, measurable, E-valued processes.E[X 191 = lim E[X, 191.8 - 0(c)15. Let A , and A, be right continuous increasing processes with A,(O) =A,(O) = 0 and €[A#)] < 00, i = 1, 2, t > 0. Suppose that A , - A2 is an{4C,}-martingaleand that Y is bounded, right continuous, has left limitsat each r > 0, and is {S,}-adapted.Show that(9.I J ) Y(s-1d(AAs) - A m= Y(s-) dA,(s) - Y(s-) dA,(s)is an {F,}-martingale.(The integrals are defined as in (5.1).)Hint: Let(9.12) <(t) = 6 - 1 Y(s)dsand apply Proposition 3.2to(9.13)16. Let Y be a unit Poisson process, and define M(r)= Y(r)- f. Show that Mis a martingale and compute [MIand (M).
  • 101. 92 STOCHASTIC PRocEssEs AND MARTINGALES17. Let W be standard Brownian motion. Use the law of large numbers tocompute [WJ.(Recall[W]= <W)since W is continuous.)18. Let 9 be the space of real-valued (9,)-progressive processes Xsatisfying 1; E[lX(t)12Jdt < 00. Note that 2 is a Hilbert space withinner product(9.14)and norm 11X 11 = d m .Let A be a bounded linear operator on 2".Then A*, the adjoint of A, is the unique bounded linear operatorsatisfying(AX, Y)= (X, AY) for all X , Y E 3.Fix s z 0 and let U(s)be the bounded linear operator defined by(X, Y)= 1ECXO)Y(t)ldr(9.15)What is U+(s)?(Rememberthat U*(s)X must be {PI}-progressive.)19. Let M , , M,. ..,M , be independent martingales. Let 3 = [O, a)"anddefinem(9.16) M(u)= n MXUi), u E #.1- I(a) Show that M is a martingaleindexed by Y.(b) Let 9,= u(M(u):u s u), and let ~ ( t ) ,t 2 0, be {f,}-stopping timessatisfying r(s)5 r(t) for s s t. Suppose that for each t there existsc, E 3 such that ~ ( t )s c, as. Let X,(t)= MXt,(t)).Show that X , , ...,X, are orthogonal IF,,,,}-martingales. More generally, show thatfor any I c {1,. ..,m}, n, I X , is an {.Ft,,,}-rnartingaIe.20. Let 1 be a metric lattice. Show that a filtration { f , ,u E 9)is rightcontinuous if and only if for every u, {u,) c f with u 5 u,, n = 1, 2,. ..,and u = lim,+,., u,, we have Pu= nSaF,.21. (a) Suppose M is a local martingale and ~up,~,IM(s)(E t for eacht > 0. Show that M is a martingale.(b) Suppose M is a positive local martingale, Show that M is a super-martingale.22. Let X and Y be measurable and {q,}-adapted. Suppose€[IX(t)l Po1 Y(s)l ds] < 00 and E[fo IX(s)Y(s)lds] < m for every 2 0.and that X is a (Q,)-martingale.Show that X(r)fo Y(s)ds-Yo X(s)Y(s)dsis a martingale. (Cf. Proposition 3.2 but note we are not assuming X isright continuous.)
  • 102. 10. NOTES 93Let X be a real-valued {$,)-adapted process, with E [ [ X ( t ) ( ]< OD forevery I 2 0. Show that X is a {9,)-martingale if and only ifE[X(r)] = E[X(O)] for every {Y,)-stopping time c assuming only finitelymany values.Let M,,...,M, be right continuous (3,)-martingales, and suppose that,for each I c {I,. ..,n}, n16I M,is also a {YJ-martingale. Let t l . ...,cnbe {Y,}-stoppingtimes, and suppose €[nl=Isupfst,lM~r)l]< OD. Showthat M(r)I n;=,M,(tA T,) is a {YJ-martingale.Hint: Use Problem 23 and induction on n. X be a real-valued stochastic process, and (9,)a filtration. ( X is notnecessarily {F,)-adapted.)Suppose E[X(t)(S,]2 0 for each t. Show thatE [ X ( r ) l F , ] 2 0 for each finite,discrete {9,}-stoppingtime c.Let (M,A, p) be a probability space, and let A, c Azc . .. be anincreasing sequence of discrete a-algebras, that is, for n = 1, 2,. ..,A, =a(A;, i = 1, 2,. ..) where the A; are disjoint, and M = U,A;. LetX E L(p),and define(9.17)(a) Show that (X,) is an martingale.(b) Suppose A = v,,An.Show thatLet (X(t):t E 3 )be a stochastic process. Show thatX , = X pa.s. and in L(,u).(9.18) o(X(s):s E f)= u a(X(s):s E I )I C Jwhere the union is over all countable subsets of $.28. Let T and u be {F,}-stopping times. Show that S,,, = Sfn 9. and9,,,= F,V F , .29. Let X be a right continuous, E-valued process adapted to a filtration{S,}.LetfE C(E)and g, h E B(E),and suppose that(9.19)andrt
  • 103. 94 STOCHASTIC PROCESSES AND MARTINGALESare {FJ-martingales. Show thatis an {s,)-martingale.10. NOTESMost of the material in Section 1 is from Doob (1953)and Dynkin (1961),andhas been developed and refined by the Strasbourgschool. For the most recentpresentation see Dellacherie and Meyer (1978, 1982). Section 2 is almostentirely from Doob(1953).The notion of a local martingale is due to It8 and Watanabe (1965).Propo-sition 3.2 is of coursea special case of much more general results on stochasticintegrals. See Dellacherieand Meyer (1982)and Chapter 5. Proposition 3.4 isdue to Doleans-Dade(1969).The projection theorem is due to Meyer (1968).Theorem 5.1 is alsodue to Meyer. See Meyer (1966),page 122,The semigroupof conditionedshifts appeared first in work of Rishel(l970).His approach is illustrated by Problem 18. The presentation in Section 7 isessentially that of Kurtz (1975). Chow (1960)gave one version of an optionalsampling theorem for martingales indexed by directed sets. Section 8 followsKurtz (1980b).Problem 4(b) is essentially Theorem 11.2.6 of Doob (1953). See Dellacherieand Meyer (1978),page 99, for a more refined version.
  • 104. 3In this chapter we study convergence of sequences of probability measuresdefined on the Borel subsets of a metric space (S. d) and in particular ofD,[O, a),the space of right continuous functions from [O, 00) into a metricspace (E,r) having left limits. Our starting point in Section I is the Prohorovmetric p on 9(S),the set of Borel probability measures on S,and in Section 2we give Prohorovs characterization of the compact subsets of SyS). in Scction3 we define weak convergenceof a sequence in 9 ( S )and consider its relation-ship to convergence in the Prohorov metric (they are equivalent if S isseparable). Section 4 concerns the concepts of separating and convergencedeterminingclasses of bounded continuousfunctions on S.Sections 5 and 6 are devoted to a study of the space Dc[O,co) with theSkorohod topology and Section 7 to weak convergence of sequences inP(DEIO,m)). In Section 8 we give necessary and suflicient conditions in termsof conditional expectations of r8(X,(c + u), X,(r))A 1 (conditioningon S,".)fora family of processes {X,) to be relatively compact (that is, for the family ofdistributionson DEIO,GO)to be relatively compacl). Criteria for relative com-pactness that are particularly useful in the study of Markov processes aregiven in Section 9. Finally, Section 10 contains necessary and sufficient condi-tions for a limiting process to have sample paths in C,[O, 00).95CONVERGENCE OFPROBABILITY MEASURESMarkov Processes Characterizationand ConvergenceEdited by STEWARTN. ETHIER and THOMASG.KURTZCopyright 01986,2005 by John Wiley & Sons,Inc
  • 105. 1. THE PROHOIOV MLTRlCThroughout Sections 1 4 . (S, d)is a metric space (d denoting the metric), @(S)is the a-algebra of Borel subsetsof S, and N S ) is the family of Borel probabil-ity measureson S.We topologizeWS)with the Prohorov metric(1.1)where Y is the collectionofclosed subsetsof Sandp(P, Q)= inf {E > 0: P(F)5 Q(Fc)+E for all F E (B),F = x E S: inf d(x, y) <E .To see that p is a metric, we need the following lemma.{ y 8 F I(1.2)1.1 Lemma Let P, Q E 9 ( S )and u, fl >0. If(1.3)(1.4)W )5 Q(Fa)+ PQ(F)s P(Fa)+Bfor all F E W,thenfor all F E W.Proof. Given FI E 6, let F, = S -fl,and note that F2E 6 and F,c S -q.Consequently,by (1.3) with F = F2,(1.5)implying(1.4) with F = F1.OF;)= 1 - P(F2)2 1 -Q(F9-B 2 Q(F,) -P,CIIt follows immediately from Lemma 1.1 that p(P, Q) = p(Q, P) for allP, Q E qS). Also, if p(P, Q)=0, then P(F) = Q(F) for all F E 6 and hence forall F E g(S); therefore, p(P, Q)=0 if and only if P = Q. Finally, ifP, Q,R t SYS),p(P, Q)< 4and AQ, R) <E, then(1.6) P(F) <Q(F? +S < Q(F)+6S R((FY)+ 6 +E 5 R(F+*)+S +Efor all F E W, so p ( f , R) s S +E, proving the triangle inequality.rov metric when Sis separable.The following theorem provides a probabilisticinterpretation of the Proho-1.2 Theorem Let (S, d) be separable,and let P,Q E P(S).Define d ( P , Q)tobe the set of all p E SyS x S) with marginals f and Q (i.e., p(A x S) = P(A)and p(S x A) = Q(A)for all A E @(S)). Then(1.7) p(P, Q)= infc 4 p . 0)inf {e > 0: pc((x,y): d(x, y) 2 E ) 5 E}.
  • 106. 1. WE PROHOROV MnRlC 97Proof. If for some E > 0 and p E M P , Q)we have(1.8) p{(x, y): 4 x . Y ) 2 6 ) s 6,then(1.9) P ( 0 = p(F x S)5 P((F x S) n {(x, y): 4x9 Y ) < 4)+ &5 p(S x P)+E = Q(F) + Efor all F E W, so p(P, Q)is less than or equal to the right side of(1.7).The reverse inequality is an immediateconsequenceof the followinglemma,01.3 lemma Let S be separable. Let P, Q E B(S), p(P, Q)< 6, and 6 > 0.Suppose that El.. ..,EN E $i?(S) are disjoint with diameters less than 6 andthat P(E,) 5 6, where E, = S - uT=,I&. Then there exist constants c,,. ..,cN E [O, I] and independent random variables X , Yo,...,YN (S-valued)and (([O, I]-valued) on some probability space (Q, 9,v) such that X has distribu-tion P, C: is uniformly distributed on [O, I],1 4 on ( X ~ E , , t r c , } , i = I ,..., N,has distribution Q,(1.11) {d(X. Y) z 6 +E ) c { X E E,} uand(1.12) v{d(X, Y) 2 6 +E } I; 6 +E.The proof of this lemma dependson another lemma.1.4 lemmaA, E a ( S )for i = I,. ..,n. Suppose that(1.13) 1 pi 5 p(;, A,) for all I c (1.. ..,n).Then there exist positive Borel measures A,, ...,I, on S such that IAA,) =A,@) = pi for i = I,. ..,n and I;=,1,(A)s p(A)for all A E a(S).Let p be a finitepositive Borel measure on S,and let pi 2 0 andi c lProof. Note first that it involves no loss of generality to assume that eachWe proceed by induction on n. For n = I, define 1, on @(S) by A1(A)Ipi > 0.
  • 107. 98 CONVERGENCE OF PROIMILIM W l J R E SpIC((An Al)/C((Al). Then JAAl) = A,@) = p,, and since pl s AA,) by (1.131,we have A,(A) s p(A n A,) S p(A) for all A E @S). Suppose now that thelemma holds with n replaced by m for m = I,. ..,n - 1 and that p, pi, andA, (1 S i 5 n) satisfy (1.13). Define q on &(S) by q(A)= p(A n A,)/p(A,), andlet E, be the largest E such that(1.14) forall I c { I , ...,n - I}.I P ICASE1. E, L p.. Let 1, -p.q and put p = p -A,. Since p. s p(A.) by(1.13), p is a positive Borel measure on S, so by (1.14) (with E = p,) and theinduction hypothesis, there exist positive Borel measures A,, ...,A,,-, on Ssuch that l,(Ai)= 1,(S) = pi for i = 1,. ..,n - I and I;:,L,(A)5 $(A) for allA s 9(S).Also, A,@,) = A,@) =p,,, so A,, ...,1, have the required properties.CASE2. E, < pm.Put p = p - ~ , q ,and note that p is a positive Borelmeasure on S. By the definition of E,, there exists I , c (I,.. .,n - 1)(nonempty) such thatwith equality holding for I = Io. By the induction hypothesis, there existpositive Borel measures Ai on S, i E I,, such that AAA,) = 11s)=p, for eachi f I . and zlclo AXA) 5 PYA) for all A E @S). Let p; =p, for i = I,. ..,n - 1and p l = p, -e0. Put Bo = U,.IOAi, define p" on g(S) by p"(A) = p(A)- p(A n Bo),and let I , = (I ,...,n) - I,. Then, for all I c I,,Here, equality in the fint line holds because equality in (1.15) holds for I -- I,,while the inequality in the second line follows from (1.14) if n 9 I and from(1.13) if n E I; more specifically,if n E I, then(1.17)
  • 108. 1. THE PROHOROV M R l C 99By (1.161,(1.18) c P; e&J/ A,) for all I c 11,i r Iso by the induction hypothesis, there exist positive Borel measures A; on S,ielI,such that &(Ai) = &(S) = pi for each i~ I , and ~ i G l l X ~ A )5 $‘(A) for allA E a(S). Finally, let A, = A; for i e I, - ( n ) and A, = A; +con. Then AAA,) =Ai(S)= pi for each i E I,, hence for i = 1,. ..,n, and(1.19)i - I i e lo l C l l= 1A,(A n B,) + C R;(A) +E,)~(A)s PYA n B,) +p”(A)+eOrt(A)= PYA) + Eo MA)= P ( 4for all A E .Sa(S), so A,,...,An again have the required properties.i r l o i C l l01.5 Corollaryand A, E a ( S )for i = 1,. ..,n. Let E > 0, and supposethatLet p be a finite positive Borel measure on S,and let p, 2 0p i S p u A, + E for all I c {I, ...,n}.LrI )(I .20)i C lThen there exist positive Borel measures A,, ...,An on S such that A,(A,) =A A S ) s p i for i = 1, ..., n, ~ ; = l A , ( S ) r ~ ; , l p , - ~ ,and C;=lA,(A)sp(A) forall A E 9?(S).Proof. Let S’ = S u {A}, where A is an isolated point not belonging to S,Extend c( to a Borel measure on S’ by defining p((A})= E. Letting A; =A, u {A} for i = 1,. ,.,n, we have(1.21)By Lemma 1.4, there exist positive Borel measures A’,, ...,A; on S such thatU A ; )= Al(S’)= pi for i = 1,. ..,n and CY-I AKA) 5 p(A) for all A E 9?(S).LetAi be the restriction of 2; to .Sa(S) for i = 1,. ..,n. Then Ai(Ai) = &(A,) 5A;(A;)= pi and A,(S - A,) = A;(S- A;) = 0 for i = I,. ..,n. Also,
  • 109. 100 CONVERGENCE OF PROMIlUTY MEASURESProof of lemma 1.3 Let P,Q,&,a, and E,, ...,EN be as in the statement of thelemma. Let p, = P(E,) and A, = Effor i = I,. ..,N.Then(1.23) Z p f S P + e forall I c { l , ...,Nj,1 6 1so by Corollary 1.5, there exist positive Bore1 measures A,, ...,AN on S suchthat AAA,) = A@) I;pffor i = 1,. ..,N,(1.24)I = I I - Iand cya,AAA) S Q(A) for all A cd?(S). Define cI,...,cNE [O, 13 byc, = (p, - A,(S))/p,, where 0/0 = 0, and note that (1 -c,)P(EI)= A,(S) fori = I, ...,N and f(Eo)+crN.p,cfP(E,)= 1 -zrNIlA,@). Consequently, thereexist Qo, ...,QN E 9(S)such that(1.25)andQ,(B)(l -c,)P(E,)= Ads), i = 1,. ..,N,N NI = 1 i=1(1.26)for all B E 4?(S).Let X,Yo,...,YN, and t be independent random variables on some prob-ability space (Q 9,v) with X,6,...,YN having distributions P, Qo,...,Q Nand t uniformly distributed on [O, 13. We can assume that Yl,...,YN takevalues in A,, ...,AN, respectively.Defining Y by (l,lO), we have by (1.25) andQd&( P(Ed + ci 4EJ) = Q(B)- c JAB)(1m,(1.27)NV { Y E B ) = C Qi(BM1 -C P O i )i = 14+PdB)(P(Ed+ I -c1 CIP(EN= Q(4for all B 8 5o(S). Noting that {XE: El, <2 cf}c (XE El, Y E Af} c(d(X,Y)e 6 +E } for i = I,. ,.,N,we haveN(1.28) {d(X, Y)2 6 -tE } c {XE Eo} u u (XB El, C: <c,}I - 1
  • 110. I. THE mottoaov m i c 101where the third containment follows from p, - AXS)5; E for i = 1,. ..,N (see(1.24)). Finally, by the first containment in (1.28) and by (1.24),(1.29)Nv(d(X, Y)2 8 +E } 5; P(E,) + ciP(E,)I - IN= P(E0)+ c (Pi - W))i = lS d + E . 01.6 Corollary Let (S,d) be separable.Suppose that X,, n = I, 2,. ..,and Xare S-valued random variables defined on the same probability space withdistributions P,, n = I,2,. ..,and P, respectively. If d(X,, X)-+0 in probabil-ity as n -+ 00,then limn-.s,p(Pm,P)= 0.Proof. For n = 1, 2,. .., let p, be the joint distribution of X, and X . Thenlim,,,p,{(x, y): d(x, y) 2 E } = 0 for every E > 0, so the result follows fromTheorem 1.2. 0The next result shows that the metric space (P(S),p) is complete and sepa-rable whenever (S, d) is. We note that while separability is a topological pro-perty, completenessis a property of the metric.1.7 Theoremcomplete,then (B(S),p) is complete.If S is separable,then 9 ( S ) is separable. If in addition (S, d) isProof. Let {x"} be a countable dense subset of S, and let 6, denote theelement of P(S)with unit mass at x E S.We leave it to the reader to show thatthe probability measures of the form cr=a,S,, with N finite,a, rational, andcr=I ai = I, comprise a dense subset of B(S)(Problem 3).To prove completenessit is enough to consider sequences {P"} c 4yS)withp(P,- P")< 2-" for each n 2 2. For n = 2, 3,. ..,choose E?,. ..,E$i E mS)disjoint with diameters less than 2-" and with P,-,(lit)s 2-", where lib" = S- urz1EY1. By Lemma 1.3, there exists a probability space (Q, 9,v) onwhich are defined S-valued random variables Yf,.,.,YE!,n = 2, 3,. ..,LO, I]-valued random variables <",n = 2, 3,. ..,and an S-valued random variableXI with distribution PI,all of which are independent, such that if the con-stants .,",...,CEE LO, 1], n = 2, 3,. .., are appropriately chosen, then therandom variable
  • 111. 102 CONVERGENCE OF fROIAMLlM MEASUREShas distribution P,and(1.31)successivelyfor n = 2,3,. ...By the Borel-Cantelli lemma,V { d ( X , - I , X,)2 2-,+) 5 2--"+,(1.32)so by the completeness of (S,d), limn-mX, =X exists as. Letting P be thedistribution of A, Corollary 1.6 implies that limn*mp(P,, P)= 0. aAs a further application.of Lemma 1.3, we derive the so-called Skorohodrepresentation.1.8 Theorem Let (S,d) be separable. Suppose P,, n = 1, 2,. .., and P in9 ( S ) satisfy limn-,mp(P,, P)= 0. Then there exists a probability space(Q, 9,w) on which are defined S-valued random variables X,, n = I, 2,. ..,and X with distributions P,, n = I, 2,..., and P, respectively, such thatlimnemX,= X as.Proof. For k = 1, 2,. ..,choose E,), ..,,E$i E wS)disjoint with diametersless than 2- and with P(l$) s 2-, where li$ = S -U;lll$), and assume(without loss of generality) that && E minIs,,,P(Ej") > 0. Define thesequence {k,) by(I .33)and apply Lemma 1.3 with Q = P,, e = ek,/k, if k, > I and E = p(P,,, P) + I/nif k, = 1, 6 = 2-", El = and N = N, for n = 1, 2,. ., .We conclude thatthere exists a probability space @,S,v) on which are defined S-valuedrandom variables Yg),...,Y& n = 1, 2,. ..,a random variable < uniformlydistributed on [O, 11, and an S-valued random variable X wifh distribution P,all of which are independent, such that if the constants c,"),...,cK!E [O, 11,n = 1,2,. ..,are appropriately chosen, then the random variablehas distribution P,,and(13)
  • 112. 2. ?ROHOROV’S THEOREM 103for n = 1, 2,. ... If K,, E min,,, k, > 1, then00s c V { X € E $ ’ } + V ( < -k = K . in]and since limn-.m K, = 00, we have limn-mX,, = X a.s. 0We concludethis section by proving the continuousmapping theorem.1.9 Corollary Let (S,d) and (S’, d’) be separable metric spaces, and leth: S4S’be Borel measurable. Suppose that P,,, n = I, 2,.. ., and P in 9 ( S )satisfy limn-00p(P,, P) = 0,and define Q,,n = I, 2,. ..,and Q in 9(S)by(1.37) Q,,= P,h-’, Q = Ph-’.(By definition, Ph-’(B) = P{s E S: h(s)E B}.) Let ch be the set of points of S atwhich h is continuous. If f(C,)= 1, then limn-mp’(Q,,,Q) = 0, where p‘ is theProhorov metric on 9(S’).Proof. By Theorem 1.8,there exists a probability space (a,9,v) on which aredefined S-valued random variables X,, n = 1, 2,. ..,and X with distributionsP,,. n = I, 2,. .., and P, respectively, such that limn*mXn= X a.s. Sincev { X E c h f = I, we have h(X,) = h(X) a.s., and by Corollary 1.6, thisimplies that p’(Qn,Q)= 0. 02. PROHOROV’S THEOREMWe are primarily interested in the convergenceof sequencesof Borel probabil-ity measures on the metric space (S,d). A common approach for verifying theconvergence of a sequence {x,} in a metric space is to first show that (x,,) iscontained in some compact set and then to show that every convergent sub-sequence of (x,) must converge to the same element x. This then implies thatx,, = x. We use this argument repeatedly in what follows, and, conse-quently, a characterization of the compact subsets of 9 ( S ) is crucial. Thischaracterization is given by the theorem of Prohorov that relates compactnessto the notion of tightness.A probability measure P E 9 ( S ) is said to be tight if for each E > 0 thereexists a compact set K c S such that P(K)L 1 - E. A family of probabilitymeasures .M c 9(S)is right if for each E > 0 there exists a compact set K c S
  • 113. such thatinf f ( K )2 1 -e.P S . U2.1 Lemma If (S, d)is completeand separable,then each P E 9(S)is tight.Proof. Let {xk} be dense in S, and let P E HS). Given E > 0, choose positiveintegers N,,N3,...such thatEp(kcl B(Xb i))2 I --2"for n = I, 2,.... Let K be the closure of nnzI u:: ,B(x,, l/n). Then K istotally bounded and hence compact,and2.2 Theorem Let (S, d) be complete and separable,and let & c 9(S). Thenthe followingare equivalent:B A is tight.fb) For each E > 0, there existsa compact set K c Ssuch that(2.4) inf P(K) 2 1 -E.P a . 4where K is as in (1.2).(c) .&is relatively compact.Proof. (a *b) Immediate.(b9 c) Since the closure of A is complete by Theorem 1.7, it is SUE-cient to show that .& is totally bounded. That is, given S > 0, we mustconstruct a finite set M c B(S)such that .# c {Q:p(P, Q)< 6 for someLet 0 < E < 6/2 and choose a compact set K c S such that (2.4) holds.By the compactnessof K,there exists a finite set ( x I , ...,x,) c K such thatK c u;-,B,, where Bi = E(x,, 28). Fix xo E S and m 2 n/E, and let Jf bethe collectionof probabilitymeasuresof the formP E Mj.where 0 5 k, 5 m and c;=,,k, = m.Given Q E A, let k,= [mQ(E,)] for i= I,...,n, where E, = 8,
  • 114. 1. -0HOROVS THEOREM 105- u;: S,, and let ko = m -c;i I k,.Then,defining P by (2.5),we havefor all closed sets F c S,so p(P, Q) 5 2~ < 6.(c 3a) Let E > 0. Since is totally bounded, there exists for n = I,2,. ..a finite subset N,c AY such that d c {Q:p(P, Q)< .~/2"~for someP E N,}.Lemma 2.1 implies that for n = I, 2,. ..we can choose a compactset K, c S such that P(K,) z 1 - ~/2"" for all P E Jv,. Given Q E ..M, itfollowsthat for n = 1,2,. .. there exists P,E N,such thatLetting K be the closure of f l , 2 1 K ~ 2 n i ,we conclude that K is compactandn2.3 CorollaryA is relativelycompact.Let (S,d) be arbitrary, and let A c *S). If A is tight, thenProof. For each m 2 I there exists a compact set Kmc S such that(2.9)1inf P(K,) 2 1 - -,P * . M mand we can assume that K ,c K, c * . For every P E d and m L 1, defineI"m)E SyS)by P")(A) = P(A n K,)/f(K,). and note that I"" may be regard-ed as belonging to 9(Km). Since compact metric spaces are complete andseparable,M")= {Pen: P E ,rV) is relatively compact in cP(K,) for each m 2 1by Theorem 2.2.
  • 115. 106 CONVERGENCE OF PROBABILITY MEASURESWe also have(2.12) P(A) 2 P(K,)P"(A) 2 (1 --!)P"yA),mfor all P E A,A E 4?(S), and m 2 1. By (2.101,Im(2.14)for all P E d a n d m T 1. Given A , , A , , ...~~(S)disjoint,(2.13)and(2.11)imply that(2.15)p(f, P"))s -GIP"(Ai) - P ( A i ) 1I2 2 4ms;m+;=-for every P E d and m > m 2 1.Let {fm}c A. By the relative compactness of &" in flK,,,), there exists(through a diagonalization argument) a subsequence {P,,,}c {P,,} andQ")E P(K,,,)such that(2.16)for every m 2 1. It follows that(2.17)for all closed sets F c S and m 2 1, and therefore the inequalities (2.11) and(2.13) are preserved for the QIm) for all closed sets A c S,hence for all A E WS)(using the regularity ofthe Q")).Consequently.we have (2.15) for the Q("),so(2.18) Q(A)s lim Q("(A)m-OD
  • 116. 3. WEAK CONVERGENCE 107exists for every A E a(S)and defines a probability measure Q tz @(S) (thecountableadditivity followingfrom (2.15)). But(2.19) P(Pn,* Q)5 P(Pn,, PP) + P(C:~Q"")+ P(Q", Q)for each k and m 2 1, implyingthat limk-,mAP,,,. Q)= 0. 0We conclude this section with a result concerning countably infiniteproduct spaces.2.4 Proposition Let (Sk, dk), k = I, 2,.. ., be metric spaces, and define themetric space (S, d ) by letting S = nL"-Sk and d(x, y) = z?=I 2-(dk(x,, y k ) A1 ) for all x, y E S. Let {Pa} c P(S)(where a ranges over some index set), andfor k = 1, 2,. ..and each a, define Pd E p(&)to be the kth marginal distribu-tion of P, (i.e., p". = Pan;, where the projection 1[k: S-r Sk is given byq(x) = xk). Then {Pa}is tight if and only if (ft}is tight for k = I, 2,. ...Proof. Suppose that {e}is tight for k = 1, 2,. ..,and let E > 0. For k = I,2,. .., choose a compact set Kk c Sk such that inf,Pt(Kk)2 I - 42. ThenK =np=K, = ()?=I n; I(&) is compact in S, and(2.20)for all a. Consequently, {P,}is tight.compact in S, andThe converse follows by observing that for each compact set K c S, q(K) is(2.21)fork= 1,2, ... .inf Pt(nk(K))2 inf P,,(K)(I U03. WEAK CONVERGENCELet C(S) be the space of real-valued bounded continuous functions on themetric space (S, d ) with norm 11/11 = sup,,,I/(x)l. A sequence {P,,}c g(S) issaid to converge weakly to P E @(S) iflim fdP, = J dP, f E C(S).The distribution of an S-valued random variable X,denoted by P X - I , is theelement of SyS) given by PX-(B) = P{X E B). A sequence {X,} of S-valued(3.1) n-m s 5
  • 117. 108 CONVERGENCE OF PROMIlLlfV MEASURESrandom variables is said to conuerge in distribution to the S-valued randomvariable X if {PX;)convergesweakly to PX-,or equivalently, if(3.2)Weak convergence is denoted by P,*P and convergence in distribution byX,*X. When it is useful to emphasize which metric space is involved, wcwrite P,=+ P on S" or "X,,=+ X in S".If S is a second metric space and/: S-+ S is continuous, we note that thenX,-X in S impliesf(X,)=./(X)in S since g E c(S)implies g 0 /E c(S). Forexample, if S = C[O, 13 and S = W, thenf(x) 3 supo,,, x(r) is continuous, soX, *X in C[O, I] implies S U ~ ~ ~ , ~X&)* S U ~ ~ ~ , ,X(t) in R. Recall that, ifS = R, then (3.2)is equivalent toIim ECf(xJI= ECf(X)I,n-ooJE CCS,.(3.3) lim P{X, r; x} = PIX s x}n-mfor all x at which the right side of (3.3)is continuous.We now show that weak convergence is equivalent to convergence in theProhorov metric. The boundary of a subset A t S is given by dA = A n (Aand A denote the closureand complement of A, respectively).A is said to be aP-continuity set if A e 9 ( S )and P(dA) = 0.3.1 Theorem Let (S, d) be arbitrary, and let {P,,}c f i S ) and P E 9(S). Ofthe following conditions, (b) through (f) are equivalent and are implied by (a).If S is separable,then all six conditions are equivalent:lim,,, P)= 0.Iim,,-- jJdP,= fdP for all uniformly continuousfc c(S).limn-- P,,(F)5 P(F)for all closed sets F c S.P,(G) 2 P(G)for all open sets G c S.limn-* PdA) = P(A)for all P-continuitysets A c S.P, 9 P.-
  • 118. 3. WEAY CONVERGENCE 109Consequently,-lim J (II/II +/)dPn 5 J (II/II + / ) d ~ ,n-m(3.6)iiii~ ( I I ~ I I-ndPa (II/II - n d pa-+ m Ifor allfe e(S), and this implies(3.1).(b*c) Immediate.(c =5 d) Let F c S be closed. For each E > 0,define/, tz c(S)by(3.7)where d(x, F) = i d y eFd(x,y). Thenfis uniformly continuous, so(3.8)for each E > 0, and thereforelim Pa(F)5 lirn $, dPa = 1;dP,I II-n-m n - * ~lirn Pn(F)s lim f, dP = P(F).(d el For every open set G c S,-a-tw 8 - 0(3.9)-(3.10) -lirn Pa(G)= I - lirn fa(G) 2 I - f(G) = P(G).a - Q a- w(c -f) Let A be a P-continuity set in S,and let A" denote its interior(A" = A - aA).Then(3.1I) tim Pn(A)s limPdJ)= 1 - clirn PA,$)s 1 - P($) = P(A)and(3.12) -lirn Pn(A) Lm P,,(A") 2 P(A")= P(A).a-m n-+m n+ma-m a- w(f-b) Let/€ C(S) w i t h J r 0 . Then d{fr t ) c { j =t ) , so (frr } is aP-continuityset forall but at most countablymany t 2 0.Therefore,(3.13) lim 1f dP, = !:I["Pa{/> t}drI1 1 IIn-m= P { f r t } dt = s / d Pfor all nonnegative/€ C(S),which clearly implies(3.1).
  • 119. 110 CONVERGENCE OF PRODAUUN MLASMES(c=a, assuming separability) Let E > 0 be arbitrary, and let E l , E, ,...E g(S) be a partition of S with diameter(E,)< ~ / 2for i = 1, 2,. ...Let N bethe smallest positive integer n such that P(u;slE,) > I - ~/2,and let Y bethe (finite) collection of open sets of the form (U,clE,)112, whereI c { 1,. ..,N}.Since Y is finite, there exists no such that P(G)s P,(G) +~ / 2forallGEYandnrno.GivenFE(81et(3.14) F, = u{Ei: 1 S i -< N, El A F # fa).Then FZ2 E Y and(3.15) P(F) s P(Ffl) +~ / 2s Pa(F y )+E 5 Pa(Fa)+Efor all n 2 no. Hence p(Pa,P) s E for each n 2 no. 03.2 Corollary Let Pa,n = I, 2,. ..,and P belong to qS),and let S E A?(S).For n = 1, 2,. .., suppose that P,(S) = P(S) = 1, and let P: and P be therestrictions of Pa and P to @(S)(of course, S has the relative topology).ThenPa-P on S if and only if Pi =aP an S.Proof. If G is open in S, then G= G n S for some open set G c S.There-fore, if Pm=+ P on S,(3.16) -lim P:(G) =Lm P,(G) 2 P(G) = P(G),a--m a-ooso PL -P on S by Theorem 3,I. The converseis proved similarly. D3.3 Corollary Let (S, d) be arbitrary, and let (X",9,n = I, 2,. ..,and X be(S x S)- and S-valued random variables. If X,+X and d(X,, G)-+0 in prob-ability, then U, X.3.4 Remark If S is separable, then @(S x S) = 9(S) x i3(S), and hence(X, Y)is an (S x S)-valued random variable whenever X and Y are S-valuedrandom variablesdefined on the same probability space. This observation hasalready been used implicitly in Section 1, and we use it again (without0mention)in later sections of this chapter.Proof. Iff€ C(S)is uniformly continuous, then(3.17) lim Eu(X,) -f(Y,,)] = 0.a-+mConsequently,(3.18)and Theorem 3.1 is again applicable.lim ET/(U,)l = lim Ec/(Xa)] = EU(W1,n-m n-m0
  • 120. 4. SEPARATING AND CONVERGENCE DETERMINING SETS 1114. SEPARATING AND CONVERGENCE DETERMINING SETSLet (S, d ) be a metric space. A sequence IS,)c B(S) is said to convergeboundedly and pointwise to / E B(S)if sup, JIf, II < 00 (where 11 . II denotes thesup norm)and limn+mfn(x)= f ( x ) for every x E S;we denote this bybp-lim f , =fn-mA set M c B(S) is called bp-closed if whenever {jn}c M./E gS),and (4.1)holds, we have /E M . The hp-closure of M c B(S) is the smallest bp-closedsubset of B(S) that contains M. Finally, if the bp-closure of M c B(S)is equalto B(S), we say that M is bp-dense in B(S). We remark that if M is bp-dense inB(S) and fE B(S). there need not exist a sequence {h}c M such that (4.1)holds.4.1 lemmasubspace.If M c B(S) is a subspace, then the bp-closure of M is also aProof. Let H be the bp-closure of M. For eachfe H,define(4.2)and note that H , is bp-closed because H is. Iff€ M, then H, 3 M, so H , =H. Iff E H, then fE H, for every g E M,hence g E H, for every g E M, andH , = {g E H : af +bg E H for all a, b E Oa},therefore H , = H. 04.2 Proposition Let (S, d) be arbitrary. Then e(S)is bp-dense in B(S).If S isseparable, then there exists a sequence {f"}of nonnegative functions in C(S)such that span {I.}is bp-dense in B(S).Proof. Let H be the bp-closure of c(S). H is closed under monotone con-vergence of uniformly bounded sequences, H is a subspace of E(S) by Lemma4.1, and zGE H for every open set G c S. By the Dynkin class theorem forfunctions (Theorem 4.3 of the Appendixes),H = B(S).If S is separable, let {xi} be dense in S. For every open set G c S that is afinite intersection of !?(xi, I/&), i,k L 1, choose a sequence (If)of nonnegativefunctions in 4s) such that bp-lim,+,f~ = xG. The Dynkin class theorem for0functions now applies to span {ft:n, G as above}.For future reference,we extend two of the definitions given at the beginningof this section. A set M c B(S) x B(S) is called bp-closed if whenever{CJn. 9,))c M ,(f, 8) e 4s)x &S), bp-limn-mfw =1; and bp-lim,+,g, = g, wehave (f, g) E M. The bp-closure of M c B(S) x B(S) is the smallest bp-closedsubset of E(S) x E(S)that contains M.
  • 121. 112 CONVERCENCEOF nowtun MEASURESA set M c 4s)is called separating if whenever P,Q E P(S)and(4.3) I/dP = JfdQ, J E M,we have P = Q.Also, M is called conoergence determining if whenever {P,,}cSyS),P E 9(S),and(4.4)we have P,, OD P.f dP = f dQ isbp-closed. Consequently, Proposition 4.2 implies that c(S) is itself separating.It follows that if M c g(S)is convergencedetermining, then M is separating.The converse is false in general, as Problem 8 indicates. However, if S iscompact, then 9 ( S ) is compact by Theorem 2.2, and the following lemmaimplies that the two conceptsarc equivalent.Gives P, Q E 4yS), the set of all fc B(S) such that4.3 Lemma Let {P,,}c 9 ( S ) be relatively cornpact, let P E as),and letM c c(S)be separating.If (4.4) holds, then P,,=+ P.Proof. If Q is the weak limit of a convergent subsequenceof {P,,),then (4.4)0implies(4.3), so Q = followsthat P,,*P.4.4 Proposition Let (S, d ) be separable. The space of functionsfE c(S)thatare uniformly continuous and have bounded support is convergence determin-ing. If Sis also locally compact, then C,(S), the space of/€ c(S) with compactsupport,is convergencadetermining.Proof. Let {x,} be dense in S, and defineh, E c(S) for isj = 1,2,. s,I(x) = 2( 1 -jd(x, xi)) V 0.Given an open set G c S,defineg,,, E M for m = I, 2,. g,,,(x) =(xh,(x))A1, where the sum extends over those i, j s m such that B(x,, l/j) c G (andax,, 10)is compact if S is locally compact).If (4.4) holds, then-lim P,(G) 2 lim /a,,, dP,, = [gm dP(4.6)for m = 1, Z..,,so by letting m+ 00, we conclude that condition (e) ofTheorem 3.1 holds. 0r - r m n-r mRecall that a collection of functions M c c(S) is said to separate points iffor every x, y E S with x # y there exists h E M such that h(x)# Yy). In
  • 122. 4. SRAMTINC AND CONVERGENCE DFl€IMMINc SETS 113addition, M is said to strongly separate points if for every x E S and S > 0there exists a finite set {h,, ...,h k ) c M such that(4.7)Clearly,if M strongly separates points, then M separatespoints.4.5 Theoremalgebra.Let (S, d ) be complete and separable, and let M c c(S) be an(a) If M separates points, then M is separating.(b) If M strongly separates points, then M is convergencedetermining.Proof. (a) Let P, Q E qS), and suppose that (4.3) holds. Thenh dP = h dQ for all h in the algebra H = {f+a:/€ M , a E R}, hencefor all h in the closure (with respect to 11 * 11) of H.Let g E c(S)and E > 0 bearbitrary. By Lemma 2.1, there exists a compact set K c S such thatP(K) 2 1 - E and Q(K)2 I -E. By the Stone-Weierstrass theorem, thereexists a sequence {g,} c H such that supxr Ig,(x) - g(x)l-+ 0 as n--r OD.Now observe thatfor each n, and the fourth term on the right is zero since g,e-h belongs tothe closure of H.The second and sixth terms tend to zero as n -+ OD, so theleft side of (4.8) is bounded by 4y&, where y = sup,., w-". Letting E-+ 0,
  • 123. 114 CONVERGENCE OF PROBMIUTV MUSUREit follows that I g d P = fgdQ. Since g E 4s)was arbitrary and C(S) isseparating, we conclude that P = Q.(b) Let (P,} c P(S)and P E: *S), and suppose that (4.4) holds. ByLemma 4.3 and part (a),it sufficesto show that {P,,)is relatively compact.Letfl ,...,& E M.Thenlim g 0 uI,...,j;)dPn = Jg 0 CT~,..., s , ) ~ Pfor all polynomialsg in k variables by (4.4) and the assumption that M is analgebra. Sincef,, ...,& are bounded, (4.9) holds for all g E C(0a)).We con-clude thatn-m s(4.9)(4.10) P n U l , . * . , j ; ) - ~ ~ , , . . . , ~ ) - ,fl,-.-,/;, E M.Let K c S be compact, and let S >0. For each x E S, choose{hf,...,hi,,)} c M satisfying(4.11) ~ ( x )3 inf max Ih;(y) -h;(x)I > 0,and let G, = ( y E S: max,sisk~,)/h~(y)-h:(x)I < 4x)). Then K cu, Gxc Kd,so, since K is compact, there exist xl,. ..,x,,, E K such thatK c u;"!I G,, c K*.Defineg,, ...,g, IZc(S) byy : l ( i . x ) ~ dI sis&(x)(4.12)and observe that (4.10) impliesthat(4.14) -lim P,,(Kd)2 &I P,, u G,,n+aD I - m= lim Pn x E S: min [gl(x) -E(xI)] < 02 P x s: min [gxx) -&(XI)] < 01I 1 s l s m-n-mI l s l s m= p(G1 Gx,)2 P(Khwhere the middle inequality depends on (4.13) and Theorem 3.1. ApplyingLemma 2.1 to P and to finitely many terms in the sequence {Pn},weconclude that there exists a compact set K c S such that inf, P,,(KJ 201 -6.By Theorem 2.2, {Pn)is relatively compact.
  • 124. 4. SEPARATING AND CONVERGENCE DRRMlNlNG W S 115We turn next to a result concerning countably infinite product spaces. Let( s k , dk), k = 1, 2,. ..,be metric spaces, and define s = nCa s k and d(x, y) =xF=l 2-(dk(Xk, yk)A I ) for all x, y e S. Then (S, d) is separable if the Sk areseparable and complete if the (Sk,d,) are complete. If the Sk are separable,then B(S)= nFmI a(&).(a) If the S, are separable and the Mk are separating, then M is separat-(b) If the (Sk, dk) are complete and separable and the Mk are con-ing.vergence determining, then M is convergencedetermining.Proof. (a) Suppose that P, Q E B(S)andand let p 1 and v1 be the first rnarginals of p and v on i4#(Sl). Since MI isseparating (with respect to Borel probability measures), it is separating withrespect to finite signed Borel measures as well. Therefore p1 = v1 and hencewhenever A , E O(Sl),n ;r 2, andfk E M, u { I } for k = 2 , ...,n. Proceedinginductively,we conclude thatwhenever n 2 1 and A, E .4a(S,) for k = 1, ..., n. It follows that P = Qon nF=,d?(S,) = B(S)and thus that M is separating.(b) Let {Pm}c B(S) and P E B(S),and suppose that (4.4) holds. Then,for k = I, 2, ..., j j d P ! = j j d P for aIlJc M k , where P! and Pdenote the kth marginals of P,, and P, and hence P: =. P. In particular, thisimplies that {Pi}is relatively compact for k = 1, 2,. .., and hence, by
  • 125. Theorem 2.2 and Proposition 2.4, {f,)is relatively compact. By Lemma174.3, P, 3P, so M is convergencedetermining.We concludethis section by generalizingthe concept of separating set. A setM c B(S)is called separating if whenever P, Q E 9(S) and (4.3) holds, we haveP = Q.More generally, if -4 c P(S),a set M c M(S)(the space of real-valuedBore1functions on S) is called separaring on .4 if(4.20) 1111dP K 00, / E M,P E -4.and if whenever P, Q Eset of monomialson w (i.e., 1, x, x2, x, ...) is separating onand (4.3) holds, we have P = Q.For example, the(Feller(1971), p. 514).Throughout the remaining sections of this chapter, (E, r) denotes a metricspace, and q denotesthe metric r A 1.Most stochastic processes arising in applications have the property thatthey have right and left limits at each time point for almost every sample path.It has become conventional to assume that sample paths are actually rightcontinuous when this can be done (as it usually can) without alteringthe finitedimensional distributions. Consequently, the spacc DEIO,00) ofright continuous functions x: [O, GO)+ E with left limits (ie., for eachr 2 0, lim,,,, x(s) = x(t) and lim,,,, x(s) ~i x(r-) exists; by convention,lim,,o- x(s) =x(0-) =x(0)) is of considerable importance.We begin by observing that functions in DEIO,a)are better behaved thanmight initially be suspected.5.1discontinuity.lemma If x E DEIO,a),then x has at most countably many points ofProof. For n = 1, 2,. .., let A, = (r >0: r(x(t), x(r-)) > l/n}, and observethat A, has no limit points in [O, ao) sincelim,,,, x(s) and Iim,-,- x(s) exist forall t 2 0. Consequently,each A, is countable. But the set of all discontinuities0The results on convergenceof probability measures in Sections 1 4 are bestsuited for completeseparablemetric spaces.With this in mind we now defineametric on DEIO,00) under which it is a separablemetric space if E is separable,of x is u."-,A,, and hence it too is countable.
  • 126. 5. THE SIACE D#, ac) 117and is complete if (E, r) is complete. Let A be the collection of (strictly)increasing functions A mapping [0, 00) onto [O, 00) (in particular, 1(0)= 0,lim14mA(c) = 00, and A is continuous).Let A be the set of Lipschitz continuousfunctionsrl E A such that?(A) 3 ess sup Ilog X(t)It 2 0A(.$) - A(r)For x, y E D,[O, 00). define(5.2) d(x, y) = inf [y(l)V e-"d(x, y, 1,u) du ,A s A 1whered(x, y, A, u) = SUP q(x(tA u), y(M) A u)).I 2 0(5.3)It follows that, given (x,,}, (y,,} c DEIO,a),lima-md(xn,y,,) = 0 if and onlyif there exists (A,,} c A such that(5.4)y(rl,) = 0 andlim m(u E [0, uo]:d(x, ,y,, ,A,,, u) 2 E } = 0a - mfor every E > 0 and uo > 0, where m is Lebesgue measure; moreover, since(5.5)for every 1 E A,implies that(5.7)lim HA,) = 0n-mlim sup I i a ( t ) - tl = 0n-m O S I S Tfor all T > 0.Let x, y E D,[O, a),and observe thatSUP q(x(t A u),y(A(t)A u))= SUP 4(x(A- (1)A u), y(t A u))for all 1 E A and u 2 0, and therefore d(x. y, 1,u) = d(y, x, A-, u). Togetherwith the fact that y(A) = y(A-) for every I 8 A, this implies thatd(x, y) = d(y, x). If d(x, y) = 0, then, by (5.4) and (5.7), x(r) = y(t) for every120 I 2 0(5.8)
  • 127. 118 CONVERGENCE OF PRCMMIUTY MEASUREScontinuity point t of y, and hence x = y by Lemma 5.1 and the right contin-uity of x and y. Thus, to show that d is a metric, we need only verify thetriangleinequality. Let x, y, z E DJO, a),A1,I2 E A, and u 2 0. Then(5.9) SUP 4(Nt A u), Z ( ~ l ( A l ( 0 )A 4)I205 SUP d x ( t A u), Y(J,(t) A 4)I20+ SUP 4(Y(A,(t) A u), Z M A l ( f ) ) A 4)120= SUP q(x(tA u), v(.W)A 4)(LO+SUP dY(t A 4,z(.tz(t) A u)),rhothat is, d(x, z,A, 0 A,, u) s d(x, y, A,, u) +d(y, z,A,, u). But since 1, 0 1,E Aand(5.10) Y(1, O 1,) r(4)+r(A2bwe obtain d(x, z)sd(x,y) +d(y, 2).topology.The topology induced on DEIO,a)by the metric d is called the Skorohod5.2 Proposition Let {x,,} c DEIO,a) and x E Ds[O, 00). Then Iirnm--d(x,, x) = 0 if and only if there exists {A,} c A such that (5.6)holds and(5.1 1) lim d(x,, x, A,, u) = 0 for all continuity points u of x.n-rnIn particular, limm-ad(xm,x ) = 0 implies that limn-= x,(u) = limm~axn(u-)= x(u) for all continuity points u of x.Proof. The sufliciency follows from Lemma 5.1. Conversely, suppose thatlirnn4md(x,, x) = 0, and let u be a continuity point of x. Recalling (5.4). thereexist {A,,} c A and {u,] c (u, a)such that (5.6) holds and(5.12)Nowlirn sup q(x,(t A u,), x(R,(t)A u,)) = 0.n-m 1 2 0
  • 128. 5. M W A a D,p, 00) 119s SUP dxn(tA Unh x(An(r) A un))4- SUP dx(s). x(4)V SUP dx(A(u)A Unh Ns))ostsu(I$ 3 S AdM) v YAdu) n Y Sr sufor each n, where the second half of the second inequality follows by consider-ing separately the cases r 5 u and t > u. Thus, limn-md(xn,x, A,, u) = 0 by0(5.12), (5.7). and the continuity of x at u.5.3 Proposition Let (x,) c &LO, 00) and x E DEIO,GO). Then the followingare equivalent:(a) limndmd(x, ,x) = 0.(b) Thereexists (A,} c A such that (5.6)holds and(5.14) lim SUP dxn(t)*x(A,(r))) 5 0for all T > 0.such that (5.7) and (5.14) hold.n-m OIIJT(c) For each T > 0, there exists {A,} c A (possibly depending on T)5.4 Remark In conditions (b) and (c) of Proposition 5.3, (5.14) can bereplaced by(5.14) lim SUP dX,(A,(t)). 41))= 0.Denoting the resulting conditions by (b)and (c), this is easily established by0n-rD OsrsTcheckingthat (b)is equivalent to (b)and (c)is equivalent to (c).Proof. (a r*b) Assuming (a) holds, there exist {A"} c A and {u,} c (0,oo)such that (5.6)holds, u,-+ GO, and d(x,, x, A,, u,,)-+ 0;in particular,(5.15) lim sup r(x,(r A u,), x(A,(r)A u,)) = 0.Given T > 0, note that u, 2 TVA,(T) for all n sufliciently large, so (5.15)implies(5.14).(b =* a)(5.16) lim sup q(x,(t A u), x(A,(t) Iu)) = 0for every continuity point u of x by (5.13) with u, > A,(u)Vu for each n.Hence (a)holds.n-m 1 2 0Let (A,) c A satisfy the conditionsof (b).Thenn-oo 1 2 0(b c) Immediate.
  • 129. 120 CoMaCENcE OF nOIAUUTY hlEAsuRES(C ab) Let N be a positive integer, and choose {A,"} c A satisfyingtheconditions of (c) with T = N, such that for each n, #(r) -R:((N) +r -Nfor all t > N.Define TO" =0 and, for k = 1,2,. ..,T: = inf t > I :r(x(t), X(TIN_,)) > -(5.17) I NIif rr-, < co, T: = co if r:-l = a.Observe that the sequence {T:} is(strictly)increasing (as long as its terms remain finite) by the right contin-uity of x and has no finite limit points since x has left limits. For each n, letuca = (A,">-(rf) for k 5 0, 1,. ..,where (A,")-(a) = a,and define p." E AbyN N - I N(5m18) C(,"(t) = r: +(1 -uk.awuk+I.a - $a) b k + 1 -7:ht E Cuta, 4+1. J CO, NI, k = 0, I, ...,p:(t) = C(,N(N)+t -N , t > N,where, by convention, 00 - I 00 = 1. With this convention,(5.19) Y(P.") = max I log (4+1, a -uL".A-(4+1 -4)INI.4.. < Nand(5.20) SUP dXa(t), xb,N(t)))OsrsNs SUP W r ) , x(C(~)))+ SUP 4x(C(t)), X(p,N(t)))O s r s N O S I S N25 SUP dxa(t), x(X(t))) +O s r s Nfor all n. Since uca= T: for k = 0, I,. .., (5.18) implies thatlim,,,y(p,") = 0, which, together with (5.20) and (5.14) with T = N,implies that we can select 1 < n, < n, < - - - such that y(r(,", s 1/N andS U ~ ~ ~ , ~ ~ ~ X , , ( ~ ) ,x(p,"(t))) 5 3/N for all n 2 nN.For I s n < n,, let 1"E A bearbitrary. For nN S n < nN+l, where N 2 I, let If,, = p.". Then {If,,) c Asatisfiesthe conditions of (b). 05.5 Corollary For x, y E Ds[O, a),define(5.21) d(x, y) = inf [q(x(r A u), y(A(c) A u))AeAv ( I n ( t ) A u - t A u l A l ) ] d u .The d is a metric on DdO, a)) that is equivalent to d. (However, the metricspace(DEIO,a),d)is not complete.)
  • 130. 5. THE SPACE D,lO, ad 121Proof. The proof that d is a metric is essentially the same as that for d. Theequivalenceof d and d follows from the equivalence of (a)and (c) in Proposi-tion 5.3. We leave the verification of the parenthetical remark to the reader. 05.6 Theoremplete, then (DJO, 00). d) is complete.If E is separable, then DEIO,00) is separable. If (E. r) is com-Proof. Let {a,} be a countable dense subset of E and letfunctionsof the formbe the collection of(5.22)where 0 = to < I , < + 3 c t, are rationals, i,, ...,in are positive integers, andn 2 1. We leave to the reader the proof that r is dense in DJO, 00) (Problem14).To prove completeness, it is enough to show that every Cauchy sequencehas a convergent subsequence. If {x,} c D,[O, 00) is Cauchy, then there existI s N , < N, < .. . such that m, n 2 Nkimplies(5.23) d(x,, x,) s 2 - k - 1 e - k .For k = I, 2,. , .,let Y k = XN, and, by (5.23)-Select Uk > k and Ak E A such that(5.24) y(&)vd(yk, ykt19 u k ) 2-;then, recalling(5.5).(5.25)exists uniformly on bounded intervals, is Lipschitzcontinuous, satisfies(5.26)and hence belongs to A. Since(5.27) sup dYk(pi (I) uk)* Y k t l(h-+!l(r) A uk)I 2 0= dYk(p; (l)A uk)* Y k + l(Ak(p; (I))A uk))120= 4(Y k ( l uk), Yk + A ud)I 2 05 2 - kfor k = I, 2,. (5.24). it follows from the completenessof (E, r) that zk ZE yk0 Irk- converges uniformly on bounded intervals to a function y: [0, a)+ E.
  • 131. 122 CONVMGENCE Of MOMBlUTY MEASUUSBut each* i t )= Oand(5.28) lim SUP d M i(Oh NO)=0for all T > 0, we conclude that limk-,md(y,,y) =0 by Proposition 5.3 (seeRemark 5.4). 0E Ds[O, a),so y must also belong to Ds[O, 00). Since Iim,--it-m Osl5T6. THE COMPACT StTS OF &lo, 00)Again let (E, r) denote a metric space. In order to apply Prohorovs theoremto @(D,[O,oo)), we must have a characterization of the compact subsets ofDs[o,00). With this in mind we first give conditions under which a collectionof step functions is compact. Given a step function x E DEIO,a),defineso(x)= 0 and,for k = 1,2,. ..,(6.1) sk(x) = inf { t >sk- I(x): x(t) # x(c-))ifs,-,(x) < oo,SAX)= 00 ifs,-,(x) = 00.6.1 lemma Let rc E be compact,let 6 >0,and define A(r,6) to be the setof step functions x e &lo, 00) such that x(t) e r for all c 2 0 and s&(x)-sk- i(x)> S for each k 2 I for which sk, ,(x) < m. Then the closure ofA(T, 8)is compact.Proof. It is enough to show that every sequence in A(T, 6)has a convergentsubsequence.Given {x,} c A(T,S), there exists by a diagonalizationargumenta subsequence{y,} of {x,,} such that, for k = 0, 1,. ..,either (a)sit(ym)< m foreach m, lim,,,-m sk(y,,,)E t k exists (possibly a),and y,(sk(y,)) E a,exists, or (b) sit(y,) = oo for each M. Sincesk( y,) -sk -I( y,,,) > 6 for each & 2 Iand m for which sk-,(y,,,)< 00, it follows easily that (y,) converges to thefunction y E DEIO,a)defined by fit) = a,, zk s t <t k + t ,k =4 1 , . ... 0The conditions for compactness are stated in terms of the followingmodulus of continuity. For x B Ds[O, a),6 >0,and T >0,define(Id i 1. I Ill- I. Ill(6.2)where {t,} ranges over all partitions of the form 0 =toc ti c -* * < r,- <T s I, with mini s,s,(r, -I,- > S and n 2 1. Note that w(x, 6, T)is nondacreasingin 6and in T,and thatw(x, S, T)= inf max sup r(x(s), x(I)),WYX, 6,T)s w(Y, 6,T)+ 2 SUP M s ) , As)).O s r c T + b(6.3)
  • 132. 6. TM COWACTSETSOF DP,a01 1236.2 Lemma (a) For each x B DJO, 00) and T > 0, w’(x, 8, T) is right con-tinuous in 6 andlirn w’(x, 8, T)= 0.&-O(b) If {x,} c DECO, oo),x E DEIO,oo), and d(xm,x ) = 0, thenfor every 6 > 0, T > 0,and E > 0.(c) For each 6 >0 and T > 0, w’(x, 6, T)is Bore1measurablein x.Proof. (a) The right continuity follows from the fact that any partition thatis admissible in the definition of w’(x, 6, T) is admissible for some 6’ > 6.To obtain (6.4), let N 2 1 and define {tr} as in (5.17). If 0 < 6 < min {tr+-t,”:t: < T}, then w‘(x, S, T) 5 2/N.(b) Let {x,} c DJO, ao), x B DEIO,oo), 6 >0, and T > 0. If1imndQd(x,, x) = 0, then by Proposition 5.3, there exists {A,} c A’ suchthat (5.7) and (5.14) hold with T replaced by T +6. For each n, let y,,(t) =x(A,(r)) for all t 2 0 and 6, = supOs,sr[A,(f +6) - A,,(r)]. Then, using (6.3)and part (a),(6.6)- -lirn w‘(x,, 6, T) = lirn w’(ym,6, T)n-m 11-9)5 lim w’(x, 8,. 1AT))1-4)s lirn w’(x, 6, V 6, T +E)= w‘(x, 6, T +E )n- mfor all E > 0.(c) By part (b) and the monotonicity of w‘(x, 6, T)in T,w’(x, 8, T+)ESlirn,,,, w’(x, S,T +6) is upper semicontinuous,hence Bore1 measurable,in x. Therefore it suflices to observe that w‘(x, 6, T) = lirn,,,, w‘(x, 6,0(T - E ) +) for every x c DEIO,00).6.3 Theorem Let (E, r) be complete. Then the closure of A c Ds[O,00) iscompact if and only if the following two conditionshold:(a) For every rational t 2 0, there exists a compact set r,c E such that(b) For each T > 0,~ ( t )E r,for all x E A.lirn sup w’(x, 6, 7’)= 0.&-0 r e A
  • 133. 124 CONVERGENCE OF MOWUTY MUSURS6.4 Remark In Theorem 6.3 it is actually necessary that for each T > 0 thereexist a compact set rTc E such that x(t) E Tr for 0 s t s T and all x E A.See Problem 16. 0Proof. Suppose that A satisfies (a) and (b), and let 12 1. Choose 6, E (0, 1)such thatand m, 2 2such that l/m,<6,.DefineY)= U~L+~)"rT,,mland, using the nota-tion of Lemma 6.1,let A, = A(T"), 6,).Given x E A, there is a partition 0 -to <tl < -* < t,- < 1 s t, <1 + I < In+,= co with min,,,,,(t, -r,-l) >6,such that(6.9)Definex E A, by x(t) = x(([m, r] -t-l)/m,) for t, s t < t,, i =0, 1,. ..,n, ThensuPo~:<lr(xl(l)rxo) s2/r, so(6.10) d(x, x) s e-" sup [r(x(t A u), x(r A u))A 13 duI20s 211 -+e- < 311.It follows that A c A:". Now I was arbitrary, so since A, is compact for each1 2 1 by Lemma 6.1,and since A c nrr,A;,, A is totally bounded and hencehas compact closure.Conversely, suppose that A has compact closure. We leave the proof of (a)to the reader (Problem 16). To see that (b) holds, suppose there exist q > 0,T >0, and {xn}c A such that w(xe, l/n, T)2 q for all n. Since A hascompact closure, we may assume that lim,4ad(xn, x ) = 0 for somex E DEIO,a).But then Lemma 6.2(b)impliesthat(6.11) q s lim w(xn,S,T)s w(x, 6, T + 1)for all 6 > 0. Letting S-, 0, the right side of (6.1I) tends to zero by Lemma0n-(o6.2(a), and this results in a contradiction.We conclude this section with a further characterization of convergence ofsequences in Os[O, m). (This result would have been included in Section 5 wereit not for the fact that we need Lemma 6.2 in the proof.) We note that(C,[O, 00). d,) is a metric space, where(6.12) ddx, y) = e-" sup Cr(x(t), y(9)A 1I du.osrsu
  • 134. 6. T M E C O k Y A C T ~ O FD,lO, m) 125Moreover, if {x,} c CJO, oo)and x E C,[O, oo),then lim,-ta, dv(x,, x) = 0 ifand only if whenever {t,} c [0, a),t 2 0, andr(x,,(t,), x(t)) = 0. The following proposition gives an analogue of this result fort, = t, we have(4TCO,a), 4.6.5 Proposition Let (E, r) be arbitrary, and let (x,} c D,[O, 00) and x ED,[O, a).Then d(x,, x) = 0 if and only if whenever (1,) c 10,w),t 2 0, and t, = t, the followingconditions hold:(a)(b) If s, = t, then(c) If limm+a,r(x,(t,),x(f-)) = 0, 0 s s, 5 t, for each n, andNx,,(t,,h x(r))A r(x,(r,,), x(r -)) = 0.r(x,(r,,), x(t)) = 0, s, 2 t, for each n, andlimn+,,,r(x,(s,), x(t)) = 0.s, = c, then r(x,,(s,), x(t -)) = 0.(6.15) 4x(UQ), ~ ( t ) )S SUP 4x(Uu))* xn(u))0suS.r+ 4Xn(tnh x(l))for each n. If also r(x(A,(r,,)), x(r)) = 0 by(6.15), so since A&) 2 A,,(t,) for each n and lim,+mA,,(s,)= r, it follows thatlimndmr(x(A,(s,)), x(c)) = 0. Thus, (b)followsfrom (6.14). and the proof of (c) issimilar.We turn to the proof of the sufliciency of (aHc). Fix T > 0 and for each ndefiner(x,(r,), x(t)) = 0, then(6.16) E, = 2 inf { E > 0 : r(r,n, 8 ) z 0 for 0 5 r 5 T } ,
  • 135. 126 CONMCEHCE OF nOUIlUTY MLASUIESwhere(6.17) r(t,n, E) = {S E (t -E, t +E ) n [O, 00):~(x,(s),dr))< e,r(X& -1, x(l- 1) E } .We claim that limn-,.,,E, =0. Suppose not. Then there exist 6 > 0, a sequence{flk) of positive integers, and a sequence {tk}c [O, T)such that r(tk,f l k , E)= 0 for all k. By choosing a subsequence if necessary, we can assume thatlimk+mfk =t exists and that t, <t for all k, f k >t for all k, or f k = t for allk. In the first case, limk,, x(tr) = limk-mx(tk-) = x(t-), and in the secondcase, lirnk-.,.,, x(fk) = limk-m%(tk -) = x(t). Since (a) implies that limn-,,, x,(s) =limn4mxn(s-)=x(s) for all continuity points s of x, there exist (byLemma 5.1 and Proposition 5.2) sequences (a,} and (6,) of continuitypoints of x such that a, c t < b, for each n and a n dt and 6,- t suliicientlyslowly that limn+wx,(a,) = limn-.,.,, x,,(a,-) = limn-- x(a,) = x(t-) andlimn+- x,(b,) = x,(b,-) = limn*- x(b,) = x(t). If t k < t (respectively,f k > r) for all k, then a, (respectively, b,,) belongs to r(rk,mk, E) for all ksuficientlylarge, a contradiction. It remains to consider the case f k = I for allk. If x(t) = x(r -), then t 6 r(t,fib, E) for all k sufficientlylarge by condition(a).Therefore we suppose that tfx(t), x(t-)) =6 > 0. By the choice of {a,) and{b,} and by condition (a),there exists no 2 1such that for all n 2 no,(6.18)(6.20)By (6.18), a, c s, S b,, and therefore s,,E (t -E. r +e), r(xm(sn),x(t)) s (6A ~)/2,and r(x,(s, -), x(t))2 (Sh~)/2. The latter inequality, together with (6.19),implies that r(x,,(s,--), x(t-)) <(6A E ) / ~ .We conclude that s, E r(t,n, 8) for alln 2 no, and this contradiction establishesthe claim that limn-.mE,, = 0.1; < * * * < r&- ,< T stk with min, s,sm,,(tr- t:- ,) > 36, such that(6.21) max sup r(x(s), Nt))4 WIX,3 ~ " ,T)+En,and put m. = max {i 2 0: t; s T}(m: is m, - 1 or m,). Define 1,(0) = 0 andA&;) = inf J-(r:, n, E,) for f = 1,. ..,m:, interpolate linearly on [O, f"n:J, and letA&) = t -t:: +A&::) for all t > r::. Then A, E A and sup,L o Ilz,(r) -rl s E,, .We claim that limn-- supos,,,r(x,,(1,(t)), x(r)) =0 and hence lima-m d(x,,x) =0 by Proposition 5.3 (see Remark 5.4). To verify the claim, it is enoughFor each n, we construct 1, E A as follows. Choose a partition 0 = r: <I S l + n . l.Ictr:.,.m
  • 136. 7. CONVEIIGENCEIN DISTUBUIIONIN DJO, Q) 127to show that if It,} c [O, TI, 0 s t s T, and limn4mrn= I, thenlim,4m r(x,,(A,(c,)), x(t,)) =0. If x(r) = X(C-), the result follows from condition(a) since limm-mA,&,,) = t. Therefore,let us suppose that X(t) # x(c-). Then, foreach n sufliciently large, r = rymfor some ints { 1,. ..,m:} by (6.21) and Lemma6.2(a). To complete the proof, it sufices to consider two cases, {I,} c [r, T ]and {t,} [O, r). In the firstcase, A,(r,) 2 A&) -A,,(r;.) and dx,(A,,(r;)), x(t)) sE,for each n suficiently large, so r(x,(A,(t,)), x(c)) =0 by condition (b),and the desired result follows. In the second case, A&,) < A&) = A$;,) andeither r(xn(A,,(~ym)-), x(t -)) < E, or r(xm(An(r;)), x(r -)) s c,, (depending onwhether the infimum in the definition of An(r:a) is attained or not) for each nsufficiently large. Consequently, for such n, there exists s, with A,(t,) < s, sA&;) such that r(x,(s,), x(t -)) s en, and therefore limn-.- ~xm(A,,(r,)),dt-))= 0 by condition (c), from which the desired result follows. This com-pletes the proof. 07. CONVERGENCE IN DISTRIBUTION IN DB[O,aJ)As in the previous two sections, ($ r) denotes a metric space. Let 9,denotethe Borel o-algebra of Dt[O, 00). We arc interested in weak convergence ofelements of P(DEIO,a))and naturally it is important to know more aboutY E .The following result states that 9, isjust the a-algebra generated by thecoordinaterandom variables.7.1 Proposition For each t 2 0,define n,:D,[O, 00)- E by n,(x)= x(t). Then(7.1)where D is any densesubset of [O, a).If E is separable,then .4ps =9’;.Y E 3 9;5 dn,:0 s r < 00)= dn,:t E D),Proof. For each E > 0, t 2 0, and/€ c((E),(7.2)defines a continuous functionfi on DJO. 00). Since Iirn,+ Jxx) =/(R,(x)) forevery x E DEIO,a),we find that f o n, is Borel measurable for everyJ E C(E)and hence for everyJE B(E). Consequently,(7.3) n; ‘(r)= {x E DsEo,a):xr(n,(x))= I 1E Y &for all r E WE),and hence Y E =YE.For each t 2 0, there exists {c,,} c D nIt, ao) such that limn+mt, = t. Therefore, A, = limn-mn,. is dn,:s E D)-measurable,and hence we have (7.1).
  • 137. Assume now that E is separable. Let n 2 1, let 0 = to < t , < . - < t, <i = O,l, ..., = ao,and fora. ,a,, ...,a,, E E define Hao,aI,...,a,,)E D,[O, a)by(7.4)Sinceq(ao,al,...,a,)(t) = al, r, s t < t i + ] ,d(rt(ao,a ] ,...,a,), ,a’,, ...,a:)) s max dotl,a;),Osisn(7.5)tf is a continuous function from En+’into DEIO,a).Since each n, isY’,-measurable and E is separable, we have that for fixed z E DEIO, a),d(z, q 0 (n,,,nt,, ..., n,J is an Ycmeasurable function from DJO, a)into R.Finally, for m = I, 2,. ..,let )I,,, be defined as was q with tl = i/m, i = 0, 1, ...,n f m2.Thenlim 4 2 , qm(qo(x),...,nlm2(x)))= d(z,x)111-m(7.6)for every x E D,[O, 00) (see Problem 12). so d(z,x) is Y6-measurable in x forfixed z E DEIO,a).In particular, every open ball B(z,e) = {x E DJO, 00):d(z,x) < E} belongs to YE,so since E (and, by Theorem 5.6, Dr[O, a))isseparable,9;containsall open sets in D,[O, ao)and hence contains 9,. 0A DEIO,a)-valued random variable is a stochastic process with samplepaths in DEIO,co),although the converse need not be true if E is not separable.Let { X J (where a ranges over some index set) be a family of stochasticprocesses with sample paths in D,[O, 00) (if E is not separable, assume the X,are DEIO,a)-valued random variables), and let {Pa}c 9(DE[0,00)) be thefamily of associated probability distributions (i.e., P,(B) = P{X, E B} for allB E LYE).We say that {X,} is relutiuely compact if {Pa)is (i.e., if the closure of{Pa}in aD6[0,00)) is compact).Theorem 6.3 gives, through an application ofProhorov’s theorem, criteria for {X,,} to be relatively compact.7.2 Theorem Let (E, r) be complete and separable, and let {Xa}be a familyof processes with sample paths in D,[O, ao). Then {X,} is relatively compact ifand only if the followingtwo conditions hold:(a) For every q > 0 and rational t 2 0, there exists a compact. setrqe,c E such that(7.7)(b) For every q > 0 and T > 0, there exists 6 > 0 such thatsup P{W’(X,, 6,T)2 tf} 5 ‘I.a
  • 138. 7. CONVERGENCE IN D(STIIIIUTI0NIN DJO, o) 1297.3 Remark In fact, if (X,} is relatively compact, then the stronger compactcontainment condition holds; that is, for every q > 0 and T > 0 there is acompact set I-qs r c E such that(7.9) inf P(X,(t) E rq. for 0 s t I T } 2 1 - q. 0:0Proof. If {X,} is relatively compact, then Theorems 5.6, 2.2, and 6.3 imme-diately yield (a)and (b);in fact, I-:, ,can be replaced by I-,,, in (7.7).Conversely, let E > 0, let T be a positive integer such that e - r < c/2,and choose 6 > 0 such that (7.8) holds with q = ~ / 4 . Let m > 1/6, putr = u;IT,, re2-l-2,1/m,and observe that(7.10)Using the notation of Lemma 6.1, let A = A(T‘,6). By the lemma, A hascompact closure.Given x E D,[O, 00) with w’(x, 6, T) < 44 and x(i/m) E fori = 0, I, ..., mT, choose 0 = lo < t , < . . * < t,- ,< T s I, such thatmin, si& - t i - ,) > 6 and&inf P{X,(i/m) E re? i = 0, I , . ..,m r } 2 I - -.a 2E(7.1I ) max sup W s ) . ~ ( 0 )< 4.and select { yl) c r such that r(x(i/m), y,) < 4 4 for i= 0, 1, ...,mT. Definingx‘ E A byI s i s n 3. f 6 I f i - I.I t )we have ~ u p ~ ~ ~ ~ ~ T ( x ( t ) ,x’(t)) < ~ / 2and hence d(x, x’) < 4 2 -te-’ < E, imply-ing that x E A‘. Consequently, inf,P(X, E A‘} 2 I - E, so the relative com-0pactness of {X,)followsfrom Theorems 5.6 and Corollary Let (E, r) be complete and separable, and let {X,,} be asequence of processes with sample paths in D,[O, 00). Then (X,,} is relativelycompact if and only if the followingtwo conditions hold:(a) For every q > 0 and rational t 2 0, there exists a compact setrq,,c E such that(7.13)(b)-lim P(X,(t) E. r;,,}2 1 -q.n-mFor every q > 0 and T > 0, there exists 6 > 0 such that(7.14)
  • 139. Proof. Fix q >0, rational t 2 0, and T > 0. For each n 2 1, there existby Lemmas 2.1 and 6.2(a) a compact set r, c E and 6, >0 such thatP{X,,(r)E r:) 2 I -q and P{w(X, ,8, ,T)2q} s q. By (7.13) and (7.14),there exist a compact set roc E, So >0,and a positiveinteger o0such that(7.15)and(7.16)inf P{X,(t) E rg}2 1 -qaznoWe can replace no in (7.15) and (7.16) by 1 if we replace Toby r = u7;or,0and So by S = A:"; S,,,so the result followsfrom Theorem Lemma Let (E, r) be arbitrary, let TI c r, csequenceof compact subsetsof E, and define- be a nondecreasing(7.17)Let {X,}be a family of processes with sample paths in S. Then {X,}isrelatively compact if condition(b)of Theorem 7.2 holds.S = {x E DEIO,a):x(t) E r,, for 0 s r s n, n = 1, 2,. ..}.Proof. The proof is similar to that of Theorem 7.2 Let E > 0, let T be apositive integer such that e- < 42, choose 6 > 0 such that (7.8) holds withq = ~/2,and let A = A(rT,6). Given x E S with w(x, 6, 7) < e/2, it is easy toconstruct x e A nS with d(x, x) <E, and hence x E (A n Sr.Consequently,inf,P(X, E ( A A Sy) ;r I -E, so the relative compactness of {X,}followsfrom Lemma 6.1 and Theorem 2.2. Here we are using the fact that (S,d) iscompleteand separable(Problem 15). 07.6 Theorem Let (E, r) be arbitrary, and let {X,}be a family of processeswith sample paths in DJO, 00). If the compact containmentcondition (Remark7.3) and condition (b) of Theorem 7.2 hold, then the X,have modifications2.that are DEIO,00)-valued random variablesand (2,)is relatively compact.Proof. By the compact containment condition there exist compact sets r, cE, n = 1, 2, ..., such that inf, P{X,(r) E r, for 0 s t s n} 2 1 -n-. LetE, = u,,rn.Note that E, is separable and P{X,(r) E E,,} = I so X, has amodification with sample paths in DEo[O,a).Consequently, we may as wellassume E is separable. Given 4 > 0, we can assume without loss of generalitythat(7.18) S, = (x E D,[O, 00): x(c) E-".,,} is a nondecreasingsequenceof compact subsetsof E. Definefor 0 s r sn, n = 1, 2,. ..}.
  • 140. 7. CONVUCENQ IN CMSfR((IWIONIN DJO, OD) 131and note that inf, P{X, E S,} 2 1 - q. By Lemma 7.4, the family {P:) c 9(S,),defined by(7.19) P p )= P{X, E B J X ,E S,},is relatively compact. The proof proceeds analogously to that of Corollary 2.3.We leave the details to the reader. 07.7 Lemma If X is a process with sample paths in Ds[O,a),then the com-plement in 10,a ~ )of(7.20)is at most countable.D(X) s {r 2 0: P{X(t)= X(c-)} = I}Proof. Let E > 0.6 > 0, and T > 0. If the set(7.21)contains a sequence {r,} of distinct points, then(7.22)contradictingthe fact that, for each x E DBIO,a),r(x(t),x(t -)) 2 E for at mostfinitelymany t E [O, TI.Hence the set in (7.21) is finite,and therefore(7.23)is at most countable.The conclusion follows by lettingE-+ 0.(0 s r 5 T :P(r(X(t),X(r-1) 2 E } 2 a)P{r(X(t,), X(r,-)) L E infinitely often}2 6 > 0,{ t 2 0: P{r(X(r),X(r-)) 2 E } > O}07.8 Theorem Let E be separable and let X,,n = I, 2, ..., and X be pro-cesses with sample paths in D,[O, m).(11 If X,*X , then(7.24) (XAti 1, * * * * xAtJ)z+ (x(ti 1, * * * 9 X(rk))for every finite set {tI. ...,t k } c D(X).Moreover, for each finite set { t i , ...,t,) t [0, 00). there exist sequences ( I ; } c It,, a),..., {r;} c [rk, 00) con-verging to t , , ...,rk, respectively, such that (X,(ti), ...,X,(t:)) 4 (X(t,),...,X(tk)).(b) If {X,} is relatively compact and there exists a dense set D c [O, 00)such that (7.24) holds for every finite set ( i l l . ...rk} c D,then X, rg X .Proof. (a) Suppose that X, rg X.By Theorem 1.8, there exists a probabilityspace on which are defined processes V,, )t = 1, 2, ...,and Y with samplepaths in Ds[O,00) and with the same distributionsas X,, n = 1, 2, ...,andX , such that lim,,-md(x, Y)= 0 as. If t E D(X)= D(Y), then, using the
  • 141. 132 CONVERGENCE OF ROMBIUTY MUSWESnotation of Proposition 7.1, n, is continuous as. with respect to the dis-tribution of Y,so Y,(f) = Y(t)8.5. by Corollary 1.9, and the firstconclusion follows. We leave it to the reader to show that the secondconclusion is a consequenceof the first, togetherwith Lemma 7.7.(b) It suffices to show that every convergent (in distribution) sub-sequence of {X,}converges in distribution to A. Relabeling if necessary,suppose that X,*Y.We must show that X and Y have the same distribu-tion. Let {r,, ...,t k } c D(Y)and f,,...,1;E C(E), and choose sequences{t;} c D n [tl, a),. .,{rr}c D n [ t k , 00) convergingto ri ,...,f k , respec-tively, and n, < n, < n3 < * * such thatThenfor each m 2 1. All three terms on the right tend to zero as m+ 00, the firstby the right continuity of X,the second by (7.25), and the third by the factsthat X,,-Y and {ti,. ..,rk} c my).Consequently,(7.27)for all {t,, ..., t r ) c [O, 00) (by Lemma 7.7 and right continuity)and allf,, ..., 1;E c(E). By Proposition 7.1 and the Dynkin class theorem(Appendix4). we conclude that X and Y have the samedistribution. 08. CRlTERtA FOR REUTIVE COMPACTNESS IN Dal0,00)Let (E, r) denote a metric space and q = r h 1. We now consider a systematicway ofselecting the partition in the definition of w(x, 6, T).Given E > 0 andx E DJO, a),definer0 = uo= 0 and, for k = 1,2,. ..,
  • 142. 8. CRITERIA FOR R E U T M COMIACWESS IN D,,IO, a01 133if Ty, - 1 < m, Ck = if Tk - 1 = 00,n k = sup t 5 T k : r(x(l),x ( T k ) ) v r(x(f-)v x ( ? k ) ) 2 -(8.2) I 27if ?k < 00, and oh = 00 if Tk 3: 00. Given 6 > 0 and T > 0. observe thatw(x, 6, T )< c/2 implies min {?k I - o k : ?k < T } > 6, for if T& I - uk5 6 andt k c T for some k 2 0, then any interval [a, 6) containing ?k with 6 - a > 6must also contain o k or T k + , (or both) in its interior and hence must satisfysup., ,* Is, 6, r(x(s),x(t))2 ~ / 2 ;in this case, w(x, 6,T) 2 42.Lettingfor k = 0, 1, ...,we have limh-,a Sk = 00. Observe that, for each k 2 0,(8.4) ak <sk T k 5 n k t I s k + I s T k + I *andif s k < 00, where the middle inequality in (8.4) follows from the fact thatr(X(Tk), x ( ? k + I)) 2 ~ / 2if ? k + I c 00. We conclude from (8.5) that min {Tk?k < T +6/2}> 6 implies-for if not, there would exist k 2 0 with s k c T, t k 2 T +612, and s, + I - sk s6/2, a contradiction by (8.4). Finally,(8.6)implies w(x, 6/2,T) 5 E.Let us now regard t k , O k , and s k , k = 0, 1,. ..,as functions from DJO, Q))into [0,003.Assuming that E is separable (recall Remark 3.4). their9,-measurability followseasily from the identities(8.7){?k < u} = ( T k - 1 < a}n ut c ( 0 ,u ) n Qand(8.8) { n k 2 u} = ( T k = m) U l(X(U-), X(?k)) 2
  • 143. 134 CONVERGENCE OFmLouuuTv ~ ~ ~ w l l l ~ ovalid for 0 < u < 00 and k = I, 2,.... We summarize the implications of thetwo preceding paragraphs for our purposes in the following lemma.8.1 lemmawith samplefor given E >Let (E, r) be separable, and let (Xm)be a family of processespaths in D,[O, a).Let zFa,u : ~ ,and .$ma, k = 0,1,...,be definedt 0 and X, as in (8.1H8.3).Then the following are equivalent:lim inf P{w’(Xm,6, T)< a } = 1,(8.9)(8.10) lim inf P(min {s$,!, -s;*’: s:‘ < T}L S}= 1,(8.11) lim inf P{min {I?,!, -a;*#:Ti.‘ < T}2 S} = I,E >0, T > 0.E > 0, T >0.E > 0, T > 0.( - 0 m6-0 ad-.O mProof. See the discussionabove. 08.2. Lemma For each a, let 0 = s: <fl <6 < * be a sequence of randomvariables with limb-,,,,s: = 00, define A: = g+,-si for k = 0,I,...,let T > 0,and put K,(T)= max (k 2 0: st < T}.Define F:[O, 00)--+ [O, 11 byF(t)=s~pmsup~,oP{A; < I, < 7’). Then(8.12) F(6)5 sup P min A,O < 6 s LF(6)+e’ Le-”F(r) dtI lfor all S > 0 and t= 1,2,. ...Consequently,(8.13) lim sup min A: <a} = 0if and only if F(O+) = 0.4-0 Q OShSiKdT)Proof. The firstinequalityin (8.12)is immediate. As for the second,min A: <6 s P{A; <6,s; < T}+P{K,(T)2 L}(8.14) P(L - 11 k - 00 $k s&(‘I
  • 144. 8. CRITERIA FOR ReAnnCOMPACTNESS IN DAO,CO) 135Finally,observe that F(O+) = 0 implies that the right side of (8.12) approacheszero as 64 0 and then L --+ 00. 08.3 Proposition Under the assumptionsof Lemma 8.1, (8.9)is equivalent to(8.15) lim sup sup P{Z:;’~ - < 6, r;.’ < T }= 0, E > 0, T > 0.d 4 0 d L Z Oproof. The result follows from Lemmas 8.1 and 8.2 together with theinequalities(8.16)The following lemma gives us a means of estimating the probabilities in(8.15). Let S(T)denote the collection of all (S;+}-stopping times boundedby T.8.4 LemmaDJO, a),and fix T > 0 and p > 0. Then, for each 6 > 0.1 > 0, and T E S(T),Let (E, r) be separable, let X be a process with sample paths in
  • 145. 1% CONVERGENCE tx riOlUUUtY MEASU~ES8.5 Remark 8 In (8.19), SUP,,^(^+^^) can be replaced by S U ~ ~ . ~ ~ ( ~ + ~ ~where So(T +26) is the collection of all discrete {9f}-stopping timesbounded by T +26. This follows from the fact that for each r e S(T+26)there exists a sequence (7,) c So(T +26) such that T" 2 T for each n andlim,,-,m r, = T ; we also need the observation that for fixed x E DJO, a),SUP0sc,r3dA,qB(x(t),x(t -u)) is right continuousin t (0 5 t < a).(b) If we take 1= e/2 E (0,lJ and T = f k A T (recall (8.1)) in Lemma 8.4,where k 2 1, then the left side of(8.17) bounds(8.20) P { T k + 1 .- rk 6, f k -bk < 6, Tk < T } ,which for each k 2 1 bounds P{?k+l - < 6, Tk < T,?I > 6).The left sideof (8.18) bounds P{r, ~r;b}, and hence the sum of the right sides of (8.17)0and (8.18)bounds P ( T k + 1 - Uk < 8, Tk < T}foreach k 2 0.Proof. Given a (9:+}-stopping time 7, let M@) be the collection of.F:+-rneasurable random variables U satisfying0 5 U 5 6. We claim that(8.21) sup E[ sup q"(X(r+ 01,X(T)V(X(r),X(r - u))]suptSS(T+& U8Yd4) O S U S l d n r~r;(a,, +4a$C@).To see this, observe that for each 7 E S(T+6)and U E M#),(8.22) 4 W ( T + U),X(d126Ia,,6-1 [q(X(r +8).X(r))+q8(X(7 +e), X(T+ V))]des a,, 8-I[ iz*q ~ x ( 7+e), x(t))deqp(x(r+ u +el, x(f+ 0))de ,1and hence
  • 146. also, r + U E S(T + 24,so (8.21) follows from (8.23).Given 0 < q < I and 7 E S(T),define(8.24)and observe thatA = inf {r > 0: q(X(r +I), X(T))> I - q),(8.25) q”(X(7 +A A S),X(t))#(X(r), X ( t - u))5 aSqO(X(r+6). X(r)h@(X(t),X(r - 0))+ajq’(X(r + S),X ( r +AAS))qc(X(r +AAS), X(r))+a: qb(X(r +6). X ( t + A A S))q’(X(r +A A 6). X(r - u))for 0 s u 5 S A t . Since t +A A S E S(T +S), S - A A S E Mr+bhd(S),andA A d + u s 26, (8.21) and (8.25) imply that(8.26) €[ sup qb(X(t -k AAd), X(t))q”(X(r), X ( r - u))1O s u S d A rs [as + 2a3ap +4a$]~(6).But the left side of (8.17) is bounded by (A - q)-@A-@times the left side of(8.26),so (8.17) follows by lettingq -+0.(8.27) qZc(X(AA 4,X(0))5 c~~[q”(x(S),X ( AA S))qc(X(AA 6). X(0))Now defineA as in (8.24) with t = 0.Then+ q”(X(6).X(0))q’(X(AA 6). X(0))l5 a8 Sp(x(6),x ( AA S))qp(X(AA 6). x(0))+ a#qO(X(d),x(o)),so (8.18) follows as above. 08.6 Theorem Let (€, r ) be complete and separable, and let ( X , } be a familyof processes with sample paths in Ds[O, 00). Suppose that condition (a) ofTheorem 7.2 holds. Then the followingare equivalent:(a) { X J is relatively compact.
  • 147. 138 CONVEICENCEOFMOW~UTV~YAU~~DS(b) For each T > 0, there exist /?>0 and a family {ye@):0 c 6 < I, alla} of nonnegativerandom variablessatisfying(8.28) ECqp(Xe(t+u), XAt))IKk’(Xdf), XAf -0)) ECYe(4 193for0 s t 5; T,Os u s 6,and 0 s u 5 6At, where 9:=9 P ; i n addition,(8.29) lim sup ECyAS)] = 0d-0and(8.30)(c) For each T > 0, there exists # >0 such that the quantitiesJim sup E[q@((X,(6),X,(O))] = 0.1-0 u(8.31) C,(6)=SUP SUP .[ SUP qC(Xg(r4-Uh Xa(t)hp(X,(d,X,(r -d)],rrS37l Osrsd O s v s i A rdefined for 0 <6 < 1 and all a, satisfy(8.32) lim sup Cg(6)=0;&-0in addition (8.30) holds. (Here S,(T) is the collection of all discrete(*:}-stopping times bounded by T.)8.7 Remark (a) If, as will typically be the case,(8.33) ECqp(Xe(t +u), XAtNISfJ s ECYJ~)I$:Iin place of (8.28), then E[:q0(XJ6), XAO))] s E[y#)] and we need onlyverify (8.29) in condition (b).(b)- For sequences {X,,},one can replace sup, in (8.29), (8.30), and (8.32)by as was done in Corollary 7.4. 0Proof. (a ;r) b) In view of Theorem 7.2, this followsfrom the facts that(8.34) dXg(t + u), Xe(t)k(XAt),XAt -u))S dxa0+4,xAtNA dXAf), XAf -0))s w’(Xe,26, T +6)A1for 0 s t 5 T,O s u $ &and0 5 u s dht, and(8.35) 4(XA4, XU(0))s W W . ,6. T )A 1.(b‘+c) Observe that f in (8.28) may be replaced by t E S,(T)(Problem25 of Chapter 2), and that we may replace the right side of (8.28) by its
  • 148. a CMRU FOIREUM CWACTNESS IN 0110, 1ssupremum over u E [O, 6A I] n Q and hence over u E [O, b A TI.Conse-quently,(8.29) implies(8.32).(c 3a) This follows from Lemma 8.4, Remark 8.5, Proposition 8.3, andTheorem 7.2. 0The following result gives suficient conditions for the existence of{ya(S): 0 < S < I, all a} with the properties required by condition (b) ofTheorem Theorem Let (E, r) be separable, and let {X,} be a family of processeswith sample paths in Dc[O,ao). Fix T > 0 and suppose there exist /l>0,C z 0,and 8 > I such that for all a(8.36) ECqP(X.(t + h), Xe(t))A q6(Xa(t),x e ( t - h))I s the,o s t s r + 1, o I;h 5 t,which is implied by(8.37) E[qC’2(Xa(t+h), X&))qb’z(Xa(r),X,(t -h))] sChe,Osrs;T+ 1,OI;hsr.Then there exists a family {ya(b):0 < b < I, all a} of nonnegative randomvariablesfor which (8.29) holds, and(8.38) qYXa(r + u),Xa(t)MXa(t),Xa(t -v)) sVJS)for 0 5 t s T,0 5 u s 6, and 0 s v s 6 Ar.8.9 Remark (a) The inequality (8.28) follows by takingconditionalexpacta-(b) Let E > 0, C > 0.8 > 1, and 0 < h s r, and supposethat(8.39)for all A > 0.Then, letting/l = 1 +e,(8.40) ECqc(Xa(t + h), XeWAqP(Xa(t),X,(t - h))]tions on both sides of (8.38).P{r(Xe(t+ h), X.(r)) 2 I, r(Xa(r),Xa(r - h))2 A} s A-’Che= P{qp(Xa(r + hh Xa(t)) L X, q’(X.(t), Xa(t -h))Z X} dx0Pmof. We prove the theorem in the case fl> 1; the proof for 0 < fl s 1 issimilar and in fact simpler since qP is a metric (satisfiesthe triangle inequality)in this case.In the calculationsthat follow we drop the subscripta. Define
  • 149. 140 CONVUCXNCE OF WOBAIlLITY MEASURES(8.41)9" = C q(X((k + 1)2-"), X(k2-"))Aq@(X(kZ-"),X((k - 1)2-"))I srszqr+1)- 1for m = 0, I,. ..,and fix a nonnegative integer n. We claim that for integersm 2 n andj , k,, k, ,and k3 satisfying(8.42) 0 SjZ-" 5 k,2-" < kz2-" K k,2-" S (j+2)2-" s T + 1,we have(8.43)(If 0 < /3 I; 1, replaceq by qcand q,!I8 by q,in (8.43).)implies that k , =j , k, =j + 1, and k, =j +2,and(8.44) q(X((j +2)2-3, X ( 0 + 1)23)Aq(X((j+ 1)2-"), X02-")) I; q,"?Suppose (8.43) holds for some m 2 n and 0 I;j2-" s k12-"-I < k, 2-"-l <k32-"-l s (j+2)2-" 5 T + 1. For i = 1, 2, 3, let e, = q(X(k;2-"),X(k,2-"-)), where if k, is even, k; = kJ2, and if k, is odd, k; = (k, f 1)/2 asdetermined by(8.45)Note that ei= 0 if k, is even and e, 5 q,!$ ,otherwise,so the triangleinequalityimplies that(8.46)q(X(k32-"),X(k,2-"))Aq(X(kz2-"), X(k,2-")) s 2 f t#@.i=rWe prove the claim by induction.For m = n, (8.43) is immediate since(8.42)6, = q(X((ki + 1)2-"-), X(ki2-"-))Aq(X(k,2-"-), X((k,- 1)2-"-)).q(X(k32-"-I), X(k22-"-I)) A q(X(kz2-"- I), X(kl2- -I))5 re3 +q(X(k32-"), X(k;2-9+811A [Ez +q(X(k;2-"), X(k; 2 - 9 ) +61J5; 2q;t +q(X(k,2-"), X(k; 2-")) A q(X(k;2-"), X(k;2-")).By the definition of k;, we still have 0 1;j2-" I; k12-"I;k;2-" I; k32-" s(j+2)2-" S T + 1,and hence the induction step is verified.and t I , t , , and t, are dyadic rational witht , - c, s 2-" for some n 2 1, then there exist j, m, k,, k z , and k, satisfying(8.42) and t, = k,2-". Consequently,If 0 s t , < t2 < t, s T +i = nBy right continuity, (8.47) holds for all 0 I;1, < c3 < t3 < T +$ witht , -t 5 2-". If 6 2 4,let y(6) = 1;if 0 < 6 < $, let nl be the largest integer n
  • 150. 9. FURTHER CRITERIA FOR RELATIVE CWIAClNESS IN D,lO, ao) 141satisfying 26 < 2 -",and define y(6)= qnr.Since ab s a A b for all Q, b E: [O, I],we concludethat (8.38)holds. Also,(8.48) E[y(G)] = 2 f ECq:@l s 2 f E[I,J/Pi==W 4-nrm5 2 c [2(T + 1)C2-"]"b.l e n dso lim,,-,, E[y(6)] 3:0(and the limit is uniform in a). 08.10 Corollary Let (E, r) be complete and separable,and let X be a processwith values in E that is right continuous in probability.Suppose that for eachT > 0, there exist /I> 0, C > 0, and 0 > I such that(8.49) E[qP(X(t+-h,), X ( t ) ) W ( x ( r ) .X(t - h d 1 5 C(h,Vh,)whenever 0 S t - h, I;t s t +. h, s T. Then X has a version with samplepaths in DEIO,00).Proof. Define the sequence of processes {X,}with sample paths in DEIO,a))by X,(t) = X(([nt] + l)/n). It suffices to show that {X,)is relatively compact,for by Theorem 7.8 and the assumed right continuity in probability of X,thelimit in distribution of any convergent subsequence of {X,} has the samefinite-dimensionaldistributions as X .Given q > 0 and t 2 0, choose by Lemma 2.1 a compact set rqslc E suchthat P{X(t)e r;.l}2 1 - q. Then (7.13) holds by Theorem 3.1 and the factthat X,(t) *X(t) in E. Consequently, it suffices to verify condition (b) ofTheorem 8.6, and for this we apply Theorem 8.8. By (8.49) with T replaced byT + 2, there exist p > 0, C > 0, and 8 > I such that for each )I(8.50) EC&Xn(t +h),XAr))A &Xn(l), Xn(l - h))I[nt] - [n(t - h)] "[n(t +h)] - [nt].C( n n >.0 5 l $ T + I , O S h h t .But the left side of (8.50) is zero if 2h 5 I/n and is bounded byC(h + 5 3Ch if 2h > l/n. Thus, Theorem 8.8 implies that (8.29) holds,0and the verification of (8.30)is immediate.9. FURTHER CRITERIA FOR RELATIVE COMPACTNESS IN DrlO,ao)We now consider criteria for relative compactness that are particularly usefulin approximating by Markov processes.These criteria are based on the follow-ing simpleresult. As usual, (E,r) denotes a metric space.
  • 151. 142 CONVERGENCE OFmoornun MEAHI~ES9.1 Theorem Let (E, r) be complete and separable, and let (X,}be a familyof processes with sample paths in D,[O, 00). Suppose that the compact con-tainment condition holds. That is, for every 1 > 0 and T >0 there exists acompact set rq,r c E for whichLet H be a dense subset of C(€) in the topology of uniform convergence oncompact sets. Then {X,)is relatively compact if and only if { / o Xa} is rela-tively compact (as a family of processes with sample paths in DJO, 00)) foreachfE H.Proof. GivenfE c(E),the mapping x - j o x from D,JO, a))into DJO, a)iscontinuous (Problem 13). Consequently, convergence in distribution of asequence of processes {X,,) with sample paths in Ds[O, a)implies convergencein distribution of c / o A,,}, and hence relative compactness of {Xu}impliesrelative compactnessof {fo X,}.Conversely, suppose that {f0 X,} is relatively compact for everyf E H.Itthen follows from (9.1), (6.3), and Theorem 7.2 that { / o A,} is relativelycompact for every f E if(E) and in particular that {q(*,z) 0 X,} is relativelycompact for each zE: E, where q = r A 1. Let 1 >0 and T >0. By the com-pactness of rq,r , there exists for each E > 0 a finite set { z ~ , ...,z N )c Tr,rsuch that min, s I s N 4 ( ~ ,z,)<E for all x E rq,r.If y E rq,r , then, for someiE {I,. ..,N},q(y, 2,) <E and hencefor all x E E. Consequently,for 0 5 t sT,0 su s 6,and 0 $ v s 6Ar.where0 < 6 < 1. Note that N dependson 1,T,and E.
  • 152. 9. FURW CRITERIA FOR RELATIVE COMIACTNESSIN DJO, 00) 143Since lim,,osup,E[w(q(*, z) 0 X,, 26, T +6)hI] = 0 for each z E E byTheorem 7.2, we may select 9 and E depending on d in such a way thatlimd+osup, E[y,(6)] = 0. Finally,(9.2)implies that(9.4) ~ ~ ( 6 ) .X,(O)) 5 V IdX,(d), 2,) - M m ( 0 ) ,zr)I + 2~+ ~w,(o)er,,rlfor all S > 0, so limd,o sup, E[q(X,(S), X,(O))] = 0 by Theorem 8.6. Thus, therelativecompactnessof {X,) followsfrom Theorem 8.6. 0Ni=I9.2 Corollary Lct (E, r) be complete and separable, and let (X,} be asequence of processes with sample paths in DJO, 00). Let M c c(E)stronglyseparate points. Suppose there exists a process X with sample paths inDc[O, 00) such that for each finiteset {g,, ...,gk} c M,(9.5) (g1,. ..,gk) X , e ( g 1 , . ..,g k ) 0 X in Dado, a>).Then X , =.X .Proof. Let H be the smallest algebra containing M u (I}, that is, the algebraof functions of the form c!Ila,/llJ12...A,,,, where 12 1, m z 1, and a, E R and/;,E M u ( 1 ) for i = 1,...,I and j = 1,...,m. By the Stone-Weierstrasstheorem, H is dense in C(E) in the topology of uniform convergence oncompact sets. Note that (9.5)holds for every finite set (gl,...,#k} c H.Thus,by Theorem 9.1, to prove the relative compactnessof (X,} we need only verify(9.1).Let r c E be compact, and let S > 0. For each x E E, choose {hf,...,hi(,,} c H satisfyingE(x) = inf max (h:(y) - hf(x)l> 0,and let U,= {y E E: mar, d l s k ( x )Jh,(y)-h,(x)l < e(x)}. Then rcU x . r U xc r, so, since r is compact, there exist x,, ...,x, E r such thatr c Ur-,U,,c r?Define o: Dl[O, 00)- DaCO, 00) by a(xKt) = suporrS,X(~)and observe that u is continuous(by Proposition 5.3). For each n,let(9.7)y : r k y ) z d I S l S k ( r )(9.6)Yn(t) = min (gXX,(t)) -e(xJ}, t L 0,where gdx) =max,s,,k(,,)Ihfl(x)- hfl(x,)l,and put Z,= a(V.). It follows from(9.5)and the continuity of e that Z,*2,where 2 is defined in terms of X asZ,is in terms of X,. Therefore Z,(T )3Z(T )for all T B D(Z),and for such TI s l b N
  • 153. 144 CONVaGENCEoFflOUIlllTYkYAPUlETby Theorem 3.1, where the last inequality uses the fact that(9.9) sup min {g,(x) -dx,)}< 0.x e r i s i s NLet q >0, let T >0 be as above, let m 2 I, and choose a compact setr0,,c E such that(9.10) f{x(r)E r0,, for 0 sr sT}z 1 -~ 2 - ~ ~ ;this is possible by Theorem 5.6, Lemma 2.1, and Remark 6.4. By (9.8), thereexistsn, 2 1 such that(9.11)Finally, for n = 1,. ..,n, - 1, choose a compact set r,,,,,c E such that(9.12) P{X,,(r)E r!!: for 0 s r s T} 2 1 -@-".Letting r, = UZ; r,,,,we have(9.13) inf P{X,(r)E r:, for 0 5 t 5 T}2 1 -$-",so if we define r,,(beingcompleteand totally bounded)and(9.14)inf P{X,(r) E r;!: for 0 st s 7) 2 1 -$-".n z l rnz Ito be the closure of n,,,,, r;",then T,, r is compactinf P{X,(t) E T,,r for 0 sr 5 T } 2 1 - q.012IFinally, we note that(9.15) (elAaI, gk 0 X,*(gl AaIr...rgk hak)0 Xfor all gl,...,gk E H and aI,...,akE R. This, together with the fact that If isdense in c(E) in the topology of uniform convergenceon compact sets, allowsone to conclude that the finite-dimensionaldistributions converge. The detailsare left to the reader. 09.3 Corollary Let E be locally compact and separable, and let Ed be itsone-point compactification. If {X,) is a sequence of processes with samplepaths in Ds[O, 00) and if { s o X,} is relatively compact for everyfE e(€)(thespace of continuous functions on E vanishing at infinity), then {X,}is rela-tively compact considered as a sequence of processes with sample paths inDEIIO,00). If, in addition, (X,,(rl),...,Xm(rk))*(X(fI), ...,X(tk))for all finitesubsets { t , ,...,tr} of some dense set D c [O, a),where X has sample paths inDEIO,a),then X,=+ X in DEIO,w).
  • 154. 9. FURTMR CRITERIA FOR RELATIVE COMPACTNESS IN D,# a01 145Proof. Iff€ C(E*), then /( a ) -J(A) restricted to E belongs to e(&).Conse-quently, ( / o X,} is relatively compact for every f E C(EA),and the relativecompactness of (X,} in D,A[O, 00) followsfrom Theorem 9.1.Under the addi-tional assumptions, X, *X in DEA[O, 00) by Theorem 7.7, and hence X, *X0in &[O. ao) by Corollary 3.2.We now consider the problem of verifying the relative compactness of(10X,}, where/€ c ( E )is fixed. First, however, recall the notation and termi-nology of Section 7 of Chapter 2. For each a, let X , be a process with samplepaths in D,[O, 00) defined on a probability space(0,.4ca,Pa)and adapted to afiltration (.at:}. Let Ip, be the Banach space of real-valued (S:}-progressiveprocesses with norm II Y II = sup,,o E[l Y(r)I]c 00. Let(Y,2)E 9,x 9,:Y(t)- 1Z(s) ds is an {.%:}-martingaleand note that completenessof (9;)is not needed here.9.4 Theorem Let (E, r) be arbitrary, and let {X,) be a family of processes asabove. Let C, be a subalgebra of C(E)(e.g., the space of bounded, uniformlycontinuous functions with bounded support), and let D be the collection pfJ E C(E)such that for every E > 0 and T > 0 there exist (c,2,) E .d,withand(9.18) sup E[ll2,~Ip, < 00 for some p E (1.a].a(IIhIt,,,=CS,Tlh(r)tPdtI1” if P < a;IIhII,,r=esssup,,,,,th(r)/.) If C,, iscontained in the closure of D (in the sup norm), then { J o X,} is relativelycompact for each /E C,; more generally, {u,,...,Jk)0 XJ is relativelycompact in DAIIO,00) for allJ,,/2,. ..,f,, E C,, 1 5 k 5 00.9.5 Remark (a) Taking p = 1 in condition (9.18)is not suficient. Forexample, let n 2 1 and consider the two-state (E = (0,l))Markov processX,with infinitesimalmatrix(9.19)(-: -:)
  • 155. 146 CONVRGENCE OF PROMIlUTV kwuaand P(X,(O) 3:0) = 1. Given a function f on (0,I), put Y. =f 0 X, and2,=(A,f) 0 X,,so that (Y., Z,,)E 2,and(9.20) ECIA~I(Xa(t))ll= nVO -f(I)IP{xa(t)= 1) +If(l)-f(OIIp{xa(l) a 0)= If([) -f(O)l(l +(n - Inn + ])-(I -e-("+l)))5 211(1) -f(O)lsfor all t 2 0. However, observe that the finitedimensional distributions ofX, converge to those of a process that is identically zero, but {X,} does notconverge in distribution.(b) For sequences {X,}, one can replace sup, in (9.17) and (9.18)7by 0
  • 156. la CONV~CENCETO A mocm H c,p, 00) 147Let l / p + l/q = 1 and l/p + I/q = I, and note that fi+lh(s)lds sPI/h Itp, f + for0 s r 5 T. Therefore,if we define(9.26) r.(4 = 2 SUP IJ2(X,(4)- WI~ c ( O . T + l l n O+411III SUP lf(X,(d) - K(4l+ 6"It z a IIpa, r + 1 + 2IIfII6"IIzaIIp . r+ 19I a (0. f + I I n Qthen(9.27) €CU(xa(t+ u)) -.f(~At)))2IFJI 1;ECya(6)ICI.Note that this holds for all 0 s r r; T and 0 s u 9 S (not just for rational Iand u) by the right continuity of X,. Since(9.28) sup ECv.@)l S (2 + 4IIfIIba+ 8" SUP ECitz&iIp*.T+rI+211JIIJ1" SUP ECIIZaIIp,r + i Jawe may select E depending on 6 in such a way that (8.29) holds. Therefore,{ f ~X,}is relatively compact by Theorem 8.6 (see Remark 8.7(a)).Let I 5 k -c oo. Given Jl,...,fk E C,,define v,((s) as in (9.26) and setydd) = x=v.cS). Thenfor 0 s t 5; T and 0 s u 5 6, and the yi(6)can be selected so that (8.29) holds.Finally, relative compactness for k = Q, follows from relative compactnessforall k < 00. (SeeProblem 23.) 010. CONVERGENCE TO A PROCESS IN C,lO,ao)Let (E,r) be a metric space, and let C,[O,oo) be the space of continuousfunctionsx: [O, 00)--+ E. For x e DEIO,oo),define(10.1)where(10.2)4 x 1 = e-"[J(x, u)A 13 du,J(x, u) = sup r(x(t), X(l-)).o s t s rSince the mapping x-+ J(x, .) from Da[O, 00) into Dfo,m,[O, 00) is continuous(by Proposition 5 3 , it follows that J is continuous on Da[O,oo). For eachx rs DJO, 00). J(x, * ) is nondecreasing,so J(x)= 0 if and only if x E C,[O, 00).
  • 157. 118 CONVERCENCE OF ROBAULITY M E W I E S10.1 Lemma Suppose (x,} c DECOYao), x E Ds[O, o),and d(xa, x) =0. Then(10.3) lim sup r(x,(r), x(r)) s J(x, u)-a-m Osrsufor all u 2 0.Let c(D,[O, a),d,) be the space of bounded, real-valued functions onDs[O, 00) that are continuouswith respect to the metric(10.5) ddx, y) = e- sup CdxW, fit))/ 11du,that is, continuousin the topology ofuniform convergenceon compact subsetsof [0, ao). Sinced 5 dU,we have YEES L€l(D,[O, a),d )c CS(D,[O, a),du).O S l S U10.2 Theorem Let X,, n = 1, 2,. ..,and X be processes with sample pathsin DEIO,a),and suppose that X , 310 X. Then (a) X is a.s. continuous if andonly if J(X,) -0, and (b) if X is as. continuous, thenf(X,) r*f(X) lor every9,-measurable fE C(D,[O, ao), d,,).Proof. Part (a)followsfrom the continuityofJ on DEIO,00).By Lemma 10.1, if {x,} c DdO, 00). x E Cs[O, ao), and limn-- d(x,, x) =0,then lima-,mdu(xa, x) 5 0. Letting F c W be closed and j be as in the state-ment of the theorem,/-(F) is d,-closed. Denoting its d-closure by/-, itfollows thatf- A CJO, 00) =f-(F) n CEIO,ao). Therefore, if P, *P on(DEW,001, d)and P(Cd0,4)r= 1,(10.6) lim P,/-I(F) <lim P,UTO)S If-)a- m a-rm= WTby Theorem 3.1, so we conclude that P,f- *PJ-. This implies(b).c,m00)) = w-W) n CJO, a))= P f - W0
  • 158. 10. CONVERGENCETO A noassIN c , ~ ,a01 149The next result provides a useful criterion for a proccss with sample pathsin D,[O, ao) to have sample paths in C,[O, 00). It can also be used in conjunc-tion with Corollary Proposition Let (E, r) be separable, and let X be a process with samplepaths in DEIO,00). Suppose that for each T > 0, there exist > 0. C > 0, and8 > 1 such that(10.7)whenever 0 5 s It 5 T,where 9 = r A 1. Then almost all sample paths of Xbelong to C,[O, ao).ECqYX(t),m)ls C@- S)OProof. Let T be a positive integer and observe that(10.8)By Fatous lemma and (10.7), the right side of (10.8) has zero expectation.andhence the left side is zero 8.5. 02.1q(X(t), X ( t - ) ) s -lim 1 qU(X(k2-"),X((k - 1)2-")).O < f S T n + w h a 110.4 Proposition For nparameter R-valued process, let a,, > 0,and define1, 2,. .., let {l-#), k = 0, 1,. ..) be a discrete-(10.9) Xn(t) = UCantl)and(10.10) zn(t) = V.(CantI)+(ant - CanrlMV.(Can11 + 1) - K(Ca,tl))for all t 2 0. Note that X, has sample paths in D,[O, 00) and Z, has samplepaths in Cw[O, 00). If limn-.w a, = OD and X is a process with sample paths inCRd[O,00). then X,*X if and only if Z,*X.Proof. We apply Corollary 3.3. It suffces to show that, if either X , = X or2,*X , then d(X,, Z,)+ 0 in probability. (The two uses of the letter d hereshould cause no confusion.)For n = 1,2,. ..,(10.11) d(X,, Z,)5 1e- sup (IX,(t) - X,(r-)IA I ) du0 st s Y +0"- IJ(X,).I;and
  • 159. 150 CONVERGENCE OF ?ROBAUUTY Muwlssprovided a; I;E. But the function J,: Du[Ol a)-,CO, 11, defined for eachE>Oby(10.13) JXx) =[e- supis continuous and satisfies lim,,,JL(x) = J(x) for all x E Dnr[Ol00). Conse-quently, if 2,-X,then (10.12) and Theorem 3.1 imply that(10.14) lim P{J(x,,)2 S} s Iim lim P{J,(z,)2 6)s lim P{J,(X)2 6)= 0sup (Ix(t)-x(s)lA 1) du ( 2 J(x)),O S f S U f - S A f $ S e f-n - a 8-40 n-tur-0for all S >0. so we conclude that J(X,,)- 0 in probability. The same conclu-sion follows from Theorem 10.2 if X,+X.In either case, (10.11) implies that0d(X,, Z,>+ 0 in probability, as required for Corollary3.3.11. PROBLEMS1. Let (S, d) be completeand separable,and let P,Q E P(S).Show that thereexistsp c M(P, Q)(seeTheorem 1.2) such thatand show that p2(P, Q)5 I{P -Q11 s 3p(P, Q).Him: Recall that $f dP =f)/fll P { f z r} df if f 2 0, and note thatII(e -d( a , F))VOJIBL5 1 for 0 < E c 1.3. Show that f l S ) is separablewheneverS is.J. Suppose (P,,)c P(R),P E LP(R),and P,PO P.Define(1 1.3) G,,(x)= inf {y E W: P,(( -00, yl)2 x}and(I 1.4)for 0 < x < 1, and let t: be uniformly distributed on (0, I). Show thatGJC) has distribution Pn for each n, G(<)has distribution P, andlimn- G,( e) = G(C)as.a x ) = inf { y E R:P((-co,y]) 2 x)
  • 160. 11. PROOlEMS 1515. Let (S, d) and (S’, d’) be separable. Let X,,n = 1, 2,..., and X beS-valued random variables with X,*X. Suppose there exist Bore1 mea-surable mappings h,, k = l, 2,. ..,and h from S into S’ such that:(a) For k = 1, 2,...,hk is continuous a.s. with respect to the distributionof x.(b) hk -+ h as k -+ 00 a.s. with respect to the distribution of X .(c) limkem P{d(hk(X,),h(X,)) > E ) = 0 for every E > 0.Show that h(X,) *h(X).(Note that this generalizes Corollary 1.9.)Let X,, n = I, 2,. .., and X be real-valued random variables withfinite second moments defined on a common probability space(Q, 9, P). Suppose that {X,]converges weakly to X in L2(P)(i.e.,ECX, Z ] = ECXZ], 2 E L?(P)),and (X,}converges in distributionto some real-valued random variable Y. Give necessary and sumcientconditions for X and Y to have the same distribution.Let X and Y be S-valued random variables defined on a probabilityspace (Q, 9, P). and let Y be a sub-a-algebra of 9. Suppose thatM c c(S) is separating and(I 1.5)for everyjE M.Show that X = Y a.s.Let M = { f ~c(W):f has period N for some positive integer N } . Showthat M is separating but not convergencedetermining.Let M c c(S)and suppose that for every open set G c S there exists asequence {f,}c M with 0 <f, 5 xG for each n such that, =xG.Show that M is convergencedetermining.10. Show that the collection of all twice continuously Frechet differentiablefunctions with bounded support on a separable Hilbert space is con-vergence determining.11. Let S be locally compact and separable. Show that M c e(S) is con-vergence determining if and only if M is dense in &) in the supremumnorm.-I6.7.N - / ( X )Ig1 =J(Y)8.9.12. Let x e DEIO,00). and for each n L I, define x, E DEIO,m) by %,(I) =x(([nt]/n) A n). Show that lim,,+md(x,, x) = 0.13. Let E and F be metric spaces, and let/: E-. F be continuous. Show thatthe mapping x -f 0 x from D,[O, 00) into D,[O, 00)is continuous.14. Show that O,[O, m)is separable whenever E is.
  • 161. 152 CONMRCENCE Of ROBMIUTY MEASURES15. Let (E, r) be arbitrary, let r, c r2c * - . be a nondecreasing sequence ofcompact subsetsof E, and define(11.6) S = {x E D,[O, 00): x(t) E r, for 0 s r In, n = 1, 2,...}.Show that (S, d) is completeand separable, where d is defined by (5.2).16. Let (E, r) be complele. Show that if A is compact in DEIO,a),then foreach T > 0 there exists a compact set Tr c E such that x(t) E TT for0 s t s Tand all x E A.17. Prove the followingvariation of Proposition 6.5.Let {x,} c DE[O, 00) and x E DEIO,a).Then d(x,, x) = 0 if andonly if whenever t, 2 s, L 0 for each n, r 2 0,,= I, andr, = t, we have(1 1*7) lim Mxn(tnh ~ ( t ) )V Hxn(sn), x(t))IA dxn(sm)v x(t -)) =0n - mand(1 1.8) lim r(x,,(t.l, x(0)A Cr(xn(tn)s x(t -1) V r(Xn(Sn1, x(r -))I = 0.18. Let (E, t)be complete and separable. Let {X,}be a family of processeswith sample paths in DEIO,OD).Suppose that for every E > 0 and T > 0there exists a family {X: } of processes with sample paths in D,[O, GO)(with X; and X,defined on the same probabilityspace)such thatsup P sup ~-(x:~(t),Xu(r))2 e < e(1 1.9)and {X:r} is relatively compact. Show that {X,}is relatively compact.19. Let (E, r) be complete and separable. Show that if {Xu}is relativelycompact in DEIO,a),then the compact containmentcondition holds.20. Let {N,} be a family of right continuous counting processes (i.e.,N,(O)=O, N , ( t ) - N , ( t - ) = 0 or 1 for all t > O ) . For k = O , 1, ..., letT; = inf ( t 2 0: N,(t) 2 k } and A; = r; - T ; - , (if T:-, < 00). Use Lemma8.2 to give necessary and sufficient,conditions for the relative com-pactness of {N,}.21. Let (E, r) be complete and separable,and let {X,} be a family of processeswith sample paths in &[O, 00). Show that {Xs}is relatively compact ifand only if for every E > 0 there exists a family {X:)of pure jumpprocesses (with X: and X, defined on the same probability space)such that sup, ~up,~~r(X#),X&)) <E a.s.. (X:(t)) is relatively compactfor each rational f z 0, and (N:) is relatively compact, where N$)is the number of jumps of X:in (0,t].I- IDI Ia O S I S ~
  • 162. 11. ROllLEMS 153(a) Give an example in which {X,)and { 5)are relatively compact inD,[O, 00). but {(X,,K)}is not relatively compact in D,,[O, 00).(b) Show that if {X,},{ X}, and {X,+ K) are relatively compact inDa[O, 00). then {(X,,Y,)) is relatively compact in Dwl[O,00).(c) More generally, if 2 5 r < 00, show that {(X:,X:, ...,Xi)}is rela-tively compact in DJO. 00) if and only if {X:} and {X:+ Xi}(k, 1 = 1,. ..,r) are relatively compact in DJO, 00).Show that { ( X i ,X.,...)} is relatively compact in D,,,[O, co)(where R"has the product topology) if and only if {(Xi,...,Xi))is relativelycompact in D,,[O, 00) for r = I, 2,. .. .Let (E, t ) be complete and separable, and let {X,)be a sequence ofprocesses with sample paths in DEIO,00). Let M be a subspace of C(€)that strongly separates points. Show that if the finite-dimensional dis-tributions of X, converge to those of a process X with sample paths inD,[O. 00). and if { g 0 X,} is relatively compact in DJO, 00) for everyg E M,then X,*X .Let (E, r) be separable, and consider the metric space (C,[O, 00). d"),where d , is defined by (10.5). Let 9denote its Bore1 a-algebra.(a) For each t 2 0, define n,:CEIO,a)---)E by R,(x) = x(f). Show thatA7 = ~ ( n , :0 -< r < a).(b) Show that d, determines the same topology on C,[O, 00) as d (thelatter defined by (5.2)).(c) Show that CJO, 00) is a closed subset of D,[O, 00), hence it belongsto .YE,and therefore .9c .YE.(d) Suppose that { P,) t .9(D,[O, to)), P E 9YDE[0,00)). and P.(C,[O,ce))= P(C,[O, 00)) = I for each it. Define {Q,}c .P(C,[O, 00)) andQ E SyC,[O, 00)) by Q, = Pelaand Q = P la. Show that P, P onD,[O, 00) if and only if Q,*Q on CJO, a)).Show that each of the following functions j k :D,[O, 00)- D,[O, 00) iscontinuous: I 1.10)27. Let .d c .@(S) be closed under finite intersections and suppose each open
  • 163. 154 CONVERGENCE OF PROBMIUTV MEASURESset in S is a countable union of sets in d.Suppose P, P, E 9(S), n = 1,2,. ..,and lime-* P,,(A) = P(A)forevery A E 1.Show that P,28. Let (S,d ) be complete and separable and let d c dit(S). Suppose for eachclosed set F and open set U with F c U,there exists A E d such thatF c A c U. Show that if {P,} c 9(S) is relatively compact andlimn-mP,(A) exists for each A E d,then there exists P E 9(S)such thatPn=$. P.P.12. NOTESThe standard reference on the topic of convergence of probability measures isBillingsley’s(1968) book of that title, where additional historical remarks canbe found.As originally defined, the Prohorov (1956) metric was a symmetrizedversion of the present p. Strassen (1965) noticed that p is already symmetricand obtained Theorem 1.2. Lemma 1.3 is essentially due to Dudley (1968).Lemma 1.4 is a modification of the marriage lemma of Hall (1935),and is aspecial case of a result of Artstein (1983). Prohorov (1956) obtained TheoremI.7.The Skorohod (1956) representation theorem (Theorem 1.8) originallyrequired that (S,d) be complete; Dudley (1968) removed this assumption. Fora recent somewhat stronger result see Blackwell and Dubins (1983). The con-tinuous mapping theorem (Corollary 1.9)can be attributed to Mann and Wald(1943) and Chernoff(1956).Theorem 2.2 is of course due to Prohorov(1956).Theorem 3.1 (without (a))is known as the Portmanteau theorem and goesback to Alexandroff (1940-1943); the equivalence of (a) is due to Prohorov(1956) assumingcompletenessand to Dudley (1968) in general.Corollary 3.3 iscalled Slutsky’stheorem.The topology on Ds[O, a)is Stone’s (1963) analogue of Skorohod’s (1956)J , topology. Metrizabilitywas first shown by Prohorov (1956). The metric J isanalogous to Billingsley’s (1968) do on D[O, I]. Theorem 5.6 is essentially dueto Kolrnogorov(1956).With a different modulus of continuity,Theorem 6.3 was proved by Proho-rov (1956); in its present form, it is due to Billingsley(1968).Similar remarksapply to Theorem 7.2.Condition (b) of Theorem 8.6 for relative compactness is due to Kurtz(1975)’as are &heresults preceding it in Section 8; Aldous (1978) is responsiblefor condition (c).See also Jacod, Memin, and Metivier (1983). Theorem 8.8 isdue to ChenEov (1956).The results of Section 9 are based on Kurtz (1975).Proposition 10.4was proved by Sato(1977).Problem 5 is due to Lindvall(1974)and can be derived as a consequenceofTheorem 4.2 of Billingsley(1968).
  • 164. 4In this chapter we study Markov processes from the standpoint of the gener-ators of their corresponding semigroups. In Section I we give the basic defini-tions of a Markov process, its transition function, and its correspondingsemigroup, show that a transition function and initial distribution uniquelydetermine a Markov process, and verify the important martingale relationshipbetween a Markov process and its generator. Section 2 is devoted to the studyof Feller processes and the properties of their sample paths, and Sections 3through 7 to the martingale problem as a means of characterizingthe Markovprocess corresponding to a given generator. In Section 8 we exploit the char-acterization of a Markov process by its generator (either through the determi-nation of its semigroup or as the unique solution of a martingale problem)togive general conditionsfor the weak convergenceof a sequence of processes toa Markov process. Stationary distributions are the subject of Section 9. Someconditions under which sums of generators characterize Markov processes aregiven in Section 10.Throughout this chapter E is a metric space, M(E) is the collection of allreal-valued, Bore1 measurable functions on E, W E )c M(E)is the Banachspace of bounded functions with 11/11 = sup,.,If(x)l, and QE) c E(E) is thesubspace of bounded continuous functions.155GENERATORS ANDMARKOV PROCESSESMarkov Processes Characterizationand ConvergenceEdited by STEWARTN. ETHIER and THOMASG.KURTZCopyright 01986,2005 by John Wiley & Sons,Inc
  • 165. 156 GENERATORS AND M M K O V PROCESSES1. MARKOV PROCESSES AND TRANSITION FUNCTIONSLet {X(t),t r 0} be a stochastic process defined on a probability space(n,SF, P)with values in €, and let 9: = o(X(s):s 5 t). Then X is a Markovprocess if(1.1) P{x(t+S) E rr4F:J = f{x(t+S) E rlx(t))for all s,t L 0 and r E 1(E).If {Yl} is a filtration with 9:c Y,, t 2 0, then Xis a Markoo process with respect to (Y,} if (1.1) holds with gF: replaced by Y,.(Of course if X is a Markov process with respect to {gJ,then it is a Markovprocess,) Note that (1.1) implies(1.2)for all s, 2 0 andfc B(E).transitiorifunction if(1.31 PO,x, * E 69, 0.4 E LO,00) x E,(1.4) P(0, x, 9 ) = 6, (unit mass at x), x E E,ELT(X(t +4)I93= E U ( X 0 +4)IX(01A function P(t, x, r)defined on [0, ao) x E x g(E)is a time homogeneous(13)(1.6) ~ ( t+s, x, r)= P ( ~ ,y, r)p(t,X, dy),P( ., ’, r)E BKO, 00) x E). r E WE),andS, t 2 0, x E E, r E q ~ ) .5A transition function P(t, x, r) is a transition function for a time-homogeneous Markoo process X if(1.7) P{X(C+S) E r1P:} = P(S,x(t), r)for all s, i 2 0and r E A?(E),or equivalently,if(1.8)for all s, r 2 0 andftz B(E).assumptiongiven (1.7). observe that (1.7) impliesELT(X(t +s))l9t,“I= If(YIP(S, X(t), dY)To see that (1.6). called the Chapman-Kolmogorov property, is a reasonable(1-9) ~ ( t+S, x(u), r)= P(X(U+t i-S) E rlsp,”}= E C P I X ( ~ + ~ + ~ ) € ~ I S ” , X , , } I ~ , Y I= E[P(S, X ( U +t), r)i*:]= sP(s. Y, WYr. X(UX dY)for all s, t, u 2 0 and r E B(E).
  • 166. 1. MARKOV PROCESSES AND TRANSITION FUNCTIONS 157The probability measure v E B(E)given by v(r)= P(X(0)E r}is called theA transition function for X and the initial distribution determine the finite-initial distribution of X .dimensionaldistributions of X by(1.10) P{X(O)E To. X(t,) E r,,...,X(r,) E r,,]P(tn - t n - 1, Y n - 1 1 rn)P(tn- I - f m - 2 3 Y n - z r dyn - 1)= 1". . . s,-, f(l1, yo ~ Y I)iu(dyo).In particular we have the following thcorem.1.1 Theorem Let P(r, x, r)satisfy (1.3Hl.6) and let v E B(E). If for eachc 2 0 the probability measure P(f,x, .)u(dx) is tight (which will be the case if(E, r) is complete and separable by Lemma 2.1 of Chapter 3), then there existsa Markov process X in E whose finite-dimensionaldistributions are uniquelydetermined by (I. 10).proof. For I c [O, a),let E denote the product space IlaE,E, where for eachs, E, = E,and let 8,denote the collection of probability measures defined onthe product a-algebra ~ J c , 4 f ( E J ) .For s E I, let X(s) denote the coordinaterandom variable. Note that nSE,4f(E,) = a(X(s):s E I).Let {si, i = 0,I, 2,. . .) c [O, 00) satisfy .Ti # s, for i # j and so = 0, and fixx, E E. For n > 1, let 1 , < t 2 < . * * < r, be the increasing rearrangement ofsl,. ..,s,. Then it follows from Tulceas theorem (Appendix 9) that there existsP, E P,silsuch that P,(X(O) E To,X(r,)E TI,. .. ,X(r,) E T,} equals the rightside of (1.10) and P,{X(s,) = xo) = I for i > n. Tulceas theorem gives ameasure Q. on E" * . * snl. Fix xo E E and define P, = Q, x S,=,,, * O . , , on&I = E ( S 1 .....Sd Eh*I.Jm+Z....I. The sequence { P,) is tight by Proposition 2.4of Chapter 3, and any limit point will satisfy (1.10) for { I , , . ..,t,) c (si}.Consequently {P,,}converges weakly to P"I E P,,,,.By Problem 27 of Chapter 2 for E E 9f(€)I0. m there exists a countablesubset {si) c [O, 00) such that B E a(X(si):i = 1, 2,. ..), that is, there existsB E si?(€)lSii such that B = {(X(sl),X(s2),...) E b}. Define P(5)= Plr4)(b).Weleave it to the reader to verify the consistency of this definition and to show0that X,defined on (Elo*m), 9(E)I0-m! P), is Markov.Let P, denote the measure on ,W(E)ro*R given by Theorem 1.1 with v = 6,,and let X be the corresponding coordinate process, that is. X ( t , w )= d r ) . Itfollows from (1.5) and (1.10) that P,(B) is a Bore1 measurable function ofx for f?=(x(O)~r,,...., X(t,,)~r,,},O < r l < r z < . . - < t , , r,, I-,,...,E B(E).In fact this is true for all 5 E 9?(E)Io,m.
  • 167. 158 GENERATORS AND MARKOV PROCESSES1.2 Proposition Let P, be as above. Then P,(B) is a Borel measurable func-tion of x for each B E O(E)Io*Proof. The collection of B for which P,(B) is a Borel measurable function is aDynkin class containing all sets of the form (X(0)E ro,...,X(t,) E r,}E0@E)ro* and hence all B E B(E)l0*m! (See Appendix 4.)Let (Y(n), n = 0, 1, 2,. ..} be a discrete-parameter process defined on(Q, 9,P) with values in E, and let 9:= a(Y(k):k s n). Then Y is a Markoilchain if(1.11) P{ Y(n +m)E ri9:) = f{Y(n +m) E ri Y(n)]for all m, n 2 0 and r E 1(E). A function p(x* r) defined on E x A#(@ is arransitionfunction if(1.12) P(X, 1E w3, x E E,and(1.13)A transition function p(x, r)is a transirion function for u time-homogeneousMarkou chain Y if(1.14) P{Y(n+ 1) E rp=,Y}= p ( ~ ( n ) ,r), 0, r E a(@.Note that* . cl(Y(n),~ Y A .As before, the probability measure v E P(E)given by v(T) = P{ Y(0)E r}iscalled the initial distribution for Y.The analoguesof Theorem 1.1 and Propo-sition 1.2 are left to the reader.Let {X(t),t ;r 0},defined on (n,9,P),be an E-valued Markov process withrespect to a filtration (3,) such that X is {Yl}-progressive.Suppose f(t, x. r)isa transition function for X,and let r be a (J,}-stopping time with 7 < co as.Then X is strong Markou at r if(1.16) PWT + t ) E rI 9,) = fit, XW, r)for all t 2 0 and l- E a(E),or equivalently,if(1.17)for all t 2 0 andf6 B(&).X is a strong Murkou process with respect to (3,)ifX is strong Markov at T for all (y,}-stoppingtimes T with T c a0 as.
  • 168. 1. MARWOV PUOCEJMS AND TRANSITION FUNCTIONS 1591.3 Proposition Let X be E-valued, {Yl}-progressive,and {Yl}-Markov,andlet P(t, x, r)be a transition function for X. Let T be a discrete (91)-sloppingtime with r < a,as. Then X is strong Markov at r.Proof.then B n (T = t i } E Y,, and hence for t 2 0,Let t,, t,, ... be the values assumed by r, and let /E B(E). If B E gr,Summing over i we obtainfor all B E Y,, which implies (1.17). 01.4 Remark Recall (Proposition 1.3 of Chapter 2) that every stopping time isthe limit of a decreasing sequence of discrete stopping times. This fact canfrequently be exploited to extend Proposition 1.3 to more-general stoppingtimes. See, for example, Theorem 2.7 below.1.5 Proposition Let X be E-valued, {Y,}-progressive,and {Y,)-Markov,andlet P(t, x, r)be a transition function for X . Let t be a {Y,}-stoppingtime,suppose X is strong Markov at T +t for all t 2 0, and let B E $#(E)ro.m). ThenProof. First consider B of the form(1.21) {X E EO. m): x(ti) E r,,i = I , . ..,n }where 0 s r , 5 t , I;* . s t,, r,,r2,...,I-,, E g(E). Proceeding by inductionon n,for B = { x E Elo* m: x(t,) E r,}we have(1.22) P { x ( ~+ .)E BIY,) = P { x ( ~+ t , ) E r,IE4,}= Or,, X(r), r,)by (l.l6),but this is just (1.20). Suppose now that (1.20) holds for all B of the
  • 169. 160 GENERATORS AND U K O V ?ROC€SSESform (1.21) for some fixed n. Thenfor 0 S tl 5 c2 s * * * s rn and&E RE).Let B be of the form (1.21)with n + 1in place of n. ThenOrdinarily, formulas for transition functions cannot be obtained, the mostnotable exception being that of one-dimensionalBrownian motion given by(1.25)1P(r, x, r)= J-2nr exp {-e}2r dy.Consequently, directly defining a transition function is usually not a usefulmethod of specifyinga Markov process. In this chapter and in Chapters 5 and6, we consider other methods of specifying processes. In particular, in thischapter we exploit the fact that(1.26)
  • 170. 1. MAIKOV PROCESSES AND TRANSITION FUNCTIONS 161defines a measurable contraction semigroup on E(E) (by the Chapman-Kolmogorov property (1 A)).Let (Tct)}be a semigroup on a closed subspace L c B(E).With reference to(M),we say that an E-valued Markov process X corresponds to { T(t)}iffor all s, I L 0 andfe L. Of course if {T(t))is given by a transition function asin (1.26), then (1.27) is just (1.8).1.6 Proposition Let E be separable. Let X be an €-valued Markov processwith initial distribution v corresponding to a semigroup { T(r)]on a closedsubspace L c B(€). If L is separating, then { T(t)}and v determine the finite-dimensional distributions of X.Proof. ForfE L and t 2 0,we have= ~C7WS(x(o))l= W ) S ( x ) W ) .5Since L is separating, u,(T) EE P ( X ( t )E r}is determined. Similarly i f f € L andg E WE),then for 0 s t , < t z rJand the joint distribution of X(r,) and X(t,) is determined (cf. Proposition 4.6of Chapter 3). Proceeding in this manner, the proposition can be proved byinduction. 0Since the finite-dimensional distributions of a Markov process are deter-mined by a corresponding semigroup (T(t)},they are in turn determined by itsfull generator A^ or by a suficiently large set A c A’. One of the bestapproaches for determining when a set is “suficiently large” is through themartingale problem of Stroock and Varadhan, which is based on the observa-tion in the following proposition.
  • 171. 162 CENEMTORS AND W K O V PROcpssEs1.7 Proposition Let X be an E-valued, progressive Markov process withtransition function P(t, x, r)and let {T(t)}and A^ be as above. If (A8) B A^then(130) 4 0 =f (X(l1)- CB ( W ) dsis an (S:}-rnartingale.Proof. For each I, u 2 0(1.31) E C W +u ) l m= ~ / W P ( u ,0We study the basic propertiesof the martingaleproblem in Sections3-7.2. MARKOV jUMP PROCESSES AND FELLER PROCESSESThe simplest Markov process to describe is a Markov jump process with abounded generator. Let p(x, r)be a transition function on E x @(E) and let1E B(E) be nonnegative.Then(2.1) A m 4 = r2(x){ M Y ) - - J ( X ) ) A X . dY)defines a bounded linear operator A on B(€), and A is the generator for aMarkov process in E that can be constructedas follows.
  • 172. (2.3) X(t) = and note that (2.1) can be rewritten asAY(O),Y(k), -!!.L I t < A0 5 L e 04Y(0))’L- I kj =0 4yo” j = o 4YO))’Let { Y’(k),k = 0, I, ...} be a Markov chain in E with initial distribution v andtransition function p’(x, r),and let Y be an independent Poisson process withparameter 1.. Define(2.4) xyt)= Y’(Y(t)), t 2 0.We leave it to the reader to show that X and X’ have the same finite-dimensional distributions (Problem 4).0bserve thatP ! ( X ) = /(Y)P’(X, dv)I(2.7)defines a linear contraction P on B(E) and that, by (2.5). A = A(P - I). Conse-quently, the semigroup { 7’(t))generated by A is given byLetfE B(E).By the Markov property of Y‘(cf.(1.15)).(2.9) ECS(Y’(k + 0)I Y’(0)...., YYOJ = P’YI Y’(N
  • 173. 164 GENERATORS AND MMKOV PROCESSESfor k, 1 = 0, 1,. ..,and we claim that(2.10) ECS(Y(k + W))Is,1= fWX(r))-- P(A n { Y(t)= I}) J Pf(Y(I))dPBPj(X(t))dP.-sA n B n (V(c)=I)Since ( A n B n { V(t)= I} :A E Sy,B E S:; I = 0, 1, 2, ...} is closed underfinite intersections and generates ,Fl,by the Dynkin class theorem (Appendix4) we haveIf(Y ( k + V(t)))dP = pL/(X(r))dP(2.13)for all A E 9,,and (2.10) follows, Finally, since Y has independent increments,I(2.14) ECf(X(t +4)I s 1 1= ECf(y(w +4 - W )+ W )I9,lfor all s, t 2 0. Hence X is a Markov process in E with initial distribution vcorresponding to the semigroup {T(t)}generated by A.We now assume E is locally compact and consider Markov romses withuous functions vanishing at infinity with norm 11f 11 .C sup, I/(x) I. Notesemigroups that are strongly continuous on the Banach space E(E)of contin-
  • 174. 2. MARKOV JUMPmocEssEs AND FELLER mocEssEs 165that e ( E )= C(E) if E is compact. Let A C E be the point at infinity if E isnoncompact and an isolated point if E is compact, and put Ed = E u (A);inthe noncompact case, EA is the one-point compactification of E. We note thatif E is also separable, then EA is metrizable. (See, for example, pages 201 and202 of Cohn (1980).)A semigroup (T(t)] on C(E) is said to be positiue if T(t) is a positiveoperator for each t L 0. (A positive operator is one that maps nonnegativefunctionsto nonnegative functions.)An operator A on e(&)is said to satisfy the positioe maximum principle ifwheneverfe 9 ( A ) ,xo E E, and supxcE f ( ~ )=f(xo) 2 0, we have Af(xo) I0.2.1 lemmathe positive maximum principle is dissipative.Let E be locally compact. A linear operator A on C(E)satisfyingProof. Let J E O(A)and 1 > 0. There exists xo E E such that If(xo)l = 11f 11.Supposef(x,) 2 0 (otherwise replacef by -f).Since supxeE / ( ~ )=f ( x o )2 0,AJ(x,) 5 0 and henceWe restate the Hille-Yosida theorem in the present context.2.2 Theorem Let E be locally compact. The closure A of a linear operator Aon C(E)is single-valuedand generatesa strongly continuous, positive,contrac-tion semigroup { T(t)}on d(E)if and only if:(a) 9 ( A )is dense in e(E).(b) A satisfies the positive maximum principle.(c) - A) is dense in c ( E )for some 2. > 0.Proof. The necessity of (a)and (c)followsfrom Theorem 2.12 of Chapter I. Asfor (b),if/€ 9 ( A ) , xo E E, and supxrEf(x)=J(xo) 2 0, thenfor each t 2 0, so Af(x,) 5 0.Conversely,suppose A satisfies (a)+). Since (b) implies A is dissipative byLemma 2.1, A’ is single-valued and generates a strongly continuous contrac-tion semigroup (T(t)}by Theorem 2.12 of Chapter 1. To complete the proof,we must show that {T(r)}is positive.
  • 175. ia GENERATORS AND MMKOV moasssLet/€ 9(A)and d > 0, and suppose that inf,,,J(x) <0. Choose {h}c9 ( A ) such that (A -A)f,-+ (A -J)f,and let x, E E and xo E E be points atwhichi, andf, respectively,take on their minimum values. Then(2.17) inf, &I- A)f(x)s &(A -A)-/&,)n-. a0s limAj&c,)= Jf(X0)<0,I--where the second inequality is due to the fact that inf,,,.&(x) =f,(x,,) 5; 0 for nsuficiently large. We conclude that iffe 9(A)and A >0, then (A -A)/z0impliesf2 0, so the positivity of(T(t))is a consequence of Corollary 2.8 ofChapter 1. 0An operator A c WE) x B(E)(possiblymultivalued)is said to be conserva-tive if (1, 0) is in the bp-closure ofA. For example, if (1, 0) is in the fullgenerator of a measurable contraction semigroup (T(t)},then T(t)l = 1 for allf 2 0. and conversely. For semigroups given by transition functions, thisproperty isjust the fact that f(t, x, E) = 1.A strongly continuous, positive, contraction semigroup on e(E) whose gen-erator is conservative is called a Feller semigroup,Our aim in this section is toshow (assumingin addition that E is separable)that every Feller semigroupone ( E )corresponds to a Markov process with sample paths in DECOYao). First,however, we require several preliminary results, including our first con-vergence theorem.2.3 lemma Let E be locally compact and separable and let {T(t)}be astrongly continuous,positive,contraction semigroup on t(E). Define the oper-ator TA(t)on C(EA)for each t 2 0 by(2.18) TA(t)f=fa)+ W ( j-f(Ah(We do not distinguish notationally between functions on EA and theirrestrictionsto E.) Then {TA(t)}is a Feller semigroupon C(EA).Proof. It is easy to verify that (Tqr)}is a strongly continuous semigroup onC(EA).Fix t z 0. To show that TA(f)is a positive operator, we must show thatif a E R, SEe(E), and a + / r0, then a + T(t)fr 0. By the positivity ofT(t), T(t)(f+)2 0 and T(t)(f-)2 0. Hence -T(c)/s T(f)(f-),and so(T(t)f)-5 T(t)(f -). Since T(t) is a contraction. ll T ( t # / - )II s llf- JI5 a.Therefore(T(t)f)- $ a, so a + T(t)f2 0.Next, the positivity of TA(c)gives ITA(t)/I5 TA(t)IIf II 5 IIf II for all/ E C(EA),so 11 Tyr)11 = 1. Finally, the generator A” of (T*(t))clearly contains(1. 0). 0
  • 176. 1. MMKOV jUMP MOCESSES AND FELLER MOCESSES 1672.4 Proposition Let E be locally compact and separable. Let {T(t))be astrongly continuous, positive, contraction semigroup on (?((E), and define thesemigroup (TA(t)}on C(EA)as in Lemma 2.3. Let X be a Markov processcorresponding to {TA(t)}with sample paths in D,,[O, a),and leti = inf{t 2 0 : X(r)= A or X(r -) = A}. Then(2.19) P{t < ao, X(T+s) L- A for all s 2 0)= P{t < 00).Let A be the generator of { T(f)}and suppose further that A is conservative. IfP{X(0) E E } = 1, then P{X E D,[O, a)}= I.Proof. Recalling that Ed is metrizable, there exists g E C(E) with g > 0 on Eand g(A) = 0. Put/= e-TA(u)gdu, and note that/> 0 on E andf(A) = 0.By the Markov property of X,(2.20) E[e-f(X(t))l .aF,"t] = e-TA(t- s)j(X(s))= e-s ~~;-TA(uMX(s))du5 e-Y(X(s)), 0 s s < t,so e-f(X(t)) is a nonnegative {.Ff+}-supermartingale.Therefore, (2.19) is aconsequence of Proposition 2.15 of Chapter 2. It also follows thatP{X(t)= A} = P{t s t ) for all r 2 0.Let A denote the generator of {TA(r)}.The assumption that A is conserva-tive (which refers to the bp-closure of A in B(E) x B(E)) implies that (xE, 0) isin the bp-closure of A (considering Ad as a subspace of B(E) x B(E*)). Sincethe collectionof (Jg) E B(Eh) x B(E) satisfying(2.2I )is bp-closed and contains A, for all t 2 0 we have(2.22)and if P{X(0)E E} = I, we concludethat P ( X E DEIO, 00)) = P(t = 00) = 1.P{r > I} = P{X(t)E E} = P{X(O)E E},0A converse to the second assertion of Proposition 2.4 is provided by Corol-lary Theorem Let E be locally compact and separable. For n = I, 2, ... let{T,(t)) be a Feller semigroup on (?(El, and suppose X n is a Markov processcorrespondingto { T&)} with sample paths in Dd[O, 00). Suppose that { T(t)}isa Feller semigroup on e ( E )and that foreach/€ e(E),(2.23)
  • 177. 168 CENauTORS AND MARKOV PROCESSESIf {XJO)} has limitingdistribution u E @E), then there is a Markov process Xcorresponding to {T(t)}with initial distribution u and sample paths inDEIO, a),and X,*X.Proof. For each n 2 1, let A, be the generator of {X(t)}.By Theorem 6.1 ofChapter I, (2.23) implies that for each/€ 9(A),there exist/; E 9(An)such that/,+/and A,f,-, Af. Sincef,(X,(t)) -PoA,L(X,,(s)) ds is an (.FaF,X"}-martingalefor each n 2 1, and since 9 ( A ) is dense in e(E), Chapter 3s Corollary 9.3 andTheorem 9.4 imply that ( X J is relatively compact in D,,[O, 00).We next prove the convergence of the finite-dimensionaldistributions of{X,}.For each n 2 1, let {Tt(r)}and {T"(t)}be the semigroups on C(EA)defined in terms of {T(r))and {T(r)}as in Lemma 2.3.Then, for each/€ C(EA)and t 2 0.(2.24) lim ECf(xn(t))l = lim ECT3t)j(Xn(O))In-m n - a= ITA(l)foc)v(dx)by the Markov property, the strong convergence of {T3r)},the continuity ofTA(r)Jand the convergence in distribution of {XAO)}.Proceeding by induc-tion, let M be a positive integer,and supposethat(2.25)existsfor allJ,, ...,fm E C(EA)and 0 s I, < - * * < r,. Then(2.26) lim ECfI(Xn(t1))..* f m ( X S t m ) ) f m + t(Xa(tm+ 1))ln-m= lim EC-f1(Xn(t1)) * - fm(Xn(tm))T%tm + 1 -tm)fm +I ( x n ( t m ) Un+ OD= lim W"(Xn(f 1)) * * * Sm(xn(tm))TA(tm +1 - t m ) f m + I(XAtm))In-mexistsforallf,, ...,f,+,cC(E")andOst, < - - * < t , , , + , .It follows that every convergent subsequenceof (X,}has the same limit, sothere exists a process X with initial distribution v and with sample paths inDEd[O, GO)such that X,=5 X.By (2.26), X is a Markov process correspondingto { T4(t)}.so by Proposition 2.4, X can be assumed to have sample paths inDEIO, 00). Finally, Corollary 9.3 of Chapter 3 implies Xnr+ X in DEIO,00). 02.6 Theoremp,(x, r)be a transition function on E x &(E) such that T.,defined byLet E be locally compact and separable. For n = 1, 2, ...let(2.27)
  • 178. 2. MADKOV lUMP PROCESSES AND FELLER FROCESSLS 169satisfiesLet E, > 0 satisfy lirn,-m E, = 0 and suppose that for everyfE e(E),:e((E)--+e((E).Suppose that { T(r)}is a Feller semigroup on C(E).(2.28)For each n 2 1, let { Y,(k), k = 0, 1. 2, . ..} be a Markov chain in E withtransition function p,(x, r),and suppose { Y,(0)} has limiting distributionv E P(E). Define X , by X,(t) E V.([t/&,J). Then there is a Markov process Xcorresponding to (T(t)} with initial distribution v and sample paths inD,[O, a),and X,= X.Proof. Following the proof of Theorem 2.5, use Theorem 6.5 of Chapter I inplace of Theorem 6.1. 02.7 Theorem Let E be locally compact and separable, and let { T(t)f be aFeller semigroup on d(&).Then for each v E 9+(E), there exists a Markovprocess X corresponding to (T(t))with initial distribution v and sample pathsin DEIO, 00). Moreover, X is strong Markov with respect to the filtration9,= *:+= nt>o*:+,.Proof. Let n be a positive integer, and let(2.29) A, = A(/ -n - A ) - = n[(/ - n - ~ ) - - /Ibe the Yosida approximation of A. Note that since (I - n-A)-- is a positivecontraction on (?(El, there exists for each .Y E E a positive Borel measurep,(x, I-) on E such that(2.30)for allJE e(E). It follows that p,,(.,r)is Borel measurable for each I- E .g(E).For each (Ag) E A, (2.30)implies(1- n - 4 -!f(x) = S(yk.(x. dy)s(2.31)Since the collection of (J,g) E B(E) x E(E) satisfying (2.31) is bp-closed, itincludes (I, 0)and hence p,,(x, E ) = I for each x E &. implying that p,(x. r)is atransition function on E x .g(E). Therefore, by the discussion at the beginningof this section, the semigroup (T,,(t)} on c(&)with generator A, corresponds toa jump Markov process X, with initial distribution v and with sample paths inNow letting n - r 00, Proposition 2.7 of Chapter 1 implies that for eachT,(t)/= T(t)f;so the existence of X follows fromDECO. 00).J EC(E)and t 2 0,Theorem 2.5.
  • 179. 170 GENERATORS AND MMKOV PROCfsSESLet r be a discrete (g,}-stoppingtime with T < 00 as. concentrated on ( I , ,r 2 , ...}. Let A E Y,, s > 0, and JE e(E). Then An{s = t i } E .%:+& for everyE > 0, so(2.32) 1 /(X(r +s)) dP = 1 AX(t, +s)) dfA n l t -111 Anlr ~ C r l= T(s-E)f(X(t, +E)) dPAnlr =ti)for 0 < E ss and i = I, 2, ... . Since {T(t)}is strongly continuous, T(s)fiscontinuouson E, and X has right continuous sample paths, we can take E = 0in (2.32).This gives(2.33) ECS(X(T + s))Ifsrl = T(s)f(x(~))for discrete r.If r is an arbitrary {S,}-stoppingtime, with r < 00 as., it is the limit of adecreasing sequence {q,} ofdiscrete stopping times (Proposition 1.3 of Chapter2),so (2.33)follows from the continuity of T(s)/on E and the right continuityof the sample paths of X.(Replace7 by T,, in (2.33), condition on Y,, and thenlet n-+ el.) 02.8 Corollary Let E be locally compact and separable. Let A be a linearoperator on c(€)satisfying(a)-@)of Theorem 2.2, and let {T(t)}be the strong-ly continuous, positive, contraction semigroup on c(E) generated by 2.Thenthere exists for each x E E a Markov process X, corresponding to (T(t)}withinitial distribution 6, and with sample paths in Ds[O, 00) if and only if A isconservative.Proof. The sufficiencyfollows from Theorem 2.7. As for necessity, let {g,,} c&(I - A ) satisfy bp-lim,,,g, = 1, and define {f,J c 9 ( A ) byl; = (I - A)-g,.Thenfor all x E €, so bp-lim,,,f, = 1 and bp-limn-.mAf, = bp-lim,,,(J-QrJ = 0. 0We next give criteria for the continuity of the sample paths of the processobtained in Theorem 2.7. Since we know the process has sample paths inDEIO,ao), to show the sample paths are continuous it is enough to show thatthey have nojumps.
  • 180. 2. MARKOV IUMP PROCESSES AND FELLER PROCESSES 1712.9 Proposition Let (E, r) be locally compact and separable,and let { T(r)}bea Feller semigroup on c(E). Let P(t, x, r)be the transition function for (T(r)}and suppose for each x E E and E > 0,(2.35) lim r-P(r, x, B(x, E)C) = 0.l-0Then the process X given by Theorem 2.7 satisfies P ( X E C,[O, a))= I.2.10 Remark Suppose A is the generator of a Feller semigroup {T(r)}onC(E) with transition function P(t, x, r),and that for each x E E and E > 0there exists f E 9 ( A ) with f ( x ) = )I f 11, supy,B,x. rF f ( y ) M < )I f 11, andAJ(x) = 0.Then (2.35) holds.To see this, note that(2.36) ( I1J II - W P ( t ,X. B(x. E)C) SJ(x)- E,CS(X(l))l= - [T(s)A/(x) ds.Divide by r and let t --• 0 to obtain (2.35). 0Proof. Note that for each x E E and 2: 0,(2.37) T;TiiP(t,y,E(y,~)C)slimY X Y-XFor each S > 0 there is a r(x, S)s 6 such that for t = f(x, 6) the right side of(2.37)is less than Bt(x, 8).Consequently, there is a neighborhood U , of x suchthat y E U,implies(2.38) W x , a), y, B(y, E)C) s 2 6 0 , 6).Since any compact subset of E can be covered by finitely many such U , , wecan define a Bore1 measurable function s(y, S) s 6 such that(2.39)and for each compact K c Ef W Y . 4,Y, B(Y, 47 5 2 W y , 6).(2.40) inf Iy,b) > 0.Y ~ KDefine to= 0 and(2.41) TII+ I = TII + ~ ( ~ ( T I I ) ,a).Note that limk+m?k= 00 since (X(s): s 5 t) has compact closure for eachr 2 0.Letn - I(2.42)
  • 181. 172 GENERATORS AND MARKOV PROCESSESand observe thatn - 1(2.43) iic Nd(n)- p(s(X(rh), x(rh), B(x(rk), &r)h - 0is a martingale. Let K c E be compact, let T >0, and defineThen by the optional samplingtheorems E[25~,] 5 2 4 T +5).Finally, observe that limd,, N&) = 1 on the set where X has a jump of sizelarger than E before T and before leaving K.Consequently, with probabilityone, no suchjump occurs. Since E, T,and K are arbitrary, we have the desiredresult. 0We close this section with two theorems generalizingTheorems 2.5 and 2.6.Much more general results are given in Section 8, but these results canbe obtained here using essentially the same argument as in the proof ofTheorem Theorem Let E, El, E,, ... be metric spaces with E locally compactand separable. For n = 1, 2,,.., let 4, :En-+ E be measurable, let (q(f)fbe asemigroup on WE,) given by a transition function, and suppose V, is aMarkov process in En corresponding to (T,(t)} such that X, = rtl, 0 Y,, hassample paths in D,[O, 00). Definen, :B(E)-t B(E,) by n , j = j o , q , , (cf. Section6 ofChapter 1). Suppose that { T(r)}is a Feller semigroup on e(E)and that foreach / E e(E) and f 2 0, T,(t)n,$-+ T(f)f (i.e., il T,(r)n,f- rr, T(t)fI1 -+ 0). If{X,(O)} has limiting distribution v E sP(E), then there is a Markov process Xcorresponding to (T(r)} with initial distribution v and sample paths inO,[O, a)),and X,*X .
  • 182. 3. THE MARTINGALE PROBLEM: CENERALl~ESAND SAMPLE PATH PROPERTIES 173Finally, we give a similar extension of Theorem Theorem Let E, E,, E l , ... be metric spaces with E locally compactand separable. For n = I, 2, ..., let q,, :En--+E be measurable, let p,(x, r)be atransition function on E, x 4?(E,), and suppose { Y,(k), k = 0, I, 2, ...} is aMarkov chain in Encorresponding to p,(x, r).Let E, > 0 satisfy limfi-.m,c, = 0.Define X n ( f ) = Vn( Yn([(Ct/&nJ))*(2.47) T J ( X ) = j/(y)pn(x* dy), SE B(En),and R,: E(E)-+ B(E,) by n,f =f 0 q,. Suppose that { T(t))is a Feller semi-group on C(E) and that for each , f ~e(E) and t >. 0, T!,"em~~,J--+T(t)J If(X,(O)) has limiting distribution v f 9(E),then there is a Markov process Xcorresponding to { T(t)}with initial distribution v and sample paths inDEIO,a),and X , *X.3.SAMPLE PATH PROPERTIESTHE MARTINGALE PROBLEM: GENERALITIES ANDIn Proposition 1.7 we observed that, if X is a Markov process with fullgenerator A, thenis a martingale for all (J g) E 4. In the next several sections we develop theidea of Stroock and Varadhan of using this martingale property as a means ofcharacterizing the Markov process associated with a given generator A. Aselsewhere in this chapter, E (or more specifically (E, r))denotes a metric space.Occasionally we want to allow A to be a multivalued operator (cf. Chapter I,Section 4), and hence think of A as a subset (not necessarily linear) ofWE) x B(E). By a solution of the martingale problemfor A we mean a measur-able stochastic process X with values in E defined on some probability space(n,9,P) such that for each (Jg) E A, (3.1) is a martingale with respect to thefiltrationNote that if X is progressive, in particular if X is right continuous, then9:= F:. In general, every event in *9(xdiffers from an event in 9: by anevent of probability zero. See Problem 2 of Chapter 2.If (9,)is a filtration with Y, 3 *9: for all r 2 0, and (3.1) is a(9,)-martingale for all (5 g) E A, we say X is a solurion oj the miartingale
  • 183. 174 CENEMTOIS AND W K O V PRocEssfsproblem for A with respect to (Y,]. When an initial distribution p e 4yE) isspecified, we say that a solution X of the martingale problem for A is asolution of the martingale problemfor (A,p) if PX(0)- = p.Usually X has sample paths in &[O, 00). It is convenient to call a probabil-ity measure P E. 9(DE[O, 03)) a solution of the martingale problemfor A (or for(A, p))if the coordinateprocess defined on (&LO, oo), Y E ,P)by(3.3) X(t, O)5 4 t h w E D,y[O, OO), t 2 0,is a solution of the martingaleproblem for A (or for (A, p)) as defined above.for A if and only ifNote that a measurable process X is a solution of the martingale problemwhenever 0 5 t , < tz < * * 4 < I,+ I, (f,g) E A, and h,, ...,h, E B(E)(or equiva-lently e(E)).Consequently the statement that a (measurable)process is a solu-tion of a martingale problem is a statement about its finite-dimensionaldistributions. In particular, any measurable modification of a solution of themartingaleproblem for A is also a solution.Let A, denote the linear span of A. Then any solution of the martingaleproblem for A is a solution for As. Note also that, if A") c A(), then anysolution of the martingale problem for A2)is also a solution for A(),but notnecessarily conversely. Finally, observe that the set of pairs (I;g) for which(3.1) is a {fB,}-martingaleis bp-closed. Consequently, any solution of the mar-tingaleproblem for A is a solution for the bp-closureof As.(See Appendix 3.)3.1 Let A" and A" be subsets of S(E)x HE). If the bp-closures of (A"))s and (A), are equal, then X is a solution of the martingaleproblem for A" if and only if it is a solutionfor A").PropositionProof. This is immediatefrom the discussionabove. 0The followinglemma gives two useful equivalencesto (3.1) being a martin-gale.3.2 lemmaThen for fixed 1 E R,(3.1) is a {Y,}-martingaleif and only ifLet X be a measurable process, Y, 3 *9:,and let j ; g E B(E).(3.5)
  • 184. 3. THE MARTINGALE PROBLEM: GENERALITIES AND SAMPLE PATH PROPERTIES 175is a {Y,}-martingale. If inf,/(x) > 0, then (3.1)is a {5fl]-martingale if and onlyif(3.6)is a {Y,}-martingale.Proof. If (3.1) is a (Y,}-martingale, then by Proposition 3.2 of Chapter 2 (seeProblem 22 of the same chapter),- l g (X(s))ds e -I - g(X(u)) du Re -Is dssbl= e-YY(X(r))+ e-"[Af(X(s)) - g(X(s))]dsJrbis a {4/,)-martingale. (The last equality follows by Fubinis theorem.) Ifinf,/(x) > 0 and (3.1)is a {Y,)-martingale,then
  • 185. 176 GENERATORS AND W K O V PUOCES-is a {91)-martingale. The converses follow by similar calculations(Problem 14). aThe above lemma gives the following equivalent formulations of the mar-tingale problem.3.3 Proposition Let A be a linear subset of B(E) x B(E)containing(I, 0) anddefinc(3.9) A + = ((1;8) E A: inf,f(x) >0).Let X be a measurable E-valued process and let 3,a*.F,".Then the followingare equivalent:(4 X is a solution of the martingaleproblem for A with respect to (Cg,},(b) X is a solution of the martingale problem for A + with respect to(c) For each (1;g) E A, (3.5)is a (9,)-martingale.(d) For each (J8)E A+, (3.6) is a {91}-martingale,{CgI}.Proof. Since (A), = A, (a) and (b) are equivalent. The other equivalencesfollow by Lemma 3.2. 0For right continuous X,the fact that (3.5) is a martingale whenever (3.1) is,is a special case of the followinglemma.3.4 Lemma Let X be a measurable stochastic process on ( R , 9 , P) withvalues in E. Let u, u: [0, 00) x E x R-, R be bounded anda[O,00) x a(E)x $-measurable, and let w: [0, a)) x [O, 00) x E x R-, Rbe bounded and B[O,00) x S[O, 00) x a(E)x 9-measurable. Assume thatu(t, x, w)is continuous in x for fixed t and w,that u(t, X(r))is adapted to afiltration {Y,}, and that u(t, X(r))and W(t, I, X(r))are {44,}-progressive.Supposefurther that the conditionsin either(a)or (b)hold:(a) For every r1 > t , 2 0,and
  • 186. 3. THE MARTINGALE PROIMLEM: GENERALITIES AND SAMRE PATH PROPERTIES 177Moreover, X is right continuous and(3.12)(b)lim E[I w(t - 6, t , X(t))- w(r, r, X(t))l] = 0, t z 0.d + O +For every t 2 > t , 2 0,andMoreover, X is left continuous and(3.15) lim E[ Iw(r + 6, t, X(r)) - w(t, t, X(r))I] = 0, t 2 0.&do+Under the above assumptions,f,(3.16)is a {Y,}-martingale.Proof.we haveFix I, > t , 2 0. For any partition f , = so < s, < s, < . . < s, = /, ,= E[ ~:*{u(s,X(S)]+ w(s", S, X(.S))]ds q,,I 1
  • 187. 178 GENERATORS AND MMKOV PROCESSESClearly, only dissipative operators arise as generators of Markov processes.One consequence of Lemma 3.2 is that we must still restrict our attention todissipativeoperatorsin order to have solutionsof the martingaleproblem.3.5 Proposition Let A be a linear subset of WE)x B(E). If there exists asolution X , of the martingale problem for (A, 6,) for each x E E, then A isdissipative.Proof. Given (S, g) E A and d > 0, (3.5)is a martingaleand henceAs stated above, we usually are interested in solutions with sample paths inD,[O, m). The follcwing theorem demonstrates that in most cases this is not arestriction.3.6 Theorem Let E be separable. Let A c c(E) x B(E) and suppose that9 ( A )is separating and contains a countable subset that separates points. LetX be a solution of the martingale problem for A and assume that for everyE > 0 and T > 0, there exists a compact set K ,,such that(3.22) P{X(t)E Kt,,. for all t E [O, T ] n QJ> 1 - E.Then there is a modification of X with sample paths in D,[O, 00).Proof. Let X be defined on (Q, 9,P).By assumption,there exists a sequence{(fi, 8,))c A such that {jJseparates points in E. By Proposition 2.9 ofChapter 2, there exists R c Q with P(W)= I such that(3.23) J;(W)- s"...))ds0has limits through the rationals from above and below for all r 2 0, all i, andall w E R. By (3.22) there exists 0"t Q with P(Q") = I such that {X(r,w):r E [O, T3 n Q} has compact closure for all T > 0 and w E Q". Supposew E W. Then for each f 2 0 there exist s, E Q such that s, > 1, lime.+*sn= t,and lim, X(s, ,w )exists,and hence(3.24)
  • 188. 3. THE MARTlNCAL€ PROBLEM: C€N€RALtTW AND S W € tATH PRO?ERNES 179where the limit on the right exists since w E R.Since {/I}separates points wehave(3.25) lim X(s)3 Y(t)*-.I+S € Qexists for all t 2 0 and w E R". Similarly(3.26) lim X(s) = Y - ( t )1-1 -3.Qexists for all t > 0 and w E R", so Y has sample paths in D,[O, 00) by Lemma2.8of Chapter 2.(3.27)Since X is a solution of the martingale problem, if follows thatECS(Y(t))I9:1 = lim ~ C S ( X ( S ) )I9:1 =S ( W )3 - 1 +S S Qfor every/€ 9 ( A ) and t 2 0. Since 9 ( A ) is separating, P( Y(t)= X(t)) = I for0all r 2 0. (See Problem 7 of Chapter 3.)3.7 Corollary Let E be locally compact and separable. Let A c e(€)x B(E)and supposc that 9 ( A ) is dense in e(€)in the norm topology. Then anysolution of the martingale problem for A has a modification with sample pathsin DEb[O,00) where EAis the one-point compactificationof E.and A" = Then any solution of the martingale problem for A consideredas a process with values in EA is a solution of the martingale problem for A".0Since A" satisfies the conditions of Theorem 3.6,the corollary follows.In the light of condition (3.22)and in particular Corollary 3.7,it is some-times useful to first prove the existence of a modification with sample paths inD,[O, 00) (where i!? is some compactification of E) and then to prove that themodification actually has sample paths in DEIO,00). With this in mind weprove the following theorem.3.8 Theorem Let (k,r) be a metric space and let A c E(E) x E(&. LetE c E be open, and suppose that X is a solution OC the martingale problemfor A with sample paths in &[O, 00). Suppose (xE,0) is in the bp-closure ofA n (C(&)x I?(&). If P(X(0)E E} = 1. then P ( X E D,[O, 00)) = 1.
  • 189. 180 GENERATORS AND MARKOV PROCESSESProof.(3.29)Then r , s t 2sB - E if and only if lim,-.m t, E T $ t . For (f,g) E: A n(c(8)x B(&)),For m = 1, 2, ...,define the {P:+)-stoppIng timet, = inf t : inf,,.g-&r(y,X(t))<- .i in7* and lirnm+mX(7,Ac) s Y(r) exists. Note that Y(r) is in(3.30)is a right continuous {Sf}-martingale, and hence the optional samplingtheorem implies that for each r L 0,(3.31) E[f(x(rm A r ) ) ~= EC/(X(O))I+ .[s"g(x(S))ds].0Letting m -+ ao,we have(3.32) W(Y(t)ll = ECf(X(O))J+ E[J^B(X(S))0 ds],and this holds for all (f,g) in the bp-closure of A n (&!?) x @)). TakingLA d = (XE ? O), we have(3.33) P{r > t ) = P{Y(r)E E} = 1, t 2 0.Consequently, with probability 1, X has no limit points in fi - E on anybounded time interval and therefore has almost all samplepaths in DJO, 00).n3.9 Proposition Let i!?, A, and X be as above. Let E c A!? be open. Supposethere exists { ( f n 9 g")} c A n (c@)x B(k))such that(3.34)(3.35)and {g,} converges pointwise to zero. If P(X(0)e E) = 1, then e { X EU,[O, m)}= 1.Proof. Substituting(f. gn)in (3.32) and letting n-+ a,Fatous lemma gives(3.36) P{Y(t)E E ) 2 P(X(0)E E} = 1. 03.10 Proposition Let A!?, A, and X be as above. Let E,, E l , ... be opensubsets of k alid let E =: nkEk. Suppose (xS, 0) is in the bp-closure ofA n (e(&x BIkU. If P{X(O)E E} = 1, then P{X B OSLO,00)) = 1.
  • 190. 3. THE MARTINGALE PROlLEM: GENERALITIES AND SAMPLE PATH PROPERTIES 181Proof. Let rk be defined as in (3.29) with E replaced by E,. Then the analogueof (3.32)gives(3.37) P(lim,,,X(r~At) E E,} 2 P{limm+mX(7~At)E E } = I.Therefore almost all sample paths of X are in DJO, no) for every k, and hencein DJO, a). 03.11 In the application of Theorem 3.8 and Propositions 3.9 and3.10, E might be locally compact and tf? = EA,or E = n,F,, where the F, are0Remarklocally compact, i? = nkF,d, and E, = n , < k F, x n,,kF f .We close this section by showing, under the conditions of Theorem 3.6, thatany solution of the martingale problem for A with sample paths in D,[O, 00) isquasi-left conrinuous, that is, for every nondecreasing sequence of stoppingtimes r, with limw-mr, = r .c co a.s., we have limn-.mX(r,,)= X(r)as.3.12 Theorem Let E be separable. Let A c C(E)x B(E)and suppose 6 ( A )isseparating. Let X be a solution of the martingale problem for A with respectto {Y,}, having sample paths in D,[O, 00). Let T~ s T~ s . . be a sequence of(9,)-stopping times and let r =(3.38)In particular, P(X(t) = X(t-)} = 1 for each