This document describes Markov and semi-Markov models for modeling the joint lifetimes of husbands and wives. It summarizes the key aspects of the models, including: 1) The models represent the possible states a couple can be in and transitions between those states. 2) Parameters are estimated by maximizing the likelihood of observed data on joint-life contracts. 3) A semi-Markov model is introduced to characterize the impact of spousal death through a function of time since bereavement.

1. MARKOVIAN APPROACHES TO JOINT-LIFE MORTALITY
ACTUARIAL SCIENCE PART 1
SHAPAH SHADRACH
(10507649)

2. INTRODUCTION
Several empirical studies in recent years suggest considerable
dependence between the lifetimes of a husband and wife.
Denuit et al. (2001) argue that a husband and wife are exposed to
similar risks, since they share common lifestyles and may encounter
common disasters.
Many insurance products provide benefits that are contingent on the
combined survival status of two lives. (Husband and Wife)

3. • To value such benefits accurately, we require a statistical model for the
impact of the survivorship of one life on another.
• In this paper, we first set up two models, one Markov and one semi-
Markov, to model the dependence between the lifetimes of a husband
and wife
• Transitions between states are governed by a matrix of transition
intensities and based on the properties of the transition intensities, we
either use (fully) Markov or semi Markov model.

4. • From the models we can measure the extent of three types of
dependence:
1. the instantaneous dependence due to a catastrophic event that
affect both lives;
2. the short-term impact of spousal death;
3. the long-term association between lifetimes.

5. • Then we apply the models to a set of joint-life and last-survivor annuity
data from a large Canadian insurance company.
• Given the fitted models, we study the impact of dependence on
annuity values, and examine the potential inaccuracy in pricing if we
assume lifetimes are independent

6. • ASSUMPTION :We assume that for any states i and j and any times t
and t+s, where s ≥0, the conditional probability Pr[Y(t+s)=j|Y(t) =i] is well
defined in the sense that its value does not depend on any information
about the process before time t.
• Intuitively, this means that the probabilities of future events for the
process are completely determined by knowing the current state of the
process. In particular, these probabilities do not depend on how the
process arrived at the current state or how long it has been in the
current state

7. • This property, that probabilities of future events depend on the
present but not on the past, is known as the Markov property. Using
the language of probability theory, we are assuming that {Y(t)}t≥0 is a
Markov process.

8. • First, we introduce to the model common shock factor, which is
associated with time of catastrophe, for the example plane crash that
affects both lives.
• Secondly, we extend Nordberg’s original Markov model to a semi
Markov model, which characterizes the impact of bereavement on
mortality through a smooth parametric function of the time elapse since
bereavement and this gives more information on how bereavement
diminishes with time.

9. • LITERATURE REVIEW
• Several empirical studies in the recent years suggest considerable
dependence between the lifetime of a husband and a wife.
• Deut et al (2001) argued that a husband and a wife are exposed to the
similar risk since thy share common lifestyle and may encounter
common disasters.
• Sutton and Jagger (1991) shows that there is an increased relative risk
of mortality following spousal bereavement.

10. • Markov multiple state models have been used in diverse ways in
Actuarial Science. Sverdrup (1965) and Waters (1984) both considered
models where the states represented different health status.
• The first application to joint-life mortality modeling may be by
Nordberg (1989). Spreeuw and Wang (2008) extended the work of
Nordberg by allowing mortality to vary with the time elapse since death
of spouse.

11. • Dickson et al (2009) explain how finite Markov models may be used in
modeling various insurance benefits including joint-life, critical illness,
accidental death and income replacements insurances.
• Another study conducted simultaneously by Spreeuw and Wang
(2008), considers difference extension of Nordberg’s model.

12. THE ORGANIZATION OF THE THESIS
1 DATA
2 MARKOVIAN MODEL
3 PARAMETER ESTIMATION
4 SEMI MARKOVIAN MODEL WITH PARAMERIC FUNCTION FOR
MODELING BROKEN HEART EFFECT.
5 COMPARISON OF ANNUITY PRICES USING THE MORKOV MODEL
WITH THE SEMI MARKOV MODEL
7 OBSERVATIONS
6 CONCLUDING REMARKS

13. DATA
• The data used in this paper were developed in research reported in
Frees et al (1996) funded by the Society of Actuaries.
• The data set contains 14947 records of joint and last-survivor annuity
contracts over an observation period from December, 29 1988 to
December, 31 1993.

14. Age at inception
(Females)
No. of persons-
at-risk
No. of deaths
60 – 69 6397 177
70 – 79 2376 188
80 – 89 192 49
90 + 10 7
Total 8975 421
Table 1a. The number of person-at-risk at the beginning of the
observation period and number of deaths during observation period.

15. Age at inception
(Males)
No. of persons-at-risk No. of deaths
60 – 69 4949 373
70 – 79 3684 586
80 – 89 318 130
90 + 24 19
Total 8975 1108
Table 1. The number of person-at-risk at the beginning of the
observation period and number of deaths during observation
period.

16. Female Male
Survived 8635 7924
Died within 5 days of spouse’s death 52 52
Died at least 5 days before/after spouse’s death 288 999
Total 8975 8975
Table 2. Breakdown of data by survival status
at the end of the observation period.

17. •In table 2 we show the breakdown of the data by the following
categories;
•1) Survived to the end of the observation period;
• 2) Died at least 5 days before or after the spouse’s death;
• 3) Died within 5 days before or after spouse’s death.

18. MARKOV MODEL
MODEL SPECIFICATION
• At this point we have to illustrate the concept of multiple state
modelling with a simple Markov model.
• 𝑆𝑡, 𝑡 ≥ 0 , is a Markov process for any u > v > 0, the conditional
probability distribution of X(u), given the whole history of the process up
to and including the time v, depends on the value of X(v) only. This
process can be conveniently represented by a diagram as shown below;

19. Figure 1.
Specification of the Markov model

20. • The boxes represent the four possible states that a couple can be in at any
time and the arrows indicate the possible transitions between these states.
• The process has state space S 0,1,2,3 . This means that, for example, 𝑆𝑡= 0
if husband and wife are alive at time t.
• We assume that the force of mortality for an individual depends on his/her
marital status but not on his/her spouse’s age.
• Let us suppose that the current ages of a wife and a husband are x and y
respectively

21. • The wife force of mortality at age x+t is denoted by 𝜇 𝑥+𝑡
∗
if she is
bereaved and 𝜇 𝑥+𝑡if husband still alive
• Likewise, the husband’s force of mortality at current age y+ t, is 𝜇 𝑦+𝑡
∗
if
he is widowed, while 𝜇 𝑦+𝑡 denotes force of mortality if wife alive.
• The ‘common shock’ factor allow the process to move from state 0 to
state 3 directly with 𝜇03
force of mortality.

22. • In many applications of the model, we require the following transition
probabilities:
𝑡𝑃𝑥
𝑖𝑗
= P (𝑆 𝑥+𝑡=j/𝑆 𝑥=i) i, j =0, 1, 2, 3 x, t ≥ 0
𝑡𝑃𝑥
𝑖𝑖
= Pr[𝑆 𝑥+𝑠 = 𝑖 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑠 ∈ 𝑜, 𝑡 ∕ 𝑆 𝑥 = 𝑖]

23. ASSUPTIONS:
• Assumption (1)
The probability of two or more transitions in a small interval h is 0(h),
where 0(⋅) is a function such that, lim
ℎ→0
0(ℎ)
ℎ
= 0
• Assumption (2)
t𝑃𝑥
𝑖𝑗
is differentiable function of t.

24. Given those two assumptions we can compute 𝑡𝑃𝑥
𝑖𝑗
by the Kolmogorov
forward equations, which can be written in the compact form as:
•
𝜕
𝜕𝑥
𝑃 𝑥, 𝑥 + 𝑡 = 𝑃(𝑥, 𝑥 + 𝑡)A(x+t), x, t≥ 0 ,
• Where P(x, x+t) is the matrix which the (i, j)th entry is 𝑡𝑃𝑥
𝑖𝑗
, i, j=1, 2, 3
and A(x+t) is the intensity matrix. The (i, j)th element is 𝜇(𝑥+𝑡)
𝑖𝑗
for i≠
𝑗 and − 𝑗=0,𝑗≠𝑖
3
𝜇 𝑖𝑗
(𝑥 + 𝑡) for i= j.

25. • Solving these system of partial differential equations, we obtain the
following expressions for the transition probabilities:
• 𝑡𝑃𝑥:𝑦
00
= exp. ( − 0
𝑡
𝜇 𝑥+𝑠 + 𝜇 𝑦+𝑠 + 𝜇03
𝑑𝑠);
• 𝑡𝑃𝑥
11
=exp. (− 0
𝑡
𝜇 𝑥+𝑡
∗
𝑑𝑠)
• 𝑡𝑃𝑥:𝑦
01
= ( 0
𝑡
𝑠𝑃𝑥:𝑦
00
𝜇 𝑦+𝑠 𝑡−𝑠 𝑃𝑥+𝑠
11
ds)
• 𝑡𝑃𝑥:𝑦
02
= ( 0
𝑡
𝑠𝑃𝑥:𝑦
00
𝜇 𝑥+𝑠 𝑡−𝑠 𝑃𝑦+𝑠
22
ds)
• 𝑡𝑃𝑥
13
= ( 0
𝑡
𝑠𝑃𝑥
11
𝜇 𝑥+𝑠
∗
ds)
• t𝑃𝑦
23
= ( 0
𝑡
𝑠𝑃𝑦
22
𝜇 𝑦+𝑠
∗
𝑑𝑠)

26. PARAMETER ESTIMATION
• Let 𝑇𝑥 and 𝑇𝑦 be the remaining lifetimes of a wife and a husband
respectively. The joint density function for 𝑇𝑥 𝑎𝑛𝑑 𝑇𝑦 can be loosely
expressed as;
• 𝑓𝑇𝑥, 𝑇𝑦 (𝑢, 𝑣) =
𝑢𝑃𝑥:𝑦 𝑣−𝑢
00
𝑃𝑦+𝑢
22
𝜇 𝑥+𝑢 𝜇 𝑦+𝑣
∗
, 𝑖𝑓 𝑢 < 𝑣,
𝑣𝑃𝑥:𝑦 𝑢−𝑣
00
𝑃𝑥+𝑣
11
𝜇 𝑦+𝑣 𝜇 𝑥+𝑢
∗
, 𝑖𝑓 𝑢 > 𝑣,
• 𝑓𝑇𝑥, 𝑇𝑦(𝑢, 𝑣) = 𝑢𝑃𝑥:𝑦
00
𝜇03

27. Assuming independence among the different couples in the data, the log-
likelihood function can be written as a sum of three separate parts,
𝑙1, 𝑙2 𝑎𝑛𝑑 𝑙3 where;
• 𝑙1 = 𝑖=1
𝑛
(− 0
𝑣 𝑖
(𝜇 𝑥 𝑖+𝑡+ 𝜇 𝑦 𝑖+𝑡 + 𝜇03
)𝑑𝑥 + 𝑑𝑖
1
𝑙𝑛𝜇 𝑦 𝑖+𝑣 𝑖
+ 𝑑𝑖
2
𝑙𝑛𝜇 𝑥 𝑖+𝑣 𝑖
+
𝑑𝑖
3
𝑙𝑛𝜇03
)
• 𝑙2 = 𝑗=1
𝑚1
(− 0
𝑢1,𝑗
𝜇 𝑥 𝑗+𝑡
∗
𝑑𝑥 + ℎ1,𝑗 𝑙𝑛𝜇 𝑥 𝑗+𝑢1,𝑗
∗
)
• 𝑙3 = 𝑘=1
𝑚1
(− 0
𝑢2,𝑘
𝜇 𝑦 𝑘+𝑡
∗
𝑑𝑥 + ℎ2,𝑘 𝑙𝑛𝜇 𝑦 𝑘+ 𝜇2,𝑘
∗
)

28. Where,
 𝑛, is the total number of couples in the data set.
 𝑚1, (𝑚2), is the number of widow (widowers) in the data set.
 𝑣𝑖, is the time until the 𝑖 𝑡ℎ
couple exits state 0, 𝑖 = 1,2,3, … , 𝑛
 time 𝑣𝑖, 𝑖 = 1,2, … , 𝑛, 𝑗 = 1,2,3.
• 𝑢(𝑢2,𝑘 𝑖𝑠 𝑡ℎ𝑒 𝑡𝑖𝑚𝑒 𝑢𝑛𝑡𝑖𝑙𝑙 𝑡ℎ𝑒 𝑗𝑡ℎ 𝑘𝑡ℎ 𝑤𝑖𝑑𝑜𝑤 𝑤𝑖𝑑𝑜𝑤𝑒𝑟 𝑠𝑡𝑎𝑡𝑒 1
or 2, j= 1,2,3, …, 𝑚1, k=1,2,3, … , 𝑚2

29. • ℎ1,𝑗 = 1 𝑖𝑓 𝑡ℎ𝑒 𝑗 𝑡ℎ
𝑤𝑖𝑑𝑜𝑤 𝑑𝑖𝑒𝑠 𝑎𝑡 𝑡𝑖𝑚𝑒 𝑡 = 𝑢1,𝑗,
 ℎ2,𝑗 = 1 𝑖𝑓 𝑡ℎ𝑒 𝑘 𝑡ℎ
𝑤𝑖𝑑𝑜𝑤𝑒𝑟 𝑑𝑖𝑒𝑠 𝑎𝑡 𝑡𝑖𝑚𝑒 𝑡 = 𝑢2,𝑘,
 𝑥𝑖 𝑎𝑛𝑑 𝑦𝑖 𝑎𝑟𝑒 𝑡ℎ𝑒 𝑒𝑛𝑡𝑟𝑦 𝑎𝑔𝑒𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑤𝑖𝑓𝑒 𝑎𝑛𝑑 ℎ𝑢𝑠𝑏𝑎𝑛𝑑, 𝑟𝑒𝑠𝑝𝑒𝑐𝑡𝑖𝑣𝑒𝑙𝑦.

30. • By maximizing the three parts of log-likelihood function separately, we
can get the maximum likelihood estimates of the transition intensities
in each states
 For the right censored data, 𝑑1
𝑗
, ℎ1𝑗 𝑎𝑛𝑑 ℎ2𝑗
will be zero.

31. • In this study we graduate the forces of mortality using Gompertz’ law,
𝜇 𝑋 = 𝐵𝐶 𝑋
.
• Assuming then, that for both sexes, mortality in state 0 follows
Gompertz’ law, we can rewrite 𝑙1 as:

32. • ,
• Where (𝐵1, 𝐶1) and (𝐵2, 𝐶2) are the Gompertz parameters for female
and male mortality in state 0, respectively. We can rewrite 𝑙2 and 𝑙3 in a
similar manner. The maximum likelihood estimate of 𝜇03
is 0.1407%,
and its standard error is 0.0195%.

33. B STANDARD ERROR C STANDARD ERROR
FEMALE
𝜇 𝑥 9.741× 10−7 2.889× 10−7 1.1331 0.0047
𝜇 𝑥
∗
2.638× 10−5 3.370× 10−5 1.1020 0.0181
MALES
𝜇 𝑦 2.622× 10−5
1.038× 10−5 1.0989 0.0058
𝜇 𝑦
∗ 3.899× 10−4 4.057× 10−4 1.0725 0.0136
Maximum likelihood estimates of other parameters are displayed in
Table 3. Parameters 𝜇 𝑥
∗
and 𝜇 𝑦
∗
have higher standard errors, since the
number of individuals who transition to states 1 or 2 is relatively small,
even in this extensive dataset.
Table 3.

34. • In Figure 2 we plot the fitted forces of mortality in different states. We
observe, for both sexes, an increased force of mortality after
bereavement, after allowing for common cause deaths using the 5-day
common ‘shock’ allocation.
• We can further observe from Figure 2 that bereavement effects vary
with age, as the mortality curves do not shift in parallel.
• Common cause deaths with more than 5 days between events could be
the cause of the increase in mortality post-bereavement;

35. Forcesofmortalitybymaritalstatus(marriedorwidowed)

36. • To examine whether the Gompertz’ laws give an adequate fit, we perform a
χ-square test.
• For µx, µ∗x and µ∗y, the null hypothesis that model gives an adequate fit is
not rejected at 5% level of significance, but for µy, the null hypothesis is
marginally rejected (the p-value is 0.042)
• The fit for µy can be improved by using Makeham’slaw, µx = A + BCx, which
increases the p-value for the χ-square to 0.13, indicating an adequate fit.
• However, for consistency reasons, we use Gompertz’ law for all four forces
of mortality, even though Makeham’s law may better fit µy.

37. • THE SEMI-MARKOV MODEL
• Model specification
• The Markov model described above is somewhat rigid, in that the
bereavement effect is assumed to be constant, regardless of the length
of time since the spouse’s death.
• While it might be reasonable that mortality of widow(er)s is generally
higher than married individuals of the same age, it also seem
reasonable to consider that the detrimental impact of bereavement on
the surviving spouse’s health might be stronger in the months
immediately following the spouse’s death than it is later on.

38. Force of mortality by period since bereavement;
𝜇 𝑥 𝑦 Ι 0
∗
in first year,
𝜇 𝑥 𝑦 Ι1
∗
in second year,
𝜇x(y)Ι 2
∗
+ after the second year.

39. • Figure 3 indicates that the bereavement mortality dynamics depend on
the period since bereavement, which implies that a semi-Markov
approach might better capture the dynamics.
• We observe that, at any given age, mortality is highest in the year
following widow(er)hood, and lowest two years later.

40. • We use the following parametric functions to model the force of mortality
after bereavement:
For widows,
𝜇∗
𝑥, 𝑡 = 1 + 𝑎1 𝑒−𝑘1 𝑡
𝜇 𝑥+𝑡 + 𝜇03
= 𝐹1(𝑡)(𝜇 𝑥+𝑡 + 𝜇03
)
For widowers,
𝜇∗
𝑦 + 𝑡 = 1 + 𝑎2 𝑒−𝑘2 𝑡
𝜇 𝑦+𝑡 + 𝜇03
= 𝐹2(𝑡)(𝜇 𝑦+𝑡 + 𝜇03
)
Where t, is the time since bereavement

41. • Initially, bereavement increases the force of mortality by a percentage
of 100𝑎1% for females and 100𝑎2% for males.
• As t increases, the multiplicative factors, 𝐹1 𝑡 1 +

43. • PARAMETER ESTIMATION
• Because the semi-Markov extension affects post-bereavement
mortality only, there is no change to the meaning and values of𝜇 𝑥,𝜇 𝑦
and 𝜇03
.
• Given the estimates of𝜇 𝑥,𝜇 𝑦and, 𝜇03
the remaining parameters can be
estimated by partial maximum likelihood estimation.

44. • The partial likelihood function 𝑙1
𝑝
for parameters 𝑎1 and 𝑘1is given by
• 𝑙1
𝑝
= 𝑗=1
𝑚1
− 0
𝑢1 𝑗
(1 + 𝑎1 𝑒−𝑘1 𝑡
( 𝐵1 𝐶1
𝑥+𝑡
+ 𝜇03
)𝑑𝑡+ ℎ1,𝑗ln( 1 +

45. • By maximizing 𝑙1
𝑝
and,𝑙2
𝑝
we can obtain estimates for the semi-Markov
parameters. The estimates of 𝑎1, 𝑎2 , 𝑘1 𝑎𝑛𝑑 𝑘2 and their approximate
standard errors are shown in Table below.
• The standard errors in Tables 3 and 4 are estimated using numerical
approximation of the second derivative of the likelihood function
Central estimates Standard error
Females
𝑎1 3.3845 0.9164
𝑘1 0.5216 0.2468
Males
𝑎2 11.0530 4.5080
𝑘2 7.9070 3.2293
Estimates of parameters 𝑎1,𝑎2, 𝑘1,𝑘2in the semi-Markov model

46. • Upper panel focuses on the first year after bereavement, and that
widowers are subject to a much higher broken heart effect shortly after
bereavement and lower panel indicates, the broken heart effect for
widows is more persistent than that for widowers.

47. IDENTIFYING COMMON SHOCK DEATHS
• In the semi-Markov model, the following two separate effects are
explicitly modeled:
1. The impact in the first five days, where we assume that deaths are
simultaneous using the common shock approach.
2. The impact after the first five days, which we have termed the
bereavement or broken-heart effect.

48. • In building the model though, the threshold for defining simultaneous
deaths is important, particularly if our aim is to separate common-
cause impact from broken-heart impact.
• If the threshold is set too long, some deaths associated with the
broken-heart effect will be misclassified as simultaneous deaths,
leading to an overestimation of 𝜇03
.

49. • If the threshold is set too short, some simultaneous deaths will be
misclassified, affecting the shape of the multiplicative factors
𝐹1 𝑡 and𝐹1 𝑡 , which are intended to model the broken-heart and not
the common shock effect.

50. OBSERVATIONS
• LAST SURVIVOR ANNUITY VALUES
• Both the Markov and semi-Markov models indicate an increase in
mortality after bereavement. However, the persistency is different.
• While the semi-Markov model allows recovery from bereavement, the
Markov model assumes that the increase in mortality is permanent.
Such a difference has an impact on annuity values.
• First, let us consider the Markov model. The three-dimensional plot
below shows the ratios of last survivor annuity values using the
Markov-model, to values assuming independent lifetimes.

51. Three-dimensional plot of the ratios of Markov-model-based
on independent last-survivor annuity values.

52. • All ratios in the plot are less than 1, confirming that last-survivor
annuities are overpriced if the assumption of independent lifetimes is
used, as with the Markov model.
• We also observe that the annuity ratios are lower when the gap
between the spouses’ ages is larger. This observation implies the effect
of long-term dependence is more significant when the age gap |x − y| is
wider.

53. • From the plots, the ratio of the last-survivor annuity values using the
semi-Markov model to those based on the assumption of
independence, we observed that these ratios are closer to 1.0; a little
lower at most age combinations.
• We also note that the plot is asymmetric, as we would expect, given
the different patterns for males and females of the impact and
likelihood of bereavement.
• Although all the values shown are less than 1.0, it is possible for the
ratio to exceed 1.0 for certain age combinations, unlike the Markov
case which requires all ratios for last survivor annuity values to be less
than 1.0.

54. • CONCLUDING REMARKS
• Intuitively there is dependence between the lifetimes of a husband
and wife is, but the nature of the dependence is not clear from pure
empirical observations.
• Through both models we fit to the annuitants’ mortality data, we have
a better understanding of two different aspects of dependence
between lifetimes.
• First, the common shock factor 𝜇03
tells us the risk of a catastrophic
event that will affect both lives.

55. • Second, in the semi-Markov model, factors 𝐹1 𝑡 and𝐹2 𝑡 , measure
the impact of spousal death on mortality and the rate at which this
reduces with time.
• Given the flexibility of Markovian models, it would be interesting to
consider in future research, how they may be applied to the valuation
of more complex financial products such as reverse mortgage
contracts, of which the times to maturity depend on not only lifetimes,
but also other factors such as the timing of long-term care and
prepayments

56. • References
• Bowers N, Gerber H., Hickman J. Jones, D., Nesbitt C. (1997) Actuarial Mathematics (2nd Edition. Society of Actuaries,
Schaumburg Il.
• Denuit, Michel, Jan Dhaene, C´eline Le Bailly de Tilleghem and St´ephanie Teghem.
• (2001). Measuring the Impact of a Dependence among Insured Lifelengths.
• Belgian Actuarial Bulletin, 1: 18-39.
• Dickson, David C.M., Hardy Mary R. and Waters Howard R. (2009). Actuarial
• Mathematics for Life Contingent Risk. Cambridge University Press, Cambridge, U.K.
• Frees, Edward W., Jacques Carriere, and Emiliano Valdez. (1996). Annuity Valuation with Dependent Mortality. The Journal
of Risk and Insurance, 63: 229-261.
• Frees, Edward W. and Emiliano Valdez. (1998). Understanding Relationships Using
• Copulas. North American Actuarial Journal, 2: 1-25.
• Jagger, Carol and Christopher J. Sutton. (1991). Death after Marital Bereavement
• – Is the Risk Increased? Statistics in Medicine, 10: 395-404.
• Ji, Min (2010) Markovian Approaches to Joint Life Mortality with Applications in
• Risk Management. PhD Thesis Proposal, University of Waterloo.
• Kim, Changki. (2005). Modeling Surrender and Lapse Rates with Economic Variables. North American Actuarial Journal, 9:
56-70.
• Klugman, Stuart, Harry H. Panjer and Gordon E. Willmot. (2008). Loss Models:
• from Data to Decisions. 3rd edition. New York: Wiley.
• Lehmann, Erich Leo. (1966). Some Concepts of Dependence. The Annals of
• Mathematical Statistics, 37: 1137-1153.

57. THANKS