This is my Master thesis in statistics from Cairo university. It is about using a continuous time Markov chains in the analysis of non-alcoholic fatty liver disease or the newly proposed name (metabolic associates fatty liver disease) .

1. Cairo University
Faculty of Graduate Studies
for Statistical Research
Mathematical Statistics
Department
Analysis of Chronic Diseases Progression
Using Stochastic Models
M.Sc. Thesis
By
Iman Mohamed Attia Abd-Elkhalik
Under Supervision
Prof. Esaam Ali Amin
Prof. of mathematical statistics
Faculty of Graduate Studies for
Statistical Research
Cairo University
Dr. Mahmoud Aboagwa
Lecturer of mathematical statistics
Faculty of Graduate Studies for
Statistical Research
Cairo University
A Thesis Submitted to the Department of Mathematical
Statisitcs in Partial Fulfillment of the Requirement for the
Degree of Master of Science in Statisitcs
2022

3. Papers and Codes Published from the Thesis
Attia, I. M. (2022). Novel Approach of Multistate Markov Chains to Evaluate Progression in the
Expanded Model of Non-alcoholic Fatty Liver Disease. Frontiers in Applied Mathematics and
Statistics, 7. https://www.frontiersin.org/article/10.3389/fams.2021.766085
Attia, I.M.(2022). Log-Linear Model and Multistate Model to Assess the Rate of Fibrosis in
Patient with NAFLD. Frontiers in Applied Mathematics and Statistics, 8.
https://www.frontiersin.org/article/10.3389/fams.2022.899247
A MATLAB code illustrating the calculations, of chapter 4, is published in the code Ocean site
at the URL:
https://codeocean.com/capsule/8641183/tree/v1 with DOI = 10.24433/CO.6022979.v2
A MATLAB code illustrating the calculations, of chapter 5, is published in the code Ocean site
at the URL
https://codeocean.com/capsule/7628018/tree/v2 with DOI = 10.24433/CO.7719785.v3
The hypothetical study, in chapter 6, is coded by STATA-14 and is published in code ocean site
with the following URL:
https://codeocean.com/capsule/4752445/tree/v3 , DOI=10.24433/CO.8778229.v3
The MATLAB code to estimate the Q transition rate matrix, for the observed transition counts
used in the hypothetical example in chapter 6, using continuous time Markov chains is published
in the code Ocean site with following URL:
https://codeocean.com/capsule/6377472/tree/v2 , DOI=10.24433/CO.2144346.v2
The MATLAB code for solving the forward Kolmogorov differential equations using the
estimated Q rate matrix, obtained for the example in chapter 6, is published in the code Ocean
site with following URL:
https://codeocean.com/capsule/7258626/tree/v1 , DOI=10.24433/CO.2985270.v1
The dataset, in chapter 6, is present on IEEE Data Port site with the following URL:
https://ieee-dataport.org/documents/fibrosis-nfld#files , with the following DOI: 10.21227/dr5j-
gs46

4. i
Acknowledgment
I thank God Allah, The All-Mighty, The most Generous and The most Glorious, for help,
easiness, and facility in making this research accomplished.
I want to express my appreciation and thanks to Prof. Dr. Elsayed Ahmed Elsherpieny, Prof.
Dr. Esaam Ali Amin, Dr. Yassmen Yousef Abdelall, and Dr. Mahmoud AboAgwa for their
tremendous support, encouragement, and guidance throughout this work.
I want to thank all my professors in the Faculty of Graduate Studies for Statistical
Researches, Cairo University, for their effort in clarifying lots of facts and ideas since I have
been in this faculty in 2015. I want to thank all who had helped me by all different means and
who had clarified ideas and knowledge that aided me in better understanding some of the
statistical methodologies.
I owe great gratitude to my family, especially my mother, father, and brother, for their
patience, support, trustworthiness, fidelity, faithfulness in my doing, and incredible graciousness
that triggered me to this achievement. I cannot thank them enough for their continuous
cooperation, delicacy, kindness, and reinforcement. I cannot offer sufficient thanks to them.
I express tremendous and profound grace to my mother. The adversity she had experienced a
few months before delivering my proposal for this thesis was the inspiring and motivating driver
for me to choose and be engaged in this research. She most probably had non-alcoholic
steatohepatitis (NASH), a complication of fatal cachexia developed due to her multiple sclerosis.
The disease had a rapidly progressive course, and she did not have a chance to be thoroughly
diagnosed before her death. NASH is a subtype of non-alcoholic fatty liver disease (NAFLD), a
prevalent disease with a rapidly increasing epidemic worldwide. Many questions have not yet
been answered, and they need more exploration and research. I hope this work will help throw
light on this disease process from the statistical point of view to help physicians and health
policymakers have better and more efficient management plans for the challenges they face
while assessing this worldwide rapidly rising epidemic.

5. ii
Abstract
Epidemiology is the science that studies the occurrence of the disease. Numerous
mathematical methods can analyze such diseases. Multistate models are one of these
mathematical methods that rely on solving differential equations to get some of the statistical
indices that describe the process of the disease. Thus multistate Markov model is a valuable tool
to model event data obtained from longitudinal studies. In medical research, this technique can
model disease evolution in which each patient starts in one initial state and eventually ends in an
absorbing or final one. Continuous-Time Markov Chain (CTMC) is one of these multistate
models. CTMCs can estimate transition intensities and probabilities between states, state
probability distribution at a specific time point, mean sojourn time in each state, life expectancy
for the patient, and the expected number of patients in each state. As the prevalence of obesity
and type 2 diabetes has reached epidemic levels that parallel the rates of the widely distributed
non-alcoholic fatty liver disease (NAFLD), CTMC can model NAFLD to get better insight into
the behavior of such a worldwide prevalent disease. CTMC helps improve the detection and
treatment of NAFLD stages to avoid morbid complications.
This work provides a new approach using maximum likelihood estimation (MLE) to predict
the transition rates among states. Once the rates are estimated, the transition probability matrix
can be estimated. This approach compensates for the missing values when patients do not
commit to the follow-up schedule by predicting the rate in each interval and taking weight from
each rate corresponding to the proportion of transition counts in each interval in relation to the
total transition counts.
The "health, disease, and death" model is the simplest form of the CTMCs to study disease
evolution. CTMC can model the expanded form of the disease constituting the nine states. Each
disease process has its unique stages and specific transitions among the states. Also, a subset of
the "nine states model" that defines the early reversible stages of the disease, pointing to how the
fibrosis evolves, was utilized to understand the factors that determine its existence, as fibrosis is
the ominous predictor of bad outcomes and death. The results have yielded that the observed
rates approximately equal the estimated rates obtained by MLE, as was the case when analyzing
the simplest and the expanded models. Exponentiation of the estimated rate matrix yielded The
transition probability matrix. The researcher used Poisson regression to relate these rates with the
covariate risk factors of the disease like age, body mass index (BMI), homeostasis measurement
assessment-insulin resistance (HOMA2-IR) reflecting insulin resistance, low-density lipoprotein
cholesterol (LDL-Chol), systolic, and diastolic blood pressure. The study results were that
insulin resistance was the most detrimental risk factor for disease progression. The more resistant
to insulin the cells were, the higher the transition rate to advanced liver fibrosis was. The study
contains hypothetical data for each model to highlight the statistical concepts used to analyze
such a widely spread disease.
Keywords: Continuous-time Markov chains, Life expectancy, Maximum Likelihood estimation,
Mean Sojourn Time, Non-Alcoholic Fatty Liver Disease, Panel Data, Poisson regression.

6. iii
List of Figures
Figure (2. 1): he alth,disease and death model ............................................................................................4
Figure (2. 2): general model for disease progression( reversible progression)............................................4
Figure (2. 3): general model for disease progression (irreversible progression)……………………………………… 5
Figure (3. 1): worldwide estimated prevalence of NAFLD and the distribution of PNPLA3 genotype.......17
Figure (3. 2): dynamic model of NAFLD ......................................................................................................21
Figure (3. 3): risk stratification and management of NAFLD patients ........................................................25
Figure (3. 4): EASL-EASD-EASO clinical practice guidelines for management of NAFLD in type 2 diabetes
....................................................................................................................................................................26
Figure (4. 1): general model structure........................................................................................................30
Figure (5. 1): expanded form of the disease model structure....................................................................63
Figure (6. 1):NAFLD with fibrosis stages…………………………..……………………………………………………… 106
Figure (6. 2.a):categorical group of patients…………………………..……………………………………………………… 112
Figure (6. 2.b): categorical group of patients…………………………..……………………………………………………… 113
Figure (6. 3.a): percentage of transition counts……………………..……………………………………………………… 114
Figure (6. 3.b): percentage of transition counts……………………..……………………………………………………… 115
Figure (6. 3.c): percentage of transition counts……………………..……………………………………………………… 116
Figure (6. 4): distribution of transition counts from 0 to 1......................................................................117
Figure (6. 5): distribution of transition counts from 1 to 2.......................................................................117
Figure (6. 6): distribution of transition counts from 2 to 3.......................................................................118
Figure (6. 7): distribution of transition counts from 3 to 4.......................................................................118
Figure (6. 8): distribution of transition counts from 1 to 0.......................................................................118
Figure (6. 9): distribution of transition counts from 2 to 1 ......................................................................119
Figure (6. 10): distribution of transition counts from 3 to 2………………………………………………………………… 119
Figure (6. 11): distribution of transition counts from 2 to 0………………………………………………………………… 119
Figure (6. 12): distribution of transition counts from 3 to 1………………………………………………………………… 120
Figure (6. 13): lowess smoother for transition counts from 0 to 1...........................................................121
Figure (6. 14): lowess smoother for transition counts from 1 to 2...........................................................122
Figure (6. 15): lowess smoother for transition counts from 2 to 3...........................................................123
Figure (6. 16): lowess smoother for transition counts from 3 to 4...........................................................123
Figure (6. 17): lowess smoother for transition counts from 1 to 0...........................................................124
Figure (6. 18): lowess smoother for transition counts from 2 to 1...........................................................125
Figure (6. 19): lowess smoother for transition counts from 3 to 2...........................................................125
Figure (6. 20): lowess smoother for transition counts from 2 to 0...........................................................126
Figure (6. 21): lowess smoother for transition counts from 3 to 1………………………………………………………126

7. iv
List of Tables
Table (4. 1): number of observed transitions in different time intervals ...................................................46
Table (4. 2): total counts in the oeriod of study( 8 years) .........................................................................46
Table (4. 3): observed counts in the time interval t=1................................................................................46
Table (4. 4): observed counts in time interval t=2 .....................................................................................46
Table (4. 5): obsered counts in time interval t=3........................................................................................46
Table (4. 6 ) :layout of data for persons studied over 8 years....................................................................47
Table (6. 1): descriptive statistical summary of patients’ characteristics ................................................109
Table (6. 2): definition of the categorical groups of the patients.............................................................109
Table (6. 3): descriptive summary of categorical groups of patients ......................................................110
Table (6. 4): correlation between continuous predictor variables...........................................................110
Table (6. 5.a): summary of transition counts among states .....................................................................110
Table (6. 5.b): correlation between the different response variables......................................................110
Table (6. 6): observed transition counts of patients over 28 years .........................................................111
Table (6. 7): location of knots for specified variables using Harrell approach .........................................121
Table (6. 8): correlation between the transformed variables ..................................................................121
Table (6. 9): estimated counts for each transition....................................................................................127
Table (6. 10): comparison between distribution of observable response rate and estimated rate ........127
Table (6. 11): comparison between null and full model (progressive transitions)...................................128
Table (6. 12): comparison between null and full model (regressive transitions).....................................128
Table (6. 13 ): results for transition from 0 to 1 .......................................................................................129
Table (6. 14): results for transition from 1 to 2 ........................................................................................130
Table (6. 15): results for transition from 2 to 3 ........................................................................................130
Table (6. 16): results for transition from 3 to 4 .......................................................................................131
Table (6. 17): results for transition from 1 to 0 ........................................................................................132
Table (6. 18): results for transition from 2 to 1 ........................................................................................132
Table (6. 19): results for transition from 3 to 2 ........................................................................................133
Table (6. 20): results for transition from 2 to 0 ........................................................................................133
Table (6. 21): results for transition from 3 to 1 ........................................................................................134
Table (6. 22): pearson dispersion statistics for different transitions .......................................................134
Table (Appendix-D. 1 ): patients’ characteristics......................................................................................172
Table (Appendix-D. 2): transition counts for each patient .......................................................................178
Table (Appendix-D. 3): time line for each patient………………………………………………………………………………..181

8. v
List of Abbreviations
Abbreviation Meaning
AAR AST to ALT ratio
ALT Alanine Transaminase
APRI AST to Platelet Ratio Index
AST Aspartate Transaminase
ATP Adenosine Triphosphate
BMI Body Mass Index= weight(kg) /height (m2
)
CAP Controlled Attenuation Parameter
CK-18 Cytokeratin-18
CT Computed Tomography
CTMC Continuous-time Markov chains
CVD Cardiovascular Disease
EASL-SASD-
EASO
European Association of Study of Liver Disease-European Association of Study of
Diabetes-European Association of Study of Obesity
ELF Enhanced Liver Fibrosis
FIB-4 Fibrosis Index Score
GGT Gamma Glutamyl Transferase
HCC Hepato-Cellular Carcinoma
HCV Hepatitis C Virus
HOMA-IR Homeostatic model Assessment of Insulin Resistance
LDL Low Density Lipoprotein-Cholesterol
LSM Liver Stiffness Measurement
MetS Metabolic Syndrome
MRE Magnetic Resonance Elastrography
MSIR Maternally Derived Immunity, Susceptible, Infectious, Recovery
NAFL Non-Alcoholic Fatty Liver
NAFLD Non-Alcoholic Fatty Liver Disease
NASH Non-Alcoholic steatohepatitis
NFS NAFLD Fibrosis Score
NIVs Non-Invasive Tests
PNPL-3 Patatin Like Phospholipase Domain Containing 3 Gene
SEIR Susceptible, Exposed, Infectious, Recovery
SEIS Susceptible, Exposed, Infectious, Susceptible
SIR Susceptible, Infectious, Recovery
SIRD Susceptible, Infectious, Recovery,Deceased
SIRV Susceptible, Infectious, Recovery,Vaccinated
SIS Susceptible, Infectious, Susceptible
U/S ultrasonography
VCTE Vibration Controlled Transient Elastography

9. vi
Table of Contents
Acknowledgment...........................................................................................................................................i
Abstract......................................................................................................................................................... ii
List of Figures ............................................................................................................................................... iii
List of Tables ................................................................................................................................................ iv
List of Abbreviations ..................................................................................................................................... v
Table of Contents......................................................................................................................................... vi
Chapter One: Introduction............................................................................................................................1
Chapter Two: Basic Definitions and Notation...............................................................................................4
2.1. Some Basic Definitions.......................................................................................................................6
2.2. Model Specification .........................................................................................................................10
2.3. Chapman-Kolmogorov Equation......................................................................................................12
2.4. Chapman-Kolmogorov Differential Equations.................................................................................12
2.5. The Mean Sojourn Time...................................................................................................................15
2.6. State Probability Distribution...........................................................................................................15
Chapter Three: Non-Alcoholic Fatty Liver Disease .....................................................................................17
3.1. Prevalence........................................................................................................................................17
3.2. Definition and Terminology .............................................................................................................19
3.3. Dynamic Model of NAFLD................................................................................................................20
3.4. Diagnosis of NAFLD..........................................................................................................................21
3.5. Treatment. .......................................................................................................................................23
Chapter Four: CTMC Analyzing NAFLD Progression (Small Model)............................................................30
4.1. Transition Probability Matrix ...........................................................................................................31
4.2. Maximum Likelihood Estimation of the Q Matrix............................................................................35
4.3. Mean Sojourn Time..........................................................................................................................40
4.4. State Probability Distribution: .........................................................................................................41
4.5. Life Expectancy of Patient in NAFLD Disease Process......................................................................43
4.6. Expected Number of Patients in Each State ....................................................................................45
4.7. Hypothetical Numerical Example.....................................................................................................45
Chapter Five: CTMC Analyzing NAFLD Progression (Big Model).................................................................63
5.1.Transition Probability matrix ............................................................................................................63

10. vii
5.2. Estimation Of The Q transition Rate Matrix.....................................................................................79
5.3. Mean Sojourn Time..........................................................................................................................88
5.4. State Probability Distribution...........................................................................................................88
5.5. Expected Number of Patients in Each State: ...................................................................................91
5.6. Life Expectancy of a Patient Suffering from NAFLD in Various Stages ............................................91
5.7. Hypothetical Model .........................................................................................................................92
Chapter six: Incorporation of Covariates in the CTMC .............................................................................106
6.1.Study Design ...................................................................................................................................108
6.2.Results .............................................................................................................................................109
6.3. Discussion.......................................................................................................................................135
6.4.Conclusions......................................................................................................................................139
Chapter Seven: Conclusions and Recommendations ...............................................................................140
7.1. Conclusions ....................................................................................................................................140
7.2. Recommendations.........................................................................................................................141
Appendix A: MATLAB Code for PDFs of the Small Model.........................................................................142
Appendix B: MATLAB Code for Testing Markovian Assumption of Small Model .....................................143
Appendix C: Differentiation of Eigenvalue Functions with Respect to Theta for Big Model....................143
Appendix D: MATLAB Code for Estimation of Rate Matrix of the Big Model ..........................................145
Appendix E: MATLAB Code for Estimation of Probability Matrix of the Big Model .................................158
Appendix F: Goodness of Fit .....................................................................................................................172
Appendix G: Selected Tables as Referred in Chapter 6 ............................................................................176
References ................................................................................................................................................188
Arabic Summary........................................................................................................................................194

11. 1
Chapter One: Introduction
The study of a disease occurrence is the scope of epidemiology. It is a science that studies the
distributions, forms, and factors of a disease in a specific population. Many mathematical models
can describe the disease process according to the stages of the disease. Ordinary differential
equations can describe the disease process. They are deterministic, but a stochastic frame close to
reality can formulate the disease process but with much more complexity to analyze.
Some of these models used in infectious diseases are the basic model, described with three
states susceptible, infectious, and recovered, thus named (SIR). SIR reflects the stages that the
patient can pass through when acquiring an infection. There are some variations of this model.
A model formulated with two states, susceptible and infectious. There is no recovery state as
in the common cold because the patient does not develop immunity. Thus this model describes
three states susceptible, infectious, and susceptible, and thus it is called (SIS) model.
Another variant describes four stages susceptible, infectious, recovered, and deceased. The
recovery stage means immunity, while the deceased stage means the chronic complications and
sequels of the disease. This variant is called the susceptible, infectious, recovered, and deceased
(SIRD) model.
The extension of the SIR model, taking into account the vaccinated stage, describes four
stages. These stages are susceptible, infectious, recovered, and vaccinated. Thus this model is
called (SIRV) model.
Another extension of the SIR model describes four stages. The extension has a first stage
reflecting the "maternally derived immunity" that the newborns have from their mothers in the
first few months after birth. Thus the model describes the following stages "maternally derived
immunity," susceptible, infectious, and recovered; thus, it is called (MSIR) model.
The model describing the following states susceptible, exposed, infectious, and recovered,
has the so-called exposed state. This state reflects the latency or incubation period during which
the patient has been infected, but the manifestations of the disease are still not apparent, and the
ability to infect others is still not acquired. Thus this model is called the susceptible, exposed,
infectious, and recovered (SEIR) model.
The model described with the states susceptible, exposed, infectious, and susceptible is the
same as the directly preceded model but with no recovery stage, and so the model name is
susceptible, exposed, infectious, and susceptible (SEIS) model. (Bailey, 1975).
Other models can describe chronic diseases other than infectious diseases like the diseases
caused by disturbed immunity, cancer, and genetic and metabolic diseases. One of these models

12. 2
is the DisMod II model with four states (susceptible, diseased, death due to illness, and death
from any other cause) (Barendregt et al., 2003). Multistate models like the continuous-time
Markov chains (CTMCs) are elaborative in analyzing chronic diseases. CTMC is frequently used
to model panel data in various fields of science, including medicine, sociology, biology, physics,
and finance. In medical studies, CTMCs model the illness-death process in which each patient
starts in one initial state and eventually ends in an absorbing or final one. The patient may
experience several events that are related to their original disease. There may be several
intermediate states in-between the initial and the final state. The patient can or cannot visit these
intermediate states. (Kruijshaar et al., 2002).
Continuous-Time Markov Chain (CTMC) has the main objective to identify all the possible
movements among the states to estimate the following:
 Transition intensities and probabilities between the states.
 State transition probabilities at a particular time point.
 Mean sojourn time in each state.
 Life expectancy for the patient.
 Expected number of the patients in each state at a specific time point.
Also, CTMC is used to incorporate and thus identify covariates that affect the transition
intensities aiming to evaluate the factors that influence various movements among these states.
In this work, the researcher throws light on some of the statistical concepts and indices
derived using CTMC to analyze the non–alcoholic fatty liver disease (NAFLD). This rapidly
increasing metabolic derangement of the liver and its associated complications have an economic
burden on society. The information obtained by this analysis helps health policymakers to
allocate human and financial resources for investigating this disease process, preventing, and
treating it. The study contains the following chapters:
 In chapter two, the researcher highlights the basic statistical and mathematical concepts.
 In chapter three, the researcher explains the medical definition of the NAFLD disease and
how to investigate it. These concepts help understand the states of the NAFLD disease. The
researcher also mentions some of the ongoing studies concerning the treatments.
 In chapter four, the researcher demonstrates the simple model of the "health, disease, and
death" process. The contained applied hypothetical numerical example illustrates how the
statistical indices can be mathematically derived.
 In chapter five, the researcher expands the disease process into nine states to describe the
states for NAFLD, the transitions among the states, and the ultimate fate of the process.
Finally, a supplemented applied hypothetical example explains these statistical indices.
 In chapter six, the researcher uses covariates in a Poisson regression model to relate the
transition rates among the states to these covariates. The researcher uses a subset of the

13. 3
disease states, taken from the extended model describing the early reversible stages of the
disease, in an applied hypothetical example.
The researcher summarizes the mathematical approach used in this work in the following items:
 Maximum-likelihood function ( MLE) followed by applying the quasi-newton method to
estimate the transition rate matrix (Q matrix).
 Exponentiation of this Q matrix to obtain the transition probability matrix. Solve the
Kolmogorov differential equations to get empirical PDFs and then substitute the estimated Q
matrix in these functions to get the final PDFs. Comparing the results of both methods,
especially if the results are equal, can support the time homogeneity of the process.
 The Poisson regression model incorporates the covariates, thus relating these covariates with
the transition rates.

14. 4
Chapter Two: Definitions and Notation
Review of literature
Many chronic diseases have a progressive course over time. The disease process passes
through successive stages comprising the disease process. CTMC can model this course of the
disease. The “health, disease, and death” model is the simplest disease model. It has three states
representing health, disease and death. Transitions are allowed to occur from health to disease
from disease to death, or from health to death. Recovery from disease to health is also permitted,
as shown in Figure (2.1):
FIGURE (2. 1): health disease death model
The model is expressed by a series of successive more sever disease stages and an absorbing
state, often a death state. The patient may advance into or recover from adjacent disease stages or
die at any disease stage. Observations of the state 𝑆𝑖(𝑡) are made on many individuals 𝑖 at
arbitrary time points 𝑡 , which may vary between individuals, as shown in the Figure (2.2) and
Figure (2.3):
FIGURE (2. 2): A general model for disease progression (reversible progression)

15. 5
FIGURE (2. 3): A general model for disease progression (irreversible progression)
Longini Jr et al. (1989) used a staged Markov model to estimate the distribution and mean
length of the incubation period of AIDS from a cohort of 603 HIV infected persons. The persons
have been followed through various stages of infection. The infection was modeled into 4 illness
stages and one final absorbing death stage.
Sharples (1993) modeled the transition rates between grades of coronary occlusive disease,
following cardiac transplantation, from each grade to death. The disease process was graded on a
three points scale according to the amount of narrowing observed in major vessels using serial
angiography.
Marshall and Jones (1995) discussed a multistate model of three transient states representing
the early stage of diabetic retinopathy. The model had one final absorbing state representing the
irreversible stage of retinopathy. They explored the effects of factors influencing the onset,
progression, and regression of diabetic retinopathy among subjects with insulin-dependent
diabetes mellitus under the assumption that CTMC determines the transition times between
disease stages.
Pérez‐Ocón et al. (2001) had applied the CTMC technique to examine the influence of three
post-surgical treatments (chemotherapy, radiotherapy, hormonal therapy) on 300 breast cancer
patients on their lifetimes and relapse times. The survival time of the patients in the group where
all the three treatment combinations were given was much more compared to either the
radiotherapy and chemotherapy group or the radiotherapy group alone.
Jackson and Sharples (2002) measured forced expiratory volume in one second (FEV1) at the
irregular intervals and used hidden Markov models for studying the staged decline in respiratory
functions as well as the influencing covariates after developing Bronchiolitis Obliterans
Syndrome (BOS) following lung transplantation.
Saint‐Pierre et al. (2003) used CTMC with time-dependent covariates and Markov model
with piece-wise constant intensities to model asthma control evolution.
Foucher et al. (2005) used the semi-Markov model to define the waiting time distribution
based on the generalized Weibull distribution. They gave an extension of the homogenous

16. 6
CTMC based implicitly on exponential waiting time distribution. They applied this concept with
an example of the evolution of HIV infected patients.
Fackrell (2009) demonstrated the usage of the CTMC structure with phase-type distribution
in modeling the health care system.
Bartolomeo, Trerotoli, and Serio (2011) employed a hidden Markov model to determine the
transition probabilities incorporating various covariates for progression of liver cirrhosis to HCC
and death.The model involved two illness states and one death state.They found that the presence
of concomitant diseases increases the risk of death in patients with HCC.
Grover et al. (2014) used a time-dependent multistate Markov chain to assess the progression
of liver cirrhosis in patients having hepatitis C virus with various prognostic factors.
Anwar and Mahmoud (2014) used CTMC to study the progression of the chronic kidney
disease, estimate the mean time spent in each stage of disease process, and estimate the life
expectancy of a chronic kidney disease patient.
Estes et al. (2018) used multistate Markov chains to model the epidemic of nonalcoholic fatty
liver disease. They forecasted the non-alcoholic fatty liver disease progression to rise by 21%,
from 83.1 million (2015) to 100.9 million (2030), and the non-alcoholic steato-hepatitis to be
elevated by 63% from 16.52 million to 27 million cases. The prevalence in the adult population
age more than 15 years will be 33.5% in 2030. The incidence of “decompensated liver cirrhosis”
will elevate 168% to 105430 cases by 2030, the incidence of liver cancer will rise by 137% to
12240 cases and deaths will increase by 178% to 78300 deaths in 2030. Also, Younossi et al.
(2016) used the multistate Markov chains to construct 5 models in the United States and 4
European countries ( Germany, France, Italy, and United Kingdom) for estimating the burden of
NAFLD in these countries. The patients move among 9 states with the following results: in the
United States, more than 64 million were estimated to have NAFLD with an annual cost of
nearly 103 billion dollars (1613 dollars per patient), while in the 4 European countries there were
approximately 52 million persons with NAFLD with an annual cost of about 35 billion.
There are numerous studies over the several past decades addressing these statistical methods
for the analysis of disease evolution and progression through time.
2.1. Some Basic Definitions:
2.1.1. Some Basic Statistical and Mathematical Definitions:
According to (Castaneda et al. 2012), these are some of basic statistical definitions.
Definition 1: Stochastic Process:
A real stochastic process is a collection of random variables {𝑋𝑡; 𝑡 ≥ 0} defined on a common
probability space (Ω, ℑ, 𝑃) with values in ℝ. T is called the index set of the process or parametric

17. 7
space, which is usually a subset of ℝ. The set of the values that the random variable 𝑋𝑡 can take
is called the state space and is denoted by S . The mapping defined for each fixed 𝜔 ∈ Ω
𝑋(𝜔): 𝑇 → S
𝑡 ↦ 𝑋𝑡(𝜔)
Stochastic processes enlarge the concept of the random variable to include time, that is to
mean, the random variable does not only map an event 𝑠 ∈ Ω, where Ω is the sample space, to
some number 𝑋(𝑠) but it maps it to different numbers at different times. So, this is not only a
number 𝑋(𝑠) but it is 𝑋(𝑠, 𝑡) where 𝑡 ∈ 𝑇 and T is called the parameter set of the process and it
is usually a set of times.
Therefore, a stochastic process is defined as a family of random variables {𝑋(𝑡, 𝑠)|𝑡 ∈ 𝑇, 𝑠 ∈
Ω } over a given probability space and is indexed by the parameter 𝑡. Such a random process is
used interchangeably with the stochastic process (Ibe 2013).
Stochastic processes are classified into four types according to the nature of the state space
and time parameter: If t is an interval of real numbers hence the process is called continuous-time
stochastic process but if the t is a countable set of positive numbers the process is called discrete-
time stochastic process. The state space is either continuous or discrete.
Examples for such classification:
 Discrete state, discrete-time stochastic process:
The number of individuals in a population at the end of the year 𝑡 is modeled as a stochastic
process {𝑋(𝑡, 𝑠)|𝑡 ∈ 𝑇, 𝑠 ∈ Ω } having 𝑇 = {0,1,2, … } and state space Ω = {0,1,2, … }
 Discrete state, continuous-time stochastic process:
The number of incoming calls in an interval [0, 𝑡]. Then the stochastic process {𝑋(𝑡, 𝑠)|𝑡 ∈
𝑇, 𝑠 ∈ Ω } has 𝑇 = {𝑡: 0 ≤ 𝑡 ≤ ∞} and Ω = {0,1,2, … }
 Continuous state, discrete-time stochastic process:
The share price for an asset at the close of trading on day 𝑡 with 𝑇 = {0,1,2, … } & Ω =
{𝑠: 0 ≤ 𝑠 ≤ ∞}
 Continuous state, continuous-time stochastic process:
The value of the Dow-Jones index at time 𝑡 such that 𝑇 = {𝑡: 0 ≤ 𝑡 ≤ ∞} and Ω = {𝑠: 0 ≤ 𝑠 ≤
∞}
Definition 2: Markov Process:
Let {𝑋𝑡; 𝑡 ≥ 0} be a stochastic process defined over a probability space (Ω, ℑ, 𝑃) and with a state
space (ℬ). {𝑋𝑡; 𝑡 ≥ 0} is a Markov process if for any 0 ≤ 𝑡1 ≤ 𝑡2 ≤ ⋯ ≤ 𝑡𝑛 and for any 𝐵 ∈ ℬ ,
𝑃(𝑋𝑡𝑛
∈ 𝐵|𝑋𝑡1
, … , 𝑋𝑡𝑛−1
) = 𝑃(𝑋𝑡𝑛
∈ 𝐵|𝑋𝑡𝑛−1
)

18. 8
Definition 3: Continuous Time Markov Process (CTMC):
Let {𝑋𝑡; 𝑡 ≥ 0} be a stochastic process with countable state space S. A process is a continuous
time Markov chain if: 𝑃(𝑋𝑡𝑛
= 𝑗|𝑋𝑡1
= 𝑖1, … , 𝑋𝑡𝑛−1
= 𝑖𝑛−1) = 𝑃(𝑋𝑡𝑛
= 𝑗|𝑋𝑡𝑛−1
= 𝑖𝑛−1)
For all 𝑗, 𝑖1, … , 𝑖𝑛−1 ∈ 𝑆 and for all 0 ≤ 𝑡1 ≤ 𝑡2 ≤ ⋯ ≤ 𝑡𝑛
Definition 4: Homogenous Continuous Time Markov Chain:
CTMC is homogenous if and only if 𝑃(𝑋𝑡+𝑠 = 𝑗|𝑋𝑠 = 𝑖) is independent of 𝑠 for all t .
Definition 5: Transition Probability:
Let 𝑝𝑖𝑗(𝑡) be the probability of the transition from state 𝑖 to state 𝑗 in an interval of length 𝑡.
𝑝𝑖𝑗(𝑡) = (𝑋𝑡+𝑠 = 𝑗|𝑋𝑠 = 𝑖) where 𝑠, 𝑡 ≥ 0. In matrix notation : 𝑃(𝑡) = [𝑝𝑖𝑗(𝑡)] 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖, 𝑗 ∈ 𝑠
It satisfies the following conditions:
1. 𝑃𝑖𝑗(𝑡 + 𝑠) = ∑ 𝑃𝑖𝑙(𝑡)𝑃𝑙𝑗(𝑠)
𝑖.𝑗.𝑙∈𝑆 , ∀ 𝑡 ≥ 0, 𝑠 ≥ 0, 𝑎𝑛𝑑 𝑖, 𝑗, 𝑙 ∈ 𝑆
𝑜𝑏𝑒𝑦𝑖𝑛𝑔 𝑘𝑜𝑙𝑚𝑜𝑔𝑟𝑜𝑣 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛𝑠
2. ∑ 𝑃𝑖𝑗(𝑡) = 1
𝑆
3. 𝑃𝑖𝑗(𝑡) ≥ 0 , ∀ 𝑡 ≥ 0 𝑎𝑛𝑑 𝑖, 𝑗 ∈ 𝑆
Definition 6: Chapman-Kolmogorov equation for CTMC
𝑝𝑖𝑗(𝑡 + 𝑠) = ∑ 𝑝𝑖𝑘(𝑡)𝑝𝑘𝑗(𝑠) 𝑠𝑜 𝑃(𝑡 + 𝑠) = 𝑃(𝑡)𝑃(𝑠)
𝑘∈𝑆
𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖, 𝑗 , 𝑘 ∈ 𝑆, & 𝑓𝑜𝑟 𝑎𝑛𝑦 𝑠, 𝑡 ≥ 0
Definition 7: Transition Rate Matrix or Intensity Matrix or Infinitesimal Generator
Let {𝑋𝑡; 𝑡 ≥ 0} be a CTMC , 𝑞𝑖𝑗(𝑡) is the rate at which transition occur from state 𝑖 to state 𝑗 at
time 𝑡 or 𝑄(𝑡) = lim
∆𝑡→0
{
𝑃(𝑡,𝑡+∆𝑡)−𝐼
∆𝑡
} = 𝑃′(𝑡 = 0) 𝑆𝑖𝑛𝑐𝑒 𝑃(0) = 𝐼 .
The Q matrix fulfills the criteria for the Q matrix which are:
1. ∑ 𝑞𝑖𝑗(𝑡) = 0
𝑆
2. 𝑞𝑖𝑗(𝑡) ≥ 0 , 𝑖 ≠ 𝑗
3. − ∑ 𝑞𝑖𝑗(𝑡)
𝑆 = 𝑞𝑖𝑖 , 𝑖 = 𝑗
Definition 8: Stationary Probability Distribution
Let {𝑋𝑡; 𝑡 ≥ 0} be a CTMC with generator matrix 𝑄 and transition probability matrix 𝑃(𝑡) .
Suppose

19. 9
𝜋 = (𝜋0, 𝜋1, … , 𝜋𝑚)𝑡
is nonnegative i.e. 𝜋𝑖 ≥ 0 for 𝑖 = 0,1,2, … , 𝑚 , where m is the number of
states. So 𝑄𝜋 = 0 𝑎𝑛𝑑 ∑ 𝜋𝑖 = 1
𝑚
𝑖=0 . This 𝜋𝑖 is called stationary probability distribution. It can
also be defined in terms of 𝑃(𝑡) such that
𝜋𝑃(𝑡) = 𝜋 , 𝑓𝑜𝑟 𝑡 ≥ 0 , ∑ 𝜋𝑖 = 1
𝑚
𝑖=0 , 𝜋𝑖 ≥ 0 𝑓𝑜𝑟 𝑖 = 0,1,2, … , 𝑚.
Definition 9: Embedded Markov Chain (EMC)
Let CTMC {𝑋𝑡; 𝑡 ≥ 0} be a CTMC. The set of the random variables {𝑌𝑛}𝑛=0
∞
is known as the
embedded Markov chain or the jump chain at the 𝑛𝑡ℎ
jump associated with the CTMC {𝑋𝑡; 𝑡 ≥
0} with a transition matrix 𝑇 = 𝑡𝑖𝑗 , where 𝑌𝑛 = 𝑋(𝑊
𝑛), 𝑛 = 0,1,2, .. , such that 𝑊𝑖 is the time at
which the 𝑛𝑡ℎ
jump occurs and the 𝑇𝑖 = 𝑊𝑖+1 − 𝑊𝑖 is the holding time or the time spent in the
state until the next jump occurs at 𝑊𝑖.
Definition 10: Accessibility
State 𝑗 can be reached from state, 𝑖 → 𝑗 , if 𝑝𝑖𝑗(𝑡) > 0 for some 𝑡 ≥ 0.
Definition 11: Communicating State
State 𝑖 communicate with state 𝑗 , (𝑖 ↔ 𝑗), if 𝑖 → 𝑗 and 𝑗 → 𝑖
The set of states that communicate is called a communication class.
Definition 12: Irreducibility.
If every state can be reached from every other state, the Markov chain is irreducible; otherwise, it
is said to be reducible.
Definition 13: Closed Class
A set of states C is closed if it is impossible to reach any state outside of C from a state inside C,
𝑝𝑖𝑗(𝑡) = 0 ; 𝑓𝑜𝑟 𝑡 ≥ 0 𝑎𝑛𝑑 𝑖𝑓 ; 𝑖 ∈ 𝐶 𝑎𝑛𝑑 𝑗 ∉ 𝐶
Definition 14: First Return Time
𝑇𝑖𝑖 is the first time the chain is in state 𝑖 after leaving state 𝑖 , it can occur for 𝑡 > 0
Definition 15: Recurrent State
State 𝑖 is recurrent in a CTMC {𝑋𝑡; 𝑡 ≥ 0} , if the first return time is finite 𝑃{𝑇𝑖𝑖 < ∞|𝑋(0) =
𝑖} = 1
State 𝑖 is recurrent in a CTMC {𝑋𝑡; 𝑡 ≥ 0} if and only if state 𝑖 in the corresponding embedded
Markov chain {𝑌𝑛}𝑛=0
∞
is recurrent.

20. 10
Definition 16: Transient State
State 𝑖 is transient in a CTMC {𝑋𝑡; 𝑡 ≥ 0} , if the first return time is finite 𝑃{𝑇𝑖𝑖 < ∞|𝑋(0) =
𝑖} < 1
State 𝑖 is transient in a CTMC {𝑋𝑡; 𝑡 ≥ 0} if and only if state 𝑖 in the corresponding embedded
Markov chain {𝑌𝑛}𝑛=0
∞
is transient.
2.1.2. Some basic medical definitions:
Definition 17 : Fibro-genesis is the mechanism through which fibrous tissue is formed i.e the
process of fibrous tissue formation.
2.2. Model Specification:
2.2.1. Transition Probability Matrix:
According to (Allen 2010), the model is specified by a transition probability matrix 𝑃(𝑡)
whose (𝑖, 𝑗)𝑡ℎ entry ,𝑝𝑖𝑗(𝑡), is the probability of a transition from state 𝑖 at time 𝑡 to some other
state 𝑗 at rate 𝑞𝑖𝑗(𝑡) per specified unit of time according to the studied process or system. So
continuous time Markov chain is modeled by its matrix of transition rates 𝑄(𝑡) at time 𝑡 . The
probability that a transition occurs from a given source state to a specific destination state
depends on both the source and the length of the interval of observation. That is to say, if the
period of observation 𝜏 = Δ𝑡 has a minimal duration so the probability of observing a transition
from state 𝑖 at time 𝑡 to state 𝑗 at time 𝑡 + Δ𝑡 during this interval [𝑡, 𝑡 + Δ𝑡) i.e. 𝑝𝑖𝑗 (𝑡, 𝑡 + Δ t)
is minimal.
So as Δ𝑡 → 0, 𝑝𝑖𝑗 (𝑡, 𝑡 + Δ t) → 0 𝑓𝑜𝑟 𝑖 ≠ 𝑗 ,
and from the conservation of probability:
𝑝𝑖𝑖 (𝑡, 𝑡 + Δ t) → 1 𝑎𝑠 Δ𝑡 → 0 .
On the other hand, as Δ𝑡 enlarges this probability 𝑝𝑖𝑗(𝑡, 𝑡 + Δ𝑡) increases to the level that the
larger the period is, the more probable multiple events will be observed. Nevertheless, the
observation periods are adequately selected to be small enough so that the probability of
observing multiple events in such a small observation period is of order 𝜊(Δ𝑡) , a quantity for
which
lim
Δ→0
𝜊(Δ𝑡)
Δ𝑡
= 0
Continuous Markov chain exhibits Markov (memoryless) property:
𝑃[𝑋(𝑡𝑘+1) = 𝑥𝑘+1 | 𝑋(𝑡𝑘) = 𝑥𝑘, 𝑋(𝑡𝑘−1) = 𝑥𝑘−1, … , 𝑋(𝑡0) = 𝑥0 ]
= 𝑃[𝑋(𝑡𝑘+1) = 𝑥𝑘+1|𝑋(𝑡𝑘) = 𝑥𝑘]

21. 11
for any 𝑡0 ≤ 𝑡1 ≤ ⋯ ≤ 𝑡𝑘 ≤ 𝑡𝑘+1 . So if the current state 𝑥𝑘 is known, then the value taken by
𝑋(𝑡𝑘+1) depends only on 𝑥𝑘 and not on any past history of the state (no state memory). Also,
the amount spent in the current state does not determine the next state (no age
memory)(Cassandras and Lafortune 2009).
2.2.2. Generator Matrix or the Transition Rate Matrix:
According to (Allen 2010), the transition probabilities 𝑝𝑖𝑗(𝑡) are used to obtain transition
rates 𝑞𝑖𝑗(𝑡). A rate of transition does not depend on the length or duration of observation period,
it is an instantaneously defined quantity that indicates the number of transitions that occur per
unit of time. The 𝑞𝑖𝑗(𝑡) is the rate of transition from state 𝑖 to state 𝑗 at time t . In non-
homogenous Markov chain both, 𝑞𝑖𝑗(𝑡) and 𝑝𝑖𝑗(𝑡) may depend on the time 𝑡 not the interval Δ𝑡.
The transition probabilities 𝑝𝑖𝑗(𝑡) are continuous and differentiable for > 0 .
At 𝑡 = 0, the transition probabilities equal:
𝑝𝑖𝑗(0) = 0 𝑎𝑛𝑑 𝑝𝑖𝑖(0) = 1
While defining:
𝑞𝑖𝑗 = lim
Δ𝑡→0
{
𝑝𝑖𝑗(Δ𝑡) − 𝑝𝑖𝑗(0)
Δ𝑡
} = lim
Δ𝑡→0
{
𝑝𝑖𝑗(Δ𝑡)
Δ𝑡
} 𝑓𝑜𝑟 𝑖 ≠ 𝑗
And
𝑞𝑖𝑖 = lim
Δ𝑡→0
{
𝑝𝑖𝑖(Δ𝑡) − 𝑝𝑖𝑖(0)
Δ𝑡
} = lim
Δ𝑡→0
{
𝑝𝑖𝑖(Δ𝑡) − 1
Δ𝑡
} 𝑓𝑜𝑟 𝑖 = 𝑗
As well as
𝑝𝑖𝑗(Δ𝑡) = 𝑞𝑖𝑗(𝑡)Δ𝑡 𝑎𝑛𝑑 ∑ 𝑝𝑖𝑗(Δ𝑡)
∞
𝑗≠𝑖
= ∑ 𝑞𝑖𝑗(𝑡)Δ𝑡 + 𝜊(Δ𝑡)
∞
𝑗≠𝑖
And from conservation of probability:
1 − 𝑝𝑖𝑖(Δ𝑡) = ∑ 𝑝𝑖𝑗(Δ𝑡)
𝑗≠𝑖
= ∑ 𝑞𝑖𝑗(𝑡)Δ𝑡 + 𝜊(Δ𝑡)
𝑗≠𝑖
1 − 𝑝𝑖𝑖(Δ𝑡) = lim
Δ𝑡→0
∑ {
𝑞𝑖𝑗(𝑡)Δ𝑡 + 𝜊(Δ𝑡)
Δ𝑡
}
∞
𝑖≠𝑗
= − ∑
𝑞𝑖𝑗 Δ𝑡
Δ𝑡
∞
𝑖≠𝑗
𝑞𝑖𝑖 = − ∑ 𝑞𝑖𝑗(𝑡)
∞
𝑗=0,𝑗≠𝑖
In homogenous continuous time Markov chain:
𝑞𝑖𝑗 = lim
Δ𝑡→0
{
𝑝𝑖𝑗(Δ𝑡)
Δ𝑡
} 𝑓𝑜𝑟 𝑖 ≠ 𝑗 ; 𝑞𝑖𝑖 = lim
Δ𝑡→0
{
𝑝𝑖𝑖(Δ𝑡) − 1
Δ𝑡
} 𝑓𝑜𝑟 𝑖 = 𝑗

22. 12
Or in matrix notation:
𝑄 = lim
Δ𝑡→0
{
𝑃(Δ𝑡) − 𝐼
Δ𝑡
}
It is called infinitesimal generator matrix or rate transition matrix of Markov chain
𝑄 = 𝑃′(0) 𝑎𝑡 𝑡 = 0 , 𝑠𝑖𝑛𝑐𝑒 𝑃(0) = 𝐼
If 𝑆 is a finite or countable state space and 𝑄 = (𝑞𝑖𝑗)
𝑖,𝑗∈𝑆
then it satisfies the following
properties:
𝑞𝑖𝑗 ≥ 0 𝑓𝑜𝑟 𝑖 ≠ 𝑗 𝑎𝑛𝑑 𝑞𝑖𝑖 ≤ 0 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖 , ∑ 𝑞𝑖𝑗
𝑗≠𝑖
= −𝑞𝑖𝑖 𝑎𝑛𝑑 ∑ 𝑞𝑖𝑗
𝑗
= 0 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖
2.3. Chapman-Kolmogorov Equations:
According to (Cassandras and Lafortune 2009), the transition probability is called 𝑝𝑖𝑗(𝑠, 𝑡) and is
defined as 𝑝𝑖𝑗(𝑠, 𝑡) = 𝑃[𝑋(𝑡) = 𝑗|𝑋(𝑠) = 𝑖], 𝑠 ≤ 𝑡
To derive the equation; the events of transitions [𝑋(𝑡) = 𝑗|𝑋(𝑠) = 𝑖] are conditioned on
[𝑋𝑢 = 𝑟]
For some 𝑢 such that 𝑠 ≤ 𝑢 ≤ 𝑡 and that the rule of total probability implies
𝑝𝑖𝑗(𝑠, 𝑡) = ∑ 𝑃[𝑋(𝑡) = 𝑗|𝑋(𝑢) = 𝑟, 𝑋(𝑠) = 𝑖]
𝑎𝑙𝑙 𝑟
. 𝑃[𝑋(𝑢) = 𝑟| 𝑋(𝑠) = 𝑖],
in addition the memory-less property ensures the following:
𝑃[𝑋(𝑡) = 𝑗|𝑋(𝑢) = 𝑟, 𝑋(𝑠) = 𝑖] = 𝑃[𝑋(𝑡) = 𝑗|𝑋(𝑢) = 𝑟] = 𝑃𝑟𝑗(𝑢, 𝑡)
Moreover , 𝑃[𝑋(𝑢) = 𝑟| 𝑋(𝑠) = 𝑖] = 𝑃𝑖𝑟(𝑠, 𝑢). Therefore,
𝑝𝑖𝑗(𝑠, 𝑡) = ∑ 𝑃𝑖𝑟(𝑠, 𝑢).
𝑎𝑙𝑙 𝑟
𝑃𝑟𝑗(𝑢, 𝑡) , 𝑠 ≤ 𝑢 ≤ 𝑡
The above Chapman-Kolmogorov equation can be rewritten in matrix notation as:
𝐻(𝑠, 𝑡) = 𝐻(𝑠, 𝑢)𝐻(𝑢, 𝑡) , 𝑠 ≤ 𝑢 ≤ 𝑡 ∶ 𝑤ℎ𝑒𝑟𝑒
𝐻(𝑠, 𝑡) = [𝑝𝑖𝑗(𝑠, 𝑡)] 𝑎𝑛𝑑 𝐻(𝑠, 𝑠) = 𝐼 ( 𝑖𝑑𝑒𝑛𝑡𝑖𝑡𝑦 𝑚𝑎𝑡𝑟𝑖𝑥)
2.4. Chapman-Kolmogorov Differential Equations:
2.4.1. Forward Chapman-Kolmogorov differential equations:
The Chapman-Kolmogorov equations for time instants 𝑠 ≤ 𝑢 ≤ 𝑡 + Δ𝑡 𝑎𝑛𝑑 Δ𝑡 > 0 is
𝐻(𝑠, 𝑡 + Δ t) = 𝐻(𝑠, 𝑡)𝐻(𝑡, 𝑡 + Δ𝑡 ) subtracting 𝐻(𝑠, 𝑡) from both sides of this equation yields

23. 13
𝐻(𝑠, 𝑡 + Δ t) − 𝐻(𝑠, 𝑡) = 𝐻(𝑠, 𝑡)[𝐻(𝑡, 𝑡 + Δ𝑡 ) − 𝐼] , then dividing by Δ𝑡 and taking the limit as
Δ𝑡 → 0 gives
lim
∆𝑡→0
𝐻(𝑠, 𝑡 + Δ𝑡 ) − 𝐻(𝑠, 𝑡)
∆𝑡
= 𝐻(𝑠, 𝑡) lim
∆𝑡→0
𝐻(𝑡, 𝑡 + Δ𝑡 ) − 𝐼
∆𝑡
lim
∆𝑡→0
𝐻(𝑠, 𝑡 + Δ𝑡 ) − 𝐻(𝑠, 𝑡)
∆𝑡
=
𝜕𝐻(𝑠, 𝑡)
𝜕𝑡
This limit represents the partial derivative of Pij(s,t) with respect to (t) if it exists .
𝑄(𝑡) = lim
∆𝑡→0
𝐻(𝑡, 𝑡 + Δ𝑡 ) − 𝐼
∆𝑡
lim
∆𝑡→0
𝐻(𝑠, 𝑡 + Δ𝑡 ) − 𝐻(𝑠, 𝑡)
∆𝑡
= 𝐻(𝑠, 𝑡)𝑄(𝑡)
lim
∆𝑡→0
𝐻(𝑠, 𝑡 + Δ𝑡 ) − 𝐻(𝑠, 𝑡)
∆𝑡
=
𝜕𝐻(𝑠, 𝑡)
𝜕𝑡
= 𝐻(𝑠, 𝑡)𝑄(𝑡) , 𝑠 ≤ 𝑡
This is called forward differential equation and in a similar fashion, the backward differential
equation can be derived. (Cassandras and Lafortune 2009)
According to (Stewart 2009) the forward differential equation can also be derived like this :
The Chapman-Kolmogorov equations for CTMC are derived from Markov property that states
𝑝𝑖𝑗(𝑠, 𝑡) = ∑ 𝑝𝑖𝑘(𝑠)
𝑎𝑙𝑙 𝑘
𝑝𝑘𝑗(𝑡) 𝑓𝑜𝑟 𝑖, 𝑗 = 0,1,2, … 𝑎𝑛𝑑 𝑠 ≤ 𝑢 ≤ 𝑡
To transfer from state i at time s to state j at time t , some state k will be visited as an
intermediate state between states i and j at an intermediate time u . When the CTMC is
homogenous, this is written as follows:
𝑝𝑖𝑗(𝑡 + Δ𝑡) = ∑ 𝑝𝑖𝑘(𝑡)
𝑎𝑙𝑙 𝑘
𝑝𝑘𝑗(∆𝑡) = ∑ 𝑝𝑖𝑘(𝑡)
𝑘≠𝑗
𝑝𝑘𝑗(∆𝑡) + 𝑝𝑖𝑗(𝑡)𝑝𝑗𝑗(∆𝑡) 𝑓𝑜𝑟 𝑡, ∆𝑡 ≥ 0
Thus
𝑝𝑖𝑗(𝑡 + Δ𝑡) − 𝑝𝑖𝑗(𝑡)
Δ𝑡
= ∑ (𝑝𝑖𝑘(𝑡)
𝑝𝑘𝑗(∆𝑡)
Δ𝑡
+ 𝑝𝑖𝑗(𝑡)
𝑝𝑗𝑗(∆𝑡)
Δ𝑡
−
𝑝𝑖𝑗(𝑡)
Δ𝑡
)
𝑘≠𝑗
= ∑ ( 𝑝𝑖𝑘(𝑡)
𝑝𝑘𝑗(∆𝑡)
Δ𝑡
+ 𝑝𝑖𝑗(𝑡) (
𝑝𝑗𝑗(∆𝑡) − 1
Δ𝑡
))
𝑘≠𝑗
Taking the limit as Δ𝑡 → 0 and recalling

24. 14
𝑞𝑖𝑗 = lim
Δ𝑡→0
{
𝑝𝑖𝑗(Δ𝑡)
Δ𝑡
} 𝑓𝑜𝑟 𝑖 ≠ 𝑗 ; 𝑞𝑗𝑗 = lim
Δ𝑡→0
{
𝑝𝑗𝑗(Δ𝑡) − 1
Δ𝑡
} 𝑓𝑜𝑟 𝑖 = 𝑗
𝑑𝑝𝑖𝑗(𝑡)
𝑑𝑡
= ∑ 𝑝𝑖𝑘(𝑡)
𝑘≠𝑗
𝑞𝑘𝑗 + 𝑝𝑖𝑗(𝑡)𝑞𝑗𝑗
That is to mean the forward differential equation is:
𝑑𝑝𝑖𝑗(𝑡)
𝑑𝑡
= ∑ 𝑝𝑖𝑘(𝑡)
𝑘≠𝑗
𝑞𝑘𝑗 𝑓𝑜𝑟 𝑖, 𝑗 = 0,1, … ; 𝑎𝑛𝑑 𝑖𝑛 𝑚𝑎𝑡𝑟𝑖𝑥 𝑛𝑜𝑡𝑎𝑡𝑖𝑜𝑛:
𝑑𝑃(𝑡)
𝑑𝑡
= 𝑃(𝑡)𝑄
2.4.2. The backward differential equation is derived as in the following steps:
𝑝𝑖𝑗(𝑡 + Δ𝑡) = ∑ 𝑝𝑖𝑘(∆𝑡)
𝑎𝑙𝑙 𝑘
𝑝𝑘𝑗(𝑡) 𝑓𝑜𝑟 𝑡, ∆𝑡 ≥ 0
= ∑ ( 𝑝𝑖𝑘(∆𝑡) 𝑝𝑘𝑗(𝑡) + 𝑝𝑖𝑖(∆𝑡)𝑝𝑖𝑗(𝑡))
𝑘≠𝑗
𝑝𝑖𝑗(𝑡 + Δ𝑡) − 𝑝𝑖𝑗(𝑡)
Δ𝑡
= ∑ (
𝑝𝑖𝑘(∆𝑡)
Δ𝑡
𝑝𝑘𝑗(𝑡) +
𝑝𝑖𝑖(∆𝑡)
∆𝑡
𝑝𝑖𝑗(𝑡) −
𝑝𝑖𝑗(𝑡)
Δ𝑡
)
𝑘≠𝑗
= ∑ (
𝑝𝑖𝑘(∆𝑡)
Δ𝑡
𝑝𝑘𝑗(𝑡) + (
𝑝𝑖𝑖(∆𝑡) − 1
Δ𝑡
) 𝑝𝑖𝑗(𝑡))
𝑘≠𝑗
Taking the limit as Δ𝑡 → 0 and recalling
𝑞𝑖𝑗 = lim
Δ𝑡→0
{
𝑝𝑖𝑗(Δ𝑡)
Δ𝑡
} 𝑓𝑜𝑟 𝑖 ≠ 𝑗 ; 𝑞𝑖𝑖 = lim
Δ𝑡→0
{
𝑝𝑖𝑖(Δ𝑡) − 1
Δ𝑡
} 𝑓𝑜𝑟 𝑖 = 𝑗
Thus:
𝑑𝑝𝑖𝑗(𝑡)
𝑑𝑡
= ∑ (𝑞𝑖𝑘 𝑝𝑘𝑗(𝑡) + 𝑞𝑖𝑖 𝑝𝑖𝑗(𝑡))
𝑘≠𝑗
That is to mean, according to (Stewart 2009), the backward differential equation is:
𝑑𝑝𝑖𝑗(𝑡)
𝑑𝑡
= ∑ 𝑞𝑖𝑘
𝑘≠𝑗
𝑝𝑘𝑗(𝑡) 𝑓𝑜𝑟 𝑖, 𝑗 = 0,1, … ; 𝑎𝑛𝑑 𝑖𝑛 𝑚𝑎𝑡𝑟𝑖𝑥 𝑛𝑜𝑡𝑎𝑡𝑖𝑜𝑛 ∶
𝑑𝑃(𝑡)
𝑑𝑡
= 𝑄𝑃(𝑡)
And the solution for this differential equation is by matrix exponential:
𝑃(𝑡) = 𝑐𝑒𝑄𝑡
= 𝑒𝑄𝑡
= 𝐼 + ∑
𝑄𝑛
𝑡𝑛
𝑛!
∞
𝑛=1

25. 15
The constant is 𝑐 = 𝑃(0) = 𝐼 .
2.5. The mean sojourn time of the continuous time Markov chain:
It is the average time spent in the state .Let {𝑋(𝑡), 𝑡 ≥ 0} be a homogenous continuous-time
Markov chain and it is in a non-absorbing state 𝑖 at time 𝑡 = 0 . Let 𝑇𝑖 be the time until a
transition out of state 𝑖 occurs, if Markov chain started at time 𝑡 = 0 in state 𝑖 and has not moved
from state 𝑖 by time 𝑠 which is equivalent to saying that 𝑃{𝑇𝑖 > 𝑠|𝑋(0) = 𝑖} = 1, then 𝑃{𝑇𝑖 >
𝑠 + 𝑡|𝑇𝑖 > 𝑠} = 𝑃{𝑇𝑖 > 𝑡} and the continuous random variable time 𝑇𝑖 is memoryless. Since the
only continuous distribution that has the memoryless property (the distribution of residual time
being equal to the distribution itself) is the exponential distribution, this leads to the fact that the
duration of the time until a transition occurs from state 𝑖 is exponentially distributed. In a
homogenous continuous-time Markov chain with a non-absorbing 𝑖 state which can move to one
or more states ≠ 𝑖 , the memoryless property of the chain compels the duration of time, until this
transition takes place, to be exponentially distributed with a rate of transition 𝑞𝑖𝑗 . So the time to
reach some state 𝑗 ≠ 𝑖 has an exponential distribution with rate 𝑞𝑖𝑗 . More to say is that upon
exiting state , more than one state can be reached, and subsequently, a race condition is started
to take place and the transition to the winning state occurs, the state which minimizes the sojourn
time in state i . Because the minimum value of several exponentially distributed random
variables is also an exponentially distributed random variable with a rate equal to the sum of the
original rates, this drives the conclusion that the time spent in state 𝑖 of a homogenous
continuous time Markov chain is exponentially distributed. This sojourn time is 𝑢𝑖 = ∑ 𝑞𝑖𝑗
𝑖≠𝑗 .
Therefore, the probability distribution of the mean sojourn time in state 𝑖 is given by:
𝐹𝑖(𝑥) = 1 − 𝑒−𝑢𝑖𝑥
, 𝑥 ≥ 0 & 𝑢𝑖 = ∑ 𝑞𝑖𝑗
𝑗≠𝑖
= −𝑞𝑖𝑖
Putting these concepts together leads to realization that the mean sojourn time in any state of a
homogenous continuous time Markov chain must be exponentially distributed. This is violated in
non-homogenous chain as it is not exponentially distributed (Stewart 2009).
2.6. State Probability Distribution:
According to (Cassandras and Lafortune 2009), it is the probability vector 𝜋(𝑡) that
represents the probability that a system will be in a particular state at a specific time point given
its initial state probability vector 𝜋(0), transition rate matrix 𝑄 and the state space 𝑋. This
analysis can be conducted in two approaches:
2.6.1. Transient Analysis:
According to (Cassandras and Lafortune 2009), define state probability 𝜋𝑗(𝑡) = 𝑃[𝑋(𝑡) = 𝑗]
and condition the event [𝑋(𝑡) = 𝑗] on the event [𝑋(0) = 𝑖] with a defined 𝜋𝑖(0) = 𝑃[𝑋(0) = 𝑖].
Thereafter the rule of total probability implies that:

26. 16
𝜋𝑗(𝑡) = 𝑃[𝑋(𝑡) = 𝑗] = ∑ 𝑃[𝑋(𝑡) = 𝑗|𝑋(0) = 𝑖]. 𝑃[𝑋(0) = 𝑖]
𝑎𝑙𝑙 𝑖 = ∑ 𝑃𝑖𝑗(𝑡)𝜋𝑖(0)
𝑎𝑙𝑙 𝑖
This relation can be rewritten in matrix notation as: 𝜋(𝑡) = 𝜋(0)𝑃(𝑡)
And since 𝑃(𝑡) = 𝑒𝑄𝑡
thus the state probability vector at time t is given by 𝜋(𝑡) = 𝜋(0)𝑒𝑄𝑡
2.6.2. Steady State Analysis:
According to (Cassandras and Lafortune 2009), this approach, the steady state behavior of
the system is of much interest, a great advantage and a huge benefit to be calculated. The system
is turned on and has been working for some time; then its performance is tested in the long run to
see how all state probabilities have reached some fixed, unchangeable values and no longer vary
as time elapses. This relies on some basic requisites such as:
Presence and evaluation of the limit: 𝜋𝑗 = ∑ 𝜋𝑗(𝑡)
𝑡→∞ . If this limit exists, 𝜋𝑗 is called steady
state, equilibrium, or stationary state probability.
If differentiating 𝜋(𝑡) = 𝜋(𝑡)𝑒𝑄𝑡
with respect to 𝑡 and substitute 𝑡 = 0 the differential equation is
𝑑
𝑑𝑡
𝜋(𝑡) = 𝜋(𝑡)𝑄. Solving such a system, to obtain an explicit solution even for a simple Markov
chain, is cumbersome. So if the 𝜋𝑗 = ∑ 𝜋𝑗(𝑡)
𝑡→∞ exists, this implies that as 𝑡 → ∞ this quantity
𝑑
𝑑𝑡
𝜋(𝑡) → 0, since 𝜋(𝑡) no longer depends on 𝑡.Therefore,
𝑑
𝑑𝑡
𝜋(𝑡) = 𝜋(0)𝑄 reduces to 𝜋(𝑡)𝑄 =
0.
In an irreducible continuous-time Markov chain consisting of positive recurrent states, a unique
stationary state probability distribution vector 𝜋 exists such that 𝜋𝑗 > 0 and is independent of
the initial state probability vector. Moreover, 𝜋 is determined by solving 𝜋𝑄 = 0 subject to
∑ 𝜋𝑗
𝑎𝑙𝑙 𝑗 = 1 .

27. 17
Chapter Three: Non-Alcoholic Fatty Liver Disease
3.1. Prevalence
NAFLD is one of the most common chronic liver diseases .Changing lifestyle behavior
during the last few decades attributed to the bad eating habits with consumption of high fat and
fructose diet resembling the western diets and the sedentary life with lack of exercise have
increased dramatically worldwide. These factors globally increase the prevalence of obesity and
type II diabetes worldwide. NAFLD is accidentally discovered during routine ultrasonography
follow-up or using magnetic resonance spectroscopy. Tests of liver enzymes underrate the true
prevalence (Younossi et al., 2017). Figure (3.1) illustrates the prevalence of the NAFLD
worldwide and the distribution of the PNPLA3 genotype:
FIGURE (3. 1): The worldwide estimated prevalence of NAFLD and the distribution of PNPLA3 genotype
3.1.1. NAFLD in USA:
The prevalence is estimated to be 24% using U/S in the USA. It is 21% if noninvasive
methods like Fatty Liver Index are used. It varies according to ethnicity. So Hispanic Americans
are first to come preceding Americans of European origin. Lastly, the African Americans are the
least affected group. The last group has the highest levels of hypertension and obesity. Some
studies reveal that the Latino patients having NASH were less aged and less physical exercise
than the white ptients of non-Latino descent; however, these patients with NASH were more
susceptible and liable to insulin resistance than the white Latino patients. Within the ethnicity
group, the prevalence exhibits variations according to the parent country: it is higher at 33% in
Americans of Mexican descent, at 16% in those of Dominican descent, and it has a prevalence of
18% in people of Puerto Rican descent, even after controlling for other risk factors (sex, age,

28. 18
waist circumference, BMI, serum HOMA, serum HDL, hypertension, serum C-reactive protein
and triglycerides as well as level of education) using multivariate analysis (Younossi et al. 2017).
3.1.2. NAFLD in South America:
in Brazil, the estimated prevalence was 30% using U/S. in Chile, it was 23% detected by U/S.
In Columbia, it was 26.6% of men using U/S. However, countries like Paraguay, Peru, Uruguay,
Ecuador, and Argentina had reported prevalence rates to be as minimum as 13% (Peru) up to as
maximum as 24 %( Uruguay) (Younossi et al. 2017).
3.1.3. NAFLD in Europe:
Although prevalence has wide variations according to the method used to reveal NAFLD,
nearly one-quarter of the European population suffers from NAFLD. A Meta-analysis study
released in 2016 documented a prevalence of 23.71% in Europe, ranging from 5% to 44% in
various countries. The data from Germany estimated a 30% prevalence rate using U/S. NAFLD
accounted for deranged liver tests is 26.4% in England. In France simple steatosis was observed
in 26.8% of liver biopsies due to questionable deranged liver tests, with 32.7 % of those biopsies
having NASH. In northern Italy, NAFLD evaluated by U/S was similar in those having liver
disease and those free of liver disease (25% in contrast with 20%, p=0.203). Statistical data from
Spain reported a resembling rate of 25.8%. From Romania, a study conducted on 3005 in-
patients population not suffering from any liver diseases revealed that NAFLD evaluated by
sonography was present in 20% of those hospitalized patients. A Hungarian study reported a
22.6% prevalence rate of fatty liver detected by U/S. (Younossi et al. 2017).
3.1.4. NAFLD in Asia-Pacific and Africa:
There are enormous variations between countries comprising this region due to huge
differences between its countries in economic, educational and political aspects and health-care
systems. These factors impact the individuals’ lifestyle, nutritional culture and sedentary
behavior. Data are not available in a comprehensive form due to the lack of statistical surveys
extending throughout the country to assess fatty liver. Subsequently, there are marked
dissimilarities in the NAFLD prevalence between different regions within the same nation.
Prevalence in Chengdu (Southwest China) was 12.5%. It was 15% in Shanghai (east china) and
17% in Guangdong (south China), while 24.5%. in central China. A study of 7152 employees in
shanghai utilizing U/S for detecting fatty liver released in 2012 estimated NAFLD prevalence to
be as great as 38.17%. In a study from Hong Kong, “proton magnetic resonance spectroscopy”
determined the quantity of fat in the liver and estimated its prevalence rate as 28.8%; it was
19.3% in non-obese individuals, and it was 60.5% among the obese. A study from Taiwan
reported that the prevalence was 11.4% in the general population. It was 50% in the elderly and
66.4% in persons with a sedentary lifestyle like taxi drivers. It was 25% in Japan in 2005 as
diagnosed by U/S. Using the same modality, the prevalence in 141610 individuals from South
Korea was 27.3% in 2013 (Younossi et al. 2017).

29. 19
South Asia and Indian regions are towards quick urbanized changes in the social and
economic aspects. In the rural India, preserving their cultural diet and traditional lifestyles, the
prevalence rate is low (9%). In contrast, the prevalence rate fluctuates between 16 and 32% in
urban areas as the lifestyle mimics the trends in western countries. Similar variations in rates
between rural and urban areas (5-30%) were obtained from smaller surveys in Indonesia,
Malaysia, Singapore, and Sri Lanka. Data from Africa are scant. In Nigeria, the rate was 9.5-
16.6% in diabetic individuals and 1.2-4.5% in non-diabetic individuals. Comparably, the
prevalence in overweight or obese South African individuals was 45-50%, while in 2014; it was
20% in the population studied from Sudan (Younossi et al. 2017).
The data about the prevalence in Egypt is scarce and it needs further studies. (Fouad et al.,
2022)
3.2. Definition and Terminology:
NAFLD covers a spectrum of hepatic diseases containing two essential disease phenotypes.
The first is nonalcoholic fatty liver (NAFL; simple steatosis), where triglycerides accumulate in
more than 5% of the hepatocytes without histologic evidence of inflammation, cellular injury, or
fibrosis. The second one is non-alcoholic steato-hepatitis (NASH), where steatosis comes with
histologic features of necro-inflammation and hepatocyte ballooning degeneration (with or
without evidence of fibrosis). These phenotypes may evolve to cirrhosis, liver-related mortality
and hepatocellular carcinoma (HCC). NAFLD, by definition, occurs when excessively
consuming alcohol is absent; a cutoff point of not more than 20g/day for women and not more
than 30g/day for men is used to distinguish and separate it from alcohol-related liver disease.
Primary NAFLD is a disease related to obesity, insulin resistance, hypertension, and metabolic
syndrome criteria. Most NAFLD patients reside in this category. The secondary NAFLD points
toward patients with causes other than metabolic syndrome-associated conditions, such as drug
or toxin-induced fatty liver and rare inherited genetic metabolic diseases (Boyer and Lindor
2016). Table (3.1) lists risk factors of NAFLD.
NAFLD is a disease characterized by wide variation in the disease seriousness and disease
consequences reflecting the interactions between extrinsic and intrinsic factors. Although it is
initiated by high fat and carbohydrate diet associated with a sedentary lifestyle composing some
of the extrinsic factors, the contributors of intrinsic factors presented in genetic background
crucially determine how the patient responds to the excess caloric intake and metabolic stressors.
These factors are known as modifiers or adaptors (Ribeiro et al. 2004)(Brand and Esteves 2005).

30. 20
Table (3. 1): Risk factors for NAFLD
Risk factor Effect
Age High risk for NAFLD and advanced fibrosis if age>45 years
Metabolic
syndrome
The more criteria there are , the higher the probability of NASH and advanced fibrosis is .
sex Male > famale
ethnicity High in Hispanics, intermediate in white , low in black
Dietary factors Diets with high contents of saturated fats, fructose and cholesterol but low in carbohydrates
increase the risk. caffeine is beneficial
Obstructive sleep apnea It elevates risk for NASH and advanced fibrosis
Genetic factors PNPLA 3 and TM6SF2 increase the risk of NASH and advanced fibrosis, PNPLA3 increase the risk for
HCC.
“Metabolic syndrome” is defined as having abdominal adiposity distinguished by waist
circumference > 94 cm for males and > 80 for females in eastern countries. It is > 102 cm for
males and > 88 cm for females in western countries. Plus two or more of the following:
1. Blood glucose ≥100 mg/dL or drug treating diabetes.
2. Arterial pressure ≥ 130/85 mmHg or drug treating hypertension.
3. Triglyceride levels ≥150 mg/dL or drug treating high level in the blood.
4. HDL cholesterol levels < 40mg/dL for males and <50 mg/dL for females or drug treating
this condition.
3.3. Dynamic model of NAFLD:
As discussed above, some NAFL patients can evolve to NASH and advanced fibrosis. On the
other hand, some of those NASH patients can regress to NAFL as time elapses. It is
hypothesized that the patient cycles between NAFL and NASH in early stages of NAFLD.
Regardless of the biopsy results being NAFL or NASH, about 80% of them are slow progressors.
That means they are unlikely to progress further beyond mild fibrosis (F0 to F2). However,
approximately 20% manifest rapid fibrosis progression and develop advanced fibrosis and
cirrhosis(F3 to F4) within a few years (De and Duseja 2020). Most NAFLD patients are slow
progressors in evolving from F0 to F1. However, a subset of these patients is rapid progressors,
evolving from F0 to F3 or F4. In a study on 108 patients having sequential liver biopsy with a
median interval of 6.6 years, 42% were progressors, 40% had stable fibrosis, while 18% were
regressors (McPherson et al. 2015). Figure (3.2) shows the dynamic model of NAFLD.

31. 21
FIGURE (3. 2): A dynamic model of NAFLD
3.4. Diagnosis of NAFLD:
The practice guidelines of the American Association for the study of Liver Disease define
NAFLD by the presence of the following features (Chalasani et al. 2018):
 The existance of hepatic steatosis established by histology or imaging.
 Exclusion of other secondary causes of hepatic steatosis, especially significant alcohol
consumption.
 Exclusion of any other causes of chronic liver disease like a drug, infection, autoimmune
diseases, hereditary, genetic, or metabolic causes like hepatitis B and C infection, HIV,
autoimmune hepatitis, primary sclerosing cholangitis, primary biliary cirrhosis, Celiac
disease, Wilson’s disease, hemochromatosis, α-1 antitrypsin deficiency, cystic fibrosis
and porphoryia .
Assessment of the patient in clinical practice should evaluate the disease activity, that is to
mean, whether the disease is NAFL or NASH, the stage of fibrosis and the severity of the risk
factors such as insulin resistance (IR) and metabolic syndrome (MetS) components.
3.4.1. Liver biopsy:
Liver biopsy permits direct examination of liver tissue to evaluate inflammatory disease
activity (grade of disease) and evolution of fibrosis to cirrhosis (stage of fibrosis). However, one
of the major limitations of liver biopsy is sampling error because < 1/50000 of the total hepatic

32. 22
volume is sampled at a specific single time point with the heterogeneity of NAFLD distribution
throughout the hepatic parenchyma. Also, accurate diagnosis is observer-dependent and
influenced by pathologist experience. (Castera et al. 2019).
The noninvasive tests can be subdivided into two major groups that can be used to diagnose
steatosis, activity, and fibrosis. Noninvasive techniques include serological or biological tests
and imaging or physical techniques.
3.4.2. Noninvasive Tests for Diagnosis of NAFLD:
3.4.2.1. Imaging Tests:
The existence of NAFLD can be detected radio-logically using the ultrasonography,
computed tomography (CT), magnetic resonance imaging (MRI), and Transient elastography.
“Magnetic resonance elastography” (MRE) is a quantitative MRI-based methodology to image
the liver stiffness. MRE is better than other biomarkers, scoring systems and US-based
elastography for the diagnosis of liver fibrosis (Yoneda et al. 2018)(Boyer and Lindor 2016).
3.4.2.2. Serological Tests:
The serological markers for recognizing stages of liver fibrosis are classified into indirect
markers that reflect the reduction in hepatic functions (AST/ALT and platelet levels) and direct
markers concerned with fibro-genesis. Serum cytokeratin (CK)-18 is an indicator or sign for
“hepatocyte apoptosis or death”. (Paul 2020).
There are many scoring systems to detect steatosis and fibrosis; the followings are some of
them:
1) One of the most commonly used systems for detecting fibrosis is the AST to platelet ratio
index (APRI). (Tapper et al. 2014)
2) Another scoring system is the FIB-4 to detect fibrosis. (Shah et al. 2009)
3) NAFLD fibrosis score (NFS) is a test for fibrosis detection. (Angulo, Jason M Hui, et al.
2007)
4) Enhanced liver fibrosis (ELF) test is a commercial panel of markers concentrating on matrix
turnover for fibrosis detection. (Lichtinghagen et al. 2013).
5) Fibro-test can be used to exclude advanced fibrosis (F3-F4). It is a commercial marker panel
with a patent algorithm. (Boyer and Lindor 2016).
6) Fatty liver index (FLI) is a test to assess hepatic steatosis.(Bedogni et al. 2006).

33. 23
7) NAFLD liver fat score and liver fat equation supply the general practitioner with a simple,
inexpensive, and non-invasive tool to predict liver fat content in susceptible individuals.
(Kotronen et al. 2009).
3.4.3. Diagnostic Algorithms of NAFLD:
Liver biopsy, as previously mentioned, is the standard gold method to diagnose the degree of
steatosis, inflammation and fibrosis as well as to determine the presence or absence of liver-
related morbidities like hepatocellular carcinoma (HCC). Many algorithms have been proposed
by many societies worldwide to illustrate the suggested plan for physicians to follow, aiming to
achieve early diagnosis, treatment and follow-up of persons with NAFLD, thereby reducing
liver-related morbidities and mortalities. Here are some of these algorithms; almost all start with
risk stratification of individuals to identify patients with advanced fibrosis who are liable for
progression to severely morbid complications. The extent of liver fibrosis is the most prognostic
factor for such progression.
* Figure (3.3) points out a suggested algorithm for risk stratification in NAFLD patients utilizing the noninvasive
tests (Castera et al. 2019).
* Figure (3.4.1), Figure (3.4.2), Figure (3.4.3), and Figure (3.4.4) demonstrate the EASL-EASD-EASO clinical
practice guidelines for the management of NAFLD in Type 2 Diabetes (Sberna et al. 2018).
3.5. Treatment.
The keystone in treatment of NAFLD must start with treating the risk factors like obesity,
dyslipidemia, diabetes and hypertension. Initially, the patients must continue to change lifestyles
as diet habits and avoidance of sedentary life
Currently, no approved drug treatment for NAFLD/NASH is available; however, abundant
drugs are being scrutinized in phase II and phase III clinical trials. The outcome from these trails
has been favorable and hopeful regarding amelioration in histological features of the disease like
steatosis, inflammation and fibrosis. As NAFLD is a heterogeneous complex disease process
with a chronic prolonged course evolving, examining the effectiveness of these drugs in the long
run and their safety and effectiveness is crucial, especially when considering that some of these
drugs have serious and possible metabolic side effects. Heading the substantial and remarkable
impediments to registration (by increasing the awareness and alertness of the seriousness of the
disease by both physicians and patients) into clinical trials is paramount, as well as improving the
design of these trials will play a tremendous role in achieving success in these trials, and hence
approval of such therapeutic drugs. Utilizing the non-invasive markers of steatosis,
inflammation, and fibrosis and the clinical trial designs can accomplish this goal.
In the FLINT study conducted on 283 non-cirrhotic NASH patients taking 25 mg daily
obeticholic acid; the improvement in histology detected by NAS was 2 points or more with no

34. 24
deterioration of fibrosis, and 35% of patients taking OCA had a decrement in fibrosis score by at
least one stage in comparison with 19 % in the placebo arm.
REGENERATE study (still in progress with an estimated primary completion date of
September 2025 as shown on clinicaltrials.gov official site) will evaluate the safety and efficacy
of Obeticholic Acid(OCA) in NASH patients with fibrosis who are randomizes to a daily dose of
25 mg, 10 mg, and placebo, with endpoints like amelioration of fibrosis by at least one stage and
decaying of NASH with no deterioration of fibrosis. At 18 month of randomization, liver biopsy
revealed statistically significant histological amelioration of fibrosis and decaying of NASH with
no deterioration in fibrosis for both 10 mg and 25 mg doses. In the GOLDEN study, conducted
on 274 NASH patients, 120 mg Elafibranor taken daily for 52 weeks induced decaying of
moderate to severe NASH in meaningfully higher percentage of patients than to the placebo,
moreover; these patients also showed lowering in fibrosis stage compared to non-resolving
NASH patients. RESOLVE-IT trial (the last update was on November 30, 2020, as shown on
clinicalrials.gov official site, but the study is still in progress according to(Guirguis et al. 2020))
emerged in May 2020 had shown that 19.2% of patients, on 120 mg daily Elafibranor, had
NASH decay without deterioration of fibrosis compared to 14.7% in the placebo group, which
was not statistically significant. Furthermore, 24.5% of patients had shown fibrosis amelioration
of more than one stage compared to 22.4% in the placebo group, which was also not statistically
significant.
In the CENTAUR trial, conducted over 289 patients taking cenicriviroc (CVC), 150 mg daily
and placebo for 52 weeks, no comparative betterment in NAS between the NASH group and
placebo was seen, however; there was one stage or more amelioration of fibrosis with no
deterioration of NASH in the group taking the CVC compared with placebo group. The
AURORA trial (primary completion dates were October 2021 according to clinicaltrails.gov site
and October 2028 according to (Guirguis et al. 2020)) will evaluate long-term safety and efficacy
of 150 mg daily CVC for the treatment of fibrosis in NASH adult patients at 2 phases, the first
has an endpoint of at least one stage amelioration of fibrosis without deterioration of NASH at
month 12, and phase 2 has an endpoint that is cirrhosis, liver-related outcome as HCC, and all
causes of mortality. In a small, open-label, randomized phase II trial including 72 biopsy-proven
NASH patients (NAS ≥ 5 and stage 2-3 liver fibrosis) receiving 18 mg daily Selonsertib for 24
weeks ,there was significant improvement in liver disease activity, fibrosis, stiffness, liver fat
content, and progression to cirrhosis (Alkhouri, Poordad, and Lawitz 2018).
FLINT, GOLDEN, and CENTAUR are phase IIb placebo-controlled RCT( randomized
control trial), while REGENERATE, RESOLVE-IT, and AURORA are randomized, placebo-
controlled, double-blinded, multicenter phase III trials

35. 25
FIGURE (3. 3): risk stratification and management of NAFLD patients
NALF is suspected if steatosis is detected by U/S or abnormal elevation of liver enzymes in
high-risk patients ( type 2 diabetes, obesity, or metabolic syndrome) excluding other causes of
chronic liver disease. Sequential tests are performed according to local availability and the
situation of use: in primary health care units, the first line tools that are inexpensive, simple,
noninvasive, and widely available are the serum biomarkers like FIB-4 or NFS with high
negative predictive value (88%-95%) to negate advanced fibrosis. Low-risk fibrosis patients (55-
58% of cases, with FIB-4<1.3 or NFS<-1.455) are given no further assessments other than
lifestyle modifications and exercises. Intermediate-risk patients (30% of cases with FIB-4 =1.3 to
3.25 or NFS=-1.455 to 0.672) as well as high-risk patients of advanced fibrosis (12%-15% of
cases with FIB-4>3.25 or NFS >0.672 and positive predictive value 75%-90%) are referred to a
specialized center for LSM( liver stiffness measurements) using transient elastography (TE) in
fasting state with M probe for patients with “skin-liver capsule distance” <25 mm otherwise XL
probe is used for more obese patients. Low-risk patients for advanced fibrosis( LSM <8 kPa;
NPV= 94%-100%) should repeat the assessment within one year . Patients with intermediate-risk
( LSM =8-10 kPa) or high-risk ( LSM ≥10 kPa; PPV=47%-70%) of having advanced fibrosis
should undergo liver biopsy. According to availability, there are alternatives to the scoring
system like the commercial patent tests such as Fibro-test or Fibro-meter. In case of failure of
XL-probe of TE, there are alternatives such as MRE, especially in patients with BMI >35 kg/m2
.
All patients should be offered lifestyle modifications and exercise.

36. 26
Sberna et al. 2017 conducted a single-center retrospective observational study to evaluate the
application of the EASL-EASD-EASO in type 2 diabetic patients; they showed up that there was
an excessive referral rate to hepatologists in liver clinic. As illustrated in the above figure, 179
diabetic patients were screened for the presence of steatosis with FLI. Any patient with increased
liver enzymes, whether with steatosis absent [FLI < 60 ,(n=1)], or with steatosis present [FLI ≥
60 ,(n=55)] was referred to a liver clinic (a total of 56). Sixteen patients with FLI <60 and with
normal liver enzymes were assigned to follow up every 3-5 years. The remaining 107 patients
with FLI ≥ 60 and with normal liver enzymes were evaluated for liver fibrosis with NFS. Of
those, 11 patients with NFS ≤ -1.455 were decided to be followed up every 2 years, 68 patients
with NFS levels between -1.455 and 0.675, and 28 patients with NFS ≥ 0.675 were referred to a
liver clinic. Thus the total referral rate was approximately 85%. Further assessment of fibrosis
score of those 56 patients with elevated liver enzyme could reduce the number of the patients to
be referred to a liver clinic.
People with T2DM(n=179)
Evaluation of Steatosis
With FLI
FLI ≥ 60 (n=162) FLI < 60 (n=17)
Any increase in ALT, AST,
GGT ( n= 55+1=56 )
Normal Liver
Enzymes (n=107)
Normal Liver
Enzymes (n=16)
Evaluation of Fibrosis With
NFS
NFS ≤ -1.455
(n=11)
-1.455 < NFS < 0.675
(n=68)
NFS ≥ 0.675
(n=28)
Follow-Up Specialist Referral (n=56+68+28=152) Follow-Up
FIGURE (3. 4.1): EASL-EASD-EASO clinical practice guidelines for management of NAFLD in type 2 diabetes

37. 27
FIGURE (3.4.2):EASL-EASD-EASO clinical practice guidelines for the management of NAFLD in type 2 diabetis using
steatotest and fibrotest
But when Sberna et al. 2017 used another system to evaluate the application of the EASL-
EASD-EASO on type 2 diabetic patients, they showed up that the referral rate to liver clinic
decreased but still high. As illustrated in the above figure, 179 diabetic patients were screened
for the presence of steatosis with steatoTest. Any patient with increased liver enzymes, whether
with steatosis absent [steatoTest < 0.57, (n=5]), or with steatosis present [steatoTest ≥ 0.57,
(n=51)] was referred to a liver clinic (a total of 56). Thirty five patients with steatoTest < 0.57
and with normal liver enzymes were assigned to follow up every 3-5 years. The remaining 88
patients with steatoTest ≥ 0.57 and with normal liver enzymes were evaluated for liver fibrosis
with FibroTest. Of those, 82 patients with FibroTest < 0.58 were decided to be followed up every
two years while 6 patients with FibroTest ≥ 0.58 were referred to a liver clinic. Thus the total
referral rate was approximately 34.6% (62 patients). Sberna et al. reported that:‘‘it would not be
possible to refer such a high proportion of people with Type 2 diabetes to a liver clinic.The
application of EASL-EASD-EASO guidelines cannot be used in clinical practice in people with
Type 2 diabetes. It is essential to develop specific steatosis scores and fibrosis scores for people
with Type 2 diabetes in order to improve the selection of patients to be referred to a liver clinic
’’.
People with T2DM(n=179)
Evaluation of Steatosis
With steato-Test
Steato-Test ≥ 0.57 (n=139)
Steato-Test < 0.57 (n=40)
Any increase in ALT, AST,
GGT (n=51+5=56)
Normal Liver
Enzymes (n=88)
Normal Liver
Enzymes (n=35)
Evaluation of Fibrosis with
Fibro-Test
Fibro-Test <0.58 (n=82)
Fibro-Test ≥0.58 (n=6)
Follow-Up
Specialist Referral (n=56+6=62)
Follow-Up

38. 28
FIGURE (3.4.3):EASL-EASD-EASO clinical practice guidelines for the management of NAFLD in type 2 diabetis using 1H-
MRS and NFS
Sberna et al. 2017 used another system. As illustrated in the above figure, 179 diabetic
patients were screened for the presence of steatosis with 1H-MRS (proton magnetic resonance
spectroscopy); it is very sensitive in detecting steatosis and can reproducibly quantify fat content
in the liver. Any patient with increased liver enzymes, whether with steatosis absent [hepatic fat
content ≤ 5.5%, (n=9)], or with steatosis present [hepatic fat content > 5.5%, (n=47)] was
referred to a liver clinic (a total of 56). Forty seven patients with hepatic fat content ≤ 5.5% and
with normal liver enzymes were assigned to follow up every 3-5 years. The remaining 76
patients with hepatic fat content > 5.5% and with normal liver enzymes were evaluated for liver
fibrosis with NFS. Of those, 9 patients with NFS ≤ -1.455 were decided to be followed up every
2 years, 45 patients with NFS levels between -1.455 and 0.675 and 22 patients with NFS ≥ 0.675
were referred to a liver clinic. So the total referral rate was 68.7%, less than 85% obtained when
FLI was initially used to detect steatosis, as; FLI was mainly designed to detect steatosis in
general population not in diabetic patients and spectroscopy is more sensitive than FLI to detect
the hepatic fat content.
People with T2DM(n=179)
Evaluation of Steatosis
With 1H-MRS
1H-MRS > 5.5 % (n=123) 1H-MRS ≤ 5.5 % (n=56)
Any increase in ALT, AST,
GGT (n=47+9=56)
Normal Liver
Enzymes (n=76)
Normal Liver
Enzymes (n=47)
Evaluation of Fibrosis With
NFS
NFS ≤ -1.455(n=9) -1.455 < NFS <0.675
(n=45)
NFS ≥ 0.675
(n=22)
Follow-Up Specialist Referral (n= 56+45+22=123) Follow-Up

39. 29
FIGURE (3.4.4):EASL-EASD-EASO clinical practice guidelines for the management of NAFLD in type II diabetis using 1H-
MRS and fibrotest
Sberna et al. 2017 used another system. As illustrated in the above figure, 179 diabetic
patients were screened for the presence of steatosis with 1H-MRS, as in the previous figure. Any
patient with increased liver enzymes, whether with steatosis absent [hepatic fat content ≤ 5.5%,
(n=9)], or with steatosis present [hepatic fat content > 5.5%, (n=47)] was referred to a liver clinic
(a total of 56). Forty seven patients with hepatic fat content ≤ 5.5% and with normal liver
enzymes were assigned to follow up every 3-5 years. The remaining 76 patients with hepatic fat
content > 5.5% and with normal liver enzymes were evaluated for liver fibrosis with FibroTest.
Of those, 72 patients with FibroTest < 0.58 were decided to be followed up every 2 years while 4
patients with FibroTest ≥ 0.58 were referred to a liver clinic. Thus the total referral rate was
33.5% (60 patients). This is the same rate when SteatoTest was initially used to detect steatosis
followed by FibroTest to detect fibrosis, which was still high.
People with T2DM(n=179)
Evaluation of Steatosis
With 1H-MRS
1H-MRS > 5.5 % (n=123) 1H-MRS ≤ 5.5 %(n=56)
Any increase in ALT, AST,
GGT (n=56)
Normal Liver
Enzymes (n=76)
Normal Liver
Enzymes(n=47)
Evaluation of Fibrosis with
Fibro-Test
Fibro-Test < 0.58 (n=72) Fibro-Test ≥ .58 (n=4)
Follow-Up Specialist Referral (n=56+4=60) Follow-Up

40. 30
Chapter Four: CTMC Analyzing NAFLD Progression (Small
Model)
Studying the natural history of disease during which individuals start at one initial state then
as time passes the patients move from one state to another, can be investigated by using
multistate Markov chains. The evolution of the disease over different phases can be monitored
by taking repeated observations of the disease stage at pre-specified time points following entry
into the study. The disease stage is recorded at the time of observation while the exact time of
state change is unobserved. NAFLD is a multistage disease process; in its simplest form has a
general structure model as depicted in Figure (4.1).
FIGURE (4. 1): general model structure .(Younossi et al. 2016)
NAFLD stages are modeled as time homogenous CTMC, meaning, 𝑃𝑖𝑗(∆𝑡) depends on
∆𝑡 and not on 𝑡 ,with constant transition intensities 𝜆𝑖𝑗 over time, exponentially distributed time
spent within each state and patients’ events follow the Poisson distribution. The states are one for
the susceptible cases (state 1) and one for NAFLD cases (state 2), and two absorbing states ; one
for the death due to NAFLD (state 3) and one for death due to any other cause (state 4). The
transition rate 𝜆12 is the rate of progression from state 1 to state 2, while the transition rate 𝜇12 is
the regression rate from state 2 to state 1. The transition rate 𝜆23 is the progression rate from
state 2 to state 3 and 𝜆24 is the progression rate from state 2 to state 4. For simplicity, all
individuals are assumed to enter the disease process at stage one, and they are all followed up
with the same length of the time interval between measurements.
In this chapter the transition probabilities and transition rates are thoroughly discussed. Also,
the mean sojourn time and its variance are reviewed as well as the state probability distribution
and its covariance matrix. This is followed by an exploration of the life expectancy of the
patients and the expected numbers of patients in each state. Lastly, a hypothetical numerical
example is used to illustrate these concepts.

41. 31
4.1. Transition Probability Matrix
The transitions can occur at any point in time. The rates at which these transitions occur are
constant over time and thus are independent of t, that is to say, the transition of patient from
𝑠𝑡𝑎𝑡𝑒 𝑖 𝑎𝑡 𝑡𝑖𝑚𝑒 = 𝑡 𝑡𝑜 𝑠𝑡𝑎𝑡𝑒 𝑗 𝑎𝑡 𝑡 = 𝑡 + 𝑠 𝑤ℎ𝑒𝑟𝑒 𝑠 = ∆𝑡 depends on difference between two
consecutive time points. And it’s defined as 𝜃𝑖𝑗 (𝑡) = lim
∆𝑡→0
𝑃𝑖𝑗(∆𝑡)−𝐼
∆𝑡
or the Q matrix.
For the above multistate Markov model demonstrating the NAFLD disease process; the
forward Kolmogorov differential equations are represented by (4.1)
𝑑
𝑑𝑡
𝑃𝑖𝑗(𝑡) = [
𝑃11 𝑃12
𝑃21 𝑃22
𝑃13 𝑃14
𝑃23 𝑃24
0 0
0 0
𝑃33 0
0 𝑃44
] [
−(𝜆12 + 𝜆14)
𝜇21
𝜆12
−(𝜇21 + 𝜆23 + 𝜆24)
0
0
0
0
0
𝜆23
𝜆14
𝜆24
0
0
0
0
] (4.1)
The Kolmogorov differential equations:
𝑑𝑃11
𝑑𝑡
= −𝑃11(𝜆12 + 𝜆14) + 𝑃12𝜇21 (4.2 )
𝑑𝑃12
𝑑𝑡
= 𝑃11𝜆12−𝑃12(𝜇21 + 𝜆23 + 𝜆24) (4.3)
𝑑𝑃13
𝑑𝑡
= 𝑃12𝜆23 (4.4)
𝑑𝑃14
𝑑𝑡
= 𝑃11𝜆14+ 𝑃12𝜆24 (4.5)
𝑑𝑃21
𝑑𝑡
= −𝑃21(𝜆12 + 𝜆14) + 𝑃22𝜇21 (4.6)
𝑑𝑃22
𝑑𝑡
= 𝑃21𝜆12−𝑃22(𝜇21 + 𝜆23 + 𝜆24) (4.7)
𝑑𝑃23
𝑑𝑡
= 𝑃22𝜆23 (4.8)
𝑑𝑃24
𝑑𝑡
= 𝑃21𝜆14+ 𝑃22𝜆24 (4.9)
𝑃33 = 1 , 𝑃44 = 1
The solution of this system of equations will give the 𝑃𝑖𝑗(𝑡)
To get P11 using equations (4.2) and ( 4.3):
𝑙𝑒𝑡: 𝛾1 = 𝜆12 + 𝜆14 , 𝛾2 = 𝜇21 + 𝜆23 + 𝜆24
𝐷𝑃11 + 𝛾1𝑃11 − 𝜇21𝑃12 = 0 (4.10)
𝐷𝑃12 + 𝛾2𝑃12 − 𝜆12𝑃11 = 0 (4.11)

42. 32
(𝐷 + 𝛾1)𝑃11 − 𝜇21𝑃12 = 0 (4.12)
−𝜆12𝑃11 + (𝐷 + 𝛾2)𝑃12 = 0 (4.13)
𝑀𝑢𝑙𝑡𝑖𝑝𝑙𝑦 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 (4.12) 𝑏𝑦 (𝐷 + 𝛾2) 𝑎𝑛𝑑 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 (4.13) 𝑏𝑦 𝜇21
to obtain
(𝐷 + 𝛾1)(𝐷 + 𝛾2)𝑃11 − (𝐷 + 𝛾2)𝜇21𝑃12 = 0 (4.14)
−𝜆12 𝜇21 𝑃11 + (𝐷 + 𝛾2) 𝜇21 𝑃12 = 0 (4.15)
Add the above equations and solve simultaneously to get the roots (value of D), let’s call it w1 & w2:
[(𝐷 + 𝛾1)(𝐷 + 𝛾2) − 𝜆12𝜇21] 𝑃11 = 0 , [𝐷2
+ (𝛾1 + 𝛾2)𝐷 + 𝛾1𝛾2 − 𝜆12𝜇21] 𝑃11 = 0
𝑤1 =
−(𝛾1 + 𝛾2) − √(𝛾1 + 𝛾2)2 − 4 𝛾1𝛾2 + 4𝜆12𝜇21
2
, 𝑤2 =
−(𝛾1 + 𝛾2) + √(𝛾1 + 𝛾2)2 − 4 𝛾1𝛾2 + 4𝜆12𝜇21
2
𝑤ℎ𝑒𝑟𝑒 ∶ (𝛾1 + 𝛾2)2
− 4 𝛾1𝛾2 + 4𝜆12𝜇21 > 0
𝑃11 = 𝑐1ew1t
+ c2ew2t
𝑡𝑜 𝑜𝑏𝑡𝑎𝑖𝑛 𝑃12 𝑀𝑢𝑙𝑡𝑖𝑝𝑙𝑦 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 (4.12) 𝑏𝑦 𝜆12 𝑎𝑛𝑑 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛 (4.13) 𝑏𝑦 (𝐷 + 𝛾1)
(𝐷 + 𝛾1) 𝜆12 𝑃11 − 𝜆12 𝜇21𝑃12 = 0 (4.16)
−(𝐷 + 𝛾1) 𝜆12 𝑃11 + (𝐷 + 𝛾1)(𝐷 + 𝛾2) 𝑃12 = 0 (4.17)
Add the above equations and solve simultaneously to get the roots (value of D), let’s call it w1 & w2:
[(𝐷 + 𝛾1)(𝐷 + 𝛾2) − 𝜆12𝜇21] 𝑃12 = 0 , [𝐷2
+ (𝛾1 + 𝛾2)𝐷 + 𝛾1𝛾2 − 𝜆12𝜇21] 𝑃12 = 0
𝑤1 =
−(𝛾1 + 𝛾2) − √(𝛾1 + 𝛾2)2 − 4 𝛾1𝛾2 + 4𝜆12𝜇21
2
, 𝑤2 =
−(𝛾1 + 𝛾2) + √(𝛾1 + 𝛾2)2 − 4 𝛾1𝛾2 + 4𝜆12𝜇21
2
𝑤ℎ𝑒𝑟𝑒 ∶ (𝛾1 + 𝛾2)2
− 4 𝛾1𝛾2 + 4𝜆12𝜇21 > 0
𝑃12 = 𝑐3ew1t
+ c4ew2t
𝑆𝑢𝑏𝑠𝑡𝑖𝑡𝑢𝑡𝑒 𝑃11 & 𝑃12 𝑖𝑛 (4.10) 𝑡𝑜 𝑠𝑜𝑙𝑣𝑒 𝑓𝑜𝑟 𝑡ℎ𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐1 w1ew1t
+ c2 w2ew2t
+ 𝑐1𝛾1 ew1t
+ c2 𝛾1ew2t
− 𝜇21𝑐3ew1t
− 𝜇21 c4ew2t
= 0
(𝑐1 w1 + 𝑐1𝛾1 − 𝜇21𝑐3 ) ew1t
+ (c2 w2 + c2 𝛾1 − 𝜇21 c4)ew2t
= 0
𝑐1 w1 + 𝑐1𝛾1 – 𝜇21𝑐3 = 0
𝜇21𝑐3 = 𝑐1 w1 + 𝑐1𝛾1
𝑐3 =
𝑐1
𝜇21
(w1 + 𝛾1)
c2 w2 + c2 𝛾1 − 𝜇21 c4 = 0
𝜇21 c4 = c2 w2 + c2 𝛾1
c4 =
c2
𝜇21
(w2 + 𝛾1)
𝑃11 = 𝑐1ew1t
+ c2ew2t
, 𝑃12 = 𝑐3ew1t
+ c4ew2t
=
𝑐1
𝜇21
(w1 + 𝛾1) ew1t
+
c2
𝜇21
(w2 + 𝛾1) ew2t

43. 33
Using the initial values at :
𝑃11(0) = 1 → 𝑐1 + c2 = 1 → 𝑐1 = 1 − c2
𝑃12(0) = 0 →
𝑐1
𝜇21
(w1 + 𝛾1) +
c2
𝜇21
(w2 + 𝛾1) = 0
∴ 𝑐2 = (
w1 + 𝛾1
w1 − w2
) and 𝑐1 = (1 − 𝑐2) = (
w2 + 𝛾1
w2 − w1
)
𝑃11 = (
w2 + 𝛾1
w2 − w1
) ew1t
+ (
w1 + 𝛾1
w1 − w2
) ew2t
𝑃12 =
𝑐1
𝜇21
(w1 + 𝛾1) ew1t
+
c2
𝜇21
(w2 + 𝛾1) ew2t
= (
w2 + 𝛾1
w2 − w1
) (
w1 + 𝛾1
𝜇21
) ew1t
+ (
w1 + 𝛾1
w1 − w2
) (
w2 + 𝛾1
𝜇21
) ew2t
𝑙𝑒𝑡: (
w2 + 𝛾1
w2 − w1
) = 𝐴1 , (
w1 + 𝛾1
w1 − w2
) = A2 , (
w2 + 𝛾1
w2 − w1
) (
w1 + 𝛾1
𝜇21
) = A3 , (
w1 + 𝛾1
w1 − w2
) (
w2 + 𝛾1
𝜇21
) = A4
∴ 𝑃11 = 𝐴1ew1t
+ A2ew2t
(4.18)
∴ 𝑃12 = A3 ew1t
+ A4 ew2t
(4.19)
𝑑𝑃13
𝑑𝑡
= 𝑃12𝜆23 = 𝜆23(A3 ew1t
+ A4 ew2t) = 𝜆23A3 ew1t
+ 𝜆23 A4 ew2t
𝑃13 = [𝜆23A3
ew1t
w1
−
𝜆23A3
w1
] + [𝜆23 A4
ew2t
w2
−
𝜆23 A4
w2
]
𝑃13 =
𝜆23A3
w1
( ew1t
− 1) +
𝜆23 A4
w2
(ew2t
− 1)
𝑙𝑒𝑡
𝜆23A3
w1
= A5 ,
𝜆23 A4
w2
= 𝐴6
∴ 𝑃13 = 𝐴5 ( ew1t
− 1) + A6 (ew2t
− 1) (4.20)
𝑑𝑃14
𝑑𝑡
= 𝜆14 (𝐴1ew1t
+ A2ew2t) + 𝜆24(A3 ew1t
+ A4 ew2t) = (𝜆14 𝐴1 + 𝜆24 A3)ew1t
+ (𝜆14 A2 + 𝜆24 A4)ew2t
𝑙𝑒𝑡 ∶ 𝐺1 = (𝜆14 𝐴1 + 𝜆24 A3) , G2 = (𝜆14 A2 + 𝜆24 A4) , thus
𝑑𝑃14
𝑑𝑡
= 𝐺1ew1t
+ G2 ew2t
𝑃14 = [𝐺1
ew1t
w1
−
𝐺1
w1
] + [G2
ew2t
w2
−
G2
w2
] =
𝐺1
w1
(ew1t
− 1) +
G2
w2
(ew2t
− 1)
𝑙𝑒𝑡 ∶
𝐺1
w1
= 𝐴7 ,
G2
w2
= 𝐴8
∴ 𝑃14 = 𝐴7(ew1t
− 1) + 𝐴8(ew2t
− 1) (4.21)
Using equations (4.6) and (4.7) to solve the set of probabilities in the second row, these set of equations
are obtained:
𝐷𝑃21 + 𝛾1𝑃21 − 𝜇21𝑃22 = 0 (4.22)

44. 34
𝐷𝑃22 + 𝛾2𝑃22 − 𝜆12𝑃21 = 0 (4.23)
(𝐷 + 𝛾1)𝑃21 − 𝜇21𝑃22 = 0 (4.24)
−𝜆12𝑃21 + (𝐷 + 𝛾2)𝑃22 = 0 (4.25)
Using similar steps as before to get P11 & P12, where w1 & w2 as previously defined , P21 & P22
are :
∴ 𝑃21 = 𝑐5ew1t
+ c6ew2t
(4.26)
∴ 𝑃22 = 𝑐7ew1t
+ c8ew2t
(4.27)
𝑆𝑢𝑏𝑠𝑡𝑖𝑡𝑢𝑡𝑒 𝑒𝑞𝑢𝑎𝑡𝑖𝑜𝑛𝑠 (4.26) & (4.27) 𝑖𝑛 (4.22)𝑡𝑜 𝑒𝑣𝑎𝑙𝑢𝑎𝑡𝑒 𝑡ℎ𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐5 w1ew1t
+ c6 w2ew2t
+ 𝑐5𝛾1 ew1t
+ c6 𝛾1ew2t
− 𝜇21𝑐7ew1t
− 𝜇21 c8ew2t
= 0
(𝑐5 w1 + 𝑐5𝛾1 − 𝜇21𝑐7) ew1t
+ (c6 w2 + c6 𝛾1 − 𝜇21 c8)ew2t
= 0
𝑐5 w1 + 𝑐5𝛾1 – 𝜇21𝑐7 = 0
𝜇21𝑐7 = 𝑐5 w1 + 𝑐5𝛾1
𝑐7 =
𝑐5
𝜇21
(w1 + 𝛾1)
c6 w2 + c6 𝛾1 − 𝜇21 c8 = 0
𝜇21 c8 = c6 w2 + c6 𝛾1
c8 =
c6
𝜇21
(w2 + 𝛾1)
𝑃21 = 𝑐5ew1t
+ c6ew2t
, 𝑃22 = 𝑐7ew1t
+ c8ew2t
=
𝑐5
𝜇21
(w1 + 𝛾1) ew1t
+
c6
𝜇21
(w2 + 𝛾1) ew2t
Using the initial values at :
𝑃21(0) = 0 → 𝑐5 + c6 = 0 → 𝑐5 = −c6
𝑃22(0) = 1 →
𝑐5
𝜇21
(w1 + 𝛾1) +
c6
𝜇21
(w2 + 𝛾1) = 1
𝑃21 = (
𝜇21
w1 − w2
) (ew1t
− ew2t) , 𝑃22 = (
w1 + 𝛾1
w1 − w2
) ew1t
+ (
w2 + 𝛾1
w2 − w1
) ew2t
𝑙𝑒𝑡 (
𝜇21
w1 − w2
) = 𝐴9 , (
w1 + 𝛾1
w1 − w2
) = 𝐴2 , (
w2 + 𝛾1
w2 − w1
) = A1
∴ 𝑃21 = 𝐴9 (ew1t
− ew2t) (4.28)
∴ 𝑃22 = 𝐴2 ew1t
+ A1 ew2t
(4.29)
𝑑𝑃23
𝑑𝑡
= 𝑃22𝜆23 = 𝜆23(A2 ew1t
+ A1 ew2t) = 𝜆23A2 ew1t
+ 𝜆23 A1 ew2t
𝑃23 = [𝜆23A2
ew1t
w1
−
𝜆23A2
w1
] + [𝜆23 A1
ew2t
w2
−
𝜆23 A1
w2
]
𝑃23 =
𝜆23A2
w1
( ew1t
− 1) +
𝜆23 A1
w2
(ew2t
− 1)