SlideShare a Scribd company logo
1 of 39
Markov Chains
Description Sometimes we are interested in how a random variable changes over time. The study of how a random variable evolves over time includes  Stochastic Processes .
What is a Stochastic Process? ,[object Object],[object Object],[object Object]
[object Object],[object Object]
The Gambler’s Ruin Problem ,[object Object],[object Object],[object Object]
What is a Markov Chain? ,[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object]
TPM: The Gambler’s Ruin Problem ,[object Object]
[object Object],[object Object],[object Object]
Question ,[object Object],[object Object]
n -Step Transition Probabilities ,[object Object],[object Object],[object Object]
The Cola Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
The Cola Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
The Cola Example ,[object Object],[object Object]
The Cola Example ,[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object]
Limiting probabilities ,[object Object],[object Object],[object Object]
Question Find the equilibrium market shares of two firms whose probability transition matrix is as follows  .5 .5 B .3 .7 A B A
Example ,[object Object],1 2 3 1/2 1/2 1/2 1/2 1
All states communicate with each other Starting from 1, the MC can return to 1 in three steps via two possible routes: Route 1:  1 to 3 to 2 to 1 with probability  .5 ×1   ×.5  =  1/4 Route 2:  1 to 2 to 2 to 1 with probability  .5  ×.5×.5   = 1/8  Hence the required probability is =1/4+1/8 = 3/8 .
Steady-State Probabilities ,[object Object],[object Object]
[object Object],[object Object]
An Example ,[object Object],In the long In the long run, what fraction of the customers purchase the  respective brands?
Solution ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Inventory Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
Inventory Example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Inventory Example: (One-Step) Transition Matrix ,[object Object],[object Object],[object Object],[object Object]
Inventory Example: Transition Diagram 0 1 2 3
Inventory Example: (One-Step) Transition Matrix
Transition Matrix: Two-Step ,[object Object]
Transition Matrix: Four-Step ,[object Object]
Transition Matrix: Eight-Step ,[object Object]
Steady-State Probabilities ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Steady-State Probabilities: Inventory Example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Mean First Passage Times ,[object Object],[object Object]
[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

More Related Content

What's hot

Markov chain and its Application
Markov chain and its Application Markov chain and its Application
Markov chain and its Application Tilakpoudel2
 
Markov Chains.pptx
Markov Chains.pptxMarkov Chains.pptx
Markov Chains.pptxTarigBerba
 
Introduction to Random Walk
Introduction to Random WalkIntroduction to Random Walk
Introduction to Random WalkShuai Zhang
 
Stat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chainStat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chainKhulna University
 
Markov analysis
Markov analysisMarkov analysis
Markov analysisganith2k13
 
Markov Chain Monte Carlo Methods
Markov Chain Monte Carlo MethodsMarkov Chain Monte Carlo Methods
Markov Chain Monte Carlo MethodsFrancesco Casalegno
 
MARKOV CHAIN ANALYSIS IN AN ORGANISATION
MARKOV CHAIN ANALYSIS IN AN ORGANISATIONMARKOV CHAIN ANALYSIS IN AN ORGANISATION
MARKOV CHAIN ANALYSIS IN AN ORGANISATIONVivek Tyagi
 
Gram-Schmidt process linear algbera
Gram-Schmidt process linear algberaGram-Schmidt process linear algbera
Gram-Schmidt process linear algberaPulakKundu1
 
2 discrete markov chain
2 discrete markov chain2 discrete markov chain
2 discrete markov chainWindie Chan
 
Marcov chains and simple queue ch 1
Marcov chains and simple queue ch 1Marcov chains and simple queue ch 1
Marcov chains and simple queue ch 1Guru Nathan
 
Time domain analysis
Time domain analysisTime domain analysis
Time domain analysisHussain K
 

What's hot (20)

Markor chain presentation
Markor chain presentationMarkor chain presentation
Markor chain presentation
 
Markov chain and its Application
Markov chain and its Application Markov chain and its Application
Markov chain and its Application
 
Markov Chains.pptx
Markov Chains.pptxMarkov Chains.pptx
Markov Chains.pptx
 
Markov presentation
Markov presentationMarkov presentation
Markov presentation
 
Markov chain
Markov chainMarkov chain
Markov chain
 
Chap 4 markov chains
Chap 4   markov chainsChap 4   markov chains
Chap 4 markov chains
 
Markov process
Markov processMarkov process
Markov process
 
Introduction to Random Walk
Introduction to Random WalkIntroduction to Random Walk
Introduction to Random Walk
 
Stat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chainStat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chain
 
Markov analysis
Markov analysisMarkov analysis
Markov analysis
 
Markov chain-model
Markov chain-modelMarkov chain-model
Markov chain-model
 
Markov Chain Monte Carlo Methods
Markov Chain Monte Carlo MethodsMarkov Chain Monte Carlo Methods
Markov Chain Monte Carlo Methods
 
MARKOV CHAIN ANALYSIS IN AN ORGANISATION
MARKOV CHAIN ANALYSIS IN AN ORGANISATIONMARKOV CHAIN ANALYSIS IN AN ORGANISATION
MARKOV CHAIN ANALYSIS IN AN ORGANISATION
 
Markov chain
Markov chainMarkov chain
Markov chain
 
Probability And Random Variable Lecture 1
Probability And Random Variable Lecture 1Probability And Random Variable Lecture 1
Probability And Random Variable Lecture 1
 
Gram-Schmidt process linear algbera
Gram-Schmidt process linear algberaGram-Schmidt process linear algbera
Gram-Schmidt process linear algbera
 
2 discrete markov chain
2 discrete markov chain2 discrete markov chain
2 discrete markov chain
 
Geometric algorithms
Geometric algorithmsGeometric algorithms
Geometric algorithms
 
Marcov chains and simple queue ch 1
Marcov chains and simple queue ch 1Marcov chains and simple queue ch 1
Marcov chains and simple queue ch 1
 
Time domain analysis
Time domain analysisTime domain analysis
Time domain analysis
 

Viewers also liked

Lesson 4.1 Extreme Values
Lesson 4.1 Extreme ValuesLesson 4.1 Extreme Values
Lesson 4.1 Extreme ValuesSharon Henry
 
3.1 Extreme Values of Functions
3.1 Extreme Values of Functions3.1 Extreme Values of Functions
3.1 Extreme Values of FunctionsSharon Henry
 
Review Of Probability
Review Of ProbabilityReview Of Probability
Review Of ProbabilityMALAKI12003
 
Extreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitationExtreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitationJonathan D'Cruz
 
4 stochastic processes
4 stochastic processes4 stochastic processes
4 stochastic processesSolo Hermelin
 

Viewers also liked (7)

Markov exercises
Markov exercisesMarkov exercises
Markov exercises
 
The markovchain package use r2016
The markovchain package use r2016The markovchain package use r2016
The markovchain package use r2016
 
Lesson 4.1 Extreme Values
Lesson 4.1 Extreme ValuesLesson 4.1 Extreme Values
Lesson 4.1 Extreme Values
 
3.1 Extreme Values of Functions
3.1 Extreme Values of Functions3.1 Extreme Values of Functions
3.1 Extreme Values of Functions
 
Review Of Probability
Review Of ProbabilityReview Of Probability
Review Of Probability
 
Extreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitationExtreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitation
 
4 stochastic processes
4 stochastic processes4 stochastic processes
4 stochastic processes
 

Similar to Markov chains1

IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docxIE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docxsheronlewthwaite
 
12 Machine Learning Supervised Hidden Markov Chains
12 Machine Learning  Supervised Hidden Markov Chains12 Machine Learning  Supervised Hidden Markov Chains
12 Machine Learning Supervised Hidden Markov ChainsAndres Mendez-Vazquez
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionbutest
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionbutest
 
Martingales Stopping Time Processes
Martingales Stopping Time ProcessesMartingales Stopping Time Processes
Martingales Stopping Time ProcessesIOSR Journals
 
Rfid presentation in internet
Rfid presentation in internetRfid presentation in internet
Rfid presentation in internetAli Azarnia
 
Book chapter-5
Book chapter-5Book chapter-5
Book chapter-5Hung Le
 
2012 mdsp pr06  hmm
2012 mdsp pr06  hmm2012 mdsp pr06  hmm
2012 mdsp pr06  hmmnozomuhamada
 
stochastic-processes-1.pdf
stochastic-processes-1.pdfstochastic-processes-1.pdf
stochastic-processes-1.pdforicho
 
Univariate Financial Time Series Analysis
Univariate Financial Time Series AnalysisUnivariate Financial Time Series Analysis
Univariate Financial Time Series AnalysisAnissa ATMANI
 
Introduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov ChainsIntroduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov ChainsUniversity of Salerno
 

Similar to Markov chains1 (20)

Stochastic matrices
Stochastic matricesStochastic matrices
Stochastic matrices
 
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docxIE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
 
12 Machine Learning Supervised Hidden Markov Chains
12 Machine Learning  Supervised Hidden Markov Chains12 Machine Learning  Supervised Hidden Markov Chains
12 Machine Learning Supervised Hidden Markov Chains
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognition
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognition
 
Martingales Stopping Time Processes
Martingales Stopping Time ProcessesMartingales Stopping Time Processes
Martingales Stopping Time Processes
 
Stochastic Processes Assignment Help
Stochastic Processes Assignment HelpStochastic Processes Assignment Help
Stochastic Processes Assignment Help
 
By BIRASA FABRICE
By BIRASA FABRICEBy BIRASA FABRICE
By BIRASA FABRICE
 
Rfid presentation in internet
Rfid presentation in internetRfid presentation in internet
Rfid presentation in internet
 
Probable
ProbableProbable
Probable
 
Book chapter-5
Book chapter-5Book chapter-5
Book chapter-5
 
Talk 5
Talk 5Talk 5
Talk 5
 
NLP_KASHK:Markov Models
NLP_KASHK:Markov ModelsNLP_KASHK:Markov Models
NLP_KASHK:Markov Models
 
A STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAM
A STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAMA STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAM
A STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAM
 
2012 mdsp pr06  hmm
2012 mdsp pr06  hmm2012 mdsp pr06  hmm
2012 mdsp pr06  hmm
 
stochastic-processes-1.pdf
stochastic-processes-1.pdfstochastic-processes-1.pdf
stochastic-processes-1.pdf
 
Univariate Financial Time Series Analysis
Univariate Financial Time Series AnalysisUnivariate Financial Time Series Analysis
Univariate Financial Time Series Analysis
 
intro
introintro
intro
 
Introduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov ChainsIntroduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov Chains
 
Queueing theory
Queueing theoryQueueing theory
Queueing theory
 

More from Kinshook Chaturvedi (20)

Working and functions_of_rbi[1]
Working and functions_of_rbi[1]Working and functions_of_rbi[1]
Working and functions_of_rbi[1]
 
Role of idfc_in_infrastucture_finance
Role of idfc_in_infrastucture_financeRole of idfc_in_infrastucture_finance
Role of idfc_in_infrastucture_finance
 
Mutual funds
Mutual fundsMutual funds
Mutual funds
 
Iifcl ppt
Iifcl pptIifcl ppt
Iifcl ppt
 
Basel ii norms.ppt
Basel ii norms.pptBasel ii norms.ppt
Basel ii norms.ppt
 
Retail banking pres
Retail banking presRetail banking pres
Retail banking pres
 
Presentation on lic of india
Presentation on lic of indiaPresentation on lic of india
Presentation on lic of india
 
Mfi dfi
Mfi dfiMfi dfi
Mfi dfi
 
Management of np as imt
Management of np as imtManagement of np as imt
Management of np as imt
 
Life insurance in india final raja
Life insurance in india final rajaLife insurance in india final raja
Life insurance in india final raja
 
Financial inclusion
Financial inclusionFinancial inclusion
Financial inclusion
 
Corporate banking v2
Corporate banking v2Corporate banking v2
Corporate banking v2
 
Corporate banking latest
Corporate banking latestCorporate banking latest
Corporate banking latest
 
Financial mgt exercises
Financial mgt exercisesFinancial mgt exercises
Financial mgt exercises
 
Csac10[1].p
Csac10[1].pCsac10[1].p
Csac10[1].p
 
Csac08[1].p
Csac08[1].pCsac08[1].p
Csac08[1].p
 
Csac05[1].p
Csac05[1].pCsac05[1].p
Csac05[1].p
 
Csac14[1].p
Csac14[1].pCsac14[1].p
Csac14[1].p
 
Csac06[1].p
Csac06[1].pCsac06[1].p
Csac06[1].p
 
Xyber001 16 12_09 (2)
Xyber001 16 12_09 (2)Xyber001 16 12_09 (2)
Xyber001 16 12_09 (2)
 

Recently uploaded

Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.Eni
 
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...Dave Litwiller
 
Sales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for SuccessSales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for SuccessAggregage
 
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...lizamodels9
 
Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...
Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...
Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...lizamodels9
 
Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
M.C Lodges -- Guest House in Jhang.
M.C Lodges --  Guest House in Jhang.M.C Lodges --  Guest House in Jhang.
M.C Lodges -- Guest House in Jhang.Aaiza Hassan
 
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...lizamodels9
 
Call Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine ServiceCall Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine Serviceritikaroy0888
 
Call Girls in Gomti Nagar - 7388211116 - With room Service
Call Girls in Gomti Nagar - 7388211116  - With room ServiceCall Girls in Gomti Nagar - 7388211116  - With room Service
Call Girls in Gomti Nagar - 7388211116 - With room Servicediscovermytutordmt
 
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Dipal Arora
 
Vip Dewas Call Girls #9907093804 Contact Number Escorts Service Dewas
Vip Dewas Call Girls #9907093804 Contact Number Escorts Service DewasVip Dewas Call Girls #9907093804 Contact Number Escorts Service Dewas
Vip Dewas Call Girls #9907093804 Contact Number Escorts Service Dewasmakika9823
 
Insurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageInsurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageMatteo Carbone
 
A DAY IN THE LIFE OF A SALESMAN / WOMAN
A DAY IN THE LIFE OF A  SALESMAN / WOMANA DAY IN THE LIFE OF A  SALESMAN / WOMAN
A DAY IN THE LIFE OF A SALESMAN / WOMANIlamathiKannappan
 
Grateful 7 speech thanking everyone that has helped.pdf
Grateful 7 speech thanking everyone that has helped.pdfGrateful 7 speech thanking everyone that has helped.pdf
Grateful 7 speech thanking everyone that has helped.pdfPaul Menig
 
Pharma Works Profile of Karan Communications
Pharma Works Profile of Karan CommunicationsPharma Works Profile of Karan Communications
Pharma Works Profile of Karan Communicationskarancommunications
 
7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...Paul Menig
 
The CMO Survey - Highlights and Insights Report - Spring 2024
The CMO Survey - Highlights and Insights Report - Spring 2024The CMO Survey - Highlights and Insights Report - Spring 2024
The CMO Survey - Highlights and Insights Report - Spring 2024christinemoorman
 
Cash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call GirlsCash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call GirlsApsara Of India
 

Recently uploaded (20)

Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.
 
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
 
Sales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for SuccessSales & Marketing Alignment: How to Synergize for Success
Sales & Marketing Alignment: How to Synergize for Success
 
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
 
Forklift Operations: Safety through Cartoons
Forklift Operations: Safety through CartoonsForklift Operations: Safety through Cartoons
Forklift Operations: Safety through Cartoons
 
Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...
Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...
Call Girls In Connaught Place Delhi ❤️88604**77959_Russian 100% Genuine Escor...
 
Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Mehrauli Delhi 💯Call Us 🔝8264348440🔝
 
M.C Lodges -- Guest House in Jhang.
M.C Lodges --  Guest House in Jhang.M.C Lodges --  Guest House in Jhang.
M.C Lodges -- Guest House in Jhang.
 
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
 
Call Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine ServiceCall Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine Service
 
Call Girls in Gomti Nagar - 7388211116 - With room Service
Call Girls in Gomti Nagar - 7388211116  - With room ServiceCall Girls in Gomti Nagar - 7388211116  - With room Service
Call Girls in Gomti Nagar - 7388211116 - With room Service
 
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
 
Vip Dewas Call Girls #9907093804 Contact Number Escorts Service Dewas
Vip Dewas Call Girls #9907093804 Contact Number Escorts Service DewasVip Dewas Call Girls #9907093804 Contact Number Escorts Service Dewas
Vip Dewas Call Girls #9907093804 Contact Number Escorts Service Dewas
 
Insurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageInsurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usage
 
A DAY IN THE LIFE OF A SALESMAN / WOMAN
A DAY IN THE LIFE OF A  SALESMAN / WOMANA DAY IN THE LIFE OF A  SALESMAN / WOMAN
A DAY IN THE LIFE OF A SALESMAN / WOMAN
 
Grateful 7 speech thanking everyone that has helped.pdf
Grateful 7 speech thanking everyone that has helped.pdfGrateful 7 speech thanking everyone that has helped.pdf
Grateful 7 speech thanking everyone that has helped.pdf
 
Pharma Works Profile of Karan Communications
Pharma Works Profile of Karan CommunicationsPharma Works Profile of Karan Communications
Pharma Works Profile of Karan Communications
 
7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...7.pdf This presentation captures many uses and the significance of the number...
7.pdf This presentation captures many uses and the significance of the number...
 
The CMO Survey - Highlights and Insights Report - Spring 2024
The CMO Survey - Highlights and Insights Report - Spring 2024The CMO Survey - Highlights and Insights Report - Spring 2024
The CMO Survey - Highlights and Insights Report - Spring 2024
 
Cash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call GirlsCash Payment 9602870969 Escort Service in Udaipur Call Girls
Cash Payment 9602870969 Escort Service in Udaipur Call Girls
 

Markov chains1

  • 2. Description Sometimes we are interested in how a random variable changes over time. The study of how a random variable evolves over time includes Stochastic Processes .
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20. Question Find the equilibrium market shares of two firms whose probability transition matrix is as follows .5 .5 B .3 .7 A B A
  • 21.
  • 22. All states communicate with each other Starting from 1, the MC can return to 1 in three steps via two possible routes: Route 1: 1 to 3 to 2 to 1 with probability .5 ×1 ×.5 = 1/4 Route 2: 1 to 2 to 2 to 1 with probability .5 ×.5×.5 = 1/8 Hence the required probability is =1/4+1/8 = 3/8 .
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30. Inventory Example: Transition Diagram 0 1 2 3
  • 31. Inventory Example: (One-Step) Transition Matrix
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.