Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due
MATH2012 STOCHASTIC PROCESSES
1. A Markov chain {Xn : n ≥ 0} on the state space S = {1, 2, 3, 4, 5} has the following transition probability matrix
Let p (n) denote the probability distribution of Xn, n = 0, 1, 2, . . ..
(a) Draw the transition diagram of this Markov chain. [5 marks]
(b) Decompose this Markov chain into intercommunicating classes, and, without performing any calculations, identify each class as transient or recurrent. [5 marks]
(c) If the Markov chain starts in state 5 at step 0, find
P{X0 = 5, X1 = 4, X2 = 5, X3 = 1, X4 = 2}. [4 marks]
(d) If the initial distribution of the Markov chain is p (0) = (1/2, 1/2, 0, 0, 0), find p(n) for n = 5. [4 marks] [Total 18 marks]
2. A Markov chain {Xn : n ≥ 0} on the infinite state space S = {1, 2, 3, . . .} has the following transition probability matrix
that is, its elements are given by
where 0 < p < 1 is a constant.
(a) Draw the transition diagram. [2 marks]
(b) Explain why the probability of first return to state 1 at step n is given by
[2 marks]
(c) Find the probability of returning to state 1. [2 marks]
(d) (i) Show that
[3 marks]
(ii) By using (b) and (d)(i), or otherwise, find the mean recurrence time of state 1. [3 marks]
(e) Find the period of the state 3. [3 marks]
(f) Explain (you may quote results from this module) why this Markov chain is Ergodic. [4 marks]
(g) Find the mean recurrence time of state 2. [7 marks] [Total 26 marks]
3. Each individual in a certain population is independent of the other individuals in the population and is subject to an independent birth process and an independent death process. Each birth event always results in one live offspring and so increases the population size by one. Each death event always results in the death of one individual and so decreases the population size by one. In the small time interval (t, t + δt], each individual in the population has probability 3λδt + o(δt) of experiencing a birth event, and probability 2µδt + o(δt) of experiencing a death event, where λ > 0 and µ > 0 are constants. The population size at time t ≥ 0 is X(t). The initial population size is X(0) = 2.
(a) (i) Explain carefully why
[1 + aδt + o(δt)]k = 1 + kaδt + o(δt),
where k is a natural number and a is a given constant. [5 marks]
(ii) By using (a)(i) or otherwise, show carefully that
P{X(t + δt) = n + 1|X(t) = n} = 3nλδt + o(δt), n = 1, 2, . . . [5 marks]
(b) Replace each ∗ with correct entry in the expressions for the transition probabilities below:
P{X(t + δt) = n − 1|X(t) = n} = (∗)δt + o(δt), n ≥ 1
P{X(t + δt) = n|X(t) = n} = 1 − (∗)δt + o(δt), n ≥ 1
P{X(t + δt) = j|X(t) = n} = (∗)δt + o(δt), j = n − 1, n, n + 1, n ≥ 1. [4 marks]
(c) Write down the generator matrix A of the process {X(t) : t ≥ 0}. [3 marks]
(d) Let pn(t) = P{X(t) = n}. Explain carefully how the following differential equations
p'0 (t) = 2µp1(t)
p'n (t) = 3(n − 1)λpn−1(t) − n(3λ + 2µ)pn(t) + 2(n + 1)µpn+1(t), n ≥ 1
can be derived from Kolmogorov’s forward equations dt/dP(t) = P(t)A. [4 marks]
(e) State the distribution of the holding time of state i, including the value of the parameter involved, where i is a given natural number. [4 marks]
(f) State the transition probability matrix of the corresponding embedded Markov chain. [4 marks] [Total 29 marks]
4. In a two-state survival model, an individual is in one of the two possible states: alive (A) or dead (D). The mortality rate of the individual at age t is given by t +1/c t where c > 0 is a given constant. Let X(t) denote the state the individual is in at age t.
(a) Write down the generator matrix A of this Markov jump process. [3 marks]
(b) Let pAA(s, t) = P{X(t) = A|X(s) = A}. Explain carefully how the differential equation
can be derived from Kolmogorov’s forward equation. [4 marks]
(c) Use your own words to define the residual holding time, Rs, of a Markov jump process. [3 marks]
(d) Find an expression for the probability P{Rs > t − s|X(s) = A}. Explain why this expression satisfies the differential equation in (b). [4 marks] [Total 14 marks]
5. Let {Bt : t > 0} be the standard Brownian motion.
(a) Use the definition of {Bt : t > 0} to show that Cov(Bs, Bt) = min(s, t), where s > 0 and t > 0 are constants. [4 marks]
(b) By Using Ito’s lemma or otherwise, show that
d exp(2Bt) = 2 exp(2Bt)dBt + 2 exp(2Bt)dt . [5 marks]
(c) By Using the result in (b) or otherwise, show that
where s > 0 is a given constant. [4 marks] [Total 13 marks]