STATS 320 – Applied stochastic modelling – 2024
Assignment 2
Due: 18 April 2024 at 5pm.
The due date for this assignment is the same day as the mid-semester test. This is intentional: doing the assignment will be helpful in studying for the test.
1. A property development company takes on one new project each year, investing all of its capital in that year’s project. If the project is successful, the company makes a proit of $1million. If the project is unsuccessful, the company is ruined (loses all its money) and ceases to exist. The success or failure of each project is independent of all previous ones. The probability that a project is unsuccessful when x million dollars is invested (for x = 1, 2, 3, . . .) is 1/(x + 1)2. Let Xn million dollars be the company’s total capital after n projects.
(a) Write down the transition probabilities for the Markov chain (Xn ).
(b) Show that when X0 = i, the probability hi of eventual ruin is i+1/1.
(c) The result of part (b) implies that the company may never be ruined. What happens to the company’s inances in that case?
2. A professor has a collection of 5 umbrellas, which are either at home or at her office. If it is raining when she is going to work in the morning, or in the evening on her return, she uses an umbrella, if possible. If it is not raining, then she doesn’t carry an umbrella with her, so they could end up all accumulating in one location, and she might not have any umbrellas available. Suppose that on each trip it rains with probability 0.3, independently of past trips. Model this system as a Markov chain, letting Xn be the number of umbrellas at home each morning before the day starts.
(a) Why is it enough to consider just the umbrellas at home as a description of the state of the system? (b) Write down the transition matrix P for your model.
(c) What is the equilibrium distribution for this chain?
(d) What is the expected number of umbrellas at home at the beginning of the day in equilibrium?
(e) What is the proportion of days on which she needs an umbrella but doesn’t have one, in equilibrium?
(f) Suppose she starts on the morning of the irst day with 2 umbrellas at home, and 3 at work (i.e. X0 = 2).
What is the expected number of working days before she has no umbrella when she needs it? (Don’t include the umbrella-less day itself.)
3. Consider the following Markov chain. There is a inite set S of states. Some pairs of states {i, jg are connected by a link; for such pairs, transitions are possible from i to j, and also from j to i. In the absence of a link, no transitions are possible in either direction. Let ni be the number of other states that state i is linked to. The transition probabilities from state i are pij = ni/1 for each state j that i is linked to, and 0 otherwise.
(a) Show that the probability distribution π on S given by
is an equilibrium distribution for the chain satisfying the detailed balance equations.
Now consider a chess piece (rook, knight, bishop, queen, or king) starting at the top left square of a chess board and making a sequence of random moves from there. At each step, one of the possible moves is chosen at random, with all legal moves being equally likely to be chosen.
(b) Find an equilibrium distribution for this Markov chain when the piece is
i. a rook
ii. a bishop
iii. a knight
(c) For which pieces (rook, knight, bishop, queen, or king) does this Markov chain converge to a limiting distribution? What is the limiting distribution? Explain why.
Hints. If you don’t know the allowed moves of the pieces in chess, the Wikipedia article “Rules of chess” is one place to look them up. It is useful to observe that the knight always ends a move on a square of the opposite colour to the one it started on, while a bishop always ends on a square of the same colour to the one it started on (and so is forever restricted to squares of a single colour).
4. The Super Lucky Strike Lottery is drawn weekly. Each week, $1million of new money is available to pay prizes. The number of winners each week (independently of previous weeks) has a Poisson distribution with mean 1:2. If there are no winners, that week’s prize-money is saved into a pool to help pay for future prizes. Otherwise, the managers attempt to give a prize of $1million to each winner, drawing on the pool when there is more than one winner. If there is insufficient prize-money available to do this, the whole amount of prize-money available is shared equally among the winners.
Let Xn denote the contents of the pool (in $millions) at the beginning of week n (before that week’snew money has arrived, or prizes have been paid out). Suppose X1 = 0 (the prize-pool account is initially empty).
(a) Explain why X1 ; X2 : : : is a Markov chain, and give the transition probabilities.
(b) Write a simulation of this system in R. Hand in your R code and an example of a typical sample path X1 ; X2 ; : : : ; X100. (Hint. The queueing model simulations in the lecture slides are a useful starting point.)
(c) Use your simulation to estimate the following things.
i. The chain’s equilibrium distribution. Include both numerical estimates of the irst few probabilities and a graph of the estimated distribution. (Hint. The table function in R might be useful.)
ii. The mean amount of money carried over from one week to the next in the prize pool, in equilibrium. Include a 95% conidence interval.
iii. The fraction of draws (in equilibrium) that empty out the prize pool, leaving no money to help pay for future prizes. Include a 95% conidence interval.