USING GAMES TO TEACH MARKOV CHAINS

Primus: Problems, Resources, and Issues in Mathematics Undergraduate Studies, Dec 2003 by Johnson, Roger W

So, to answer the question "What's the chance we secure some cargo?", we need only compute (P^sup 3^)^sub 0,3^ which is just slightly less than 0.5400. Note that this probability is concerned with securing a Ship, Captain, Crew within three rolls. The chance of securing Ship, Captain, Crew in exactly three moves, on the other hand, is (P^sup 3^)^sub 0,3^ - (P^sup 2^)^sub 0,3^, or about 0.18375.

PROBABILITY CALCULATIONS AND SUMMARY MEASURES

The games mentioned above deal with stationary Markov chains having a single absorbing state with the other states being transient. Now we find the probability mass function for the number of moves taken by the winner of such a game when n players are playing the game and, consequently, summary measures such as the average number of moves by the winner in such a game. (Note, however, that as far as Ship, Captain, Crew is concerned, we are now looking at a modified version of the game where each person has his or her own five dice and takes one roll at a time; the first one to secure Ship, Captain, Crew wins, with cargo being irrelevant.) This material may be presented to students once they understand how (P^sup m^)^sub i,j^ may be interpreted.

Tables 1 and 2 give the expected game length and standard deviation of game length for one to four players in Hi Ho! Cherry-O and Chutes and Ladders, respectively (c. f. [7],[9]).

Turning to Monopoly, I tell my students that it would be nice to know the long-run proportion of time a player could expect to land on certain properties as this could partly guide a player's strategy (e.g. which properties to buy, which properties to build upon first). In the course I teach I do take the time to give sufficient conditions under which limits of the form (G) occur (being especially careful to present and illustrate the associated language). In particular, I give a simple heuristic why [pi] = ([pi]^sub 0^,[pi]^sub 1^, ...)^sup t^ is the eigenvector of P^sup t^ corresponding to eigenvalue 1 (i. e. P^sup t^[pi] = [pi]) properly scaled so that the entries add to one. In regards to Monopoly, however, what I find especially interesting is the preliminary modeling process that needs to be undertaken before considering the numeric problem of determining the long-run probabilities, [pi]. The Monopoly board has 40 squares but one of them, the "GO TO JAIL" square, you do not remain on at the completion of the throw of the dice. Also, one of these squares can be occupied two different ways - on the Jail square one can be in Jail or just visiting Jail. This gives us a total of 40 different states to be in at the end of a throw. I find that most of my students have played Monopoly and remember a good number of the rules for the game. Consequently, when I ask them if Monopoly can be modeled as a 40-state stationary Markov chain, my students usually determine that it cannot and identify one or two problems. One problem focuses on the throwing of doubles. If, when moving around the board, a player throws three consecutive doubles, then (s)he is sent to Jail (so that where the player goes next doesn't depend just on where (s)he is now). To get around this problem, any square can be transformed to one having three states - landing on that square with no doubles, landing on that square with doubles once, and landing on that square with doubles twice. Jail is also a problem. As there is no room for human decision in a Markov chain model, two possible adaptations to the rules are to (a) remain in Jail as long as possible attempting to leave by throwing doubles until being forced out on the third failed try, or (b) leave jail immediately and pay the $50 fine. If one adopts the remain in Jail rule, then again we can model Jail as having 3 states - in Jail with no doubles attempts, in Jail with one failed doubles attempt, and in Jail with two failed doubles attempts and this then gives a total of 120 states. If one adopts the leave Jail immediately rule, then Jail can be modeled as a single state and this then gives a total of 118 states. Note that to implement these Jail rules one may not make use of "Get of Jail Free" cards - these cards can be removed before play, disregarded during play, or give some monetary award ($50?) to those that draw it during the game. A traditionally more difficult problem area for my students to recognize is that the usual way of drawing Chance arid Community Chest cards - drawing the top card and then placing it on the bottom - leads to non-stationarity. The simple solution here is to simply reshuffle the appropriate Chance or Community Chest card pile each time a card is drawn.